MAPS-L Archives

Maps-L: Map Librarians, etc.

MAPS-L@LISTSERV.UGA.EDU

Options: Use Forum View

Use Monospaced Font
Show HTML Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Subject:
From:
Jeff Essic <[log in to unmask]>
Reply To:
Maps-L: Map Librarians, etc.
Date:
Wed, 23 Jan 2019 09:10:47 -0500
Content-Type:
multipart/related
Parts/Attachments:
text/plain (4 kB) , text/html (9 kB) , image001.png (4 kB)
All the answers so far have been good, but I also want to point out that it
is possible to bulk download images from EarthExplorer.  You must have an
account and be logged in.

On the interface, after you have your results, there's a link above the
results frame: "Click here to export your results."  After several steps,
you can get yourself a csv of the results.  The last column in the csv is a
link to the thumbnail.   In the link path, the folder and file name of the
thumbnail will tell you also where the zip file is and it's name.

The next step is to click to download one of the individual images.  When
the download popup appears, highlight the area with the download button,
and in Chrome right click and go to Inspect.  In the in Elements console,
you can see the full path for that particular image.  Use that path info
and in your csv, copy/replace what you need to construct the full path to
all the zip files you want to download.

At this point, use your favorite downloader app to get all the files or a
python script.  I would probably construct a batch file and use wget but
there are numerous GUI options as well.

Best Regards,
Jeff Essic


------------------------------------------------------------------------------------
Jeff Essic
Data & Visualization Librarian
North Carolina State University
D.H. Hill Library, Data & Visualization Services
Box 7111, Raleigh, NC  27695-7111
(919) 515-5698
http://www.lib.ncsu.edu/gis




On Tue, Jan 22, 2019 at 2:06 PM Lauer, Greg <[log in to unmask]> wrote:

> Hi Francine,
>
>
>
> As your student is looking at processing a significant of data I would be
> looking at utilizing some cloud processing tools on a platform such as
> Amazon Web Services (AWS). This would be much more efficient than batch
> downloading terabytes of imagery data
>
>
>
> AWS hosts some significant imagery holdings including Landsat, Sentinel
> and NAIP:
>
>
>
> https://registry.opendata.aws/sentinel-2/
>
> https://registry.opendata.aws/landsat-8/
>
> https://registry.opendata.aws/sentinel-1/
>
> https://registry.opendata.aws/naip/
>
>
>
> They basic idea is that you set up a virtual service (such as an EC2
> instance) to process the data within the cloud. The results can be stored,
> viewed and downloaded as necessary. The links above point to some examples.
>
>
>
> In most cases the data access is free (sentinel-2 is user pays), and you
> only pay for the time the EC2 instance is running/processing plus and
> storage costs.
>
>
>
> If the student would prefer to batch download the raw data there are many
> tools that allow you to connect the ‘buckets’ referenced above and download
> (for example S3 Browser). Several scripts and command line tools can also
> be used to pull the data from S3.
>
>
>
> Hope the above helps
>
>
>
>
> *Greg Lauer *Business Development Manager – Asia/Pacific
>
> *East View Geospatial, Inc. *10601 Wayzata Blvd, Minneapolis, MN
> 55305-1526 USA
> Tel : AU +61 405 258 631 | US +1 (909) 979 3822
> [log in to unmask] www.geospatial.com
>
>
>
>
>
> *From:* Maps-L: Map Librarians, etc. <[log in to unmask]> *On Behalf
> Of *Francine Berish
> *Sent:* Monday, January 21, 2019 3:56 PM
> *To:* [log in to unmask]
> *Subject:* batch global imagery downloading
>
>
>
> [please forgive the cross posting]
>
>
>
> Good afternoon folks,
>
>
>
> Helping a student who is using artificial intelligence (AI) to interpret
> global imagery for geological purposes. The student has been testing this
> workflow using LandSat 8 (bands 6-7), ASTER shortwave infrared, and Digital
> Globe RGB. Since the student is looking to download and process imagery in
> large batches – the Earth Explorer and Digital Globe portals propose some
> unique challenges as imagery practically has be downloaded tile by tile.
> Wondering if there’s a way to access any of these sources through a
> directory or batch download?
>
>
>
> The researcher is open sources other than LandSat 8, ASTER & Digital
> Globe. The project is not geographically specific, and is using global
> imagery sources. It would acceptable if there were a license or cost
> associated with this data, as long as the batch processing and possible
> automation capability is there.
>
>
>
> Your insight and expertise are greatly appreciated!
>
>
>
> Best&Cheers,
> Francine
>
> ________________________________________­­________________________
>
> Francine Berish | Geospatial Data Librarian | Queen’s University Library
>
> 613-533-6000 x 36952 | [log in to unmask]
>
> [image: https://library.queensu.ca/files/qul-email-signature.png]
>
>
>


ATOM RSS1 RSS2