Archive

Posts Tagged ‘imagery’

The Landsat Explorer Web Mapping Application, Revisited

August 10, 2020 Leave a comment

In four years since we last wrote about what was then the “Landsat thematic bands application,” many features have been added to this tool. Furthermore, as it is, I believe, one of the very most useful mapping tools for research and instruction, ’tis time for an updated review. Now known as the Landsat Explorer app, this web mapping application is now part of the Living Atlas set of apps, a very useful set of tools that includes the Wayback imagery (which we have written about here), the water balance app, and others. I have used the Landsat Explorer app for research and also in a wide variety of instructional settings, from upper primary to university level, and in remote sending and GIS courses, but also in geography and environmental science courses. You could also use it for research purposes. With this app, Landsat images for the whole world are available to you at your fingertips, over a 45 year span of time. Truly amazing.

The Landsat Explorer app’s main tools are its renderer, identify, time-swipe-change, mask, save layer to ArcGIS Online, export, and add data from ArcGIS Online.

The renderer allows you to choose from about 10 band combinations, from agriculture to vegetation index to geology and more. This is incredibly responsive and useful. The only slight misgiving I have from an instructional standpoint is that I wish the band combinations would be visible, so students and others understand some of remote sensing theory. To the app’s credit, though, you can choose your own custom bands and indicate the band number you wish to display, shown below.

Identify allows you to learn the image scene ID, acquisition date, and cloud cover.

Most likely, most users will use this application for understanding change over time via imagery. You can use the swipe tool and the change detection tool for this. Before you can compute change, you will need to select Primary and Secondary Dates. Use the Time Slider to select an earlier date and click the Set as Secondary Layer button. Then, select a later date and move to the Swipe tool or the Change Detection tool to compare them. The Swipe tool for the Mt St Helens area for 1990 and 2019 are shown below. The swipe tool is my favorite but admittedly sometimes a frustrating tool to use, to get exactly the images you wish to have on the left and right side. It is also a bit confusing in the app to understand which image corresponds to which dates. All I can say here is to go through the very helpful tutorial, available on the lower left via the ‘graduation cap’ symbol, and carefully follow the steps.

The change detection tool can calculate changes in vegetation health (NDVI or SAVI), burned area (Burn Index), water content (Water Index), and urban area (Urban Index). Three items of interest: First, I am using the vegetation index, but you can use this drop-down to change to the soil adjusted vegetation index, the burn index, the water index, and the urban index. Second, the sliders can be used in your change detection. Third, you can define your own area of interest rather than take the default scene that is in your browser window.

Save allows you to save the top layer to your ArcGIS Online account (a log in to your own account is required). Export allows you to extract whatever map (the top layer in the app) you have built as a local file–a high resolution TIFF with customizable pixel sizes and spatial reference. This is a file that you can then bring into ArcGIS Pro or any GIS software for further analysis.

As the name implies, “Add data from ArcGIS Online” allows you to bring in your content from your own ArcGIS Online account into the Landsat viewer (a log in to your own account is required). After doing so, however, it wasn’t clear how to obtain a list of layers so I could turn my added layer off, so in my view this is the only tool I would probably would not use much. “Stories” shows a selection of 5 areas around the world where Landsat imagery is applied to solve real-world challenges, including in flood analysis. The “points of interest” choice here is particularly useful for instruction, guiding you to numerous places around the planet with explanations of each human-built and natural feature with a specific combination of Landsat bands.

Finally, if you wish to create your own similar type of app in the ArcGIS platform, go to the GitHub imagery zone and to the ArcGIS Developers site. Give this tool a try and I look forward to your comments.

–Joseph Kerski

Be Critical of the Data: Imagery too!

July 12, 2020 5 comments

As we have written about many times in this blog and in our book, being a critical consumer of data is essential for successful, wise decision making in the modern world.  Geospatial data offers a vast array of capabilities, but also offers numerous examples of discrepancies that often are found only through paying attention to details.  Even imagery is not immune to offsets and discrepancies, as we have detailed here and here.  The following represents another example of why this all matters.

A colleague of mine was working on a project outside of the town of Redcliff, in western Colorado USA.   In the first image, the red point on the older World Imagery (Esri Clarity) layer is at the corner of the house, and its coordinates (in WGS84) were calculated on that image.  In the second image, the white point at the corner of the house is on the current Esri World Imagery layer, and the coordinates (in WGS84) that were calculated based on that image.

bonar1
bonar2

Below are the same points on Google Earth. They match up pretty well with the World Imagery (Clarity) imagery.

bonar_google_earth

Further investigating the extent of what seems to be an offset between the two image layers are three screen captures below in the Paonia, Colorado area. These show a Delta County local road crossing the North Fork Gunnison River.   The map is cast in NAD83 UTM Zone 13.  The roads shapefile is from the Colorado Department of Transportation, Delta County Local and Major Roads (2019), and also in NAD83 UTM13.  The intersection point shows WGS1984 Lat-Long coordinates.   The two Esri World Imagery layers are both in WGS_1984_Web_Mercator_Auxiliary_Sphere.   The latest USDA NAIP (2019) is in NAD83 UTM13.   The World Imagery (Clarity) basemap and the NAIP both line up well with the Colorado DOT roads shapefile. The Esri World Imagery basemap does not.

bonar5
bonar6
bonar7

After reading these notes from my colleague, I spent some time looking at this, and then, along the lines of “being critical of the data” I did some of my own investigating.  I drew the yellow line in ArcGIS Online on the image along the bridge following the Clarity layer, and a cream-colored line along the bridge following the base imagery layer, noting that about a 12 meter offset exists, largely along a north-south axis, here.  But, I panned to other locations in Colorado and outside Colorado and found no offset.

bonar3
bonar4

I relayed this to my Esri colleagues who upon investigation, told me that this is not a projection issue, but likely a shift introduced in Maxar’s orthorectification process (related to ground control or DEM or both).   They told me that they will be publishing CONUS wide updates over the next several months which should resolve this particular localized shift.   Hence, repeat my above experiment whenever you happen to be reading this essay, and you may find different (and hopefully better) results!

I think at least five take-away points are important here (feel free to add more in the comments section).

(1)  GIS and remote sensing tools and data are continually evolving particularly in our data-as-services world.

(2)  Sometimes a wealth of information can be obtained by asking questions of the data and services providers.  Sometimes they can even hasten changes that need to be made.

(3)  The user needs to determine “fitness for use” regarding the spatial accuracy, completeness, date, and other characteristics for their project:  The project’s needs determine whether the data under consideration will be useful.  In the above example, if you were laying water or gas pipe or conducting land surveying for the assessor’s department, the above offsets would likely require you to use the Clarity layer or seek another source.  But if you are assessing regional or even local land use patterns, either layer would likely be just fine.

(4) Be critical of the data.   Be curious and ask questions.

(5)  Pay attention to detail.  Investigate.

–Joseph Kerski

Categories: Public Domain Data Tags: ,

Top 7 Satellite Imagery Sources

April 19, 2020 5 comments

My colleagues at EOS.com recently wrote a summary of what they consider to be the top 7 satellite imagery sources:  https://eos.com/blog/7-top-free-satellite-imagery-sources-in-2019/.

These include sources that we have reviewed on this Spatial Reserves blog, here:  https://spatialreserves.wordpress.com/2019/02/18/the-top-10-most-useful-geospatial-data-portals-revisited/   as well as the top Landsat sites that we reviewed, here:  https://spatialreserves.wordpress.com/2019/08/04/the-top-10-landsat-image-sites/, but it also includes a few we have not reviewed, such as EOS’ own Landviewer.

It is good to see reviewed collections such as these that have an aim to make life a bit easier for us as data consumers and analysts.

landviewer.PNG

Landviewer data portal, from EOS.  

–Joseph Kerski

Teaching Location Privacy and Resolution with a Big Pixel Image

October 13, 2019 Leave a comment

Ever since those ultra-high-resolution “gigapan” images began appearing from Microsoft and other sources a decade ago, I have been fascinated by them for their use in education.  Today, I frequently use the following image taken off of the Oriental Pearl Tower in China (at 468m, the tallest tower in China from 1994-2007):   http://sh-meet.bigpixel.cn/?from=groupmessage&isappinstalled=0     This image, compiled from billions of pixels, is amazing in its resolution.  A video on how I teach with it is here. 

pixel1

Big pixel image from Oriental Pearl Tower in China–initial view.

I have, for example, included this image in a university cartography and geo-visualization course that I teach online.  I first ask the students to examine the cultural geography, assessing the land use, zoning, traffic, and other aspects.  Then, I ask them to examine the physical geography–the terrain, the vegetation, the river winding through the city, and so on.

Third, I ask them to consider the resolution, reflecting on what we have discussed thus far in the course.  I ask them: Can you see inside office buildings and residential windows? Can you read license plates on cars?  Can you determine what pedestrians look like?  I ask them to think about:  Do your answers and the resolution of this image bring up any ethical concerns?

pixel2

Big pixel image from the Oriental Pearl Tower in China–detailed view. 

Fourth, I ask them to consider another topic we have discussed:  The Internet of Things and our connected world.  Where does information come from?  Increasingly, it is from webcams, sensors, and humans.  We have a chat about face recognition software and how none of the faces in this image (as of this writing) are blurred.  What are the implications for blurring and not blurring?  Finally, I ask them to take a random sample of 10 people in the gigapixel image.  How many people are holding a tablet or smartphone?  What implications does this have on information, and for society?

–Joseph Kerski

Copernicus Earth Observation Programme – Data and Information Access Services (DIAS)

August 13, 2019 1 comment

To facilitate access to the European Space Agency’s (ESA) Copernicus earth observation data (Sentinel satellites), the European Commission has funded five cloud-based platforms, known as the Data and Information Access Services (DIAS), to provide users with easy access to data from the various satellite missions. The data are available on the platforms, subject to account registration, as both open data and on a pay-per-use basis.

The following examples of data search and download options were taken from the WEkEO data service. The other data hosting services include ONDA, numdi, sobloo  and CREODIAS.

The raw data are available as NetCDF (.nc) files on the WEkEO site, an example of which is shown here using the Panoply data viewer.

There are various tools available to convert NetCDF files to more GIS friendly formats such as the Marine Geospatial Ecology Tools, a free add-on for ArcGIS that converts NetCDF to ASCII and ArcGrid formats.

 

 

 

Categories: Public Domain Data Tags: , ,

The Top 12 Most Useful Landsat Image Sites

August 4, 2019 5 comments

Recently, I wrote an essay about the sites that are, in my judgment, the top 10 in terms of containing useful geospatial data.   Now, I would like to describe what I consider to be the top sites for Landsat satellite imagery in terms of content and ease of use.  Let’s limit it to the Top 12.  Why might such a list be helpful?  First, there is no “one single site” to obtain Landsat data, and second, the sites are in continual flux, with some such as the Global Land Cover Facility disappearing and some having recently been created.  As with any consideration of data portals, make sure you have done a careful assessment of your data needs–band combinations, resolutions, formats, streamed services vs. downloaded files, dates, how many files you need, and so on, to guide you before you start searching.

(1)  The DevelopmentSeed’s Libra Portal.  I recently used this resource to include in the update (to ArcGIS Pro) for the Brazil land use change lesson that we host on the Spatial Reserves set of 10 hands-on exercises.  We wrote about the Libra portal here, and it remains in my judgment a no-nonsense resource that is easy to use with a wealth of options and data.

(2)  The EOS Data Analytics Landviewer, as we described here, is very useful and user friendly.   The EOS staff also wrote this helpful review of imagery sites.  Like the DevelopmentSeed portal, I find its user interface to be very straightforward.   The Landviewer includes Sentinel-2 and other imagery, as well.

(3)  Esri’s ArcGIS Living Atlas of the World has made amazing strides in content and usability since we first wrote about it here.  Most of Esri’s ArcGIS Living Atlas data is provided as streaming services instead of download, but for an increasing number of workflows, this is actually perfect.  The Living Atlas has in a few short years become probably the largest collection of spatial data on the planet, and so I recommend keeping it in mind not just for satellite imagery, but vector data as well, some of which can be downloaded, and all of it can be streamed.  Plus, you can contribute your organization’s data to the Living Atlas.  On a related note, be sure to check ArcGIS Online for imagery as well, the web GIS platform that Esri’s ArcGIS Living Atlas is based on.

(4)  The Esri Landsat Thematic Bands Web Mapping Application.   As we described in this post, through this application, you can access a variety of up-to-date and historical images in various band combinations, and save specific configurations and locations to share with others.

(5)  The USGS Earth Explorer.  While the Earth Explorer is in my view in need of improvement from the user’s perspective, it is functional and does contain a wealth of data, and sometimes is the best source for specific image sets.

(6)  The USGS Landsat Look Viewer.  I prefer the Landsat Look viewer’s interface over the Earth Explorer, as I described here.

(7)  The USGS GloVIS viewer.  I also prefer this interface over Earth Explorer.  GloVIS dates back to 2001 and was redesigned in 2017.

(8)  Landsat 8 archive on Amazon AWS.  As we described here, this has emerged as an amazing archive of data.  The user, as one might expect, is faced with a list of files rather than a fancy User Interface, but sometimes accessing specific files is exactly what one needs.

(9)  Landsat archive in Google Cloud.   Like the AWS experience, the UI is spartan but its data sets are vast, which is what one would expect from Google.

(10)  The FAO GeoNetwork.  This site focuses on vector data sets, but its raster holdings include many useful Landsat mosaics for specific geographic areas such as countries.

(11) Remote Pixel.  This is incredibly easy to use and blazing fast to zoom, pan, and query, and largely the work of a single individual.  It is my hope that if its developer does not maintain it in the future, that someone else will, because it is so marvelous.  Fortunately, the developer shows others how to host something like this themselves.

(12)  The Copernicus Open Data Access Hub, as its name implies, focuses on Sentinel data, but if you are interested in Landsat imagery, you probably are interested in other imagery as well.

satellite_image_portals_collageA few of the image portals described in this article.   We look forward to your feedback!

–Joseph Kerski

Imagery: It is what it is: Well, not always.

May 12, 2019 5 comments

Being critical of all data, including geospatial data, is a chief theme of this blog and our book:  Use it wisely.  Know where it came from.  Know what has been done to it. Imagery has always been an important component of geospatial data, and today, it takes many forms, ranging from images taken on the ground with a phone or camera to those taken by webcams, UAVs, aircraft, satellites, and more.  Doesn’t this imagery “tell it like it is” and is it therefore exempt from us casting a critical eye on it?   I contend that no, images must be viewed critically so that you can determine if they are fit for your use in whatever project you are working on.  For more, read below and see my video here.

As one example in a larger story, consider the example of the removal of individuals from 20th Century photographs in the USSR. But lest we think such image doctoring was a thing of the past, there are dozens of articles that arise in current web searches highlighting modern day image manipulation, from fashion magazines to science journals, some inadvertent, some on purpose, some for purposes to benefit society and some unfortunately to mislead.  In the geospatial world, consider the image offsets on Google Maps in China that we wrote about not long ago.

Let’s discuss another aspect of image manipulation.  Consider the satellite images below.  They cover many different dates and are derived from many different sources.  But pay attention to what’s on–or off-the highways in the images.   These images all cover the same location—Interstate 30 – 35E interchange on the southwest side of downtown Dallas Texas USA, known locally as the Mixmaster.  Constructed between 1958 and 1962, you can spot, in some images, recent much-needed improvements due to the substantial increase in traffic volume in the past 60 years.

arcgis_online_dallas..PNGmapquest_dallas.png

Study area in ArcGIS Online (first image) and Mapquest (second image).

yahoo_maps_dallas.PNGbing_dallas.PNG

Study area in Yahoo maps (first image) and Bing maps (second image).

gmaps_web_dallas

Study area in Google Maps (it appears the same way in Google Earth). 

Notice anything missing on the Google Maps image?  No traffic. This would be unheard of, day or night, in this busy part of one of the largest cities in the USA.  Obviously, a vehicle-removal algorithm has been applied to the image. It is actually quite impressive, because the algorithm seems to only be applied to moving vehicles–cars, buses, trains–and not to stationary vehicles.  No doubt it is applied to make the process of viewing and interpreting the now-clearer imagery easier.  Interestingly, the moving-vehicle removal seems to only, at the present time, be present in running Google Maps on the web on a tablet or laptop:  Running the Google Maps app on a smartphone shows moving vehicles (below).

gmaps_dallas_app_on_phone.png

Study area in Google Maps app on smartphone. 

Key takeaway points and teachable moments here include:  (1) Images that can be obtained in web mapping services, in geodatabases, and portals should not be considered the same view that would be seen if someone were to be peering over the side of a UAV, airplane, spacecraft, or standing next to a webcam when the image was captured.  They can and are manipulated for a variety of purposes.   They need to be viewed critically.  At the same time, we can marvel and appreciate the many choices we have nowadays in imagery for use in geospatial analysis.   (2)  The machine learning and artificial intelligence techniques applied to imagery are further evidence for the enterprise nature of geotechnologies:  GIS is increasingly connected to mainstream IT trends and innovations.

–Joseph Kerski

Global Land Cover Facility goes offline

January 7, 2019 2 comments

The world of geospatial data portals is dynamic; new sites appear and others disappear.  Sites are shut down due to the end of a funding period, changes in technologies, or as a result of mission or personnel changes. One of the earliest and most useful sites particularly for remotely sensed imagery recently went off-line–the Global Land Cover Facility from the University of Maryland.  Their notice said, “The GLCF has had a very good run since 1997! Originally it was funded under NASA’s Earth Science Information Partnership (under the inspired leadership of Martha Maiden of NASA). Subsequently it was maintained to support our NASA-funded research activities especially those concerned with Landsat data.   We feel we have attained what we wanted to accomplish, and now it’s time for us to move on and explorer other ventures. The data and services provided by GLCF are now mostly available via government agencies, especially USGS and NASA.”

To expand on the last note above, what should you, the GIS user who loves imagery, do?  For the time being, the GLCF data are still on a no-graphics FTP site, here:  ftp://ftp.glcf.umd.edu/.   But better yet, we have examined numerous functioning imagery portals on this blog; start here.  These include, for example, LandViewer, EOS Data Analytics, NASA AVIRIS, the GeoPortal, Lidar from USGS, DevelopmentSeed, Sentinel-2, and many others.

glcf

Thank you, GLCF!  You provided a wonderful service, and will be missed. 

–Joseph Kerski

Hangar: A new on-demand UAV Data Service

October 28, 2018 2 comments

We have written about the rapid evolution of imagery platforms and portals many times in these blog essays over the years.  One major recent advancement is the UAV service from Hangar, an Esri business partner.  This service allows data users to order UAV imagery and receive it according to their project specifications:
https://www.spar3d.com/news/uav-uas/hangar-esri-reality-data-arcgis/

In my way of thinking, it is sort of like an” Uber for Drones”.  Let’s say you don’t have a pilot’s license, or time, or equipment, or expertise to fly your own UAV imagery.  Hangar is a new UAV service covering all areas that may be of interest to a client requiring imagery.  For this service, Esri partners with Hangar, a company that holds hundreds of waivers to fly almost anywhere and the expertise and equipment to serve clients from just about any discipline and with any need.   For more information, read the article “Hangar Joins Esri Startup Program to Add ‘Task & Receive’ Aerial Insights ArcGIS:”   https://www.prweb.com/releases/2018/05/prweb15514744.htm. 

And two of the best examples of some of the Hangar imagery is this story map of the devastation from the Carr fire in California and this story map showing some of their imagery for Kilauea, shown below.   Be sure to zoom and pan the 360-degree UAV imagery shown in these story maps.  Warning!  They are highly addictive and fascinating.  And for those of us in education, they make for an attention-getting teaching tools which I have already used numerous times from primary school to university and beyond to teach about wildfires and volcanic hazards.

uav_hangar.JPG

-Joseph Kerski

Categories: Public Domain Data Tags: , ,

Historical Imagery for the entire world now available via Wayback Service in ArcGIS from Esri

July 5, 2018 2 comments

I know that many of you regularly want to examine changes-over-space-and-time with imagery and GIS for research or instruction purposes.   As of last week, 81 different dates of historical imagery for the past 5 years now reside in ArcGIS via the World Imagery Wayback service.   For more information, see: https://www.esri.com/arcgis-blog/products/arcgis-living-atlas/imagery/wayback-81-flavors-of-world-imagery/

You can access this imagery in ArcGIS Online, ArcMap, and ArcGIS Pro.  A great place to start is the World Imagery Wayback app – just by using a web browser  – https://livingatlas.arcgis.com/wayback/    A fascinating and an incredible resource for examining land use and land cover change, changes in water levels of reservoirs, coastal erosion, deforestation, regrowth, urbanization, and much more.  This resource covers the entire globe.

However, in keeping with the theme of our book The GIS Guide to Public Domain Data and this blog of being critical of the data, caution is needed.  The dates represent the update of the Esri World Imagery service.  This service is fed by multiple sources, private and public, from local and global sources.  Thus, the date does not mean that every location that you examine on the image is current as of that date.  I verified this in several locations where my ground observations in my local area show construction as of June 2018, for example, but that construction does not appear on the image.  In addition, several other places I examined from wintertime in the Northern Hemisphere were clearly “leaf-on” and taken during the summer before, or even from the summer before that.  Therefore, as always, know what you are working with.  Despite these cautions, the imagery still represents an amazing and useful resource.

wayback1wayback2

Sample from this imagery set for 30 July 2014 (top) and four years later, 27 June 2018 (bottom) for an area outside Denver, Colorado USA.