Historical Imagery for the entire world now available via Wayback Service in ArcGIS from Esri

July 5, 2018 2 comments

I know that many of you regularly want to examine changes-over-space-and-time with imagery and GIS for research or instruction purposes.   As of last week, 81 different dates of historical imagery for the past 5 years now reside in ArcGIS via the World Imagery Wayback service.   For more information, see: https://www.esri.com/arcgis-blog/products/arcgis-living-atlas/imagery/wayback-81-flavors-of-world-imagery/

You can access this imagery in ArcGIS Online, ArcMap, and ArcGIS Pro.  A great place to start is the World Imagery Wayback app – just by using a web browser  – https://livingatlas.arcgis.com/wayback/    A fascinating and an incredible resource for examining land use and land cover change, changes in water levels of reservoirs, coastal erosion, deforestation, regrowth, urbanization, and much more.  This resource covers the entire globe.

However, in keeping with the theme of our book The GIS Guide to Public Domain Data and this blog of being critical of the data, caution is needed.  The dates represent the update of the Esri World Imagery service.  This service is fed by multiple sources, private and public, from local and global sources.  Thus, the date does not mean that every location that you examine on the image is current as of that date.  I verified this in several locations where my ground observations in my local area show construction as of June 2018, for example, but that construction does not appear on the image.  In addition, several other places I examined from wintertime in the Northern Hemisphere were clearly “leaf-on” and taken during the summer before, or even from the summer before that.  Therefore, as always, know what you are working with.  Despite these cautions, the imagery still represents an amazing and useful resource.

wayback1wayback2

Sample from this imagery set for 30 July 2014 (top) and four years later, 27 June 2018 (bottom) for an area outside Denver, Colorado USA. 

Advertisements

Access data and analytical tools online from EOS Data Analytics

A new platform from EOS Data Analytics (https://eos.com/platform/) allows data users not only to access data, but even perform online image processing in a web browser.  It is a set of mutually integrated cloud products for searching, analyzing, storing, and visualizing geospatial data.  This is a representation of what we have been discussing on this blog, namely, the increasing adoption of Software as a Service, and in addition, the combination of SaaS with data services and analytical services.  Thus, GIS professionals can search for, analyze, store, and visualize large amounts of geospatial data in one platform.  Doing all this in one system and also in a browser is, quite frankly, quite amazing.

With the EOS Platform, GIS users have access to an ecosystem of four mutually integrated EOS products, which together provide a powerful toolset for geospatial analysts. Image data is stored in cloud-based storage and is available for image processing or remote sensing analysis at any time; this can be a raw user file, an imagery obtained from their LandViewer data portal, or an output file from their online EOS Processing tools.  The EOS Platform is currently available for free during an open Beta.   The LandViewer tool has been freely available for some time and will continue to be.

eos1

There are at least two reasons why image processing is the platform’s major asset: the processing of large data amounts runs online and offers as many as 16 workflows with even more coming soon. On top of that, users can get the best cartographic features of EOS Vision for vector data visualization and soon to come, analysis.

eos2

A list of EOS Processing workflows that can be filtered by industry and input data type.

I found the LandViewer tool easy to use, with a wide variety of data sets to choose from, including Landsat, MODIS, NAIP, and others.

eos3.png

The LandViewer tool interface.

eos4

Choosing one’s own area of interest via the LandViewer interface.

Ordnance Survey GB to provide OS MasterMap data for free.

June 19, 2018 1 comment

The UK government announced last week that key parts of the OS MasterMap dataset (OSMM) are to be made available free of charge (see full announcement from OS). The following two datasets are due to be released under the Open Government licence (OGL) agreement:

  • OS MasterMap Topography Layer property extents
  • OSMM Topography Layer TOIDs (TOpographic IDentifiers), built into the features in the OS OpenMap-Local dataset.

In addition, a number of datasets will be made available (through an API) for free, up to a threshold, including:

  • OS MasterMap Topography Layer, including building heights and functional sites
  • OS MasterMap Greenspace Layer
  • OS MasterMap Highways Network
  • OS MasterMap Water Network Layer
  • OS Detailed Path Network

The announcement didn’t included any information on what the threshold for free access was, but no doubt details will start to filter out shortly as organisations start making use of these new data assets.

Imaging Spectrometer data from NASA AVIRIS

June 11, 2018 1 comment

The NASA AVIRIS mission has generated imaging spectrometer data for many areas of the USA since the 1990s.  The AVIRIS download portal for data from 2006 onward is on a node at NASA, here.  The AVIRIS sensor collects data that can be used for characterization of the Earth’s surface and atmosphere from geometrically coherent spectroradiometric measurements. This data can be applied to studies in the fields of oceanography, environmental science, snow hydrology, geology, volcanology, soil and land management, atmospheric and aerosol studies, agriculture, and limnology.  Applications under development include the assessment and monitoring of environmental hazards such as toxic waste, oil spills, and land/air/water pollution. With proper calibration and correction for atmospheric effects, the measurements can be converted to ground reflectance data which can then be used for quantitative characterization of surface features. In short, AVIRIS can collect in over 200 bands and therefore it can help analysts work out details such as vegetation health, or even species type, from the data.

The AVIRIS portal, presented in a Google map with popups with download links, as well as the  metadata file (in plain text format, available here)  both look very dated.  But this is a case where we encourage the user to give it a try–the portal may not look modern, but the data behind the portal is incredibly useful.   One can toggle data layers in the right hand corner of map to show All AVIRIS data or the Attrib. Filtered data (data that meets the attribute criteria but ignores the spatial filter).  One can also bound a box on the map, which has long been a favorite feature of mine on data portals, using the red rectangle to activate.  To update the spatial filter, click the “Update Map” button below the map.   The files are not streamed, but must be downloaded; perhaps because of their large size (typically over 1 GB).  Again, think “old school” formats – zip files and TAR files, but again, the data are plentiful and useful.   A set of previews are available, for example, here, and shown below.  For more information about AVIRIS data, see this link and this link.

aviris

 

The AVIRIS data portal.

aviris2.JPG

A sample AVIRIS image. 

An update on the World Bank’s Spatial Agent

May 28, 2018 5 comments
It sounds like a modern detective novel, but the Spatial Agent is actually a new, free app from the World Bank that offers one-stop access to interactive maps and charts of national, regional, and global datasets.  Jill Clark reviewed this site on our blog here. As we have written about data sources on this blog for nearly six years, and covered this topic in our book, the phrase “one-stop access” naturally caught my attention.  Could the Spatial Agent truly be all that it claims to be?

 

To find out, I began by watching a webinar that Mr Harshadeep recently conducted, which is as of this writing, still available online, here, after a short registration process.  In the webinar, and after my subsequent investigations, I was amazed at how the Spatial Agent as an app could bring together on-demand thousands of free, public-domain spatial data and analytical services (from in-situ and earth observation sources and also live cloud computing services).  It represents the data from sources such as the UN, NASA, NOAA, ESA, World Bank, many universities, and thousands of other sources, covering themes such as social (poverty, water supply), environmental (land use, biodiversity), economic (GDP, energy), and climate (snow cover, precipitation, for example).

The goal of the Spatial Agent is to offer solutions to many of the development challenges faced across the globe, which are often hampered by the poor availability of spatial data. For example, the app can be used to determine the areas in Madagascar that are susceptible to cyclones, or the areas in India that have high child malnutrition, or discovering the major exports of Vietnam, or determining how fast the population in Lagos is rising.  As these examples show, the Spatial Agent’s data cross boundaries, disciplines, and cover many different scales.  The Spatial Agent is the creation of Nagaraja Harshadeep, the lead environmental specialist and global lead for watersheds at the World Bank.  Mr Harshadeep has decades of experience working with spatial data and the application reflects his knowledge and passion.  There is much more than maps and imagery here, but rich tabular databases and other services, and the metadata for each of the data sets is quite robust.

I have been a long time fan of the spatial data from the World Bank, and use their data in several systems, including many layers available in ArcGIS Online.  The major limitation with the Spatial Agent app at this point that I can see is that it is just that — an app.  Therefore it only works on mobile phones and tablets.  I understand in part why it is focused on these devices–these are what many people are using day to day in their work.  Still, to bring the data sets into a GIS and more fully use them, I would love to see its capabilities inside of a series of user-driven interfaces that could be run in a standard web browser on a computer where I also have GIS and statistics tools available to me.  But I was glad to see this note about this very thing on the project’s site:  “The web version is being developed with the Bank’s Global Reach effort for launch later this year.”  Since the data and documentation are so rich on this site, I look forward to finding out how we will be able to use the services in a GIS.  Even without a GIS, the Spatial Agent is already very useful, because it is helping to bring data-driven decisions to daily decision making.

 

Two views of the hundreds of data layers and statistics available via the Spatial Agent.

For more information, including the links to access the apps, and the tutorials, see this page.

A review of the ArcGIS Pro Cookbook

May 14, 2018 4 comments

GIS Professional Tripp Corbin’s book, the “ArcGIS Pro 2.x Cookbook” (2018, Packt Publishing) is new but I believe will quickly become a valued and oft-used resource. Mr Corbin’s goal in writing this extensive (694 pages) resource is to help GIS professionals “create, manage, and share geographic maps, data, and analytical models using ArcGIS Pro.” The audience for this book includes all who are learning GIS, or learning Pro, as well as those migrating from ArcMap to Pro.

Tripp’s “cookbook” theme is evident throughout the book’s format, where in each section and problem to be solved, he shows how to get ready, how to do it, how it works, and … “there’s more” (additional resources). That the book is from Packt is excellent, because Packt (www.packtpub.com) offers eBook versions of every one of its books, and also offers newsletters and tech articles. That Tripp is a full time trainer and instructor is evident–he understands the challenges in learning a rapidly-changing and complex technology inherent in GIS with just enough tips to keep the reader engaged. He also encourages the reader to think about how to apply each tool and method to his or her own work. He offers the reader the ability to download the sample data for the book, and the data bundle is also on GitHub. He also includes PDFs of all images of screen shots and diagrams.

I like Tripp’s approach because, similar to my own instruction, and central to the theme of this blog, he starts with data. He’s not hesitant to discuss the benefits but also the limitations of each data format such as shp, gdb, and CAD files. He spends quality time in the book helping the reader understand how to convert data to the format that best fits his or her needs. His sections on linking tables from outside sources to existing data, on editing (in particular, a focus on topologies to improve data accuracy and increasing editing efficiency), and on 2D and 3D analysis are very helpful. I was pleased to see much attention to what I consider to be a chief advantage of Pro–the ability to more easily share content from Pro to ArcGIS Online and hence the wider community. Another wonderful new function in ArcGIS Pro is also included in the book–writing and using Arcade scripts, applied to symbology, classification, and analysis.

As a GIS book author, I know the challenges faced in writing such a book–what should be included, and what should be left out? Tripp does a nice job here as well, including the fundamentals that most users will touch. The book’s chapters include: 1: Capabilities and terminology. 2: Creating and storing data. 3: Linking data together. 4: Editing spatial and tabular data. 5: Validating and editing data with topologies. 6: Projections and coordinate systems. 7: Converting data from one format to another. 8: Proximity analysis. 9: Spatial statistics and hot spots. 10: 3D maps and 3D analyst. 11: Arcade, labeling and symbology expressions. 12: ArcGIS Online, 13: Publishing your own content to ArcGIS Online. 14: Creating web apps using ArcGIS Online.

These chapters cover a great deal of ground. In the editing chapter, for example (Chapter 4), configuring editing options, reshaping existing, splitting, merging, aligning, creating new point line polygon features, creating new polygon feature using autocomplete, and editing attributes using attribute pane and in the table view, are all examined. The examples in the book are interesting and relevant, and not without some humor (Trippville is a community that is often studied). In my view, the book contains just the right amount of graphics. Tripp provides answers to the questions he poses, and then gives the explanation for each answer. Despite the “recipes” provided in the cookbook, not all of them require the previous recipe to be used, which is excellent for all of us in GIS who have limited time and want to select sections in a non-sequential order.

I highly recommend using this book in conjunction with Tripp’s other book on this topic, “Learning ArcGIS Pro.” The Learning book focuses on installing, assigning licenses, navigating the interface, creating and managing projecrts, creating 2D and 3D maps, authoring map layouts, importing existing projects, creating standardized workflows using tasks, and automating analysis and processes using modelbuilder and python. The Learning ArcGIS Pro book ideally should be used first, before the ArcGIS Pro 2.x Cookbook, but if you are pressed for time, these two books could be used in tandem. Keep both of them handy–they will be very useful to you.

tripp4

Covers of Tripp Corbin’s ArcGIS Pro books. 

tripp3

An example of the detail provided in Tripp Corbin’s ArcGIS Pro 2.x Cookbook. 

Need access to thousands of historical aerials and topographic maps at your fingertips? Try www.historicaerials.com

April 30, 2018 2 comments

Imagine having instant access to thousands upon thousands of historic aerial photographs and topographic maps to be able to examine change over time.  Thanks to a resource called Historic Aerials, you do have this wealth of information at your fingertips.  These aerials and maps, which go back 50, 60, and even 70 years or more, can be used for research, for instruction, for planning, and for other purposes.  Being in the field of geography and GIS education, I can think of many disciplines in which this can be used — urban and rural geography most certainly, but also biology, environmental science, city planning, history, agriculture, and also in GIS courses.  These resources foster spatial thinking about changes in time and space, from natural causes, such as volcanic eruptions or changes in river meanders, or from human causes, such as urbanization or the construction of reservoirs.  And given the connection that often exists between human and natural changes, sometimes these causes are intertwined–the construction of jetties along barrier islands often influences the naturally occurring longshore sediment transport and the migration of the islands themselves, as can be seen by comparing the historic to modern aerials of Ocean City, Maryland, for example.

The interface for the Historic Aerials is intuitive and provides tools that allow the user to compare USGS topographic maps of various years as well as the aerials themselves.  In the example below, I compare a 1958 with a 2009 aerial of a section of Grand Junction Colorado, before and after Interstate 70 and some surrounding housing was constructed.  You can also use the spotlight tool to “see back in time” for wherever you pan your cursor, and you can turn on the streets to see where  streets would one day be constructed on top of historical imagery.

The site comes from the Nationwide Environmental Title Research group, which has spent over 20 years collecting the worlds largest database of historical aerial images and topographical maps of the USA.  Their sources include USGS and USDA imagery, several private collections, and they are continually acquiring more. All the imagery they collect is orthorectified to provide the data in a searchable and precise geo-locatable format.  A print or digital image (for GIS users, GeoTIFF will be especially useful, delivered in lat-long coordinates, but JPG and PNG are also offered) of any of the maps or aerials is available.  In addition, a subscription service allows anyone to access the site with the following advantages: Full screen viewer, no advertising, PDF builder, quick JPEG downloads, and multiple user accounts.   I had a pleasant chat with the good people behind the site and found that their prices are quite reasonable.  Their FAQ, forum, and tutorials make it clear that they are committed to user success with these resources, and there are human beings behind this site to help as well.

Quite frankly, during my years working at the USGS, I always dreamed that my agency would create something like this.  Kudos to the HistoricAerials staff for making this a reality!

historic_aerials

The interface for HistoricAerials.com is quite intuitive and allows for fascinating investigations back in time for lands across the USA.