Archive

Archive for September, 2020

Modern strategies for finding geospatial data

September 21, 2020 Leave a comment

What are modern best strategies for finding geospatial data?  Despite the increase in volume and variety of spatial data even in the 8 years since we have been writing this column, I still receive this question several times each week.  And for good reason–(1) data needs are often very specialized, in theme, scale, region, attributes, and other characteristics, and often it is difficult to find a thread of a similar search by someone else online; (2) the ephemeral nature of the web makes it challenging to learn from others’ strategies (hence the reason for this blog).  My advice is still specifically tailored to the data requester, but there are some elements that are common to many of my responses, which I share below in the hopes it will be helpful to others.

(1) First and foremost, have a clear vision of the data you need, the scale, the date, the extent, the resolution, and other characteristics before you spend time searching.  Also set yourself a limit on how much time you will devote to processing the data before you can use it for analysis.  This will reduce the behavior that I think is similar to “wandering through a grocery store without a clear idea of what you are looking for”.  Walking up and down aisles in a grocery store, you might take something that you really didn’t want or originally intend to pick up, and waste valuable time with it in your project.  But once you’ve thought carefully about your needs and what you will and will not accept, then you are ready to search.

(2)  While you are searching, here is an aid that we reviewed to help you decide whether the data will meet your needs.

(3) Start your search with the ArcGIS Living Atlas of the World. In my judgment, the 8,000 + layers, maps, apps, and services represent the single most useful set of curated geospatial resources in formats that are easily ingestible into GIS software, especially the ArcGIS platform.  If the data you are seeking is not in Living Atlas or not in the format that you desire, then:  (4) Go to selected “top 10” (such as satellite imagery, Landsat, and vector data portals that we have reviewed in this blog.   These include such useful tools such as  ArcGIS Hub search engines.  If these do not net fruitful results, then dig into (5) portals organized by theme that would cover your topic (such as WRIs on land use and natural resources),  (6) portals organized by location, that would cover at least in part, your study area (such as Ordnance Survey for the UK, and state, province, region, or city (examples here for Oregon, here for a large city of Los Angeles, here for the small city of Oak Hill West Virginia.

Just like there is often no “single best source” for one specific topic on the web, no single “geospatial gold mine” exists that is the end-all site for spatial data.  However, this is actually a good thing, in my opinion.  Why?  It forces the user to think critically about the data sets that user may or may not choose.  As is clear on this data blog, choices abound.  Those choices require the user to investigate and evaluate each of them.

Here is a sample data request I recently received:   We are working on a project involving drought in Colorado and we are looking for the following data:

  • Soil Moisture: Map of Soil Moisture for April 2013 – we found this source but the pixel size might be too large to be meaningful for a storymap that focuses on Colorado.
  • Precipitation: Is there a map of total precipitation for Colorado, 2013 water year (October 2012-September 2013)?
  • Streamflow: Time series of streamflow of Colorado River October 2012-September 2015 (with average streamflow marked).
  • Groundwater: Is there a spatial map of groundwater availability?
  • Reservoir Levels: Time series of Lake Powell water level for October 2012-September 2015.
  • Temperature – is there a map of temperature in April 2013 for across Colorado?
  • Agricultural Health: Is there an index for agricultural health? Would there be a time series or some data for 2013?
  • Water use: Is there a map that shows water use?

My reply included the following:  A search for spatial data in a political unit such as Colorado USA, will yield many sites, from which you will need to evaluate and make choices.  There are data portals via the Geospatial Centroid at Colorado State University,  via the Colorado Information Marketplace (data.colorado.gov), the OpenColorado data catalog, ColoradoView, and also those portals hosted by individual state agencies such as the Department of Natural Resources, the Colorado Geological Survey, and the Department of Transportation.   The sources require close evaluation to determine if they meet your needs.  Some sources are formatted as lists of data or map-based clickable links that require you to download the data to your device, while increasingly, others provide data as streaming services.  For example, this map service “DNR Viewer” from the Department of Natural Resources does not allow for the downloading of geodatabases, imagery, or shapefiles, but this data portal from the same agency does.

On the topic of streamflow:  From the USGS, this Current Water Data resource has been a longstanding favorite of mine:  https://waterdata.usgs.gov/nwis/rt.  Nowadays, however, it is also a live data feed in ArcGIS Online:  The metadata is here:  https://www.arcgis.com/home/item.html?id=4d8b9ff2d9e74ad18daecb2db7eaa87f    Examine the metadata and open it in ArcGIS Online or in ArcGIS Pro.  This is a premium service so you need to sign in to ArcGIS Online to access it, but once you do, it looks like this, below (there is also a global stream gauge layer, here):

streamflow

On the topic of groundwater:  See this report and sets of data.  Reservoirs are available here at USGS and here in the ArcGIS Living Atlas of the World, but reservoir levels are more difficult to obtain:  It may involve calling or emailing water resources agencies.  Yes, sometimes these “20th Century methods” are best!

On the topic of Agricultural Health:  I am not certain if there is a globally accepted measure for it analogous to the HDI for education-health-life expectancy from the UN.  There is, however, an “agricultural stress index” map and measure, from the FAO, here.  You might have to speak with someone at the state department of agriculture.  Here is one of my favorite agriculture-related story maps entitled “Farming for the Future“.   This story map contains links to individual maps with data layers for the type of crop.

On the topic of temperature, I recently worked with an Esri colleague on the data team, and, good news!… he created this average global air temperature layer.

On the topic of precipitation:  In conjunction with an educational project, I compiled this layer as well as other Colorado layers, and display them in this gallery.  The precipitation data came from COCORAHS – the Community Collaborative Rain Hail & Snow Network.  In terms of water use, this remains challenging, but this Living Atlas collection contains water related items.  When I worked at USGS, I made extensive use of this water use report:  https://waterdata.usgs.gov/nwis/wu.   From this report, you may need to go through this workflow: Query the system > obtain a CSV table > bring that table into ArcGIS Online or ArcGIS Pro to create a map from it.

Feel free to share your strategies, in the comments, below.

–Joseph Kerski

Categories: Public Domain Data

A review of the ArcGIS Hub Gallery

September 7, 2020 Leave a comment

As we have described in this blog, ArcGIS Hub is a set of templates and tools that allow organizations to more easily share their geospatial data holdings and also to engage the community. ArcGIS Hub has emerged over the past few years as a primary way that organizations are sharing geospatial data, and thus is germane to the theme of this blog and our book.

One of the resources that is worth frequently checking is the ArcGIS Hub gallery. The sites in this gallery range from those created by local, regional, and national governments, international agencies, nonprofits, universities, to those created by private companies. At the current time, you can search by these “sectors” but also on Focus Areas:  Transportation  |  Elections  |  Census 2020  |  Financial Transparency  |  Volunteerism  |  Open Data.

I advise “frequently checking” because the gallery is actively curated and changes often. But the resources are vast and varied!

A portion of the ArcGIS Hub Gallery.

–Joseph Kerski