The Top 10 Most Useful Geospatial Data Portals, Revisited

We have been writing this geospatial data column for 7 years now, beginning when our book The GIS Guide to Public Domain Data, was published.  Over those years, in addition to keeping issues such as data quality, copyright, privacy, and fee vs. free at the forefront of the conversation, we have tested and reviewed many geospatial data portals.  Some of these portals promise more than they deliver, some have been frustrating, but many have been extremely valuable in GIS work.  Back in 2017 we listed 10 of those that we have found most useful, rich with content, easy to use, and with metadata that is available and understandable.  A few are no longer functioning, and a few have emerged that merit inclusion in the top 10 list.  In creating such a list, we realize that “most useful” really depends on the application that one is using GIS for, but the list below should be useful for GIS users across many disciplines. Some allow for data to be streamed from web servers into your GIS software, and all allow data to be downloaded.

1. The Open Data portal based on ArcGIS Hub technology.   This portal’s simple “what” and “where” interface is the entry point to a vast, curated, and growing list of valuable open data sites, along with a helpful story map described here.
2. The Esri Living Atlas of the World is an expanding, curated set of data and maps on thousands of topics that can be used and also contributed to by the GIS community.
3. The European Space Agency’s Sentinel Online data portal includes a wide variety of image-related data sets on the five themes of land, marine, atmosphere, emergency, and security.
4. CIESIN at Columbia University has been serving data for over 20 years on climate, population, soil, econonics, land use, biodiversity, and other themes, including its Socioeconomic Data and Applications Center (SEDAC).
5. The Atlas of the Biosphere serves global data, largely in grid format, of human impact, land use, ecosystems, and water resources themes.
6. Natural Earth is a public domain dataset at small scale (1:10,000,000, 1:50,000,000, and 1:110,000,000) for the globe, in vector and raster formats that are easily ingestible in GIS software.
7. The World Resources Institute hosts a variety of data geospatial data sets for specific areas of the world, such as Kenya and Uganda.
8. The FAO GeoNetwork.  This portal contains global to regional scale data from administrative  boundaries and agriculture to soils, population, land use, and water resources.
9. OpenTopography.   This NSF data facility from UC San Diego focuses on “Earth science-related, research-grade, topography and bathymetry data”, including a mountain-load of Lidar data.
10. Many “lists of data sites” have appeared over the years.  Most are not kept up to date and end up being less useful over time.  However, those that are still quite helpful that we have reviewed are Dr Karen Payne’s list from the University of Georgia, and Robin Wilson’s list of free spatial data.  A few others that are useful are this list from the USGS that I started back when I worked at that organization, and this list from Stanford University.

A few others “almost make the top 10” :  The National Map from the USGS, data.gov from the US Government (though I am still frustrated that they removed the zebra mussels data that I used to access all the time), environmental and population data from TerraPopulus, Diva-GIS’s data layers for each country, the UNEP Environmental Data Explorer, the NEO site at NASA Earth Observations, and OpenStreetMap (which besides roads, also includes buildings, land use, railroads, and, waterways)

For more details on any of these resources, search the Spatial Reserves blog for our reviews, remain diligent about being critical of the data you are considering using, and as always, we welcome your feedback.

Working with Lidar data obtained from the USGS National Map data portal. 

–Joseph Kerski

Key statements about the importance of spatial data

Sometimes it is helpful to have some research results and quotes in support your data advocacy efforts at your own organization–in your promotion of  “why this all matters!”.   And, of course, why your efforts need to be funded and supported!  Here are a few key quotes about the importance of spatial data–and what happens when the data doesn’t exist.

Kathryn Sullivan, former NASA astronaut, recently commented, The power of a map to put time and place and phenomena together, to give it to our brains through the most potent input sensor human beings have — our eyes — is a remarkable accelerator for the comprehension and engagement and use of the data that tell us what’s on Earth, where are things happening on our planet …   as reported in Cheney, S. (2017). How GIS can help us understand our changing oceans.  (quoted on the 2017 Esri Ocean GIS Forum on https://www.devex.com/news/how-gis-can-help-us-understand-our-changing-oceans-91366).

Another extremely useful statement is, “Advances in research on resilience and vulnerability are hampered by access to reliable data” can be found in Barrett, C. B. and D.D. Headey. 2014. Measuring resilience in a risky world: Why, where, how, and who? 2020 Conference Brief 1. May 17-19, Addis Ababa, Ethiopia. Washington, D.C: International Food Policy Research Institute.

“The lack of data is one of the biggest obstacles to progress toward development goals” was a part of the statements from the United Nations Independent Expert Advisory Group (UN). 2014. A World That Counts: Mobilising the Data Revolution for Sustainable Development. A Report to the UN Secretary General. New York, NY: United Nations, pp.28.

Perhaps the strongest argument for more and better data comes from ODI’s report The Data Revolution:  Finding the Missing Millions, where the authors cite numerous cases where inadequate data leads to sub-optimal policy decisions.  These cases “confirm some of the anecdotal evidence about the lack of good data in developing country ministries.”    The full citation is:  Stuart, E., E. Samman, W. Avis, and T. Berliner. 2015. The data revolution: finding the missing millions. ODI Research Report 03. London: Overseas Development Institute, pp. 51. (https://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/9604.pdf).

Based on the research that Jill Clark and I have done in this area over the past decade, I would add to the ODI statement that in developed countries, some similar challenges exist.  We have documented those in these blog essays for six years.

I sometimes use the statements from this National Academies of Sciences report and these written by vterrain.org.  I have created videos on this topic such as here and articles such as in Directions Magazine here.

My own contribution to these quotes is, “We have made much progress, to be sure, but the world’s increasingly complex and serious issues are not going to wait around another generation for us to get our data act together.”

What are the quotes and studies you are using in your own data advocacy efforts?  Please share those in the comments section.

Photograph by Joseph Kerski.

 

Fantasy island

A widely reported story, courtesy of the BBC, appeared last week describing the unusual case of Sandy Island, a south Pacific island between Australia and New Caledonia.  Although the island appeared on Google Maps and Google Earth, a research team from Australia were unable to locate it when they went to investigate. Expecting to find a ‘sizeable’ strip of land, the researchers found instead 1,400 m of deep blue sea. In their defence Google commented that they did consult a number of authoritative data sources for their maps, and the island did appear to be a genuine feature.

It seems the island never really existed and was most likely the product of an error that had gone unnoticed and had been perpetuated over the years.  This story touched on a number of issues we discussed in The GIS Guide to Public Domain Data including the use of assertive versus authoritative data sources. What does authoritative really mean and how far should we go to get the definitive answer?  Should we no longer rely solely on the traditional sources of geographic data when it seems even they can’t be guaranteed, and always get a second opinion? Prof. Michael Goodchild (Univ. of California, Santa Barbara) discusses a hybrid solution in a paper entitled ‘Assertion and Authority: The Science of User Generated Geographic Content’  and sees the merits in taking advantage of the expertise and knowledge accumulated by the traditional data providers (national mapping agencies, survey companies and so on) but also taking advantage of independent verification from volunteer groups, research teams and other interested individuals and organisations.

Future developments in data capture and verification will probably mean cases like this should be rare. However given the rate of change in both the physical and man-made environment and the ever-present possibility of mis-interpretation, the  ‘definitive map’ will probably remain an elusive goal.