Welcome to the Spatial Reserves blog.
The GIS Guide to Public Domain Data was written to provide GIS practitioners and instructors with the essential skills to find, acquire, format, and analyze public domain spatial data. Some of the themes discussed in the book include open data access and spatial law, the importance of metadata, the fee vs. free debate, data and national security, the efficacy of spatial data infrastructures, the impact of cloud computing and the emergence of the GIS-as-a-Service (GaaS) business model. Recent technological innovations have radically altered how both data users and data providers work with spatial information to help address a diverse range of social, economic and environmental issues.
This blog was established to follow up on some of these themes, promote a discussion of the issues raised, and host a copy of the exercises that accompany the book. This story map provides a brief description of the exercises.
In this blog and in our book, we have reviewed many geospatial data portals. One of the oldest and yet most useful of all regional or state portals is that of the Texas Natural Resources Information System, or TNRIS. Indeed, TNRIS predated digital spatial data, for it was founded in 1968, housing paper topographic and geologic maps and aerial photographs for years before hosting digital spatial data. TNRIS allows searches by county or by data theme. If one zooms in on a the statewide map with county boundaries, the familiar USGS 7.5-minute grid is displayed, from which one can download such data as digital raster graphics, elevation, wetlands, geology, and historical and current satellite imagery. Statewide themes include bathymetry, land cover, soils, census data, transportation, and many others. Metadata is not only available but it is conveniently packaged, and the site doesn’t burden the data user with needless frills and fancy ways to download–it is, in my view, what a data portal should be–with the ability to quickly go in and get what one needs, in a variety of formats.
As GIS technologies have evolved, the TNRIS portal has evolved as well. One of the most innovative and useful sections of their site is its online mapping services. Here, high resolution imagery (30 cm in many places), land cover, and other themes are hosted as ArcGIS services and OGC WMS services. The site conveniently enables the data user to preview the services on their website or to copy the URL for the service so that it may be used in ArcGIS Online. Therefore, not everything from TNRIS needs to be downloaded–a growing amount can be streamed.
Texas is an excellent location for other useful data portals as well: The General Land Office hosts data on habitat, minerals, oil and gas, and other themes. The Texas Commission on Environmental Quality hosts data on air and water quality, toxic hazards, and other layers. Texas Parks and Wildlife hosts data on bays, ecosystems, trails, and wildlife management areas. And other gems exist, such as the railroads and other data hosted by Entergy on the Texas Site Selection Center.
A section of the Texas Natural Resources Information Systems geospatial data portal.
We have been writing this geospatial data column since 2012, when our book The GIS Guide to Public Domain Data, was published. Over the 5 years that have elapsed, in addition to keeping issues such as data quality, copyright, privacy, and fee vs. free at the forefront of the conversation, we have tested and reviewed many geospatial data portals. Some of these portals promise more than they deliver, some have been frustrating, but some have been extremely valuable in GIS work. We have decided to list 10 of those that we have found most useful, rich with content, easy to use, and with metadata that is available and understandable. In considering such a list, we realize that “most useful” really depends on the application that one is using GIS for, but the following sites should be useful for users in many disciplines. Some allow for data to be streamed from web servers into your GIS software, and all allow data to be downloaded.
- The FAO GeoNetwork. This portal contains global to regional scale data from administrative boundaries and agriculture to soils, population, land use, and water resources.
- The Esri Living Atlas of the World is an expanding, curated set of data and maps on thousands of topics that can be used and also contributed to by the GIS community.
- The European Space Agency’s Sentinel Online data portal includes a wide variety of image-related data sets on the five themes of land, marine, atmosphere, emergency, and security.
- CIESIN at Columbia University has been serving data for over 20 years on climate, population, soil, econonics, land use, biodiversity, and other themes, including its Socioeconomic Data and Applications Center (SEDAC).
- The Global Land Cover Facility at the University of Maryland remains one of the best and easiest to use sources and methods to obtain Landsat, MODIS, Aster, SRTM, and other satellite imagery.
- The Atlas of the Biosphere serves global data, largely in grid format, of human impact, land use, ecosystems, and water resources themes.
- Natural Earth is a public domain dataset at small scale (1:10,000,000, 1:50,000,000, and 1:110,000,000) for the globe, in vector and raster formats that are easily ingestible in GIS software.
- The World Resources Institute hosts a variety of data geospatial data sets for specific areas of the world, such as Kenya and Uganda.
- The GIS Data Depot from the GeoCommunity is one of the oldest data depositories, dating back to the 1990s, but still very useful for international and USA specific data on such themes as elevation, transportation, imagery, scanned topographic maps, and hydrography, many of which have been re-served from more-difficult-to-use government sites.
- There have been many “lists of data sites” over the years, and these invariably are not kept up to date and end up being less useful over time. However, those that are still quite helpful that we have reviewed are Dr Karen Payne’s list from the University of Georgia, and Robin Wilson’s list of free spatial data. A few others that are useful are this list from the USGS that I started back when I worked there, and this list from Stanford University.
A few selected others are also useful that “almost make the top 10” above are The National Map from the USGS, data.gov from the US Government, environmental and population data from TerraPopulus, Diva-GIS’s data layers for each country, the UNEP Environmental Data Explorer, the NEO site at NASA Earth Observations, and OpenStreetMap.
For more details on any of these resources, search the Spatial Reserves blog for our reviews, remain diligent about being critical of the data you are considering using, and as always, we welcome your feedback.
A Top 10 List of Useful Geospatial Data Portals.
An article co-authored by Benjamin Pross, Christoph Stasch, and Albert Remke, of the 52°North Initiative for Geospatial Open Source Software GmbH; and Satish Sankaran and Marten Hogeweg of Esri describes a development that should interest anyone who uses geospatial data. The 52°North Initiative for Geospatial Open Source Software has developed an open-source extension to ArcGIS for Desktop that enables access to Open Geospatial Consortium, Inc. (OGC), Web Processing Services (WPS). The result? This initiative makes it possible for these services to be used in the same manner as native ArcGIS geoprocessing tools. In other words, they appear in the list of tools just as a standard buffer or overlay tool would appear. Yes, it could be just that easy.
The article explains that “while ArcGIS allows geoprocessing tools to be published as a WPS, [ArcGIS] does not offer a WPS client interface. Consequently, it is not easy to access external non-ArcGIS geoprocessing tools such as simulation models, rich data interfaces, or processing capabilities from any other legacy software that supports the WPS interface.” This points to the reason why this initiative offers such promise: “The 52°North Extensible WPS Client for ArcMap was implemented as an open-source extension to ArcGIS that fully integrates into the ArcGIS for Desktop environment. It enables OGC WPS to be accessed and used in the same manner as native ArcGIS geoprocessing tools. This makes it easy to run WPS-based processes and integrate the results of that processing into ArcMap for use with other applications.”
In plain language, because the complex issues grappled with by GIS analysts often require major investments of time to generate models, services, and customized workflows and code, why should each analyst have to create all of this from scratch? An enormous time savings could be realized if there was an easy way to share these things. This article both explains recent progress in this area but also encourages the community to think creatively about how to pursue further collaborative methods.
Despite the growing volume of geospatial data available, and the ease of use of much of this data, finding and using data remains a challenge. To assist data users in these ongoing challenges, I have written a new activity entitled “Key Strategies for Finding Content and Understanding What You’ve Found.” The goal of this activity ” Key Strategies for Finding and Using Spatial Data” is to enable GIS data users to understand what spatial analysis is, effectively find spatial data, use spatial data, and become familiar with the ArcGIS platform in the process. I tested the activity with a group of GIS educators and now would like to share it with the broader GIS community.
The document makes it clear that we are still in a hybrid world–still needing to download some data for our work in GIS, but increasingly able to stream data from online data services such as those in ArcGIS Online. But these concepts don’t make as much sense unless one actually practices doing this–hence the activity.
In the activity, I ask the user to first practice search strategies in ArcGIS Online, using tags and keywords. Then, I guide the user through the process of downloading and using a CSV file with real-time data. After a brief review of data types and resources, I guide the user of the activity through the process of downloading data from a local government agency to solve a problem about flood hazards. The next step asks users to compare this process of downloading data with streaming the same data from the same local government’s site (in this case, using data from Boulder County, Colorado) into ArcGIS Online. The activity concludes with resources to discover more about these methods of accessing data.
Jill Clark and I have created other hands-on activities on this theme of finding and understanding data as well, available here. We look forward to hearing your comments and I hope this new activity is useful.
Over the last four years we have discussed some of the many challenges posed by the volume of data now available online – issues of quality, determining provenance, privacy, identifying the most appropriate source for particular requirements and so on. Being overwhelmed by the choice of data available or not always knowing what resources are available or where to start looking have been common responses from geospatial students and practitioners alike.
A recent report from the BBC on laser technology highlighted some current and future applications that have or will transform geospatial data capture, including the use of LiDAR and ultra precise atom interferometers that could be used to develop alternate navigation systems that do not rely on GPS. The article also discusses the inherent limitations of our current electronics-based computing infrastructure and the potential of silicon photonics, firing lasers down optical fibres, to help meet the demand for instant or near-instant access to data in the Internet-of-Everything world. If many feel overwhelmed now by the volumes of data available, what will technologies like silicon photonics mean for data practitioners in the future? Just because data may be available at unprecedented speeds and accessed more easily, that alone doesn’t guarantee the quality of the data will be any better or negate current concerns with respect to issues such as locational privacy. A critical understanding of these issues will be even more important if we are to make the most of these advances in digital data capture and transmission.
We wrote extensive reviews of local, regional, state, provincial, national, and international government data portals in our book and from time to time do so in this blog. One of the finest state geospatial data portals in our judgment is the Montana Digital Atlas.
We have been critical here and in our book about data portals that were obviously set up simply to satisfy some organizational mandate without regard to those who will actually use the data portal. I have spent time with the MAGIP (Montana Association of Geographic Information Professionals) community, and most recently was honored to give the keynote at their annual conference. I am happy to report that they have built their data portal with the end user in mind. What’s more, the Montana State Library has been a leader in the GIS community there for years, and I have found that when library information professionals are involved–people who really understand data–their resources will be extensive, the metadata will be rich, and the services are actually going to work.
The Digital Atlas features geographic databases, aerial photos, and topographic maps of lands in Montana. The functions begin with an interactive map, where you can select base maps, thematic map layers, and tabular data from which you can draw on the screen, generate reports from in XLS and CSV formats, and download in various GIS formats. The site features functionality that I wish all data portals had, such as the ability to move popup boxes to locations most convenient for you, choices on datums and projections for your data, the ability to clip data to specific geographic areas, and the ability to search the state library catalogs for articles, books, and other resources for the area you are investigating. You can even copy the map link to get a web link to the current map that you have created on the site. Furthermore, you can load some of the layers directly from the Montana State Library to ArcGIS Online, via the services in the ‘MSDI_Framework’ and ‘MSL’ folders, on https://gisservicemt.gov/arcgis/rest/services/MSDI_Framework and https://gisservicemt.gov/arcgis/rest/services/MSL.
Lastly, the portal managers graciously say to contact them at MSLDA @ mt.gov if you experience difficulty with the application, if you see a problem with any of the data, or even–and this is wonderful–to suggest additional map layers.