Welcome to the Spatial Reserves blog.
The GIS Guide to Public Domain Data was written to provide GIS practitioners and instructors with the essential skills to find, acquire, format, and analyze public domain spatial data. Some of the themes discussed in the book include open data access and spatial law, the importance of metadata, the fee vs. free debate, data and national security, the efficacy of spatial data infrastructures, the impact of cloud computing and the emergence of the GIS-as-a-Service (GaaS) business model. Recent technological innovations have radically altered how both data users and data providers work with spatial information to help address a diverse range of social, economic and environmental issues.
This blog was established to follow up on some of these themes, promote a discussion of the issues raised, and host a copy of the exercises that accompany the book. This story board provides a brief description of the exercises.
The popular music streaming service Spotify recently announced an updated set of terms and conditions. In addition to stated intentions to access contact information and photographs stored on a mobile device, for those using the Spotify Running feature the service would also collect location data.
Depending on the type of device that you use to interact with the Service and your settings, we may also collect information about your location based on, for example, your phone’s GPS location or other forms of locating mobile devices (e.g., Bluetooth). We may also collect sensor data (e.g., data about the speed of your movements, such as whether you are running, walking, or in transit). – Spotify
For many service providers such as Spotify, personal location information is just one of the data sources they can tap in to provide a more personal service and Spotify are not the first service to want access. The trade off we have as consumers of those services is does the service we want to use justify trading some of our personal information. Before consenting to that trade off we need to understand how, what, and when data is collected, who will use it (third party access?) and if we can opt out of sharing this information.
In responding to some of the negative feedback to the announcement from users concerned about how their personal information was being used, Spotify acknowledged that they didn’t do a good job communicating the updated terms and conditions and what they meant to say was…..
Location: We will never gather or use the location of your mobile device without your explicit permission. We would use it to help personalize recommendations or to keep you up to date about music trending in your area. And if you choose to share location information but later change your mind, you will always have the ability to stop sharing. – Spotify
For some the new personal service will appeal, for others it will be a tracked step too far. The important thing for all users is sufficient information to make an informed decision before accepting the new terms and conditions.
I have taught numerous workshops using Lyme Disease case counts from 1992 to 1998 by town in the state of Rhode Island. I began with an Excel spreadsheet and used Esri Maps for Office to map and publish the data to ArcGIS Online. The results are here.
As the first decade of the 2000s came to a close, my colleague and I wanted to update the data with information from 1999 to the present, and so we contacted the people at the Rhode Island Department of Health. They not only provided the updated data, for which we were grateful, but they also provided valuable information about the data. This information has wider implications for data quality in general that we frequently discuss on this Spatial Reserves blog.
The Public Health staff told us that the Lyme disease surveillance is time and resource intensive. During the 1980s and 1990s, as funding and human resource capacity allowed, the state ramped up surveillance activities including robust outreach to healthcare providers. Prioritizing Lyme surveillance allowed the state to obtain detailed clinical information for a large number of cases and classify them appropriately. The decrease observed in the 2004-2005 case counts was due to personnel changes and a shift in strategy for Lyme surveillance. Resource and priority changes reduced their active provider follow up. As a result, in the years since 2004, the state has been reporting fewer cases than in the past. They believe this decrease in cases is a result of changes to surveillance activities and not to a change in the incidence of disease in Rhode Island.
If this isn’t the perfect example of “know your data”, I don’t know what is. If one did not know the above information, an erroneous conclusion about the spatial and temporal patterns of Lyme disease would surely have occurred. This kind of information often does not make it into standard metadata forms. This therefore is also a reminder that contacting the data provider is often the most helpful way of obtaining the “inside scoop” on how the data was gathered. I created a video highlighting these points. And rest assured that we made certain that this information was included in the metadata when we served this updated information.
Recent updates to Esri’s ArcGIS Editor for OpenStreetMap (OSM) add-on and a new OSM edit option in GitHub highlight the continuing popularity of OSM as one of the go-to base layers for many online mapping applications. In April this year, Esri announced the release of ArcGIS Editor for OSM 10.3x, providing an updated free and open source desktop toolset to download, edit and publish updates to OSM.
Two years ago we wrote about the then new option to upload and visualise geoJSON format spatial data in GitHub against a base map provided by OSM. GitHub have now extended the options for viewing and collaborating on spatial data sets to include the base map itself, with a new option to improve the underlying map for registered GitHub and OSM users.
Registered users can either edit the base map themselves or for those who haven’t registered with OSM, leave a note for another editor to review and resolve. Use of the OSM data remains subject to the terms and conditions of the Open Data Common Open Data Licence.
A theme running throughout our book The GIS Guide to Public Domain Data is to be critical of the data that you are using–even data that you are creating. Thanks to mobile technologies and the evolution of GIS to a Software as a Service (SaaS) model, anyone can create spatial data, even from a smartphone, and upload it into the GIS cloud for anyone to use. This has led to incredibly useful collaborations such as Open Street Map, but this ease of data creation means that caution must be employed more than ever before, as I explain in this video.
For example, analyze a map that I created using Motion X GPS on an iPhone and mapped using ArcGIS Online. It is shown below, or you can interact with the original map if you prefer. To do so, access www.arcgis.com/home (ArcGIS Online) and search for the map entitled “Kendrick Reservoir Motion X GPS Track” or go directly to http://bit.ly/Rx2qVp. Open the map. This map shows a track that I collected around Kendrick Reservoir in Colorado USA. This map was symbolized on the time of GPS collection, from yellow to gradually darker blue dots as time passed.
Note the components of the track to the northwest of the reservoir. These pieces were generated when the smartphone was just turned on and the track first began, indicated by their yellow color. They are erroneous segments and track points. Notice how the track cuts across the terrain and does not follow city streets or sidewalks. Change the base map to a satellite image. Cutting across lots would not have been possible on foot given the fences and houses obstructing the path. When I first turned on the smartphone, not many GPS satellites were in view of the phone. As I kept walking and remained outside, the phone recorded a greater number of GPS satellites, and as the number of satellites increased, the triangulation was enhanced, and the positional accuracy improved until the track points mapped closely represented my true position on the Earth’s surface.
Use the distance tool in ArcGIS Online to answer the following question: How far were the farthest erroneous pieces from the lake? Although it depends on where you measure from, some of the farthest erroneous pieces were 600 meters from the lake. Click on each dot to access the date and time each track point was collected. How long did the erroneous collection continue? Again, it depends on which points you select, but the erroneous components lasted about 10 minutes. At what time did the erroneous track begin correctly following my walk around the lake? This occurred at 11:12 a.m. on the day of the walk. [Take note of the letters I drew along the southwest shore of the reservoir!]
This simple example points to the serious concern about the consequences of using data without being critical of its source, spatial accuracy, precision, lineage, date, collection scale, methods of collection, and other considerations. Be critical of the data, even when it is your own!
The Un-Spider Knowledge Portal (United Nations Platform for Space-based information for Disaster Management and Emergency Response) recently reported the launch of the Bhuvan Ganga web portal and the Bhuvan Ganga mobile application. This new monitoring initiative will use existing geospatial information and crowd-sourced reporting to monitor pollution levels in the River Ganga (Ganges). The data portal already provides access to a variety of geospatial information including as flood hazard zones and environmental data and visitors to the site will be able to contribute to the project by uploading shapefiles and WMS layers. The accompanying mobile app will also allow users to collect and report information on pollution sources affecting water quality in the River Ganga basin.
The host geospatial platform, Bhuvan, was one of the projects we discussed in The GIS Guide to Public Domain Data. Impressed by geospatial resources such as Google Earth but concerned about potential misuses of the information following the terrorist attacks in Mumbai in 2008, the Indian Government launched its own version, describing Bhuvan as a gateway to the geospatial world. The benefits of providing open access to national, regional and local geospatial information outweighed lingering concerns over potential future attacks. Over the last seven years the site has developed into a comprehensive resource of geospatial datasets and services.
One of the most robust data portals is The Open Geoportal (OGP). It is a collaboratively developed, open source, federated web application framework to rapidly discover, preview and retrieve geospatial data from multiple curated repositories. The Open Geoportal Federation is a community of geospatial professionals, developers, information architects, librarians, metadata specialists and enthusiasts working together to make geospatial data and maps available on the web and contribute to global spatial data infrastructure. Patrick Florance at Tufts University and others have been diligently working to make this resource one that will be valued and useful for the GIS community for years to come. The project’s code repository is hosted on github. Documentation can be found here. To search the repository, you can enter information using the “where” and/or “what” search fields or zoom in on a location using the map,
Like any large data depository, this one takes some getting used to–but I found it to be straightforward: You enter where you are interested in searching, and what you are interested in searching for. Where and What: It doesn’t get much more straightforward than that. The only thing I could not get to work was the “Help” link on the page. After selecting and viewing your data on the map, you add it to a Cart. The Cart acts like something you would see on Amazon, and you can add to it and delete from it as you are searching, which I found to be quite convenient. Another nice touch is that you can adjust the symbology of the data that you are examining on the map before you download it. Even better, you can stream web services directly to your desktop, web, or mobile applications from the Cart. After you have made your selections, you access your Cart, whereupon you are presented with download options. If a layer is restricting by licensing agreement, you can add them to the cart but you must log in to preview or download restricted layers. Spending time with the OpenGeoportal will be well worth it given its ease of use, but moreso for the thousands of international data layers accessible here.
Additional tools that the OpenGeoPortal community is in the process of building include a Harvester–an open source web application that provides the automation of customized harvesting from partner metadata nodes and XML metadata files within a web or local directory. Also in progress is a Metadata Toolkit–a publicly available website that provides tools to easily create guided, geospatial metadata, and a Dashboard to analyze and visualize massive spatial data collections.