Welcome

April 16, 2012 5 comments

Welcome to the Spatial Reserves blog.

The GIS Guide to Public Domain Data was written to provide GIS practitioners and instructors with the essential skills to find, acquire, format, and analyze public domain spatial data. Some of the themes discussed in the book include open data access and spatial law, the importance of metadata, the fee vs. free debate, data and national security, the efficacy of spatial data infrastructures, the impact of cloud computing and the emergence of the GIS-as-a-Service (GaaS) business model. Recent technological innovations have radically altered how both data users and data providers work with spatial information to help address a diverse range of social, economic and environmental issues.

This blog was established to follow up on some of these themes, promote a discussion of the issues raised, and host a copy of the exercises that accompany the book.  This story map provides a brief description of the exercises.

Data Quality on Live Web Maps

June 19, 2017 3 comments

Modern web maps and the cloud-based GIS tools and services upon which they are built continue to improve in richness of content and in data quality.  But as we have focused on many times in this blog and in our book, maps are representations of reality.  They are extremely useful representations, to be sure, particularly so in the cloud, but still are representations.   These representations are dependent upon the data sources, accuracy standards, map projections, completeness, processing and rendering procedures used, regulations and policies in place, and much more.  A case in point are offsets between street data and the satellite image data that I noticed in mid-2017 in Chengdu in south-central China.  The streets are about 369 meters southeast of where they appear on the satellite image (below):

china-google-maps

Puzzled, I panned the map to other locations in China.  The offsets varied, but they appeared everywhere in the country; for example, note the offset of 557 meters where a highway crosses the river at Dongyang, again to the southeast:

china-google-maps2

As of this writing, the offset appears in the same cardinal direction and only in China; indeed; After examining border towns with North Korea, Vietnam, and other countries, the offset appears to stop along those borders.  No offsets exist in Hong Kong nor in Macao.  Yahoo Maps Bing Maps both show the same types of offsets in China (Bing maps example, below):

china_bing

MapQuest, which uses an OpenStreetMap base, showed no offset.  I then tested ArcGIS Online with a satellite image base and the OpenStreetMap base, and there was no offset there, either (below).  This offset is a datum issue related to national security that is documented in this Wikipedia article.  The same data restriction issues that we discuss in our book and in our blog touch on other aspects of geospatial data, such as fines for unauthorized surveys, lack of geotagging information on many cameras when the GPS chip detects a location within China, and seeming unlawfulness of crowdsourced mapping efforts such as OpenStreetMap.

But furthermore, as we have noted, the satellite images are processed tiled and data sets, and like other data sets, they need to be critically scrutinized as well.  They should not be considered “reality” despite their appearance of being the “actual” Earth’s surface.  They too contain error, may have been taken on different dates or seasons, may be reprojected on a different datum, and other data quality aspects need to be considered.

china-agol

Another difference between these maps is the wide variation in the amount of detail in terms of the streets data in China.  The OpenStreetMap was the most complete; the other web mapping platforms offered a varying level of detail; some of which were seriously lacking, surprisingly especially in the year 2017, in almost every type of street except major freeways.  The streets content was much more complete in other countries.

It all comes back to identifying your end goals in using any sort of GIS or mapping package.  Being critical of the data can and should be part of the decision making process that you use and the choice of tools and maps to use.  By the time you read this, the image offset problem could have been resolved.  Great!  But are there now new issues of concern? Data sources, methods, and quality vary considerably among different countries. Furthermore, the tools and data change frequently, along with the processing methods, and being critical of the data is not just something to practice one time, but rather, fundamental to everyday work with GIS.

IndianaMap: Data and Visualization for Indiana

June 4, 2017 3 comments

IndianaMap is a resource for visualizing and accessing spatial data for the US state of Indiana.   The data source contains elements that other provinces, regional, and state governments might wish to adopt because as I see it, they are incredibly useful to the data user.  In addition, I keep talking with people who state that such adoption has helped them build internal support for their organization’s mission, and recommend examining IndianaMap for that reason as well as a model for how it could work.

One of my favorite things about IndianaMap is that it contains a map viewer, a map gallery, and a layer gallery, linked right at the top of the user interface.   At least 75 layers exist in this resource at the time of this writing.  Two that I was particularly glad to see were the geology layers and the historical 1990s Digital Orthophotoquads.  New imagery at 1 foot spatial resolution is also available.  Yes, 1 foot!  Each of the layers can be examined in more detail, previewed, its metadata viewed, or downloaded for use in desktop GIS software. Layers as map services can also be examined in a web based client, or one can choose to add the layer to the interface’s own Map View. Once you have explored layers of interest, you can use the “Add Content” tool on IndianaMap to quickly add, remove, and manage each layer. Each layer can be examined and saved as a favorite.  Sure enough, after I had used the tool, revisiting the site showed me the layers I had favorited in “My layers” so I could resume my work from a few days ago, right away.

On the Spatial Reserves blog, we often feature sites with a particularly useful user interface.  IndianaMap definitely achieves high quality marks in this regard.  Give it a try!

 

indianamap

User Interface for the Indiana Map showing bedrock geology, karst springs, and selecting one of my favorite parts of Indiana’s geology–its limestone.

Accessing Landsat 8 Data through DevelopmentSeed’s Libra Portal

May 21, 2017 1 comment

Libra is a very useful browser for open Landsat 8 satellite imagery. You can use it to browse, filter, sort, and download imagery for the entire planet.  Libra was developed by DevelopmentSeed, an engineering team solving complex problems with open software and open data, and AstroDigital, a company focused on providing imagery analyzed in real time and streamed to applications via their API.

Libra’s map interface is one of the most straightforward and useful that I’ve ever seen for imagery as evidenced in the screenshot below.  On the interface, each circle on the map represents the number of available images at that location. Filters at the top of the map can be used to select a date range, cloud cover percentage, and sun azimuth angle. Bundled downloads are available within one week of the image being taken. Individual bands are available for all 2015 scenes.

Once your download is complete, the site organizers recommend Landsat-util for processing the raw image and getting it ready for publication and analysis.  Search results are powered by Landsat 8 Metadata API and images from USGS Earth Explorer. Downloads are provided via Google Earth Engine and Amazon Web Services.

More information and the official announcement about this resource can be found here.   Give it a try and let us know what you think.

Interface for the Libra Development Seed Landsat Site.

Interface for the Libra Development Seed Landsat Site.

New LandViewer Tool for Quickly Finding and Analyzing Satellite Imagery

May 7, 2017 2 comments

The LandViewer tool and data portal quickly and painlessly allows you to browse and access satellite imagery for the planet.  The tool, developed by the Earth Observing System Inc.’s Max Polyakov, currently features Landsat 8 and Sentinel 2 imagery with more image sets soon to arrive.  Landsat 8 carries two instruments: The Operational Land Imager (OLI) sensor includes refined heritage bands, along with three new bands: a deep blue band for coastal/aerosol studies, a shortwave infrared band for cirrus detection, and a Quality Assessment band. The Thermal Infrared Sensor (TIRS) provides two thermal bands. Sentinel 2 is an Earth observation mission developed by the ESA as part of the Copernicus Programme to perform terrestrial observations in support of services such as forest monitoring, land cover changes detection, and natural disaster management.

Using the LandViewer tool, you can quickly zoom on an interactive web map to your area of interest.  You can filter on geography and time, including cloudiness, sun angle, and other parameters. At the time of this writing, 18 filters such as Atmospheric Removal, Panchromatic, NDVI, Thermal Infrared, False Color, and more, are available so that you can obtain the band combinations most suitable to your analysis in the areas of agriculture, geology, or other applications. A very helpful image interpretation screen is available to help you choose the combination that are best for your analysis goals.  You can do some contrast stretching in the web tool itself.  Then after signing in to the site, you can download the images in GeoTIF for further analysis using your favorite GIS tools.

The tool was also reviewed on the Geoawesomeness web site, and I wholeheartedly agree with their sentiments expressed–this is one of the most useful and fastest satellite image portals I have used. It is useful for research but also, given its ease of use, can even be used effectively to teach concepts of remote sensing.  Give it a try and let us know in the comments section what you think.

landsat_viewer.JPG

Landsat scenes with band combinations possible for an area on the southwest side of Costa Rica.

 

A review of the Los Angeles GeoHub

April 23, 2017 2 comments

The Los Angeles GeoHub represents, in many ways, the next generation GIS data portal. It is in my view what a data portal should be, and given the population and areal size of Los Angeles, that the portal is robust makes it even more impressive.  The data user can search the city’s open data site, and also do something that not all sites allow:  “Explore all data”.  At the time of this writing, “exploring all data” resulted in 554 results, which one can then add to “my favorites” for later investigation and download.  One can also explore the data by category, including business, boundaries, health, infrastructure, planning, recreation and parks, safety, schools, and transportation.  Most data sets can be downloaded as a spreadsheet, as a KML file, or a shapefile.  These layers include grasslands, fire stations, cell phone towers, road work projects, traffic, parcels, and dozens and dozens more–even bus stop benches and other treasures.  Each download is quick and painless.

A unique and very useful characteristic of the GeoHub is that each layer lists the number of attributes, which are easily displayed on the site.  Another wonderful feature is that each layer is displayed above its metadata listing as a web service inside ArcGIS Online, which can be opened immediately in ArcMap or ArcGIS Pro or streamed as a GeoJSON or GeoService as a full or filtered data set. Applications based on the data can also be accessed on the site, such as the CleanStat clean streets index and the social determinants of health app.  And yet there is even more–charts can be generated straight from the data, and a whole set of ArcGIS Online mapping applications that the city has generated are displayed in a gallery here.  Because of these applications, the site can be used effectively even by someone who is not familiar with how to run a GIS to understand Los Angeles better and to make smarter decisions.

If you are a data user, explore the data on the GeoHub today.  If you are a data administrator, consider using the GeoHub as a model for what you might develop and serve for your own data users in your own location.

la_geohub

Los Angeles GeoHub results from examining cell phone towers.  Note the many data-user-friendly items and choices to stream and download.

Latitude Mark II: All change and no change

April 18, 2017 2 comments

A recent article on BusinessInsider reported the re-launch of Google’s location sharing feature as an update to Google Maps. Originally available as Google Latitude, the first version prompted a report highlighting the risks of inadvertently sharing personal location information. Although the location sharing options seem similar second time around, the focus seems to be on the benefits of sharing this type of information and as the article notes, although the privacy concerns haven’t away, they are a footnote rather than the headline.

What has changed in the intervening years appears to be the perceptions about sharing personal location information. Is this because consumers of such services heeded the warnings and shared with discretion so fears were unfounded, or because the risks were not as great as originally thought? Other location sharing applications, such as Glympse and Swarm, stayed the course and developed their niche products away from the spotlight that tends to focus on Google. Have these services paved the way for Google to try again? Whatever the reason, Google is confident enough of a favourable reception to re-release their location sharing technology as part of their flagship application.

 

An Open Letter to the Open Data Community: Reaction

April 9, 2017 3 comments

A group of people at the Civic Analytics Network recently wrote “An Open Letter to the Open Data Community” that focuses on topics central to this blog and to our book. The Civics Analytics Network, is “a consortium of Chief Data Officers and analytics principals in large cities and counties throughout the United States.”  They state that their purpose is to “work together to advance how local governments use data to be more efficient, innovative, and in particular, transparent.”

The letter contained 8 guidelines the group believed that if followed, would “advance the capabilities of government data portals across the board and help deliver upon the promise of a transparent government.”  The guidelines included the following:

  1.  Improve accessibility and usability to engage a wider audience. 
  2. Move away from a single dataset centric view.
  3. Treat geospatial data as a first class data type.
  4. Improve management and usability of metadata. 
  5. Decrease the cost and work required to publish data. 
  6. Introduce revision history.
  7. Improve management of large datasets.
  8. Set clear transparent pricing based on memory, not number of datasets.

It is difficult to imagine a letter that is more germane to what we have been advocating on the Spatial Reserves blog.  We have been open about our praise of data portals that are user friendly–and critical of those that miss the mark–over the past five years.  We have noted the impact that the open data movement has had on the data portals themselves–becoming in many cases more user friendly and encouraging adoption of GIS beyond its traditional departmental boundaries.  The principles we have adhered to are also mentioned in this letter, such as being intuitive, data-driven, and with metrics.  The letter highlights a continued need, the ability to tie together and compare related data sets, which is at times challenging given “data silos.”

One of my favorite points in the letter is the authors’ admonition to “treat geospatial data as a first class data type.”  The authors claim that geospatial data is an underdeveloped and undervalued asset; and it “needs to be an integral part of any open data program”, citing examples from Chicago’s OpenGrid and Los Angeles’ GeoHub as forward-thinking models.

On the topic of metadata, the authors call for portals and managers to allow “custom metadata schemes, API methods to define and update the schema and content, and user interfaces that surface and support end-user use of the metadata.”  Hear, hear!  Equally welcome is the authors’ call to decrease the cost and work required to publish data. Through their point #6 about revision history, they advocate that these data sets need to be curated and updated but also allow historical versions to be accessed.

What are your reactions to this letter?  What do we need to do as the geospatial community to realize these aims?

Categories: Public Domain Data Tags: