Data Practitioner Profile Document Reviewed

July 31, 2017 2 comments

The recent document entitled “Profile of the Data Practitioner” (created by a panel with a diverse background, published by EDC Newton Massachusetts USA) is useful in several ways.  First, it succinctly outlines many of the issues we have focused on in this blog and in our book–data quality, critical thinking, domain knowledge, and others.  Second, it lists skills, knowledge, and behaviors, and therefore is an excellent though brief supplement to the Geospatial Technology Competency Model.  Third, it lists equipment, tools, and supplies, future trends, and industry concerns.  Fourth, page 2 of the document is a practical application of the Geographic Inquiry Model, as it describes how the data practitioner initiates a project, sources the data, transforms the data, analyzes the data, closes out the project, and engages in professional development.

The document should be helpful for those pursuing their own career path in GIS and data science, and for those designing and teaching courses and workshops in GIS in academia, nonprofit organizations, private companies, and government agencies.  I only wish the document was longer or linked to a longer report that would provide more detail.  Still, for a succinct document summarizing some key items that data practitioners need to have in place, this document is worth spending time reviewing and telling others about.

Advertisements

Data Portals for the Chesapeake Bay reviewed

July 17, 2017 3 comments

The Chesapeake Bay, situated along the east coast of the United States, in part because it has long been a focus for environmental restoration, is also a rich source of geospatial data. One primary source is the Virginia Institute on Marine Science, which maintains a Submerged Aquatic Vegetation (SAV) resource.  It also includes an interactive GIS map.  The SAV maps and vegetation data contain information dating all the way back to 1971.

The reasons why VIMS maps the SAV is because this vegetation is one of the best barometers of the water quality; its beds filter polluted runoff, provide food for waterfowl, and provide habitat for blue crabs, juvenile rockfish (striped bass), and other aquatic species; the beds are associated with clear water, and their presence helps improve water quality.  Even if you are not interested in analyzing the vegetation per se, the site is an excellent resource for data and imagery on the Chesapeake Bay.

Some of my other recommended data sites in the region include the Chesapeake Bay Data Hub, the Maryland Open Map portal, (which we reviewed here), the Susquehanna River Basin Commission, Virginia’s open data portal, other Virginia portals, and the USGS’ Chesapeake Bay’s site.  Try these resources and we look forward to your comments below.

savmap

Portion of submerged aquatic vegetation imagery and mapped data from the Virginia Institute of Marine Science.

Reviewing the US City Open Data Census Portal of Geospatial Content

July 2, 2017 1 comment

The US City Open Data Census portal is “an ongoing, crowdsourced measure of the current state of access to a selected group of datasets in municipalities across the United States.”  The portal represents another example of a trend we have been noting in this blog for quite some time, a catalog that is a combination of crowdsourced and created by the authors.  In this case, “Any community member can contribute an assessment of these datasets in their municipality at any time. Census content will be peer-reviewed periodically by a volunteer team of Census librarians. [..]  The US City Open Data Census began as a partnership between Code for America, the Sunlight Foundation, and Open Knowledge International. It is maintained by Sunlight Foundation staff members, with technical support from Open Knowledge, local outreach by Code for America brigades, advising from the Open Government Data working group, and contributions from many members of the wider community.”

In the case of this site, don’t think “Census” in terms of demographic data gathered by statistical agencies, but rather, “census” as a catalog of geospatial data for municipalities.  The 18 themes currently cataloged for urban areas include crime, parcels, zoning, and others, but also those that are of interest but may be outside typically considered and sometimes a-spatial categories, such as lobbyist activity, web analytics, and spending.  At this time, the site’s focus is on the U.S. only.  Cities are ranked by the variety and amount of data in the catalog, and at the time of this writing, Las Vegas achieved top score. Testing this site, I was able to find quite a volume of data, in many formats that I could use, and in some formats I was not familiar with but was able to find out more about them.  If the data set I needed was not available, which occurred on more than one occasion, the site tells me who to contact.

If a data user wanted to obtain a set of data to compare across cities, this data set would save that data user quite a bit of time scouring each city’s GIS data site.  Therefore, even though the site’s ambitious list of themes are empty for many cities, and in many ways this project is just getting started, this resource may be valuable for your needs.  And in part because it is crowdsourced and curated, it could become even more valuable in the future.  Time will tell if it persists.  And, like any resource, be critical of its sources and use it if you deem that it will meet your needs.

 

opencity_census.jpg

Visualizing data cataloged by the US City Open Data Census portal, ranked by “score”, with a lower number indicating that a greater volume and wider variety of data is available for that city.

Data Quality on Live Web Maps

June 19, 2017 3 comments

Modern web maps and the cloud-based GIS tools and services upon which they are built continue to improve in richness of content and in data quality.  But as we have focused on many times in this blog and in our book, maps are representations of reality.  They are extremely useful representations, to be sure, particularly so in the cloud, but still are representations.   These representations are dependent upon the data sources, accuracy standards, map projections, completeness, processing and rendering procedures used, regulations and policies in place, and much more.  A case in point are offsets between street data and the satellite image data that I noticed in mid-2017 in Chengdu in south-central China.  The streets are about 369 meters southeast of where they appear on the satellite image (below):

china-google-maps

Puzzled, I panned the map to other locations in China.  The offsets varied, but they appeared everywhere in the country; for example, note the offset of 557 meters where a highway crosses the river at Dongyang, again to the southeast:

china-google-maps2

As of this writing, the offset appears in the same cardinal direction and only in China; indeed; After examining border towns with North Korea, Vietnam, and other countries, the offset appears to stop along those borders.  No offsets exist in Hong Kong nor in Macao.  Yahoo Maps Bing Maps both show the same types of offsets in China (Bing maps example, below):

china_bing

MapQuest, which uses an OpenStreetMap base, showed no offset.  I then tested ArcGIS Online with a satellite image base and the OpenStreetMap base, and there was no offset there, either (below).  This offset is a datum issue related to national security that is documented in this Wikipedia article.  The same data restriction issues that we discuss in our book and in our blog touch on other aspects of geospatial data, such as fines for unauthorized surveys, lack of geotagging information on many cameras when the GPS chip detects a location within China, and seeming unlawfulness of crowdsourced mapping efforts such as OpenStreetMap.

But furthermore, as we have noted, the satellite images are processed tiled and data sets, and like other data sets, they need to be critically scrutinized as well.  They should not be considered “reality” despite their appearance of being the “actual” Earth’s surface.  They too contain error, may have been taken on different dates or seasons, may be reprojected on a different datum, and other data quality aspects need to be considered.

china-agol

Another difference between these maps is the wide variation in the amount of detail in terms of the streets data in China.  The OpenStreetMap was the most complete; the other web mapping platforms offered a varying level of detail; some of which were seriously lacking, surprisingly especially in the year 2017, in almost every type of street except major freeways.  The streets content was much more complete in other countries.

It all comes back to identifying your end goals in using any sort of GIS or mapping package.  Being critical of the data can and should be part of the decision making process that you use and the choice of tools and maps to use.  By the time you read this, the image offset problem could have been resolved.  Great!  But are there now new issues of concern? Data sources, methods, and quality vary considerably among different countries. Furthermore, the tools and data change frequently, along with the processing methods, and being critical of the data is not just something to practice one time, but rather, fundamental to everyday work with GIS.

IndianaMap: Data and Visualization for Indiana

June 4, 2017 3 comments

IndianaMap is a resource for visualizing and accessing spatial data for the US state of Indiana.   The data source contains elements that other provinces, regional, and state governments might wish to adopt because as I see it, they are incredibly useful to the data user.  In addition, I keep talking with people who state that such adoption has helped them build internal support for their organization’s mission, and recommend examining IndianaMap for that reason as well as a model for how it could work.

One of my favorite things about IndianaMap is that it contains a map viewer, a map gallery, and a layer gallery, linked right at the top of the user interface.   At least 75 layers exist in this resource at the time of this writing.  Two that I was particularly glad to see were the geology layers and the historical 1990s Digital Orthophotoquads.  New imagery at 1 foot spatial resolution is also available.  Yes, 1 foot!  Each of the layers can be examined in more detail, previewed, its metadata viewed, or downloaded for use in desktop GIS software. Layers as map services can also be examined in a web based client, or one can choose to add the layer to the interface’s own Map View. Once you have explored layers of interest, you can use the “Add Content” tool on IndianaMap to quickly add, remove, and manage each layer. Each layer can be examined and saved as a favorite.  Sure enough, after I had used the tool, revisiting the site showed me the layers I had favorited in “My layers” so I could resume my work from a few days ago, right away.

On the Spatial Reserves blog, we often feature sites with a particularly useful user interface.  IndianaMap definitely achieves high quality marks in this regard.  Give it a try!

 

indianamap

User Interface for the Indiana Map showing bedrock geology, karst springs, and selecting one of my favorite parts of Indiana’s geology–its limestone.

Accessing Landsat 8 Data through DevelopmentSeed’s Libra Portal

May 21, 2017 1 comment

Libra is a very useful browser for open Landsat 8 satellite imagery. You can use it to browse, filter, sort, and download imagery for the entire planet.  Libra was developed by DevelopmentSeed, an engineering team solving complex problems with open software and open data, and AstroDigital, a company focused on providing imagery analyzed in real time and streamed to applications via their API.

Libra’s map interface is one of the most straightforward and useful that I’ve ever seen for imagery as evidenced in the screenshot below.  On the interface, each circle on the map represents the number of available images at that location. Filters at the top of the map can be used to select a date range, cloud cover percentage, and sun azimuth angle. Bundled downloads are available within one week of the image being taken. Individual bands are available for all 2015 scenes.

Once your download is complete, the site organizers recommend Landsat-util for processing the raw image and getting it ready for publication and analysis.  Search results are powered by Landsat 8 Metadata API and images from USGS Earth Explorer. Downloads are provided via Google Earth Engine and Amazon Web Services.

More information and the official announcement about this resource can be found here.   Give it a try and let us know what you think.

Interface for the Libra Development Seed Landsat Site.

Interface for the Libra Development Seed Landsat Site.

New LandViewer Tool for Quickly Finding and Analyzing Satellite Imagery

May 7, 2017 2 comments

The LandViewer tool and data portal quickly and painlessly allows you to browse and access satellite imagery for the planet.  The tool, developed by the Earth Observing System Inc.’s Max Polyakov, currently features Landsat 8 and Sentinel 2 imagery with more image sets soon to arrive.  Landsat 8 carries two instruments: The Operational Land Imager (OLI) sensor includes refined heritage bands, along with three new bands: a deep blue band for coastal/aerosol studies, a shortwave infrared band for cirrus detection, and a Quality Assessment band. The Thermal Infrared Sensor (TIRS) provides two thermal bands. Sentinel 2 is an Earth observation mission developed by the ESA as part of the Copernicus Programme to perform terrestrial observations in support of services such as forest monitoring, land cover changes detection, and natural disaster management.

Using the LandViewer tool, you can quickly zoom on an interactive web map to your area of interest.  You can filter on geography and time, including cloudiness, sun angle, and other parameters. At the time of this writing, 18 filters such as Atmospheric Removal, Panchromatic, NDVI, Thermal Infrared, False Color, and more, are available so that you can obtain the band combinations most suitable to your analysis in the areas of agriculture, geology, or other applications. A very helpful image interpretation screen is available to help you choose the combination that are best for your analysis goals.  You can do some contrast stretching in the web tool itself.  Then after signing in to the site, you can download the images in GeoTIF for further analysis using your favorite GIS tools.

The tool was also reviewed on the Geoawesomeness web site, and I wholeheartedly agree with their sentiments expressed–this is one of the most useful and fastest satellite image portals I have used. It is useful for research but also, given its ease of use, can even be used effectively to teach concepts of remote sensing.  Give it a try and let us know in the comments section what you think.

landsat_viewer.JPG

Landsat scenes with band combinations possible for an area on the southwest side of Costa Rica.