Archive

Posts Tagged ‘Data Portal’

Using GeoSeer to find geospatial data

GeoSeer (https://www.geoseer.net) is a search engine for spatial data covering (at the time of this writing) over 1.2 million distinct spatial datasets from over 180,000 public OGC services (Web Map Services (WMS), Web Feature Services (WFS), Web Coverage Services (WCS), and Web Map Tile Services (WMTS)).

There are a huge number of OGC services online but they’re largely invisible. GeoSeer is designed to solve this “discoverability problem”, similar to how regular search engines like Google, Bing, and DuckDuckGo find web pages, but GeoSeer is focused on OGC services.  In fact, I was originally drawn to GeoSeer because of their statement, “We created GeoSeer to solve a problem: it’s an absolute pain to find spatial data.”  Indeed, this was one driving force for our book and this blog!  Another thing that attracted me was its simple interface (see below), which reminded me, after years of using 37.com, Webcrawler, AltaVista, and other web search tools over the 1990s, the first time I saw the simple but powerful Google search interface.

The GeoSeer bot scrapes over 350 Open Data portals looking for OGC services to add to the index, including the ArcGIS Hub Open Data Portal [1], the Global Earth Observation System of Systems (GEOSS) Portal [2], and many others. By scraping all of these portals and combining all of the discovered services into a single search engine, GeoSeer makes it easy to find open and public spatial data.  One can search by bounding box, lat-long, and service type, as explained here.

For end users, the benefits are a much easier data discovery process, while for the data providers it improves uptake of services and data that would otherwise be invisible and unused.   GeoSeer also includes an API to allow organisations to use the search functionality in their own WebGIS or application, allowing non-expert users to easily find and use these services.  I also liked working with the map-based interface to find data (see below).

How do you obtain data once you have found it?   GeoSeer is designed to demonstrate how the API can be integrated into a webGIS. Rather than trying to be a full webGIS, it was created to demonstrate how smooth the entire search-add process can be for end-users with the API.  It is a search engine to data, but unlike some of the other resources we have reviewed here, itself does not contain data.  Data can be downloaded via WFS or WCS.  WFS are raw vector/Feature data, while WCS are raw raster/Coverage data. If data is WMS/WMTS, then what the user sees is a pre-rendered map only.  Some datasets are available via multiple services, which is why GeoSeer says “distinct” in its “1.2 million distinct spatial layers” statements.  A statistics page shows how many of each data type GeoSeer has in its index: https://www.geoseer.net/stats/

To properly interact with the resulting data, the user will need to load the data into a proper (web) GIS.  The simplest way to do that is to use the regular search, which for me was to search for trails along the Front Range of Colorado, which netted me this link to the Denver Regional Council of Governments page: https://www.geoseer.net/rl.php?ql=7aa26a5ae9fe3e4c.    Not all services findable on GeoSeer are available via WFS or WCS. For example, if my trails data service was only a WMS, I could not download the data.  At the top of my search results, I had two URLs: “WMS GetCapabilities URL” and “WFS GetCapabilities URL”. The WMS version gets me to the pre-rendered map, which is what I saw displayed on the GeoSeer map screen; and the WFS allowed me to download the raw vector data.

I invite you to give GeoSeer a try!

[1] – https://spatialreserves.wordpress.com/2018/12/10/finding-data-on-arcgis-hub-open-data-portal/

[2] – https://spatialreserves.wordpress.com/?s=geoss

geoseer_mapping_interface0

Search screen for GeoSeer. 

geoseer_mapping_interface

GeoSeer Mapping Interface.

With thanks to the GeoSeer team for technical assistance with this post.

–Joseph Kerski

 

 

Nebraska: New Digital Hub and Geospatial Data Portal

May 26, 2019 2 comments

I recently gave the new Nebraska digital hub and geospatial data portal a test run.  As some of the GIS-based lessons I teach are based in Nebraska, some of which are in the Exercises section of this blog (including the fire tower site selection exercise), I have been using GIS portals in the state for at least 20 years.  As I have stated in reviews of other data portals, my preference is to have searching tools as well as the ability to browse categories of data, and this site provides both.

As detailed in a recent article in Enterprise Digitalization, the unique characteristic of the Nebraska portal is that it includes data from every single state agency, served in one location.  Crime, childcare, gas stations, and other information is all supposed to be here.  My searches for fuel or gas came up with nothing, as was my search for robberies, crime, and soils.   My search on land cover netted me unrelated items.   Admittedly, these were my first experiences with the site, and no doubt the site as well as my skills in using it will improve over time.  I did find some interesting data sets that I had not been able to find in the Nebraska data portals I had been using up to now.  I was also pleased to see that many of the data sets are offered as downloads and also as  streaming data services.  Plus, metadata is easily found and understandable.

I know it is a challenge to pull together a state portal, and I salute the efforts of those behind this one.  Admittedly, all I am interested in is geospatial data, and not the other data on the site.  For now, I will use this portal, but because I could not find all of the data sets I am interested in, I will continue to use the Nebraska DNR site, the state’s open data portal, and the sites I have found useful in the past at the University of Nebraska.   I only hope that if those other sites are removed at some point, that the developers do not do so before verifying that every single GIS dataset is moved to the new portal (please!).

Nebraska data portal.

–Joseph Kerski

Accessing data with the WRI Open Data Portal

April 1, 2019 1 comment

The good folks at Blue Raster recently announced the creation of the World Resources Institute’s (WRI) Open Data Portal.  As their article explains, this portal is built with the open source platform CKAN, and provides a centralized, searchable catalog of all data provided by WRI.  We have written about specific WRI resources in this blog in the past, such as here, and one of our exercises makes use of WRI data for Kenya.  WRI has long been one of my favorite organizations and I have made extensive use of their data over the past 15 years.  Before this new portal, WRI researchers would publish data to their own WRI website, but this site only contained data from only a fraction of their projects.  The advantage of the new data portal is that it provides for the first time a full catalog of WRI datasets. Internally, the portal  also provides a more streamlined process for uploading and hosting data, which benefits WRI staff and data users, alike.

I tested the WRI Open Data Portal and was impressed at its clean interface and its ability to filter by file format and region.  I was puzzled, though, because I did not find that many additional data sets beyond what I have seen on WRI in the past.  But perhaps it is because of the search terms I used.  I also could not discover how to stream in data from ArcGIS Online as the article above mentions.  Nevertheless, the portal holds great promise; I highly encourage you to try it.

wri

WRI Open Data Portal interface.

–Joseph Kerski

Access data and analytical tools online from EOS Data Analytics

A new platform from EOS Data Analytics (https://eos.com/platform/) allows data users not only to access data, but even perform online image processing in a web browser.  It is a set of mutually integrated cloud products for searching, analyzing, storing, and visualizing geospatial data.  This is a representation of what we have been discussing on this blog, namely, the increasing adoption of Software as a Service, and in addition, the combination of SaaS with data services and analytical services.  Thus, GIS professionals can search for, analyze, store, and visualize large amounts of geospatial data in one platform.  Doing all this in one system and also in a browser is, quite frankly, quite amazing.

With the EOS Platform, GIS users have access to an ecosystem of four mutually integrated EOS products, which together provide a powerful toolset for geospatial analysts. Image data is stored in cloud-based storage and is available for image processing or remote sensing analysis at any time; this can be a raw user file, an imagery obtained from their LandViewer data portal, or an output file from their online EOS Processing tools.  The EOS Platform is currently available for free during an open Beta.   The LandViewer tool has been freely available for some time and will continue to be.

eos1

There are at least two reasons why image processing is the platform’s major asset: the processing of large data amounts runs online and offers as many as 16 workflows with even more coming soon. On top of that, users can get the best cartographic features of EOS Vision for vector data visualization and soon to come, analysis.

eos2

A list of EOS Processing workflows that can be filtered by industry and input data type.

I found the LandViewer tool easy to use, with a wide variety of data sets to choose from, including Landsat, MODIS, NAIP, and others.

eos3.png

The LandViewer tool interface.

eos4

Choosing one’s own area of interest via the LandViewer interface.

Data Portals for the Chesapeake Bay reviewed

July 17, 2017 3 comments

The Chesapeake Bay, situated along the east coast of the United States, in part because it has long been a focus for environmental restoration, is also a rich source of geospatial data. One primary source is the Virginia Institute on Marine Science, which maintains a Submerged Aquatic Vegetation (SAV) resource.  It also includes an interactive GIS map.  The SAV maps and vegetation data contain information dating all the way back to 1971.

The reasons why VIMS maps the SAV is because this vegetation is one of the best barometers of the water quality; its beds filter polluted runoff, provide food for waterfowl, and provide habitat for blue crabs, juvenile rockfish (striped bass), and other aquatic species; the beds are associated with clear water, and their presence helps improve water quality.  Even if you are not interested in analyzing the vegetation per se, the site is an excellent resource for data and imagery on the Chesapeake Bay.

Some of my other recommended data sites in the region include the Chesapeake Bay Data Hub, the Maryland Open Map portal, (which we reviewed here), the Susquehanna River Basin Commission, Virginia’s open data portal, other Virginia portals, and the USGS’ Chesapeake Bay’s site.  Try these resources and we look forward to your comments below.

savmap

Portion of submerged aquatic vegetation imagery and mapped data from the Virginia Institute of Marine Science.

IndianaMap: Data and Visualization for Indiana

June 4, 2017 4 comments

IndianaMap is a resource for visualizing and accessing spatial data for the US state of Indiana.   The data source contains elements that other provinces, regional, and state governments might wish to adopt because as I see it, they are incredibly useful to the data user.  In addition, I keep talking with people who state that such adoption has helped them build internal support for their organization’s mission, and recommend examining IndianaMap for that reason as well as a model for how it could work.

One of my favorite things about IndianaMap is that it contains a map viewer, a map gallery, and a layer gallery, linked right at the top of the user interface.   At least 75 layers exist in this resource at the time of this writing.  Two that I was particularly glad to see were the geology layers and the historical 1990s Digital Orthophotoquads.  New imagery at 1 foot spatial resolution is also available.  Yes, 1 foot!  Each of the layers can be examined in more detail, previewed, its metadata viewed, or downloaded for use in desktop GIS software. Layers as map services can also be examined in a web based client, or one can choose to add the layer to the interface’s own Map View. Once you have explored layers of interest, you can use the “Add Content” tool on IndianaMap to quickly add, remove, and manage each layer. Each layer can be examined and saved as a favorite.  Sure enough, after I had used the tool, revisiting the site showed me the layers I had favorited in “My layers” so I could resume my work from a few days ago, right away.

On the Spatial Reserves blog, we often feature sites with a particularly useful user interface.  IndianaMap definitely achieves high quality marks in this regard.  Give it a try!

 

indianamap

User Interface for the Indiana Map showing bedrock geology, karst springs, and selecting one of my favorite parts of Indiana’s geology–its limestone.

New LandViewer Tool for Quickly Finding and Analyzing Satellite Imagery

May 7, 2017 2 comments

The LandViewer tool and data portal quickly and painlessly allows you to browse and access satellite imagery for the planet.  The tool, developed by the Earth Observing System Inc.’s Max Polyakov, currently features Landsat 8 and Sentinel 2 imagery with more image sets soon to arrive.  Landsat 8 carries two instruments: The Operational Land Imager (OLI) sensor includes refined heritage bands, along with three new bands: a deep blue band for coastal/aerosol studies, a shortwave infrared band for cirrus detection, and a Quality Assessment band. The Thermal Infrared Sensor (TIRS) provides two thermal bands. Sentinel 2 is an Earth observation mission developed by the ESA as part of the Copernicus Programme to perform terrestrial observations in support of services such as forest monitoring, land cover changes detection, and natural disaster management.

Using the LandViewer tool, you can quickly zoom on an interactive web map to your area of interest.  You can filter on geography and time, including cloudiness, sun angle, and other parameters. At the time of this writing, 18 filters such as Atmospheric Removal, Panchromatic, NDVI, Thermal Infrared, False Color, and more, are available so that you can obtain the band combinations most suitable to your analysis in the areas of agriculture, geology, or other applications. A very helpful image interpretation screen is available to help you choose the combination that are best for your analysis goals.  You can do some contrast stretching in the web tool itself.  Then after signing in to the site, you can download the images in GeoTIF for further analysis using your favorite GIS tools.

The tool was also reviewed on the Geoawesomeness web site, and I wholeheartedly agree with their sentiments expressed–this is one of the most useful and fastest satellite image portals I have used. It is useful for research but also, given its ease of use, can even be used effectively to teach concepts of remote sensing.  Give it a try and let us know in the comments section what you think.

landsat_viewer.JPG

Landsat scenes with band combinations possible for an area on the southwest side of Costa Rica.