Archive

Posts Tagged ‘Esri’

Finding Data on ArcGIS Hub Open Data Portal

December 10, 2018 Leave a comment

The ArcGIS Hub Open Data Portal  has in a short period of time become a very useful means by which geospatial data can be searched, found, and used.   I believe that there are two main reasons why:  The ArcGIS Hub (1) allows organizations to easily host their own data, and (2) provides an easy to use but powerful set of tools for users to find data.  At the time of this writing, nearly 111,000 data sets were linked to the ArcGIS Hub Open Data Portal from nearly 6,000 organizations worldwide.

In keeping with the theme of our book and this blog, pay close attention to each of the data sets listed here that you are interested in using, and make sure you understand the usage restrictions, if any.  Not all data sets listed are necessarily “open” for any conceivable use, so again, understand the licensing and usage for your desired data set.

One advantage to using the ArcGIS Hub Open Data Portal from the user’s perspective is its simple layout (Figure 1):  The user is presented with a search category box along with a location box; i.e. “near <location x>.”  This surprisingly straightforward interface reminds me of how simple I found Google search to be nearly 20 years ago after years of using WebCrawler, AltaVista, and other search engines.

My education outreach team recently used the ArcGIS Hub in an educational context, in our Esri MOOC entitled “Do It Yourself Geo-Apps”.  In the MOOC, we had participants leverage open data to build web apps using Washington D.C.’s Vision Zero Safety data to help people learn more about pedestrian and bicyclist safety within the community. Specifically, students in the course searched and found data on commuting in Washington DC, downloaded the data as a shapefile, and uploaded it to their ArcGIS Online account (Figure 2) and began analyzing it.

An alternative workflow becoming rapidly adopted, as we have documented in this blog, rather than download and upload, is to obtain the link for the data as a REST endpoint and add it directly into a working session in ArcGIS Online, from which analysis tools can be run.  To do this using the ArcGIS Hub, use the APIs link on the right side after you find your desired data set, with one modification:  The GeoService full dataset are often tagged with a query statement.  For example, the Michigan hydrography polygons are listed as:  https://gisago.mcgi.state.mi.us/arcgis/rest/services/OpenData/hydro/MapServer/17/query?outFields=*&where=1%3D1.  To view the data in ArcGIS Online, remove everything after MapServer/, as shown in Figure 3.

Another fascinating feature in that same right-hand zone on the metadata results page is “create story map”, which, as the name implies gets you started right away creating and displaying the data in a story map (Figure 4) – in my case, a map series story map.  From this point, you could add additional layers, audio, video, photographs, and narrative to this same story map.

It is understandable with any open portal such as this, with contributions from a wide variety of organizations, that some challenges will exist.  From the perspective of the data user, one of those current challenges is finding results to searches on medium sized polygon areas, such as “Colorado” or “Platte River drainage”.   However, in the above Washington DC example, even if you did not know the term “Vision Zero”, a search on bicycle safety near Washington DC would provide you the result you are seeking.  The data extent for the Washington DC Vision Zero covered the entire North Atlantic Ocean, but that’s no doubt the result of an improperly encoded data point.

There is much more to ArcGIS Hub than this open data portal.  ArcGIS Hub includes community engagement tools such as event management, comment management, engagement dashboards, and initiatives.  One of the most appealing things about the ArcGIS Hub is that if you have an ArcGIS Online subscription, you can share your own authoritative open data with ArcGIS Hub.  By using your existing ArcGIS Online groups to identify data to share, you can set up public-facing websites for people to easily find and download your data in a variety of open formats. Your open datasets are connected to the source and are automatically updated.   I highly recommend spending time with the ArcGIS Hub, beginning with the open data portal.

openhub.JPG

Figure 1.  ArcGIS Hub Open Data interface, a very useful tool for finding geospatial data. 

hub1

Figure 2.  Vision Zero safety data for Washington DC from the ArcGIS Hub streamed into ArcGIS Online. hub2

Figure 3.  Michigan hydrography data from the ArcGIS Hub streamed into ArcGIS Online.

hub4

Figure 4.  Story map from Michigan hydrography polygons. 

Advertisements

Crowdsourcing Story Maps and Privacy

As we have pointed out in this blog, we have had the capability to create story maps (multimedia-rich, live web maps) for a few years now, and we have also had the capability to collect data via crowdsourcing and citizen science methods using a variety of methods.  But now the capability exists for both to be used at the same time–one way is with the new crowdsourcing story map app from Esri.

The crowdsource story map app joins the other story map apps that are listed here.  To get familiar with this new app, read this explanation.  Also, you might explore a new crowdsourced story map that, after selecting “+ Participate”, prompts you for your location, photograph, and a sentence or two about attending, in this case, the Esri User Conference.  If you did not attend, examining the application will give you a good sense for what this new app can do.

It’s not just this story map that has me interested.  It is that this long-awaited capability is now at our fingertips, where you can, with this same app, create crowdsourced story maps for gathering data on such things as tree cover, historic buildings, noisy places, litter, weird architecture, or something else, on your campus or in your community.  It is in beta, but feel free to give this crowdsourcing story map app a try.

We have also discussed location privacy concerns both here and in our book.  The story Map Crowdsource app is different from the other Story Maps apps in that it enables people to post pictures and information onto your map without logging in to your ArcGIS Online organization.  Thus, the author does not have complete control over what content appears in a Crowdsource story. Furthermore, the contributor’s current location, such as their current street address or locations they have visited, can be exposed in a Crowdsource app and appear with their post in these maps as a point location and as text. This may be fine if your map is collecting contributions about water quality, invasive plant species, or interesting places to visit in a city, where these location are public places. But it may not be desirable for other subject matter or scenarios, especially if people may be posting from their own residence.

Thus, it is up to you as the author of a Story Map Crowdsource app to ensure that your application complies with the privacy and data collection policies and standards of your organization, your community, and your intended audience.  You might wish to set up a limited pilot or internal test of any Story Map Crowdsource project before deploying and promoting it publicly in order to review if it meets those requirements. And for you as a user of these maps, make sure that you are aware that you are potentially exposing the location of your residence or workplace, and make adjustments accordingly (generalizing your location to somewhere else in your city, for example) if exposing these locations are of concern to you).

Thus, the new crowdsource story map app is an excellent example of both citizen science and location privacy.

ccc

Example of the new crowdsourcing story map app.

New and Updated OpenStreetMap Editors

August 10, 2015 1 comment

Recent updates to Esri’s ArcGIS Editor for OpenStreetMap (OSM) add-on and a new OSM edit option in GitHub highlight the continuing popularity of OSM as one of the go-to base layers for many online mapping applications. In April this year, Esri announced the release of ArcGIS Editor for OSM 10.3x, providing an updated free and open source desktop toolset to download, edit and publish updates to OSM.

Two years ago we wrote about the then new option to upload and visualise geoJSON format spatial data in GitHub against a base map provided by OSM. GitHub have now extended the options for viewing and collaborating on spatial data sets to include the base map itself, with a new option to improve the underlying map for registered GitHub and OSM users.

GitHub_OSM

Registered users can either edit the base map themselves or for those who haven’t registered with OSM, leave a note for another editor to review and resolve. Use of the OSM data remains subject to the terms and conditions of the Open Data Common Open Data Licence.

 

 

 

 

 

Landsat 8 Data Now Available on Amazon AWS

April 13, 2015 Leave a comment

Last month Amazon announced the release of Landsat 8 data on its AWS S3 platform. The data are freely available in GeoTiff format and are not subject to any restrictions on use. The imagery is updated on a 16 day cycle and is available on AWS within hours of reception by USGS.

All of the scenes from 2015 are available, along with a selection of scenes from 2013 and 2014. For those interested in downloading the data (no Amazon account required), each scene’s directory includes the following:

  • a .TIF GeoTIFF for each of the scene’s up to 12 bands
  • .TIF.ovr overview file for each .TIF
  • a _MTL.txt metadata file
  • a small rgb preview jpeg, 3 percent of the original size
  • a larger rgb preview jpeg, 15 percent of the original size
  • an index.html file including the RGB preview and links to the GeoTIFFs and metadata files

As a partner in the initiative to provide easier access to the imagery, Esri has created a set of Landsat Web Services that are available through ArcGIS Online. The services provide dynamic access to the entire collection of Landsat 8 data on AWS.  The web services show one Landsat 8 service that has a subset of the Landsat 8 imagery. If you are interested in downloading more, see the “downloading” link  here.

Cambridge Gulf in Australia

Cambridge Gulf in Australia

 

 

 

New data portals and open data initiatives

August 12, 2013 Leave a comment

The past few months have seen the launch of a number of new spatial data portals and open data initiatives as governments and organisations continue to liberate their data stores. The portals include the Open Geography Portal, providing access to the geographic information behind the national statistics published by the UK Office of National Statistics. The datasets have been made available for free under the terms and conditions of the Open Government Licence. Visitors to the site can search for a variety of statistics related spatial data sets including administrative boundaries, habitat, agriculture, postcode and INSPIRE compliant data themes in the extensive data catalog.

ONS Geography Portal

ONS Geography Portal

Although having open access to spatial data resources such as the ONS repository is beneficial, the current design of the ONS portal highlighted some of the same issues with map portals we identified in an earlier post, primarily the lack a focussed and intuitive front end to identify relevant data sets quickly.

As for open data initiatives, the European Commission has recently agreed to provide free access to data captured by its new Sentinel Earth satellites. After a protracted evaluation process, and following the example of the US Government’s Landsat program, the Commission concluded that the benefits of making the data available for free, with the anticipated growth in value-added services based on the data, outweighed any potential harm to private sector satellite operators (Source: SpaceNews). The first three Sentinel satellites are expected to be launched within the next year. Originally known as the GMES (Global Monitoring for Environment and Security) project, the new program of data capture has been renamed Copernicus.