Archive

Archive for September, 2019

Detailed Political Boundaries and Thematic Data from the William and Mary geoLab

September 29, 2019 Leave a comment

Obtaining detailed, accurate political boundaries of countries and administrative districts within those countries has long been a challenge.  The William and Mary geoLab, run by geographer Dan Runfola, provides a novel boundary dataset (geoBoundaries) designed to improve on the Database of Global Administrative Areas (GADM) by providing meticulous metadata on boundary sources, including detailed license information.  This dataset is accessible through either the geoLab site (geoLab.wm.edu), or through GeoQuery, an online tool designed to enable easy use to GIS data.  Funded by a combination of USAID, the Cloudera Foundation, and a variety of other non-profits, GeoQuery allows users to generate a CSV based on selected administrative boundaries, satellite data, and survey data (including dollar amounts of international aid sent to each administrative zone).  Behind-the-scenes, a large-scale cluster computing environment (SciClone) processes raster and vector boundaries dynamically to respond to each user query, leveraging a combination of open source python (rasterIO) and Hadoop (CDH) tools.

I began my test of the resource by requesting Armenia’s internal administrative boundaries.  It was easy to select what I needed because of the text-based folder structure of the site.  Oftentimes, despite the wonderful map-based selection tools we now have at our fingertips, this text method is the fastest and most straightforward.

wm_data_portal4

Folder for Armenia boundary data.

I unzipped the resulting data set and brought it into ArcGIS Pro, shown below in light red atop the topographic basemap from ArcGIS Online.

wm_data_portal2

Boundaries (in light red) downloaded from the site vs. the ArcGIS topographic basemap.

As we have advocated throughout this blog and our book, be critical of the data.  For example, on the map above, my newly downloaded boundaries do not completely match either this basemap or other basemaps.  Why?  I am tempted to say that given the focus and rigor of the WM GeoLab data, that their boundaries are the more accurate.  But again, dig deep into the data sets you are using, make you understand how they were obtained, and make sure they meet your needs.

With the geoLab’s detailed boundary information, you can also obtain a selected set (approximately 50 layers) of curated thematic data within specific regions in specific countries.  You can also generate your own thematic data based on flows of international aid to each country on the fly (for example, all environmental aid to each region, or infrastructure aid from the World Bank).   In the example below, my goal was to obtain mean temperature for provinces within Armenia.

wm_data_portal

Downloading the thematic data from the William and Mary research lab.

Once I submitted my request, within a few minutes, I received an email about the data’s availability and was able to download a zip file containing everything I needed.   The email contained a link for me to review my request and download the results and documentation, and a link to download the results directly, (and one nice feature is that this link is permanent, so you can go back to it later), and another link to review all of my current and previous requests.

The site also helpfully provided a citation reminder and information:  “Don’t forget to cite both AidData’s GeoQuery tool as well as each dataset you selected within GeoQuery. All citations can be found in the Documentation PDF at the link above. Here’s the correct citation for GeoQuery:   Goodman, S., BenYishay, A., Lv, Z., & Runfola, D. (2019).   GeoQuery:   Integrating HPC systems and public web-based geospatial data tools. Computers & Geosciences, 122, 103-112. ”

I brought the CSV into ArcGIS Pro and joined it on the province name to the boundaries layer that I previously downloaded.   Within a few minutes, I had the data joined, classified, and symbolized, and was ready to start my analysis.  The only thing that was less than ideal was that I had to join on province name, as I could not find a common code between the boundaries and the thematic data.  This worked fine in my case but prior experience with joining makes me always want to join on a code, if possible.   This was because I used GADM for the boundaries.  If I had selected “GeoBoundaries” on the first page of GeoQuery (where I chose my boundary), then I would have been provided a unique ID for joining across the entire world.

wm_data_portal3

Mapping the thematic data with the province data. 

According to the site manager, they are moving towards a system that simply sends the boundary data to the user when they make the request, rather than me having to download the boundary information separately.   Stay tuned, as the site continues to evolve and improve.

I highly encourage you to give this resource a try!

–Joseph Kerski with gratitude to Dan Runfola at the College of William and Mary.

Categories: Public Domain Data

A review of the Oak Hill West Virginia Open Data Site

September 1, 2019 Leave a comment

Recently at the Esri User Conference, I met the amazing and innovative GIS coordinator for the city of Oak Hill West Virginia.  The open data portal that this coordinator created represents an excellent example of what we have been describing in this blog–the open data movement combining with tools that enable GIS administrators to create and maintain the resources that will serve their internal and external data users.

Oak Hill Open Data works alongside its main website to provide information and enhance transparency to constituents through the power of GIS. Oak Hill Open Data is the repository for data, maps, and apps being generated by the City of Oak Hill. It features the city’s zoning map viewer, “Oak Hill CPR” for submitting citizen complaints directly to the city, an Operations Dashboard for dilapidated structures, and Story Maps about Needleseye Park and the city’s monthly city council meetings.

The site is easy to use, and includes web applications, pages, and even social media feeds and videos.  A search in the top search box on zoning, structures, transportation, and other words and phrases immediately netted me exactly what I was looking for, as streaming data services or as downloadable files in a variety of formats.  I have already used it to create several GIS-based lessons, such as investigating traffic accidents, and intend to do so in the future.  I salute those involved with putting this resource together and I encourage other governmental, nonprofit, academic, and private organizations to make use of the tools such as ArcGIS Hub to do something similar.  Fortunately, many are doing just that!

oak_hill_open_dataA section of the open Data Portal for Oak Hill, West Virginia (https://gis-cityoh.opendata.arcgis.com/).  

–Joseph Kerski