Archive

Posts Tagged ‘Open Data’

A review of the Los Angeles GeoHub

April 23, 2017 2 comments

The Los Angeles GeoHub represents, in many ways, the next generation GIS data portal. It is in my view what a data portal should be, and given the population and areal size of Los Angeles, that the portal is robust makes it even more impressive.  The data user can search the city’s open data site, and also do something that not all sites allow:  “Explore all data”.  At the time of this writing, “exploring all data” resulted in 554 results, which one can then add to “my favorites” for later investigation and download.  One can also explore the data by category, including business, boundaries, health, infrastructure, planning, recreation and parks, safety, schools, and transportation.  Most data sets can be downloaded as a spreadsheet, as a KML file, or a shapefile.  These layers include grasslands, fire stations, cell phone towers, road work projects, traffic, parcels, and dozens and dozens more–even bus stop benches and other treasures.  Each download is quick and painless.

A unique and very useful characteristic of the GeoHub is that each layer lists the number of attributes, which are easily displayed on the site.  Another wonderful feature is that each layer is displayed above its metadata listing as a web service inside ArcGIS Online, which can be opened immediately in ArcMap or ArcGIS Pro or streamed as a GeoJSON or GeoService as a full or filtered data set. Applications based on the data can also be accessed on the site, such as the CleanStat clean streets index and the social determinants of health app.  And yet there is even more–charts can be generated straight from the data, and a whole set of ArcGIS Online mapping applications that the city has generated are displayed in a gallery here.  Because of these applications, the site can be used effectively even by someone who is not familiar with how to run a GIS to understand Los Angeles better and to make smarter decisions.

If you are a data user, explore the data on the GeoHub today.  If you are a data administrator, consider using the GeoHub as a model for what you might develop and serve for your own data users in your own location.

la_geohub

Los Angeles GeoHub results from examining cell phone towers.  Note the many data-user-friendly items and choices to stream and download.

An Open Letter to the Open Data Community: Reaction

April 9, 2017 3 comments

A group of people at the Civic Analytics Network recently wrote “An Open Letter to the Open Data Community” that focuses on topics central to this blog and to our book. The Civics Analytics Network, is “a consortium of Chief Data Officers and analytics principals in large cities and counties throughout the United States.”  They state that their purpose is to “work together to advance how local governments use data to be more efficient, innovative, and in particular, transparent.”

The letter contained 8 guidelines the group believed that if followed, would “advance the capabilities of government data portals across the board and help deliver upon the promise of a transparent government.”  The guidelines included the following:

  1.  Improve accessibility and usability to engage a wider audience. 
  2. Move away from a single dataset centric view.
  3. Treat geospatial data as a first class data type.
  4. Improve management and usability of metadata. 
  5. Decrease the cost and work required to publish data. 
  6. Introduce revision history.
  7. Improve management of large datasets.
  8. Set clear transparent pricing based on memory, not number of datasets.

It is difficult to imagine a letter that is more germane to what we have been advocating on the Spatial Reserves blog.  We have been open about our praise of data portals that are user friendly–and critical of those that miss the mark–over the past five years.  We have noted the impact that the open data movement has had on the data portals themselves–becoming in many cases more user friendly and encouraging adoption of GIS beyond its traditional departmental boundaries.  The principles we have adhered to are also mentioned in this letter, such as being intuitive, data-driven, and with metrics.  The letter highlights a continued need, the ability to tie together and compare related data sets, which is at times challenging given “data silos.”

One of my favorite points in the letter is the authors’ admonition to “treat geospatial data as a first class data type.”  The authors claim that geospatial data is an underdeveloped and undervalued asset; and it “needs to be an integral part of any open data program”, citing examples from Chicago’s OpenGrid and Los Angeles’ GeoHub as forward-thinking models.

On the topic of metadata, the authors call for portals and managers to allow “custom metadata schemes, API methods to define and update the schema and content, and user interfaces that surface and support end-user use of the metadata.”  Hear, hear!  Equally welcome is the authors’ call to decrease the cost and work required to publish data. Through their point #6 about revision history, they advocate that these data sets need to be curated and updated but also allow historical versions to be accessed.

What are your reactions to this letter?  What do we need to do as the geospatial community to realize these aims?

Categories: Public Domain Data Tags:

Digital Globe’s Open Data Program

February 26, 2017 1 comment

Open Data continues to make progress as manifested in data portals, organizations adopting it, and associated literature.  Are private companies also involved in Open Data? Yes. As early as two years ago, we wrote about Esri’s initiatives in ArcGIS Open Data. Imagery and geospatial data company DigitalGlobe have created DigitalGlobe’s open data portal, as part of their efforts to provide “accurate high-resolution satellite imagery to support disaster recovery in the wake of large-scale natural disasters”.  This includes pre-event imagery, post-event imagery and a crowdsourced damage assessment. Associated imagery and crowdsourcing layers are released into the public domain under a Creative Commons 4.0 license, allowing for rapid use and easy integration with existing humanitarian response technologies.  For example, their imagery for areas affected by  Hurricane Matthew in 2016 is available here.

On a related note, I have worked with DigitalGlobe staff for years on educational initiatives.  They provided me with high resolution imagery for an area in Africa I was conducting a workshop in, and more recently with imagery in Southeast Asia that I needed in conjunction with helping Penn State prepare exercises for their GEOINT MOOC (Massive Open Online Course in Geointelligence).  They have always been generous and wonderful to work with and I salute their Open Data Portal initiative.  In the MOOC we also used their Tomnod crowdsourcing platform with great success and interest from the course participants.

digitalglobe

Digital Globe’s Open Data Program.

New Exercise Using Open Data Portals from Local Governments

December 18, 2016 1 comment

Despite the growing volume of geospatial data available, and the ease of use of much of this data, finding and using data remains a challenge.  To assist data users in these ongoing challenges, I have written a new activity entitled “Key Strategies for Finding Content and Understanding What You’ve Found.”   The goal of this activity ” Key Strategies for Finding and Using Spatial Data” is to enable GIS data users to understand what spatial analysis is, effectively find spatial data, use spatial data, and become familiar with the ArcGIS platform in the process.  I tested the activity with a group of GIS educators and now would like to share it with the broader GIS community.

The document makes it clear that we are still in a hybrid world–still needing to download some data for our work in GIS, but increasingly able to stream data from online data services such as those in ArcGIS Online.  But these concepts don’t make as much sense unless one actually practices doing this–hence the activity.

In the activity, I ask the user to first practice search strategies in ArcGIS Online, using tags and keywords. Then, I guide the user through the process of downloading and using a CSV file with real-time data.   After a brief review of data types and resources, I guide the user of the activity through the process of downloading data from a local government agency to solve a problem about flood hazards.  The next step asks users to compare this process of downloading data with streaming the same data from the same local government’s site (in this case, using data from Boulder County, Colorado) into ArcGIS Online.  The activity concludes with resources to discover more about these methods of accessing data.

Jill Clark and I have created other hands-on activities on this theme of finding and understanding data as well, available here.  We look forward to hearing your comments and I hope this new activity is useful.

bouldercounty_data.png

Accessing data from the Boulder County local government GIS portal through the lesson described above.

On Geospatial Information and the Sustainable Development Goals

August 28, 2016 4 comments

In a white paper entitled Transforming Our World:  Geospatial Information Key to Achieving the 2030 Agenda for Sustainable DevelopmentDigitalGlobe and Geospatial Media and Communications tie the need for geospatial data to meeting the UN Sustainable Development Goals.

On related topics, we have written about the UN resolution on geospatial data, and the UN Future Trends in geospatial information management, and in our book we wrote about the 8 Millennium Development Goals adopted by UN member states.  The white paper brings together some key connections between the Sustainable Development Goals (SDGs) and GIS.  The 17 goals include–no poverty, zero hunger, good health and well being, quality education, gender equality, clean water and sanitation, affordable and clean energy, decent work and economic growth, industry, innovation, and infrastructure, reduced inequalities, sustainable cities and communities, responsible consumption and production, climate action, life below water, life on land, peace and justice/strong institutions, and partnerships to achieve the goals. The 17 SDGs and the 169 associated targets seek to achieve sustainable development balanced in three dimensions–economic, social, and environmental.  The article focuses on a topic that is central to this blog and our book--the need for data, specifically geospatial data, to monitor progress in meeting these goals but also to enable those goals to be achieved.

The report ties the success of the SDGs to the availability of geospatial data.  One finding of the report was that many countries had not implemented any sort of open data initiatives or portals, which is an issue we have discussed here and in our book.  The main focus of the report is to identify ways that countries and organizations can work on addressing the data gap, such as creating new data avenues, open access, mainstreaming Earth observation, expanding capacities, collaborations and partnerships, and making NSDIs (National Spatial Data Infrastructures) relevant.  For more information on the authors of the paper, see this press release by Geospatial World.

I especially like the report because it doesn’t just rest upon past achievements of the geospatial community to make its data holdings available to decision makers   To be sure, there have been many achievements.  But one thing we have been critical of in this blog in our reviews of some data portals is that many sound fine in press releases, but when a data user actually tries to use them, there are many significant challenges, including site sluggishness, limited data formats and insufficient resolution, and the lack of metadata about field names, to name a few.  The report also doesn’t mince words–there have been advancements, but the advancements are not coming fast enough for the decisions that need to be made.

The report’s main message is that the lack of available geospatial data is not just a challenge to people in the geospatial industry doing their everyday work, but that the lack of available geospatial data will hinder the achievement of the SDGs if not addressed fully and soon.

sdg_doc

White paper connecting the UN Sustainable Development Goals (SDGs) to geospatial information, from DigitalGlobe and Geospatial Media and Communications.

Boulder County Colorado’s New Geospatial Open Data Platform

One of the exercises in our book involves accessing Boulder County Colorado’s GIS site to make decisions about flood hazards.  We chose Boulder County for this activity in large part because their data covers a wide variety of themes, is quite detailed, and is easy to download and use.  Recently, Boulder County went even further, with the launch of their new geospatial open data platform.  This development follows other essays we have written about in this blog about open data, such as the ENERGIC OD, ArcGIS Open Data, EPA flood risk, Australian national map initiative, and open data institute nodes.  Other open data nodes are linked to a live web map on the ArcGIS Open Data site. 

Accessible here, Boulder County’s open data platform expands the usability of the data, such as providing previews of the data in mapped form and in tabular form. The new platform allows for additional data themes to be accessed; such as the lakes and reservoirs, 2013 flood channel, floodplain, and streams and ditches, all accessible as a result from a search on “hydrography” below.  Subsets of large data sets can also be accessed. In addition, the services for each data set are now provided, such as in GeoJSON and GeoService formats, which allows for the data to be streamed directly to such portals such as ArcGIS Online, and thus avoid downloading the data sets altogether.

Why did the county do this?  Boulder County says they are “committed to ensuring that geospatial data is as open, discoverable and usable as possible in order to promote community engagement, stimulate innovation and increase productivity.”  The county is providing an incredibly useful service to the community through their newest innovative efforts, and I congratulate them.  I also hope that more government agencies follow their lead.

boulderhydro

Boulder County’s Open Data Portal showing results for a search on hydrography.

US Census Business Builder App and the Opportunity Project

April 24, 2016 2 comments

The Census Business Builder app and the Opportunity Project are two new tools from the US Census Bureau that make accessing and using data, and, we hope, making decisions from it, easier for the data analyst.  Both of these applications are good representatives of the trend we noted in our book and in this blog— the effort by government agencies to make their data more user-friendly.  While I would still like to see the Census Bureau address what I consider to be the still-cumbersome process of downloading and merging data from the American Community Survey and the Decennial Census with the TIGER GIS files, these two efforts represent a significant step in the right direction.  While GIS users may still not be fully satisfied by these tools, the tools should expand the use of demographic, community, and business data by non-GIS users, which seems to be sites’ goal.

The Opportunity Project uses open data from the Census Bureau and from communities along with a Software Development Kit (SDK) to place information in the hands of decision makers.  Because these decision makers are not likely to be familiar with how to conduct spatial analysis within a GIS, the appeal of this effort is for wiser decision making with the geographic perspective.  A variety of projects are already on the site to spark ideas, including Streetwyze, GreatSchools, and Transit Analyst.

The Census Business Builder is a set of web based mapping services that provides selected demographic and economic data from the Census Bureau.  You can use it to create customized maps and county and city level reports and charts.  A small business edition presents data for a single type of business and geography at a time, while the regional analyst edition presents data for all sectors of the economy and for one or more counties at a time.  These tools are based on Esri’s online mapping capabilities and offer some of the functionality of Esri’s Business Analyst Online.  Give them a try and we look forward to your comments below.

census

A map that I created using the Census Business Builder.