DataPortals.org: A Global Catalogue of Public Domain Data Portals

The DataPortals.org site, hosted by the Open Knowledge International organisation in conjunction with the LOD2 project, provides a comprehensive repository of over 500 open data portals. The registered portals, published by local, regional and national governments, international organisations and a number of Non Government Organisations (NGOs), provide access to a variety of spatial data sources including administrative boundaries, land use, economic activity and environmental indicators.

All data sets referenced by the DataPortals catalogue, including those that form part of a database collection, are published under the Open Data Commons Public Domain Dedication & Licence. The data sets are available to download in a variety of formats including .xls, JSON/GEOJSON and shapefile.

 

 

“More eyes, better data”

We have written many posts over the last two years on open data and the many data portals that are now available, providing open access to a range of datasets. However as Anders Pedersen (Open Knowledge Foundation) recently remarked during a data skills training initiative, open data does not end with setting up an open data portal;  it’s not enough to just make the data available, the data also has to be ‘reusable and redistributable’.That means publishing the data in more open formats, such as .csv and .txt as opposed to pdfs, to provide the widest possible access to the data.

Pedersen also urged those responsible for establishing data portals to remember that once a data portal is operational the work doesn’t end there. Much remains to be done to keep the site and its content up to date, and promoting the portal to make sure people know about it and what information it provides access to. This means those who maintain portals must have the necessary data collection, management and visualisation skills to support this ongoing effort. Improved access should widen the potential audience for the data, something Pedersen argues will be good for data quality; other agencies and interested citizens will help validate the data, hence ‘more eyes, better data’.

We have reported some examples of portals that have slipped into obsolescence due to a lack of continued support and the comments from Frank Biasi (National Geographic Maps) who reflected on the demise of a conservation geoportal noting, amongst other things, that  “.. the concept of sharing data is much more advanced than the practice“. Training initiatives like those offered by the OKF will hopefully help those involved with open data learn from the experiences of others and avoid some of the mistakes of earlier projects.

 

More data does not mean better data

In a review of government open data initiatives David Buxton, CEO of Arachnys, makes a good point that although more and more governments are making their data available online (World map of open government data initiatives) simply having open access to such data doesn’t necessarily mean that the data will be better; there are no assurances that the data will more accurate, current, useful or even relevant. He does however point to the growing evidence that opening up access to data is generally having a positive influence and cites the success of an initiative in Mauritius to map land ownership across the entire country, the results of which have been a decrease in land grabs and better public scrutiny.

Open Government Data Initiatives

Added these government initiatives, the sheer volume of data that is increasingly being collected and made available via the various resources we have discussed in earlier posts (Internet of Things/Everything, UAVs ( unmanned aerial vehicles) and crowd sourcing) means that it is all the more important for data analysts and end users to understand the provenance, quality and relevance of the data. With increasing choice of data to work with, comes increasing responsibility to make sure it’s the appropriate data for any given application.

The Open Geoportal project

The Open Geoportal (OGP) project is ‘…. a collaboratively developed, open source, federated web application to rapidly discover, preview, and retrieve geospatial data from multiple organizations‘. The project, lead by Tufts University in conjunction a number of partner organisations including Harvard, MIT, Stanford and UCLA, was established to provide a framework for organizations to share geospatial data layers, maps, metadata, and development resources through a common interface. Version 2.0 of the OGP was released in April 2013, providing an improved interface and interoperability for a number of web mapping environments.

OGP currently supports four production geoportal instances:

• Harvard Geospatial Library: Geospatial data catalog from the Harvard University libraries

Harvard Geospatial Library

• UC Berkeley Geospatial Data Repository: Geospatial data from UC Berkeley Library
• MIT GeoWeb: Geospatial data from the MIT Geodata Repository, MassGIS, and Harvard Geospatial Library
• GeoData@Tufts: Geoportal developed and maintained by Tufts University Information Technology, providing search tools for data discovery and for use in teaching, learning, and research.

The data may be streamed, downloaded or shared as required. Although many of the data layers are publicly available, access to some of the layers is restricted and requires registration with the geoportal.

Access to data layers

A number of geoportals are currently in development including those from the universities of Colombia, Washington and Yale.