Archive

Archive for September, 2021

More Big Data: Microsoft Buildings Layer

September 27, 2021 2 comments

We have written about the Microsoft building footprints layer before, but given its rapid advance and improvement, an update is warranted. Version 2 of all USA buildings data by USA state is available on Github.  The following represents the Download Links for the Version 2 State Datasets (as of 2021): github.com/microsoft/… [gcc02.safelinks.protection.outlook.com]

As fitting for the “being critical of the data quality” theme of this Spatial Reserves blog, one user of the data wrote, “I find that some polygons are perfectly located; some are not so good; there are a few polygons where buildings have not existed for years and polygons on new construction. [But] Overall it is a good reference.” A fascinating study of this data set was reported on in this article in the journal Nature. On a side note, this is one of the most rigorous studies I have seen of a spatial data set, approaching the data like any other scientific data, and I encourage the geospatial community to create additional studies like this one.

The dataset is in .geojson format. Many GIS software sets will be able to ingest it. For example, in ArcGIS Pro, you can use the tool “JSON To Features” on this file. An example of the results is below.

Many states have enhanced this data and we recently wrote about the efforts in New York state.

Microsoft buildings layer in light outlines atop satellite image. You can see the outlines are good in some places, not as good in others, and missing in others (new housing development), but overall, given the extent and number of features in the data set, truly amazing and an example of GIS “big data.” –Joseph Kerski

Categories: Public Domain Data

A Map to Access the Open Data Portals of the World

September 13, 2021 16 comments

My colleague Nicolas Holm has created a very useful map showing the locations of the open data portals of the world. In our geospatial data-rich environment in which we all now work, this map is valuable because it allows data users to zoom in on specific locations where data is likely to be served for that location or region. It helps fill the gap in the need for a ‘central repository or library’ for geospatial data and acts as “data on data”. Nicolas created the map from the database from Open Knowledge International and OpenDataSoft which he wishes to gratefully acknowledge.

The resource is simple yet powerful–zoom in on area of interest, find points that represent data portals, and click on the data portal where you suspect your desired data set(s) will be. Most of the portals that are featured on this map are what we have featured in this blog as exemplifying “modern data portals”; ones that offer streaming and/or downloading options, many formats to choose from, and the ability to view the data before accessing it, typically using ArcGIS Hub or other open data sharing tools.

After testing the map, I was pleased to find many of my favorite international and also local and regional data portals included. We have reviewed many of these in this blog over the past decade.

In keeping with the theme of this blog, be critical and closely examine each portal to determine whether it meets your needs. Also keep in mind that the data you are seeking for a specific area might not necessarily be served by a data portal located in that area. For example, wetlands data for Area X may actually be served by a portal in Area Y, which may be on the other side of the state/province or even in the national capital by the national mapping or science agency. Still, for many applications, the local data portal might be the most suitable starting point. For example, if I needed data on a specific county’s floodplains, buildings, geologic hazards, and other layers, with this map I can zoom in to my area of interest and find the local and regional data portals from which I could stream and/or download the data that I need.

The map of the open data portals of the world.

–Joseph Kerski

Categories: Public Domain Data