Accessing USGS Topographic Maps on The Internet Archive – Archive.org

September 30, 2018 1 comment

For years, I have used the Internet Archive (https://archive.org/about/) for many things, from archiving multimedia that I created for my story maps to looking up information on historical web pages through their Wayback Machine, (as well as for listening to some old wonderful sound recordings) and through those efforts became aware of the wealth of information on the site.   And when I say wealth, I truly mean enormous – 279 billion web pages, 11 million books and texts, 4 million audio recordings (including 160,000 live concerts), 3 million videos (including 1 million Television News programs), 1 million images, and100,000 software programs. But did you know that The Internet Archive also houses some geospatial data?  The Internet Archive, a 501(c)(3) non-profit organization that has existed since 1996, states that its mission is to “provide Universal Access to All Knowledge,” so it makes sense that some geospatial data for the public good is there.

Let’s focus here on the USGS topographic map data on The Internet Archive, also known as Digital Raster Graphics (DRGs).  Start here for a list of these maps by state, and then underneath each state, a variety of search options are available.  It isn’t the most intuitive unless you know the specific map name that you are looking for, so a topographic map index may still come in handy; a scanned version of these is not easy to come by, but one such archive is here.  Formats include GeoTIFF, essential for use in a GIS.

archive-org.JPG

Interface on The Internet Archive for USGS Digital Raster Graphics. 

While I still find the interface on the other main DRG archive, LibreMap, to be a bit easier to use, LibreMap is not maintained any longer, and is starting to return some errors during certain searches.  The Esri USGS Historical Map Explorer, and the USGS TopoView, reviewed here, is more modern approach to obtaining topographic maps, with the added benefit of historical editions.  USGS topographic maps are part of the set of basemaps available inside ArcGIS Online as data services, which is increasingly part of modern GIS workflows, rather than downloading the data and using it locally.  Still another archive is that from Historical Aerials, which I reviewed here. 

orleansin.JPG

A section of my all-time favorite USGS topographic map, for Mitchell Indiana, simply because of the intricacies of the depression contours and disappearing streams in the magnificent karst landscape depicted. 

Advertisements

A review of Google’s search engine for Open Data

September 17, 2018 4 comments

An article in Nature described Google’s new search engine for open data, and since geospatial data is a fundamental part of open data, and after all these years, still challenging at times to find, I was immediately interested in testing it.

The tool, called Google Dataset Search, is accessible on this link.  Like Google Scholar and Google Books both of which I make heavy use of, this is a “specialized search engine.”

The utility of this tool will depend on metadata tagging. Indeed, as the article points out, “those who own the data sets should ‘tag’ them, using a standardized vocabulary called Schema.org, an initiative founded by Google and three other search-engine giants (Microsoft, Yahoo and Yandex)”. The schema.org dataset markup is the standard used here, but others are supported, such as CSVs, imagery, and proprietary formats.  I had to laugh at the open-ness of the last line in the list of what could qualify as a dataset, “Anything that looks like a dataset to you.”

Many data search engines and portals have a vast amount of data but very little geospatial data.  But with this tool, in my test searches, I found many useful geospatial data sets, some of which I knew and some that were new to me.  I have had challenges finding stream gauging data services for Australia recently, and with this tool found some new leads to investigate.  Being Google based, the searches were rapidly returned, with what I considered enough information to decide whether or not to investigate more fully (see screenshot below).  The data format was featured prominently, as was the coverage, both of which I appreciated.  NOAA was an early adopter of the indexing, and so it makes sense that I could find many NOAA data sets using this search engine.

I wonder if data in Github, or in Esri’s Living Atlas, or on state, national, and international portals will be findable.  I also wonder how the sheer importance of Google will influence how organizations tag their data in the future, and the influence this will have on agencies that perhaps did not put as much time on metadata as they perhaps should have.  Time will tell, but if Google Scholar and Google Books are any indication, the Google Dataset Search could indeed prove to be extremely useful for many of us in GIS research and education.

1111

Result of stream gauge search in the new Google data search engine. 

Location, Privacy and Google

September 10, 2018 1 comment

Recent revelations about Google’s continued tracking of personal location information despite the Location History setting being disabled have been widely reported (Business Insider, The Guardian). Google responded robustly, acknowledging incidental location information was also collected under other application settings (specifically Web and App Activity) but insisting they provided clear guidelines on what the various settings entailed. Many observers and users of Google services remain unimpressed by what appears at be the rather insidious tracking of location information. Disabling Location History should mean NO location information is tracked regardless of which Google service was used.

Having just reread their Privacy Policy, Google does make it clear that location information is tracked through a variety of other sources, including public data, business and marketing. The Location History setting means location information is saved to a Google account timeline – if that’s what it means, then perhaps the setting should be Save Location History to Timeline.

All of this leads into a more general discussion on privacy; what do we assume privacy means, what do we expect to remain private and what information about us are we prepared to be in the public domain. I typed define: privacy into Google search and the response was … ‘a state in which one is not observed or disturbed by other people‘. Clearly this is not the baseline for Google’s policy and perhaps another example where some rewording of the policy headline would help clarify exactly what a user of Google’s services can expect. Maybe Information Collection and Reuse Policy would be more transparent so there is no misunderstanding, no expectation of privacy and users make informed decision at to what personal information they are prepared to handover in exchange for access to online services.

 

 

 

Categories: Public Domain Data

The use of administrative data to improve state decision-making, with connections to GIS

August 20, 2018 1 comment

An article on the use of administrative data to improve state decision making has, I believe, key implications for those of us involved with creating, coordinating, and using geospatial data.  “As state leaders seek to harness data in innovative ways, what common themes, noteworthy successes, and notable challenges have the 50 states in the USA experienced across a broad cross section of issue areas? To address these questions, The Pew Charitable Trusts interviewed state leaders across the U.S. in 2016 and reviewed relevant laws, documents, and policies in all 50 states. This report is the culmination of that research, and the first comprehensive overview of how data is being utilized in all 50 states.”

Using data from over 350 interviews with key state officials, the report highlights ways in which some government leaders have employed sophisticated data analytics, to craft policy responses to complex problems, improve service delivery, manage resources, and examine program effectiveness.  The five key actions identified that state leaders could take are:  Plan ahead, including the use of data!, build the capacity of stakeholders to use data, create meaningful information from data, and sustain support for data efforts.  The complete 61 page report is linked to the above summary.

The report’s thoughtful recommendations go far beyond the “hey, let’s employ data driven decisions” that we hear so much about these days.  For large organizations such as state or regional government agencies to effectively use data, data literacy needs to be built in that organization, so that people can make sense of that data, and those efforts need to be supported and funded.  I would even go beyond this and say that the efforts of the education outreach team on which I serve at Esri and all those involved with GIS education play a key role, because the development of geographic literacy takes time and must include all levels of education–primary, secondary, university, and lifelong learning, and include formal and informal settings.

I will also state that while geospatial data is not mentioned in the summary, and only a few times by name in the report itself, GIS is the force behind and needs to be recognized as the force behind each one of these data initiatives.   Examples of the importance of GIS are all over this report, from the vehicle crash incidents map in Indiana, to natural hazards assessment in Oregon, to opioid overdoses in Massachusetts.  But I would also say that it is up to us in the GIS community to make sure we draw out the importance of GIS in reports and recommendations such as this in the presentations we give, meetings we attend, and items that we author.   As GIS becomes more of an enterprise solution, embedded in the day-to-day work that organizations do, I am concerned that GIS could get “buried” or taken for granted.  I believe that it must be continually cited so that it is supported and funded into the future.

IMG_0403

Transportation is one obvious ongoing need that large government agencies will need to address into the future with data and GIS technology, but there are many others, identified by the Pew Trust’s report that I describe here.  Photograph by Joseph Kerski, sitting at a dead stop along Highway 60 in California.

 

 

Categories: Public Domain Data Tags:

Key statements about the importance of spatial data

August 6, 2018 3 comments

Sometimes it is helpful to have some research results and quotes in support your data advocacy efforts at your own organization–in your promotion of  “why this all matters!”.   And, of course, why your efforts need to be funded and supported!  Here are a few key quotes about the importance of spatial data–and what happens when the data doesn’t exist.

Kathryn Sullivan, former NASA astronaut, recently commented, The power of a map to put time and place and phenomena together, to give it to our brains through the most potent input sensor human beings have — our eyes — is a remarkable accelerator for the comprehension and engagement and use of the data that tell us what’s on Earth, where are things happening on our planet …   as reported in Cheney, S. (2017). How GIS can help us understand our changing oceans.  (quoted on the 2017 Esri Ocean GIS Forum on https://www.devex.com/news/how-gis-can-help-us-understand-our-changing-oceans-91366).

Another extremely useful statement is, “Advances in research on resilience and vulnerability are hampered by access to reliable data” can be found in Barrett, C. B. and D.D. Headey. 2014. Measuring resilience in a risky world: Why, where, how, and who? 2020 Conference Brief 1. May 17-19, Addis Ababa, Ethiopia. Washington, D.C: International Food Policy Research Institute.

“The lack of data is one of the biggest obstacles to progress toward development goals” was a part of the statements from the United Nations Independent Expert Advisory Group (UN). 2014. A World That Counts: Mobilising the Data Revolution for Sustainable Development. A Report to the UN Secretary General. New York, NY: United Nations, pp.28.

Perhaps the strongest argument for more and better data comes from ODI’s report The Data Revolution:  Finding the Missing Millions, where the authors cite numerous cases where inadequate data leads to sub-optimal policy decisions.  These cases “confirm some of the anecdotal evidence about the lack of good data in developing country ministries.”    The full citation is:  Stuart, E., E. Samman, W. Avis, and T. Berliner. 2015. The data revolution: finding the missing millions. ODI Research Report 03. London: Overseas Development Institute, pp. 51. (https://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/9604.pdf).

Based on the research that Jill Clark and I have done in this area over the past decade, I would add to the ODI statement that in developed countries, some similar challenges exist.  We have documented those in these blog essays for six years.

I sometimes use the statements from this National Academies of Sciences report and these written by vterrain.org.  I have created videos on this topic such as here and articles such as in Directions Magazine here.

My own contribution to these quotes is, “We have made much progress, to be sure, but the world’s increasingly complex and serious issues are not going to wait around another generation for us to get our data act together.”

What are the quotes and studies you are using in your own data advocacy efforts?  Please share those in the comments section.

Back CameraPhotograph by Joseph Kerski.