Archive

Archive for the ‘Public Domain Data’ Category

Imagery: It is what it is: Well, not always.

Being critical of all data, including geospatial data, is a chief theme of this blog and our book:  Use it wisely.  Know where it came from.  Know what has been done to it. Imagery has always been an important component of geospatial data, and today, it takes many forms, ranging from images taken on the ground with a phone or camera to those taken by webcams, UAVs, aircraft, satellites, and more.  Doesn’t this imagery “tell it like it is” and is it therefore exempt from us casting a critical eye on it?   I contend that no, images must be viewed critically so that you can determine if they are fit for your use in whatever project you are working on.

As one example in a larger story, consider the example of the removal of individuals from 20th Century photographs in the USSR. But lest we think such image doctoring was a thing of the past, there are dozens of articles that arise in current web searches highlighting modern day image manipulation, from fashion magazines to science journals, some inadvertent, some on purpose, some for purposes to benefit society and some unfortunately to mislead.  In the geospatial world, consider the image offsets on Google Maps in China that we wrote about not long ago.

Let’s discuss another aspect of image manipulation.  Consider the satellite images below.  They cover many different dates and are derived from many different sources.  But pay attention to what’s on–or off-the highways in the images.   These images all cover the same location—Interstate 30 – 35E interchange on the southwest side of downtown Dallas Texas USA, known locally as the Mixmaster.  Constructed between 1958 and 1962, you can spot, in some images, recent much-needed improvements due to the substantial increase in traffic volume in the past 60 years.

arcgis_online_dallas..PNGmapquest_dallas.png

Study area in ArcGIS Online (first image) and Mapquest (second image).

yahoo_maps_dallas.PNGbing_dallas.PNG

Study area in Yahoo maps (first image) and Bing maps (second image).

gmaps_web_dallas

Study area in Google Maps (it appears the same way in Google Earth). 

Notice anything missing on the Google Maps image?  No traffic. This would be unheard of, day or night, in this busy part of one of the largest cities in the USA.  Obviously, a vehicle-removal algorithm has been applied to the image. It is actually quite impressive, because the algorithm seems to only be applied to moving vehicles–cars, buses, trains–and not to stationary vehicles.  No doubt it is applied to make the process of viewing and interpreting the now-clearer imagery easier.  Interestingly, the moving-vehicle removal seems to only, at the present time, be present in running Google Maps on the web on a tablet or laptop:  Running the Google Maps app on a smartphone shows moving vehicles (below).

gmaps_dallas_app_on_phone.png

Study area in Google Maps app on smartphone. 

Key takeaway points and teachable moments here include:  (1) Images that can be obtained in web mapping services, in geodatabases, and portals should not be considered the same view that would be seen if someone were to be peering over the side of a UAV, airplane, spacecraft, or standing next to a webcam when the image was captured.  They can and are manipulated for a variety of purposes.   They need to be viewed critically.  At the same time, we can marvel and appreciate the many choices we have nowadays in imagery for use in geospatial analysis.   (2)  The machine learning and artificial intelligence techniques applied to imagery are further evidence for the enterprise nature of geotechnologies:  GIS is increasingly connected to mainstream IT trends and innovations.

–Joseph Kerski

 

List of Open Geospatial Data Portals from Recent GIS MOOC Discussions

April 29, 2019 Leave a comment

In our recent offerings of Do-It-Yourself GeoApps, an Esri MOOC, we’ve had some wonderfully rich discussions about open data portals.  I thought that the readers of this blog might appreciate viewing the listing of these portals from local to global that my colleague compiled, here:  diy_geoapps_data_sources_m19_combined 

As always, investigate each of these before using them, but this extensive list is a further reflection of the open data movement that we have been discussing in this blog, and it is exciting to see.

data_door

–Joseph Kerski

New open location data services for Europe

April 22, 2019 1 comment

New open location data services developed by the Open European Location Services (ELS) project were recently demonstrated at the Geospatial World Forum earlier this month. Included in the test data services on show were EuroGlobalMap—a 1:1 million topographic dataset covering 45 countries and territories across Europe—and a geographical names gazetteer.

EuroGlobalMap

EuroGlobalMap

 

The Open ELS project is a two-year programme (started May 2017) aimed at improving access to location-based information captured and maintained by various public bodies across Europe. Among the participating organisations are national mapping authorities from Great Britain, Poland, Germany, Finland, Norway, Spain, Sweden and the Netherlands. Although there is no universally accepted definition of open data, the Open ELS project has defined open data as … Date from nation sources available free of charge under an open licence that are free to access and made available to the public without any restriction that impedes reuse.

With the exception of one EuroGlobalMap web feature service, which is licensed separately, the remaining data services developed for the ELS project are available under the Open ELS licence. Any one using the Open ELS data services is free to:

  • Copy, publish, share and re-use the data
  • Adapt the data and services
  • Use the data for both commercial and non-commercial applications

There no indication yet when the data services will be generally available; a demonstration site is available at https://demo.locationframework.eu/.

 

Testing positional accuracy underwater

April 15, 2019 2 comments

I recently had the opportunity to test positional accuracy while underwater.  And I did not even have to get wet!  While I was doing some GIS work with the excellent faculty at the University of Hamburg, I walked through the St. Pauli Elbe Tunnel while collecting a track.  As I did so, I reflected on the fascinating cultural and physical geography of this 1911 engineering masterpiece that is still in use:  The tunnel is 426 m (1,398 ft) long; it was a technical sensation when constructed; photos at the entrance show Kaiser Wilhelm II dedicating it.  It connected central Hamburg on the north side of the river with the docks and shipyards on the south side of the river Elbe.  The most amazing part was the four massive elevators, capable of carrying bikes and whole vehicles, and of course, 100 years ago, carriages and horses.  These elevators and tunnel are still functional and being used today!

While pondering these thoughts, I collected a track in the Runkeeper app, and mapped it as a GPX file in ArcGIS Online as a 2D webmap and as a shapefile in a 3D scene.  I wanted to test how spatially accurate a track underwater would be, in the x and y dimensions, but also in the z dimension.  First, let’s consider the x and y:  As I walked through the tunnel 24 m (80 ft) beneath the surface through one of the two 6 m (20 ft) diameter tubes, I expected the my app to lose sight of the GPS, Wi-Fi, and cell phone towers, but I did not know how far off my position would be.   My recent experiments on an above-ground track gave me a ray of hope that perhaps my position would be recorded as somewhere in Germany rather than in the North Sea or the Atlantic Ocean.

I was told by a local source who said that the tunnels are 8 m below the bottom of the river, making the water 16 m deep here (this depth here allowed Hamburg to become of the largest container ports in the world).  Thus, above me was 8 m of sediment (glacial, in this area), and 16 m of water for a total of 24 meters above me.  The elevation at the water surface here is approximately 5 m above sea level.  Thus, my elevation in the tunnel should be 5 – 24 = -19 meters, minus 3 more meters because I was standing on the bottom of the tunnel rather than the top, so -22 meters.

My results as a 2D webmap and as a 3D scene are shown below.  As is evident, the recorded elevations are all above sea level, at around 4 or 5 meters, so they were 22+5=27 meters off of my tunnel elevation.

elbe_tunnel_experiment_screen

A 2D map in ArcGIS Online showing the results of my experiment, with elevations in meters above sea level shown as labels. 

Feel free to open and interact with the data!  For example, to test the X and Y:  Using the measure tool, measure the distance between the tunnel as shown on the OpenStreetMap basemap and the position recorded by my track.  As I left the train station on the north side, my position was fairly accurately recorded, but once I descended the stairs into the tunnel, my position was off to the east by about 140 meters, and then shifted to the west and was off by about 240 meters.  But as I continued walking south, for the last 1/3 of my trek through the tunnel, my XY positional accuracy was only off by 50 meters.   I ascended the stairs and circled the parking lot on the south side, and was only 1 to 2 meters off once more.  I descended into the tunnel and walked north.  This time, my position was about 100 meters off, becoming worse as I kept walking.  My position overcorrected 80 meters to the north as I ascended the stairs, and “settled back” to being a few meters off as I walked to the train station.

To test the Z position:  The elevations were, as I suspected, not displaying their correct number below sea level; that is, 19 meters below sea level. However, you can see that the elevations are actually quite close to the elevation of the surface of the river in this area; at about 4.5 meters.

elbe_tunnel_gps_3D_scene.JPG

A 3D scene in ArcGIS Online showing the results of my experiment, with elevations in meters above sea level shown as labels and symbolized as cylinders.   Feel free to open and interact with this 3D scene!

Overall, with only a smartphone and a fitness app, displaying the data in ArcGIS Online, I was rather pleased with the fact that my positions all around were usually only in the tens or a few dozen meters off of true. This aligns with my recent reports of above-ground experiments and is further evidence of the improvements in spatial accuracy with all location based services.

Interested in further exploration?  See the evidence of my field trip in the photographs below.

tunnel1.jpgThe enormous elevators that carry pedestrians, bicycles, and even vehicles from the street level to the level of the tunnels.  This one is at the north side of the river with a photo of the opening ceremony with Kaiser Wilhelm II dedicating it.

tunnel2Standing at the entrance to the tunnel; photo also shows one of the glazed terra cotta art sculptures.

Now, go conduct your own accuracy experiments!

–Joseph Kerski

 

Accessing data with the WRI Open Data Portal

April 1, 2019 1 comment

The good folks at Blue Raster recently announced the creation of the World Resources Institute’s (WRI) Open Data Portal.  As their article explains, this portal is built with the open source platform CKAN, and provides a centralized, searchable catalog of all data provided by WRI.  We have written about specific WRI resources in this blog in the past, such as here, and one of our exercises makes use of WRI data for Kenya.  WRI has long been one of my favorite organizations and I have made extensive use of their data over the past 15 years.  Before this new portal, WRI researchers would publish data to their own WRI website, but this site only contained data from only a fraction of their projects.  The advantage of the new data portal is that it provides for the first time a full catalog of WRI datasets. Internally, the portal  also provides a more streamlined process for uploading and hosting data, which benefits WRI staff and data users, alike.

I tested the WRI Open Data Portal and was impressed at its clean interface and its ability to filter by file format and region.  I was puzzled, though, because I did not find that many additional data sets beyond what I have seen on WRI in the past.  But perhaps it is because of the search terms I used.  I also could not discover how to stream in data from ArcGIS Online as the article above mentions.  Nevertheless, the portal holds great promise; I highly encourage you to try it.

wri

WRI Open Data Portal interface.

–Joseph Kerski

Location Tracking: Getting Under The Skin

March 11, 2019 Leave a comment

We’ve written many times over the last few years about the varying ways devices can track our location, with or without our explicit consent. In most cases our tracks are determined by our interaction with and/or our adjacency to a tracking device; the boundaries of our location privacy determined by the range and sophistication of those devices and to some extent our application preferences. To stop the location tracking, all we needed to do was change the settings, turn off the device or leave it at home and avoid the cameras. However, recent advances in microchip technology look set to change the boundaries of location tracking once more.

Microchip implants have been around for over 20 years, from early experiments proving RFID (radio frequency identification) implants could be used to open doors and turn on lights, to pet and patient microchips for storing identification and medical information. The next generation of microchips will see the inclusion of location tracking technology. Within the last year some new microchips have been introduced that can be read from a distance, are connected to the internet and GPS-enabled. With an embedded GPS-enabled microchip, we become the tracking device.

One company involved in this area, Three Square Market (32M), have been working on a voice-activated, body-heat-powered chip that will monitor an individual’s vital signs and track their location via GPS. With plans to test the new chips this year, 32M are focussing initially on dementia patients. With all such new technology, there always the potential to misuse and abuse the information collected, and as with the introduction of other tracking devices such as drones, the legislation governing the use of GPS-enable microchips lags behind. As Weiss notes (2018) ‘… how will lawmakers and experts in security and tech react when required to define consent for a patient with advanced dementia?

With embedded microchip devices that can transmit and receive location and other information over an increasingly wide area, can there be any guarantees that the individual hosting the device will have complete control over who has access to their location information?