Archive

Archive for January, 2018

Privacy concerns from fitness maps and apps

January 31, 2018 2 comments

We frequently write about the need to teach about and be aware of location privacy with the rapid advancement and web-enablement of GIS.  Thus it wasn’t a surprise when recent concerns arose over an amazing map from Strava Labs.  Maps generated from GPS-enabled fitness devices and other recreational uses of GPS such as GPS Drawing, as well as those from the fitness tracker market such as Fitbit and Garmin, have for several years been sharable and viewable.  Strava has been one of the leaders in helping people stay motivated to meet their fitness goals by providing tools such as apps and maps.  But perhaps the Strava map attracted more attention than others because it contains an amazing “over 1 billion activities and 13 trillion data points”, or perhaps because the map is so responsive and contains some stunning cartography that the web map user can customize.

Whatever the reason, as reported in USA TodayPopular MechanicsWired, and elsewhere, location privacy concerns have arisen recently over the new Strava map.  Specifically, “Security experts over the weekend questioned whether the user-generated map could not only show the locations of military bases, but specific routes most heavily traveled as military personnel unintentionally shared their jogging paths and other routes.”  Some of the posts have reported that it may even be possible to scrape the data to discover the person behind each of the tracks, and the Strava CEO has responded to these and other concerns.  Any GIS user knows that much can be discovered through mapped layers and satellite imagery these days, shedding new light on what is really “secret” in our 21st Century world, but maps aimed at the recreational user are bringing these discussions to the general public.  The particular concern with the Strava data is not so much just the location information, but the temporal data tied to the location, and potential identification of individuals.

Much of it comes down to what we have been saying in this blog–understand the defaults for whatever you are doing in GIS, whether it is the projection of your geospatial data or the location-based app on your phone.  Ask yourself, “What is the default–is my data public by default? Is my projection Web Mercator by default?  Can I override the default, and if so, how?  What is the best way to represent this spatial information?  Do I need to share this information?  If I need to share the information, how should I do it?”  and then act accordingly.   For more on this topic, I encourage you to read some of our short essays, such as Why Does a Calculator App need to know my location?, Making the Most of Our Personal Location Dataposting cat pictures and The Invasion of the Data Snatchers.

stravamap

A section of the Strava heat map, showing the results of people who have recorded and shared their fitness walks and runs.  As one might expect, city park and a high school track stand out as places where more people conduct these activities.  As with other maps showing locations where people are now or where they have been, location privacy concerns have been raised. 

Advertisements

Planet’s Imagery Now Viewable by the Public

January 29, 2018 2 comments

Back in 2014, we wrote about inexpensive and the miniaturization of remote sensing, as exemplified in Planet Labs then-new small satellites.  A year later, we wrote about the company’s (now called “Planet”) Open Region initiative with the United Nation to share imagery under a Creative Commons license.  As described in this National Geographic post, Planet has now created a web mapping tool that allows users to examine two million images, updated monthly.  The tool, called Planet Explorer Beta, contains images dating back to 2016, at anywhere from 3 to 40 meters.  My favorite feature so far on the Explorer Beta is the ability to drag-and-drop two images to create a swipe map, to compare changes over time for any given area.  If you create an account and log in, you can explore daily, rather than just monthly, imagery.  Whether logged in or not, the tool is an excellent and amazing resource for teaching and research.  It could also serve as a great way to introduce students and faculty to imagery and encourage them to go further and deeper with remote sensing.

As most of the readers of this blog are work in the field of GIS, they will want to know how to use this imagery in a GIS.  The viewer described above is just that–a viewer.  You can only view the images online.  To actually access the data for use in your GIS or remote sensing work, begin with Planet’s Documentation.  As Planet is a professional satellite image company, it comes as no surprise that users have a multitude of options from which to choose–bands, date and time, cloud cover, sun elevation and azimuth, rectification, data format, and much more.  The imagery is available via a Planet Explorer interface and a Data API, which requires installing a Python client.

planetlabs.JPG

Comparing imagery from two time periods in Colorado, USA, using Planet’s Planet Explorer Beta.

New working lists of US Federal and State GIS portals

January 15, 2018 2 comments

Joseph Elfelt of MappingSupport.com has compiled a very helpful working list of addresses for over 40 federal ArcGIS servers with open MapServer and ImageServer data:

https://mappingsupport.com/p/surf_gis/list-federal-GIS-servers.pdf

And a list of over 50 state server addresses:

https://mappingsupport.com/p/surf_gis/list-state-GIS-servers.pdf

The lists also contain some key caveats and tips for finding local GIS data as well.  Joseph is open to the community contacting him with additional federal or state servers to add them; his contact information is at the top of the lists.  That these already excellent resources will continue to be updated is very good news.

elfelt.JPG

A section of the very helpful federal and state lists of servers with open MapServer or ImageServer data, compiled by Joseph Elfelt. 

Accessing and Using Lidar Data from The National Map

January 8, 2018 1 comment

We have written about the USGS data portal NationalMap numerous times in this blog and in our book, but since the site keeps getting enhanced, a re-examination of the site is warranted.  One of the enhancements over the past few years is the addition of Lidar data to the site.  I did some recent testing of searching for and downloading Lidar data on the site and wanted to report on my findings.  For videos of some of these procedures, go to the YouTube Channel geographyuberalles and search on Lidar.

From a user perspective, in my view the site is still a bit challenging, where the user encounters moments in the access and download process where it is not clear how to proceed.  However, (1) the site is slowly improving; (2) the site is worth investigating chiefly because of its wealth of data holdings:  It is simply too rich of a resource to ignore.  One challenging thing about using NationalMap is, like many other data portals, how to effectively narrow the search from the thousands of search results.  This in part reflects the open data movement that we have been writing about, so it is a good problem to have, albeit still cumbersome in this portal.  Here are the procedures to access and download the Lidar data from the site:

  1. To begin:  Visit the National Map:  https://nationalmap.gov/ > Select “Elevation” from this page.
  2. Select “Get Elevation Data” from the bottom of the Elevation page.  This is one of several quirks about the site – why isn’t this link in a more prominent position or in a bolder font?
  3. From the Data Elevation Products page left hand column:   Select “1 meter DEM.”
  4. Select the desired format.  Select “Show Availability”.   Zoom to the desired area using a variety of tools to do so.  In my example, I was interested in Lidar data for Grand Junction, in western Colorado.
  5. Note that the list of  available products will appear in the left hand column.  Lidar is provided in 10000 x 10000 meter tiles.  In my example, 108 products exist for the Grand Junction Lidar dataset.  Use “Footprint” to help you identify areas in which you need data–the footprints appear as helpful polygon outlines.  At this point, you could save your results as text or CSV, which I found to be quite handy.
  6. You can select the tiles needed one by one to add to your cart or select “Page” to select all items.  Select the Cart where you can download the tiles manually or select the “uGet Instructions” for details about downloading multiple files.  Your data will be delivered in a zip format right away, though Lidar files are large and may require some time to download.

 

lidar_results.JPG

The National Map interface as it appeared when I was selecting my desired area for Lidar data.

Unzip the LAS data for use in your chosen GIS package.  To bring the data into ArcGIS Pro, create a new blank project and name it.  Then, Go to Analysis > Tools > Create LAS dataset from your unzipped .las file, noting the projection (in this case, UTM) and other metadata.  Sometimes you can bring .las files directly into Pro without creating a LAS dataset, but with this NationalMap Lidar data, I found that I needed to create a LAS dataset first.

Then > Insert:  New Map > add your LAS dataset to the new map. Zoom in to see the lidar points.  View your Lidar data in different ways using the Appearance tab to see it as elevation, slope, aspect (shown below), and contours.  Use LAS dataset to raster to convert the Lidar data to a raster.  In a similar way, I added the World Hydro layer so I could see the watersheds in this area, and USA detailed streams for the rivers.

lidar_results2.JPG

Aspect view generated from Lidar data in ArcGIS Pro.

There are many things you can do with your newly downloaded Lidar data:  Let’s explore just a few of those.  First, create a Digital Elevation Model (DEM) and a Digital Surface Model (DSM).  To do this, in your .lasd LAS dataset > LAS Filters > Filter to ground, and visualize the results, and then use LAS Dataset to Raster, using the Elevation as the value field.  Your resulting raster is your digital elevation model (DEM).  Next, Filter to first return, and then convert this to a raster:  This is your digital surface model (DSM).  After clicking on sections of each raster to compare them visually, go one step further and use the Raster Calculator to create a comparison raster:  Use the formula:  1streturn_raster – (subtract) the ground_raster.  The first return result is essentially showing the objects or features on the surface of the Earth–the difference between “bare earth” elevation and the “first return”–in other words, the buildings, trees, shrubs, and other things human built and natural.  Symbolize and classify this comparison surface to more fully understand your vegetation and structures.  In my study area, the difference between the DEM and the DSM was much more pronounced on the north (northeast, actually) facing slope, which is where the pinon and juniper trees are growing, as opposed to the barren south (southwest) facing slope which is underlain by Mancos Shale (shown below).

lidar_veght

Comparison of DEM and DSM as a “ground cover” raster in ArcGIS Pro.

My photograph of the ridgeline, from just east of the study area, looking northwest.  Note the pinon and juniper ground cover on the northeast-facing slopes as opposed to the barren southwest facing slope.

Next, create a Hillshade from your ground raster (DEM) using the hillshade tool.   Next, create a slope map and an aspect map using tools of these respective names.  The easiest way to find the tools is just to perform a search.  The hillshade, slope, and aspect are all raster files.  Once the tools are run, these are now saved as datasets inside your geodatabase as opposed to earlier—when you were simply visualizing your Lidar data as slope and aspect, you were not making separate data files.

Next, create contours, a vector file, from your ground raster (DEM), using the create contours tool.  Change the basemap to imagery to visualize the contours against a satellite image.  To create index contours, use the Contour with Barriers tool.  To do this, do not actually indicate a “barriers” layer but rather use the contour with barrier tool to achieve an “index” contour, as I did, shown below.  I used 5 for the contour interval and 25 (every fifth contour) for the index contour interval.  This results in a polyline feature class with a field called “type”.  This field receives the value of 2 for the index contours and 1 for all other contours.  Now, simply symbolize the lines as unique value on the type field, specifying a thicker line for the index contours (type 2) and a thinner line for all the other contours.

lidar_results4

Next, convert your 2D map to a 3D scene using the Catalog pane.  If you wish, undock the 3D scene and drag it to the right side of your 2D map so that your 2D map and 3D scene are side by side.  Use View > Link Views to synchronize the two.  Experiment with changing the base map to topographic or terrain with labels.  Or, if your area is in the USA like mine is, use the Add Data > USA topographic > add the USGS topographic maps as another layer.  The topographic maps are at 1:24,000 scale in the most detailed view, and then 1:100,000 and 1:250,000 for smaller scales.

 

lidar_results3.JPG

2D and 3D synced views of the contours symbolized with the Contours with Barriers tool in ArcGIS Pro. 

At this point, the sky’s the limit for you to conduct any other type of raster-based analysis, or combine it with vector analysis.  For example, you could run the profile tool to generate a profile graph of a drawn line (as I did, shown below) or an imported shapefile or line feature class, create a viewshed from your specified point(s), trace downstream from specific points, determine which areas in your study site have slopes over a certain degree, or use the Lidar and derived products in conjunction with vector layers to determine the optimal site for a wildfire observation tower or cache for firefighters.

Profile graph of the cyan polyline that I created from the Lidar data from the National Map in ArcGIS Pro.

lidartrace

Tracing downstream using the rasters derived from the lidar data in ArcGIS Pro.

lidar_over40

Slopes over 40 degrees using the slope raster derived from the lidar data in ArcGIS Pro.

I hope these procedures will be helpful to you.