Archive

Posts Tagged ‘raster’

Accessing and Using Lidar Data from The National Map

January 8, 2018 Leave a comment

We have written about the USGS data portal NationalMap numerous times in this blog and in our book, but since the site keeps getting enhanced, a re-examination of the site is warranted.  One of the enhancements over the past few years is the addition of Lidar data to the site.  I did some recent testing of searching for and downloading Lidar data on the site and wanted to report on my findings.  For videos of some of these procedures, go to the YouTube Channel geographyuberalles and search on Lidar.

From a user perspective, in my view the site is still a bit challenging, where the user encounters moments in the access and download process where it is not clear how to proceed.  However, (1) the site is slowly improving; (2) the site is worth investigating chiefly because of its wealth of data holdings:  It is simply too rich of a resource to ignore.  One challenging thing about using NationalMap is, like many other data portals, how to effectively narrow the search from the thousands of search results.  This in part reflects the open data movement that we have been writing about, so it is a good problem to have, albeit still cumbersome in this portal.  Here are the procedures to access and download the Lidar data from the site:

  1. To begin:  Visit the National Map:  https://nationalmap.gov/ > Select “Elevation” from this page.
  2. Select “Get Elevation Data” from the bottom of the Elevation page.  This is one of several quirks about the site – why isn’t this link in a more prominent position or in a bolder font?
  3. From the Data Elevation Products page left hand column:   Select “1 meter DEM.”
  4. Select the desired format.  Select “Show Availability”.   Zoom to the desired area using a variety of tools to do so.  In my example, I was interested in Lidar data for Grand Junction, in western Colorado.
  5. Note that the list of  available products will appear in the left hand column.  Lidar is provided in 10000 x 10000 meter tiles.  In my example, 108 products exist for the Grand Junction Lidar dataset.  Use “Footprint” to help you identify areas in which you need data–the footprints appear as helpful polygon outlines.  At this point, you could save your results as text or CSV, which I found to be quite handy.
  6. You can select the tiles needed one by one to add to your cart or select “Page” to select all items.  Select the Cart where you can download the tiles manually or select the “uGet Instructions” for details about downloading multiple files.  Your data will be delivered in a zip format right away, though Lidar files are large and may require some time to download.

 

lidar_results.JPG

The National Map interface as it appeared when I was selecting my desired area for Lidar data.

Unzip the LAS data for use in your chosen GIS package.  To bring the data into ArcGIS Pro, create a new blank project and name it.  Then, Go to Analysis > Tools > Create LAS dataset from your unzipped .las file, noting the projection (in this case, UTM) and other metadata.  Sometimes you can bring .las files directly into Pro without creating a LAS dataset, but with this NationalMap Lidar data, I found that I needed to create a LAS dataset first.

Then > Insert:  New Map > add your LAS dataset to the new map. Zoom in to see the lidar points.  View your Lidar data in different ways using the Appearance tab to see it as elevation, slope, aspect (shown below), and contours.  Use LAS dataset to raster to convert the Lidar data to a raster.  In a similar way, I added the World Hydro layer so I could see the watersheds in this area, and USA detailed streams for the rivers.

lidar_results2.JPG

Aspect view generated from Lidar data in ArcGIS Pro.

There are many things you can do with your newly downloaded Lidar data:  Let’s explore just a few of those.  First, create a Digital Elevation Model (DEM) and a Digital Surface Model (DSM).  To do this, in your .lasd LAS dataset > LAS Filters > Filter to ground, and visualize the results, and then use LAS Dataset to Raster, using the Elevation as the value field.  Your resulting raster is your digital elevation model (DEM).  Next, Filter to first return, and then convert this to a raster:  This is your digital surface model (DSM).  After clicking on sections of each raster to compare them visually, go one step further and use the Raster Calculator to create a comparison raster:  Use the formula:  1streturn_raster – (subtract) the ground_raster.  The first return result is essentially showing the objects or features on the surface of the Earth–the difference between “bare earth” elevation and the “first return”–in other words, the buildings, trees, shrubs, and other things human built and natural.  Symbolize and classify this comparison surface to more fully understand your vegetation and structures.  In my study area, the difference between the DEM and the DSM was much more pronounced on the north (northeast, actually) facing slope, which is where the pinon and juniper trees are growing, as opposed to the barren south (southwest) facing slope which is underlain by Mancos Shale (shown below).

lidar_veght

Comparison of DEM and DSM as a “ground cover” raster in ArcGIS Pro.

My photograph of the ridgeline, from just east of the study area, looking northwest.  Note the pinon and juniper ground cover on the northeast-facing slopes as opposed to the barren southwest facing slope.

Next, create a Hillshade from your ground raster (DEM) using the hillshade tool.   Next, create a slope map and an aspect map using tools of these respective names.  The easiest way to find the tools is just to perform a search.  The hillshade, slope, and aspect are all raster files.  Once the tools are run, these are now saved as datasets inside your geodatabase as opposed to earlier—when you were simply visualizing your Lidar data as slope and aspect, you were not making separate data files.

Next, create contours, a vector file, from your ground raster (DEM), using the create contours tool.  Change the basemap to imagery to visualize the contours against a satellite image.  To create index contours, use the Contour with Barriers tool.  To do this, do not actually indicate a “barriers” layer but rather use the contour with barrier tool to achieve an “index” contour, as I did, shown below.  I used 5 for the contour interval and 25 (every fifth contour) for the index contour interval.  This results in a polyline feature class with a field called “type”.  This field receives the value of 2 for the index contours and 1 for all other contours.  Now, simply symbolize the lines as unique value on the type field, specifying a thicker line for the index contours (type 2) and a thinner line for all the other contours.

lidar_results4

Next, convert your 2D map to a 3D scene using the Catalog pane.  If you wish, undock the 3D scene and drag it to the right side of your 2D map so that your 2D map and 3D scene are side by side.  Use View > Link Views to synchronize the two.  Experiment with changing the base map to topographic or terrain with labels.  Or, if your area is in the USA like mine is, use the Add Data > USA topographic > add the USGS topographic maps as another layer.  The topographic maps are at 1:24,000 scale in the most detailed view, and then 1:100,000 and 1:250,000 for smaller scales.

 

lidar_results3.JPG

2D and 3D synced views of the contours symbolized with the Contours with Barriers tool in ArcGIS Pro. 

At this point, the sky’s the limit for you to conduct any other type of raster-based analysis, or combine it with vector analysis.  For example, you could run the profile tool to generate a profile graph of a drawn line (as I did, shown below) or an imported shapefile or line feature class, create a viewshed from your specified point(s), trace downstream from specific points, determine which areas in your study site have slopes over a certain degree, or use the Lidar and derived products in conjunction with vector layers to determine the optimal site for a wildfire observation tower or cache for firefighters.

Profile graph of the cyan polyline that I created from the Lidar data from the National Map in ArcGIS Pro.

lidartrace

Tracing downstream using the rasters derived from the lidar data in ArcGIS Pro.

lidar_over40

Slopes over 40 degrees using the slope raster derived from the lidar data in ArcGIS Pro.

I hope these procedures will be helpful to you.

 

 

 

Advertisements

Daily Satellite Imagery for Planet Earth

December 1, 2014 1 comment

In this blog, we have written about the revolution occurring in the remote sensing world, centered on inexpensive and crowdsourced remote sensing. As described in this TED talk from Planet Labs’ Will Marshall, Planet Labs has launched small satellites of the dimensions 10 x 10 x 30 cm, weighing 4 kg, which can take images at 10 times higher resolution than conventional large satellites.  Early in 2014, the International Space Station launched 28 of these small satellites.  They plan to launch more than 100 that will image the Earth from a single orbital plane as the planet rotates beneath it.  Will refers to this system as a “line scanner for the planet.”

While our book and this blog discuss geotechnologies from a technical point of view, we also highlight the societal implications of these innovations.  Planet Labs’ work fits in well with these themes, because they are not only technically innovative, but their goal is to democratize remote sensing data.  They are asking:  “If you had access to imagery for the whole planet on a daily basis, what would you do with it?” Every point on the planet will be imaged every day with their platform.

And while the partnerships and avenues of dissemination data are still being worked out, this and similar efforts in the remote sensing world will surely impact data availability, crowdsourcing, copyright, privacy, decision-making, and other topics important to science, education, and society, in the months and years ahead.

 

Planet Labs:  Imagery and its democratization

Planet Labs: Imagery and its democratization.

 

 

High-resolution SRTM elevation data to be released globally

October 12, 2014 1 comment
Comparing SRTM data resolutions

Comparing SRTM data spatial resolution:  90 meters on the left, 30 meters on the right.

High-resolution elevation data from the Shuttle Radar Topography Mission-Level 2 (SRTM-2), previously only available for the USA, will be made publicly available over the next 12 months, the White House announced recently at the United Nations Heads of State Climate Summit. The first elevation data set to be released will be over the African continent and is available on the United States Geological Survey’s Earth Explorer website, by choosing the “SRTM 1 Arc-Second Global” data set, with future regions to be released within the coming year.

“I look forward to the broader impact that the release will have on the global scientific and capacity building community,” said National Geospatial-Intelligence Agency (NGA) Director Letitia Long.  Until now, SRTM data was only publicly available at a lower 90-meter resolution (see above image). The newly-released global 30-meter SRTM-2 dataset will be used worldwide to improve environmental monitoring, climate change research including sea-level rise impact assessments, and local decision support, the White House said.

The SRTM mission began in 2000 as a venture between NASA and NGA that used a modified radar system on board the Space Shuttle Endeavour to acquire elevation data for over 80% of the Earth’s land mass. The Department of Defense and intelligence community continues to use this topographic data for multiple applications – from developing navigation tools and supporting military operations, to geological and environmental purposes.  In August 2014, Long authorized the removal of the Limited Distribution caveat from the SRTM-2 dataset, making it available to the public on a phased-release schedule. The 30-meter topographic dataset was then sent to USGS for public distribution.

When I heard Shuttle pilot Dom Gorie speak about his work with the SRTM at a GIS conference about 10 years ago, it was one of the most memorable keynote addresses I have ever heard.  I look forward to investigating this new data set and the delivery mechanism.  Keep an eye on this blog for further updates.

 

The National Atlas of the USA is Disappearing

July 6, 2014 2 comments

One of the most useful sites of the past 15 years for GIS users, in my judgment, has been the National Atlas of the United States.  It contains a “map maker” that allows you to create online maps of climate, ecoregions, population, crime, geology, and many other layers, and a “map layers” repository that houses all of the raster and vector data layers that are displayable in the map maker.  All of those hundreds of layers are downloadable in standard formats that are easy to use with GIS.

Sadly, the National Atlas is scheduled to disappear on 30 September 2014.  According to the transition FAQ, “the National Atlas and The National Map will transition into a combined single source for geospatial and cartographic information.  This transformation is projected to streamline access to maps, data and information from the USGS National Geospatial Program (NGP).  This action will prioritize our civilian mapping role and consolidate core investments while maintaining top-quality customer service.”  Thus, the National Map is scheduled to be the content delivery mechanism for the National Atlas content.

But, data users take note:  Not all of the National Atlas content is migrating to the National Map.  According to the FAQ’s question of “Will I still be able to find everything from the National Atlas on The National Map web site”, the answer is, “No. Most National Atlas products and services that were primarily intended for a broad public audience as well as thematic data contributions from outside the National Geospatial Program (NGP) will not be available from nationalmap.gov.”

I think this is most unfortunate news.  In my opinion, and that of many students and educators that I work with in courses and institutes, and the other data users I have worked with over the years, the National Map is almost as clunky and difficult to use as it was 10 years ago.  I use it frequently because it is still one of the richest sources of data, but it is by no means easy to obtain that data.  And equally importantly, it serves a different audience than the National Atlas does.   Yes, the National Atlas viewer is dated, but it requires little bandwidth, making it accessible to schools and other institutions contending with poor connectivity. How much effort is required just to leave national atlas alone and leave it online, with an understanding that it will not be updated?

In an era where more geospatial data are needed, not less, and improved geographic literacy is increasingly critical to education and society, the disappearance of the National Atlas seems like a giant step backward.

National Atlas website with Map Maker and just a few of the many data layers available.

National Atlas website with Map Maker and just a few of the many data layers available.

Spatial Analyst videos describe decision making with GIS

I have created a series of 22 new videos describe decision making with GIS, using public domain data.  The videos, which use the ArcGIS Spatial Analyst extension, are listed and accessible in this YouTube playlist.  Over 108 minutes of content is included, but in easy-to-understand short segments that are almost entirely comprised of demonstrations of the tools in real-world contexts.  They make use of public domain data such as land cover, hydrography, roads, and a Digital Elevation Model.

The videos include the topics listed below.  Videos 10 through 20 include a real-world scenario of selecting optimal sites for fire towers in the Loess Hills of eastern Nebraska, an exercise that Jill Clark and I included in the Esri Press book The GIS Guide to Public Domain Data and available online.

New Spatial Analyst videos explain how to make decisions with GIS.

New Spatial Analyst videos explain how to make decisions with GIS.

1)  Using the transparency and swipe tools with raster data.
2)  Comparing and using topographic maps and satellite and aerial imagery stored locally to the same type of data in the ArcGIS Online cloud.
3)  Analyzing land cover change with topographic maps and satellite imagery on your local computer and with ArcGIS Online.
4)  Creating a shaded relief map using hillshade from a Digital Elevation Model (DEM).
5)  Analyzing a Digital Elevation Model and a shaded relief map.

6)  Creating contour lines from elevation data.
7)  Creating a slope map from elevation data.
8)  Creating an aspect (direction of slope) map from elevation data.
9)  Creating symbolized contour lines using the Contour with Barriers tool.
10)  Decision making using GIS:  Introduction to the problem, and selecting hydrography features.

11) Decision making using GIS:  Buffering hydrography features.
12)  Decision making using GIS:   Selecting and buffering road features.
13)  Decision making using GIS:  Selecting suitable slopes and elevations.
14)  Decision making using GIS:  Comparing Boolean And, Or, and Xor Operations.
15)  Decision making using GIS:   Selecting suitable land use.

16)  Decision making using GIS:  Selecting suitable land use, slope, and elevation.
17)  Decision making using GIS:   Intersecting vector layers of areas near hydrography and near roads.
18)  Decision making using GIS:  Converting raster to vector data.
19)  Decision making using GIS:  Final determination of optimal sites.
20)  Creating layouts.

21)  Additional considerations and tools in creating layouts.
22)  Checking Extensions when using Spatial Analyst tools.

How might you be able to make use of these videos and the processes described in them in your instruction?

Historical and seasonal Landsat data: WELD Resource

The Web-enabled Landsat Data (WELD) project generates 30-meter composites of Landsat 7 Enhanced Thematic Mapper Plus (ETM+) terrain corrected (Level 1T) mosaics at weekly, monthly, seasonal and annual periods for the conterminous United States (CONUS) and Alaska. These mosaics provide consistent data that can be used to derive land cover as well as geophysical and biophysical products for regional assessment of surface dynamics and to study Earth system functioning.

Landsat WELD resource

Landsat WELD resource

A collaboration between the United States Geological Survey (USGS) Earth Resources Observation and Science (EROS) Center and academic partner South Dakota State University Geographic Information Science Center of Excellence, this is an excellent resource for all who seek to compare land use through time and through seasonal variation using Landsat data in the continental USA and in Alaska.  The WELD documentation site describes the WELD products on the site, known issues, and future plans.

WELD products are available as custom GeoTiff subsets via a new interactive web ordering system and as tiled HDF products via FTP.  I found the site fairly intuitive, simple, and straightforward to use.  Its products are directly importable into GIS software and hence it provides much more than visualizations, but rather, products useful to the GIS analyst.  The “good news, bad news” is that the GIS data user is confronted with an array of Landsat sites from which they may obtain data.  Each has its own interface and formats, but the situation is still far better than 10 years ago when nearly all of it was either for fee or difficult to obtain. Because it is not well linked to other sites, the WELD site is difficult to “stumble across” unless the data user is familiar with the acronym.  However, it is well worth a visit as it is one of the most intuitive and resource-rich.

Maps as representations of reality: The deciduous-coniferous tree “line”

November 5, 2012 3 comments

One of the themes running through our book The GIS Guide to Public Domain Data is that maps are representations of reality.  While almost everyone reading this statement is likely to agree with it, in the fast-paced world that GIS analysis and creating maps has become, it is easy to lose sight of this fact when staring at tables, maps, and imagery.  In a recent video, I discuss just one place where care needs to be made in making decisions based on spatial data.  In the video, observe my surroundings as I stand near the traditional “line” that divides the deciduous forest to the south from the coniferous forest to the north in North America. Is the “line” really a line at all, or is it better described as a gradual change from deciduous to coniferous as one travels north?  Is that vector line then better symbolized as a “zone”, or is vegetation better mapped as a raster data set, with each cell representing the percentage of deciduous and coniferous trees?

How many other data sets do we tend to see as having firm boundaries, when the boundaries are not really firm at all in reality?  How does that affect the decisions we make with them?  Even the boundary between wetlands and open water were originally interpreted based on land cover data or a satellite or aerial image.   As we state in the book, even contour lines were often interpreted originally from aerial stereo pairs.  And each data set was collected at a specific scale, with certain equipment and software, at a specific date, and within certain margins of error that the organization established.  Maps are representations of reality.  They are incredibly useful representations to be sure, but care needs to be taken when using this or any abstracted data.