Archive

Posts Tagged ‘ArcGIS Pro’

A review of the ArcGIS Pro Cookbook

May 14, 2018 4 comments

GIS Professional Tripp Corbin’s book, the “ArcGIS Pro 2.x Cookbook” (2018, Packt Publishing) is new but I believe will quickly become a valued and oft-used resource. Mr Corbin’s goal in writing this extensive (694 pages) resource is to help GIS professionals “create, manage, and share geographic maps, data, and analytical models using ArcGIS Pro.” The audience for this book includes all who are learning GIS, or learning Pro, as well as those migrating from ArcMap to Pro.

Tripp’s “cookbook” theme is evident throughout the book’s format, where in each section and problem to be solved, he shows how to get ready, how to do it, how it works, and … “there’s more” (additional resources). That the book is from Packt is excellent, because Packt (www.packtpub.com) offers eBook versions of every one of its books, and also offers newsletters and tech articles. That Tripp is a full time trainer and instructor is evident–he understands the challenges in learning a rapidly-changing and complex technology inherent in GIS with just enough tips to keep the reader engaged. He also encourages the reader to think about how to apply each tool and method to his or her own work. He offers the reader the ability to download the sample data for the book, and the data bundle is also on GitHub. He also includes PDFs of all images of screen shots and diagrams.

I like Tripp’s approach because, similar to my own instruction, and central to the theme of this blog, he starts with data. He’s not hesitant to discuss the benefits but also the limitations of each data format such as shp, gdb, and CAD files. He spends quality time in the book helping the reader understand how to convert data to the format that best fits his or her needs. His sections on linking tables from outside sources to existing data, on editing (in particular, a focus on topologies to improve data accuracy and increasing editing efficiency), and on 2D and 3D analysis are very helpful. I was pleased to see much attention to what I consider to be a chief advantage of Pro–the ability to more easily share content from Pro to ArcGIS Online and hence the wider community. Another wonderful new function in ArcGIS Pro is also included in the book–writing and using Arcade scripts, applied to symbology, classification, and analysis.

As a GIS book author, I know the challenges faced in writing such a book–what should be included, and what should be left out? Tripp does a nice job here as well, including the fundamentals that most users will touch. The book’s chapters include: 1: Capabilities and terminology. 2: Creating and storing data. 3: Linking data together. 4: Editing spatial and tabular data. 5: Validating and editing data with topologies. 6: Projections and coordinate systems. 7: Converting data from one format to another. 8: Proximity analysis. 9: Spatial statistics and hot spots. 10: 3D maps and 3D analyst. 11: Arcade, labeling and symbology expressions. 12: ArcGIS Online, 13: Publishing your own content to ArcGIS Online. 14: Creating web apps using ArcGIS Online.

These chapters cover a great deal of ground. In the editing chapter, for example (Chapter 4), configuring editing options, reshaping existing, splitting, merging, aligning, creating new point line polygon features, creating new polygon feature using autocomplete, and editing attributes using attribute pane and in the table view, are all examined. The examples in the book are interesting and relevant, and not without some humor (Trippville is a community that is often studied). In my view, the book contains just the right amount of graphics. Tripp provides answers to the questions he poses, and then gives the explanation for each answer. Despite the “recipes” provided in the cookbook, not all of them require the previous recipe to be used, which is excellent for all of us in GIS who have limited time and want to select sections in a non-sequential order.

I highly recommend using this book in conjunction with Tripp’s other book on this topic, “Learning ArcGIS Pro.” The Learning book focuses on installing, assigning licenses, navigating the interface, creating and managing projecrts, creating 2D and 3D maps, authoring map layouts, importing existing projects, creating standardized workflows using tasks, and automating analysis and processes using modelbuilder and python. The Learning ArcGIS Pro book ideally should be used first, before the ArcGIS Pro 2.x Cookbook, but if you are pressed for time, these two books could be used in tandem. Keep both of them handy–they will be very useful to you.

tripp4

Covers of Tripp Corbin’s ArcGIS Pro books. 

tripp3

An example of the detail provided in Tripp Corbin’s ArcGIS Pro 2.x Cookbook. 

Advertisements

Using the Data Interoperability Extension to import SDTS DLG files into ArcGIS Pro

April 16, 2018 Leave a comment

One of the themes of this blog and our book has been the wide variety of spatial data formats in existence.  Some of these spatial data formats have remained challenging to import into a GIS right up to the present day.  To meet this challenge, Esri’s Data Interoperability Extension has been a longstanding and useful set of tools that enables a wide variety of spatial data formats to be imported for use in a GIS.  It is an integrated spatial ETL (Extract, Transform, and Load) toolset that runs within the geoprocessing framework using Safe Software’s FME technology. It enables you to integrate data from multiple sources and formats, use that data with geoprocessing tools, and even publish it with ArcGIS Server.

I recently tested the Data Interoperability Extension in ArcGIS Pro and was thrilled with the results.  Read about how to install and authorize the extension here.  The extension does many things, but one that is particularly useful is that the extension creates a toolbox directly in ArcGIS Pro (graphic below).  I used this toolbox’s Quick Import tool to import a SDTS Format DLG (USGS Digital Line Graph) file directly to a file geodatabase.  The tool, like other ArcGIS Pro geoprocessing tools, walked me right through the process:  Data Interoperability > Quick Import > I then pointed to my DLG files in SDTS format > I named the resulting gdb (file geodatabase).  Once imported, I was then able to work with my hydrography, hypsography, roads, boundaries, and other data.

DLG files have existed since the early 1990s.  Why are we still working with them?  The reasons are that (1) They are dated but still useful vector data sets; (2) Many geospatial data portals still host data only in this format, such as the USGS Earth Explorer.  Another way to import these DLG files into ArcGIS Pro or ArcMap is to use the DLG2SHP tools that I wrote about in this set of guidelines using a standalone program.  See below for step-by-step instructions with the Data Interoperability Extension with screen shots.

data_interoperability

1. Use Toolboxes > Data Interoperability Tools > Quick Import, as shown above.data_interoperability_use_for_dlg_screen1

2.  Using QuickImport pulls up a “specify data source” dialog box, as shown above.

data_interoperability_use_for_dlg_screen3

3.  In the specify data source dialog box, use “find other source” and then specify SDTS format.

data_interoperability_use_for_dlg_screen2

4.  Selecting SDTS format.

data_interoperability_use_for_dlg_screen4

5.  Pointing to the SDTS file (after it has been unzipped and un-TAR’d) and saving it into a geodatabase.

data_interoperability_use_for_dlg_screen5

6. Once the file has been imported into a geodatabase, it can be added to a new map in ArcGIS Pro.  The data is now ready for use, as shown for this hydrography example, above. 

 

Accessing and Using Lidar Data from The National Map

January 8, 2018 Leave a comment

We have written about the USGS data portal NationalMap numerous times in this blog and in our book, but since the site keeps getting enhanced, a re-examination of the site is warranted.  One of the enhancements over the past few years is the addition of Lidar data to the site.  I did some recent testing of searching for and downloading Lidar data on the site and wanted to report on my findings.  For videos of some of these procedures, go to the YouTube Channel geographyuberalles and search on Lidar.

From a user perspective, in my view the site is still a bit challenging, where the user encounters moments in the access and download process where it is not clear how to proceed.  However, (1) the site is slowly improving; (2) the site is worth investigating chiefly because of its wealth of data holdings:  It is simply too rich of a resource to ignore.  One challenging thing about using NationalMap is, like many other data portals, how to effectively narrow the search from the thousands of search results.  This in part reflects the open data movement that we have been writing about, so it is a good problem to have, albeit still cumbersome in this portal.  Here are the procedures to access and download the Lidar data from the site:

  1. To begin:  Visit the National Map:  https://nationalmap.gov/ > Select “Elevation” from this page.
  2. Select “Get Elevation Data” from the bottom of the Elevation page.  This is one of several quirks about the site – why isn’t this link in a more prominent position or in a bolder font?
  3. From the Data Elevation Products page left hand column:   Select “1 meter DEM.”
  4. Select the desired format.  Select “Show Availability”.   Zoom to the desired area using a variety of tools to do so.  In my example, I was interested in Lidar data for Grand Junction, in western Colorado.
  5. Note that the list of  available products will appear in the left hand column.  Lidar is provided in 10000 x 10000 meter tiles.  In my example, 108 products exist for the Grand Junction Lidar dataset.  Use “Footprint” to help you identify areas in which you need data–the footprints appear as helpful polygon outlines.  At this point, you could save your results as text or CSV, which I found to be quite handy.
  6. You can select the tiles needed one by one to add to your cart or select “Page” to select all items.  Select the Cart where you can download the tiles manually or select the “uGet Instructions” for details about downloading multiple files.  Your data will be delivered in a zip format right away, though Lidar files are large and may require some time to download.

 

lidar_results.JPG

The National Map interface as it appeared when I was selecting my desired area for Lidar data.

Unzip the LAS data for use in your chosen GIS package.  To bring the data into ArcGIS Pro, create a new blank project and name it.  Then, Go to Analysis > Tools > Create LAS dataset from your unzipped .las file, noting the projection (in this case, UTM) and other metadata.  Sometimes you can bring .las files directly into Pro without creating a LAS dataset, but with this NationalMap Lidar data, I found that I needed to create a LAS dataset first.

Then > Insert:  New Map > add your LAS dataset to the new map. Zoom in to see the lidar points.  View your Lidar data in different ways using the Appearance tab to see it as elevation, slope, aspect (shown below), and contours.  Use LAS dataset to raster to convert the Lidar data to a raster.  In a similar way, I added the World Hydro layer so I could see the watersheds in this area, and USA detailed streams for the rivers.

lidar_results2.JPG

Aspect view generated from Lidar data in ArcGIS Pro.

There are many things you can do with your newly downloaded Lidar data:  Let’s explore just a few of those.  First, create a Digital Elevation Model (DEM) and a Digital Surface Model (DSM).  To do this, in your .lasd LAS dataset > LAS Filters > Filter to ground, and visualize the results, and then use LAS Dataset to Raster, using the Elevation as the value field.  Your resulting raster is your digital elevation model (DEM).  Next, Filter to first return, and then convert this to a raster:  This is your digital surface model (DSM).  After clicking on sections of each raster to compare them visually, go one step further and use the Raster Calculator to create a comparison raster:  Use the formula:  1streturn_raster – (subtract) the ground_raster.  The first return result is essentially showing the objects or features on the surface of the Earth–the difference between “bare earth” elevation and the “first return”–in other words, the buildings, trees, shrubs, and other things human built and natural.  Symbolize and classify this comparison surface to more fully understand your vegetation and structures.  In my study area, the difference between the DEM and the DSM was much more pronounced on the north (northeast, actually) facing slope, which is where the pinon and juniper trees are growing, as opposed to the barren south (southwest) facing slope which is underlain by Mancos Shale (shown below).

lidar_veght

Comparison of DEM and DSM as a “ground cover” raster in ArcGIS Pro.

My photograph of the ridgeline, from just east of the study area, looking northwest.  Note the pinon and juniper ground cover on the northeast-facing slopes as opposed to the barren southwest facing slope.

Next, create a Hillshade from your ground raster (DEM) using the hillshade tool.   Next, create a slope map and an aspect map using tools of these respective names.  The easiest way to find the tools is just to perform a search.  The hillshade, slope, and aspect are all raster files.  Once the tools are run, these are now saved as datasets inside your geodatabase as opposed to earlier—when you were simply visualizing your Lidar data as slope and aspect, you were not making separate data files.

Next, create contours, a vector file, from your ground raster (DEM), using the create contours tool.  Change the basemap to imagery to visualize the contours against a satellite image.  To create index contours, use the Contour with Barriers tool.  To do this, do not actually indicate a “barriers” layer but rather use the contour with barrier tool to achieve an “index” contour, as I did, shown below.  I used 5 for the contour interval and 25 (every fifth contour) for the index contour interval.  This results in a polyline feature class with a field called “type”.  This field receives the value of 2 for the index contours and 1 for all other contours.  Now, simply symbolize the lines as unique value on the type field, specifying a thicker line for the index contours (type 2) and a thinner line for all the other contours.

lidar_results4

Next, convert your 2D map to a 3D scene using the Catalog pane.  If you wish, undock the 3D scene and drag it to the right side of your 2D map so that your 2D map and 3D scene are side by side.  Use View > Link Views to synchronize the two.  Experiment with changing the base map to topographic or terrain with labels.  Or, if your area is in the USA like mine is, use the Add Data > USA topographic > add the USGS topographic maps as another layer.  The topographic maps are at 1:24,000 scale in the most detailed view, and then 1:100,000 and 1:250,000 for smaller scales.

 

lidar_results3.JPG

2D and 3D synced views of the contours symbolized with the Contours with Barriers tool in ArcGIS Pro. 

At this point, the sky’s the limit for you to conduct any other type of raster-based analysis, or combine it with vector analysis.  For example, you could run the profile tool to generate a profile graph of a drawn line (as I did, shown below) or an imported shapefile or line feature class, create a viewshed from your specified point(s), trace downstream from specific points, determine which areas in your study site have slopes over a certain degree, or use the Lidar and derived products in conjunction with vector layers to determine the optimal site for a wildfire observation tower or cache for firefighters.

Profile graph of the cyan polyline that I created from the Lidar data from the National Map in ArcGIS Pro.

lidartrace

Tracing downstream using the rasters derived from the lidar data in ArcGIS Pro.

lidar_over40

Slopes over 40 degrees using the slope raster derived from the lidar data in ArcGIS Pro.

I hope these procedures will be helpful to you.