Ensuring data quality through the ArcGIS Data Reviewer

One of the central themes of our book and this blog has been data quality. And for good reason! Data provide the foundations behind the workflows that people create as they use geospatial and other tools that result in decisions being made that impact people, their communities, and the environment. Coupled with data quality is the fact that more organizations are sharing their data with those outside their own organization. How can an organization understand the quality of the data they are using and serving, and be informed about any potential data sets that do not meet a standard? ArcGIS Data Reviewer, an extension to ArcGIS Pro and ArcGIS Enterprise, reduces the risk of using low-quality data by implementing workflows that highlight data that does not meet specified quality requirements, leading to increased confidence in data, enhanced productivity, and reduced costs.

For more, see this essay from my Esri colleague Jay Cary: https://www.esri.com/arcgis-blog/products/data-reviewer/data-management/data-quality-matters-arcgis-data-reviewer-esri/

In addition, I highly recommend this series of videos in the data reviewer channel. One of these videos starts with why data quality matters–to reduce risk, help make decisions more confidently, drive increased usage, and maximize productivity of the organization, all of which as an instructor I find very useful grounding for “why I should care about this in the first place.” There is also an active Esri Community space on the Data Reviewer, with questions, answers, new releases, and more, here.

An organizational leader, professor, or someone else reading this is probably thinking “I don’t have time to routinely examine all the geometry and attributes of every single data set I am generating and using”, and rightly so. Well, one of the tool’s capabilities is automated data review — a capability that evaluates a feature’s quality without human intervention. The Data Reviewer aids with: How to take data requirements and convert those into automated checks? The Data Reviewer contains over 37 configurable checks. The include feature attributes (checking for outliers, mis-formatted strings, invalid domain values, and so on), feature integrity (especially helpful if you are importing from another source), and spatial relationships (such as “this feature should not overlap that feature”). Data Reviewer provides a library of checks to validate data based on your unique quality requirements. The checks are designed to assess various aspects of a feature’s quality, including its attribution, integrity, or spatial relationship to other features.  The checks can be implemented when editing, and/or after the data has been created. The ArcGIS Data Reviewer also engages subject matter experts, and can help track, manage, and monitor data quality.

The helpful poster below can be found on: https://pro.arcgis.com/en/pro-app/latest/help/data/validating-data/pdf/data-reviewer-poster.pdf

Data Reviewer enables the management and tracking of errors through a defined lifecycle. Data Reviewer also tracks the details of who, when and how errors are corrected and whether the correction has been verified as acceptable. This additional information enables a manager to report progress on data quality goals and forecast when quality goals will be achieved. How can a manager efficiently review these error reports? The error reports indicate who found it, who checked it, what was done, and more.

This landing page about the Data Reviewer contains some compelling user stories from a utility in Utah and a water department in Arizona, among others, about how organizations have implemented the Data Reviewer. We have a dynamic planet, changing needs and requirements of organizations and their stakeholders, and changing tools within GIS, all of which make data quality an elusive goal. However, I am confident that with tools like the Data Reviewer, data quality can be as high as possible, minimizing risk and serving people’s needs.

I look forward to hearing your comments.

–Joseph Kerski

Notes from the GeoAI and the Future of Mapping: Implications for 21st-Century Digital Resilience Symposium

I recently attended a workshop from the National Academy of Sciences on GeoAI and the Future of Mapping: Implications for 21st-Century Digital Resilience and, as its topic relates to our book and this blog, felt that it was appropriate to share what occurred and my reactions.

Geospatial science and technologies can be core to forging a path toward resilience. New innovations in digital data collection, spatial analysis and geovisualization now allow us to map and understand human impacts at global to local scales, to identify big‐picture patterns and processes and to generate actionable geographic knowledge. However, if new geospatial mapping and tools are to help communities, the tools must also be resilient. Digital resilience means that to the greatest extent possible, the data and tools that communities use to build resilience should be freely accessible, up‐to‐date, interchangeable, operational, explainable and principled in the sense that they are consistent with theory and settled science. 

This meeting of the Geographical and Geospatial Sciences Committee addressed the implications of the future of mapping for digital resilience with a specific focus on geoAI, the integration of location, spatial relations, and place to the broader paradigm of AI. This promising and yet disruptive digital technology may help to automate the understanding and prediction of spatial patterns, tendencies, and relationships and to predict the location of future behaviors, events, or trends. Can we leverage these new advances in geospatial science and technologies to build greater digital resilience, and ultimately community resilience, to the shocks and disruptions that will occur in a world facing accelerating change?

Some of the most innovative professors I know in the community spoke including Dr Gao from the University of Wisconsin, Dr Nelson from UCSB, Bo Zhao from the University of Washington, and others, as well as scholars from the US Census Bureau, the USGS, and Dr Dawn Wright and Andrew Turner from Esri.

For more details about the speakers and their topics, see:

https://www.nationalacademies.org/documents/embed/link/LF2255DA3DD1C41C0A42D3BEF0989ACAECE3053A6A9B/file/D3735D428198AACDD84F77494A9629ECB9E4EC66B25E?noSaveAs=1

Pat McDowell and Harvey Miller, Co-Chairs, Geographical and Geospatial Sciences Committee, and Kristen Kurland began the symposium. Esri Chief Scientist Dr Dawn Wright started the symposium’s content discussions with some excellent challenges for the community and presented a focused way to think about all this, including a wider notion of resiliency–data resiliency. This theme ran through all workshops, so I thought it was very useful that Dr Wright spent time defining it.

This includes the following tenets:

• Making the data and code available is not enough – we need to share the workflows.
• Integrate, integrate, integrate – via interoperability and crosswalking.
• Findable accessible interoperable – and reusable – that is, the FAIR principles (which we wrote about, here). To make data reproducible means in part to make it virtual.   This includes the use of digital object identifiers to connect journals, containers, and more.

• Ease leads to exposure, and exposure leads to adoption (arising from some statements from “Yoda”!).
• Make it explainable. 
• Let ethics, empathy, and equity guide you.
• Promote a culture of sharing, engagement, and collaboration.

More of Dr Wright’s writings on this topic are here:  https://esriurl.com/resilientdata. We have also reflected upon data resiliency in this essay in Spatial Reserves.

Other speakers discussed exemplars for basic and applied science, and opportunities and risks for GeoAI for digital resilience. The discussions illustrated how in many cases given the relative newness of much of the topic that the community is still settling and deciding on the terms of definitions: How can we conceptualize geo-AI in light of rapid change? This symposium also showed how important these topics are to all of us in education, and beyond – in the greater society. GeoAI has enormous implications for ethics, as we have written about many times in this blog–see these essays for example. The focus of the symposium was on the positive benefits GeoAI brings to R&D and instruction but it recognized the need for caution and numerous examples were sprinkled in as to why this caution is needed.

To me, much of these important discussions and attention reinforces what we are always telling students and our colleagues: Being a critical thinker about methods, data, workflows, tools matters more now than ever before.

–Joseph Kerski