ArcGIS Data ReviewerAdvanced Data Validation Concepts

Francisco Perez & Shankar Chandrasekaran

orkshop Agenda

Importance of Data QualityWhat is ArcGIS Data Reviewer?Automated ValidationExtending Automated ValidationSemi-Automated ReviewAnalyzing & Reporting Data QualityReferences/Resources

hy Quality Control Matters

In 1976 the Teton dam collapsed due to improper materials used in construction.

In 2009 Toyota Recalled 9 Million cars due to faulty floor mat.

In 2010 BP Deepwater Horizon Rig exploded due to a number of quality failures.

In 2015 Refugio State Beach Oil Spill -The pipeline did not have an automatic shut-off valve that would have been required on an intrastate pipeline

arbage in Garbage Out

Buildings

Roads

Water Bodies

arbage in Quality Out

Buildings

Roads

Water Bodies

hat is ArcGIS Data Reviewer?ata Quality Management for ArcGIS

System to support data quality management for ArcGISAutomates quality control processMonitor and Measure data quality

ReviewValidate Summarize

hat is ArcGIS Data Reviewer?latform data quality challenges

GIS Data Server

Data Revie

hat is ArcGIS Data Reviewer?enefits of Data Reviewer

Provides accurate data for your applications

Measure and report data quality

Eliminate costs associated with poor data quality

Automated Validation

utomated Data Quality Ruleslementing business rules with Reviewer Checks

Reviewer Batch Jobs- Define and encapsulate quality requirements- Captures domain knowledge- Designed once and executed many times

- ArcGIS for Desktop, ArcGIS for Server

utomated Validation Categoriesifferent Type of Issues in GIS Data

Attribution

Spatial Relationships

Feature Integrity

Metadata

utomated Validation Business Scenario- Validate Data Submissionata Revieweridate mobile data from data contributors

RequirementsRequirements Automated ChecksAutomated Checks

om Requirements to Data Reviewer Automated Checksata Quality Management for ArcGIS

QA Functional Requirements

Business Needs

• Automated Checks

• Extended

• Semi-Auto

Contributor Portal

quirementsata Reviewer Automated Checks

les:Add information about the corresponding Site Address PointsEnsure that the driveway and building entrance points overlap the corresponding parcel feature For driveway entrance points the location must be in the driveway

tending Automated Validationata Reviewer Automated Checks

idate Existing Data: eck the accuracy of existing Site address nts and identify whether they have been orrectly attributed

Organization

utomated Review Demo

Contributors

Requirements

Results

Batch Job

Contributor Portal

Questions ?

Semi-Automated Review

sual Quality Control ProcessReviewer Service

Write Results

Editor

ArcGIS Server

Reviewer

GeodatabaseUsers

OrganizationSMEs

mi-automated review methods

Positional Accuracy

Data Inspection tools

Visual Review

• Identifying missing or misplaced features

• Random Sampling

mi-automated review methods

Positional Accuracy

Data Inspection Tools

Data Inspection Tools

• Report Features

• Draw Geometries on the map

lue of performing visual review

Finding Missing Features

Compare to Trusted Sources

Discover Patterns

Visual Review

nefits of Visual Quality Control

Get feedback from Subject Matter Experts (SME)

Find issues not detectable by automated validation

Track issues reported

Demo

Visual Review

usiness Scenario –Visual Reviewa Quality Management for ArcGIS

QC Editors

On-Site Field Crew

Visual Quality Control

uestions ?

Report & AnalyzeData Quality

data quality

dtransparency to stakeholders on data qualityr stakeholders to make informed and accurate decisions

vely summarize results based on different perspectiveses are derived from results

data quality

What are the issues on a geographic extent?

How severe are the issues with my data?

How many types of issues do I have in my data?

How many issues have been reviewed and fixed?

How many issues do I have by feature class?

esults

nd non-spatials describe check, problem, and validation informationprovides link back to feature that failed validation

esults

e records that are stored in Reviewer workspacenformation about a feature or a row which failed QC standardultsrs

Resultsutomated alidation

nalyze Architecture

Data Reviewer

Web API

ArcGIS Server

Reviewer Workspace

Dashboard

Reviewer Map Service

REST

API

Results

Analyze - Demo

Contributoration

uestions ?

y management - ArcGIS platform

ta Quality with iewer for Desktopp Pro

with iewer for Server

om Apps with iewer JavaScript API

dard Apps with Builder for ArcGIS

Server

Online Content and Services

Desktop Web Device

ArcGIS OnlinePortal

artates

ewer templates l utilitiesvernmentsourceslities

Esri industry modelsrting point

Day and Description Type Time Location

June 28

Data Reviewer: An Introduction Technical Workshop 8:30 – 9:45am Room 15B

Data Reviewer: Assessing Positional Accuracy Demo Theater 10:30 – 11:15am Demo Theater 4Geodata

Data Reviewer: Integrating ArcGIS Data Reviewer and ArcGIS Workflow Manager to Automate Data Quality Control Workflows

Demo Theater 11:30 – 12:15pm Demo Theater 4Geodata

Data Reviewer: Leveraging Geoprocessing for Data Validation Demo Theater 12:30 – 1:15pm Demo Theater 4

Geodata

Data Reviewer: Quality Assessment for Elevation Raster Datasets Demo Theater 2:30 – 3:15pm Demo Theater 4

Geodata

day June 29

Data Reviewer: An Introduction Technical Workshop 3:15 – 4:30pm Sapphire Ballroom

A/B

y June 30

Data Reviewer: Advanced Data Validation Concepts Technical Workshop 8:30 – 9:45am Room 32 A/B

Data Reviewer: Implementing Data Quality Reporting in Web Clients Demo Theater 10:30 – 11:15am Demo Theater 6

Server

Data Reviewer: Integrating Data Validation Capabilities into Web Applications Demo Theater 12:30 – 1:15pm Demo Theater 6

Server

arn more this week?

rn more?

formationm/datareviewer

rn more?

ation

erver.arcgis.com)

rn more?

ation

raining.esri.com)g Data Quality using ArcGIS Data Reviewer (Seminar)

ng Positional Accuracy Using ArcGIS Data Reviewer for Desktop (Seminar)with ArcGIS: Automating Validation (Web Course)with ArcGIS: Visual Review (Web Course)

Control Using ArcGIS Data Reviewer for Desktop (Instructor-Led)

rn more?

ation

g Data Quality using ArcGIS Data Reviewer (Seminar)ng Positional Accuracy Using ArcGIS Data Reviewer for Desktop (Seminar)with ArcGIS: Automating Validation (Web Course)with ArcGIS: Visual Review (Web Course)

Control Using ArcGIS Data Reviewer for Desktop (Instructor Led)

eonet.esri.com)iewer place

!!

out the session survey using the Esri Events pvance Data Validation Concepts” session

Search Feature to quickly find this title

wn to the “Feedback” sectionfew short questions, enter any comments and bmit”!