GIS & Tax Collections Chad Hoover, GISP Steuben County GIS Coordinator.
GIS Database ManagementGIS Database Management in a Near Real-Time World53 Mary Buffington, GISP,...
Transcript of GIS Database ManagementGIS Database Management in a Near Real-Time World53 Mary Buffington, GISP,...
GIS Database Management in a Near Real-Time World
Mary Buffington, GISP, Project Manager
Eastern Shore Regional Shore Cooperative (ESRGC)
Doug Adams, GISP, GIS Program Manager
Baltimore County Department of Public Works
Maryland Society of Surveyors53rd Annual Fall Conference &
Exhibition
Jedi Surveying
•Mason Dixon Boundary - 1767• Survey of the Coast – 1807
• U.S. Coast and Geodetic Survey – 1878• National Geodetic Service - 1970
•National Spatial Reference System• Horizontal Datums
• North American Datum of 1927 (NAD 27)• North American Datum of 1983 (NAD 83)• North American Datum of 1983 HARN (NAD 83 HARN)• North American Terrestrial Reference Frame of 2022 (NATRF2022)
• Vertical Datums• National Geodetic Vertical Datum of 1929 (NGVD 29)• North American Vertical Datum of 1988 (NAVD 88)• North American-Pacific Geopotential Datum of 2022 (NAPGD2022)
Defining the Boundaries of Practice …
•Common Interest and concerns• They create and use map-related information• They serve the public• They are concerned for the health, safety and welfare of the
public• They share interest in technologies for location-based data• They provide answers to questions that start with: “Where?”
https://www.urisa.org/clientuploads/directory/Documents/Advocacy/DefiningBoundaries_WhitePaperDec2018.pdf
Issues Observed by Surveyors Concerning Use of GIS
•Using GIS maps and data to determine precise locations on the ground•Using GIS maps and data without being aware of their quality
• Accuracy• Currency• Data Source• Method of Compilation
This information is referred to as metadata.
Surveyors’ Expertise Includes
•Measuring and recording the location of boundaries, fixed works and terrain•Assessing and mitigating the locational errors in maps•Producing AUTHORITATIVE (source) data/documents for the location of boundaries, monuments, reference points, fixed works, elevation, and the shape of the earth• Interpreting cadastral law and practices
GIS Professionals’ Expertise Includes:
•Creating geospatial databases and software applications•Analyzing the location and characteristics of mapped features•Compiling and combining geographic data into new, map-based visualizations•Developing new spatial data from analysis of multiple sources•Managing GIS systems and operations
Derivative
Derivative 1
Derivative 1.1
Derivative 1.2
Derivative 2
Derivative 2.1
Derivative 2.2
Property Line Survey
PlatCadastral
Map
Plats Feature Class
Subdivision Feature Class
Zoning Map
Master Water & Sewer Plan
Property Feature Class
… and the list goes on. Most of my organizations non-asset data can be linked back to property data in some way.
Survey Control
Utilities
Sewer
Water
Storm
Transportation
Street Centerlines
Edge of Pavement
Changing Technology
•Manual vs. Analytical vs. Softcopy photogrammetry• Tablet digitizing vs. Heads-up digitizing• Film camera vs Digital camera•Airplane vs. Drone• ...and it’s going to continue to change
Geospatial Resources
•Plats.Net (http://www.plats.net/pages/index.aspx)•Maryland Land Records (https://mdlandrec.net/main/index.cfm)•Map Service Center (https://msc.fema.gov/portal/home)
National Geospatial Data Act 2018Codifies• Federal Geographic Data Committee (DOI) - 1990• National Spatial Data Infrastructure - 1994• National Geospatial Advisory Committee - 2008
• Data Themes• Geospatial Data Standards
• Geospatial Data Portal – GeoPlatform 2011 (https://www.geoplatform.gov/)
2M
• Organizational tool to store and manage data• Data organization using feature datasets
• Set coordinate system of the data• Groups similar data together - sewer, water, stormwater, etc.
• Allows tabular rules for data integrity• Domains - set of valid values for data
• Pipe material: CMP, PVC, etc.• Pipe size: 8 inch, 12 inch, etc.• Set default
• Subtypes - classify different types of features• Building type - primary, secondary• Road - paved, gravel, dirt• Set default
Geodatabase Benefits
• Allows spatial rules for data relationships• Topology - defines and enforces data integrity rules
• “Gravity Mains must not intersect”• “Gravity Mains must not have dangles”• “Manholes must not overlap”• “Parcels must not have gaps”
• Geometric Network - model common networks and infrastructure in the real world• Find disconnected network elements• Find loops or circuits• Calculate the shortest path upstream from one point to
another• Google maps!
Geodatabase Benefits
• Relationship Class• Objects can relate to other objects (buildings related to
parcels)• Objects can relate to tables (parcels can be associated with
owner information)• Manage the associations between objects in one class to
objects in another class• Parcel data related to Real Property Data
Geodatabase Benefits
Geodatabase Design
• Features as points, lines, polygons•Continuous surfaces - rasters, TIN...•Map graphics - text labels (annotation)•How do these items relate?•Model the user’s view
• Identify the functions that support the organization’s goals and objectives
• Identify the data required to support the functions• Organize the data into logical sets of features
Database Types• Enterprise
• Data stored in SQL Server, Oracle, etc.• Recommended for data stored and managed in relational database• Storage size limited to DBMS limits
• File Geodatabase• Recommended for data stored and managed in file system folder• Size limited to 2 TB per dataset (can be raised to 256 TB)
• Personal Geodatabase• Original format for gdb stored and managed in Access data files• File limit size of 2 GB• Concurrent use eventually degrades for large numbers of readers
File Types
• File Geodatabase (FGDB)• One editor• Many Users
•Personal Geodatabase (PGDB)• One editor• One User
• Shapefile - very prevalent...very limiting• Triangular Irregular Network (TIN)•GRID
Lidar Data Storage
Quality LevelPoints Per
MeterLAZ
(sq mi)
LAS(2.5X)(sq mi)
LAS(10X)
(sq mi)
Baltimore County
(682 sq mi)
1 8+ 0.38 GB 0.95 GB 3.8 GB 648–2591 GB
2 2-8 0.16 GB 0.4 GB 1.6 GB 273-1091 GB
3 1-2 0.05 GB 0.125 GB 0.5 GB 85-341 GB
In comparison, a days worth of data from a single CORS station is:• 1 second interval – 25 MB• 5 second interval - 5 MB
Geodatabase Features
•Points, lines, polygons• Feature type is dependent on scale
• River - line vs polygon• City - point vs polygon
• Topology• Within a layer and between layers• Reinforces geographic relationship between features
Importing data to a Geodatabase
• Shapefiles• CAD(ish)• Other geodatabase feature classes/features• Tables (coming soon in Baltimore County…)• Raster data
• Raster datasets• Raster catalog • Mosaic dataset
• ...the list goes on • If you can import it - you can export it!
Attribute Table
•Provide information about the feature• Set of data elements arranged in rows and columns
• Row = one record/feature• Column = Field
• Joins and Relates• “Join” geographic features and tables based on a common field
• ACCTID...• One to one - One owner owns one parcel
• Relate - establishes a connection between records in two tables• One to many - one parcel has many permits
Data Types- Raster/Vector
•Vector• Points, lines, polygons• Is the feature represented on a map?• Is the shape of a feature important?• Will the feature have different representations at different scales?
•Raster• Cells with values, continuous• DEM, imagery…• LiDAR not meant to be survey grade
Attachments
Advantages• Stored in geodatabase• Security• Enable deduplication•Any application supported format
• Image• Video• Word• Excel
https://bc-gis.maps.arcgis.com/apps/webappviewer/index.html?id=66c733159a024a9cbb20134f93610b4e
Related Objects•Attachments - deduplication!• External Data (joins, relates, etc.)• Hyperlinks
Data Models
• Local Government Information Model (LGIM)• Transportation•Water Utilities• ElectricUse as a guide - easily customizable
http://appsforms.esri.com/products/download/index.cfm?fuseaction=download.all
2D
Baltimore CountyDPWData Themes
Vertical Data Models
Sewer Treatment
Plants
Sewer Pump
Stations
Sewer Grinder Pumps
Water Treatment
Plants
Water Wells
Dams Levees
Theme Documentation - Spreadsheet
• Attribute Listing• Logical Model• Physical Model• Domains• Relationship Classes• Data Quality• Metadata• Business Rules• Schema Modifications• Topology
Theme Documentation – WordOverview• Purpose• Description• Stakeholders
• Program Manager• Support Staff
• Update Frequency• Date of Last Update• Supported Program(s)• Asset Group• Requirements• Data Locations• Views• Applications• Business Rules• Tools• Source Materials
Overview (con’t)• Geodatabase Development• Schema• Layer Files• Definition Queries• Map Documents• Queries• Supporting Documents
Map Production• Page Size• Marginalia• Symbology• Special Notes• Description
Geodatabase Development• Compilation Rules• Assumptions• Remaining Tasks• Compilation Statistics• Questions• Compilation Issues
Maintenance• Maintenance Workflows• Metadata Updates
Maintenance Record
File Management – Theme Based• Applications – Documentation and specifications for application used to support the creating, maintenance
and quality control of this data.• Documentation – This folder is where the Geodatabase Design (xls) and Geodatabase Documentation (docx)
documents should be stored. Any schema change documents should also be located in this folder. • Expressions – This folder should contain any expressions used in map production, definition queries, etc. saved
from ArcMap.• Historic – This folder should contain any documentation created prior to the current documentation standard.
Folders can be created as required to organize the materials. The contents of this folder can be compressed.• Layer Files – This folder should contain any layer files created for web applications or map production. The
layer files should be documented in the geodatabase documentation document.• Maps – This folder should contain any maps produced specifically from the related data. Subfolders can be
created for successive years as required.• Maintenance – This folder should include the maintenance record (if not included in the geodatabase
documentation document) and source materials.• Research – This folder should contain information on research performed related to the design, compilation or
maintenance of the geodatabase. The contents of this folder can also be compressed.• Scripts – This folder should contain any scripts used in the compilation, maintenance, presentation of the data.• Source Material – This folder should contain the source material used in the compilation of the data. The
folder should be organized to sufficiently ensure the content is clearly understood.• Views – The documentation and specification for materialized or dynamic views created specifically for this
data. 2M
https://tidesandcurrents.noaa.gov/map/index.html
https://mdfloodmaps.net/map/
https://coast.noaa.gov/slr/#/layer/fld/0/-8514647.371503906/4764471.237285254/14/satellite/none/0.8/2050/interHigh/midAccretion
https://waterdata.usgs.gov/nwis/rt
Data Services•Map - view
• https://bcgis.baltimorecountymd.gov/arcgis/services• https://geodata.md.gov/imap/services• https://hazards.fema.gov/gis/nfhl/rest/services/public/NFHL/MapServ
er• https://idpgis.ncep.noaa.gov/arcgis/services• https://hydro.nationalmap.gov/arcgis/rest/services/nhd/MapServer
• Feature - view, query, download...• https://geodata.md.gov/imap/rest/services/Environment/MD_Protect
edLands/FeatureServer• LiDAR
• https://lidar.geodata.md.gov/imap/rest/services
https://viewer.nationalmap.gov/services/
I - IdentificationII - Data QualityIII - Spatial Data OrganizationIV - Spatial ReferenceV - Entity and AttributesVI - DistributionVII - Metadata Reference
… and then there’s Feature Level Metadata
Metadata - FGDC
Data OrganizationNSDI Core/Expanded
Core• Geodetic Control• Hydrology• Elevation• Cadastral• Orthoimagery• Transportation• Governmental Units
Extended• Biota• Climate & Weather• Geology• Land Use-Land Cover• Real Property• Soils• Utilities
Data Organization - ISO 19000 Metadata Theme Categories
• Farming• Biota• Boundaries• Climate/Meteorology/Atmosphere• Economy• Elevation• Environment• Geoscientific Information• Health
• Imagery/Base Maps/ Earth Cover• Intelligence Military• Inland Waters• Location• Oceans• Planning Cadastral• Society• Structure• Transportation• Utilities/Communication
Codified Layers
•Chesapeake Bay Critical Area•Commercial Revitalization Districts•Growth Tiers• Enterprise Zones•Master Land Use Plans•Master Water Sewer Plans•Metropolitan District• Zoning• Zoning Variance
Open Data Portals
Data.gov - https://www.data.gov/
HIFLD Open - https://services1.arcgis.com/Hp6G80Pky0om7QvQ/ArcGIS/rest/services
Md iMAP - https://data.imap.maryland.gov/
ReferencesBolstad, Paul. 2019. GIS Fundamentals, 6th Ed. XanEdu
DeMers, Michael N., 2009. Fundamentals of Geographical Information Systems, 4th Edition. John Wiley & Sons
Harmon, John E. and Anderson, Steven J., 2003. The Design and Implementation of Geographic Information Systems, John Wiley & Sons. ISBN 978-0-471-20488-6
Robinson, Arthur, et al., 1995. Elements of Cartography, 6th Edition. John Wiley & Sons, Inc.
Tomlinson, Roger. 2013. Thinking About GIS: Geographic Information System Planning for Managers, 5th Ed. Esri Press
Wolf, Paul, et al., 2014. Elements of Photogrammetry with Application in GIS, 4th Edition. McGraw-Hill.
Zeiler, Michael, 2010. Modeling Our World, Second Edition: The ESRI Guide to Geodatabase Concepts, Esri Press
Zeiler, Michael, et al. 2004. Designing Geodatabases Case Studies in GIS Data Modeling, Esri Press
QuestionsMary Buffington, [email protected], 410.548.3844Douglas Adams, [email protected], 410.887.3451