Spatial Analysis in Government Lecture 2- Data & Information
description
Transcript of Spatial Analysis in Government Lecture 2- Data & Information
Spatial Analysis in GovernmentLecture 2- Data & Information
By Guy Thigpen, M.Phil., MUSA
Purpose of Lecture
Review the Data and Information contained in GEODB2
Explore an example (ULRS) of the information available in the data warehouse
Examine the data structure of a parcel layer
Frameworks of Understanding
Spatial Analysis
GeographySystems Thinking
Cartography
Philadelphia’s Data Warehouse
Data and Information
282 feature files (tables, points, etc)
20 departments provide information
26 geodata folders with additional feature files
GEODATABASEA database
that is in some way referencedto location on the earth.
VECTOR DATAspatial data represented. as
points, lines and polygons
ATTRIBUTE DATAadditional information about each spatial feature housed
in tabular format.
RASTER DATAcell-based data such as aerial imagery and digital elevation
models
METADATA Documentation of GIS datasets.Contains information such as,
•the coordinate system, •when the data was created,•contact of who created it •definitions for any of the code attribute data.
ArcGIS Services
• Caching
One of the main characteristics that makes the server very capable of publishing maps over web is Caching. Additionally, we have more control over how the cache gets built and updated. Caching particular areas on the map and On Demand caching are the other main features.
• Security
Management of services with ArcGIS Server's role based security has been a major advantage. Also, the product supports token based security for web services and applications.
• Imagery
ArcGIS Server 10.1's Imagery capabilities enable us to handle large amount of GIS data and provides access to it enterprise wide. It also allows processing and analysis of images on the client side.
ESRI Arc GIS Server 10.1
Some of the services that we provide include, City Aerial Photography, City Hybrid Map, Parcel Layer, Building Footprint Layer, Septa Lines etc.
All these services can be overlaid upon each other according to the user's choice in any
web application or desktop around the city.
Pictometry
Pictometry is the leading provider in digital, geo-referenced, oblique aerial imagery and innovative geospatial technologies.
ORTHO Shot straight down and oriented North-up.
OBLIQUEshot at a 40~45 degree angle. Individual image is available from North, South, East or West.
CAPTURE AREA: The oblique imagery is captured for the City
of Philadelphia, including • center city area, • the entire Philadelphia International
Airport, which is located in Delaware County, and
• the Wyndmoor portion of Montgomery County.
With Pictometry Electronic Field Study (EFS) software, licensed users are allowed to view and work with the aerial images contained in the Pictometry Image Library
SPECIFICATION:RESOLUTION: Pictometry offers two levels of resolution, “community” level and “neighborhood” level. The average resolution of oblique imagery at community level (shot from around 4,500 ft) is 1-foot per pixel, and at neighborhood level (shot from around 3,500 ft) is 4-inch per pixel.
DATE/ TIME STAMP : Each image is date and time stamped with the time of capture as reported by the time information transmitted from the Global Positioning System.
How to make a connection to GEODB2?
1) In ArcCatalog:
The easiest way to create a connection to the central data warehouse, is by using ArcCatalog and connect to the ArcSDE application server (aka. Geodb2). Geodb2 will broker all your operations to the database.
2) Connecting with Oracle Client:
The central warehouse is in essence an Oracle database, and for power users out there who perform more complex operations and queries, you can connect to the warehouse directly using any Oracle Client .
GIS Spatial Data Sharing and Distribution
• In general the staff and departments who have been in the forefront of the development of the City’s spatial data support the sharing of this data within the City and to the public where appropriate and where the City’s costs to provide this data access can be reimbursed.
• The interns at the Philadelphia redevelopment Authority are also provided access to the database .
Publically Available Data from GEODB2@http://maps11.phila.gov/EnterpriseGIS/Downloads.aspx
Philly GIS Forum • Bring together the GIS users in all the departments of City of Philadelphia.• The topics discussed here are expected to get attention from the concerned users
who may make posts pertaining to the discussion.• Threads can serve as future reference for various issues should the need arise.
Group structure with restrictions:• Threads in the forum pertaining to certain specific topics will be having access
limitations; consequently, the forum users will be divided into groups according to the topics that are discussed and these groups will be having access those threads. One user can be present in multiple groups depending on the topics he is engaged in. Another one will be the read only restriction for certain threads and for certain users.
• Solutions can be found soon without numerous email tags and phone call exchanges as the problems are viewed by the community.
• Betterment in speed, resource utilization and problem-solving of city-wide GIS analysis and development.
Data and Information Process
Regulation
ComplianceStandard
DistributionMedium
Resources Committed
Primary DataCapture
DataProcessing
DATA InformationKnow-ledge
Under-standing WISDOM
Data and Information The foundation of an analysis include DATA and INFORMATION.
The City of Philadelphia has solid foundation of data and information.
W = Wisdom U = Understanding K = Knowledge
I = Information (URLS parcel layer and batch geocoder) D = Data (various shapefiles geospatially located)
O CPS IOutputs CustomersProcessesSuppliers Inputs
COMPLIANCESTANDARD:
Programs/projects are required or encouraged by
federal, state, and local sources to
collect data
carried out by contractors and city agencies
DATA:
Programs and projects
produce data
DATAPROCESSING:
Data is uploaded, stored and
collated to be available as
data or information
PLATFORM:
Open Data Philly
PASDA
phila.gov/map
Public
Clients
City
Agencies
Data and Information Relates to Analysis
Data
InformationBasic
Analysis
Knowledge
Understanding
Wisdom
Parcel Polygons
ULRS: Information in GEODB2
A simple task – displaying an address spatiallyWhat do we know about an address?
- What are the sources for that address?- (process of standardization)
- What parcel from the Department of Records does this address belongs too?- (process of geocoding)
Address standardization
An ADDRESS_ID is any unique combination of
STREET_NUMBERSTREET_NUMBER_SUFFIXSTREET_PREFIXSTREET_NAMESTREET_TYPESTREET_SUFFIX
Incoming addresses are scrubbed to one or more ADDRESS_IDs
(~ similar to LOC field in REAL_ESTATE table)
!!! ADDRESS_ID is an artifact of the ULRS !!!
STREET_CENTERLINE + any other street known by OPA informs the “valid” streets
Parcel geocodingFor each ADDRESS_ID, assigned to active DOR parcel
2 methods- by direct reference to DOR: if the address is in the parcel layer- by estimation: if the address is not in the parcel layer, ULRS will perform “educated guess”
!! Not every ADDRESS_ID can be associated with a parcel !!
Relations in ULRS
PARCEL
145N140168
026S050256
068N220032
070N090134
...etc...
ADDRESSES
1234 MARKET ST
2700 SPRING GARDEN ST
1206 NAUDAIN ST
...etc….
BRT Tax Accounts
023150808
453137110
883333239
...etc...
LNI Hanssen Address Keys
141144
141145
141147
141148
..etc...
PWD WaterRev
757315026001
5614004537001
8088001300002
...etc...
NON-SPATIALSPATIAL
Process of address standardizationProcess of geocoding
ULRSDB
(“address switchboard”)
BRT
LNI Hanssen
Water Revenue
GIS web services
Other spatial data (e.g. police district boundaries, trash
pickup, …)
CityMaps
BRT Web
Zoning Archive
LegistarStreets
Workorder 311
DHS Facts 2
Map Plugin
CRS Online
LNI Workorder
311
Online Zoning Map
Batch Geocoder
BUILD
OHCD Prop lookup
FPC Treemanag
ement
Crime Map
Police PHA interface
PhiCAMS
PWD Doc managemernt
PWD Plumming
PWD workorder
311
ParcelExplorer
Conveyance Fraud Tracker
Revenue Online
RESA
Streets Snowcategory
Pothole Reporter
Address Explorer
Unified Land Records System
Authentication Geocoding
Reverse Geocoding
Land Record Extraction
ULRS system lookups
Service area lookups
Generate map thumbs
Address standardization &
verification
Published Data(~reports~)
MASTER_ADDRESSMASTER_ADDRESS_ALL
PARCEL_ULRSBRTREALESTATE_DORPARCELMASTER_ADDRESS_BRTPICT
….(upon request)
….
(monthly-ish)
(Weekly)
ULRS access?
Exposed through various means
» Webservices + their clients» Ready-made applications: AddressExtractor,
AddressExplorer» bulk data reports in the “GIS_ULRS_DATA” schema
Web Services
• = developer technology• Two flavors
– CityMaps Webservices• “generic GIS” functionality: • relies on ULRS for only two pieces: the address cleanup logic &
parcel-based geocoding
– ULRS WebServices• Main window in the ULRS.
– Allows you to extract any information that is stored in the ULRS
• Provides simple read access to other db systems.
Web Services
• Who uses the webservices?– Around 25 clients from various departments: lni, police, revenue, dot, brt, ohcd, dhs, water, sherrif’s
office,…– 2,000,000+ lookups against the web services per month– Supports web sites + batch processing– Majority of ULRS clients do not rely on “geocoding logic”
• mainly to resolve address input• switchboard to retrieve account information based on address
(!!! after migration to ArcGIS Server, some functionality was lost!!! )
Existing ApplicationsAddress Explorer
Replaces the ULRS Explorer – which was discontinued after June 30, 2009.
Why the rename? - address is central – not the parcel, not the BRT#,…- also, underline the fact that ULRS does not store any other information other than relationships between addresses & parcels on the one hand, and addresses & accounts on the other hand. (nothing more, nothing less)
Mimic three basic ways of retrieving information from the web services- by address- by parcel #- by account # (topic key)
~corresponds to 3 ways of organizing the data
Existing ApplicationsBatch Geocoder/Address Extractor
-Allows you to process address based data
- Standardize; clean up an address(= most basic operation)- Validate: clean up an address & check if imported by ULRS- Geocode: clean up an address + find location- Extract land records (~~~ topic pane from Address Explorer)- Extract DOR parcel info: retrieve parcel identifiers by address
Data Reports- bulk delivery of ulrs data under GIS_ULRS_DATA on geodb2 (on monthly basis)
- Replaces ulrs publisher: low usage + data not always meaningful
GOALS-organize data in meaningful way-secure core applications-flexible schema
- data added upon request: make sure that every report actually has a user- not recommended for application use – but not enforced either.
-SO FAR:-ALIAS_ADDRESS-MASTER_ADDRESS-MASTER_ADDRESS_ALL-MASTER_ADDRESS_BRT_PICT-PARCEL_ULRS-BRTREALESTATE_DORPARCEL
This Square
Exploring DATA: how it is generated
Legal Description and Survey Plan
Consistency - some legal descriptions do not indicate streets and just use bearings where others do indicate the streets with the bearings.
Understanding Bearings
Example- South 03 degrees 42 minutes 30 seconds East S03º42’30”E
• 1: First it gives the rotation of the line with respect to the geographical north in degrees, minutes and seconds. It is in the middle part between the directions.
• 2: Second it gives the direction of the line from the starting point to the end point in one of the four predefined ways - South to East, South to West or North to East, North to West. These directions are at the start & end.
Reading Bearings From a Legal Description
55º
15º
60º
20º
Bearing Definition
NENW
SESW
55º43’59” 15º21’45”
60º10’25” 20º46’12”
N
S
W E
O
g
b
W
L
LO = N 55º43’59” WOL = S 55º43’59” E
bL =?Lb =?
gL =?Lg =?
How to Define Bearings
Each Segment should have bearing
Angles in Bearings are never greater than 90º
Bearings are referenced from North or SouthFour predefined ways - South to East,
South to West
North to East, North to West
A Parcel Should have Bearings either in a clockwise or Anticlockwise Direction
Minutes & Seconds should not be greater than 60º
bL = S 15º21’45” WLb = N 15º21’45” E
gL = N 60º10’25” WLg = S 60º10’25” E
How to Check Bearing of Parcel
Example: Parcel 61B
Either – 74.417’Or – 74’5”
Missing Bearing
Two Bearing Segments Missing from Legal Description
Missing Segments Assumed At Right Angle
Bearing To Be Verified
2nd Deed
2nd Shape in identified parcel
Final Remainder
This Square
• ALL THAT CERTAIN lot or piece of ground Situate in the 6th Ward of the City of Philadelphia and described as follows:
• Beginning at a point of intersection of the Southeasterly side of 46th Street ( ) and the Southeasterly side of Fairmount Avenue (60 feet wide);
• Thence extending N86°17'30"E, along the Southeasterly side of Fairmount Avenue a distance of 243.417 feet;
• Thence extending S03°42'30"E, along a line at right angles to said Fairmount Avenue a distance of 249.000 feet;
• Thence extending S86°17'30"W, along a line parallel with said Fairmount Avenue, a distance of 243.417 feet to a point on the Southeasterly side of said 46th Street;
• Thence extending N03°42'30"W, along the said Southeasterly side of said 46th Street, a distance of 249.000 feet to the point of beginning.
• Containing 60,610.83 square feet or 1.3914 acres, more or less.
• BEING KNOWN AS:?
Remainder Legal Description
This Square
The resulting shape in GIS layer
Next week’s Topic: The Front Door
We will discuss the challenges and analysis that led to the implementation of the ‘Front Door’