FME Delicacies:Tasty Examples from the City of St. Albert

Tammy KobliukGIS Coordinator

On the Menu:

Appetizer: 2010 Municipal Census Salad Course: Business Licensing Main Course: Fire Services 9-1-1 Dispatch Dessert: Questions? (you’re all on a diet)

APPETIZER

Municipal Census

Appetizer: 2010 Municipal Census

The challenge for the kitchen: City-conducted municipal census All residences within City boundaries First on-line census No previous spatial census-base layer available No spatial address point layer available No single address database available All residences require a secure PIN number All residences will be mailed their PIN number A complete residential address listing is required

Available Ingredients

Census Landbase

Postal Codes

Enumeration Areas

EnumerationDistricts

Desired Outputs

Primary and secondary PIN numbers for each address

A mail merge database including postal codes and PIN numbers

A list of addresses requiring in person PIN letter delivery

A list of addresses requiring institutional enumeration

A final GIS census polygon landbase including assigned PIN numbers

Finding the Right Recipe

The goals for the PIN numbers: No less than 5 characters long No more than 8 characters long Primary and secondary PINs No duplicate PINs Repeatable method No confusing characters (e.g. 0 vs O)

Trial 1: Using Existing Info

The City of Airdrie did it so why can’t we? Available Info:

Street names Legallot ID numbers Centroid X/Y coordinates Neighbourhood names

Strip them down, mash them together, in short make an unproductive mess…

Sampling Ingredients

1. Input Landbase

2. Fetch Centroid Coodinates

3. Trim off decimal places

4. Replace decimal

5. Generate random 9-digit number

6. Generate single random number

7. Extract first letters of street names

And It Tastes Like?

Ummm…best not to be sampled: Generated duplicates Complicated at best Seemed like it would take more work than it

was worth to get it to work Like Plan B would be better: why use existing

ingredients when you can just create some tasty new ones?

Plan B: Just Random

Use individually generated random numbers that match the ASCII character designations for capital letters A to Z.

Stick with just letters instead of mixing in numbers

Generate 7 individual letters Combine in order for the initial 7-digit PIN Recombine to create an 8-digit PIN Little chance of generating duplicates as

there are now 26 choices, instead of 10, for each character

Appetizer: The Model

1. GenerateRandomLetters

2. Create first PIN

3. Check for duplicates

4. Create second PIN5. GIS Output

Appetizer: The Model (more yummies)

6. Addressesmust be sorted

In ascending order of:• Enumeration Area• Enumeration District• Street• House number• Suite number

10. Get rid of offending commas

7. Address mergefile for Admin staff(Access table)

8. Addresses forhand delivery ofPIN letters(Access table)

9. Addresses forCity staff enumeration(Access table)

11. Blank databasefor census contractor

(CSV file)

Appetizer in Review

We went with simple instead of fancy for PIN number generation.

We chose to optimize our development time. We had difficulties in joining postal codes to

address points in FME so ended up doing that in ArcGIS as pre-processing.

Overall development time was approximately 1.5 days, including initial sandbox time.

Very happy with final result.

SALAD COURSE

Mapping Home-Based Businesses

Salad Course:Mapping Business Licenses

The challenge for the kitchen: To map the home-based business licenses The license database is not spatial The license database is address-based We don’t have a spatial point address layer We do have a census landbase Address format structure does not match

between the two databases We wish to have an automated process for

producing updated business license datasets

Available Ingredients

Census Landbase

Business LicenseDatabase

NAICS CodeLookup Table

Desired Outputs

GIS point layer of home-based businesses Business trade name Address (full and parsed) License number License category/subcategory NAICS code NAICS code full description NAICS sector NAICS class

Salad Course: The Model

Create addressjoin string

Create address points;Match to BL data

Merge with NAICSlookup table

Resulting Delicacy

Salad Course in Review

Overall development time was approximately 5 days, including initial sandbox time.

Very happy with final result. Have already updated the data with the latest monthly data dump.

Challenges: Understanding the tools Chaining multiple models together Parameterizing the model for flexibility

MAIN COURSE

Fire Services 9-1-1 Dispatch

Main Course:Fire Services 9-1-1 Dispatch

The challenge for the kitchen: 9-1-1 calls come in from Telus Telus feed has specifically formatted addresses 9-1-1 addresses must be structured to match

Telus feed data as matching is automated 9-1-1 system requires address point dataset All non-relevant addresses are to be stripped Each address must have a unique ID, repeatable

through each data load Addresses cannot be duplicated The City does not have a spatial address dataset

Available Ingredients

Parcel Polygons

Townhouse Polygons

Apartment/CondoSuites

Additional AddressPoints

Desired Outputs

GIS address file Shapefile format Point feature type Containing only addresses that will be

referenced by Telus Containing only those fields required by CriSys Formatted to match Telus feed

Challenge for the Kitchen

Must be repeatable Must be automated as much as possible Must be able to easily QC for duplicate IDs

and Addresses Must be fast Must be flexible to deal with changing inputs

Main Course: The Model

Building the Crust

1. Convert polygons to points2. Filter out non-relevant addresses3. Filter out Null addresses/IDs – multiple methods required

Building the Crust

4. Convert UPPER CASE street names to Sentence Case5. Fix McKenney Avenue (inner capital letter)6. Adjust spelling of key road names to match CriSys/Telus

Building the Crust

7. Checking for duplicate IDs8. Filtering out or dealing with duplicate IDs9. Filter out specific addresses for replacement

Building the Crust

10.Converting LegallotID value to CriSys ID11. Automated removal of duplicate addresses12.Output QC datasets for parcel addresses and duplicates

Making the Filling

1. Convert townhouse polygons to points2. Change UPPER CASE street names to Sentence Case3. Fix McKenney Ave case

Making the Filling

4. Filter for addresses with/without suite numbers5. Create unique ID numbers from LegallotID, suite and house

numbers6. Deal with two specific problem properties

7. Output QC dataset

Making the Topping

1. Converting multi-story suite polygons to points2. Change UPPER CASE street names to Sentence Case3. Fix McKenney Ave case

Making the Topping

4. Create final ID string from LegallotID, house number and suite number (when applicable)

5. Output QC dataset

Putting It All Together

13.Adding in the extra points14.Streaming all the outputs into

a single final address point file

Main Course: The Model

Resulting Delicacy

Main Course in Review

This was actually our first (and most complex) FME project

This is the second iteration of the model Taking the course helped in refining the

model (thanks!) It’s a live model and is expected to have to

constantly evolve to deal with data oddities The model is currently pretty robust Challenges were mostly related to discovering

and dealing with data oddities

A QUICK DESERT

Recap and Thoughts

Recap and Thoughts

We LOVE FME! The software is intuitive It’s also powerful and complicated It definitely meets our needs We see it playing a key role in data integration We use it from soup to nuts – everything from

simple field rearranging to complex translation and integration models

We see it being useful for more than just spatial information

Thank You!

Questions?

For more information: Tammy Kobliuk tkobliuk@st-albert.net City of St Albert

Enter other resources