Open Data in San Francisco

Joy Bonaguro Chief Data Officer

City and County of San Francisco XLDB 2015, May 20, 2015

Phases of Open Data Phase 1: The Executive Edict

Phase 2: The Publishing Scramble

Phase 3: The Period of Stagnation

Phase 4: The Resource Reckoning

Phase 5: The Integration of the Initiative

CENTRAL HYPOTHESIS: DataSF can help solve our internal data sharing challenges

How to Broaden the Value of Open

Data?

Knowledge

Ability Access

© Emw / Wikimedia Commons / CC-BY-SA-3.0

Knowledge, mural by Robert Lewis Reid. Second Floor, North Corridor. Library of Congress Thomas Jefferson Building, Washington, D.C.

IGNORANCE IS THE CVRSE OF

GOD

KNOWLEDGE IS THE WING

WHEREWITH WE FLY TO

HEAVEN.

“Within six months, each

Department shall publish on

DataSF a catalogue of

their Department's

data that can be made

public, including both raw

datasets and APIs”

http://irisclasson.com/2013/03/14/stupid-question-163-what-is-dogfooding/istock_000015884355small/

Knowledge

Ability Access

© Emw / Wikimedia Commons / CC-BY-SA-3.0

Hey! You got this data?

Maybe…

Can I have it?

Hello????

Here it is…

Thanks! Some ?s…

Question death spiral

Task: Update Data

Task: Obtain Data

See previous

Publication Automation

The Balancing Act

Privacy Transparency & Open Data

HIPAA

FERPA HMIS Rule 603

Child & Welfare

CMIA

SB1386

42 CFR 2

Health & Safety Code

Welfare & Institutions Code

AD Ch 20, 16, 12m….

Health Code

Penal Code

Police Code

Business & Tax Code

Data Inventory and Raw Classification

Raw Classification Description

Public This data could be publicly disseminated without any concerns

Protected This data is protected by law or regulation and can only be shared or accessed internally and per organizational procedures; OR this information includes individually identified information

Sensitive In its raw form, this data poses security concerns, could be misused to target individuals or poses other concerns.

Survey of Applicable Laws

Raw Classification Description

Public This data could be publicly disseminated without any concerns

Protected This data is protected by law or regulation and can only be shared or accessed internally and per organizational procedures; OR this information includes individually identified information

Sensitive In its raw form, this data poses security concerns, could be misused to target individuals or poses other concerns.

More Process & Control Points

Fast releases Thorough process

Defining your methods

Standard Methods

Consistent technical implementation

Plug and play regardless of

backend Probabilistic matching with

expert curation Role based access control

down to the column level Legally aware

Joy’s

Holiday

Wishlist

(cheap too, please!!

;-)

Knowledge

Ability

Access

© Emw / Wikimedia Commons / CC-BY-SA-3.0

Within an EcoSystem of Shared Data and Technology

1. Increase individual skills & capacity

2. Support analytics in departments

3. Foster a data-enabled policy environment

Which leads to… Increased use of data in decision-

making

Schedule friendly

Free ;-) Demand driven

What…does the city do about affordable housing?

Where…do you go for reliable data about

affordable housing?

The answers are the problem…

not sure

X and Y and Z and Q and…

Not possible!

The solution…Housing Data Hub Single place: • Housing overview • Description of policies and

programs • Supporting data & viz

housing.datasf.org

enduring sources of truth

&

unbroken data lineages

Strategic releases help

democratize how

governments disseminate their data

for a local audience that may be focused on issues

(not just apps).

THANK YOU @datasf | datasf.org |datasf.org/blog

Questions?