IOGDC Open Data Tutorial

Open Government Data Tutorial

2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE

Hadley BeemanJim HendlerJeanne Holm

Presenter Contact Information

Organized by the World Bank and Data.gov

IOGDC Open Data Tutorial 22012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV

DATA TUTORIAL

Introductions!

• Please introduce yourself– Name– Organization– Three (3) words that explain either why you are

here or what you hope to learn

9 July 2012


DATA TUTORIAL

Understanding the Foundations of Open Data

• Why do countries and people share data?• What will citizens, businesses, scientists, and

journalists do with the data?• How can we manage it?

9 July 2012


DATA TUTORIAL

Why Countries Share Data

• Meet regulatory compliance• Provide transparency into government

operations

9 July 2012


DATA TUTORIAL

Why Countries Share Data

• Anticipate economic development• Initiate innovation

9 July 2012


DATA TUTORIAL 9 July 2012


DATA TUTORIAL

Why People Want Open Data

Swati Ramanathan9 July 2012

2012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV DATA TUTORIAL

Real Outcomes = Better Lives

• Open data eases the burden on families in caring for a sick child or seeking medical information

• More importantly, the data as it’s aggregated empowers communities to make changes that improve the quality of life of citizens– In California, ReLeaf plants trees in areas identified by Together We

Breathe as danger areas for asthma sufferers– Cities see hot spots that trigger asthma problems for their citizens

• Each company in Health 2.0 used government data to innovate and create high-value jobs in America

• Civic Commons has a great collection of good open use cases: http://civiccommons.org/

IOGDC Open Data Tutorial 89 July 2012

http://civiccommons.org/

http://civiccommons.org/


Energy Drives Innovation• Energy.Data.gov

connects innovators, industry, academia, and government at federal, state, and local levels



Challenges Spark Ideas• Energy.Data.gov

connects works with challenges across the nation to integrate federal data and bring government personnel to code-a-thons



Data Drives Decisions• Apps transform data

in understandable ways to help people make decisions



DATA TUTORIAL

Changing Economic Equations

Study from Malaysian government: http://www.transknowformance.com/article.cfm?id=53

9 July 2012

http://www.transknowformance.com/article.cfm?id=53

http://www.transknowformance.com/article.cfm?id=53


DATA TUTORIAL

Why People Want Open Data

9 July 2012


DATA TUTORIAL

What Makes Data Open

• Open Format– The US Government through the Open

Government Directive (http://www.whitehouse.gov/omb/assets/memoranda_2010/m10-06.pdf ) defines an open format as “one that is platform independent, machine readable, and made available to the public without restrictions that would impede the re-use of that information.”

9 July 2012

http://www.whitehouse.gov/omb/assets/memoranda_2010/m10-06.pdf




DATA TUTORIAL


• Example Open Formats:– PDF for documents (but not data)– CSV for data– Web standards for publishing, sharing or linking

• HTML, XML, RDF

– Web standards for syndication• RSS, Atom, JSON

9 July 2012


DATA TUTORIAL


• Metadata– The information about the data being shared

• Who produced it• Where• When• Use restrictions• Etc.

– Use standards such as ADMS or Dublin Core– New (Search-oriented) Embedded dataset

metadata coming9 July 2012


DATA TUTORIAL

What Topics of Data Are Published

• Analytics based on over 1,000,000 datasets from around the world can be seen at – http://logd.tw.rpi.edu/iogds_data_analytics

• The examples that follow are from that page

9 July 2012

http://logd.tw.rpi.edu/iogds_data_analytics

http://logd.tw.rpi.edu/iogds_data_analytics


DATA TUTORIAL

Countries Sharing Data

Important note:quantity is not really the most important issue

9 July 2012


DATA TUTORIAL

Example: US

Data.gov

9 July 2012


DATA TUTORIAL

Example: UK

Data.gov.uk

9 July 2012


DATA TUTORIAL

Example: Spain

9 July 2012


DATA TUTORIAL

Topics (Across All Catalogs)

9 July 2012


DATA TUTORIAL

What Topics of Data Are Published (UK)

9 July 2012


DATA TUTORIAL

Data “Mashups” of Many Kinds

More than 50 of these at http://logd.tw.rpi.edu 9 July 2012


DATA TUTORIAL

Making Data Open, Accessible, and Discoverable

• Architecture for systems and technology• Processes for publishing data• Policies for ensuring data is open, accessible,

and obtainable

9 July 2012


DATA TUTORIAL

Creating an Open Data Architecture

• Key components– Workflow for release approval (often overlooked)– Dataset storage

• Can be centralized or via linking

– Data Cataloging• Metadata critical to a good open data site

– Data API• Can be via download or via access• Technical issues with syndication, usage rules, etc.

9 July 2012


DATA TUTORIAL

Processes

• Publication (and cleaning)

• Data reuse and integration

• Community input

9 July 2012


DATA TUTORIAL

Policies Become Essential

• Policies are needed to help drive the ecosystem and “motivate” departments to continue to share data openly

• Build the policies based around issues that are universal • Licensing, provenance: http://creativecommons.org/licenses/

9 July 2012

Open data on food, security,

transportation, and transparency

http://creativecommons.org/licenses/

http://creativecommons.org/licenses/

312012 INTERNATIONAL OPEN GOVERNMENT DATA CONFERENCE—OPEN GOV

DATA TUTORIAL IOGDC Open Data Tutorial9 July 2012


DATA TUTORIAL

Generations Share Differently

• 1930-50’s era generation– Focus on society– Friendships are forged through adversity

• 1960-70’s era generation– Focus on community– Friendships forged through identification with a cause

• 1980-90’s era generation– Focus on the individual– Friendships forged through individual goal accomplishment

• 2000’s era generation– Focus on common interests– Friendships are created or thrive virtually…

9 July 2012


DATA TUTORIAL

Creating Change is a Social Construct• The social aspects of an organization that help to

facilitate, create, and ensure that the goals of the organization are met– Social networks– Norms and values– Social trust– Networks and communities– Physical spaces– Storytelling– Social capital consists of the stock of active connections among people;

the trust, mutual understanding, and shared values and behaviors that bind the members of human networks and communities and make cooperative action possible

• Robert Putnam, Harvard University

9 July 2012


DATA TUTORIAL

Trust and Reciprocity

9 July 2012

• Trust can be built on• Personal experience

• “I know you”• Shared experience

• “We both worked on the same project”

• Transfer of trust• “We know the same

person who trusts us”• Shared values

• “We agree to operate by the same rules”


DATA TUTORIAL

Semantic Web and Linked Data (UK)

County Council

Ordnance Survey

Royal Mail

9 July 2012


DATA TUTORIAL

Linking Data Via Common Naming (Usually URLs)

9 July 2012


DATA TUTORIAL

Example: Agency Names

9 July 2012


DATA TUTORIAL

Can Be Lots of Things

9 July 2012


DATA TUTORIAL

“Linking” Data goes beyond govt

http://linkeddata.org/

Government Data is currently over ½ the cloud in size (~17B triples), 10s of thousands of links to other data (within and without)

9 July 2012


DATA TUTORIAL

5 Star Data

9 July 2012


DATA TUTORIAL

Creating an Open Data Ecosystem

9 July 2012


DATA TUTORIAL

Sustaining Data Publishing

• A steward (someone tied to the business process)– Defines the resource– Purpose, audience, content, context, policies, operations,

metrics, and access method – Manages the resource– Measures performance– Identifies needs and opportunities – Applies and enforces policies– Oversees daily operations– An open data project then is an enabler to creating the

environment in which the processes can occur

9 July 2012


DATA TUTORIAL

Embedding Publishing• There are three ways to look at creating a sustainable

open data initiative– Step 1: Process: Oriented on the way in which people do

their day-to-day work in the organization (the how and why)

– Step 2: Services: Focused on who will help people share their knowledge and who will maintain tools and processes (the who)

– Step 3: Systems: Are the IT infrastructure and tools necessary to deliver the processes and services efficiently and effectively to the end users (the what and where)

9 July 2012


DATA TUTORIAL

Creating Community

• Communities are public-facing spaces that present data, information, and subject matter knowledge about a single topic from many organizations in one place– The topics for communities can be

chosen based on priorities from the public, departments based on their mission, or issues of national importance

9 July 2012


DATA TUTORIAL

Community Vision

• These questions help to guide early discussions– Vision: What will the community connection and collaboration look

like in the future?– Leaders: Who will help to lead the community?– Participants: Who will participate?– Outcome: What are the expected outcomes, metrics, and

measurements that will show success? How will this community work to improve the lives of citizens?

– Functionality: What types of activities will be conducted on the site (forums, blogs, wikis, ranking, rating, challenges, or apps)?

– Content: What content should be displayed– Interactivity: What ways will the community interact with the leaders,

with each other, and with the public?

9 July 2012


DATA TUTORIAL

Open Communities

IOGDC Open Data Tutorial

Community

Developers ✓

Open Data ✓

Semantic Web ✓

Health ✓

Law ✓

Energy ✓

Education ✓

Ocean ✓

Safety ✓

Manufacturing ✓

Business ✓

Ethics ✓

ConsumerResearch and DevelopmentCities + many more…

9 July 2012


Supporting Global EventsJapanese tsunami, earthquake,

and radiation monitoring

Restore the Gulf: Deepwater Horizon

Response



Health.Data.gov

48

Champion: Todd ParkChief Technology OfficerUS Dept. of Health and Human Services

Apps Forums

Challenges

Blogs

IOGDC Open Data Tutorial9 July 2012


Publicizing Data to Innovators

49

• Challenges and code‐a-thons (health2challenge.org)

• Many innovator “meetups” and conferences

• Annual health data-paloozas• Over 139 applications• 50 new businesses• Thousands of lives improved

each day• 1700 attendees at the Health

Data Palooza in 2012



Creating Apps That Improve Lives: Asthmapolis

50IOGDC Open Data Tutorial9 July 2012


DATA TUTORIAL

Creating Apps That Save Lives: iTriage and Hospital Compare



Use Case: Economic• Healthtap

– Provides free, reliable, and independent health information through the use of open data and by building the world’s largest medical expert network tailored for young mothers and their children

– Won 2010 Health 2.0 challenge– Success invigorated expansion

and new hiring at HealthTap– Reaches to Data.gov datasets

(along with others)

52IOGDC Open Data Tutorial9 July 2012


DATA TUTORIAL

Ensuring Use and Economic Benefits European Union market: EUR 28 billion in

2008 (Vickery) £6 billion in the UK (Newbury, Bently and

Pollock 2008) The Open Data Institute

9 July 2012


Used in Collaborative Planning and Governance Processes (from CTG SunyA)

STEP 1 STEP 5

Initiative Description:

· Title · Purpose· Supporting

strategic goals· Program or policy

area· Tactics

Describe Initiative

Review Open Government Public

Value Portfolio

· Review public value assessments across all initiatives to inform decision making.

· Does our open government portfolio, taken as a whole, optimize our resources and capabilities while meeting our mission and delivering maximum public value to all stakeholders?

STEP 6

· Establish a summary assessment for the initiative for each public value type across all the primary stakeholders

· All summary assessments are populated into the Portfolio Review Workbook

(Step 6)

· Identify a full list of initiative stakeholders

· Prioritize by rating each stakeholder as A, B, or C

Identify and Prioritize

Stakeholders

STEP 2

Initiative One-Five Workbooks

· Identify the type and level of Public Value the initiative is expected to create for each of the primary stakeholders

Public Value is expressed in terms of: · Economic value · Social Value · Political Value· Strategic Value· Quality of Life

Value · Ideological Value · Stewardship

Value

Identify and Rate the Public Value

STEP 3

· For each stakeholder and each change mechanism describe how the expected value will be produced

Change mechanisms include impacts on: · Efficiency· Effectiveness· Intrinsic

Enhancement · Transparency· Collaboration· Participation

Identify Stakeholder

Interests

STEP 4

Summarize the Public Value Assessments

Portfolio Review Workbook


DATA TUTORIAL

What Can You Do With Open Data?

• What are the uses of open data you have seen in your city, country, or organization?

• Write down on the post-it notes one or more of the following:– What has been done with open data– What could be done with open data– What are the challenges you are facing– What have been your successes

9 July 2012


DATA TUTORIAL

Additional Topics

• Licensing, provenance, languages• Metadata design (international)• Trust – government data is controversial, who

controls it?• Scaling – over 1M datasets and growing fast

– How to search, store, link, translate, and archive• Versioning and updating• Visualization beyond the single dataset• Boundaries of open data9 July 2012


DATA TUTORIAL

Questions

9 July 2012


DATA TUTORIAL

Summary and Next Steps

• Join a community– W3C eGovernment Interest Group

• http://www.w3.org/egov/wiki/Main_Page

– Open Data Innovation Network on LinkedIn• http://bit.ly/ODNetwork

9 July 2012

http://www.w3.org/egov/wiki/Main_Page

http://www.w3.org/egov/wiki/Main_Page

http://bit.ly/ODNetwork



IOGDC Open Data Tutorial

Technology

Transcript of IOGDC Open Data Tutorial