Swarm Intelligence or Lost in the Crowd. Open Data, Public Scrutiny of Public Action, and How Evaluations

Change

Aline Pennisi e Laura Tagle EES Annual Conference Helsinki, October 2012

Outline

• State of the art

the increasing use of ICT in all domains

• Increasingly citizens/businesses are making their own

information on the internet and consuming information made

by others

• The internet is increasing the value of information created by

government

There is economic and social use of all this information…

…. and new opportunities to evaluate government policies

what open data is

“A piece of content or data is open if anyone is free to use, reuse, and

redistribute it —without restrictions from copyright, patents or other

mechanisms of control.”

• collected by governments while performing its tasks (open

government data)

• created and shared by users

• Restrictions: technological or legal features

Focus: how open data changes the way the public sector relates to the external world and works

internally

how open data and ICT affect the way the public sector works

• Dissemination of raw administrative data (while dissemination of

statistics is not a novelty ...) and accessibility to anyone

• Interaction with a huge number of external bodies (citizens,

businesses, other public authorities) on a much wider range of

possibilities, with fewer filters

• The connection/integration between your data and the data produced by other parties ... public but also non-public

• Crowd-sourcing and collaborative data collection/validation

• The increased uncertainty about official/truth, given a less

marked border between "certified" information and not

• The possibility that others provide a service that was previously

the sole prerogative of the Government / public sector (in competition,

replacement or in addition)

Open government and e-government

• Overlapping yet not the same

• Different approach to use of ICT

• Both share similar limitations

– Digital divides (territorial, age, socio-economic)

– New vulnerabilities

E- government

• Stress on technology: use of ICT to provide services

• Allows, but not necessarily requires:

– new services

– collaboration

– openness

– Scrutiny

Frustratingly interacting with an automated system from

home not intrinsically better than walking to the counter

to frustratingly interact with a human

Open government

• Stress on activities

• Data as public good: push for publication of data changes internal functioning of government

• Openness relates to – Possible collaboration: accomodates or invites interactions with

other public and private entities

– New services: no fixed bundle of services, but invites and accomodates creative use of existing public goods (among which data) to provide new services

– Multi-centric, democratic governance

– Scrutiny of government action

9

Minnesota Tagged Fish http://www.dnr.state.mn.us/fisheries/tagged_fish_reporting/index.htm

l

10

FixMYStreet http://www.fixmystreet.com/

11

OpenCoesione http://www.opencoesione.gov.it/

first claim on benefits of open data / ICT

• efficiency: with open data the role of Government/public

administration in informing the citizens of public affairs and in providing utilities can be largely reduced (e.g., Robinson & Yu, 2010)

o other (private intermediaries, profit or not-) can do this better,

because they are on the "user’s side“ and having to compete every

day in the market they need to continuously innovate and do it better

o others can do this at lower cost, extrapolating data from more and

more numerous sources, and giving it more meaning (through

research, representation, processing, data update---beyond simple

data delivery)

But some questions are left unresolved:

who is responsible for ensuring these services and making sure they are fair/

for everyone?

are we really willing to pay "public" services supplied by third parties at a cost?

does “everyone” really mean everyone? How many people are digitally literate?

second claim on benefits of open data / ICT

• democracy: Open data allows you to re-invent the relationship

between rulers and ruled, between public service providers and citizens

— in favor of the latter (e.g., Tim O'Reilly, 2009; Maier-Rabler & Huber,

2011)

o Government/public administrations can offer an unbiased platform

where all actors interact (government as a platform ... from Donald

Kettle's "vending machine" to Eric Raymond's "Bazaar")

o everyone can monitor the actual conduct of the

Government/Administration

But some questions are left unresolved:

It is not clear whether we have enough or the right data to determine accountability

If there isn’t a first “authentic/explicit” interpretation of data will we actually know

which are the objectives of policies?

If there is no official info who will lead the citizen in choosing between alternatives?

Which groups are more active on the internet? How are vulnerable groups/territories

reaching out on the internet?

third claim on the benefits of open data / ICT

• effectiveness: Open data improves the services provided and

supports better policy decisions (e.g.,)

o It reduces the information asymmetry and citizens/businesses can

more easily report about the situation on the ground

o allows you to rebuild confidence between the parties and share

goals, and then to generate co-operative behaviors between policy

makers and policy beneficiaries

o dissemination, extreme detail and speed of information facilitates

coordination among the various policy actors

o it can provide valuable data for informing the choices of citizens and

economic operators

o You can create a stronger pressure towards results

however: is voice enough to identify policies "that work“? how to deal with moral

hazard? How to select among all the information available?

opportunities for evaluation o increased availability of (often free!) data on the programs, projects or

services that you want to evaluate

o possibility to have many more reviews from many more evaluators

(since they all have the same chances to have access to crucial data)

o open government data is changing the way to collect and make

available data on social phenomena (e.g., urban decor, environment,

crime)

o and it pushes the observation of social phenomena towards more

detailed territorial level (using more administrative archives and fewer

samples/estimates)

MORE ?

challenges for evaluations

• Input data overwhelm the discourse already

• Government-produced information on programs, projects and services are not necessarily sufficient for the assessment of effects ..

• Open government data can only concern secondary data (created or collected or archived by public sector for its purposes)

• Difficulties in discerning sense, reliability, biases in primary data—not exactly new issues, but with NEW technical features and at a (potentially) much larger scale

• Judgment criteria multiply exponentially

• Speaking truth to power has a new meaning: main client is not anymore public sector but the public

• Independence becomes a must: reputation vis-à-vis the masses, not only the policy or professional community

• Quality control not only Steering Groups or advisory groups but the collective intelligence

Evaluation needs primary data

challenges for evaluators

• Skills

• Demand: will open data stimulate demand for evaluation? Or will the

availability of multiple analyses fulfill the need for evaluation—in a

time of shrinking resources?

• Status

expert, non-expert, and differently-expert scrutiny

o how is the data going to be interpreted now that everyone is

legitimated to do it?

o Myriads of analyses and new “evaluators”

Who needs to pay an evaluator when there are thousands of self-

appointed analysts?

Where do we acquire the new skills?

New frontiers

• opportunities opened by the ability to conjure collective intelligence in

evaluation processes--using concepts already developed in the

participation tradition

• evaluators also now have new tools to gather information, which can

change the relationship with what is observed or evaluated

• Evaluation as expert knowledge, as an application of methods by

“evaluators”, questioned.

• Clearer what has always been there:

• evaluation is one among many practices of public scrutiny,

• New, different expertise come into the play-field

• People with no voice & power can have their say.

Which one are we going for?