tcij.org Twitter.com/cijournalism

Computer Assisted Reporting

• What is CAR?

• Examples

• Tools and methodologies

• Mistakes and misconceptions

Computer Assisted Reporting

New approaches to Journalism born from:

1. New technology

2. A cultural shift towards openness

3. Application of social and computer science techniques to news gathering and reporting.

Computer Assisted Reporting

• “CAR is the use of computers and social science methods to acquire and analyze information to do stories that otherwise would be difficult or impossible”

– Steve Doig

• Pioneered by Philip Meyer and others in late 1960s (Detroit Riots, Pulitzer prize)

• Increasingly used across newsrooms in Europe and America

Government Increasingly open

Open Data Campaigners opening the Vault Political, corporate, economic data

Free, accessible and reusable

• Wikileaks: a watershed moment

• 250,000 US Embassy Diplomatic Cables

• 391,832 Iraq War Logs

Scale of data releases demands new approaches and techniques.

Within CAR, 2 Key StrandsData Driven Journalism

• Requires trained journalists to analyze data, and uses traditional publication model.

Database Journalism

• Produces databases of ‘facts’ that are continually maintained and improved

Data Driven Journalism

• Traditional reporting – but using Data as your ‘source’

• Data is the primary source material that is analyzed by journalists who report the findings to their readers.

• Requires trained journalists to analyse data, and uses traditional publication model.

Data Driven JournalismExamples:• MP’s Expenses• Iraq War logs• Baby P Investigation• Election and campaign finance investigations• Anything produced by Guardian, NYT, LATimes

Data Desks.

Database Journalism

• Produces databases of ‘facts’ that are continually maintained and improved

• Instead of stories, products are interactive, offering ordinary people instant access to information that previously required curation and publication by journalists or public officials.

• Requires programming expertise to build platforms for sharing and accessing data, such as websites and APIs.

Database Journalism Examples:

• TheyWorkforyou.com Database of MPS, their registered interests and Voting habits.

• OpenCorporates.com/CompaniesHouse.gov.ukDatabase of all registered companies.

• Police.ukMaps crime data. See the crimes reported in your area.

• FixMyTransport.comDatabase of local transport issues – Allows direct reporting of problems to authorities.

• FarmSubsidies.orgDatabase of EU farm subsidies and their recipients.

Methodology

• Analysis – Process ‘raw numbers’ into narratives and stories your readers can understand.

• Visualization – Present abstract statistics to readers in a way that makes them accessible and impactful to your audience.

• Mapping – Allow readers to understand and check information as it relates to their neighborhood/region. Check/Compare service provision in your area

Mapping Data

Visualisation

Invest in Data projects – Ongoing returns

• 12 member multi-lingual team• 1 database• Nearly 600 data sheets• 21 languages• More than 640,000 records• More than 100 agencies• 2 dozen stories … and counting

Where Can I find Data?• Data is everywhere

• Get in a “data state of mind”: forms, databases, statistics, press releases with numbers. Don’t take numbers given to you at face value

• Online databases (publicly available)

• Freedom of Information requests (requesting whole databases)

• “Scraping data”

• Databases constructed ‘from scratch’

• Plain and simple Excel

• Data cleaning tools (Google Refine…)

• Database managers (SQL, Access)

• Scraping software (or code) and tools (imacros, OutwitHub, ScraperWiki…)

• Visualisation tools (Tableau, Omniscope)

Tools of the Trade

Mistakes and Misconceptions• Giving up because you are told no

• It’s too expensive• We can’t download it• It’s private information• We don’t hold it

• Keeping your data to yourself: publish and be DAMNED!

• Pretty visualisations are not an investigation

• Being geeky and knowing the latest tools does not an investigative journalist make

Conclusion• “Analysing data is the future for journalists”- Tim Berner’s

Lee

• Transparency in Data is accountability – Demand data releases from authorities and governments

• Data journalism demands new technical skills from journalists – Reach out to those with programming and technical skills. Will augment your existing digital efforts

• Don’t be left behind – Cannot ignore the added value to your publication that a data desk will bring. Not an optional ‘extra’. ‘I don’t do numbers’ no longer acceptable.

Train or Fail• Russell Square, Algate, Edgware Road, Reported as most dangerous

(crime) stations in London – Stations were attacked by terrorists. -Context

• Telegraph claimed public sector pensions cost each citizen £4000 a year. More than most people pay in income tax. -Common Sense

• 100% Increase in risk. From 1 in a million to 2 in a million. 13% increase in chance of death. To 113%? – Basic Numeracy.

• Correlation and causation – Do mobile phone masts increase fertility?

Buzz Data – Collaborative data sharing platformhttp://buzzdata.comData.Gov – Main hub for UK Gov Datahttp://data.gov.uk/London Datastorehttp://data.london.gov.uk/datastoreUK Public Spendinghttp://www.ukpublicspending.co.uk/Scotlandhttp://www.esds.ac.uk/government/docs/ScotlandGuide.pdfUK Policehttp://www.police.uk/UK National Statisticshttp://www.statistics.gov.uk/hub/index.html

Some suggestions for finding UK Data