Biomedical citizen science: successes and

opportunitiesBenjamin Good

The Scripps Research Institute @bgood

Types of Biomedical Citizen Science

• Personal data: 23andme surveys, ubiome!, …

• Microtasks: large in number, small in difficulty

• Megatasks: Small in number, high in difficulty

Good & Su 2013. Crowdsourcing for Bioinformatics. Bioinformatics

Citizens enhance the capacity of traditional science

by performing microtasks

• Processing images (tagging, tracing)

• Annotating concepts in text: mark2cure.org

Keys for success: volume, redundancy, aggregation, gold standards for training and quality assessment

More than microtasks, not quite megatasks: Visual Reasoning

Foldit: 3D Protein Structure Puzzles

DNA Sequence Alignment

Key to making it fun and productive is a good automated scoring function

Citizen scientists performing the megatasks usually reserved for

professional scientists

(Khatib et al 2011) “Algorithm discovery by protein folding game players” PNAS

Open Innovation Contest Platforms

Successful megatasks

• Large, diverse user population

• Well-defined problem

• Rapid, high quality feedback on proposed solutions

EteRNA http://eterna.cmu.edu/

1. Its a game! - rapid feedback, visual, beautiful

2. Its a weekly competition! the winners are rewarded with real laboratory tests of their hypotheses about how a string of RNA will fold in 3 dimensions

3. Its a community forum for creating RNA design rules

4. Its a machine learning system for learning RNA design rules

5. Its a MASSIVE OPEN ONLINE LABORATORY !!!!

6. Its working! (Lee 2013) RNA design rules from a massive open laboratory, PNAS

Identifying new opportunities

• “Measurement is the beginning of [citizen] science”

• Look for problems where it is possible to provide rapid, high quality feedback about progress

Needs

• Frame the process

• Reduce barriers to entry

Games are the Gateway Drugs!

• Short, high-quality feedback loops

• Deeper learning, discovery paths

Motivation is the easy part

Opportunities

• Biology and medicine provide a heroic purpose for citizen science – not unlike the more traditional purpose of saving the world from aliens. Scientist