Michael BernsteinMIT COMPUTER SCIENCE AND ARTIFICIAL INTELLIGENCE LABORATORY

MIT HUMAN-COMPUTER INTERACTION

DARPA’s Network Challenges

…new technologies to rapidly create theoretically-informed, data-driven models of complex human, social, cultural, and behavioral dynamics that are instantiated in near-realtime simulations…

- DARPA TASC

…the roles the Internet and social networking play in the timely communication, wide-area team-building, and urgent mobilization required to solve broad-scope, time-critical problems.

- DARPA Network Challenge

…harness the unique cognitive and creative abilities of large numbers of people to enhance dramatically the knowledge derived from ISR systems.

- Deep ISR Processing by Crowds

Crowdsourcing is not simple.THE PRESIDENT: Three point five million people voted. I have to say that there was one question that was voted on that ranked fairly high and that was whether legalizing marijuana would improve the economy -- (laughter) -- and job creation. And I don't know what this says about the online audience -- (laughter) -- but I just want -- I don't want people to think that -- this was a fairly popular question; we want to make sure that it was answered. The answer is, no, I don't think that is a good strategy -- (laughter) -- to grow our economy. (Applause.)Digital Town Hall

Republican CrowdsourcedPolicy Suggestions

A 'teacher' told my child in class that dolphins were mammals and not fish! And the same thing about whales! We need TRADITIONAL VALUES in all areas of education. If it swims in the water, it is a FISH. Period! End of Story.

data isready andwaiting

Realtime Social Media Analytics

Countries are made of people, and people produce social media.

Realtime Social Media Analytics

Microblogging produces an up-to-the moment, publicly minable snapshot of people in crisis.

Realtime Social Media Analytics

The challenge is to identify:• Who can we trust?• What’s happening, where?• Which information is the most important?

…in seconds or minutes.

Red River FloodsRed River at East Grand Forks is 48.70 feet, +20.7 feet of flood stage, -5.65 feet of 1997 crest. #flood09

Post-hoc analytics of a crisis scenario

there's an emergency animal shelter set up at the Fargo fairgrounds

we are on the western central edge of town, so we are a fair distance from any water for now.

Tweets

[Starbird et al. CSCW ‘10, Vieweg et al. CHI ‘10]

From MWCFD: Eastwood addition, a lot of damage. Part has power, part doesn't. Won't open it back up

Red River Floods

White dots: tweets on the flood topic

[Starbird et al. CSCW ‘10, Vieweg et al. CHI ‘10]

Address/Intersection

Placename

Highways

City Name

County Name

0% 2% 4% 6% 8%10%12%14%

Geolocation information in tweets

Tweet the Debates

[Diakopoulos and Shamma CHI ‘10]

Twee

t Sen

timen

t Val

ence

Twee

t Vol

ume

Financial Crisis

McCain War Story Terrorism

Post-hoc analytics of McCain-Obama

Approach and Success Metrics

1) Archive datasets from global events.2) Can we mine the data that would help us

aid the situation, within seconds or minutes of the news breaking? Can we detect spammers?

3) Apply the techniques to new onset events.

people areready andwaiting

The ProblemUAV pilot interfaces are held back by AI-hard problems like computer vision.

MIT professor Missy Cummings used to fly F/A-18 Hornet fighters for the Navy. “I spent whole time complaining — who was the moron who designed this thing?” she recalled. If you’ve ever peeked inside a fighter cockpit, you’ll understand her gripe. Dials, displays and controls pack every nook and cranny. It’s the farthest thing from ergonomic.

- Wired

Crowd-Powered Interfaces

Can these cockpit interfaces:1. Learn from each others’ expertise?2. Benefit from outsourcing tasks

to non-experts?

The Interface User as a Crowd

Gathering pilot data and then mining it can lead to user interfaces that:• Prevent error

(“This is an uncommon operation!”)• Speed up laborious tasks• Provide trainee pilots with realtime feedback

on what expert pilots would do in a given situation

[Hartmann et al. CHI ‘10]

Outsourcing

Non-pilots can solve AI-hard problems in the background to support the user interface of the pilot.

There are ~25 million veterans and 850,000 reservists in the United States who could help from home.

Soylent: A Word Processor with a Crowd InsideAmazon Mechanical Turk workers will solve basic cognition tasks for cents. We can embed these workers inside the user interface:

[Bernstein et al. UIST ‘10]

Approach and Success Metrics

1) Design and build novel interfaces.2) Demonstrate that they improve performance

and reduce error with complex military or intelligence user interfaces

3) Deploy the interfaces and work through personnel training/scaling issues

Research ProblemsIntegrating social media into our on-the-groundsituational knowledge in real-time.

Teaching user interfaces to learn from other users and to outsource AI-hard problems to humans who can solve them quickly.

“there's an emergency animal shelter set up at the Fargo fairgrounds”

“From MWCFD: Eastwood addition, a lot of damage. Part has power, part doesn't.”

• Why: nations are increasingly producing social media, which breaks news faster than any other known method• If we don’t: we won’t be able to solve burgeoning problems before they become huge

• Why: highly trained individuals are still performing basic interface actions•If we don’t: our interfaces will continue to be more of a hindrance than a help to personnel

Related Research

Social data mining1. Vieweg, S., Hughes, A. L., Starbird, K., and Palen, L. 2010. Microblogging during two natural hazards

events: what twitter may contribute to situational awareness. CHI '10. ACM Press.2. Starbird, K., Palen, L., Hughes, A. L., and Vieweg, S. 2010. Chatter on the red: what hazards threat

reveals about the social life of microblogged information. CSCW '10. ACM Press.3. Diakopoulos, N. A. and Shamma, D. A. 2010. Characterizing debate performance via aggregated

twitter sentiment. CHI '10. ACM Press.

Crowd-powered interfaces4. Bernstein, M., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D., Panovich,

K. Soylent: A Word Processor with a Crowd Inside. UIST ‘10. ACM Press.5. Hartmann, B., MacDougall, D., Brandt, J., and Klemmer, S. R. 2010. What would other programmers

do: suggesting solutions to error messages. CHI '10. ACM Press.

What these researchers need: data sets, and the ability to roll out testbed interfaces to small groups.

How to Engage

1. Create TREK-style datasets and competitions for social media data analysis: can we extract the data we need to, quickly, and avoid spammers?

2. Encourage the development of novel crowdsourced interfaces for complex sensemaking scenarios

Success Metrics

1. Interfaces demonstrably improve efficiency and accuracy, and lessen the learning curve.

2. Embedded, distributed human help becomes a critical part of complex user interfaces.

3. We can use the Twitter Firehose to detect, track, and send support to crises when they occur

Proposed DARPA Initiatives

1. Archive all tweets about #iranelection and #peru earthquake, make them publicly available, and hold a competition to produce a time-dependent model of the crises as they unfold

2. Call for interfaces that can integrate latent human knowledge and expertise (e.g., in veterans and reservists) to support the main user.