Web as a Laboratory for Understanding Humanity

Jure LeskovecStanford University

3

Web – The Lab for Humanity

Large on-line applications

with hundreds of millions of

users

The Web is my “laboratory”

for understanding

the pulse of humanity.

4

Digital Traces Networks

Model as an interaction network

Analyzing Online Media

Collect 40 million articles, posts / dayStudy media ecosystem at large

2008 U.S. presidential election campaign

8

How is News Made?

Blogs trail mass media for 2.5h on average(but professional blogs lead!)

Influence of Media Types

NewspapersPro BlogsTVNews AgenciesBlogs

Entertainment

10

Media Networks

BlogsMainstream media

Media Networks: Zoom-in

BlogsMainstream media

Modeling Human Behavior

Predicting new links on FacebookOut of 20k we select 20 and get 8 right!

Friend or Foe?

?+

+

++

+

– –

––

–

> 90% accuracy

–

14

But, where are we going?

Observations:

Big data

Models: Predictions

Algorithms: Applications

Actively influencing the system

15

Changing the Behavior

… to evolve into a “happy” network?

How to change human

behaviors..

Observations: Data

analysis

Models: Predictions

Algorithms: Applications

Actively influencing the system

16

THANK YOU

20

Predicting links onWe get ~50% of them right

But, what is global utility?

Observations: Data

analysis

Models: Predictions

Algorithms: Applications

Actively influencing the system

Example: Utility of links

vs.

How to evolve into a “happy” network?

Too sparseToo

“cluttered”

Reflections

Why are networks organized the way they are?Build models/understandingMake predictions

Large dataObserve/model patterns not

visible at smaller scales

“The future is already here — it just is unevenly distributed”

William Gibson

THANK YOUhttp://

snap.stanford.edu

Physchohistory

Three axioms of Physchohistory:Detailed data about the population

Population should be sufficiently large

The population should remain in ignorance of the results

Can we do it today?

What’s next? combines history, sociology, and

mathematical statistics to make (nearly) exact predictions of the collective actions of very large groups of people, such as the Galactic Empire.

What’s next? psychohistorians, masters of an

esoteric mathematical science that enables them to plan for the future of their galactic civilization and realize their plans by making minimal changes in the society around them

Seriously Psychohistory depends on the idea that, while one cannot

foresee the actions of a particular individual, the laws of statistics as applied to large groups of people could predict the general flow of future events.

Asimov used the analogy of a gas: an observer has great difficulty in predicting the motion of a single molecule in a gas, but can predict the mass action of the gas to a high level of accuracy

If we’d want to predict future we would need: The population whose behavior is modeled should be sufficiently

large The population should remain in ignorance of the results