Web Intelligence and Big DataIntroduction to the Web Intelligence

Mangesh R. Wanjari

Lecture 1

1/4/2017 1

1/4/2017 2

Outline

• Turing test

• Intelligence of a machine

• Web scale AI applications

• Big Data?

• Analytics

1/4/2017 3

Turing test

Textual typewritten messages only

Party Game

Which is man and which is

woman?

1/4/2017 4

Turing test

Textual typewritten messages only

Party Game: Replace one of the humen with a Machine

Which is man and which is

machine?

1/4/2017 5

Turing test

Textual typewritten messages only

Party Game: Replace Judge with a machine

Which is man and which is

machine?

1/4/2017 6

Reverse Turing test

Textual typewritten messages only

Like/Dislike Shopper/Surfer Rich/Poor

Which is man and which is

woman?

1/4/2017 7

Web Scale AI

1/4/2017 8

Moore’s Law

Moore's law refers to an observation made by Intel co-founder Gordon Moore in 1965. He noticed that the number of transistors per square inch on integrated circuits had doubled every year since their invention.

1/4/2017 9

Kryder’s Law

The density of information on hard driveshas been growing at an even faster rate,increasing by a factor of 1000 in 10.5 years,which corresponds to a doubling roughlyevery 13 months

1/4/2017 10

Big Data?

• Lots of web pages• A billion Facebook users• Billion+ facebook pages• Hundreds of millions Twitter account• Hundreds of millions Tweets per day• Billions of Google queries per day-may be

more• Millions of servers, petabytes of data

1/4/2017 11

Big corporations work on

• 5000-50000 servers, may be some more

• Terabytes of data, millions transactions per day

1/4/2017 12

Big Data technology

• Traditional BI using databases

• Google, Facebook, LinkedIn, eBay, Amazon, … did not use traditional databases for their data

Statistical report

Databases Data Warehouse

More Databases

1/4/2017 13

What they use?

• Parallel programming

• Massive parallelism

• Map-Reduce paradigm

1/4/2017 14

Data and intelligence

• “Any fool can know. The point is to understand.”

-Albert Einstein

The goal of understanding is to predict.

1. Reactive Intelligence

2. Predictive intelligence

1/4/2017 15

Data and intelligence

Look

Listen

Learn

Connect

Predict

Correct

1/4/2017 16

Thank you!!!