CMU SCS U Kang (CMU) 1KDD 2012 GigaTensor: Scaling Tensor Analysis Up By 100 Times – Algorithms...

CMU SCS

U Kang (CMU) 1KDD 2012

GigaTensor: Scaling Tensor Analysis Up By 100 Times –

Algorithms and Discoveries

U Kang

ChristosFaloutsos

School of Computer ScienceCarnegie Mellon University

EvangelosPapalexakis

AbhayHarpale

CMU SCS


Outline

Problem Definition

Algorithm

Discoveries

Conclusions

CMU SCS


Background: Tensor

Tensors (=multi-dimensional arrays) are every-where Hyperlinks and anchor texts in Web graphs

URL 1

URL 2

Anchor Text

Java

C++

C#

11

1

1

1

11

CMU SCS


Background: Tensor

Tensors (=multi-dimensional arrays) are every-where Sensor stream (time, location, type) Predicates (subject, verb, object) in knowledge base

“Barrack Obama is the president of U.S.”

“Eric Clapton playsguitar”

(26M)

(26M)

(48M) NELL (Never Ending

Language Learner) dataNonzeros =144M

CMU SCS


Problem Definition

Q1: How to decompose a billion-scale tensor? Corresponds to SVD in 2D case

CMU SCS


Problem Definition

Q2: What are the important concepts and syn-onyms in a KB tensor? Q2.1: What are the dominant concepts in the

knowledge base tensor? Q2.2: What are the synonyms to a given noun

phrase?

(26M)

(26M)



CMU SCS


Outline

Problem Definition

Algorithm

Discoveries

Conclusions

CMU SCS


Algorithm: Problem Definition

Q1: How to decompose a billion-scale tensor? Corresponds to SVD in 2D case

CMU SCS


Challenge

Alternating Least Square (ALS) Algorithm

• •

: pseudo-inverse

How to design fast MapReduce algorithm for the ALS?

: Hadamard: Khatri-Rao

(J=26M)

(I=26M)

(K=48M)

Details

CMU SCS


Main Idea

1. Ordering of Computation Our choice

FLOPS (NELL data)𝟖 ⋅𝟏𝟎𝟗

FLOPS (NELL data)𝟐 .𝟓⋅𝟏𝟎𝟏𝟕

Details

CMU SCS


Main Idea

2. Avoiding Intermediate Data Explosion

Size of Intermediate Data (NELL) - Naïve: 100 PB

(J=26M)

(I=26M)

(K=48M)

Details

CMU SCS


Main Idea

2. Avoiding Intermediate Data Explosion

Size of Intermediate Data (NELL)- Proposed: 1.5 GB

Details

Size of Intermediate Data (NELL) - Naïve: 100 PB

(Before) (After)

CMU SCS


Experiments

GigaTensor solves 100x larger problem

Number of nonzero= I / 50

(J)

(I)

(K)

GigaTensor

Tensor

Toolbox Out ofMemory

100x

CMU SCS


Outline

Problem Definition

Algorithm

Discoveries

Conclusions

CMU SCS


Discoveries: Problem Definition

Q2: What are the important concepts and syn-onyms in a KB tensor? Q2.1: What are the dominant concepts in the

knowledge base tensor? Q2.2: What are the synonyms to a given noun

phrase?

(26M)

(26M)



CMU SCS


A2.1: Concept Discovery

Concept Discovery in Knowledge Base

CMU SCS


A2.1: Concept Discovery

CMU SCS


A2.2: Synonym Discovery

Synonym Discovery in Knowledge Base

a1 a2 aR…

(Given) noun phrase

(Discovered) synonym 1

(Discovered) synonym 2

CMU SCS


A2.2: Synonym Discovery

CMU SCS


Outline

Problem Definition

Algorithm

Discoveries

Conclusions

CMU SCS


Conclusion

GigaTensor: scalable tensor decomposition al-gorithm for billion-length modes tensors Algorithm: avoid intermediate data explosion Discoveries: concept discovery and contextual syn-

onym detection on KB tensor

CMU SCS


Thank you !www.cs.cmu.edu/~pegasuswww.cs.cmu.edu/~ukang

CMU SCS U Kang (CMU) 1KDD 2012 GigaTensor: Scaling Tensor Analysis Up By 100 Times – Algorithms...

Documents

Transcript of CMU SCS U Kang (CMU) 1KDD 2012 GigaTensor: Scaling Tensor Analysis Up By 100 Times – Algorithms...