Label Embedding Trees for Large Multi-class Tasks
-
Upload
brennan-matthews -
Category
Documents
-
view
49 -
download
1
description
Transcript of Label Embedding Trees for Large Multi-class Tasks
![Page 1: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/1.jpg)
Label Embedding Trees for Large Multi-class Tasks
Samy Bengio Jason WestonDavid Grangier
Presented by Zhengming Xing
![Page 2: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/2.jpg)
Outline
• Introduction• Label Trees• Label Embeddings• Experiment result
![Page 3: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/3.jpg)
Introduction
Main idea: propose a fast and memory saving multi-class classifier for large dataset based on trees structure method
Large scale problem:
the number of example Feature dimensionNumber of class
![Page 4: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/4.jpg)
Introduction Label Tree:
Label Predictors:
Indexed nodes:
Edges:
Label sets:
The root contain all classes, and each child label set is a subset of its parent
K is the number of classes
Disjoint tree: any two nodes at the same depth cannot share any labels.
![Page 5: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/5.jpg)
IntroductionClassifying an example:
![Page 6: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/6.jpg)
Label TreesTree loss
I is the indicator function
is the depth in the tree of the final prediction for x
![Page 7: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/7.jpg)
Label treeLearning with fixed label tree: N,E,L chosen in advanceGoal: minimize the tree loss over the variables F
Given training data
Relaxation 1
Relaxation 2
Replace indicator function with hinge loss and
![Page 8: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/8.jpg)
Label treeLearning label tree structure for disjoint tree
Treat A as the affinity matrix and apply the steps similar to spectral clustering
Basic idea: group together labels into the same label set that are likely to be confused at test time.
![Page 9: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/9.jpg)
Label embeddingsis a k-dimensional vector with a 1 in the yth position and 0 otherwise
Problem : how to learn W, V
define
solve
![Page 10: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/10.jpg)
Label embeddingsMethod 1:
The same two steps of algorithm 2Learn V
Learn W
minimize
minimize
![Page 11: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/11.jpg)
Label embedding
Combine all the methods discussed above
Method 2: join learn W and V
minimize
minimize
![Page 12: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/12.jpg)
ExperimentDataset
![Page 13: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/13.jpg)
Experiment
![Page 14: Label Embedding Trees for Large Multi-class Tasks](https://reader031.fdocuments.us/reader031/viewer/2022020111/56812d17550346895d920694/html5/thumbnails/14.jpg)
Experiment