Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf ·...
Transcript of Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf ·...
![Page 1: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/1.jpg)
Data Visualization for Machine LearningFernanda Viégas @viegasfGoogle Brain
![Page 2: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/2.jpg)
![Page 3: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/3.jpg)
Open Source toolsand platforms
Public presentations,sharing best practices
Public Symposia& meetings
Educational Materials
AcademicPublications
PAIR | People + AI Research InitiativeBringing Design Thinking and HCI to Machine Learninggoogle.ai/pair
Visiting Faculty,Faculty Grants
![Page 4: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/4.jpg)
Training data is crucial Debug your data before
debugging your model
![Page 5: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/5.jpg)
Let's start with a data set you might have heard of 😉
![Page 6: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/6.jpg)
32 x 32 images10 classes
![Page 7: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/7.jpg)
FacetsOpen-source pair-code.github.io/facets
![Page 8: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/8.jpg)
ATTORNEY CLIENT PRIVILEGED & CONFIDENTIAL
Gender Shades Joy Buolamwini MIT Media Lab
![Page 9: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/9.jpg)
What-If Tool open sourcecode-free ML probingpair-code.github.io/what-if-tool
![Page 10: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/10.jpg)
What-If Tool Fairness metrics
![Page 11: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/11.jpg)
Model Understanding Looking into high-dimensional
spaces
![Page 12: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/12.jpg)
Warm up:MNIST
![Page 13: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/13.jpg)
Images as vectors
![Page 14: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/14.jpg)
Images as vectors
0
0.5
1
![Page 15: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/15.jpg)
Images as vectors
0
0.5
1
(1,1,1,1,1,1,1,1,1,.8,0.,5,1,1,1,1,1,.8,.3,0,0,0,0,0,.1,3,1,1,1,1,1,1,1,1,1,.3,.1,0,0,0,0,0,.4,.9,1, … )
![Page 16: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/16.jpg)
We’ve turned this image into this vector
(1,1,1,1,1,1,1,1,1,.8,0.,5,1,1,1,… )
784 pixels → 784 dimensions
![Page 17: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/17.jpg)
We’ve turned this image into this vector
(1,1,1,1,1,1,1,1,1,.8,0.,5,1,1,1,… )
784 pixels → 784 dimensions
(1,1,1,1,1,1,1,1,.6,.7,0.,4,1,1,1,… )
(1,1,1,1,1,1,1,1,.4,.5,0.,3,.2,1,1,… )
.
.
.
![Page 19: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/19.jpg)
Model interpretability use caseMulti-lingual translationWhat does the language embedding space look like?
https://arxiv.org/abs/1611.04558Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, Jeffrey Dean
![Page 20: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/20.jpg)
Training: English ← → Japanese English ← → Korean
![Page 21: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/21.jpg)
Training: English ← → Japanese English ← → KoreanJapanese ← → Korean (zero shot)
![Page 22: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/22.jpg)
Visualize internal representation ("embedding space")
![Page 23: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/23.jpg)
Research questionWhat does the multi language embedding space look like?
or
Note: not real data
![Page 24: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/24.jpg)
What does a sentence look like in embedding space?(points in 1024-dim space: the data that the decoder receives)
E.g. “The stratosphere extends from 10km to 50km in altitude”
![Page 25: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/25.jpg)
What does a sentence look like in embedding space?
Note: simplification of real situation!
![Page 26: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/26.jpg)
What does a sentence look like in embedding space?
![Page 27: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/27.jpg)
What do parallel sentences look like in embedding space? (same meaning, different language)
<2pt>
<2en>
EnglishPortuguese
like this?
![Page 28: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/28.jpg)
What do parallel sentences look like in embedding space? (same meaning, different language)
EnglishPortuguese
or like this?
![Page 29: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/29.jpg)
Interlingua? Sentences with the same meaning mapped to similar regions regardless of language!
![Page 30: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/30.jpg)
Distance between bridge / non-bridge sentences is inversely related to translation quality
![Page 31: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/31.jpg)
word embeddings
![Page 32: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/32.jpg)
Word embeddings
Distributed Representations of Words and Phrases and their Compositionality
Mikolov et al. 2013
![Page 33: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/33.jpg)
Meaningful directions(word2vec)
![Page 34: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/34.jpg)
![Page 35: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/35.jpg)
![Page 36: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/36.jpg)
![Page 37: Data Visualization for Machine Learningintrotodeeplearning.com/2019/materials/2019_6S191_L7.pdf · English Portuguese or like this? Interlingua? Sentences with the same meaning mapped](https://reader034.fdocuments.us/reader034/viewer/2022042312/5eda5fd9b3745412b5713cd8/html5/thumbnails/37.jpg)
Data Visualization for Machine LearningFernanda Viégas @viegasfgoogle.ai/pair