01. Message Passing Interface (MPI) for Python Mohsan Jameel
Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan...
Transcript of Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan...
![Page 1: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/1.jpg)
Turning a Blind Eye:Explicit Removal of Biases and
Variation from Deep Neural Network Embeddings
Mohsan Alvi, Andrew Zisserman, and Christoffer NellåkerBias Estimation in Face Analytics
ECCV 2018 Workshop, 14/09/2018
![Page 2: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/2.jpg)
Introduction
Convolutional Neural Networks are the state-of-the-art in image classification
Rely on “good” training data
Not clear what the network has
learned
Networks can cheat by learning
spurious variations
Lack of comprehensively annotated data
![Page 3: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/3.jpg)
Contents
• Removing a bias from a network• Removing multiple spurious variations from a network• LAOFIW – Labeled Ancestral Faces in the Wild
![Page 6: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/6.jpg)
IMDB Faces Dataset [1]
• Dataset consists of celebrity faces from International Movie DataBase• Contains Age, Gender, Identity Labels
• Created two subsets of the dataset• Age• Gender
• Cleaned labels
[1] Rothe, Timofte, Van Gool. (2015)
![Page 10: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/10.jpg)
Training on Extremely Biased DataEvaluated on a gender/age balanced test dataset
Gender classification accuracy: 70%
![Page 11: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/11.jpg)
Turning a Blind Eye
• Primary task is the attribute of interest• Gender Classification
• Secondary task denotes bias to be unlearned• Age Classification
• Objective:learn feature representation that is informative for primary task, and uninformative for secondary task• Repurpose work in field of domain adaptation [2]
[2] Tzeng, Hoffman, Darrell, and Saenko. (2015)
![Page 12: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/12.jpg)
Methods (1/3)
Based on VGG-M Network [2]
[2] Chatfield, Simonyan, Vedaldi, and Zisserman. (2014)
Minimize:
![Page 14: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/14.jpg)
Methods (3/3)
Cross-entropy between classifier output and uniform distribution
Minimize:
Confusion Loss
Secondary Loss
Secondary Classification Secondary
Confusion
Act in opposition to each other
Alternate:
Primary Classification
&Secondary Confusion
Secondary Classification
Primary Classification
Secondary Confusion
![Page 15: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/15.jpg)
Results – Removing a bias
Baseline Gender classification accuracy: 70%Age-Blind Gender classification accuracy: 86%
![Page 16: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/16.jpg)
Removing multiple spurious variations (1/2)
Problem 1: Multiple biases may be present in dataset
AgeGender
Pose
Expression
Ancestry
![Page 17: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/17.jpg)
Removing multiple spurious variations (2/2)
• Problem 2: no single dataset contains labels for all biases
• Each labeled for a single purpose• Leverage information from multiple datasets
Dataset 3:Ancestry
Dataset 1:Gender
Dataset 4:Pose
Dataset 2:Age
![Page 18: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/18.jpg)
14,000 images in four classes:• Sub-Saharan Africa• Western Europe• East Asia• Indian subcontinent
Labeled Ancestral Origin Faces in the
Wild
LAOFIW
![Page 20: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/20.jpg)
Removing multiple spurious variations experiments
AncestryGender* PoseAge
GenderAncestry PoseAge
Experiment(1)
Experiment(2)
* Not extremely biased
![Page 21: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/21.jpg)
Results - Removing multiple spurious variations (1/2)
Primary task: Gender Secondary tasks: Age, Ancestry, Pose
![Page 22: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/22.jpg)
Results - Removing multiple spurious variations (2/2)
Primary task: Ancestry Secondary tasks: Age, Gender, Pose
![Page 23: Turning a Blind Eye - University of Oxfordvgg/publications/2018/Alvi18/... · 2018-10-04 · Mohsan Alvi, Andrew Zisserman, and Christoffer Nellåker Bias Estimation in Face Analytics](https://reader034.fdocuments.us/reader034/viewer/2022043017/5f3971eec8b66d6a4336febb/html5/thumbnails/23.jpg)
Conclusions
• Can improve generalizability of models train on biased datasets• Can remove multiple spurious variations from feature representation
of network• LAOFIW – ancestral origin dataset