convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with...

22
Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda

Transcript of convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with...

Page 1: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Categorisation images of clothes with convolutional neural networks (CNN)

Author: Taras Sereda

Page 2: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

LeNET - 5 (Zip code recognition)Black

Yan LeCuNAI lab director at Facebook(NYU)

[1] http://yann.lecun.com/exdb/lenet/

http://yann.lecun.com/exdb/lenet/
Page 3: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Convolution operation

Dot product between image patch and kernel shifted by bias.

[2]http://intellabs.github.io/RiverTrail/tutorial/

http://intellabs.github.io/RiverTrail/tutorial/
Page 4: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Max Pooling

pooling neighboring features, reducing parameters amount.

[3]http://cs231n.github.io/convolutional-networks/

http://cs231n.github.io/convolutional-networks/
Page 5: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

AlexNet (ImageNet 2012 winning solution )

[CONV -> POOL] * 2 -> CONV -> CONV -> CONV -> POOL -> FC -> FC -> SOFTMAX

[4] imagenet-classification-with-deep-convolutional-neural-networks

http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks
Page 6: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

ReLU activation function

f = max(0,x)

● speeds up training● more expressive features

[5]http://cs231n.github.io/neural-networks-1/

http://cs231n.github.io/neural-networks-1/
Page 7: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

GoogleNet (ImageNet 2014 winning solution )

[6] Going deeper with convolution

http://arxiv.org/abs/1409.4842
Page 8: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

CONV1 weights

Similar to V1 layer of visual cortex

Page 9: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Gathering datasets

● Nearly 800 k of user photos● ~120 classes in total● Noize ~30-40%● manually pick 100 images for each class(human)● train CNN● use network for gathering data.

Page 10: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Finetuning for Clothes classification with Caffe

● Small learning rate: 0.001● Batch_size: 256 ● Classes: 56● Dataset: 20k● time: 20 hours on K520 GPU● Amazon: g2.2xlarge

Page 11: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Wrong

Correct

1,8k Test set accuracy

Finetuning Results

Page 12: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Online data augmentation #0

● Random crops ● Random rotations [-50;50] ● For generalization and rotation invariance.

Page 13: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Online data augmentation #1

-60 degree "accessory.glasses": "0.90990"

original image"accessory.glasses”: “1.0000”

-90 degree”accessory.glasses": "0.78519”

Page 14: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Online data augmentation #2

90 degree"feet.sport.low trainers": "0.38777", "feet.sport.high trainers": "0.27408",

original image "feet.sport.low trainers": "1.00000"

-30 degree"feet.sport.low trainers": "0.95052"

Page 15: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Client sideGarment adding process.

Page 16: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Client sideResponse as text + icone.

Page 17: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

tt CAFFE

Python API

Docker

Cloud server

Deployment

Page 18: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Classes Hierarchy

legs

entity

body feat head

Page 19: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

DARTS (hadging your bets)

Solwing accaracy specificity tradeof

accuracy

reward

[7] DARTS, hedging your bets

http://www.image-net.org/projects/hedging/
Page 20: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

DARTS Benefits

● get still correct predicts but less specific● body.knitwear.cardigan is also body.knitwear● needs only posterior probabilities on leaf nodes.

Page 21: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Future plans

● add more classes,move to 120 in total.● build ansambel of specialized models● add image segmentation● apply attention concepts for image captioning● move towards online learning

Page 22: convolutional neural networks (CNN) Categorisation … · Categorisation images of clothes with convolutional neural networks (CNN) Author: Taras Sereda. LeNET - 5 (Zip code recognition)

Thanks!Any questions?

[email protected]


DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor

DeepPose & Convolutional Pose Machinescseweb.ucsd.edu/.../Presentations/12_Human_Pose_Estimation_Rant… · DeepPose & Convolutional Pose Machines. Main Concepts 1. CNN with regressor

CNN Explainer: Learning Convolutional Neural Networks with ... · CNN EXPLAINER aims to bridge this critical gap. Contributions. In this work, we contribute: • CNN EXPLAINER, an

CNN Explainer: Learning Convolutional Neural Networks with ... · CNN EXPLAINER aims to bridge this critical gap. Contributions. In this work, we contribute: • CNN EXPLAINER, an

CNN-MGP: Convolutional Neural Networks for Metagenomics ... › content › pdf › 10.1007 › s12539-018-0313-4.pdfA convolutional neural network (CNN) is a special type of neural

CNN-MGP: Convolutional Neural Networks for Metagenomics ... › content › pdf › 10.1007 › s12539-018-0313-4.pdfA convolutional neural network (CNN) is a special type of neural

Convolutional Neural Network - CNN · Introduction CNN Layers CNN Models Popular Frameworks Papers References Convolutional Neural Network - CNN Eduardo Todt, Bruno Alexandre Krinski

Convolutional Neural Network - CNN · Introduction CNN Layers CNN Models Popular Frameworks Papers References Convolutional Neural Network - CNN Eduardo Todt, Bruno Alexandre Krinski

Convolutional Neural Networks (CNN)

Convolutional Neural Networks (CNN)

Research Trends on Convolutional Neural Network (CNN ...

Research Trends on Convolutional Neural Network (CNN ...

A-CNN: Annularly Convolutional Neural Networks on Point Clouds · Figure 1: The proposed annularly convolutional neural networks (A-CNN) model on point clouds to perform classification,

A-CNN: Annularly Convolutional Neural Networks on Point Clouds · Figure 1: The proposed annularly convolutional neural networks (A-CNN) model on point clouds to perform classification,

05-Deep Neural Networks-2-CNN - Pattern Recognition in ...pattern-recognition.weebly.com/.../05-deep_neural_networks-2-cnn.… · A Convolutional Neural Network(CNN) ... • Handwritten

05-Deep Neural Networks-2-CNN - Pattern Recognition in ...pattern-recognition.weebly.com/.../05-deep_neural_networks-2-cnn.… · A Convolutional Neural Network(CNN) ... • Handwritten

Convolutional Neural Networks Analyzed via Convolutional ... · Convolutional neural networks (CNN) lead to remarkable results in many fields Clear and profound theoretical understanding

Convolutional Neural Networks Analyzed via Convolutional ... · Convolutional neural networks (CNN) lead to remarkable results in many fields Clear and profound theoretical understanding

Convolutional Neural Networks - TJ Machine Learning Clubtjmachinelearning.com/lectures/cnn/cnn.pdf · 2018-05-23 · Convolutional Neural Networks Mihir Patel ... Traditional neural

Convolutional Neural Networks - TJ Machine Learning Clubtjmachinelearning.com/lectures/cnn/cnn.pdf · 2018-05-23 · Convolutional Neural Networks Mihir Patel ... Traditional neural

CNN 101: Interactive Visual Learning for Convolutional ... · CNN 101 is an interactive system for illustrating how a trained CNN model classifies an image (Figure 1). It en-ables

CNN 101: Interactive Visual Learning for Convolutional ... · CNN 101 is an interactive system for illustrating how a trained CNN model classifies an image (Figure 1). It en-ables

Convolutional Networks: Overviewsrihari/CSE676/9.0 CNN-Overview.pdf · •Motivation behind using convolution in a neural network •Pooling, which almost all convolutional networks

Convolutional Networks: Overviewsrihari/CSE676/9.0 CNN-Overview.pdf · •Motivation behind using convolution in a neural network •Pooling, which almost all convolutional networks

Convolutional Neural Network Hyper-Parameters Optimization ...€¦ · applications, Convolutional Neural Network (CNN) is the most widely used technique for image classification.

Convolutional Neural Network Hyper-Parameters Optimization ...€¦ · applications, Convolutional Neural Network (CNN) is the most widely used technique for image classification.

POP-CNN: Predicting Odor’s Pleasantness with Convolutional ...

POP-CNN: Predicting Odor’s Pleasantness with Convolutional ...

T-CNN: Tubelets with Convolutional Neural Networks for ...1 T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos Kai Kang, Hongsheng Li, Junjie Yan,

T-CNN: Tubelets with Convolutional Neural Networks for ...1 T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos Kai Kang, Hongsheng Li, Junjie Yan,

Tube Convolutional Neural Network (T-CNN) for Action ...openaccess.thecvf.com/...Tube_Convolutional_Neural... · propose Tube Convolutional Neural Network (T-CNN) for action detection.

Tube Convolutional Neural Network (T-CNN) for Action ...openaccess.thecvf.com/...Tube_Convolutional_Neural... · propose Tube Convolutional Neural Network (T-CNN) for action detection.

Convolutional Neural Network - CNN - UFPR

Convolutional Neural Network - CNN - UFPR

Convolutional Neural Networks (CNN) in Medical and General ...€¦ · Convolutional Neural Network -CNN • Imaging physics and convolution mathematics to understand how each component

Convolutional Neural Networks (CNN) in Medical and General ...€¦ · Convolutional Neural Network -CNN • Imaging physics and convolution mathematics to understand how each component

Convolutional Networks - Université de Montréal · Convolutional Neural Networks •Object detection systems based on the deep convolutional neural network (CNN) have recently made

Convolutional Networks - Université de Montréal · Convolutional Neural Networks •Object detection systems based on the deep convolutional neural network (CNN) have recently made

Convolutional Neural Networkscis581/Lectures/Fall2018/CNN-intr… · Convolutional Neural Network (CNN) Pipeline: A Monitor. Convolutional Neural Nets (CNNs) in a nutshell: • A

Convolutional Neural Networkscis581/Lectures/Fall2018/CNN-intr… · Convolutional Neural Network (CNN) Pipeline: A Monitor. Convolutional Neural Nets (CNNs) in a nutshell: • A