DEFECT INSPECTION FROM SCRATCH TO PRODUCTION · 2 AGENDA Defect Inspection and its challenges NGC...
Transcript of DEFECT INSPECTION FROM SCRATCH TO PRODUCTION · 2 AGENDA Defect Inspection and its challenges NGC...
Andrew Liu, Ryan Shen
Deep Learning Solution Architect
DEFECT INSPECTION FROM SCRATCH TO PRODUCTION
2
AGENDA
Defect Inspection and its challenges
NGC Docker images
Model set up - Unet
Data preparation - DAGM
Production - Speed up with TensorRT
3
INDUSTRIAL DEFECT INSPECTION
4
INDUSTRIAL DEFECT INSPECTION
5
NGC DOCKER IMAGES
Benefits for Deep Learning WorkflowHigh Level Benefits and Feature Set
Single software stack
Develop once, deploy anywhere
Scale across teams of practitioners
Developer, DevOp, QC
7
Defect inspection Workflowfrom scratch to production within container
Training Inference
NGC optimized docker image TENSORRT GRE
docker pull
nvcr.io/nvidia/tensorflow
:18.02-py3
docker pull
nvcr.io/nvidia/tensorrt:
18.02-py2
8
MODEL SET UP
9
WAYS FOR DEFECT INSPECTION
Depends on domain adaptation
Image SegmentationObject Detection
Image Classification
Label: 0 / 1 (Defect / Non Defect)
Label: Bounding-Box Label: Polygons Mask
10
CNN STRUCTURELeNet
Source: Design of Deep Convolutional Neural Network Architectures for Automated Feature Extraction in Industrial Inspection, D. Weimer et al, 2016
11
“defect
class1”
1000-dim vector
< 1 millisecond
convnets perform classification
end-to-end learning
https://arxiv.org/abs/1605.06211
12
~1/10 second
end-to-end learning
???
lots of pixels, little time?
https://arxiv.org/abs/1605.06211
51
22
51
22
51
22
1 16 16
16 32 32
32 64 64
64 128 128 256
128 256 256
25
62
25
62
25
62
12
82
12
82
12
82
64
2
64
2
64
2
32
2
32
2
32
2
12
82
12
82
12
82
128 64 64
128 128
64
2
64
2
64
2
64 32 32
25
62
25
62
25
62
51
22
51
22
51
22
32 16 16
3X3 Conv2d+ReLU
2X2 MaxPool
2X2 Conv2dTranspose
copy and concatenate
U-Net structure
14
KERAS IMPLEMENTATIONConvolution
15
DECODINGdeconvolution
16
Image segmentation on medical imagesSame process among various use cases
MRI image
Left ventricle
heart disease
Data Science BOWL2016
Data Science BOWL 2017
CT image
Nodule
Lung cancer
Image
Nuclei
Drug discovery
Data Science BOWL2018
17
DATA PREPARATION
18
INDUSTRIAL OPTICAL INSPECTIONGerman Association for Pattern Recognition
• http://resources.mpi-inf.mpg.de/conferences/dagm/2007/prizes.html
19
INDUSTRIAL OPTICAL INSPECTIONGerman Association for Pattern Recognition
Pass NG Pass
NGNGPass Pass
NG
20
DATA DETAILS
• Original images are 512 x 512 grayscale format
• Output is a tensor of size 512 x 512 x 1
• Each pixel belongs to one of two classes
• Training set consist of 100 images
• Validation set consist of 50 images
21
MORE EXAMPLES
22
IMBALANCE DATA
23
Dice Metric (IOU)• Metric to compare the similarity of two samples:
2𝐴𝑛𝑙________________________________
𝐴𝑛 + 𝐴𝑙
• Where: • An is the area of the contour predicted by the network • Al is the area of the contour from the label• Anl is the intersection of the two
• The area of the contour that is predicted correctly by the network
• 1.0 means perfect score.
• More accurately compute how well we’re predicting the contour against the label
• We can just count pixels to give us the respective areas
24
LOSS FUNCTION WITH KERAS
24
25
LEARNING CURVES
25
LOSS
26
APPLICATION: INDUSTRIAL INSPECTIONNVIDIA
• 1000 defect-free, 150 defect images
• Challenges: Not all deviations from the
texture are necessarily defects.
27
PRODUCTION
28
FINAL DECISIONPlus Human Logic
Size, Position, … etc
29
INFERENCE PIPELINE
Post-processing and decision makingInference
Camera
Machine
P40
DGX Server
TensorRT+GRE
Fetch image
TensorRT+GRE
Detectors/Classifiers/Segment Composite Result Metadata
DataCenter
Edge
Defect Pattern Ratio Defect Level Defect region size Defect counts
…
Domain Criteria Determine threshold
Precision/ Recall
30
Development flow
Called UFF, Universal Framework FormatONNX
31
Defect inspection Workflow
from scratch to production within container
Training Inference
NGC optimized docker image TENSORRT GRE
docker pull
nvcr.io/nvidia/tensorflow
:18.02-py3
docker pull
nvcr.io/nvidia/tensorrt:
18.02-py3
32
SUMMARY
Challenges Delivers
Training , inference environment is hard to build, maintain, share. Using NGC Docker images.
Model optimizations and speed up throughput. TensorRT + GRE SDK
So many deep learning model out there, how to choose the right
model?
If your dataset, demand requirement fit the scenario
like we do. U-Net model is great choice for
segmentation task.
33
Thank You