Object Detection Using Semi-Naïve Bayes to Model Sparse Structure
description
Transcript of Object Detection Using Semi-Naïve Bayes to Model Sparse Structure
![Page 1: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/1.jpg)
Object Detection Using Semi-Naïve Bayes to Model Sparse
Structure
Henry Schneiderman
Robotics InstituteCarnegie Mellon University
![Page 2: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/2.jpg)
Object Detection
• Find all instances of object X (e.g. X = human faces)
![Page 3: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/3.jpg)
Examples of Detected Objects
![Page 4: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/4.jpg)
Sparse Structure of Statistical Dependency
Chosen variable Chosen variableChosen variable
![Page 5: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/5.jpg)
Sparse Structure of Statistical Dependency
Chosen coefficientChosen coefficient Chosen coefficient
![Page 6: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/6.jpg)
Sparse Structure of Statistical Dependency
Chosen coefficient Chosen coefficient Chosen coefficient
![Page 7: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/7.jpg)
Detection using a Classifier
“Object is present” (at fixed size and alignment)
“Object is NOT present”(at fixed size and alignment)
Classifier
![Page 8: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/8.jpg)
Proposed Model: Semi-Naïve Bayes
• Kononenko (1991), Pazzini (1996), Domingos and Pazzini (1997), Rokach and Maimon (2001)
)|(
)|(log...
)|(
)|(log
)|(
)|(log),...(
2
1
22
12
21
111
n
nr SP
SP
SP
SP
SP
SPxxH
},...,{,..., 11 rn xxSS
e.g. S1 = (x21, x34, x65, x73, x123) S2 = (x3, x8, x17, x65, x73, x111)
subsets input variables
![Page 9: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/9.jpg)
Goal: Automatic subset grouping
S1 = (x21, x34, x65, x73, x123)S2 = (x3, x8, x17, x65, x73, x111) . . .Sn = (x14, x16, x17, x23, x85, x101, x103, x107)
)|(
)|(log...
)|(
)|(log
)|(
)|(log),...(
2
1
22
12
21
111
n
nr SP
SP
SP
SP
SP
SPxxH
![Page 10: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/10.jpg)
Approach: Selection by Competition
x1 x2 x3 . . . xm
S1 S2 . . . Sq
Generate q candidate subsets
log [p1(S1|1) / p1(S1|2)] log [p2(S2|1) / p2(S2|2)] . . . log [pq(Sq|1) / pq(Sq|2)]
Train q log likelihood function
Select combination of n candidates
log [pj1(Sj1|1) / pj1(Sj1|2)] log [pj2(Sj2|1) / pj2(Sj2|2)] . . . log [pjn(Sjn|1) / pjn(Sjn|2)]
H(x1,…,xr) = log [pj1(Sj1|1) / pj1(Sj1|2)]+log [pj2(Sj2|1) / pj2(Sj2|2)] +. . .+ log [pjn(Sjn|1) / pjn(Sjn|2)]
q functions
n functions, n << q
![Page 11: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/11.jpg)
Approach: Selection by Competition
x1 x2 x3 . . . xm
S1 S2 . . . Sq
Generate q candidate subsets
log [p1(S1|1) / p1(S1|2)] log [p2(S2|1) / p2(S2|2)] . . . log [pq(Sq|1) / pq(Sq|2)]
Train q log likelihood function
Select combination of n candidates
log [pj1(Sj1|1) / pj1(Sj1|2)] log [pj2(Sj2|1) / pj2(Sj2|2)] . . . log [pjn(Sjn|1) / pjn(Sjn|2)]
H(x1,…,xr) = log [pj1(Sj1|1) / pj1(Sj1|2)]+log [pj2(Sj2|1) / pj2(Sj2|2)] +. . .+ log [pjn(Sjn|1) / pjn(Sjn|2)]
![Page 12: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/12.jpg)
Generation of Subsets
• “modeling error for assuming independence”
qxP
xP
xPxP
xPxP
xxP
xxP
q dxdxabsxxPCq
q
q
q ...]...log[log)...(... 1)|(
)|(
)|()|(
)|()|(
)|...(
)|...(
11 2
1
22
12
21
11
21
11
q is size of the subset
![Page 13: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/13.jpg)
Generation of Subsets
• Selection of variables - “discrimination power”
qxxP
xxP
q dxdxabsxxPCq
q ...][log)...(... 1)|...(
)|...(
12 21
11
q is size of the subset
![Page 14: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/14.jpg)
Pair-Wise Measurement
• pair-wise measurements
kj
k
k
j
j
kj
kj
xxkjxP
xPxP
xP
xxP
xxP
kjkj dxdxabsxxPxxC,
)|()|(
)|(
)|(
)|,(
)|,(
1 ]log[log),(),(2
1
2
1
2
1
),(),(),( 21 kjkjkj xxCxxCxxC Pair-affinity
kj
kj
kj
xxkjxxP
xxP
kjkj dxdxabsxxPxxC,
)|,(
)|,(
2 ][log),(),(2
1
![Page 15: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/15.jpg)
Visualization of C(x,*)(frontal faces)
xx x
![Page 16: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/16.jpg)
Measure over a Subset
),()(,
)1(1
kj
xxSxx
NNi xxCSD
kj
ikj
iSN Subset-affinity
![Page 17: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/17.jpg)
Generation of Candidate Subsets
x1 x2 x3 . . . . . . . . . . . . . . . . . . . . . . . . xm
S1 S2 . . . . . . . . . . . . . . . . . . Sp
Heuristic search and selective evaluation of D(Si)
C(x1, x2) C(x1, x3) . . . . . . . . C(xm-1, xm)
![Page 18: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/18.jpg)
subset size vs. modeling power
• Model complexity limited by number of training examples, etc.
• Examples of limited modeling power– 5 modes in a mixture model– 7 projection onto principal components
![Page 19: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/19.jpg)
Approach: Selection by Competition
x1 x2 x3 . . . xm
S1 S2 . . . Sq
Generate q candidate subsets
log [p1(S1|1) / p1(S1|2)] log [p2(S2|1) / p2(S2|2)] . . . log [pq(Sq|1) / pq(Sq|2)]
Train q log likelihood function
Select combination of n candidates
log [pj1(Sj1|1) / pj1(Sj1|2)] log [pj2(Sj2|1) / pj2(Sj2|2)] . . . log [pjn(Sjn|1) / pjn(Sjn|2)]
H(x1,…,xr) = log [pj1(Sj1|1) / pj1(Sj1|2)]+log [pj2(Sj2|1) / pj2(Sj2|2)] +. . .+ log [pjn(Sjn|1) / pjn(Sjn|2)]
![Page 20: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/20.jpg)
Log-likelihood function = Table
)...1( ii Nf
Si = (xi1, xi2, . . ., xiq)
)log( )|()|(
2
1
i
i
fPfP
vector quantization
table look-up
![Page 21: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/21.jpg)
Sub-Classifier Training by Counting
Pi (fi |1)
Pi (fi |2)fi
fi
![Page 22: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/22.jpg)
Example of VQ
xi1 xi2 xi3 . . . xiq
c1 c2 c3
z1 z2 z3
f = z1m0 + z2m1 + z3m2
quantization to m levels
projection on to 3 principal components
![Page 23: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/23.jpg)
Approach: Selection by Competition
x1 x2 x3 . . . xm
S1 S2 . . . Sq
Generate q candidate subsets
log [p1(S1|1) / p1(S1|2)] log [p2(S2|1) / p2(S2|2)] . . . log [pq(Sq|1) / pq(Sq|2)]
Train q log likelihood function
Select combination of n candidates
log [pj1(Sj1|1) / pj1(Sj1|2)] log [pj2(Sj2|1) / pj2(Sj2|2)] . . . log [pjn(Sjn|1) / pjn(Sjn|2)]
H(x1,…,xr) = log [pj1(Sj1|1) / pj1(Sj1|2)]+log [pj2(Sj2|1) / pj2(Sj2|2)] +. . .+ log [pjn(Sjn|1) / pjn(Sjn|2)]
![Page 24: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/24.jpg)
h1(S1) h2(S2) . . . hP(SP)
E1,1 E1,2 E2,1 E2,2 . . . Ep,1 Ep,2 Evaluate on training data
ROC1 ROC2 . . . ROCPEvaluate ROCs
Order top Qlog-likelihoodfunctions
hj1(Sj1) hj2(Sj2) . . . hjQ(SjQ)
Candidatelog-likelihoodfunctions
![Page 25: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/25.jpg)
hj1(Sj1) + h1(S1) . . . hjQ(SjQ) + hp(Sp)Form pQ pairsof log-likelihoodfunctions
Ej1,1+ E1,1 Ej1,2+ E1,2 . . . EjQ,1+ Ep,1 EjQ,2+ Ep,2 Sum Evaluations
ROC1 . . . ROCQPEvaluate ROCs
Order top Qpairs of log-likelihoodfunctions
hk1,1(Sk1,1) + hk1,2(Sk1,2) . . . hkQ,1(SkQ,1) + hkQ,2(SkQ,2)
. . . Repeat for n iterations
![Page 26: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/26.jpg)
H1(x1, x2, . . ., xr) . . . HQ(x1, x2, . . ., xr)
Cross-Validation Selects Classifier
Q Candidates: H1(x1, x2, . . ., xr) = hk1,1(Sk1,1) + hk1,2(Sk1,2) +. . .+ hk1,n(Sk1,n) . . . HQ(x1, x2, . . ., xr) = hkQ,1(SkQ,1) + hkQ,2(SkQ,2) + . . . + hQ,n(SkQ,n)
H*(x1, x2, . . ., xr)
Cross-validation
![Page 27: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/27.jpg)
Example subsets learned for telephones
![Page 28: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/28.jpg)
Evaluation of Classifier
“Object is present” (at fixed size and alignment)
“Object is NOT present”(at fixed size and alignment)
Classifier
![Page 29: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/29.jpg)
1) Compute feature values
f2 = #3214
f1 = #5710
fn = #723
![Page 30: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/30.jpg)
2) Look-Up Log-Likelihoods
P1( #5710 | 1)
P1( #5710 | 2)
f2 = #3214
f1 = #5710
fn = #723
= 0.53
P2( #3214 | 1)
P2( #3214 | 2)= 0.03log
log
Pn( #723 | 1)
Pn( #723 | 2)= 0.23log
![Page 31: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/31.jpg)
3) Make Decision
P1( #5710 | 1)
P1( #5710 | 2)= 0.53
P2( #3214 | 1)
P2( #3214 | 2)= 0.03log
log
Pn( #723 | 1)
Pn( #723 | 2)= 0.23log
> <0.53 + 0.03 + . . . + 0.23
![Page 32: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/32.jpg)
Detection using a Classifier
“Object is present” (at fixed size and alignment)
“Object is NOT present”(at fixed size and alignment)
Classifier
![Page 33: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/33.jpg)
View-based Classifiers
FaceClassifier #1
FaceClassifier #2
FaceClassifier #3
![Page 34: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/34.jpg)
Detection: Apply Classifier Exhaustively
Search in position
Search in scale
![Page 35: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/35.jpg)
Decision can be made by partial evaluation
P1( #5710 | 1)
P1( #5710 | 2)= 0.53
P2( #3214 | 1)
P2( #3214 | 2)= 0.03log
log
Pn( #723 | 1)
Pn( #723 | 2)= 0.23log
> <0.53 + 0.03 + . . . + 0.23
![Page 36: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/36.jpg)
Detection Computational Strategy
Computational strategy changes with size of search space
Apply log [p1(S1|1) / p1(S1|2)]exhaustively to scaled input image
Apply log [p2(S2|1) / p2(S2|2)]reduced search space
Apply log [p3(S3|1) / p3(S3|2)]further reduced search space
![Page 37: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/37.jpg)
Repeat for N2 Candidates
Compute M2 feature values Look-up M2 log-likelihood values
Candidate-Based Evaluation
![Page 38: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/38.jpg)
Repeat for N2 Candidates
Compute N2 + M2 +2MN feature values Look-up M2 log-likelihood values
Feature-Based Evaluation
![Page 39: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/39.jpg)
Create candidate subsets
Train candidate log-likelihood functions
Select log-likelihood functions
Retrain selected log-likelihoodfunctions using Adaboost
Training imagesof object
Cross-validationimages
Training images of non-object
Determine detection threshold
Automatically select non-objectexamples for next stage
Images that donot contain object
Increment stage
Cascade Implementation
Adaboost using confidence-rated predictions [Shapire and Singer, 1999]
Bootstrapping [Sung and Poggio, 1995]
![Page 40: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/40.jpg)
Face, eye, ear detection
![Page 41: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/41.jpg)
Frontal Face Detection
Recognition rate 85.2% 89.7% 92.1% 93.7% 94.2%
False detections(this method)
6 13 44 64 79
False Detections[Viola and Jones, CVPR, 2001]
-- 31 50 167 --
• MIT-CMU Frontal Face Test Set [Sung and Poggio, 1995; Rowley, Baluja and Kanade, 1997]
– 180 ms 300x200 image– 400 ms 300x500 image
• Top Rank Video TREC 2002 Face Detection• Top Rank 2002 ARDA VACE Face Detection algorithm evaluation
AMD Athalon 1.2GHz
![Page 42: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/42.jpg)
Face & Eye Detection for Red-Eye Removal from Consumer Photos
CMU Face Detector
![Page 43: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/43.jpg)
Eye Detection
• Experiments performed independently at NIST
• Sequested data set: 29,627 mugshots• Eyes correctly located (radius of 15 pixels)
98.2% (assumed one face per image)• Thanks to Jonathon Phillips, Patrick
Grother, and Sam Trahan for their assistance in running these experiments
![Page 44: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/44.jpg)
Realistic Facial Manipulation:Earring Example
With Jason Pinto
![Page 45: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/45.jpg)
Telephone Detection
![Page 46: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/46.jpg)
Cart, pose 1
![Page 47: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/47.jpg)
Cart, pose 2
![Page 48: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/48.jpg)
Cart, pose 3
![Page 49: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/49.jpg)
Door Handle Detection
![Page 50: Object Detection Using Semi-Naïve Bayes to Model Sparse Structure](https://reader035.fdocuments.us/reader035/viewer/2022062410/56815a91550346895dc807be/html5/thumbnails/50.jpg)
Summary of Classifier Design
• Sparse structure of statistical dependency in many image classification problem
• Semi-naïve Bayes Model• Automatic learning structure of semi-naïve Bayes
classifier:– Generation of many candidate subsets– Competition among many log-likelihood functions to
find best combination
CMU on-line face detector:http://www.vasc.ri.cmu.edu/cgi-bin/demos/findface.cgi