Debajyoti MondalYang Wang Stephane Durocher Department of Computer Science University of Manitoba.
21
Robust Solvers for Square Jigsaw Puzzles Debajyoti Mondal Yang Wang Stephane Durocher Department of Computer Science University of Manitoba
-
Upload
sharon-bates -
Category
Documents
-
view
217 -
download
0
Transcript of Debajyoti MondalYang Wang Stephane Durocher Department of Computer Science University of Manitoba.
Robust Solvers for Square Jigsaw Puzzles
Debajyoti MondalYang Wang Stephane Durocher
Department of Computer ScienceUniversity of Manitoba
CRV 2013 231/05/2013
What are Jigsaw Puzzles?
CRV 2013 331/05/2013
Square Jigsaw Puzzles
24×18 = 432 puzzle pieces
CRV 2013 431/05/2013
State-of-Art Solvers
Pomeranz, Shemesh and Ben-Shahar
CVPR 2011
Cho, Avidan and Freeman
CVPR 2010
CVPR 2012
Andrew Gallagher
Solved puzzles with 432 pieces
Average 10% accuracy on432 piece puzzles
Solved puzzles with 3300 pieces
Average 94% accuracy on432 piece puzzles
Solved puzzles with 9600 pieces
Average 95% accuracy on432 piece puzzles
http://www.cs.bgu.ac.il/faculty/person/dolevp.html http://www.cs.bgu.ac.il/faculty/person/shemeshm.html http://www.cs.bgu.ac.il/~ben-shahar/
http://www.eng.tau.ac.il/~avidan/ http://people.csail.mit.edu/taegsang/ http://people.csail.mit.edu/billf/
http://chenlab.ece.cornell.edu/people/Andy/
CRV 2013 531/05/2013
Why Solving Jigsaw Puzzles ?
Restore Torn Apart Documents
http://www.bouldercitysocial.com/wp-content/uploads/2011/04/paperShredding.jpg
Fossil Reconstructionhttp://www.aim.uzh.ch/morpho/wiki//CAP/3-2
Ancient art/document reassemblyhttp://www.edgarlowen.com/n1/b7300.jpg
CRV 2013 631/05/2013
Our Robust Jigsaw Solver (Noise and Missing Boundary)
CRV 2013 731/05/2013
Our Robust Jigsaw Solver (Noise and Missing Boundary)
CRV 2013 831/05/2013
How to Solve a Puzzle?
XiXj
XiXk
XiXj
XiXk
XiXj
XiXk
XiXj
XiXk
XiXj
XiXk
XiXj
XiXk
CRV 2013 931/05/2013
Successful Strategies
Pomeranz et. al. [CVPR 2011]Sum of Squared Distance (SSD)
Gallagher [CVPR 2012]Mahalanobis Gradient Compatibility (MGC)
SSD ( xi , xj ) = DLR ( xi , xj ) MGC ( xi , xj ) = f (μi , Gij)
CRV 2013 1031/05/2013
Our Approach: M+S
(M+S) Compatibility Score = MGC( xi , xj ) SSD( xi , xj )1/q.
MGC
SSD
M+S,4
M+S,5
M+S,6
M,S,7MGC
SSD
M+S
20 images, each with 432 Puzzle Pieces of size 28×28×3
CRV 2013 1131/05/2013
Further Refinements
MGCScoring matrix
(M+S) Compatibility Score = MGC( xi , xj ) SSD( xi , xj ) 1/q.
5 3 9 1 6 7 2 4 8
| MGC(3,1) - MGC(3,2) | < σ
Row 3
CRV 2013 1231/05/2013
How to Refine this further?
MGCScoring matrix
(M+S) Compatibility Score = MGC( xi , xj ) SSD( xi , xj ) 1/q.
5 3 9 1 6 7 2 4 8
| MGC(3,1) - MGC(3,2) | < σ
Row 3
Greedy choice!No global Agreement!
CRV 2013 1331/05/2013
Selectively Weighted MGC (wMGC)
MGCScoring matrix
3
2 2
3
CRV 2013 1431/05/2013
Selectively Weighted MGC (wMGC)
MGCScoring matrix
A bijection with optimum weight
CRV 2013 1531/05/2013
Selectively Weighted MGC (wMGC)
5
2
MGCScoring matrix
5 3 9 1 6 7 2 4 8
Row 2
wMGC (xi , xj) =
Column 4
(M+S) Score, if ‘Conflict’
MGC Score, otherwise.
CRV 2013 1631/05/2013
Selectively Weighted MGC (wMGC)
5
2
MGCScoring matrix
5 3 9 1 6 7 2 4 8
Row 2
wMGC (xi , xj) =
Column 4
(M+S) Score, if ‘Conflict’
MGC Score, otherwise.
CRV 2013 1731/05/2013
Experimental ResultsMGC
SSD
M+S
wMGC,4
(M+S) Compatibility Score = MGC( xi , xj ) SSD( xi , xj )1/q.
wMGC (xi , xj)
=(M+S) Score, if ‘Conflict’
MGC Score, otherwise.
20 images, each with 432 Puzzle Pieces of size 28×28×3
MGC
SSD
M+S
wMGC
CRV 2013 1831/05/2013
Gallagher’s Reassembly [CVPR 2012]
Scoring Matrix
Construct Spanning Tree Trimming Filling
CRV 2013 1931/05/2013
Results
Perfect Noisy Cropped 0
50
100
150
200
250
300
350
Forest
SSDMGCOurs
Perfect Noisy Cropped 0
50
100
150
200
250
300
350
City
SSDMGCOurs
MIT scene database, 328 images of forest, 308 images of city81 pieces per puzzle, each piece of size 28×28×3
CRV 2013 2031/05/2013
Future Research
Image Filtering?
How much does it help if we know the image category?
Robust functions for compatibility scoring.
Thank You
