Shape Recognition for Multi-Touch...
Transcript of Shape Recognition for Multi-Touch...
![Page 1: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/1.jpg)
Shape Recognitionfor Multi-Touch Table
Yehoraz Kasher Project Competition10/06/2012
Students: Itay Hubara
Amit Nishry
Supervisor: Yair Moshe
![Page 2: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/2.jpg)
Introduction
System
Algorithm
Implementation
Demo Game
Conclusion
Introduction
System
Algorithm
Implementation
Demo Game
Conclusion
Outline
2
![Page 3: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/3.jpg)
Background
• Multi-touch is starting to appear in every day
computers
• New User Interfaces (NUI) are extensively
pursued
3
![Page 4: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/4.jpg)
Motivation
• Existing multi-touch surfaces are expensive
• Current technology is unable to recognize
shapes
• Shape recognition enhances user interface
4
![Page 5: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/5.jpg)
Goals
• Design a shape recognition algorithm
• Track and recognize combined shapes
• Integrate into open-source environment
• Real-Time performance
5
![Page 6: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/6.jpg)
Challenges
• Noisy input image
• Real-time constrains
• Implementing into open-source environment
• Creating a complete system solution
• Shape combinations and movement
6
![Page 7: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/7.jpg)
• Real-world view• IR input
Challenges (cont.)
7
![Page 8: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/8.jpg)
Outline
Introduction
System
Algorithm
Implementation
Demo Game
Conclusion8
![Page 9: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/9.jpg)
System
• Hardwareo Multi-Touch Table
o FTIR Technology
• Softwareo Community Core Vision
o Client / Server Model
• Algorithmo Image Processing
o Shape Detection
o Tracking
• Applicationo Game
Hardware
9
Software
Algorithm
Application
FTIR - Frustrated Total Internal Reflection
![Page 10: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/10.jpg)
Frustrated Total Internal Reflection
10
![Page 11: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/11.jpg)
Multi-Touch Table
11
![Page 12: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/12.jpg)
Outline
Introduction
System
Algorithm
Implementation
Demo Game
Conclusion12
![Page 13: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/13.jpg)
Shape Recognition
Pre-Processing
VideoFeature
ExtractionClassification Tracking
LearningShape
ModelApplication
13
Learning Phase
On-line Phase
![Page 14: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/14.jpg)
• Global thresholding leads to
inconsistent performance
• Each frame has a different
dynamic range
• Solution: Adaptive thresholding
Pre-Processing
Cropping
Stitching
Backgroundremoval
Noise filtering
Thresholding
Segmentation
14
![Page 15: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/15.jpg)
Pre-processing (cont.)
IR input Background removed
15
![Page 16: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/16.jpg)
Pre-processing (cont.)
Global threshold Adaptive threshold
16
![Page 17: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/17.jpg)
Shape Recognition
Pre-Processing
VideoFeature
ExtractionClassification
LearningShape
Model
17
![Page 18: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/18.jpg)
Contour RegionStructural Global
Feature Extraction
Shape
features
[Zhang & Lu, 2004] 18
Shape
Region
Global
moments area axis …
![Page 19: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/19.jpg)
• Area
• Major & minor axis
• Orientation
• Eccentricity
• 6 Hu moments
• 7 complex moments
Candidate Features
19
![Page 20: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/20.jpg)
Image Moments
• Complex moments:
• Moments are centralized for translation
invariance
( , ) - image indicator function
,p q
pq
x y
c x iy x iy x y dxdy
20
![Page 21: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/21.jpg)
• In polar coordinates:
• Rotation by
• In general
• Which leads to
2
1
0 0
,i p qp q
pqc r e r drd
Rotation Invariance
i p q
pq pqc e c
0
0n
j j jjk p q
1
j
j j
Nk
p q
j
I c
21
![Page 22: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/22.jpg)
• Seven rotation and translation invariant
moments
• Useful for visual pattern recognition
• Rotation invariance via complex moments
1 11
2 20 02
c
c c
[Hu, 1962][Flusser, Suk & Zitová, 2009]
Hu Moments
22
![Page 23: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/23.jpg)
Shape Recognition
Pre-Processing
VideoFeature
ExtractionClassification
LearningShape
Model
23
![Page 24: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/24.jpg)
Learning
Feature
selection
Shape
model
• Many features high computation
• Dimensionality reduction required
• Extract features at various locations
• Approx. 50 samples per shape
• Independent Gaussian model
24
![Page 25: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/25.jpg)
Feature Selection• Exhaustive algorithm based on cross-validation
25
Features
First 10% of samples
Second 10% of samples
Tenth 10% of samples
Class
Third 10% of samples
Error 1
Error 2
Learning set
Test set
Test set
Learning set
Error 3
Error 10
Result: 1 2 10, , ,totale mean e e e
![Page 26: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/26.jpg)
Feature Selection (cont.)
26
• Initialization o Set n=1 feature(s) out of a total of N features
• Stepo Calculate for all possible combinations
o Set n=n+1
• Stopping condition
o When
n
totaleN
n
1n n
total totale e
![Page 27: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/27.jpg)
Feature Selection (cont.)
• Chosen features:o Area
o Major & minor axis
o Eccentricity
o 4 Hu moments:
• Empirical selection
• Similar to other shape recognition works
1 4 6 7, , ,
27[Hu, 1962][Sarfraz, 1993]
![Page 28: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/28.jpg)
Shape Recognition
Pre-Processing
VideoFeature
ExtractionClassification
LearningShape
Model
28
![Page 29: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/29.jpg)
Classification
• Independent Gaussian distributions
• Naïve Bayes Classifier
•
29
2
1
1
1
1
( | ) ,
( | ,..., ) ( ( | )
ˆ classify( , , ) argmax ( | ,...,
)
)j
j i
j
m
j j
i i i
m i
m
j
j m
i
s
s f
s
s
p f N
p f f p p f
s f f
s
p
s
f fs
For each shape and feature
assume :
![Page 30: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/30.jpg)
Naïve Bayes Classifier
• Advantages:o Generic
o Fast
o Allows thresholding
o Good performance
• Disadvantages:o Long offline learning process
30
![Page 31: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/31.jpg)
Shape Recognition
Pre-Processing
VideoFeature
ExtractionClassification Tracking
Application
31
![Page 32: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/32.jpg)
Tracking
NN tracking
Combined
shapes
Internal
movement
False
positive
detection
• Nearest neighbors tracking
• Simplifies data handling
• Prevents false detections
• Allows complex situationso Shape combinations
o Internal movement in a combination
32
![Page 33: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/33.jpg)
Complex Situations
• Shape combinations can appear as one
(legitimate) shape
• Possible classification mistakes:
33
![Page 34: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/34.jpg)
Internal Shape Movement
• Complex shape modification
• Combined shape structure is maintained
34
![Page 35: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/35.jpg)
Tracking
Segmentation
Classification
False Positive
Detection
Combined Shapes
Detection
Internal Movement Detection
Status Update
Blobs
Shapes
CombinedShapes
Blobs
Shapes
CombinedShapes
Current framePrevious frame
35
Video
![Page 36: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/36.jpg)
36
Complex Situations
![Page 37: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/37.jpg)
Shape Recognition
Pre-Processing
VideoFeature
ExtractionClassification Tracking
Application
37
![Page 38: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/38.jpg)
Application
Client /
Server
Data
transfer
protocol
• Many possible applications
• Client / Server model established
• Community Core Vision
• Tangible User Interface Objects (TUIO) protocol for data transfer
38
![Page 39: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/39.jpg)
Community Core Vision
• Open-source / cross-platform solution for
computer vision and machine sensing
• Supports multi-touch input
• Supports multi-camera feed
• Commonly used by developers
39
![Page 40: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/40.jpg)
TUIO Protocol
• Open framework communication protocol and API for tangible multi-touch surfaces
• Supports various multi-touch informationo Location
o Orientation
o Acceleration
o …
• Commonly used with Community Core Vision
40
![Page 41: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/41.jpg)
Outline
Introduction
System
Algorithm
Implementation
Demo Game
Conclusion41
![Page 42: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/42.jpg)
Implementation
42
• Simulations (Matlab)
• Implemented into CCV (C++)o Adaptive thresholding
o Learning process
o Classification
o Shape tracking
o Import / export classifiers
o Additions to GUI
• Implemented into TUIO (C++ & C#)o Sending shape information
o Distinguishing between touch and shape
![Page 43: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/43.jpg)
Graphical User Interface
43
![Page 44: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/44.jpg)
Learning
44
![Page 45: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/45.jpg)
Outline
Introduction
System
Algorithm
Implementation
Demo Game
Conclusion45
![Page 46: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/46.jpg)
Tangram
• Chinese dissection puzzle with seven shapes
• The objective is to fill a given silhouette
• Demonstrates the algorithm capabilities
46
![Page 47: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/47.jpg)
SIPL Tangram
47
![Page 48: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/48.jpg)
Outline
Introduction
System
Algorithm
Implementation
Demo Game
Conclusion48
![Page 49: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/49.jpg)
Demo
49
![Page 50: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/50.jpg)
Performance Analysis
• Resourceso Intel i7 processor, 8 GB of memory
o Three IR cameras, 60 fps
• Real-Time performance
• Computing time < 2 miliseconds per frame
50
![Page 51: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/51.jpg)
Summary
Multi-Touch table
Feature Extraction
Shape Classification
Tracking
Complex situations
CCV & TUIO
Tangram51
Shapes
![Page 52: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/52.jpg)
Future Work
• Extending shape recognitiono Additional shapes
o Shape color
• Improving camera registration
• Cross-Platform implementations
• Finalize implementation of the demo game
52
![Page 53: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/53.jpg)
Thanks
53
AviYairKobi
![Page 54: Shape Recognition for Multi-Touch Tablesipl.eelabs.technion.ac.il/.../sites/6/2016/12/PresentShapeReco2012f.… · Introduction System Algorithm Implementation Demo Game Conclusion](https://reader033.fdocuments.us/reader033/viewer/2022042807/5f77f47a3e63f63bdf0a66ad/html5/thumbnails/54.jpg)
References
• D. Zhang and G. Lu, "Review of shape representation and
description techniques“, Pattern Recognition, vol. 37, pp. 1–19,
2004.
• M. K. Hu, "Visual Pattern Recognition by Moment Invariants“, IRE Trans. Information Theory, vol. 8, pp. 179–187, 1962.
• J. Flusser, T. Suk and B. Zitová, “Moments and Moment
Invariants”, John Wiley & Sons Ltd., 2009.
• C. M. Bishop, “Pattern Recognition and Machine Learning”,
Springer, 2007.
• "CCV, Community Core Vision“, [Online],
http://ccv.nuigroup.com/.
• "TUIO.org“, [Online], http://www.tuio.org/
54