Assaf Shacham , Member, IEEE, Keren Bergman, Senior Member, IEEE, and
REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE,...
-
Upload
ralf-stanley -
Category
Documents
-
view
215 -
download
0
Transcript of REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE,...
![Page 1: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/1.jpg)
REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES
Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE,
Alessandro Mulloni, Student Member, IEEE, Tom Drummond, and
Dieter Schmalstieg, Member, IEEE Computer Society
![Page 2: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/2.jpg)
Outlines
Introduction Nature Feature Tracking System FAST detector Adopted Trackers Performance & Analysis Conclusion
![Page 3: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/3.jpg)
Introduction
Reason: Limited performance on phones
(limited computational resources)
Leads to: Natural feature tracking not feasible
(Needs long waiting time for large computation)
Goal: Speed improvement
(enough speed for AR processing & displaying)
![Page 4: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/4.jpg)
Natural Feature Tracking System
1. SIFT2. Ferns (subsets of features)
Both are accurate but not fast enough for phones
Need faster approach New approaches are called:
1. PhonySIFT2. PhonyFerns
![Page 5: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/5.jpg)
FAST detector
Ref from:<Machine learning for high-speed corner detection>By Edward Rosten and Tom Drummond, University of Cambridge
A corner detector many times faster than DoGbut not very robust to the presence of noise,
Can be trained to be much faster
![Page 6: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/6.jpg)
Adopted Trackers
PhonySIFT (Refined from SIFT) PhonyFerns (Refined from Ferns) Patch Tracker (developed by authors) Combined Tracking
PhonySIFT + PatchTracker PhonyFerns + PatchTracker
![Page 7: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/7.jpg)
Ferns
Ref from:<Fast Keypoint Recognition in Ten Lines of
Code>by Mustafa Özuysal Pascal Fua Vincent Lepetit, Computer Vision
Laboratory ,Switzerland
An keypoint tracker using statistical algorithmand can be trained to get higher matching rate As good as SIFT, or even better performance
![Page 8: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/8.jpg)
SIFT to PhonySIFT
Changes: Uses FAST corner detector to all scaled
images to detect feature points instead of scale-crossing DoG
Only 3x3 subregions, 4bins each , creates 36-d vector
![Page 9: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/9.jpg)
Ferns to PhonyFerns
Changes: Uses FAST detector to increase detection
speed Reduces each ferns size Uses 8-bit size to store probability instead
of using 4 bytes float point value modifying the training scheme to use all
FAST responses within the 8-neighborhood
![Page 10: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/10.jpg)
PatchTracker
Both the scene and the camera pose change only slightly between two successive frames
New feature positions can be successfully predicted by old one with defined range search.
Speed is less dependency with the camera resolution
![Page 11: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/11.jpg)
Combined Tracking
![Page 12: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/12.jpg)
Performance & Analysis
Platform:Asus P552W (Cellphone) 624Mhz CPU 240x320 screen resolution No float point unit No 3D acceleration
Platform:Dell Notebook (PC) 2.5Ghz , limited to use single core With float point support
![Page 13: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/13.jpg)
Performance & Analysis
![Page 14: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/14.jpg)
Performance & Analysis
Robustness results over different tracking targets
![Page 15: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/15.jpg)
Performance & Analysis
The following graph shows the statistics of above situations
![Page 16: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/16.jpg)
Performance & Analysis
![Page 17: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/17.jpg)
Conclusion
Successfully worked with tracking system on phones
In the future, faster CPUs could come, and the choice of next generation of tracking technique may be different, and may enable more expensive per-pixel processing
![Page 18: REAL-TIME DETECTION AND TRACKING FOR AUGMENTED REALITY ON MOBILE PHONES Daniel Wagner, Member, IEEE, Gerhard Reitmayr, Member, IEEE, Alessandro Mulloni,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649d965503460f94a7f5e7/html5/thumbnails/18.jpg)
The End
Thank you for listening