Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO...
Transcript of Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO...
![Page 1: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/1.jpg)
Marco Cavallo Riverwalk: Incorporating Historical Photographs in Public Outdoor Augmented Reality Experiences
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
![Page 2: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/2.jpg)
First of all…
… some discussions about terminology used could be done, but for now we will stick to the term «Augmented Reality»
(Milgram, 1994)
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
![Page 3: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/3.jpg)
Before…
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
![Page 4: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/4.jpg)
…after
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
![Page 5: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/5.jpg)
The Chicago 0,0 Project
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
![Page 6: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/6.jpg)
The Chicago 0,0 Project
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
• Ongoing project >> in collaboration with Geoffrey Alan Rhodes
• Chicago History Museum’s massive photo archive
![Page 7: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/7.jpg)
The limits of markerless AR
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
• Flat images • Highly textured • Feature distribution • Avoid organic shapes • Avoid repetitive patterns • Good local contrast
![Page 8: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/8.jpg)
Guess what?
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
• Repetitive patterns (windows!) • Sometimes uniform in color • Lighting and weather conditions • Flat…?
![Page 9: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/9.jpg)
Additional complaints
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
What if… • No features to be tracked? • No flat images to track? • We wanted to create more complex behaviors? • We wanted to improve the design process?
![Page 10: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/10.jpg)
Our solution: geolocating content
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
… we live in a 3D world after all :)
Towards an absolute camera pose
Both overlays and fiducials are geolocated!
![Page 11: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/11.jpg)
The dual camera approach
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
• ARCamera >> pose estimated through
pattern-based image tracking • SensorCamera
>> leveraging geomagnetic field, accelerometer and gyroscope
… an abstraction for Markerless & Location-based AR
![Page 12: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/12.jpg)
The ARCamera
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
• Temporary solution -> general purpose markerless AR and… oversampling :-/ • Future solution -> specific algorithm for architectural features
… approach by itself algorithm-independent
![Page 13: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/13.jpg)
The ARCamera
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
But… what’s the novelty?
Absolute positioning!!
![Page 14: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/14.jpg)
The SensorCamera
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
• Absolute position: (A-)GPS >> Corrected with step detectors, multi-sensors odometry, visual odometry
• Absolute orientation: Inertial Measurement Unit (IMU) >> Accelerometer, Gyroscope, Magnetic field
![Page 15: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/15.jpg)
Estimating the pose of the camera
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
4 cases to be handled: • Fiducial found • Fiducial lost • Multiple fiducials • No fiducials available
![Page 16: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/16.jpg)
General advantages
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
• Absolute Geolocation
>> Abstract location-based and marker-less AR as a single experience
>> Load and unload content based on user location
>> Display virtual content even if no tracking is available
>> Know virtual content close to user or to other virtual content >> Prune the dataset of pattern images based on proximity
>> Virtual environment where to edit and preview the application
>> Dynamic resource management
![Page 17: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/17.jpg)
General advantages
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
• Absolute orientation: >> Know how much user is moving away from a tracked object >> Display objects even if tracking has been lost >> Know how to orient user towards other virtual content >> Know how to filter virtual content if overlapping >> Signaling incoherent situations and false positives / negatives >> Intelligent camera smoothing (stabilization)
![Page 18: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/18.jpg)
Navigation & information browsing
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
• Color coding and showing available content
• Showing target where the user has to aim
• Indicating whete to move / rotate to next object
• Indicate when device needs calibration
• Allow transitions between adjacent / overlapping content
• Suggest the user where to position to see from a particular perspective
• Map + optional navigation
• Allow user to move / modify / correct by himself?
![Page 19: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/19.jpg)
Just to be sure…
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
• Weather conditions • Absence of light • Device limitations
… always keep a classic user interface to allow users visualizing the content
![Page 20: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/20.jpg)
The designer perspective
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
… ever wondered?
![Page 21: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/21.jpg)
The designer perspective
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
![Page 22: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/22.jpg)
Towards a first authoring tool
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
• Selecting view points • Placing overlays • Previewing them offline…
…simulating the user’s experience!!
![Page 23: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/23.jpg)
What’s next?
Ops… now we’ll go a bit off-topic :)
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
… or maybe not?
![Page 24: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/24.jpg)
Not that again!
Let’s broaden our term… let’s use <<Mixed Reality>>
(Milgram, 1994)
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
![Page 25: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/25.jpg)
Let’s think outside the box
What if we could… ● Create a better way to design and edit MR experiences
● Modify at runtime the MR experience
● Study how users behave while using our application
● Interact selectively with them
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
![Page 26: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/26.jpg)
Merging the two worlds
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
1:1 mapping to real-world locations
![Page 27: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/27.jpg)
An editor but also…
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Not only a nice way to edit virtual content
![Page 28: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/28.jpg)
A portal between realities
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Content modified real-time on clients
Users represented as avatars with 4+3 DOF
![Page 29: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/29.jpg)
Many new possibilities!!
POLITECNICO DI MILANO Marco Cavallo – Research Presentation
• Real-time content customization in order to solve problems or satisfy user needs
• Possibily to analyze the behavior of users and to store significant data
• Possibility to interact with users in MR >> Create private audio and video channels >> See from their real-world perspective >> Debug the whole MR solution >> Add ourselves to their MR experience
![Page 30: Marco Cavallo - evlGeneral advantages Marco Cavallo – Research Presentation POLITECNICO DI MILANO • Absolute orientation: >> Know how much user is moving away from a tracked object](https://reader033.fdocuments.us/reader033/viewer/2022060309/5f0a75337e708231d42bbb29/html5/thumbnails/30.jpg)
Thanks for watching ;)
POLITECNICO DI MILANO Marco Cavallo – Research Presentation