Slide Pieter Jonker_Yolande Kolstee
-
Upload
culturainnovazione -
Category
Technology
-
view
730 -
download
7
description
Transcript of Slide Pieter Jonker_Yolande Kolstee
![Page 1: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/1.jpg)
01/07/2010 15:36 1
Pieter Jonker Professor of Vision based Robotics
Dept Bio-Mechanical Engineering, Faculty 3MEDelft University of Technology, Delft
Augmented Reality
Yolande KolsteeLector of Innovative Visualisation Techniques for Art & Design
Royal Academy of Art, The HagueThe [email protected]
![Page 2: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/2.jpg)
01/07/2010 15:36 2
Het volgen van een enkele hypothese (toegepast in AR)Augmented Reality vs Virtual Reality
![Page 3: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/3.jpg)
01/07/2010 15:36 3
Sorts of Augmented Reality
Screen BasedHead-Set Based
![Page 4: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/4.jpg)
01/07/2010 15:36 4
Sorts of Augmented Reality
Video see through AR2 cameras + 2 displays + video in video
Optical see through AR1 camera + 2 half translucent mirrors + 2 displays
![Page 5: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/5.jpg)
01/07/2010 15:36 5
History of our headsets
![Page 6: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/6.jpg)
01/07/2010 15:36 6
The Measurement Problem
How to hold the virtual objects in a perfect position and orientation in the physical world if I move my head?
By searching and tracking (camera + image processing) one or more known landmarks!
![Page 7: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/7.jpg)
01/07/2010 15:36 7
How to anchor onto a marker?
Grab an image Edge detectionSelect closed
contoursSimple corner
detection
Keep contours with 4 corners
Split contour into 4 lines
Sub-pixel fit of the 4 lines
Intersect 4 lines for corners
Calculate pose of marker
Determine the ID
Calculate pose of camera
Grab an image Edge detectionSelect closed
contoursSimple corner
detection
Keep contours with 4 corners
Split contour into 4 lines
Sub-pixel fit of the 4 lines
Intersect 4 lines for corners
Calculate pose of marker
Determine the ID
Calculate pose of camera
1
32
1024
2
64
2048
4
128
4096
8
256
8192
512
16384
161
32
1024
2
64
2048
4
128
4096
8
256
8192
512
16384
16
![Page 8: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/8.jpg)
01/07/2010 15:36 8
Why not tracking at natural features?
This costs too much time fort now, but soon…
![Page 9: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/9.jpg)
01/07/2010 15:36 9
Is only a camera & image processing enough?
No, we must know where the head will be in 30 msecsbecause rendering a virtual image costs time. We have to predict where the head will be bij measuring the motion of the head.
That is why we have to fuse the data of the camera and image processing with the date from a so called inertia tracker to track fast head movements.This is comparable with the yes and the inner ear / vestibular organ of a human.
An inertia tracker has 3 accelerometers to measure linear acceleration in 3 dimensions (x, y, z) and 3 gyroscopes to measure 3 rotational velocities (roll, pitch, yaw).This gives relative displacements at 100Hz that are combined with the 25Hz absolute position and orientation of the camera.
![Page 10: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/10.jpg)
01/07/2010 15:36 10
Fusion of camera and inertia tracker data
100 Hz
60 Hz
Camera measurement
Inertia measurement
Data Fusion
Rendering Imagetime
Image processingImage acquisition 15 Hz
![Page 11: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/11.jpg)
01/07/2010 15:36 11
So how accurate can one place virtual objects in the physical world?
Statical misalignment< 0.6 ° (< 0.03° = eye) Dynamical Misalignment< 2.3° (< 0.5° = eye)
With a 20 x 20 cm marker at 5 m and a 1MPix camera
Own and object motion will camouflage this
![Page 12: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/12.jpg)
01/07/2010 15:36 12
Future elektronics to loose the backpack and headset
![Page 13: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/13.jpg)
01/07/2010 15:36 13
Conclusions so far
• Augmented Reality is not Virtual Reality• Augmented Reality is screen or headset based• Headset based AR is video or optical see-through• The AR problem is dominated by the the allignment
problem of virtual and real world objects• This is solved by inertia and camera tracking• Due to the imperfect technology one can still see a
bit alignment errors, but far better than GPS! • Happily the human visual system compensates a bit• Future hardware will make it as small as iPods.• AR is a new and complex medium to discover!
![Page 14: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/14.jpg)
01/07/2010 15:36 14
The Vision of Yolande Kolstee
Visualization in Art and Design
![Page 15: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/15.jpg)
01/07/2010 15:36 15
Royal Academy of Art (KABK)
The Hague
The Netherlands
www.ARLAB.nl
Undergraduate / Bachelors
� Design: Textile, Fashion, Interior,
Furniture, Graphic, Interactive Media
� Fine Arts & ArtScience
� Photography
Graduate / Masters
� Type and Media
� ArtScience
� Interior Architecture
� Research in the Arts
� Industrial Design
![Page 16: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/16.jpg)
01/07/2010 15:36 16
Delft University of Technology
Delft
The Netherlands
Undergraduate, Graduate & Doctorate
Faculties:
� Aerospace Engineering
� Applied Sciences
� Architecture
� Civil Engineering and Geosciences
� Electrical Engineering, Mathematics & Computer Sc.
� Industrial Design Engineering
� Mechanical, Maritime & Materials Engineering
� Technology, Policy and Management
![Page 17: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/17.jpg)
01/07/2010 15:36 17
The fusion of perception of real and virtual imagesThe AR Lab is for all students, also outside students, who need help in the AR domain.
Institutes of cultural heritage come to us, like Museums:•Van Gogh•Boijmans van Beuningen•Stedelijk Museum•Kröller-Müller•Escher
Manifestations of:Multi-mediaModern-ArtICT
![Page 18: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/18.jpg)
01/07/2010 15:36 18
For example we used AR to show animated chairs and tables in the 2008 exhibition Salone del Mobile in Milan, Italy
![Page 19: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/19.jpg)
01/07/2010 15:36 19
AR was used to demonstrate the construction process
![Page 20: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/20.jpg)
01/07/2010 15:36 20
AR can be used to demonstrate the possibilities of the product
Melanie Luchtenveld
![Page 21: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/21.jpg)
01/07/2010 15:36 21
The virtual objects can be studied from all viewpoints
![Page 22: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/22.jpg)
01/07/2010 15:36 22
Transportation and duplication of furniture is quite easy
![Page 23: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/23.jpg)
01/07/2010 15:36 23
Connecting with the virtual world, experiments with interfaces
Audience likes AR & Interaction �Wii remote controller�RFID�Data glove: 5 flexor strips in fingers�Markers 'stand-alone' or incorporated�Fragments of pottery
![Page 24: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/24.jpg)
01/07/2010 15:36 24
We use RFID technology to let (tagged) physical objects influence virtual objects
![Page 25: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/25.jpg)
01/07/2010 15:36 25
We use a data-glove to manipulate virtual manipulated objects. Here to realize a virtual hand-puppet
![Page 26: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/26.jpg)
01/07/2010 15:36 26
One can interact with virtual objects in several ways
One can us real physical objects to change behavior of virtual puppets e.g.
Bath-sponge � towel and foam
Pot-holder � cook
Fay-wand � blow fish is crowned
Cigarettes � blow fish blows
![Page 27: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/27.jpg)
01/07/2010 15:36 27
Interaction can also be performed by moving the markers
![Page 28: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/28.jpg)
01/07/2010 15:36 28
Interaction with a WII mote
� Escher in the Palace:� Wii remote controller:
throw a visual to thewall
![Page 29: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/29.jpg)
01/07/2010 15:36 29
Headset Design
Students of the Royal Academy designed a new AR headset based on the VR/AR headset of the Dutch SME Cybermind who made a working version of the student’s mock-up
![Page 30: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/30.jpg)
01/07/2010 15:36 30
Design requirements:� More ease to wear� Don't isolate the user� Don't block the reality� Contact with the environment� Unique experience / attractive
![Page 31: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/31.jpg)
01/07/2010 15:36 31
Headset 'George'
� camera at the right side� inertia tracker left� open view below� wide screen� Side view
![Page 32: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/32.jpg)
01/07/2010 15:36 32
Various other designs have been made, but not yet prototyped
Design Niels Mulder
![Page 33: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/33.jpg)
01/07/2010 15:36 33
What to display?
Man made:There is a variety of 3D design programs and skilled and talented people who master this:Fine Art Artists, Animators, 3D Designers, Architects
Computer Generated:� 3D imagery (industrial) machines
especially from the bio-medical research� CT scan generated 3D images, yield an
inexhaustible amount of CT images
Data visualization:� There are various huge databases, e.g. geo
information, annual reports
![Page 34: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/34.jpg)
01/07/2010 15:36 34
Art: Drops of White
Marina de Haas
![Page 35: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/35.jpg)
01/07/2010 15:36 35
Geo Info
Data Visualization of Acid level in the first AR MAP on behalf of GSDI 11 World Conference, RotterdamJune 16-20 2009
Global Spatial Data Infrastructure association
![Page 36: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/36.jpg)
01/07/2010 15:36 36
AR Art: first experiment for general public
Marina de Haas
![Page 37: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/37.jpg)
01/07/2010 15:36 37
Cultural Heritage: Displaying computer generated data:Using bio-medical imaging equipment: CT scans for visualizing sgraffito-pottery:
Pre industrial original (1250-1450)Post industrial means Scanning- Digitalizing
3D printing � physical copy3D Archiving � virtual copy
![Page 38: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/38.jpg)
01/07/2010 15:36 38
![Page 39: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/39.jpg)
01/07/2010 15:36 39
Lessons learned
� Let the public view on a beamer what is seen by the HMD
� The “empty” third dimension is very useful for information display and interaction and detaches the application from gravity. One can have art or any info in the air ( e.g. drops of white)
� Motion of the virtual objects covers up for misalignment and jitter
� Design packages such as Cinema 4D make design with animated figures possible. For real 3D animated films with large plots, we are busy to incorporate game engines like Unity
![Page 40: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/40.jpg)
01/07/2010 15:36 40
� Headset based AR is at its best when a full immersive experience is required and people can walk around or within larger objects; buildings, DNA, airplanes to be refurbished
� For outdoor AR it is necessary that the ambient light intensity and the intensity of the LCD displays on the headset is in balance (automatic sunglasses)
� Augmented Reality can be fruitfully used to attract a broad public to displays of cultural heritage. Its narrative power is huge.
� The collaboration between researcher in the area of image processing with artists, designers and curators appeared to be very fruitful and has lead to many amazing productions and exhibitions
![Page 41: Slide Pieter Jonker_Yolande Kolstee](https://reader031.fdocuments.us/reader031/viewer/2022020110/548c10cfb47959fb0c8b628f/html5/thumbnails/41.jpg)
01/07/2010 15:36 41
“It begins with a blessingand it ends with a curse;making life easy,by making it worse.”
‘Why more realities?’
the old man in Milan said:
‘Live is already so complicated’.