Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian...
Transcript of Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian...
![Page 1: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/1.jpg)
Stereoscopic 3D Videos and Panoramas
Christian Richardt
![Page 2: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/2.jpg)
1. Capturing and displaying stereo 3D videos
2. Viewing comfort considerations
3. Editing stereo 3D videos (research papers)
4. Creating stereo 3D panoramas
Stereoscopic 3D videos and panoramas
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 2
![Page 3: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/3.jpg)
Parallel Converged (‘toed-in’)
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 3
Stereo camera rigs
![Page 4: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/4.jpg)
Parallel Converged (‘toed-in’)
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 4
Stereo camera rigs
©2012
Oliv
er
Kre
ylo
s
©2012
Oliv
er
Kre
ylo
s
![Page 5: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/5.jpg)
Computational stereo 3D camera system
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 5
Computational stereo camera system with programmable control loop
S. Heinzle, P. Greisen, D. Gallup, C. Chen, D. Saner, A. Smolic, A. Burg, W. Matusik & M. Gross
ACM Transactions on Graphics (SIGGRAPH), 2011, 30(4), 94:1–10
©2011
Hein
zle
et
al./
AC
M
![Page 6: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/6.jpg)
Polarised projection Wavelength multiplexing
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 6
Commercial stereo 3D projection
©R
ao
ul N
K, 3d
natu
reg
uy/
Wik
imed
ia C
om
mo
ns/
CC
-BY-S
A-3
.0
e.g. Dolby 3De.g. RealD 3D, MasterImage 3D
©2011
Sco
tt W
ilkin
son
/So
un
d a
nd
Vis
ion
![Page 7: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/7.jpg)
Active shutter glasses Autostereoscopy
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 7
Medium-scale stereo 3D displays
e.g. NVIDIA 3D Vision, 3D TVs
©C
mg
lee/W
ikim
ed
ia C
om
mo
ns/
CC
-BY-S
A-3
.0
©2011
MTB
S3D
/NV
IDIA
![Page 8: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/8.jpg)
Head-mounted displays (HMDs) Anaglyph stereo
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 8
Other stereo 3D displays
©2016
HTC
Co
rpo
ratio
n
e.g. HTC Vive, Oculus Rift, Google Cardboard
©2016
Tari
ng
a
e.g. red cyan glasses, ColorCode 3-D, Inficolor 3D
![Page 9: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/9.jpg)
Cinema 3D
Cinema 3D: large scale automultiscopic display
N. Efrat, P. Didyk, M. Foshey, W. Matusik & A. Levin
ACM Transactions on Graphics (SIGGRAPH), 2016, 35, 59:1–12
©2016
Efr
at
et
al.
Narrow angular range that spans a single seat
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 9
![Page 10: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/10.jpg)
Cinema 3D
Cinema 3D: large scale automultiscopic display
N. Efrat, P. Didyk, M. Foshey, W. Matusik & A. Levin
ACM Transactions on Graphics (SIGGRAPH), 2016, 35, 59:1–12
©2016
Efr
at
et
al.
Narrow angular range that spans a single seat
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 9
![Page 11: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/11.jpg)
ScreenX
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 10
ScreenX: public immersive theatres with uniform movie viewing experiences
J. Lee, S. Lee, Y. Kim & J. Noh
IEEE Transactions on Visualization and Computer Graphics, 2017, 23(2), 1124–1138
©2017
Lee e
t al./
KA
IST V
isual M
ed
ia L
ab
![Page 12: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/12.jpg)
Pictorial depth cues:
– size, occlusion, perspective,
aerial perspective, texture
gradient, motion parallax,
depth of field, …
Ocular depth cues:
– Accommodation
– Vergence
Binocular disparity
Depth cues
Vergence
Accommodation
Slid
e c
ou
rtesy
of
Petr
Kelln
ho
fer
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 11
![Page 13: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/13.jpg)
How does disparity work?
Screen
Object in left eye
Object in right eyeObject perceived in 3D
Pixel disparity
Slid
e c
ou
rtesy
of
Petr
Kelln
ho
fer
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 12
![Page 14: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/14.jpg)
How does disparity work?
Vergence
Depth
Accommodation
(focal plane)
Slid
e c
ou
rtesy
of
Petr
Kelln
ho
fer
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 12
Vie
win
g d
isco
mfo
rt
![Page 15: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/15.jpg)
How does disparity work?
Comfort zone
Vergence
Depth
Accommodation
(focal plane)
Slid
e c
ou
rtesy
of
Petr
Kelln
ho
fer
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 12
![Page 16: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/16.jpg)
Comfort zone
Preventing viewing discomfort
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 13
Viewing discomfort
Slid
e c
ou
rtesy
of
Petr
Kelln
ho
fer
![Page 17: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/17.jpg)
Comfort zone
Preventing viewing discomfort
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 13
Viewing discomfort Viewing comfortScene manipulation
Slid
e c
ou
rtesy
of
Petr
Kelln
ho
fer
![Page 18: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/18.jpg)
Disparity manipulation
strong
weak
Perceived
distortions
Slid
e c
ou
rtesy
of
Petr
Kelln
ho
fer
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 14
OSCAM – Optimized stereoscopic
camera control for interactive 3D
Oskam et al., SIGGRAPH Asia 2011
Input disparityOu
tpu
t d
isp
ari
ty
Nonlinear disparity mapping
for stereoscopic 3D
Lang et al., SIGGRAPH 2010
Perceived
distortions
A perceptual model for disparity
Didyk et al., SIGGRAPH 2011
![Page 19: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/19.jpg)
Production rules for stereo acquisition
Zilly et al., Proc. IEEE 2011
Predicting stereoscopic viewing comfort using a coherence-based computational model
Richardt et al., CAe 2011
A luminance-contrast-aware disparity model and applications
Didyk et al., SIGGRAPH Asia 2012
A metric of visual comfort for stereoscopic motion
Du et al., SIGGRAPH Asia 2013
Modeling and optimizing eye vergence response to stereoscopic cuts
Templin et al., SIGGRAPH 2014
What makes 2D-to-3D stereo conversion perceptually plausible?
Kellnhofer et al., SAP 2015
GazeStereo3D: seamless disparity manipulations
Kellnhofer et al., SIGGRAPH 2016
Causes of discomfort in stereoscopic content: a review
Terzic & Hansard, arXiv:1703.04574
Additional reading on viewing comfort
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 15
![Page 20: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/20.jpg)
2D-to-3D conversion
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 16
Perceptual real-time 2D-to-3D conversion using cue fusion
Leimkühler et al., IEEE TVCG 2017
StereoBrush: interactive 2D to 3D conversion
using discontinuous warps
Wang et al., SBIM 2011
©2011
Wan
g e
t al./
AC
M
©2016
Leim
kü
hle
ret
al.
![Page 21: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/21.jpg)
Deep3D: fully automatic 2D-to-3D video conversion
with deep convolutional neural networks
Xie et al., ECCV 2016
Hallucinating stereoscopy from a single image
Zeng et al., CGF (Eurographics) 2015
Video stereolization: combining motion analysis with user interaction
Liao et al., IEEE TVCG 2012
Depth Director: a system for adding depth to movies
Ward et al., IEEE CG&A 2011
Stereoscopic video synthesis from a monocular video
Zhang et al., IEEE TVCG 2007
Additional reading on 2D-to-3D conversion
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 17
![Page 22: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/22.jpg)
Video de-anaglyph
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 18
©2014
Ro
o&
Ric
hard
t; v
ideo
: Eri
c D
ere
n/D
zig
nlig
ht
Stu
dio
s
Temporally Coherent Video De-Anaglyph
Roo & Richardt, SIGGRAPH Talks 2014
![Page 23: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/23.jpg)
Stereo 3D video deblurring
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 19
Simultaneous stereo video deblurring and scene flow estimation ↑
Pan et al., CVPR 2017
Stereo Video Deblurring
Sellent et al., ECCV 2016
©2017
Pan
et
al.
©2016
Selle
nt
et
al.
Blurry input image Deblurred image
![Page 24: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/24.jpg)
Stereo 3D video stabilisation
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 20
Joint Subspace Stabilization for Stereoscopic Video
Liu et al., ICCV 2013
Input video frames (anaglyph)
Stabilised video frames (anaglyph)
©2013
Liu
et
al./
IEEE
![Page 25: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/25.jpg)
Correspondence finding
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 21
← Consistent binocular depth and scene flow with
chained temporal profiles
Hung et al., IJCV 2013
Real-time spatiotemporal stereo matching using
the dual-cross-bilateral grid
Richardt et al., ECCV 2010
©2013
Hu
ng
et
al.
![Page 26: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/26.jpg)
Retargeting:
– Object-coherence warping for stereoscopic image retargeting
Lin et al., IEEE TCSVT 2014
– Stereo seam carving a geometrically consistent approach
Basha et al., IEEE TPAMI 2013
Compositing:
– StereoPasting: interactive composition in stereoscopic images
Tong et al., IEEE TVCG 2013
– Stereoscopic 3D copy & paste
Lo et al., SIGGRAPH Asia 2010
Warping:
– Perspective-aware warping for seamless stereoscopic image cloning
Luo et al., SIGGRAPH Asia 2012
– Enabling warping on stereoscopic images
Niu et al., SIGGRAPH Asia 2012
Image-only techniques
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 22
![Page 27: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/27.jpg)
Capturing 3D panoramas
232017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
![Page 28: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/28.jpg)
Capturing 3D panoramas
232017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
![Page 29: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/29.jpg)
Capturing 3D panoramas
242017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
![Page 30: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/30.jpg)
Capturing 3D panoramas
252017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
![Page 31: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/31.jpg)
Capturing 3D panoramas
262017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
![Page 32: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/32.jpg)
Capturing 3D panoramas
272017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
Omnistereo: Panoramic Stereo Imaging
Peleg et al., IEEE TPAMI 2001
![Page 33: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/33.jpg)
Capturing 3D panoramas
272017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
Omnistereo: Panoramic Stereo Imaging
Peleg et al., IEEE TPAMI 2001
![Page 34: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/34.jpg)
Capturing 3D panoramas
292017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
![Page 35: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/35.jpg)
Capturing 3D panoramas
302017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
![Page 36: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/36.jpg)
Input video:
Capturing 3D panoramas
31
©2013
Ric
hard
t et
al.
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
![Page 37: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/37.jpg)
Capturing 3D panoramas
322017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
©2013
Ric
hard
t et
al.
Megastereo: Constructing High-Resolution Stereo Panoramas
Richardt et al., CVPR 2013
![Page 38: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/38.jpg)
Image alignment
33
image-based alignment
SfM-based alignment
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
©2013
Ric
hard
t et
al.
![Page 39: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/39.jpg)
Strip blending artefacts
34
far: duplication
near: truncation
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
© d
ata
set
‘refa
im’ b
y R
av-
Ach
aet
al.,
IJC
V 2
008
![Page 40: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/40.jpg)
Duplication + truncation
35
near objectsfar objects
panoramic
imaging surface
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
![Page 41: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/41.jpg)
Flow-based ray interpolation
36
near objectsfar objects
panoramic
imaging surface
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
![Page 42: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/42.jpg)
Flow-based ray interpolation
36
near objectsfar objects
panoramic
imaging surface
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
![Page 43: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/43.jpg)
Strip blending artefacts
37
far: duplication
near: truncation
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
© d
ata
set
‘refa
im’ b
y R
av-
Ach
aet
al.,
IJC
V 2
008
![Page 44: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/44.jpg)
Flow-based blending
38
far: stretching
near: squeezing
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
©2013
Ric
hard
t et
al.;
data
set
‘refa
im’ b
y R
av-
Ach
aet
al.,
IJC
V 2
008
![Page 45: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/45.jpg)
No blending Flow-based blending
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 39
Blending comparison
©2013
Ric
hard
t et
al.
![Page 46: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/46.jpg)
Stereo 3D panorama
40
©2013
Ric
hard
tet
al.
Megastereo: Constructing High-Resolution Stereo Panoramas
Richardt et al., CVPR 2013
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
![Page 47: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/47.jpg)
Stereo 3D panorama
40
©2013
Ric
hard
tet
al.
Megastereo: Constructing High-Resolution Stereo Panoramas
Richardt et al., CVPR 2013
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas
![Page 48: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/48.jpg)
360 º
zoom
©2013
Ric
hard
tet
al.
![Page 49: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/49.jpg)
360 º
100% zoom
140 MP stereo panorama
©2013
Ric
hard
tet
al.
![Page 50: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/50.jpg)
stereo video = videos for left + right eyes
– good: binocular disparity provides depth perception
– bad: does not react to head motion
accommodation–vergence conflict:
– excessive disparity causes viewing discomfort
editing stereo video needs to preserve consistency of views
– many tasks still difficult to achieve, even with research software
high-quality stereo panoramas created with Megastereo
– SfM-based alignment + flow-based blending
Quick recap
2017-08-03 Christian Richardt – Stereoscopic 3D Videos and Panoramas 43
![Page 51: Stereoscopic 3D Videos and Panoramas - Christian Richardt · 2017-08-08 · 2017-08-03 Christian Richardt –Stereoscopic 3D Videos and Panoramas 3 Stereo camera rigs. Parallel Converged](https://reader034.fdocuments.us/reader034/viewer/2022042313/5edc94caad6a402d66674e5c/html5/thumbnails/51.jpg)
Stereoscopic 3D Videos and Panoramas
Christian Richardt