ORT (other random topics) CSE P 576 Larry Zitnick ([email protected])[email protected].
Using Photographs to Enhance Videos of a Static Scene Pravin Bhat 1, C. Lawrence Zitnick 2, Noah...
-
date post
21-Dec-2015 -
Category
Documents
-
view
213 -
download
0
Transcript of Using Photographs to Enhance Videos of a Static Scene Pravin Bhat 1, C. Lawrence Zitnick 2, Noah...
Using Photographs to Enhance Videos of a Static
ScenePravin Bhat1, C. Lawrence Zitnick2, Noah Snavely1, Aseem
Agarwala3, Maneesh Agrawala4, Michael Cohen1,2, Brian Curless1, Sing Bing Kang2
EGSR 2007
University of Washington1, Microsoft Research Redmond2 University of California3, Adobe Systems4
Motivation
• Low quality video• Reconstructed video
– Reconstructed from photos– Good spatial reconstruction– Bad temporal reconstruction
Input Video
Reconstructed Video
Motivation
• Spacetime Fusion result– Spatial properties of reconstruction– Temporal properties of input video
Input Video
Spacetime Fusion Result
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video
Spacetime Fusion
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video– Key Idea
• Temporal gradients defined betweenmotion compensated temporal neighbors
Spacetime Fusion
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video– Key Idea
• Temporal gradients defined betweenmotion compensated temporal neighbors
Video frame: t Video frame: t - 1
Spacetime Fusion
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video– Key Idea
• Temporal gradients defined betweenmotion compensated temporal neighbors
Video frame: t Video frame: t - 1
Gt
Gt(x, y, t) = V(x, y, t) - V(x, y, t - 1)
Spacetime Fusion
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video– Key Idea
• Temporal gradients defined betweenmotion compensated temporal neighbors
Video frame: t Video frame: t - 1
Gt
Gt(x, y, t) = V(x, y, t) - V(x - u, y - v, t - 1)
Spacetime Fusion
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video– Key Idea
• Temporal gradients defined betweenmotion compensated temporal neighbors
• Increases compatibility betweentemporal gradients and spatial gradients
Spacetime Fusion
• Define a 3D gradient field – Spatial gradients from reconstruction– Temporal gradients from input video– Key Idea
• Temporal gradients defined betweenmotion compensated temporal neighbors
• Increases compatibility betweentemporal gradients and spatial gradients
• Integrate the 3D gradient field
Spacetime Fusion
• Integrating the gradient field
Solve linear system:Av = b
Constraints:vx, y, t – vx-1, y, t = Gx(x, y, t)vx, y, t – vx, y-1, t = Gy(x, y, t)vx, y, t – vx-u, y-v, t = Gt(x, y, t)
Spacetime Fusion
Conclusion
• Spacetime fusion – Combines spatial and temporal gradients
from two different sources– Requires motion vectors for temporal
source• stereo (static scenes)• flow (dynamic scenes)