Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo...
-
Upload
kailyn-horner -
Category
Documents
-
view
214 -
download
0
Transcript of Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo...
![Page 1: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/1.jpg)
Super-Resolution Texturing for Super-Resolution Texturing for Online Virtual GlobesOnline Virtual Globes
Diego Rother, Lance Williams and Guillermo SapiroDiego Rother, Lance Williams and Guillermo SapiroUniversity of Minnesota and Google, Inc.University of Minnesota and Google, Inc.
Internet Vision Workshop (CVPR 2008)
![Page 2: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/2.jpg)
Online Virtual Globes
![Page 3: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/3.jpg)
Online Virtual Globes
ClipMap pyramid.
Server Client
Requests
Tiles
• Problems:– Huge storage space required (one of the largest organized collections of imagery on the Internet).– Expensive acquisition and high transmission bandwidth.– Interpolation beyond available resolution: unnatural.
User
![Page 4: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/4.jpg)
Earth surface
– Stereotypical. – Rapidly changing.– Identity important, details not.
![Page 5: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/5.jpg)
Proposed Solution
• Proposition:– Synthesize, on the client, details for the
lower pyramid levels.– Using super-resolution techniques.– Harnessing labels and textures samples
from users (wiki model).
• Requirements:– Fast.– Seamless transition between layers.
![Page 6: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/6.jpg)
Results
![Page 7: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/7.jpg)
User input 1: Labels
Using interactive segmentation as in:Bai, X. and Sapiro, G., "A geodesic framework for fast interactive image and video segmentation and matting." ICCV, 2007.
Original frame User provided labels
Class1
(grass)
Class2
(path)
![Page 8: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/8.jpg)
User input 2: Keyframed Texture
Keyframe1
Keyframe2
Keyscale1
Keyscale2
• User provides the texture pyramid:
Texture pyramid.
e.g., in meters/pixel
![Page 9: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/9.jpg)
Synthesis of a New Layer
Input:from Server
Output:New Layer
Input:from Users
ClipMap pyramidLabels
Texture pyramid.
![Page 10: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/10.jpg)
System Overview
Undo Color Matching
Texture Transfer
Color Matching
Interpolation
Interpolation
Pyramid of Training Textures
(from users)
Inputs
Last
Lay
er(fr
om s
erve
r)La
bels
(from
use
rs)
Outputs
New
Lay
erN
ew L
abel
s
Selection of the Training Image
![Page 11: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/11.jpg)
Texture transfer: 1st passTraining Texture (from the texture pyramid)
Color matched image, without high frequencies
• Wei, L. and Levoy, M., "Fast Texture Synthesis using Tree-structured Vector Quantization." SIGGRAPH, 2000.• Efros, A. A. and Leung, T. K., "Texture Synthesis by Non-parametric Sampling." ICCV, 1999.
Small contexts → Fast
Y Channel(luminance)
I and Q Channels (chrominance)
Mean and Gradient Only Mean
Similarity between contexts considers
Source Locations
Produces 1-Pixel Patches:Contiguous areas copied verbatim from the training texture.
![Page 12: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/12.jpg)
Texture transfer: 2nd, 3rd and 4th passes
• Ashikhmin, M., "Synthesizing Natural Textures." ACM Symposium on Interactive 3D Graphics. 2001.
Few candidates → Fast
Training TextureColor
ChannelPass
2nd
3rd and 4th
Y Channel(luminance)
I and Q Channels (chrominance)
Similarity between contexts considers
Produces Bigger Patches
![Page 13: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/13.jpg)
Texture transfer from the same texture
Texture Pyramid
1st synthetic frame
2nd synthetic frame
• Patch interior (lilac and violet) directly copied.
• Patch boundaries (pink) synthesized in 4 passes as before.
• Doubles the patch size.
ClipMap Pyramid
![Page 14: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/14.jpg)
Results: Maracanã
No texture transferred
No texture transferred
No texture transferred
No texture transferred
Texture pyramid (user input)
ClipMap pyramid (result)
![Page 15: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/15.jpg)
Results: Maracana
No texture transferred
No texture transferred
No texture transferred
No texture transferred
ClipMap pyramid (result)
Source Locations (result)
![Page 16: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/16.jpg)
Results: Field
No texture transferred
No texture transferred
No texture transferred
No texture transferred
No texture transferred
No texture transferred
No texture transferred
No texture transferred
No texture transferred
No texture transferred
ClipMap pyramid (result)
Texture pyramid (user input)
Source Locations (result)
![Page 17: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/17.jpg)
Results: BeachClipMap pyramid (result)
![Page 18: Super-Resolution Texturing for Online Virtual Globes Diego Rother, Lance Williams and Guillermo Sapiro University of Minnesota and Google, Inc. Internet.](https://reader036.fdocuments.us/reader036/viewer/2022081516/551791615503460e6e8b5941/html5/thumbnails/18.jpg)
Conclusions
• Proposed solution:– Reduces storage, bandwidth, and acquisition costs.– Improves appearance and information content.– Is fast (low dimensional contexts).