On-Demand Sharing of a High-Resolution Panorama Video from Networked Robotic Cameras Supported in...
-
date post
19-Dec-2015 -
Category
Documents
-
view
215 -
download
0
Transcript of On-Demand Sharing of a High-Resolution Panorama Video from Networked Robotic Cameras Supported in...
On-Demand Sharing of a High-Resolution Panorama Video from Networked Robotic
Cameras
Supported in part by
CPSC 643
Dezhen Song
Texas A&M University
Vide framesSphere
wrappingImage
alignment
1
i1
im N
Server side
IBBBBBBIBBBBBBIBBBBBB1
IBBB
i
1
IBBBBBB IBBBBBBN
……
Client i
IBBBBBBIBBBBBBIBBBBBB……
i
m
Client iClient iClient i
Decoding and composing panorama
Rendering, Display
User request [area i, at time t1~tk]
… …
Live video
Time k
BBB
2
Panosonic HCM 280
– PTZ Robotic Camera: • 350° Pan, 120° Tilt, 42x Zoom
• Maximum spatial resolution: 500 Megapixel per
steradian
• 3 Gigapixels panorama
– Network Video Camera:• Built-in streaming server
• 640x480 pixels video
• >30 frames per second
Network PTZ Robotic Camera for Nature Observation
5
Giga-pixel Motion Panorama VS. Fixed Lens Camera
• Fixed lens with mirror• 10M Pixel CCD• $ 20.0 K• 2M Pixel / Steradian
• Pan, Tilt, Zoom (21x)• 0.37M Pixel CCD• $ 1.2 K• 500M Pixel / Steradian
6
Existing Panoramic Video Systems
SystemCamer
a
Bandwidt
hVideo Output
Sample System
s
Wide angle lens/mirrors
Single fixed
Low Low quality live stream
[Baker 1999], [Nayar 1997], [Xiong
1997], [Ng 2005]
Multiple camera
panorama video
Multiple fixed
High Live panoramic video[Foote2001], [Swaminath
an 2000]
Panoramic video texture
Single pan
HighPseudo-live panorama
video by changing video temporal display
[Agarwala 2005]
Dynamosaics Single pan
HighPseudo-live panorama
video by changing space-time volume
[Rav2005]
Motion panorama
Single LowStatic panorama overlaid with living moving objects
trajectory
[Irani 1996], [Bartoli 2004]
Our systemPTZ
Cameras
Low Partial live panorama
7
Evolving Panorama: High Resolution Live Panoramic Video Using PTZ
Camera
Tilt
Pan
Frame sequence
Panorama
Tilt
Time
Panorama
Live frame sequence
Updated Part in
Panorama
8
robotic video cameras
Collaborative Observatories for Natural Environments (www.c-o-n-e.org)
motion sensors
timed checks
sensor networks
humans: amateurs and profs.
2005-2008
9
On-demand Panoramic Video
SharingVide frames
Sphere wrapping
Image alignment
1
i1
im N
Server side
I B B B B B B I B B B B B B I B B B B B B1
I B B B
i1
I B B B B B B I B B B B B BN
……
Client i
I B B B B B B I B B B B B B I B B B B B B
……
im
Client iClient iClient i
Decoding and composing panorama
Rendering and Display
User request [area i, at time t1~tk]
… …
Live video
Time k
B B B
10
Challenges:
– Dynamic video coverage
– High resolution panorama coverage
– Multiple different spatial-temporal client requests.
On-demand Panoramic Video
Sharing
11
User Request
Live Live
Timek-1 k
… ……
Live video
…
Client i
User i request: ri=[u, v, w, h, ts, te]
12
Camera Coverage
Camera Coverage
pan-180o -180o
60o
tilt
N
1 pjk
Patch-based Panorama Video
Snapshot at time k
Live patch Static patch
Camera Coverage
13
Patch-based Panorama Video
Live video
Live
Timek-1 k
… ……k-2
…
Camera coverage at time kPatch j at time k
14
On-demand Patch-based Panorama Video Sharing
Vide frames
Sphere wrapping
Image alignment
1
i1
im N
Server side
I B B B B B B I B B B B B B I B B B B B B1
I B B B
i1
I B B B B B B I B B B B B BN
……
Client i
I B B B B B B I B B B B B B I B B B B B B
……
im
Client iClient iClient i
Decoding and composing panorama
Rendering and Display
User request [area i, at time t1~tk]
… …
Live video
Time k
B B B
15
Frame Insertion Algorithm
Input: Ft
Output: Updated evolving panorama video
Wrap Ft onto the spherical surface;
Estimate Ft’s registration parameters by aligning it with previous frames;
Project Ft onto the sphere panorama surface;
for each pj and pj ∩ Ft ≠ Ø do
Insert pjt into pj’s GOP buffer;
for each pj, j=1, …,N do
if pj’s GOP buffer is full then
Encode patch video segment;
Store patch segment start position and time data into lookup table;
Reset GOP buffer for incoming data;
16
On-demand Patch-based Panorama Video Sharing
For User i request:
Send patch data:
ri=[u, v, w, h, ts, te]
ri ∩ Pt = { pjk | j Є {1,…,N}, k Є [ts, te], pjk ∩ ri ≠Ø , pjk ≠Ø }
17
User Query Algorithm
Input: ri
Output: ri ∩ P in MPEG-2 format
Identify patch set S= { pj | j Є { 1,…,N }, pj ∩ ri ≠Ø };
for each pj Є S do
Find the nearest I frame pjb earlier or equal to ts;
Find the nearest I frame pjc later or equal to te;
Transmit the patch segments between pjb and pjc;
18
Experiments and Results
Hardware configuration:
•Dell Dimension DX, 3.2Ghz Pentium dual-core processor, 2GB RAM
•Panasonic HCM 280A video camera
Software configuration:
•Visual C++ in Microsoft Visual Studio 2003 .NET
•MPEG-2 encoder/decoder from MPEG Software Simulation Group
Input data set: •Frame number: 609
•Frame resolution: 640x480 pixels
•Frame rate: 25 fps
•Raw RGB data size; 536 MB
•Panorama resolution: 2742x909 pixels