Sub-Pixel Layout for Super-Resolution with Images in the ...hangzhao/posters/ECCV14_Poster.pdf ·...

Post on 31-Oct-2019

1 views 0 download

Transcript of Sub-Pixel Layout for Super-Resolution with Images in the ...hangzhao/posters/ECCV14_Poster.pdf ·...

Sub-Pixel Layout for Super-Resolution with Images in the Octic GroupBoxin Shi1,2, Hang Zhao1, Moshe Ben-Ezra1, Sai-Kit Yeung2, Christy Fernandez-Cull3, R. Hamilton Shepard3, Christopher Barsi1, and Ramesh Raskar1

1 2 3

This paper presents a novel super-resolution framework by exploring the propertiesof non-conventional pixel layouts and shapes. We show that recording multipleimages, transformed in the octic group, with a sensor of asymmetric sub-pixel layoutincreases the spatial sampling compared to a conventional sensor with a rectilineargrid of pixels and hence increases the image resolution. The example above showsOTCCD (Orthogonal-Transfer CCD, MIT Lincoln Lab) images under 4 rotations canperform 4x Super-Resolution (SR).

Idea illustration

Take-home message:

Result

Contributions

23.47

10.71

4.67

1.56

Ground truth

Image observed witha conventional sensorwith a rectilinear grid of pixels

SR results varying with sub-pixel layouts

All full-rank sub-pixel layoutsachieve SR reconstructionwith designated magnification.

Simulated images with various sub-pixel layouts for 4x SR.

RMSE24.39 1.75

16.57 1.70

22.92 3.87

4x SR ResultOTCCD layoutGT | LR | SR

8x SR ResultLR | SR

Key proposition

rank: 64

Sub-pixel layouts for 8x SR.

Real-simulation result for 4x (left) and 8x (right) SR

Notehow higher-rank layoutsincrease theresolution.

4x SR needs 4 imagesrotated by 90o; 8x

SR needs all 8images in the

octic group.

This is the gap amongsub-pixels (treat as a dumb pixel). It also contributes to SR.

o A novel view to the SR problem by using an asymmetric sub-pixel layout to form multiple images in the octic group

o The theoretical bound of SR performance w.r.t. the number and distribution of sub-pixels

o A sub-pixel layout selection algorithm to choose good layouts for well-posed SRo A simple yet effective SR reconstruction algorithm

Potential hardware

Given a group of pixels with poses in the octic group with each pixel containing sub-pixels, for a designated magnification factor , rank( ) is bounded as

rank( ) is determined by number and distribution of sub-pixels; It evaluates the well-posedness of SR reconstruction.

OTCCD + Dove prism + rotation mount

“Rotation-asymmetric sub-pixel layout + rotated images” play an important role in increasing spatial sampling to break the limit of reconstruction-based super-resolution.

Conventional CCD OTCCD images rotated by 0, 90, 180, and 270 degrees SR result

An OTCCD pixel (with 4 sub-pixels) in the octic group

rank: 4

rank: 7

rank: 12

rank: 16

rank: 16

Octic group (4th order dihedral group)O

rigin

alO

rigin

al

90o

rota

tion

90o

rota

tion

180o

rota

tion

180o

rota

tion

270o

rota

tion

270o

rota

tion

ReflectionReflection ReflectionReflection ReflectionReflection ReflectionReflection

rank( ) min