1

25 Years of Computer Graphics:Back to the Future?

John PATTERSON

Computer Vision and Graphics Group

Dept of Computing Science

University of GLASGOW

25 Years of Graphics at Bath

University of Bath

14-September-04

2

Back to the Future?

• ..maybe back to the 70s at 21st century clock rates

• e.g. graphics CPU/GPU relationship periodically gets reinvented

• What about relationship between Vector and Raster graphics?

• Advocate putting vector formats on same basis as raster formats

• Many pitfalls

• low-level approach because these issues need fixing first

3

Topics

• Fiddling at the margins

• Resizing and Resampling (and inter alia an outrageous proposal)

• What is a pixel anyway?

• Vectors Rule OK

4

Fiddling at the Margins (1)

Catmull-Rom patch with control points (x, y, h)

(x,y): pixel coordinates

h: pixel grey-value

Isochromic surface for original raster image:

What happens at the edges?

5

Edge PixelCurve segmentPP P Petc.12 3 4

Curve segmentP P Petc.2 3 4P0P1

Fiddling at the Margins (2)

curve is only defined between and

€

P2

€

P3

Need to ‘hallucinate’ additional point to accommodate edge pixel

€

P0otherwise get image erosion. Also applies to convolution filters, which may need more pixels ‘hallucinated’ to avoid erosion e.g. 2 for 5 x 5, 3 for 7 x 7 etc.

6

Fiddling at the Margins (3)

Set to zero

Set to

Extrapolate from (linear)

Extrapolate from (quadratic)

Extrapolate from (cubic)

..or?

€

P0

€

P0

€

P1

€

P2,P1

€

P3,P2,P1

€

P4,P3,P2,P1

The options are:

Not clear which, if any is ‘appropriate’.

7

Method: Crimesti, Perez & Toyama “Object removal by Exemplar-Based Inpainting “ CVVG 2003

This implementation favours reinforcement of linear features

Original Image Segment cut out Synthesised result

..or - use in-painting?- (1)

8

Hole regionImage region

Hole regionImage region

Hole regionImage region

..or - use in-painting (2)

•Find highest confidence point on fill front

•Track along isochromic contour for best match

•Replace point by matched region, confidence values = pixel match accuracy

Local isochromic contour

Patch size should be big enough to contain texture cell, here 9 x 9

9Texture synthesis example:

Synthesised result

Example shows repair of linear features. While a characteristic of the algorithm a different ‘data’ term was used to prioritise in-filling points here

Original Image

Segments cut out

10

Application to edge extension

High confidence point

In-painting involves reducing hole using high confidence points

Edge extension involves extending image with less support from local context

11

Resizing and Resampling (1)

€

1 −tan θ 2( )0 1

⎛

⎝ ⎜

⎞

⎠ ⎟

€

1 −tan θ 2( )0 1

⎛

⎝ ⎜

⎞

⎠ ⎟

€

1 0

sin θ( ) 1

⎛

⎝ ⎜

⎞

⎠ ⎟

Any whole-image transformation requires (a) resampling stage(s)

Example: rotation through using 3 skew operations:

Skew along x axis, resample in x

Skew along y axis, resample in y

Skew along x axis, resample in x

12

Resizing and Resampling (2)

This is what happens when you don’t resample in a skew

Fairly horrible!

€

1 −tan θ 2( )0 1

⎛

⎝ ⎜

⎞

⎠ ⎟

13

Resizing and Resampling (3)Even re-sizing requires resampling: carry out in 2 passes,

scale in x, resample in x; scale in y, resample in y

Ideal reconstruction filter:

€

sinc x( ) =sin x( )x

=1−x2

3!+x4

5!−x6

7!etc. is 1 when x=0

Gaussian is not very good approximation

Catmull-Rom basis is better

14

Resizing and Resampling (4)

A small entertainment:

Gaussian kernel usually modelled as right:

Usually applied in convolution using weighted samples, here weights modelled

as heights to match curve above

Alternative is to use stochastic sampling: treat Gaussian function as a probability

field and take spatially distributed samples which are averaged

Could do this for the sinc function also (and subtract samples in negative lobes)

Defeated by need for too many samples? What happens at edges?

15

Resizing and Resampling (5)So- how to resample?

• Catmull-Rom is popular, but is only first-order continuous

• Central convolution interpolation advocated- still first-order order continuous

•Resampling assumes continuity between samples. What about edges? Zero crossings (sign reversals in the second difference) should give these away (e.g. Canny)

• Also assumes you can trust your pixel values:

Invented by actuaries 100+ years ago

Use differences up to third degree and interpolate ‘up the middle’ using weights (convolution). Various formulae map out more sinc lobes

Point spread function

Noise

Blur (can blur edges away)

16

What is a pixel anyway?

32 x 32 pixel

image

An honest topography

17

The pretence

Image

ContinuousSurface (here Catmull-Rom)

Slicing the surface gives an isochromic contour (but need interpolation)

18

Girl’s face photoInput Image

Contour or ‘Vector’ form

Rendered from vectors

19

Direct Comparison at same sizes

Original image x4 bicubic Rendered image from contour map

20

Level Sets

Level Sets and Fast Marching methods originally developed to solve flame-front advancement calculations involving PDEs

Similar to reaction-diffusion equations originally worked on by Turing in the 1930s, so collectively known as ‘diffusion’

Outer contour

Inner contour

Outer boundary ‘advances’ towards inner boundary according to PDEs controlled by boundary conditions, e.g. curvature

Intermediate contour defines boundary position at set ‘time’ or parametric distance

Many image indexing problems may be cast in this form

21

Conclusion

• Images can be represented in terms of control points for isochromic contours

• Images shown do not take full advantage of smoothing options (data reduction) and do not yet use level set diffusion

• Whole image operations, including matting, matte-pulling, and texture synthesis can be expressed in vector form

• Computational topology (Morse theory, Homotopy, Critical point analysis etc.) applicable

• Level sets and Fast marching methods applicable both to spatial indexing between contours but also to contour evolution between images in a sequence (in-betweening) - originally developed as a numerical technique

• .. but in the end images are vectors. Isn’t this where we came in? (metafiles, SVG)?