Astrophysics Applied What to do with Images and Wavelets Plato's Republic, Book vii: Socrates: Shall...
-
date post
18-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of Astrophysics Applied What to do with Images and Wavelets Plato's Republic, Book vii: Socrates: Shall...
Astrophysics Applied What to do with Images and
Wavelets
Plato's Republic, Book vii:
Socrates: Shall we set down astronomy among the subjects of study?
Glaucon: I think so, to know something about the seasons, the months and the years is of use for military purposes, as well as for agriculture and for navigation.
Socrates: It amuses me to see how afraid you are, lest the people should accuse you of recommending useless studies.
Socrates goes on to say that the use of astronomy is not to add to the vulgar comforts of life, but to assist in raising the mind to the contemplation of things which are to be perceived by the pure intellect alone.
28th. June 2006 Bernard60 - Valencia, Spain 2
What this is about – a history
1984: Started a small hi-tech venture – to offset the stress of the academic job situation (!). The venture focused on CCTV imaging on the then-new PC platform.
1988: BBC TV Tomorrow’s World
1995 started work in earnest after winning UK government contracts
2003 developed leading video acquisition, storage and retrieval system based on wavelet methodology, modern statistical methods and nonlinear image processing algorithms.
Currently happily spend a week per month at the Kapteyn Institute in Groningen: mind-to-mind resuscitation.
28th. June 2006 Bernard60 - Valencia, Spain 3
Achievements
Best real-time video compression algorithm based on adaptive wavelet technology: up to 9 days on one DVD!
Sophisticated video event detection through adaptive scene analysis.
Automated video synopses based on event detection allows searching terra-bytes in minutes
Earning a reasonable living without having to suffer the slings and arrows of arbitrary high level funding decisions – independence!!
Even more stress and anxiety
28th. June 2006 Bernard60 - Valencia, Spain 4
Image processing
Static images
Weiner filters
Fourier transforms
Moving image sequences
Nonlinear reconstruction
Wavelet transforms
Classical Modern
Static images (generally)
Huge dynamic range
Measured quantities have important physical meaning
Dynamic images
Limited dynamic range
Data is not photometric
TV ImagingAstronomy
28th. June 2006 Bernard60 - Valencia, Spain 5
CCTV – Closed circuit television
The UK has some 8x106 CCTV cameras– 10% of world total– Main detector is small CCD (mostly colour)– Sensitivity down to 10-5 lux (mono cameras)– Resolution at best 576x704– Output is analogue
Future is to go to HDTV and digital IP delivery– Resolution 1024x768 (p) or 1280x1024 (i)– Data format MPEG-4 (ideally AVC – level 10)
starlight 0.00005 lux
Moonlight 1 lux
Office light 100 lux
midday 100,000 lux
(1 footcandle = 10.76 lux)
Note that even the new HDTV has nowhere near the 5 megapixel resolution of a modest still digital camera.
28th. June 2006 Bernard60 - Valencia, Spain 6
The CCTV data nightmare
One camera generates 30 Megabytes of data per second.
A small 100 camera system delivers 3 Gigabytes per second or 1 Petabyte per year
All CCTV cameras in the UK generate about 100 Exabytes per year
There is no way of storing all this, let alone looking at it or searching for anything.
G=109, T=1012,P=1015, E=1018,Z=1021,…,W=1030
Library of Congress = 10 Tbytes
Cut down on frame rate (time-lapse)
Save only 1-2 weeks
Compress while trying to retain quality
Save only “interesting” data
Such strategies may compromise data integrity if data is eventually to be used as part of a legal investigation.
Some numbers … Traditional strategies for handling this …
While acquiring the images
1. Do far better compression
2. Analyze the data
3. Enhanced real time scene analysis
4. Store data synopses
Improved search methodology:
1. Cross-correlate what is seen on different cameras
2. Learn from previous searches
3. Provide a search query mechanism
28th. June 2006 Bernard60 - Valencia, Spain 7
Acquiring video pictures to PC
The video frame grabber converts analogue video signals into digital data array that can be processed. Much depends on the quality of this device: this one supports broadcast quality.
Analogue video in (16 channels)
Digital data outputSent to PC processor
Analogue to digital converters
High performance digital signal
processor (VLIW multi-core
4 billion instructions per second)
One or more of these sit inside a
“PC” running Linux
We design and manufacture
these.This is the 6th in 20
years.
28th. June 2006 Bernard60 - Valencia, Spain 8
Typical site monitoring
28th. June 2006 Bernard60 - Valencia, Spain 9
One dimensional wavelet filter
)(][
)(][)(
,
,
xkd
xksxf
kLjk
LjjL
kjk
j
The yellow bits at level j
Scaling function at level j
(orthogonal to the wavelet)
The red bits. Contributions come from all
levels, L
Wavelet Function at level L
Reconstruction of data S0 from multi-resolution analysis {Sj, Dj, Dj-
1}.
Schematically:
Levels used in reconstruction
You don’t have to use the same wavelets at all levels, and nor do you have to keep all the {dj}.
The choices can be made to depend on image content and dynamics. This is a key trick.
28th. June 2006 Bernard60 - Valencia, Spain 10
Multi-resolution hierarchy
2D S
2SS 2SD
Level 1
Level 2
Level 3
1 SS
2D D
1D S 1 D D
1SD
O riginal Im ageLevel 0
STAR T
FINISH
1D D1D S
1S D
2SD
2DS 2DD
3SS 3SD
3D S 3D D
The image is shrunk by linear factors of two. The residuals that would allow the picture at any level to be reconstructed from the higher levels are stored alongside.
The shrinking process is important, but need not use wavelets.
28th. June 2006 Bernard60 - Valencia, Spain 11
Image representation - wavelets
Two level wavelet transform
Censorship of coefficients for compression
Rie
n va
n de
Wey
gaer
t
Images have been enhanced for clarity
The scene analysis might as well be done at the same time as the censorship and compression of the data. That way the compression can adapt locally to scene content.
28th. June 2006 Bernard60 - Valencia, Spain 12
Wavelet compression vs. MPEG
Wavelet compressed frame from CCTV video sequence.
MPEG-1 compressed frame from same video sequence. Note the blocky artefacts and colour contouring.
The goal is to achieve high levels of compression without sacrificing image quality. This is particularly difficult when there is a lot of movement in the scene. Using our locally adaptive wavelets makes a huge difference over normal wavelets.
28th. June 2006 Bernard60 - Valencia, Spain 13
Sunlight, shadows & trees
This is full-screen video event detection. No areas have been excluded.
Note the moving shadows and trees, together with varying illumination. These do not produce alarms.
The first detection is in fact behind the window before walking through the door.
The picture turns grey on detection for demo purposes only: we need to show colour blocks.
This is an MPEG-1 version of an Astraguard wavelet sequence
The main issue is to detect what you want to detect without missing anything, while at the same time avoiding things you do not want like sunlight, shadows and trees. This sequence demonstrates some successes. There are also some failures which can in fact be handled with a little more algorithm refinement.
28th. June 2006 Bernard60 - Valencia, Spain 14
Application Areas
Large government and industrial sites
Hospitals
Police Work
Fire detection on gas rigs
Oil Pipelines
Power line monitoring
High value premises
The goal is to provide equipment to handle video streams in situations where an understanding of what is taking place in the scene, or has taken place at an earlier time, is relevant.
The system must be easy to configure and even easier to use. Application areas are often mission-critical, so close to 100% reliability is essential.
Systems range from one camera to many hundreds of cameras. Upper limit is in the 1000’s of cameras.
28th. June 2006 Bernard60 - Valencia, Spain 15
What next?
Retire to a vineyard in Australia?
Write the book that has taken over 10 years to not finish.
Continue to share in the excitement of cosmology.
Keep healthy if not fit
28th. June 2006 Bernard60 - Valencia, Spain 16
Signal recognition using wavelets
The same signal (top) is analyzed using two different wavelets.The Gabor wavelet is capable of localizing the signal in both frequency and
time.
“fre
qu
en
cy”
Time
am
plit
ud
e
28th. June 2006 Bernard60 - Valencia, Spain 17
Signal recognition using wavelets
“fre
qu
en
cy”
Time
am
plit
ud
e
The same signal (top) is analyzed using two different wavelets.The Gabor wavelet is capable of localizing the signal in both frequency and
time.