Prof. Aaron D. Lanterman School of Electrical & Computer Engineering
description
Transcript of Prof. Aaron D. Lanterman School of Electrical & Computer Engineering
![Page 1: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/1.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Target Recognitionin Cluttered Infrared Scenes via
Pattern Theoretic Representations and Jump-Diffusion Processes
Prof. Aaron D. Lanterman
School of Electrical & Computer EngineeringGeorgia Institute of Technology
![Page 2: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/2.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Variability in Complex ScenesVariability in Complex Scenes– Geometric variability
• Position• Orientation• Articulation• Fingerprint (in the Bob Hummel sense)
– Environmental variability• Thermodynamic variability in infrared• Illumination variability in visual
– Complexity variability• Number of objects not known
![Page 3: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/3.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Pattern Theory: The Grenander ProgramPattern Theory: The Grenander Program• Representation:
– Incorporate variability in the parameter space• Possibly many nuisance parameters
– Model mapping from parameters to data• Inference:
– Build inference algorithms using the representation– Ex.: Markov chain Monte Carlo (jump-diffusion)
• General notion: avoid “preprocessing” or “feature extraction” as much as possible to avoid loss of information– Recall Biao Chen’s mention of the Information Processing
Inequality• Apply tools of Bayesian inference to weird things
![Page 4: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/4.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Legacy WorkLegacy Work
• Sponsored by– U.S. Army Center for Imaging Science
(ARO - David Skatrud/Bill Sander)– ONR (Bill Miceli)
• Collaborators– Michael Miller (now at Johns Hopkins)– Donald Snyder (Washington Univ.)– Anuj Srivastava (now with Dept. of Stat., Florida State)
• Airborne targets – radar– Matt Cooper (now with Xerox)
• Thermodynamic variability of targets
![Page 5: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/5.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Parameter Space for ATRParameter Space for ATR• Parameter space for a single target:
• Number of targets not known in advance:
A )2,0[)1( 2 X...}62,60,2{ TMMA
• Parameter space for an n-target scene:nAn ])2,0[[)( 2 X
0
2 ])2,0[[n
nAX
![Page 6: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/6.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Ingrid’s Third ApproachIngrid’s Third Approach• Data , parameters• Likelihood
– Render infrared scene onto detector plane – Model sensor noise effects
• Prior • Bayesian posterior
– Analytically forboding!• Sample via jump-diffusion processes
– Jump from subspace to subspace– Diffuse to refine estimates within a subspace )|( xyp
x
y
p(x)
(x) p(x | y) p(y | x)p(x)
![Page 7: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/7.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Take Home MessageTake Home Message
Go readUlf Grenander’s
papers and books.They are very cool.
![Page 8: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/8.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Perspective ProjectionPerspective Projection
![Page 9: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/9.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Sensor EffectsSensor Effects
Optical PSF
Poisson Photocounting Noise
Dead andSaturated Pixels
![Page 10: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/10.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
FLIR LoglikelihoodFLIR Loglikelihood• CCD loglikelihood of Snyder, Hammoud, and White
LCCD (y | ) (i)i y(i)ln(i)
i
( j) psf(ij | j)( j)where
L(y | x) LCCD (y | render(x))• Cascade with
render : x
• Sensor fusion natural; just add loglikelihoods
![Page 11: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/11.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Langevin Processes ProcessLangevin Processes Process
• Consider a fixed number of N targets and target classes• Simulate Langevin diffusion:
)())}(({)( NnXN dWXHdXN
• Distribution of • Computed desired statistics from the samples• Generalizes to non-Euclidean groups like rotations• Gradient computation
– Numeric approximations– Easy and fast on modern 3-D graphics hardware
)()( nNN xX
(x) exp{H(x)}/Z• Write posterior in Gibbs form:
![Page 12: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/12.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Diffusion Example on AMCOM DataDiffusion Example on AMCOM Data
![Page 13: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/13.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Jump MovesJump MovesBirth Death Type-change
![Page 14: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/14.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Helpful Properties of Jump ProcessesHelpful Properties of Jump Processes
• Connectedness: can go from any point to any other in a finite number of jumps
(x)Pr(x z) (z)Pr(z x)(continuous form slightly more complicated)
{states reachable from }
T 1(x) T 1(x)
T 1(x)
T 1(x) {states from which can be reached}
x
x
• Jump at exponentially distributed times
• Detailed balance (in discrete form)
• Move reversability:
![Page 15: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/15.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Jumping StrategiesJumping Strategies
• Gibbs–Sample from a restricted part of the posterior
• Metropolis-Hastings style–Draw a “proposal” from a “proposal density”–Accept (or reject) the proposal with a certain probability
![Page 16: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/16.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Example Jump-Diffusion ProcessExample Jump-Diffusion Process
![Page 17: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/17.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
How to Model Clutter?How to Model Clutter?• Problem: Algorithm only knows about tanks
(which Bob doesn’t like anyway), and will try to explain everything with tanks!– Cows, swimming pools, school buses
• Solution (?): Let the algorithm use flexible representation in addition to rigid objects– Blobs: Simple connected shapes on the lattice to
represent structured clutter– Could use active curves, level set methods (Clem
Karl), active polygons (Hamid Krim)– Clutter might be interesting in its own right!
![Page 18: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/18.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Random Sampling for BlobsRandom Sampling for Blobs
• Set of jump moves–Add a pixel along the boundary–Remove a pixel along the boundary–Keep the blob a blob
• Pick move based on posterior probability
![Page 19: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/19.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Blob Estimation ExamplesBlob Estimation Examples
M60 spreading
M60 decaying
Ship decaying
![Page 20: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/20.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
NVESD M60 ExampleNVESD M60 Example
![Page 21: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/21.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Saccadic DetectionSaccadic Detection
• Current implementation “births” specific target types
• May be better to birth simple shapes, and later change them to more specific target types (clutter or target)
• Example:– Birth squares– Deform into rectangles– Then jump to more detailed targets
![Page 22: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/22.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Initial Detection
Low-dimensionalrefinement
AMCOM Data Ex.: Finding Tank 1AMCOM Data Ex.: Finding Tank 1
High-dimensionalrefinement
Equilibrium
![Page 23: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/23.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Initial Detection
AMCOM Data Ex.: Finding Tank 2AMCOM Data Ex.: Finding Tank 2
Low-dimensionalrefinement
Equilibrium
![Page 24: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/24.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
AMCOM Data Ex.: Finding ?????AMCOM Data Ex.: Finding ?????
Initial Detection
Low-dimensionalrefinement
![Page 25: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/25.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Factory ExampleFactory Example
![Page 26: Prof. Aaron D. Lanterman School of Electrical & Computer Engineering](https://reader035.fdocuments.us/reader035/viewer/2022062810/56815e04550346895dcc4ce3/html5/thumbnails/26.jpg)
Aaron D. Lanterman 404-385-2548 [email protected] of Electrical and Computer Engineering
Unified AlgorithmUnified Algorithm
• Extended jump moves– Saccadic blob– Saccadic rigid– Blob rigid– Break/combine blobs– Change rigid target types
• Difficulty: Make parameters make sense between representation types