M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) •...
43
________________________________________________________________________________________________________________________________ ________________________________________________________________________________________________________________________________ M.S. Thesis Defense ________________________________________________________________________________________________________________________________ ________________________________________________________________________________________________________________________________ Bryan David Thompson March 29 th , 2007 ________________________________________________________________________________________________________________________________ Adviser: Dr. Mahmood R. Azimi-Sadjadi Committee: Dr. Robert Liebler Dr. Edwin K. P. Chong
Transcript of M.S. Thesis DefenseRoadmap • Literature Review (5 min) • Research Objectives (5 min) •...
________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
M.S. Thesis Defense________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Bryan David ThompsonMarch 29th, 2007
________________________________________________________________________________________________________________________________
Adviser:Dr. Mahmood R. Azimi-Sadjadi
Committee:Dr. Robert Liebler
Dr. Edwin K. P. Chong
Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)
– Implementation– Experimental Results
• Theoretical Contribution (10 min)– Development– Simulation Results
• Q&A (15 min)• Additional slides
Literature Review
Buried Underwater Target Detection and Classification• Sensor development & Data acquisition
[Schock] BOSS, AUVs, SAS Processing, Imagery[Harbaugh] Synthesized data sets
• Feature Extraction[Maussong] SAS Imagery (mean-standard deviation)[Sternlicht] Sediment volume imagery (clustering and ellipse)[Azimi] Canonical Correlation Analysis
• Detection & Classification[Dobeck] Detection & Classification Algorithm Fusion[Sternlicht] 2D & 1D Fusion (Gaussian-Bayesian classifiers)[Azimi] HMMs, NNs, D/F-level Fusion, CMAC,
Success of CCA as a coherence-based Feature extraction procedure
Research Objectives
• To implement a multi-channel extension of CCA as a feature extraction procedure
• To demonstrate its effectiveness via practical application
• To develop an iterative learning algorithm providing numerous advantages
Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)
– Implementation– Experimental Results
• Theoretical Contribution (10 min)– Development– Simulation Results
• Q&A (15 min)• Additional slides
Algorithm Review: Multi-channel Coherence Analysis
Assume n data channels
Composite data channel vector
Corresponding correlation matrix
MCA Review (2)
MCA Review (3)CCA: 1st coordinate pair and correlation
MCA: 1st coordinate set and correlations
MCA Review (4)Constraint (1) Unit Variance:
SUMCOR + Unit Variance = No unique solution (thesis)
Constraint (2) Unit Trace:
SUMCOR + Unit Trace = Generalized eigensystem
MCA Review (5)CCA: Higher order coordinates/correlations
MCA: Higher order coordinates/correlations
s.t.
Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)
– Implementation– Experimental Results
• Theoretical Contribution (10 min)– Development– Simulation Results
• Q&A (15 min)• Additional slides
Practical Application• Date: May of 2004 • Platform: Disk BOSS• Location: St. Andrews Bay, Panama City, FL • 11 objects
– 5 mine-like (ML)• Boundary markers• Bomb-shaped object• Large cylinders
– 6 non-mine-like (NML)• Artillery shells• Small cylinder• Bullet-shaped objects• Spheres
• Naturally occurring objects (rocks / clutter)
Implementation
Implementation (2)Experimentally determined ping separations
Experimental Results
• MCA-based single-ping classifier – Outperforms CCA-based single-ping classifier– Outperforms Decision-level Fusion for testing data
• Uses 67% more sonar returns per decision
– Highest generalization ability
Experimental Results
Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)
– Implementation– Experimental Results
• Theoretical Contribution (10 min)– Development– Simulation Results
• Q&A (15 min)• Additional slides
Theoretical Contribution
An Iterative Learning Algorithm for MCA• Data-driven• Real-time• Circumvented calculations:
– Calculating sample covariance matrices for all data channel pairs
– Solving generalized eigenvalue problems• Recent IJCNN 2007 acceptance
– Best Paper Award nomination by one reviewer
Development1st coordinate set
Iterative method of steepest descent (or ascent… maximizing)
Concisely
Development (2)1st correlation sum
Higher order (ith) coordinate sets / correlation sums
subject to:
Development (3)Lagrange multiplier method with penalty parameter
Iterative method of steepest descent (ascent)
Development (4)Conversion to Least-Mean-Square (LMS) based updating
For the composite coordinate mapping vector
For the correlation sum
Simulation Results
• Simulated three-channel data set– -
• Guided by the linear model
Simulation Results (2)
Squared estimation error plots vs. iteration
Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)
– Implementation– Experimental Results
• Theoretical Contribution (10 min)– Development– Simulation Results
• Q&A (15 min)• Additional slides
Q&A
• Supported by ONR– Contract #N00014-05-1-0014
• Special Thanks– Dr. Azimi, Dr. Liebler, and Dr. Chong– Gordon, Nicholas, Jered, Amanda, Neil, Ali,
Jaime, Makoto, Derek, and Michael.– Family and friends– Coworkers at Agilent Technologies, Inc.
• Incredible experience because of the people
Roadmap• Literature Review (5 min)• Research Objectives (5 min)• Algorithm Review (10 min)• Practical Application (15 min)
– Implementation– Experimental Results
• Theoretical Contribution (10 min)– Development– Simulation Results
• Q&A (15 min)• Additional slides
Experimental Results
Additional: MCA
FONC: (interior case)
SOSC: Satisfied because A (the Hessian) is positive semi-definite
Additional: MCA (2)
Additional: Mutual Information and Coherence
Additional: BOSS Data set
Additional: Preprocessing
Additional: Concatenation
• For each channel, a preprocessed sonar return yields a a time series of 200x1
• Concatenated together to yield 9600x1• Partition into overlapping blocks 40x1
– 50% overlap– Statistically rich
Additional: Feature Extraction• To capture ML vs NML discriminatory factors in sonar
returns using coherence-based feature extraction• CCA
– Input two preprocessed sonar ping partitions (each of 40x1)– Output vector of canonical correlations 40x1
• MCA– Input three preprocessed sonar ping partitions (each of 40x1)– Output vector of correlation sums 40x1
• Use dominate 20 as feature vector– To simplify classifier complexity
Additional: Clarity• Preprocessing
– To remove undesirable portions of the signal– Matched filtering (requires incident signal)– Windowing– Inverse Matched Filtered
• Concatenation– Concatenate kth return for each channel into one time series– Form overlapping blocks with 50% overlap– Statistically rich
• Feature extraction– To capture ML and NML discriminatory factors in sonar
returns using coherence-based feature extraction– Canonical Correlation Analysis (two-channel)– Multi-channel Coherence Analysis (three-channel)
Additional: Clarity (2)
• Single-ping classification– Back propagation neural network (BPNN)– Training data set: ½ of Line 4– Validation data set: ½ of Line 4– Testing data set: Lines 2 and 8
• Decision-level fusion– Nonlinear (BPNN-based)
Additional: Single-ping Classifier• To capture ML vs. NML discriminatory factors in feature vectors• Batch process training using entire set of training patterns
– 20 dominant correlations (sums) extracted via CCA (MCA)– Equally split ML and NML features between training and validation
data• Back propagated error (between the desired and actual outputs)
through the system and the various weight layers are adjusted. – Mine-like output is [1 0]T– Non-mine-like output is [0 1]T
• Structure – Several different two- and three-layer BPNNs were tested– Two-layer implementation determined best– Tested ranged from 40 to 50 hidden layer neurons– Randomly initialized layer weights and biases (several times)– Optimal structure: 20 inputs, 50 hidden layer neurons, 2 output
neurons
Additional: Single-ping Classifier (2)
• Optimal classifier chosen based on training and validation data set classification performance– ML classification performance favored over NML
• Training– Fixed learning rate (heuristically determined)– Levenberg-Marquardt (LM) learning rule [54]– No momentum factor
• Line 2 and 8 testing data sets determined classifiers’ generalization ability
Additional: Decision-level Fusion
• To further exaggerate the captured ML vs. NML discriminatory factors of previously made decisions
• Similarly training as BPNN-based Single-ping classifiers
• Similarly obtained optimal structure– Optimal structure: 6 inputs, 6 hidden layer
neurons, 2 output neurons
Additional: Implementation
Additional: Implementation (2)
No more additional slides
