SENSOR FUSION LABORATORY Director: Thad Roppel, Associate Professor AU Electrical and Computer...
-
date post
21-Dec-2015 -
Category
Documents
-
view
213 -
download
0
Transcript of SENSOR FUSION LABORATORY Director: Thad Roppel, Associate Professor AU Electrical and Computer...
SENSOR FUSION LABORATORY
Director: Thad Roppel, Associate Professor AU Electrical and Computer Engineering Dept.
EXAMPLES
• Infrared / Millimeter wave radar for vehicle detection and identification
• Chemical sensor arrays – “artificial nose”
• Biomimetics – imitating animal sensorimotor behaviors
• Biomedical – using electrical and optical probes to study cardiac arrhythmias
MISSION: Study the benefits of using simultaneous information from multiple sensors to probe the environment.
SENSOR FUSION LABORATORY
Problem Complexity: Human vs. Machine
HUMAN
MA
CH
INE
EASY HARD
EASY
HARD Maximum
Potential Benefit
• Object recognition• Linguistics• Extraction of Relevant
Features from Sensor Arrays
• Arithmetic• Logic
• Thresholding• Tallying
• Judging
Personnel and PublicationsPERSONNEL
•Ting-To Lo (PhD): Molecular Switching in Biosensors
•Rama Narendran (PhD): Biomimetic Simulations of Organized Machine Behavior
•Jun Pan (PhD): Wireless Protocol for Electrical and Optical Cardiac Microprobes
•Aroldo Couto (MS): Flight Stabilization Using Adaptive Artificial Neural Networks
•Brian Wingfield (MS): Silicon Processing for Lateral Emission Fiber-Optic SensorsREPRESENTATIVE RECENT PUBLICATIONS
• D. M. Wilson, T. Roppel, and R. Kalim, "Aggregation of Sensory Input for Robust Performance in Chemical Sensing Microsystems," Sensors and Actuators B, 64(1–3), 107-117, June 2000.
• T. Roppel and D. M. Wilson, "Biologically-Inspired Pattern Recognition for Odor Detection," Pattern Recognition Letters, 21(3), 213–219, March 2000.
• D. M. Wilson, K. Dunman, T. Roppel, and R. Kalim, "Rank Extraction in Tin-Oxide Sensor Arrays," Sensors and Actuators B, 62(3), 199-210, April 2000.
• T. Roppel, R. Kalim, and D. Wilson, "Sensory Plane Analog-VLSI for Interfacing Sensor Arrays to Neural Networks, " Virtual Intelligence and Dynamic Neural Networks VI-DYNN '98, Stockholm, Sweden, June 22-26, 1998.
IR / MMW DATA FUSIONSupport: AFOSR 1992-93
Project Goal: Improved identification of military vehicles from aerial scenes.
LANCE Missile Launcher
T-62 Tank
M-113 Armored Personnel Carrier (APC)
IR / MMW Fusion, cont’d
APPROACH:
IR SCENE PIXELS
MMW RADAR DATA
NEURAL NETWORK
APCTANKLAUNCHER
PERFORMANCE ASSESSMENT: A T L
A + - -
T - + -
L - - +
•Multiple permutations
•Confusion matrix
•Average result
OVERALL RESULT: 14 % improvement with sensor fusion
Chemical Sensor Arrays
Support: DARPA 1997-99
PROJECT GOAL: Improved identification and detection of chemical plumes in non-laboratory conditions.
VEHICLE
SENSORS
PLUME COMMANDSTATION
RF LINK
ROAD
WIND
Chemical Sensor Arrays, cont’d
Odor Sensor Array
0 100 200 300 400 5000
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Timestep
Sen
sor
Vol
tage
Sensor Outputs
Sensor Array Dynamic Response
Chemical Sensor Arrays, cont’d
0 100 200 300 400 5000
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Timestep
Sen
sor
Vol
tage
10 20 30 40 50
2
4
6
8
10
12
14
Sen
sor
Num
ber
Timestep
Sensors 1-15
Raw Output Thresholded Binary Output
Above ThresholdBelow ThresholdPreprocessing
Chemical Sensor Arrays, cont’d
ace
Sample 1 Sample 2 1
20
Sample 3 1
20
amm
dal
g87
g89
g93
oil
pth
Sensor #
xyl
5 10 15Sensor #5 10 15
Sensor #5 10 15
Chemical Sensor Arrays, cont’d
input categories
net
wo
rk r
esp
on
se 1 timestep
aceammdalg87g89g93oilpthxyl
5 timesteps 10 timestepsn
etw
ork
res
po
nse 20 timesteps
aceammdalg87g89g93oilpthxyl
50 timesteps Ideal Response
Time Evolution of Confusion Matrix: Forward SequenceTrained for 20 timesteps
00.10.20.30.40.50.60.70.80.91
Chemical Sensor Arrays, cont’d
00.10.20.30.40.50.60.70.80.91
Time Evolution of Confusion Matrix: Random SequenceTrained for 20 timesteps
1 timestep 5 timesteps 10 timesteps
20 timesteps 50 timesteps Ideal Response
net
wo
rk r
esp
on
se
aceammdalg87g89g93oilpthxyl
net
wo
rk r
esp
on
se
aceammdalg87g89g93oilpthxyl
input categories
Chemical Sensor Arrays - Summary
A recurrent neural network was trained to recognize 9 odors presented in an arbitrary time sequence.
Response time is reduced by an order of magnitude by threshold preprocessing.
Well-suited for use as a front-end for a hierarchical suite of NN’s in a portable, near-real time odor classification device.
BIOMIMETICS
PROJECT GOAL: Learn about optic flow-based sensor fusion from animals. Apply this to flying a drone to target using onboard video.
Flies land accurately
Bees find flowers
Bats catch evading insects in flight
BIOMIMETICS, cont’d
These are natural examples of optic flow.
Represent sensory image field by motion vector field.
Image Sequence
Optic Flow Field
BIOMIMETICS, cont’d
EXAMPLES
A fly can land by maintaining constant optic flow.
A dog can track by maintaining constant sensory flow across olfactory epithelium and following the gradient (using sniffing as a form of “chopper amplifier.”
Can we navigate a drone or guide a missile to target with a similar approach?
Neuro-fuzzy Control Systems
FUZZY LOGIC
•Multivalued logic based on established, rigorous mathematical theory
•Allows intermediate values to be defined between extremes (yes/no, fast/slow)
•Permits valid computational decisions to be made using syntactic input
•Yields fast, rule-based computations which can be validated systematically
NEURAL NETWORK
•Nonlinear dynamical system “black box”
•Trained using field data
•Hard to validate
Neuro-fuzzy Application
Example: Optic Flow for 2D Translation and RotationSpecifying the initial membership functions for the fuzzy sets
Fuzzy Rules
Two example rules (implement multi-camera fusion)...
•If angle is zero and angular velocity is zero and speed is zero, then translation is zero and rotation is zero.
•If angle is zero and angular velocity is pos. low and speed is pos. low, then translation is pos. low and rotation is CW low.
Example meta-rule...
•If GPS is unreliable then do not use.
Italics: predicate Bold: consequence Underlined: fuzzy value
Neuro-fuzzy Training
•Membership functions and rule weights are trained on field data using a neural network.
•Training can be off-line or on-line (adaptive).
•Any suitable NN can be used.
•After training, validation can be achieved to any desired degree by systematically firing rules and observing system behavior.
•The system dynamics is embedded in the fuzzy rules, not the neural network.
De-fuzzification
Ref.: “A brief course in Fuzzy Logic and Fuzzy Control”
http://www.flll.uni-linz.ac.at/pdw/fuzzy/fuzzy.html
Neuro-fuzzy Navigator
OPTIC FLOW PROCESSOR
SENSOR FUSION & FEATURE EXTRACTION
NEURAL NET NAVIGATOR
Channel 1
GPS
IMU
GPS/INS NAVIGATION
Channel 2Magnetometer
NEURAL NETWORK TRAINING
-
-
+
+
Measured Velocity & Position
Velocity & Position Estimate
Attitude
Attitude
STATE ESTIMATOR
OPTIC FLOW PROCESSOR
SENSOR FUSION & FEATURE EXTRACTION
NEURAL NET NAVIGATOR
Channel 1
GPS
IMU
GPS/INS NAVIGATION
GPS
IMU
GPS/INS NAVIGATION
Channel 2Magnetometer
NEURAL NETWORK TRAINING
NEURAL NETWORK TRAINING
-
-
+
+
Measured Velocity & Position
Velocity & Position Estimate
Attitude
Attitude
STATE ESTIMATOR
Neuro-fuzzy Processing