Understanding Multi-Instrument Measurement Systems
Transcript of Understanding Multi-Instrument Measurement Systems
![Page 1: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/1.jpg)
Peter Bajcsy, Ph.D.Research Scientist, ALG NCSAAdjunct Assistant Professor, CS and ECE Departments, UIUC
Understanding Multi-Instrument Measurement Systems
![Page 2: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/2.jpg)
2
Research Focus
Theoretical modeling and experimental understanding of multi-instrument measurement systems that deal with multi-dimensional multi-variate dataand include sensor networks, wireless communication, data acquisition, fusion, spectral & spatial analysis and synthesis components
![Page 3: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/3.jpg)
3
R&D Components
Sensors
Data Fusion
Analyses
Syntheses
Data Acquisition
Building SolutionsSensorsData AcquisitionData FusionAnalysesSyntheses
![Page 4: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/4.jpg)
4
R&D Components: SensorsMeasurement Sensors
Measured physical entities, e.g., displacement, shear, temperature, reflectance, orientation
Positioned as satellite, aerial or ground sensors
Attached to moving or stationary objects
Connected as wireless or wired sensors
Sensors of Interest
Spectral sensors: EO, IR, HS, UV, SAR, X-ray, OCT, MRI, CT
Raster and point sensors“Smart” MEMS sensors
Sensors
Data Fusion
Analyses
Syntheses
Data Acquisition
![Page 5: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/5.jpg)
5
Hardware EquipmentCameras
Hyperspectral (visible & near IR), multispectral (R,G,B, near IR), thermal IR, visible (wireless)
Robot with armOrientation sensors IS300 ProCrossbow MEMS (wireless)Audio (wireless)
![Page 6: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/6.jpg)
6
EXAMPLE:Toward Hazard Aware Spaces
![Page 7: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/7.jpg)
7
Problem Description
Deploy, Sense, Calibrate, Analyze, Trigger,
Confirm, Alarm
![Page 8: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/8.jpg)
8
Problem DescriptionGiven wired and wireless multi-spectral and other sensors,
assess hazards in an indoor environment.Data acquisition:
Wired multi-spectral sensors – thermal infrared, hyperspectral, visible and multi-spectral camerasMicro-Electro-Mechanical Systems (MEMS) with CPU – MICA sensors with tinyOS
Wireless data transmission and receptionSensor registrationSensor calibrationData fusionHazard detection
![Page 9: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/9.jpg)
9
Motivation & ChallengesMotivation:
Proactive surveillanceBandwidth requirement reductionSpatially adaptive indoor hazard detection
Challenges: Sensor selection (ruggedness, weight, capabilities)Power/CPU/Memory constraintsWireless communication limited range, delays and packet losesSensor deploymentOptimal sensor layoutSensor and camera calibrationCamera controlHazard event recognition (image and signal analysis)Sensor based confirmationAlarm propagation
![Page 10: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/10.jpg)
10
Deployable Sensor Network
![Page 11: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/11.jpg)
11
Optimal MICA Sensor LayoutWhat is the most optimal “smart”MEMS system design?
What is the most effective mechanism to receive information from the wireless sensor network?How to synchronize data acquisition from all “smart” MEMS sensors?How to assign time stamps to every sensor reading?Is there an optimal spatial arrangement of sensor networks?What is the impact of other wireless devices on sensor networks?
![Page 12: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/12.jpg)
12
Thermal IR Camera CalibrationThermal IR Image
uncalibrated
calibrated
Color Image
Engineering Units [degrees Celsius]
Goal: Information Content
![Page 13: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/13.jpg)
13
R&D Components: AcquisitionData Acquisition
Sensor SetupsOrientation of sensorsPlacement distribution, e.g., density, of sensorsDistance from objects of interestField of view and spatial resolutionSensor attachment mechanismInterference of multiple sensorsExternal dependencies, e.g., light sources, media propagation
Synchronization of multiple sensorsTriggers and timersTemporal resolution
Wireless data transmissionSpatial coveragePacket lossBandwidth and frequency bandsCommunication protocols
Input DataPhysical entityData type (raster or vector types)Datum precision
Data of Interest
Any combination of miscellaneous data sets combined with image data
Sensors
Data Fusion
Analyses
Syntheses
Data Acquisition
![Page 14: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/14.jpg)
14
EXAMPLE:Robot Control Using Arm Gestures
![Page 15: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/15.jpg)
15
Problem StatementRecognize human gestures according to a gesture lexicon for controlling remotely autonomous vehiclesConstraints:
Environment: Limited line of sight, EM Interference Acoustic noise, Motion (Catapult and shock vibrations, Aircraft deck motion ~5-6° roll), Many emittersOperation Conditions of Yellow Shirts: YS is in all weather conditions, YS is on the deck 14-16 hours, YS is exposed to temperatures –10°F to 140°F
-150
-100
-50
0
50
100
0 2000 4000 6000 8000 10000 12000
Gesture Lexicon Command:Move Ahead
Robot Instructions:Set Velocity X
Sensor Output Robot Action
![Page 16: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/16.jpg)
16
Motivation & ChallengesMotivation:
Allow UAVs to mix seamlessly with the existing traffic on an aircraft carriers deck
Challenges: Sensor selection: active & passiveSensor attachmentData acquisition and measurement repeatabilityAutomatic classification of sensed arm movements into distinct hand signalsReal-time parsing and classificationTranslation of hand gesture sequence into a sequence of autonomous vehicle commands Execution feedback
![Page 17: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/17.jpg)
17
Developed Solution
![Page 18: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/18.jpg)
18
Real-Time Classification Example: Turn to Left
![Page 19: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/19.jpg)
19
R&D Components: FusionData Fusion
Transformation of Measured VariablesAnalytical and Statistical Relationships
Coordinate System RegistrationRaster & Raster/ Vector & Raster/ Vector & VectorGeoreferencing & Medical Atlas Registration & Generic RegistrationAutomation
Spatial Interpolation/Extrapolation of VariablesInterpolation/Extrapolation ModelsBoundary Conditions (constraints)
Temporal Interpolation/Extrapolation of VariablesInterpolation/Extrapolation ModelsStationary & Non-stationary variables
Uncertainty Modeling & FusionPropagation errorsModeling errorsInterpolation/Extrapolation errors
VisualizationData OverlaysUncertainty maps
Fusion of Interest
Any combination of raster, boundary and point data
Sensors
Data Fusion
Analyses
Syntheses
Data Acquisition
![Page 20: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/20.jpg)
20
EXAMPLE:Data Fusion of Raster and Vector Data Using Uncertainty Modeling
![Page 21: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/21.jpg)
21
Problem DescriptionGiven Multiple Advanced Sensors
Measuring Point and Raster Types of Physical Entities, integrate all measurements and compare them with simulations
• Transformation of measured variables• Sensor registration (coordinate system
registration)• Continuous approximation of measured
physical entity • Data fusion• Location dependent
accuracy/confidence prediction• Visualization and comparison of
measured and simulated data
LoadWall
![Page 22: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/22.jpg)
22
Sensor Fusion
Interpolated ImageFrom Vector Data
Raster Image
Image Mask
SelectRasterData
Select InterpolatedData
![Page 23: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/23.jpg)
23
R&D Components: AnalysesData Analyses
Understanding Measurements & Problem Statement & Constraints & Solution Requirements
Underlying physics, e.g., Maxwell EquationsMeasurement constraintsRepeatability of measurementsReliability of measurements
Feature ExtractionFeature SelectionFeature Processing & Data Mining & Pattern Recognition
Supervised or unsupervisedTesting and verification
Output DataPhysical entityResult accuracyResult validation
Data Analyses of Interest
Any application that involves scene understanding, knowledge mining, or optimal decision making
Sensors
Data Fusion
Analyses
Syntheses
Data Acquisition
![Page 24: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/24.jpg)
24
R&D Components: SynthesesData Syntheses
Understanding Phenomenology Data driven prediction & Physics based modelingData variability & phenomenon variability
ModelingModel dimensionalityFeature selectionComputational tradeoffsStatistical & Analytical modeling
Output DataPhysical entityValidation metricsSpatial, temporal and spectral accuracy
Data Syntheses of Interest
Spectral scene modeling using BRDFs
Environmental predictions
Atmospheric forecast
Sensors
Data Fusion
Analyses
Syntheses
Data Acquisition
![Page 25: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/25.jpg)
25
EXAMPLE:Spectral Scene Modeling Using BRDFs
![Page 26: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/26.jpg)
26
Problem Statement
![Page 27: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/27.jpg)
27
Challenges: Geometric & Photometric Properties
specular surfaces (artificial grass)diffuse surfaces (concrete)isotropic surfaces (leather)anisotropic surfaces (straw)surfaces with large height variations (pebbles)surfaces with small height variations (sandpaper)pastel surfaces (cotton)colored surfaces (velvet)natural surfaces (lettuce)man-made surfaces (sponge)
![Page 28: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/28.jpg)
28
Challenges: Outdoor Environment
Wheat Canopy– May 8, 1999, 55° Solar Zenith Angle, (36.77 N; 97.13 W) near Ponca City, Oklahoma
Parallel to the principal plane of the Sun
Perpendicular to the principal plane of the Sun
15 75-75 -15
15
-15
-75
15
75
![Page 29: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/29.jpg)
29
Reflectance Modeling
Reflectance [ ,%]
( ); ( ); ( 2 ); ( );( ); ( 2 ); (2 ); ( );(2 2 )
reflected
incident
i r i r i i r
i r i r i r
dBρ
ρ ω ω ρ ω ω ρ ω π ρ ω ωρ ω ω ρ ω π ρ π ω ρ ω ωρ π π
Φ= Φ
→ →∆ → ∆ →∆ →∆ ∆ → → →∆
→
![Page 30: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/30.jpg)
30
BRDF Definition
Helmholtz Reciprocity Principle( ) ( )
Isotropic Surface( , ; , ) ( , ; , )
r i r r r i
r i i r r r i i r r
f f
f f
ω ω ω ω
θ φ φ θ φ φ θ φ θ φ
→ = →
+ + =
1( , ): ( , ; , ) [ ]( , ) cos
r r rr i i r r
i i i i i
LBRDF f srL d
θ φθ φ θ φθ φ θ ω
−=
1( , ; ): ( , ; , ; ) [ ]( , ; ) cos
r r rr i i r r
i i i i i
LBRDF f srL d
θ φ λθ φ θ φ λθ φ λ θ ω
−=
1( , ; , ): ( , ; , ; , ) [ ]( , ; , ) cos
r r r i rr i i r r i r
i i i i r i i
LBRDF f srL d
θ φ λ λθ φ θ φ λ λθ φ λ λ θ ω
−=Fluorescent surfaces
Wavelength dependent absorbent surfaces
![Page 31: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/31.jpg)
31
Aerial Imagery: Verification
Mask Generated Using Isodata Clustering With 8 Mask Labels
Measured
Synthetic
Mask
![Page 32: Understanding Multi-Instrument Measurement Systems](https://reader033.fdocuments.us/reader033/viewer/2022050604/62730815c6a9de68cd44c818/html5/thumbnails/32.jpg)
32
SummaryUnderstanding of Multi-Instrument Measurement Systems
Cover very broad range of problemsProblems consists of fundamental theoretical issues, experimental issues and applied issuesFrom sensors to knowledge – complex systemsData heterogeneity and dimensionality – processing and understanding challenges
Ubiquitous applicationsSensor-tools (ad-hoc networks, hazard aware spaces)Fusion-tools (Structural engineering, GIS, Medicine)
BioMed-tools (Bio-informatics, Medicine, Biology, Plant Science)Geo-tools (GIS, Remote Sensing, Agriculture, Hydrology, Water Quality Survey, Atmospheric Science, Military)Sim-tools (Military, Structural Engineering, Atmospheric Science, Agriculture)Edu-tools (Education)