Post on 11-Jul-2020
University of Minnesota Department of Computer Science and Engineering
Active Sensing for Terrain Active Sensing for Terrain Classification on an Agile RobotClassification on an Agile Robot
Richard VoylesCollaborative Systems Lab
Department of Computer Science and Engineering
University of Minnesota Department of Computer Science and Engineering
OutlineOutline
Motivation: Urban Search and Rescue– TerminatorBot Robotic Platform
NSF Center for Safety, Security, and Rescue RobotsTerrain ClassificationSummary and Future Work
University of Minnesota Department of Computer Science and Engineering
The Platform: The Platform: TerminatorBotTerminatorBot
University of Minnesota Department of Computer Science and Engineering
Emergency Response TrainingEmergency Response Training
Sponsored by NSFIndiana Task Force 1 (FEMA)Robotics Profs
R4: Rescue Robots for Research and Response
University of Minnesota Department of Computer Science and Engineering
Emergency Response RobotsEmergency Response Robots
University of Minnesota Department of Computer Science and Engineering
Standard Tools: Standard Tools: CoreCore--Bored SearchBored Search
University of Minnesota Department of Computer Science and Engineering
TerminatorBotTerminatorBot to Augmentto AugmentCoreCore--Bored SearchBored Search
University of Minnesota Department of Computer Science and Engineering
Custom Custom TerminatorBotTerminatorBotUnder DevelopmentUnder Development
Compact Flash
Interface to PDA
On-board Atmel RISC Microcontroller for Local ControlTeleoperated Through PDA and/or GesturesStructural Tether for Retrieval, Power, and Video/Audio/CmdCommunicationsPop-Up Vision CompactFlash for Full Autonomy with on-board PDA
University of Minnesota Department of Computer Science and Engineering
NSF Center for Safety, Security,NSF Center for Safety, Security,
and Rescue Researchand Rescue Research
A Collaborative Effort between the University of Minnesota and the National Science Foundation to
Join with Local and National Corporations in Research and Development
University of Minnesota Department of Computer Science and Engineering
What is an NSF I/UCRC?What is an NSF I/UCRC?
Cooperative Research Venture between Academe and Industry– Medium-term, Industrially-Relevant Research
Funding– NSF Provides Administrative Money– Industry Provides Seed Research Money– Academe Leverages Seed Money to Attract Federal
Agency Money– Targeted Appropriations may Provide Block Grant
Money
University of Minnesota Department of Computer Science and Engineering
A Year in the Life of SSRRCA Year in the Life of SSRRC……
Spring SymposiumPoster Session½ Day Tutorial
Field Demo & Trade FairStandards Committee Mtg
Fall MeetingReports from prev. yearProposal presentations
PSAC, IAB mtg.
Universities
CompaniesCompanies
Companies
ProposalsProposals
Proposals
Companies
Ranking byPSAC, IAB
FundedProjectsFunded
ProjectsFundedProjects
ProgressPosters
Awards made
Work begins Jan 1
Results
New topics and opportunities
identified
Submitted to I/UCRC early Fall
Work together or in teams to refine topics
University of Minnesota Department of Computer Science and Engineering
Research and Development AgendaResearch and Development Agenda
Robotic Mechanisms for RubbledTerrainSensors and Distributed Sensor NetworksSecure, Low-Power, Wireless NetworksHuman Activity MonitoringReconnaissance and Surveillance
University of Minnesota Department of Computer Science and Engineering
Prospective Member CompaniesProspective Member Companies
Robot CompaniesSensor CompaniesWireless and Networks CompaniesCompanies with Unique Fabrication or Software ExpertiseCompanies Established in the Homeland Security FieldCompanies Eager to Enter the Homeland Security FieldGovernment Labs, Centers, Agencies, etc.
University of Minnesota Department of Computer Science and Engineering
University of MinnesotaUniversity of Minnesota
Technical Capabilities– Robot Design and Prototyping– Sensor Design and Interfacing– Secure Networks– Human Activity Monitoring– Ultra-Wideband Wireless– MEMS Design and Fabrication
Voyles, Papanikolopoulos, Gini, Roumeliotis, Giannakis, plus ???
University of Minnesota Department of Computer Science and Engineering
Robot MechanismsRobot Mechanisms
University of Minnesota Department of Computer Science and Engineering
Heterogeneous SensorsHeterogeneous Sensors
(1) Audio Sensor(2) Vibration Monitor(3) Video Reconnaissance Module
M203grenadelauncher
CollaboratingHeterogeneousAgents
Magnetometer,tiltmeter
Ranger
Scout
(1)
(2)
(3)
Communications,control, planning,reconfiguration,navigation andscout launching
Coordination& data routing
Sensor data
Length: 600mm
CPU, GPS, sensors, actuators, communications array
Video
TerminatorBot
University of Minnesota Department of Computer Science and Engineering
Chemical Plume EstimationChemical Plume Estimation
Problem: Response to Chemical, Biological and Radiological Threats Requires Early Detection of Sources and Diffusion PatternsAnswer: Estimation of Gradients from Sparsely Distributed Sensors can Predict “Hotspot” Locations, Suggest Re-Deployment of Sensors to Optimize Observability, and Predict Evacuation Zones Due to Airborne Drift
University of Minnesota Department of Computer Science and Engineering
Visual Visual ServoingServoing and Estimationand Estimation
University of Minnesota Department of Computer Science and Engineering
Distributed Control SoftwareDistributed Control Software
ARC ARC ARC
RCRC RC RC
Behaviorpriority 1
Behaviorpriority 1
Behaviorpriority 2
60% 20%20%
University of Minnesota Department of Computer Science and Engineering
Activity Recognition Based on Activity Recognition Based on Position and VelocityPosition and VelocityTrack each pedestrian throughout the scene using Kalman filter estimates– Record the position and velocity– Develop a position and velocity
path characteristic for each pedestrian
Send a warning signal under the following conditions:– Pedestrian enters a secured area– Pedestrian moves in “tagged” way
• Application-specific “tags”• Running, loitering, meeting, etc
– Pedestrian’s motion statistics do not meet the norm of crowd behavior
University of Minnesota Department of Computer Science and Engineering
OutlineOutline
Motivation: Urban Search and RescueNSF Center for Safety, Security, and Rescue RobotsTerrain Classification with Agile RobotSummary and Future Work
University of Minnesota Department of Computer Science and Engineering
TerminatorBotTerminatorBot
University of Minnesota Department of Computer Science and Engineering
TerminatorBotTerminatorBot -- Alternate Scout Alternate Scout
Two 3-DoF Arms that Stow Inside BodyDual-Use Arms for both Locomotion and ManipulationFour Locomotion Gait Classes:– “Swimming” Gaits (dry land)– Narrow Passage Gait (no wider
than body)– “Bumpy Wheel” Rolling Gait– “Body-Roll” Dynamic Gait
University of Minnesota Department of Computer Science and Engineering
TerminatorBotTerminatorBot Form FactorForm Factor
Stowed Configuration Deployed Configuration
Hemispherical side for smooth manipulation
Concave claw for traction/digging
University of Minnesota Department of Computer Science and Engineering
Visual Visual ServoingServoing and and TerminatorBotTerminatorBot: Goals: Goals
Visual Servoing for Navigation and Homing– Fixate on (many) distant features and center
(object or FOE) while moving forward (like eye-in-hand)
– Visual odometry (3D estimation)Visual Servoing for Object Manipulation– Body-fixed camera (not eye-in-hand)
Visual Servoing for Terrain Identification
University of Minnesota Department of Computer Science and Engineering
SSD TrackingSSD Tracking
SSD(dx,dy)=Σi, j ∈N [ I(x+dx+i, y+dy+j)-T(x+dx+i, y+dy+j) ]2
University of Minnesota Department of Computer Science and Engineering
RobotRobot’’s Eye Views Eye View
University of Minnesota Department of Computer Science and Engineering
RobotRobot’’s Eye Views Eye View
University of Minnesota Department of Computer Science and Engineering
RobotRobot’’s Eye Views Eye View
University of Minnesota Department of Computer Science and Engineering
Vertical Servo Error for Vertical Servo Error for Different SurfacesDifferent Surfaces
Foam
Peanuts
Hard
University of Minnesota Department of Computer Science and Engineering
RobotRobot’’s Eye Views Eye View
University of Minnesota Department of Computer Science and Engineering
Bounce NormalizationBounce Normalization
2 Features (min)– 3 DOF: Homing,
Terrain, Body RollCompensate for Body Roll Assuming Fixed Features
University of Minnesota Department of Computer Science and Engineering
Terrain Classification ApproachTerrain Classification Approach
Prep
roce
ss
Cla
ssifi
er
00100
cf : Xf : θ X
University of Minnesota Department of Computer Science and Engineering
Feature SpaceFeature SpaceGait Bounce Normalization
f : θ → θ
Fast Fourier Transformf : θ → F
Fast Fourier Transform(segments)
f : θs → Fs
θ = [ θ1 θ2 … θn ]
University of Minnesota Department of Computer Science and Engineering
ClassifiersClassifiersArtificial Neural NetworksBackpropogation with MomentumLogarithmic Sigmoid Transfer Function
Variations in Network Structure
Support Vector Machines
Polynomial(x⋅y)d
Radial Basisen where n = -||x-y||2 / (2σ2)
Sigmoidtanh(κ(x ⋅ y) + Θ)
Discriminant AnalysisLinearQuadraticLogarithmic
University of Minnesota Department of Computer Science and Engineering
Experimental MethodsExperimental Methods
Data Collection
5 terrains (carpet, BBs, woodchips, foam, rock)150+ gait samples collected over each terrain
Feature Vector Raw (Identity Function)FFT all (Frequency Space)FFT segmented (Concatenated Frequency Space)
Classifiers Artificial Neural Networks (x2)Support Vector Machines (x3)Discriminant Analysis (x3)
Evaluation 50 Random Divisions of Sample DataTerrain Classification Rate = mean(mean(c(Xi))t)
University of Minnesota Department of Computer Science and Engineering
Experimental Results Experimental Results –– Raw ClassifiersRaw Classifiers
0102030405060708090
Raw FFTall FFTsec
ann(1)
ann(2)
svm(p)
svm®
svm(s)
da(l)
da(q)
da(g)
University of Minnesota Department of Computer Science and Engineering
Spatiotemporal Patterns and Motion PrimitivesSpatiotemporal Patterns and Motion Primitives
PositionArm
TouchG
round
DragB
ody
DropB
ody
PositionArm
LiftB
ody
University of Minnesota Department of Computer Science and Engineering
HMMsHMMs As ClassifiersAs Classifiers
Train to maximize P(O|λ)by adjusting λ1 parameters A,B
RBRRBYBRRYBBRYRR…BBRYBBRY
Class 1
Train to maximize P(O|λ)by adjusting λ2 parameters A,B
YYBYRRBRBBYBBBRY…RBRBYYBR
Class 2
RBRYBRRBWhich model best fitsthis observation sequence??
University of Minnesota Department of Computer Science and Engineering
Observation SequenceObservation Sequence
o1
f : θ → F preprocesspreprocess preprocess
QDA
o9. . . . . . .
classify classify classify
o2
University of Minnesota Department of Computer Science and Engineering
Experimental MethodsExperimental Methods
X3
O3
All
Sam
ple
Dat
a
Prep
roce
ss
For eachSegmentf : θ → F
QDA ClassifiersQDA Classifiers
QDA ClassifierTrainer
X1Segment Classifier/
HMMTraining Data
HMMTraining Data
Testing Data
X1
X2
O1 O2
HMMTrainer
HMM ClassifiersHMM Classifiers
University of Minnesota Department of Computer Science and Engineering
0102030405060708090
100
Total
Carpet BBs
ChipsFoa
m
Rock
HMM Classification Rates
Experimental ResultsExperimental Results
020406080
100
Total
Carpet BBs
ChipsFoa
mRoc
k
1 Cycle1 Cycle3 Cycles3 Cycles
Classification Over Multiple Gaits
University of Minnesota Department of Computer Science and Engineering
General Applicability Tracked VehicleGeneral Applicability Tracked Vehicle
Sample "Gait" Bounce from Tank
University of Minnesota Department of Computer Science and Engineering
Summary and Future WorkSummary and Future WorkDescribed Search-and-Rescue MotivationDetoured to NSF CenterTerrain Classification– Classification nearly 90% on single gait cycle– Seems generally applicable to non-legged platforms
Future Work:Estimating Physical Terrain ParametersAdapting Gait to Terrain ParametersMulti-Legged, Reconfigurable Mechanisms