Active Sensing for Terrain Classification on an Agile Robot · Voyles, Papanikolopoulos, Gini,...

University of Minnesota Department of Computer Science and Engineering

Active Sensing for Terrain Active Sensing for Terrain Classification on an Agile RobotClassification on an Agile Robot

Richard VoylesCollaborative Systems Lab

Department of Computer Science and Engineering


OutlineOutline

Motivation: Urban Search and Rescue– TerminatorBot Robotic Platform

NSF Center for Safety, Security, and Rescue RobotsTerrain ClassificationSummary and Future Work


The Platform: The Platform: TerminatorBotTerminatorBot


Emergency Response TrainingEmergency Response Training

Sponsored by NSFIndiana Task Force 1 (FEMA)Robotics Profs

R4: Rescue Robots for Research and Response


Emergency Response RobotsEmergency Response Robots


Standard Tools: Standard Tools: CoreCore--Bored SearchBored Search


TerminatorBotTerminatorBot to Augmentto AugmentCoreCore--Bored SearchBored Search


Custom Custom TerminatorBotTerminatorBotUnder DevelopmentUnder Development

Compact Flash

Interface to PDA

On-board Atmel RISC Microcontroller for Local ControlTeleoperated Through PDA and/or GesturesStructural Tether for Retrieval, Power, and Video/Audio/CmdCommunicationsPop-Up Vision CompactFlash for Full Autonomy with on-board PDA


NSF Center for Safety, Security,NSF Center for Safety, Security,

and Rescue Researchand Rescue Research

A Collaborative Effort between the University of Minnesota and the National Science Foundation to

Join with Local and National Corporations in Research and Development


What is an NSF I/UCRC?What is an NSF I/UCRC?

Cooperative Research Venture between Academe and Industry– Medium-term, Industrially-Relevant Research

Funding– NSF Provides Administrative Money– Industry Provides Seed Research Money– Academe Leverages Seed Money to Attract Federal

Agency Money– Targeted Appropriations may Provide Block Grant

Money


A Year in the Life of SSRRCA Year in the Life of SSRRC……

Spring SymposiumPoster Session½ Day Tutorial

Field Demo & Trade FairStandards Committee Mtg

Fall MeetingReports from prev. yearProposal presentations

PSAC, IAB mtg.

Universities

CompaniesCompanies

Companies

ProposalsProposals

Proposals

Companies

Ranking byPSAC, IAB

FundedProjectsFunded

ProjectsFundedProjects

ProgressPosters

Awards made

Work begins Jan 1

Results

New topics and opportunities

identified

Submitted to I/UCRC early Fall

Work together or in teams to refine topics


Research and Development AgendaResearch and Development Agenda

Robotic Mechanisms for RubbledTerrainSensors and Distributed Sensor NetworksSecure, Low-Power, Wireless NetworksHuman Activity MonitoringReconnaissance and Surveillance


Prospective Member CompaniesProspective Member Companies

Robot CompaniesSensor CompaniesWireless and Networks CompaniesCompanies with Unique Fabrication or Software ExpertiseCompanies Established in the Homeland Security FieldCompanies Eager to Enter the Homeland Security FieldGovernment Labs, Centers, Agencies, etc.


University of MinnesotaUniversity of Minnesota

Technical Capabilities– Robot Design and Prototyping– Sensor Design and Interfacing– Secure Networks– Human Activity Monitoring– Ultra-Wideband Wireless– MEMS Design and Fabrication

Voyles, Papanikolopoulos, Gini, Roumeliotis, Giannakis, plus ???


Robot MechanismsRobot Mechanisms


Heterogeneous SensorsHeterogeneous Sensors

(1) Audio Sensor(2) Vibration Monitor(3) Video Reconnaissance Module

M203grenadelauncher

CollaboratingHeterogeneousAgents

Magnetometer,tiltmeter

Ranger

Scout

(1)

(2)

(3)

Communications,control, planning,reconfiguration,navigation andscout launching

Coordination& data routing

Sensor data

Length: 600mm

CPU, GPS, sensors, actuators, communications array

Video

TerminatorBot


Chemical Plume EstimationChemical Plume Estimation

Problem: Response to Chemical, Biological and Radiological Threats Requires Early Detection of Sources and Diffusion PatternsAnswer: Estimation of Gradients from Sparsely Distributed Sensors can Predict “Hotspot” Locations, Suggest Re-Deployment of Sensors to Optimize Observability, and Predict Evacuation Zones Due to Airborne Drift


Visual Visual ServoingServoing and Estimationand Estimation


Distributed Control SoftwareDistributed Control Software

ARC ARC ARC

RCRC RC RC

Behaviorpriority 1

Behaviorpriority 1

Behaviorpriority 2

60% 20%20%


Activity Recognition Based on Activity Recognition Based on Position and VelocityPosition and VelocityTrack each pedestrian throughout the scene using Kalman filter estimates– Record the position and velocity– Develop a position and velocity

path characteristic for each pedestrian

Send a warning signal under the following conditions:– Pedestrian enters a secured area– Pedestrian moves in “tagged” way

• Application-specific “tags”• Running, loitering, meeting, etc

– Pedestrian’s motion statistics do not meet the norm of crowd behavior


OutlineOutline

Motivation: Urban Search and RescueNSF Center for Safety, Security, and Rescue RobotsTerrain Classification with Agile RobotSummary and Future Work


TerminatorBotTerminatorBot


TerminatorBotTerminatorBot -- Alternate Scout Alternate Scout

Two 3-DoF Arms that Stow Inside BodyDual-Use Arms for both Locomotion and ManipulationFour Locomotion Gait Classes:– “Swimming” Gaits (dry land)– Narrow Passage Gait (no wider

than body)– “Bumpy Wheel” Rolling Gait– “Body-Roll” Dynamic Gait


TerminatorBotTerminatorBot Form FactorForm Factor

Stowed Configuration Deployed Configuration

Hemispherical side for smooth manipulation

Concave claw for traction/digging


Visual Visual ServoingServoing and and TerminatorBotTerminatorBot: Goals: Goals

Visual Servoing for Navigation and Homing– Fixate on (many) distant features and center

(object or FOE) while moving forward (like eye-in-hand)

– Visual odometry (3D estimation)Visual Servoing for Object Manipulation– Body-fixed camera (not eye-in-hand)

Visual Servoing for Terrain Identification


SSD TrackingSSD Tracking

SSD(dx,dy)=Σi, j ∈N [ I(x+dx+i, y+dy+j)-T(x+dx+i, y+dy+j) ]2


RobotRobot’’s Eye Views Eye View


Vertical Servo Error for Vertical Servo Error for Different SurfacesDifferent Surfaces

Foam

Peanuts

Hard


RobotRobot’’s Eye Views Eye View


Bounce NormalizationBounce Normalization

2 Features (min)– 3 DOF: Homing,

Terrain, Body RollCompensate for Body Roll Assuming Fixed Features


Terrain Classification ApproachTerrain Classification Approach

Prep

roce

ss

Cla

ssifi

er

00100

cf : Xf : θ X


Feature SpaceFeature SpaceGait Bounce Normalization

f : θ → θ

Fast Fourier Transformf : θ → F

Fast Fourier Transform(segments)

f : θs → Fs

θ = [ θ1 θ2 … θn ]


ClassifiersClassifiersArtificial Neural NetworksBackpropogation with MomentumLogarithmic Sigmoid Transfer Function

Variations in Network Structure

Support Vector Machines

Polynomial(x⋅y)d

Radial Basisen where n = -||x-y||2 / (2σ2)

Sigmoidtanh(κ(x ⋅ y) + Θ)

Discriminant AnalysisLinearQuadraticLogarithmic


Experimental MethodsExperimental Methods

Data Collection

5 terrains (carpet, BBs, woodchips, foam, rock)150+ gait samples collected over each terrain

Feature Vector Raw (Identity Function)FFT all (Frequency Space)FFT segmented (Concatenated Frequency Space)

Classifiers Artificial Neural Networks (x2)Support Vector Machines (x3)Discriminant Analysis (x3)

Evaluation 50 Random Divisions of Sample DataTerrain Classification Rate = mean(mean(c(Xi))t)


Experimental Results Experimental Results –– Raw ClassifiersRaw Classifiers

0102030405060708090

Raw FFTall FFTsec

ann(1)

ann(2)

svm(p)

svm®

svm(s)

da(l)

da(q)

da(g)


Spatiotemporal Patterns and Motion PrimitivesSpatiotemporal Patterns and Motion Primitives

PositionArm

TouchG

round

DragB

ody

DropB

ody

PositionArm

LiftB

ody


HMMsHMMs As ClassifiersAs Classifiers

Train to maximize P(O|λ)by adjusting λ1 parameters A,B

RBRRBYBRRYBBRYRR…BBRYBBRY

Class 1

Train to maximize P(O|λ)by adjusting λ2 parameters A,B

YYBYRRBRBBYBBBRY…RBRBYYBR

Class 2

RBRYBRRBWhich model best fitsthis observation sequence??


Observation SequenceObservation Sequence

o1

f : θ → F preprocesspreprocess preprocess

QDA

o9. . . . . . .

classify classify classify

o2


Experimental MethodsExperimental Methods

X3

O3

All

Sam

ple

Dat

a

Prep

roce

ss

For eachSegmentf : θ → F

QDA ClassifiersQDA Classifiers

QDA ClassifierTrainer

X1Segment Classifier/

HMMTraining Data

HMMTraining Data

Testing Data

X1

X2

O1 O2

HMMTrainer

HMM ClassifiersHMM Classifiers


0102030405060708090

100

Total

Carpet BBs

ChipsFoa

m

Rock

HMM Classification Rates

Experimental ResultsExperimental Results

020406080

100

Total

Carpet BBs

ChipsFoa

mRoc

k

1 Cycle1 Cycle3 Cycles3 Cycles

Classification Over Multiple Gaits


General Applicability Tracked VehicleGeneral Applicability Tracked Vehicle

Sample "Gait" Bounce from Tank


Summary and Future WorkSummary and Future WorkDescribed Search-and-Rescue MotivationDetoured to NSF CenterTerrain Classification– Classification nearly 90% on single gait cycle– Seems generally applicable to non-legged platforms

Future Work:Estimating Physical Terrain ParametersAdapting Gait to Terrain ParametersMulti-Legged, Reconfigurable Mechanisms

Active Sensing for Terrain Classification on an Agile Robot · Voyles, Papanikolopoulos, Gini,...

Documents

Transcript of Active Sensing for Terrain Classification on an Agile Robot · Voyles, Papanikolopoulos, Gini,...