USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an...
-
date post
21-Dec-2015 -
Category
Documents
-
view
220 -
download
5
Transcript of USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an...
![Page 1: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/1.jpg)
USARsim & HRI Research
Michael Lewis
![Page 2: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/2.jpg)
Background..
USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in Urban Search & Rescue
Katia Sycara CMU- Multi agent systems
Illah Nourbakhsh CMU/NASA- Field robotics
Mike Lewis Pitt- human control of robotic teams
![Page 3: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/3.jpg)
USARSim Design Objective
Leverage technology developed by the $30 billion/year game industry to focus on building and validating high fidelity models of robots
Piggyback on rapidly evolving technology for game engines Photorealistic graphics to allow study of human-robot interaction and
machine vision Best available physics engines to replicate control/mobility challenges of
real environments Availability of modeling tools to create realistically complex
environments in a reasonable time Compatibility with robotics software such as Player or Pyro and game
content such as America’s Army
![Page 4: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/4.jpg)
Graphics and Vision
The Unreal Engine supports the rapidly evolving graphics acceleration on the new video cards
USARSim’s image server captures in-memory video so that images can be made available for:
Machine vision algorithms Addition of realistic noise and distortion
We are engaged in an on-going validation effort to identify aspects/algorithms that are/are not accurately modeled by the simulation
![Page 5: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/5.jpg)
the PER one of our first robots
![Page 6: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/6.jpg)
Black Arena- Nike Silo fixed reference environment
![Page 7: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/7.jpg)
Red Arena real & simulated
![Page 8: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/8.jpg)
Brief history2002-2003 Developed USARsim simulation
•Limited to our own robots
•Limited to our own (RETSINA) control architecture
2003-2004 Extended simulator for general access
•Added robots widely used in robocup USAR
•Added api’s for Player & Pyro
2004-2005 Began cooperative development
•Involved NIST in maintenance
•Demo competition at robocup in Osaka
2005-2006 Simulation matured
Virtual Robots USAR competition in Bremen
Rationalization of units, modularization of classes, etc.
![Page 9: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/9.jpg)
www.sourceforge.net/projects/usarsim
•Used for Virtual Robots Competition in USAR League
•Maintained by NIST
![Page 10: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/10.jpg)
Fixed Camera Illusion
Can gravity referenced view (GRV) help us maintain awareness of attitude?
Less time Less backtracking
![Page 11: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/11.jpg)
Camera Control Experiment
Video Feed is the strongest perceptual link to remote environment Disorientation Failure to take precaution against hazards Non-detection of mission critical information
![Page 12: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/12.jpg)
Camera Control
Fixed Camera PTZ Camera Dual ptz Cameras
Results
More targets for PTA & dual camera
Dual camera twice as likely to be “disjoint”
![Page 13: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/13.jpg)
MultiRobot ControlFully autonomous cooperation (Machinetta)
Manual
Cooperative
![Page 14: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/14.jpg)
Multi-robot results
More complete searches for autonomous & cooperative
More victims found in cooperative (followed by manual)
Cooperative participants switched more frequently between robots and Frequency of switching correlated with finding
victims
![Page 15: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/15.jpg)
Validation
All robots in USARsim model real robots and so are candidates for validation
Gives indication of the degree to which experimental results might be generalizable
Provides a common reference for comparing experimental results
Provides a mechanism for sharing advances in control code and interfaces among researchers
Provides reassurance that software developed using simulation can be ported to hardware
![Page 16: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/16.jpg)
![Page 17: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/17.jpg)
![Page 18: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/18.jpg)
![Page 19: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/19.jpg)
![Page 20: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/20.jpg)
![Page 21: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/21.jpg)
Sensor validation for vision
Conventional wisdom is that synthetic images are NOT useful for work in machine vision because of intrinsic regularities, etc.
Similar to arguments wrt congruential random number generators
Question should be empirical not theory
![Page 22: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/22.jpg)
Feature Extraction Algorithms
Edge detection Template matching Snakes OCR
Tested in: Camera well lit Camera dimly lit Simulation well lit
![Page 23: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/23.jpg)
Canny Edge Detectionoriginal
Gaussian Filter to remove noise
Sobel operator separatesHigh horizonal/vertical regions
Canny operator with thresholding
![Page 24: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/24.jpg)
Edge Detection Performance
![Page 25: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/25.jpg)
Template MatchingTemplate Image with feature correlation
Inverse of Fourier transform of image x Fourier transform of template
![Page 26: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/26.jpg)
Template Convolution
simulation real camera images
![Page 27: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/27.jpg)
Template convolutiondistance in pixels estimate & target feature
![Page 28: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/28.jpg)
Snake algorithm extraction on simulated image
![Page 29: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/29.jpg)
Snake algorithm extraction on real camera image
![Page 30: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/30.jpg)
Pitt/CMU Validation
Participants controlled either robot or simulation from lab at Pitt
Robot testing was conducted in replica of NIST’s Orange Arena at CMU
Control was either Direct teleoperation or Command where operator specified waypoint
Two robot types, the experimental Personal Explorer Rover (PER) and the Pioneer P3-AT (simulated as P2-AT) were tested
![Page 31: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/31.jpg)
Simple & Complex Navigation Tasks
3 Meters
1 Meter
1 Meter
1 Meter
3 Meters 3 Meters
![Page 32: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/32.jpg)
All Conditions:Task Completion Times
Task Competion Times
0.0
50.0
100.0
150.0
200.0
250.0
300.0
350.0
400.0
Wood, Straight Paper Straight Rock Straight Wood, Complex Paper Complex Rock Complex
PER Direct
PER Sim Direct
PER Command
PER Sim Command
Pioneer Direct
Pioneer Sim Direct
![Page 33: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/33.jpg)
PER Direct vs. Command:Number of Turns
Number of Turns
0.0
5.0
10.0
15.0
20.0
25.0
Wood, Straight Paper Straight Rock Straight Wood, Complex Paper Complex Rock Complex
PER Direct
Sim Direct
PER Command
Sim Command
![Page 34: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/34.jpg)
Direct Control PER & P3-AT: Average Forward Sequence Average F
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
Wood, Straight Wood, Complex Paper Straight Paper Complex Rock Straight Rock Complex
Sec
on
ds
PER
Sim-PER
P3AT
Sim-P3AT
C
![Page 35: USARsim & HRI Research Michael Lewis. Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in.](https://reader037.fdocuments.us/reader037/viewer/2022110207/56649d585503460f94a37cd3/html5/thumbnails/35.jpg)
& now Accelerated Physics
Next engine release will support Aegia PhysX Continually improving simulation quality (ex: 3 order of
magnitude improvement in physics with hardware acceleration) & validation
Let us do tracked robots, collapsing buildings, etc.