From Teleoperation to Autonomy

Introduction to AI Robotics (MIT Press) Chapter 1 1

1 From Teleoperation to Autonomy

• Define Intelligent Robot

• Be able to describe at least two differences between AI and engineering approaches to robotics

• Be able to describe the difference between telepresence and semi-autonomous control

• Have some feel for the history and societal impact of robotics

History-AI-EngineeringTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview


1 Intelligent Robot

• Mechanical creature which can function autonomously

– Mechanical= built, constructed

– Creature= think of it as an entity with its own motivation, decision making processes

– Function autonomously= can sense, act, maybe even reason; doesn’t just do the same thing over and over like automation

History-AI-EngineeringTeleopCase StudiesProgrammingSummaryReview


1 What are Robots?• Autonomous mechanical creatures

– Capek 1921: R.U.R.

• Intelligent because teleoperation doesn’t work, doesn’t scale

• Physically situated, but now software agents or softbots– Principles from robotics influenced AI

community, esp. planning

– Combines programming, networks, operating systems, algorithms, … everything about CS into a system (the ultimate software engineering project)

www.fradulent.org/rur.htm



1 Robots Constantly in the Press

www.sony.com

www.irobot.com

courtesy of Honda

courtesy of MIT AI Lab


KismetAsimo

AIBO


1 Less Famous Cousins at WTC

Inuktun microTracks

½ iRobot PackBot

iRobot PackBot at WTC


1 Why Robots? Dirty, Dangerous, Dull Tasks

• Joint Vision 2010, TRADOC (Army), Joint Forces COM, all branches even down to the organic level– Reconnaissance, Military Operations on Urbanized Terrain (MOUT),

denial of area, consequence management, logistics, demining

Replace Humans with Robots

www.friendlymachines.com



1 Why Robots? Better Than Bio

• Robots at WTC…– voids smaller than person

could enter

– voids on fire or oxygen depleted

• nuclear/biological/chemical (NBC) Response– Lose ½ cognitive attention

with each level of protection• Level A=12.5% of normal

ability

Do Things that Living Things Can’t

Void on fire

Void:1’x2.5’x60’History-AI-EngineeringTeleopCase StudiesProgrammingSummaryReview


1 Major Robot Modalities: UAV, UGV, UUV

• Unmanned Aerial Vehicles– drones since Vietnam: Global Hawk, UCAV

– easy: nothing to hit

– hard: mission sensing, human-in-the-loop control

• Unmanned Ground Vehicles– since 1967

– easy: can always stop and think, a priori maps

– hard: perceiving, e.g., light vegetation vs. wall

• Unmanned Underwater Vehicles

– Remotely Operated Vehicles (ROVs) since 1960s

– easy: run tethers

– hard: platform operation in unfriendly environment

Mobility (platform), Perception,Communications +HRI, Control (Intelligence), Power



1 A Brief History…


1 Industrial Manipulators

• “Tommy” type of robots: deaf, dumb, and blind

• High precision, fast repetition

• Usually no sensing of the environment– Welding can be off by an inch…



1 3 Ways of Controlling a Robot

• “RC-ing”– you control the robot

– you can view the robot and it’s relationship to the environment

– ex. radio controlled cars, bomb robots

– operator isn’t removed from scene, not very safe

HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview



• teleoperation– you control the robot

– you can only view the environment through the robot’s eyes

– don’t have to figure out AI


Video

Foster-Miller Talon



• semi- or full autonomy– you might control the robot sometimes

– you can only view the environment through the robot’s eyes

– ex. Sojouner with different modes

– human doesn’t have to do everything



1 Components of a Telesystem(after Uttal 89)

• Local – display

– Local control device

• Communication• Remote

– sensor

– mobility

– effector

– power

• Local – display

– Local control device

• Communication• Remote

– sensor

– mobility

– effector

– power

Display

Control

Sensor

Mobility

Effector

Power

Communi-cation

Local

Remote


1 Example

Local

Remote



1 Typical Run


video


1 Problems That You Saw• no feedback, couldn’t really tell that the robot was

stuck but finally got free– robot doesn’t have “proprioception” or internal sensing to tell

you what the flippers were doing. No crunching noises, no pose widget to show the flippers

• no localization, mapping-> no idea how far traveled• partial solution: better instrumentation (but can’t do dead

reckoning well)

– operator doesn’t have an external viewpoint to show itself relative to the environment

• solution: two robots, one to spot the other

• communications dropout, even though ~3 meters away• lighting conditions went from dark to very bright

– hard for computer vision or human to adjust



1 But good for unmodeled events



1 Communications is Important:DarkStar+7 seconds=DarkSpot

• 7 second communications lag (satellite relay)

• “interruption” lag on part of operator

HistoryTeleop-Motivation-Components-Problems-AlternativesCase StudiesProgrammingSummaryReview DarkStar


1 Predator:~7:1 human to robot ratio

• 4 people to control it (52-56 weeks of training)– one for flying

– two for instruments

– one for landing/takeoff

• plus maintenance, sensor processing and routing

• lack of self-awareness– in Kosovo, come along side in helicopter and shoot down

Leo’s unofficialPredator page



1 Summary of Teleop Problems

• cognitive fatigue

• communications dropout

• communications bandwidth

• communications lag

• too many people to run one robot (hidden cost)



1 Telesystems Best Suited For:

• the tasks are unstructured and not repetitive

• the task workspace cannot be engineered to permit the use of industrial manipulators

• key portions of the task require dexterous manipulation, especially hand-eye coordination, but not continuously

• key portions of the task require object recognition or situational awareness

• the needs of the display technology do not exceed the limitations of the communication link (bandwidth, time delays)

• the availability of trained personnel is not an issue



1 Teleop Improvements: Telepresence

• Telepresence– improves human control, reduces simulator sickness

and cognitive fatigue by providing sensory feedback to the point that teleoperator feels they are “present” in robot’s environment

– increases demands on bandwidth



1 Teleop Improvements:Supervisory Control

• Semi-autonomous– Supervisory Control

• human is involved, but routine or “safe” portions of the task are handled autonomously by the robot

• is really a type of mixed-initiative

• Shared Control/ Guarded Control– human initiates action, interacts with remote by adding

perceptual inputs or feedback, and interrupts execution as needed

– robot may “protect” itself by not bumping into things

• Traded Control– human initiates action, does not interact



1 Teleop Improvements:Mixed-Initiative

• Levels of Initiative

– do only what told to do (teleoperation)

– recommend or augment (cognitive augmentation)

– act and report

– act on own and supervise itself (autonomy)



1 “No Hands Across America”

• 1994

• CMU NavLab

• Pittsburgh to San Diego– 2897 miles total

– 2849 autonomously

• Autonomous or Mixed-Initiative?



1 Collaborative Teleoperation

Urban is stuck, Inuktun can’t help from current perspective

1. Driven off 3rd floor2. Hoisted to 2nd floor by tether3. Has better view, changing

configuration & rocking extend view

mpg: June 2, 2000 SRDR Miami Beach: view from Inuktun as it falls mpg: June 2, 2000 SRDR Miami Beach: view from Inuktun from hoisted position

1

2

3

still: June 2, 2000 SRDR Miami Beach


1 2000 AAAI Mobile Robot

• 2 robots helping each other reduced collision errors, sped up time navigating confined space, righting

HistoryTeleopCase StudiesProgrammingSummaryReview


1 Example:Mixed-Initiative & Collab. Teleop

• 9/2000 DARPA Tactical Mobile Robots demonstration

• Robot used an intelligent assistant agent to look for signs of snipers hiding in urban rubble

– motion– skin color – difference in color– thermal (IR camera)

• Human navigated mother robot using viewpoint of 2nd robot (not in picture)

• Once deposited the human moved the daughter robot, and either saw a sniper or was alerted by the agent



1 AI provides the “other stuff”

• knowledge representation• understanding natural language• learning• planning and problem solving• inference• search• vision



1 Example User Expectation of AI

• Proposed Goal: 1:1 soldier:any robot, where 1 soldier is responsible for 1 or more active robots but does not have to pay continuous attention to them.

4 specialists:1 vehicle

1 specialist:1 vehicle

1 specialist:1 modality 1 specialist 1 soldier

1 specialist:n vehicles

MAV-UGVcooperativemonitoring

Flocks of MAVs

UAVs astheater assets

MAVs asorganic

assets

Field recon-figurable

UUVs

Young Frankenstein


1 More Reasonable Expectionsagents with“tactical”autonomy,toolkits

Mass-produceddedicated

agents

Field-reconfigurable

agents

Cooperating“pack” or “herd”

agents

Vehicle success isstill based on human,but robot is “in front”

Human intermittentattention as teamcoordinator, notwith individuals

Human primaryresponsibility as a

tool builder, expertadvisor. Peer-level

communication

Consolidation

Dedicated Autonomy Systems

Reconfigurable AutonomySystems



1 Programming Notes• You always need telesystem or human intervention as a

backup – at some point a human will need to take control– embed in your design

• “Roboticists automate what is easy and leave the rest to the human”- Don Norman

• The user interface is absolutely critical– User interface make up 60% of commercial code– Useful= is the program purpose useful?

• usually given to designer via specifications and requirements

– Usable= can a human use it efficiently?• designer must conduct usability studies • avoid “if I can use it, some one else will”


See RI paper


1 Example of How an “Internal” Display Can Hurt

• gamer joystick plus laptop with video & audio• robot state: battery, comms, orientation, camera, encoders• was not used on rubble pile at WTC because it scared off rescuers: too complicated, too long to boot,

too toy– now integrated with Land Warrior– used in Afghanistan

iRobot PackBotvideo, FLIR, 2 way audio



1 Summary

• Teleoperation arose a partial solution to autonomy– cognitive fatigue, high comms bandwidth, long

delays, and many:one human to robot ratios– Telepresence tries to reduce cognitive fatigue

through enhanced immersive environments– Semi-autonomy tries to reduce fatigue, bandwidth by

delegating portions of the task to robot• mixed-initiative

• Teleop isn’t simple and improvements aren’t just “better user interfaces”



1 Review Questions

• What is an intelligent robot?

• What is the difference between engineering and AI robotics?

• What are 3 types of control?

• What are the parts of a telesystem?

• What are problems with teleoperation?

• What’s the difference between telepresence and semi-autonomous control?

• What are the levels of initiative (mixed-initiative)?

• What are alternatives to traditional teleoperation?


From Teleoperation to Autonomy

Documents

Transcript of From Teleoperation to Autonomy