NASA FISO Presentation: Exploration Telepresence: Value and Challenges

Exploration Telepresence: Value and Challenges

Dan Lester University of Texas

FISO Telecon

May 27, 2015

The cognitive mandate for telepresence!depends on !

control latency ! control bandwidth!

Telepresence and human space flight!as enablers for space exploration.!

Human-robot collaboration!

Human cognition makes use of presence!in distant places. Cell phone, Skype, etc.!

Once achievable only with human bodies. !Congress endorses human presence in spaceflight.!

! ! ! ! !Telepresence!

FISO Telecon 5/27/15 2

FISO Telecon 5/27/15

The idea of space telepresence was considered in some detail at the Exploration Telerobotics Symposium, May 2-3, 2012 at GSFC.

This symposium explored

Space Exploration via Telepresence: A New Paradigm for Human-Robotic Cooperation

See presentations and final report at

http://telerobotics.gsfc.nasa.gov

3


Control bandwidth for quality telepresence?! Human vision ~10 Mb/s per eye! A bit more with haptics, dexterity, and audio!

(with compression, and some generosity, lets say 1 Mb/s)!

Control latency for quality telepresence?! Threshold for latency detection is ~50 ms but !Human reaction time for visual stimulus is ~200 ms!

That defines a cognitive horizon of c * 200 ms / 2 ~ 30,000 km!

But everywhere we want to go is way beyond 30,000 km from Earth!!

Earth-Moon 2.6 seconds Earth-Mars 8-40 minutes!

WE CANT HAVE QUALITY TELEPRESENCE AT THESE !LOCATIONS FROM THE EARTH. BUT WE CAN IF WE GET CLOSE.!

Recipe for Telepresence!

4


But lets talk about latency!

5

Dave Scotts Apollo 15 stopwatch


Whats wrong with a little control latency, anyway?!For driving around, and avoiding obstacles, not much. !(10-20s latency endured by Lunokhod controllers made driving around hard, but they did it) !

Few careful assessments of the latency penalty on cognition for >1 second.!

Oscillopscia (perceptual instability relative to inertial and gravitational frame of reference)!and dynamic head-tracking errors. Latency screws with your brain. !

Allison et al. (2001, 2004) assessed !perceived stability by a user with a stereo!head-mounted display while panning head.!

Additional tracking delay (abcissa) is the!added latency between the motion of their!head, and the motion of the image.!

100ms two-way latency isnt that bad.!Just go slow.

22.5 deg/s

45 deg/s

90 deg/s

6


Whats wrong with a little control latency, if were trying to do something? Well, it depends on what youre trying to do. If youre just trying to guide a 7 DOF arm to do a positioning task, its not that hard, but there are latency penalties.!

Lane 2001 saw that the required completion time using an unmitigated situation, where the user simply saw what was going on, rises steeply with latency. !

It helps if you have predictive assistance, and also if you have a graphical indication of what you commanded. !

7


Whats wrong with a little control latency, if were trying to do something? It depends on how hard it is.

Piece of telerobotic history!-- work of Bill Ferrell, back in the 1960s. !

Simple 2D minimal manipulator.!

Task completion time versus latency and difficulty.!

Difficulty defined as how far you had !to move, divided by how precisely you got there. !

Not surprisingly task completion time depends !critically on latency and difficulty. !

8


Whats wrong with a little control latency, if were trying to do something? How about doing really hard stuff?!

Telerobotic surgery is exquisitely difficult. High-dexterity w/ precision cutting and stitching!of compliant material. Low error tolerance. !

A LOT of work assessing latency penalty. 500ms is considered practical limit.!

Assessment by Rayman 2006 !task time for average of four!laproscopic surgery manipulations.!

Things get bad fast beyond 500ms.!

Errors also go up quickly.!

Lesson of telerobotic surgery is!for high dexterity operations, 500ms !is a practical limit for latency.!

9


and then there is easy stuff that is hard (unlike surgery)

The NEEMO crew experimented with undersea habitat-to-shore telerobotics, and found that tying a shoelace with Earth-Moon latency (2.6 seconds) took almost ten minutes.

10


Whats wrong with control latency, if were trying to do something? What hard stuff are we talking about?

Picking up rocks is hard with latency.! Different sizes, random shapes. Turning over to inspect.!

Assembly and maintenance is hard with latency. ! Drilling, digging, stacking and cabling.!

In general, dexterity is hard with latency. Situation awareness as well. !

Haptic feedback extremely intolerant of latency. !

For field geology, we have NO experience with low latency telerobotics. !

Lessons from high latency telerobotics (MER, MSL) dont !necessarily transfer well to low latency telerobotics. !

Analog studies on the Earth will be essential to this understanding.!

Latency makes life HARD. !So lets do something about it. !(But landing humans in gravity wells is hard too.)!

11

So what are we talking about? On-orbit telerobotics.!Human presence from a (short) distance away. LOW LATENCY.!

(images from Boeing, GSFC, etc.)!FISO Telecon 5/27/15 12


Controlling telerobots from space microgravity! may pose some challenges !

Human factors are an important element of exploration telepresence. !

Sensorimotor deficits in eye-hand coordination in microgravity, space-fog?! Cognitive complication of control in a gravity field by an astronaut in microgravity.! Space habitat fatigue and work overload on cognitive functionality.! Teleoperation control station design in microgravity display screens, dexterous manipulators.! Candidate screening & skill assessment.!

HARI NASA ARC/JSC !Otmar Bock/DLR! Chuck Oman/MIT ! !PMDIS - Barry Fowler/York U. !

Doing telerobotics in space is serious issue for exploration telepresence.!ISS probably at least as important for assessing !

human factors as technical factors. !(Canadarm/DEXTRE not that relevant anymore.)!

13


6 Crew Human Exploration Using Real-time

Robotic Operations (HERRO)

Schmidt, Oleson, Landis et al. Glenn Research Center

Low latency telerobotic control process?!


Lessons about telepresence!architecture from terrestrial !

applications.!

Drones/UAVs

Undersea oil, gas, cable

Surgery

Mining

15


"##!

Evolution of space !telepresence!

Lunokhod control from Earth (control latency ~5-10

seconds)!

ISS Remote !Workstation !

control latency ~ ms)!

Lunar workstation at cis-lunar habitat -

latency ~few x 100 ms!

Boeing

Space telepresence

enabled

16


Low Latency Telerobotics - not a new concept for space exploration!Paine Report (1986) Pioneering the Space Frontier!"We recommend that: NASA explore the limits of expert systems, and tele-presence or tele-science for remote operations!including ties to spacecraft and ground laboratories. In working toward these goals, a broad examination of the non-space !applications of tele-science should be included. !

Telepresence: The use of real-time video communications coupled with remote control techniques which would provide !an operator on Earth's surface or other location with the capability to carry out complex operations in space or on the !surface of a planet or moon.!

Stafford Report (1991) America at the Threshold: America's Space Exploration Initiative! Telepresence robots can conduct some geologic field work. While such a technique might greatly enhance the scientific !return, the details of how such robots might work with people remain to be developed. Operators for these telepresence !robots need near-instant radio contact with the robots. This may be marginally obtainable by having the controllers on the !Earth, but operators on or near the Moon have a near-zero time lag for robotic teleoperations.!

and more recently (including HERRO!)!

Mars DRM5 (2009)!Low-Latency telerobotics operation may be a useful strategy, particularly if human missions stay out of gravity wells for some time.!

Augustine Commission (2009) Review of United States Human Space Flight Plans Committee!The goal focuses human exploration on producing exciting new science at each step of the way. The emphasis would be !on obtaining multi-kilogram samples from a variety of solar system bodies through tele-robotic exploration in concert with !The human missions. In the case of the Moon and Mars, humans would remain in orbit.!

A Phobos-based teleoperated exploration of the Martian surface, returning with samples from that surface, would likely!precede a crewed Mars landing mission, and would provide dramatically more responsive remote control than with the !communication delays incurred between Mars and Earth.!

NASA Technology Roadmap TA04 (2012)!Top technical challenges in human-robot interfaces are full immersion telepresence with haptic, multi sensor feedback, understanding!and expressing intent between humans and robots, and supervised autonomy of dynamic/contact tasks across time delay.!

17

and most recently !

Human Spaceflight Architecture Team (HAT) DRM 8a (2013)!Mars Orbital Mission (crew tele-operates surface assets)!

Global Exploration Roadmap (2013)!New mission concepts, such as human-assisted sample return and tele-presence should be further explored, increasing understanding of the important role of humans in space for achieving common goals.!Human-assisted sample return and tele-presence represent new and integrated approaches to space exploration with the potential to increase benefits.!Tele-presence is a capability which could significantly enhance the ability of humans and robots to explore together, where the specific exploration tasks would benefit from this capability.!

Evolvable Mars Campaign (2015)!Exploration Augmentation Module: Tele-operations in space and on lunar surface, report in progress!

Humans Orbiting Mars workshop (2015)!Teleoperation of Mars surface assets is presumed, but report in progress

Somewhat oddly, the 2013 NRC Human Spaceflight Report referred to this once in a footnote

The committee defines as human exploration ambiguous cases in which humans are in martian or lunar orbit telerobotically conducting surface operations because of the astronauts proximity to the target and their remoteness from Earth.



Opportunities and value presented by space telepresence!

Reduced cost impact, by not landing humans in strong gravity well. !

One party of humans on one trip achieves presence at many sites.!

Exploration duration not EVA-limited. !

Better visual perception (compared with the view through a helmet visor)!

Better dexterity (compared with dexterity achievable with EVA gloves)!

Benefits from rich commercial terrestrial technology development investments.!

Credible forward planetary protection. Dont put people where youre looking for life. !

Lower human risk. Put presence on the surface where humans might not dare tread.!

>> Increase in the number of potential destinations for human presence.! Mercury? Venus? Scuba diving in the lakes of Titan? !

Thats not to say that we dont want to land humans!! (thats another subject entirely) !

19

2-5 seconds latency 24/7

~50 ms latency for ~10 min overpass

Testing from ISS?!Exercising comm

challenges.!

On-orbit, low-latency control has been proven on ISS, !

e.g. METERON (shown here), !HET Surface Telerobotics K-10

ISS in LEO TDRSS in GEO

KONTOUR2 in Zvezda

Weilheim DLR

White Sands

FISO Telecon 5/27/15 Justin

20


So where do we try to use space telepresence?

LEO-to-Earth demo from ISS!!e.g. Ames K-10 (Fong)!

ESA METERON (Schiele)!

Good feasibility demo, but incomplete operational trial for Mars or Moon!applications. TDRSS-reliant or very brief connect-time. !

Lunar orbit-to-Moon!!e.g. from Orion or cis-lunar hab!

But from where exactly?!

LLO? ! Not good. R~100 km, unstable, short duration LOS contact.!EM L1/L2? Needs some stationkeeping, r~70,000km 500ms latency is a bit large.!Elliptical ! ! Changing latencies, limited LOS durations!

ARM SDRO? Similarly large 2-way latency. Strongly captured but precarious. !

PSDRO ! Proximal SDRO ARM orbit > r > 5,000km giving real-time connectivity. !

21


Lunar human-assisted sample return ?!

OSCAR Orion Sample Capture and Return / Orion-MoonRise is an excellent concept proposed by Lockheed and JPL (Leon Alkalai, FISO telecon, 2-27-13) (and others)!

Orion at EM L2 controls sampler-cacher in the SPAB, stow samples, transfer them to Orion waiting at EM L2, which transports them back to Earth.!

Concept worked out in some detail!

Getting Orion back and forth to EM L2! Lunar ascent vehicle to bring rocks back! Rendezvous, docking, and sample transfer to Orion! Target sites for sampling!

Sample selection procedure not obviously worked out.!Grab & go? Sampling decisions this one, and not that one?!

Human-in-the-loop sample selection, or human taxi service?!

That being said, about EM L2 operations, . !

22


PSDRO characteristics (Adamo et al., AIAA Space2014) and Jeff Parker at Colorado Close to Moon ; ~180ms 2-way latency at 25,000km. Ro~25,000 km makes for a convenient 4-day orbit. Good launch flexibility. Both-side operations. PSDROs are slightly chaotic, but strongly captured no stationkeeping. High inclinations possible, permitting polar operations. Retrograde orbit offers free-return capability.

The value of PSDROs for lunar surface telepresence, better than EM Lagrange point orbits?

23

EM L1/L2 orbit

25,000km

70,000km

25k km PSDRO

Suppose we had a habitat at a 25,000 km PSDRO ! an enabling site for lunar telepresence!

Latencies 15x less than from Earth, real time operations! Continuous access at a site over a whole workshift, both sides of Moon! No orbit maintenance required! Convenient trajectories to lunar surface! Similar LOI deltaVs for small PSDROs ! and, like EM L1 and L2! Very short eclipses of Earth and Sun! Control options for multiple surface sites!

This habitat need not obviate human visits to the lunar surface!! (But such visits can be expensive )!

!Most potent long term goal of lunar on-orbit telerobotics is ! conops validation for future visits to Mars and beyond.!


How could telepresence offer value on the Moon? !

Things youd want humans to do there. !Especially things that might be inefficient to do from the Earth.!

Inspection and assaying of regolith! Manipulation and collection of samples! Site surveying and reconnaissance! Instrument emplacement, maintenance! Regolith grading, piling, and paving for dust suppression! Deployment of habitats and support facilities!

(Yes, you could do this all from the Earth, !but it would take a lot longer for complicated tasks )!



25,0

00 k

m

L1/2

Operationally, comm is easy at a small PSDRO ! (and not bad at L1 or L2)!

habitat

surface

habi

tat

Where we are now (beyond ISS experiments)!(reports being produced)!

NASA Evolvable Mars Campaign planning on-orbit telepresence for Mars, perhaps on martian moons, as informed by activities in cis-lunar space.!(Crusan, Drake et al.)!

Humans Orbiting Mars workshop highlighted on-orbit telepresence for Mars exploration. (Hubbard, Ney, Naderi, Price et al.)!

ESA HERACLES, end-to-end plan for cis-lunar space consistent with GER, include exploration telepresence explicitly. Partnering with NASA, CSA, ROSCOSMOS, JAXA. (Hufenbach, Landgraf et al.) !

Telepresence study group formed by IAA Planetary Robotic Exploration Coordinating Group. Possible workshop organization. (Coradini)!

Informal Exploration Telepresence Working Group active for a year. !Biweekly telecons. Proposal for KISS workshop on exploration telepresence. (Anderson) !


So where do we go from here?!

On-orbit telerobotics is important rationale for in-space habitats!

!Hab-specific value!!Control functionality?!!Optimal orbit?!

!Telerobotics-specific value!!Modes for fully immersive telepresence?!!Extensibility to other destinations?!! Sharing low- and high-latency tasks?!

Policy-specific value!! Reconcile historical exploration with!

on-orbit telepresence exploration without ! physically being all the way there? !



Dan Lester University of Texas

[email protected] The evolution

of exploration?

29

NASA FISO Presentation: Exploration Telepresence: Value and Challenges

Documents

Transcript of NASA FISO Presentation: Exploration Telepresence: Value and Challenges