Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT...

Monitoring, Modelling, and Predicting with Real-Time Control

Dr Ian OppermannDirector, CSIRO ICT Centre

Understanding the world …in real time

CSIRO. Monitoring, Modelling, and Predicting with Real-Time Control

It is so natural for us.Nothing is “natural” in Robotics How do we do it in “real time”?We focus on four components.

Sensing Perception, Control Actuation

Understanding the world : A Robot’s Viewpoint


operations performed, from sensing to actuation, within the timeframe required for effective operation in a dynamic environment

Globally Consistent Real-time Perception

Sensing(Lidar. Radar, Stereo)

Sensing(Lidar. Radar, Stereo)

Semantics(Path, Obstacles)

Semantics(Path, Obstacles)

SLAM

Long Term Mapping

The difference between MOT and CD is in the time

scale and whether the object is transitory or

permanent

The difference between MOT and CD is in the time

scale and whether the object is transitory or

permanent

The difference between Localization and MOT is the difference between recording the pose of

ourselves versus recording the pose of

somebody else

The difference between Localization and MOT is the difference between recording the pose of

ourselves versus recording the pose of

somebody else

Globally Consistent Model(map, trajectories etc.)


Mobile Mapping: What’s available now?

Big, Bulky and Expensive• Rooftop-mounted sensors

• 2D lidars with high-end GPS/INS

• Cost: $ X00,000

Limited Environments• Reliance on GPS which is

challenged in urban canyons, underground, near large infrastructure, forests, mines

Not Real Time• Processing straightforward if

accurate position is known at all times but NOT REAL TIME



What problems to address?

Cost

move from expensive hardware to software

Accuracy

improve location estimation

loop closure (knowing where you have been)

A dynamic environment

Long term versus short term changes in environment

Speed

Real time means being able to sense, aggregate, decide or re-plan in the time frames affected by limits of safety, fuel, task completion constraints


Mapping and Location Improving accuracy, reducing cost

3D Map created with spinning 2D LIDAR on Bobcat

With Scan Matching (without any additional sensors)

Data A

ssociation R

obust Optim

izationD

ata Association

Robust O

ptimization

• No GPS• No odometry• No encoders• No IMU (inertial measurement Unit)


Loop Closure – Improving accuracy

With Place Recognition (loop closure)

2D LIDAR on moving 4WD

Data A

ssociation R

obust Optim

izationD

ata Association

Robust O

ptimization


Loop Closure Reliability and Accuracy over long times and long distances

2 Lasers

No GPS

No odometry

No encoders

No IMU

No Calibration

Object Tracking

Long term Tracking

Under different conditions Passive actuationCSIRO. Monitoring, Modelling, and Predicting with Real-Time Control

Example : Mapping Jenolan Caves


Example : Mapping Jenolan Caves Sample Point Cloud


Example : Mapping Jenolan Caves Watertight Surface Generation


Example : Mapping Jenolan Caves Chifley Cave Surface Model


Example : Mapping Jenolan Caves Registered Point Cloud


Example : Mapping Jenolan Caves Point Cloud Overlay


Example: Bringing it all together


Example : Airborne Terrain Mapping and Static Object Avoidance


The Future?Connecting the Virtual to the Real (Robot)

Mixed-RealityTele-Robotic

Robot

Tele-Operation

Machine Autonomy

SharedAutonomy

Autonomy

Manual

User Interface

Intelligent Behavior

Extent of Knowledge

CommunicationsLatency

Global

LocalReactive

Proactive

AugmentedReality

AugmentedVirtuality

Supervisory

Assistive

Real

Virtual


The Future? Convergence of Technology

Realistic Simulation of Environment.

Realistic Visualization of

Environment

Realistic Interaction with

Environment


Simulation and Computation Spectrum

Off-line

Analysis

Multiple

Scenarios


Data Importance Latency Bandwidth

Safety (automation) Failsafe Low Low

Control & State Critical Med Med

Sensing Desirable Variable High

Peer-to-Peer

MBWA

Peer-to-Peer

WiMAXBackhaul Network

WiFi/MiMo Mesh Access Network

Whole of system view


The Challenge: Unified User Interface

Command & Control

Tele - Robotic SCADA

Whole of Mine Planning

by Paul BourkeCSIRO. Monitoring, Modelling, and Predicting with Real-Time Control

http://local.wasp.uwa.edu.au/~pbourke/miscellaneous/

Summary

• Automation and Teleoperation in Realtime Environments require:• Knowing state of environment [Mapping]• Knowing where you are [Localization]

• SLAM provides:• Localization in GPS denied areas• Mapping over large scales• Independent real-time sensor• Integration with other sensors (GPS,IMU etc)

• Future Realtime Automation will require:• Pervasive Tracking

• SLAM-MOT (SLAM with Moving Object Tracking)• Collaborative / Distributed real time mapping

• Deal with uncertainty, latency, trust and different sensing modalities• Life long mapping and long term map management

• Scalable (semantic mapping)• Maps get better with age, rather than “blurry


CSIRO – more than 80 years of achievements


Contact UsPhone: 1300 363 400 or +61 3 9545 2176

Email: [email protected] Web: www.csiro.au

Thank you

With Thanks to Dr Rob ZlotDr Mike BosseDr Elliot DuffDr Jonathan Roberts

For further information:Dr Ian OppermannDirector, CSIRO ICT Centre

Email: [email protected]: www.csiro.au/ict

Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT...

Documents

Transcript of Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT...