User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters

User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters

Affligem, October 19, 2009

Elvina Motard Space Applications Services (Belgium)

Jeremi Gancet Space Applications Services (Belgium)

Amir M. Naghsh Sheffield Hallam University (UK)

Chris R. Roast Sheffield Hallam University (UK)

Outline

1. Introduction and background– the EU FP6 Guardians project

2. Fire-fighting context, setting and operations

3. The Guardians baseline setup

4. Direct Human Swarm Interactions (HSI)

5. Remote Human Swarm Interactions

6. Conclusion and future work

1. Introduction and background – the EU FP6 Guardians project

Guardians: EU-FP6 project, 2006-2010 (38 months)

=> Group of unmanned assistant robots deployed in aggregative navigation supported by scent detection

=> Swarm robotics

=> Human-Robot Interactions / Human-Swarm Interactions

=> Application domain: search & rescue, support to fire-fighting

=> Main scenario: large warehouse fire


=> Smoke – means heavy breathing apparatus, little to no vision, guidelines

=> Noise – means limited audio communication, establish protocols

=> Heat – physical stress

=> Touch – limited dexterity

=> 20 minutes maximum, risks to lives – mental stress


=> Fire event: little information available a priori...

=> 1st task: to assess the incident and the primary risks

=> Fire-fighters safety: high(est) priority!

=>Organized into teams, then groups of 2 – 3 fire-fighters

Incident Commander

Operations Commander

Sector Commander

Fire-fighter

Fire-fighter

Fire-fighter

Safety Officer

ECO

Zone 3

Zone 2Zone 1

Sector 3Sector 2

Sector 1

3. The Guardians baseline setup

Incident Commander


Fire-fighter

Fire-fighter

Fire-fighter

Safety Officer

ECO

Zone 3

Zone 2Zone 1

Sector 3

Sector 2

Sector 1

R

RRR

Guardians’ Base Station

Coordinator

Operator(s)

Sensor Data Specialist(s)

Sector Commander

Incident Commander


Fire-fighter

Fire-fighter

Fire-fighter

Safety Officer

ECO

Zone 3

Zone 2Zone 1

Sector 3

Sector 2

Sector 1

R

RR

Sector Commander

R


Coordinator

Direct Human-Swarm Interactions

Operator(s)


4. Direct Human Swarm Interactions


Purpose: supporting local fire-fighters operations

=> Guidance / navigation (e.g. toward exit in case of increasing risk)

=> Informing fire-fighters about current ambient conditions in the vicinity (e.g. heat level, chemicals)

=> Small distance forward scouting


Light Array Visor: Baseline setup


Light Array Visor (LAV): version 1

=> Phidgets toolkit

=> Iterative development process:

• Implementing the prototype

• Followed by exploration sessions with Fire-fighters

=> Analogical visual information

Light Array Visor (LAV): version 2

=> Real operational fire-fighting helmet

=> Experimenting with a group of two fire-fighters

=> Logical commands encoding and display

=> Integration of internal measurement unit (IMU)

=> More stress (reporting, counting light flashes and memorising the mental map)



Incident Commander


Fire-fighter

Fire-fighter

Fire-fighter

Safety Officer

ECO

Zone 3

Zone 2Zone 1

Sector 3

Sector 2

Sector 1

R

RR

Sector Commander

R


Coordinator

Operator(s)


Remote Human-Swarm Interactions


Purpose: remotely supervising robots and fire-fighters operations

=> Integrated data for the global supervision of the situation

=> Provides means to control the robots of the swarm:

• High level commands / policy level control of groups of robots in the swarm

• Individual robot control, in case of specific need

=> Interface with decision makers in the fire-fighters hierarchy, thus allows to take informed decision


Base station baseline setup

Khepera III

HNI Bebotx80Erratic

Robudem

Robotnik Guardian

Atrv-JR


1

2

5

3

4

1. Main visualization zone

2. Viewport navigation

3. Commands formulation area

4. Overall map view

5. Mission log

Remote interaction: main user interface


1

Fire-fighter and robots 3D representation

Robots sensors + video - joint visualization (ecological display paradigm)

High level command generation wizard

6. Conclusion and future work

Direct HSI

- LAV2 more expressive for fire-fighters, although more stressful

- Further experiments planned with a number of swarm support scenarios

Remote HSI

- Operator role usability evaluation so far => further work to focus on Sensor Data Specialist and Coordinator roles

- Multi-user, joint operations to be experimented and evaluated

Interfaces design for fire-fighters support

- Complex but valuable approach to integrate user inputs into the design and development process

- Incremental usability evaluation process (formative feedback)

Thanks!

http://www.guardians-project.eu/

User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters

Technology

Transcript of User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters