Is Motion Capture-Based Biomechanical Simulation

Valid for HCI Studies? Study and Implications

Myroslav Bachynskyi

Antti Oulasvirta

Gregorio Palmas

Tino Weinkauf

Saarbrücken

http://resources.mpi-inf.mpg.de/biomechanics

There are more opportunities in HCI for gestural and full body controls

Larger design spaceMore freedom for interface design

Traditional interfaces

Modern interfaces

Physical ergonomics is very important for the success of an interface

Gorilla arm Trapezius fatigueJoint stress

Traditional ergonomics instruments are too expensive, invasive or cumbersome

Goniometers QuestionnairessEMG (Surface

Electromyography)Needle EMG

SubjectiveUnreliable

CumbersomeNot accurate

Limited

Only surface musclesCross-talk

Unreliable in dynamics

Too invasiveOnly expert useHard to move

Motion-capture based biomechanical simulation allows “looking inside the body”

Optical motion capture (MoCap) Biomechanical simulation

Biomechanical simulation produces wide range of inside-body ergonomics indices

– Moments

– Forces inside joints

1. Model Scaling

2. Inverse Kinematics

3. Inverse Dynamics

4. Static Optimization

Output for observable movement:

Further processing

• Physical work

• Energy expenditure

• Fatigue index

• Per muscle:– Force exerted

– Activation by neural system

• Per joint:– Angles

MoCap data

Subject weight

Performance and ergonomics measured within single experiment synchronously

Single HCI experimentwith MoCap recording Performance of movements:

speed, accuracy, throughput

Ergonomics of movements: Joint angles and momentsMuscle forces and activationsEnergy expenditure

Synchronized in time and registered in 3D movement space

Our goal is to adapt biomechanical simulation for HCI scenarios

Select HCI task Record MoCap Simulate Analyze the data

Another HCI task

Medicine and sports:• Educated experimenters• Model fine-tuned to subject• Goal: highly focused analysis• Focus on lower body, gait and run

HCI:• Non-expert experimenters• No fine-tuning of the model• Multiple user groups• Overview of movement space• Upper extremity and full-body• Specific types of movements

Is it valid for HCI?

Sources of error:• Marker mapping• Marker drift• Suit drift• Marker trust• Model scaling• Mass distribution• Muscle properties• Activation optimality ?

Upper extremity model with muscles must be validated for HCI tasks

HCI Biomechanics

[Honglun2007]

[Du2007]

[Chang2007]

Bio

mec

han

ical

co

vera

ge

Simple ergonomics toolsintegrated with MoCap

Biomechanical simulationwith EMG for muscles

Complete biomechanical simulation

[This paper]

Moment at joint [Lloyd2003]

Muscle activations [Hamner2010]

[Pronost2011]

3 muscles and specific movement [Daly2011]

8 muscles and whole space movements

[This paper]

Bio

mec

han

ical

co

vera

ge

The paper reports 2 experiments

Study 1: Applicability across HCI tasks

Study 2: Validity against EMG data

Small movements Finger and arm

Full body

Run simulations and qualitatively inspect outputs

EMG

Simulation

Correlation as similarity measure?

EMG data

Predictions

Setup

PhaseSpace Impulse

Force plate OpenSim and Full-Body model

Main results

1. The simulation is successful for movements larger than 4cm

Failure Partial success Success

Hand model

2. Problems caused by strong users

Extremely fast movements Inverse Kinematics Static Optimization

3. Dynamic contact forces require external force measurements

Movement Without external forces the model is up in the air fixed onto pelvis

Correct simulation

Incorrect simulation

Part of parachute harness to fix pelvis

4. Large muscles are better predicted

Good

Bad

5. Fast movements are better predicted

Good

Bad

FastMediumSlow

6. Large individual differences due to age, gender and anatomy

Good

Bad

Conclusions

Biomechanical simulation is valid for the following HCI scenarios

• Movements longer than 4cm

• No extreme angles

• No strong participants

• Recording of external forces, if present

• Focus on bigger muscles

• Longer and faster movements

Possible improvements and future work

• Computed Muscle Control should produce better results than Static Optimisation

• Small finger movements may be successful with more comprehensive motion capture

• Simple-to-use index of muscular fatigue needs to be developed based on biomechanical simulation

Despite the restrictions, biomechanical simulation CAN be effectively applied for a wide range of HCI tasks

Select HCI task Record MoCap Run the simulation Analyze the data

Another HCI taskhttp://resources.mpi-inf.mpg.de/biomechanics

Is Motion Capture-Based Biomechanical Simulation Valid for HCI Studies?

Study and Implications

http://resources.mpi-inf.mpg.de/biomechanics

Myroslav Bachynskyi

Antti Oulasvirta

Gregorio Palmas

Tino Weinkauf

Saarbrücken