How to calculate joint kinematics using motion capture data

Sumin Park

Motion capture system • What is the motion capture? -> digitalizing movements of human, animals, robots and so on in 3D space -> movements of human : joint angles and body parts trajectories

• Many applications using motion capture systems… • Biomechanics, sport science, rehabilitation, animation, robotics etc.

• Many different software, devices and marker set for different purposes

Motion capture system • Types of motion capture systems -> optical, magnetic, X-ray fluoroscopy …

• Company for optical motion capture system -> Vicon, Qualisys, Kwon3D

• Vicon provides lots of software Vicon Nexus -> a motion capture platform for Life Sciences applications Vicon BodyBuilder -> designed specifically to make the implementation

of biomechanical models quick and easy Vicon Polygon -> reporting and presentation tool

• What can we get from the motion capture?

Motion capture

Inverse kinematics: joint angles Inverse dynamics: joint forces

Vicon Nexus (Plug-in-Gait model)

Vs. OpenSim

Joint kinematics • In general, cannot measure bone motion directly.

• Estimate motion of the bones from markers attached to the segment

• Motion of the bones = Joint kinematics : relative translation and rotation of adjacent bones (anatomical joint axes)

• Human has 206 bones. It is difficult to obtain the joint kinematics for all the 206 bones.

• Simplify the human skeletal model to express major motion : for example, flexion, adduction, rotation for hip, knee, ankle, shoulder/flexion for elbow..

• Ignore immovable joints, syndemoses (인대결합), synchondroses (연골결합)

Vicon Nexus Plug-in-Gait Model

OpenSim Gait 2392(2354) Model

Inverse kinematics Inverse dynamics

Inverse kinematics Inverse dynamics

Musclotendon dynamics

VS

Vicon Nexus Plug-in-Gait Model

OpenSim Gait 2392(2354) Model

Inverse kinematics Inverse dynamics

Inverse kinematics Inverse dynamics

Musclotendon dynamics

VS

Vicon Nexus Plug-in-Gait Model

OpenSim Gait 2392(2354) Model

Inverse kinematics : Link by link

Inverse kinematics : optimization

VS Head

Foot

Tibia

Femur

Pelvis

Thorax Humerus

Radius

Hand

Clavicle

Toe

Vicon Nexus Plug-in-Gait Model

Inverse kinematics

VS

Plug-in-Gait Model • Different models are needed to express different human motions

• Vicon Nexus is the motion capture platform, but provides Plug-in-Gait Model (PIG Model) for gait analysis

• PIG Model is developed especially for gait motion (full body) Bell, A.L., Pedersen, D.R. & Brand, R.A. (1990). A comparison of the

accuracy of several hip centre location prediction methods. Journal of Biomechanics, 23, 617-621.

Kadaba, M.P., Ramakrishnan, H.K. & Wooten, M.E. (1990). Lower extremity kinematics during level walking. Journal of Orthopaedic Research, 8, 849-860.

Winter, D.A. (1990) Biomechanics and motor control of human movement. John Wiley & Sons, Inc.

• Lower body of PIG Model is validated by several articles, but upper body of PIG Model has not been validated

• Joint kinematics : relative translation and rotation of adjacent bones -> PIG Model dose not consider translation

What is the Plug-in-Gait Model?

Head

Foot

Tibia

Femur

Pelvis

Thorax Humerus

Radius

Hand

Clavicle

Toe

• Define the bones and joint axes based on the markers • Use 39 or 35 markers and make 19 bones for the full body model

• Need more inputs about body information -> Subject Measurements (24개) • Body mass, height, leg length, knee width, elbow width, wrist width….

What is the Plug-in-Gait Model?

Head

Foot

Tibia

Femur

Pelvis

Thorax Humerus

Radius

Hand

Clavicle

• Approximate joint angles from the bones, joint axes, subject measurements. • 26*3 angles, among them some L angles overlap R angles

PIG Model outputs

Neck angle

Spine angle

Pelvis angle

Ankle angle

Knee angle

Hip angle

Wrist angle

Elbow angle

Shoulder angle

Toe

• Angles calculated from the YXZ Euler angles derived by comparing the relative orientations of the two bones

PIG Model outputs

Neck angle Head

Thorax Rotation angles

• Proximal(몸의 중심에서 가까운) frame 기준으로 Distal(몸의 중심에서 먼) frame의 angle -> Relative

• Pelvis의 경우는 Progression frame 기준 -> Absolute

• Flexion이 인체 동작에서 가장 주요한 모션

• Flexion (Y-axis) • Adduction (X-axis) • Internal rotation (Z-axis)

Head

Foot

Tibia

Femur

Pelvis

Thorax Humerus

Radius

Hand

Clavicle

Spine angle

Pelvis angle

Ankle angle

Knee angle

Hip angle

Wrist angle

Elbow angle

Shoulder angle

Toe

Z-axis

X-axis

Y-axis

Z-axis

X-axis Y-axis

PIG Model outputs

Head

Foot

Tibia

Femur

Pelvis

Thorax Humerus

Radius

Hand

Clavicle

Toe

How to define the bones and joint frames from the marker? - Lower body (in detail) - Upper body (in short)

Technical pelvis frame definition

RASI LPSI RPSI LASI Midpoint of RPSI/LPSI (SACR)

Midpoint of RASI/LASI

(Pelvis origin) -> Y-axis

Plane formed by RASI/LASI/SACR

-> X-axis

Perpendicular to the plane -> Z-axis

Pelvis angle • Progression 기준으로 Pelvis frame 회전각

• Anterior Tilt (Y-axis, 앞쪽 기울임) • Upward Obliquity (X-axis, 옆쪽 기울임) • Internal Rotation (Z-axis, 옆으로 돌림)

• LPelvis angle/RPelvis angle X, Z 회전 반대 X-axis

Y-axis

Z-axis [ Pelvis frame ]

X-axis (Forward)

Y-axis

Z-axis (Vertical)

[ Progression frame ]

Y, X, Z order

X-axis Y-axis

Z-axis

Pelvis frame

Hip joint center definition

X-axis

Z-axis

Pelvis frame

Y-axis RASI LASI

[측면] [정면]

RASI LASI

𝜽 = 𝟎.𝟓 (𝒓𝒓𝒓) 𝜷 = 𝟎.𝟑𝟑𝟑 (𝒓𝒓𝒓) 𝒓𝒎𝒓𝒓𝒎𝒎𝒓 = 𝒎𝒓𝒓𝒎𝒎𝒓 𝒓𝒓𝒓𝒓𝒓𝒓 𝑪 = 𝑴𝒎𝒓𝑴 𝑳𝒎𝑳 𝑳𝒎𝑴𝑳𝑳𝑳 ∗ 𝟎.𝟑𝟑𝟓 − 𝟑𝟓.𝟑 𝒎𝒎 𝒙𝒓𝒓𝒓 = 𝑳𝒎𝑳 𝑳𝒎𝑴𝑳𝑳𝑳 ∗ 𝟎.𝟑𝟏𝟏𝟏 − 𝟑𝟏.𝟓𝟓 𝒎𝒎 𝒓𝑨𝑨𝑨𝑨 = 𝒓𝒓𝒓𝑳𝒓𝑴𝒅𝒎 𝒃𝒎𝑳𝒃𝒎𝒎𝑴 𝑹𝑨𝑨𝑨 𝒓𝑴𝒓 𝑳𝑨𝑨𝑨 𝑨 = +𝟑 𝒇𝒇𝒓 𝒓𝒓𝑳𝑳𝑳 𝒓𝒓𝒓𝒎,𝒓𝑴𝒓 − 𝟑 𝒇𝒇𝒓 𝒍𝒎𝒇𝑳 𝒓𝒓𝒓𝒎

X-axis

Z-axis

Y-axis

RHJC

Anatomical pelvis frame

X-axis

Z-axis

Y-axis

Anatomical pelvis frame • Once the hip joint center are defined, the origin of the technical pelvis frame is shifted to the mid point of the hip joint centers.

• This frame is the PEL Plug-in-Gait Bone in NEXUS.

Pelvis frame

Hip joint center (HJC)

LHJC

Knee joint center definition

RTHI

RKNE RKNE

KneeOS = (Marker radius +Knee width)/2

RKNE

RHJC

RTHI Knee joint center (KJC)

Y-axis X-axis

Z-axis RHJC

RKNE

RHJC

RTHI

RKJC

Y, X, Z order

Anatomical Femur frame • This frame is the RFE(right) Plug-in-Gait Bone in NEXUS. Hip angle • Pelvis frame 기준 Femur frame 회전각

• Flexion (Y-axis, 앞쪽 회전) • Adduction (X-axis, 안쪽 오므림) • Internal Rotation (Z-axis, 안쪽 돌림)

• LHip angle/RHip angle X, Z 회전 반대

Z-axis

RKNE

RHJC

RTHI

RKJC

Z-axis

Y-axis

X-axis

Plane formed by RTHI/RKNE/RKJC/RHJC

RKJC

Perpendicular to the plane -> X-axis

Z-axis

Y-axis

X-axis

[ Femur frame ]

X-axis Y-axis

Z-axis

[ Pelvis frame ]

KJC

AJC AJC

[ Tibia frame ] [ Tibia frame ]

RKJC

RANK

RTIB

Plane formed by RTIB/RANK/RAJC/RKJC

Ankle joint center definition

X-axis

Perpendicular to the plane -> X-axis

Z-axis Y-axis

X-axis RKJC

RANK

RTIB

RANK

RTIB Ankle joint center (AJC)

AnkleOS = (Marker radius +Ankle width)/2

RKJC

RANK

RTIB

Z-axis

RAJC RAJC

RAJC

RKJC

RANK

RTIB

Y-axis

Z-axis

Y-axis X-axis

Z-axis

Torsioned tibia ‘Tibial Torsion’ (Measurement input)

Z-axis

X-axis Y-axis

Z-axis

Untorsioned tibia For ankle angle For knee angle

Y-axis X-axis AJC

Torsioned tibia/Untorsioned tibia Knee angle • Femur frame 기준 Untorsioned Tibia frame 회전각

• Flexion (Y-axis, 뒤쪽 굽힘) • Adduction (X-axis, 안쪽 꺽임) • Internal Rotation (Z-axis, 안쪽 돌림)

• LKnee angle/RKnee angle X, Z 회전 반대

• ‘Tibial Torsion’ indicates the angle between the knee Y- axis and ankle Y-axis

Y, X, Z order

Z-axis

Y-axis X-axis

[ Femur frame ]

KJC

[ Tibia frame ]

X-axis Y-axis

Z-axis

Untorsioned tibia

AJC

Anatomical shank frame • The Torsioned tibia frame is the RTI(right) Plug-in-Gait Bone in NEXUS.

Z-axis

Y-axis

X-axis Z-axis

Y-axis

X-axis

RTOE

RAJC

Z-axis Y-axis

RHEE

RANK

X-axis

Dynamic trial

Ankle angle • Torsioned tibia frame기준 foot frame 회전각 • Dorsiflexion (Y-axis, 위쪽 올림) • Internal Rotation (Z-axis, 안쪽 돌림) • Inversion (X-axis, 안쪽 회전) • This frame is the RFO(right) in NEXUS.

Foot frame definition

Y, Z, X order

Z-axis

Y-axis X-axis

AJC RANK

RHEE

RTOE

RHEE

RTOE

RANK

RAJC

HEE and TOE markers are at the same height

Z-axis

Y-axis X-axis

[ Tibia frame ]

AJC

Z-axis

RHEE

RTOE

RAJC

Perpendicular to the plane formed by TOE/AJC/KJC -> Y-axis

Z-axis

RKJC

Y-axis RANK

Foot static frame (no flat option) • AJC를 이용한 frame과 • HEE을 이용한 frame 사이의 각도 계산 • Plantar-flexion offset (Y-axis 회전) -> α • Rotation offset (X-axis 회전) 구함 -> β

• Dynamic 때 Y, X-axis 순서로 offset 적용

RTOE

RAJC

Z-axis

Y-axis

RHEE X-axis

Static trial α β

β α

TOE Z-axis

Y-axis

X-axis [ Foot frame ]

Torsioned tibia

Thorax/Head frame definition

Z-axis

Y-axis X-axis

RFHD LFHD

RBHD LBHD CLAV

STRN

C7

T10

Anterior view Posterior view Anatomical pelvis frame

X-axis

Z-axis

Y-axis

Thorax frame Head frame

Y, X, Z order Y, X, Z order

Z-axis

Y-axis

X-axis

Anatomical Thorax frame • This frame is the THO Plug-in-Gait Bone in NEXUS. Spine angle • Pelvis frame 기준 Thorax frame 회전각

• Backward Tilt (Y-axis, 뒤쪽 꺽음) • Side Tilt (X-axis, 좌우 기울임) • Rotation (Z-axis, 좌우 돌림)

• LSpine angle/RSpine angle X, Z 회전 반대

Anatomical Head frame • This frame is the HEA Plug-in-Gait Bone in NEXUS. Neck angle • Thorax frame 기준 Head frame 회전각

• Forward Tilt (Y-axis, 앞쪽 꺽음) • Side Tilt (X-axis, 좌우 기울임) • Rotation (Z-axis, 좌우 돌림)

• LNeck angle/RNeck angle X, Z 회전 반대

• Kinematics calculation Define bones and joint frames using markers and skeletal geometry Joint frames only have rotations, no translations allow Link (segment) by link calculation

• Some problems of PIG Model kinematics

The location of a joint axis for a segment may have some errors depending on the marker placement error

The changed joint axes lead the change of segment’s length (For example, the length between femur frame and tibia frame which is considered as the femur bone length could be change. In real, the bone length can not be changed.)

PIG Model kinematics summary

[측면] [정면]

RASI LASI

Hip joint center (HJC)

OpenSim Gait 2392(2354) Model

Inverse kinematics

Gait 2392(2354) Model • OpenSim provides many different Models. Gait 2392(2354) is one of them.

• 23->number of joints/92->number of musclotendon actuators for 76 muscles

• Gait 2392 is also developed for gait motion (lower body, torso) Yamaguchi G.T., Zajac F.E.: A planar model of the knee joint to

characterize the knee extensor mechanism." J . Biomecl7. vol. 21. pp. 1-10. 1989.

Delp, S.L., Loan, J.P., Hoy, M.G., Zajac, F.E., Topp E.L., Rosen, J.M.: An interactive graphics-based model of the lower extremity to study orthopaedic surgical procedures, IEEE Transactions on Biomedical Engineering, vol. 37, pp. 757-767, 1990.

Anderson F.C., Pandy M.G.: A dynamic optimization solution for vertical jumping in three dimensions. Computer Methods in Biomechanics and Biomedical Engineering 2:201-231, 1999.

• Model itself, already has the bones, joint axes, muscles, markers, constraints, contact geometry…

• The bones, joint axes, muscles etc. of a model are defined differently depending on the motions to be analyzed and analysis purpose

OpenSim Joint Types • Complex and multiple joint models for complex motions

• Available Joint Types Weld Joint: introduces no coordinates (degrees of freedom) and fuses

bodies together Pin Joint: one coordinate about the common Z-axis of parent and child

joint frames Slider Joint: one coordinate along common X-axis of parent and child

joint frames Ball Joint: three rotational coordinates that are about X, Y, Z of B in P Ellipsoid Joint: three rotational coordinates that are about X, Y, Z of B

in P with coupled translations such that B traces and ellipsoid centered at P

Free Joint: six coordinates with 3 rotational (like the ball) and 3 translations of B in P

Custom Joint: user specified 1-6 coordinates and user defined spatial transform to locate B with respect to P

Slider Ball Ellipsoid

Gait 2392(2354) Model • Gait 2392 Model geometry

Marker (called virtual

marker)

Muscle

Bone

Joint axis

Gait 2392(2354) Model • Gait 2392 Model configuration

Gait 2392(2354) Model • Pelvis frame: The pelvic reference frame is fixed at the midpoint of the two anterior superior iliac spines

• Femur frame: The femoral frame is fixed at the center of the femoral head

• Tibia frame: The tibial frame is located at the midpoint of the line between the medial and lateral femoral epicondyles

• Patella frame: The patellar frame is located at the most distal point of the patella

• Talus frame: The talar frame is located at the midpoint of the line between the apices of the medical and lateral malleoli

• Calcanus frame: The calcaneal frame is located at the most interior, lateral point on the posterior surface of the calcanus

• Toe frame: The toe frame is located at the base of the second metatarsal

• Lumbar frame: Same as the pelvis frame

Gait 2392(2354) Model • 23DOF for the lower limb and torso

• Pelvis joint -> 6DOF = 3 rotations + 3 translations (Free joint) • Hip joint -> 3DOF = 3 rotations (6DOF for left, right) (Ball joint) • Knee joint -> 1DOF = 1 flexion (2DOF for left, right) (Ellipsoid joint) • Ankle joint -> 1DOF (2DOF for left, right) (Revolute joint) • Subtalar joint -> 1DOF (2DOF for left, right) (Revolute joint) • Metatarsophalangeal joint-> 1DOF (2DOF for left, right) (Revolute joint) • Lumbar joint -> 3DOF = 3 rotations (Ball joint)

Knee joint ANK/ST/MTP joint

Gait 2392(2354) Model • Knee joint -> 1DOF = 1 flexion (2DOF for left, right)

• Ankle joint -> 1DOF (2DOF for left, right)

• Subtalar joint -> 1DOF (2DOF for left, right)

Ankle joint

Subtalar joint

[Yamaguchi G.T., Zajac F.E.: A planar model of the knee joint to characterize the knee extensor

mechanism." J . Biomecl7. vol. 21. pp. 1-10. 1989]

OpenSim Scaling • The dimensions of each segment in the model are scaled so that the distances between the virtual markers match the distances between the experimental markers.

• Distances between experimental markers (ei) relative to the distances between corresponding virtual markers (mi) are used to compute scale factors.

• s1=e1/m1 -> s=(s1+s2)/2 (‘s’ means a segment of the body) • After scaling the model, the next step is to move the model's markers to match experimental marker locations in a static pose.

OpenSim Inverse Kinematics

X-axis

Y-axis

Z-axis

X-axis

Y-axis

Z-axis

X-axis

Y-axis

Z-axis

Defined joint axes and markers

Experiment markers

• Inverse Kinematics steps through each time frame to have “best matches” of experimental data and positions the model in a pose

OpenSim Inverse Kinematics • This "best match" is the pose that minimizes a sum of weighted squared errors of markers and/or coordinates.

• where q is the vector of generalized coordinates being solved for, xiexp is the

experimental position of marker i, xi(q) is the position of the corresponding marker on the model, qj

exp is the experimental value for coordinate j.

• qjexp can be joint angles obtained directly from a motion capture system (i.e.,

built-in mocap inverse kinematics capabilities), or may be computed from experimental data by various specialized algorithms.

• Prescribed coordinates are set to their experimental values. For instance, in the gait2354 and gait2392 examples, the subtalar and metatarsophalangeal (mtp) joints are locked and during IK they are assigned the prescribed value of 0˚.