Finding the Truth: Interview and Interrogation Training Simulations

Janet Mulkern

Concurrent Technologies Corporation

Ronald Punako, Jr.

Concurrent Technologies Corporation

• Welcome!• Instructional Systems Design

– Intended Audience– Design Considerations– Commonalities of Instructional Design

• Simulation – Visual and Interaction– Interview System– Interrogation System

Topics

2

Our Purpose

Interview a Child Interrogate a Suspect

Creating interview and interrogation

Immersive Learning Simulations (ILS)

3

Intended Audience

• The interview training system was designed as capstone experience.

• The interrogation training system was designed for Special Agents with little or no interviewing or interrogating suspects of major crimes.

• The audiences include mostly male individuals between 18-40 years of age.

4

Design Considerations

• Predominant Millennial learners (1982-2002):– Predominant training question to answer is: “How do we do it?”

(Coomes and DeBard, 2004)– The “Net Generation.”

• Instructional strategies support the learner’s needs as described by Svinicki (1999): – Provide reinforcement for activities you wish to encourage such

as praise and positive feedback. – Emphasize internal reinforcement and motivation. – Set challenging yet attainable goals for learning, and provide

feedback on progress.

5

Kinesics

• “The study of nonlinguistic bodily movements”– Communication is 93% body language and paralinguistic cues

(including pitch, volume and tone)– Words only provide 7% of the communication. (Borg, 2009)

6

Commonalities of Instructional Design

• Challenges: – Recognition of behavioral indicators of abuse– Recognition of Kinesic indicators of truth or deception.

• Models: – Scenarios were modeled after a session by an experienced

investigator and expert instructor. – Correct answers are based on real-life experiences and

consequences.

7

Commonalities of Instructional Design (Continued)

• Controls: – System’s graphical user interfaces (GUIs) were designed to

include a number of controls. – In the interrogation training system an additional control was

added where the learner can acquire points to increase the motivation to “win” by obtaining the most points.

• Manipulation: – The learner controls the interview and interrogation by asking the

right question, reading the non-verbal indicators correctly and then asking the next question correctly.

– Depending on how poorly the learner asks the question, the consequences become more severe.

8

Commonalities of Instructional Design (Continued)

• Authenticity: – Written from case material– Reviewed by the internal SMEs and the client for authenticity of

content and interaction.

• Consequences: – If the learner does not pay attention to and adapt their line of

questioning to the suspects non-verbal signs, the subject avatar is designed to shut down the interview and ask for a lawyer.

– To the learner, the course consequences are passing or failing.

9

Interview System

• Goals• Subject• Environment• Simulation of Body Language• Simulation of Facial Expression• Proxemics• Attention to Child’s Needs

10

Interview ILS Goals

• Develop a proof-of-concept simulation to train investigators to conduct interviews of child abuse victims.

• Develop a training scenario to benefit interagency organizations for both military and civilian applications.

• Provide a virtual interactive training module avatar system, with all necessary motion and appearance to project the behavioral indicators of abuse.

• Enable investigators to practice interviewing child abuse victims and to receive feedback after the training has reached completion.

11

Subject: Cynthia Baker

12

Physical profile• 6 years old• Petite• Brown hair• Brown eyes

Environment: Child’s Interview Room

13

Playroom environment• Sofa, adult chairs, child

chairs• Toys

• Posters• One way glass

Simulation of Body Language

14

Shrugging off Coloring

Casual dialogue Closing off/hiding

Challenge: Communicate emotional state using body language

• Designed 20 animations to communicate emotional state

• Body animations designed to be blended across 2 layers including idle and emotive

Simulation of Facial Expression

15

Happy

Challenge: Attempt to accurately portray facial expression

•Designed for sadness, fear, anger, indignant, disgust, happiness, confusion and shame across 4 layers including: blink, idle, lip-sync and emotive•Selected expressions modeled after Facial Action Coding System action unit combinations.

Angry

Proxemics

• First question deals with seating arrangements relative to Cynthia

16

• Experimented with the Xbox 360 Kinect controller to provide proxemics

Attention to Cynthia’s Needs

Challenge: Simulating the

special needs of a child

interviewee

• Talking at the child’s level• Sitting at the child’s level• Paying attention to the

child's special needs (requests for mother, playing and coloring)

17

Playing under the table

Interrogation System

• Subject• Environment• Simulation of Body Language• Simulation of Voluntary Facial Expression• Simulation of Micro Expression• Detection of Micro Expression• Continuation of Disposition/Personality

18

Interrogation ILS Goals

• Develop a proof-of-concept simulation to train investigators to conduct interrogation of criminals suspected of sexual assault.

• Develop scenarios that can be manipulated to provide challenging interrogation exercises that are real and relevant to the current threat of perpetrators of serious crimes against persons.

19

Subject: Sergeant Mike Hagan

Physical profile• 30 years old• 71 inches• 156 lbs• Brown hair• Brown eyes

Criminal record• First time offender

20

Environment: Interrogation Room

21

Minimalist design• One table • Security Camera• Two chairs

Simulation of Body Language

22

Shrugging off Stop pressuring me

Defensive rebuke Closing off/hiding

Challenge: Communicate

emotional state using body

language

• Designed 35 animations to communicate emotional state

• Body animations designed to be blended across 2 layers including idle and emotive

Simulation of Voluntary Facial Expression

23

Happy Angry

Worried Disgusted

Challenge: Attempt to accuratelyportray facial expression with ahigh-fidelity model.

• Designed for sadness, fear, anger, contempt, disgust,happiness and surprise across 4 layers including: blink, idle, lip-sync and emotive

• Selected expressions modeled after Facial Action Coding System action unit combinations.

Simulation of Micro Expressions

Challenge: Can the architecture

handle extremely fast animation

transitions and blending?

• Designed for sadness, fear, anger, contempt, disgust,happiness and surprise

• Achieved required 1/25 secondmicro expression duration.

24

Worried micro expression

Detection of Micro Expressions

Challenge: How to integrate a

micro expression mini-game into

the simulation?

• A possible micro expression occurs while a detect button is shown

• Not a test of dexterity• False-positives possible.

25

Worried micro expression

Continuation of Disposition/Personality

26

Disposition Threshold Disposition Visual Output

5+ Angry/Uncooperative

2-4 Upset

0-1 Idle/Cooperative

Disposition Modifiers(First time offender)

Red = +1Yellow = +0Green = -1

Disposition Modifiers(Repeat offender)

Red = -1Yellow = +0Green = +1

Common Design Elements: Decision Sequencing

• Linear - sequencing of events with no opportunity for deviation from sequence trunk.

• Branching - sequencing of events that allows deviation from the sequence trunk.

• Recursive – revisit previous events.

• Any order – events presented in a discreet group in any order.

27

Common Design Elements: Decision Making

28

Select a response which would help assess the child’s episodic memory level.

A. You were telling me about your Christmas. (Green = Best)B.I’ll bet Christmas day was fun, right? (Yellow = Average)C.Specifically, where were you and what did you do Christmas day. (Red = Poor)

Common Design Elements: Performance Scoring

29

Learner makes a choice Score is averaged by objective and recoded

Final score is averaged across objective scores

 Objective 1: 50%Objective 2: 100%…

 Final: 75%1. Choice A

2. Choice B3. Choice C

Learner makes a choice Feedback is recorded by objective.

Feedback is provided to learner

 Objective 1: Good Job! …

 Good Job! You chose choice A. Choice A was best because…

1. Choice A2. Choice B3. Choice C

Quantitative

Qualitative

Common Design Elements: Confession or Disclosure

Challenge: How tosimulate a meaningfulconfession/disclosure?

• Culminating point in final act

• Reached through majority green and yellow path decisions

• Must avoid terminal red path decisions

30

Common Design Elements: Notebook

Challenge: How to provide the learner with an effective record of kinesics exhibited?

•Running log of all kinesics, dialogue, choices, decisions and mentor feedback•Designed for learner reflection, performance review and memory/analysis aid

31

Common Design Elements: After Action Review

Summary Panel• Rollup of objectives• Scored or un-scored against

objective thresholds

Details Panel• For each decision displays screen

shot, decision response, related objective and mentor feedback

32

Technical and Integration Details

• Work with internal and client SME to develop a script that details what content (objectives, voice over, feedback, remediation, interactivity, others) to integrate

• Staff works to produce content

• Custom development pipeline built on the Unity Game Development Tool supports integration of content

• Simulations currently deployed to Web, PC, Mac, iPad

33

Next Steps

• System to be evaluated for a Certificate of Networthiness

• Pursuing evaluation by learners at Fort Leonard Wood, MO

• Advanced feature integration– Full implementation of continuation of disposition/personality– Implementation of motion capture for Proxemics– Photo-realistic body language and facial expression– Improve development pipeline

• Evaluate potential for application in broader industry.

34

Questions

Questions ?

35