Literature review of Deception Detection...deception detection on inter- and intra-difference. For...

Literature review of Deception Detection

Guozhen An Speech lab Queens College

1

Outline• Motivation

• Deception detection

• Emotion recognition

• Personality recognition

• Conclusion and future work

Motivation

• Who interested about detecting deception?

business, jurisprudence, law enforcement, and national security, etc.

• Why interested about detecting deception?

Identifying potential harm for people and society.

3

Definition of Deception

• A successful or unsuccessful attempt, without forewarning, to create in another a belief which the communicator considers to be untrue. (Granhag, Strömwall, 2004)

• Any behavior is not intentionally act to make belief is not deception, such as honest mistake, misremembering.

4

Type of Deception• Face saving deception

Lie to protect themselves, avoid tension and conflict in social interaction, minimize hurt feelings and ill will

• Malicious deception

politic, academic, consumer, job market, security,

5

Deception Detection Method• Equipment

Polygraph

• Verbal cues

pitch, accent, lexical meaning and speech event

• Non-verbal cues

eye gaze, body movement, facial, posture

7

Verbal and Non-verbalWe will mainly discuss the importance of verbal cues for deception detection, because ..

1. Vrij (2008b) states that police usually pays more attention to the non-verbal cue than verbal cue, and the result of paying attention to only non-verbal cue is less accurate than take the verbal cue into account.

2. Meta-analysis of verbal and nonverbal cues for deception shows that speech related cues are more accurate than nonverbal cues (DePaulo et al. 2003, Vrij et al. 2008a).

8

Difficulty of Deception Detection

• Huge individual difference among the liars (inter-difference).

• Differences between truth tellers and liars are typically small. (intra-personal)

• Embedded lie in truth.

9

Human Performance of Detecting DeceptionGroup Subjects Accuracy

Teachers 20 70%Social workers 20 66.25%

Criminals 52 65.40%Secret service agents 34 64.12%

Psychologists 508 61.56%Judges 194 59.01%

Police officers 511 55.16%Customs officers 123 55.30%Federal officers 341 54.54%

Students 8876 54.20%Detectives 341 51.16%

Parole offices 32 40.42%Total 11052 54.50%

10

Deception Dataset• One of biggest limitation for deception research is

that there is very few good annotated data sets available which are accurate and well designed.

• Columbia SRI Colorado corpus was introduced by Hirschberg(2005), and it is the one of best designed deceptive speech corpus.

• Tripadviser-gold dataset was introduced by Myle(2011), and it is another widely used deceptive dataset which is text based.

11

Columbia SRI Colorado corpus

• 32 native American English speakers with a balanced gender from the Columbia university community.

• Subjects answered questions and performed activities in six areas: music, interactive, survival skills, food and wine knowledge, NYC geography, and civics.

• 'little lie' and 'big lie'

Tripadviser-gold dataset

• 400 truthful reviews from www.tripadviser.com, and 400 deceptive reviews from Amazon Mechanical Turk.

• All reviews evenly distributed across 20 most popular Chicago hotels, 20 deceptive reviews and 20 truthful reviews for each hotel with minimum 150 characters length reviews.

Deception Detection nonverbal approach

• Direct approach

• Lexical analysis

• Acoustic and prosodic analysis

• Speech event analysis

• Indirect approach

• Emotion

• Personality

14

Direct approach




Lexical Analysis

• Motivation: different patterns of word usage than truth teller (Newman et al. 2003, Qin et al. 2004, Zhou et al. 2004).

• Techniques: LIWC, DAL, POS, N-gram,

16

LIWC• Linguistic Inquiry and Word Count (LIWC)

calculates the degree to which people use different categories of words, and can determine the degree any text uses positive or negative emotions, self-references, causal words, and 70 other language dimensions. (Pennebaker et al. 2001).

• LIWC is often used for detecting emotion and cognitive changes for the speech content for deception detection. (Newman et al. 2003, Hirschberg et al 2005, Ott et al. 2011)

17

LIWC• Hirschberg (2005) found that positive emotion

words is the best indicator of deception by applying LIWC. Deceptive speech has a greater proportion of positive emotion words than truthful speech.

• Ott(2011) is another example of using LIWC and N-gram to detect deception. They found that truthful opinions tend to include more sensorial and concrete language than deceptive opinions, truthful opinions are more specific about spatial configurations.

18

DAL• The Dictionary of Affect in Language (DAL) lists

approximately 4500 (8,742 at 2009) English words, a rating for Pleasantness (Evaluation) and rating for Activation (Arousal) is associated with each word in the Dictionary (Whissell et al 1986). The difference between DAL and LIWC is that DAL focus is more narrow than LIWC, it only addresses the emotional meaning of words.

• Hirschberg (2005) used DAL to distinguish the different emotional state between deceptive and truthful speech. They found that higher average pleasantness score is more likely to be deceptive, and higher pleasantness standard deviation is less likely to be deceptive.

19

POS and N-gram• Part-of-speech tagging is the process of marking

up a word in a text (corpus) as corresponding to a particular part of speech, based on both its definition, as well as its context—i.e. relationship with adjacent and related words in a phrase, sentence, or paragraph.

• N-gram is a contiguous sequence of n items from a given sequence of text or speech. Usually many approaches combine the N-gram with other features to detect deception.

20

Direct approach




Acoustic and prosodic analysis

• Motivation: Literature suggest that pitch, energy, speaking rate, and other stylistic factors vary when speaker deceive (Depaulo et al 2003).

• Features: Energy, Pitch, Statistical features based on Energy, Pitch,

22

Acoustic and prosodic features

• Hirschberg(2005) used a wide range of potential acoustic and prosodic features, they extracted and modeled features including durational, pausing, intonational, and loudness, associated with multiple time scales from a few milliseconds to an entire speaker turn. (Pitch, energy and durational features were computed.)

• At the end, they found energy and f0 features play positive roles for deception detection among the wide range of acoustic and prosodic features.

23

Direct approach




Speech event analysis

• Motivation: Speech event such as filled pause, laughter is important to distinguish the interpersonal difference on communication behavior(Bortfeld, 2001), (Clark,1994), therefore, it might be also a useful predictor of deception speech.

25

“Um” and “Uh”• In Benus(2006), they found that subjects used the filled pauses significantly

more frequently in locally truthful than locally deceptive statements; turn internal silent pauses also occurred more frequently in truthful than in deceptive speech; there are significantly fewer pauses in lie than the truth and there is a tendency for latencies to be longer before lies than before truthful statements.

• They also did research for the differences between um and uh, and found that um was more likely to be followed by a silent pause than uh; a tendency for uhs to occur in utterances that were locally truthful but the subjects were expressing a global lie.

• In conclusion, people tend to be more careful about their word during lying than during telling the truth, and local deception does correlate with the use of um more than with the use of uh, um is longer, and has longer latencies with more silent pause surrounded.

26

Performance of Deception approaches

27

Performance of speech based deception detection

64

65.25

66.5

67.75

69

Accuracy

Newman (2003) Hirschberg (2005)Graciarena (2006) Benus (2006)Enos (2007)

Emotion• Motivation: Same person may feel different emotion

in different time period during telling the lie versus telling the truth. Vrij (2010) states that, compared with truth tellers, liars may experience stronger emotions, may experience higher levels of cognitive load, and are inclined to use more and different strategies to make a convincing impression on others.

• Automatic emotion recognition is a good way to help deception detection because of intra-personal difference during deception.

29

Emotion dataset• Acted speech

Berlin database of emotional speech (Burkhardt et al 2005)

Danish Emotional Speech Corpus (Schuller et al 2007)

• Elicited speech

Deception speech

• Natural emotions

30

Label of Emotion• The labeled emotions usually can be labeled in two

dimensions, arousal and valence.

• Arousal is a state of heightened activity in both our mind and body that makes us more alert (Berger, 2010). The dimension of valence ranges is from positive to negative

• Valence is the intrinsic attractiveness (positive valence) or aversiveness (negative valence) of an event, object, or situation (Frijda,1986). The dimension of arousal ranges is from high to low.

31

Arousal and Valence

32

Automatic Emotion Recognition

• There are usually three main steps for automatic emotion recognition.

1. Finding appropriate audio units

2. Feature extraction

3. Classification

33

Audio Unit• In order to recognize the emotion, digitalization and

acoustic preprocessing is required, and also segment the speech into meaningful units. Especially the audio segmentation is the goal for segmenting the speech signal into units address proper emotions.

• A good emotion unit should be fit two requirement.

1. Long enough to calculate features.

2. Short enough to have stable acoustic properties with emotion segmentation.

34

Feature

• Pitch and Energy

• Mel Frequency Cepstral Coefficients (MFCC)

Thomas (1998), Ververidis(2006), Truong(2012)

35

Personality• Motivation: Vrij (2010) states that one of major difficulty of deception

detection is interpersonal difference, and we think personality recognition is a key to solve this difficulty.

• Psychologist (DePaulo & Friedman, 1998) states that there are large individual differences in people’s behavior.

• Enos (2006) also found that judges with different personality perform different accuracy when they detect deceit, and we can hypothesize that personality test may also provide useful information in predicting individual differences in deceptive behavior of the speakers they judged.

• Therefore, personality analysis will be a useful cue to analyze interpersonal differences in different lying styles.

37

Big five or five factor model (Norman, 1963)

• O: Openness.

• C: Conscientiousness.

• E: Extraversion.

• A: Agreeableness.

• N: Neuroticism.

38

Automatic Personality Recognition

• Text based personality computing

For the text based analysis for the personality computing, many approaches usually used several very fundamental and widely known techniques, such as LIWC, MRC, N-gram, part of speech.

• Paralinguistic for personality perception

The “Interspeech 2012 Speaker Trait Challenge” has led to the first rigorous comparison of different approaches over the same data and using the same experimental protocol. A large number of features and machine intelligence approaches have been proposed, but none of them appears to clearly better than others.

39

Paralinguistic for personality perception

• Feature selection method

• Prosody features

• Other approaches

ASR + LIWC + Bayesian Networks ( SAIL )

40

PerformanceThe best performance of each paper in Interspeech 2012

0

25

50

75

100

Pohjalainen Chastagnol Ivanov Sanchez Attabi Audhkhasi Wagner

O C E A N

41

ConclusionCurrent approaches for detecting deception build upon the experiment of computer scientist based on the psychological theory of deception.

For the feature extraction, variety techniques were introduced for extracting lexical features, acoustic and prosodic features.

The emotion and personality may be a useful way to help deception detection on inter- and intra-difference.

For classification, various kinds of classifier were used for detecting deception, such as Support Vector Machine, neural networks and decision trees.

43

Future research• Continue work on Lexical, acoustic prosodic,

speech event feature.

• Develop a better performing system for personality recognition, and apply personality to deception detection.

• Find the best extractable features for verbal cues of deception across cultures.

44

Thank you 谢谢

감사합니다

45

Thank you 谢谢

감사합니다

Literature review of Deception Detection...deception detection on inter- and intra-difference. For...

Documents

Transcript of Literature review of Deception Detection...deception detection on inter- and intra-difference. For...