Towards Recommender Systems for Police Photo Lineup

Towards Recommender Systems

for Police Photo Lineup

Ladislav Peškaa and Hana Trojanováb

a Department of Software Engineering,

b Department of Psychology,

Charles University, Prague,

Czech Republic

Ladislav Peska & Hana Trojanova: Towards

Recommender Systems for Police Photo Lineup

2

Recommender Systems

Propose relevant items to the right persons at the right time

Successful on many various domains

Multimedia

E-commerce

POIs

Events

Recepies

…

However, some novel (relevant, important) tasks are yet to

discover

DLRS@RecSys2017, Como



3

Police Photo Lineup

Eyewitness identification of the suspect / offender during

criminal proceedings

Most common case: select suspect among other persons (fillers)

Either „in natura“ or based on photographies


- Suspect is positioned at

random

- 3-7 fillers are added, witness

should not know them

- Wittness is instructed that the

offender may or may not be

present



4

Errors in Photo Lineup


1) Observe the event

2) Identify the offender - among other candidates



5

Errors in Photo Lineup


1) Observe the event. - numerous sources of error (poor lighting,

large distance, short time, fear, fatigue…)

- affects memory and recognition processes

- decrease witness certainty and reliability

2) Identify the offender - decreased certainty may lead

to incorrect identification

- and conviction of innocent

persons



6

Assembling Fair Photo Lineups

System should be set to eliminate testimony of uncertain

witnesses

Double-blind administration

Fair (unbiased) assembling of lineups

Out of the dataset of candidates, lineup administrator should select

fillers similar to the suspect

Currently, only feature-based filtering systems are available

Better automatization in content processing is necessary

How to better approximate human-perceived similarity of

persons?

Convolutional Neural Networks?




7

Assembling Fair Photo Lineups

First approach: item-based recommendations

Propose top-k candidates for each suspect

CB-RS, cosine similarity of candidates’ and suspect’s CB features

Nationality, Age, Appearence features

Baseline (mimic feature-based filters)

Visual-RS, cosine similarity of candidates’ and suspect’s VGG-

Face probability layers

Approximate the similarity of a candidate and the suspect through their

similarities with other persons

VGG-Face1 network

VGG network for facial recognition problem

Trained as a multi-classification problem

Dataset of 2,622 celebrities with 1,000 images each

1Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015). Deep Face Recognition. British Machine Vision Conference.




8

Research Questions

Is candidates recommendation (based on VGG-Face’s

features) suitable for the lineup assembling?

How well is the human-perceived similarity approximated within top-k

candidates?

What is the role of diversity in lineup assembling?

Is there a space for personalized recommendations (i.e. personalized per

lineup administrator)?

Can there be „too similar“ candidates?




9

Evaluation – Protocol

Dataset of missing and wanted persons in Czech Republic

4423 males

User study, 30 different lineups, 7 forensic technicians

For each lineup, both CB-RS and Visual-RS proposed top-20 candidates

Merged into one list, forensic technician should select relevant fillers for

particular suspect

Follow-up questionaire and discussion




10

Results

In total 202 completed lineups, 800 selected candidates

Recommendations were seemingly relevant in most cases

Visual-RS considerably outperformed CB-RS

The performance difference was much smaller for suspects from outside

of central Europe

Effect of different ethnicity? (Known in the forensic psychology literature).


Total lineups Selected candidates Level of agreement

All lineups

Visual-RS 202 466 (58%) 0.178

CB-RS 298 (37%) 0.138

Lineups with suspects from countries outside Central Europe

Visual-RS 53 105 (51%) 0.128

CB-RS 82 (40%) 0.205



11

Results II.

Very low intersection (1.8%) of top-20 CB-RS and Visual-RS candidates

Potential to apply some combination in the future work

Average position of selected candidate

CB-RS: 8.9 (+/- 5.2); Visual-RS: 8.2 (+/- 5.7)

Ordering seems relevant even in top-20, however can be improved

Low level of agreement among participants on the selected candidates (0.178, 0.138)

Limits of the global ordering?

Potential for personalized recommendation?

Less diversity is better

Participants agreed on the need for providing homogeneous lineups

Dynamical recommendation based on the selected candidates?

No seemingly too similar candidates

However, the dataset contained some duplicates

Should be addressed in the future work


Conclusions

Recommending candidates based on CNN’s features seems

plausible in police photo lineup Item-based recommendations seems like a good first approach

There is a potential for:

Session-based personalized recommendations (need for uniformity of the final list)

Long-term preference learning (due to the level of disaggreeement among lineup

administrators)

Approaches combining CNN’s features with content-based attributes

Future work Move from recommended candidates to recommended lineups

Evaluation through mock witnesses

Deployable demo

DLRS@RecSys2017, Como Ladislav Peska & Hana Trojanova: Towards


12

DLRS@RecSys2017, Como Ladislav Peska & Hana Trojanova: Towards


13

Thank you!

Questions, comments?

Supplementary materials: http://github.com/lpeska/lineups, https://tinyurl.com/yat3rtr2

Slides: https://www.slideshare.net/LadislavPeska/towards-recommender-systems-for-police-photo-lineup

http://github.com/lpeska/lineups

http://github.com/lpeska/lineups

https://tinyurl.com/yat3rtr2

https://tinyurl.com/yat3rtr2

https://www.slideshare.net/LadislavPeska/towards-recommender-systems-for-police-photo-lineup















Towards Recommender Systems for Police Photo Lineup

Science

Transcript of Towards Recommender Systems for Police Photo Lineup