Automatic Shoeprint Retrieval System Automatic Shoeprint Retrieval System for use in Forensic Investigationsfor use in Forensic Investigations

Lin ZhangLin Zhang and and Nigel AllinsonNigel Allinson

Electronic Systems Design Research GroupDepartment of Electronic & Electrical EngineeringUniversity of Sheffield

Crime Scene Investigation

DNA

Shoeprint

Bullet

Cartridge case of firearm

FingerprintMake unique Make unique identificationidentification

Helpful in Helpful in recognitionrecognition

Face

Shoeprint Recognition

Shoeprints are often found at crime scenes and contribute considerably to forensic intelligence.

Identify linked crime scenes

Link suspects in custody to other crime scenes

Permit the targeting of prolific offenders

Provide strong courtroom evidence when detailed matching of mark and shoe exist

Database of impressions made by shoes available on the market

Database of footwear impressions found at other crime scenes

Database of impressions made by shoes from suspects

Forensic analysis requires comparison of shoeprint images against specific databases.

An image of the shoeprint can be obtained using photography, gel or electrostatic lifting or by making a cast when the impression is in soil.

Current solutions

There’s a danger of inconsistency between the codes and users.

Modern shoes have increasingly more intricate outsole patterns that are difficult and tedious to describe with only a few basic coding shapes.

Manually: search through paper catalogues

Slow, tedious, need considerable training!!!

Semi-automatically: Computer databasesHuman coding of shoeprint outsole patterns based on shape primitives (e.g. lines, circles, logos, zigzag, etc)

Aim Aim

The aim of this study is to develop a fully automatic shoeprint recognition system.

Sort the database in response to a query imageSort the database in response to a query image

Functions with minimum user interventionFunctions with minimum user intervention

Insensitive to scale, rotational and translational Insensitive to scale, rotational and translational variance in query imagevariance in query image

Database Database

A subset of 512 images from Foster & Freeman Ltd’s shoeprint database SoleMate (includes over 8000 different sole patterns)

System Overview System Overview

Feature Extraction

Feature ExtractionShoeprint

Image Databas

e

Pre Processing

Query

Correct Match

Pattern Matching

Display Ranked

list of Images

User SelectionUser Selection

image pre-processing feature extraction pattern matching

Image pre-processing Image pre-processing

PDE (partial differential equation)-based de-noising approach to implement edge preserving smoothing under controlled curvature motion

Evolving the image I as a surface is equivalent to repeatedly iterating the edge-preserving anisotropic filter:

2 2

1 2 2 3/ 2

(1 ) 2 (1 )

2(1 )xx y x y xy yy x

t tx y

I I I I I I II I

I I

Results of applying the filter to typical noisy images for 40 iterations. Noise effects are attenuated and useful edges are preserved.

Feature Extraction Feature Extraction

Canny edge detector edge image

An edge direction histogram of 72 bins is used to record edge directions quantized at 5o intervals.

Matching such histograms is sensible to rotational and scale variances

Normalize the histogram

( ) ( ) / , [0,1,...,71]eH i H i n i

H(i): count in bin I

ne: total number of edge points

The DFT coefficient vector is used as the feature extracted from image.

Calculate 1-D DFT coefficients on the normalized histogram

Pattern Matching Pattern Matching

0 10 20 30 40 50 60 70 800

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 10 20 30 40 50 60 70 800

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

DFT vector

of input image

DFT vectorof database

images

Euclideandistance

Sorted list of database images

Images with similar patterns as query image will stay on TOP of the ranking list.

Edgeimage

0 10 20 30 40 50 60 70 800

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0 10 20 30 40 50 60 70 800

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0 10 20 30 40 50 60 70 800

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

NormalizedEdge direction

histogram

0 10 20 30 40 50 60 70 800

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 10 20 30 40 50 60 70 800

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 10 20 30 40 50 60 70 800

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

DFT vector

De-noised shoeprint

(a)

(b)

(c)d(a,b)=0.260; d(a,c)=0.911; d(b,c)=0.802

AccuracyAccuracyRank the 512 database images from best to worst match and return top 20 for further investigation.

Stability Stability Original: All images in the database. Rotated: Every image in the database rotated randomly and

used as query. Scaled: Every image in the database scaled randomly and used

as query. Noisy: Random noise added to every image in the database

and used as query.

SpeedSpeed matching process needs about 1s for one image (excluding the pre-processing time)

Experimental Results Experimental Results

2 4 6 8 10 12 14 16 18 2030

40

50

60

70

80

90

100

110

n

%

OriginalRotated

ScaledNoised(10%)Noised(20%)

Probability of correct retrieval in the first n positions

Query result n =1(%)

n ≤ 2(%)

n ≤ 3(%)

n ≤ 4(%)

n ≤ 5(%)

n ≤ 20(%)

Not Retrieved(%)

Original 99.41 100 100 100 100 100 0

Rotated 48.83 58.97 63.87 67.77 71.09 87.50 12.50

Scaled 96.09 97.27 97.66 98.05 98.05 99.61 0.39

Noisy (10%) 85.35 90.04 92.77 93.95 94.53 97.66 2.34

Noisy (20%) 57.42 62.89 67.19 69.73 71.29 84.96 15.04

Pre-align images: rotated about the centroid, major axis parallel to y-axis

Highly deteriorated

Future workFuture work

Identify partial shoeprintsIdentify partial shoeprints

Incorporate some neural network Incorporate some neural network methodsmethods

Investigate and test alternative de-Investigate and test alternative de-noising methodsnoising methods

AcknowledgmentsAcknowledgments

Foster + Freeman