Human Identity Recognition in Aerial Images

Omar OreifejRamin MehranMubarak Shah

CVPR 2010, JuneComputer Vision Lab of UCF

Outline• Introduction• Challenges• Problem Definition• Weighted Region Matching (WRM)– Pre-processing steps

• Human Detection• Blob Extraction• Alignment

– Measuring the Distance Between Blobs– Determining the Voter’s Weight

• Experiments and Results

Introduction

• Identity recognition from aerial platforms is a daunting task.– Highly variant features in different poses– Vanish details under low quality images

• In tracking, objects are usually considered to have small displacements between observations.– Mean Shift [4]– Kalman filter-based tracking– with long temporal gaps, all assumptions of the continuous

motion models become weak

Challenges

• Low quality images• High pose variations• Possibility of high density crowds

• We employ a robust region-based appearance matching.

Problem Definition

• A user is able to identify a target person over a short period of time.

• Humans maintained their clothing and general appearance.• We define the problem as a voter-candidate race.

Weighted Region Matching (WRM)

where P(vi) is the voter’s prior.

Weighted Region Matching (WRM)

• Equation (1) can be rewritten in a form similar to a mixture of Gaussians:

• where τ is a constant parameter

• Provide a robust representation of the distance between every voter-candidate pair.

• Specify the weight of every voter.

Human Detection• We train a SVM classifier based on the HOG descriptor

[6].• 6000 positive images: – humans at different scales and poses

• 6000 negative examples: – the background and non-human objects

• Train over a subset of 9000.• Validation using the rest of the dataset.

Blob Extraction

• The background regions contained in the bounding boxes do not provide any information about a specific person.

• Segmentation method: kernel density estimator [12, 15]

i

ii xxKxf )()(ˆ

Estimate the pdf directly from the data without any assumptions about the underlying distributions.

Alignment

• To eliminate the variations from camera orientation and human pose.

• Edge detection is noisy.• A coarse alignment:– eight point head, shoulders and torso (HST) model– The model captures the basic orientation of the

upper part of the body.

Alignment

• Find the best fit of the HST model over human blobs– we train an Active Appearance Model (AAM)

Alignment

• We employ to compute an affine transformation to a desired pose.

• Align all the blobs to the mean pose generated by the AAM training set.

Measuring the Distance Between Blobs

• Treat blob as a group of small regions of features.

• These features compose:– Histograms of HSV channels– The HOG descriptor

• We apply PCA on the feature space and extract the top 30 eigen vectors.

Measuring the Distance Between Blobs

• Using Earth Mover Distance [16, 14] (EMD)Compute the minimum cost of matching multiple regions.

Having each region represented as a distribution in the feature space

Measuring the Distance Between Blobs

Number of pixelsNumber of pixels

bin bin

• Total cost in the example : 1·1+2·2=5, EMD=5/3• For two distributions, P = {pi} and Q = {qi}

P Q

Determining the Voter’s Weight

• We rank the collection of input images according to the value of information.

• Given the set of regions from all voters, R = {rk}– We assign a weight for every region such that the

most consistent regions are given higher weights– Use the PageRank algorithm [3]

PageRank

• Conception– Vote– based on a random walk algorithm A

B D C

PR(A) = PR(B) + PR(C) + PR(D)

VisualRank: Applying PageRank to Large-Scale Image Search,余償鑫 

PageRank

A

B

D

C

VisualRank: Applying PageRank to Large-Scale Image Search,余償鑫 

PageRank

VisualRank: Applying PageRank to Large-Scale Image Search,余償鑫 

In G, we connect every region from voter i to the K nearest neighbor regions of voter j where i != j.

The final weight for a region rk:

Region sizePR

the voter’s weight wi =normalized sum of weights of its regions

Matching

• Substituting the distances and the weights in equation 2, we compute a probability for every candidate to belong to the target.

• The best match should be the candidate with the highest probability.

Experiments and

Results

Experiments and

Results

Experiments and

Results