1

Tennis Video Shot Classification Based On Support Vector Machine

Computer Science and Automation Engineering (CSAE), 2011 IEEE International Conference on

2

Outline Introduction Feature extraction Shot classification Experiment results Conclusion

3

Introduction Source: tennis video Present method: mostly based on color distribution Propose a method based on SVM Shot type: long shot , audience shot , close shot , close

up shot

4

Feature extraction Edge Distribution Descriptor – 5 values

Optical Flow Feature – 2 values

Shot Classification Feature – 7 mean values

5

Feature extraction(cont.) Edge Distribution Descriptor

Each key frame partition into five windows Use sobel edge detector to detect edge pixels

6

Feature extraction(cont.) Edge Distribution Descriptor

Calculate edge pixels ratios of each window

MxN : total pixel

G(x,y):gray-scale value

i:window ID

P(I):edge pixels ratio

7

Feature extraction(cont.) Optical Flow Feature

The optical flow of an image sequence is a set of vector fields Each vector field represents the apparent displacement of each

tracked point from one image to the other

From Lihi Zelnik-Manor’s Slide

8

Feature extraction(cont.) Optical Flow Feature

The Optical Flow Field : find for each pixel a velocity vector which says

(high computation cost)

– How quickly is the pixel moving across the image

– In which direction it is moving

From Lihi Zelnik-Manor’s Slide

9

Feature extraction(cont.) Optical Flow Feature

Corner detection: the improvement of Kitchen and Rosenfeld method Detect corner points

Adopt pyramidal Lucas-Kanade method Use selected corner points to calculate motion vector between

two consecutive images

10

Feature extraction(cont.) Optical Flow Feature

Figure 3. optical flow field:(a)audient shot,(b) close shot (c) long shot (d) close up shot

11

Feature extraction(cont.) Optical Flow Feature

Extract two features from optical flow field FPR (foreground tracked points ratio) MLV (mean length of motion vectors)

Optical flow filed varies with the difference of camera behavior Global movement : whole image space Local movement : concentrate on the area of players Use motion vector histogram descriptor to estimate

12

Feature extraction(cont.) Optical Flow Feature

Motion type The whole optical flow field is split into ten slices

Count total number of motion vectors whose length is nonzero for each slice

i : 1,2,…,10Ci :the set of motion vector which locate in the ith slicehi :total number of motion vector in the ith slice

1 2 3 4 5 6 7 8 9 10

Figure 4(a)

local

global

13

Feature extraction(cont.) Optical Flow Feature

N(I)h : count the number of nonzero hi

Figure 4(b) motion vectors histograms of (a)

local

global

14

Feature extraction(cont.) Optical Flow Feature

FPR (foreground tracked points ratio) Classify the motion vectors into eight different directions Define MaxD(I) as the main orientation of motion vectors N(I)maxd :count the number of vectors whose direction is

MaxD(I) N(I)con : count the number of vectors whose length is nonzero

MLV (mean length of motion vectors)

15

Feature extraction(cont.) Shot Classification Feature

the mean value of all feature data of the key frames in the same shot should be calculated for shot classification define Kij as the jth key frame of the ith shot

the mean value of the wth window’s edge pixels ratio in the ith shot

16

Shot classification A novel method of tennis shot classification based on

support vector machine (SVM) SVM is a binary classifier as we all known Construct a four-class classifier because of four tennis

shot types Use seven features as the input vectors for SVM Construct 4 ×(4 - 1) / 2 binary SVMs by training on

data

17

Experiment results Tennis videos :Tennis Masters Cup of Shanghai, French

Tennis Open and America Tennis Open Videos size :720x404 All together 2285 key frames in different shots divided

into two subsets respectively for training and testing

18

Experiment results We compare our approach with the color distribution

based approach [1] over the same dataset

19

Conclusion Our approach obtains a higher accuracy of

classification because of our improvement of the sensitivity to light compared to traditional methods.

More effective methods of selecting the motion vectors in the foreground will be paid more attention in the future work.