AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is...

13
http://www.iaeme.com/IJARET/index.asp 59 [email protected] International Journal of Advanced Research in Engineering and Technology (IJARET) Volume 6, Issue 11, Nov 2015, pp. 59-71, Article ID: IJARET_06_11_006 Available online at http://www.iaeme.com/IJARET/issues.asp?JType=IJARET&VType=6&IType=11 ISSN Print: 0976-6480 and ISSN Online: 0976-6499 © IAEME Publication ___________________________________________________________________________ AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS Hari R Department of Computer Science, University of Kerala ABSTRACT Video summarization is a useful technique in present world to view all the important aspects of a lengthy video. Video summarization extracts important representative shots from the video and it conveys all the semantics of the entire video in a short span of time. In sports, it plays a very important role to generate highlight of the game video spanning over many hours. Hence we propose an algorithm for automatic summarization of lengthy hockey game videos. In this method, different efficient algorithms are devised to find shot detection, penalty corner and penalty stroke detection, Umpire detection and foul detection. The method also detects all the replay shots along with logo shots. The method finally combines all the important events detected to form a summarized video. It also allows the user to create customized video summary containing user preferred events. The method is tested with many international Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical Flow, Hough Transform. Cite this Article: Hari R. Automatic Summarization of Hockey Videos. International Journal of Advanced Research in Engineering and Technology , 6(11), 2015, pp. 59-71. http://www.iaeme.com/IJARET/issues.asp?JType=IJARET&VType=6&IType=11 1. INTRODUCTION Content based video retrieval technique helps the user to browse the preferred events in the sports video in a short span of time. Enormous research works are going on in the automatic highlight generation of various sports video that includes all the relevant events happening in the game. Since the sports videos are in unscripted pattern, we have to extract out the video scenes containing few semantically related and continuously record image sequence [1] [2]. Most of the sports video summarization works exploits the visual and audible features of the game video [3]- [6] such as detecting the whistle sounds, ground excitement, text boxes etc. in the video. Another approach is to exploit the semantic descriptions of the sports video, which can be done either by using a Bayesian network [7], Ontology modeling [8] [9],

Transcript of AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is...

Page 1: AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical

http://www.iaeme.com/IJARET/index.asp 59 [email protected]

International Journal of Advanced Research in Engineering and Technology

(IJARET) Volume 6, Issue 11, Nov 2015, pp. 59-71, Article ID: IJARET_06_11_006

Available online at

http://www.iaeme.com/IJARET/issues.asp?JType=IJARET&VType=6&IType=11

ISSN Print: 0976-6480 and ISSN Online: 0976-6499

© IAEME Publication

___________________________________________________________________________

AUTOMATIC SUMMARIZATION OF

HOCKEY VIDEOS

Hari R

Department of Computer Science, University of Kerala

ABSTRACT

Video summarization is a useful technique in present world to view all the

important aspects of a lengthy video. Video summarization extracts important

representative shots from the video and it conveys all the semantics of the

entire video in a short span of time. In sports, it plays a very important role to

generate highlight of the game video spanning over many hours. Hence we

propose an algorithm for automatic summarization of lengthy hockey game

videos. In this method, different efficient algorithms are devised to find shot

detection, penalty corner and penalty stroke detection, Umpire detection and

foul detection. The method also detects all the replay shots along with logo

shots. The method finally combines all the important events detected to form a

summarized video. It also allows the user to create customized video summary

containing user preferred events. The method is tested with many international

Hockey match videos and the average efficiency is found to be 0.88.

Key words: Video Summarization, Colour Segmentation, SSIM, Optical

Flow, Hough Transform.

Cite this Article: Hari R. Automatic Summarization of Hockey Videos.

International Journal of Advanced Research in Engineering and Technology,

6(11), 2015, pp. 59-71.

http://www.iaeme.com/IJARET/issues.asp?JType=IJARET&VType=6&IType=11

1. INTRODUCTION

Content based video retrieval technique helps the user to browse the preferred events

in the sports video in a short span of time. Enormous research works are going on in

the automatic highlight generation of various sports video that includes all the

relevant events happening in the game. Since the sports videos are in unscripted

pattern, we have to extract out the video scenes containing few semantically related

and continuously record image sequence [1] [2]. Most of the sports video

summarization works exploits the visual and audible features of the game video [3]-

[6] such as detecting the whistle sounds, ground excitement, text boxes etc. in the

video. Another approach is to exploit the semantic descriptions of the sports video,

which can be done either by using a Bayesian network [7], Ontology modeling [8] [9],

Page 2: AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical

Hari R

http://www.iaeme.com/IJARET/index.asp 60 [email protected]

classification of temporal structure of the game [10] [11], etc. A novel motion

analysis method for generating highlights of racket sports video is proposed in [12],

which exploits the player behavior and audience response and it was tested on

broadcast tennis and badminton videos. A team-sports video summarization based on

the knowledge about displayed content and the individual preferences of the user

which was experimented on soccer, basketball and volleyball videos is presented in

[13]. In [14], the storyboard summary generation of snooker video is carried out using

a hierarchical event representation framework and importance based selection

algorithm. Highlight based summary generation of cricket videos by the detection of

events like FOUR, SIX, OUT, RUN etc. can be done using visual and aural features

as well as semantic description techniques of the game video [15] – [18]. From all

these, it is clear that novel attempts were carried out to summarize sports videos like

soccer, snooker, tennis, cricket etc. whereas efforts for the detection of events in a

Hockey game is still in the infant stage.

Hockey is a game played between two teams, each having eleven players

including the goal keeper. The players hit a small leather ball with a curved stick and

if the player of team A strikes the ball and if it is made to enter into the goal post of

team B, then team A is said to score a ‘goal’. The Hockey field is rectangular in shape

having a large center line which is termed as the ‘Halfway line’. In addition to this,

the field also consists of side lines and goal lines on either side of the center line

nearer to the goal posts that are situated at the two ends of the field as shown in Figure

The goal keepers of each team will take the position in front of the goal post and they

will always try to block the ball hit by the players of the opposite team from entering

his/her goal post. There will be two Umpires in the field wearing a jersey with a

colour other than the colour of team players’ jersey, who continuously monitor the

game and decide penalty serving and foul by displaying green, red and yellow cards

to the players in the game.

The present work concentrates on highlighting the major three events in the

Hockey game video like ‘goal’, ‘penalty corner’, ‘penalty stroke’ and major ‘fouls’.

The organization of the paper is as follows: Section 2 describes the proposed method

along with the details of the techniques used for shot detection and classification,

event detection and summarization. Section 3 discusses the experimental results

obtained and section 4 concludes the work.

2. PROPOSED WORK

In this work, we propose a new methodology for automatic summarization of Hockey

game video by using the inherent features of the game as well as the game dependent

structured motion pattern and recording fashion of the cameras. In a Hockey match,

the most important event is scoring of a goal as well as the attempt to score a goal. In

addition, penalty stroke, penalty corner and serious fouls can also be treated as

important events. Here, the methodology developed can automatically create a

summarized video containing all the above events. The overview of the system is

given in figure 1.

Initially, all the frames of the Hockey video are undergone preprocessing steps for

enhancing the features by removing noises. These frames are then analyzed to find the

shot boundaries for grouping the frames into different shots. After this, the replay

shots are found out and removed from further processing. The frames of the

remaining shots are then analyzed to find the features contained in them for their

classification into the following categories.

Page 3: AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical

Automatic Summarization of Hockey Videos

http://www.iaeme.com/IJARET/index.asp 61 [email protected]

1. Long shots: The shots recorded by the camera without zooming such that the

prominent area in each of the frame in a shot is occupied by the field.

2. Close-up shots: These shots contain frames within which the face of the player or

Umpire appears.

3. Umpire shots: Here the frames in the shot contain the Umpire, whose presence

can be identified by the unique color of the Umpire uniform that can be easily

distinguished from the players.

4. Goal post/Goal mouth shots: These shot frames contain goal post.

5. Logo shot: In all game videos, computer generated logo graphics are flashed

frequently. Normally, the logo shots are displayed at the beginning and end of

replay shots.

6. Replay shot: Slow motion replays are always telecasted just after the occurrence

of an important event.

Figure 1 Overview of the system

Now the occurrence pattern of the above said shots are examined to deduce the

presence of important events in those shots. These events are finally combined to

create summarized Hockey video. In the following sections the details of the

techniques used to detect and classify the shots are explained.

(a) (b) (c)

(d) (e)

Figure 2 Sample Frames of (a) long shot, (b) close-up shot, (c) logo shot, (d) umpire

shot and (e) goal post shot

Shot

detection Input

video

frames

Shot classification

Long

shots

Replay

shots

Goal post

shots

Audience

shots

Umpire

shots

Event detection Summari-zed

video

Page 4: AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical

Hari R

http://www.iaeme.com/IJARET/index.asp 62 [email protected]

2.1. Shot detection

In Hockey video, multiple cameras are used to capture the events. In this context, a

shot can be designated as a group of frames continuously recorded by a single camera

and hence all the frames in a shot exhibit structural and spatial similarity. Structural

similarity index measure (SSIM) [19] is used to find the similarity between adjacent

frames. If the similarity value between adjacent frames is above a threshold value,

then those frames belong to a single shot. To proceed further, the groups obtained so

far will be termed long shots, close-up shots, Umpire shots and goal post shots.

2.1.1 Detection of Logo shot

The duration of logo shots are less than a second. Compared to other shots in the

video, this shot is of very small. Moreover the colour contrast in this shot is found

higher than that of other shots. Hence these two cues are used to distinguish the logo

shots from others. All the shots whose duration is less than a second, i.e., shots

containing less than 30 frames, are considered for selecting as logo shots. In addition,

the contrast in the selected frames is calculated and if this is greater than that of

frames in other shots, the shot is finally designated as logo shot.

2.1.2 Detection and removal of replay shots

In Hockey video, the important events are always followed by its replay in slow

motion. These replay shots are always telecasted between two logo shots. Hence all

the shots coming between two adjacent logo shots with duration less than 2 minutes

are considered as replay shots and they are removed immediately from further

processing, i.e., all shots excluding the logo shots and replay shots are given to next

modules for further processing. Since this is preceded by important shots, once a

replay shot is detected by the system, the system tags the preceding shot as important.

2.2. Shot classification

All the shots except replay shots and logo shots are thoroughly analyzed for detecting

the presence of objects like field, goal post, players and umpires. Based on these

features, shots can be classified into one of the following using different techniques.

Long shot: In long shots, the frames contain a distant view of the playground. Hence

the major portions of the frames in these shots are occupied by the field. Using color

segmentation technique, we can detect the field and if its area is greater than ¾ of the

frame, after the removal of minor objects in the frame using morphological

operations, shot is tagged as long shot. In different hockey matches, synthetic fields

of different colures are used. Our system automatically selects the major colour in the

first few frames of the video as the field colour for processing the remaining frames.

Close-up shot: During a game, the camera focuses either on a single player or players

or the umpire depending on the events occurring in the shot. For example, when a

player scores a goal, the camera starts focusing on the goal post, then on the player

who scored the goal and the players who gathered around the scorer to express their

immense emotions. To detect a close-up shot, our method tries relies on two methods,

namely field colour detection and skin colour detection. When a close up view is

displayed, the field is either missing or partially present in the frames. Hence the ratio

of field colour pixel to total pixel in the frame is computed and if it is less than or

equal to ¼, the shot can be considered as a candidate shot for close up shot selection.

To confirm its candidature as close up shot, skin colour segmentation is carried out

next to detect the face as well as the hands of the players. If the method can detect the

skin within the frames of the above shot, it is finally tagged as a close up shot.

Page 5: AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical

Automatic Summarization of Hockey Videos

http://www.iaeme.com/IJARET/index.asp 63 [email protected]

Umpire Shot: The game is controlled by Umpires, wearing distinct colour dress.

Using colour segmentation technique, the Umpires can be easily distinguished from

the players of both the team. Since the Umpires wear different colour uniforms in

different matches, the system needs human intervention to select the colour of

Umpires’ uniform in the initial phase.

(a) (b)

Figure 3 The output of Umpire detection procedure. (a) Original image, (b) Colour

segmented image.

Goal post shots: Two goal posts are located at the two ends of the field, which are

characterized by vertical strips / poles that are connected by a horizontal strip. Hough

transform is used to detect the vertical and horizontal strips of the goal post and

morphological operations are carried out to find the rectangular goal post. If any one

of the frames in a shot contains the goal post, the shot is classified as goal post shot.

(a) (b) (c)

(d) (e)

Figure 4 Illustration of goal post detection process. (a) original goal post frame, (b)

binary image of (a), (c) image after performing morphological operations on (b), (d)

lines detected by Hough transform in (c) and (e) the detected boundary lines of the

goal post

2.3. Event detection

In Hockey video, the major events include scoring of goals, attempt to score a goal,

foul, penalty stroke etc. All these events in the video are characterized by unique

features like presence of goal post, signals of umpire and the emotions shown by both

the players and spectators. In the proposed work, the gestures of umpires and the

presence of objects or events are taken into account for detecting important events and

thereby creating effective summary of the Hockey video.

2.3.1. Goal detection

If a goal is scored, the shot contains the goal post and hence goal event is always

included in the goal post shot. The immediate shot coming after this contains the

close-up view of the player or players showing emotions. The goal event is always

followed by a set of replay shots also. The system checks the occurrence of this

sequence and if the sequence is detected, the system selects the goal post shot as goal

shot or goal event and the shot is also tagged with the name ‘goal’. Hence the same

Page 6: AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical

Hari R

http://www.iaeme.com/IJARET/index.asp 64 [email protected]

shot is having two tags, namely ‘goal post’ and ‘goal’. A failed goal attempt also have

the same structure, since in the summarization point of view, both the events i.e.,

scoring a goal and failed attempt to score a goal are equally relevant. Hence the

technique can be well adopted in this work.

2.3.2. Penalty corner/ Penalty stroke detection

If a foul occurs in the penalty area of a team, the Umpire awards a penalty corner or

penalty stroke for the opposite team. In all these events, the players take a preparation

time before performing it. During this time, the camera focuses on the penalty area

wherein the goal post exists. To detect the penalty corner, our system estimates

motion in all goal post shots. If the motion of such a shot is very less and below a

threshold and if the shot spans over a minimum of three seconds, it can be taken as a

penalty corner. In this work, optical flow method [20] is used to extract the motion in

the goal post shots. If the above conditions are met in a goal post shot, it is named

with ‘Penalty corner/ Penalty stroke’ tag. If the sequences club with that of goal

detection sequence, then the penalty corner may end in goal event. In this case the

same shot has three tags, namely, ‘goal post’, ‘Penalty corner/Penalty stroke’ and

‘goal’.

2.3.3. Foul detection

The Umpire signals the occurrence of fouls by showing either green or yellow or red

cards. Of these both red and yellow cards indicate major fouls. Hence our system

detects the fouls indicated by both yellow and red cards and the respective Umpire

shot and its preceding shot in which the foul occurs are tagged as ‘foul’ shot. The

method to detect these foul events is implemented through the following steps.

1. Skin color cum yellow and red color segmentation is carried out in frames of all

Umpire shots.

2. The resulting frame in step (1) is analyzed for the presence of yellow or red color.

If either yellow or red color is present, step (3) is performed.

3. Connected component analysis is carried out to find whether the yellow or red

color of the card appears immediate to skin color which is contributed by the

hand of the Umpire.

(a) (b) (c)

Figure 5 Illustration of foul detection through Umpire signal. (a) the Umpire shows

yellow card after a foul committed by a player, (b) the colour segmented image of (a),

(c) connected component analysis is performed on (b) to verify the presence of yellow

colour in hand.

2.4. Video summarization

All the shots are detected and tagged with the name of the event contained in it as

mentioned in the above sections. In this module, the method selects all the shots

tagged with important events. This includes penalty stroke, penalty corner, goal and

major fouls. Then all these events are stitched together in the same order as they

appear in the original video to form the summary of the video. In addition, the method

Page 7: AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical

Automatic Summarization of Hockey Videos

http://www.iaeme.com/IJARET/index.asp 65 [email protected]

also enables the user to create his own summary, containing those interested events

like all goal post shot alone or all goal shots using the tags of the shots.

3. EXPERIMENTAL RESULTS

The method is experimented with a large number of Hockey matches being played in

many places under both sunlight and flood lights. All the matches are in MPEG

format recorded in full HD (1920x1080) resolution recorded at 30 fps. For the

evaluation purposes, clips from first half of eight games are randomly chosen and the

general details of the games are given in table 1.

Table 1 General details of the game dataset

Game Match Place Date Details

Length of

the video

clip (min)

H1 England Vs

Pakistan Bhubaneswar India 7/12/2014 1st round match 38:58

H2 Belgium Vs

Australia Bhubaneswar India 7/12/2014 1st round match 41:57

H3 Netherlands Vs

India Bhubaneswar India 9/12/2014 1st round match 35:55

H4 Argentina Vs

India Bhubaneswar India 7/12/2014 1st round match 34:48

H5 New Zealand Vs

England Mendoza, Argentina 4/12/2014

Quarter Final

(Women’s

World Cup)

39:26

H6 Germany Vs

India

Delhi

India 13/1/2014 Final 36:03

H7 Argentina Vs

Australia Bhubaneswar India 11/12/2014 Quarter Final 2 37:38

H8 Belgium Vs

India Bhubaneswar India 11/12/2014 Quarter Final 4 17:43

The results for individual modules can be analyzed separately. For the evaluation,

recall, precision and F-score measures are used.

The performance of the shot detection module is evaluated by , , and

parameters. If is the number of correctly detected events verified

manually , is the number of events detected by this method and is the actual

number of events identified by the human, then we can define these parameters as,

The value of all the parameters range from 0 to 1. A high value indicates

the effectiveness of the method in finding correct shots. The reflects the ability of the method to avoid false shot detection.

Page 8: AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical

Hari R

http://www.iaeme.com/IJARET/index.asp 66 [email protected]

3.1. Shot boundary detection

In the games taken for experimentation, the total shots is the summation of shots

containing the game, logo shots, one or more replay shots included between a pair of

logo shots. The shot detection results are given in table 2.

Table 2 Shot boundary detection results

Game Ms Ts Cs Recall

(Cs/Ms)

Precision

(Cs/Ts) F-score

H1 297 289 271 0.91 0.94 0.92

H2 368 370 339 0.92 0.92 0.92

H3 324 332 308 0.95 0.93 0.94

H4 318 314 296 0.93 0.94 0.93

H5 418 426 375 0.9 0.88 0.89

H6 340 345 322 0.95 0.93 0.94

H7 315 322 287 0.91 0.89 0.9

H8 217 223 202 0.93 0.91 0.92

Note: Ms- Total number of shots manually detected, Ts- Total number of shots detected by the

method and Cs-Number of correctly detected shots

From the table, it can be found that the method very well detects the shot

boundaries leaving an average recall, precision and f-score values of 0.93, 0.92 and

0.92 respectively.

3.2. Replay detection

Since the logo shots are computer generated graphics shots, the detection of logo

shots and replay shots can easily be carried out with high accuracy. Each replay may

contain multiple shots featuring the view of the same shot in different angles. The

replay results are given in table 3. It can be seen from the table that most of the replay

shots are correctly detected by the system, proving the efficiency of the replay

detection algorithm. Table 3 Replay detection results

Game MR TR CR Recall

(CR/MR)

Precision

(CR/TR) F-score

H1 34 35 33 0.97 0.94 0.95

H2 44 43 40 0.91 0.93 0.92

H3 50 50 47 0.94 0.94 0.95

H4 23 24 22 0.96 0.92 0.94

H5 36 38 33 0.92 0.87 0.89

H6 32 35 31 0.97 0.89 0.93

H7 27 28 26 0.96 0.93 0.94

H8 20 20 18 0.9 0.94 0.95

Note: MR-Total number of replays manually detected, TR- Total number of replays detected

by the method and CR-Number of correctly detected replays.

3.3. Goal post shot detection

Since the goal post or goal mouth is clearly demarked by the white strips of the post,

the detection rate is also very high. The results of goal post shot detection are given in

table 4. In certain games the recall and precision values are less, since the frames

including the side view of the goal post are not detected correctly by the method and

Page 9: AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical

Automatic Summarization of Hockey Videos

http://www.iaeme.com/IJARET/index.asp 67 [email protected]

in certain other cases, the goal post is partially covered by the players, making the

detection of the goal post difficult.

Table 4 Goal post shot detection results

Game MGP TGP CGP

Recall

(CGP /

MGP)

Precision

(CGP /TGP) F-score

H1 68 70 65 0.96 0.93 0.94

H2 77 82 70 0.91 0.85 0.88

H3 60 57 52 0.87 0.91 0.89

H4 66 65 61 0.92 0.94 0.93

H5 95 98 89 0.94 0.91 0.92

H6 46 45 42 0.91 0.93 0.92

H7 62 65 58 0.94 0.89 0.91

H8 29 30 27 0.93 0.9 0.91

Note: MGP-Total number of goal post shots manually detected, TGP- Total number of goal post

shots detected by the method and CGP- Number of correctly detected s goal post shots.

3.4. Umpire shot detection

Color segmentation algorithm works well in identifying the umpire in the game from

players. Since the umpires wear different color dress in different games, the color has

to be selected manually in the system. In some games the color of the umpire dress

matches with that of the goal keeper, very few false alarms occurred. Once the

Umpire shot is identified, again color segmentation is used to find the different cards

like red, yellow or green cards. The results of Umpire shot detection are given in table

5.

Table 5 Umpire shot detection results

Game MRS TRS CRS Recall

(CRS/MRS)

Precision

(CRS/TRS) F-score

H1 24 25 23 0.96 0.92 0.94

H2 32 30 29 0.91 0.97 0.94

H3 28 32 25 0.89 0.78 0.83

H4 52 55 49 0.94 0.89 0.91

H5 44 43 42 0.95 0.98 0.96

H6 26 26 24 0.92 0.92 0.92

H7 40 51 32 0.8 0.63 0.7

H8 19 25 16 0.84 0.64 0.73

Note: MRS-Total number of umpire shots manually detected, TRS- Total number of goal post

shots detected by the method and CRS- Number of correctly detected s goal post shots.

In games H3, H7 and H8, we can see that the recall and precision values are less.

This is because, in these games the colour of Umpires’ uniform matches with that of

the goal keeper of a playing team

3.5. Goal detection

A goal shot is identified by the occurrence of the following sequences – goal post shot

with a close up shot. Here all goals are detected fairly and hence good recall and

precision as given in table 6.

Page 10: AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical

Hari R

http://www.iaeme.com/IJARET/index.asp 68 [email protected]

Table 6 Goal detection results

Game MG TG CG Recall

(CG/MG)

Precision

(CG/TG) F-score

H1 5 6 4 0.8 0.67 0.73

H2 5 5 4 0.8 0.8 0.8

H3 0 1 - - - -

H4 2 4 2 1 0.5 0.67

H5 3 4 3 1 0.75 0.86

H6 3 2 2 0.67 1 0.8

H7 2 3 2 1 0.67 0.8

H8 1 2 1 1 0.5 0.67

Note: MR- Total number of goals manually detected, TR-Total number of goals detected by

the method and CR- Number of correctly detected goals

3.6. Penalty corner detection

Our system detects the penalty corner with high degree of accuracy, which is given in

table 7.

Table 7 Penalty corner detection results

Game MPC TPC CPC Recall

(CPC/MPC)

Precision

(CPC/TPC) F-score

H1 2 4 2 1 0.5 0.67

H2 4 5 3 0.75 0.6 0.67

H3 3 2 2 0.67 1 0.8

H4 1 3 1 1 0.33 0.5

H5 7 8 6 0.86 0.75 0.8

H6 5 4 4 0.8 1 0.89

H7 2 3 1 1 0.67 0.8

H8 2 4 2 1 0.5 0.67

Note: MPC- Total number of penalty corners manually detected, TPC- Total number of penalty

corners detected by the method and CPC-Number of correctly detected penalty corners.

Most of the penalty corners are detected correctly and few false alarms are due to

the motion of the camera as well as the focusing of the camera on the players instead

of the goal post.

3.7. Foul detection

In all the games considered, the major fouls are signaled by the Umpire using yellow

cards only and our system has detected all the fouls correctly leading to recall and

precision values of 1. In the games chosen experimentation, the method detected all

the yellow cards displayed by the Umpire. But in some cases, the Umpire showing the

cards will not be focused by the camera, leading to the failure of the methods.

Table 8 Foul detection results

MF TF CF Recall

(CF / MF)

Precision

(CF / TF) F-score

3 3 3 1 1 1

Note: MF - Total number of yellow/red cards manually detected, TF - Total number of

yellow/red cards detected by the method, CF - Number of correctly detected yellow/red cards

Page 11: AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical

Automatic Summarization of Hockey Videos

http://www.iaeme.com/IJARET/index.asp 69 [email protected]

3.8. Summarization Result

For evaluating the result of summarization, we have considered the number of events

included in the automatically created video. In this video all the important events

related to scoring of a goal, penalty corner and fouls are included. Summarization

efficiency is evaluated using recall, which is defined as the ratio of total number of

events correctly detected by the method to the total number of events in the game. The

final results are given in table 9.

Table 9 Results of Summarization

Game Total number of

events in the game

Total number of events

correctly detected Recall

H1 8 7 0.88

H2 11 8 0.73

H3 3 3 1

H4 3 3 1

H5 10 9 0.9

H6 8 6 0.75

H7 4 3 0.75

H8 3 3 1

From this, we can see that the method provides average recall of 0.88, showing

that the method is highly efficient in generating summary with all the major events..

4. CONCLUSION

A new method for extracting important events in Hockey game videos to create

effective summary of the game is proposed. Even though many different methods are

available for summarizing other sports like soccer, basketball, cricket, etc, very few

efforts were made to summarize the hockey video game. In our method, all the

inherent features of the game are considered for extraction of important events. The

method involves many sub-modules and efficient algorithms are devised in each of

these modules. Experimentally all these modules are tested and the result obtained

from the modules are found promising. An average summarization efficiency of 88%

is obtained. In future the score card extraction and audio commentary detection can

also be incorporated to increase the efficiency of the system.

REFERENCES

[1] Chong Wah Ngo, Yu-Fei Ma and Hong-Jiang Zhang, Video Summarization

and Scene Detection by Graph Modeling, IEEE Trans. on Circuits ans

Systems for Video Technology, Vol. 15, No. 2, Feb. 2005, pp. 296-305.

[2] Xinghao Jiang, Tanfeng Sun, Jin Liu, Juan Chao and Wensheng Zhang, An

Adaptive Video Shot Segmentation Scheme Based on Dual- Detection

Model, Journal on Neurocomputing, 116, 2013,pp. 102-111.

[3] Baoxin Li, Hao Pan and Sezan, I., A General frame work for sports video

summarization with its application to soccer, IEEE International Conference

on Acoustics, Speech and Signal Processing, (ICASSP ’03), 2003, vol. 3, pp.

169-172.

[4] Dian Tjondronegoro, Yi-Ping Phoebe Chen and Binh Pham, Integrating

Highlights for More Complete Sport Video Summarization, Published by the

IEEE Computer Society 2004 IEEE, pp.22-37.

Page 12: AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical

Hari R

http://www.iaeme.com/IJARET/index.asp 70 [email protected]

[5] A. Hanjalic, Generic Approach to highlighte extraction from a sport video,

ICIP 2003.

[6] F. Coldefy, P. Bouthemy, M. Betser and G. Gravier, Tennis video abstraction

from audio and visual cues, Proc. of IEEE 6th workshop on Multimedia

Signal processing, Sep. 2004,pp. 163-166.

[7] Xueming Qian, Huan Wang, Guizhong Liu and Xingsong Hou, HMM based

soccer video event detection using enhanced mid-level semantic, Multimedia

Tools Appl. (2012) 60:233-255.

[8] Lamberto Ballan, Marco Bertini, Alberto Del Bimbo and Giuseppe Serra,

Semantic annotation of soccer videos by visual instance clustering and

spatial/temporal reasoning in ontologies, Multimedia Tools Appl (2010) 48:

pp. 313-337.

[9] Jian-quan Ouyang and Renren Liu, Ontology reasoning scheme for

constructing meaningful sports video summarization, IET Image Processing,

2013, vol.7, Issue 4, pp. 324-334.

[10] Takahashi, Y. , Nitta, N. and Babaguchi, N., Video Summarization for Large

Sports Video Archives, Proc. of IEEE International Conf. on Multimedia and

Expo, ICME 2005, pp. 1170-1173.

[11] Shu-Ching Chen, Miami F. L. ,Min Chen, Chengcui Zhang, Mei-Ling Shyu ,

Exciting Event Detection Using Multi-level Multimodal Descriptors and

Data Classification Eighth IEEE International Symposium on Multimedia,

2006. ISM'06, pp. 193-200.

[12] Guangyu Zhu, Qingming Huang, Changsheng Xu and Liyuan Xing, Human

Behavior Analysis for Highlight Ranking in Broadcast Racket Sports Video,

IEEE Trans. on Multimedia, vol.9, Issue 6,2007, pp. 1167-1182.

[13] Fan Chen, Damien Delannay and Christophe De Vleeschouwer, An

Autonomous Framework to Produce and Distribute Personalized Team-Sport

Video Summaries: A Basketball Case Study, IEEE Trans. on Mutimedia, vol.

13, No. 6, December 2011, pp. 1381-1394.

[14] Dr. K. Devaraju. Effect of S.A.Q Training on Vital Capacity among Hockey

Players. International Journal of Advanced Research in Engineering and

Technology, 5(1), 2015, pp. 102 - 105.

[15] K. Devaraju and A. Needhiraja, Prediction of Playing Ability in Kabaddi

from Selected Anthropometrical, Physical, Physiological and Psychological

Variables Among College Level Players, International Journal of

Management, Volume 3, Issue 2, 2012, pp. 150 - 157.

[16] Matthew L. Parry, Philip A. Legg, David H. S. Chung, Iwan W. Griffiths and

Min Chen, Hierarchichal Event Selection for Video Storyboards with a Case

Study on Snooker Video Visualization, IEEE Trans. on Visualization and

Computer Graphics, vol. 17, No.12, December 2011, pp.1747-1756.

[17] Namuduri, K., Automatic Extraction of Highlight from a Cricket Video

Using MPEG 7 Descriptors, IEEE First International Conference on

Communication Systems and Networks and Workshops, 2009, pp. 1-3.

[18] A. Kokaram and P. Delacourt, A New Global Motion Estimation Algorithm

and its Application to Retrieval in Sports Events, IEEE fourth Workshop on

Multimedia Signal Processing, 2001 pp. 251 – 256.

[19] N. Harikrishna, Sanjeev Satheesh, S. Dinesh Sriram and K. S. Easwarakumar,

Temporal Clssification of Events in Cricket Videos, IEEE National

Conference on Communications, 2011, pp. 1-5.

Page 13: AUTOMATIC SUMMARIZATION OF HOCKEY VIDEOS€¦ · Hockey match videos and the average efficiency is found to be 0.88. Key words: Video Summarization, Colour Segmentation, SSIM, Optical

Automatic Summarization of Hockey Videos

http://www.iaeme.com/IJARET/index.asp 71 [email protected]

[20] Maheshkumar H. Kolekar and Somnath Sengupta, Semantic concept mining

in cricket videos for automated highlight generation, Multimedia Tools and

Applications, Springer, vol. 47, No. 3, May 2010, pp. 545-579.

[21] Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh and Eero P.

Simoncelli, Image Quality Assessment: From Error Visibility to Structural

Similarity, IEEE Trans. on Image Processing, vol. 13, no. 4, pp. 600-

612,April 2004.

[22] B. Horn and B. Schunck. Determining optical flow. Artificial Intelligence,

16:185–203, Aug. 1981.

[23] K. Devaraju and A. Needhiraja, “Prediction of Kabaddi Playing Ability from

Selected Anthropometrical and Physical Variables Among College Level

Players”, International Journal of Advanced Research in Engineering &

Technology (IJARET), Volume 3, Issue 1, 2012, pp. 118 – 124.