Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player...
-
date post
19-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player...
![Page 1: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/1.jpg)
Haojie LiJinhui TangSi WuYongdong ZhangShouxun Lin
Automatic Detection and Analysis of Player Action in Moving Background Sports
Video Sequences
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 20, NO. 3, MARCH 2010
![Page 2: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/2.jpg)
IntroductionGlobal motion estimationPlayer body shape segmentationAnalysis of actionExperimental results
OUTLINE
![Page 3: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/3.jpg)
Presents a system for automatically detecting and analyzing complex player actions in moving background sports video sequences
Providing kinematic measurements for coach assistance and performance improvement
Video-based approach : low cost, no interference to the performance of players, can analyze the rich archived video clips
INTRODUCTION
![Page 4: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/4.jpg)
Block diagram of the proposed system
INTRODUCTION
Video Sequenc
e
Global motion
estimation
Action clips detection
Action recognition
Player shape
segmentation Visual
analysis
Kinematic analysis
Highlights library
1. The detected highlights are stored into library as video summaries for user’s quick browsing2. Action recognize using CHMM ( continuous hidden markov models )
Action-based video indexing
Kinematic parameters
![Page 5: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/5.jpg)
The result accuracy of most global motion estimation methods are influenced by outliers.
Some methods used experimentally determined or manually specified thresholds to remove outliers, thus are not adaptive to other data.
In this paper 6-parameter affi ne model & Fisher linear discriminant analysis are used. is point pair.
GLOBAL MOTION ESTIMATION
𝑢𝑖=𝐻 𝑖 𝐴
(𝑥𝑖𝑦 𝑖) (𝑥 𝑖
❑ ′ 𝑦 𝑖❑′ 1
0 0 00 0 0𝑥𝑖
❑ ′ 𝑦 𝑖❑ ′ 1) (𝑎 ,𝑏 ,𝑐 ,𝑑 ,𝑒 , 𝑓 )𝑇
: Point p in current frame
: Point p in frame
Global motion parameter
![Page 6: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/6.jpg)
Finding point pairs in and Calculate the global standard variance of pixel values in Scan and check each n*n block. If standard variance of a
block is large enough ( > ) the upper left corner of the block is selected as
is obtained by searching nearby blocks in
GLOBAL MOTION ESTIMATION
𝐼𝑘 𝐼𝑘−1
![Page 7: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/7.jpg)
By solving , we can obtain initial solution Since is the approximate solution and motion of
outliers is not consistent with GM according to residual errors, we can separate point pairs into inliers and outliers
We can use inliers to refine A
GLOBAL MOTION ESTIMATION
𝑟 𝑖=𝑢𝑖−𝐻 𝑖𝐴∗
𝑢𝑖∈ {𝑜𝑢𝑡𝑙𝑖𝑒𝑟 ,|𝑟 𝑖|2≥𝑇
𝑖𝑛𝑙𝑖𝑒𝑟 ,|𝑟 𝑖|2<𝑇
[24] N. Ostu, “A threshold selection method from gray level histogram,” IEEETrans. Syst. Man. Cybern., vol. 9, no. 1, pp. 62–66, Jan. 1979.
![Page 8: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/8.jpg)
GLOBAL MOTION ESTIMATION
𝐼𝑘𝐼𝑘−1Global motion vectors between two frames
Outlier filtering Aligned image using estimated GME parameters
Difference image between (b) (e)
Background is accurately aligned
![Page 9: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/9.jpg)
An algorithm has been proposed[29] (algo1)Main steps of Algorithm 1
Global motion estimation Foreground separation : three-frame-difference Background construction :
is aligned to GM Using temporal median
Object segmentation Background subtraction Significance test[30] is used to decide threshold to binarize Connected component analysis Snack model[31] is adopted to smooth each remaining
component’s boundary
PLAYER BODY SHAPE SEGMENTATION
𝐼𝑘 𝐼𝑘+1 𝐼𝑘+𝐿𝐼𝑘−𝐿 … …𝐼𝑘−1Consecutive 2L-1 frames
𝐼𝑘−1 𝐼𝑘 𝐼𝑘+1
d1 d2
𝐷1 𝐷2∩ :
An algorithm has been proposed[29] (algo1)Main steps of Algorithm 1
Global motion estimation Foreground separation : three-frame-difference Background construction :
is aligned to GM Using temporal median
Object segmentation Background subtraction Significance test[30] is used to decide threshold to binarize Connected component analysis Snack model[31] is adopted to smooth each remaining
component’s boundary
![Page 10: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/10.jpg)
Results of Algo1
PLAYER BODY SHAPE SEGMENTATION
Problem:Work well only when object has apparent motion
Reason:Doesn’t consider the object motion between frames
![Page 11: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/11.jpg)
Improved version (Algo2) take object motion between frames into consideration select frames with apparent object motion to construct background image
We use global motion between frames as the measurement of object motion
Key-frame selection: Neighboring frames with global motion > Th1 A frame’s cumulative global motion to the nearest key-frame >
Th1 When the cumulative global motion from to key-frame > TH2 no more frames are needed!!
PLAYER BODY SHAPE SEGMENTATION
Only selected key-frames are used to construct background
𝐼𝑘 𝐼𝑘+1 𝐼𝑘+𝐿𝐼𝑘−𝐿 … …𝐼𝑘−1Consecutive 2L-1 frames
… …kf1 kfL1Kf-1kfL2
L1L2
𝐿𝑖=min❑ (𝐿 , argmin𝐽 𝐶𝑀 𝑖 (𝑘 , 𝐽 )≥ h𝑇 2)
![Page 12: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/12.jpg)
Results of Algo2
PLAYER BODY SHAPE SEGMENTATION
Th1 = 4TH2 = 50L = 11
![Page 13: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/13.jpg)
Kinematic analysis : we present an automatic method through 2-D articulated human body model fi tting, to get the joint angles. S=( x, y, θ, θ1, θ2, θ3, d )
THE ANALYSIS OF ACTION
Human body model
Test body shape
Edge mapDistance transform map [35]
The initial parameter is refine by searching with annealed particle algorithm[34]
Global position & rotation parameter
neck, hip, knee angle
![Page 14: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/14.jpg)
Visual analysisMotion Panorama
Overlay composition
THE ANALYSIS OF ACTION
Temporal median filtering
• Compare actions performed by different players or by the same player at different time
• No constraint that two clips should be of same scene
![Page 15: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/15.jpg)
Global motion estimation
EXPERIMENTAL RESULTS
( Interframe transformation fidelity )
Aligned image to Ik by global motion compensated on Ik-1
RANSAC & LTS with refinement procedures
![Page 16: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/16.jpg)
Player body segmentation
EXPERIMENTAL RESULTS
![Page 17: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/17.jpg)
Action recognition
EXPERIMENTAL RESULTS
![Page 18: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/18.jpg)
Kinematic analysis
EXPERIMENTAL RESULTS
![Page 19: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/19.jpg)
Visual analysis
EXPERIMENTAL RESULTS
![Page 20: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/20.jpg)
Experiments on jump videos
EXPERIMENTAL RESULTS
![Page 21: Haojie Li Jinhui Tang Si Wu Yongdong Zhang Shouxun Lin Automatic Detection and Analysis of Player Action in Moving Background Sports Video Sequences IEEE.](https://reader038.fdocuments.us/reader038/viewer/2022110207/56649d2c5503460f94a02ac2/html5/thumbnails/21.jpg)
EXPERIMENTAL RESULTS