UNSUPERVISED UNCERTAINTY ANALYSIS FOR VIDEO SALIENCY … · 2017. 3. 8. · Chenlei Guo; Liming...
Transcript of UNSUPERVISED UNCERTAINTY ANALYSIS FOR VIDEO SALIENCY … · 2017. 3. 8. · Chenlei Guo; Liming...
-
Motivation
Moving from lab-controlled saliency detection to real world application requires more than “better AUC numbers”
Comparing different saliency detection algorithms requires a mechanism to evaluate uncertainty and statistical significance
To be applicable to wide range of videos, estimation needs to be HVS-inspired rather than data-fitted, hence unsupervised.
UNSUPERVISED UNCERTAINTY ANALYSIS FOR VIDEO SALIENCY DETECTIONTariq Alshawi, Zhiling Long and Ghassan AlRegib
{talshawi, zhiling.long, alregib}@gatech.eduSchool of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
1. Chenlei Guo; Liming Zhang, "A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression," in Image Processing, IEEE Transactions on , vol.19, no.1, pp.185-198, Jan. 2010
2. F. W. M. Stentiford, “Attention based Auto Image Cropping,” Workshop on Computational Attention and Applications, ICVS, Bielefeld, March 21-24, 2007.3. K. Debattista, L.P. Santos, A. Chalmers, Accelerating the irradiance cache through parallel component-based rendering, in 6th Eurographics Symposium on
Parallel Graphics Visualization. Eurographics, May 2006, pp. 27-34.
Uncertainty Framework
Proposed Method Experiments
Data
Public CRCNS database
50 video clips (640 x 480), 5-90 seconds each, 30 frames/sec
Street scenes, TV programs, video games, etc.
Ground truth eye fixation data from human subjects under freeview condition
Feature
Extraction
Feature
Extraction
Localized
3D FFT
Spectral
Decomposition
Center-Surround
ComparisonF
Ft
Fs
Center-Surround
Comparison
+
St
Ss
S
Et
Es
ft
fx
fy
ᶱ
M (a0, b0, c0)
a0
b0
c0
N (a0, b0, 0)
O
Auto-Cropping2
Rendering3
Video Processing Algorithm
Saliency Detection Algorithm
Decision Making Module
Risk Assessment
Module
Uncertainty Estimation
Input Video
Saliency Map
UncertaintyMap
DecisionMap
Output Video
Cost
Compression1
Evaluation Methodology
Input Video
Sa
liency D
ete
ctio
nE
va
luatio
n D
ata
se
tR
esu
lts
Expanded Eye-fixation map
Saliency Map
True Uncertainty
-
Uncertainty Estimation
Estimated Uncertainty
Fixed Threshold
Receiver Operation Characteristics
(ROC)
M
N D
Saliency Map S
Spatial
Neighbors
Frame# N Frame# N+1Frame# N–1
Pixel of
Interest
Temporal
Neighbors
0.6 1 0.6
1 -0.75 1
0.6 1 0.6
Estimated
U
ncertain
ty Ue
Tru
e U
nce
rtai
nty
Utr
for beverly05, frame 5
Saliency value at pixel x
Average value around pixel x