Mahfuzul Haque and Manzur Murshed

Panic-driven Event Detection from Surveillance

Video Stream without Track and Motion Features

Presentation Outline

• Introduction

– Area

– Problem

– Objective

• Event Detection

• The Idea

– Why not track or motion features?

• The Proposed Method

• Experimental Results

• Q & A

Research Area

… Event

Model Stage 2 Stage 1

Intelligent Video Surveillance

Automated Alert

Smart Monitoring

Context-aware Environments

Event Detection

Action / Activity Recognition

Behaviour Recognition

Behaviour Profiling

Video Stream Analytics Real-time Processing

Dynamic Scene Understanding

The Problem

… Event

Model Stage 2 Stage 1

Large Surveillance Network

Thousands of video feeds

Ad-hoc remote surveillance

Dynamic scene variations

Scene specific tuning

Availability of training data

Video Stream Analytics Real-time Processing

Dynamic Scene Understanding

How to develop a generic scene

understanding framework that

would reliably work on a wider

range of scenarios?

Research Objectives

… Event

Model Stage 2 Stage 1

A generic scene understanding framework

Developing the building blocks for the essential processing

stages

Scope:

Panic-driven abnormality detection

A fixed set of specific events

Video Stream Analytics Real-time Processing

Dynamic Scene Understanding

Event Detection

Specific types of events vs. abnormality

An event persists for a certain duration of time

The duration is variable

Event characteristics of the same event

Variable in the same environment

Variable from one scene to other

time

How to identify the generic

characteristics of an event?

The Idea

Event detection as temporal data classification problem

A distinct set of temporal features can characterise an event

Which/how frame-level features are extracted?

How the observed frame-level features are transformed in

temporal-features?

time

f1

f2

f3

.

.

.

fn

Event

Model

Frame-level

Features

Temporal

Features Classifier

The Idea

Key points detection

Point matching in successive frames

Flow vectors: position, direction, speed

Motion based approaches Tracking based approaches

Object detection

Object matching in successive frames

Trajectories: object paths

Inter-frame association

Context specific information

Event models are not generic

Common characteristics

Object detection

Global frame-level descriptor:

independent of scene characteristics

Proposed generic approach No Inter-frame association

Independent frame-level features =>

temporal features considering speed

and temporal order

Hu et al. (ICPR 2008) Xiang et al. (IJCV 2006)

The Idea

Object based approach

Independent frame-level features– no object / position

specific information, no spatial association

Frame-level features are transformed into temporal features

considering speed and temporal order

Supposed to be more context invariant

time

f1

f2

f3

.

.

.

fn

Event

Model

Frame-level

Features

Temporal

Features Classifier

Summary

The Proposed Method

Background

Subtraction

Selective

Frame-level

Feature Extraction

Selective

Temporal

Feature ExtractionIncoming frames Foreground blobs

Trained

Event Models

Detection

Results

Background

Subtraction

Frame-level

Feature Extraction

(30 features)

Temporal

Feature Extraction

(270 features)Labelled frames Foreground blobs

Feature Ranking

and Selection

Event Model

Training

Model Training

Real-time Execution

Event

Models

Foreground

Detector

Frame-level

Feature Extractor

Temporal

Feature Extractor

Architecture

The Proposed Method

Frame-level features

Blob Area (BA)

Filling Ratio (FR)

Aspect Ratio (AR)

Bounding Box Area (BBA)

Bounding box Width (BBW)

Bounding box Height (BBH)

Blob Count (BC)

Blob Distance (BD)

Frame #

1

2

3

4

5

6

The Proposed Method

Temporal features

Overlapping sliding window

Temporal order

Speed of variation

The Proposed Method

Blob Count (BC), Blob Area (BA)

The Proposed Method

Blob Distance (BD)

The Proposed Method

Aspect Ratio (AR)

The Proposed Method

Top five features for four different events

Feature ranking using absolute value criteria of two sample t-test, based on

pooled variance estimate.

Experimental Results

Specific Event Detection

• Four different events: meet, split, runaway, and fight

• CAVIAR dataset with labelled frames

• 80% of the test frames for model training

• 100 iterations of 10-fold cross validation

• Remaining 20% of the test frames for testing

• SVM classifier as event models

• Separate model for each event

Experimental Results

Experimental Results

Specific Event Detection

Predicted

Actual

Severity

Experimental Results

Abnormal Event Detection

• University of Minnesota crowd dataset (UMN dataset)

• The Runaway event model

• No additional training or tuning

• Three different sites

Experimental Results

Abnormal Event Detection (UMN-9)

Experimental Results

Abnormal Event Detection (UMN-10)

Experimental Results

Abnormal Event Detection (UMN-01)

Experimental Results

Abnormal Event Detection (UMN-07)

Experimental Results

Method AUC

Our Method 0.89

Pure Optical Flow [1] 0.84

Performance Comparison

[1] R. Mehran, A. Oyama, and M. Shah, “Abnormal crowd behavior detection using social force model,” in Proc. IEEE

Conference on Computer Vision and Pattern Recognition CVPR 2009, 20–25 June 2009, pp. 935–942.

Publication

Mahfuzul Haque and Manzur Murshed, “Panic-driven Event Detection

From Surveillance Video Stream without Track and Motion Features,”

IEEE International Conference on Multimedia & Expo (ICME), 2010.

Q&A Mahfuzul.Haque@infotech.monash.edu.au

Thanks!