Human Activity Recognition ModuleProject: Automatic Surveillance System for Video Streams (ASSVS)

November 24, 2016

Contents

1

Chapter 1

Introduction

1.1 Purpose

This document describes the requirements specifications of human activity recognition module. It definesthe scope of human activities and their recognition that will be covered in this module. It further coversthe constrains, assumptions and dependencies of the module. Moreover, the functional requirements ofthe module are mentioned.

1.2 Module Scope

This module will cover single human activity, human-object interactions and human-human interactions.

1.2.1 Single Human Activity

Single human activity will include following states or actions:

Locomotion

1. Walking

2. Running

3. Jumping

4. Jogging

5. Pushing

6. Pulling

7. Bending

8. Hand waving

Static

1. Sitting

2. Standing

3. Carrying

2

CHAPTER 1. INTRODUCTION 3

1.2.2 Human-Object Interaction

Human-object interaction will cover the following scenarios:

1. Chair: Sitting

2. Mobile: Calling

3. Bag: Carrying, dropping

4. Bus, car: Driving

5. Bike, cycle: Riding

6. Door: Pushing, pulling

7. Food: Eating

8. Bat, ball etc.: Playing

1.2.3 Human-Human Interaction

1. Meeting

2. Hand shaking

3. Hugging

4. Gathering

5. Fighting

Chapter 2

Overall Description

2.1 Module Perspective

Activity recognition module is an important component of ASSVS. The chief objective of human activitydetection sub-module is to find the class of activity that is present in input images or video frames. Thiscomponent does not work on its own and it requires human body segmented from human detection andrecognition module. The output of this module would be name of activity class and will be used as inputin the natural language generation module. The diagram of this sub-module and interaction with othermodules is shown in figure ??:

Figure 2.1: Module importance in ASSVS

4

CHAPTER 2. OVERALL DESCRIPTION 5

2.2 Module Functions

The functions of human activity recognition are revealed in table ??.

Functions DescriptionSingle human activity This function will detect single human activities in the video

scene.Human-object interaction Interaction between humans and objects will be covered in

this function.Human-human interaction This function will cover interactions between multiple hu-

mans.

Table 2.1: Functions of human activity recognition module

2.3 User Classes and Characteristics

User classes for the sub-module are described below:

• Security officers in Law enforcement agencies including Court, Police, Rescue, Traffic Police.

• Administration staff, patient monitoring staff.

• Store keepers for asset monitoring.

• Security staff for student surveillance and lecture feedback

2.4 Operating Environment

Being computationally extensive module, it requires reasonable hardware resources.

2.5 Design and Implementation Constraints

2.5.1 Occlusions

Obstructions or occlusions will decrease the accuracy of human activity recognition and the module requiresclear visual scenes where complete human body is visible.

2.5.2 Camera position

Camera placed at greater elevations or at a lengthy distance may also create problems in the recognitionof human activity particularly when human arms and legs are not distinguishable.

2.5.3 Activity Coverage

The system will only recognize activities covered in the scope of the module.

2.5.4 Illumination

Human activity recognition will become difficult in dark and dim light conditions.

CHAPTER 2. OVERALL DESCRIPTION 6

2.6 Assumptions and Dependencies

The assumptions and dependencies of the module are described below:

1. Cameras have moderate resolution.

2. Cameras are adjusted in proper altitude.

3. There are no occlusions when taking video frames.

4. Lighting conditions are stable.

5. Only a single activity is being performed in the video.

2.7 User Documentation

Installation guide and user manual will be provide along with the software.

Chapter 3

External Interface Requirements

3.1 User Interfaces

The user can take input from two sources i.e. camera or recorded video. The interface will show the videoand activities in the video in a hierarchical format. The screen shots of last detected videos will also bedisplayed in the right panel. The probability of each activity and action will be displayed in the form ofgraph. Figure 3.1 shows the GUI of the module.

Figure 3.1: User Interface

7

CHAPTER 3. EXTERNAL INTERFACE REQUIREMENTS 8

3.2 Software Interfaces

Operating System Windows.NET Framework v4.0IDE Visual Studio 2013Database MS SQL SERVER 2008Image Processing Libraries OpenCV 3.0, EmguCV 3.0Machine Learning API Accord.NETDatabase IDE SQL Server Management StudioWeb Server IIS 8

3.3 Hardware Interface

Processor Quad-Core 3.4 GHzRAM DDR3 16GB RAMCamera FHDDVR AnyStorage Device 1TBNetwork Bandwidth 1MBpsGraphics Card Nvidia

3.4 Operating Environment

As this module is a part of surveillance system, monitoring the behavior of humans. This can includeobserving a subject from a distance using electronic devices like CCTV cameras. System can work inindoor environment as well as outdoor environment with the concerns that minimum lighting conditionsare satisfied. This module is an independent system that can act as an input of different applications andcan also act as a subsystem for a surveillance system.

Chapter 4

System Features

This chapter provides module perspective and features. The feature list is priorities based on importanceof each feature in the module.

4.1 Module Perspective

Figure 4.1: Activity recognition module use-case

9

CHAPTER 4. SYSTEM FEATURES 10

4.2 Module Features

4.2.1 Single Human Activity Recognition

Description and Priority

Single human activity recognition sub-module will recognize the different actions performed by singlehuman. In terms of usage, Single human activity recognition has equal priority as human-object interactionand human-human interaction.

Stimulus/Response Sequences

The video sequences from camera or storage media will be processed to recognize single human activityas soon Retrieve Video button is pressed The single human activity will be shown on bounding boxes oneach individual human presence.

Functional Requirements

FR-1: The video/image sequences from the camera interface or storage media.

4.2.2 Human-Object Interaction

Description and Priority

Human object interaction sub-module will classify the different types of interaction between human andobjects.

Stimulus/Response Sequences

The annotated human and object recognition module will provide their feeds to human object interactionand it will start processing as soon Retrieve Video button is pressed

Functional Requirements

FR-1: The function shall detected humans and objects along with bounding boxes around them in theimage sequences.FR-2: The function will produce the interaction between human and object.

4.2.3 Human-Human Interaction

Description and Priority

Human-human interaction sub-module will classify the different types of interaction between human andhuman.

Stimulus/Response Sequences

The annotated human action recognition module will provide their feeds to human interaction and it willstart processing as soon Retrieve Video button is pressed.

Functional Requirements

FR-1: The detected humans along with bounding boxes around them in the image sequences.FR-2: The function will produce the interaction between humans.