Virtual 3D Shield for Asset Protection

Depth Sensing Systems for People Safety

1 ICCE – Berlin, 7 – 10.09.2014

Virtual 3D Shield for Asset Protection Daniel Moldovan

AIT, Vienna


2 ICCE – Berlin, 7 – 10.09.2014

Outline

• D-SenS (Depth Sensing Systems for People Safety)

• Intrusion Detection System (IDS)

– Motivation / applicability (Why?)

– Relevant state of the art

– What & How?

– Concrete experiments.

• Learned lessons & Conclusion


3 ICCE – Berlin, 7 – 10.09.2014

D-SenS

Supervise a predefined volume by using depth-sensing technology.

Create new business potential for applications related to people safety.


4 ICCE – Berlin, 7 – 10.09.2014

D-SenS focus areas

People Tracking

Fall Detector

Left-Luggage

3D from moving

platform

http://d-sens.eu/

Intrusion

Detection


5 ICCE – Berlin, 7 – 10.09.2014

Outline





– What & How?




6 ICCE – Berlin, 7 – 10.09.2014

Intrusion Detection System

Museum exhibit Private property

Employs a virtual 3D shield for asset protection.


7 ICCE – Berlin, 7 – 10.09.2014

Requirements - Functional

• Detection rate: 4 – 10 times/sec

• Position of the 3D screen: user definable

• Ground plane calibration: automatic

• Variable configuration: 2 or 3 different sensors.

• Accuracy:

– Asset size: any (indoor)

– Minimum object size: >= 10x10x10 cubic cm.


8 ICCE – Berlin, 7 – 10.09.2014

Requirements - Accuracy

• Object size:

– Asset size: any (indoor)

– Minimum object size: >= 10x10x10cm3 at 5m distance between virtual 3D screen and sensor.

• Surveyed volume:

– Small, medium, big rooms (10 – 100m2)

– Maximum distance between sensor and object: 10m.


9 ICCE – Berlin, 7 – 10.09.2014

Motivation and applicability

• 3D monitoring: – Increased safety without barriers; – Economical / High productivity; – High level of flexibility:

• simple configuration of detection zones; • rapid installation.

• Application areas: – Smart building; – Assisted living; – Security.


10 ICCE – Berlin, 7 – 10.09.2014

Relevant state of the art

Multiple Camera systems Depth Sensing Camera

S. Kawabata, S. Hiura and K. Kato, 3D Intrusion

Detection System with Uncalibrated Multiple Cameras,

ACCV 2007, Tokyo, Japan, 2007

SafetyEye (Pilz GmbH)


11 ICCE – Berlin, 7 – 10.09.2014

Outline





– What & How?




12 ICCE – Berlin, 7 – 10.09.2014

Framework of the proposed IDS

Data/control flow

Response to Intrusive Activities


13 ICCE – Berlin, 7 – 10.09.2014

Architecture overview


14 ICCE – Berlin, 7 – 10.09.2014

Depth Acquisition – Passive 3D Stereo

Stereo-camera

system developed by

AIT

(1280*1024)

• Run-time: ~ 14 fps (Core-i7, multithreaded, SSE-

optimized)

• Resolution: typically

~1 Mpixel.

• Excellent “depth-

quality /

computational-costs”

ratio. (1280*1024)


15 ICCE – Berlin, 7 – 10.09.2014

(1280*1024) (640*480)

ASUS Xtion Pro Live IR

Depth Camera

Depth Acquisition – IR Camera

• Run-time: ~ 20 fps.

• Excellent “depth-quality

/ computational-costs”

ratio.

• Resolution: typically

VGA size.

• Robustness against

illumination, shadows.


16 ICCE – Berlin, 7 – 10.09.2014

Architecture – Sensing & Calibration


17 ICCE – Berlin, 7 – 10.09.2014

Dominant flat surface detection

Disparity image 3D point cloud of the scene (top view).

Horizontal grid parallel with the floor.


18 ICCE – Berlin, 7 – 10.09.2014

V-Disparity Image

V-Disparity image

Disparity

[pix]

Image

Height

1 150

Disparity image


19 ICCE – Berlin, 7 – 10.09.2014

Flat Surface Estimation

V-Disparity image Straight lines corresponding to flat surfaces

detected by using Hough Transform


20 ICCE – Berlin, 7 – 10.09.2014

Defining the location of the spherical shield

Placing the center

of the sphere

Adjusting the location of

the sphere in 3D


21 ICCE – Berlin, 7 – 10.09.2014

Architecture – Analytics


22 ICCE – Berlin, 7 – 10.09.2014

1. Removal of 3D points located outside of the virtual shield.

2. Background = Depth image of the remaining 3D points.

Background processing steps

Background image = filtered depth Full depth image


23 ICCE – Berlin, 7 – 10.09.2014

Object detection - connected components

Neighboring pixels are connected in 3D space if the difference of their

disparities is smaller than a certain threshold (usually one pixel).

Connected pixels are

asigned to the same label.

Red rectangles correspond to

the location of the labels.

Label image


24 ICCE – Berlin, 7 – 10.09.2014

Object detection within volume

Background subtraction Red rectangles surrounding

the intruding objects

1. Depth-based background subtraction.

2. Labeling of intruding objects.

Depth image


25 ICCE – Berlin, 7 – 10.09.2014

Verify the intruding objects

In order to verify the size of the intruding

object, we generate a concentric inner sphere.

The gap between the spheres represents the

minimum size of the object for which an alarm

will be generated.


26 ICCE – Berlin, 7 – 10.09.2014

Verify the intruding objects

False alarm True alarm


27 ICCE – Berlin, 7 – 10.09.2014

Architecture – Communication


28 ICCE – Berlin, 7 – 10.09.2014

Outline





– What & How?




29 ICCE – Berlin, 7 – 10.09.2014

Intrusion detection in a factory-like

scenario

Safety zone is marked by the

red square Top view of the protected zone


30 ICCE – Berlin, 7 – 10.09.2014

Detection of an unauthorized entry in the safety zone

Intrusion detection in a factory-like

scenario


31 ICCE – Berlin, 7 – 10.09.2014

Intrusion detection by using both

passive and active sensors

Two cameras were placed oppositely;

they were both facing the candy plate.

The target to be protected

marked by a red square.


32 ICCE – Berlin, 7 – 10.09.2014

Learned lessons

• Depth sensing sensors: improve the efficiency and robustness of IDS by delimiting in a precise way the volume of interest.

• The great advantage: easiness of setup and flexibility.

• Combination of multiple depth sensors: – useful when one of the sensors becomes occluded;

– interchangeably use of Passive / Active stereo cameras.


33 ICCE – Berlin, 7 – 10.09.2014

Conclusion

• Practical system for detecting volumetric intrusion in a predefined restricted area by using a depth-based camera.

• Experimental scenarios show a great potential for a mixed passive/active stereo vision IDS.


34 ICCE – Berlin, 7 – 10.09.2014

Acknoledgements

• Funding:

– European Unions Seventh Framework Programme managed by REA (Research Executive Agency) (FP7/2007-2013) under grant agreement no. FP7-SME-2012 - DSenS.

• Technical support:

– Rdnet company (http://www.rdnet.fi/).

– Xtrust company (http://www.xtrust.net/).


35 ICCE – Berlin, 7 – 10.09.2014

Thank you!

Virtual 3D Shield for Asset Protection

Documents

Transcript of Virtual 3D Shield for Asset Protection