Surveillance sensor systems using CMOS imagers

8/7/2019 Surveillance sensor systems using CMOS imagers

http://slidepdf.com/reader/full/surveillance-sensor-systems-using-cmos-imagers 1/9

Surveillance sensor systems using CMOS

imagers&mems(inertia) sensor

abstract Surveillance sensors are being applied in factory automation systems, trafficcontrol, entrapment protection, automotive safety systems and in other

applications where information about the occupancy of a scene is required. Inorder to detect object motion several methods exploiting distinct physicalphenomena, e. g. passive infrared sensors or active microwave sensors, have

been realized. When comparing all of the applicable methods, the electro-optical

approach outperforms with respect to the spatial resolution of the monitoredarea. Therefore, electro-optical sensors are able to provide additional information,

e. g. to predict the direction of motion, or to localize and identifyobjects.However, convenient image processing systems using CCD sensors for

image acquisition and DSP or ?P boards for signal processing and classification

are not well suited for dedicated, powerful and cost-effective optical sensorsolutions. In contrast to this mainstream approach CMOS based imaging

technologies offer novel solut ions in both the design and applications of electro-optical surveillance sensors.This contribution discusses CMOS imagers operatingprinciples and describes certain architectures and applications for passive and

active surveillance sensors. The capabilities to realize on-chip motion detection

and range sensing using fast shutter devices are illustrated. We conclude with adiscussion of the status of CMOS surveillance sensors and suggest trends for

future applications.

Citation: A. Teuner, M. Hillebrand, B.J. Hosticka, S.-B. Park, J.E. Santos Conde, N.

Stevanovic, "Surveillance Sensor Systems Using CMOS Imagers," iciap,pp.1124, 10th International Conference on Image Analysis and Processing(ICIAP'99), 1999

control Real-time human motion analysis and IK-based human figure

Austin, TexasDecember 07-December 08ISBN: 0-7695-0939-8S. Yonemoto, Div. of Intelligent Syst., Kyushu Univ., Fukuoka, JapanD. Arita, Div. of Intelligent Syst., Kyushu Univ., Fukuoka, Japan

R. Taniguchi, Div. of Intelligent Syst., Kyushu Univ., Fukuoka, JapanThe paper presents real-time human motion analysis based on real-time inversekinematics. Our purpose is to realize a mechanism of human-machine interactionvia human gestures, and, as a first step, we have developed a computer-vision-

based human motion analysis system. In general, man-machine "smart"

interaction requires a real-time human full-body motion capturing system withoutspecial devices or markers. However, since such a vision-based human motion

capturing system is essentially unstable and can only acquire partial informationbecause of self-occlusion, we have to introduce a robust pose estimation

strategy, or an appropriate human motion synthesis based on motion filtering. Tosolve this problem, we have developed a method based on inverse

kinematics, which can estimate human postures with limited perceptual cues such

as positions of a head, hands and feet. We outline a real-time and on-line humanmotion capture system and demonstrate a simple interaction system based on

the motion capture system.



Index Terms: real-time systems; image motion analysis; kinematics; user interfaces; computervision; real-time human motion analysis; inverse kinematics based human figurecontrol; real-time inverse kinematics; human-machine interaction; human

gestures; computer-vision-based human motion analysis system; robust poseestimation strategy; human motion synthesis; motion filtering; perceptual cues;

head position; hand position; foot position; real -time on-line human motion

capture system

Citation: S. Yonemoto, D. Arita, R. Taniguchi, "Real-time human motion analysis and IK-

based human figure control," humo, pp.149, Workshop on Human Motion

(HUMO'00), 2000

motion detection From Wikipedia, the free encyclopedia

Motion can be detected by measuring change in speed or vector of an object or

objects in the field of view. This can be achieved either by mechanical devices

that physically interact with the field or by electronic devices that quantify and

measure changes in the given environment.

When motion detection is accomplished by natural organisms, it is called motion

perception.

Motion can be detected by: sound (acoustic sensors), opacity (optical and

infrared sensors and video image processors), geomagnetism (magnetic sensors,

magnetometers), reflection of transmitted energy (infrared laser radar, ultrasonic

sensors, and microwave radar sensors), electromagnetic induction (inductive-loop

detectors), and vibration (triboelectric, seismic, and inertia-switch sensors).

Acoustic sensors are based on: electret effect, inductive coupling, capacitive

coupling, triboelectric effect, piezoelectric effect, and fiber optic transmission.

Radar intrusion sensors have the lowest rate of false alarms.

Probably the best radar intrusion sensor is a pair of leaky coaxial lines hidden

(buried) in the soil. A chirp frequency modulation provides a continuous target

response having a baseband frequency that is proportional to the distance along

the length of the cables.



Chip

The PrimeSensor Reference Design is built around PrimeSense¶s PS1080 system

on a chip (SoC). The PS1080 SoC houses extremely parallel computational logic,

which receives a Light Coding infrared pattern as an input, and produces a

VGA-size depth image of the scene.



The PS1080 SoC is a multi-sense system, providing a synchronized depth image,

color image and audio stream. The PS1080 includes a USB 2.0 PHY, whose USB

2.0 interface is used to pass all data to the host. The PS1080 makes no

assumptions about the host device CPU ± all depth acquisition algorithms run on

the PS1080, with only a minimal USB communication layer running on the host.

This feature provides depth acquisition capabilities even to computationally

limited host devices.

The PS1080 is PrimeSense¶s second chip generation, and is already produced in

mass-market quantities.The PrimeSensor Reference Design is an end -to-end solution that

enables a device to perceive the world in th ree dimensions and to

translate these perceptions into a synchronized image, in the same

way that humans do. The solu tion includes a sensor component,

which observes the scene (users and their surroundings), and a

perception component, or brain, which comprehends the user

interaction within these surroundings.

The PrimeSensor Reference Design is able to see and track user

movements within the scene and provide the application layer with

control widgets ± a simple, clear API that translates user gestures

or postures into known, deterministic application inputs. All activity

is performed without any assumptions about the host, the user orthe environment. No wearable equipment is required, making the

solution practical, convenient, intuitive and easy to use. The sensor

provides a natural interface to living-room devices (such as game

consoles and set-top boxes), mobil e devices and more.

PrimeSense is dedicated to bringing a novel, Natural Interaction

solution to the mass consumer market. PrimeSense provides both a

thin-host PrimeSensor depth acquisition device, as well as

embedded middleware components that perform depth processing.

To support a fast-growing market, PrimeSense has teamed up withdepth-processing middleware partners and application providers to

co develop a unique holistic solution. The benefits of this novel,

collaborative API are clearly visible ± enabling companies that adopt

the natural interface to quickly capitalize on their investment by

means of new applications, new experiences and new target

audiences.



IMPLEMENTATION Microsoft project natal:-

A prototype of 3DV Systems' motion-sensitive video camera, the technology that is

most likely behind Microsoft's Project Natal

A s l i de f rom Mic roso f t ' s E3 Conference show ing a d iagram o f the techno log ies in K inec t

Kinect is based on software technology developed internally by Rare, a subsidiary

of Microsoft Game Studios owned by Microsoft and range cameratechnology by

Israeli developer PrimeSense, which interprets 3D scene information from a

continuously-projected infrared structured light.[20][21] This3D scanner system is

called Light Coding [22], employing a variant of image-based 3D

reconstruction.[23][24]

The Kinectsensor[10] is a horizontal bar connected to a small base with a

motorized pivot and is designed to be positioned lengthwise above or below the

video display. The device features an "RGB camera, depth sensor and multi-array

microphone running proprietary software",[25] which provide full-body 3D motion



capture, facial recognition and voice recognition capabilities. At launch, voice

recognition was only made available in Japan, the United Kingdom, Canada and

the United States. Mainland Europe will receive the feature in spring 2011. [26] The

Kinect sensor's microphone array enables the Xbox 360 to conduct acoustic

source localization and ambient noise suppression, allowing for things such asheadset-free party chat overXbox Live.[10]

The depth sensor consists of an infrared laser projector combined with a

monochrome CMOS sensor, which captures video data in 3D under any ambient

light conditions.[10][27] The sensing range of the depth sensor is adjustable, with

the Kinect software capable of automatically calibrating the sensor based on

gameplay and the player's physical environment, such as the presence of

furniture. [28]

Described by Microsoft personnel as the primary innovation of

Kinect,[29][30][31][32] the software technology enables advanced gesture recognition,

facial recognition and voice recognition.[33] According to information supplied to

retailers, the Kinect is capable of simultaneously tracking up to six people,

including two active players for motion analysis with a feature extraction of 20

joints per player.[34]

Through reverse engineering efforts,[35] it has been determined that the Kinect

sensor outputs video at a frame rate of 30 Hz. The RGB video stream uses 8-bit

VGA resolution (640 × 480 pixels) with a Bayer color filter, while the

monochrome depth sensing video stream is in VGA resolution (640 × 480 pixels)

with 11-bit depth, which provides 2,048 levels of sensitivity. The Kinect sensor

has a practical ranging limit of 1.2±3.5 m (3.9±11 ft) distance when used with the

Xbox software. The area required to play Kinect is roughly 6m², although the

sensor can maintain tracking through an extended range of approximately 0.7±6

m (2.3±20 ft). The sensor has an angular field of view of 57° horizontally and 43°

vertically, while the motorized pivot is capable of tilting the sensor up to 27°

either up or down. The horizontal field of the Kinect sensor at the minimum

viewing distance of ~0.8 m (2.6 ft) is therefore ~87 cm (34 in), while the vertical

is ~63 cm (25 in), resulting in a resolution of just over 1.3 mm (0.051 in) per

pixel. The microphone array features four microphone capsules [36] and operates

with each channel processing 16-bit audio at a sampling rate of 16 kHz.



Living Room Entertainment

The living room environment is naturally the heart of home entertainment. While living

room entertainment technologies have advanced tremendously over recent years, the

interaction and control mechanisms used in these technologies are seriously lagging

behind, undeveloped for decades.

Thin displays, home theater PCs, advanced set-top boxes, IP TV and other technologies (as

recent as 3D TV) deliver genuine entertainment experiences. Nonetheless, they are all

controlled by standard infrared remote control units, invented over half a century ago.

While remote control unit manufacturers continuously crammed more and more

functionality into a single hand-held apparatus, this functionality was unusable to the

average user. It was as if the market need threshold was dramatically overshot by remote

control technology, as illustrated in the figure below. In essence, because interaction with

complex remote control functions is unnatural for most users, a usability gap resulted,

leaving a vacant space for a new technology that could provide users with a more natural

approach to control functionality.

A hands-free, gesture-based control paradigm is the most-suitable natural interaction

method. In recent years, this method has reached a consumer market price-performance

point, where the user acquisition accuracy, resolution, robustness to ambient light,

processing speed and cost have all passed the mass market threshold.

Low-cost, hands-free, gesture-based control is enabled via a three-dimensional view of the

living room scene. While traditionally such a 3D view has been computationally expensive

and therefore prohibitive to the consumer market, PrimeSense is now offering a consumer

mass-market depth sensor that can provide hands-free control utilizing a computationally

thin host.

There are numerous examples of living room applications that can make excellent use of a



natural interaction paradigm. These include TV watching, content browsing and

consumption (for example, TV content, pictures and music), web browsing, social

interaction, fitness exercising and many more.

Consumer Electronics

There are numerous consumer electronics applications that can benefit

great ly from the use of computat ional ly cheap, three -dimensional vis ion.

Products such as video communications, security systems, digital signage,

air conditioning, touch screens and many others can have their

performance and resulting user experience significantly enhanced by

util izing depth information.

PrimeSense's goal is to provide immer sive interacti on between the user

and the technology, simplifying the interface and making the experience

seamless and intuitive. For that purpose, the technology has to be awareof the user, his/her location, gestures and intentions. This is enabled

primarily by computer vision. Until today, computer vision was a very

expensive component to add to consumer electronic devices, gi ven the

intensive computational power required to add a depth measurement for

every pixel in the image.

However, with the emergence of a low -cost depth sensor, computer vis ion

has become significantly more accessible and affordable. Simple

interaction paradigms previously beyond commercial viability are now

readily deployable in the mass consumer electronics market.

Surveillance sensor systems using CMOS imagers

Documents

Transcript of Surveillance sensor systems using CMOS imagers