February 2010 | Concurrent Vision ApS | +4530328964

Real Time Object Recognition and Tracking Concurrent Vision ApS

Contents

Intelligent active video surveillance

Biomedical Image Analysis

Visual Feedback control of robots

Diverse Object recognition applications

Automation systems, especially in the world of robotics, are

becoming faster creating an increasing need to track objects at

higher speeds than ever before.

Systems which rely on computer vision analysis to make artificial

intelligence decisions and provide control, extend from high speed

production lines and robot arms to autonomous guided vehicles,

missiles and planes. Such systems use computer vision algorithms

to extract information from images in a video sequence to identify

and track objects in a scene. Usually these algorithms require high

computational resources from a general purpose processor or a

DSP, causing high computational latencies. High latencies act as a

prohibitive factor for providing true, real time recognition and

tracking of objects moving at high velocities.

Company Concurrent Vision ApS develops real time, high speed,

vision-based systems that identify and track objects in a continuous

video stream. These systems are based on the digital ASIC and

FPGA technologies to implement high speed parallel computations

providing true real time recognition and tracking of objects moving

at speeds above 200 km/h. Typical applications of these systems

include active video surveillance, vision- based robotic arms motion

control, providing cognitive characteristics to robots and tracking

high speed moving targets. Of other applications can be mentioned

video stabilizing, augmented reality, image stitching, real time

demosaicing for high definition video cameras, 3D imaging,

intelligent toy and physical interactive computer games.

Concurrent Vision also provides solutions for the acceleration of high

speed content based image retrieval systems that search for digital images in large databases. An example of such systems is retrieval and matching medical images for computer aided diagnosis.

1.issue, February 2010

Concurrent Vision ApS

Intelligent Active Video Surveillance

STATE-OF-THE-ART COMPONENT

TECHNOLOGIES IN VIDEO ANALYSIS

FOR SURVEILLANCE

Automated video surveillance in commercial, law

enforcement, and military applications is concerned with

real-time observation of people and vehicles in crowded

environments. A type of observation that tends to describe

actions and interactions and probably predict behavior.

Active surveillance as a real-time medium creates effective

deterrence systems protecting people and businesses from

crime and criminal activity. In continuous automated

monitoring of surveillance video, security alerts are issued

responding to burglary in progress or to suspicious

individuals, moves or objects in a scene. Automated Video

surveillance technology has also been proposed in

applications to measure traffic flow, detect accidents on

highways and log routine maintenance tasks at nuclear

facilities. Military applications include patrolling national

borders, measuring the flow of refugees in troubled areas,

monitoring peace treaties, and providing secure perimeters

around bases and embassies. Such video surveillance

presents a number of technical issues including moving

object detection and tracking, object classification, human

motion analysis, and activity understanding.

Concurrent Vision’s solutions solve or aid solving technical

issues of Automated Video surveillance by providing high

speed techniques for the following:

Detection and tracking which involves real-time

extraction of moving objects from video and continuous

tracking over time to form persistent object trajectories.

Human motion analysis which is concerned with

detecting periodic motion signifying a human gait and

acquiring descriptions of human body pose over time.

Activity analysis deals with parsing temporal sequences

of object observations to produce high-level descriptions

of agent actions and multiagent interactions.

2

Concurrent Vision, In

key areas of Video-based detection and

tracking,

Video-based person

identification, and

Large-scale surveillance

systems.

Biomedical Image Analysis

Computer Aided

Diagnosis Reduces

Human Errors Matching Medical Images for Computer Aided Diagnosis

Some studies show that 20 to 40 percent of statements made on radiological reports by radiologists or radiology residents were found to be erroneous. Errors can be classified as observational and interpretational errors. Observational errors can be linked to incomplete or faulty search patterns. Observation is for instance enhanced by taking advantage of the computer ability to see shades of gray beyond the range of human vision and ability to use sophisticated search patterns. Computers can store and analyze all 1000 shades of gray in the photon beam exiting the patient during radiologic scans. Shades representing differences in bone and tissue density, whereas Human visual range can only see 32 or fewer shades of gray. Errors of interpretation can be linked to the practitioner’s failure to link abnormal radiologic signs to relevant clinical data. Using object recognition, image retrieval and matching algorithms, computers can access and process huge amounts of stored clinical data and produce accurate interpretations

Medical databases contain huge amount of information relevant to illnesses

and their cures. These databases contain radiologic medical images that give

pictures of small details of organs in the body. Benefiting from these images is

however quite difficult, since data sets to be analyzed by radiologists is

increasing substantially. Automatic image retrieval and matching Systems

based on scale and rotation invariant object recognition techniques, can be

used to collect or classify the statistical information obtained from the

databases, and perform computer aided diagnosis of diseases and

abnormalities. Computer aided diagnosis improve accuracy of statements made

on radiological reports and reduce both observational and interpretational

human errors in these statements. Automatic retrieval and matching of medical

images takes advantage of the merging of medical imaging with multimedia

technology in networked multimedia systems for image-assisted medical care.

Object recognition techniques depend greatly on extracting and detecting features in

2-D scalar images. Feature points are used to establish correspondence between pairs

of images which is important for landmark based image registration and for building

statistical models of shape and appearance. Extracting and matching Features in

images can for instance be used in content based image retrieval from a database of

fracture images for the purpose of planning surgical interventions after fractures.

Image retrieval and matching can be used to supply similar cases to an example to

help treatment planning and find the most appropriate technique for a surgical

intervention. The Figure below shows examples images of a fracture database used for

retrieval.

Concepts used in the computer vision technique for extracting and matching features

in 2D scalar images can be extended to scalar images of arbitrary dimensionality.

Retrieval and matching of 3D human Magnetic Resonance Imaging (MRI) brain scans

and 4D computed tomography (CT) cardiac scans are examples.

3

Continuous real-time monitoring of vasospasm using TCD Cerebral aneurysm refers to the localized dilation or ballooning of the cerebral artery due to the weakening of the wall of the blood vessel. As the size of the aneurysm grows, the chance of it to rupture increases. Rupture of the cerebral aneurysm will lead to subarachnoid hemorrhage (SAH), which is a serious condition with a mortality rate of 30-60%. The primary treatment for this condition includes open surgery aneurysm clipping and endovascular coiling. Regardless of the treatment, patients suffering from SAH may undergo vasospasm, which is a condition when blood vessels spasm, leading to decreased oxygen delivery. It is most likely to occur within 3-7 days after treatment. As a result, continuous monitoring of the blood vessels within the first 3-14 days of after SAH is desired to assess the presence of vasospasm. At the present time, there are various accepted clinical methods to diagnose vasospasm. A non-invasive technique to monitor vasospasm includes the use of Transcranial Doppler Ultrasound (TCD), which is cost-effective, easy to use and potentially available 24-7. TCD is a tool that transmits ultrasound to measure the blood flow velocity in the blood vessels, which acts as an indicator for the occurrence of vasospasm. However, the use of TCD requires the presence of a skilled ultrasonographer, and suffers from operator dependence. The use of computerized monitoring improve sthe current TCD technology and minimizes the need of dedicated ultrasonographer.

Object Recognition in Multimodal

Biomedical Imaging

Extracting and matching features for

object recognition can be exercised to all

types of medical images acquired by any

existing image acquisition modality. It can

for example be used to automatically

detect and diagnose knee meniscus tears

from MR medical images. It can also be

used to perform real-time analysis of

Transcaranial Doppler Ultrasound

(TCD) image streams for such purposes

as the study of cerebrovascular ischemia

(stroke), the monitoring of blood flow

velocity during intensive care, general

anesthesia and carotid endarterectomy

(CEA), the detection of vasospasms after

subarachnoid hemorrhage (SAH) and the

assessment of arteriovenous malformations (AVM).For instance, many morphological

and dynamic properties of the common carotid artery (CCA), e.g. lumen diameter,

distension and wall thickness, can be measured non-invasively with ultrasound (US)

techniques. This however requires as a preliminary step the manual recognition of the

artery of interest within the ultrasound image. In real-time US imaging, such manual

initialization procedure interferes with the difficult task of the sonographer to select

and maintain a proper image scan plane. Even for off-line US segmentation, the

requirement for human supervision and interaction precludes full automation to

eliminate user interference and to speed up processing for both real-time and off-line

applications.

Automatic object recognition and tracking can also be extremely useful in conjunction

with Optical Coherence Tomography (OCT). OCT is an imaging technique that

allows non-invasive, high resolution, cross sectional-imaging of both transparent and

non-transparent structures. The greatest advantage of OCT is its resolution. Standard

resolution OCT can achieve axial resolution of 10-15 µm. A high resolution OCT

increases the resolution to the sub-cellular level of 1-2 µm. Below, a figure showing a

true subcellular image using OCT with a resolution of 4 µm

OCT has demonstrated feasibility for high-resolution imaging of the vascular system

and other vulnerable tissue. This includes the central nervous system and the cartilage

of joints. OCT can be applied to a variety of applications such as

Diagnosing and monitoring of retinal diseases

Imaging atherosclerotic plaque

Tumor detection in gastrointestinal, urinary, and respiratory tracts

Detection of skin Cancer

Early detection of osteoarthritic changes in cartilage

4

Real-time in vivo Brain Tumor Microvasculature Monitoring Using Combined Laser Scanning Confocal Fluorescence Microscopy and Optical Coherence Tomography in Preclinical Window-chamber Models

Glioblastoma multiforme (GBM) is a common primary brain tumor with aggressive, lethal, and malignant characteristics. Its high proliferative and invasive nature leads to T-cell immunosuppression and drug inefficiency and hinders surgical resection. There is a need to investigate GBM in vivo in preclinical animal models, at the macro and micro levels, that are also potentially translatable to the clinic. Combined intravital microscopy using confocal fluorescence (CF) and optical coherence tomography (OCT) can be used for the purpose. The potential applications include surgical guidance and monitoring of tumor response to photodynamic therapy (PDT). Using this imaging technique enables real-time microvasculature imaging of brain tumors and normal brain tissue in order to track the tumor growth pattern in vivo and to monitor and quantify the tissue responses to PDT treatment. A computerized system based on tumor boundaries recognition would provide a real-time and potentially available 24-7 monitoring and quantification.

OCT is typically used for assessing arterial wall pathology in vivo. It may provide more

detailed structural information than other techniques. With high resolution OCT, and

automatic object recognition, atherosclerotic plaques can be diagnosed in real-time

with high accuracy including measurement of the thickness of thin fibrous caps less

than 65µm. This represents a step towards in vivo assessment of the risk of rupture.

Insight into the physiology of a plaque is complementary to the structural information

offered by the OCT grayscale image. While the OCT image presents morphological

information in highly resolved detail, it relies on interpretation of the images by trained

readers for the identification of vessel wall components and tissue type. Computerized

image retrieval and matching as well as object recognition can help the interpretation

and identification process. It can be used to characterize different atherosclerotic

plaque components by their distinctive signal patterns as shown in the next figure. The

Figure shows histopathologic (hematoxylin and eosin staining; magnification ×40) and

OCT images of a predominantly lipid-rich plaque in quadrants I—III.

The OCT( right) shows the lipid-rich plaque (lip) with a low signal appearance and poorly delineated borders

compared with the signal-rich appearance of the fibrous plaque material (fib). (Courtesy of Meissner OA, Rieber J, Babaryka G, et al: Intravascular optical coherence tomography: comparison with histopathology in

atherosclerotic peripheral artery specimens. J Vasc lnterv Radiol 17:343–349, 2006. © SCVIR.)

OCT is also proving valuable in the differentiation

between cancerous and normal tissues as it is

sensitive to the disruption of normal tissue

architecture. The picture to the left shows the

image of a sarcoma, or muscle tumor, obtained

using (OCT). In the picture, the tissue looks

healthy and normal on the left. To the right, the

structure appears cancerous and irregular. Images

like this one, obtained by using OCT in real-time,

can help detecting tumors early during image-

guided procedures. Due to its inherent

compatibility with the use of compact fiber-based

probes, and the ability to construct portable

systems, OCT appears to be promising in the early detection of several types of cancer

in clinics. OCT aided with computerized recognition and classification of tissue structure

has capabilities for in-vivo detection of bladder cancers, colon cancers, oral cancers

and skin cancer. In detecting breast cancer for instance, compact optical fiber probes

permit access within the ductal structure of the breast or to a suspicious lesion via the

tip of biopsy needle making possible to perform localized optical imaging of tissues at

the needle tip, this along with real-time feedback, has the potential to enhance the

guidance accuracy of the biopsy and reduce “miss-rate” compared to large-core needle

biopsy obtained under ultrasound guidance.

Higher resolution and acquisition rates of OCT images improve real-time imaging

capability. Given the data acquisition rates possible with the state-of-the-art OCT

systems, rigorous human interpretation of every image is not possible in real-time.

Thus high speed computer algorithms are essential for real-time feedback during

biopsy or surgical guidance procedures to enable synchronization of motor rotation

with the high-speed OCT frame acquisition in mechanically actuated probes, and to

enable real-time tissue classification and suspicious object recognition.

Visual Feedback control of robots

Both industrial robot arms and mobile robots require sensing capability to

adapt to new tasks without explicit intervention or reprogramming. Visual sensing capability of robots overcomes many of the difficulties of uncertain

models and unknown environments which limit the domain of application of robots used without external sensory feedback. The image-based structure is an approach to visual servo control, which uses image

features (e.g., image areas, and centroids) as feedback control signals, thus

eliminating a complex interpretation step (i.e., interpretation of image features to

derive world-space coordinates).

In this approach, a 2-D image from a video sequence is processed in order to

extract image features. The extracted features are matched to precompiled object

model features in order to identify, pick or track desired objects or provide an

absolute measure of the robot state (localization). In particular, the information

gathered by a vision system about the environment makes it possible to detect

natural landmarks, navigate among unknown obstacles, and achieve a reactive

robot behavior. Vision-based sensing however has some drawbacks, such as the

need to recognize and extract a huge number of characteristic features from the

image, an increased computational burden, and a critical dependence on lightning

conditions of the environment.

Concurrent Vision ApS delivers solutions in digital logic that compensate for

these drawbacks using a proven algorithm for feature extraction which is invariant

to scale, illumination, occlusion and viewing angle. Moreover, the solution from

Concurrent Vision ApS implements a novel method of feature matching that

accelerates the process up to real time speeds.

Increased speed and

enhanced safety of visual

based robot control

An Example of a vision-based

control task is for a robot arm to

acquire an unoriented object

from a pallet without prior

knowledge of the object position.

A CCD camera attached to the

robot arm provides visual

sensing capability. Images

acquired by the camera are

processed by a computer vision

system in order to identify the

object and infer relationships

between the spatial position of

the object and the camera

position. Such relative position

information is used to guide the

robot to acquire the object. The

same problem arises in the

navigation of a mobile robot with

respect to objects in an

unstructured environment using

visual feedback. The Use of

computer vision to infer position

and orientation of objects or

interpret general three-

dimentional relationships in a

scene is a complex task

requiring extensive computing

resources which may render

robots slow in reacting to

unexpected events or

constraints the robotic system

with respect to speed. Using the

high speed parallel processing

approach, speed limits can be

removed and system safety can

be enhanced. This can especially

be desired in robotic systems

designed to remove defect

objects from high speed

automated production lines.

6

Diverse Object Recognition Applications

Visio-haptic wearable systems for the blind Object recognition and tracking algorithms for visio-haptic

information analysis, i.e., the conversion of visual data

into haptic (tangible) features, can be utilized in wearable

assistive devices for blind individuals. Touch is an

important modality for individuals who are blind, but it is

limited to the extent of one's reach. By estimating how an

object feels from its visual image, we are able to

overcome this limitation.

Common haptic devices and systems allow blind people

recognize three-dimensional (3D) objects that exist in

virtual environment. Such systems allow blind people to

touch, grasp and manipulate objects that exist in the hap-

tic enabled virtual environment. In the regard object

recognition reduces the overall time needed to understand the shape of objects and

provide better immersion to the virtual environment.

Tracking Objects in Augmented Reality

Augmented reality (AR) is a term for a live direct or indirect view of a physical

real-world environment whose elements are merged with virtual computer-

generated imagery - creating a mixed reality. The augmentation is conventionally

interactive in real-time and in semantic

context with environmental elements,

such as sports scores on TV during a

match. With the help of advanced AR

technology the information about the

surrounding real world of the user

becomes interactive and digitally usable.

Artificial information about the

environment and the objects in it can be

stored and retrieved as an information

layer on top of the real world view.

The ability to track visible objects in real-time provides an invaluable tool for the

implementation of Augmented Reality. Once an object has been detected, it’s

location in future frames can be used to position virtual content, and thus annotate

the environment. Object recognition and tracking solutions provided by Concurrent

Vision ApS can effectively be used in real-time AR systems.

Concurrent Vision ApS

Steen Koldsø steen.koldsoe@gmail.com Mobile: + 4550168167

Moatasem Chehaiber moatasem@abs2real.dk Mobile: +4530328964

Object Recognition aid

to the Blind

Obstacle Avoidance:

Object Recognition aids provide advance warning of obstacles and allow the blind to find a safe, clear path. A popular system is the SonicGuide Also known as the Binaural Sensory Aid. This device uses ultrasound to scan the space in front of the user and creates a stereo audio signal that varies in pitch to indicate the distance

of obstacles. The system fits conveniently in the frame of a pair of glasses. Although the rich information provided by the SonicGuide can be extremely useful, learning how to decode this signal requires significant effort There is also fear that the audio signal could mask important environmental sounds. While audition is already tapped for echolocation, the sense of touch is largely unused while traveling. It is thus possible to stimulate the skin without interfering with the normal activities and environmental cues used by the blind. Moreover, it may be easier to represent spatial information on the skin rather than through audition. The best known such aids can be classified as vision substitution systems since they provide sufficiently rich information to be used. These systems use vision by CCD to scan the space and can be integrated in an earpiece on a hearing aid device as well in a haptic device. Such systems are required to be small, lightweight and ultra low power consuming which can be achieved by implementing algorithms in HW using the ASIC technology.

.

7

Company Details Moatasem Chehaiber: Senior Electronics Engineer and CTO Steen Koldsø: Senior Management Consultant and CEO MD. Mohammad Chehaiber: Consultant In Endocrinology MD. Tahseen Chouheiber: Consultant In Orthopedics MD. Mohammad Elhashimy: Consultant In General medicine

Join the biomedical Engineering group on LinkeIn. A forum connecting biomedical

engineers and biomedical imaging experts where people share biomedical engineering ideas based on biomedical image analysis. http://www.linkedin.com/groups?gid=2788350&trk=myg_ugrp_ovr

Concurrent Vision ApS

Hvidovrevej 44 2610 Rødovre +4530328964 moatasem@abs2real.dk Moatasem Chehaiber moatasem@abs2real.dk Steen Koldsø steen.koldsoe@gmail.com