Post on 18-Jan-2016
description
Image Recognition using Hierarchical Temporal Memory
Radoslav ŠkovieraÚstav merania SAV
Fakulta matematiky, fyziky a informatiky UK
Image Recognition
• Applications: Digital image databases, surveillance, industry, medicine
• Tasks: Object recognition, automatic annotation, content based image search
• Input: Digital Image– Single object– Scene (multiple objects – clutter, occlusion, merging)
• Output: Description of the input image– Keywords, scene semantics, similar images
• Subtasks: image segmentation, feature extraction, classification
Motivation
• Image recognition – Very easy for us humans (and [other] animals)– Computers can‘t do it neither quickly, nor
accurately enough, yet• Good motivation for the researchers in the
field of AI – bio-inspired models
Hierarchical Temporal Memory (HTM)
• Developed by Jeff Hawkins and Dileep George (Numenta)• Hierarchical tree-shaped network• Bio-inspired – based on large scale model of the
neocortex• Consists of basic operational units – nodes
– Each node uses the same two-stage learning algorithm:1) Spatial Learning (Pooling)2) Temporal Learning (Pooling)
– Learning is performed layer-by-layer– Nodes have receptive fields – each (except for the top node)
can look only at a portion of the input image
Spatial Learning
• Observe common patterns in the input space (training images)
• Group them into clusters of spatially simillar patterns
• Use only one representative of each cluster– Generate „codebook“
• Input space and spatial noise reduction
Temporal Learning
• Uses time sequences to learn correlations of spatial patterns
Temporal Learning
Temporal Learning
Temporal Learning
• In each training step, TAM is increased at the locations corresponding with the co-occurring codebook patterns according to the update function defined as follows:
Inference & Classification
• Uses simlar dataflow as learning• Two stages of inference in each node:– Spatial inference – find the closest pattern in the codebook– Temporal inference – calculate membership into temporal
groups
• Classification – HTM itself does not classify images, it only transforms input space into another (hopefully more inviariant) space– External classifier must be used
ATM Security
• ATM (automatic teller machine) semiatomatic fraud detection system– Detection of masked individuals interacting with
the ATM through the ATM‘s camera – possibility of illegal activity
• Pilot system implemented and tested in an experimental environment
• Using Kinect as an input device
Kinect
• RGB camera developed for the XBOX game console– Capable of providing depth image for the scene and a
„skeleton“ if a person is detected on the scene
Experiment Setup
Face Image Segmentation using Kinect
Face Image Segmentation using Kinect
• Two image classes: normal and anomalous faces
ATM Security – Results• Image set inflated with translated, rotated and mirrored copies of the
original images• k-NN classifier in the input space was compared with the combination
of the HTM and k-NN and HTM and SVM classifier
• Scenario 1: The whole data set was used and• Scenario 2: Translated images were excluded from the training set
New features and algorithmsfor the HTM
• New temporal pooler
• Images transformed to different image spaces– different image features
• Various settings for the temporal pooler• SOM as spatial pooler
Testing of new image features
• Dataset: selected images from Caltech 256– 10 classes, 30 testing and 30 training images per class
• Single layer network– With 1-NN classifier as top node– Image features extracted from image patches
corresponding to the receptive fields of nodes
Results
%
TE window step size in pixels
s1 s2 s4 s8
RGBCA 42,87 41,61 40,86 38,00
med 42,50 41,33 41,00 38,17
GreyCA 40,13 39,63 38,41 34,68
med 39,67 39,33 37,83 35,67
CannyCA 40,35 42,33 43,66 43,55
med 40,50 41,83 43,00 43,00
LabCA 44,92 44,17 44,23 43,17
med 44,83 44,50 43,67 43,67
GLDCA 45,95 46,01 46,43 46,10
med 46,00 46,12 46,17 46,00
problems - background
problems - background
Thank you for your attention