Street Smarts: Visual Attention on the Go
description
Transcript of Street Smarts: Visual Attention on the Go
![Page 1: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/1.jpg)
Street Smarts:Visual Attention on the Go
Alexander PatrikalakisMay 13, 2009 6.XXX
![Page 2: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/2.jpg)
Vision of Attention
• For machines to recreate human visual attention, we must accept that humans:– Maintain multi-scale orientation, intensity, and
color feature neuronal maps in parallel– Combine multi-scale features into a central
conspicuity (saliency) map– Maintain a Winner-Take-All neural network that
saccades to and subsequently inhibits decreasingly salient points
![Page 3: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/3.jpg)
ExampleObject recognition at all points of an image is infeasible time-wise
Visual attention allows us to find the interesting points quickly
Ullman agrees: “Recognition over the whole scene leads to a combinatorial explosion.”
![Page 4: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/4.jpg)
Implementation Steps
• Analyzed previous work done by Ullman, Itti, and Koch on visual attention
• Implemented visual saliency model in C++ using Intel OpenCV, IPP, and TBB
• Implemented FOA shifting by saccading to points with decreasing saliency map values; same effect as a 2D neuronal matrix
![Page 5: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/5.jpg)
Results
• Tested algorithm on 13 geometric scenes, and obtained plausible salient winners in each
• Tested algorithm on 40 natural scenes (roads and highways) and found that signs and signals are very salient (usually saccaded to first)
• Algorithm resilient to noise and takes advantage of multi-scale analysis
![Page 6: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/6.jpg)
Itti: Normalization• Promote maps with small
numbers of strong maxima• Suppress maps with large
numbers of equally strong maxima
• Method: scales maps by the difference between global maximum and mean of remaining maxima
![Page 7: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/7.jpg)
Ullman, Itti, Koch: Multi-scale features
Multi-scale Architecture Three Feature Maps
![Page 8: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/8.jpg)
Ullman: The Winner-Takes-All (WTA)
![Page 9: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/9.jpg)
Simple Example
![Page 10: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/10.jpg)
Noise Resilience
![Page 11: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/11.jpg)
Multi-scale Advantage 1
![Page 12: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/12.jpg)
Multi-scale Advantage 2
![Page 13: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/13.jpg)
Problematic distractions
![Page 14: Street Smarts: Visual Attention on the Go](https://reader035.fdocuments.us/reader035/viewer/2022062323/56816367550346895dd43fe9/html5/thumbnails/14.jpg)
Contributions
• Reviewed past work done on biologically inspired visual attention models
• Identified Itti’s algorithm as a candidate for saliency detection in natural scenes involving road signs
• Demonstrated algorithm’s effectiveness on many natural scenes involving road signs
• Created a prototype saliency heuristic for evaluating sign effectiveness