By: Grace Sze-en FooTIES Intern (Summer 2009 )

Grocery Shopping Assistant for the Blind/Visually Impaired (GroZi)

“….The real problem of blindness is not the loss of eyesight.

The real problem is the misunderstanding and lack of information that exist.

If a blind person has proper training and opportunity, blindness can be reduced to a physical

nuisance….”

10 million blind and visually impaired people live in the U.S (American Foundation for the Blind)

Goal:empower blind/visually impaired individuals to

shop at their own convenience and privacy using computer vision.

What is GroZi

Winter 07Creation of blind-accessible web interface that

allows users to create a shopping list.

Spring 07 (TIES & MAE156B)Further development of work from Winter 07.Remote Sighted Guide (RSG), incorporating

vibration and audio commands

Previous Work

Fall 08 & Winter 09Creation of Soylent Grid. Customized

database and user interface (UI) for labeling products.

Winter 08Replaced Fall 08 prototype with wristbands

with motors attached.

Previous Work

Spring 09Mechanical design & creation of software to

control wiimote.Mouse-Click Recorder Program that allows

review of digital images.

Previous Work

Summer 2009

The Team

Supervisor:Dr. Serge Belongie

Client:Dr. John Miller

Grace Sze-en Foo

Tommy Mueller

Masaaki Kokawa Jeffrey Wurzbach

Implement a Remote Sighted Guide (RSG)AudioHaptic (Vibration)

Goals for Summer

Set Up

Sound processed to give the listener the impression of a sound source within a three-dimensional environment.

Relatively new technology.

Used in Stage 3 to provide approximate location of product on the product shelf.

Spatialized Audio

CamerasInitially

Wireless analog camera Radio frequency (RF)

transmission. VGA/RCA feed.

Finalized cameraMicrosoft LifeCam VX-

6000OpenCV compatibleStable USB connectionHigh resolution

Primary functions harnessed by GroZi : VibrationIR Sensor Basic Audio

Tasks: Audio Sampling Rate (Audacity)Created software implementable by

the Remote Sighted Guide interfaceProvided audio cues for the 7

stages.

Wiimote

Remote Sighted Guide (RSG)

Been working on the project since January

Learnt a tremendous amount

Gratifying to see progress of project

Motivation and cultivation of a heart for service engineering

Immensely impressed with TIES program and would love it if I could bring it back with me to Australia/Malaysia.

Personal Touch

GroZi is an innovative shopping assistant designed for the blind/visually impaired.

Summer accomplishments: Investigated feasibility of using spatialized

audio.Packaged wiimote functions for easy

integration in the RSG interfaceFinalized choice of cameraImplemented an RSG interface

Summary

Spatalized Audio

http://www.qsound.com/demos/london-tour.htm

http://listenwithyourownears.com/3d-audio-demo-showdown/

Wiimote functionality

Remote Sighted Guide (RSG)

Demos