1

Tactile Graphics with a Voice

2015

Richard LadnerUniversity of Washington

2

Text Book Translation Problem

text

math

images

Tactile Perception

3Somatosensory system

4

Tactile Perception Limitations

• Resolution of human fingertip: 25 dpi

• Tactual field of perception is no bigger than the size of the fingertips of two hands

• Color information is replaced by texture information

• Visual bandwidth is 1,000,000 bits per second, tactile is 100 bits per second

Resolution Study

5

.35mm 1 mm 2 mm

• On average, blind participants could sense 1 mm separation but no smaller.

• On average, sighted participant could sense 1.5 mm separation but no smaller.

Blind Spatial Perception

• Tactile perception can lead to a “spatial” model in the brain.

Tactile Graphic

7

Tactile Graphics Kit

8

Tactile Graphics Printing

• Embossing Printer

• Thermoform

• Swell Paper

9

10

Tiger Embosser

• 20 dpi (raised dots per inch)• 7 height levels (only 3 or 4 are distinguishable)• Prints Braille text and

graphics• Prints dot patterns for

texture• Invented by a blind physicist,

John Gardner

Refreshable Tactile Display

11

12

Graphics

13

Graphic Translation<LocationInformation><NumLabels>16</NumLabels><Resolution>100.000000</Resolution><ScaleX>1.923077</ScaleX><ScaleY>1.953125</ScaleY>-<sd><x1>121</x1><y1>45</y1><x2>140</x2><y2>69</y2><Alignment>0</Alignment><Angle>3.141593</Angle></Label>

preprocesstext extract

cleanimage

originalscannedimage

puregraphic

textimage

locationfile

14

Graphic Translation<LocationInformation><NumLabels>16</NumLabels><Resolution>100.000000</Resolution><ScaleX>1.923077</ScaleX><ScaleY>1.953125</ScaleY>-<Label><x1>121</x1><y1>45</y1><x2>140</x2><y2>69</y2><Alignment>0</Alignment><Angle>3.141593</Angle></Label>

puregraphic

textimage

locationfile

y(0,20)x=1515105Ox510152020x+y=20(15,0)(15,5)

y(#0,#20)x.k#15#15#10#5Ox#5#10#15#20#20x+y.k#20(#15,#0)(#15,#5)

text Braille

15

16

Available Books

• Computer Architecture: A Quantitative Approach, 3rd Edition 25 minutes per figure (230 figures)

• Advanced Mathematical Concepts, Precalculus with Applications 6.3 minutes per figure (1,080 figures)

• An Intoduction to Modern Astrophysics 10.2 minutes per figure (467 figures)

• Discrete Mathematical Structures 8.8 minutes per figure (598 figures)

• Introduction to the Theory of Computation, 2nd Edition 13.3 minutes per figure (180 figures)

17

TGA Workflow

• Advantages– Much faster production– Batch processing instead of one figure at a

time– Much tedious work is avoided

• Disadvantages– May be of lower quality than custom

translation– A lot of technology needs to be mastered

18

One-offs vs. Mass Production

1916 WoodsDual Power

Model T1906 Reo

Audio-Tactile Graphics

19

Talking Touch Tablet

Digital Pen audio-tactile graphic

Tactile Graphics with a Voice: using QR codes to access text

in tactile graphics

• C.M. Baker, L.R. Milne, J. Scofield, C.L. Bennett, RE Ladner. Proceedings of the 16th international ACM SIGACCESS conference on Computers. ASSETS 2014.

20

21

Tactile Graphics with a Voice

System Use

22

23

Modes

Top

Silent Verbal Finger Pointing

Study

• 10 blind users

• 4/10 non-Braille

• 4 tasks

• 3 modes

• 6 sessions

24

Tasks

Line Parabola

Triangle Bar Chart

25

Time

26

F5, 640=2.268, p<.05

F2, 640=0.619,p=.5391

Time outs

27

  Silent Verbal Finger Pointing

Line 97% 97% 93%

Parabola 93% 95% 95%

Triangle 88% 88% 90%

Bar Chart 73% 70% 79%

All Tasks 88% 88% 89%

Accuracy

28

• 4 preferred silent– To be honest, I use screen readers every day and I am so sick

of electronic noise.

• 4 preferred finger pointing– I like the concept of the finger pointing. I feel more confident that

since it looks for a finger it’s getting the right QR code if you have multiple on the same page.

• 2 preferred verbal– The other thing I like about Verbal mode is that every time I hear

a zero I think so I need to move it a little bit

Preference

29

In a study of 82 images from a pre-calculus textbook, on average Braille took up more space

– Braille was larger in 326 out of 626 labels• Braille labels: 327 sq. mm

• QR codes: 225 sq. mm

Braille comparison

30

Braille - Time

31

Braille - Accuracy

32

Questions

33