Created in:

Autism Language Program,

Children’s Hospital Boston

Howard Shane, Ph.D.

howard.shane@childrens.harvard.edu

Use Of Electronic Media To Promote Acquisition of Action Verbs

and Prepositions

www.aac-rerc.com

This work is funded by the National Institute on Disability and Rehabilitation Research of the U.S. Department of Education, under grant number H133E030018. The opinions contained in this presentation are

those of the grantee and do not necessarily reflect those of the U.S. Department of Education.

Acknowledgement

Contributors to this presentation

• Patients and parents in the Autism Language Program

• Monarch School students, parents, and staff

• Model Autism Program, Boston Public Schools

• Sharon Shaham, James Sorce, Meghan O’Brien, Marie

Duggan & John Costello (Center for Communication

Enhancement, Children’s Hospital Boston)

Today’s Agenda

1. Overview of Teaching Language Concepts Program (TLC)• Why is learning language concepts difficult for some persons with

ASD?• Why are visuals needed?

2. Three Phases of Instruction • Dynamic Scene Cues• Static Scene Cues• Element Cues

3. Developing Effective and Efficient Implementation Procedures for Teaching Element Cues • Using typically developing children

• Adapting and applying findings to individuals with autism

Overview of VIPGoal

Utilize visual supports to teach learners on the autism spectrum how to better communicate during everyday home, school and community interactions.

Key ingredients

1. Visual language for comprehension and expression

2. A ‘closed language’ with restricted pragmatic opportunity -- targeting basic communicative operations

3. Mentors and learners use the same visual symbols

4. Use of advanced computer and video technologies

What is Visual Immersion?

A symbol-rich environment

An environment where visual symbols clarify spoken language, support expressive communication, assist with transitions, and facilitate learning.

• Used in home, school, and community

• Easy access to– Non-electronic symbols – Electronic Screen Media (Monitors and TVs)

What Does VIP Target?• Seven concrete Communicative Operations :

1. Protests/Refusals2. Requests3. Directives4. Comments5. Questions6. Social conventions7. Transitions

• VIP is not intended to teach advanced communicative operations: – Abstract concepts: “…with liberty and justice for all.”

– Passive voice: “The book was read by the boy.”

– Complex syntactic structures: “If he hadn’t checked the weather in the morning, then he would have forgotten to bring his umbrella.”

– Figurative language: “She flew to the bookstore.”

– Humor : “Why did the chicken cross the road?”

• VIP provides foundation for more advanced language-based intervention programs

VIP: Three Instructional Settings

1. Natural setting: Familiar, everyday people,objects, actions, locations and events as opportunities to overlay language. Takes advantage of adding language to already learned behavioral skills:

1. Daily living/self-help routines2. Operational knowledge3. Preferred play routines

2. Tabletop setting: A controlled environment involving one-to-one instruction

3. Virtual setting:Electronic (television or computer) display for presentation of materials

Setting selection varies as a function of child and the skill to be learned.

Overview of Teaching Language Concepts Program (TLC)

Why Teach Language Concepts?Autism Language Program (ALP): • Population: Moderate to severe autism (from initial diagnosis to adulthood)

• Core deficit: Difficulties with language comprehension and expression, in addition to pragmatic language skills

– Comprehension: • Children on the spectrum often experience difficulty comprehending

spoken language and/or auditory processing impairment (Novick et al., 1980; Boddaert et al., 2003).

• Many children are able to follow routine-based, familiar directives, but have not yet demonstrated comprehension of more abstract linguistic concepts (e.g., verbs, prepositions, attributes)

– Expression:• 50% of individuals with autism spectrum disorders do not use speech

functionally (Wetherby and Prizant, 2005; Lord & Paul, 1997; Rutter, 1978)• Many children are able to reliably use PECS to request and label, but have

not yet developed the ability to describe, comment, or ask questions

Model of Language Comprehension(Typical Development)

SpokenLanguage

Short-Term Memory

LanguageProcessor

Comprehension

Model of Language Comprehension(Moderate-Severe ASD)

Verbal Information

Short-Term Memory

LanguageProcessor

Comprehension

= Clinical insight suggests these impairments are due to:•Attention to auditory stimuli •Not understanding language as meaningful/symbolic •Fleeting nature of spoken language•Language processor ‘broken’•Combination of these

Language comprehension is often compromised• Occasional comprehension of noun labels and familiar directives (in context)• Impaired comprehension of relational linguistic concepts (e.g., verbs, prepositions, attributes)

‘Teaching Language Concepts’ (TLC) Program

• A visual instruction system for hard to learn language concepts (e.g., verbs, prepositions and attributes).

• TLC is a closed visual language• Instruction in the virtual environment (video clips), tabletop

environment (photographs, toy figurines and miniature objects), and natural environments (home, school and community)

Three Phases of TLC Instruction

TLC Instructional Phases:

• Learner progresses through all/some of three phases of visual language symbols, from concrete to abstract representations:

1. Dynamic Scene Cues: full-motion video clips of action scenes

Bypassing Spoken Language Comprehension Using Dynamic Scene Cues

(Moderate-Severe ASD)

Verbal Information

Short-Term Memory

Visual Information

(dynamic scene cues)

Imitation(as inferred from

dynamic scene cues)

*Note: No involvement of language processor requiredfor comprehension of dynamic scene cues

TLC Instructional Phases:

• Learner progresses through all/some of three phases of visual language symbols, from concrete to abstract representations:

• Dynamic Scene Cues: full-motion video clips of action scenes

• Static Scene Cues: photographs that capture a prototypical moment in the action scene

Bypassing Spoken Language Comprehension Using Static Scene Cues

(Moderate-Severe ASD)

*Note: No involvement of language processor requiredfor comprehension of static scene cues

Verbal Information

Short-Term Memory

Visual Information(static scene cues)

Imitation(as inferred from static scene cues)

Implications of Static Scene Cue MasteryUse of dynamic and static cues bypasses language

processor– Due to load on language processor when using

elements, some children may use static scenes as their communication system

Mastery of static scene cues is a significant accomplishment– Can be used to promote general understanding and

communication related to:• Daily Living Skills• Play• Transitions• Requesting• Commenting• Clarifying• Directives

TLC Instructional Phases:

• Learner progresses through all/some of three phases of visual language symbols, from concrete to abstract representations:

1. Dynamic Scene Cues: full-motion video clips of action scenes

2. Static Scene Cues: photographs that capture a prototypical moment in the action scene

3. Language Element Cues: graphic icons representing each of the individual linguistic components that comprise an action scene (e.g., subject, object, verb, preposition, adjective, etc.)

TLC Phase 3: Element Cues

Symbolate: Language Comprehension?

• Stringing symbols together does not automatically result in comprehension.

• Learners must first have a knowledge of language elements and semantic relationships, as provided in VIP.

• Paradoxically, stringing symbols together may actually interfere with comprehension.

Model of Visual and Spoken Language Comprehension

(Moderate-Severe ASD)

Verbal Information

Short-Term Memory

LanguageProcessor

Visual Information(element cues)

Comprehension ofVisual

Representation(element cues)

Comprehension ofVerbal

Representation

Clinical Observations About Element Understanding

• Some label elements without meaning attached (echolalic-like) • Some comprehend agent and object elements -- struggle with relational

elements• Some comprehend isolated elements -- difficulty interpreting element

strings

What improves element comprehension?• Immersion in symbol-rich environment

• Mass trials across multiple settings and communication partners

• Computer-based instruction

• Persistence over extended period

Bottom line: A more effective, efficient way of teaching language elements must be developed.

TLC 3 Phases: Clinical Observations

• Progressing from – Dynamic Scene Cues to Static Scene Cues -- is easy

• No involvement of language processor required

– Static Scene Cues to Element Cues -- is difficult• Requires involvement of language processor

But

• The payoff is worth it -- foundation for generative language

– Goal: Combine language element cue vocabulary with their understanding of semantic relationships to generate novel sentences for expressing requests, comments, replies, etc. in natural settings.

The Empirical Challenge

• Can we better understand the process?

• Can we expedite the process?

Developing Effective and Efficient Implementation Procedures for

Teaching Element Cues

Top Down Approach

Example of Mixed Display - with Referencing

The TLC ApproachStrategy: First study typical children, then apply findings to children with

autism Rationale:

– Not yet literate – Able to verbalize thought process– Compliance

Subjects: Ages 3 yrs, 11 mos to 6 yrs, 4 mos

Procedure: General task - sequence three linguistic elements: left-to-right reading order

– agent + action + object– agent + preposition + object

– Experimenter provides no spoken language to label elements– Learner demonstrates comprehension of element string by acting out directives

Multiple design-prototype-test cycles to refine TLC instructional approach

Iteration 1: Top-Down Approach - Sequentially Through Three Instructional Phases

Iteration 1 Procedure• Learner presented with

– Dynamic scene cues; imitate action– Static scene cues; imitate action– Element cues; demonstrate comprehension by

acting out directives

• For each instructional phase, learner viewed the visual cues and was provided with spoken prompt, “Now you do it!”

Outcome:– Able to accurately imitate dynamic and static cues

– Difficulty understanding task when presented with element cues (often attempted to match objects to elements rather than carry out directive)

– Difficulty comprehending left-to-right sequence of ‘reading’ elements

– Difficulty understanding meaning of verb/preposition (abstract) symbols

– Difficulty comprehending de-contextualized element strings.

– Tendency to act on objects in a familiar manner (e.g., ‘man on ladder’ rather than ‘man climbs ladder’)

Iteration 1: Top-Down Approach Sequentially Through Three Instructional Phases

Iteration 2: Modified Top-Down Approach

Modifications to Iteration 1 Procedure:– Introduced visual template to focus attention on left-

to-right order of elements, and presented elements in left-to-right sequence

– Introduced Mixed Display (Static scene cue along with Element cues) and demonstrated association between scene cue and its elements

– To enhance understanding of task expectation:• Presented task as ‘game’ to discover meaning of

the relational (verb or preposition) symbol • Experimenter modeled the task

Iteration 2: Visual Template

Outcome:– Initially learners did not appear to attend to the

relational element -- verb or preposition (e.g., “You don’t need that one”)

– Still initially attempted to match the agent and object elements to their physical objects

– Given repeated trials, demonstrated comprehension of the individual elements, but continued to experience difficulty with the left-to-right ‘reading’ order for elements

Iteration 2: Modified Top-Down Approach

Iteration 3: Bottom-Up Approach

Modifications to Iteration 2 ProcedureInstruction started at element rather than dynamic phase

• Presented elements on computer monitor– Experimenter modeled the task– Guided discovery to select combination of element

strings to play corresponding video clip

• Then moved to tabletop– Presented non-electronic task with identical element

cues and physical objects; – Experimenter modeled the task– Learner encouraged to create element strings to direct

the actions of communication partners

Software Applications

Learning Together with Music Puddingstone Place

Iteration 3: Bottom-Up Approach

Iteration 3: Bottom-Up Approach

Outcome:– Older learners were able to execute each directive

presented and label each element

– Younger learners had difficulty attending to screen and task; more likely to engage in their own independent play with the materials

Iteration 4: Modified Bottom-Up Approach with Compelling Scenes

• Modifications to Iteration 3 Procedure:– Created ‘fun’ scenes depicting engaging, motivating

activities using human models and life-sized objects

– Clicked on message window to activate video clip of corresponding element string (in hopes of enhancing attention to relational element)

– Presented learner with an ‘agent’ element cue representing him/herself

– Encouraged learner to use elements to direct his/her own play and actions of communication partners

Iteration 4: Modified Bottom-Up Approach

Iteration 4: Modified Bottom-Up Approach with Compelling Scenes

Outcome:– Actively engaged

– Successfully combined elements - although still initial tendency not to attend to relational element (verb or preposition)

– Prepositions more difficult than verbs

Iteration 5

Modifications to Iteration 4 Procedure:

– Focus attention to relational element (verb or preposition):

• Animated relational element

• Click on relational element to play video

Key Ingredients to Teaching Element Cues

• Present elements electronically

• Guided discovery with element combinations (to play video)

• Provide children with symbols representing themselves and familiar communication partners to allow them to direct the activity

• Begin instruction with fun, motivating tasks. Gradually expand to language-arts activities.

• Animate center elements to direct attention to element and enhance likelihood of comprehension

Directions for future TLC research

• Improve symbols– Animated symbols (verbs, preposition, attributes)

• Generalization of language skills targeted in TLC to natural

setting

• Facilitating generative language