1

Final PresentationFinal PresentationSpring 2006 T/TAC Online Project Spring 2006 T/TAC Online Project Virginia AssessmentsVirginia AssessmentsMay 18, 2006May 18, 2006

Matt HumphreyAala’a MashaalPamela Sharpe

Tim SmithMaria Washington

Graduate School of Education George Mason University

2

T/TAC Online RegionsT/TAC Online Regions

3

Our Faculty and FacilitatorsOur Faculty and Facilitators Dr. Michael Behrmann Dr. Nada Dabbagh Dr. Kevin Clark Anya Evmenova Gary Dickelman Jocelyn DelSignore

4

ClientsClients John Eisenberg – Severe Disabilities/Assistive

Technology Specialist, Virginia Department of Education

Mary Wilds – Statewide Coordinator of Distance Education, Old Dominion University

Dr. Michael Behrmann – Professor, Director of Helen A. Kellar Institute for Human disAbilities (KIHd), George Mason University

Nathan Sparks – Coordinator, Division of Assessment and Reporting, Virginia Department of Education

5

Presentation OverviewPresentation OverviewPart One

Part Two

Part Three

Part Four

Part Five

• Instructional Design Process

• Background & Introduction• Mission and Goal of Project

• Analyses• Design • Development

• Formative Evaluation• Prototype Demonstration

• Future T/TAC Site Recommendations

• Questions and Answers

6

Project BackgroundProject Background No Child Left Behind Act (NCLB) 2002

Every child must be assessed according to the state standards

Individuals with Disabilities Education Improvement Act of 2004 (IDEA)

Accommodations for all children with disabilities

7

Project BackgroundProject Background Assessment options for students who are

exempt from traditional assessment programs Virginia Alternate Assessment Program (VAAP)

Students assessed at grade levels K-5 Virginia Grade Level Assessment (VGLA)

Students assessed at grade levels 3-8 Virginia Substitute Evaluation Program (VSEP)

Students assessed at grade levels 9-12

8

Project Problem StatementProject Problem StatementVA AssessmentsVA Assessments

Students with severe disabilities were only assessed on life skills prior to new mandates

Transition to cognitive standards based assessments

Communication skills are now considered in the assessment process

New process for teachers Our project concentrates on the VAAP

assessments

9

Instructional Design ProcessInstructional Design ProcessThe ADDIE ModelThe ADDIE Model

Analysis Design Development Implementation Evaluation

10

The Instructional Design ProcessThe Instructional Design Process Who

Learner What

Learner needs Instructional Strategies

How Evaluation

11

AnalysisAnalysis

Performance AnalysisNeeds AnalysisLearner Analysis Task Analysis

Analysis Design Development Implementation Evaluation

12

Analysis - FindingsAnalysis - FindingsPerformance AnalysisPerformance Analysis

New assessment process for teachers Teachers were not using standardized

procedures for assessments Teachers did not have sufficient

access to resources on the VA DOE website

Teachers were not targeting the Aligned Standards of Learning or communication skills standards

13

Analysis - FindingsAnalysis - FindingsNeeds AnalysisNeeds Analysis

Teachers must begin aligning lesson plans to new ASOL guidelines Include cognitive and communication skills Scaffolding needed

Skills will be measured in four subject areas: 1) Reading, 2) Math, 3) History & Social Science, 4) Science

Lesson plans will be scored on a scale of 1 to 4 A score of 3 is needed to qualify

Teachers should be able to differentiate and adapt existing SOL Enhanced lesson plans

14

Analysis - FindingsAnalysis - FindingsLearner AnalysisLearner Analysis

Varied experience levels amongst teachers to be addressed in solution system

Using the data collected from the surveys and interviews with SMEs, teachers, and administrators we found that our users were: Novice Special Education Teacher Intermediate Special Education Teacher Expert Special Education Teacher Reluctant Special Education Teacher T/TAC Online Administrator

15

Analysis - FindingsAnalysis - FindingsTask AnalysisTask Analysis

Existing lesson plan document Instruction is focused on the process of

creating lesson plans Support the teachers in aligning them to the new

standards Task analyze the lesson plan process

Understand the process used by a special education teacher

Support those processes in the solution system

16

Analysis – FindingsAnalysis – FindingsTask AnalysisTask Analysis

Creating a lesson plan is a procedural task Problem solving tasks are embedded

Break down each step of the lesson plan creation process into its constituent parts

Identify what the system should provide to support teachers throughout the process Supports will be utilized in the solution system

17

Instructional ObjectivesInstructional Objectives Facilitate and support lesson plan creation

process EPSS guides teachers through each step of

creating a lesson plan Additional supports assist teachers throughout

the process Encourage knowledge sharing

Lesson plan database Allows teachers to search and download

exemplary lesson plans Search VAAP Manual

18

DesignDesign

Analysis Design Development Implementation Evaluation

Design ApproachStoryboards

19

Design ApproachDesign Approach Support the process of creating lesson

plans aligned to new standards and allow knowledge sharing

Develop an electronic performance support system (EPSS) to be used by teachers

EPSS should include guides and necessary supports that will: Guide and scaffold the lesson plan creation

process Allow special education teachers the ability to

develop, share, and download lesson plans

20

Design ApproachDesign ApproachRationaleRationale

Supporting an existing process Teachers know how to create lesson

plans New constraints on that process

Teacher experience levels vary Just-in-time support in the classroom

21

Design ApproachDesign ApproachEPSS CharacteristicsEPSS Characteristics

Electronic Performance Support System (EPSS)

Computer software program that improves performance by: Reducing complexity Providing the performance information to

perform a task Providing a decision support system that

enables the user to identify appropriate actions

22

Design ApproachDesign ApproachEPSS CharacteristicsEPSS Characteristics

Job aid Controlled by the user, used on

demand Reduce the need for prior training in

order to accomplish the task Supports new constraints on an

existing process

23

DevelopmentDevelopment

Analysis Design Development Implementation Evaluation

FlowchartsPrototypeUsability Testing

24

Development of PrototypeDevelopment of Prototype Operationalize instructional objectives VAAP section will include three main

sections:1. Create Lesson Plans

Teachers will be guided through the lesson process with additional just-in-time supports

2. Search the VAAP Manual Enable teachers to find VAAP information more

efficiently3. Search for Created Lesson Plans

Teachers can search and download exemplary lesson plans created by other teachers

25

Flowchart 1:Flowchart 1: Searching the VAAP Searching the VAAP ManualManual

26

Flowchart 2: Lesson Plan DocumentFlowchart 2: Lesson Plan Document

27

Flowchart 3: VA Assessments Site FlowFlowchart 3: VA Assessments Site Flow

28

DevelopmentDevelopmentLesson Plan Generator (LPG)Lesson Plan Generator (LPG)

29

Lesson Plan GeneratorLesson Plan GeneratorFinal PrototypeFinal Prototype

30

EvaluationEvaluationFormative EvaluationFormative Evaluation

The formative evaluation consisted of three stages of usability testing:

Stage one - novice users One on one, think aloud protocol

Stage two - experienced users. One on one, think aloud protocol

Stage three - expert users Small group, expert review

31

EvaluationEvaluationResultsResults

Positive Aspects of the System as Identified by the Testers:

Users could easily sign in with their existing accounts and new users could create an account easily

The navigation is clear and consistent with lesson plan objectives of the teachers simulating the real process for this task

The supports are appropriate for each section of the system with adequate information to support a teacher in creating a great lesson plan

The system is easy and clear in choosing organizing topics and having the standards auto populate dynamically is a necessary and strong feature

32

EvaluationEvaluation System Improvements

Usability Instructional Content Interface Design

33

EvaluationEvaluationUsabilityUsability

Add a descriptor to the lesson plan generator system so users are aware of what they can do (This is an area where you can create your own lesson plan, ideas are provided through links on the right)

Make text throughout the site bolder with a sharper font for varying visual needs of users

Create a flashing button for “select” next to the organizing topic to assure that the users get the correct set of standard

34

Development Changes Development Changes Instructional ContentInstructional Content

Create documents defining the standards and organizing topics

Reword the directions for the activities and strategies section to be clearer to the users

Add scoring rubrics to the evidence collection page to identify significance of peer interaction

Add examples and clarification documents to the evidence page

Remove the list of AT Vendors as it will not assist the user with the task at hand.

35

Development ChangesDevelopment ChangesInterface DesignInterface Design

Made the Navigation bar have actual section titles as well as highlight where the user is in the system.

Made the logon screen bigger and the word “new user” larger to accommodate different visual needs

Made the user name and date auto populate based on logon user information.

36

Next Steps - RecommendationsNext Steps - RecommendationsGeneral recommendations include the following:1. Add a spell check feature.2. Create a flashing button for “select” next to the organizing topic to

assure that the users get the correct set of standard3. Convert help files to PDF instead of Microsoft Word documents.4. Add a Help button on Preview screen. Also an enlarge font button.5. The age appropriate materials document should be changed to a

pop-up box instead of a word document since the content is short to make loading it quicker.

6. Add visuals for better comprehension.7. System implementation – experienced teachers entering lesson

plans into the Lesson Plan Generator8. Make the lesson plan search feature functional9. Summative evaluation

37

AcknowledgementsAcknowledgements Shuangbao Wang – T/TAC Online

Programmer Seunghun Ok – T/TAC Online Site

Administrator MSHA Immersion Team Kilmer Center - Vienna, VA Courthouse Road Elementary –

Spotsylvania County, VA Vienna Elementary School – Vienna, VA Assistive Technology Taskforce T/TAC Staff

38

Thank You for ListeningThank You for Listening

Are there any questions?