Design Treatment Presentation Fall 2005 T/TAC Online Project VA Assessments December 15, 2005

1

Design Treatment PresentationDesign Treatment PresentationFall 2005 T/TAC Online Project Fall 2005 T/TAC Online Project VA AssessmentsVA Assessments

December 15, 2005December 15, 2005

Matt HumphreyAala’a MashaalPamela Sharpe

Tim SmithMaria Washington

Graduate School of Education George Mason University

2

Our Faculty and FacilitatorsOur Faculty and Facilitators

Dr. Nada Dabbagh Dr. Kevin Clark Dr. Mike Behrmann Gary Dickelman Jocelyn DelSignore Anya Evmenova

3

Clients & Information SourcesClients & Information Sources

Clients Information Sources•Dr. Patricia Abrams •Dr. Michael Behrmann •John Eisenberg •Nathan M. Sparks

•Mary Wilds

Human: •All clients•Jocelyn DelSignore •Anya Evmenova •Teachers

Content:•T/TAC & VA DOE websites•Assessment Manuals

4

Presentation OverviewPresentation OverviewPart One

Part Two

Part Three

Part Four

Part Five

• Immersion Team Process• Instructional Design Process

• Background & Introduction• Mission and Goal of Project

• Analyses• Needs Analysis• Performance Analysis• Learners Analysis• Task Analysis• Findings and Results

• Design Approach• Development

• Next Steps• Questions and Answers

5

Immersion Team ProcessImmersion Team Process

The Learning Process Teamwork Portfolios and Reflections

Challenges Benefits

6

The Instructional Design ProcessThe Instructional Design Process

Who Learner

What Learner needs Instructional Strategies

How Evaluation

7

Instructional Design ProcessInstructional Design ProcessIntegrated Learning Design FrameworkIntegrated Learning Design Framework

Enactment

Evaluation

Exploration

Online Learning Developer

8

Instructional Design ProcessInstructional Design ProcessThe ADDIE ModelThe ADDIE Model

DesignAnalysis Develop Implement Evaluate

9

Project BackgroundProject Background

No Child Left Behind Act (NCLB) Individuals with Disabilities Education

Improvement Act of 2004 (IDEA) Federal Mandates and VA Assessments

Standard of Learning (SOL) Assessment Alternative SOL (VGLA and VSEP) Alternate SOL (VAAP)

10

Training/Technical Assistance Centers Training/Technical Assistance Centers (T/TAC)(T/TAC)

T/TAC Online is a web-based community linking people and resources to help children and youth with disabilities

http://ttaconline.org

11

Project Problem StatementProject Problem StatementVA AssessmentsVA Assessments

Transition to Cognitive Standards Based Assessments

Communication Skills State Mandated Assessments New Process for Teachers

12

AnalysisAnalysisAnalysisAnalysis

Design Develop Implement Evaluate

Analysis

Needs Analysis

Performance Analysis

Learner Analysis

Task Analysis

13

Needs AnalysisNeeds AnalysisOverviewOverview

The process of determining, analyzing, and prioritizing needs and, in turn, identifying and implementing solution strategies to resolve high-priority needs.

14

Needs AnalysisNeeds AnalysisPurposePurpose

Determine the best solution for T/TAC Online Actuals and Optimals

Generate data to support in making a recommendation

Provide specific recommendations Decide priorities for action

15

Needs AnalysisNeeds AnalysisSteps TakenSteps Taken

Determine Current Status Performance Analysis

Analysis of Results Teacher Expectations Identify Discrepancies Determine the Goals Set Priorities for Action

16

Performance AnalysisPerformance AnalysisOverviewOverview

Used to determine what needs to be done in order to accomplish the client’s stated goals and objectives

Focuses on determining the elements that support the project

Investigates the current situation (actuals) and desired performance (optimals)

17

Performance AnalysisPerformance AnalysisResultsResults

Initial Actuals Teachers were not using standardized

procedures for assessments Teachers did not have sufficient

access to resources on the VA DOE website

Teachers were not targeting the Aligned Standards of Learning

18

Performance AnalysisPerformance AnalysisResults Results

Initial Optimals Teachers will use standardized

procedures for assessment Teachers will have easy access to

resources Teachers will target the ASOLs for

assessment

19

Learner AnalysisLearner Analysis

A learner analysis is performed to determine who the users will be, what skills and knowledge are needed by the learners, the environment in which they work, and their motivations and goals for performing.

20

Learner Analysis Learner Analysis Why Create Personas?Why Create Personas?

Personas identify the user motivations, expectations and goals responsible for driving behavior.

Although personas are fictitious, they are based on knowledge of real users. Some form of user research is conducted before they are written to ensure they represent end users rather than the opinion of the person writing the personas.

There are many varying personas that represent our users.

21

Learner AnalysisLearner Analysis How We Developed Our PersonasHow We Developed Our Personas

Using the data collected from the surveys and interviews with SMEs, teachers, and administrators we found that our users were:

Novice Special Education Teacher Intermediate Special Education

Teacher Expert Special Education Teacher Reluctant Special Education Teacher T/TAC Online Administrator Parent/Guardian

22

Needs AnalysisNeeds AnalysisData CollectionData Collection

Reviewed the VAAP Manual Attend conferences to observe teacher’s reactions Administer surveys Visit schools to interact with special education

teachers Interview:

Dr. Behrmann Mary Wilds John Eisenberg Jocelyn DelSignore Anya Evmenova

23

Needs AnalysisNeeds AnalysisSurveySurvey

Gathered information in the following topic areas: Demographics Technology skills Attitudes toward the VAAP Attitudes toward new assessments

24

Needs AnalysisNeeds AnalysisFindings – New VAAPFindings – New VAAP

New VAAP

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5

Agree w /assessment

Method of Assessment

Time Assessing

Tools for access

Lesson Plans to VAAP

Agree w /assessment Method of Assessment Time Assessing Tools for access Lesson Plans to VAAP

Figure 1

•85.6% of the teachers indicated that they agree with the assessments.

•67% of teachers surveyed indicated that they spent a lot of time assessing the students.

•61% of teachers surveyed agreed that they have five or more tools for assessments

Surveys conducted 10/25/05, Fairfax County and 11/3/05, Fredericksburg, VA 89 SPED teachers

25

Needs AnalysisNeeds AnalysisFindings - AssessmentFindings - Assessment

0.0

1.0

2.0

3.0

4.0

5.0

Hindrances PrepareLessons

Interacting-Transition

ShareLessonPlans

Help Aligning Know ledgeof T/TAC

Use T/TAC

Assessments

Hindrances Prepare Lessons Interacting-Transition Share Lesson Plans Help Aligning Know ledge of T/TAC Use T/TAC

Figure 2


74% 72% 77% 75% 79% 62% 50%

26

Needs AnalysisNeeds AnalysisFindings - Age RangesFindings - Age Ranges

16 16.5 17 17.5 18 18.5 19

Number of Teachers

20-30

30-40

40-50

50+

Age

20-30 30-40 40-50 50+


27

Needs AnalysisNeeds AnalysisFindings - Years of Teaching ExperienceFindings - Years of Teaching Experience

Years Teaching

21 +38%

6 - 1012%

1 - 54%

16 - 2027%

11 - 1519%

1 - 5 6 - 10 11 - 15 16 - 20 21 +


28

Needs AnalysisNeeds AnalysisFindings - LicensureFindings - Licensure

License

Provisional15%

Professional85%

Professional

Provisional


29

Needs AnalysisNeeds AnalysisCurrent Situation/ActualsCurrent Situation/Actuals

Special Education teachers have not been doing assessments for their students in a standardized format

Special Education teachers’ collaboration outside the classroom is limited

The assessment documents are difficult to locate on the Virginia Department of Education website

The challenge of assessing special education students creates a barrier

Special education teachers of students with severe disabilities have not been required to conduct SOL based assessments

Teachers spend an inordinate amount of time developing portfolios which are not aligned with SOL expectations

30

Needs AnalysisNeeds AnalysisDesired Performance/OptimalsDesired Performance/Optimals

For students designated to be assessed under the VAAP, teachers must begin aligning lesson plans to new aligned standards of learning (ASOL) guidelines, which include cognitive skills.

Skills will be measured in the four subject areas: Reading, Math, History & Social Science and Science.

Teachers must develop lesson plans that align to SOL guidelines and will receive a rating of three 3 or above on a scale of 1-4.

Differentiate the SOL enhanced lesson plans that are currently available on the SOL Enhanced area of T/TAC Online for their students assessed under VAAP guidelines.

31


It is important to note that if a student does not perform well in the four subject areas they may still garner a passing score by performing well in the following communication areas: Student performance Context Settings Socialization Support

32

Needs AnalysisNeeds AnalysisDiscrepancies in Teacher PerformanceDiscrepancies in Teacher Performance

Teachers do not have experience in aligning their lesson plans to the new standards.

Some of the teachers do not have experience assessing using the former VAAP standards.

The greatest discrepancy lies in the issue that the cognitive skill assessment process is new to all teachers in Virginia.

Teachers have limited resources to help them with the assessment process.

Teachers have limited time outside the classroom to dedicate to this process.

33


Use computer: Download blank forms Internet access Be able to use word-processing software

Be familiar with state and district-wide assessment Have knowledge of learner’s skill level, needs, and

characteristics Have knowledge/access to proven research based

instruction techniques Be able to develop lesson plans that fit the student’s

current stage of learning for a particular skill.

34


The solution system should: Assist teachers with assessing their students using the

different assessment procedures outlined in the VAAP, VGLA, and VSEP manuals

Allow teachers to access the documents in one location or a “one stop shop”

Facilitate the sharing of lesson plans with other teachers also involved in the process of creating lesson plans

Facilitate the submission of standardized assessment portfolios with streamlined evidences thereby reducing the amount of time teachers spend compiling evidence of student performance

The documents should be searchable for easy access and downloading

35

Needs AnalysisNeeds AnalysisPriorities for ActionPriorities for Action

Analyze lesson plan process Determine potential solution based on

the needs Create prototype of the solution

system

36

Task AnalysisTask AnalysisOverviewOverview

The process of analyzing and articulating the kind of learning that you expect the learners to know how to perform

A blueprint of the instructional sequence and it assures the instructional designer that there are no gaps in the instruction

37

Task AnalysisTask AnalysisLesson Plan ProcessLesson Plan Process

Instruction is focused on the process of creating lesson plans Support the teachers in aligning them to

the new standards Task analyze the lesson plan process

Understand the process used by a special education teacher

Support those processes in the solution system

38

Task AnalysisTask AnalysisInformation Processing Analysis (IPA)Information Processing Analysis (IPA)

Used most often for procedural and cognitive tasks First step in decomposing or breaking down a goal into its constituent parts Identifies what instruction is needed to attain the goal

39

Task AnalysisTask AnalysisIPA/Lesson Plan ProcessIPA/Lesson Plan Process

Creating a lesson plan is a procedural task Break down each step of the lesson plan

creation process into its constituent parts Identify what we need to provide to the

teachers to support them throughout the process Supports will be utilized in the solution system

40

Task AnalysisTask AnalysisWhat the Teacher Needs To DoWhat the Teacher Needs To Do

Complete prerequisite tasks1. Review present level of performance,

assessment data, and IEP team recommendations

2. Review student profile document3. Identify communication skills4. Identify the student’s entry points5. Plot entry level points and

communication skills

41


Complete the lesson plan process1. Identify a main theme or event of the lesson plan2. Record the date of creation3. Choose an ASOL subject matter

Organizing topic, standard used, support in activity context

Problem-solving cognitive task4. Choose a communication skill that can be

incorporated into the lesson Can be used multiple times throughout the

lesson Problem-solving cognitive task

42


5. Identify the student tasks and activities Main tasks, sub tasks, prerequisites, instructional

strategies Rule-using cognitive task

6. Identify the materials needed Rule-using cognitive task

7. Identify the assistive technology needs Ensure that all students have appropriate assistive

technology throughout the lesson Rule-using cognitive task

8. Identify the evidence collections materials Problem-solving cognitive task

43

Task AnalysisTask AnalysisChallengesChallenges

Is this the only lesson plan process? Identifying types of cognitive tasks Identifying appropriate supports for

each task Identifying instructional strategies What does the final result of each

task look like?

44

Task AnalysisTask AnalysisLesson Plan DocumentLesson Plan Document

45

Solution System Solution System

Based upon the results of the needs and task analysis phases, our goals for the solution system are to:

1. Provide scaffolding for teachers to develop lesson plans aligned with the new SOLs.

2. Produce a searchable VAAP manual.3. Provide additional resources for

teachers to utilize throughout the assessment process.

46

Analysis Analysis Design DesignAnalysis Analysis Design Design

Performance Analysis

Needs Analysis

Personas

Task Analysis

Develop Implement Evaluate

Design

Analysis

Design Approach

Storyboards

47

Design ApproachDesign ApproachEPSS CharacteristicsEPSS Characteristics

Electronic Performance Support System (EPSS)

Computer software program that improves performance by: Reducing complexity Providing the performance information to

perform a task Providing a decision support system that

enables the user to identify appropriate actions

48

Design ApproachDesign ApproachEPSS CharacteristicsEPSS Characteristics

Used on the job Controlled by the user Reduce the need for prior training in

order to accomplish the task Supports new constraints on an

existing process

49

Design ApproachDesign ApproachRationaleRationale

Supporting an existing process Teachers have created lesson plans New constraints on that process

New standard format for lesson plans is a system requirement

Teacher experience levels vary Just-in-time support in the classroom

50

Design Design Development Development

DesignAnalysis Implement Evaluate

FlowchartsSchemasPrototype

Develop

51

Development of PrototypeDevelopment of Prototype

Users will be able to use the VA Assessments section on T/TAC Online to: Search the VAAP Manual Create Lesson Plans Search for Created Lesson Plans

52

Searchable Online VAAP ManualSearchable Online VAAP Manual

Users should be able to search the VAAP Manual by: Subject Subject and ASOL code Subject and Skill Group Instructional Strategy

53

Flowchart 1:Flowchart 1: Searching the VAAP Searching the VAAP ManualManual

55

Creating a Lesson PlanCreating a Lesson Plan

Users will use the system to create lesson plans using a wizard with built in just-in-time supports

Users can also receive assistance if they need help with a particular part of the lesson plan creation process

56

Flowchart 2: Lesson Plan DocumentFlowchart 2: Lesson Plan Document

57

Lesson Plan Database SchemaLesson Plan Database Schema

58

Searching for Created Lesson PlansSearching for Created Lesson Plans

Users will be able to search for previously created lesson plans by: Subject Subject & Skill Group Subject & ASOL Code

59

Design ApproachDesign ApproachPrototypePrototype

http://immersion.gmu.edu/ttac/fall2005/epss

60

Next StepsNext Steps Identify supports to be integrated into the

EPSS Interview expert teachers Surveys Resources Collaboration

Continue Development of Prototype Implement Prototype Usability Testing Formative Evaluation

61

Thank You for ListeningThank You for Listening

Are there any questions?

Design Treatment Presentation Fall 2005 T/TAC Online Project VA Assessments December 15, 2005

Documents

Transcript of Design Treatment Presentation Fall 2005 T/TAC Online Project VA Assessments December 15, 2005