Data Analysis - Tools and Processes · 2013. 10. 24. · Data Analysis - Tools and Processes...

Data Analysis - Tools and Processes (School Level)

Food for Thought

How does your school use data to inform instruction and

improve student achievement?

Tuesday, February 21, 2012

Hawaii Department of Education Office of Curriculum, Instruction and Student Support

Webiquette

1. Before the meeting starts, close

any other applications running

on your computer.

2. Use the “Hand” icon

if you wish to speak

or have a question.

3. Mute your microphone

to eliminate ambient

noises.

4. Make sure that your

“Chat” box is set for

“Everyone.”

5. Use the “Chat” box

for questions.

6. Restrict the use of text-speak,

please respond using standard

English to text.

7. Your collaboration

is vital. Every

perspective

contributes to the

whole picture.

Muted

Unmuted

2

Group Norms for Webinar

Self-directed Learner

› Make personal connections to your position

Community Contributor

› Honor the expertise of ALL

Complex Thinker

› Synergize – Collective thoughts

Quality Producer

› Grow professionally

Effective Communicator

› Seek first to understand, then to be understood

Effective & Ethical User of

Technology

› Remove all other distractions

3

Hawaii’s Five RTTT Pillars

Improved Student

Outcomes

Data for School Improvement

Longitudinal Data System

Balanced Scorecard

Data Governance

Using data to inform instruction

Common Core Standards

Career & College Ready Diploma

Curriculum Framework

Common Instructional Materials

Formative Assessments

Interim Assessments

Summative Assessments

STEM

Focused support on

lowest-performing schools

-

Zones of School

Innovation

•Flexibility

•Great teachers and great

leaders

•Remove barriers to

learning

Performance-based

evaluation system

New Teacher Induction &

Mentoring

Incentives

Leadership development

Alternative pathways

Systems of Support to enable schools to do their best work – reprioritize and reorganize State resources;

establish Human Resources Unit in Zones of School Innovation; automate

6

Essential Question

How does data analysis help in school

improvement efforts?

7

Desired Outcomes

A common understanding of the various

purposes for analyzing data

An understanding of how to analyze data

using a variety of tools and processes

8

Data Analysis (School Level)

Agenda

Reason We Analyze Data

Basic Information We Use to Analyze Data

Processes We Can Use to Analyze Data

Finding Root Causes

9


Why do we need to use data?

Why do we want to use data?

10

Why We Need to Use Data


=


•Formative Assessment / Instruction

•Data for School Improvement (DSI) as a formative assessment tool

•Using DSI Reports to inform instruction

•Deconstructing the Standards Process K-12

Why We Want to Use Data 1. Pick a number from 1-10.

2. Multiply that number by 9.

3. Add up the digits of the answer.

4. Subtract 5 from the number.

5. Find the letter that corresponds to the number (example: 1=A, 2=B, 3=C, etc.)

6. Think of a country whose name starts with that letter and write it down.

7. Take the 2nd letter in the country's name, and think of an animal whose name starts with that letter and write it down.

8. Write down the color of that animal.

9. Take the last letter of the country's name and write down an animal whose name starts with that letter.

10. Use the last letter of that animal and write down a fruit that starts with that letter.

18

Types of School Teams Principal

Counselor

Curriculum Coordinator

GL Reps/Dept Chairs

Kindergarten

First

Second

Teacher Teacher

Science

Math

Language Arts

Teacher Teacher

Leadership Team

Grade Level/

Department

Classroom

19

Take a Minute

How does your school use data to help

students and teachers succeed?

What types of data teams do you have at

your school?

20

State Goals

Vision of a Hawaii high school graduate is that

all public school graduates will:

Realize their individual goals and aspirations;

Possess the attitudes, knowledge and skills necessary to

contribute positively and compete in a global society;

Exercise the rights and responsibilities of citizenship; and

Pursue post-secondary education and/or careers without

need for remediation.

21

State Goals

22

Vision of a Hawaii high school graduate

There are six General Learner Outcomes (GLOs) that are the goals of standards-based learning in all content areas:

Self-Directed Learner: The ability to be responsible for one's own learning

Community Contributor: The understanding that it is essential for human beings to work together

Complex Thinker: The ability to be involved in complex thinking and problem solving

Quality Producer: The ability to recognize and produce quality performance and quality products

Effective Communicator: the ability to communicate effectively

Effective and Ethical User of Technology: the ability to use a variety of technology effectively and ethically.

Inverted Data Pyramid

Summative High Stakes Assessment

Other

23

N. Love, 2010

Data Pyramid: What kinds of data do

teams and coaches use?

Summative High Stakes

Assessments

Demographic, Process and

Perceptual Data

Benchmark Common Assessment

Formative Common Assessment

Formative Classroom Assessment

24

N. Love, 2010


Agenda




Finding Root Causes

25

Using Data at the School Level

Where are we now?

• Requires measures and data

• What measures do you have available to help to determine what may be the problem?

Where do we want to go?

Where are we going?

• Determine the goal – know the target

• What are the needs of the school or the students?

How do we get there?

• Identify a strategy or process

• What will you put in place in order to achieve the outcome?

26

Too much data?

Diverts attention away from the primary purpose:

improving instruction.

Leads to overload – creating long, “comprehensive”

plans that few read.

Reveals too many things to address – so too many

goals and initiatives are created.

Multiple

Measures Demographics

Perceptions

Student Learning

Processes

28 V. Bernhardt

Demographic Data

Clarifies who our “clients” are.

Builds on the context of the school

Helps to predict future conditions to best serve the needs of our future students.

“Demographic information is crucial in data analysis as it helps us understand the context within which schoolwide change is planned and takes place.”

(V. Bernhardt, 1998, p-25)

29

Examples of Demographic

Data Number of students in the school

Number of students with special needs

Ethnicities of the students in the school

Number of graduates

Number of disadvantaged students

30

Tools –Finding Data Demographic

(Type any other places that you get this type of data into the Chat box)

Longitudinal Data System (LDS) http://employees.hidoe.k12.hi.us



United States Census http://quickfacts.census.gov/qfd/states/15000.html



Hawaii Department of Business, Economic Development & Tourism http://hawaii.gov/dbedt/info/census/



School Documents Online http://iportal.k12.hi.us

Perceptual Data A view, judgment or appraisal formed in the mind about a

particular matter.

A belief stronger than impression and less strong than positive knowledge.

A judgment one holds as true.

“ In organizations, if we want to know what is possible . . .we need to know the perceptions of the people who make up the organization.”

V. Bernhardt, 1998,pg. 41

35

Examples of Perceptual Data

Observations

Person-to-person interviews

Telephone surveys

Focus groups

Parent surveys

36

Tools –Finding Data Perceptions


School Quality Survey (SQS) http://arch.k12.hi.us

Process Data

Programs can include a wide variety of offerings, from

specially funded programs to academic curricular

sequences to extracurricular programs.

38

Examples of Process Data (Type any other places that you get this type of data into the Chat box)

Grant data

Program data

Comprehensive Needs Assessment (continuous improvement process)

Curriculum mapping

Data Teams

39

Student Learning Data Most important type of data to focus on.

Annual Large-Scale Assessment Data

Periodic Assessment Data

Ongoing Classroom Assessment Data

40

Examples of Student Learning Data

Hawaii State Assessment (HSA)

Terra Nova

DIBELS/DIBELS Next

Reading Inventories

Classroom Assessment Data

41

LDS

Tools –Finding Data Student Learning


Accountability Resource Center Hawaii (ARCH) http://arch.k12.hi.us

Tools –Finding Data Covers Demographic, Perceptual, and Student Learning


School Status & Improvement Report (SSIR) http://arch.k12.hi.us

Disaggregating Data

44

Disaggregating Data

Typically, student achievement data are reported for

whole populations, or as aggregate data. It is not,

however, until the data are disaggregated that

patterns, trends and other important information are

uncovered.

** Disaggregated data simply means looking at test

scores by specific subgroups of students.

45

Disaggregating Uncovers . . .

Achievement gaps are differences in academic

achievement amongst different groups of students.

It is important to examine these differences in order to

find ways that we can address some of the inequities.

The disaggregated data and the dialogue that arises

can transform beliefs and practices.

46

Trend Data

Data that shows a pattern over time.

Time is the variable over which one constant is being compared.

The more years of data that you have, the more reliable are the trends and patterns.

Statistically three years of data just barely indicates a trend. Five years provides more confidence to your inferences.

N. Love, 2008

47

Several Ways to Disaggregate

Data Gender

Socio-Economic Status

Mobility

Special Education and Disability

ELL – English Language Learners

Grade level

Classroom or course

48

Take a Minute

What student data/information does your

school use to make decisions?

49


Agenda




Finding Root Causes

50


Where are we now?




Where are we going?






51

Problem Solving Processes

52

Root Cause Analysis - Process

Evaluate Programs

Improvement Planning

Determine Root Causes

Conduct Data Analysis

Define Problems

Organize Teams

53 P. Preuss, 2003

Structured Collaboration

BUILD

• Foundation

IDENTIFY

• Student Learning Problems

VERIFY

• Causes

GENERATE

• Solutions

RESULTS

54

N. Love, 2010

The Problem Solving Cycle

1. Identify the Problem

2. Describe Hunches and Hypotheses

3. Identify Question and

Data

4. Analyze Multiple

Measures

5. Analyze Political Realities

6. Develop Action Plan Resolution

7. Implement Action Plan

8. Evaluate Implementation

9. Improve the Process and

System

55

V. Bernhardt,

2011

Step 2: Analyze Data to

Prioritize Needs

Step 3: Establish SMART Goals

Step 4: Select

Specific Strategies

Step 5: Determine

Results Indicators

Step 6: Monitor and

Evaluate Results

Step 1: Conduct a Treasure

Hunt

Decision

Making for

Results –

Doug Reeves

56

Inquiry Cycle Annenberg Institute for School Reform

Establish Desired

Outcomes

Define the Questions

Collect and Organize the

Data

Make meaning of the data

Take Action

Assess and Evaluate actions

57


Agenda




Finding Root Causes

58


Where are we now?




Where are we going?






59

What are the Causes?

60

“Data helps us get to the root

causes of a problem so we solve

the problem and not just the

symptom.”

V. Bernhardt

61

What is Root Cause?

The deepest underlying cause,

or causes, of positive or negative

symptoms within any process

that, if dissolved, would result in

eliminating or substantial

reduction of the symptom.

62

P. Preuss, 2003

Levels of Root Cause

External Level

Systemic Level

Programmatic or Process Level

Incident/Procedural Level

• Families

• Communities

• Supporting agencies

• Leadership

• Values/Beliefs

• Instructional Process

• Time

• Staff Development

• Students

• Teachers

• Incidents

63

When is a Cause a Root Cause?

Would the problem

have occurred if the

cause had not been

present?

Will the problem

reoccur as the result of

the same cause if the

cause is corrected or

dissolved?

Will correction or

dissolution of the cause

lead to similar events?

If yes, then it is a contributing

cause.

If no, then it is a

root cause

64

Tools for Root Cause Analysis

The Five Whys (Roots Toyota Corporation)

The Questioning Data Process (P. Preuss)

Causes for Student Learning Problems (N. Love)

System Planning Process (P. Preuss)

The Diagnostic Tree Process (P. Preuss)

65

The Five Whys – Why is it happening? Problem: 10th grade is not making significant progress in reading

Why? • Students not progressing are not doing work

Why?

• Students not progressing miss multiple days a quarter

Why? • Students miss instruction because they are not in class

Why? • Students are not in class because they babysit siblings

Why?

• Elementary school and high school schedules are different

66

Questioning Data Process (P. Preuss)

Step 1: Ask “What do you see in this data

set?”

Step 2: Ask “What questions do you

have about what you see?”

67

Possible Causes for Student

Learning Problems (N. Love) • Did we teach it? In enough depth? Placed in the right sequence?

Frequently enough? Curriculum:

• Did we use a variety of research-based instructional approaches? Are we sharing successful practices? Did we reteach using a different approach to individuals or groups who didn’t yet get it?

Instruction:

• Do we use ongoing formative assessment to explore student thinking and built on it in our instruction? Communicate to students how to improve? Help them self-assess?

Assessment:

• Did we examine attitudes or practices that might contribute to achievement/relationship/teaching gaps? Equity:

• Did we identify students who need additional help and provide them with it?

Individual Assistance:

• What knowledge/skills would help us improve student achievement? Teacher

Development:

68

System Planning Process –

EXAMPLE (P. Preuss) • 18 of 20 IEPs for high school students with

disabilities lack post secondary goals. Where are we

now?

• Post secondary goal statements must reflect goals after leaving high school

Where are we going?

• Involve the student. career interest Inventory

How will we get there?

• Making time for students to explore options after leaving high school

What is holding us back?

69

Sample Diagnostic Tree (P. Preuss)

HSA Math Scores below standard

Math Achievement Score in Grade 3

Student Demographics

Curriculum Instruction


System Processes

Organizational Culture


70


Agenda




Finding Root Causes

71

How does data analysis help in

school improvement efforts?

My answer:

It allows us to see everything that may affect

student learning

It allows school level leaders identify ways to

support student learning

It complements classroom (instructional) level

data teams

It allows us to predict success!

72

Question & Answer

73

Thank you for joining us!

A recording of this webinar will be posted

on the Standards Toolkit website.

If there are any questions, please e-mail: Dewey Gottlieb, Mathematics Specialist

Monica Mann, Acting Administrator

Petra Schatz, Language Arts Specialist, or

Derrick Tsuruda, Science Specialist

74

Data Analysis - Tools and Processes · 2013. 10. 24. · Data Analysis - Tools and Processes...

Documents

Transcript of Data Analysis - Tools and Processes · 2013. 10. 24. · Data Analysis - Tools and Processes...