CRESST’s Evaluation of the Artful Learning Program: “Findings,” Contexts, and

Future Explorations

Noelle Griffin,Ph.DUCLA Graduate School of Education & Information

StudiesNational Center for Research on Evaluation,Standards, and Student Testing (CRESST)CRESST Conference, September 10, 2004

Artful Learning Program

Arts-based education model: Arts infused across the curriculum

Currently implemented in 7 states, 22 schools

Four phases: Experience, Inquire, Create, Reflect

Use of Masterworks of art

Where Does ArtfulLearning Fit?

Comprehensive school reform

Instructional, organizational, assessment components

Focused on helping kids improve academic performance

Depends upon trained teachers and administrators

Theory of Action for Artful Learning

The underlying theory motivating the Artful Learning model is that the joy, discipline, and commitment required by the arts will provide a useful framework for the overall improvement of education and for individual growth

Evaluation Background

CRESST/UCLA

Beginning in 2001

Both external and internal applications

Qualitative and quantitative strategies

Emphasis: Capacity building

Evaluation Methods

Yearly implementation survey (participating schools, n=@400 per year)

Pre-post professional development surveys (on-going)

Teacher/administrator interviews (n=@30 per year)

School-level achievement data

Supplemental information

Academic Achievement

Student Engagement/Motivation

QualityImplementation

Organization And

LeadershipClassroom Instruction

Connections Processes

Professional Development/Support

District/School Context

Contextual Factors

Need for both administrative and district support

Middle/high school vs. primary

Teacher/student turnover

Role of arts teachers/specialists

Based on qualitative data

Professional Development and Support

Overall quality/utility high (quantitative and qualitative)

Consistency across levels/experience

Differences across program components

Professional Development Quality: Percent “Agree” or “Strongly Agree”

0

20

40

60

80

100

Val

id P

erce

nt

Usefulknowledge or

skills

Links topractice

Well-connectedcontent

L1

L2

Significant Pre/Post Development Increase in Reported Teacher Expertise

(MANOVA)

Level I and Level II

Artful Learning Instructional Process

Assessment Practices

Organizational/structural Practices

General Instructional Quality

How Well Did Development Prepare You to Implement Classroom Component?

Level 1

Not at all

Slightly

Somewhat

Adequately

More than adequately

Level 2

Not at all

Slightly

Somewhat

Adequately

More than adequately

How Well Did Development Prepare You to Implement Assessment Component?

Level 1

Not at all

Slightly

Somew hat

Adequately

More than adequately

Level 2

Not at all

Slightly

Somewhat

Adequately

More than adequately

Implementation Survey Multiple Regression Results: Implementation of

Artful Learning Classroom Process

Artful Learning Process

Standards Use/Understanding

Assessment Use/Understanding

High Quality Instruction

Reported Impact

Shared Mission

*

*

*

*

*

*=Significantly Predicts p<.05

Multiple Regression: No Significant Relationship to Implementation

Artful learning experience

Teaching experience/Teacher background

Parent involvement

Shared leadership

Implementation: Other Findings

Teacher evaluation of program after use is high

Continued growth of program/use over time

Assessment/parent involvement weakest components

Based on Qualitative and Quantitative Data

Additional Findings: Shared Efficacy

Teacher perceptions of school’s overall effectiveness in teaching process

Student achievement links

Group competency and task analysis components

Included in implementation survey

Multiple Regression Findings: Shared Efficacy

Shared LeadershipShared Leadership

Years ExperienceYears Experience

Grade Level TaughtGrade Level Taught

Combined Shared Efficacy

Combined Shared Efficacy

Group CompetencyGroup Competency

Task AnalysisTask Analysis

*All Significant relationships are negative*No significant relationship to other Artful learning implementation variables

Student Outcomes

Engagement and other teacher-reported outcomes

Achievement

Quantitative and qualitative consistency

Access to information – what teachers “don’t know” re: assessment results

Teacher Reported Program Impact: Student Engagement

Level of Increase

0

20

40

60

80

100

Val

id P

erce

nt

No Impact

Slightly

Moderately/Greatly

Teacher Reported Program Impact: Quality of Student Work

Level of Increase

0

20

40

60

80

100

Val

id P

erce

nt

No Impact

Slightly

Moderately/Greatly

Teacher Reported Program Impact: Classroom Tests/Assessments

Level of Increase

0

20

40

60

80

100

Val

id P

erce

nt

No Impact

Slightly

Moderately/Greatly

School-Level Achievement Data: Limitations

Cohort group issues

Dilution of effects

Test changes/lack of comparability

Insufficient information statistically

Tentative Findings: Comparative Average Growth in Students

Meeting/Exceeding Standards

GRAMMY DISTRICT COMP. SCHOOLS

READING 16 10 9

ELA 9 8 5

MATH 12 9 6

CRESST Project Synergy: QSP use in Artful Learning

QSP: Quality School Portfolio assessment use software

Address program difficulties collecting achievement data in multiple districts/states

Build project self-evaluation capacity

Potential QSP Applications for Artful Learning:

Track student-level data

Compare participating vs. non-participating students

Compare students based on “dosage”

Compare teachers based on program experience

Enduring Questions Related to Arts Education

What should we value as attainment of serious learning?

Can arts programs in schools be sustained as political and parental pressure increasingly focuses on traditional academic attainments?

What should count as quality education?

How scalable is change away from a compartmentalized view of learning?

How can such change be sustained and expanded?