Mixing Authentic and Large-Scale Visual Arts Assessment: What might

work for you?

Thomas BrewerUniversity of Central Florida

NAEP Research Team Members Read Diket, William Carey University

Lihua Xu, University of Central FloridaAnne Grey, University of Central Florida

2012 FAEA ConferenceFriday, November 9St. Petersburg, FL

Proposal DescriptionThis presentation will address the issue of developing authentic visual arts assessment that might be used in larger scale school, district, and state-wide applications. As a National Assessment of Educational Progress (NAEP) Visual Arts Consortium member, the presenter is charged with informing current art educators and administrator about the various curriculum and assessment resources available through the NAEP Arts website and the NAEP Data Explorer (NDE). Whether you are interested in developing assessments for your classroom, school, district, or state, this presentation will provide some foundational entry points.

What Might Work for You?My Role with TeamRater procedures for the NAEP assessment and implicationfor results and student learning.

Possible teacher assessment procedures and how teacherscould use NAEP testing blocks to assess their students’ artlearning. (Assessment, curriculum, policy implications)

Provide discernible and useful arts education policyinformation and interpretations to aid in the

improvementof arts education.

What Might Work for You?Possible teacher assessment procedures and how teachers could use NAEP

Testingblocks to assess their students’ art learning. (Assessment, curriculum, policyimplications.)

What is NAEP Arts Report Card?http://nces.ed.gov/nationsreportcard/ > Sample Questions,

Analyze Data, Data Explorer or Sample Questions

Examples of authentic and large-scale assessment test blocksand instruments:Siegesmund and Diket (2000)Brewer (2011), Developed Bundled AssessmentBrewer and Grey with Mother/Child exemplars NAEF 2010

What Might Work for You? Relevant Findings IPossible teacher assessment procedures and how teachers could use NAEP

Testingblocks to assess their students’ art learning. (Assessment, curriculum, policyimplications.)

UCF Graduate Research classes use NAEP Data Explorer(NDE) to more closely look at 192 variables.

One very important finding was discovered by Kathy Arndt in2011 were students scores on a 300 point scale weresignificantly higher only for those who received artinstruction from full time or part time art specialist andnot from: Artists-in-residence (this might also be applied toteaching artists); Classroom teachers (though we love them);Volunteers (though we love them too), or other faculty (PE).

What Might Work for You? Relevant Findings IPossible teacher assessment procedures and how teachers could use NAEP

Testingblocks to assess their students’ art learning. (Assessment, curriculum, policyimplications.)

Specialist in this study only meant being certified but nospecifics were given about what path the art educators took tocertification.

None the less Arndt’s (2012) finding have enormous Teacherpreparation and education policy implications and isextremely important to art education.

What are some possible meanings and implications?

What Might Work for You? Relevant Findings IIProvide discernible and useful arts education policy

information and interpretations to aid in the improvement of arts education.

One very important arts education policy position comes from

our current Mother/Child Block analysis where we may beunearthing existing or recognizing new models of art learningand discovering new paths to teaching methodologies giventhe relationships between students’ art knowledge, technicalknowledge, aesthetic properties, and meaning. Dr. Xu

What Might Work for You? Relevant Findings II

These conclusions seem to have significant implication for curriculum designers, principals, decision makers in curriculum design in the field of visual arts education.

At the same time, it provides statistical evidence for the significance of proper question layout to maximize test takers’ performance in arts assessment. Assessment experts and visual arts educators are advised considering presenting the visual arts assessment questions in the order of arts knowledge, technical knowledge and skills, and aesthetic properties, and interpretation of art works.

8SEM analysis

What Might Work for You? Relevant Findings III The 2008 Mother/Child block remained essentially the same as 1997. However, the 2008 sample of eighth grade students included a disproportionate number of Hispanic students and a likely increase in high poverty schools. Catholic Schools were over represented in the private school designation.

Despite these factors and a small loss of art specialist expertise in the classrooms (65% in 2008; in 1997, 73%), and with coding changes, the basic performance did not decline between the 1997 and 2008 samples.

Implications/ApplicationsSo what does any of this have to do with you?

What sort of assessment practices are you aware of and are using?

Are you being ask to assess student learning in art? If so what resources do you use?

Do you need some assessment items?

10SEM analysis

Some foundations for the 2008 NAEP Secondary Analysis

Diket, R. M., Xu, L., and Brewer, T. (2012). Discussing Educational Policy and Implications of Efficacy in the Arts as School Subject Areas: NAEP>Teacher-Made Assessment,> Subject Area Growth. (AERA Symposium Presentation).

Diket, R., Xu, L., & Brewer, T. (2012). Toward an Aspirational Learning Model gleaned from large-scale assessment. (Paper under editorial review Studies in Art Education).Diket, R. M., & Brewer, T. (2011). NAEP and policy: Chasing the tail of

the assessment tiger. Arts Education Policy Review, 112(3), 35-47.Brewer, T. (2011). Lessons learned from the Bundled Visual Arts

Assessment. Visual Arts Research, 37(1), 79-95.Brewer, T. (2008). Developing a Bundled Visual Arts Assessment

Model. Visual Arts Research, 34(1), 63-74.

Thank you for attending

Mother and Child Portraits for NAEP 1997/08 - AG

William Sergeant Kendall. Beatrice. Courtesy of Pennsylvania Academy of the Fine Arts, Philadelphia.Philadelphia.

Nigerian. Figure of Woman and Child. Courtesy of National Museum of African Art, Washington, D.C.

Mother and Child Portraits for NAEP 1997/08 - AG

Raphael. The Small Cowper Madonna. Courtesy ofNational Gallery of Art, Washington, D.C.

Courtesy of Joan Miro. Series I. The Family.

Byzantine. Madonna and Child on a Curved Throne. Courtesy of National Gallery of Art, Washington, D.C.

Mother/Child ConstructsPath Diagram

17SEM analysis

Hypothesized modelTwo structural models were conducted with the hierarchical

representation from art knowledge to technical to aesthetic properties to meaning and with or without the path from art knowledge to meaning.

The hypothesized model is displayed in the path diagram. – represents constructs (latent variables)

represents measured items/questions (variables)– Line connecting variables implies a hypothesize direct effect– Absence of a line connecting variables implies lack of a

hypothesized direct effect.

18SEM analysis

Diket, R., Xu, L., & Brewer, T. (2012). A (Aspirational) Theory of How Students Progress to Meaning in Art. (Paper under editorial review NUMBERS, Online?).

Informing Assessment Practices of Art Education Teachers: How Test Blocks from the National Assessment of

Educational Progress Arts Report Cards (1997 and 2008) Can Serve as General Models When Designing Classroom

Assessment Using Contemporary Art

2010 NAEF Research GrantRead Diket, William Carey University

Thomas Brewer and Anne Grey, University of Central Florida