MEASUREMENT AND MODELING OF HIGHER ORDER SKILLS IN STEM TALENT DEVELOPMENT USING THE CWRAChristopher Kolar

NCSSSMST/NAGC Annual Convention 2013, Indianapolis

The challenge

Exceptional students

Exceptional claims of exceptional results

Exceptional Evidence

Standards of Significant Learning (SSLs)

Assessment at IMSA

Competence In Inquiry

Making accurate observations.

Evaluating evidence.

Challenging assumptions.

Making connections. Making sound

judgments. Communicating

powerfully.

CWRA: Performance task and document

In addition to an overall Performance Task score, subscores include:

Analytic Reasoning

Writing Effectiveness

Writing Mechanics

Problem Solving

MEASURING UP

Admissions data

Admissions Benchmarking

Prior GPA SAT-M SAT-CR SAT-We SAT-Wmc

Admissions Benchmarking

CWRA performance task score significantly correlated with all admissions metrics.

Prior GPA SAT-M SAT-CR SAT-We SAT-Wmc

MEASURING UP

Initial academic performance

First year grades in writing intensive courses: Literary Explorations, American Studies, and Scientific Inquiries (Biology)

Prior GPA SAT-CR SAT-We SAT-Wmc CWRA+

First Year Grade Point Average

First year grades in writing intensive courses: Literary Explorations, American Studies, and Scientific Inquiries (Biology): 23% of variance explained.

Prior GPA SAT-CR SAT-We SAT-Wmc CWRA+

First Year Grade Point Average

MEASURING UP

English core assessment

No reliability as outcome measures

Thesis Paragraphs Quote Usage

Analytic Reasoning

Writing Effectiveness

Writing Mechanics Problem Solving

IMSA CWRA+

THEREFORE

What this means in practice

Student Support

CWRA+ individual results as a diagnostic Further grade and retention modeling using this

data

Assessment Development

Performance task academy School improvement plan Revision of core assessment

Value added

“Small” effect size for time by junior year. Senior exit testing next.