Measuring Principal Performance: How Rigorous Are Publicly Available Assessment Instruments?
13
Copyright © 2010 Learning Point Associates. All rights reserved. Measuring Principal Measuring Principal Performance: Performance: How Rigorous Are Publicly How Rigorous Are Publicly Available Assessment Available Assessment Instruments? Instruments? Matthew Clifford Learning Point Associates [email protected] Presentation for PEAC July 2010
-
Upload
rosalyn-lynch -
Category
Documents
-
view
36 -
download
0
description
Measuring Principal Performance: How Rigorous Are Publicly Available Assessment Instruments?. Matthew Clifford Learning Point Associates [email protected] Presentation for PEAC July 2010. Learning Point Associates Overview. www.learningpt.org. - PowerPoint PPT Presentation
Transcript of Measuring Principal Performance: How Rigorous Are Publicly Available Assessment Instruments?
Copyright © 2010 Learning Point Associates. All rights reserved.
Measuring Principal Performance: Measuring Principal Performance: How Rigorous Are Publicly Available How Rigorous Are Publicly Available
Assessment Instruments?Assessment Instruments?
Matthew CliffordLearning Point Associates
Presentation for PEACJuly 2010
www.learningpt.org
Learning Point Associates Overview
www.learningpt.org/expertise/educatorquality/METworks.php
www.learningpt.org
www.learningpt.org
What is Your Perspective onPrincipal Performance Assessment?
• What instruments are you or others currently using?
• How systematically are the instruments being administered?
• How are results being used?
• How confident are you in the assessment instruments and process?
www.learningpt.org
What Research Tells Us about Principal Performance Assessment
• School districts’ principal performance assessments appear to be inconsistently administered and measured (Goldring et al, 2009; Thomas et al, 2000; Murphy et al, 2007).
• Principal performance assessments are not always aligned with existing state or national professional standards for practice, and lack justification or documentation of psychometric research (Heck & Marcoulides, 1996).
www.learningpt.org
Learning Point Associates: Managing Educator Talent
www.learningpt.org/expertise/educatorquality/METworks.php
Performance Management
CoherentSystematic
www.learningpt.org
Principal Performance Assessment: Recommended Attributes
• Transparent
• Trusted
• Consequential
• Systematic and consistent
• Involving multiple measures
• Validate and reliable
(Brown-Sims, 2010)
www.learningpt.org
Our Study: Question & Method
Question: How valid and reliable are publicly available principal performance assessment instruments?
www.learningpt.org
Our Study: Review Criteria and Results
Criteria for inclusion:• Claim of use as a principal performance assessment• Recent (within past 15 years or so)• Publicly available technical information
Results of scan: • 2000 articles identified by keyword search & snowball sampling• Approx. 400 examined• 20 articles reviewed on 8 assessment instruments
www.learningpt.org
Our Study: Instruments Reviewed
Instrument Approach Validity Reliability
Change Facilitator Style Questionnaire(1988)
77-item survey focusing on 6 domains
360-degree of principal as change facilitator
Content: Lit review
Construct: Factor analysis
Alpha Range: .64 to .95
Diagnostic Assessment of School and Principal Effectiveness(1992)
213-item survey
360-degree
Content: Expert review
Construct: Factor analysis and inter-item correlation
Alpha Range:.8 to .97
Instructional activity questionnaire(1987)
34-item assessment
Focus on instructional leadership
Content: Lit review
Construct: Factor analysis
Alpha Range: .7 to .9
www.learningpt.org
Our Study: Instruments Reviewed
Instrument Approach Validity Reliability
Instructional activity questionnaire(1987)
34-item assessment
Focus on instructional leadership
Content: Lit review
Construct: Factor analysis
Alpha Range: .7 to .9
Leadership Practices Inventory (2002)
30-items
Principal and supervisor completes
Leadership effectiveness
Content: Interviews and surveys of leaders
Construct: Factor analysis
Concurrent: Other management measures
Test/re-test: .79
Performance Review Analysis and Improvement System for Education (1985)
81-item with 9 sub-scales
Two-dimensional profile with +/-
Content: Lit review with Delphi panel
Construct: Unknown
Alpha Range: .88 to .98
Test/re-test: .59 to .8
www.learningpt.org
Our Study: Instruments Reviewed
Instrument Approach Validity Reliability
Principal Instructional Management Rating Scale(1985)
71-item questionnaire with 11 subscales
Content: Lit review and Delphi panel
Construct: Correlation within subscale and school document review
Alpha: .75
Principal Profile(1986, 1987)
Interview-based protocol characterizes principal effectiveness and leadership style
Content: Lit review
Construct: Factor analysis
Inter-rater agreement range from .5 to 1
VAL-ED(2006)
72-items
360-degree, produces profile
Content: Lit review
Construct: Factor analysis
Alpha= .98 for 12 scales on different forms
www.learningpt.org
Our Study: What did we learn?
• If there are many principal performance assessment being used in the field, then evidence of their reliability and validity is not publicly available.
• Of those reviewed, 5 of 8 were developed in tested in the mid-to-late1980s.
• Survey-based assessment gather self, and others (3), perceptions of principal performance. One used a trained rater.
• No information on consequential validity, and two examples of concurrent validity.
www.learningpt.org13
Matt Clifford
E-Mail:
Phone: 630-689-8017
1120 East Diehl Road, Suite 200Naperville, IL 60563-1486General Information: 800-356-2735
