Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference...
-
Upload
thomasine-chapman -
Category
Documents
-
view
214 -
download
2
Transcript of Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference...
![Page 1: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/1.jpg)
Including Quality Assurance Within The Theory of Action
Presented to:CCSSO 2012 National Conference on Student Assessment
June 27, 2012
![Page 2: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/2.jpg)
• Quality Assurance - A continuous cycle that improves and refines the processes necessary to produce valid assessments to the highest standards. Ensures that excellence is inherent in all components of the process. – Plan– Do– Study– Act
Quality Assurance as a Continuous Cycle
![Page 3: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/3.jpg)
• Evaluated assessment systems in most states and the U.S. territories through the federal standards and assessment peer review process.
• Developed interpretative arguments for the validity of assessments and conducted studies to test the claims and assumptions in the interpretive argument for a number of states.
• Provided technical assistance for reviewing the technical quality of assessment and data quality of assessment systems.
Edcount’s relevant work:
![Page 4: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/4.jpg)
• Students are appropriately tested for identified testing options
• Test scores accurately reflect student knowledge and skills
• Reports provide valid scores and are used appropriately
Quality Assurance Within the Theory of Action - Claims
![Page 5: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/5.jpg)
• The Learner Characteristics Inventory (LCI)– A survey designed to collect information about
the population of students who participate in alternate assessments based on alternate achievement standards (AA-AAS).
– Helps states identify and evaluate “who” is participating in the alternate assessments. Provides information to help direct participation rules and guidance.
Students Are Appropriately Tested for Identified Testing Options – Special Studies
![Page 6: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/6.jpg)
• Accommodation Studies– The extent to which accommodations are
provided on the state assessments.– Literature review on best practices for
accommodation use. – Provides information and evidence to improve
best practices, guidance, and monitoring of accommodation use.
Students Are Appropriately Tested for Identified Testing Options – Special Studies
![Page 7: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/7.jpg)
• Performance Level Descriptor (PLD) studies– Examine the extent to which assessment cut scores
appropriately distinguish between student performance levels, and whether the performance level descriptors adequately capture meaningful distinctions in grade-level performance.
– Engage teachers in rating and feedback processes to help administrators understand how teachers use and interpret PLD’s
– Evaluate the PLD’s for coherence, consistency, and accuracy.
Test Scores Accurately Reflect Student Knowledge and Skills – Special Studies
![Page 8: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/8.jpg)
• Performance Level Descriptor (PLD) studies cont.– Provides information to help states refine their
standard setting processes, focus their professional development and support their interpretations on the state assessments.
Test Scores Accurately Reflect Student Knowledge and Skills – Special Studies
![Page 9: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/9.jpg)
• Alignment Studies– Evaluations of the alignment between academic
content standards and alternate assessments.– Evaluations of the alignment between academic
content standards and general assessments.– Provides information to support and improve the
design and development of assessments.
Test Scores Accurately Reflect Student Knowledge and Skills – Specialized Studies
![Page 10: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/10.jpg)
• Cognitive Labs– Collect information about whether assessments
are eliciting the intended cognitive processes. – The cognitive labs can help states understand why
students may struggle with certain items or content areas on an assessment, and, as a result, provide feedback about ways to develop accessible assessments and ways to improve instruction.
Test Scores Accurately Reflect Student Knowledge and Skills – Special Studies
![Page 11: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/11.jpg)
• Consequential Validity Studies– Evaluate and collect evidence about the intended
and unintended consequences of a states large scale assessment system.
– Provides evidence to inform policies and processes and focus professional development.
Reports Provide Valid Scores and Are Used Appropriately - Special Studies
![Page 12: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/12.jpg)
• Special Studies– PLD study– Accommodation Study– Consequential Validity Evaluation– Cognitive labs
• Professional Development– Data Use Workshop– Curriculum Mapping Boot Camps
Work Within Puerto Rico
![Page 13: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/13.jpg)
• Special Studies– PLD studies– Cognitive Process Validity Study
• Technical Assistance For:– Development of a balanced assessment system
with formative and interim assessments– Development of an accountability System – Small
school review– Development of a data management system
Work Within the Laurent Clerc Center at Gallaudet University
![Page 14: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/14.jpg)
• Provided focus for professional development• Provided information to improve the design
and development activities for assessments and reports
• Provided context as well as empirical evidence to support policies and processes
How Did the Work Inform Quality Assurance and Processes
![Page 15: Including Quality Assurance Within The Theory of Action Presented to: CCSSO 2012 National Conference on Student Assessment June 27, 2012.](https://reader036.fdocuments.us/reader036/viewer/2022081603/56649ea15503460f94ba4247/html5/thumbnails/15.jpg)
Contact Information• Jennifer Stegman, Senior Associate• [email protected]
• edCount LLC• 5335 Wisconsin Avenue NW• Washington DC 20015• edCount.com