Adding Value to the MSP EvaluationsNorman Webb, Rob Meyer, and Paula White

Wisconsin Center for Education ResearchUniversity of Wisconsin-Madison

1025 W. Johnson St., Madison, WI 53706www.addingvalue.org

   Contact Information:Principal Investigator: Norman Webb, nlwebb@wisc.edu, 608-263-4287Senior Personnel:Rob Meyer, rhmeyer@aol.com, 608-265-5663Paula White, pawhite1@wisc.edu, 608-263-4353

Project Principles 

• Build on what has been learned about evaluating large-scale systemic reform

• Develop a learning community among the MSP evaluators

 

Project Activities• Provide technical assistance to MSP evaluators regarding

MSP evaluation challenges, the use of appropriate analytic tools, and techniques for studying the impacts of MSPs

• Two-day semiannual meetings each spring and fall to present, discuss, and analyze evaluation techniques

• Teleconference meetings to identify evaluation needs, make recommendations, and discuss alternatives for evaluation

• Site visits to interact with evaluators on specific evaluation needs and issues

• Develop a value-added and an alignment tool for conducting value-added analyses and alignment analysis of standards, assessments, and curriculum

Project Goals1. Increase the knowledge of MSP evaluators

about design, indicators, and conditions needed to successfully measure change in student learning over time.

2. Develop useful tools and designs for evaluators to attribute outcomes to MSP activities.

3. Apply techniques for analyzing the relationship between student achievement and MSP project activities to evaluate the success of MSP projects.

 

How does the Added Value Project benefit

MSP projects?

Partnership Challenges and Opportunities

• The Adding Value Research Team applies what has been learned about evaluating large-scale systemic reform over the past decade to evaluating the MSPs

• The Adding Value Web site www.addingvalue.org and the Web Alignment Tool www.wcer.wisc.edu/wat serve as resources for the evaluation of the MSPs

• The technical assistance conferences assist MSP evaluators to identify and clarify critical evaluation needs

• The Adding Value Project activities including technical assistance conferences, the evaluation teleconferences, the site visits, and the Adding Value Web sites, serve to establish a network of MSP project evaluators

ADDING VALUE TO THE MATHEMATICS AND SCIENCE PARTNERSHIP EVALUATIONS

PROJECT MISSION CONFERENCES STAFF RELATED BIBLIOGRAPHY RESOURCES/LINKS ONLINE DISCUSSION

Adding Value to the Mathematics and Science Partnership Evaluations is designed to provide technical assistance to the National Science Foundation (NSF) Mathematics and Science Partnership (MSP) program, a massive effort to produce improved student learning in mathematics, science, technology, and engineering (STEM).

This project is funded by the National Science Foundation

http://www.addingvalue.org

Teacher Quality, Quantity, and Diversity

• The Adding Value Project activities contribute to existing knowledge about measuring the impact of large-scale education reform initiatives and enhancing the capacity to measure and evaluate teacher quality, quantity, and diversity within the MSPs

Challenging Courses and Curricula

• The Value Added Tool is used to analyze student achievement data and the impact of the MSPs on content and curricula

• The Web Alignment Tool is designed to analyze the alignment of curriculum standards and student assessments

– The Web Alignment Tool has been alpha tested in several states and will undergo a beta test in January, 2004

– Several states have used the Web Alignment Tool to analyze science, mathematics, and language arts

– Several states plan to use the Web Alignment Tool to examine alignment including social studies

– The Web Alignment Tool is designed to produce measures on the following five criteria: categorical concurrence, depth-of-knowledge consistency, range-of-knowledge correspondence, balance of representation, and source of challenge

Evidence-based Outcomes

• The Adding Value Project assists evaluators to develop effective techniques and reliable tools that can be applied to measure student learning

• A key outcome of the project is more knowledgeable MSP project evaluators and more effective evaluations to produce valid studies that measure the impact of the MSPs on student learning

Institutional Change and Sustainability

• The Adding Value Project assists evaluators to determine the impact of the MSPs on participating schools and universities and to investigate the sustainability of the interventions

Products• A technical assistance plan

• A Web-based alignment analysis tool and documentation

• A value-added analysis tool on CD

• A technical report categorizing and analyzing the set of MSP project evaluations

• Six semiannual technical assistance meetings

• A technical report on the value-added analysis of student outcomes

• A final report