Designing and conducting formative evaluations

Designing and Conducting Formative Evaluations

“Real improvement of our schools is not simply a matter of spending more: It’s a matter of asking more-expecting more-of our schools , our teachers, of our kids, of our parents, and ourselves.” George Bush Sr. 1989-1993

Michael W. Moore cwid # 010028336


“The problem of untested materials was magnified in the 1960’s with –the advent of large curriculum development projects. When such studies were carried out, researchers often found a relatively low level of student achievement with the new curriculum materials. In reviewing this situation, Cronbach (1975) and Scriven (1967) concluded that we must expand our concept of evaluation.” Dick et.al. 2005


Describe the purpose for and various stages of formative evaluation of instructor developed materials, instructor-selected materials, and instructor-presented instruction.

Describe the instruments used in a formative evaluation. Develop a appropriate formative evaluation plan and construct instruments for a set of instructional materials or an

presentation.


“Formative Evaluations; is the process designers use to obtain data that can be used to revise their instruction to make it more efficient and effective. The emphasis in formative evaluation is on the collection and analysis of data and the revision of the instruction.” Dick et.al. 2005


“When a final version of the instruction is produced, other evaluators may collect data to determine its effectiveness. This latter type of evaluation is often referred to as summative evaluation. It is summative in that the instruction is now in its final form, and it is appropriate to compare it with other similar forms of instruction.” Dick et. al. 2005


“There are three basic phases of formative evaluation. The first is one-to-one, or clinical evaluation. In this initial phase the designer works with individual learners to obtain data to revise the materials.” Dick et.al. 2005


“The second stage of formative evaluation is a small-group evaluation. A group of eight to twenty learners who are representative of the target population study the materials on their own and are tested to collect the required data.” Dick et.al. 2005


“The third stage of formative evaluation is usually a field trial. The number of learners is not of particular consequence; often thirty are sufficient. The emphasis in the field trial is on testing of the procedures required for installation of the instruction in a situation as close to the ‘real world’ as possible.” Dick et.al. 2005


“The three phases of formative evaluation are typically preceded by the review of instruction by interested specialists who are not directly involved in the instructional development project, but have relevant expertise.” Dick et.al. 2005


“The purpose of the formative evaluation is to pinpoint specific errors in materials in order to correct them, the evaluation design including instruments, procedures, and personnel needs to yield information about the location of and the reasons for any problems.” Dick et.al. 2005


“Focusing the design only on the goals and objectives of instruction would be to limited. Data on learners’ achievement of the goals and objectives would be insufficient, though important, because these data will only provide information about where errors occur, not why they occur.” Dick et.al. 2005


“Similarly, a shotgun approach to the collection of data would also be inappropriate. Although collecting data on everything you can imagine will produce a variety of information, it may yield some data that are irrelevant and incomplete.” Dick et.al. 2005


“Perhaps the best anchor or framework for the design of the formative evaluation is the instructional strategy. Since the strategy was the foundation for creating the materials, it is likely to hold the key to the nature of errors you made in producing them.” Dick et.al. 2005


“Using the instructional strategy as the frame of reference for developing evaluation instruments and procedures should help you avoid designing a formative evaluation that is either too narrowly focused or too broad.” Dick et. al. 2005


“Instructional strategy can be used to aid the design of the formative evaluation by creating a matrix. Listings the components of the instructional strategy along one side and the major areas of questions about the instruction along the other.” Dick et. al. 2005


“In the intersecting boxes of the component-by-question matrix, you can generate questions that should be answered in the evaluation related to each area and component. Using these questions, you can then plan the appropriate instruments and procedures to use and the appropriate audiences to provide the information.” Dick et. al. 2005


“The different components of the strategy should be quite familiar to you by now. What general areas of questions should be quite familiar to you by now. What general areas of questions should be asked about each component of materials? Although undoubtedly there are questions that would be unique for a given set of materials.” Dick et. al. 2005


“The questions included in the intersecting blocks of the matrix should help you develop other instruments such as checklists to guide your observations and questions to include in your interviews and questionnaires.” Dick et. al. 2005


Summary: “Formative evaluations of instructional materials is conducted to determine the effectiveness of the materials and to revise them in areas where they are ineffective. Formative evaluations conducted on newly developed materials as well as existing materials as well as existing materials selected by Instructional strategy.” Dick et.al. 2005

MICHAEL WADE MOORE2937 Alta Road Montgomery, Alabama

36110 Home Phone (334) 593-2548 Cell Phone (334) 306-7502 Personal E-mail: [email protected] School E-mail:[email protected] Skype Account: michael.moore 238 Dropbox.com:[email protected]

mailto:[email protected]

Designing and conducting formative evaluations

Education

Transcript of Designing and conducting formative evaluations