Teachers' perspectives on competency-based testing

11
Teachers' Perspectives on Competency-Based Testing [] Norman Higgins Elizabeth Rice Norman Higgins is a Professor of Education and Elizabeth Rice is a Faculty Associate at the College of Education, Arizona State University, Tempe. The study described in this article was designed to gather information about teachers" per- spectives on achievement testing and to describe where the competency-based testing in Conti- nuous Uniform Evaluation Systems (CUES) programs fits into teachers" overall plans for teaching and assessing students. The specific objectives of the research were (1) to develop a taxonomy of techniques that teachers use to assess students, (2) to describe teachers" uses and perceptions of various assess- ment techniques, and (3) to describe the imple- mentation of various assessment activities in natural classroom settings. The results of the study have implications for the design and implementation of instructional systems in school settings. [] In recent years a great deal has been writ- ten about the need to implement instructional systems development in the public schools (Martin & Clemente, 1990; Schiffman & Gans- neder, 1987; Shrock & Byrd, 1988; Snelbecker, 1988). In Arizona, as in many other states, instructional systems development was imple- mented in schools as a state-mandated Con- tinuous Uniform Evaluation System (CUES). The CUES legislation required individual school districts to develop instructional objec- fives, objectives-based tests, and objectives- based instructional activities in the areas of reading, writing, and arithmetic for kinder- garten through twelfth grade. The study described in this article was designed to gather information about teach- ers' perspectives on achievement testing and to describe where the competency-based test- ing in CUES programs fits into the teachers' overall plans for teaching and assessing stu- dents. Our aim was to produce a study that would contribute both to the literature on teacher planning and decision making (see Shavelson, 1980) and to the literature related to the implementation of instructional systems development in public schools. The specific objectives of the research were (1) to develop a taxonomy of techniques that teachers use to assess students, (2) to describe teachers' uses and perceptions of various assessment techniques, and (3) to describe the ETR&D, VOL 39, No, 3, pp, 59-69 ISSN 1042-1629 59

Transcript of Teachers' perspectives on competency-based testing

Page 1: Teachers' perspectives on competency-based testing

Teachers' Perspectives on Competency-Based Testing

[ ] Norman Higgins Elizabeth Rice

Norman Higgins is a Professor of Education and Elizabeth Rice is a Faculty Associate at the College of Education, Arizona State University, Tempe.

The study described in this article was designed to gather information about teachers" per- spectives on achievement testing and to describe where the competency-based testing in Conti- nuous Uniform Evaluation Systems (CUES) programs fits into teachers" overall plans for teaching and assessing students.

The specific objectives of the research were (1) to develop a taxonomy of techniques that teachers use to assess students, (2) to describe teachers" uses and perceptions of various assess- ment techniques, and (3) to describe the imple- mentation of various assessment activities in natural classroom settings. The results of the study have implications for the design and implementation of instructional systems in school settings.

[] In recent years a great deal has been writ- ten about the need to implement instructional systems development in the public schools (Martin & Clemente, 1990; Schiffman & Gans- neder, 1987; Shrock & Byrd, 1988; Snelbecker, 1988). In Arizona, as in many other states, instructional systems development was imple- mented in schools as a state-mandated Con- tinuous Uniform Evaluation System (CUES). The CUES legislation required individual school districts to develop instructional objec- fives, objectives-based tests, and objectives- based instructional activities in the areas of reading, writing, and arithmetic for kinder- garten through twelfth grade.

The study described in this article was designed to gather information about teach- ers' perspectives on achievement testing and to describe where the competency-based test- ing in CUES programs fits into the teachers' overall plans for teaching and assessing stu- dents. Our aim was to produce a study that would contribute both to the literature on teacher planning and decision making (see Shavelson, 1980) and to the literature related to the implementation of instructional systems development in public schools.

The specific objectives of the research were (1) to develop a taxonomy of techniques that teachers use to assess students, (2) to describe teachers' uses and perceptions of various assessment techniques, and (3) to describe the

ETR&D, VOL 39, No, 3, pp, 59-69 ISSN 1042-1629 5 9

Page 2: Teachers' perspectives on competency-based testing

60 ETR&D, Vol. 39, No. 3

implementation of various assessment activ- ities in natural classroom settings. The results of the study have implications for the design and implementation of instructional systems in school settings.

BACKGROUND

The increased use of testing to accomplish educational accountability in schools has stimulated considerable research regarding teachers' perceptions and uses of tests in the classroom. Much of this research has focused on the teachers' uses of standardized achieve- ment tests. Three major findings have emerged from this research:

1. There is often a mismatch between what is taught in the classroom and what is tested on achievement tests (Cole & Nitko, 1979; Leinhardt & Seewald, 1980).

2. Achievement tests are of very little use to teachers (Salmon-Cox, 1980).

3. Teachers use a variety of nonstandardized assessment methods in their day-to-day teaching activities (Salmon-Cox, 1980; Yeh, 1978).

Competency-based testing emerged as a response to the inadequacies of both formal standardized testing and informal assessment methods used by teachers. Most research in the area has focused on technical issues related to the development of competency-based tests (Haladyna, 1991; Hambelton, Swaminathan, Algina, & Coulson, 1978; Nitko, 1980). Very little has been written about teachers' percep- tions or uses of competency-based tests. How- ever, the literature on teacher planning does not augur well for programs of competency- based teaching or testing. Investigators fre- quently find that objectives, direct instruction, and objectives-based assessment--three crit- ical components of most instructional systems development programs---appear not at all or very far down on the list of things teachers mention in talking about preparing lesson plans (Eisner, 1967; Morine, 1976; Zahorick, 1975). Instead, teachers seem more concerned with the implementation of classroom activi- ties (Joyce, 1978-1979; Yinger, 1979).

The suburban school district in which this investigation was conducted provided a rich setting for investigating teachers' uses and per- ceptions of tests. During the period of our investigation, teachers in this district im- plemented a wide variety of tests, including formal achievement tests, formal competency- based (CUES) tests, and a wide variety of infor- mal observation and testing methods. The district competency-based tests consisted of interim tests, which could be administered at any time by the teachers, and summative tests, which were administered once at mid-year by all teachers.

PROCEDURES

The study was conducted by an interdisci- plinary research team that consisted of five university faculty members, two graduate stu- dents, three school district administrators, and six classroom teachers. The university faculty members on the team consisted of two anthro- pologists and three teacher educators (an edu- cational psychologist, a language arts specialist, and an educational technologist). The school district personnel consisted of an assistant superintendent for instruction and two class- room teachers with temporary supervisory appointments. The two teachers from the district office, together with two graduate stu- dents from the university, served as inter- viewer/observers. The interviewer/observers received training that included discussions of the methods and goals of ethnographic inter- viewing and observation, role-playing inter- view practice, and practice sessions in making observational notes from videotapes of class- room activities.

The study covered a one-year period. The methods used in the study were designed to provide a description of the teachers' perspec- fives on the process of assessing students and on the role of testing in that process.

The subjects of the study were six classroom teachers and their fourth-, fifth-, and sixth- grade classes. The teachers were treated as integral participants in the research effort as key informants (see Mehan, 1979; Pelto & Pelto, 1978; Quinsaat, 1980). Teachers were asked to participate based on their nomination by their

Page 3: Teachers' perspectives on competency-based testing

COMPETENCY-BASED TESTING 61

principals as "good teachers." This tactic was used to avoid clouding the issue with ques- tions of teacher ability. The teachers' partici- pation was completely voluntary and they were assured of anonymity in reporting the results of the study.

A three-stage design was developed that began with relatively informal initial inter- views, followed by classroom observations, and concluded with formal interviews. This design enabled the study to begin with relatively open, unstructured techniques intended to uncover the concepts of interest to the teach- ers, and progress to relatively more structured techniques for gathering and displaying data. Each classroom teacher was matched with an interviewer/observer who conducted all inter- views and observations with that teacher.

Initial Interviews. Two interviews were con- ducted with each teacher. The initial interview was open ended and was designed to elicit a taxonomy of evaluation methods used by teachers as well as situations in which those methods were employed. The teachers were asked to describe the methods they used to assess the performance of their students and to describe in detail the classroom procedures followed for each method. They were also asked to list the situations in which they assessed the progress of their students. They were then asked to match the situations with the assessment methods they described. The interview questions were deliberately non- directive in nature, so as to allow each teach- er's view of the assessment process to be expressed. The initial interview took from one to three hours to complete.

Classroom Observations. After the initial inter- view, the interviewer/observer assigned to each teacher conducted about ten hours of class- room observation, generally in five two-hour sessions. The classroom observations were recorded as written field notes. The major pur- pose of the observations was to collect descrip- tions of actual classroom examples of the assessment methods the teachers had de- scribed in their initial interviews. Because the timing of the observations coincided with the statewide administration of the California

Achievement Test (CAT), the administration of at least one session of this test was observed. The anthropologist on the team accompanied each interviewer/observer on one observation session for purposes of training, data control, and establishing rapport with the teachers.

Second Interview. The second interview, con- ducted after the completion of the classroom observation sessions, was more structured than the initial interview. The teachers were asked to rate the ~lidity of the eight most com- monly mentioned assessment methods. They were then asked to compare each assessment method in a triads comparison task, identify- ing which method was the most different from the other two methods in the triad. In this interview, the teachers were also asked to rate the extent of their agreement with various statements about assessment that were taken from the initial interview. The second inter- view lasted about one hour.

Analyses

In addition to the general compilation of responses from the interviews, three specific analysis procedures were used: a taxonomic analysis, an assessment method-by-use matrix analysis, and a multidimensional scaling analysis.

Taxonomic Analysis. A taxonomy of assessment methods was prepared for each teacher based upon information provided during the initial interview. A taxonomy is an outline that shows the hierarchical or inclusion relations among a set of items (see Spradley, 1979). These tax- onomies were presented to the teachers dur- ing the final interview for verification and modification. The individual taxonomies were then combined into one summary taxonomy of assessment techniques.

Assessment Method-by-Use Matrix Analysis. A matrix was constructed for each teacher that contained the assessment methods mentioned in the initial interview matched against the sit- uations in which the teacher said the meth- ods were used (for a similar method, see

Page 4: Teachers' perspectives on competency-based testing

6 2 Ell~&D, vol. 39, No. 3

Observations

Assigned Work

Tests

I l

Classwork

Homework

Projects/Reports

m

Teacher-Made

Program

Diagnostic

District CUES

Standardized

-- Watching - - ~

' I Interacting---~

In class On playground Among selves

Ask questions Oral tests Guidance with work

-- Individual I -- Dittos

Workbooks Program pages Directed exercises Small group activities

__ Group(s) {

Discussion Oral reading Board work Response Cards Games

E Dittos Workbook Directed exercises

F Written Oral Graphic

C Formal Tests Informal quizzes

Pretests/posttests Unit Reassessment

E Spache Houghton-Mifflin (Rdg) Corrective reading

....... r - - Summative L-. Interim

California Achievement Test

Figure I [] Taxonomy of Assessment Methods

Page 5: Teachers' perspectives on competency-based testing

COMPETENCY-BASED 1ESTING 63

D'Andrade, 1976). The individual matrices were then combined into one summary matrix. Idiosyncratic responses--those mentioned by only one teacher--were not included in the summary matrix.

Multidimensional Scaling. The responses to the triads comparison task were combined into a similarity matrix. This similarity matrix pro- vides a numerical representation of how sim- ilar or different the teachers perceive each pair of the assessment methods to be. The matrix was submitted to a multidimensional scaling program (ALSCAL), which provides a two- dimensional visual representation of the sim- ilarity measures.

RESULTS AND INTERPRETATION

The results from the study are presented in two parts. The first part presents data describ- ing the teachers' general evaluation strategies. The second part contains a more specific description of how teachers perceive their dis- trict's competency-based assessment program.

Teachers' Assessment Methods

The data from this study indicate that teach- ers use a wide variety of methods to evaluate student performance. The taxonomy in Fig- ure I presents an outline of the methods men- tioned by the six teachers interviewed. The items on the left of the taxonomy are the more general assessment categories mentioned by the teachers; those on the right are the more specific categories.

The teachers mentioned an average of 15 assessment methods during their interviews. Three broad categories emerged from the inter- views: observation, assigned work, and tests. Within these categories, the one with the most methods specified was assigned work, specif- ically class work. In general, the most elabo- rated categories in a taxonomy are the most salient or important to the respondents. The

significant role of class work for the teacher was confirmed by the classroom observations. Classroom observations indicated that the bulk of the teachers' days is spent conducting activ- ities categorized as class work (see Figure 1). It is clear from the taxonomy that the various tests available to the teachers constitute only a part of their overall student assessment information.

Teachers' Uses of Assessment Information

A more complete picture of the degree to which the various assessment methods are used by teachers is presented in Table 1. This is a matrix in which assessment methods the teachers mentioned are matched with the sit- uations in which teachers reported using stu- dent assessment information.

It is important to note that the "use" col- umns indicate only student assessment situ- ations and not all situations in which student assessment data might be used. For example, one teacher mentioned that she used standard- ized test results for "self-evaluation." Because self-assessment is not a situation in which a student is being evaluated, it does not appear in the matrix. The numbers appearing in each cell represent the number of teachers report- ing each use. The most common situations in which teachers mentioned using assessment results were reporting progress, either on report cards or in parent conferences, and checking for student mastery of what had been taught in order to make instructional decisions.

Four assessment methods stand out as being those used most often. In order of frequency, they are written class work, program tests, observation, and teacher-made tests. Aside from the program tests that accompany pub- lished instructional programs and the teach- ers' own tests, other tests appear to be seldom used by teachers. The prime role of class work in the teachers' assessment schema, as sug- gested by the taxonomy, is further confirmed in this matrix analysis.

Page 6: Teachers' perspectives on competency-based testing

64 ETR~D, Vol. 39, No, 3

Teachers' Rankings of Assessment Methods

In the second interview, teachers were asked

to rank eight assessment methods that were

frequently mentioned in the first interview.

Teachers were asked to rank the methods from

1 (best) to 8 (worst) according to which method

provides " the best evaluation of what a stu-

dent knows." The rankings for each teacher

are presented in Table 2. There is a clear

bimodal distribution of the teachers' rankings.

Teachers A, B, and C ranked teacher-made tests

as the best method; teachers D, E, and F

ranked observation as the best method. Dis-

cussion, class work and program tests appear

to constitute the next category of methods,

with competency tests and standardized tests

being judged as the least effective methods.

The even split between teachers who pre-

fer observation and teacher-made tests is con-

firmed by other data. When asked to judge

the statement, 'q 'here is no better way to assess

students than by talking to them," half of the

teachers agreed strongly and half disagreed.

None of the teachers was neutral on that item.

The same pattern exists for responses to the

statement, "The CAT (California Achievement

Test) is mass cruelty." Three teachers agreed

with the statement and three disagreed. Class-

room observations indicated that these differ-

ences in opinion have behavioral correlates in

the classroom. For example, teachers who be-

lieved the CAT to be cruel had a more informal

classroom style and provided more occasions

for individual teacher-student interaction than

teachers who did not think the CAT is cruel.

TABLE 1 [ ] Assessment Methods by Assessment Uses Matrix: Numbers of Teachers Reporting Each Use of Each Assessment Method (N = 6),

ASSESSMENT USES

Check Mastery, Determine Written

ASSESSMENT Plan Group Attitude/ Report Parent Parent Determine Determine METHODS Lessons Students Effort,,, Cards Conferences Reports Competence Retention

Observation 5 1 4 4 5 1 1 1

Written class work 6 2 2 6 5 3 3 2 Group class work 4 1 4 1 1 1 Homework 3 2 3 3 1 I Projects/reports 2 1

Teacher-made tests 6 1 5 4 1 3 2 Program tests 6 3 6 4 2 2 2 Diagnostic tests 1 2 1 Competency tests 2 1 1 3 2 Standardized tests 2 1

Discussions with 2 1 1 2 teachers

TABLE 2 [ ] Teacher Rankings of Assessment Methods [1 = Best, 8 = Worst)

Teachers

Assessment Methods A B C D E F

Observation 8 4 4 1 1 1 Discussion 5 5 3 2 3 2 Class work 4 3 2 3 5 3 Teacher-made tests 1 1 1 4 4 4 Program tests 2 2 5 5 2 5 Competency tests (interim) 6 6 8 6 6 6 Competency tests (summative) 7 7 6 7 7 7 California Achievement Test 3 8 7 8 8 8

Page 7: Teachers' perspectives on competency-based testing

COMPETENCY-BASED TESTING 6 5

While there is considerable disagreement among the teachers as to the degree of for- mality they believe is appropriate for assess- ment, there is agreement as to which methods enable them to determine what objectives are assessed and when assessment will occur. The methods they prefer are those that are most easily integrated into their classroom routines.

Scaling Teachers' Perceptions of Assessment Methods

A mult idimensional scaling of the teachers' percept ions of assessment methods is pre- sented in Figure 2. This analysis uses the same eight assessment methods the teachers ranked earlier. This multidimensional scaling is based upon the teachers' responses to a triads com- parison task. The analysis gives a graphic rep- resentation of the teachers' perceptions of how similar the assessment methods are. The doser together two methods are on the graph, the more similar they are perceived to be; con- versely, the further apart two methods are, the more different they are perceived to be. In this analysis, for example, observation and

discussion are perceived as very similar by the teachers in our sample, while observation and the California Achievement Test are perceived as being quite different. The two-dimensional solution presented in Figure 2 accounts for 90% of the variance in the responses.

The interpretat ion of the two dimensions of the graph in Figure 2 comes from the teach- ers' explanations of their choices in the triads comparison task. Based upon the teachers ' descriptions, the horizontal axis on the graph represen t s a " fo rmal i ty" d imens ion . The assessment methods on the right side are the least formal and those on the left side are the most formal. Thus, class work, observation, and discussion are informal assessment meth- ods, while program tests, district competency tests, and standardized tests are formal assess- ment methods. Teacher-made tests fall near the middle, tending toward the formal side of the graph. The vertical axis on the graph represents a dimension of "curriculum rele- vance." Along this dimension, teacher-made tests, program tests, and class work are con- sidered most relevant to the curriculum, while s tandardized achievement tests like the CAT are cons idered least relevant. The distr ict

e- o w t - o _s

8 e., m => O n-

2.0

1.5

1.0

0.5

0.0

-0.5

-1.0

-1.5

-2.0 -3.0

Program [] Tests

Teacher-Made Tests

Competency Tests (Inter~n) []

Competency Tests

(Summative) [] CAT Test

I I I l I I

-2.5 -2.0 -1.5 -1.0 -0.5 0.0

Formality Dimension

a Classwork

Figure2 [ ] Multidimensional Scaling of Assessment Methods

Discussion a a Observation

I I I I I

0.5 1.0 1.5 2.0 2.5 3.0

Page 8: Teachers' perspectives on competency-based testing

66 ETR&D, Vol. 39, No. 3

competency tests are slightly more relevant than standardized tests, but are perceived by teachers as being not very relevant to the cur- riculum. Discussion and observation occupy about the same position on the curriculum rel- evance dimension as the district competency- based tests.

TEACHERS' PERCEPTIONS OF COMPETENCY- BASED TESTING

As was noted in the previous section, teach- ers perceive their district competency-based tests to be very similar to standardized achieve- ment tests: formal, not closely related to the instructional program, and as relatively poor for assessing what a student knows.

All of the teachers were familiar with their district CUES program and reported keeping the required records of student performance. All but one teacher felt that the CUES pro- gram had had an impact on the way they assessed their students. In general, the teach- ers felt the impact has come not from the test- ing component of the program, but rather from the CUES objectives that help teachers to deter- mine what their students should know. All of the teachers agreed with the statement, "I find the CUES objectives helpful in determin- ing what to teach my students."

The teachers voiced a number of concerns about the CUES program, chief among them the amount of record keeping required. All but one teacher agreed with the statement, "There is too much record keeping with CUES, and that takes away from my teaching." One teacher indicated that she had abandoned the teach/test/mark process in the competency- based program because she felt her class was suffering from the reduced teaching that resulted from the record keeping that was involved.

There was a variety of opinions about the interim and summative tests in the CUES pro- gram. Two teachers appear to be using the interim tests to assess student progress. Oth- ers complained that the tests contained mis- takes and inappropriate questions and that the small number of multiple-choice questions was inadequate to assess what a student knows. There was also the complaint that the summa-

rive tests were given too early in the school year to serve their intended function. In gen- eral, the district competency tests do not appear to be well integrated in the teachers' procedures for assessing students.

Only one teacher expressed a concern which questioned the philosophy of competency- based testing. Her concern was that the emphasis on minimal competencies was over- stated. She felt that teachers should consider the whole child when making decisions affect- ing the child's education. In general, however, the teachers were quite willing to implement the competency-based program, provided the amount of paperwork could be made more manageable. They were content with the objectives that had been written for their grade levels, but were not convinced that the dis- trict had provided them with appropriate or valid methods for assessing mastery of the objectives.

SUMMARY

It is clear from the results obtained in this study that teachers use a wide range of methods for assessing students. The two assessment tech- niques used most often by teachers to obtain useful information were written class work and program tests. Classroom observation and teacher-made tests were also used by a high proportion of the teachers studied. Formal tests such as the district competency tests and stan- dardized tests were not used or were used only by a very small proportion of the teachers for planning lessons or for reporting achievement. Furthermore, the teachers ranked district com- petency tests and standardized achievement tests the lowest of all of the assessment meth- ods that they used. The low rankings are supported by the multidimensional scaling analysis of the teachers' comparisons of assess- ment methods.

The two primary uses of student assessment information that were mentioned by all of the teachers were (1) planning instruction, i.e., grouping students and determining when the content of a lesson had been sufficiently taught; and (2) reporting student achievement to parents and school administrators. These observations form the basis for the recommen- dations that follow.

Page 9: Teachers' perspectives on competency-based testing

COMPEENCY-BASED TESTING 67

RECOMMENDATIONS

The findings of this study have implications for both the development of competency-based testing programs and for the preservice and inservice training of teachers. Considering teachers' preferences for assessment methods that have curriculum validity and that they can control, and considering the purposes for which teachers use assessment information, the following recommendations are offered.

instructional Design Recommendations

Competency-based testing programs should be designed to be integrated into the day-to- day activities of the classroom. The following three recommendations are geared toward such integration.

1. Index the competencies (objectives) of the assess- ment program with the instructional materials and assessment materials used in the school district. Such an index would assist teachers with les- son planning and enable them to select appro- priate assessment materials for use at different times during the school year, particularly those times when they must make instructional deci- sions and report student achievement to par- ents and school administrators.

2. Review the competencies and the materials used to teach and assess them for proper alignment. The teachers in this study felt that the competency- based tests in their district CUES program were not relevant to the way that content was pre- sented and practiced in their instructional materials. A cursory examination of the dis- trict CUES tests and the textbooks used revealed that they were matched on content (e.g., synonyms), but that the skills taught in the textbooks and those assessed on the tests were not properly aligned. The tests were designed in a multiple-choice format for machine scoring. The practice activities and the program tests that were included in text- books taught children how to use synonyms in productive writing and reading activities. Most educators would probably agree that the skills taught and assessed in the textbooks were more like the way synonyms are used

in everyday life, and were therefore more worthwhile, than the way synonyms were assessed on the CUES competency tests.

Developers of competency-based testing pro- grams should ensure that the tests that they develop assess worthwhile instructional objec- tives that are properly aligned in both con- tent and skills with the way the objectives are taught and practiced in the instructional program.

3. Integrate the objectives of the competency-based testing program into the parent reporting materi- als and enable the teachers to schedule assessment activities to correspond with the reporting sched- ule. This recommendation speaks directly to the teachers' concerns about the amount of paperwork associated with recording student progress in the CUES program. If the CUES records of student progress were combined with reporting to parents, the amount of paperwork would be minimized and the value of the competency tests and records to teach- ers would be enhanced. Tying the competency- based tests to parent reports addresses the teachers' primary purpose for assessing stu- dent achievement.

Teacher Training Recommendations

The results of this study indicate that both teachers and administrators could benefit from further training in principles of competency- based instruction and in the interpretation of assessment information. Thus, three recom- mendations for teacher training follow.

1. Provide teachers and administrators with train- ing in the skills needed to implement competency- based teaching programs. The taxonomic analysis of teachers' assessment methods indicates that the distinction between classroom practice activities and competency assessment activi- ties is blurred. The teachers' perceptions that the competency tests are not relevant to the curriculum indicates that assessment materials and instructional materials are not properly aligned. The districts' reliance on machine- readable formats for their competency-based assessment materials compromises the curric- ulum relevance of their tests. These examples

Page 10: Teachers' perspectives on competency-based testing

68 ETR~, Vol. 39, NO. 3

point up the need for basic training in how to design and implement competency-based instruction in school settings. (See Sullivan & Higgins, 1983, or Dick & Reiser, 1989, for competency-based teacher training materials.)

2. Provide teachers and administrators with train- ing in how to interpret and use the information gath- ered from various assessment methods. As the teachers in this study reported, there is little or no uniformity in the way that teachers inter- pret or use student assessment information. The value of formal standardized assessment methods as perceived by teachers is particu- larly low. There appear to be two problems with standardized tests, first, that the tests are not perceived as being relevant to the curric- ulum, and second, that the tests are given too late in the school year to be of value to the teachers. Hanson and Schutz (1978) provide a compelling argument for formal, program- based assessment as opposed to standardized assessment. They describe very positive effects on student learning of program-based assess- ment when it is properly implemented by teachers and properly supported by school administrators.

3. Provide instruction aimed at integrating teach- ers" preferred assessment methods with competency- based instruction. The results of this s tudy indicate that teachers find teacher-made tests, class work, discussion, and observation to be useful methods for assessing student learning. Instruction on how to prepare competency- based items for written tests, for product check- lists, and for performance checklists is very important in teachers' evaluation schema.

CONCLUSION

One of the most important contributions of the naturalistic approach used in this study has been to focus attention on the classroom teacher as an essential and active force in the implementation of competency-based instruc- tional systems. Such systems are being man- dated by state legislatures across the country, and developers must recognize the importance of the classroom teacher in implementing corn-

petency programs. Systems must be designed to be integrated into the teachers' role in plan- ning, implementing, and assessing instruction if they are to contribute to the educational achievement of children. []

REFERENCES

Cole, N., & Nitko, A. (1979). Instrumentation and bias: Issues in selecting measures for educational evaluation. Paper presented at the National Symposium on Educational Research, Johns Hopkins University.

D'Andrade, R. (1976). A propositional analysis of U. S. American beliefs about illness. In K. Basso & H. Selby (Eds.), Meaning in Anthropology. Albu- querque, NM: University of New Mexico Press.

Dick, W., & Reiser, R. (1989). Planning effective instruc- tion. Englewood Cliffs, NJ: Prentice Hall.

Eisner, E. (1967). Educational objectives: Help or hindrance? School Review, 75, 250-266.

Hambelton, R., Swaminathan, H., Algina, J., & Coulson, D. (1978). Criterion-referenced testing and measurement: A review of technical issues and developments. Review of Educational Research, 48, 1-47.

Haladyna, T. (1991). Generic questioning strategies for linking teaching and testing. Educational Tech- nology Research and Development, 39(1), 73-81.

Hanson, R., & Schutz, R. (1978). A new look at schooling effects from programmatic research and development. In D. Mann (Ed.), Making Change Happen. New York: Teachers College Press.

Joyce, B. (1978-1979). Toward a theory of information- processing in teaching. Education Research Quar- terly, 3, 66-67.

Leinhardt, G., & Seewald, A. (1980). Overlap: What's tested, what's taught? Manuscript available from Learning Research and Development Center, Uni- versity of Pittsburgh.

Martin, B., & Clemente, R. (1990). Instructional sys- tems design and public schools. Educational Tech- nology Research and Development, 38(2), 61-75.

Mehan, H. (1979). Learning lessons. Cambridge: Har- vard University Press.

Morine, G. (1976). A study of teacher planning. San Francisco: Far West Regional Laboratory.

Nitko, A. (1980). Distinguishing the many varieties of criterion-referenced tests. Review of Educational Research, 50, 461-485.

Pelto, P., & Petto, G. (1978). Anthropological research: The structure of inquiry (2nd ed.). New York: Cam- bridge University Press.

Quinsaat, M. (1980). 'But it's important datE: Mak- ing the demands of a cognitive experiment meet the educational imperatives of the classroom. The Quarterly Newsletter of the Laboratory of Compara- tive Human Cognition, 2, 70-74.

Salmon-Cox, L. (1980). Teachers and tests: What's really

Page 11: Teachers' perspectives on competency-based testing

COMPETENCY-BASED TEStiNG 69

happening? Paper presented at the meetings of the American Educational Research Association, Boston.

Schiffman, S., & Gansneder, B. (1987). Graduate programs in educational technology: Their chap acteristics and involvement in public education. Journal of Instructional Development, 10(3), 22-28.

Shavelson, R. (1980). Research on teachers" pedagogi- cal thoughts, judgments, decision, and behavior. Wash- ington, DC: National Institute of Education.

Shrock, S., & Byrd, D. (1988). An instructional devel- opment look at staff development in the public schools. Journal of Instructional Development, 10(4), 45-53.

Spradley, J. (1979). The ethnographic interview. New

York: Holt, Rinehart, and Winston. Snelbecker, G. (1988). Instructional design skills for

classroom teachers. Journal of Instructional Devel- opment, 10(4), 33-40.

Sullivan, H., & Higgins, N. (1983). Teachingfvrcom- petence. New York: Teachers College Press.

Yeh, J. (1978). Test use in the schools. Los Angeles: Center for the Study of Evaluation, University of California, Los Angeles.

Yinger, R. (1979). A study of teacher planning: Descrip- tion and a model of pre-active decision making (Re- search Series 55). East Lansing: Institute for Research in Teaching.

Zahorik, J. (1975). Teachers' planning models. Edu- cational Leadership, 33, 134-139.

UMI reproduces this publication in microform: micro- fiche and 16 or 35mm microfilm. For information about this publication or any of the more than 16,000 periodicals end 7,000 newspapers we offer, complete end mail this coupon to UMI, 300 North Zeeb Road, Ann Arbor, M148106 USA. Or call us toll-free for an im- mediate response: 800-521-0600. From Alaska and Michigan call collect 313-761-4700. From Canada call toll-free 800-343-5299.

U-M-I A Bell & Howell Company 300 North Zeeb Road, Ann Arbor, M148106 USA 800-521-0600 toll-free 313-761-4700 collect from Alaska and Michigan 800-343-5299 toll-free from Canada

Please send me information about the titles rve listed below:

Name Title

Company/Institution Address

City/State/Zip Phone ( ~ J