Test design of reading comprehension in college English
Transcript of Test design of reading comprehension in college English
A Probe into Test Design of Reading Comprehension in College English
Yingfei Qu
Foreign language School, Huaqiao University, Xiamen Campus, No. 668, Jimei Avenue, Xiamen, Fujian Province, 361021, China
[email protected], [email protected]
Keywords: Test design; Reading comprehension; College English
Abstract. Evaluation is one of the indispensable steps in foreign language teaching. Traditionally, people in China focused too much on the scores and ignored to a large extent the gradual
development. Reading is an important part in college English in China and should be evaluated
scientifically. This paper, based on the theories of language testing, discussed the testing range of
reading in college English.
Introduction
In most countries around the world, educational testing is increasingly administrated on a large
scale. In heavily populated countries a single test may be administrated to millions of students in a single year, this is especially true for China. Generally, a large-scale testing serves three major
functions: 1) selecting and placing students for further educational and career opportunities; 2)
certifying that students posses requisite knowledge and skills, and 3) monitoring the educations of
educational system to insure that it is functioning effectively. In order to construct tests that carry
out system-oriented functions with large numbers of students, test makers have traditionally been
forced to rely on multiple-choice questions. Defenders of multiple-choice tests often point to a
further advantage such as money saving, objective and easy manipulation. Within such a framework,
the users of the test only focus on the final stage of scoring and thus ignore the fact that language
learning achievement is built in a step by step manner. In some cases, the competence even can not
be reflected in the so called objective testing. Reading, as one of the most important part in college
English in China, has always been a hot topic in testing. This paper probes into the merits and demerits of traditional English testing in China, and provides some suggestions in the range and
scope of reading in college English testing.
Relative Theories about Language Assessment
Basic Testing Theories. In resent years, researchers on educational testing and assessment have
become increasingly aware that multiple-choice questions can set up a misleading opposition
between different ways of reading a text. The test is given at two levels: first level, a basic
qualification, and executive level, and advanced qualification.[1] when test maker design questions, they should pay attention on it. Although not much information is available in the specimen material
on the scope of the investigation into the target language use situation, it would appear that some
consultations with experienced business professional were carried out. In fact, the lack of
information about precisely what the criteria for correctness are is a rather serious deficiency in the
specimen material.
In specific purpose language testing, we must be careful to ensure that the intended discourse
domain, that of the target language use situation, is well signaled in the test[1]. Researchers use the
term discourse domain to refer to the internal interpretation of context. They define discourse
domain as a cognitive construct created by a language learner as a context for interlanguage
development and use. There is considerable research in language testing that demonstrates the effects of test method on test performance. The characteristics of the tasks used are always likely to
affect test scores to some degree, so that there is virtually no test that yields only information about
the ability we want to measure.
6th International Conference on Management, Education, Information and Control (MEICI 2016)
© 2016. The authors – Published by Atlantis Press
6th International Conference on Management, Education, Information and Control (MEICI 2016)
© 2016. The authors – Published by Atlantis Press 1240
Pragmatic Knowledge. Pragmatic knowledge enables us to create or interpret discourse by
relating utterances or sentences and texts to their meanings, to the intentions or language users, and
to relevant characteristics of language use setting. According to Dai and He [2], there are two areas
of pragmatic knowledge: Pragmalinguistics and Sociopragmatics. Pragamalinguistics is applied to
the more linguistic end of pragmatics, i.e. how the linguistic forms are used to serve specific pragmatic purpose. Sociopragmatics is the sociological interface of pragmatics; it is concerned with
the customary ways in which people of a particular culture behave to achieve a particular purpose.
Functional Knowledge. Functional knowledge is what Bachman calls “illocutionary
competence” [3]. It enables us to interpret relationship between utterances of or sentences and texts
and the intentions of language users. Quite frequently the appropriate interpretation of a given
utterance also involves the language users‟ prior knowledge of the language setting, including the
characteristics of the participants, their background knowledge of the world, etc. Meanwhile, to
understand the so called „illocutionary” act, one must be able to analyze meaning in the language, or
in other words, becomes more analytical in dealing with the indirect meaning. This, of course is the
abilities test designer are interested in during their preparation of the test.
The Nature of Reading Comprehension in College English
Researchers in native language and foreign language teaching have contributed much to the
understanding of reading comprehension. Reading is to get information, but it is not a unilateral
process. As many researchers argued, reading is in reality a bilateral or interactive process in which
there could be certain “talking” between the readers and the author. In other words, while reading,
the reader is trying to find what the author wanted to express. As it is, reading comprehension must
care if the material is authentic or not, that is, if there is real interaction between the reader and the author [4]. On the other hand, considering the process of language acquisition, reading is in fact a
channel to provide students input which aims to promote students‟ learning. One important of the interactive feature of reading is its “schemata”. That is, in reading, readers
tend to use the existing schemata existing in their mind, which is useful in reading. Meanwhile,
reading ability lies in the understanding of the contents. This ability cannot be observed directly as
it is per se the mental manipulation. However, it can be observed indirectly through the examination
of some items by testing. For college English, students are supposed to master more than 5000
thousand words and be able to read passages of about 500 or more words. All this must be taken
consideration in the testing system.
The Issues of Testing Reading Comprehension in College English
Relative Matters about the Design. In spite of the wide range of reading material specially written
or adapted for English language learning purposes, there are few comprehensive systematic
programmers which have been constructed from a detailed analysis of the skills required for
efficient reading. Much test material is still limited to short reading extracts on which general
„comprehension‟ questions are based. Characteristics of the test rubric include the objective,
procedures for responding, structure of the test/ task, time allotment [1].
The term rubric is a somewhat obscure one, originally used in Christian prayer books to refer to
procedural information about the form of worship, separate from the religious content. In testing, the term has been defined by Bachman as characteristics „that specify how test takers are expected
to proceed in taking the test, and include the instructions, time allocation, and test organization. [1]
The characteristic of the rubric relate to the nature of the communicative event, whether in the target
language situation or the language test, including its objective, procedures for responding, structure,
format, the time available for completing it, and evaluation criteria and procedures.
According to what have been discussed, it is clear that multiple-choice testing of reading
comprehension cannot satisfy the requirement of measuring the abilities of reading comprehension
of Chinese college students, the author present some test design to improve this problem on the
basis of some relative assessment theories. The following four items about testing reading
6th International Conference on Management, Education, Information and Control (MEICI 2016)
© 2016. The authors – Published by Atlantis Press
6th International Conference on Management, Education, Information and Control (MEICI 2016)
© 2016. The authors – Published by Atlantis Press 1241
comprehension are proposed.
Completion Items. Completion items measure recall rather than recognition. Although such
items are similar in many ways to open-ended questions in tests of reading comprehension, they
often regarded as belonging more to the objective category of test items. Usually completion items
require the testees to supply a word or a short phrase. Unless great care is taken to ensure that there is only one correct answer, the making will prove very difficult when the tester is confronted with a
variety of answers ranging from acceptable. All valid interpretations, whether or not these were in
the test writer‟s mind at time of the construction of the test, must be regarded as correct. In
designing reading comprehension, the item is like “The author was surprised to meet Dr Steven
because ____” rather than “Why was the author surprised to meet Dr Steven?”This kind of testing
is often used to relatively detailed illustration which call forth students recall rather than recognition
[5]. This is especially useful in classroom evaluation as well as in formal testing.
Rearrangement Items. These two item types are particularly useful for testing the ability to
understand a sequence of steps of steps in a process or events in a narrative. While in an exercise for
classroom practice the students will often be required to rewrite the jumbled sentences in their correct sequence, it is obviously preferable for testing purposes to instruct them to write simply the
numbers or letters of the jumbled sentences.
Type 1: the first of these items takes the form of jumbled sentences which the students have
to “unscramble” and arrange in the correct order.
Type 2: the jumbled sentences in the second item are based on a reading comprehension text
and should be unscrambled in the light of the information contained in the text. The students
must write the letters of the sentences again in their correct order in the boxes at the end of
the item.
Open-ended and Miscellaneous Items. The term “open-ended” is used to refer to those
questions which elicit a completely subjective response on the part of the testees. The response
required may range from a one-word answer to one or two sentences. When making open-ended items which require answers in sentences, it is frequently advisable to award at least two or three
marks for each correct answer. When constructing a reading comprehension test, the test writer
should remember to let the text itself determine the types of items set. Indeed, if this principle is
observed, it follows that different parts of a particular text will frequently require different item
types .It is now becoming common practice for more than one type of item to be used to test
comprehension of the same text.
Short-answer Questions. First of all, think-aloud protocols will be used to elicit the response of
college English students to perspective test material. Second, this reader response will be used to: a)
construct different ways of reading the passage and b) develop SAQs that are appropriate for
Chinese university student.
Third, the SAQs will be pilot tested to determine not only psychometric properties of reliability
and validity but also the degree to which they elicit: a) correct responses that are stable in form and
b) in correct response that can be lexically differentiated from the correct ones (these two conditions
must be met if artificial intelligence techniques are to be used in scoring response to SAQs).
Fourth, on the basis of the pilot testing, the SAQs will be revised and administrated on a large
scale. Finally, a computer-based program will be developed to score selected samples of the revised
SAQs (the goal here is simply to demonstrate that such programme can be developed for scoring
responses to SAQs).
Conclusion
Reading comprehension is an indispensable part in college English testing system. For years in
China, the tests of various types adopted merely multiple choices due to the influence of
psychometric theories of the western world. Although it is admitted that this type of items are
feasible in manipulation and used for years, its disadvantages are found by many researchers. For
instance, it cannot reflect the real abilities of testees. Therefore, in college English teaching
framework, a useful and multi-dimensional testing system must be adopted so that students‟ abilities
6th International Conference on Management, Education, Information and Control (MEICI 2016)
© 2016. The authors – Published by Atlantis Press
6th International Conference on Management, Education, Information and Control (MEICI 2016)
© 2016. The authors – Published by Atlantis Press 1242
like skimming, scanning, understanding of the details, memory, recognition, etc can be measured
objectively. This paper is theoretically based, and it is hoped that in the future, more and more
teachers, researchers will be involved for the reform and practice of testing students‟ reading
comprehension in college English.
References
[1] D., Douglas, Assessing language for specific purposes. Cambridge: Cambridge University
Press, 2000.
[2] W. D., Dai, & Z. X., He. A New Concise Course in Linguistics for Students in English.
Shanghai: Shanghai Foreign Language Education Press, 2010.
[3] L., Bachman. Fundamental Consideration in Language Testing. Oxford: Oxford University Press, 1998.
[4] F. F., Zhao. An Analysis of the Authenticity of Reading Comprehension in CET-4. Journal of
Hubei Correspondence University, Vol.28, 2015(23), Pp168-169.
[5] J. B., Heaton. Writing English Language Tests. Beijing: foreign Language Teaching and
Research Press, 2000.
6th International Conference on Management, Education, Information and Control (MEICI 2016)
© 2016. The authors – Published by Atlantis Press
6th International Conference on Management, Education, Information and Control (MEICI 2016)
© 2016. The authors – Published by Atlantis Press 1243