file.upi.edufile.upi.edu/Direktori/FPBS/JUR._PEND._BAHASA_INGGR… · Web viewGrammar names the...
Transcript of file.upi.edufile.upi.edu/Direktori/FPBS/JUR._PEND._BAHASA_INGGR… · Web viewGrammar names the...
The Effectiveness of Using Multiple Choice Tests versus Short-Answer Test in Testing Grammar
Multiple choice is a form of assessment in which respondents are asked to select the best possible answer. Multiple choice format is most frequently used in educational testing. Meanwhile short answer test is the test of people’s ability to demonstrate an understanding of relationship between the course content, course objective and the practical and theoretical perspectives used to understand the course. When the students are going to have short answer test they study hard since they have to integrate what they have learned in lectures, readings, and discussions over the duration of semester. Unfortunately, the students’ score of short answer test is lower than students’ score of multiple choice test. Thus, this study was aimed to investigate the effectiveness of using multiple choice versus short answer question in testing grammar. The method that was used in this study was quantitative research where the data were analyzed using t-test. From the means of two groups (participants) were known that H1 was accepted. It means that there was difference between multiple choice test and short answer test in testing grammar. Multiple choice test more effective than short answer test in testing grammar.
Key words: Short answer, Multiple choice, Testing Grammar.
Introduction
The importance of testing in education makes it an important topic of conducting
research in testing field. Evaluation by teacher-made tests in schools is an important part of the
educational system and crucial area for research (Haynie, 1990a, 1990b, 1991, 1992; Mehrens &
Lehmann, 1987, Wiggins, 1993). According to Heaton’s study, there are many reasons for giving
a test, but however the teacher prepare a test, he or she must think of the real purpose of the test
which is given to the students.
One of the most important reasons is to find out how well the students have mastered the
language, areas and skills that have just been taught. These tests look back and they are called
Progress tests. If the teacher tests what has been recently taught and also practiced then there is
an expectation that students will score fairly high marks. If it does not happen and many students
fail, something must be wrong either with the teaching itself, the materials or students’
understanding of the syllabus. Also progress tests at the end of a unit or a semester should reflect
progress, not failure. The teacher should try to give progress tests regularly, but it is very
important to avoid over-testing, which de-motivates the students. “The best progress test is one
which students do not recognize as a test but see as simply an enjoyable and meaningful
activity”, states Heaton (9).
On the other hand, a classroom test does not focus only on students’ knowledge or
problems with the language, but furthermore, it is concerned with evaluation for the purpose of
enabling teachers to increase their own effectiveness by making adjustment in their teaching. As
a result, those classroom tests should enable groups of students or individuals in the class to
benefit more. The inquiry helps the teacher to select and evaluate the effectiveness of the
syllabus as well as the methods and materials that he or she uses. Furthermore, the test results
may indicate the whole area of the language syllabus, which have not been understood well by
the class. If, for example, eight or more students make a similar mistake in the Present Perfect
Tense, the teacher should take this problem area into account when planning further teaching.
Further, Heaton notes that another important function of testing is to encourage students.
Because most people like doing things at which they are good, the test should be made to show
students’ progress, and on the other hand, students will be usually good at the things, which they
like. It means that a well-constructed classroom test may be effectively used to motivate
students. What is more, if the details of their performance are given as soon as possible after the
test, the students should be able to learn from their weakness. Heaton concludes, that in these
ways a good test can be used as a valuable teaching device (Heaton, 1990).
Jeremy Harmer (2001) mentions two basic characteristics of a good test: Validity and
Reliability. According to Harmer’s statement, a test is valid if it tests what it is supposed to test.
Thus it is not valid, for example, if it is supposed to test writing ability with an essay question
that requires special knowledge of history or biology. In addition, to be reliable, a good test
should give consistent results. For example, if the same group of students took the same test
twice within two days, without reflecting on the first test before they wrote it again, they should
get the same results on each occasion. In practice, these principles should be kept: make the test
This paper actually will discuss more about testing grammar using two kind different
method of testing. Those two methods of testing grammar that will be discussed here are
multiple choice test and short answer test. My students say that they will study hard if they are
going to have a short answer test for their testing and usually they will be less of study if they are
going to have multiple –choice test for their testing, in this case the testing is in testing grammar.
Even though they say that they have studied hard for short answer testing but their score
in this type of testing is low. On the contrary, they are more successful at completing multiple
choice items as they may also guess and it is probably easier work for them. That is the reason
why I have decided to investigate the effectiveness of using Multiple –choice test versus Short
answer test in testing grammar.
Description
Multiple choice
Multiple choice is a form of assessment in which respondents are asked to select the best
possible answer out of choice from the list
(http://www.en.m.wikipedia.org/wiki/multiple_choice). Thus, the respondent have to choose the
answer that they think is right from several possible answer that are listed in the question paper.
Multiple-choice tests have been used extensively in recent years for assessment purposes.
Multiple choice questions were considered to be an ideal test instrument for measuring students’
knowledge of grammar and vocabulary (Harmer 2001). In addition, Heaton (1990) describes
multiple choice questions as a device that tests the ability to recognize sentences which are
grammatically correct. Wherever possible, the items of tests should be set in context. If the
teacher wants to concentrate on a certain area of grammar, he or she should put the item into a
short two-line dialogue. This is better than providing no context at all. (Zlabkova, 2007)
Multiple-choice tests have reportedly been used to good effect in courses ranging from computer
programming (eg Kuechler & Simkin, 2003) to physical education (eg Ayers, 2001).
The advantages of multiple-choice tests to be found in the literature (eg Epstein et al,
2002; Higgins & Tatham, 2003; Kuechler & Simkin, 2003) include that they can:
• test knowledge quickly within large groups
• be used to provide quick feedback
• be automatically scored
• be analysed with regard to difficulty and discrimination, and
• be stored in banks of questions and re-used as required
The literature (eg Wesolowsky, 2000; Paxton, 2000; National Center for Fair & Open
Testing, n.d.) also lists many disadvantages, including that multiple-choice tests can :
• take a lot of time to construct
• test knowledge and recall only
• never test literacy, or ability to analyse
• never test creativity, or unique thinking, and
• encourage students to take a surface approach to learning
There are four steps of constructing multiple choice items described by Heaton (1988)
Step 1: The first step is to know what the teacher wants to test and to have the sentence
testing that. The teacher may think the sentence up or use sentences and errors made by
students in their free composition and open-ended answers to questions. I like tea but I
haven’t / don’t like coffee.
Step 2: Next the teacher writes out the sentence substituting a blank for the area being
tested and then writes in the correct option and the distractor which the student has
provided. I like tea but I .......................coffee. A don’t like B haven’t like
Step 3: Now the teacher adds two more distractors. Again he or she may go to the written
work of the students to provide these distractors. But if he or she cannot find any suitable
errors without too much difficulty, the teacher uses his or her own experience and
knowledge of the target and native languages. Of course, it is necessary to be very careful
and to not give more than one correct option. There is also a possibility to change the
positions of the distractors. I like tea but I .......................coffee. A doesn’t like B don’t
like C haven’t like D like
Step 4: Finally the teacher should check to be sure that other options are not correct and
ask his or her fellow teachers to try the test to find possible imperfections.
Short answer test
One Method of testing that has received little attention in the literature, however, which is
popular in many educational settings, is the use of short answer test items (Haynie, 1983). Short
answer test is testing that test people’s ability to demonstrate an understanding of the relationship
between the course content, the course objectives and the practical and theoretical perspectives
used to understand the course (http://ar.cetl.hku.hk/am_saq.htm). Short answer question are
open-ended question that require the students to create the answer. They are commonly used in
examination to assess the basic knowledge and understanding. Short answer test does not have a
generic question. Question may have complete the sentence, supply the missing word, short
descriptive or diagrams with explanation. The answer usually short, from one to a few line.
(http://ar.cetl.hku.hk/am_saq.htm).
Example of Short answer test
1. Diana is _________ (write) a letter.
Like multiple choice test, short answer test has several advantages as well. The advantages of
short answer test are:
Short answer test are relatively fast to mark and can be marked by different assessors,
as long as the question are set in such a way that all alternative answers can be
considered by the assessors.
Short answer test are also relatively easy to set compared to many assessment
methods.
Short answer test can be used as a pert can be use for summative and formative
assessment, as the structure of short answer tests are very similar to examination
question.
Short answer test are relatively easy to prepare. (http://ar.cetl.hku.hk/am_saq.htm)
It is not an easy thing to make short answer test. People need to follow the step in constructing
the short answer test. These are several steps to construct the short answer test:
1. Design short answer items that are appropriate assessment of the learning objective.
2. Make sure that the content of the short answer question measures knowledge appropriate
to the desired learning goal.
3. Express the question with clear wordings and language which are appropriate to the
student population.
4. Ensure that the item clearly specifies how the question should be answered (e.g. Students
should answer it briefly and concisely using a single word or short phrase?
5. Write the instruction clearly so as to specify the desired knowledge and specify response.
(http://ar.cetl.hku.hk/am_saq.htm)
Why Testing Grammar
Grammar is the structural foundation of our ability to express ourselves. Grammar
is important because it is the language that makes it possible for us to talk about language.
Grammar names the types of words and word groups that make up sentences.
The testing grammar is one of the mainstays of language testing. While such tests
test the ability to either recognize or produce correct grammar and usage, people do not tests
the ability to use the language to express the meaning. However, it can be argued that a basic
knowledge of grammar underlies the ability to use language to express meaning, and so
grammar test do have an important part to play in language programs.
People associate grammar with errors and correctness. Grammar can be part of
literature discussion, when teacher and the students read the poetry or stories, thus, knowing
about grammar means finding the out that all languages and all dialects follow grammatical
patterns. (http://www.grammar.about.com/od/basicsentencegrammar/a/grammarintro.htm)
From the statement above, can be concluded that grammar is very important in
Language where in this case is English, thus testing grammar is chosen in this research to be
analyzed in case of comparing the effectiveness of using multiple choice test and short
answer test in testing grammar.
Methodology
Population and Sample
There were 64 students tested. All of them attended the XI grade at Senior High
School, they were from the same school, the students at grade XI chosen since they have got
more materials about grammar. Class XII actually had more knowledge about grammar but
they were deleted from this population since in this grade the students had to focus on
National examination. The 64 students divided into 2 groups: (a) Multiple-choice test
(Group A, n= 32), (b) Short answer test (Group B, n=32). Those 64 students has the same
level of intellectual, it can be seen from the result of the mid test which had been conducted
before. The scores of those 64 students are quite similar or did not reveal the significant
difference.
Design
The design of this research was a quasi-experimental, a type of research design which
includes experimental and control groups without random sampling (Hatch and Lazarraton,
1991). Those two groups of students were taught by the same materials whether it was a
Group A or a Group B got the same steps in teaching materials. The steps began from the
easiest one for example the material of to be, and then move to the other steps until they
reached tenses steps. After knowing all the materials then the test was prepared. The test
was well-arranged and, of course, clear instructions and the example were given at the
beginning. For the time testing, each group has equal time in doing the test.
Instrument and Data Collection
The score of the students’ tests were used to know the effectiveness of using the
multiple choice test versus short answer test in testing grammar. They were collected
through thirty multiple choice items and twenty short answer question. The test was divided
into three parts: the first part was focused on Present Simple tense, the second on Present
Continuous and the third one concern both the tenses mixed together.
Data Analysis
The results of this study were based on the tests which were conducted. Quantitative
analysis was used to see the effectiveness of using multiple choice versus short answer test in
testing grammar. Score of the students were analyzed through t-test since this test used to
determine whether there was a different significant between two means of the group then was
going to be known whether H1 or Ho which was rejected. If H1 accepted it means that multiple
choice more effective than short answer in testing grammar, however, if Ho accepted then short
answer more effective than multiple choice in testing grammar.
Finding and Discussion
This section presents the research findings and the discussion based on the analysis of the
data collected from the use of multiple choice items versus short answer test in testing grammar.
The research findings and the discussion are presented on the basis of the result of the tests. The
procedure, compared the means of the Group A and Group B to find out their significant
difference. If any, there might be a possibility that the means score of the two groups are
different. The significant difference can be seen after the t-value has been obtained. The formula
is:
t = Difference between two sample means Standard error of differences between means
t = _______________ M1-M2____________________________
∑ x 12−¿¿ + ∑ x 22−¿ 1_ + 1_ (N1-1) + (N2-1) N1 N2
Explanation: M1 : Mean score of group AM2 : Mean score of Group B¿ : Square the total score group A
¿ : Square the total score group B∑ x 12 : Sum the square in Group A∑ x 22 : Sum the square in Group BN1 : The number of participants of group A
N2 : The number of participants of group B
The first step to get the data to be analyzed was conducting the test. In conducting the test
both two group are given the test with different type of test, Group A with multiple choice test
and Group B with short answer test. If the scores have been got, then continued by analyzing the
data could be started. From the test which had been conducted, the scores which are got by the
students are presented in the following table:
Table of students’ scores NO Group A ( Multiple
Choice test)Squares Group B ( Short
answer tets)Squares
1 78 6084 66 43562 77 5929 71 50413 81 6561 47 22094 73 5329 57 32495 71 5041 64 40966 78 6084 51 26017 74 5476 74 54768 72 5184 77 59299 71 5041 59 348110 69 4761 53 280911 76 5776 49 240112 74 5476 65 422513 72 5184 66 435614 71 5041 51 260115 77 5929 59 348116 77 5929 48 230417 76 5776 66 435618 69 4761 65 422519 78 6084 68 462420 77 5929 49 240121 79 6241 71 532922 82 6724 73 230423 86 7396 61 372124 88 7744 68 462425 75 5625 73 532926 73 5329 69 4761
27 80 6400 67 448928 75 5625 70 490029 72 5184 74 547630 85 7225 73 532931 82 6724 78 608432 71 5041 76 5776
M1 : 76.26M2 : 64.41¿ : 5948721
¿ : 4235364∑ x 12 : 186633∑ x 22 : 135080N1 : 32N2 : 32
The normal distribution scores are needed to fulfill the requirement, in this case, t-test
(Hatch and Lazaraton, 1991: 164). Since the sample of the study was more than 30 (32 students
in experimental and 32 students in control group) the normality test was not conducted. Hatch
and Farhady (1982: 64) state that if the N (number of cases) is 30 or more, the distribution of
scores of that random sample is close enough to a normal distribution.
The score of the students which were as the data had been got, it means the next steps
could be started. The next step was analyzing the data, counted the several numbers which were
needed to get the t-value of the data. Here, t-test was used to test or compare two means of both
of the groups. The significance criteria were gained by calculating t-value with the assumption
that Ho is accepted if t-value < t-table while Ho is rejected if t-value > t- table.
From the begin calculating process which had been done, could be explained that the 32
students of Group A were got the result of mode, mean, median and SD were like these; mode
of this group was 77, the median was 76, mean of this group was 76.26 and the SD of this group
was 4,65. Meanwhile for 32 students of Group B were got the result of mode, mean ,median and
SD were like these; for mode of Group B was 66, median of 32 students of Group B was 66 as
well, for mean of Group B was 64.41 and SD of the short answer test group was 8,96.
For getting the t-value, it needed to calculate the square total and sum square of each
group; Group A and Group B. The sum square Group A was 186633, meanwhile the sum square
Group B was135080. After getting the sum square of each groups then the result of square total
were going to be known. For Group A the result of the square total was 5948721 then for Group
B the result of the square total was 4235364.
The calculation results of independent t-test revealed that the t-value was 6.53, while t-
table with alpha .05 and df 60 was 2.00. Thus, t-value was greater than t-table which indicated
that there was a significant difference between the Group A or Multiple choice group and Group
B or Short answer test group. Thus, Ho was rejected and H1 was accepted. It can be concluded
that testing grammar using multiple choice more effective than using short answer .
CONCLUSIONS
This last section presents the conclusions drawn from the research findings and
discussion in the previous chapter. This research aimed to discover the effectiveness of using
multiple choice items versus short answer test in testing grammar. The data was collected from
tests. Both of groups were conducted the test at the same times but different types of the test. The
first group or Group A used multiple choice test items and the second group used the short
answer test. Those both groups have the same level of intellectual, so that, the validity of the test
result can be guaranteed. Regarding to aimed or purposed of this research, the data gained from
the independent t-test, it showed that there was a significant difference between Group A and
Group B, who were taught by us of the scores result. It can be concluded that testing grammar
using multiple choice more effective than using short answer test.
REFFERENCES
Ayers. S.F. (2001). Developing quality multiple-choice tests for physical education Journal of Physical
choice questioning Assessment and Evaluation in Higher Education. Bath: Jun 2000. Vol. 25, Iss. 2;
p. 109 Education, Recreation & Dance. Reston: Aug 2001. Vol. 72, Iss. 6; p. 23
Epstein, M. L., Lazarus, A. D., Calvano, T. B., Matthews, K. A. et al. (2002). Immediate feedback
assessment technique promotes learning and corrects inaccurate first responses
Harmer, Jeremy.( 2001.) The Practice of English Language Teaching. Harlow: Pearson Education.
Hatch and Farhady. 1982. Research Design and Statistic for Applied Linguistics. Massachusetts:
Newbury House Publisher, Inc.
Hatch and Lazaraton. 1994. The Research Manual; Design and Statistic for Applied Linguistics.
Massachusetts: Heinle & Heinle Publishers.
Haynie, W, J. (1992). Post hoc analysis of test items written by technology education teaqchers. Journal
of Technology Education, 4(1), 27-40.
Haynie, W.J. (1990a). Effects of tests and anticipation of tests on learning via videotaped materials.
Journal of Industrial Teacher Education, 27 (4), 18-30.
Haynie, W.J. (1990b). Anticipation of tests and open space laboratories as learning variables in
technology education. Proceedings of the 1990 North Carolina Council on Technology Teacher
Education Winter Conference. Camp Caraway, NC: NCCTTE.
Haynie, W.J. (1991). Effects of take-home and in-class tests on delayed retention learning acquired via
individualized, self-paced instructional texts. Journal of Industrial Teacher Education, 28(4),
52-63.
Haynie,W. J. (1983). Student evaluation: The teacher’s most difficult job. Monograph Series of the
Virginia Industrial Arts Teacher Education Council, Monograph Number 11.
Heaton, J. B.( 1988) Writing English Language Tests. Harlow: Longman.
Heaton, J. B.( 1990) Classroom Testing. Harlow: Longman.
http://ar.cetl.hku.hk/am_saq.htm. Accessed On December, 4th 2013
http://www.en.m.wikipedia.org/wiki/multiple_choice. Accessed On December, 4th 2013
http://www.grammar.about.com/od/basicsentencegrammar/a/grammarintro.htm. Accessed On
December, 4th 2013
Illinois State University (2001). Centre for theAdvancement of Teaching. Journal of Applied Statistics.
Sheffield: Sep 2000. Vol. 27, Iss. 7; p. 909
Kuechler, W. L. & Simkin. M. G. (2003). How Well Do Multiple Choice Tests Evaluate Student
Understanding in Computer Programming Classes? Journal of Information Systems Education.
West Lafayette: Winter 2003. Vol. 14, Iss. 4; p. 389
Paxton. M. (2000). A linguistic perspective on multiple
choice questioning Assessment and Evaluation in Higher Education. Bath: Jun 2000. Vol. 25, Iss.
2; p. 109
Wesolowsky. G. O. (2000). Detecting excessive similarity in answers on multiple choice exams.
Zlbkova, L. 2007. Testing Grammar: Using Multiple Choice Tests versus Translation. Masaryk
University Brno.
The Effectiveness of Using Multiple Choice Tests versus Short-Answer Test in Testing Grammar
(Research Based Paper of Language Testing)
ARRANGED BY
TIARA GARDENIA RESMITA
1202213
School of post GraduateEnglish Education Study ProgramIndonesia University of Education
2013