Assessment Test

43
Running Head: ASSESSING SECOND LANGUAGE SKILLS Assessing Second Language Skills: Final Project Brian Doebbeling and Kathleen Hamel Colorado State University

Transcript of Assessment Test

Running Head: ASSESSING SECOND LANGUAGE SKILLS

Assessing Second Language Skills: Final Project

Brian Doebbeling and Kathleen Hamel

Colorado State University

ASSESSING SECOND LANUGAGE SKILLS 1

Assessing Second Language Skills: Final Project

This project concerns the creation, revision, administration and reliability study of a

grammar test. The reason that we have chosen this language area is because grammar is needed

in all four skills (Purpura, 2004). Grammar questions come up in the other core courses, i.e.

reading, writing, listening and speaking, and it is helpful if we can answer these questions when

they are asked. Furthermore, a good understanding of the ways to assess grammar is helpful in

order to be able to successfully teach it. Since we are both currently working at INTO, the

likelihood of us teaching a grammar course is high because teachers are required to teach any

type of class.

In addition, since we both value a communicative teaching approach, it is necessary to

give students the correct grammar skills needed in order to achieve communicative competence.

According to Purpura (2013), “More recent research shows that grammatical knowledge, when

defined in terms of form-meaning mappings, is not only a critical component of L2 knowledge,

but also a strong predictor of the ability to communicate in semantically meaningful and

pragmatically appropriate ways” (p.1). Therefore, testing students on their ability to recognize

and produce the correct form and meaning of specific grammatical structures is imperative.

Additionally, in order to be communicatively based, the assessment must bring students to,

“accomplish complex and significant tasks, while bringing to bear prior knowledge, recent

learning, and relevant skills to solve realistic or authentic problems” described by Herman,

Aschbacher, and Winters (as cited in O’Malley & Pierce, 1996, p. 4). We are also personally

motivated to create a test at this intermediate level because both of us are currently teaching

classes at that level.

ASSESSING SECOND LANUGAGE SKILLS 2

Description of the Test

Test Purpose

The week 10 grammar achievement test (referred to by INTO as an Achievement

Assessment) for intermediate students assesses material covered in weeks 6-9 in the 14-week

semester program; thus, this test is syllabus-based rather than theory-based. The scores from the

week 10 test must be interpreted as criterion-referenced because students need to know how they

did based on course objectives and outcomes, not upon their peers’ scores. It is a medium-stakes

test and totals 20% of the students’ final grades.

It is then predicted that there will be an impact on the students more than any other party

involved, i.e. teachers or administrators. However, there still could be an impact on teachers as

they could use the results formatively. Consequently, if they see that one or more of the key

grammar areas was especially difficult for the students, there is still the possibility of bringing up

the difficult structures in the weeks before the final achievement test.

Target Language Use Domain

The Target Language Use (TLU) domain for this test is concerned with the grammatical

editing of students’ papers. At a university, students will be required to write a lot of papers and

they will need to edit their own work. They will need to revise their papers’ organization and

content in the future, but that is beyond the scope of this test. In this test, we are assessing

students’ ability to edit a paragraph for grammatical errors. The TLU task, as shown in Appendix

B, involves editing a paragraph. Students are required to identify and repair 10 mistakes, all of

which deal with grammatical structures they have been studying recently: comparatives with

adverbs, adverb cause and effect clauses, and conditionals.

ASSESSING SECOND LANUGAGE SKILLS 3

Construct definition

The construct being assessed with this test is concerned with both receptive and

productive uses of the target grammatical structures. Examinees will be able to choose forms of

stative verbs, parallel structure, and use of articles. Examinees will also be able to choose the

meaning of comparatives with adverbs, present perfect progressive, reduced adjective clauses,

and used to/would with past habits. Choosing both the forms and meanings of these grammatical

structures demonstrates students’ grammatical knowledge in these areas.

In contrast, students must demonstrate grammatical ability in several tasks and content

areas. Examinees will be able to create sentences and evaluate the grammaticality of items in a

paragraph, then revise them with correct comparative adverbs, adverb clauses, and conditionals.

Examinees ability to speak and to listen will not be tested. Ability to correctly use and identify

simple present/past, progressive present/past and the present perfect is assumed.

Description of Tables of Specifications

A common method of test planning and construction is to create a Table of Specifications

(TOS), as shown in Appendix A, which helps organize topics covered and general instructional

objectives (Miller, Linn, & Gronlund, 2009). This method was used to organize content and

objectives in the development of this test. The content areas covered during the weeks since the

last achievement test are listed on the far left hand side of the table, and the instructional

objectives are listed in the top two rows. Each cell in the center of the table has the number of

items total, followed directly by the numbers of the actual items in the test. We met with the

curriculum supervisor for grammar classes at INTO CSU and discussed the content areas that

were being taught, and how these needed to be assessed. At INTO CSU, the terms grammatical

knowledge and grammatical ability are used. Here, teachers define grammatical knowledge as

ASSESSING SECOND LANUGAGE SKILLS 4

the assessment of the capacity to recognize meaning and form of grammatical structures.

Grammatical ability is the capacity to produce limited or extended responses.

As seen in the Table of Specifications, the biggest focus for grammar instruction in the

week 10 test was on adverb clauses for cause/effect and time (43%), followed by comparatives

(21%), and then followed by conditionals (16%). Less time (3% each) was spent on several other

structures: present perfect progressive, reduced adjective clauses, parallel structure, stative verbs,

used to/would for past activities, and articles (a/an, the and ∅). The latter three structures were

items originally covered in their previous class, but were reviewed briefly in the past few weeks.

It should be noted that item number 2D.10 could be correctly answered with either a conditional

or an adverb clause of time. Because we feel that students will be more likely to answer it as a

conditional, it is listed under this content area in the TOS.

Description of Test Tasks

For our test, there are six different types of test tasks. We ordered our tasks from easiest

to most difficult and verified this order of task with the curriculum supervisor. Therefore, each

task will take more time for students to complete than the prior one. We have not included

examples for students of how to perform the tasks for every part because they will work through

a practice test one day before the actual test. The practice test will enable them to be familiar

with the types of tasks and reduce the amount of time students need to understand the task. We

also ordered our tasks starting with grammatical knowledge and then grammatical ability. An

example of grammatical knowledge would be the ability to recognize the correct form; students

could be given the following sentence: I do not enjoy running, swimming, nor (played, playing,

plays). When students choose the correct form of the parallel structure, this demonstrates

grammatical knowledge. In this assessment, there are four main types of limited or extended

ASSESSING SECOND LANUGAGE SKILLS 5

response items which are used to assess students’ grammatical ability: constructed response,

sentence combination, paragraph editing, and paragraph writing, which will be further described

below.

The first task, ‘Multiple Choice,’ which tests grammatical knowledge, consists of

multiple choice questions. Instructions for the task state: ‘Circle the letter of the best answer;’ for

directions on all tasks, we attempted to avoid using technical vocabulary and to be written

between a 6th-8th grade reading level (Becker, 2016). The use of these multiple choice items

allow us to “measure a variety of learning outcomes” (Miller et al., 2009, p. 196). Because there

is only one “single criterion for correctness” for these questions, the students receive one point

for the correct response (14 points total) (Bachman & Palmer, 2010). The expected response

would then be a circle around the correct answer of the letter of the question; these questions test

students’ knowledge of parallel structure, present perfect progressive, articles and other

grammatical knowledge as previously mentioned. Due to the similarity between the expected

response and the input, there is a direct relationship with a narrow scope (Bachman & Palmer,

2010). Because of the strategies of ordering tasks, we expect the students to take the least amount

of time per question in this section. We predict that low-leveled students take a maximum of 13-

15 minutes, while higher-leveled students will take 9-10 minutes. A multiple choice item for

non-native speakers may take as long as 1-2 minutes to answer; however, that may vary

according to the length of the stem and options (T. Becker, personal correspondence, March 29,

2016). Most of the items in this part are short, so we have reduced the allotted time accordingly.

Our second test task, ‘Completion,’ asks students to ‘Fill in the blank with the correct

from the verbs provided in parentheses.’ The expected response of this limited response task is to

have students provide the correct verb tense/aspect based on the rules of the conditional forms

ASSESSING SECOND LANUGAGE SKILLS 6

(types 0-2). There is a direct relationship between the input and expected response, while there is

a narrow scope of the relationship (Bachman & Palmer, 2010). Because students only fill in one

blank, we expect that for low-level students the task would take a maximum of 10-12 minutes,

while higher-leveled students would take a maximum of 7-10 minutes. Additionally, each blank

is worth one point for the correct verb tense/aspect (7 points total). Having students provide an

answer, rather than just selecting one, reduces the chances that students will guess the correct

answer (Miller et al., 2009, p. 175).

For the third task, ‘Sentence Combining.’ there are two sections within this task. The first

asks students to, ‘Using the given words/phrases in the parentheses, combine ideas from ‘a’ and

‘b’ to create a new sentence. Keep the meaning the same.’ Under ‘a’ and ‘b’ two sentences are

given in each. The ‘a’ set of sentences gives a ‘reason,’ while ‘b’ sentences show the ‘result’ of

these reasons. The expected response for this task is a complete sentence that uses the adverb,

preposition or transition word accurately in both form and meaning. Due to this, there is a direct

relationship between the input and expected response and a narrow scope of the relationship

(Bachman & Palmer, 2010). Since students are creating new sentences, we anticipate that for

low-level students the task would take a maximum of 10-12 minutes, while higher-leveled

students would take a maximum of 7-10 minutes. Each sentence constructed is worth two points,

one point for correct form and one point for correct meaning (10 points total). The use of this

partial-credit scoring allows teachers “to capture more precise information from test-taker

responses” (Purpura, 2004, p. 117).

The second part of the task has different directions than the first part; the directions state

‘Create new sentence(s) by combining the sentences using the word given in the parentheses.

Keep the meaning the same.’ Students are expected to write a new sentence with an adverb

ASSESSING SECOND LANUGAGE SKILLS 7

clause, of time or reason/result, or a preposition phrase to show reason/result. Each sentence is

worth two points, one for meaning and one for form (8 points total). Like the first section, this

section also has a direct relationship with a narrow scope (Bachman & Palmer, 2010).

Additionally, we also expect the same amount of time taken for both low-level and high-leveled

students.

The second part of the third task has many similarities to that of the fourth task,

‘Sentence Creation.’ In this task, students are asked to ‘Use the sentences to write a sentence to

make a comparison. Keep the meaning the same.’ We expect students to write a new sentence

using either the comparative, superlative, equative and other ways of showing comparing, as can

be seen in the answer key (Appendix D). One similarity between tasks three and four is the

amount of points each answer is worth, and their rationale, which means that each answer is

worth two points, one for form and one for meaning. Another similarity is that ‘Sentence

Creation’ has a direct relationship between input and expected result, as well as a narrow scope

(Bachman & Palmer, 2010).

Another task that has both a direct relationship with a narrow scope is the fifth task,

‘Editing Paragraph.’ The directions for this task state, ‘Find and correct the 10 grammar errors in

the paragraph. Make your corrections in the space above the error. Do not change the meaning.’

The expected response for this task is that ten corrections are made relating to adverb clauses, of

time and reason/result, conditionals, and comparatives, which can be seen on the Table of

Specifications. Each correction successfully changed earns the student one point. However, if

students correct an error that does not need to be changed, they will not lose points. Since

students must be aware of varying plausible errors, we expect that low-level students the task

ASSESSING SECOND LANUGAGE SKILLS 8

would take a maximum of 14-16 minutes, while higher-level students would take a maximum of

10-12 minutes.

The final task for our test is ‘Short Answer,’ where students are to ‘Write a paragraph (3-

4 sentences) to answer the following question.’ This tasks asks students to answer What causes

students to think school is boring? What effects does this feeling (boring) have on students in

school? The expected response from this question is to show multiple ways of reason/result (or

cause/effect); the various ways that these can be shown are adverb clauses, prepositional phrases

and transition words, which are all classified as ‘current grammar.’ Because students are

expected to create an extended response, there is a narrow scope with an indirect relationship

(Bachman & Palmer, 2010). For this task, an analytic rating scale is the most appropriate option

(Purpura, 2004). Therefore, students’ responses are worth five points total: 1 point for a complete

answer, 2 points for previous grammar (i.e. punctuation, subject/verb agreement, etc.), and 2

points for current grammar, as described above. Even though this task is worth five points, we

predict that students will take the most time completing this task. We foresee low-level students

to take 18-20 minutes to complete, while high-level students would take 16-18 minutes.

Development of Criteria

In order to write the items for the test, we focused on the TLU domain, the construct, and

the course objectives as laid out by the Table of Specifications (Appendix A). We created the

items for the different parts of the test while maintaining a progression from easier to more

difficult items. After having written the items, we got feedback from experienced grammar

instructors at INTO CSU, the curriculum supervisor, and our professor. For example, initially on

the paragraph editing task (Part 2D), some grammar mistakes occupied two different lines of the

text. In other words, journalism isn’t as difficult engineering was separated in the middle so that

ASSESSING SECOND LANUGAGE SKILLS 9

it would be much more difficult for students to see the mistake. After receiving input from these

teachers about this potential problem, we made the necessary changes.

After all of the items were written and had been edited, we developed rubrics for items

2C through 2E. All of these items were worth at least 2 points. In most cases, the rubric was

simply one point for meaning, one point for form. However, Part 2E needed a special rubric as

can be seen on the test (Appendix C). Finally, after administering the test, some elements needed

to have other answers added to the key that would also be acceptable, which is further elaborated

on in the Discussion section.

Pilot Test Procedure

This grammar test was administered during a regular class period, meaning that

participants had 90 minutes to complete it. The 15 participants in this study were considered by

INTO CSU to be of mid-intermediate English proficiency. The students’ L1s are Arabic (10),

Chinese (4), and Thai (1). Of the 10 Arabic participants, 7 were female and 3 were male, while

there was 1 female Chinese speaker, and the rest of the participants were male.These students

were in the second level of the program; thus, they had been in the program prior to this

semester. Because of this experience in the program, these students were familiar with the testing

methods that were implemented. This was the second major achievement test in this course, but

the types of tasks that were included on this test are tasks that were also included on the last test

in this class. The grammatical structure content was different from the first achievement

assessment. The paragraph editing was the only new type of task. As a review before the test, a

practice test was given in exactly the same format as the actual test, and the grammar content

was similar. The instructions of each part were read and the test takers were able to ask

questions. The test takers completed most parts of the practice test, and answers were discussed.

ASSESSING SECOND LANUGAGE SKILLS 10

In order to enable students to keep track of time, a digital clock was displayed on the

board where everyone could see it. Before distributing the test, students were asked to do the

following: distance themselves from other students, put their phones on silent and in their bags,

go to the restroom if need be, and if they had a question to raise their hand. Students were also

told that once they left the room, then they must turn in their test and not come back in the

classroom; only in the case of emergencies would students be allowed to leave and come back.

Before the test was administered, the test key was created (Appendix D). Within this key,

each section was assigned certain points as laid out by particular criteria. These criteria allowed

us to score the test accurately and fairly. After the students had taken the test, they were graded

by one instructor due to practicality reasons and because there was a deadline to have the grades

into the administration. Even though only one person was grading the tests, when questions arose

the grammar curriculum supervisor was able to provide valuable input.

Through this input, we were able to have conversations about adding more potential

answers to the key. For two questions we expanded the possible correct answers, like under Part

2A #4, which states:

If people live in the city, they (have) _______ more possibilities for shopping.

And the initial answer that was included on the key was:

If people live in the city, they (have) _have_ more possibilities for shopping.

When reviewing students’ answers, this came up frequently:

If people live in the city, they (have) _will have_ more possibilities for shopping.

Since it is possible to use the future in the independent clause in the first conditional, the

curriculum supervisor and I decided to add it to the key. We also saw a similar trend with 2C #3,

and add the same as as a potential answer.

ASSESSING SECOND LANUGAGE SKILLS 11

Test Results

After the administration of the test, the instructor graded the tests. Following this, the

tests were scanned, copied and the corrections made by the instructor were whited out. Then, the

other researcher graded all the students’ tests individually as well. The names of the students had

also been replaced by numbers so that there was no chance for bias. The raters then met up to

compare the correction results. The items on each test were compared and the raters discovered 4

main areas of difference. One mistake in correcting was simply overlooked. Five mistakes of the

second rater were due to a lack of knowledge of level appropriate grammar. Lastly, minor

scoring differences and misinterpretations of the rubrics caused 6 mistakes in grading. These

items were discussed and it was discovered that only one of the mistakes out of 62 points on 15

tests was made by the original instructor (i.e. Rater 1).

After these items had been discussed we analyzed which students were masters or non-

masters. As the cut score was 74.5%, all 15 students in the class passed the test, so in general,

they mastered the material from the construct established.

Table 1 Summary of Rater 1 Scores for Pilot Study

Scoring Criteria N Min Max Mean SD

Rater 1

Knowledge: Form & Meaning 15 11 14 13.01 1.03

Ability: Conditionals 15 4 6 5.73 0.88

Ability: Adverb clauses & Connectives 15 15 18 17.01 1.16

Ability: Comparisons 15 5 8 7.33 0.98

Ability: Paragraph Editing 15 7 10 8.23 1.03

Ability: Paragraph Creation 15 4 5 4.53 0.39

ASSESSING SECOND LANUGAGE SKILLS 12

Table 2 Summary of Rater 2 Scores for Pilot Study

Scoring Criteria N Min Max Mean SD

Rater 2

Knowledge: Form & Meaning 15 10 14 12.73 1.28

Ability: Conditionals 15 4 7 5.73 0.88

Ability: Adverb clauses & Connectives 15 16 18 17.4 0.63

Ability: Comparisons 15 4 8 7.2 1.21

Ability: Paragraph Editing 15 7 10 8.4 0.91

Ability: Paragraph Creation 15 3.5 5 4.5 0.5

As shown by Table 1 and Table 2, both raters, at first glance, appeared to grade similarly

throughout all six parts to the test. This was confirmed for some parts of the test by running

interrater reliability using Pearson’s r. Displayed in Table 3 below, there was a very high

correlation (1) between raters on Part 2A Ability: Conditionals. Despite this, the correlation for

Part 2C was rather low with 0.36; our hypothesis of why this correlation was so weak is included

in the Discussion.

After calculating the variance on the mean scores for all of the parts, the Cronbach’s

Alpha was calculated for all six tasks and was 0.50. This means that there was a medium-low

correlation that all six tasks were measuring the same grammatical ability. Then the Standard

Error of Measurement was 2.01. Therefore, a student’s score should be (+/-) 2 points of the final

score calculated.

ASSESSING SECOND LANUGAGE SKILLS 13

Table 3 Interrater Reliability by parts

Test Section Correlation between raters

Part 1 0.88

Part 2A 1.0

Part 2B 0.54

Part 2C 0.36

Part 2D 0.71

Part 2E 0.45

Item Analysis

With the multiple choice items in Part 1, item analysis was performed to see how difficult

each item was and how much each item discriminated between the test takers (Miller, Linn &

Gronlund, 2009). Taken globally, the average difficulty of the items was 93%, and the

discrimination was 14%. These calculations were performed by separating students into 2

groups: those who had done well, and those who, in comparison, had done poorly. Then we

looked at each item to see how these two groups performed. The distractors were analyzed to see

how effective they were. Six of the items had distractors which were chosen at least once, and

two of the items had distractors where both of the distractors were chosen. Eight of the items

were correct for all of the students, so they had a difficulty level of 1.00 and a discriminatory

power of zero. The other 6 items had a difficulty level ranging from .75 to .92 and a

discriminatory power ranging from 0.17 to 0.5.

ASSESSING SECOND LANUGAGE SKILLS 14

Discussion

Test revisions made

Since this was the first time this test was piloted, there are several revisions that have

been made. In Part 1, some of the items had individual directions in addition to the general

directions at the beginning. We made changes to keep only the general directions. Furthermore,

the answer to Part 1.9 was too easy and has been changed. In Part 2D, the paragraph editing

activity, one of the items to be corrected was missed by most students in the class. After

discussing this with the curriculum supervisor, this item was not counted in students’ final

scores. Although Part 2D is an authentic activity from the TLU domain, it needs more

scaffolding for students at the intermediate level, so it has been changed to include a list of

possible mistakes that students could look for based on the grammar that has been covered

during that part of the semester. A sample of how corrections could be made has also been

included.

Suggested changes to answer key

After correcting the tests, some issues were raised about the scoring methods used on the

rubrics in the answer key, and there are several suggested changes to the answer key. In Part 2A,

two answers that were also correct have been added to the answer key as mentioned in the test

piloting section. Furthermore, in Part 2D, we added acceptable answers to the key for item

numbers 5 and 7. For example, for item number 5, we accepted ‘,so’ because it also shows result.

If this test were to be used again in the future, several changes need to be made to the

answer key for the short and extended response test items. For example, in Part 2E, it would be

helpful to list the main items that belong to both previous grammar and current grammar. We

would also add a comment that no points should be deducted for spelling except where the

ASSESSING SECOND LANUGAGE SKILLS 15

mistakes occurred in the grammar, like verb endings. Another possibility for this item is to

design a holistic rubric with general guidelines for the main elements in the task.

Critique of item performance

As mentioned in the results section, a simplified item analysis (Miller, Linn & Gronlund,

2009) was done for the multiple choice section. This kind of item analysis enables the inspection

of each multiple choice item to look at item difficulty and discriminating power of each item.

Since this is not a norm-referenced test, our goal is not to differentiate among students, so the

fact that over half of the items in this part were answered correctly does not mean they are not

effective items (Miller, Linn & Gronlund, 2009). As mentioned earlier, this test follows good

assessment practices of having the easier items towards the beginning of the test, so it is to be

expected that students would do well on this first section.

Reliability

In order to evaluate the reliability of our test, two raters scored the two tests, based upon

the answer key. After computing interrater reliability, it seems as though we were fairly

inconsistent. As previously mentioned, we had a very high correlation on Part 2A, and Part 1 and

2D were also fairly high. Even though half of the parts had a high correlation, the other three had

a medium-low to a very low correlation, specifically Part 2C, with a 0.36. Our reasoning as to

why there was a low correlation to that part was because one of the raters was not sufficiently

trained enough in the grammatical structures acceptable. The same goes for Part 2E, which was

the second lowest correlation between raters. If both raters had been trained to the same

standards, then reliability could have been higher.

ASSESSING SECOND LANUGAGE SKILLS 16

Evidence for validity

There are several pieces of evidence that demonstrate a valid use of test scores

(Cumming, 2013). One of the most important pieces of evidence for validation is related to

construct validity, which means the scores on the test can be interpreted to show that students

have ability in the areas prescribed in the construct (Miller, Linn & Gronlund, 2009). This

process of working towards the construct must be integral to the actual planning of the test, so

that you are ensuring you’re testing what was taught. As previously noted in the construct

definition, the Table of Specifications was established with the construct in mind, namely the

recognition and differentiation of form and meaning of certain structures and the ability to use

other structures. The TOS was used in deciding exactly how many items of each part of the

construct should be created. A further aspect of the argument for construct validity comes from

analyzing the tasks in the test to see if they “elicit the intended behavior” (Cummings, 2013, p.

5). While Parts 2B and 2C elicit the desired ability on the sentence level, 2D and 2E do it at the

discourse level.

A further argument for validation of the test concerns criterion related evidence. One way

to demonstrate criterion related evidence is to look for similarities between the test scores from

the test, and in “predicting individuals’ long-term success” (Cummings, 2013, p. 5). Looking at

the scores in general, it can be seen that every test taker who passed the week 10 achievement

test, also passed the class. Furthermore, there were 6 test takers who did well on the on the week

10 test, i.e. scored higher than 90 %, who also scored higher than 90% in the course. Likewise, 4

students who scored lower than 90% on the week 10 test, also scored lower than 90% in the

course. Taken together, these two groups represent 66% of all students. Even though each test is

different and there are always other factors that influence how people do on a test, this statistic is

ASSESSING SECOND LANUGAGE SKILLS 17

a fairly good indicator that we can interpret the week 10 achievement test for students’ overall

grammatical ability, as represented by the objectives and outcomes in the syllabus.

Consistency among raters is also a criteria for validation of the use of scores. Since some

parts of the test did not have high inter-rater reliability, this would be an argument against

validity. At the same time, it should be noted that the first rater was in constant communication

with the curriculum supervisor who confirmed the actual scores that the students received.

Therefore the second rater’s scores did not directly affect the students’ final scores on the test.

Self-report methods can also be used to show the correct use of scores. When students

were asked about the test, they commented that it was a fair assessment of their abilities in the

areas of instruction based on the standards in the syllabus.

Authenticity

The authenticity of the assessment is measured by the degree of correspondence between

the test tasks and the TLU domain. Part 2D was designed and analyzed through the use of a TLU

domain table (Appendix B), and was judged to be fairly authentic by the instructor. It was only

fairly authentic because the kinds of mistakes were of limited scope. On the original test, there

were no instructions about what kind of mistakes test takers should look for. However, this may

have been too challenging for the students, and we decided to list the kinds of mistakes test

takers should look for as can be seen on the revised test (Appendix C).

Impact and Practically

Since parts of this test were also used by other instructors, their input about the relevance

and similarity to previous tasks were obtained. Two out of the four instructors who taught similar

sections of the course felt as though this test assessed students fairly due to the fact that students

had practiced with similar types of tasks throughout the semester. Their students also felt as

ASSESSING SECOND LANUGAGE SKILLS 18

though that this test had been fair because the tasks and knowledge being tested had been

thoroughly covered prior. One of these instructors did make revisions, as to be tailored to their

particular class, which is why those results are not included.

The other two instructors did not necessarily feel the same way, and made major

revisions prior to the administration of the test. They did not give more details about this, but it

can be assumed that they did not use similar types of tasks during their course; because this test

was not standardized, this was acceptable by INTO. In order to determine the impact on students,

and the applicability to their future studies, an administrator would need to evaluate the different

tasks utilized.

In terms of practicality, this test was administered during normal class time and did not

require extraneous resources nor training to administer. That being said, some training would

have been necessary in order to properly score the test, especially with Part 2E, even though

there was a guiding rubric. Additionally, it did take some time to score all of the tests, and some

discussion about certain items were discussed with the grammar curriculum supervisor, in order

to properly record students’ grades. The more authentic parts 2D and 2E took the longest to

correct, but this is a tradeoff we were willing to make in order to have a high degree of

correspondence with real-life tasks.

Overall estimate of test purpose

Altogether, it seemed as though the test achieved its purpose in assessing students in their

knowledge of their study on material from weeks 6-10, as stated by the syllabus. There was

evidence for construct validity, but in order to gather more data about the reliability and

effectiveness of this test, it would have been beneficial to have obtained results from all four

sections of intermediate grammar.

ASSESSING SECOND LANUGAGE SKILLS 19

Reflection

One of the major weaknesses seen in this project is not directly an aspect of the test, but

of the way it was rated. Since I, Brian, had not taught the class nor was I familiar with the topics

being assessed, there was a need for more training before I corrected the test. A way this could

have been resolved is by correcting 3-4 tests from another teacher who gave the test and then

discussing the corrections in detail. Furthermore, we could have put more detail into the rubrics

in parts 2B, 2C, and 2E so that some of these errors would have been avoided. For example, in

Part 2E, clearer instruction could have been given concerning the method of awarding or taking

away points. This is an aspect of the test that could be developed if it is given again in the future,

as INTO typically reuses tests if the same constructs are being tested.

The use of the TLU domain table was very helpful in analyzing every aspect of the

editing paragraph. However, in hindsight, Part 2E was also a very authentic task. Accordingly,

we would need to create a TLU domain table for Part 2E to really analyze it in detail for

authenticity; notwithstanding, it does seem very close to what students are asked to do at the

university level. We had discussed putting more detail into the directions so that future students

would have to respond with the grammar that we were trying to elicit. When students are given

an assignment in one of their future classes, they are free to use the grammar structures that are

most appropriate for the response. The actual tests answers to that part showed that test takers

answered using cause/effect and conditional grammar, which comprised a major part of the

grammatical structures which were taught.

From my perspective, I, Kathleen, felt very similarly to Brian that the training for him

could’ve been more substantial so that mistakes could’ve been avoided. One way that I could’ve

have better prepared him for those mistakes would’ve been to update the answer key as needed.

ASSESSING SECOND LANUGAGE SKILLS 20

Instead, I wrote down errors (not correct answers) that I needed to discuss with the curriculum

supervisor, which ended up confusing him, and understandably so. Nonetheless, so much of my

time was consumed with copying, and removing indicators from the marks that I had made on

students’ tests, that I hadn’t thought about the notes I had made that could’ve been interpreted

wrong.

Another point that I wish could’ve gone better was the collaboration between other

grammar teachers. Even though we had been receiving comments from them throughout the

process of creating this test, one instructor had decided to not share all suggestions they had.

Instead, they informed us the day of the exam, specifically an hour before it was to be

administered, that they had made “enough” changes to the test. If we were informed about this

earlier, we could have had a discussion about these potential changes, so that we could have had

more data to provide for the reliability and effectiveness of our test. This was a learning

experience for me that both input from others can be highly beneficial, and that others might also

not want to truly be a part of that collaborative process, which at times can be both

understandable and frustrating, depending on your perspective in the matter.

ASSESSING SECOND LANUGAGE SKILLS 21

References

Bachman, L., and Palmer, A. (2010). Language assessment in practice. New York, NY: Oxford

University Press.

Becker, T. (2016). Developing classroom assessment [Powerpoint slides]. Retrieved from the

class’ Canvas page.

Cummin, A (2013). In C. A. Chapelle (Ed.), The Encyclopedia of Applied Linguistics (pp. 1-10).

New York, NY: Blackwell Publishing Ltd.

Miller, M. D., Linn, R. L., and Gronlund, N. E. (2009). Measurement and assessment in

teaching. Upper Saddle River, New Jersey: Pearson Education.

O’Malley, J.M., & Pierce, L.V. (1996) Authentic assessment for English language learners:

Practical approaches for teachers. Boston, MA: Addison-Wesley Publishing Company.

Purpura, J. E. (2004). Assessing grammar. New York, NY: Cambridge University Press.

Purpura, J. E. (2013). Assessing grammar. In C. A. Chapelle (Ed.), The Encyclopedia of Applied

Linguistics (pp. 1-10). New York, NY: Blackwell Publishing Ltd. DOI:

10.1002/9781405198431.wbeal004

ASSESSING SECOND LANGUAGE SKILLS 22

Appendix A

Table of Specifications Content Areas Grammatical ability Ss. produce with grammar

# of items Grammatical knowledge. Ss recognize meaning and form. # of items

# of items/# of points

% of points

Para. writing

Para. editing Sentence combining Completion M/C choose form

M/C choose Meaning

Wk 6: comparatives with adverbs, ch. 13

(3) 2D.1 + 2D 2 + 2D.9

(4) 2C 1-4

(2) 1.1 + 1.14

9/13 21 %

present perfect progressive (2) 1.12 + 1.13

2/2 3 %

Wk 6: Adverb clauses (cause/effect)

(1) 2E.1

(4) 2D. 3-5 + 2D.8

(7) 2B.1-5 + 2B.2.7 + 2B.2.9

12/23 37 %

Wk. 7: Adverb clauses: time (2) 2B.2.6 + 2B.2.8

2/4 6 %

Wk 8 ch. 12 Conditionals types 0-2

(3) 2D.6,7+2D.10

(7) 2A 1-7

10/10 16 %

Wk. 6 Reduced adjective clauses, (2) 1.9 + 1.10

2/2 3%

Review stative verbs (2) 1.3 + 1. 6

2/2 3 %

Wk. 8 Parallel structure (review simple lists)

(2) 1.2 + 1.7

2/2 3 %

Wk 9: Used to/ would (past habit)

(2) 1.8 + 1.11

2/2 3 %

review use of articles (a/an/the) (2) 1.4 + 1.5

2/2 3 %

# of items /points in the part 1/5 10/10 13/26 7/7 6/6 8/8 45/62

% of points in the part 8% 16 % 41 % 11 % 10 % 13 % 100% 100 %

ASSESSING SECOND LANGUAGE SKILLS 23

Appendix B Target Language Use task

At the university, students will numerous papers to hand in. They will need to edit their own writing for grammatical errors at the discourse level of these assignments.

TLU Task Characteristics Characteristics of the setting

physical characteristics library or study setting participants student alone time of task morning, afternoon, evening

Characteristics of the scoring rubric instructions

language channel specification of procedures and tasks

structure time allotment scoring method

criteria for correctness procedures for scoring the response explicitness of criteria and procedures

Characteristics of the input format

channel visual form language language target language length from 1 to 30 pages type input for interpretation degree of speededness less speeded than native speaker vehicle live

language of the input language characteristics

organizational characteristics grammatical syntax, vocabulary, graphology textual academic or technical

pragmatic characteristics functional ideational, heuristic sociolinguistic genre, register, natural/idiomatic,

topical characteristics personal Characteristics of the expected response

format channel visual form language language target length 1-30 pages type extended production degree of speededness not speeded as students have time to revise

language of the expected response language characteristics

organizational characteristics grammatical syntax, vocabulary, phonology, graphology textual rhetorical and cohesion

pragmatic characteristics functional ideational, heuristic sociolinguistic genre, register, natural/idiomatic, cultural/figures of

speech topical characteristics technical/academic

Relationship between input and response reactivity non-reciprocal scope of relationship broad scope directness of the relationship indirect

ASSESSING SECOND LANGUAGE SKILLS 24

Assessment Task Characteristics

On the test, students will edit a given paragraph only with errors in the grammar covered leading up to this test. Characteristics of the setting

physical characteristics classroom participants ESL students and instructor time of task daytime

Characteristics of the scoring rubric instructions

language English channel written specification of procedures and tasks briefly explained in writing

structure edit para. 11 items time allotment 10 – 13 minutes scoring method

criteria for correctness finds mistake and corrects it procedures for scoring the response 1 pt for correcting mistake explicitness of criteria and procedures Each correct answer 1 point, written in directions

Characteristics of the input format

channel written form language language English length 12 lines type paragraph degree of speededness somewhat speeded vehicle reproduced

language of the input language characteristics

organizational characteristics grammatical incorrect syntax items in context of para. textual cohesion and organization is correct

pragmatic characteristics functional ideational sociolinguistic academic register, genre

topical characteristics personal story Characteristics of the expected response

format channel visual form language-based language English length phrases type limited production degree of speededness somewhat speeded

language of the expected response language characteristics

organizational characteristics grammatical correct syntax that fits context textual cohesion

pragmatic characteristics functional ideational sociolinguistic correct register, genre

topical characteristics personal Relationship between input and response

reactivity non-reciprocal scope of relationship narrow scope directness of the relationship relatively direct

Degree of authenticity: Fairly authentic. The assessment task is similar to the TLU task, but the limited scope of possible errors reduces the task’s authenticity.

ASSESSING SECOND LANGUAGE SKILLS 25

Appendix C

AEIN8213: Week 10 Achievement Test The purpose of this test is to assess your knowledge and ability to use grammatical structures from level 2 texts. You have 90 minutes to complete this test and all answers should be written on this test. Each section has directions, please read them carefully. Unless otherwise noted, each question on the test is worth 1 point. There are a total of 62 points. CSU Honor Pledge- Write this statement on the lines below and initial. “I will not give, receive, or use any unauthorized assistance.”

_________________________________________________________________Initials:______

Part 1: Knowledge 1) Multiple Choice (14 points) Directions: Read each item and circle the letter that best answers the question or completes the sentence. 1. John isn’t as happy as Mary is.

Who is happier?

a. Mary b. John c. Neither 2. I do not enjoy running, swimming nor ____________. a. played b. playing c. plays 3. Paul ____________ when people are mean. a. is hating b. hates c. is hated

ASSESSING SECOND LANGUAGE SKILLS 26

1) Multiple Choice (Continued) Directions: Read each item and circle the letter that best answers the question or completes the sentence.. 4 . I read ____________________ interesting article yesterday. ___________________ article was about St. Augustine, Florida ___________________oldest city in the United States. (Note: ∅ = no article).

a). an / The / the b). an / The / an c). the / An / the d). a / The / ∅ 5. One old city is in Virginia. __________________ city name is Jamestown. ___________________ interesting fact about Jamestown is that it was ________________ first British colony. (Note: ∅ = no article). a). The / ∅ / a b). The / An / the c). A / An / ∅ d). ∅ / A / the 6. I ___________________ your wife’s wonderful spaghetti sauce. a). want b). am wanted c). am wanting 7. Choose the sentence with the correct parallel structure. a). Yangping had her pens, got her notebooks, and her bookbag. b). Yangping had her pens, her notebooks, and her bookbag. c.) Yangping wanted her pens, notebooks, and remembered her bookbag. d). Yangping had one pen, notebooks, and many bookbags.

ASSESSING SECOND LANGUAGE SKILLS 27

1) Multiple Choice (Continued) Directions: Read each item and circle the letter that has the closest meaning to the underlined part of the item. 8. Mary used to smoke. a). Mary smokes now. b). Mary has never smoked in her life. c). Mary smoked in the past, but she does not smoke now. d). Mary smoked in the past, and she still smokes today. 9. The buildings destroyed by the fire were very old. a). The old buildings are gone. b). The buildings destroyed the fire. c). The fire was very old. 10. Students regularly doing the homework understood the lessons better in their

classes.

a). Students not doing their homework had a better understanding. b). Students who were doing the homework had better understanding. c). Textbooks understood the lesson better. 11. When I was a child, I would walk to school. a). One time action b). Past habit c). Conditional 12. I have been cooking dinner. a). Past activity b). Finished action before now c). Recent activity 13. I haven’t been studying.

a. Recent activity b. Past activity c. Finished action before now

ASSESSING SECOND LANGUAGE SKILLS 28

14. Either my mother or my father will come to the soccer game.

Who will come to the soccer game?

a. My mother b. My father c. Neither d. We don’t know

Part 2: Ability 2A) Completion (7 points) Directions: Fill in the blank with the correct form of the verbs provided in parentheses. 1. If tomorrow (be) _______________ a nice day, we might go swimming.

2. People (travel) ________________ more in the U.S. if it was less expensive.

3. Fatimah will be sad if her friends (not, write) _______________ to her.

4. If people live in the city, they (have) ___________________ more possibilities for

shopping.

5. If I (find) ________________$1 million on the street, I would not give it to the police.

6. If you (not, study) ______________ conditionals, you will have problems with grammar.

7. If I were you, I (not, be) _________________ late to class for the exam next Friday.

ASSESSING SECOND LANGUAGE SKILLS 29

2B) Sentence Combining (18 points total) 2B.1) Adverb Clauses and Connectives (10 points) Directions: Using the given words/phrases in the parentheses, combine ideas from ‘a’ and ‘b’ to create a new sentence. Keep the meaning the same. a) My dad is a good cook. (or) My dad is a bad cook. b) I always enjoy the meals he makes. (or) I don’t enjoy the meals he makes. Example: Due to him being a bad cook, I don’t enjoy the meals he makes.

1. (although)__________________________________________________________________

2. (because)___________________________________________________________________

3. (thus)_______________________________________________________________________

4. (because of)__________________________________________________________________

5. (so) _______________________________________________________________________

2B.2) Adverb Clauses (8 points)

Directions: Create new sentence(s) by combining the sentences using the word given in the parentheses. Keep the meaning the same. 6. I bought food at the grocery store. Then, my dad cooked me dinner. (after)_________________________________________________________________________ 7. There was no class yesterday. The teacher was sick. (due to)_______________________________________________________________________ 8. I study often at night. Then, I go to bed. (before)_______________________________________________________________________ 9. I don’t eat meat. I am a vegetarian. (since)________________________________________________________________________

ASSESSING SECOND LANGUAGE SKILLS 30

2C) Sentence Creation (8 points) 2C.1) Comparisons Directions: Use the sentences to write a sentence to make a comparison. Keep the meaning the same. 1. Mohamed doesn’t work hard. Ali works hard. ______________________________________________________________________________ 2. Louis is bad at math. Betty is okay at math. Tina is good at math. ______________________________________________________________________________ 3. Ashlee likes to read books. Angela also likes to read books. ______________________________________________________________________________ 4. At the beginning, the class was the easiest. Later, the class became harder. At the end, the class was the hardest. ____________________________________________________________________________

ASSESSING SECOND LANGUAGE SKILLS 31

2D) Editing paragraph (10 points) Directions: Find and correct the 10 grammar errors in the paragraph (3 errors with comparatives, 4 errors with adverb clauses, and 3 errors with conditionals. Make your corrections in the space above the error as in the example in the second line). Do not change the meaning.

My name is Rachael and I am studying mechanical engineering at Colorado State because University. My brother is studying journalism although he has always wanted to be a reporter. I think journalism isn’t as difficult engineering. My says engineering is more harder.

So my dad forced me to study engineering, now I am studying it. Well actually, he didn’t really

force me. Since my dad is an engineer he wanted my brother or me to follow him and study

engineering. My brother is very good at writing, therefore, I was chosen to study the same thing

that my dad studied. I have always liked mathematics, and I am very good with numbers. My dad

says that I work hard in college, if I will get a job anywhere in the world. He says that a lot of

people want to hire engineers. I am excited about my future job possibilities, and I hope to work

in many different countries. I know that if I will stay in one place for a long time, I will get

bored. Due to I want to travel, I need to work hard in school. Recently, I have gotten the higher

grades in my classes, so my father is very proud. So I graduate, we will have a party.

ASSESSING SECOND LANGUAGE SKILLS 32

2E) Short Answer (5 points; 1pts completion, 2pts previous grammar, 2pts current grammar) Directions: Write a paragraph (3-4 sentences) to answer the following questions. What causes students to think school is boring? What effects does this feeling (boring) have on

students in school?

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

ASSESSING SECOND LANGUAGE SKILLS 33

Appendix D Achievement Test Answer Key

Name _____________________________________ TOTAL: ______/62pts

AEIN8213: Week 10 Achievement Assessment KEY

The purpose of this test is to assess your knowledge and ability use grammatical structures from level 2 texts. You have 90 minutes to complete this test and all answers should be written on this test. Each section has directions, please read them carefully. Unless otherwise noted, each question on the test is worth 1 point. There are a total of 62 points. CSU Honor Pledge- Write this statement on the lines below and initial. “I will not give, receive, or use any unauthorized assistance.” __________________________________________________________________Initials:_____

Part 1: Knowledge

1) Multiple Choice (13 points) Directions: Circle the letter of the best answer. 1. Who is happier?

John isn’t as happy as Mary is. a. Mary

b. John

c. Neither 2. Circle the correct form to complete the parallel structure in the sentence.

I do not enjoy running, swimming nor ____________. a. played

b. playing

c. plays

3. Circle the correct form of the verb. Paul ____________ when people are mean. a. is hating

b. hates

c. is hated

ASSESSING SECOND LANGUAGE SKILLS 34

1) Multiple Choice (Continued) Directions: Circle the letter of the best answer. 4. Choose the correct articles for the spaces given(∅ = no article).

I read _____ interesting article yesterday. _____ article was about St. Augustine, Florida, _____oldest city in the United States.

a. an / The / the

b. an / The / an

c. the / An / the

d. a / The / ∅

5. Choose the correct articles for the spaces given(∅ = no article). One old city is in Virginia. _____ city name is Jamestown. _____ interesting fact about Jamestown is that it was _____ first British colony.

h. The / ∅ / a

i. The / An / the

j. A / An / ∅

k. ∅ / A / the

6. Circle the correct form of the verb. I __________________your wife’s wonderful spaghetti sauce.

a. want b. am wanted

c. am wanting

7. Choose the sentence with the correct parallel structure. a. Yangping had her pens, got her notebooks, and her bookbag. b. Yangping had her pens, her notebooks and, her bookbag. c. Yangping wanted her pens, notebooks, and remembered her bookbag. d. Yangping had one pen, notebooks, and many bookbags. 8. What is the meaning of the following:

Mary used to smoke. a. Mary smokes now. b. Mary has never smoked in her life. c. Mary smoked in the past, but she does not smoke now. d. Mary smoked in the past, and she still smokes today. 1) Multiple Choice (Continued)

ASSESSING SECOND LANGUAGE SKILLS 35

Directions: Circle the letter of the best answer. 9. Choose the sentence with the same meaning as example sentence.

The buildings destroyed by the fire were very old. a. The buildings which were destroyed were very old. b. The buildings destroyed the fire. c. The fire was very old.

10. Choose the sentence with the same meaning. Students regularly doing the homework understood the lessons better in their

classes. a. Students were doing the homework. b. Students who were doing the homework had better understanding. c. Textbooks understood the lesson better.

11. What is the meaning of the underlined phrase?

When I was a child, I would walk to school. a. One time action

b. Past habit c. Conditional

12. What is the meaning of the underlined phrase?

I have been cooking dinner. a. Past activity

b. Finished action before now

c. Recent activity

13. Which is the correct meaning?

I haven’t been studying. a. Recent activity

b. Past activity

c. Finished action before now

ASSESSING SECOND LANGUAGE SKILLS 36

1) Multiple Choice (Continued) Directions: Circle the letter of the best answer. 14. Who will come to the soccer game?

Either my mother or my father will come to the soccer game. a. My mother b. My father c. Neither d. We don’t know

Part 2: Ability

2A) Completion (7 points) Directions: Fill in the blank with the correct form of the verbs provided in parentheses. Contractions are also possible where applicable. Other acceptable answers are in parenthesis after the item. 1. If tomorrow (be) _is____ a nice day, we might go swimming.

2. People (travel) _would travel___ more in the U.S. if it were less expensive.

3. Fatimah will be sad if her friends (not write) __do not write__ to her. (can’t write)

4. People who live in the city, (have) _______have_______ more possibilities for shopping.

(will have or might have)

5. If I (find) _____found__$1 million on the street, I would not give it to the police. (could

find).

6. If you (not, study) __do not study___ conditionals, you will have problems with grammar.

7. If I were you, I (not, be) ____would/might not be___ late to class for the exam next Friday.

ASSESSING SECOND LANGUAGE SKILLS 37

2B) Sentence Combining (18 points total) 2B.1) Adverb Clauses and Connectives (10 points)(2 pts/answer; 1 pt- meaning, 1 pt- form) Directions: Using the given words/phrases in the parentheses, combine ideas from ‘a’ and ‘b’ to create new sentence(s). Use correct sentence structure and punctuation.

a) My dad is a good cook. (or) My dad is a bad cook. b) I always enjoy the meals he makes. (or) I don’t enjoy the meals he makes.

1. (although) Although my dad is a good cook, I don’t enjoy the meals he makes/ I don’t enjoy

the meals my dad makes although he is a good cook. OR Although my dad is a bad cook, I

always enjoy the meals he makes/ I always enjoy the meals my dad makes although he is a bad

cook.

2. (because) Because my dad is a good cook, I always enjoy the meals he makes/ I always

enjoy the meals my dad makes because he is a good cook. OR Because my dad is a bad cook, I

don’t enjoy the meals he makes/ I don’t enjoy the meals my dad makes because he is a bad cook.

3. (thus) My dad is a good cook. Thus, I always enjoy the meals he makes. OR My dad is a

bad cook. Thus, I don’t enjoy the meals he makes.

4. (because of) Because of my dad’s good cooking skills/abilities, I always enjoy the meals he

makes. / I always enjoy the meals my dad makes because of his good cooking skills/abilities.

Because of my dad’s bad cooking skills/abilities, I don’t enjoy the meals he makes. / I don’t

enjoy the meals my dad makes because of his bad cooking skills/abilities.

5. (so) My dad is a good cook, so I always enjoy the meals he makes./ My dad is a bad cook, so

I don’t enjoy the meals he make.

ASSESSING SECOND LANGUAGE SKILLS 38

2B.2) Adverb Clauses (8 points) Directions: Create a new sentence by combining the sentences using the word given in the parentheses. Keep the meaning the same. 6. I bought food at the grocery store. Then, my dad cooked me dinner. (after)______After I bought food at the grocery store, my dad cooked me dinner._/ My dad cooked me dinner after I bought food at the grocery store. 7. There was no class yesterday. The teacher was sick. (due to)______Due to the teacher’s sickness, there was no class yesterday. / There was no class yesterday due to the teacher’s sickness_

8. I study often at night. Then, I go to bed. (before)________Before I go to bed, I study often at night. / I study often at night before I go to bed. 9. I don’t eat beef. I am a vegetarian. (since)_________Since I am a vegetarian, I don’t eat beef. / I don’t eat beef since I am a vegetarian. 2C) Sentence Creation (8 points) 2C.1) Comparisons (2 points/answer- 1pt- form, 1 pt- meaning) Directions: Use the sentences to write a sentence to make a comparison. Keep the meaning the same. 1. Mohamed doesn’t work hard. Ali works hard. ____Mohamed does not work as hard as Ali / Ali works harder than Mohamed. / Either Ali or Mohamed works hard/ Mohammed works less hard than Ali.

2. Louis is bad at math. Betty is okay at math.

ASSESSING SECOND LANGUAGE SKILLS 39

Tina is good at math. Tina is the best at math. / Louis is the worst at math. / Betty is better than Louis at math. / Tina is better than Betty at math.

3. Ashlee likes to read books. Angela also likes to read books. Angela/Ashlee likes to read books as much as Angela/Ashlee. / Both Ashlee and Angela like to read/ Angela/Ashlee likes to read books the same as Angela/Ashlee. 4. At the beginning, the class was easiest. Later, the class became harder. At the end, the class was the hardest. The class is getting harder and harder.__/ At the beginning, the class was easier than it is now. / The class is harder now than it was before/ The class is becoming more and more hard.

2D) Editing paragraph (10 points) Note for Teachers: Other possibilities for correct answers are included at the bottom of the page. Directions: Find and correct the 10 grammar errors in the paragraph. Make your corrections in the space above the error. Do not change the meaning. My name is Rachael and I am studying mechanical engineering at Colorado State University.

My brother is studying journalism because he has always wanted to be a reporter. I think

journalism isn’t as difficult1 as engineering. My brother agrees and says engineering is2 more

harder. So Because3 my dad forced me to study engineering, now I am studying it. Well actually,

he didn’t really force me. Since my dad is an engineer4, he wanted my brother or me to follow

him and study engineering. My brother is very good at writing. 5Therefore, I was chosen to study

the same thing that my dad studied. I like mathematics and I am good with numbers. My dad

says that 6if I work hard in college, if I will get a job anywhere in the world. He says that a lot of

people want to hire engineers. I am excited about my future job possibilities, and I hope to work

ASSESSING SECOND LANGUAGE SKILLS 40

in many different countries. I want to see the world. I know that if I 7will stay in one place for a

long time, I will get bored. Due to8 the fact that I want to travel, I need to work hard in school.

Recently, I have gotten the 9highest grades in my classes, so my father is very proud. 10Whenever

I graduate, we will have a party.

Paragraph editing correction notes

Do not take off points if students correct parts of the paragraph that are not incorrect. In addition to the correct answers highlighted in the paragraph, here are some other answers that should be considered correct. 2. More difficult, more challenging etc. 3. Since my dad forced me…

My dad forced me to study engineering, so now I am studying it. 5. ...good at writing; therefore, I was…/ good at writing, so, I was …

7. ...if I stayed in one place...I would get bored

8. Since or Because I want to travel, …

10 After or other time phrase

ASSESSING SECOND LANGUAGE SKILLS 41

2E) Short Answer (10 points) Directions: Write a paragraph (3-4 sentences) to answer the questions. RUBRIC FOR WRITTEN ASSIGNMENTS IN GRAMMAR

1pt - Fully developed and logical response to the prompt (instructions/questions) which shows understanding of the assignment. 2pts- Correct use of previous grammar structures from previous levels (subject/verb, plurals, word forms, punctuation, capitals, tenses etc. ). 2pts- Correct form and accurate use of current grammatical structures focus*. TOTAL ______/ 5points each

Current grammatical structures = All grammatical structures on Table of specifications for Ability. What causes students to think school is boring? What effects does this have on students in

school?

ASSESSING SECOND LANGUAGE SKILLS 42

Appendix E

Score Reporting Form

Week 10 Grammar Achievement Assessment

Name of Student_________________________________

Part of test points/points

possible per part

Percentage correct

Part 1. Knowledge/ Form and Meaning Multiple choice/ Mixed

/14 %

Part 2A. Ability: Completion (conditionals)

/7 %

Part 2B.1 + 2. Ability: Sentence combining: Adverb clauses and connectives

/18 %

Part 2C. Ability: Sentence Creation/ Comparisons

/8 %

Part 2D. Ability: Paragraph Editing/ Mixed grammar

/10 %

Part 2E. Ability: Paragraph Editing/ Adverb clauses (Cause and Effect)

/5 %

Total

/62 %

Score needed to pass 74.5 percent Your score ___________________ (out of 62)