Assessing French Language Skills: Creating the...

Running Head : ASSESSING FRENCH LANGUAGE SKILLS: FINAL PROJECT

Assessing French Language Skills: Creating the Horizons Chapter 9 Test

Anne Sheriff

Colorado State University

Assessing French Language Skills 1

Assessments at the end of an instructional unit have remained important in second and

foreign language instruction as a way to determine the extent that instructional goals for that unit

have been achieved (Miller, Linn & Gronlund, 2009). Continuing with this tradition, this

assessment is a summative assessment to be administered at the end of Chapter 9 in Horizons

(2010), the textbook currently used in first year French classes at Colorado State University.

Jamieson (2011) explains “the function of summative assessments is mainly reporting on

students’ achievement” (p. 3). The function of this assessment will be to show student

achievement on a chapter whose theme is travelling and vacations. In order to incorporate all of

the grammar and vocabulary competencies discussed in this chapter, this will be an integrated

skills test. Students will provide written output to listening and reading input. As the French

107 syllabus (2014) states, “the exams encourage open-ended writing as well as discrete-point

responses to grammar and vocabulary questions.” This test uses a format (the types of tasks, the

order of tasks) similar to the other tests students have taken so that it blends seamlessly into the

pre-established curriculum in the Foreign Languages and Literatures department at Colorado

State University. By the time students in French 107, a second semester French course, take this

exam, they should have witnessed and taken ten exams with this format.

In the Horizons textbook (2010), “the four Compétences…each contain three parts.”

Each content area (grammar skills, vocabulary set, reading strategy) is assessed by this test. It

also “emphasizes the structures needed for common communication situations” (2010, p. IG-11),

as in Chapter 9, where the communication situation is vacationing. Because students often report

that they are studying French so that they can travel in French-speaking countries, their results on

this particular travel-themed assessment should be of particular interest to them.


Bachman and Palmer (2010) point out that “certain assessment tasks that presuppose

cultural or topical knowledge on the part of test takers may be easier for those who have that

knowledge and more difficult for those who do not” (p. 41). Even though students may already

have knowledge about travelling overseas, before instruction, they are lacking the necessary

vocabulary and skills to communicate in French with people in the travel industry. In addition,

all of the cultural information necessary to pass this exam (cities, countries, tourist attractions),

were mentioned during instruction to reduce any sort of topical knowledge interference, focusing

around travel in French overseas departments and French-speaking countries.

This paper provides a detailed report of the development of this unit test for French

students at Colorado State University. First, we provide a description of the test, including

information on its purpose and the individual tasks among others. Next, we describe the pilot

administration of the test, including information of the participants and the scoring procedure.

Third, the results of this pilot administration are summarized and analyzed. Finally, we critique

and evaluate the overall test performance and usefulness.

Test Description

Test Purpose

This test serves a summative purpose to gauge learning at the end of a unit. Through this

exam, an instructor should be able to infer whether or not students have mastered the skills

presented in Chapter 9 of the Horizons (2010) textbook. After the test, the scores are averaged

with the other four test scores in the class. This average counts as 35% of the students’ grade in

the class. If there are any major problem areas on the test, they will be briefly mentioned in

class. The instructor as well as the student can use scores in specific areas of the test to

determine what will need to be reviewed before the final. This assessment is low-stakes, as it


will not have a huge impact on the individual test takers. Due to the weight of grades in this

class, this one assessment will not be the only deciding factor on whether or not a student passes

the class. For the instructor, this test may help them improve their instruction of the content in

the future and to create a list of vocabulary and grammar to review before the final exam. While

this assessment itself will have little impact on the French section of the Foreign Languages and

Literatures department, the designing of this test could influence the ways in which future unit

tests are developed.

Type of Test and Interpretation of Scores

As this test is an achievement test, it measures the knowledge of specific course content

and what a student has learned (Miller, Linn & Gronlund, 2009). It is syllabus-based and

criterion-referenced. It provides a description of the specific performance of the learning tasks

presented within Chapter 9.

Target Language Use Domain

As this chapter is based on the theme of travelling, the target language use domain also

falls into this category of explaining a future trip. Within this domain, there are two language

use tasks that this test will attempt to reflect: 1) communicating with a travel agent about details

of a trip and 2) explaining a trip to friends and family. In both of these cases, two people are

having a conversation about a future travelling event. In the conversation with the travel agent,

the language is quite formal. The travel agent directs the conversation asking questions about

what the traveler wants to do. In the other language use domain, the language is less formal.

The traveler can talk about what they will do while the friend asks questions, or the traveler can

write to a friend to explain what they are going to do. However, the traveler can also ask the


friend for advice and suggestions. They can also promise to write or call while they are

travelling. These two language use domains are delineated in Appendix A.

Construct Definition

The construct this test represents is based on the syllabus of the First Year French II class,

which requires students to learn the content of Chapter 9 in Horizons (2010). The skills

represented and assessed by this chapter include listening, reading and writing. Grammatical

knowledge will also be assessed, as students will be expected to use proper syntactic structures

as well as vocabulary knowledge. Their failure to do so will result in loss of points. Textual

knowledge will also be tested, especially in the writing portions of the exam, where non-cohesive

responses will cause the loss of points. The ideational and manipulative functions as well as

strategic competence will be assumed. Sociolinguistic knowledge will not be included.

Table of Specifications

As Jamieson (2011) explains, “one method for displaying overall test design is to create a

matrix, called a ‘Table of Specifications’” (p. 6). This table’s purpose is to help test developers

(such as teachers) avoid 1) leaving out key material when creating tests based on books, and 2)

overemphasizing relatively unimportant material (Miller, Linn & Gronlund, 2009). For these

reasons, a table of specifications was created to help organize and develop this test.

Since Horizons (2010) puts the greatest emphasis on the writing skill, this skill comprises

the most points on the exam. However, Chapter 9 also addresses the listening and reading skills.

For the purposes of this test, these two skills provide the input for students’ written responses.

This chapter presents five new content areas and structures: travel vocabulary, the simple future,

object pronouns, new verbs related to communication and knowing and prepositions with place


names. All of these areas will be assessed by this test. To see the distribution of points in

relation to each structure and skill, see the Table of Specification in Appendix B.

Description of Test Tasks

This test is made up of 8 different tasks that address the skills and objectives presented in

the table of specifications. For every task, the instructions are in the target language (French).

Each task will be briefly described in the order it appears on the test under that tasks’ title.

A l’agence de voyage. This test starts with a short listening task to leave students free to

finish the test at their own speed with no need to wait for the listening input. The instructions for

this task explain to students that they should imagine they are at a travel agency and a travel

agent is asking them questions. The input is read by the instructor. Each question is read two

times, and then after the students have heard all of the questions, the instructor reads each

questions a final time to give students the opportunity to verify that their answers respond to the

question. Students are expected use the same tense in their answers that was used in the question

and respond in complete sentences. Scores are based on the content of their answers and the

form of the verb.

Les voyages de Paul. The next task is a gap-fill task where students are expected to read

a short text and supply the missing geographical prepositions. The instructions explain that

students should complete the paragraph with a preposition if needed or an X if nothing is needed.

This choice is based on the context of the sentence and topical knowledge about cities and

countries in Africa and North America. Scores are assigned one point for correct, zero points for

incorrect.

Compréhension de lecture. The next task is a short reading task. In the instructions,

students are asked to read the itinerary and answer questions based on what they have read. In


order to assess grammatical objectives as well as reading, students are expected to respond in

complete sentences. Each question is worth two points: one point for giving the correct answer

and one point for the grammatical accuracy of the answer.

Les vacances de Camille. In this task, students must complete a text using the simple

future. Students are told to choose the correct verb from a list and write it in the future to

complete the text. In this gap-fill exercise, the short reading provides the context for picking the

correct verb from the list. Each verb is used only once and there are no distractor verbs. As

Purpura (2004) explains, when answers require both grammar and vocabulary knowledge, it is

often best to use multiple criterion scoring. Following this logic, students will receive one point

if they choose the correct verb and one point if they use the correct form of the verb they chose

in the future. This type of scoring helps separate out the grammatical and vocabulary aspects of

this task.

A l’aéroport. The next task is another gap-fill exercise, but one that focuses on

vocabulary. Students read a short story and choose words from a word bank to fill in the blanks.

They are also reminded to provide the article when necessary. Students will receive two points

per gap filled: one for choosing the correct word and one for using the correct article, which in

French is part of word knowledge as defined by Nation.

Des connaissances pour voyager. In the next task, students need to choose between the

two verbs in French for ‘to know,’ savoir and connaître, and conjugate them correctly. The

instructions ask students to pick between these two verbs and then to conjugate them correctly in

the present tense, using complete sentences. Each short answer will be worth two points: one

point for picking the correct verb, one point for the correct conjugation.


Répondez aux questions de vos parents. The next task focuses on object pronouns.

Students are told to answer questions by replacing underlined phrases in the questions with either

a direct object or indirect object pronoun. This task gives an example to eliminate any confusion

about which word to replace with object pronouns. This task asks students to use object

pronouns with the past, present and future tenses as the placement of the pronoun changes for

each of these tenses. Student responses should logically answer the questions, maintaining

negation when ‘no’ is given, and transforming subjects between the question and the response.

This item is this task will be worth two points: one point for pronoun choice, one point for the

correct placement.

Un voyage à venir. The last task of this assessment is a writing task. The instructions

tell students that they are going on vacation and they are writing an e-mail to a friend that

explains what they will do during their trip. There is also short list of other requirements along

with this prompt. Students are expected to write a paragraph (at least 7 sentences) that addresses

all of the listed requirements. They are allowed to be as creative as they want as long as it makes

sense. This writing task is out of 20 points and scored with a rubric that gives five points for the

focus and organization of the response, five points for the development of the answer, five points

for grammatical accuracy and five points for using a varied vocabulary.

Item Writing and Revising

As is always important in test development, before this test was administered, several

people looked at the items and provided feedback for revisions. One major change between the

first draft and the final draft is the addition of scoring information for each item so students know

what areas are most important to their responses. A word bank was added to the vocabulary

section to reduce the number of possible answers. Any negative instructions, such as “Don’t


forget agreement if necessary” were changed to positive constructions such as “Use agreement

when it is necessary.” The instructions were made clearer in the “Les vacances de Camille” task

as students were told to use each verb only once. The format of the “Des connaissances pour

voyager” section was a bit unclear, so it was changed into a gap-fill task instead of a short

answer task.

Pilot Test Procedure

Participants

This test was piloted to 26 French language learners at Colorado State University. These

students are all currently enrolled in French 107 in the Foreign Languages and Literature

Department. They all have novice-high proficiency in French. Twenty-four have English as

their L1, one is L1 Spanish, and one is L1 Persian.

Administration

Students were told about the test and its purpose several days before administration.

They were also told that they would receive extra credit in their French 107 class, but that the

amount they received would depend on their score on this test. On the day of the test, after all of

the participants had arrived, they were reminded that their results on this test would not hurt their

grade in their French class. Then the tests were handed out. After giving the participants five

minutes to look over the test and read the instructions, the instructor started the listening portion.

This task took 10 minutes, and participants were given another 35 minutes (50 minutes total) to

complete the test.


Scoring Procedure

The test developer and administrator scored this test using the scoring key (see Appendix

C). Due to lack of time and resources, there was only one rater for the written portion of this

exam.

Test Results

The results of the pilot test were analyzed using SPSS and Excel software.

Item Statistics

The mean and standard deviation for each of the 46 items was calculated. These results

are presented in Appendix D. There were no items that every participant answered 100%

correct. Every item, besides the preposition items, which received one or zero points, and the

letter writing task, which received a score out of 20, received a score out of two points.

Item difficulty and the B-index for each item were also calculated (see Appendix D). The

B-index shows how much each item contributes to the pass-fail distinction (Brown, 2003). For

the purposes of this test, we set the cut-off score at 72 out of 100, as this is the Foreign Language

Department’s cut-off score for a D in a course and as the French 107 syllabus (2014) says, “If [a

student] earns less than 73% in the course, [they] do not have the skills necessary to succeed at

the next level,” meaning they have not mastered the objectives of the course or in this case, the

chapter.

Descriptive Statistics

Descriptive Statistics for the test as a whole were also calculated. All scores are out of

100 points. The overall mean for the test is 78.2. The minimum score was 41.5. The maximum

score is 98.5. The median score was 80.5. The standard deviation for the exam is 12.9. The

statistics show that the scores are relatively spread out.


Reliability and Standard Error of Measurement

As there was only one rater for and only one version of the test, the only reliability that

could be calculated was Cronbach’s alpha, a measure of internal consistency. It measures the

correlation between items. For this test, Cronbach’s alpha was 0.82. According to Chapelle

(2013), reliabilities above .90 are very good, and those below .70 indicate scores contain lots of

error. As the reliability for this exam is between those two numbers, we consider this test to

have good reliability. Because it is not a high-stakes exam, such as the TOEFL, we are not

concerned that this number is below .9. In fact, a Cronbach’s alpha above .75 is considered good

for classroom tests such as this one.

The standard error of measurement for this test is 5.4. This value creates a large range of

true scores for this exam. If thinking in terms of percentages and Colorado State University’s

letter scale, with this standard error of measurement, each student’s true score could potentially

be the equivalent of a letter above or below the score they received on the exam.

Description of Masters/Non-Masters

As previously stated, the cut-off score for this exam was 72 points as students who

receive less than a 72% in the class do not have the language knowledge to continue to advance

in French classes. The students above this level are able to read, write and understand oral

speech of the skills assessed by this exam. None of the students who scored above this level

receive zero points on any one task, meaning they all had a certain level of language ability in all

of the skills.


Discussion

Item Performance

Overall, most items performed as expected. Looking at the B-Index for all of the items,

there are a few with negative-values, meaning that the group of students who did not meet the

cut-off score performed better than the students that did. If this test were to be revised, these

items should be deleted and replaced with others. In examining the item difficulty, it is evident

that there are not many items that were incredibly difficult and quite a few that were incredibly

easy for most students.

In addition, the relative facility of this exam was shown by the speed with which students

finished this exam. Compared to other unit tests that these participants have taken, including the

original Chapter 9 test this test is intended to replace, this test took significantly less time, with a

majority students completing the test in less than forty minutes. If this test were to be revised,

adding more items should be considered.

Evaluation of Test Usefulness

Reliability. The internal consistency of this test is appropriate for its purpose as it is not

a high-stakes test. This value of internal consistency shows the degree to which the items in the

assessment are measuring similar characteristics (Miller, Linn & Gronlund, 2009), in this case,

language learning. Inter-rater reliability could not be calculated as there was only one rater;

however, using multiple raters and calculating this measure would have been useful for the

writing task found on this exam as a way to show the reliability of the rubric used. For

subjective tasks, such as the final task on this assessment, it is preferable to have multiple trained

raters and to check for inter-rater reliability (Bachman & Palmer, 2010).


Construct related evidence of validity. Construct related evidence of validity is

demonstrated by the fact that students’ scores on this exam were relatively consistent with their

scores on previous exams in this course. The participants who scored high on this exam usually

have high scores on all of their French exams and, vice versa, students who usually have low

scores had low scores. This correlation helps to show that the construct actually is assessed by

this exam without other interference. In addition, as the construct definition contains a wide

range of language knowledge characteristics, this exam has a high level of interactiveness.

This test also showed a certain level of authenticity. There is a degree of overlap between

the TLU domain and the test. Bachman and Palmer (2010) explain that the more similarities

between the test and the actual language use (represented by the TLU domain), the more certain

the instructor can be about the inferences being made. Students are asked to answer realistic

questions from travel agents. They obtained information from an authentic travel itinerary.

They answered questions using pronouns as they would in conversation and writing. They had

to write emails to friends, and as many of the participants currently have French pen pals that

they regularly send emails to, they could see the purpose of this task.

Consequential evidence of validity. There is little evidence of consequential validity for

this test, because feedback from the students was not officially obtained. However, when going

over the results of this exam with students, it was observed that they were using their results to

start thinking about what sorts of things they needed to review before the final exam. Cumming

(2013) explains that consequential validity has to do with the effects of the exam on all of the

stakeholders, and ensuring the information from and about the assessment is used appropriately.

The instructor for this class began to think about new ways to teach this chapter in the future to


improve students’ learning and retention. No feedback was gathered from administrators as to

the effect.

Practicality. This test is practical to administer. It takes the same amount of time and

effort from the instructor as the unit test that it replaced in the French 107 curriculum at

Colorado State University. In addition, it is relatively easy to score as most tasks require

objective scoring.

Overall Estimation of Test Purpose

This test did achieve its purpose of assessing the achievement of students in Chapter 9 of

Horizons (2010). After this test, an instructor can infer which language skills from this chapter a

student has mastered. There is sufficient proof of reliability for low-stakes test such as this one

as well as proof of validity.

One drawback to this test is its length. There is room to add more items without risking

students running out of time to complete all of the tasks. More items can increase reliability and

help to separate out the masters from the non-masters.


References

Bachman, L. & Palmer, A. (2010) Language Assessment in Practice. Oxford: Oxford

University Press.

Brown, J. D. (2003). Criterion-referenced item analysis (The difference index and B-index).

SHIKEN: The JALT Testing & Evaluation SIG Newsletter, 7(3), 18-24.

Chapelle, C. A. (2013). Reliability in Language Assessment. In C. A. Chapelle (ed.) The

Encyclopedia of Applied Linguistics. Hoboken, NJ: Blackwell Publishing Ltd.

Cumming, A. (2013). Validation in Language Assessment. In C. A. Chapelle (ed.) The

Encyclopedia of Applied Linguistics. Hoboken, NJ: Blackwell Publishing Ltd.

Colorado State University Department of Foreign Languages and Literatures. (2014). LFRE

107 First Year French II Syllabus.

Jamieson, J. (2011). Language assessment and learning. In E. Hinkel (Ed.), Handbook of

second language teaching and research.

Manley, J. H., Smith, S., McMinn, J. T., & Prévost, M. A. (2010). Horizons (5th ed.). Boston,

MA: Heinle.

Miller, M. D., Linn, R. L. & Gronlund, N. E. (2009). Measurement and Assessment in Teaching

(10th ed.). Upper Saddle River, NJ: Pearson.

Nation, I. S. P. (2001). Learning Vocabulary in Another Language. Cambridge: Cambridge

University Press.

Purpura, J. E. (2004). Assessing Grammar. Cambridge: Cambridge University Press.


Appendix A

TLU Task Characteristics

TLU task #1: Someone is planning on taking a trip and talks to a travel agent about their plans Characteristics of the setting

physical characteristics travel agency

participants traveler, travel agent

time of task day

Characteristics of the test rubric

instructions

language

channel

specification of procedures and tasks

structure

time allotment

scoring method

criteria for correctness

procedures for scoring the response

explicitness of criteria and procedures

Characteristics of the input

format

channel aural and visual

form both language and non-language

language target language

length short conversational exchanges

type series of “items”

degree of speededness normal speed (~160-180 wpm)

vehicle live

language of the input

language characteristics

organizational characteristics

grammatical full set of phonological features, vocab related to travelling and countries, statements & questions

textual short series of questions and answers—conversation

pragmatic characteristics

functional ideational and manipulative

sociolinguistic formal, polite, natural

topical characteristics travelling—countries, cities, air travel, activities, possibilities

Characteristics of the expected response

format


form language (some non-language)

language target




language of the expected response



grammatical full set of phonological features, vocab related to travel desires, statements & questions,

textual short series of responses and questions—conversation


functional ideational and manipulative

sociolinguistic formal, polite, natural

topical characteristics travelling—countries, cities, air travel, activities, choices

Relationship between input and response

reactivity reciprocal

scope of relationship narrow

directness of the relationship direct


TLU task #2: Someone is planning on taking a trip and tells their friend what they are going to

do Characteristics of the setting

physical characteristics anywhere two friends could meet (restaurant, café, home)

participants friends

time of task anytime (night, day)

Characteristics of the test rubric

instructions

language

channel

specification of procedures and tasks

structure

time allotment

scoring method

criteria for correctness

procedures for scoring the response

explicitness of criteria and procedures

Characteristics of the input

format

channel aural, maybe some visual

form both language and non-language

language target language




vehicle live

language of the input



grammatical full set of phonological features, vocab related to travelling and countries, statements & questions

textual majority narration with the possibility for questions and responses


functional ideational

sociolinguistic informal, colloquial, natural, idiomatic

topical characteristics travelling—countries, cities, air travel, activities

Characteristics of the expected response

format


form language (some non-language)

language target




language of the expected response



grammatical full set of phonological features, vocab related to travel desires, statements & questions,

textual responses and reactions to story, possibility to ask questions


functional ideational

sociolinguistic informal, colloquial, natural, idiomatic

topical characteristics travelling—countries, cities, air travel, activities

Relationship between input and response

reactivity reciprocal

scope of relationship narrow

directness of the relationship direct


Appendix B

Table of Specifications

Horizons (2010) Chapter 9 Test

Structures/Skills Listening +

Writing

Reading +

Writing Writing # points % points

Travel

Vocabulary 6 5 17 28 28

Simple Future 3 4 20 27 27

Object Pronouns 0 0 10 10 10

Verbs 2 0 20 22 22

Geographical

Prepositions 1 1 11 13 13

# points 12 10 78 100

% points 12 10 78 100


Appendix C

Test, Scoring Key and Answer Sheet

Examen Chapitre 9 Votre nom_____________________

LFRE 107 Printemps 2014 (version 2)

I. À l’agence de voyages. Imaginez que vous êtes à l’agence de voyages et que l’agent vous

pose des questions. Ecrivez vos réponses à ses questions en faisant des phrases complètes.

Vous aurez un point pour des réponses logique et un point pour la grammaire. (12 points)

1.___________________________________________________________________________

2.___________________________________________________________________________

3.___________________________________________________________________________

4.___________________________________________________________________________

5.___________________________________________________________________________

6.___________________________________________________________________________

II. Les voyages de Paul. Paul partira bientôt en vacances et il imagine les endroits qu’il aimerait

visiter. Remplissez les blancs avec la préposition correcte. Si vous n’avez pas besoin de

préposition, écrivez X. (10 points)

Paul habite ______ Montpellier, une jolie ville _____ France. Il n’a jamais quitté son pays, mais

le mois prochain, il va faire le tour du monde. D’abord, il a l’intention d’aller _____ Afrique. Il

va aller _____ Abidjan _____ Côte d’Ivoire, où il passera une semaine. Ensuite, il restera deux

semaines _____ Sénégal. Après, il va aller _____ Etats-Unis ; il a envie d’aller _____ Miami et

_____ Las Vegas. Il voudrait aussi visiter _____ Montréal parce qu’il peut y parler français.

Après tout cela, il aura encore un mois de vacances pour aller visiter d’autres continents !


III. L’itinéraire de Camille. L’agent de voyages donne à Camille son itinéraire. Lisez

l’itinéraire et répondez aux questions suivantes en faisant des phrases complètes. Vous aurez 1

point pour la bonne réponse et 1 point pour la grammaire (10 points)

1. Où est-ce que Camille partira en vacances ?

2. Quand partira-t-elle ?

3. Combien de valises peut-elle prendre ?

4. En quelle classe voyagera-t-elle ?

5. Selon l’itinéraire, qu’est-ce qu’il faut faire avant le départ ?


IV. Les vacances de Camille. Camille rêve de ce qu’elle fera au Maroc. Choisissez le bon

verbe de la liste ci-dessous et écrivez-le au futur. Vous utiliserez chaque verbe une fois. Vous

aurez 1 point pour le choix du verbe et 1 point pour la conjugaison (20 points).

ECRIRE ALLER VISITER FAIRE CONNAITRE

FINIR LIRE VOIR POUVOIR AVOIR

Camille _________________ ses études cette année et elle voudrait faire un voyage au

Maroc avec ses amis. Elle se dit : « J’________________ beaucoup de temps libre cet été

pour faire un beau voyage ! »

Elle se retrouve avec son ami Barney au café pour parler de son voyage. Barney lui

demande : « Est-ce que tu _________________ des guides touristiques avant de partir ? Tu

penses que tu _______________ La Médina à Fès ? » Au Maroc, elle _____________

parler français et arabe et elle _______________ au restaurant pour goûter la cuisine locale.

Camille va rencontrer une amie marocaine, Aïcha, et elle dit à Barney : « Nous

________________ des randonnées dans le désert et nous _________________ des

chameaux (camels). Nous _______________ aussi des cartes postales à nos familles. Après

tous ces aventures, on ______________________ un peu plus de l’histoire et de la culture du

Maroc. »

V. A l’aéroport. Choisissez un mot correcte de la liste pour compléter l’histoire ci-dessous.

N’oubliez pas de rajouter un article quand il est nécessaire! Vous aurez un point pour le

choix du mot et un point pour le choix d’article (8 points).

VACANCES CARTE DE CRÉDIT AÉROPORT PASSEPORT

GUIDE TOURISTIQUE GUADELOUPE DOUANE EUROS

Mon ami Barney et moi avons envie de nous reposer, alors nous allons partir en

______________ au Maroc. Nous achetons notre billet sur Internet, donc nous utilisons

___________________ pour payer sans utiliser d’argent liquide (cash). Nous allons à

___________________ pour prendre l’avion. Là, nous devons avoir une pièce d’identité

pour prendre l’avion, donc nous montrons notre ___________________ à l’agent de la

compagnie aérienne.


VI. Des connaissances pour voyager. Choisissez entre le verbe savoir ou connaître et

remplissez les blancs pour compléter les phrases. Vous aurez 1 point pour le choix du verbe

et 1 point pour la conjugaison (10 points)

1. Est-ce que vous _________________ parler arabe ?

2. Nous _____________________________la France.

3. Les américains ne ______________________ pas le président de l’Allemagne?

4. Je ___________________________les guides de Lonely Planet ?

5. Le jeune homme _____________________ où se trouve l’Australie.

VII. Répondez aux questions de vos parents. Remplacez les mots soulignés par un

pronom d’objet direct ou un pronom d’objet indirect. Accordez (use agreement) quand il

est nécessaire ! Vous aurez 1 point pour le choix du pronom et 1 point pour son placement.

(10 points) Un exemple a été fait.

Exemple : Est-ce que tu manges le croissant ? = Je le mange.

1. Est-ce que tu lis une histoire à ton petit frère ?

___Oui________________________________________________________________

2. Ton petit frère et toi, donnez-vous de bonnes nouvelles à vos parents ?

___Non________________________________________________________________

3. Est-ce que vous allez écrire des lettres à Camille quand vous allez être en vacances ?

___Oui________________________________________________________________

4. Est-ce qu’elle dit toujours la vérité (the truth) ?

___Non________________________________________________________________

5. Ton professeur a-t-elle lu ta lettre à tous les élèves ?

___Oui________________________________________________________________


VIII. Un voyage à venir. Imaginez que vous allez bientôt partir en voyage. Vous écrivez un

mail ou une lettre à un ami pour expliquer ce que vous avez l’intention de faire pendant ce

voyage. Votre note pour cette section reflètera le contenu de votre lettre et la qualité de votre

grammaire (20 points)

Votre mail/lettre doit :

Employer le futur simple.

Employer au moins (at least) une phrase avec si… (si je ________ je ________) OR

quand…(quand je ______, je_________)

Dire où vous allez voyager.

Décrire au moins trois activités que vous ferez pendant ton voyage

Écrivez au moins 7 phrases.

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________


Script for the Listening Task

I. A l’agence de voyage

1. Dans quel pays est-ce que vous voudriez aller ?

2. Qu’est-ce que vous ferez pendant vos vacances?

3. Savez-vous parler la langue de ce pays ?

4. Quand est-ce que vous voyagerez ?

5. Désirez-vous un billet aller-retour ou un aller simple ?

6. Est-ce que vous écrirez des mails ou des cartes postales à vos amis ?

Translation of Script for the Listening Task

I. At the travel agency. Read each question two times. Then repeat all the questions a

third time.

1. To which country would you like to go?

2. What will you do there during your vacation?

3. Do you know how to speak the language of this country?

4. When will you leave?

5. Do you want a roundtrip ticket or one way?

6. Will you write emails or post cards to your friends?


Scoring and Answer Key for Chapter 9 Test

Examen Chapitre 9 Votre nom_____________________

LFRE 107 Printemps 2014

I. À l’agence de voyages. Vous êtes à l’agence de voyages et l’agent vous pose des questions.

Répondez aux questions en faisant des phrases complètes. (12 points)

1. _Je voudrais aller au/en + pays____

2. . _Je + activité au futur__________

3. __Je (ne) sais (pas) parler la langue du pays_

4. __ Je partirai + date__

5.___Je désire un billet aller-retour/un aller simple____

6.____Je (n’) écrirai (pas) de/des mails ou des cartes postales à mes amis________

II. Les voyages de Paul. Paul partira bientôt en vacances et il imagine les endroits qu’il aimerait

visiter. Remplissez les blancs avec la préposition correcte. Si vous n’avez pas besoin de

préposition, écrivez X. (10 points)

Paul habite ___à___ Montpellier, une jolie ville __en___ France. Il n’a jamais quitté son pays,

mais le mois prochain, il va faire le tour du monde. D’abord, il a l’intention d’aller __en___

Afrique. Il va aller ___à__ Abidjan __en___ Côte d’Ivoire, où il passera une semaine. Ensuite, il

restera deux semaines __au___ Sénégal. Après, il va aller _aux___ Etats-Unis ; il a envie d’aller

__à___ New York et __à___ Las Vegas. Il voudrait aussi visiter __X___ Montréal parce qu’il

peut y parler français. Après tout cela, il aura encore un mois de vacances pour aller visiter

d’autres continents !

1 point logical response

½ point tense

½ point conjugation

1 point correct response


III. Compréhension de lecture. L’agent de voyages donne à Camille son itinéraire. Lisez

l’itinéraire et répondez aux questions suivantes en faisant des phrases complètes. (10 points)

1. Où est-ce que Camille partira en vacances ?

Elle partira à Rabat/au Maroc/à Rabat au Maroc.

2. Quand partira-t-elle ?

Elle partira le jeudi 26 février.

3. Combien de valises peut-elle prendre ?

Elle peut prendre 2 valises.

4. En quelle classe voyagera-t-elle ?

Elle voyagera en classe touriste.

5. Selon l’itinéraire, qu’est-ce qu’il faut faire avant le départ ?

Il faut arriver à l’aéroport deux heures à l’avance.

1 point correct answer to the

question

1 point grammar


IV. Les vacances de Camille. Camille rêve de ce qu’elle fera au Maroc. Choisissez le bon

verbe de la liste ci-dessous et écrivez-le au futur simple pour

compléter le texte. (20 points)

ECRIRE ALLER VISITER FAIRE CONNAITRE

FINIR LIRE VOIR POUVOIR AVOIR

Camille ___finira/fera________ ses études cette année et elle voudrait

faire un voyage au Maroc avec ses amis. Elle se dit : « J’___aurai____

beaucoup de temps libre cet été pour faire un beau voyage ! »

Elle se retrouve avec son ami Barney au café pour parler de son voyage. Barney lui

demande : « Est-ce que tu ____liras_____ des guides touristiques avant de partir ? Tu penses

que___visiteras/verras______ La Médina à Fès ? » Au Maroc, elle ____pourra______

parler français et arabe et elle ____ira_____ au restaurant pour goûter la cuisine locale.

Camille va rencontrer une amie marocaine, Aïcha, et elle dit à Barney : « Nous

___ferons_______ des randonnées dans le désert et nous ____verrons/visiterons______ des

chameaux (camels). Nous __écrirons______ aussi des cartes postales à nos familles. Après

tous ces aventures, on _____connaîtra________ un peu plus l’histoire et la culture du

Maroc. »

V. A l’aéroport. Choisissez un mot correcte de la liste pour compléter l’histoire ci-dessous.

N’oubliez pas de rajouter un article quand il est nécessaire! Vous aurez un point pour le

choix du mot et un point pour le choix d’article (8 points).

VACANCES CARTE DE CRÉDIT AÉROPORT PASSEPORT

GUIDE TOURISTIQUE GUADELOUPE DOUANE EUROS

Mon ami Barney et moi avons envie de nous reposer, alors nous allons

partir en __vacances____ au Maroc. Nous achetons notre billet sur

Internet, donc nous utilisons ___une carte de crédit_____ pour payer

sans utiliser d’argent liquide (cash). Nous allons à

___l’aéroport_______ pour prendre l’avion. Là, nous devons avoir une

pièce d’identité pour prendre l’avion, donc nous montrons notre ____passeport______ à

l’agent de la compagnie aérienne.

1 point verb choice

1 point verb form

1 point Word choice

1 point article


VI. Des connaissances pour voyager. Choisissez entre le verbe savoir ou connaître et

remplissez les blancs pour compléter les phrases. Vous aurez 1 point pour le choix du verbe

et 1 point pour la conjugaison (10 points)

6. Est-ce que vous ____savez____ parler arabe ?

7. Nous ____connaissons________la France.

8. Les américains ne ___connaissent___ pas le

président de l’Allemagne?

9. Je ___connais____les guides de Lonely Planet ?

10. Le jeune homme ___sait____ où se trouve l’Australie.

VII. Répondez aux questions de vos parents. Remplacez les mots soulignés par un pronom

d’objet direct ou un pronom d’objet indirect. Accordez (use agreement) quand il est

nécessaire ! Vous aurez 1 point pour le choix du pronom et 1 point pour son placement. (10

points) Un exemple a été fait.

Exemple : Est-ce que tu manges le croissant ? = Je le mange.

1. Est-ce que tu lis une histoire à ton petit frère ?

___Oui je lui lis une histoire________________________________________________

2. Ton petit frère et toi, donnez-vous de bonnes nouvelles à vos parents ?

___Non nous ne leur donnons pas de bonnes nouvelles_________________________

3. Est-ce que vous allez écrire des lettres à Camille quand vous allez être en vacances ?

___Oui je vais lui écrire des lettres._______________

4. Est-ce qu’elle dit toujours la vérité (the truth) ?

___Non elle ne la dit pas toujours.________________

5. Ton professeur a-t-elle lu ta lettre à tous les élèves ?

___Oui elle l’a lue à tous les élèves___________________

1 point choice of pronoun

1 point placement of pronoun

1 point verb choice

1 point conjugation


VIII. Un voyage à venir. Imaginez que vous allez bientôt partir en voyage. Vous écrivez un

mail à un ami pour expliquer ce que vous ferez pendant ce voyage. (20 points)

Employez le futur.

Employez au moins (at least) une phrase avec si… (si je ________ je ________) OR

quand…(quand je ______, je_________)

Dites où vous allez voyager.

Les activités que vous ferez pendant votre voyage. (au moins trois activités)


Rubric for Writing Task (VIII. A l’agence de voyage)

Focus/Organization Development Accuracy Word Choice/Use

5 Brings full understanding Provides substantial Nearly perfect; few Appropriate and

to the question; uses support/examples, errors with current varied vocabulary;

connectors/transitions; all appropriate and and prior grammar spelling, gender, and

varies sentence structure convincing accents nearly perfect

4 Stays mostly focused on topic; Provides some Occasional errors in Mostly appropriate

uses some transition words; support/examples, current and/or prior vocab; occasional

understood question mostly appropriate grammar spelling, gender and/or

fairly well and convincing accent problems; not

very repetitive

3 Does not address all parts Provides weak Frequent error with Frequent vocab errors;

of question; few transitions; support/examples most current and uses some English;

not very focused on topic and/or not enough prior grammar many spelling/gender

of them problems or lack of

accents; repetitive

2 Illogical, disorganized Provides minimal Recurring errors Recurring problems

and unfocused; feels more or irrelevant impede understanding impede understanding

like a list than a paragraph; support/examples or not enough written

only tangentially relevant to show knowledge of grammar

1 No focus or engagement with Provides no examples Unreadable Unreadable

topic; ignored question or support for ideas


Translation of the Test

I. At the travel agency. Imagine you are at a travel agency and the travel agent is

asking you questions. Write your answers to her questions with complete sentences.

You will get one point for a logical response and one point for grammar (12 points).

II. Paul’s trip. Paul will soon go on vacation and he is imagining the places that he

would like to visit. Complete the paragraph with the correct preposition. If you do

not need a preposition, write an X. (10 points).

Paul lives ______Montpellier, a pretty city _____France. He has never left his

country, but next month, he is going to on a round the world trip. First, he is going to

go ______Africa. He is going to go ______Abidjan ______Ivory Coast where he

will spend a week. Then, he will spend two weeks________ Senegal. After, he is

going to go _______ United States; he wants to go _______New York et _______Las

Vegas. He would also like to visit______Montreal because he can speak French

there. After all of this, he will still have a month of vacation to visit other continents.

III. Reading Comprehension. The travel agent gives Camille her itinerary. Read the

itinerary and answer the questions using complete sentences. You will earn one

point for the correct answer and one point for your grammar (10 points)

1. Where will Camille go on vacation?

2. When we she leave?

3. How many suitcases can she take?

4. In what class will she travel?

5. According to the itinerary, what must she do the day of the departure?

IV. Camille’s vacation. Camille has been dreaming of what she will do in Morocco.

Chose the correct verb from the list below and write it in the simple future to


complete the text. You will use each verb once. You will get one point for your verb

choice and one point for its conjugation. (20 points)

WRITE GO VISIT DO KNOW

FINISH READ SEE BE ABLE TO HAVE

Camille______ her studies this year and she would like to take a trip to Morocco with

her friends. She says to herself, “I _________ a lot of free time this summer to take a

lovely trip.”

She meets up with her friend Barney at the café to talk about her trip. Barney asks

her, “Will you_______ guidebooks before you leave? Do you think you_________

La Medina in Fez?” In Morocco, she __________ to speak French et Arabic and she

_______ at restaurants to taste the local cuisine.

Camille is going to meet up with a Moroccan friend, Aicha, and she tells Barney

“We__________hikes in the desert and we__________ camels. We________ post

cards to our families too. After all of these adventures, we __________ a bit more the

history and culture of Morroco.”

V. At the airport. Choose the correct word from the list to complete the story below.

Add an article when it’s necessary. You will get one point for your word choice

and one point for the article (8 points).

VACATION CREDIT CARD AIRPORT PASSPORT

GUIDEBOOK GUADELOUPE CUSTOMS EUROS

My friend Barney and I want to relax, so we are going to go on __________to

Morocco. We buy our ticket online, so we use___________ to pay without using

cash. We go to __________ to catch our flight. There we have to have a form of

identity to get on the place so we show our_________ to the airline company agent.

VI. Knowledge for travelling. Choose between the verb “to know” or “to know” and fill

in the blanks to complete the sentences. You will get 1 point for your verb choice and

one point for the conjugation.

1. Do you __________________ how to speak Arabic?


2. We ____________________ France.

3. Americans do not __________________ the president of Germany.

4. I__________________ Lonely Planet guidebooks.

5. The young man______________________ where Australia is located.

VII. Answer your parents’ questions. Replace the underlined words with a direct object

pronoun or an indirect object pronoun. Use agreement when it’s necessary. You get

one point for the pronoun choice and one point for its placement. (10 points)

An example has be done.

Ex: Are you eating the croissant ? = I eat it.

1. Are you reading a story to your little brother?

Yes…

2. Your little brother and you, do you give good news to your parents?

No…

3. Are you going to write letters to Camille when you are on vacation?

Yes…

4. Does she always tell the truth?

No…

5. Your teacher did she read your letter to all of the students?

Yes…

VIII. An upcoming trip. Imagine that you are soon going to take a trip. You are writing

an e-mail or a letter to a friend in order to explain what you will do during this trip.

Your score for this section will reflect the content of your letter and the quality of

your grammar. (20 points).

Your letter/email must:

Use the simple future tense

Use at least one sentence with

Say where you are going to go.

Describe at least three activities that you will do during your trip.

Write at least 7 sentences.


Appendix D

Item Statistics

L=listening task (I), PP=preposition task (II), R=reading task (III), F=future task (IV), SC= savoir and

connaître task (V), V=vocabulary task (VI), PN=pronoun task (VII), W=writing task (VIII)

Item Mean Standard Deviation Item Difficulty B-Index

L1 1.81 0.25 0.90 0.04

L2 1.71 0.32 0.86 0.11

L3 1.67 0.45 0.84 0.13

L4 1.73 0.45 0.87 0.12

L5 1.38 0.55 0.69 0.23

L6 1.75 0.32 0.88 0.23

PP1 0.92 0.27 0.92 -0.11

PP2 0.96 0.2 0.96 0.13

PP3 0.81 0.40 0.81 0.26

PP4 0.69 0.47 0.69 0.64

PP5 0.73 0.45 0.73 0.33

PP6 0.88 0.33 0.88 0.19

PP7 0.88 0.33 0.88 0.38

PP8 0.81 0.40 0.81 -0.1

PP9 0.54 0.51 0.53 0.06

PP10 0.19 0.40 0.19 0.28

R1 1.67 0.55 0.84 -0.01

R2 1.74 0.3 0.87 -0.01

R3 1.90 0.25 0.95 0.02

R4 1.78 0.30 0.89 0.13

R5 1.69 0.31 0.85 0.07

F1 1.38 0.7 0.69 0.46

F2 1.31 0.74 0.65 0.49

F3 1.50 0.71 0.75 0.36

F4 1.35 0.69 0.67 0.52

F5 1.19 0.63 0.6 0.41

F6 1.35 0.8 0.67 0.43

F7 1.31 0.79 0.65 0.31

F8 1.23 0.82 0.62 0.26

F9 1.42 0.64 0.71 0.31

F10 1.27 0.60 0.63 0.38

SC1 1.77 0.51 0.88 0.10

SC2 1.62 0.5 0.81 0.26

SC3 1.46 0.58 0.73 0.24

SC4 1.27 0.60 0.63 0.10

SC5 1.5 0.58 0.75 0.36

V1 1.73 0.45 0.87 -0.01

V2 1.40 0.49 0.70 0.25

V3 1.73 0.53 0.87 0.17

V4 1.48 0.5 0.74 0.12

PN1 1.81 0.49 0.90 0.22

PN2 1.5 0.65 0.75 0.36

PN3 1.23 0.59 0.62 0.17

PN4 1.69 0.47 0.85 0.05

PN5 1.11 0.52 0.56 0.26

W1 17.08 4.55 0.85 0.26

Assessing French Language Skills: Creating the...

Documents

Transcript of Assessing French Language Skills: Creating the...