Writing processing in a trial Writing processing in a trial version of the Computer version of the Computer

Based University Entrance Based University Entrance Examination: A way to Examination: A way to

motivate curriculum change motivate curriculum change in Spainin Spain

Dr. Jesús García Laborda, Universidad Dr. Jesús García Laborda, Universidad AlcalaAlcala

Dr. Teresa Magal Royo, U. Politecnica Dr. Teresa Magal Royo, U. Politecnica ValenciaValencia

Dr. Emilia Enríquez Carrasco, U. Dr. Emilia Enríquez Carrasco, U. Politecnica ValenciaPolitecnica Valencia

… In this presentation…In this presentation…

IntroductionIntroduction Computer processes: Observation Computer processes: Observation

results results Educational changes associated Educational changes associated

to the trialling stageto the trialling stage ConclusionsConclusions

INTRODUCTIONINTRODUCTION Spanish Government is researching Spanish Government is researching

implications of implementing an Internet implications of implementing an Internet based language test for the national University based language test for the national University Entrance Examination (PAULEX project Entrance Examination (PAULEX project HUM2007-66479-C02-01/FILO). (Washback)HUM2007-66479-C02-01/FILO). (Washback)

The team found that their main concern The team found that their main concern should be the writing and speaking tasks. should be the writing and speaking tasks.

Extensive study in relation to oral tasks has Extensive study in relation to oral tasks has been done by testing companies such as the been done by testing companies such as the Educational Testing Service (TOEFL) and NCS Educational Testing Service (TOEFL) and NCS Person e-Measuremet Services (Pearson PTE).Person e-Measuremet Services (Pearson PTE).

INTRODUCTIONINTRODUCTION

In the research the main concern was In the research the main concern was posed on the written tasks. Reasons: posed on the written tasks. Reasons: (1) the oral tasks have not implemented (1) the oral tasks have not implemented

in the University Entrance Examination in the University Entrance Examination yet (scheduled for 2012),yet (scheduled for 2012),

(2) extensive trialling is necessary and (2) extensive trialling is necessary and given the current funds, it may take two given the current funds, it may take two years to test a minimal sample to take years to test a minimal sample to take decisions.decisions.

Research (1)Research (1)

in November 2009, 260 students in November 2009, 260 students took a computer based version of the took a computer based version of the University Entrance Examination in University Entrance Examination in Valencia (Spain). Valencia (Spain).

Two main assessments: a holistic Two main assessments: a holistic one and an error analysis. one and an error analysis.

Research (1)Research (1) Both analysis indicated a number of issues:Both analysis indicated a number of issues: The holistic The holistic results of both the group and the results of both the group and the

individuals was not very different from the results individuals was not very different from the results obtained by the candidates who took the same test obtained by the candidates who took the same test only a few months before;only a few months before;

Students were happy about the testing platform Students were happy about the testing platform and felt that it was a reliable way to measure their and felt that it was a reliable way to measure their language proficiency;language proficiency;

Errors due to computer use and typos were a small Errors due to computer use and typos were a small percentage of the total number of errors and the percentage of the total number of errors and the average word count was also similar to the control average word count was also similar to the control test (the one that was used.test (the one that was used.

Computer processes: Computer processes: Observation results Observation results

Two 1 hour sessions were recorded. Two 1 hour sessions were recorded. The writing speed and their body The writing speed and their body language expressions were also language expressions were also annotated.annotated.

Computer processes: Computer processes: Observation resultsObservation results

three main conclusions:three main conclusions: Students showed no sign of additional difficulty. Students showed no sign of additional difficulty.

Students did not require much support to work. Students did not require much support to work. Most assistance, in fact, was provided to Most assistance, in fact, was provided to support the students with the software support the students with the software deficiencies (mostly due to connectivity);deficiencies (mostly due to connectivity);

Students worked regularly and typed at an Students worked regularly and typed at an adequate speed to fulfil the main written adequate speed to fulfil the main written task in the time given;task in the time given;

Their keyboard use was adequate, at least, Their keyboard use was adequate, at least, for the tasks they were assigned.for the tasks they were assigned.

Educational changes Educational changes associated to the associated to the

trialling stagetrialling stage Evidence to support the implementation of Evidence to support the implementation of

the Computer based University Entrance the Computer based University Entrance examination with a few premises:examination with a few premises:

It is necessary to motivate both teachers and It is necessary to motivate both teachers and students to accept, understand and integrate students to accept, understand and integrate the changethe change. García Laborda (2010) verified . García Laborda (2010) verified that although teachers may be reluctant to that although teachers may be reluctant to change at the beginning, they may also be change at the beginning, they may also be willing to take the necessary training that willing to take the necessary training that may help them to understand and accept their may help them to understand and accept their attitudes’ change.attitudes’ change.

Educational changes Educational changes associated to the associated to the trialling stage (2)trialling stage (2)

Effective and simple applications are Effective and simple applications are necessary to motivate studentsnecessary to motivate students. Simplicity . Simplicity in interface and flow design may have a in interface and flow design may have a significant effect in students accepting the significant effect in students accepting the application. If writing was considered to be application. If writing was considered to be the most troublesome aspect in the change, the most troublesome aspect in the change, as it was observed, when students feel at as it was observed, when students feel at ease they may be prone to perform, at ease they may be prone to perform, at least, as well in a computer environment as least, as well in a computer environment as they would in a different context (pen and they would in a different context (pen and paper). paper).

Educational changes Educational changes associated to the associated to the trialling stage (3)trialling stage (3)

Motivation should also be an Motivation should also be an important aspect to consider for the important aspect to consider for the changechange. Since foreign language . Since foreign language teachers in Spain tend to be reluctant teachers in Spain tend to be reluctant to integrate computers in writing to integrate computers in writing (although they may be more willing to (although they may be more willing to use it with multiple choice exercises use it with multiple choice exercises or reading activities), it is important or reading activities), it is important to facilitate meaningful training.to facilitate meaningful training.

Educational changes Educational changes associated to the associated to the trialling stage (3)trialling stage (3)

Overall, the results obtained in the Overall, the results obtained in the experimental stage of the project may experimental stage of the project may motivate dramatic educational changes. motivate dramatic educational changes.

Therefore, as observed the teachers’ and Therefore, as observed the teachers’ and students’ reconceptualization may be students’ reconceptualization may be supported by three main pillars: knowledge supported by three main pillars: knowledge of the software, acquisition of working of the software, acquisition of working strategies (both for teaching and learning), strategies (both for teaching and learning), and security that the results will be reliable and security that the results will be reliable and, at least, as good in the new delivery and, at least, as good in the new delivery context. context.

Educational changes Educational changes associated to the associated to the trialling stage (4)trialling stage (4)

Even more important could be these Even more important could be these results in light of their institutional results in light of their institutional effects and the effects in the regular effects and the effects in the regular teaching (washback). Since students teaching (washback). Since students can adapt easily to the new delivery can adapt easily to the new delivery context, it could be plausible believing context, it could be plausible believing that with some economical effort that with some economical effort computer testing can benefit computer testing can benefit economically the educational boards economically the educational boards around Spain on the medium term. around Spain on the medium term.

ConclusionsConclusions

If students are able to adapt themselves to the If students are able to adapt themselves to the written section of computer assisted language written section of computer assisted language tests without previous training, there is little tests without previous training, there is little doubt that they could adapt to the rest of the doubt that they could adapt to the rest of the test. There is also little debate on whether test. There is also little debate on whether computers can be used to improve the computers can be used to improve the assessment process while reducing the costs assessment process while reducing the costs related to testing. This facility should motivate related to testing. This facility should motivate the incorporation of computer based testing the incorporation of computer based testing platforms for the University Entrance platforms for the University Entrance examination. examination.

Thank you for your attentionThank you for your attention

jesus.garcialaborda@uah.esjesus.garcialaborda@uah.es