Full Terms & Conditions of access and use can be found athttps://www.tandfonline.com/action/journalInformation?journalCode=ncal20

Computer Assisted Language Learning

ISSN: 0958-8221 (Print) 1744-3210 (Online) Journal homepage: https://www.tandfonline.com/loi/ncal20

The impact of using machine translation on EFLstudents’ writing

Sangmin-Michelle Lee

To cite this article: Sangmin-Michelle Lee (2019): The impact of using machine translation on EFLstudents’ writing, Computer Assisted Language Learning, DOI: 10.1080/09588221.2018.1553186

To link to this article: https://doi.org/10.1080/09588221.2018.1553186

Published online: 07 Feb 2019.

Submit your article to this journal

Article views: 199

View Crossmark data

The impact of using machine translation on EFLstudents’ writing

Sangmin-Michelle Lee

School of Global Communication, Kyung Hee University, Yongin, South Korea

ABSTRACTAlthough it remains controversial, machine translation (MT)has gained popularity both inside and outside of the class-room. Despite the growing number of students using MT,little is known about its use as a pedagogical tool in theEFL classroom. The present study investigated the role ofMT as a CALL tool in EFL writing. Most studies on MT as atool for L2 learning have focused on student postediting ofthe translation that MT provides; however, the presentstudy employed a different design with students translatingtheir L1 writing into L2 without the help of MT and thencorrecting their L2 writing using the MT translation forcomparison. Text analysis of students’ writing outcomesrevealed that MT helped to decrease lexico-grammaticalerrors and improve student revisions. Using MT for revisionsalso positively affected student writing strategies andhelped them think of writing as a process. The interviewsand reflection papers demonstrated that students viewedthe use of MT during writing positively. This study foundthat MT can be a useful aid to language learning, but for itto benefit student learning, teachers must be aware of itslimitations and provide adequate guidance to students.

KEYWORDSMachine translation; L2writing; revision;writing strategies

Introduction

Machine translation (MT) is not widely used in the academic context,mainly because of concerns about its reliability with regard to whether itcan accurately render the source text in the target language. However,MT technologies have developed considerably during the past decadeand offer significantly improved grammatical and lexical accuracy. MT isdefined as “the process by which computer software is used to translateand [is] compatible with PC systems and smart phone systems”(Alhaisoni & Alhasysony, 2017, p. 73). Since MT became available onboth PCs and mobile phones, it has become increasingly widespread invarious settings because of its convenience, multilingualism, immediacy,

CONTACT Sangmin-Michelle Lee sangminlee@khu.ac.kr� 2019 Informa UK Limited, trading as Taylor & Francis Group

COMPUTER ASSISTED LANGUAGE LEARNINGhttps://doi.org/10.1080/09588221.2018.1553186

efficiency, and free cost (Alhaisoni & Alhaysony, 2017; Ni~no, 2009;Sukkhwan & Sripetpun, 2014).Undoubtedly, students are increasingly using MT for diverse academic

purposes as well as in their daily lives. They use MT for vocabularylearning, translation, reading comprehension, and writing assignments,considering it to be a good supplementary tool for language learning(Alhaisoni & Alhasysony, 2017). According to the literature, MT benefitsstudent language learning from the cognitive, linguistic, and affectiveperspectives. From the cognitive perspective, it reduces cognitive load bydoing preliminary translations (Baraniello et al., 2016; Lewis, 1997) andpromotes self-directed learning (Godwin-Jones, 2015; Wong & Lee,2016). From the linguistic perspective, MT supports lexico-grammaticalknowledge (Bahri & Mahadi, 2016; Doherty & Kenny, 2014; Wong &Lee, 2016), promotes reading comprehension and writing (Alhaisoni &Alhaysony, 2017; Garcia & Pena, 2011; Kumar, 2012), and ultimately fos-ters language learning (Belam, 2003; Ni~no, 2009; Shei, 2002a; Wong &Lee, 2016). Williams (2006) pointed out that the use of MT can “forcestudents to think about language as a communication tool, not as a setof decontextualized vocabulary words or phrases” (p. 574). From theaffective perspective, it lowers language anxiety (Bahri & Mahadi, 2016;Jin, 2013), increases motivation and confidence (Kliffer, 2008; Ni~no,2008), and creates a nonthreatening learning environment (Ni~no, 2009).At the same time, studies also report the drawbacks of MT, such as erro-neous sentences, incorrect lexis, and inaccurate grammar (Bahri &Mahadi, 2016; Josefsson, 2011; Ni~no, 2008, 2009).The current situation can be summarized as follows: the demand for

MT is increasing in the learning context, but the reliability of MT hasnot yet been fully established. Therefore, we need to determine how bestto use MT, an imperfect tool, to produce positive learning outcomes inthe language classroom. To date, little research on this issue has beenconducted. Hence, it is important to investigate how MT can facilitatestudent language learning based on actual evidence and acknowledge itspotential benefits, as well as its dangers, in education. Most studies onMT as a tool for L2 learning focus on MT of students’ L1 writing andpostediting the translation provided. Consequently, these reports holdMT as a bad model (Ni~no, 2004, 2009), full of lexico-grammatical errorsthat must be corrected through postediting. In contrast, this researchtakes MT as a CALL tool that can provide students with lexico-grammat-ical references in the target language and support language learning. Aspostediting does not enable direct measurement of student improvement,this study employs a different task design with students translating theirL1 writing into L2 without the help of MT and then correcting their L2

2 S.-M. LEE

writing using the MT translation for comparison. The purpose of thecurrent study is to investigate the value of MT for improving students’L2 writing.

Literature review

Prior studies confirmed that MT can serve as an effective supplementarylearning tool during L2 writing. The studies of both Garcia and Pena(2011) and Ali and Alireza (2014) revealed that MT helped studentswrite faster and produce more fluent and natural writing with fewererrors. Garcia and Pena (2011) further found that beginning studentsbenefited most from MT and that MT helped them express themselvesbetter and communicate more in L2 writing. According to Godwin-Jones’s (2015) study, MT promoted students’ writing by scaffolding theirlearning. Ni~no’s (2008) study also showed that MT particularly improvedstudents’ paraphrasing skills, which are essential to successful writing(Chen, Huang, Chang & Liou, 2015). Correa (2014) ascertained thatpostediting with MT helped students develop metalinguistic awareness,contributed to the writing process, and improved the final outcome.Similarly, Amaral and Meurers (2011) claimed that MT can provide lin-guistic modeling, raise linguistic awareness, and increase lexical, seman-tic, syntactic, and pragmatic knowledge of L2 during L2 writing.MT can further facilitate student writing by increasing lexical fluency

(Chen et al., 2015). In fact, studies have often compared MT to diction-aries and other electronic reference tools, and they concluded that MTperforms better in terms of translating technical jargon, phrases, and col-locations than traditional dictionaries (Bahri & Mahadi, 2016; Frodesen,2007). In addition, MT could better cater to a language learner’s needs,whereas other electronic reference tools and dictionaries are inadequatein helping learners develop lexical knowledge (Chen et al., 2015).Bernardini (2016) maintained that MT in language classrooms functionssimilar to a corpus that is “fully adaptable to the learner’s individualneeds and preferences” (p. 16). This, in turn, leads to data-driven learn-ing, which has been shown to be effective for ESL/EFL students, particu-larly those at lower levels (Nation, 2001; Wong & Lee, 2016).Several studies also found that MT could help student revisions during

L2 writing. Garcia and Pena (2011) claimed that MT postediting allowsstudents to focus more on editing and the process of writing. Kliffer(2005) stated that MT postediting enables students to apply their existingknowledge of the target language and make corrections in writing. Inaddition, MT can assist EFL students who often find it difficult to obtainindividualized feedback about their writing in the classroom. It can offer

COMPUTER ASSISTED LANGUAGE LEARNING 3

individualized feedback, provide lexical and syntactical alternatives, andtherefore help students detect and fix errors. Furthermore, MT focusesmore on lexico-grammatical errors and forces students to find their ownerrors, solve the problem, and make a decision. In doing so, studentsacquire self-directed and independent learning skills related to L2 writing(Bernardini, 2016; Garcia & Pena, 2011).Furthermore, MT seems to be helpful not only from the linguistic per-

spective but also from the affective perspective. According to Amores(1997), students tend to become defensive toward the teacher’s editing oftheir writing. In contrast, students might be less defensive and morecomfortable with MT. Ni~no (2004) argued that using MT in posteditingtasks heightens learner motivation and participation and, as a conse-quence, increases confidence in language learning. According to severalstudies (Bahri & Mahadi, 2016; Jin, 2013; Kliffer, 2008; Ni~no, 2008,2009), MT helps lower student anxiety and build a nonthreatening lan-guage learning environment.Studies have also reported the drawbacks of MT. Although it is helpful

in developing lexical knowledge and increasing awareness of grammarrules in context, it is less useful for providing grammatical solutions(Josefsson, 2011). Moreover, MT supports language learning mostly atthe lexico-grammatical level, but not at the discourse level (Groves &Mundt, 2015). Shei’s studies (2002a, 2002b) on MT pre-editing reportedidiomatic, lexico-semantic, and cultural limitations. He also pointed togrammatical errors in MT outputs due to ill-formed language input andinsufficiencies of MT grammar systems. The dependence of MT qualityon the language pair, size, text type, and subject area is also problematic(Godwin-Jones, 2015; Ni~no, 2009).Although it remains controversial, previous research has found diverse

educational benefits of MT in language learning. Moreover, as MT tech-nologies advance, MT is expected to be used more widely in the aca-demic EFL context. Therefore, it is imperative to investigate the effectsof MT on L2 writing. However, mainly because researchers and practi-tioners are still skeptical of its accuracy and usefulness (Barr, 2013),adequate amounts of empirical research on this issue have not yet beenpublished to prove its pedagogical effectiveness. Most previous studieshave focused on the reliability of MT and have held MT to be a bad lan-guage model. Moreover, the research methods used in those studies weremostly limited to surveys that did not measure student learning out-comes after using MT (Case, 2015). Therefore, this study investigatesMT as a CALL tool in L2 writing based on multiple data sources: studentwriting outcomes, interviews, and reflection papers. Whereas prior stud-ies focused mainly on MT postediting (student editing of the MT version

4 S.-M. LEE

to enhance the accuracy and quality), this study asked students to edittheir own writing with the help of MT. In the current study, the follow-ing questions are addressed:

1. In what ways did MT influence EFL students’ English writing?2. How did MT influence students’ writing strategies during revision?

Method

Participants and task description

Data for the present study were collected over six weeks in a multi-media-assisted language learning course at a university in Korea. Thesubjects were 34 students (21 females and 13 males) who majored inEnglish. Their L1 was Korean and English proficiency generally rangedbetween 70 and 95 on the iBT TOEFL (between intermediate and high-intermediate). The students were given a writing task about the textinglanguage of today’s young people that involved the following five steps:(1) watching a TED video (15minutes) on the topic, (2) writing a one-page paper about the topic in their L1 (source text), (3) translating theirwriting into English (initial English version without the help of MT), (4)translating their writing into English solely using MT (MT version), and(5) editing their initial English translation by comparing it with the MTversion (final version with the help of MT). During Step 3 and 5 the stu-dents were allowed to use other resources, including dictionaries. Thestudents were allowed to choose MT, and except four students who usedPapago (MT developed in Korea), all of the students used GoogleTranslate. Both tools were similar in function and performance and bothwere easy to use. After completing the task, the students were

Figure 1. Data collection and analysis.

COMPUTER ASSISTED LANGUAGE LEARNING 5

interviewed during individual writing conferences with their instructor.They also submitted reflection papers at the end of the task.

Data collection and analysis

The present study employed a mixed method, which included quantita-tive text analysis of all the versions of the students’ writing and qualita-tive analysis of their interviews and reflection papers. Data collection andanalysis are summarized in Figure 1.All versions of the students’ writing were analyzed using text analysis with

multiple steps. During the first step of the analysis the L1 writing was ana-lyzed to check its overall quality. Particular attention was paid to the contentand meaning because they were supposed to remain the same in both thestudents’ translation and the MT version. The structure and logic of the sen-tences were also checked since they can influence the quality of the transla-tion, particularly with MT (Godwin-Jones, 2015; Ruiz & Federico, 2014).During the next steps, the errors appearing in the students’ English

writing (without the help of MT) and their final version (with the helpof MT) were calculated and categorized as lexical or grammatical errors.While previous MT studies used error analysis to evaluate the accuracyof MT (Aiken & Balan, 2011; Groves & Mundt, 2015; Kliffer, 2008), thecurrent study was unique in that it used error analysis to measure howmuch the students’ L2 writing improved with the help of MT. In add-ition, previous studies measured only grammatical errors, but the currentstudy examined errors in two dimensions, the level and type of errors.Error levels included the use of symbols, words, phrases, or clauses/sen-tences; error type distinguished grammatical and lexical errors.According to Barkaoui & Knouzi (2012), lexical errors include two cate-gories, type and form. A lexical type error refers to wrong word choicefor the given context, such as “use texting distinctively� in reality andon the internet.” A lexical form error indicates the wrong form of aword, such as “destruct� of the language.” However, because lexicalform errors overlap with one of the grammatical error categories, misin-formation, they were categorized as grammatical errors and only lexical-type errors were counted as lexical errors in this study.Concerning grammatical errors, the four categories proposed by

Dulay, Burt, and Krashen’s (1982) Surface Structure Taxonomy, addition,omission, misinformation, and misordering, were utilized. While lexicalerrors were simply counted in terms of occurrence, grammatical errorswere weighted differently depending on their significance. For instance,the omission of content words, such as subjects and verbs, was takenmore seriously than the omission of function words, such as articles, or

6 S.-M. LEE

the misuse of prepositions. Thus, student writing with more errors didnot necessarily score lower than that with fewer errors. It should be alsonoted that the error-free T-unit ratio, which is widely used in ESL writ-ing research, was not used in this study because it cannot count multipleerrors in a single T-unit. The multi-trait scoring system was used, as itwas more suitable for the purposes of this study than holistic or analyticscoring systems, which incorporate more criteria than required for thisstudy. Based on the number of errors, the significance of each error, theclarity of meaning, and the correctness of lexical usage, each piece of thesecond and final versions was scored from 1 to 6. Then the lexical andsyntactic complexity of each piece of writing was calculated.The last step was investigating changes appearing in the final version

and identifying each change on the basis of its level and purpose. Levelrefers to symbols, words, phrases, clauses/sentences, or paragraphs.Purpose indicates the reason for the change: mechanics corrections,word use corrections, grammatical corrections, changes to better expressthe same meaning, and changes in content. Two trained raters scoredthe students’ writing outcomes and identified the errors. The interraterreliabilities were 91% and 93%, respectively, for the initial and final ver-sions of the students’ English writing.On completion of the task, interviews and reflections papers were col-

lected. All of the participants participated in interviews with the instructorand each interview took about 5–10minutes based on the analysis of thestudents’ writing. The interview questions covered the advantages and dis-advantages of MT in L2 writing, procedures for correcting errors duringrevision, and the reasons underlying the changes (Appendix). Interviewswere individually conducted during writing conferences and recorded. Inthe reflection papers, the students evaluated MT as a supplementary learn-ing tool for L2 writing in half to one page. Interviews and reflection papershelped to elucidate the students’ perceptions of MT and their intentions inmaking the changes they did in their final versions, which could not havebeen determined otherwise. The interviews and reflection papers werecoded with multiple steps from open to selective coding to indicate emerg-ing themes and used to triangulate the results of writing analysis.

Results

Analysis of student writing

Most students generated acceptable writing quality in L1. The students’ ini-tial L2 versions had scores ranging from 2 to 6 with a mean of 3.76 on a6-point scale (four students with a score of 2; 10 students, 3; 14 students, 4;two students, 5; and four students, 6). Compared with the initial versions,

COMPUTER ASSISTED LANGUAGE LEARNING 7

the mean score significantly increased in the final version (mean¼ 4.59).The average number of total errors in the students’ initial English versionwas 21.94, with grammatical errors (mean¼ 15.67) occurring more thantwice as often as lexical errors (mean¼ 5.97). The average number of totalerrors decreased significantly in the final version (mean¼ 13.64), with theaverage number of lexical errors and grammatical errors falling to 3.82 andto 9.82, respectively, because the students meticulously edited their initialEnglish version by comparing it with the MT version. The average numberof changes that the students made to the final version was 16.0, withchanges at the lexical level (mean¼ 8.57) occurring twice as frequently aschanges at the phrase level (mean¼ 3.55) or clause/sentence level(mean¼ 3.91). No changes were found at the symbol and paragraph levels.It was noticeable that the students did not simply adopt the MT translationin their revision. This will be further discussed later in the paper.With regard to purpose, replacing expressions appeared the most frequently

(mean¼ 7.8), followed by editing grammar (mean¼ 6.4) and fixing vocabulary(mean¼ 3.6). Changes to vocabulary were mostly limited to the lexical level,while changes to grammar and expressions ranged from changing a single wordto phrase- or sentence-level revisions. Whereas changes in vocabulary andgrammar contributed to correcting errors, changes in expressions did not neces-sarily reduce the number of errors in the students’ writing, although they didenhance comprehensibility and authenticity. However, sometimes grammaticalor lexical errors were corrected concurrently when one expression replacedanother. It should also be noted that the sum of the changes in each purpose cat-egory exceeded the number of total errors marked in some students’ writingbecause a single error was sometimes edited for two or more purposes. Forexample, a student changed “the destruct* of the language” to “the decline ofthe language,” simultaneously fixing both a lexical and a grammatical error. Thestudents did not make any change for mechanics or content in the final version.Among the changes, 68.9% turned out to be correct, whereas the rest were stillincorrect (Table 1).On statistical analysis, t-tests supported significant differences in writ-

ing scores and the number of lexical and grammatical errors in the initialand final versions. On the other hand, there were no statistically mean-ingful differences in lexical complexity (lexical diversity and density) orsentence complexity between the initial and final versions (Table 2).

Qualitative data results

Two types of qualitative data, interviews and reflection papers, were col-lected, and both types were consistent with the other results of thisstudy. Most of the students mentioned in their reflection papers that

8 S.-M. LEE

using MT helped their writing (N¼ 30, 88%). They wrote that they wereinitially skeptical about its accuracy based on their own previous experi-ences or hearing from others about MT, which made them dismissive ofMT, especially in an academic setting. However, during the task, theywere “very impressed to find that MT was a lot more accurate thanexpected,” as one student remarked.Most of the students mentioned that MT was useful, particularly for

finding a more accurate word or an authentic expression for a given con-text (N¼ 23, 67%), and about 70% of them (N¼ 23) mentioned that theywould use MT to find vocabulary next time. They explained that theywere often flooded with words in the dictionary, did not know which onewas the most appropriate for their context, and often ended up choosingthe wrong word. The students mentioned that MT was effective for lexicalchoices (N¼ 19, 55%) because it automatically ruled out some options,saying that “MT, unlike a dictionary, selected a word based on the con-text.” The students felt that MT’s selection of a word was more accurateand contextualized than their own choices from the dictionary in certaincases. They further said that having another option could make their writ-ing better. They compared MT’s choice with theirs and then selected theword that best fit the context. When they were uncertain, they checkedthe usage of the word on the internet or in the dictionary.

Table 1. Sample of changes.Category Original version Final version

Level Symbol No changes foundWord The standard for

right and wrongshouldbe definite.

The standard forright and wrongshould be clear.

Phrase At this very minute Even atthis moment

Clause/Sentence It is not obviousthat the arrival ofthe texting lan-guage is profit-able to one’slinguistic area.

It is doubtfulwhether emer-gence of the textlanguage is bene-ficial in the indi-vidual lan-guage domain.

Paragraph No changes foundPurpose language Mechanics No changes found

Vocabulary Destruct ofthe language

Decline ofthe language

Grammar A new technology isemergedconstantly.

A new technologyemergesconstantly.

Expression If someone arguesthat the transi-tion of languageadverselyaffects it…

If we argue that lan-guage changeshave had a nega-tive effecton language…

Contents No change found

COMPUTER ASSISTED LANGUAGE LEARNING 9

In a similar way, when they noticed sentence structure different fromtheirs in the MT version, they “became aware” of potential issues and“reconsidered” the grammar they used to determine the right choice. Inother words, MT improved the students’ lexico-grammatical awarenessand pushed them to correct errors. This study revealed that the studentstook advantage of MT in expressions as well. The students commentedthat MT introduced them to many authentic expressions and showedthem an alternate way to express the same meaning. As one studentremarked during the interview, “I realized there was an easier way toexpress the same meaning and, moreover, it made more sense.” In add-ition, 12 students (34%) mentioned that they did not usually edit beforesubmitting writing assignments; however, they spent more time editingduring the MT task.On the other hand, the students noted several drawbacks with MT.

First, they questioned its accuracy (N¼ 9, 27%). When they foundungrammatical sentences in the MT version, they said that they becamemore cautious and did not directly adopt the MT version. Second,although MT translated text in a more contextualized manner than a dic-tionary, the students still found awkward literal translations (N¼ 8,23%). Third, certain writing styles were less accurately translated by MT.For instance, run-on sentences tended to generate more grammaticalerrors. A student remarked in the reflection paper that he would “writein Korean more simply and clearly and then translate into another lan-guage next time” to minimize the errors generated by MT. Several stu-dents also mentioned that MT should be able to understand the contextand culture of the target language. Additionally, words with elusive ordouble meanings were often inaccurately translated.During analysis of the students’ reflection papers, two strikingly differ-

ent groups of responses emerged in terms of students’ perceptions of theaccuracy of MT. A closer look at the data revealed that they viewed MTdifferently depending on their English proficiency. That is, the studentswith initial English writing scores �4 on a scale of 6 (N¼ 28) mentionedthe positive side of MT more frequently. They wrote that MT was

Table 2. t-test results.

Measure

Students’English version Final version t-test

M SD M SD t Sig (2-tailed)

Writing score 3.76 1.13 4.59 1.05 –5.524 0.000No. of lexical errors 5.97 3.07 3.82 2.52 6.742 0.000No. of grammatical errors 15.67 7.81 9.82 4.87 5.759 0.000Lexical diversity 141.67 28.82 141.20 34.33 0.158 0.876Lexical density 67.62 10.32 67.64 9.93 –0.027 0.979Sentence complexity 20.39 5.19 20.83 5.53 –1.32 0.195

N¼ 34.

10 S.-M. LEE

“useful,” “helpful,” and “effective” for correcting grammar and vocabu-lary, in that it “helped improve my writing” and “I could learn a lotabout grammar from MT.” On the other hand, students with scores �5(N¼ 6) often mentioned the downsides of MT. They remarked that“although MT was helpful at the vocabulary level, it did not help muchat the sentence level.” They also pointed out that MT was not accuratein translating longer text, and therefore it was not appropriate for aca-demic purposes. Interestingly, one student added that although MT wasnot particularly helpful for her, she believed that “it would benefit elem-entary-level students.” In summary, the reflection papers showed thatMT was more beneficial to lower-level students, particularly with regardto appropriate word choice as well as identifying and fixing lexico-gram-matical errors.

Discussion

MT’s influence on student revisions

Compared with their own English writing, the students significantlyimproved their final version using MT. The number of errors in bothvocabulary and grammar decreased dramatically in the final version.Editing with MT in this study ultimately heightened grammatical accur-acy in student writing. Previous studies claimed that improved grammaris characteristic of enhanced text quality (Min, 2006) and that grammarcorrection facilitates the communicative effectiveness of writing (Rahimi,2009). While lexico-grammatical accuracy is not a single criterion forgood English writing, it can certainly contribute to the quality of writing.Students at the intermediate and lower levels are challenged by variouslexical and grammatical errors in their English writing, which often hin-der communication and lower the quality of their writing (Lee, 2014).Even when students can produce quite good writing in their L1, likethose in the current research, lexico-grammatical errors and inauthenticexpressions are detrimental to their English writing. In such cases,improvements in vocabulary, grammar, and expressions enhance theoverall quality of writing. In the current study the students improved thequality of their writing by decreasing lexico-grammatical errors with thehelp of MT.The students also found MT useful for context-appropriate word

choice and developing more authentic expressions. Whereas dictionariesshow vocabulary without considering the context in which it will beused, MT is generally more sensitive to context because it translatesentire sentences (Doherty & Kenny, 2014). Therefore, students regardedMT as a useful substitute for a dictionary. In summary, MT helped

COMPUTER ASSISTED LANGUAGE LEARNING 11

students minimize grammatical errors, select more appropriate and con-textualized vocabulary and more authentic expressions, and thusenhanced the overall quality of their writing. In this study, as in Ni~no’sargument (2004), editing with MT could “promote accuracy and there-fore grammatical, lexical and stylistic correctness in the foreign languageclassroom” (p. 183). Additionally, in terms of types of feedback, previousstudies found that direct correction was most effective for grammar revi-sion and students were most satisfied with it compared to indirect cor-rection and other types of feedback (Chandler, 2003; Saito & Fujita,2004). In this study, MT provided immediate and individualized directcorrection of grammatical errors, and the students found grammaticalcorrection with MT during revision easy.The current study established that MT functioned similar to peer-edit-

ing: neither is perfect, but both are helpful to student writing. Numerousstudies support the positive role of peer-editing. Those studies assert thatstudents obtain more positive attitudes toward writing (Garcia & Pena,2011; Min, 2005), acquire insight into the revision process, and generatebetter revisions (Lee, 2014; Min, 2006) after peer-editing. Just as studentswere shown to be selective when adopting from peer-editing in previousstudies (Enkin & Mejias-Bikandi, 2016; Lee, Wong, Cheung, & Lee, 2009;Min, 2006), the students in this study also selectively embraced MT.However, a noticeable difference between peer-editing and editing withMT was observed. In previous studies (Lee, 2009; Min, 2006; Sengupta,1998), changes in grammar were less frequent among the various pur-poses of revision after peer-editing, whereas the students in this studywere more open to making grammatical changes after comparing theirwriting with the MT version. The main cause of the low rate of gram-matical revisions that appeared in both Sengupta’s (1998) and Lee’s(2009) study was the students’ low language proficiency. In Min’s study(2006), the cause was assumed to be that the students’ revisions weredone for a variety of purposes at various levels as they focused on mean-ing, and, accordingly, peer-editing also included more than grammar. Onthe other hand, the students’ more frequent grammatical changes in thisstudy indicate two things. On the one hand, most of the students in thisstudy were proficient enough to notice their grammatical errors afterconsulting with MT, and moreover, as Lee’s (2009) study pointed out,students at the intermediate level were most sensitive to their grammat-ical errors, so they actively made grammatical corrections. On the otherhand, too much focus on grammatical changes indirectly shows the limi-tations of revision using MT. Although MT was a good resource for lex-ical and grammatical revisions, it could not provide any feedback beyondthe sentence level, unlike peer-editing. In other words, MT was helpful

12 S.-M. LEE

for micro-revisions at the sentence level, but it was not helpful formacro-revisions at the discourse or content level.In addition, no significant differences in lexical or grammatical com-

plexity were found before and after revision. Both lexical density and lex-ical diversity remained the same in the final version because the studentsreplaced one word with another, which did not change the overall lexicalcomplexity. There were no significant differences in syntactic complexityeither, presumably because the students adopted only parts of sentencesrather than entire sentences from the MT version, which did not resultin differences in structure.The current study thus shows that MT greatly improved the quality of

student English writing in terms of vocabulary, grammar, and expression,but it could not help the students beyond that. Therefore, teachers needto remind students to focus on issues beyond lexico-grammatical changesduring revision. Moreover, the changes that the students made were notalways correct. As MT is not perfectly accurate yet, it is hard for the stu-dents with low language proficiency to select proper use of the language.Accordingly, teachers should advise them to double-check the translationof MT with other resources rather simply adopting it. Additionally,teachers should list the drawbacks of MT, such as grammatical inaccur-acy, literal translations, the influence of L1 writing styles, and erroneoustranslations of words with ambiguous meanings, so students can avoidthese potential pitfalls. With the help of MT, students can produce writ-ing or make revisions more easily in a low-stress environment. As MThelps to reduce errors in spelling, vocabulary, and grammar, they canput more effort into global revisions for meaning instead, or, as Grovesand Mundt (2015) argued, they can “begin to focus on the deep liter-acies” (p. 119).

MT influence on student writing strategies

In the current study, the students made use of diverse writing strategieswhile editing with MT, such as double checking, using previous know-ledge, inferencing, paraphrasing, and rewriting. Comparing their versionswith the MT version encouraged the students to identify and fix theirown errors. During revision, MT functioned as an individualized feed-back system for each student. It could not suggest the best languagemodel to the students, but it was sufficient to show alternatives, whichrequired them to give their word choice more thought and double-checktheir grammar and expression. EFL students’ reading proficiency is usu-ally higher than their writing proficiency; therefore, even though the stu-dents could not generate accurate text in English on their own, they

COMPUTER ASSISTED LANGUAGE LEARNING 13

were often able to distinguish the best choice for a given context whengiven concrete options. Thus, rather than simply adopting everythingfrom the MT version, the students critically and strategically selectedfrom alternatives based on their previous knowledge, and sometimescombined their original with the MT version to produce a sentence.When they were uncertain, the students consulted other resources, suchas dictionaries or the internet.This process fostered not only writing but also language learning in

general because it raised their metalinguistic awareness of L2, which ispositively correlated with L2 proficiency (Enkin & Mejias-Bikandi, 2016).Bernardini (2016) stated that MT can help students become more awareof patterns, correlations between form and meaning, lexical choices, andcollocational patterns. In the same vein, MT in this study alerted the stu-dents to errors by suggesting alternatives and raised their metalinguisticawareness, which helped them correct erroneous sentences and benefitedtheir L2 proficiency. According to Wong and Lee (2016), language learn-ing with MT can cultivate student language awareness and noticing skills.During the learning process with MT, students first notice their lack ofL2 knowledge and identify items to be learned from the MT version(noticing). Then, they recall the newly learned items (retrieving) and usethem in new contexts to create new sentences (creating). Similar toWong and Lee’s argument, the students in the current study compared,detected errors, considered alternatives, and rewrote, and in doing sothey learned grammar and new words in context and consolidated thenewly learned items by using them in the final version. As Carroll andSwain (1993) emphasized, error detection and correction enhances gram-matical accuracy in FL writing and promotes interlanguage development.Moreover, being exposed to alternatives can help students become awarethat more than one form or expression can have the same meaning andthat there is no one-to-one translation between languages (Baraniello etal., 2016).As described above, producing the final version using MT entailed a

complex revision process that encouraged the students to view writing asa process. Previous studies pointed out that EFL students do not focusmuch on revision and usually view writing as a product (Lee, 2009,2014; Min, 2006), but the students in this study acknowledged theimportance of writing as a process. As in Garcia and Pena’s study(2011), in which students viewed translation and editing as a process ofwriting during MT editing, the students in this study focused more onediting and shifted their view of writing from a product to a process.They even began to pay attention to their L1 writing (pre-editing) afterrealizing that their source text could determine the quality of the MT

14 S.-M. LEE

version. Viewing writing as a process can in turn promote learner auton-omy and self-directed learning strategies, which is essential for successfulwriting, especially in an EFL writing classroom with limited instructorfeedback (Bernardini, 2016; Ni~no, 2008; Saito & Fujita, 2004).

Conclusion

The present study investigated the role of MT as a CALL tool by exam-ining students’ L2 writing outcomes with MT and their perceptionsregarding its use. Text analysis of two versions of student writing indi-cated that MT improved their vocabulary, grammar, and expressions,which ultimately resulted in writing quality improvement. This studyalso found that MT positively influenced student writing strategies dur-ing revision. At the same time, the students found various drawbacks ofMT, such as inaccuracy, literal translations, and dependence on L1 writ-ing style.Despite the mixed results of prior studies, there is generally agreement

about the potential of MT as a pedagogical tool for language learning,with more recent studies confirming that MT technology has advancedand become more accurate (Correa, 2014; Enkin & Mejias-Bikandi,2016). As Myles (2002) mentioned, many FL students have problemswith even basic grammar, which makes them prone to producing errors.Moreover, instructor feedback for individual students is limited in theclassroom. In this situation, MT can be an effective supplementary tool.However, to benefit student learning, the teacher must understand therole of MT in language learning and provide adequate guidelines on theuse of MT, explaining both its strengths and weaknesses to students,along with ways to use MT as a language resource. Clifford, Merschel &Munn�e (2013) reported that teachers held negative attitudes toward theuse of MT in an academic setting. This is understandable because of thepotential pitfalls, such as being exposed to inaccurate language models,cheating, or becoming overdependent on MT; nonetheless, the accuracyof MT seems highly likely to improve (Case, 2015; Groves & Mundt,2015), with its use becoming more prevalent in the near future. Giventhe increasing demand for MT, teachers need to find ways to embrace itrather than ignoring it. Ni~no (2009) pointed out that MT systems werenot originally designed for language learning; therefore, it is crucial thatteachers prepare students for how to deal with it in a language learningcontext (Bahri & Mahadi, 2016; Davis, 2006).The current study is significant in that it empirically examined the

role of MT based on multiple data sources from multiple perspectives.However, the sample size was insufficient to generalize the results. Other

COMPUTER ASSISTED LANGUAGE LEARNING 15

variables, such as different types of MT, writing procedures, languagepairs, and language proficiency levels, might cause different results. Inaddition, the students in the study did not correct errors that were notindicated by MT, possibly due to task design. Different task designs maylead to different student behaviors during revision. Therefore, moreresearch needs to be done on these issues.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes on contributor

Sangmin-Michelle Lee is Professor in the School of Global Communication at KyungHee University, S. Korea. She is the author of English textbooks, digital language learn-ing materials and articles related to technology-supported language learning, second lan-guage writing and creativity.

ORCID

Sangmin-Michelle Lee http://orcid.org/0000-0002-7686-3537

References

Aiken, M., & Balan, S. (2011). An analysis of Google Translate accuracy. TranslationJournal, 16(2). Retrieved from http://translationjournal.net/journal//51pondering.htm.

Alhaisoni, E., & Alhaysony, M. (2017). An investigation of Saudi EFL university stu-dents’ attitudes towards the use of Google Translate. International Journal of EnglishLanguage Education, 5(1), 72–82. doi:10.5296/ijele.v5i1.10696

Ali, K., & Alireza, F. (2014). The effect of computer-assisted translation on L2 leaners’mastery of writing. International Journal of Research Studies in Language Learning, 3,29–44.

Amaral, L., & Meurers, D. (2011). On using intelligent computer-assisted language learn-ing in real-life foreign language teaching and learning. ReCALL, 23(01), 4–24. doi:10.1017/S0958344010000261

Amores, M. (1997). A new perspective on peer-editing. Foreign Language Annals, 30(4),513–522. doi:10.1111/j.1944-9720.1997.tb00858.x

Bahri, H., & Mahadi, T. (2016). Google Translate as a supplementary tool for learningMalay: A case study at Universiti Sains Malaysia. Advances in Language and LiteraryStudies, 7(3), 161–167.

Baraniello, V., Degano, C., Laura, L., Lozano, M., Zahonero, M., & Petroni, S. (2016). Awiki-based approach to computer-assisted translation for collaborative language learn-ing. In Y. Li, M. Chang, M. Kravcik, E. Popescu, R. Huang, Kinshuk, & S. Chen(Eds.), State-of-the-art and future directions of smart learning: Lecture notes in educa-tional technology (pp. 369–379). Singapore: Springer.

16 S.-M. LEE

Barkaoui, K., & Knouzi, I. (2012). Combining score and text analyses to examine taskequivalence in writing assessment. In E. Sttendam, M. Tillerma, G. Rijlaarsdam, & H.Bergh (Eds.), Measuring writing: Recent insights into theory, methodology and practices(pp. 83–116). Boston: Davers.

Barr, D. (2013). Embedding technology in translation teaching: Evaluative considerationsfor courseware integration. Computer Assisted Language Learning, 26(4), 295–310. doi:10.1080/09588221.2012.658406

Belam, J. (2003). Buying up to falling down: A deductive approach to teaching post-editing. InProceedings of theWorkshop on Teaching Translation Technologies and Tools (pp. 1–10).

Bernardini, S. (2016). Discovery learning in the language for translation classroom.Cadernos de Traduc~ao, 36, 14–35. doi:10.5007/2175-7968.2016v36nesp1p14

Carroll, S., & Swain, M. (1993). Explicit and implicit negative feedback: An empiricalstudy of linguistic generalisations. Studies in Second Language Acquisition, 15(03),357–386. doi:10.1017/S0272263100012158

Case, M. (2015). Machine translation and the disruption of foreign language learningactivities. eLearning Papers, 45, 4–16.

Chandler, J. (2003). The efficacy of various kinds of error feedback for improvement inthe accuracy and fluency of L2 student writing. Journal of Second Language Writing,12(3), 267–296. doi:10.1016/S1060-3743(03)00038-9

Chen, M., Huang, S., Chang, J., & Liou, H. (2015). Developing a corpus-based para-phrase tool to improve EFL learners’ writing skills. Computer Assisted LanguageLearning, 28(1), 22–40. doi:10.1080/09588221.2013.783873

Clifford, J., Merschel, L., & Munn�e, J. (2013). Surveying the landscape: What is the roleof machine translation in language learning? The Acquisition of Second Languages andInnovative Pedagogies, 10, 108–121.

Correa, M. (2014). Leaving the “peer” out of peer-editing: Online translators as a peda-gogical tool in the Spanish as a second language classroom. Latin American Journal ofContent and Language Integrated Learning, 7(1), 1–20. doi:10.5294/laclil.2014.7.1.1

Davis, R. (2006). Utopia or Chaos? The impact of technology on language teaching. TheInternet TESL Journal, 12(11). Retrieved from http://iteslj.org/Articles/DavisImpactOfTechnology.html.

Doherty, S., & Kenny, D. (2014). The design and evaluation of a Statistical MachineTranslation syllabus for translation students. The Interpreter and Translator Trainer,8(2), 295–315. doi:10.1080/1750399X.2014.937571

Dulay, H., Burt, M., & Krashen, S. (1982). Language two. New York: Oxford UniversityPress.

Enkin, E., & Mejias-Bikandi, E. (2016). Using online translators in the second languageclassroom: Ideas for advanced-level Spanish. Latin American Journal of Content &Language Integrated Learning, 9(1), 138–158. doi:10.5294/laclil.2016.9.1.6

Frodesen, J. (2007). Linguistic challenges of summary and paraphrase. Paper presented atthe 2007 Symposium on Academic Language Development. San Francisco, CA.

Garcia, I., & Pena, M. (2011). Machine translation-assisted language learning: Writingfor beginners. Computer Assisted Language Learning, 24(5), 471–487. doi:10.1080/09588221.2011.582687

Godwin-Jones, R. (2015). Contributing, creating, curating: Digital literacies for languagelearners. Language Learning & Technology, 19(3), 8–20.

Groves, M., & Mundt, K. (2015). Friend or foe? Google Translate in language for aca-demic purposes. English for Specific Purposes, 37, 112–121. doi:10.1016/j.esp.2014.09.001

COMPUTER ASSISTED LANGUAGE LEARNING 17

Jin, L. (2013). Foreign language learners’ use and perception of online dictionaries: Asurvey study. MERLOT Journal of Online Learning and Teaching, 9(4), 513–533.

Josefsson, E. (2011). Contemporary approaches to translation in the classroom: A studyof students’ attitudes and strategies. Retrieved from http://du.diva-portal.org/smash/get/diva2:519125/FULLTEXT01.pdf.

Kliffer, M. (2005). An experiment in MT post-editing by a class of intermediate/advancedFrench majors. In Proceedings EAMT 10th Annual Conference (pp. 160–165).

Kliffer, M. (2008). Post-editing machine translation as an FSL exercise. PortaLinguarum, 9, 53–67.

Kumar, A. (2012). Machine translation in Arabic-speaking ELT classrooms: Applicationsand implications. International Journal of Social Science and Humanity, 2(6), 442–445.doi:10.7763/IJSSH.2012.V2.142

Lee, S.-M. (2009). Exploring the writing and editing patterns of the different languageproficiency groups on a wiki. Foreign Languages Education, 16(3), 95–122.

Lee, S.-M. (2014). Grammatical error patterns in EFL students’ writing across differentlanguage proficiency levels. Foreign Languages Education, 21(2), 1–28. doi:10.15334/FLE.2014.21.2.1

Lee, C., Wong, K., Cheung, W., & Lee, F. (2009). Web-based essay critiquing systemand EFL students’ writing: A quantitative and qualitative investigation. ComputerAssisted Language Learning, 22(1), 57–72. doi:10.1080/09588220802613807

Lewis, D. (1997). Machine translation in a modern language curriculum. ComputerAssisted Language Learning, 10(3), 255–271. doi:10.1080/0958822970100305

Min, H. (2005). Training students to become successful peer reviewers. System, 33(2),293–308. doi:10.1016/j.system.2004.11.003

Min, H. (2006). The effects of trained peer review on EFL students’ revision types andwriting quality. Journal of Second Language Writing, 15(2), 118–141. doi:10.1016/j.jslw.2006.01.003

Myles, J. (2002). Second language writing and research: The writing process and erroranalysis in student texts. TESL-EJ, 6(2)Retrieved from http://www.teslej.org/word-press/issues/volume6/ej22/ej22a1/?wscr.

Nation, I. (2001). Learning vocabulary in another language. Cambridge: CambridgeUniversity Press.

Ni~no, A. (2004). Recycling MT: A course on foreign language writing via MT post-editing.In Proceedings of 7th Annual CLUK Research Colloquium, 179–187.

Ni~no, A. (2008). Evaluating the use of machine translation post-editing in the foreignlanguage class. Computer Assisted Language Learning, 21(1), 29–49. doi:10.1080/09588220701865482

Ni~no, A. (2009). Machine translation in foreign language learning: Language learners’and tutors’ perceptions of its advantages and disadvantages. ReCALL, 21(2), 241–258.doi:10.1017/S0958344009000172

Rahimi, M. (2009). The role of teacher’s corrective feedback in improving Iranian EFLlearners’ writing accuracy. Reading and Writing, 22, 219–243. doi:10.1007/s11145-008-9139-5

Ruiz, N., & Federico, M. (2014). Complexity of spoken versus written language formachine translation. In Proceedings of the Annual Conference of the EuropeanAssociation for Machine Translation, 173–180.

Saito, H., & Fujita, T. (2004). Characteristics and user acceptance of peer rating in EFLwriting classrooms. Language Teaching Research, 8(1), 31–54. doi:10.1191/1362168804lr133oa

18 S.-M. LEE

Sengupta, S. (1998). From text revision to text improvement: A story of secondaryschool composition. RELC Journal, 29(1), 110–137. doi:10.1177/003368829802900107

Shei, C.-C. (2002a). Combining translation into the second language and second languagelearning: An integrated computational approach. PhD Thesis. University of Edinburgh,Edinburgh.

Shei, C.-C. (2002b). Teaching MT through pre-editing: Three case studies. In Proceedingsof 6th EAMT Workshop Teaching Machine Translation (pp. 89–98).

Sukkhwan, A., & Sripetpun, W. (2014). Use of Google Translate: A survey of SongkhlaRajabhat University students. In Proceedings of L-SA Workshop and Colloquium:“Speaking” for ASEAN, (pp. 88–104).

Williams, L. (2006). Web-based machine translation as a tool for promoting electronicliteracy and language awareness. Foreign Language Annals, 39(4), 565–578. doi:10.1111/j.1944-9720.2006.tb02276.x

Wong, T., & Lee, J. (2016). Corpus-based learning of Cantonese for Mandarin speakers.ReCALL, 28(2), 187–206. doi:10.1017/S0958344015000257

Appendix

Interview questions

1. What were the advantages and disadvantages of using MT during revision?2. Why did you accept or reject the translation of MT in your revision? How did

you decide?3. Did you use any other resources during writing other than MT? How did you

use it?4. Did you experience any technical difficulties while using MT?

COMPUTER ASSISTED LANGUAGE LEARNING 19