Automated user-centered task selection and input modification

Rintse van der Werf

Geke Hootsen

Anne Vermeer

MASLA project

Tilburg University

Outline• Background

• Research

• Discussion and future research

User-centered learning

• Approaches in educational research– Authentic– User initiated– Motivating– Individual needs

MASLA project

• Models of Adaptive Second Language Acquisition

• Combination of Computer Science and Second Language Acquisition

• Goal: building a model for personalized digital language learning web based applications

• How can learning materials automatically be adapted to fit the characteristics and preferences of the language learner?

• Criterion is learning effect.

Requirements for adaptivity

• Annotated learning material– domain model

• Knowledge about learner characteristics– user model

• User model + domain model -> adaptation model (rules)

(Dexter model, 1990; AHAM model (De Bra, 2000))

MASLA Framework

Graphical User Interface

Curriculum

L2 - proficiencies

Learning contents

Learning styles

Learner backgrounds

Task: Vocabulary learning through reading

• Incidental vocabulary learning (side effect of reading for comprehension)

• ZOPD (Vygotsky, 1962); Comprehensible input (Krashen, 1987)

– Assessing learner proficiency

– Assessing text difficulty

based on frequency information from corpora

=> combined in text coverage

Text Coverage

70

75

80

85

90

95

100

1 2 3 4 5 6 7 8 9

learner profficiency (x1000 lemmas)

lem

ma c

overa

ge (

%)

more difficult

easier

Interpreting text coverage

• Hazenberg, 1994; Laufer, 1989; Vermeer, 1998

• Lemma Coverage:

– 85%: Global understanding– 90%: Good understanding– 95%: Almost complete understanding

Effective Instruction

• Comprehensible but challenging

• Lemma coverage 85% - 92%

• Support from input modification– Dictionary/glossary (see Hulstijn et al., 1996; Plass et al.,

1998; Watanabe, 1997)– User initiated “focus on form”

Text Coverage

70

75

80

85

90

95

100

1 2 3 4 5 6 7 8 9

learner profficiency (x1000 lemmas)

lem

ma c

overa

ge (

%)

Top criterion

Bottom criterion

Summary of research background

• Web based tool for automatic adaptive selection of the appropriate text for a specific user.

• Automated analysis of text difficulty.

• User proficiency calculation from score on vocabulary test.

• User gets text that is comprehensible but challenging and has input modification for unknown words to support for understanding the text.

Research questions

• A. Adaptive selection of texts leads to:

• A learning effect for all users• No difference between learners with different proficiency

levels

• B. Using input modification:

• There is a relation between noticing and retention• (There is no difference in this relation for different

proficiency levels)

Method (1)

• Subjects (N=32)

• Reading Texts (16)– 4 clusters

• Input modification

Text coverage for selected texts

60

65

70

75

80

85

90

95

100

1 2 3 5 8

Almost complete comprehension

Global comprehension

Mean text coverage per cluster

60

65

70

75

80

85

90

95

100

1 2 3 5 8

Almost complete comprehension

Global comprehension

Method (1)

• Subjects (N=32)

• Reading Texts (16)– 4 clusters

• Input modification

Method (2)

• Data collection:

– User logging and tracking

– Testing material• Vocabulary proficiency test• Text specific vocabulary tests• Comprehension questions

• Procedure

Learning gains

Learning gains

Learning gains

Learning gains

Procedure

Results (1)

• A mean learning effect occurred for all clusters– 5% learning gains

• No significant difference between groups– both pre and posttest scores– learning gains

Results (2)

• Correlation between noticing and retention– Mean Φ correlation for subjects: .28– Mean Φ correlation for items: .50

• in general, the use of the dictionary was limited – No significant difference between proficiency groups

• In lookup behavior• In correlation

Conclusion

• Automated assessment of texts based on corpora information is a useful indication of text (task?) difficulty.

• Adaptive selection of texts based on vocabulary proficiency works.

• Open, web based learning environment provides flexibility in the curriculum and opportunities for individualized tasks.

Discussion and future work

• Increase learning gains– More adaptivity in text selection

• Increase exposure to target words• Based on observed behavior

• Increase usability of input modification– Individualize annotation

• Based on observed behavior• More focus on form

• Use different corpus for text coverage– Now children’s corpus, future Celex/CGN– Unknown lemmas – Multiword expressions