PORTFOLIO ASSESSMENT OF SPEAKING WITH...
21
Transcript of PORTFOLIO ASSESSMENT OF SPEAKING WITH...
P O RT F O L I O A S S E S S M E N T O F S P E A K I N G W I T H
R O B S H E P PA R D
A N A C T I O N R E S E A R C H P R O J E C T
C O N D U C T E D A T
S O U N D C L O U D
F U N D E D B Y
S O U N D C L O U D
I S S U E S O F A S S E S S I N G S P O K E N L A N G U A G E
• grammar of spoken English incompletely codified
• inverse relationship between reliability and authenticity
• issues related to interacting with technology
• scoring validity
W H AT M A K E S A P O R T F O L I O A P O R T F O L I O ?
Portfolios are“purposeful collections of students’ work that tell the story of their achievements, skills, efforts, abilities, and contributions to a particular class” (Brown and Hudson, 1998, p. 164).
Others (e.g. O’Malley and Valdez-Pierce, 1996) stress the importance of student involvement in the selection of portfolio entries.
P O R T F O L I O A S S E S S M E N T I N L A N G U A G E T E A C H I N G
• authenticity/content validity
• contextualized performance
• increased motivation
• positive backwash
• increased student agency/empowerment
C H A L L E N G E S S P E C I F I C T O O U R C O N T E X T
!
!
!
!
• use of TABE at higher levels
• student goals related to speaking
F E AT U R E S O F S O U N D C L O U D
• simple interface
• accessible from anyplace with a web connection
• mobile apps
• timed comments
E N V I S I O N I N G A S P E A K I N G P O R T F O L I O• Students and teachers have checklist of speaking objectives for the
course.
• Teachers design speaking tasks aligned with speaking objectives.
• Groups record and upload tasks to class account.
• As homework, students listen to and comment on their own and peers recordings from their own accounts. Comments are specific and focused on aspects of the language.
• In the second half of the course, students begin to collect samples of their work, embedding recordings in a blog, including typed account of their own progress. This blog is the final product.
T H E E X T E N T A N D L I M I T S O F T H E P I L O T P R O J E C T
• not a test of spoken portfolios; a test of the viability of SoundCloud as a platform for such portfolios
• recording and commenting, but no compiling of a proper portfolio
• one semester (15 weeks)
• anonymous questionnaires at end of project
A C T I V I T I E S U N D E R T H E G R A N T
• purchased iPads and SoundCloud premium
• trained teachers on iPads
• trained teachers on SoundCloud
• worked with teachers to develop classroom objectives and activities for this project
• met periodically to discuss progress and setbacks
0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 01 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 10 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 11 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 00 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 01 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 01 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 01 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 10 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 01 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 00 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 01 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 00 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 01 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 10 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 10 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 10 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 00 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 01 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 01 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 11 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 01 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 11 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0
D ATA C O L L E C T I O N M E T H O D S
• pre-semester tech survey
• post-semester questionnaire to students
• post-semester questionnaire to teachers
• SoundCloud reports
• triangulation across methods when possible
C O M M E N T S B R O K E N D O W N B Y T Y P E
Other 13%
Language-Focused 13%
Response to Content 35%
Praise 38%
“I was so nervous !!!”
“B— speaks very eloquently.”“I should said 'when I was single.’’’
“Same to me, I came to here by myself and I don't know what the people said to me too.”
W H AT W E M E A N B Y SPEC I F IC A N D LANGUAGE -FOCUSED
“Here I said for but I should have used to.” !
“Consonant sounds at the ends of words are difficult for me, but here I did a good job
pronouncing first”
C O M M E N T S B Y T Y P E O V E R T I M E
First Half of Semester
Other 30%
Language-Focused 4%
Response to Content 30%
Praise 35%
Second Half of Semester
Other 3%
Language-Focused 19%
Response to Content 38%
Praise 41%
0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 01 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 10 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 11 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 00 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 01 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 01 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 01 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 10 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 01 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 00 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 01 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 00 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 01 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 10 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 10 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 10 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 00 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 01 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 01 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 11 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 01 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 0 0 1 0 1 0 1 1 11 1 0 1 0 1 1 0 1 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 1 0 0
R E S U LT S
18 respondents (1 response = 5.5%)
S I G N I F I C A N T Q U E S T I O N N A I R E R E S U LT S
• 83% of students agreed or strongly agreed that
• using SoundCloud improved their English
• using SoundCloud made them speak more in class
• they felt good about listening to and commenting on other students’ recordings
T E C H R E C O M M E N D AT I O N S
• as much tech training, practice, and support to staff as early as possible and ongoing
• train in stages: hardware, operating system, specific apps, specific functions
• make tech available on-site after hours (with staff support if possible)
• assume nothing about tech familiarity; conduct questionnaires
P E D A G O G I C A L A N D L O G I S T I C A L R E C O M M E N D AT I O N S
• clearly communicate the structure and breadth of the project to staff and students; reiterate several times in the early stages
• be prepared to change course slightly
• consider scalability at the outset (Which elements are desired-not-required?)
• we don’t want to be prescriptive about commenting, but students do need guidance; language-focused comments do not come naturally to students
• be prepared to provide more scaffolding than you think will be necessary
• practice commenting in class, and possibly separate from SoundCloud
R E F E R E N C E SBrown, J. D. and Hudson, T. (1998). The alternatives in language assessment. TESOL
Quarterly 32(4), 653-675.
Ellis, R. (1993). The structural syllabus and second language acquisition. TESOL Quarterly 27(1), 91-113.
Hoekje, B. and Linnell, K. (1994). “Authenticity” in language testing: Evaluating spoken language tests for international teaching assistants. TESOL Quarterly 28(1), 103-126.
Hughes, A. (2003). Testing for language teachers (2nd ed.). Cambridge, UK: Cambridge University Press.
Luoma, S. (2004). Assessing speaking. Cambridge, UK: Cambridge University Press.
Lynch, B. and Shaw P. (2005). Portfolios, power. and ethics. TESOL Quarterly 39(2), 263-297.
McCarthy, M., and Carter, R. (1995). Spoken grammar: What is it and how can we teach it? ELT Journal 49(3), 207-218.
O’Malley, J. M. and Valdez Pierce, L. (1996). Authentic assessment for English language learners: Practical approaches for teachers. Boston, MA: Addison-Wesley.
Richards, J. C. (2008). Moving beyond the plateau: From intermediate to advanced levels in language learning. New York, NY: Cambridge University Press.
