Scalable, Scaffolded Writing Assignments with Peer...
Transcript of Scalable, Scaffolded Writing Assignments with Peer...
Scalable, Scaffolded Writing Assignments with Peer Assessment in an Economic Literacy Course
AviJ.Cohen(Economics,UniversityofTorontoandYorkUniversity)
AndreaWilliams(WritingInstruction,UniversityofToronto)
ABSTRACT
Thispaperdescribesacollaborationbetweenaneconomistandawritingspecialistinthecontextofafaculty-widewritingacrossthecurriculumprogram,WIT(WritingInstructionforTAs).Thecollaborationinvolveddevelopingscaffoldedwritingassignmentsforalarge(800students)introductorycoursewithaneconomicliteracyapproach.Aneconomicliteracyapproachfocusesoncoreconceptsstudentscanusetomakesmartchoicesasconsumers,businesspeople,andinformedcitizens.Informedcitizenswillmakebetterdecisionsabouteconomicissuesthatmattertothemandtheircommunities,andcommunicatethoseviewseffectivelytoothers.Theop-edassignmentgivesstudentspracticedevelopingaconvincingargumentaboutaneconomicissueforgeneralreaders.
Theassignmentisscalablebecauseofitslength(300-500words),detailed,yeteasy-to-applygradingrubrics,andpeerassessmentsoftware.Studentsgiveeachotherdetailedfeedbackthattheyusetorevisetheirinitialdraft,whilethegraduateteachingassistantsgradethefinalversion.
Sincestudentslackexperiencewritingargumentsforgeneralreaders,wedevelopedtwoscaffoldedassignmentstopreparethem.ThefirstisanonlinemoduleonhowtoreadcriticallythatmodelstheprocessofreadinganarticlefromTheEconomist.Studentsthenwritea250-300wordabstractforanarticlefromanapprovedlist.Thesecondonlinemoduleteachesstudentshowtowriteanabstract,emphasizingthetransitionfromwriter-basedtoreader-basedprose,andtheimportanceofrevision.Thelow-stakesabstractassignmentalsohasstudentsdopeerreviewbeforetheyrevisetheirwriting,alongwithasimplerubrictohelpTAsgrademoreconsistently.Afinalmodulehasstudentswritetheirownargumentasanop-edassignment.
Likeanearlierpaper(CohenandSpencer1993)aboutintegratingwritingintotheeconomicscurriculum,thispaperincludesassignmentdetails,processes,andlessonslearnedforinstructorsinterestedinimplementingsimilarassignments.
Cohen&Williams,May2017
1
INTRODUCTION
Writingiswidelyconsideredbothanessentialskillforcollegegraduatesandakeypart
oftheundergraduatecurriculum.TheWritingAcrosstheCurriculum(WAC)movementandits
closecounterpart,WritingintheDisciplines(WID),advocateforcollegeresponsibilitytoteach
studentstowriteintheirchosenmajor,usingwritingasakeytoolforthinkingandlearning
throughwriting-to-learnactivities.TheInternationalNetworkofWACPrograms(INWAC)--
asubgroupofthelargestpostsecondaryorganizationofwritingteachers,theConferenceon
CollegeCompositionandCommunication--in2014issuedaStatementofWACPrinciples
assertingthat“writingshouldbeanintegralpartofthelearningprocessthroughoutastudent’s
education,notmerelyinrequiredwritingcoursesbutacrosstheentirecurriculum.”1Because
waysofknowingandwritingarediscipline-specific,writinginstructionismosteffectivewhen
guidedbythosewithdisciplinaryexpertise.Whilethecentralroleofwritingisobviousin
English,history,andphilosophy,commonlyreferredtoas“writing-intensivedisciplines,”the
roleofwritingislessvisibleinmorequantitativedisciplinessuchaseconomics.Yetwritingisan
essentialskillineconomics,andwritinginstructionandwritingassignments,asCohenand
Spence(1993)demonstrated,canbeintegratedintoeconomicscoursesinwaysthatenhance
ratherthandetractfromthecorecoursecontentandlearningobjectives.
Thispaperreportsonasuccessfulcollaborationbetweenaneconomistandawriting
specialistindevelopingmulti-partreadingandwritingassignmentsinalarge(over500
students),full-year,introductoryeconomicscoursethattakesaneconomicliteracyapproach,
ECO105IntroductiontoEconomicsforNon-Specialists.2Thecoursestructureandmaterials
describedherecouldbeworkinavarietyofcontexts,includinglarge,publicuniversities(like
Cohen&Williams,May2017
2
theonewhereweteach)whereclasssizesarelargeandinstructorsmustrelyonacadreof
graduatestudentswithlittletonoexperienceinteachingwritingtorespondtoandevaluate
studentwriting.Wewillshowhowwritingassignmentsandrelatedinstructionalmaterialscan
helpstudentslearnfundamentaleconomicreasoning.
Althoughwebelievethatwriting,becauseitiscloselylinkedtothinking,deservesa
placeinmosteconomicscourses,writingisaparticularlygoodfitforaneconomicliteracy
coursewheretheaimistopreparestudentstobecomeeconomicallyliteratecitizens–
thoughtfulandengagedpeoplecapablenotonlyofunderstandingbasiceconomicconcepts
andargumentsbutwhocanalsoconstructpersuasiveeconomicargumentsoftheirown.Hall
andPodemska-Mikluch(2015)showthebenefitsofop-edwritingassignmentsinencouraging
studentstoengagewithkeyeconomicconcepts.Similarly,theassignmentswepresenthere,
becausetheyinvolvecriticalthinking,reading,andwriting,helpstudentsengagewithkey
economicconceptsappliedtopolicyissues.Theiterativestructureoftheseassignments
requiresstudentstoread,draft,reviseandedit,andthenreceivefeedbackandread,draft,
revise,andeditagain,teachingstudentsfundamentaltransferableskillssuchascriticalreading
andwritingprocess.
Wepresenttheseteachingmaterialssothatinstructorsinterestedinintegratingwriting
intotheireconomicscourseshavepre-existingassignments,rubrics,andrelatedinstructional
activitiesandresourcestoadoptoradapt,ratherthanhavingtoreinventthewheel.The
assignmentandrelatedresourcesincludedherehavebeentestedforsixyearsonover3000
undergraduatesand,aswewillshow,areefficientandscalable.3
Cohen&Williams,May2017
3
Effectiveinstructionmustbealignedwithdisciplinary,curricular,andcoursegoalsand
contexts.Wefirstdiscussthecourseobjectivesrelatedtowritingandhowtheassignments,
activities,andresourcesmeetthecourselearninggoalsandfitourparticulardepartmentaland
institutionalcontexts.Efficiencyisnotonlyakeyconceptineconomics,butincreasinglya
centralconcerninhighereducationasinstitutionsareexpectedtoteachmorestudentswith
fewerdollars.Second,weaddresshowtheassignmentsmakeefficientuseoflimitedresources,
particularlyteachingassistant(TA)hours.Thethirdwayourcourseservesasapotentialmodel
isitsscalability:theseassignments,rubrics,andrelatedactivitiesworkinhigh-enrolment
courses(thiscourseenrolsapproximately500studentsannuallyintheface-to-faceversionand
300intheonlineversion).Forthosewhohavetheluxuryofteachingsmallernumbersof
students,theseassignmentscouldbeexpandedtoprovidestudentswithevenmore
opportunitiestoread,write,receivefeedback,andrevise.
INSTITUTIONAL,DEPARTMENTAL,ANDCOURSECONTEXTS
Writingisacomplexcognitiveskill.Teachingstudentstowritewellischallenginginany
institutionalcontext,andevenmoresoataninstitutionsuchasours.TheUniversityofToronto
hasahighpercentageofmultilingualinternationalstudentsaswellasalargecontingentof
multilinguallocalstudents.With71,000undergraduatestudents,manyclassesattheUniversity
ofTorontoenrolhundredsofstudentsofwhichover20%areinternationalstudents,mostof
whomaremultilinguallearners,alongwiththemanyCanadianstudentswhoaremultilingual.
Becauseofthelargeclassessizes,TAsplayakeyroleinteachingandevaluatingwriting.The
Cohen&Williams,May2017
4
TAsinthecoursediscussedherearemostlyeconomicsgraduatestudents(bothMAandPhD
students),manyofwhomarethemselves,liketheundergraduatestheyteach,multilingual
learners.4Ourinstitution,likemostCanadianuniversities,hasnorequiredfirst-yearwriting
coursesoifstudentsaretolearntowrite,theymustdosoindisciplinarycourses.Althoughthe
vastmajorityofourundergraduatesdonottakededicatedwritingcourses,theyhaveaccessto
writingcentreswheretheyobtainhelponwritingassignmentsinone-on-oneappointments
fromhighlyskilledinstructorsandgroupworkshops.5Ourassignmentsaredesignedtoutilize
theseexistinginstitutionalresources.Weencouragestudentstobookappointmentsatoneof
thecampuswritingcentreswheretheycangetfeedbackonadraftoftheirwritingassignment
inaone-on-oneappointment.TheeconomicsTAsdonotprovideguidanceorfeedbackon
assignmentsbeyondgradingthefinalsubmissions.
TheDepartmentofEconomicsispartoftheFacultyofArtsandSciences.6The
departmentalcontextforECO105issimilartotheinstitutionalcontextexceptthatEconomics
hasanevenhigherpercentageofmultilinguallearnersandalanguageotherthanEnglishas
theirfirstlanguagethantherestoftheFacultyanduniversity.7Economicshasparticipatedina
Faculty-wideWACinitiativecalledWIT(WritingInstructionforTAs)since2012andnowhasten
participatingcourses.Participationinvolvesincludingatleastsomewritinginacourseand
consultingwiththeWITCoordinatorandthetwograduatestudentswhoworkwiththe
CoordinatorasLeadWritingTAs(WITconsultants)ondesigningassignments,rubrics,and
tutorialactivities.ParticipatingintheWITprogramgivesinstructorsguidanceoncourseand
assignmentdesign,aswellasadditionalhoursforcourseTAstoreceivetraininginwriting
instruction,particularlyhowtorespondtoandevaluatestudentwriting.8TheLeadWritingTAs
Cohen&Williams,May2017
5
aretrainedandmentoredbytheWITCoordinator(AndreaWilliams)andinturntrainthe
courseTAsinprovidingeffectiveandefficientfeedbacktostudentsontheirwriting.
AviCohenhastaughttheECO105coursesince2011.Asanintroductoryeconomics
coursewithaneconomicliteracyapproach,thecourseexplicitly“introducesmicroeconomics
andmacroeconomicsasthebasisformakingsmartchoicesinlifeasconsumers,
businesspeople,investorsandinformedcitizensjudginggovernmentpolicies.”(SeeAppendixA
forthesyllabus).Althoughlessmathematically-orientedthanitscounterpartcourseformajors,
ECO105isarigorouscoursethatstudentsmayusetowardsaneconomicsmajoraslongasthey
earn80%ormore.9Thecourseistaughtinaface-to-faceandanonlinesection,bothofwhich
haveavarietyofassessments,includingonlinequizzes,groupprojects,termtests,finalexam,
andwritingassignments.Theface-to-facecoursehastwowritingassignment(articleabstract
andoneop-edoneitheramicroormacrotopic),theonlineversionhasthreewriting
assignments:anabstractofanarticleandtwoop-eds(onemicroandonemacro).10
WRITINGASSIGNMENTSANDRUBRICS
Webegancollaboratingonthefirstop-edwritingassignmentin2011andhavesince
revisedtheassignmentstructurealongwithrelatedrubrics,resources,activities,andTA
trainingtobettersupportboththeundergraduatesinthecourseandthegraduateTAswho
gradethestudents’writing.OnekeytenetofWACisthatwritingassignmentsbeclosely
alignedwithcoursegoalsandthattheytaketheformofcommongenresinthedisciplineso
thatstudentsseethewritingasalegitimatepartofthecourseandthefieldratherthanasbusy
work.AppendixAshowsthecoursesyllabus,whichincludestenspecificlearningoutcomes.The
Cohen&Williams,May2017
6
mostsalientoutcomeforthewritingassignmentsisforstudenttolearnto“Writepersuasive,
informedopinionpiecesaboutmicroeconomicandmacroeconomicpolicyissuesforageneral
audience.”Beyondthelearningoutcomes,thesyllabusalsoprovidesarationaleforthewriting
assignments:
Partofyourroleasaninformedcitizeninvolvesmakingsoundargumentsabouteconomicissuesthatmattertoyouandcommunicatingtheseargumentseffectivelytoothers.Theseassignmentswillgiveyoupracticeidentifyinganddevelopingconvincingwrittenargumentsaboutaneconomicissueforageneralaudience.
Theassignmentsasksstudentstowriteforgeneralratherthanexpertaudiencesbothtobetter
assesstheirunderstanding(it’seasytohidemisunderstandingbehindjargon)andgivethema
writingsituationmorecloselyresemblesthewritingtheywilldoaftergraduation:explaining
complex(economicorotherwise)ideastogeneralreadersratherthantoeconomicexperts.11
Scaffolding
Scaffoldingassignmentsinvolvesstructuringinwaysthatbreaklarger,morecomplex
tasksintosmaller,moremanageableonesthatsupportstudentlearning.Theideaisto
sequenceassignmentssothatstudentsgraduallyprogresstomorecomplexanddifficulttasks
withpracticeandformativefeedback.Thearticleabstractandop-edwritingassignmentsare
scaffoldedinmultipleways.Theop-edassignmentismostcloselyalignedwiththelearning
objectiveto“Writepersuasive,informedopinionpiecesaboutmicroeconomicand
macroeconomicpolicyissuesforageneralaudience.”Butundergraduates,especiallyfirstyear
students,areoftendaunted,ifnotparalyzed,attheprospectofmakinganoriginalargument.
Thearticleabstractassignmentprecedestheop-edassignment,providingpracticein
identifyingsomeoneelse’sargument,andgivingasenseofthestructureandcompositionof
Cohen&Williams,May2017
7
whatthestudentwillhavetodoherselfintheop-ed.Writinganeffectiveabstractinvolves
knowinghowtoreadcritically–toreadforargument.Beforethearticleabstractassignment,
wewalkstudentsthroughthestepsofreadingcriticallyanarticlefromTheEconomistabout
Ubersurgepricing.
Theassignmentsequencingtakestheformofthreemodulesthatwedescribeinmore
detailbelow.
1. HowtoReadCritically
2. Howto(Re-)WriteanAbstract
3. Howto(Re-)WriteanOp-Ed
Noticetheemphasisonre-writing.Thereisasecondformofscaffoldingwithineach
writingassignment.Aspartofteachingstudentstowrite,weprovideanexplicitthree-step
processforthewritingprocessthatasksthemto(1)draft,(2)revise,and(3),edittheirwriting.
Thereisanexplicit,grade-incentiverationaleforthisprocessintheassignmentdescription
(AppendixD):“Manynovicestudentwritersoftensubmitfirstdraftsandwonderwhytheyearn
poormarks.However,goodwritingalmostalwaysrequiresrewriting.”
Becauseoneofouraimsistogivestudentsaframeworkandvocabularyforwritingthat
theycantransfertoothercourses,disciplines,andcontexts,theassignmentdescriptionnot
onlynamesthisthree-stageprocess,butexplainsitinconsiderabledetail.Providingstudents
withaframeworkforwritingisparticularlyimportantforthemanymultilinguallearnersinthe
coursewhostrugglewithreadingandwritinguniversity-levelEnglish.Wedevelopedavideo
modulethatnotonlydescribesbutalsomodelsthecriticalreadingprocessandgivesstudents
practicereadingcriticallyasamplearticlefromTheEconomist(showninAppendixC).Aspartof
thewrittenassignmentforHowto(Re-)WriteanAbstractandrelatedvideomodule,weinclude
Cohen&Williams,May2017
8
ourownversionsofabstractsforthesamplearticle,includingaversionwithAviCohen’sre-
draftingandrevisionsshownintrackchanges(AppendixE).Bypresentingtwodifferent
abstractsofthesamearticlealongwiththerevisionprocess,weshowthatthereismorethan
onewaytowriteagoodabstractandthatevenexpertwritersmustrevisetheirdraftsto
producehighqualitywriting.ThethirdmoduleonHowto(Re-)WriteanOp-Ed,buildsonthe
processesinthepreviousmodules.
HowtoReadCritically12
Manystudentsentercollegewithinadequatereadingstrategies.Becausestudent
successonthewritingassignments(andinthecourseasawhole)requireseffectivereading
strategies,wedevelopedamoduletoteachthiscrucialskill.Themoduleconsistsofavideo
tutorialthatbothexplainscriticalreadinganddemonstrateshowexactlytodoitusingThe
Economistarticle,“AFareShake.”Thatarticleissimilartotheonesstudentsmustwritetheir
ownabstractsfor.AswereadthroughTheEconomistarticleandbeganseparatelywriting
respectiveabstracts,wecametoseejusthowcomplexthestepsinvolvedincriticalreadingare.
Ourvideooncriticalreadingbeginsbydefiningtheterm,distinguishingitfrom
criticizingordisagreeingwithatext,acommonmisconceptionthatmanystudentshavewhen
askedtocriticallyreadoranalyzeatext,event,orphenomenon.13Ratherthanemphasizing
criticizing,criticalreading(mostcollege-levelreading)involvesseveralkeymovesonthepartof
readers.Thefirstisunderstandingtheoverarchingargumentthatisbeingpresentedrather
thansimplyidentifyingtheparticularinformation,factsordatapresented.Thesecondis
identifyingthespecificclaimsbeingmadeandhoweffectivelytheseclaimsaresupportedwith
Cohen&Williams,May2017
9
evidence.Thethirdmoveisaskingquestionsofthetext.Thismayinvolvechallenging
assumptionstheauthormakes,identifyinggapsorcontradictionsintheargumentorevidence,
questioningsourcesofdataorevidence(asinthearticlewheredataprovidedbyUberisused
tosupportaclaim),challengingthetestimonyofexperts(suchasUberemployees),andnoting
counter-argumentstheauthorhasprovided(ornot).
Thesecriticalreadingstepsapplytomostdisciplines,soweexplainwhatcriticalreading
meansspecificallyineconomics,whichwedescribeas“beingabletoreadandidentifythe
economicconcepts,models,assumptions,reasoning,andevidenceanauthorpresents”
(AppendixG).Becausemoststudentshavereadlittleabouteconomicsbeforetakingthis
course,itisnotenoughtomerelyexplainwhattheconceptofcriticalreadingis.Werealized
thatweneedtoshowstudentshowtoreadcriticallyaneconomicstext.Wedecidedthebest
waytodothiswastopayattentiontoourownreadingprocessandAndreaWilliam’sin
particular.Althoughunlikethestudentssheisanexpertreader,likethestudents,sheisnotan
economist.
SoAndreaWilliamsdemonstratesherowncriticalreadingprocess:howsheapproaches
anunfamiliartextinanunfamiliardisciplineandmakessenseofit.Becauseofthemany
multilinguallearners,wethoughtitbestnottoassumethatanyreadingstrategywastoo
obviousorbasic.Andreabeginshercriticalreadingdemonstrationatthebeginning,withthe
title,whichshepointsoutissometimesoverlookedbyhurriedreadersbutwhichtheauthor
haschosencarefullyandwhichusuallyforecaststhearticle’sargument.Thetitleofoursample
articleonUber(“AFareShake”)involvesapunontheword“fare,”whichwepointoutforthe
benefitoftheEnglishlanguagelearnerswhomaynotrecognizethehomonym.Andrea
Cohen&Williams,May2017
10
William’sdemonstrationofcriticalreadinganalyzeschronologicallythemovestheauthor
makesandaddressesbothlocalissuessuchastopicsentencesthatforecastthesubjectofthe
paragraph(“Agoodtopicsentenceshouldtellyouwhataparagraphisabout”)andmoreglobal
observationsaboutthetext’sorganization(e.g.,thatitisstructuredasaproblem-solution).But
morethanmerelycommentoneachrhetoricalmovetheauthormakes,AndreaWilliam’s
criticalreadingdemonstrationincludesquestionsshehasofthetextsuchas:“Asa...
consumerwho...usesUberI’maskingofthearticle,‘Whynot?’Whyshouldn’tlocal
governmentsbansurgepricing?I’mexpectingthenextparagraphtoanswerthisquestion.”
(AppendixG).
Themodelofcriticalreadingwegivethestudentsincludesanalyzingthelanguageofthe
article.Forexample,AndreaWilliamspointsoutthatdescribingpayingdriversmoreduring
periodsofsurgepricingas“sharingwithdrivers”istheauthor’sattempttomakethecompany
seemgeneroustowardsitsdriversratherthanmerelyfairlycompensatingthem.Inadditionto
encouragingstudentstoattendcarefullytolanguageastheyread,ourdemonstration
addressestheuseoftwographs,whichlikeallvisualdisplaysofdata,playakeyroleinmost
economicarguments.Althoughthegraphsusedinthesampleargumentarequite
straightforward,asAndreapointsout,othersourcesofevidencearelessso.Forexample,
astudythearticleusestomakeacaseforsurgepricingwassuppliedbytheless-than-
independentsourceofUberitself,whichAndreaexplainsmakesitalessconvincingsource
hadthisevidencebeenprovidedbyaneutralthirdparty.
Cohen&Williams,May2017
11
Howto(Re-)WriteanAbstract14
Followingthecriticalreadingmodule,thefirstwritingassignmentsasksstudentsto
chooseoneofsixdifferentarticlesfromTheEconomistandwriteashort(250-300words)
abstractforitthatisunderstandabletoageneralaudience.Becausethegoalistoteach
studentshowtoanalyzeandconstructbasiceconomicarguments,wedecidedthatanabstract
wouldhelppreparestudentstowritethemorechallenginganalysisassignment(theop-ed
writingassignment).Theabstractassignmentisworth5%tomakeitalow-stakesassignment
thatgivesstudentspracticereadingandwritingwithoutunduepressure.Theassignment
containsthegradingrubric,providingadditionalguidancetostudentsaboutwhatweexpect,
andwhattheyneedtodotoreceivehighmarks(AppendixB).
Weallowstudentschoicetoensurethattheyhavethefreedomtoexploreatopicthey
findinteresting,butlimitthemtoarticlesthataregoodmodelsofwriting–TheEconomistis
thegoldstandardofeconomicsjournalism.TheselectionofsixarticlesalsomakestheTAs’
gradingtaskeasier.Allabstractswrittenaboutoneofthesixarticlesweregradedbyasingle
TA.
Tofurtherscaffoldthisassignment,weusepeerfeedbacktoensurestudentstheygive
andreceivecommentsontheirinitialdraftssincetheTAhoursareonlysufficienttogradeone
versionofeachassignmentandbecausestudentslearnfromreadingandrespondingtoeach
other’swork.UsingpeerScholarsoftware(www.peerScholar.com),eachstudentresponds
onlineanonymouslytothreepeers’assignmentsandinturnreceivesfeedbackfromthree
differentpeersontheirownassignment.Weaskstudentstoratetheassignmentona4point
scale,identifyonethingthattheirpeerdidwell,onethingthatneedsimprovement,andthen
Cohen&Williams,May2017
12
giveanoverallcomment.ThepeerScholarsoftwarealsohasahighlightingandcommenting
featuresostudentscanidentifyspecificwords,phrases,andsentencesthatareproblematic.
TheTAsmarkstudentsonthequalityofthefeedbacktheyprovide,assigningoneof
fourlevelsonasimplerubric:(1)Strengthsandweaknessesforeachassignedabstract,with
specifics;(2)Fairattemptbutweak,lackingspecifics;(3)Norealeffort;and(4)Missing.This
fostersaccountability,asthequalityoffeedbackstudentsprovidecountsaspartoftheiroverall
gradeonthewritingassignment.
UsingpeerScholaralsomakesiteasyforTAsnotonlytogradestudents’assignments
usingthescoringrubric,butalsoreviewthefeedbackeachstudenthasprovidedtotheirpeers.
Studentsdescribethefeedbackprocessasextremelyvaluableinmultipleways.Notonlydo
theyreceivesuggestionsforimprovingtheirabstracts,theylearnfromreadingothers’
assignmentsandprovidingfeedback.Manycommentongainingperspectivefromseeingother
students’work,ofbothhighandlowquality.
StudentssubmitafinaldraftoftheirabstracttopeerScholar,incorporatingsuggestions
frompeersoranyotherrevisions.TominimizethevariabilityacrossthedifferentTAgraders
andmaximizetheuseofthelimitedTAshours,wedevelopedarubricthatgivesstudentsas
muchinformationaspossibletoexplaintheirperformanceandjustifytheirgrade,yetiseasy
forTAstouse.AsAppendixBshows,therubricfortheabstracthasthreemaincriteria(main
economicargument,structure,andqualityofwriting),whichareassignedoneoffourlevelsof
competence(excellent,good,competent,andproblematic).Inweightingmostheavilythe
economicargument,therubricsignalstostudentsthattheirunderstandingoftheissuesis
Cohen&Williams,May2017
13
whatmattersmost;yetbyalsoincluding“structure”and“qualityofwriting”ascriteriathe
rubricremindsstudentsthathowclearlytheyexpresstheirunderstandingofeconomicissues
matters.
Todiscourageplagiarism,studentsmustsubmittheirfinaldraftstoTurnitin,the
software-matchingprogramourinstitutionsubscribesto.WeusedtheTurnitinsettingsthat
allowstudents,whentheysubmit,toseetheoriginalityreporthighlightingpassagesthat
containtextualsimilaritiestopapersinthedatabase.Theonusisonthestudentforfixingany
suspiciouspassages,anddraftscanberesubmitted(withneworiginalityreports)untilthedue
date.Thisdrasticallyreducesmarkingtimeforplagiarismdetection.15
Withtherubricsforpeerassessmentandthefinaldraft,markingtimeaveragedabout
5-6minutesperabstract.MarksareentereddirectlyintopeerScholar,andTAssimplyhaveto
pickonenumberforassessment,and3numbersforthefinaldraft.Themarksaretransferred
directlytotheLMS(inthiscase,Blackboard)gradebook.
Howto(Re-)WriteanOp-Ed16
Buildingontheabstractassignment,thesecondwritingassignment(AppendixF)and
associatedvideomodule(AppendixI)asksstudentstomovefromsummarizingtoanalyzingby
identifyingacontroversialeconomicargumentfromarecentarticle,editorial,oropinionpiece
(thepiecemusthavebeenpublishedsincethestartdateofthecourse).Studentsthenwritea
commentary(300-500words)inresponsetothearticleoropinionpieceinwhichtheymakean
argumentaboutaneconomicissue.Allowingstudentstobasetheirop-edasareactiontoa
publishedstoryisanotherformofscaffolding.Wedecideditwastoodifficultforstudentsto
Cohen&Williams,May2017
14
comeupwithanargumentontheirown,andusingapublishedsource(whichtheyhadto
includeintheirsubmission)minimizedtheneedforadditionalresearch.
Aswiththeabstractassignment,studentsareinstructedtofollowthethree-stage
writingprocess(draft,revise,andedit)andpeerreviewisarequiredcomponenttoprovide
studentswithformativefeedbackontheirinitialdraftandguidetheminmakingtheirrevisions.
Liketheassignmentitself,therubricfortheop-edassignment(showninAppendixF)ismore
complexthantheabstract.Themaincriteriaoftherubricare(1)Title,introduction,audience
(howappropriatethewritingisforageneralreader),andbackgroundinformation;(2)
Economicargument,reasoning,counter-argument,andevidence;(3)Structureandclosing;and
(4)Qualityofwriting.
Thisop-edassignmentiscloselyalignedwiththecorelearningoutcomesofthecourse
becauseitasksstudentstoengagewithkeyeconomicissuessuchasgainsfromtrade,
governmentversusmarketfailures,andenvironmentalpoliciesastheyexplainandargueforor
againstvariouspolicies.Byhavingtopersuadeanimaginedgeneralratherthanexpertreader,
studentsmustdomorethanmerelyparrotlecturesorthetextbookandinsteaddemonstrate
theirunderstandingintheirownwords.
Despitethedetailedassignmentinstructions,videotutorials,andpeerresponse,some
studentsstillstrugglewiththereadingandwritingelementsofthecourseandrequire
additionalsupport.Werecommendinallofourwrittenandvideoinstructionsthatstudents
seekhelpontheirwritingassignmentsatoneofourcampuswritingcentres.Inthisway,we
provideadditionalsupporttostudentswhoneedthiswithoutexceedingourcourseTAbudget.
Cohen&Williams,May2017
15
Sincemostcollegesanduniversitieshavewritingcentres,thisstrategywouldworkwellina
varietyofcontexts.
TATRAININGANDRESOURCES
OnedistinctivefeaturesoftheWITprogramattheUniversityofToronto,whichthe
initiative’snamesuggests--WritingInstructionforTAs--isitsdualfocusonimproving
undergraduatewritingandprovidingGraduateTeachingAssistantswithtrainingand
professionaldevelopmentinwritingpedagogy.WhereasmanyWACprogramsexportTAsfrom
Englishtoserveaswritingfellowsinotherunits,theapproachofWITistoprovideTAswiththe
necessarytrainingandsupporttoteachwritingintheirhomedisciplines.TAsarenotteaching
designatedwritingcourses;rather,theyareteachingwritingskillswithindisciplinarycourses
suchaseconomics.
Nowinitstenthyear,akeyfeatureofWITisitsuseofbenchmarkingorgrading
calibrationsessionstoimprovethequalityoffeedbackstudentsreceiveontheirwritingandthe
consistencyofthegrading,butalsoprovideprofessionaldevelopmentinteachingwritingfor
TAs.Foreachwritingassignment,thecourseinstructorandWITCoordinatorco-facilitate
sessionswiththeTAswhereweanalyzestudentwritingandscoreitaccordingtotherubric.
ThesesessionsgiveTAsachancetoaskquestionsabouttheassignmentandclarifythe
expectationsoftheinstructor.
Cohen&Williams,May2017
16
GradingCalibration
Thereweregradingcalibrationsessionsforeverywritingassignment–thearticle
abstract,andeachop-ed.Thesesessionslastedabout2hourseach,withtheTAs,Andrea
WilliamsandAviCohenaroundabigtablewithalargewhiteboard.Everyonebringsalaptop
foraccessingpeerScholar,wherethegradingtakesplace.Wejointlyselect5-6studentpapers
thatcovertherangeofqualityandgradingissues–strong/weakarguments,structure,writing,
andprosewrittenbymultilinguallearners.AfterAndreaWilliamsandAviCohengothrough
therubriccategories,everyonereadsandprivatelygradesthesameop-ed.Weputamatrixon
theboard,withtherubriccategoriesasrows,andthenamesofeachgrader(TAs,Andrea,Avi)
acrossthecolumns.
Wegoaroundthetable,haveeachpersonstatetheirmarksforeachrubriccategory,
andmoreimportantly,theirthinkingbehindthatmark.Wedon’tallowhalfwaymarksbetween
rubricgrades(excellent,good,competent,problematic),forcingquickerdecisionsandless
agonizing.Therewasoftenconsensusonmarksforindividualcategories,andmoreoften,close
totalmarksdespitedifferencesinindividualcategories.Butsituationswithlargedifferencesin
markswereoftenmostvaluable,fortheensuingdiscussionwouldclarifywhatwewerelooking
for.Wedonotprivilegeourownmarks(whichoftendisagreed),andusuallyspokeafterthe
TAs.WhenTAsweretornbetweenmarksforanindividualcategory,weencouragedthemto
compensateinadifferentcategory.Forexample,ifagraderwasequivocalbetweenexcellent
andgoodforthecategoryofEconomicArgument,shemightgiveexcellent,butthengofora
lowergradeforthecategoryofQualityofWritingwhereshewasequivocalbetweenexcellent
andgood.ThesediscussionshelptheTAsfeelcomfortableandconfidentintheirmarking.
Cohen&Williams,May2017
17
Theop-edassignmenthasanadditionalrubric(notpresentintheabstractassignment)
fortheReflection(“onwhatyoulearnedfromthewritingandassessmentprocess”)that
studentsmustsubmitwiththerevisedversionoftheirop-ed.TheReflectionrubrichasfour
levels:(1)Thoughtfulanddetailed,referstopeercomments,incorporatedrevisions;(2)Honest
attemptbutlackingspecifics—maximumifmissedassessments;(3)Norealeffort;and(4)
Missing.AccordingtothecognitivepsychologyresearchbehindpeerScholar,thecombination
ofcriticallyassessingothers,receivingandincorporatingassessments,andreflectingonthe
processenhanceslearning.17
HavingusedpeerScholarforyears,AviCohendevelopedanefficientworkflowforthe
orderofactivitiesinvolvedingradingeachassignment.Fortheop-eds,thoseactivitiesinclude
lookingatthemediastorylinkonwhichtheop-edisbased,readingthefinaldraftoftheop-ed
andassigninggradesineachcategory,readingthepeerassessmentsgiventothestudent,
comparingtheoriginalandfinaldrafts(thesoftwaremakesthiseasy,similarlytotrack
changes),andgradingtheReflection.Wethenlookatthepeerassessmentsgivenbythis
student,andassignamarkusingtheAssessmentrubric.Hecreatedashortscreencapture
video18walkingtheTAsthroughtheworkflow.WeestimatethatittakesaTA7-9minutesto
gradeeachop-edassignment.
OneofthemostchallengingissuesourTAsfaceingradingstudentwritingconcernshow
toevaluatethewritingofEnglishlanguagelearners(ELL).Tocounteractthewidespreadbut
falsenotionsthatallgoodwritingiserror-freeandthaterror-freewritingisalwaysgood,atthe
gradingcalibrationsessionweintroducetheideathatmanygoodwriterswriteEnglishwithan
“accent.”Inconversation,someonemaybearticulateandeasilyunderstoodbylistenerseven
Cohen&Williams,May2017
18
thoughtheyspeakwithanaccent(notlikeanativespeaker).Similarly,goodwritersmaywrite
withanaccentandmakeminorerrorsthatdon’timpedemeaning.Althoughweencourage
studentstoeditandproofreadcarefullytheirwritingassignments,wealsorecognizethatfor
mostEnglishlanguagelearnersitisimpossibletoproduce100%error-freeproseontheirown.
DuringthesessionwediscussstudentwritingsamplesthathelpTAsdistinguishbetweenmajor,
meaning-impedingerrors(suchasentencefragmentsandmajorproblemswithverbtense)and
moretrivialones(suchasmissingarticles).ManyoftheTAsarethemselvesmultilingual
learnerswhohavenevergradedwritingbefore.
SeeinghowtheTAsusetherubricandgettingtheirfeedbackonitenablesusto
improveitovertheyears.Forexample,thisyearwenoticedthatmanystudentsandTAsalike
werestrugglingwitharubriccategorythatincludedthephrase“useofeconomicconcepts.”
Therubricwasintendedtogetstudentstoactuallyapplykeycourseconceptslikeelasticityto
explainrealeventslikepricingdecisions.Instead,studentsinterpretedtherubricasrewarding
thementionofeconomicconcepts,whichtheythensprinkledinwithoutnecessarilyexplaining
them.Asaresult,werevisedtherubrictosay“useofeconomicreasoning”andthewritinghas
sinceimproved,withfewerstudentsresortingtousingeconomicjargon.
TAHours
Whatdothesewritingassignments“cost”intermsofTAhours?Beyondtheestimates
of5-6minutespergradedabstract,and7-9minutespergradedop-ed,wecanonlygiverough
approximationsbecauseTAwritingresponsibilitiesarecombinedwithtraditional
Cohen&Williams,May2017
19
responsibilitieslikeinvigilatingandmarkingtestsandexams.Eventheonlinesectionhasan
in-personmidtermandfinalexam.
Forthethreeacademicyearsof2013/14–2015/16,theECO105full-yearcoursewas
allocated1050TAhoursfor500students,or2.1TAhoursperstudentperyear.The2016/17
academicyearwasthefirstwithbotha500studentface-to-facesectionanda300studentfully
onlinesection.TAresourceswerecombinedforthesections,with1505TAhoursfor800
students,or1.88TAhoursperstudentperyear.SomeoftheTAhourswereunused,sowe
estimatethatthecourses,includingtheabstractandop-edwritingassignments,need
approximately1.75TAhoursperstudentyear.
OtherTAresponsibilitiesincludeinvigilating6hoursoftermtests(4onehourtestsfor
theface-to-facesection,1twohourmidtermfortheonlinesection)and5hoursoffinalexams
(2hoursforface-to-face,3hoursforonline).Termtestsandexamsconsistofacombinationof
true/false,multiplechoice,andshortanswerproblems.WhiletheheadTAdealswith
computerizedmarkingoftrue/falseandmultiplechoicequestions,TAsgradetheshortanswer
problems,whichconstituteabout25%oftotaltest/exammarks.Theothermajorresponsibility
isansweringstudentquestionsoncoursediscussionboards(forbothsections).Wehave
eliminatedin-personofficehoursforlackofdemand,andtherearenotutorialsforthese
courses.
Wehopethesenumbersgiveyouatleastsomeabilitytojudgethe“affordability”of
implementingwritingassignmentsinyourowncourses.Suggestionsaboutwhatother
informationwouldbeusefultohavearemostwelcome.
Cohen&Williams,May2017
20
CONCLUSION
Readingandwritinginstructionandassignmentscanbeseamlesslyintegratedintoan
economicliteracycoursetohelpstudentslearnhowtoapplykeyeconomicconceptswhile
improvingtheirwriting.Weprovidestrategiesforrealizingpedagogicaleffectivenessand
efficienciesinwritingpedagogies—short,scaffoldedassignmentswithpeerfeedbackandarich
arrayofwrittenandvideotapedinstructionsandguidedpractice—thatcanbeusedinavariety
ofcontexts.Theseassignmentscanbeadoptedinlargeclassesinotherinstitutionalcontexts,
orusedinsmallerclasseswhereinstructorscouldintroduceadditionalassignmentsormeet
one-on-onewithstudentstoimprovetheirwriting.Wehavepresentedherepracticalstrategies
forintegratingwritingintolargecourses.
Anecdotally,weseesignificantimprovementsinthestudents’writingbetweenthe
draftsoftheabstractandthefinalversions,butitwouldbeusefultoanalyzestudentwriting
moresystematicallytoseeinwhatareasstudentsmostandleastimproveandhowwecan
adapttheassignmentsandactivitiesaccordingly.Althoughwehavedevelopedrobustcourse
andwritingassignments,goodteaching,likegoodwriting,involvesongoingrevision.Having
developedeffectiveassignmentsandrubricsweplantopilothavingTAsaddageneral
commenttostudentsontheirwritingassignmentasmeansofprovidingadditionalformative
feedbacktostudents.Weanticipatecontinuingtorefinethewritingelementstoensurethat
studentsgetasmuchguidedpracticeandfeedbackpossiblegiventheconsiderableresource
constraints.
Cohen&Williams,May2017
21
REFERENCES
Cohen,A.andJ.Spencer.1993UsingWritingacrossthecurriculumineconomics:IsTakingthePlungeWorthIt?TheJournalofEconomicEducation.24,no.3,(Summer):219-230.http://links.jstor.org/sici?sici=00220485%28199322%2924%2924%3A3%3C219%3AUWTCI%3E2.0.CO%3B2-P
Hall,J.andM.Podemska-Mikluch.2015.TeachingtheEconomicWayofThinkingthroughOp-eds.InternationalReviewofEconomicsEducation,19,(May):13-21https://doi.org/10.1016/j.iree.2015.05.002
TheInternationalNetworkofWACPrograms(INWAC),StatementofWACPrinciplesandPracticeshttps://wac.colostate.edu/principles/statement.pdf
King,A.2002.Structuringpeerinteractiontopromotehigh-levelcognitiveprocessing.TheoryintoPractice,41,33-39.
McLeod,SusanH.1992.WritingAcrosstheCurriculum:AnIntroduction.InWritingAcrosstheCurriculum:AGuidetoDevelopingPrograms.Eds.SusanH.McLeodandMargotSoven.NewburyPark:SagePublications.1-11.
Russell,DavidR.2002.WritingintheAcademicDisciplines:ACurricularHistory.2ndEd.Carbondale:SouthernIllinoisUniversityPress.
Venables,A.,&Summit,R.2003.Enhancingscientificessaywritingusingpeerassessment.InnovationsinEducationandTeachingInternational,40,281-290.
Vgotsky,L.S.1978.Mindinsociety:Thedevelopmentofhigherpsychologicalprocesses.Cambridge,MA:HarvardUniversityPress.
Cohen&Williams,May2017
22
ENDNOTES
1ForanoverviewofWACandWIDseeRussell(2002)andMcLeod(1992).2AttheUniversityofToronto,studentsmustchooseeitheranintensivespecialistmajor,adoublemajor,oramajorandtwominors.With40one-semestercoursesrequiredforadegree,aspecialistmajorhas24courses,amajorhas14courses,andaminorhas8courses.Specialistmajorsgenerallyarepreparingforgraduateschool.StudentswhotakeECO100IntroductiontoEconomics,areeligibleforalleconomicsdegrees.TheunsexynameforECO105,IntroductiontoEconomicsforNon-Specialists,isaccurateinthatitallowsstudentstopursueaneconomicsmajororminor,butnotaspecialistdegree.ECO105attractsstudentsdoingmajorssuchasinternationalrelationsandpublicpolicy,whichrequireatleastoneeconomicscourse,andstudentsinotherscience,socialscience,andhumanitiesdepartmentslooksforanelectiveeconomicscourse.http://calendar.artsci.utoronto.ca/crs_eco.htm#ASSPE14783AppendicesA-Fcontainallprintmaterials.AppendicesG-Icontaintheslidesfromthevideomodules.4MostTAsareeconomicsMAorPhDstudents,butbetween¼and½oftheTAsaregraduatestudentsinthe2yearMaster’sofPublicPolicy(MPP)program,whichrequiresafullyearmicro/macrocoursetaughtbyanEconomicsfacultymember.WhiletheEconomicsqualificationsoftheseMPPstudentsarenotasstrong,theyhaveturnedouttobeexcellentTAs–moreversedinwriting,policyapplications,andtheexperienceoflearningsomeeconomicsforitsapplicability.Atthestartofeachyear,AviCohengatherspreferencesfromtheTAsformarkingproblemsontests/examsversuswritingassignments.MPPTAsarefarmoreinterestedin,andusuallybetterat,markingwritingassignments.5TheFacultyofArtsandScience,wherethedepartmentofeconomicsissituated,hassevencollegewritingcenterswithapproximately30full-timeequivalentinstructorsandtwiceasmanyadjunctinstructors.TheUniversityofTorontoasawholehas9undergraduateWritingCenterson3campuses.http://writing.utoronto.ca/writing-centres/6TheDepartmentofEconomicshas62full-timetenuretrackfaculty,and7full-timeteachingstreamfaculty.7It’simpossibletogetaprecisenumberforthepercentageofEnglishLanguageLearners,sincethereisnodefinitionofsuchacategoryandtheUniversitydoesnotadministeralanguageaptitudetesttoallincomingstudents.WehavedataonthenumberofincomingstudentsrequiredtoshowproofofEnglishlanguageproficiency(i.e.TOEFLorothertestscore),butthisappliesonlytopeoplewhodidn'tcomplete4yearsofhighschoolinOntarioanddoesnotcoverthemanystudentswhoaremorecomfortableinalanguageotherthanEnglish,whethertheyareCanadians,permanentresidents,orvisastudents. 8NormallyWIT-designatedcoursesreceiveupto280additionalTAhoursovertheirbasehoursforteachingandgradingwriting.9Studentswhoreceiveagradeof67%inECO100or80%inECO105areeligibletogoontoupper-levelcoursesandtomajororminorineconomics.Allmajorsmustalsoreceiveagradeof63%inafull-yearcalculuscourseasapre-requisiteforintermediatemicroormacro.
Cohen&Williams,May2017
23
10Theonlinesectionwasofferedforfirsttimethisyear.Differencesinthestructureoronlinecoursescontributedtodecisiontoaddasecondop-edwritingassignment.11AccordingtoCohenandSpencer(1993,221),“Studentsfaceauniquedifficultyinclarifyingtheaudienceforwhomtheyarewriting.Inmostwritingsituations,thewriterwritestoinformareaderwhoisgenerallylessknowledgeable.Insharpcontrast,studentsareaskedtowriteforareader(theinstructor)whoisclearlymoreknowledgeable.Thisplacesthestudentwriterinanartificialandhencedifficultsituation.’TheECO105writingassignmentsallspecifytheaudience:“Youraudienceisthegeneralreadingpublic.Assumeyouraudiencehassomeeducationandbackgroundincurrentaffairsandunderstandsbasiceconomicconcepts,butisnotknowledgeableaboutthedetailsofeconomictheoryorpolicy.Inotherwords,you’llneedtoexplainanyeconomicconceptsandspecializedvocabularyinawaythatkeepstheirinterestandrespectstheirintelligence.YouraudienceisnottheprofessorortheTAs.Youneedtowritemorelikeajournalistthananacademic.”Becauseofthepeerreviewprocess,theactualaudienceisfellowstudents.12SlidesforthevideoareinAppendixG.13Weallowthatcriticalreadingsometimesinvolvesdisagreeing.14SlidesforthevideoareinAppendixH.15Theinstructionstostudentsare:“TurnitingeneratesanOriginalityReporthighlightingsuspiciouspassages.Clickonhighlightedpassagestolinktotherelatedinternetsource.Ifthesourceisastudentpaperfromyourclassthiswillbeindicated.Ifthesourceisastudentpaperfromanotherclass,yourinstructorwillreceivecontactinformationforthatinstructor.EachOriginalityReporthasascorebasedonthepercentageofmatchingtext.ThesereportsdoNOTindicatewhetheryouhaveplagiarized,butsimplyhelptoidentifysourcesthatcontaintextualsimilaritiesinsubmittedpapers.Ifyoufindsignificanttexturalsimilaritiestouncreditedsources,youshouldmodifyyourpaper.YoumaysubmityourpaperasmanytimesasyoulikebeforetheduedatetorechecktheOriginalityReport.”16SlidesforthevideoareinAppendixI.17ThepowerofpeerassessmenttoenhancelearningfitswellwiththeviewsofVygotsky(1978)andhasbeensupportedempiricallyandtheoreticallyviamanyresearchstudies(e.g.,King,2002;Venables&Summit,2003).18https://play.library.utoronto.ca/o8oeVCYLTXRn