Nkte DurhanMETU, Northern Cyprus Campus, School of Foreign Languages(Ankara, 30 May 2012)

General look at socio-cognitive frameworkCognitive validity (theory-based) and routine skillsNeeds analysis results of the Academic Speaking Skills (211) CourseDescription of the speaking tasks in the course and evaluation from cognitive validity aspectsContext validity: considering its relevant aspects and the speaking tasksAreas that need work: scoring validity

Performance tests are characterised by the relationship of the test task to the world beyond the test. Ideally tasks aredirect and authenticsituationally authenticinteractionally authenticBut a test can never fully represent reality.

We want candidates toperform relevant language tasksadapt their speech to circumstancesmake decisions under time pressureimplement them fluentlymake any necessary adjustments as unexpected problems arise

Test-taker characteristicsContext focusedCognitive focused (theory-based)Scoring focusedCriterion focused (external)Consequentially focused

The abilities to be tested are demonstrated by the mental processing of the candidate (the cognitive dimension)

The use of language in performing tasks is viewed as a social rather than a purely linguistic phenomenon (the socio- dimension)

Test construct resides in interactions of underlying cognitive ability and context of use, i.e. socio-cognitive model

Executive processesConceptualizerPre verbal messageLinguistic formulatorPhonetic planArticulatorOvert speechAuditionSpeech comprehension

(Weir,2005)Executive resourcesLanguage knowledgeGrammaticalTextualFunctional SocialContent knowledgeInternal External

Routine Skills (Bygate, 1987)

Informational Routinesexpository routines: involve factual informationevaluative routines: involve drawing of conclusions, expression of reasoning; explanation, predictions, justifications, preferences and decisions

Routine Skills (Bygate, 1987)

Interactional Routines : telephone conversationsinterviewsmeetingsdiscussionsdecision making tasks

Routine Skills (Bygate, 1987)

Interactional Routines : They are assisted byinteraction management skills involving content-focused agenda management and interaction-focused turn-takingImprovisational Skills brought into play when an interaction falters

Informative Speech (individual) (10%)Persuasive Speech (group) (20%)Final Presentation (individual) (30 %)

No assessment of discourse and strategic competences

Participants want more of the following:More focus on spontaneous speech A variety of speaking activitiesMore presentations with smaller percentagesDifferent types of presentationsFocus on fluencyInteractive activitiesCreativity and individualitySpeech critiques

Debate Role playsDiscussionsImpromptu speechPronunciation activitiesCritical thinking activitiesAsking & answering questions

Reporting the teams observation of a stores merchandising strategies and store design

Organizing and sharing points in the groupPresenting findingsGiving evidence from the observationsEvaluating the effectiveness of the strategies

To inform and enlighten the audience about a given topic on the general theme of technology

Choosing a news item related to art, analyzing it and presenting the analysis to classIncludes: justification of the selection of the itemreaction to points and styleevaluation of the ideas

Giving a balanced view of the selected topic (related to the five themes in the book)Includes:extensive literature researchreaction to the presented material/discussion or evaluation of the topic

DebatesDiscussionsPanel discussion Role playsConflict resolution tasks

Three different activities should be carried out in class throughout the semester.

Setting: taskPurposeResponse formatKnown criteriaWeightingOrder of itemsTime constraintsSetting: administrationPhysical conditionsUniformity of administrationSecurity

(Weir,2005)

Demands: taskLinguisticChannelDiscourse modeText length Nature of informationTopic familiarityLexical rangeStructural rangeFunctional range

1. Rubric

Is the rubric accurate and accessible?

2. Purpose

Is the purpose of the test made unequivocally clear for the candidate?

3. Weighting

Are any weightings for different test components adequately justified? 4. Known criteria

Are the criteria to be used in the marking of the test explicit for the candidates and the markers?

5. Time constraints: particularly important for speech as normal speech takes place under time pressure

Is the timing for each task (e.g. preparation and completion) appropriate?

6. Discourse mode:Reciprocity conditions are concerned with the dimension of interpersonal interaction, the relation between speaker and listener.

Are there reciprocal exchanges where speakers have to adjust vocabulary and message and take each others contributions into account?

7. Type of informationIs the type of information appropriate for the target situation requirements of the students being tested?

8. Content knowledgeIs the content sufficiently familiar to candidates so that they have sufficient existing schemata to enable them to deploy appropriate skills and strategies? (Urquhart & Weir, 1998)

8. Content knowledgeevery attempt should be made to allow background knowledge to facilitate performance rather than allowing its absence to inhibit performance (Alderson, 2000)*

9. Linguistic knowledgeLexical rangeAre the lexical items in the test both in input text and required as output appropriate for the level of the candidates?Syntax Are the grammatical items in the test both in input text and required as output appropriate for the level of the candidates?

9. Linguistic knowledgeFunction: is a term used to describe the illocutionary force of what is said.Are the functions in the test both in input text and required as output appropriate for the level of the candidates? The variety of tasks with different functional purposes and increasing number of tasks improve the validity.

RatingCriteria / rating scale

Rating proceduresRater selectionRater trainingStandardisationModerationRating conditionsStatistical analysis

dnukte@metu.edu.tr

******