Influencing critical engagement with assessment criteria

Darrin Beattie, Careers Service Curriculum Development Officer

Coherent Curriculum themes: assessment and feedback; student engagement

Career Development Modules (CDM)

• Suite of 4, year long, 20 credit work based learning modules for levels 5, 6 & 7

• 500+ students from a range of academic schools representing all three faculties

• Part of Project 2012 • Assessment for learning• Transparency in assessment processes

Intervention: annotations

• Student use of annotations within CDM assignment 1 to identify specific passages that they believe are the strongest evidence for each assessment criterion.

Intervention’s aim and objectives

• Raise lower quartile of CDM marks by:• Promoting critical engagement with

assessment criteria• “Testing” student understanding of

assessment criteria• Generating more effective feedback• Evolving assessment criteria

Assessment criteria

Newcastle(CDM)/Northumbria(SiS) work-related learning modules assessment criteria 2011/12 Final Year Students Classification Fail Third 2:2 2:1 First

Strand and theme 0 25 30 39 40 49 50 59 60 69 70 79 80 90 100 Critically analyses and evaluates how goals and key issues relevant to self and placement influenced actions

Shows little or no understanding of goals or key issues relevant for self or placement

Identifies goals and key issues for self and/or placement that influenced actions

Discusses how goals and key issues for self and placement influenced actions

Consistently uses critical analysis when discussing how goals and key issues for self and placement influenced actions

Meets all criteria to this point and consistently uses critical analysis and evaluation across all strands of the assessment criteria

Consistently uses critical analysis and evaluation across all strands of the assessment criteria to instigate innovative practices within the host organisation

Applies research relevant to self and placement linked to goals and key issues

Provides little or no evidence of applying knowledge in practice

Provides one example of new or existing knowledge applied in practice or refers to sources of limited quality

Provides examples of knowledge applied in practice from a combination of appropriate primary and secondary sources

Consistently provides examples of knowledge applied effectively in practice from a combination of appropriate primary and secondary sources

Evidences development of competence in Graduate Skills at placement and during assessment, linked to goals and key issues

Provides no evidence of developed competence

Provides evidence of basic competence only, from placement and during assessment

Provides some evidence of appropriately developed competence from placement and during assessment

Consistently provides evidence of appropriately developed competence from placement and during assessment

Evidences beneficial outcomes for self and placement linked to goals and key issues

Provides no evidence of explicit outcomes

Provides limited evidence of explicit outcomes that benefit self and/or placement

Provides evidence of a range of explicit outcomes that benefit self and placement

Consistently provides evidence of a range of explicit and significant outcomes that benefit self and placement.

Notes 1. The classifications as described across the four strands of the criteria assume that all lower classifications have been satisfied. 2. The higher levels of Competence and Beneficial Outcomes are also addressed through Supervisor Evaluation via outstanding examples. 3. The expected level of academic skills and competent performance are those that are appropriate for a final year university student. 4. Engagement with the module will be considered where the final module mark is close to a grade boundary. 5. See the Glossary provided in the Module Handbook for an explanation of all terms used.

Assignment 1: Exemplar. student annotations and related feedbackRelated passage within feedback :

...there is limited evidence of analysis in your discussions of goals, key issues and actions and this will improve by adopting a more critical and evaluative approach e.g. did you consider what may be causing teacher attitudes so as to ensure that how you made your offer to support sporting activities was sensitive to such issues? Related passage within feedback :

“...there is evidence of both primary and secondary research in your assignment but when evident, has not been suitably developed as examples of applied learning e.g. what specifically did you learn about leadership through observing your supervisor that you then applied when leading small groups”?

Passage from feedback referring to unconscious competence?: ...Learning RX is a better example of applied learning in that you have clearly related the new knowledge acquired from this source (learning styles) and presented an explicit example of your applying this knowledge (forces activity). The highlighted passage relates general learning Learning RX relates an explicit example of applied new knowledge...

Unit reflections

Students are critically engaging with the

assessment criteria prior to submission

Commonly misunderstood criterion easily identified

Student selected passages for annotation suggest

unconscious competence in evidencing criteria

Easier to write formative feedback

“Whispered” student feedback

Glad it was not worth any marks but very useful and

unusual

I understand the need for a word limit but it was at times difficult to explain yourself to the fullest extent within the word limit

It helped me grow from last year’s assignments and fully integrate each of the graduate skills in my role as a student tutor

Student feedback from MEQLikes: “The increased ability to evaluate self-performance”; “the level of learning independence. I also enjoyed the assessments”; “Improves critical analysis and other skills”; “Focussed on assessment criteria”; “Improved focus and drive in general towards work and studies as a result of this module”; “The modes of assessment were good”

Dislikes: “Inflexible marking standard”; “Extremely specific marking scheme”; “The marking of the essay was very strict”; “The marking scheme is too "tick box" style”; “The formality of some examples, I struggled to understand some points”; “The assessment was repetitive and lacked clarification as to how to score higher marks”

“I don't think I could give three reasons, I love this module!”

Review of aim and objectives

• No discernible effect on CDM lower quartile mark, but intervention: • Has promoted critical engagement with

assessment criteria• Does “test” understanding of assessment

criteria• Produce more effective feedback• Will develop the assessment criteria

Issues

• Assessment criteria appearing in non-assessed annotation rather than assessed script

• Perception of “jumping through hoops”• “Blunderbuss” approach to annotations

render information meaningless

Developments

• Annotation minus rationale• More “effective” pitch to students• Feedback relating effective as well as less

effective examples to specific criterion• Deeper analysis of impact