Post on 03-Dec-2014
description
Wanted! Evidence based guidelines for
unseen invigilated examinations
Ieva StupansUniversity of South Australia
Three interrelated objectives for quality in student assessment in
higher education1. Assessment that guides and encourages effective
approaches to learning
2. Assessment that validly and reliably measures expected learning outcomes, in particular the higher-order learning that characterises higher education; and
3. Assessment and grading that defines and protects academic standards
EDUCATIONAL EFFECTIVENESS
(McInnis, 2004)
Wanted! Evidence based guidelines for unseen invigilated examinations
• EDUCATIONAL EFFECTIVENESS
• RESOURCE EFFICIENCY
Wanted! Evidence based guidelines for unseen invigilated examinations
• Analysis of marks on 1,712 modules at Oxford Polytechnic. Modules with 100% coursework had an average mark 3.5% higher than modules with 100% examinations, and there were three times as many failed students on modules where there were only examinations. (Gibbs & Lucas, 1997)
• Four universities - coursework marks higher by one third of a degree classification in English and History and higher by two thirds of a degree classification in Biology, Business Studies, Computer Studies and Law. (Bridges et al. 2002)
Wanted! Evidence based guidelines for unseen invigilated examinations
• Audit from an Australian university indicated that approximately 50% of first year subjects had final examinations within their assessment profile, within sciences this was approximately 80%. (Taylor, 2006)
• With respect to certification or accreditation of learning by an external body such as a health profession accreditation body, for example, FIP recommends ‘a final examination should lead to the granting of a diploma or degree’ . (FIP, 2000)
Wanted! Evidence based guidelines for unseen invigilated examinations
Assessment
for learning = coursework type assessments
of learning = invigilated examinations
Assessment
‘traditional’= learning at the lowest levels of Bloom’s cognitive domain
‘alternative’ = positioned to measure learners’ higher-level thinking skills of synthesis, analysis, and evaluation
Wanted! Evidence based guidelines for unseen invigilated examinations
Assessment of learning is not mutually exclusive to “alternative” assessment
but
is frequently associated with question spotting, cramming and short-term knowledge retention
therefore
the concept of structuring questions which allow students to demonstrate higher-level thinking skills is one which needs to be developed.
Returning to the unseen invigilated exam!
• “Blueprinting”
Assessment and grading that defines and protects academic standards
• problem solving MCQ
Assessment that validly and reliably measures expected learning outcomes, in particular higher-order learning
• “cheat sheets” and open book
Assessment that guides and encourages effective approaches to learning
Higher order learning
“Blueprinting” e.g. statsLevel of skill required
Standards Definitions Compre-hension
Application Analysis Problem solving
Total
Analyse data using mean, median etc
2 MCQ 2 MCQ 8 MCQ 1 written response
12 MCQ+written
Predict from data
6 MCQ 2 MCQ 1 written response
8 MCQ+written
“Blueprinting”
• Tension between efficiency and effectiveness (reliability)
• What is the optimal number of items, balance of items- MCQ, short answer and extended answer?
• Should components be weighted?• Assessment of different learning
outcomes but these are interrelated.
Problem solving MCQ
MCQ in Pharmacology• Questions which require application, analysis,
synthesis and evaluation can be written• Writing of such questions needs to be
intentional ( importance of “blueprinting”) • For NESB students to achieve similar results,
attention needs to be paid as to whether questions are assessing application of pharmacology knowledge or application of pharmacology knowledge and language proficiency.
Cheat sheets and open book
• Students have access to “essential reference books” in examination
• Does this impact on “quality” of answers particularly for weak students.
• Is there a way to optimise student learning using open book and “cheat sheets”
Cheat sheets and open book
Questions• Strategies such that there is motivation for
students to organize all of the materials in their minds prior to the exam
• Strategies so that exam time is not the time for students to work through the texts for the first time
Wanted! Evidence based guidelines for unseen invigilated examinations
Learning logs/diaries
Peer assessment
Annotated bibliographies Projects/case studies
Educational Effectiveness
Resource efficiency
Hornby, 2003
Unseen Examination