A Pedagogical Imperative of Pedagogical Imperatives. Lewis Gordon.
Professor, dr.philos. Arild Raaheim · Digital assessment –pedagogical possibilities Keynote at:...
Transcript of Professor, dr.philos. Arild Raaheim · Digital assessment –pedagogical possibilities Keynote at:...
Digital assessment – pedagogical possibilities
Keynote at:
Pedagogical Conference, October 27th 2016
Uppsala University
Professor, dr.philos. Arild Raaheim
Exams…Assessment…Digitalization…Torgny Roxå:
«It’s NOT OK to just have an opinion about assessment.” “It is a whole lot of difference between
having and opinion and putting forward an informed opinion.” https://www.youtube.com/watch?v=MEXJF1ncNwk
UNIVERSITETET I BERGEN2017-01-17 SIDE 2
We can all learn what we want …
but we have to believe in it…
and others must have faith in us!A. Raaheim, 2011.
Was shocked when I saw
the «examination room».Psychology students were sitting
for 5 hours in a zebra-crossing.
(BT 29.05.13)
Scholarship of teaching and learning
(SoTL):
• Knowledge/information
• What do we know about learning and teaching?
• Planning/reflection
• When, how, what, and what when?
• Communication
• What is effective in different settings?
• Evaluation
• Why did I (not) succeed?
UNIVERSITETET I BERGEN2017-01-17 SIDE 3
The way we assess students determines:
- What students learn
- How students learn
- Students’ conceptions of knowledge (what does it mean to “know”?)
• Assessment is the most powerful tool we
have to direct students’ learning.
• What is it that we want to assess?
UNIVERSITETET I BERGEN2017-01-17 SIDE 4
Why should we digitalize?
• Because students demand it of us?
• Because we as teachers wish to develop our
skills or demonstrate that we are up-to-date?
• Because this is the future?
• Because there are good reasons to believe that
this will have a positive effect on students’
learning?
UNIVERSITETET I BERGEN2017-01-17 SIDE 5
Teaching in the digital age – two routes
Low-threshold route
• More advanced PP-
lectures
• Use of clickers/Kahoot
• Smartboards
• LMS: in/out
• Exam via PC
High threshold route
• Reduce number/length of lectures
• Introduce SMOOC’s
• Open source exam
• Online-conferencing
• Local-interactive «distance» teaching
• Learning labs for students’ (video) productions
• New architecture
UNIVERSITETET I BERGEN2017-01-17 SIDE 6
Digital exams at the University of Bergen
UiB is Norway’s leading institution when it comes to digital exams:
• Inspera Assessment
– 55% of all school exams are digitalized in spring 2016
– an estimated 61% in autumn 2016
– max 62% with today’s system
Technical needs:
Of the remaining 38%:
• 60% formulas (math, chemical, economic, etc)
• 74% special characters (field specific)
• 47% drawings
• 22% third party integrations (56% of these are drawings of chemical
structures, 37% compiling programming code, 7% other systems).
UNIVERSITETET I BERGEN2017-01-17 SIDE 7
Exams – historical background
• The examination system was systematized in the Sui Dynasty (581–618) as an official method for recruiting bureaucrats. It was intended to ensure that appointment as a government official was based on merit and not on favoritism or heredity.
• Theoretically, any male adult in China, regardless of his wealth or social status, could become a high-ranking government official by passing the imperial examination.
• Candidates were tested on their knowledge of the Confucian classics, their ability to write, and the "Five Studies:" military strategy, civil law, revenue and taxation, agriculture, and geography. Though only about 5 percent of those who took them passed, the examinations served to maintain cultural unity and consensus on basic values and ensured the identification of the educated elite with national, rather than regional, goals and values.
http://www.newworldencyclopedia.org/entry/Imperial_Examinations_(Keju)
UNIVERSITETET I BERGEN2017-01-17 SIDE 8
The exam – some key features
• Control
• Standardization
• Objectivity
• Confidensiality (external examiner/second opinion)
• Reliability
• Validity– internal - external
UNIVERSITETET I BERGEN2017-01-17 SIDE 9
Assessment – more than exams
• Summativ assessment
• Formativ assessment
• Peer assessment
• Self assessment
Assessment of, for, and as learning.
UNIVERSITETET I BERGEN2017-01-17 SIDE 10
The «Big-five» of assessment
1 2 3 4 5
Reliable
Valid
Authentic
Transparent
Fair
UNIVERSITETET I BERGEN2017-01-17 SIDE 11
Very low Very high
Assessment drives learning
Which rises the following question:
• What is it that we want students to learn?
- recall?
- describe?
- explain?
- discuss
- assess?
- evaluate?
- lead a critical discussion?
- demonstrate?
And: What gives us reason to believe that all of the above
can (best) be assessed by using a traditional exam?
Constructive alignment
An ocean of possibilities.
Variation along the following dimensions:
• Level– Whether students are assessed alone or together with others
• Arena– Where assessment is carried out (at the institution, in natural
situation outside the institution, in a virtual room)
• Form– Whether assessment is based on written material, practical
(artistic) work or an oral presentation
• Format– Summative or formative assessment, or a combination of both
Constructive alignment - Rubrics
A rubric is a scoring tool that lists the criteria for a piece
of work, or “what counts” (for example, purpose,
organization, details, voice, and mechanics are often
what count in a piece of writing); it also articulates
gradations of quality for each criterion, from excellent to
poor.
Learning for an unknown future.Content-focused vs capability-focused curriculum.
Knowledge capability:
• work out the key aspects to be dealt with in each new situation
• relate these aspects to knowledge already acquired and/or to other
knowledge the person knows how to access
• determine what the task or problem in the new situation might be
• design a process or solution to deal with the situation, and then
• have the ability to follow through and complete the task or solve the
problem, either alone or with others.
«…knowledge capability … is achieved through experience of variation,
rather than merely having varied experiences» (s. 233).
Baillie, C., Bowden, J.A. & Meyer, J.H.F. (2013). Threshold capabilities: threshold concepts and knowledge
capability linked through variation theory. Higher Education, 63, 227-246.
If there are 50 ways to leave your
lover….
Ooh, slip out the back, JackMake a new plan, StanYou don’t need to be coy, RoyJust listen to meHop on the bus, GusYou don’t need to discuss muchJust drop off the key, LeeAnd get yourself free
© 1975 Words and Music by Paul Simon
There must be a million ways to assess students…
40 alternatives to the traditional written exam:Eksamensrevolusjonen. Gyldendal Akademisk, 2016.
Skriftlig eksamen
Alternativ 1: Med medbrakt «fuskelapp»
Alternativ 2: Med innlagt mulighet for innhenting av informasjon/diskusjon med medstudenter
Alternativ 3: Med åpenhet til alle typer kilder
Alternativ 4: «Take away» eksamen
Alternativ 5: Individualisert eksamen
Alternativ 6: Objektiv prøve (multiple choice)
Alternativ 7: Omvendt objektiv prøve
Alternativ 8: Studenten som sensor
Alternativ 9: Stasjonseksamen
Alternativ 10: Fagartikkel
Muntlig eksamen
Alternativ 11: Med tid til forberedelse
Alternativ 12: Som forberedt klage/ankemulighet
Alternativ 13: Posterpresentasjon
Alternativ 14: Praktisk muntlig
Alternativ 15: Disputas
Alternativ 16: Foredrag
Alternativ 17: Intervju
Andre vurderingsformer
Alternativ 18: Bidrag på vitenskapelig konferanse
Alternativ 19: Mappevurdering
Alternativ 20: Virtuell konferanse
Alternativ 21: Praktisk oppgave utenfor institusjonen
Alternativ 22: Oppdragsvirksomhet
Alternativ 23: Logg
Alternativ 24: Intervju av fagperson
Alternativ 25: Utplassering
Alternativ 26: Prosjektpresentasjon
Alternativ 27: Vurdering av medstudenter
Alternativ 28: Kronikk
Alternativ 29: Litteraturanmeldelse
Alternativ 30: Kursanmeldelse
Alternativ 31: Planlegging av undervisningsopplegg
Alternativ 32: Blogginnlegg
Alternativ 33: Facebook-gruppe
Alternativ 34: Film
Alternativ 35: TBL-aktivitet
Alternativ 36: Arrangere faglig aktivitet
Alternativ 37: Analyse av vurderingsformen
Alternativ 38: Studenten som underviser
Alternativ 39: Pasienten som sensor
Alternativ 40: Ekstern sensur
Team based learning (TBL)
Four pillars/principles:• Permanent teams (5-7students)
• Readiness assurance process (RAP)
– Prereading
– iRat
– tRat
– Appeals
• 4-S application activities
– S1 – significant problems
– S2 – specific choices/answers
– S3 – Same problems
– S4 – simultaneous reporting
• Peer evaluation
http://www.teambasedlearning.org/
The student as examiner – critical
thinking
The student is provided with a full text (answer)
and asked to assess it:
Is this a good answer/paper?
- why/why not?
- what is lacking?
- which part(s) of the literature is lacking, why?
- which mark would you suggest, why?
Assessment based on a set of explicit criteria.
Full access to all sources
A fully digitalised exam instead of today’s digital
school exam;
- puts high demands on formulation of tasks (as the point is not to
assess students’ ability to reproduce what they have read, but their
critical use of different sources)
- puts high demands on those who assess students
- assessment criteria have to be spelled out
Flipped MCQ-test
Objective tests have limited value; may test lower cognitive skills like memory and understanding (declarative knowledge not procedural knowledge).
a) - provide students with an already finished MCQ
test
- ask students to argue why particular answers are
correct/wrong
b) - students develop an MCQ-test
- write a reflective note in which they argue for
choice of questions/answers
A virtual conference
Use existing LMS:
- conference «here and now» (x no. days)
- group assignment/work
- oral presentation
- accepted paper (supervision account)
- full paper distributed in advance
- conference organised in sessions with moderator
- feedback – external/international expert
- moderator and expert assess (peer assessment?)
Interview an expert
A good interview requires much preparation and insight in topic/field.
And that one has understood the interviewee and is able to refer
him/her correct.
- interview an expert
- write an article (newspaper)
- citation check: the interviewee writes a short note:
- did the student get the message?
- was I correctly referred?
- did the student ask relevant/good questions?
- some questions that was not asked?
- how well had the student prepared him/herself?
Publish the article? Own net page or «public»?
Workplace training: illustrate relevance, promote
motivation and learning, and build identity!
And write individual blog:
http://biopraksis.b.uib.no/
Blog
Workplace controls/accepts attendance (3 weeks of practice). Students complete a portfolio:
– 4 blogs
– Reflective paper
– Oral presentation
• Pass/no pass
http://biopraksis.b.uib.no/
Gaute Velle Torstein Nielsen Hole
UNIVERSITETET I BERGEN2017-01-17 SIDE 26
Teach to Learn – creativity,
communication and learning.
TE2LE
The Teach to Learn (TE2LE) project aims to teach BSc and MSc students in biology cross-disciplinary transferable skills through the application of an innovative pedagogical practice.
– 4 min instructional videos • Biolab (Bio 101)
• bioST@TS (AB 204) co operation with MatRIC (Univ. of Agder)
Jonathan Soulé Anne-Laure Simonelli Lucas M. Jeno
UNIVERSITETET I BERGEN2017-01-17 SIDE 28