guidelines for the setting of examination papers in languages
General Framework for Setting Examination Papers and Test Papers
-
Upload
william-kapambwe -
Category
Education
-
view
6.942 -
download
2
description
Transcript of General Framework for Setting Examination Papers and Test Papers
GENERAL FRAMEWORK FOR SETTING EXAMINATION
PAPERS: TEST SPECIFICATIONS
PAPER PRESENTED BYW. M. KapambwePrincipal - TEVET
Examinations Council of Zambia
INTRODUCTION
You are a teacher, one day a colleague phones you and she tells you that she is ill. According to the programme she has to develop a test. Due to the illness he/she is not able to develop that test. Of course you are willing to help your colleague.What kind of information do you need to be able to develop the test?
According to The Standards for Educationaland Psychological Testing:
“…the validity of an intended interpretation of test scores relies on all the available evidence relevant to the technical quality of a testing system. This includes evidence of careful test construction…”
Content Definition and MappingWhat content is being measured?How is the measurement of the content domain(s) being operationalised?
Test Specifications– Attainment targets– Assessment objectives (Behaviour or
content)– Taxonomy– Test format– Test plan– Test matrix or test grid
Assessment Options for Different Learning DomainsAll human behavior can be divided into oneof three categories or domains: Cognitive
Affective
Psychomotor
The three categories promote different learning behaviors or objectives. The most commonly assessed behavior domain is the cognitive. Behaviours in the cognitive have been organized into general categories. Such an organization is called taxonomy (system of classification).Owing to the different learning behaviours, different assessment techniques are utilized.
Bloom’s Taxonomy
The Blooms taxonomy is the maintaxonomy used
Test Matrix or GridBehaviours
Know
Comp
Application
Synthesis
Evaluation
Total
1. Knowledge – define, recognize
2. Comprehension – explain, compare
3. Application – Calculate, apply4. Analysis – Analyze, Plan5. Synthesize – Design,
schematize6. Evaluation – Criticize,
Choose, judge
Taxonomy: Bloom
Bloom Taxonomy
Verbs for the cognitive domain Cog. Area
Know. Comp. App. Analysis Synthesis
Evaluation
Behaviours CountDefine
Identify
Label list
Match
Name
Outline
Point out
Quote
Recite
Repeat
Reproduce
Select state
trace
Classify
Compare
Convert
Discuss
Distinguish
Estimate
Explain
Generalise
Give
Examples
Infer
Interpret
Para
Rewrite
Summarize
translate
Change
Compute
Construct
Solve
Demonst-rate
Illustrate
Predict
Breakdown
Differentiate
Investigate
Relate
Separate
Subdivide
Arrange
Combine
Construct
Create
Design
Formulate
Genera-lise
Generate
Group
Integrate
Organise
summarise
Appraise
Conclude
Critique
Critcize
Grade
Judge
Justify
Interpret
Support
Recommend
Old & New Bloom’s Taxonomy
The six levels of Blooms taxonomy were changed from nouns to verbs as shown below.
The New Bloom’s Taxonomy incorporates not only the cognitive domain but also the affective and the psychomotor. This represents a holistic treatment of all the learning domains. The New Blooms Taxonomy has the following domains:– The cognitive - knowledge based domain,
consisting of six levels – The affective - attitudinal based domain,
consisting of five levels, and – The psychomotor - skills based domain,
consisting of six levels.
New Terms Explained
• Remembering: Retrieving, recognizing, and recalling relevant knowledge from long-term memory.
• Understanding: Constructing meaning from oral, written, and graphic messages through interpreting, exemplifying, classifying, summarizing, inferring, comparing, and explaining.
• Applying: Carrying out or using a procedure through executing, or implementing
• Analyzing: Breaking material into constituent parts, determining how the parts relate to each other and to an overall structure or purpose through differentiating, organizing, and attributing.
• Evaluating: Making judgments based on criteria and standards through checking and critiquing.
• Creating: Putting elements together to form a coherent or functional whole; reorganizing elements into a new pattern or structure through generating, planning or producing.
Assessment Options for the Affective Domain
– Difficult to assess due to the personal and internal qualities of affect.
– Inquiry/problem-solving commonly used.– Receiving: observations of learners’discussions
and questionnaires.– Responding: observation of learners’
participation and interviews.– Valuing: Interviews, questionaiires and essays.– Organisation: Observation of learners’choices.– Characterisation: Learners’
responsibilities,projects and debates.
Assessment Options for the Psychomotor
– Hierarchy of difficulty levels ranging from reflex movements to skilled movements.
– Levels generally assessed by observation of either behaviour or performance.
– Observational data can be recorded as an anecdotal record, or by using ckecklists or rating scales.
Romiszowski Taxonomy Knowledge
Facts Concepts
Skills
Reproduction Production
Cognitive (RC) Interactive (Ri)
Psychomotor (Rpm)Reactive (Rr)
Cognitive (PC) Interactive (Pi)Psychomotor
(Ppm)Reactive (Pr)
Romiszowski : Knowledge and Skills (a) Knowledge:
– Factual knowledge: recognizing and remembering of facts and action prescription
– Conceptual knowledge: understanding principles of strategies actions
(b) Skills– Reproductive skills: Routines and vocational
activities based on standard procedures executed according to a clear prescription or protocol.
– Productive skills: Appeals to the creativity and problem solving skills of the student. Application of principles and strategies in new situations.
(c ) Sub-skills for Reproductive and Productive– Cognitive skills: Application of knowledge,
eg. Interpretation, analysis, deciding.– Psychomotor skills: Execution of physical,
motoric actions using knowledge and comprehensive.
– Reactive skills: Attitudes according to a value system.
– Interactive skills: Application of social and communicative aspects in daily life in relation to others, communicating, collaborating.
Romiszowki Taxonomy:
VerbsFactual Knowledge
Conceptual knowledge
Reproductive skills
Productive skill
RecognizeRepeatListNameDefinePoint outUnderline
ExplainInterpretClassifyComparePut in orderCombineDistinguishDescribeGive examples
DemonstrateConstructApply (in concrete situations)
ModelDesignPlanAnalyzeArgueDiscuss
Romiscowski Test Matrix
Subject School Type Year/Class
Author: Period of Testing:
Test Number: Test Format Status: Draft Final
Test Goal/Sub-domain
Content (Summarized/End Term)
Knowledge
Knowledge
Skills Skills
Conclusion
Test construction requires knowledge of the syllabus/course outline on which an examination syllabus is based. There should be congruency or agreement between the learning outcome or objectiveand the testing technique used to assess a learning outcome.
The taxonomy used should be determined by the nature of learning outcome. Not all learning outcomes can be assessed by pencil and paper and within a short time. Most learning outcomes in the affective and psychomotor can be observed by observations and recording over a long time. Performance tasks are also assessed by observation.A mismatch between learning outcomes and assessment procedures leads to what is called curriculum distortion. This distortion could lead to lower learning achievement due to wrong enforcement from the assessment process.
Bibliography
American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1999). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research AssociationAnderson, L. W. and David R. Krathwohl, D. R., et al (2000) A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. Allyn & Bacon
Bloom, B.S. and Krathwohl, D. R. (1956) Taxonomy of Educational Objectives: The Classification of Educational Goals, by a committee of college and university Examiners. Handbook I: Cognitive Domain. NY, NY: Longmans, Green. Barnes, J. (1976) 'Introduction' to Aristotle The Nicomachean Ethics ('Ethics'), Harmondsworth: Penguin.
Barrow, R. (1984) Giving Teaching back to Teachers. A critical introduction to curriculum theory, Brighton: Wheatsheaf Books.MoE, 2003: The Zambia Basic Education Syllabi.Kelly, A.V. (1999) The Curriculum – Theory and Practice, Paul Chapman Publishing Limited, London.
• Cito International(2009) Theory and Practice of Test Construction, Netherlands Fellowship Programme, Arnheim
• Kapambwe, W.M. (2006) The Formative Evaluation of the Continuous Assessment Pilot Programme at Basic School Level in Zambia, Unpublished M.Ed Thesis, University College Dublin
Pickard M.J. New Bloom’s Taxonomy: An Overview for Family and Consumer Sciences. Journal of Family and Consumer Sciences Education, Vol.25, No.1, Spring/Summer 2007.Stenhouse, L. (1975) An Introduction to Curriculum Research and Development, Heinemann Educational Books, London.