OECD Programme for International Student Assessment
(PISA) Mathematics
Margaret Wu
ACER & University of Melbourne
OECD PISA Project - led by ACER, with ETS, Westat, NIER, Cito
• 15 year-old students– end of compulsory education– not intact class sample
• About 40 countries
• 3-year cycle:– 2000 Reading– 2003 Mathematics– 2006 Science
• Student and School questionnaires
Mathematics Framework - 1
• Experts driven, not curricula driven– TIMSS based on common curricula– PISA based on “definition of Mathematics”
deefined by a group of “expert” mathematics educators.
• Expert Group Members– Jan de Lange, Werner Blum, Mary Lindquist,
Vladmir Burjan, Sean Close, John Dossey, Zbigniew Marciniak, Mogens Niss, Kyungmee Park, Luis Rico, Yoshinori Shimizu
Mathematics Framework - 2
• Definition of PISA Mathematics Literacy:– Mathematics literacy is an individual’s capacity
to identify and understand the role that mathematics plays in the world, to make well-founded judgements and to engage in mathematics, in ways that meet the needs of that individual’s life as a constructive, concerned, and reflective citizen.
Mathematics Framework - 3• Organisation of content
– TIMSS by topics and subtopics of mathematics • Number, Algebra, Measurement, Geometry, Data
– PISA by overarching ideas (phenomenological approach)
• Quantity, Space and Shape, Change and Relationships, Uncertainty
• Conception of assessment items is different; making a distinction between teaching and assessment
Mathematics Framework - 4
• Processes– Emphasis on Mathematisation - the processes
involved from encountering a real-world problem to generating a solution
– Three competency classes• Reproduction - practised routine procedures
• Connection - making judgements, reasoning
• Reflection - making generalisations
Item Context Emphasis on authenticity - 1
• No “naked” drills items, e.g.,– Solve a linear or quadratic equation– Construct an angle– Simplify a fraction
• Few intra-mathematics items, e.g.,– pattern observation in sum of odd numbers– properties of numbers, e.g., perfect numbers.
Item Context Emphasis on authenticity - 2
• Problem context is not just for the sake of adding context, but for real-world application.
How far is the foot of a 2 m ladder from the wall when the top of the ladder is 1.92 m above the ground?
X
Not a good PISA item!
Another example of better context
– Farmer Dave keeps chickens and rabbits. Dave counted altogether 65 heads and 180 feet. How many chickens does Dave have?
Tickets to the school concert costs $4 for an adult and $2 for a child. 65 tickets were sold for a total of $180. How many children’s tickets were sold?
Item Context Emphasis on authenticity - 3
• Problems with contexts that influence the solution and its interpretation are preferred for assessing mathematical literacy.
You must be over 21 to drink alcoholic drink. Which people should you check?drinking beer 22 yrs old drinking coke 16 yrs old
If a card has a vowel on one side, it must have an even number of the other side. Which cards should you turn over to check?
A 6 J 7 (Wason. Griggs&Cox)
Item Format
• MC and Open-ended; about half of each kind.
• Raw responses captured as much as possible
• Double-digit coding to keep track of different approaches
Example Mathematics Item - 1
The picture shows a spinner used in playing games. For a game, the spinner is used to choose a person at random to start the game. Explain how you will use this spinner to choose a person at random if there are (1) three players, and (2) nine players.
5
4
3
2
1
6
The picture shows a spinner used in playing games. (1) What is the probability of spinning a “3”? (2) What is the probability of spinning an even number?
Fits PISA framework
better
Example Mathematics Item - 2Break-in
On the radio, an advertisement for an insurance company ran as follows: “Every 10 minutes, a car is stolen in Zedland. Every 21 minutes, a house is broken into. Take up an insurance policy today.”
Using only the information given in the advertisement, can you conclude
(1) anything about the chance a car will be stolen in Zedland?
(2) that it is more likely to have a car theft than a house break-in?
Give reasons to support your answer.
Example Mathematics Item - 3
LaneReaction time (secs)
Final time (secs)
1 0.147 10.092 0.136 9.993 0.197 9.874 0.18 Did not finish race5 0.21 10.176 0.216 10.047 0.174 10.088 0.193 10.13
To date, no humans can react in less than 0.110 of a second.
Challenges for Test Developers
• Real-world mathematics are not easy to find (for 15 year-olds)
• Predominantly about price, cost, discounts in everyday life (too domestic?).
• PISA is somewhat “middle-class” - telephone, internet, cars, computers
Hong Kong Performed Well
370
420
470
520
570
620
Hon
g K
ong
Japa
n
Kor
ea
New
Zea
land
Fin
land
Aus
tral
ia
Can
ada
Sw
itzer
land
Uni
ted
Kin
gdom
Bel
gium
Fra
nce
Aus
tria
Den
mar
k
Icel
and
Liec
hten
stei
n
Sw
eden
Irel
and
Nor
way
Cze
ch R
epub
lic
Uni
ted
Sta
tes
Ger
man
y
Hun
gary
Rus
sia
Spa
in
Pol
and
Latv
ia
Italy
Por
tuga
l
Gre
ece
Luxe
mbo
urg
Mex
ico
Bra
zil
Ma
the
ma
tics
Me
an
Sco
re
Compare TIMSS and
PISA
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
sta
nd
ard
ised
sco
res
jpn
jpn
kor
kor
nzl fin
fin
can
can
aus
aus
uk
uk
cze
cze
usausahun
hun
rus
rus
lav
lav
ita
PISA TIMSS
ita
nzl
hkg
hkg
Compare TIMSS and PISA
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
Standardised TIMSS score
sta
nd
ard
ised
PIS
A s
core
korjpn
ita
can
hun
rus
aus
cze
finuk
nzl
usa
lav
hkg
Apples M136Q02
• Number of apple trees = n2
• Number of conifer trees = 8n– where n is the number of rows of apple trees.
• There is a value of n for which the number of apple trees equals the number of conifer trees. Find the value of n and show your method of calculating this.
Speed of Racing Car M159Q03Speed(km/h)
180
160140
120
100
80
60
40
20
00 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0
0.5 1.5 2.5
Starting line Distance along the track (km)
Speed of a racing car along a 3 km track(second lap)
What can you say about the speed of the car between the 2.6 km and 2.8 km marks?
A. The speed of the car remains constant.
B. The speed of the car is increasing.
C. The speed of the car is decreasing.
D. The speed of the car cannot be determined from the graph.
Top Related