Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting...
-
Upload
alyson-mckinney -
Category
Documents
-
view
213 -
download
0
Transcript of Analysing ANA results Nicky Roberts Wits School of Education Kelello Consulting...
Analysing ANA results
Nicky RobertsWits School of EducationKelello [email protected]
Context
Focus on Primary Maths project Errors in marking, adding and recording
results (2012) Using ANAs as a diagnostic tool for teaching
and development (2013)
Research in two schools in Cape Town• Focus on Primary Maths 2012-2014
– School based maths teams– Professional development for FP teachers in
maths• Mini seminars• Co teaching interventions• Mathematical thinking course
– Responding to ANA attainment– Sharing lessons
Responding to ANA attainment
Tracking ANA results year on year Question-by-question capture and analysis Planning for fourth term based on ANA
analysis Rasch analysis comparing the two schools
Background to the two schools• 2 schools are equal partners in development project, chose
to be named at school level• Capricorn Primary School (CPS)
– 700 learners, 2011 below provincial mean (ANA and systemics)– Township, backyard dwellers, poverty, gangsterism and taxi
violence, drugs (Tic), alcohol (FAS). Mostly households in poverty.
– Minority not English main lang, LOLT is English• The Grove Primary school
– 700 learners, history of excellent attainment (ANA and systemics)
– Suburban, model C, mostly affluent households– Majority English main lang, LOLT is English
No anchor questions Predictable test Not year-on-year
comparison Assume system is
stable, then performance relative to national mean is indicator of overall performance
Indicator of proportion of learners meeting grade level requirements (relative to national pass rate, on that ANA)
WCED systemic (standardized questions and marking, so year-on-year comparison)
Question by question data capture
Real power for instructional design
Q1 Q2 Q3 Q4...
Learner 1Learner 2Learner 3Learner 4Learner 5….
CorrectIncorrectFacility
ANA analysis template
Class capture sheet Summary sheet Question summary sheet Spreadsheet template suburban school Spreadsheet template township school
What we did in 2013
Before ANA marking Import the learner names from EMIS into classes. One class
per sheet. Check formulas and that the template works as you want it
to
On the marking day Laptops/ computers with template in marking venue Capture marks for each script Queries – go back to teacher who was marking
What we did in 2013
After marking daysCreate grade summariesCompare to previous yearsLook for questions of common difficulty
Very poor: colour code in green Poor: colour code code in pink Revise maths teaching and development plan for
fourth term
Functions used in Excel template
SUM AVERAGE MIN MAX STDEV
Some notes on using Excel
IF-THEN COUNT IF
What we did in 2014
Repeated the descriptive statisticsUsed this to create grade level ‘mini lesson studies’
cycles, eg: Teaching bonds in Grade 1 Teaching views and positional language in Grade 2 Teaching analogue time in Grade 3
Plan – teach- reflect cycleTeacher presentations at mini conference, AMESA
regional and AMESA national
Mini conference (Day 1 internal)
Mini conference (Day 2 district)
Mini conference
AMESA regional
What we did in 2015
Repeated the descriptive statistics Used the data to conduct Rasch analysis:
Misfitting itemsBiases between schools (based on DIF values)
Used this to target problematic questions and review these qualitatively
Qualitative analysis (from initial qunat data)
Total marks 1Mean 0.02Standard deviation 0.14Mean percentage 2%Number writing 80Number getting 0 79Percentage getting 0 99%
Identified as a major area of weakness
Errors on Q 25.1
Some completely off
Most common error was to break up number rather than converting
Conclusion
ANA is a major investment in SA education It gives annual standardised data across schools If anchor questions were included, it would also give year-on
year comparisons It gives schools rich data for error analysis (if item level analysis
is conducted) which can lead to lesson study cycles/ interventions
Teachers need support in marking ANAs (again directs attention to the mathematics) and reporting common errors to district/ province