Dr Sarah Richardson Senior Research Fellow, Higher Education Australian Council for Educational...
-
Upload
adan-herdman -
Category
Documents
-
view
215 -
download
2
Transcript of Dr Sarah Richardson Senior Research Fellow, Higher Education Australian Council for Educational...
Insights from the AHELO Feasibility Study – why, what and how
Dr Sarah RichardsonSenior Research Fellow, Higher Education
Australian Council for Educational [email protected]
Eternal challengesHow do we measure quality in teaching and
learning?How effective are institutions at educating
students?What knowledge and skills do students
possess at the end of their degrees?How do students’ learning outcomes compare
to international benchmarks?
Final-year Bachelor degree students
To determine a robust approach to measuring learning outcomes in ways that are valid across cultures and languages, and across the diversity of institutional settings and missions
To give institutions data on student learning outcomes which they can use to improve quality in teaching and learning
How do the learning outcomes of our students measure up against international benchmarks?
Benchmarking against like institutionsStudents – demographics, time usage,
performance, motivation, plansTeaching – curriculum structure, class sizes,
staff contact, feedback, industry experience Institutional characteristics – size, location,
funding, mission, student characteristics
To give institutions data on student learning outcomes which they can use to improve quality in teaching and learning
AHELO: international participation – 17 countries (including Egypt, Kuwait and Abu Dhabi) and 12 languages (including
Arabic)
Is it feasible to develop frameworks and instruments to test discipline-specific learning outcomes?
Test fields: Economics, Engineering
23000 students
5000 faculty
Generic Skills Economics Civil Engineering
250 institutionsOnline
International agreementExpert Groups – Engineering and EconomicsDomain parameters / conceptual structureMeasurement priorities – real worldReview and map draft assessment materials Scoring guide – desired responsesStructure reporting and review resultsGeneric Skills – different approach used
Economics learning outcomessubject knowledge and understandingsubject knowledge and its application to real
world problemsability to make effective use of relevant data
and quantitative methodsability to communicate to specialists and non-
specialistsability to acquire independent learning skills
Assessment instrumentsCombine pre-existing / new materialsInternational collaboration / resourcesAdapted / created for assessment frameworkConstructed response and MCQsPilot & qualitative feedback Localisation – equivalency across languagesOnline delivery
Engineering
Engaging participantsCountries / institutions / faculty / studentsWhat’s in it for me?Marketing and awareness raisingCultural variationsSample / populationAuthority / persuasionIncentives and inducementsIntrinsic motivation
Analysis and reportingPsychometric analysis
How did the test items function?What was the spread of difficulty?Did difficulty vary according to student and
institution characteristics?Reporting – international, national and
institutional levels
Using results to enhance teaching and learningPerformance by student group
Which students need greatest support?Performance by competency
Which aspects of teaching need enhancing?Performance by country
What is missing in the curriculum?
Insights from AHELOMeasure learning outcomes internationally? International agreement on skills and knowledge? Assessment materials to suit diverse contexts? Maintain equivalence in translation? Engage institutions, faculty and students? Online delivery in different languages/alphabets? Useful benchmark reports without identification? AHELO Feasible?
Learning
Employment
Curricula
Faculty
Improve Quality
Selection Diagnostic
Progression Outcomes