CETL-impact : case studies from S104 Exploring Science

Sally Jordan, Diane Butler and Paul Hatherly

3rd Open CETL Conference, 24th September 2008

Enabling student planning and preparedness: use of diagnostic quizzes and the Maths Skills ebook

Sally Jordan3rd Open CETL Conference

The problem…• Making sure that students are on the ‘right’ course and

sufficiently prepared for study is of crucial importance for retention.

• S104 is a 60 point course: we want to make sure that students have sufficient time available for studying it.

• S104 assumes some basic mathematical skills, taught in S154 Science Starts Here and elsewhere. Where appropriate, we want students to study S154 before S104.

• Other students may be generally well prepared, but rusty in a particular skill assumed by S104.

• So we use an online diagnostic quiz ‘Are you ready for level 1 science?’ and the Maths Skills ebook.

Are you ready for Level 1 Science?• A successor to the printed ‘Are you ready for…’

diagnostic quizzes;• Three linked quizzes: online, interactive;• Designed to help students assess their preparedness for

S104, S154 and other level 1 science courses;• The S104 quiz makes extensive use of iCMA questions

written for S154 (with different data-sets).

AYRF

Are you ready for Level 1 Science? : Evaluation• 50,000 people have accessed the quizzes since April

2007; 14,000 have accessed ‘Are you ready for S104?’• Students have used the quizzes to

Confirm choice of course;

Decide between two or more courses;

Alter choice of course;

Identify small areas of weakness.• Of S104 students who used the quizzes, 90.3% found

them useful or very useful.• But 50.4% of S104 students surveyed didn’t use the

quizzes before starting S104.

The Maths Skills ebook

• Developed as a piCETL resource, initially for Science Short Course students.

• Available on the S104 course website and the new Studying Science website, to enable students to access it before the S104 site opens.

• Content can be revised and extended easily; now includes all the maths taught in S154.

• Accompanied by its own set of iCMA questions.

Maths Skills ebook

Maths Skills ebook and iCMA questions : Evaluation• Of students replying to an S104 survey, 42.5% had used

the Maths Skills ebook and 97.3% of these had found it useful.

• The Maths Skills questions were attempted by 1490 people between 28th August 2007 and 24th August 2008.

• Around half of the 118 students who responded to a feedback question had not looked at the ebook, only the interactive questions, to ‘tell me what I need to revise’.

Acknowledgements• Ruth Williams (S154 Course Team, AYRF)• Linda Fowler (S154 Course Team, AYRF)• Greg Black (LTS, AYRF and Maths Skills questions)• Richard Jordan (Consultant, Maths Skills ebook and

questions)

s.e.jordan@open.ac.uk

Enabling practical work for all students: using an interactive screen experiment

Paul Hatherly and Sally Jordan3rd OpenCETL Conference

The Open UniversityPractical work is taught by:• Laboratories at Residential School• Home Experiments

But there are issues when students…• can’t do the experiment for themselves [because of

disability];• can’t attend residential school;• need extra practice.

• The role of interactive screen experiments (ISEs):– Widen accessibility of experimental work;– Extra practice; – Improve confidence.

Before we can understand what an ISE is, we have to understand what it is NOT

It is NOT a programmed simulation (eg, Hooke’s Law Flash simulation from PhET, U of Colorado)

Required physics is explicitly coded Much may be left out!

What an ISE is not

An ISE is a generalised very interactive movie that is delivered remotely via an ordinary web browser without special software

The best way to understand what this means is through an example…

All physics is automatically present

What an ISE is

S104 Book 3 Energy and lightActivity 11.1 Investigating light

Students:• Compare and analyse the spectra produced by an energy-

saving and a tungsten filament light bulb

Feedback on the experimentFrom tutors:• [students] are normally quite blown away that you can

look inside the atom with just day to day equipment…• I am really glad that it is compulsory in S104.

From students:• I really enjoyed this experiment. I was surprised at how

interesting and successful it was. I even got my parents to observe the difference in the spectrums of the bulbs, and was able to explain to them how the energy-efficient bulbs work. It was good because I was able to share my new knowledge.

• My eyesight is not as good is it used to be.

So we made an ISE…

The S104 ISE

Evaluation of the S104 ISEWe wanted to know:

1. Can students who have used the ISE instead of the ‘real thing’ complete the activity in their course book? Can they complete related assessment tasks?

2. How easy is the ISE to use?

Students have been observed in the IET Research Lab and students who use the ISE instead of the real experiment (and their tutors) will be approached for feedback.

Early conclusions

• this ISE is easy to use;• the ‘real thing’ gives a wider experience of practical work

than the ISE, but the ISE is invaluable for those who are unable to attempt the real experiment.

Note: the ISE is doing something that a simulation couldn’t

Acknowledgements• John Macdonald (University of Reading)• Wez Charlton (University of Reading)

Paul Hatherly

p.a.hatherly@open.ac.uk

www.open.ac.uk/picetl/

Supporting student learning through Tutor Group Forums

Diane Butler3rd OpenCETL Conference

Tuition and SupportCustom and practice in the science faculty to use FirstClass forums for support of large groups of students (course basis)

For S104 we wanted to • reposition the focus of students onto TGF –tutor group

forums• make TGFs vehicles for group tuition as well as group

support

Tuition and AssessmentIn order to improve students’ perceptions of the value of online collaboration and deliver an appropriate tuition strategy for a globally delivered course we

• embedded some online activities into assessments• allowing us to assess Learning Outcomes relating to

group working and information collation

Tutor Support To prepare our ALs we produced the following resources

and stored them online on the course web site (Tutor view)

• A tutor guide to each book of the course outlining both optional and assessed activities which were suitable for online tuition and how these were to be delivered

• Detailed teaching resources for student use but guided by a tutor

The assessed activitiesThree assessed activities in S104, all relying on student-student collaboration

• Reporting results of a home experiment, sharing and comparing data

• Accessing specified internet resources for information on primate species –leading to a discussion on which species should be targeted for conservation action (threats, phylogentic importance, geographic range)

• Accessing specified internet resources to investigate the topic of water on Mars

Optional Activities• Prepared by AL consultants in conjunction with the

course team• Wealth of material to use according to identified student

need by the tutor – online or at f2f events• Material requires facilitation by the tutor –it is teaching

material not student resource• Aims to sustain activity and tutor-student-student contact

in the TGF

Supporting student learning through assessment: TMAs and iCMAs

Valda Stevens and Sally Jordan 3rd OpenCETL Conference

Centre for Open Learning of Mathematics, Science Computing and Technology (COLMSCT)

The Formative Assessment in Science Teaching (FAST) Project

Gibbs and Simpson identified 11 conditions under which assessment supports student learning.(Gibbs, G and Simpson, C (2004) Conditions under which assessment supports students’ learning. Learning and teaching in higher education, Issue 1, pp3-31)

3 of the conditions relate to pacing and 4 of the conditions relate to the provision of timely feedback.

Feedback matters!

• Within the Open University context, learners are geographically separated and we cannot assume that they will meet their tutor in order to receive feedback. Feedback on assignments therefore has a particularly important teaching and learning function.

• We are seeking to provide students with feedback which is personalised, can be understood by the student and is received in time to be used in future learning and assessment tasks.

• So feedback becomes feedforward.

S104 Assessment strategy• Seven tutor marked assignments (TMAs)• Eight interactive computer marked assignments (iCMAs)

(summative but low stakes)• Synoptic component:

Written end of course assignment (ECA)

Longer iCMA (25 questions) with questions ranging across the course

• We want to help students keep up to date in their studies;

• Frequent small assessment tasks to help students monitor their own progress.

The OpenMark system• Uses a range of question types, going far beyond what

is possible with multiple choice;• Question types include:• Numerical input; text input; drag and drop; hotspot• Students are allowed three attempts with an increasing

amount of teaching guidance, wherever possible tailored to the student’s previous incorrect answer;

• Different students receive variants of each question so each has a unique assignment;

• We have included a few questions requiring free text answers of a phrase or sentence in length.

Evaluation• We didn’t get the timing of the iCMAs quite right;• The free text short answer questions are proving to be

remarkably robust;• There have been some surprises (in student responses

to all questions). It is important to monitor student responses;

• Students respond to iCMA questions in summative use in a very different way to when they are used purely formatively.

Acknowledgments• The S104 Course Team and our LTS media developers;• The Centre for Open Learning of Mathematics, Science,

Computing and Technology (COLMSCT) especially Barbara Brockbank, Phil Butcher and Laura Hills

• Tom Mitchell of Intelligent Assessment Technologies Ltd.

v.a.m.stevens@open.ac.uk

s.e.jordan@open.ac.uk

http://www.open.ac.uk/colmsct/projects/sallyjordan