In-depth case studies of students’ use of technology

to support assessment

Gráinne ConoleInstitute of Educational Technology

g.c.conole@open.ac.uk

Maarten de Laat, Jonathan Darby, Theresa DillonCAA conference, 5th July 2006

LXP project

How do learners engage with and experience e-learning?What is their perception of e-learning?What do e-learners do when they are learning with technology?What strategies do e-learners use and what is effective?

How is e-learning integrated?How does e-learning relate to and contribute to the whole learning experience?How do learners manage to fit e-learning around their traditional learning activities?

Aim To distil out subject discipline issues in using e-learning:

Collecting data on students’ experiences of using e-learning describing the students’ personal background & learning contextdrawing out learner beliefs and e-learning strategies

SampleHE Academy subject centres

Medicine, Dentistry and Veterinary Sciences Economics Information and Computer Sciences Languages and linguistics

Data collectionOnline survey:

To gain a wider understanding of learner XP around particular artefacts and select effective e-learners for case studies

Case studies of individual learners: Describing e-learning activities carried out by the learnerExploring e-learner context and background

Data• Survey

– Economics: 134– Languages: 84– Medicine: 35– Computing: 170– Other: 16– Unknown: 8– Delete: 44– Total:

491

• Audio diaries– Economics:    3– Languages:     47– Medicine:    16– Computing: 19– Total: 85

• Interviews– Economics:    2 – Languages:     2– Medicine:       4– Computing: 4– Total: 12

Survey and audio diaries

I use email t

o communicate with

everyone,

especially le

cturers; arra

nging meetin

gs,

asking questions about w

ork and queries

over assignments etc I w

rite all m

y assignments

using Word

and to sort

through th

e inform

ation I f

ind,

make notes of what I

still need to

do and spell check

my emails th

at I'm sending to

lecturers.

Search engines are used to fin

d news articles

I use them to find out information for

assignments, and also to help me clarify

my notes on each subject area that I study.

Instant messaging is used to discuss

issues with friends if a topic is not understood

Use it to manage my data and thoughts... endnote for referencing spss - stats software - easier Data entry, producing stats

The first thing i do when given any piece of word is type it into a search engine! this gives me the opportunity to see how different people interpret the title. from there i can focus on one main idea and use the electronic resources to support my initial findings or indeed rule them out. e-mail is always vital with communicating with different mediums. teachers for support

Survey and audio diaries

The use of the in

ternet h

as replaced m

any aspects

of my studying before I u

sed the in

ternet.

I generally

start by using th

e intern

et for r

esearch

and then m

ove to books etc w

hen I have th

e backgro

und info

My PDA is useful for reading

things when I'm on the move.

I use my laptop to store data and type my course works. The MP3 player serves as a storage media used to save most of my assignments, electronic journals and articles, while I use MS word application to type most of my course works. The electronic library gives me access to books, journals and articles all of which are important for my study

Instant messenger and skype for communicating with students, powerpoint for laying out slides and revision notes. Do not use any graphical packages for my studies but do use them for extra-curricular activities

Interviews

“I think that technology is becoming more and more important for my studies” “I think is technology also provides a means for me to communicate with tutors and the other students and when we are doing this research training course in the university we have this blackboard and the tutors most of them not all of them some tutors put their notes on blackboard and we can always check back and we can download it before the course started so we can get ourselves familiar with the material and then when we come to the discussion of the course we feel it just make more sense the topic makes more sense to us”

Technology essential and integral

Use of Internet to find

information and for communication

Online material to refer to and reflect

Technology integral across work and social life

Sharing with peers

Interviews

“If I can get something online I will go for that first because it is always good you can get electronic form to keep in your computer and you can always trace back but for paper – and also you can print out if you want you can read it first and then if it is useful you can print out but if something is not so relevant you can just leave it”“[Communication] through email because I think everyone most other people check their email quite often and so that’s the yeh and also this is the sort of communication media is less interruptive for me because its not like you phone somebody and you are not sure if she or he or she is convenient or not at that moment but you email and he can always pick up when he got time”

Subtle and sophisticated usesInternet as first port of call, difficulty of using paper-based journals

Care in selecting appropriate resources; using time online effectively

Interviews

“And you like through this media [Video Conferencing] you bring people together and because sometimes seeing people you can really boost kind of motivation thing because you know who you are talking to and you get more human interaction with it”“Yes sometimes I contact with the tutor [via mobile phone] cos like some tutors busy you have no other way to find them so mobile phone is the only way”“I remember this one day just something wrong with my computer with my internet connection the whole day because I couldn’t get internet so I think finally I sat down and I got a lot of work done so I think that might be one thing one of the caution you have to be careful”

Opportunities to interact globally:with peers and experts

Technology integral to learning

Interviews

“Once I came here and I am doing my master and phd I started using the technology very very much and it was very helpful and easy and for collecting data and for receiving information”“I have attended more than four online conferences during the last three years and the last one it was an e-learning conference seminar … it was very helpful and the research of their website was the powerpoints which would be downloaded

Increased use of technology in recent years

Easy to use and find information

Value of online resources

Communicating with friends, peers and tutors

Information Communication

Perceptions Environment

Uses

Perceptions/expectationsAll comfortable with using technologies

See it as integral to their learning

Sophisticated and varied use of different

communication tools for different purposes

Access to up to date and relevant

information and resources vital

Don’t see technology as anything special

– another tool to support their learning

Mismatch between institutions perceptions

of student use of technology and actual use

Perceptions/expectionsUse of technology at school

Computer as central learning tool

Critically aware of the pros and cons of e-learningUse it where its appropriate – ‘vote with their feet’Still value face to face interaction with tutorUse pen and paper where appropriateCombination of print and screen based material

Aware of varied and relevant

resources for their subject area

Sophisticated and focused users

InformationContentChanging nature of content (more available, lower intrinsic value, higher interactivity and presentational standard)

Cost and valueMaterials and information freely available on the InternetPerceived worth and value of material

PresentationUse of multimedia to improve presentation expectations of good quality content & material

EvaluationNew skills needed in terms of assessing worth of content over the internet and ensuring work is that of the student

Communication

CommunicationUsing tools in a variety of ways to communicate – with friends, family, peers and tutors Use of the internet to access expert knowledge (indirect form of communication)Expectation of being able to communicate with anyone about anything when they want to

CollaborationNew forms of collaboration possible both with peers and via new ‘smart’ and adaptive technologies – distributed cognition and shared enterprise with tools

EnvironmentMediaChanging media – increase of USB pens, ipods, mps players, integrated phones, better screen displays for reading

MotivationGaming generation, used to highly, engaging and entertaining environment

InteractivityEvidence of a shift from passive to interactive interactions across all aspects of their learning

Near ubiquitousMany now have their own PCs and wireless internet access – becoming accustomed to being able to access information or contact people on demand, anywhere

UsesIntegratedUse of tools in a combination of ways to suit individual needs Evidence of mixing and matching, comfortable with switching between media, sites, tools, content, etc. Provides more flexibility in terms of undertaking learning anytime, anywhere

PervasiveExtensive use of technologies to produce find, manage and produce content Used for all aspects of their study

Niche adaptive, utilitarian useIncreased evidence of use of free self-assessment quizzes to test knowledge, utlising their community of peers – to share resources, ask for help, peer assess

PersonalisedAdapting to suit their own needsUse of computer, internet and books simultaneouslyAnnotation and adaptation of materials

UsesManagementSophisticated at finding and managing information (searching and structuring)Use of PC as central learning toolUsed to having easy access to information (for travel, entertainment etc) and therefore have an expectation of the same for their courses

TransferabilityTransfer of practices to the learning context: MSN chat, Amazon, ebay, Skype

TimeConcept of ‘time’ changing – both in terms of expectation of information and results on demandFragmentation of the learning timetable

Changing work patternsNew working practices using an integrated range of toolsUse of tools is changing the way they gather, use and create knowledgeValue and perceived intrinsic worth of knowledge is differentSkills – shift from lower to higher levels of Blooms taxonomy

Issues for assessmentWhat to assesDevaluing of content and increased interactivity, raises the question of the value of assesses knowledge rather than critical awareness, as well as raising issues about how we can assess process rather than product

How to assessChanging practices – multi-tasking and integrated use of tools, resources and community for a variety of purpose

Who is being assessedAuthentication – of material submitted (plagiarism) and exams undertaken

OutcomesWhat constitutes a valued educational outcome? What is its value and purpose?