Research into the effectiveness of learning through ICT for people with basic skills needs

Dr Harvey Mellar, Dr Maria Kambouri,

Professor Alison Wolf, Tina Goodwin, Violet Windsor

Pavlos Koulouris, Annette Hayton

Reports

Ufi, the DfES and the Basic Skills Agency have been involved in a research project looking at the effectiveness of ICT as a learning tool for people with basic skills needs. The project is now complete and has resulted in the production of a guide for learning centres (Using ICT to develop literacy and numeracy: A guide for learning centres working with adult learners) as well as a research report.•http://www.ufi.com/partners/hubandlearning/Ictnumeracy.htm

Overview• Introduction

1. How is ICT being used?

2. Is the use of ICT in learning attractive?

3. How effective is ICT in promoting learning?

4. What are the factors that engender successful learning with ICT?

5. How can tutors be trained to use ICT effectively?• Conclusions

Introduction

• Moser Report:

"At the heart of improved quality in delivery and materials must be increased use of Information and Communication Technologies (ICT) to improve basic skills"

• Brooks, G. et al (2001) Progress in Adult Literacy: Do learners learn? London: Basic Skills Agency.

Methodology

• 11 centres selected• Interviews:

– 71 managers and tutors – 124 learners.

• Pre and post testing– numeracy and literacy– two and a half months between pre and post– 70 learners completed the post tests

How is ICT being used?

• Changing nature of literacy and numeracy

• Accessing and using ICT

• CD-ROMS and learning packages

• Word-processing and office software

• Email

• WWW

• Producing multimedia presentations

Incorporating ICT into teaching

• Using materials

• Student autonomy and the role of the tutor

Is the use of ICT in learning attractive?

• Increased number of learners

Characteristics of learners

• 68% male, 32% female– NFER study - 45% male, 55% female– learndirect learners on basic skills courses -

40% male 60% female

• 46% of post-interview sample have access to IT at home (36% of pre-interview sample)

Characteristics of learners - level

0

10

20

30

40

50

60

70

80

90

Numeracy Listening Reading Writing

%

E1E2E3L1L2

Reading levels

0

10

20

30

40

50

60

NFER IoE

%

E1E2E3L1L2/ 3

Characteristics of learners - age

Mean age 32

0

10

20

30

40

50

60

70

under 30 30-39 40-49 50 plus

%

I oE study

NFER study

learndirect

Is the use of ICT motivating?

• 92% of learners thought so

• Motivating aspects of ICT use– self esteem– keeping up with the changing times– different from previous learning experiences– rewards and accreditation– gamesaspects

• Some demotivating aspects

Using ICT

• Starting to use ICT

• Changing experience of ICT use, the web is:– a source of information for older learners– a source of entertainment for younger learners

How effective is ICT in promoting learning?

• Learning– interviews with learners– tests

• Retention

• Monitoring and communication

• Computer based assessment

Learner interviews

• 64% thought that ICT had had a significant impact on their learning

• 45% expressed strong reliance on the tutoring arrangements

• 20% expressed their willingness or desire for more independent activity and saw the computer as an aid to this

Learner interviews

0

10

20

30

40

50

60

70

EAL Lit Num Other Conf Work

% GoalGains

Comments

• Meeting their expectations as regards literacy, but not numeracy

• Making gains in ICT skills even when these had not been one of their initial learning goals

• Expectations about preparation for specific areas of work were often not being met

ICT or Literacy?

• Occasionally there was some evidence of tension between the use of ICT and the learning of literacy and numeracy

Changing aspirations

• 26% indicated that their work aspirations had altered so as to include a desire for a greater involvement with ICT in their work

A story

John is 35, and has a 7 year old daughter, he used to work in the motor industry but is now unemployed. He has used his girl friend’s computer to produce some fliers for an event he was organising and to play some games. He is aware of his difficulties with literacy and numeracy. He had tried going to a learning centre in a college before but he drifted away quite quickly. He decided to give it another try. The college put him on to a computer based assessment program. – he had not expected to use a computer and he quite liked the fact that he was using it. He felt he was learning how to use computers as well as doing the work he needed to do.

Over the next couple of months he worked through a number of computer based learning units. During this time he is going home and talking with his daughter and girl-friend about what he is doing with computers and comparing it with their experiences of using computers in school and work. Then he moves on to using the World Wide Web and email in projects - he enjoys this because it further extents his use of computers. He is starting to feel like an expert and is helping other learners in the centre with their use of ICT. He starts to talk about taking a course about the internet, he moves on.

Testing

• Average time between pre and post-test was two and a half months

• Sample sizes were small and results are indicative only

Numeracy

• No significant difference in average score

• Two further specific hypotheses were tested:

that ICT use might reinforce place value/arithmetical operations in particular

that doing literacy as well as numeracy might improve understanding and interpretation of graphs and figures.

No significant differences were found

Literacy: Entry level 1/2

• Very high standard deviations for change scores, large and erratic changes

• Reading - no significant differences found

• Writing - a small but not statistically significant improvement in grammar

Reading: Entry level 2/3

Pre Post

Mean 17.48 19.46

Standard Deviation

5.41 5.02

Change: mean 1.97, SD 4.05

Significant p<0.05

Equivalent to an improvement of about 7 percentage points.

Writing: Entry level 2/3

• Small increase in word totals

• Some improvement in spelling and grammar

• Noticeable increase in complexity of writing, as measured by number of clauses

Effect size

• EFFECT.XLS

Literacy: Level 1/2

• Reading - no significant changes

• Free writing - slight improvement in grammatical accuracy

• Rewriting - definite improvement in grammar, spelling and punctuation (but not statistically significant)

EAL

• Reading - no significant change

• Writing– a definite increase in number of words used

for middle band (Entry 2/3)

– no consistent pattern of change in word length, complexity or spelling

– some improvement in grammar for the lower band (Entry 1/2)

– slightly more spelling mistakes

Retention

• The retention rate between pre and post literacy tests was 64%

• NFER study had a retention rate of 57% for the reading and 54% for the writing test

Monitoring and communication

• Little effective use of ICT for monitoring, and no integration with Educational Management Information Systems

• Positive response by both learners and tutors to the communication possibilities opened up by ICT

Computer based assessment

• Learners - positive (though could be time consuming)

• Tutors - divided opinions

What are the factors that engender successful

learning with ICT?

• Pedagogic factors– assessment

– learning styles

– integration in teaching

– learners’ views on ICT and learning

– tutors’ views on ICT and learning

• Organisational factors

Learners’ views on ICT and learning

• ICT helps you to learn

• ICT helps you to concentrate

• ICT gives you feedback

• ICT enables you to do your work more quickly

• ICT enables you to produce better presented work

Tutors’ views on ICT and learning - 1

• ICT provides a wide range of multimedia resources

• ICT enables the tutor to individualise materials

• ICT enables concentration

• ICT provides practice

Tutors’ views on ICT and learning - 2

• ICT removes the need for handwriting

• ICT supports literacy skills

• ICT supports speaking and listening

• ICT supports collaborative work

• ICT encourages independence

Organisational factors

• Tutors

• Accessibility

• Physical organisation

• College culture

• Technology management

• Planning

Case studies of centres

• College: vocational focus

• College: communication workshop:

• Community based class

• Outreach project

• Prison

College: vocational focus

• Strong commitment by management, and strategic college level planning

• Specific vocational focus. Intensive courses

• Full time, well trained tutors, working in teams, weekly meetings

• Ready access to high quality computer suites• Basic skills suite integrated in college facilities• Tutors have strong relationship to technology staff.

Involvement of staff in planning. Technical support understands needs of basic skills learners.

How can tutors be trained to use ICT effectively?

• Do generic trainers need special skills to provide support to people with basic skills needs?

• Do basic skills tutors have the necessary skills to make effective use of ICT in their work?

• ICT training for tutors of basic skills

Conclusions - 1

• ICT has a crucial role in enticing people into learning

• There is some evidence for the impact of ICT use on literacy (but not numeracy)

• For most learners the role of the tutor is very important, but there is a sizeable group for whom greater autonomy is promised by the use of ICT

• ICT use in basic skills provision has not yet reached the critical threshold in terms of access, development of pedagogy, or tutor training for really effective implementation

Conclusions - 2

• There is a need for tutor training for basic skills tutors:– basic ICT skills– specific pedagogic approaches to integrating

ICT into basic skills provision– the impact that ICT might have on the nature

of learning

Further research

• The impact of learndirect products

• Intervention studies • Observational studies• Development of a principled approach to

instructional design. • Reluctance of some learers to use ICT in

their learning• Reluctance of some tutors to use ICT