University of Northampton

BLITT Evaluation Data

Teaching and Learning Research

2015-16

Program Theory for Blended Learning in ITT QTS

1

IMPACT

High quality beginning primary teachers who consistently make use of theory and practice in their teaching

Increased use of technology enhance learning by students

Tutors develop blended teaching and learning resources

Tutors are trained / enabled to use a range of technological tools (used for teaching, learning and assessment)

Tutors have improved knowledge and skills in using technological tools (used for teaching, learning and assessment)

Improved knowledge and understanding of theory and practice (in relation to becoming a primary teacher)

Increased use of technology enhanced teaching by tutors

Blended Learning Program Logic

2

EXTERNAL FACTORS.

Instability in HEI ITT QTS.

University move to Waterside.

University move towards Blended Learning.

New technologies used in teaching and learning.

ASSUMPTIONS

Connections

Provision of blending learning will positively influence knowledge, motivation and participant behaviour.

Tutors engagement improves teaching and learning.

Blended Learning leads to high quality employable beginning teachers.

Tutors use of Blended Learning can lead to recognition within HEA.

ASSUMPTIONS

Pre-conditions

Students and tutors have access to technology

Students and tutors are willing to engage and use technological tools

Effective IT infrastructure to support (NILE)

Tutors value teaching which demonstrates theory and practice

INPUTs

StaffingFundingLearning TechnologistsInfrastructureMaterialsTechnological Resources

OUTPUTS

Programme/Modules produced

Learning Events and Online Learning Package sessions planned and available on NILE

Students enrolled on Programmes/Modules

Blended Learning delivered.

SHORT TERM OUTCOMES

Students participate in Blended Learning

Positive response to resources.

Increased knowledge (for tutors and students) in relation to learning outcomes for modules.

Improved confidence and competence of tutors to use Blended Learning.

Students’ increased skills in using in Blended Learning.

Students report changes in confidence and competence in linking theory and practice.

MEDIUM TERM OUTCOMES

Tutors change T&L strategies to regularly include technology in teaching.

Students’ increased engagement with online learning communities.

Increased links made between theory and practice by students.

Increased links made between theory and practice by tutors.

IMPACT

High quality beginning teachers who consistently make use of theory and practice in their teaching.

TRAINING PROVISION

ITT QTS Staff meetingsCAIROs School ForumCurriculum GroupsLearning TechnologistsPeer to Peer Experts within ITT QTS team – inc apps team

ACTIVITIES

Existing Programmes/Modules reviewedNew T&L strategies developed (Learning Events and Online Learning Packages LE and OLP)New Programme and Module developedStaff meetings organisedTraining designed and deliveredNew T&L resources developedBlended Learning promoted to students

BLITT Project 2015-16- Appropriateness Focus

To what extent is Blended Learning considered suitable in meeting the needs of ITT staff and ITT students.

(BA Primary Education year 1, BA Early Years year 1, PGCE Primary 5-11 and PGCE Primary 3-7) all with QTS.

The responses to questionnaires given to staff and students in December 2015 have been compared in terms of the total % for strongly agree and agree.

Staff BA Primary 5-11 Year One

BA Early Years 3-7 Year One

PGCE Primary 5-11

PGCE Primary 3-7

Cohort Size 15 90 12 41 6Question QuestionI understand what blended learning involves

93 I understand what blended learning involves

83 100 87 100

I understand why blended learning has been introduced

80

The need to introduce blended learning in the University was clearly articulated

67 I understand why the University uses blended learning

76 92 79 83

The need to introduce blended learning in ITT QTS was clearly articulated

87 I understand why teacher training includes blended learning

74 75 88 83

It is not clear which ITT QTS modules will have blended learning in them

80 I am clear which modules include blended learning

69 92 51 50

The theory diagram underpinning blended learning captures the intent of blended learning

87 The diagram shows clearly the intention of blended learning

N/A N/A 69 N/A

The blended learning *LE and OLP, are suited to ITT QTS provision

73 Blended learning* LE and OLP, are an appropriate approach to learning to teach

86 83 72 67

3

The blended learning *LE and OLP, meets University policies on learning and teaching

73 Blended learning *LE and OLP, meets University policies

63 92 69 67

Blended learning meets the needs of the University

93

Blended learning meets the ITT QTS criteria

87

Blended learning meets the needs of teacher educators

53 Blended learning meets the needs of the tutors for teaching trainee teachers

74 83 77 100

Blended learning meets the needs of ITT QTS students

60 Blended learning meets the needs of trainee teachers

77 91 71 83

Blended learning meets the needs of partnership schools

60

Staff Comments (December 2015)

The evidence for and justification of blending learning has not been made convincingly for the University nor ITT QTS. There are some good elements for the LE and 0LP idea but alas some significant drawbacks.

8, 12 & 13 - poor research questions! - Cannot be certain of impact in school life. Overall agree with 8, 12 & 13 but not instances - pure online procedures may/may not be appropriate - I am unsure whereas blended pre-F2F/F2/F/post F2F certainly works well. I strongly believe online procedures will work with TAR but am not convinced with respect to some courses where replaced F2F.

The fact that packages can be restricted can be very beneficial to students. 9 - Unsure of University policies so found this difficult to answer. 10 - Unsure of this question. Would

need to discuss. 14 - Unsure of this question. Would need to consult PSG. 11/12 - Arrangements to secure engagement / assessment not yet in place to ensure impact. 13 - In

theory! There is a lack of openness on the part of the University about the drive to blended learning. It is

financial not pedagogical in its approach. This makes some of the questions above difficult to answer. Over time as staff and students become more familiar with use then impact may become more

apparent. The use of blended learning should not be used as a reason for less face-to-face time with students. - ITT students in particular need this collaborative interaction.

4

BLITT Project 2015-16- Effectiveness Focus

How well has blended learning been implemented to achieve its intended aims.

(BA Primary Education year 1, BA Early Years year 1, PGCE Primary 5-11 and PGCE Primary 3-7) all with QTS.

The responses to questionnaires given to staff in December 2015 have been compared to trainee responses to questionnaires given in December 2015 in terms of the total % responses to

Intended Aims – see Program Theory

Tutors are trained/enabled to use a range of technological tools Tutors have improved knowledge and skills in using technological tools Tutors develop blended teaching and learning resources Increased use of technology enhanced teaching by tutors and increased use of technology enhanced

learning by students Improved knowledge and understanding of theory and practice (in relation to becoming a primary

school teacher)

Staff perceptions on training / being enabled to know and use technology (Dec 2015)

Technology staff supportive but access can be an issue re room in Sulgrave. There have been equipment issues due to multiple department use of room in past. I suggest more support from Technology staff will be required after this at Waterside

Discrepancies and inefficient IT support have seriously impacted upon use of technology, iPads in particular. IT support sporadic. Some IT not Apple TV ready.

IT support have made it very difficult to engage further with IT in sessions. Need Apple TV in all teaching rooms. In early stages so creating e.g. 0LP time consuming.

Time has been allocated for creation. (2) Infrastructure is weak. No video support services for technical issues and editing. (3) Room infrastructure is much weaker. (4) IT support through Belinda is excellent.

As we were rushed into implementing this within modules, time was not used efficiently as training was 'broad-brush' approach rather than tailoring to individuals need as they fuller assessed how to introduce appropriately for their own modules.

Trainees’ perceptions on factors which have prevented their use of blended learning (December 2015).

The responses were coded into themes although these are not independent of each other.

Lack of time/other priorities Navigation Lack of clarity/information Technology issues Other

5

BLITT Project 2015-16- Efficiency Focus

To what extent have the inputs into implementing blended learning been converted into outputs.

(BA Primary Education year 1, BA Early Years year 1, PGCE Primary 5-11 and PGCE Primary 3-7) all with QTS.

The response to questionnaire given to staff in December 2015 in terms of the total % for each of the resources listed below as being used ‘efficiently’ and ‘very efficiently’ to achieve outputs on Program Logic.

Questions %All intended outputs were achieved (Dec 2015)

(with reference to the Program Logic)

53% strongly agree and agree

TIME - time to identify tutor's needs (skills, knowledge) to be able to implement blended learning

86

TIME - time to train individual tutors 60TIME – to train all ITT QTS tutors 60TIME - to access University support 47TIME - to create and load resources on NILE 33RESOURCES – availability of ipads/hardware 20RESOURCES - availability of software 40RESOURCES - availability of IT in teaching rooms 20STAFFING - availability of IT support staff 36STAFFING - availability of Learning Technologists 57STAFFING - availability of peer support within ITT QTS 79BUDGET - availability of funding to support outputs 29

TIME - time to indentify tutor's needs (skills, knowledge) to be able to implement blended learning

TIME - time to train individual totors

TIME - time to train all ITT QTS tutors

TIME - to access University support

TIME - to create and load resources on NILE

RESOURCES - availability of iPads/hardware

RESOURCES - availability of software

RESOURCES - availability of IT in teaching rooms

STAFFING - availability of IT support staff

STAFFING - availability of Learning Technologists

STAFFING - availability of peer support within ITT QTS

BUDGET - availability of funding to support outputs

Production of Learning Events and Online Learning Packages.

6

Responses to monitoring checkpoint Dec 2015 to identify how many and which technological tools had been developed/used.

How many? 0 1-5 6-10 11-15 16+Learning Events developed for yourself

1 8 5 4 9

Learning Events developed for others

3 8 6 3 2

Learning Events used technology in pre F2F

4 9 5 1 3

Learning Events used technology in F2F

5 9 3 5

Learning Events used in post F2F

4 10 5 1 2

Online Learning Packages developed for own modules

7 11 2 1 1

Online Learning Packages developed for others

5 13 4

Technology Tools used by staff – at Dec 2015

a) Most Used (above 6)

Padlet – 14 responses

Kaltura – 9 responses

Video Clips – 7 responses

b) 2- 4 responses for each of these

Google docs, ipads, NILE, YouTube, panopto, ppts, Thinglink, prezi, pic collage, switch, imovie,

c) One response for other tools (generally one or two individual staff only)

Pastoon, google plus, google forms, popplet, book creator, photos, blog + others

Monitoring Implementation Nov 2015

Module Code

Module Title Blended Learning - have we got online learning ?

Blended Learning - have we got Learning Events

ITT1001 Professional Studies 17

ITT1004 Primary Foundation Subjects - 1ITT1005 Subject Specialism 1- HistoryITT1006 Subject Specialism 1- EnglishITT1007 Subject Specialism 1- MathsITT1008 Subject Specialism 1- ScienceITT1009 Subject Specialism 1- PEITT1015 Professional Studies 1- Early YearsITT1016 EYFS: Communication and Language

and LiteracyITT1017 EYFS Areas of Learning

ITT1018 EYFS MathematicsITT1019 Specialism 1: Learning and Teaching

(3-7)ITT1020 Early Years ICT and PedagogyITT1021 Mathematics - 1ITT1022 Art and Design Specialism 1ITT1023 English- 1ITT1024 Science, Design & Technology -1ITT1025 Computing/Technology Enhanced

Learning and RE/PSHE

BLITT Project 2015-16- Impact Focus

What results, intended or unexpected, have been produced directly or indirectly by blended learning.

(BA Primary Education year 1, BA Early Years year 1, PGCE Primary 5-11 and PGCE Primary 3-7) all with QTS.

The responses to questionnaires given to staff in December 2015 and April 2016 have been compared to trainee responses to questionnaires given in December 2015 and April 2016 in terms of the total % responses to ‘completely’ and ‘mostly’.

Intended Short term Outcomes – see Program Logic Students participate in Blended Learning Positive responses to resources Increased knowledge (for tutors and students) in relation to learning outcomes for modules Improved confidence and competence of tutors to use Blended Learning Students’ increased skills in using Blended Learning Students report changes in confidence and competence in linking theory and practice.

Staff BA Primary and BA Early Years

PGCE Primary 5-11 and PGCE Primary 3-7

Cohort 20 99 52QuestionTrainees participate in Blended Learning

80 85 59

Trainees participate in pre F2F activities

60 82 66

Trainees participate in F2F activities

100 99 93

8

Trainees participate in post F2F activities

20 65 31

Trainees respond positively to technological resources used

100 83 87

Trainees have increased knowledge of the learning outcomes of the module

79 90 88

Staff have increased knowledge of the learning outcomes of the module

80

Trainees have increased skills in using blended learning

65 71 73

Staff are more confident in using blended learning

80

Staff are more competent in using blended learning

75

Trainees are more confident in linking theory and practice

55 84 83

Trainees are more competent in linking theory and practice

65 83 85

1. Initial staff perceptions on impact of technology tools on teaching and learning (December 2015)

Teaching and learning

enhances the teaching and learning, enables teaching off site, good ideas for pF2F, F2F and post F2F

a good way to making sessions more visual/classroom based it feels like it has raised the breadth of our teaching styles to enhance student experience just increased variety, impact on learning - not sure means I am aware so can adapt LE as necessary online have helped a lot positive range, opportunities, to merge on/off site, efficiency time some different approaches for students to share ideas, video always good when IT works speed up progression in PE on specific skills they are an essential component of my teaching they have provide variety to the methods of engagement yes, data collection generating ideas, capturing events eg panel discussion for further use in

various ways

Development of resources

easier to access during sessions and for students, however IT services have been most unhelpful

frustration at the provision at times, it takes time to develop the teaching using these, and this has not been available to the right extent

hugh amount of time preparing, relatively short delivery time, very positive but IT has been universally poor

9

lack of reliable IT has made me less confident to use these techniques

Not made a difference

not really undecided - need to get feedback from year 1 trainees I have felt too much under time pressure to try many of the elements Belinda taught us,

although I can see the advantages of being able to use them. Difference to teaching currently, - minimal.

2. Initial Trainee perceptions on benefits from blended learning (December 2015)

Trainees were asked to comment on

How is the use of blended learning approach benefitting your development as a teacher? How is blended learning benefitting the development of your skills in engaging in technology

enhanced learning? What are the benefits to you as a student of blended learning?

Their responses were coded into themes although these are not independent of each other. These initial themes have been identified:

Access to learning –flexibility of being able to access off site Increased knowledge and understanding Reinforcement of learning Linking theory to practice Approaches to learning Resources Organisation Clarity, relevance, workload, timing Not beneficial

3. Staff perceptions on impact of blended learning in supporting trainees to link theory and practice (April 2016).

the links between the 3 components of the BL enable a more structured development, of these links - supporting exploration of them in more depth

better prioritisation of essential information, and then providing opportunities for trainees to apply learning

need to access material on theory prior which is then applied in F2F showing practice

4. Trainee perceptions on impact of blended learning in supporting them to link theory and practice (April 2016). – see full list

UG

Able to link what I have learnt in lesson to practise and understand what to do in certain situations

10

Blended learning reinforces the theory so in a classroom environment background learning allows effective practice

Post-F2F activities allow you to further knowledge and actually have time to reflect on how you would put theory in to practice. Time that you may not yet in the F2F sessions.

The pre-F2F allow for reading around the topic as do the F2F this means when applying and discussing with mentors as trainees it can be seen where it fits practically.

PG

Some of the information online has been useful for better exploring the link between theory and practice.

Blended learning is helpful in my teaching practice, linking technology and teaching- which is paramount in the 21st century.

Research tasks have helped to understand the theory behind certain techniques used within the classroom.

Intended Medium Term Outcomes – see Program Logic

Tutors change T&L strategies to regularly include technology in teaching. Students increased engagement with online learning communities Increased links made between theory and practice by students Increased links made between theory and practice by tutors.

Staff BA Primary and BA Early Years

PGCE Primary 5-11 and 3-1

Cohort 20 99 52QuestionStaff regularly use technology in their teaching

70 92 88

Trainees engage with online learning communities

35 63 54

1. Staff perceptions on how blended learning is supporting trainees to engage with online learning communities.(April 2016)

Increased repertoire of digital tools to support online collaboration increased technological awareness and experience, unintended learning communities being created

2. Trainee perceptions on how blended learning is supporting them to engage with online learning communities. April 2016

The G+ communities have helped develop understanding on a new platform as well as introducing activities for the classroom.

Allows us to learn about different communities - improves knowledge and understanding. Ideas posted on learning communities provide resource/lesson ideas. Helps give resources and ideas of what to input into the online community giving opportunity to

share. It gives you a fantastic bank of learning. We have too many communities to keep up with. It's very

time consuming and keeps you doing work which you may need F2F contact for. It could encourage more people to use online learning communities. For those that do use it, it works

well as it is a place to share ideas that will permanently stay there.

11

BLITT Project 2015-16- Sustainability Focus

Focus Group of 3 staff responses collated

1. What do you think might be the future benefits, if any, from Blended Learning for

a) yourself –

Greater flexibility to my teaching and learning - opportunity to go beyond the here and now e.g using onsite film clips when considering LOTC and more meaningful pre FTF tasks - less sure about some post F2F . At its best when online linked to F2F as opposed to separate can enable more effective use of F2F.

Professional development, evaluation of and reflection on own practice. Developing quality and sustainable models of learning.

b) trainees

Where resource base expanded ( see above) , for tool box types packages - e.g.TAR tailored more effectively to student need.

Taking responsibility individually and collectively for own learning. Raising expectations and challenge.

c) ITT as a whole

Given that so much of our course is offsite, suggest that it ought to be used to link up with these sites to enhance cohesiveness of the course rather reducing out F2F more -we are in a different position to most courses which are essentially university based.

12

Developing communities of enquiry and practice between academic tutors, students and school partners. ·

Less face to face time – double edged: May help trainees to develop greater independence in learning or could just alienate themFocus of sessions can relate more to pedagogy rather than subject knowledge

2. How do you think our Blended Learning approach has been achieved?

A positive ethos in the team coupled with significant opportunities for us to interpret it to our needs Stages to series of activities - my initial fear that it would be an online lecture never came about

although have seen some dreadful stuff on youtube Putting teaching and learning first - including offsite and online Very strong support from Belinda and members of IT team when needed.

Team work and a ‘driver’ in a leadership role. Time allotted to development. A collective commitment to embracing change.

Staged introduction required across whole programmes Staff training Staff meeting time – establish basis and need for Staff time – development of modules

3. How do you think our Blended Learning approach could be used by other areas of the University?

I think the PRE F2F and F2F (some need to refine what we mean post F2F in current model) could be easilyapplied elsewhere.

I think that the models we have created are readily transferable.

Don’ think it should be a one approach fits all

4 What will make our Blended Learning approach sustainable ?

reviewing packages over time - there is a danger that could become ossified knowing how far to go, where it introduce it -using the best balance re student needs - e.g advise

against reducing contact for much of year 1 - strong case for F2F in this year to help transition to HE. developing links with offsite partnerships -schools etc to diversity learning contexts

Continued investment in training and time to develop quality resources approaches collaboratively, and share this good practice. I would really value the opportunity to create DVD footage of teachers in our partnership that we could use in seminars.

Technical support and professionally produced online learning packages Identifying the critical balance of F2F and online to ensure trainees are not put off the course Ensuring expectations on student workload are balanced across the course Ensuring a balance of different types of activity

13

Monitoring of activities and response from tutors to ensure trainees feel work is valued for some designated activities

Clear expectations identified at the outset – trainees clear about their responsibilities

Gathering Evidence for the Blended Learning Study 2015-16

Blended Learning – Initial Evaluation Timescale for BLITT Project

The evaluation framework will use the five themes of Appropriateness, Effectiveness, Efficiency, Impact and Sustainability. The evaluation schedule will be as follows:

Theme December 2015 March 2016 CommentsAppropriateness Students – Questionnaire

See below for C

Staff – QuestionnaireSee below for A

Project Team Focus Group

Student Questionnaires – BA Primary and BA Early Years OnePGCE Primary 5-11 and PGCE Primary 3-7

Staff Questionnaires – all full time and Associate Tutors who teach ITT QTS – ethical issues shared,

Questionnaires – likert and open ended

Project Team Focus Group – Elaine, Jo BB, Anna, Gill C, Paul and Karen L (and Dave)

Effectiveness Students – Questionnaire D and Focus Group

StudentsQuestionnaire

StaffQuestionnaire

Student Questionnaires - BA Primary and BA Early Years Year OnePGCE Primary 5-11 and PGCE Primary 3-7

Student Focus Group – BA Primary Year One – offered to all students to be involved

Efficiency Staff – Questionnaire B StaffQuestionnaire

Staff Focus Group E

Staff Questionnaire – all full time and Associate Tutors who teach on ITT QTS

Staff Focus Group – Gill Chambers, Paul Bracey, Karen Lindley

Impact Students and Staff Questionnaire

Staff Questionnaire – all full time and Associate Tutors who teach on ITT QTS

Student Questionnaire – BA Primary and BA Early Years Year One and PGCE Primary 5-11 and PGCE Primary 3-7

Sustainability Staff Questionnaire

Student Questionnaire

Staff Focus Group

All full time and Associate staff who teach ITT QTS

Student Questionnaire - BA Primary and BA Early Years Year One and PGCE Primary 5-11 and PGCE Primary 3-7

Staff Focus Group – Elaine, Jo BB, Anna, Gill C, Paul, Karen L (and Dave)

14

Research Ethics – confidentiality, anonymous (need to use tool which does not identify person), responses for research questions will be part of the project but the expectation is that staff will be expected to complete the monitoring documents as part of University expectations of monitoring new teaching and learning strategies.

Date Teacher Educator Voice Trainee (Learner) VoiceMarch 25th 2015

Individual feedback via email – to Head of ITT after staff meeting.

June 8th and 9th 2015

Individual reflections and feedback via email to Head of ITT after staff training day on learning theories and technological tools.

Nov 27th 2015

Monitoring use of technological tools – Checkpoint One questionnaire issued during teacher educators meeting.

Focus: Use of technology and Impact on Teaching

Dec 2nd 2015

Staff and Student Meeting – verbal feedback on using technology enhanced tools from first year trainees.

Dec 4th 2015

Blended Learning Questionnaire for first year trainees.

Focus: Appropriateness and Effectiveness

Dec 7th and 8th

Blended Learning Questionnaire for post graduate trainees.

Focus: Appropriateness and Effectiveness

Dec 11th 2015

Staff Questionnaire

Checkpoint Two Focus: Appropriateness

Checkpoint Three – EfficiencyDec 11th 2015

Undergraduate Staff- Student Liaison Committee - feedback on training provision

Jan 6th 2016 Monitoring and development – teacher educator meeting to review module specifications for blended learning and future development hours

March 7th 2016

Undergraduate Staff- Student Liaison Committee - feedback on overall training provision

March 9th 2016

Staff and Student Meeting – verbal feedback on using technology enhanced tools from first year trainees.

March 14th 2016

Postgraduate Staff- Student Liaison Committee – feedback on

15

overall training provisionApril 13th 2015

Staff Questionnaire

Checkpoint Three Focus: Impact

April 22nd 2016

Blended Learning Questionnaire Undergraduates Year One

Focus: ImpactMay 16th 2016

Blended Learning Questionnaire: Postgraduates

Focus: ImpactMay 16th 2016

Staff Project Meeting

Focus: Teaching and Learning PresentationStaff Focus Group Meeting

Focus: Sustainability

There has been a change to the planned use of questionnaires to staff in phase two of the project. The project team reviewed the evidence from phase one and considered if during the time between January and April there would have been any significant differences in teaching using blended learning for trainees and decided there could be limited changes. The two reasons for this decision are firstly in terms of time spent during this time period at University; all postgraduate trainees have been in schools and not attending University Learning Events and the year one trainees have been only attending their specialism modules. Secondly, the data collected in phase one questionnaires provides an insight into perceptions on ‘effectiveness’ which will be followed up with a staff focus group rather than a separate questionnaire as planned.

16