Quality assurance and monitoring of

online/remote learning

platforms

By Alassane OUEDRAOGO, Ed Specialist (M&E), UNICEF MENA RO

Highlights:

1. Ranging from fully online to fully face to face and including every variation in between

2. High tech, low tech and low-cost solution were used— i.e. TV, Radio, Online, Offline, Face to Face

3. An estimated 40% of enrolled school age children did not have access to any form of remote learning

1.

2.

3.

4.

METRICS to monitor and evaluate E-leaning or Online learning

•Relevance to developmental stage and age, etc, Adherence to curriculum, grade level, level of quality

•What was the change in knowledge, skills, attitudes, or behaviors?

•Was content used as intended , relevant to needs, and captivating to users?

•Use of content, Time spent

•Who accessed technology, programming, and content relative to intended users?

•Who accessed education and learning opportunities and resources

REACH ENGAGEMENT

QUALITY ASSURANCE

OUTCOMES

1.

2.

3.

4.

Eleven quality aspects which we consider crucial whenassessing quality in e-learning / online learning:

1. Educational Material/content2. Structure/virtual environment/classroom3. Communication, cooperation and interactivity4. Student assessment 5. Flexibility / adaptability of education6. Support (student and staff and parent)7. Staff qualifications/skills and experience8. Institutional leadership and vision on e-learning 9. Resource allocation10. The holistic process aspect (Learning

processes)11. Security / online safety

Q1: How should we understand quality

assurance when it comes to

online/remote learning?

Enabling environment for learning

– Conceptual framework

1.

2.

3.

4.

Q2: What measurement tools do we have to monitor and evaluate

online/remote learning?

Inspection (QA) of and support to

schools and teachers

Benchmarking to inform development monitoring checklists

Standards setting for quality assurance

1.Framework for national quality

assurance system

Standards setting and quality assurance : Criterion-related references to judge performancea. National standards for quality online programsb. National standards for quality online coursesc. National standards for quality online teaching

BenchmarkingTo inform the development of checklists for monitoring

a. Institution policy and governance for technology-supported learning and teaching

b. Planning for and quality improvement of the integration of technologies for learning and teaching

c. Information technology infrastructure to support learning and teaching

d. Staff support for using technologies for learning and teaching

Integrate in-person and remote approaches, use multi-modal interfaces,

and employ mixed methods to inspection (QA) of and support schools

and teachers (E-classroom observation of inspection, spot checks,

review recordings, student testimonials, Focus Groups Discussion)

1.

2.

3.

4.

Q3: What challenges are we facing in-terms of quality assurance

measurements?

Methodological development

Capacity gaps within National

Agency for quality assurance

Benchmarking

Lack of effective Learning

Management System - (LMS)

Lack of national / regional

assessment framework

Adapting existing quality

assurance measurement tools to digital

context

Integration of e-learning criteria

in the nation quality

assurance system

Standards

1.

2.

3.

4.

Q4: How do we ensure quality assurance of online/remote learning isn’t unfair

compared to in-class learning?

1. Integration of e-learning criteria in the national quality assurance system

2. National benchmarking established for remote learning and in-person learning

3. National standards established for remote learning and in-person learning

4. Holistic and comprehensive national framework for quality assurance for both online and face to face—i.e Adjustment of the existing national frameworks to cover online learning, where transferability exists.

5. Capacity building of National Agency in charge of quality assurance

6. Periodic audits of the quality assurance systems

7. Revision of existing guidelines –Reinterpretations or adjustments of existing aspects and criteria

1.

2.

3.

4.

Q5: What are the best practices for quality assurance that we have seen

globally/regionally?

Australasian Council on Open, Distance and E-learning : “Benchmarks for the use of technology in learning and teaching in universities”

Swedish National Agency for Higher Education

USA : Council for Higher Education Accreditation (CHEA) : “publication and good practices on Accreditation and assuring quality in distance leaning”

UK Open University (OU) : internal quality processes• Quality control, a process that operates post-production or delivery to determine that everything works• Quality assurance, a process applied during production to assure fitness• Quality enhancement, a process that assures improvementwww.universitiesuk.ac.uk/use of technology in learning and teaching in universities

1.

2.

3.

4.

Q6: What practices can we bring to the Arab region to support quality

assurance and monitoring?

Framework development – i.e: Methodological framework and toolkit for monitoring remote and or hybrid learning during and post COVID-19

Harmonized assessment tool for quality assurance assessment and monitoring system

Standard development for quality assurance assessment and monitoring system

Benchmarking for quality assurance assessment and monitoring system

Capacity building of national Agency in charge of quality assurance assessment and monitoring system

Capacity building of teacher in online delivery

Promoting embedded quality assurance assessment and monitoring system into online and distance learning platforms

1.

2.

3.

4.

Q7: How can we nurture local eco-systems to enhance quality,

contextualization and sustainability of remote learning?

Sharing best practices

South-South cooperation and collaboration

Network of practitioners and academia and Tech companies

Partnership— i.e : (i) for enhancing capacity of national teacher training providers; (ii) The promotion of blended products.

24 June 2020

DAPM, PD, EMOPS, & ICTD

Joint initiative

Methodological framework and toolkit for monitoring remote and or hybrid learning during and post COVID-19

Lay the foundation for improving the national capacities to generate critical data

needed by countries themselves for their own use and to ready the statistical data

collection systems for emerging dataset in dealing with health emergency contexts that

require system changes to ensure continued monitoring and evaluation.

PURPOSE

1.

2.

3.

Provide guidance to establish and maintain a set of standard indicators, operational

data collection processes and tools at country level to generate up-to-date

evidence needed at country level to gauge the effectiveness and appropriateness

of national response and education systems in pandemic or lockdown contexts

Support interactions and dialogue with Ministry of Education on how MoE can

incorporate monitoring mechanisms of distance and or remote learning into their

existing systems— i.e: integrate formal assessment and statistical modules into their

existing national distance learning platforms which can automatically generate key data

on coverage, time spent using, SABER for monitoring distance learning delivery. 4.

Where

are the children not

accessing remote

learning?—

i.e: geographic

location

What

are the changes in the

children learning gain or loss

WHO

are the children

reached and not

reached or at risk of

not being reached by

one particular form of

remote learning

Why

are children not reached

by any or no form of

remote learning delivery

modes

Looking at specific groups

Unpack interplay barriers

STRUCTURE OF THE

FRAMEWORK

Conceptual framework to guide the implementation of monitoring process of remote

learning and or blended learning towards integration in national M&E systems

Expected deliverables

1.

2.

3.

Planning, monitoring and evaluating remote learning implementation operational guide—

i.e suggested set of indicators to be tracked will be documented

Suggested set of data collection instruments / questions databank—The toolkit includes a

suggested organized set of data collection instruments in adherence with the conceptual

framework for targeted audience— MoE, Schools, Teachers, Parents, Students

4.

4. Guide for sampling design, data analysis plan and data collection options

5.Features and characteristics of an effective distance learning system – From an IT and a

statistics viewpoint/standpoint, this module will discuss technical capabilities and

requirements.

15

Thank you!