Teacher Educators’ Quality Standardskpese.gov.pk/Downloads/KPESPS-Papers/TEQS Pilot...
74
Transcript of Teacher Educators’ Quality Standardskpese.gov.pk/Downloads/KPESPS-Papers/TEQS Pilot...
Published by: Khyber Pakhtunkhwa - Education Sector Plan Support Programme (KP-ESPSP) House # 26 CA-1, Circular Road, University Town Peshawar, Pakistan T: +92 91 – 584 3293-4 (Ext 32) F: +92 91 – 584 3296
Report Author: Nighat Lone – Consultant Contributing Editor: Dr. Philip Powell-Davies – Senior Strategy Advisor, KP-ESPSP Design: Amjad Hussain Alvi – Technical Advisor, KP-ESPSP January 2019
Khyber Pakhtunkhwa Education Sector Plan Support Programme (KP-ESPSP)
Technical Cooperation Services
This publication has been produced with the assistance of the European Union. The contents of this publication are the sole responsibility of the KP-ESPSP Programme and can
in no way be taken to reflect the views of the European Union.
Contents
Acronyms
Executive Summary & Recommendations ................................................................... 1
Recommendations ................................................................................................................ 3
1. Recommendations for the Selection Stage of Teacher Educators:........................... 4
2. Recommendations for the Training of Trainers: ....................................................... 5
3. Recommendations for the Delivery and Monitoring of Training: ............................. 6
Chapter 1: Introduction ............................................................................................. 8
1.1. Background ............................................................................................................... 8
1.2. Piloting the TEQS ....................................................................................................... 9
Chapter 2: Pilot Study Methodology ...................................................................... 10
2.1. Key Contributors to the TEQS Pilot ......................................................................... 11
2.2. Design of the Pilot ................................................................................................... 11
2.3. Monitoring and Evaluation Approach for the Pilot ................................................. 12
2.3.1. Monitoring and Evaluation Tools ............................................................... 13
2.3.2. The Monitoring Team ................................................................................ 13
2.4. Training Stage 1: Selection of Trainers .................................................................... 14
2.4.1. Orientation of DEOs ................................................................................... 14
2.4.2. Shortlisting of CVs ...................................................................................... 14
2.4.3. Demonstration of Training Ability .............................................................. 14
2.5. Training Stage 2: Training of Trainers ..................................................................... 15
2.5.1. Module Development ................................................................................ 15
2.5.2. Training of Trainers (TOT) .......................................................................... 16
2.6. Training Stage 3: Delivery & Monitoring of Training............................................... 16
Chapter 3: Findings of the Pilot Study .................................................................... 17
3.1. Observation of Training........................................................................................... 17
3.1.1. The Observation Form ............................................................................... 17
3.1.2. Rubric ......................................................................................................... 17
3.2. Findings from the Observation Study of Training ................................................... 18
3.2.1. Section 1: PREPAREDNESS ......................................................................... 18
3.2.2. Section 2: DELIVERY ................................................................................... 19
3.2.3. Section 3: COMMUNICATION .................................................................... 27
3.2.4. Section 4: TIME MANAGEMENT ................................................................. 30
3.3. Findings from the Focus Group Discussions ........................................................... 32
3.3.1. FGD 1: Master Trainers .............................................................................. 32
3.3.2. FGD 2: Trainees/Participants of the CPD Training ..................................... 35
3.3.3. FGD 3: TEQS Design Team (PITE/DCTE/RITE/DE&SE) ................................ 37
Chapter 4: Recommendations and Conclusions ..................................................... 40
4.1 Implications for Further Capacity Building .............................................................. 42
4.2 Recommendations .................................................................................................. 43
4.2.1 Training Stage 1: Selection of Teacher Educators ...................................... 43
4.2.2 Training Stage 2: Training of Trainers ........................................................ 45
4.2.3 Training Stage 3: Delivery and Monitoring of Training .............................. 46
4.3 Conclusions ............................................................................................................. 47
Annexures .................................................................................................................... 49
Annex I: Pilot Training Centres ................................................................................ 50
Annex II: TEQS Pilot Monitoring Team .................................................................... 51
Annex III: Observer Code of Conduct ...................................................................... 52
Annex IV: Criteria for Short-listing the CVs of Trainers ........................................... 53
Annex V: CV Template - Haripur & Peshawar ......................................................... 54
Annex VI: Training Demonstration Observation Tool ............................................. 56
Annex VII: Observation Tool for Monitoring of Training ........................................ 57
Annex VIII: Focus Group Discussion Templates x 3 ................................................. 61
Annex IX: Pilot Implementation: Monitoring and Evaluation Framework .............. 64
Acronyms
ASDEO Assistant Sub-Divisional Education Officer
CPD Continuous Professional Development
DCTE Directorate of Curriculum & Teacher Education
DE&SE Directorate of Elementary & Secondary Education
DEO District Education Officer
E&SED Elementary and Secondary Education Department
EU European Union
FGD Focus Group Discussion
KP Khyber Pakhtunkhwa
KP- ESPSP Khyber Pakhtunkhwa Education Sector Plan Support Programme
JRF Joint Review Framework
LT Lead Trainer
M&E Monitoring & Evaluation
MNSQE Minimum National Standards for Quality Education
MT Master Trainer
NPST National Professional Standards for Teachers
OT Observation Tool
PITE Provincial Institute for Teacher Education
RITE Regional Institutes for Teacher Education
SDEO Sub-Divisional Education Officer
SLO Student Learning Outcome
SOP Standard Operating Procedure
SQMI School Quality Management Initiative
TE Teacher Educators
TEQS Teacher Educators Quality Standards
TORs Terms of Reference
TOT Training of Trainers
1
Executive Summary & Recommendations
This report describes the findings of a pilot study for implementation of the Quality Standards for Teacher
Educators (hereafter TEQS) carried out in two districts of KP from October to December 2018 in
conjunction with the E&SED, DCTE, and PITE under the EU-funded KP-ESPSP programme of technical
assistance to the E&SED.
Context
The Khyber Pakhtunkhwa Elementary and Secondary Education Department has been endeavouring to
improve the quality of education in the province over the recent years. KP became the first province to
start the implementation of the Minimum National Standards for Quality Education (MNSQE) approved
at the National level. The recent emphasis of the KP government on standards based reform efforts
culminated in the development and approval of the Teacher Educators’ Quality Standards (TEQS) in 2018,
which was envisioned as an essential extension of the MNSQE. From a systemic efficiency point of view,
the focus on standards presents a significant step forward. Standards present a unique opportunity to
align all quality indicators towards a singular objective – the improvement of students learning outcomes.
The role of teacher educators is of vital importance as they prepare teachers to effectively and efficiently
improve their professional competence. The TEQS are designed to support the E&SED, DCTE, RITEs and
PITE in assessing, selecting and professionally developing quality trainers so that they can become an
established pool of trainers and strengthen the management of teacher education across the province.
The standards were developed with the technical support of the EU’s KP-ESPSP programme, which has
been promoting the idea of developing teacher educator standards in KP since 2017. The aim of
developing these standards was to help PITE and DCTE enhance the skills of their teacher educators,
professionalise their way of working, and contribute more effectively to the delivery of teaching and
learning across KP. Currently, PITE draws upon a pool of trainers from the field on a needs basis to meet
its training requirements. These trainers, typically teachers or school heads, are taken out of their regular
responsibilities for the duration of the training, deliver the required training and then return to their
duties.
The TEQS provide a more strategic option for PITE and DCTE and enable them to define the minimum
levels of competence expected of teacher educators and set a clear reference point against which to
measure their practice, performance, knowledge and behaviours. The standards also serve as a tool to
help define professional development programmes, and help the E&SED, DCTE and PITE in establishing a
permanent pool of specialist teacher educators.
2
Objectives of the Pilot
After the approval of the TEQS in 2018, the first recommended step towards its implementation was the
piloting of the TEQS on an ongoing training to measure their effectiveness, to assess their fitness for
purpose and to measure the extent to which they promote better quality training.
The specific objectives of the pilot were to:
• Measure the effectiveness and applicability of the TEQS in an upcoming training of PITE
• Assess the potential of the Standards to improve the quality of trainings, when applied, and
• Identify potential gaps and areas of improvement or modification in the TEQS.
The focus of the pilot study (which ran from October- December 2018) was the Continuous Professional
Development (CPD) of Head Teachers programme. This was selected by PITE and DCTE and covered 6
training centres in Haripur (for female trainees) and 6 training centres in Peshawar (for male trainees)
Annex I contains the details.
Findings of the Pilot
The pilot was designed to assess whether all the process revisions and quality inputs managed to produce
good trainers who could demonstrate the desired level of proficiency in line with the TEQS. Another
purpose was to see whether the standards have the capacity to identify candidates who can become part
of a sustainable pool of teacher educators.
The aggregated findings of the pilot show that :
• Almost 40% of the trainers selected using the TEQS principles demonstrated a good level of skill
aligned with the TEQS indicators by scoring 3 on the Likert scale for most of the indicators in the
observation tool. These trainers performed well within the desired level of effectiveness in all
categories.
• An additional 14% performed especially well demonstrating consistent and high level of skill,
knowledge and professional competency. This means that more than half of the selected trainers
(54%) performed satisfactorily or very well against the criteria laid down by the TEQS.
• A further 33% of the trainers performed in line with expectations but inconsistently. In particular
they struggled with some of the higher order professional skills which take more time to
understand, embed and exemplify in a training context. This signifies the need for further capacity
building to raise their levels of competency so that is consistently in line with the TEQS indicators.
This also points to another value of using the TEQS – they provide a level of confidence, a
benchmark against which DCTE and PITE can make their judgments about the quality of trainers
and training, and provides specific guidance on what action needs to be taken to build professional
3
capacity and capability. Of the entire sample of selected trainers only 14% performed consistently
poorly and did not meet the TEQS quality standards in most instances.
The results imply that a positive number of trainers (54%) who scored 3 and 4 on the Likert scale had
benefitted from all the additional efforts that had been planned for the TEQS pilot, including a more
rigorous selection process and additional TOT training days. Based on the results these are trainers that
can be confidently recommended to be included in an established pool of teacher educators.
The findings also enabled the identification of those trainers requiring further assistance and capacity
building. These (33%) were the ones who showed potential but with some limitations. The results also
clearly showed which trainers not to include in future trainings, as they did not show any potential in most
areas.
The principal areas of weakness identified through the monitoring of the TEQS pilot centred on higher-
order pedagogical skills, such as facilitating the active involvement of participants in the learning process;
assessment of learning; dealing with questions and discussion; flexibility in the use of materials and
activities; structuring a session; conflict management, and some aspects of communication and time
management.
The findings of the FGDs further support these observational study findings as the trainers themselves felt
that the selection process had helped in securing good quality trainers. The training on the standards and
facilitation skills had contributed to enhancement of their capacities as trainers. They were able to apply
most of the skills learned to make their training meaningful and engaging.
The participants of the CPD training courses also supported this claim, highlighting the positive aspects of
the training as well as the trainers’ approach, stating that the training was more active and beneficial
than most of their previous training experiences.
Participants of the FGD from among the team of experts involved in the standards’ development and
piloting process expressed the view that implementation of the TEQS must be enforced without delay as
the TEQS forms an important link in the education quality chain in KP.
Recommendations
Based on the results it can be claimed that the pilot study does in fact validate the importance of the
TEQS. The next step is to look into the scaling up and institutionalize the processes that would continue
to ensure the implementation and application of the TEQS. However, in order to scale up the
implementation of the processes for engaging large numbers of trainers it may be necessary to make
some further adjustments.
The summary of recommendations for up-scaling the pilot fall into three categories:
4
1. Recommendations for the Selection Stage of Teacher Educators:
Step 1
Orientation of
DEOs
• Orientation of DEOs should be continued as a key step in trainers’ selection
• For future requirements, for example CPD in 8 new districts, all the relevant DEOs should be invited to PITE for a half-day session on the principles of the TEQS and its implications for the selection of trainers and other elements of the training.
• This model should be applied to all training and become part of PITE’s SOPs.
• An additional benefit of this step would be the continuous widespread dissemination and understanding of the TEQS and ownership of the district management in the quality aspects of teacher professional development.
• The DEOs should not just send nominations only but merit-based CVs of the recommended candidates.
• The initial filtering of the CVs should be done at the DEOs office in line with the requirements highlighted in the orientation session. An IT platform can be used to store the results of filtering and enable rapid sharing of results.
• The E&SED should facilitate this process by ensuring direct access of PITE to the districts.
Step 2
Shortlisting of
CVs
This is another important step for filtering the nominations. This step prevents the engagement of irrelevant and unsuitable applicants and helps to ensure a good balance of qualifications, skills and experience. It is recommended that:
• PITE continue with this process as regular practice after the nominations and the districts forward the long-list of CVs.
• The tool for shortlisting developed for the pilot should be used for this step linked to IT platforms to store and share the results.
• The CV format developed for the pilot should be made a regular part of the selection process. Using an IT platform to share the templates online for candidates to fill and submit will facilitate the process and speed up the selection process.
Step 3
Practical
Demonstrations
of Training
Ability
This was the final step in the TEQS pilot for final shortlisting of the trainers based on their actual performance. Though an important step it is the most challenging to take to scale. Observing demonstrations of 24 plus candidates was possible given the time frame but for PITE to organise demonstration sessions for more than 2500 candidates for CPD alone would require a solution involving other groups working hand in hand with PITE.
5
• As demonstration gives the best insight into a candidate’s potential this step should continue in some form.
• Options could include:
o Inclusion of demonstration sessions in the TOT (with room for rejecting some candidates during the TOT)
o Training district staff (DEOs and ASDEOs and possibly RITEs) in conducting demonstration sessions (after PITE finalises the shortlist of CVs) to join hands with PITE to select the most suitable candidates.
o Capacity building of all experts conducting demonstrations should be ensured to minimise the risk of selecting poor trainers.
2. Recommendations for the Training of Trainers:
Additional days
of training
• PITE should continue to arrange for additional days of training for all the future candidates for the multiple training courses it manages throughout the year. This is important to strengthen skills and competencies in core skills, especially the more complex higher-order ones.
Resourcing • The Department, DCTE and PITE should advocate for additional resources to facilitate the roll-out of a strengthened TOT approach.
TOT Content • The training of trainers on standards and facilitation techniques should be spread over 3 instead of 2 days. This is because the 2-day approach could not capture all the areas that require proficiency in trainers, which was evident through some of the weaker areas identified.
• Include practice and demonstration sessions during the TOT to compensate for any deficiencies in utilising demonstration during selection and ensure filtering of candidates at this stage,
• PITE should build up a pool of highly qualified Lead Trainer Teacher Educators from around the province to conduct the TOTs to complement staff at PITE who will not be able to carry out all the tasks themselves at scale.
Development of
a permanent
pool of Teacher
• PITE can use the three-day TOT to run annual courses for potential master trainers to keep strengthening its pool of trainers.
6
Educator
trainers
• An online database of all trainers in the pool needs to be established with full records of their skills and the training courses they have designed and/or delivered.
• The pool needs to be managed and quality-controlled by PITE so that the TEQS principles are maintained and operationalised.
• The pool is, to some extent, an interim measure. Its effectiveness can be evaluated over time to assess the feasibility and desirability of establishing a teacher educator cadre in KP. This would greatly support the continued development and strengthening of the teacher educator function in KP.
3. Recommendations for the Delivery and Monitoring of Training:
Observation
Tools
• PITE should continue to use this tool for monitoring all training and develop a database of all trainers recommended for the pool based on observational data of skills, competencies and behaviours.
Monitoring
Processes
• A strengthened standardised approach to monitoring (and evaluation) of all training courses should be instituted at PITE
• PITE should ensure that monitoring becomes an essential and regular part of every training programme.
• Staff at PITE and all Lead Trainers should be trained in the use of the monitoring tools and how to report on the data gathered.
• An online form to enter and collate results is already developed and can be used as part of the monitoring approach.
• The monitoring framework needs to be aligned with the M&E framework in operation at PITE and DCTE and aligned to wider quality assurance mechanisms that have been recommended for E&SED.
Conclusions
Delivering quality education to each school child is one of the core purposes of an education system.
Quality in education is, however, a multidimensional concept which means that reaching this goal may
not be straightforward. It requires multiple, parallel inputs from all stakeholders with a well-
coordinated approach which links policy and strategy to sound management of the systems to create
the conditions for effective and efficient implementation.
7
The TEQS is an essential component of the quality chain, which requires concerted efforts for its sustained
institutionalisation in the system. An encouraging factor is the evidence of strong ownership of the TEQS
in the Department, Directorate, DCTE and PITE, which can guide the efforts for its continued
implementation alongside other quality-focused initiatives. This is due, in no small part, to the highly
participatory and consultative way in which the TEQS was designed over many months with
representatives of DCTE, PITE, the RITEs, DE&SE and the Planning Cell of the E&SED. The TEQS pilot study
confirms the value and applicability of the standards and their potential in achieving the purpose they
were designed for – better quality training to facilitate better quality teaching and learning in the
classroom.
The system already provides opportunities for the professional development of teachers through in-
service provision and the pre-service induction programme (managed by DCTE, PITE and RITEs). The
teacher educator quality standards are designed to strengthen the quality assurance mechanisms within
these organisations and bring new energy and strategic coherence to existing practices. As such the TEQS
strengthen the accountability of teacher education institutions in delivering their services.
For this purpose, DCTE and PITE need to continue this work - extending their own efforts and providing
strategic management of teachers in-service training and also assessing the feasibility of extending the
approach to the pre-service induction programme. Clear roles and responsibilities of all stakeholders for
every stage of the training process should be written and agreed in detailed SOPs.
Additional resources should be allocated for the special purpose of developing the capacity and capability
of teacher educators and developing, as a matter of priority, a pool of teacher educator trainers to serve
as a resource that PITE can draw upon and manage strategically and operationally.
The Teacher Educator Quality Standards open up the discussion on systemic and procedural reforms
required to make their regular implementation as business as usual a reality in KP. This includes
strengthening existing practices as well as looking at long-term sustainable solutions. Setting up a
permanent cadre for teacher educators within the system with a clear chain of command and career
ladder is one such longer-term solution to streamline the professional development landscape for
teachers and teacher educators in the province. A detailed feasibility study analysis is required to chart a
way forward with policy, strategy, implementation and cost implications, and a phased action plan that
could be implemented.
8
Chapter 1: Introduction
1.1. Background
The development of the Teacher Educators’ Quality Standards (TEQS) is a significant step towards
improving the quality of education in Khyber Pakhtunkhwa (KP). The TEQS paves the way for quality
improvement across all tiers of teachers’ professional development. The quality of teacher educators (TEs)
directly impacts the performance of teachers and, in turn, students’ learning.
The aim of developing these Standards was to help PITE and DCTE enhance the skills of their teacher
educators, professionalise their way of working, and contribute more effectively to the delivery of
teaching and learning across KP. Currently, PITE draws upon a pool of trainers from the field on a needs
basis to meet its training requirements. These trainers, typically teachers or school heads, are taken out
of their regular responsibilities for the duration of the training, deliver the required training and then
return to their duties.
The TEQS provide a more strategic option for PITE and DCTE and enable them to define the minimum
levels of competence expected of teacher educators and set a clear reference point against which to
measure their practice, performance, knowledge and behaviours. The standards can also be used as a tool
to help define professional development programmes, and help the E&SED, DCTE and PITE in establishing
a pool of competent teacher educators.
The Teacher Educator Quality Standards are built around 5 themes, namely:
Quality Standards
for Teacher Educators
1. Knowledge of content and
methodologies
2. Demonstrating knowledge and skills through
practice
3. Planning, delivery and monitoring
4. Communication and facilitation
5. Professional development for teacher educators
9
Each theme is further divided into sub-themes and indicators that detail measurable and observable
proficiencies at different stages of the training process. A range of assessment tools are indicated to
determine the proficiency of TEs at induction and post-induction level.
1.2. Piloting the TEQS
It was agreed with DCTE and PITE in late 2018 that once the Standards had been developed, they needed
to be piloted in a real-world context to assess their fitness for purpose and to measure the extent to which
they promote better quality training.
The specific objectives of the pilot were to:
• Measure the effectiveness and applicability of the TEQS in an upcoming training of PITE
• Assess the potential of the Standards to improve the quality of trainings, when applied, and
• Identify potential gaps and areas of improvement or modification in the TEQS.
The pilot focused on the application of the first four of the TEQS themes as they relate directly to selection
of trainers and training delivery. The fifth theme, which relates to professional development, can only be
measured in the longer term, and only after themes 1 to 4 have been embedded in the system.
The focus of the pilot study (which ran from October- December 2018) was the Continuous Professional
Development (CPD) of Head Teachers programme. This was selected by PITE and DCTE and covered 6
training centres in Haripur (for female trainees) and 6 training centres in Peshawar (for male trainees)
Annex I contains the details.
10
Chapter 2: Pilot Study Methodology
PITE is charged with in-service training of all teachers as well as school and district level managers. For
this purpose, it relies on a pool of trainers from within the education system. These trainers are typically
teachers, head teachers, and RITE instructors (with a preference towards those who have already
conducted training in the past). As a general practice PITE asks the DEOs for nominations of candidates.
PITE has developed basic criteria for short listing candidates nominated by the districts. This tends to focus
on academic qualifications, but has not proven to be entirely successful, as the requisite skills set for
training teachers does not necessarily correlate with formal qualifications. During the teacher training,
master trainers may be observed by PITE but it is not regular practice. If during such an observation the
trainer’s performance is found to be less than satisfactory s/he is restricted from imparting further
training. In management terms, this is not an effective way to manage the pool of trainers or to develop
capacity in the system.
The TEQS provides an opportunity for PITE and DCTE to align their processes and tools to accommodate
the application of quality standards at each stage of the training from identification and selection of
trainers to their preparation, capacity building, delivery and feedback.
All training conducted by the E&SED is structured in the following tiers of responsibility:
DCTE Planning & oversight of training
PITE Design, materials
development & training delivery
Lead Master Trainers: trained by PITE
Master Trainers: trained by Lead Master Trainers
11
2.1. Key Contributors to the TEQS Pilot
All relevant stakeholders including representatives of E&SED, PITE and DCTE and the KP-ESPSP programme
team participated during the various stages of the pilot, as follows:
Planning
Representatives of E&SED, Directors of DCTE and PITE and
selected specialists supported by the KP-ESPSP team participated
in the design of the pilot study
Selection of Trainers DCTE, PITE, Directorate, KP-ESPSP and District management
Module Development DCTE and PITE supported by KP-ESPSP
Training Of Trainers PITE (including Lead Master Trainers)
Master Trainers Candidates from the field nominated by the DEOs
Design of Monitoring and
Evaluation System KP-ESPSP in consultation with DCTE, PITE
Observation team PITE and DCTE
FGDs KP-ESPSP team, PITE
2.2. Design of the Pilot
As a first step in designing of the pilot, workshops were held to scope out the pilot outline and processes,
the roles of different organisations and the tools for monitoring and evaluating the pilot. The current
processes for running ongoing training were assessed in terms of their relative strengths and effectiveness
(in relation to the application of TEQS). Based on this, a range of revised processes were developed to
align the training to the quality standards in a more structured way and finalise the elements of the pilot
study:
• The pilot was focused on the CPD training of head teachers being implemented by PITE covering 6
training centres in Haripur (for females) and 6 centres in Peshawar (for males)
• A total of 24 master trainers (12 male and 12 female working in pairs in each centre) would be
selected to deliver the training
• The different stages of the training (e.g. selection of trainers, TOT, delivery of training) would be
modified in terms of processes and tools to ensure alignment to the quality standards framework
• It was decided that a control district or centre would not be used due to time and human resource
constraints. While a control group would have been desirable to assess comparability between
12
approaches using and/or not using the quality standards approach, in this case the focus was more
to assess the suitability of the approach to the needs of PITE and DCTE
• Tools were developed for each stage of training from selection to delivery. Existing tools were used
as the basis for review and adjustment
• Roles and responsibilities of PITE and DCTE were clearly delineated for each stage of the pilot, and
• A monitoring and evaluation framework was designed to measure the effectiveness of the pilot
with tools focused on observation of training and FGDs with participants in the pilot study itself.
2.3. Monitoring and Evaluation Approach for the Pilot
The M&E framework was designed to assess the effectiveness of the TEQS approach and principles, and
develop a feedback loop which could provide insight to stakeholders on the fitness for purpose of the
TEQS approach in strengthening the effectiveness of the teacher educator function in KP (details of the
M7E approach are contained in Annex IX).
With these objectives in mind, the M&E approach would address the implementation arrangements as
well as investigate the short-term impact of the TEQS approach on the teacher training activities
contained in the CPD programme. Therefore, part of the Kirkpatrick model1 was used as the basis for
analysing evidence on the effectiveness of the TEQS in this pilot.
As far as the piloting of the draft standards developed by the TEQS activity, it was important to collect
monitoring data on the first two levels of performance (reactions and learning), given the relatively short
timeframe of the CDS. The other two levels (transfer and results) refer to longer-term impact and change,
which fall outside the scope of the pilot. The details of the four levels of the Kirkpatrick Model and their
measurement in relation to the pilot are outlined below:
1 The Kirkpatrick Model is recognised as a standard to evaluate the effectiveness of training. It recommends the measurement of training value across four levels. Each level is important as it draws attribution linkages between the various levels. Each advancing level requires more effort in assessment and measurement but also provides more information pertinent to training effectiveness.
13
After much debate, it was agreed to monitor only the selected districts and centres and not include a
control district. The rationale was that the pilot was designed to allow DCTE and PITE to practise the
application of the standards through revised processes on a smaller scale and then use the pilot
experience to decide on scaling-up based on what worked and what did not. If more time had been
available, it would have been desirable to have a control group in order to analyse the comparability of
the TEQS approach against the usual practice for carrying out such training. Consequently, the findings of
this study need to be understood as a qualitative-dominant description of a pilot exercise to assess the
fitness for purpose of the TEQS approach. Recommendations made in this report are made on that basis.
2.3.1. Monitoring and Evaluation Tools
Two main tools were designed to gauge what the teacher educators have learned in terms of knowledge
and professional skills as a result of the pilot. These include an observational tool (OT) based on the TEQS
to document the degree to which the MTs had learned from the process and how it had influenced their
performance. In addition, FGDs were conducted with three groups: selected trainers, trainees and the
experts who participated in the pilot (Annexes VII & VIII).
2.3.2. The Monitoring Team
DCTE and PITE nominated 3 Male and 3 Female staff members each to form the monitoring team. Some
of these members had been part of the standards development team while some members had to be co-
opted (see Annex II). All of the monitors attended standardization training to ensure they understood the
nature and purpose of the tools being used and crucially to standardise the way in which the tools were
assessed and reported on.
The OT rubric (scored on a 4-point Likert scale) was discussed in detail and finalised. The participants were
apprised of their task and a code of conduct for observation was discussed and finalised (Annex III).
Participants were also coached on how to fill the form and enter the scores on the online form
immediately after the observation. Each member of the observation team was assigned a training centre
where they would be observing the training. They were each required to fill out two observation forms;
one for each trainer at the training centre.
A total of 5 Focus Group Discussions were conducted as part of the M&E approach with three different
sets of participants. Two FGDs were conducted per district with selected MTs and trainees to understand
their perceptions and experiences regarding the training. An additional FGD was conducted with
members of the team who had helped to design the pilot (and the TEQS) to gather their impressions about
the process and the effectiveness of the TEQS pilot. For this purpose, three FGD questionnaires were
developed (see Annex VIII).
14
2.4. Training Stage 1: Selection of Trainers
Application of the TEQS required modifications in the selection process to accommodate the standards
and ensure the induction of the best possible candidates. To identify potential trainers, PITE currently
relies on nominations from the DEOs, which it secures through a letter directed to the DEOs’ office and
channeled through the E&SED. Despite the fact that the letter includes some minimum requirements for
the MTs the nominated candidates may not demonstrate the essential skills expected of a good trainer.
This constitutes a risk to the quality of those selected and the quality of the training being delivered.
For the purpose of the pilot certain steps were added to the selection process of the master trainers to
ensure identification of the best candidates in line with the TEQS. This is described in more detail below.
2.4.1. Orientation of DEOs
This was an additional step as both DCTE and PITE felt that the DEOs should be briefed face to face about
the purpose and requirements of the training and the trainers. For this purpose, DCTE, PITE and KP-ESPSP
team representatives visited the DEO’s offices in Peshawar and Haripur on 5th and 6th November 2018
respectively and met with the DEO, SDEOs and ASDEOs to share with them the relevant details of the
pilot. DEOs were given an overview of the standards and how they impacted the selection process of the
trainers and discussed the important role of the DEO’s office in ensuring the application of standards on
the nominations they forward to PITE. The DEOs were asked to submit 20 CVs against the requirement of
12 trainers per district.
2.4.2. Shortlisting of CVs
Director PITE nominated a team of 3 experts to shortlist the CVs. A tool was developed for grading the
CVs against a set of criteria derived from the standards (Annex IV). The team’s task was to apply the criteria
designed during the workshop to score each CV and prepare a long-list of selected candidates .
A simple CV format was developed and sent to the DEO’s office for the incumbents to fill and submit
(Annex V). As a result, 17 filled out forms were received from Haripur and 27 CVs received from Peshawar
were short listed in line with the criteria. 13 candidates from Haripur and 20 candidates from Peshawar
were subsequently shortlisted.
2.4.3. Demonstration of Training Ability
After the shortlisting of CVs an additional step was introduced to further filter and refine the selection
process in accordance with the TEQS. The shortlisted candidates were invited to take part in a
demonstration session to assess their training competency. Director PITE nominated a 4-member team
to observe the demonstration of each candidate and score him/her according to an observation tool (OT),
which was a checklist of criteria developed specifically for this purpose (see Annex VI). The indicators of
the OT were derived from the TEQS and focused on delivery and communication skills. The
15
demonstrations took place at Haripur on 23rd November and Peshawar on 5th December 2018 where
each candidate was given a pre-prepared session selected from the CPD module to demonstrate in front
of the selection committee. The committee scored each candidate’s performance based on the indicators
of the demonstration OT. 33 candidates participated in the demonstrations out of which 25 qualified to
attend the TOT as potential master trainers.
2.5. Training Stage 2: Training of Trainers
According to current practice PITE carries out a TOT for the selected Master Trainers for all training events.
The duration of the TOT equals the number of training days for teachers/head teachers plus one extra day
for planning and preparation. For the ongoing CPD training programme for head teachers, PITE had
already designated 4 training days for the MTs which included 3 days for CPD training and 1 day for
planning and preparation.
For the pilot it was agreed to add two additional TOT days to introduce the TEQS and cover facilitation
techniques in more detail. The purpose of these two additional days was to train and equip the master
trainers in the requisite skills of training and facilitation as envisioned in the TEQS.
PITE had already developed a generic three-day training module on facilitation skills. It was agreed to take
that module as the base document and modify it in line with the TEQS.
2.5.1. Module Development
A joint team of PITE and DCTE was formed to adapt the module for the additional two days of training.
This was done in a working session with the nominated team on 11th November 2018. The idea was to
review the existing module on facilitation techniques and align it to the standards.
The existing module had been designed for three days of training. During the working session this was
reduced to two days. A session on orientation to the TEQS principles and indicators was added, while the
sessions on communication skills were further enriched based on the indicators of the standards.
Overall each session was linked to the TEQS and participants were encouraged to identify this link and
repeatedly refer to the standards throughout the training. The last session of the module focused on
introduction of the observation tool against which their performance and professional competencies
would be assessed. The module was carefully reviewed by PITE and DCTE, and finally approved by the
DCTE.
16
2.5.2. Training of Trainers (TOT)
The TOT took place at PITE from 10th – 15th December, with two days of training on standards, three days
training on CPD and one day for planning and preparation. All 25 candidates participated in the TOT. PITE
designated two trainers to lead the two-day sessions on facilitation skills in the light of the TEQS.
Observations of day two of the training showed a positive degree of satisfaction on the part of the trainers
as well as trainees with the process so far. The trainees were especially appreciative of the additional two
days training that were added. Most observed that it was the first time they had participated in such
training.
The trainees also positively reacted to the TEQS document. They reflected positively on the robust process
employed by PITE for their selection and expressed a sense of accomplishment on being amongst the
selected candidates.
2.6. Training Stage 3: Delivery & Monitoring of Training
The CPD training programme for head teachers was a three-day training course which was held across the
12 pilot centres in Peshawar and Haripur. The trained master trainers carried out the training. PITE and
DCTE jointly monitored the training. A 12-member monitoring/observation team consisting of 3 male plus
3 female staff members both from DCTE and PITE was formed and trained in their task. The monitoring
framework was based on the Kirkpatrick Model, which is recognised as a standard to evaluate the
effectiveness of training interventions..
The purpose of monitoring the TEQS pilot was to assess the effectiveness of the TEQS approach and
principles, and develop a feedback loop, which could provide insight to stakeholders on the fitness for
purpose of the TEQS approach to enhancing the effectiveness of the teacher educator function.
17
Chapter 3: Findings of the Pilot Study
3.1. Observation of Training
Observations of training were carried out in the pilot centres as a means of assessing the applicability and
suitability of the TEQS approach to the delivery of good quality trainer training and teacher training. The
12-member monitoring team were assigned to different centres where they carried out the observation
of training (a total of 21 trainers were observed in 12 centres). Each observer spent a full day in their
centre observing training activity and completing the observation forms manually and online.
3.1.1. The Observation Form
The Observation Tool for teacher educators (Annex VII) was based on the content of the TEQS and
designed to be filled out by an observer to determine the quality of teacher training. This tool was
developed over several working sessions with a mixed team from PITE, DCTE, DE&SE and Planning Cell,
and complemented work from the ongoing parallel activity of designing an M&E framework for PITE.
The OT has a total of 35 indicators covering the four core aspects of observation: preparedness, delivery,
communication and time management. For 29 of these indicators a standard 4-point Likert scale was
applied. For the remaining 6 indicators, which mostly focused on preparation and a few aspects of
delivery, a ‘Yes or No’ response was included. A remarks column was added to each indicator in case any
clarification was required. Additionally, two questions at the end of the OT allowed for deeper reflections
on the trainers’ performance by the observer.
3.1.2. Rubric
The rubric to ensure uniform interpretation of the 4-point Likert scale was as follows:
4 Evidence observed exceeded expectation throughout, resulting in a very effective training
session
3 Consistent evidence observed throughout, resulting in an effective session
2 Evidence observed, but limited in nature and not observed throughout the session
1 No evidence observed during the session
The scores that would indicate a satisfactory and excellent level of effectiveness in terms of performance
are 3 and 4. Anything below this would signify some gaps and inconsistencies in the expected
competencies.
18
3.2. Findings from the Observation Study of Training
The findings of the observational study are discussed below and organised according to the four core
aspects of the observation form: preparedness, delivery, communication and time management.
3.2.1. Section 1: PREPAREDNESS
Indicators in this section were designed to assess whether the TE was well prepared for the training as
per the instructions, guidelines and discussions during the TOT. Preparedness spanned punctuality,
readiness in terms of materials and training schedules, along with arrangement of materials to be used
during training.
Indicators 1-3
1. Reached venue at least 30 minutes before the start of the session
2. Ensured availability of teaching aids
3. Training schedule is prominently displayed
Collated findings from the 21 observations reflect that most trainers (73%) reached the venue at least 30
minutes before the start of the session (Table 1), and 95% ensured the availability of teaching aids (Table
2). Some of the remarks made by the observers stated that multimedia was not available at the venue,
which made it difficult to deliver certain aspects of the training. Only 32% of the trainers had the training
schedule displayed in a prominent place in the training room with 68% having no training schedule
displayed (Table 3). In a few cases the previous day’s schedule was displayed as mentioned in the remarks
column of the OT.
Table 1 Table 2
73%
27%
0%0%
50%
100%
Yes No NA
1. Reached venue at least 30 minutes before the start of
the session 95%
5% 0%0%
20%
40%
60%
80%
100%
Yes No NA
2. Ensures availability ofteaching aids
19
Table 3
3.2.2. Section 2: DELIVERY
Delivery is the most important aspect of training and refers to all elements of imparting the content,
including setting the ground rules, subject matter knowledge, effective use of techniques, engaging the
participants through Q&A and keeping all trainees involved. Effective delivery also encompasses
appreciation of work done by the trainees to keep them motivated while paying attention to the
participants’ learning, recapping the key content areas and encouraging trainees throughout.
The delivery section of the OT had 18 indicators - three marked as either Yes/No and 15 to be scored on
a 4-point Likert scale in line with the rubric.
Indicators 4 - 21
4. Introduces her/himself
5. Describes the norms of participation and ground rules
Indicators 4 and 5, which dealt with introductions and setting of ground rules, were designed for day 1 of
the training. However, it was expected that the ground rules chart would be displayed in the training room
for the remaining training days. Results show that half of the trainers (50%) introduced themselves. 14%
did not introduce themselves while in 36% of the cases it was found not applicable. The NA rating may
have resulted from the fact that this was not expected on day 2 (Table 4). 55% described ground rules,
recapping day 1 of training while 23% did not talk about ground rules (Table 5).
Table 4 Table 5
32%
68%
0%0%
20%
40%
60%
80%
100%
Yes No NA
3. Training schedule is prominently displayed
50%
14%
36%
0%
20%
40%
60%
80%
100%
Yes No NA
4. Introduces Himself/herself
55%
23% 23%
0%20%40%60%80%
100%
Yes No NA
5. Describes the norms of participation and ground rules
20
6. Explains the objectives at the beginning of each session
An important aspect of good training delivery includes clearly explaining the objectives at the beginning
of each session. Only 36% of trainers explained the objectives before beginning each session while 50%
started the activities without explaining the objectives (even though this was an important aspect covered
in the TOT). In 9% of the cases this was scored as not applicable which is not clear, as every session has
clearly laid out objectives in the module, which should be explained before starting the session (Table 6).
Table 6
7. Uses a variety of training techniques to address individual learning needs to ensure maximum
participation
41% of the trainers observed were able to consistently demonstrate variety in their training style at the
desired level of effectiveness to ensure participation. Some of the trainers (5%) showed exceptional
results, exceeding expectations in this category. 45% applied only some variation in their teaching style
but not consistently and only to a limited extent, while 9% failed to demonstrate any evidence of variety
in their teaching style to meet individual needs throughout the training day (Table 7).
Table 7
36%50%
9%
0%
20%
40%
60%
80%
100%
Yes No NA
6. Explains the objectives at the beginning of each session
9%
45%
41%
5%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
7. Uses a variety of training techniques to address individual learning needs to ensure
maximum participation
21
8. Uses resources effectively, including technology and AV aids, to enhance trainees’ learning
The next indicator (8) is closely linked to the previous one (7) and focused on use of appropriate resources
during training. 45% demonstrated a good understanding of the use of teaching aids (scoring 3). These
trainers consistently made effective use of resources to enhance their trainees’ knowledge. Additional to
this 9% performed exceptionally well in this category and scored 4 points. Of the remaining trainers 27%
did not consistently use the resources as required and 19% , rather worryingly, did not use any teaching
aids at all during their sessions (Table 8). Though the TOT did address these important facets (teaching
techniques coupled with use of teaching aids) of training, more emphasis needs to be placed on both
these aspects to enhance learning.
Table 8
9. Demonstrates good command over subject matter knowledge
Most trainers (59%) showed good command over subject matter knowledge with a score of 3 points with
an additional 5% exceeding expectations in this category. Of the remaining trainers, 23% scored 2 points,
showing some gaps in subject knowledge with room for improvement and 14% demonstrated no or poor
command over the subject they were training (Table 9).
Table 9
18%
27%
45%
9%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
8. Uses resources effectively, including technology and AV Aids, to enhance
trainees knowledge
14%
23%
59%
5%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
9. Demonstrates good command over subject matter knowledge
22
10. Manages contributions from all participants by involving trainees in discussion and activities
Ensuring trainees’ active participation in the learning process is an important skill for a good teacher
trainer. In this category 45% scored 3 points, which shows that were able to consistently and competently
involve all participants in activities and discussions throughout the sessions. Half of the trainers (50%)
involved the trainees to some extent but not consistently, leaving room for improvement. The remaining
5% did not involve the participants at all (Table 10). More focus needs to be assigned to participants’
involvement in future TOTs in order to maximise learning and reflection.
Table 10
11. Follows the structure, instructions and flow
12. Stays focused on and meets the stated objectives (activities) of the training module
13. Conveys clear instructions for the activities
The trainers were given a pre-designed module with step-by-step instructions that they were required to
follow. The next three indicators (11, 12, 13) related to the trainers’ performance in following the
module’s instructions. Results show that 55% of trainers were able to follow the instructions and flow of
activities to the desired level of effectiveness according to the module while the rest fell short of following
even the written instructions in a satisfactory manner (Table 11). Similarly, the same number (55%) of
trainers were able to effectively and consistently stay focused on the objectives of the training (Table 12).
About 50% were able to deliver clear instructions for the activities, while the remaining half of those
observed either did not consistently give clear instructions (45%) or did so poorly (5%) (Table 13).
5%
50%
45%
0%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
10. Manages contributions from all participants by involving trainees in discussion and activities
23
Table 11 Table 12
Table 13
This shows that close to half the participants were not able to follow the module’s instructions or stay
focused on the objectives or give clear instructions for activities despite the fact that all these elements
were clearly outlined in the module. There is a clear need for a sharper focus on this area during TOT
sessions in order to strengthen the way in which tasks and activities are described. This is particularly
important because teachers need to follow good models in the training room and apply them to their own
teaching.
14. Applies appropriate questioning techniques to engage all participants
15. Motivates Participants to ask questions
Effective use of questioning techniques is an essential skill to keep participants engaged. This includes
both asking and allowing questions by participants and responding appropriately. The results show that
45% trainers were able to apply questioning techniques consistently to the desired level of effectiveness
(scoring 3 on the rubric), while 50% of the trainers did not consistently engage all participants through
questions. 5% trainers did not engage participants through questions at all (Table 14).
On the other hand, only 32% of trainers encouraged participants to ask questions for clarifications and/or
responded with confidence, while 55% allowed questions but not consistently. The rest (14%) didn’t show
5%
41%
55%
0%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
11. Follows the structure, instructions and flow (activities) of the training
module
5%
36%
55%
5%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
12. Stays focused on and meets the stated objectives
5%
45%
50%
0%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
13. Conveys clear instructions for the activities
24
any evidence of allowing the participants to ask questions. (Table 15). This is also a significant finding and
has implications for the types of skills that need to be practiced in TOT in order to improve the way that
trainees experience a training session. Learner-centred approaches require that question and answer and
discussion is prioritised in the training room and in the classroom. It is clear that even specially selected
trainers rely heavily on trainer-centred approaches, which diminishes the opportunities for purposeful
discussion in the training sessions.
Table 14 Table 15
16. Motivates less active participants by encouraging and assigning specific tasks to them
17. Manages dominant people or participants who talk too much without directly offending them
Indicators 16, 17, 18 and 19 are connected as they deal with the trainers’ response to the types of
participants present in a training session and the ability of the trainer to adapt their approach accordingly.
This appears to be a weaker area as a high number of trainers (73%) performed less than the desired mark
on involving less active participants in the discussions (scoring 2 on the Likert scale). Only 18% trainers
demonstrated the ability to involve participants to an acceptable standard with none demonstrating an
exceptional level of competence. 9% of trainers did not demonstrate any evidence of this competency
(Table 16). Similarly, for management of dominant participants 41% were not able to manage dominant
trainees effectively. In this category 32% altogether failed to control dominate participants. Only 14%
scored 3 points in this category and 5% scored 4 points. In 9% of the cases this was marked as not
applicable which indicates that there were no dominant participants in the group (Table 17).
5%
50%
45%
0%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
14. Applies appropriate questioning techniques to engage all participants
14%
55%
32%
0%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
15. Motivates participants to ask questions and responds effectively and
confidently
25
Table 16 Table 17
18. Appreciates contribution of participants verbally and non-verbally
Participants’ management also covers appropriately acknowledging and appreciating their contributions
through verbal and non-verbal responses. As far as appreciation of participants’ contributions is
concerned 41% of the trainers showed appreciation at the desired level of effectiveness and 5% exceeded
expectations. The remaining either did so inconsistently (50%) or not at all (5%) (Table 18).
Table 18 Table 19
19. Applies different assessment techniques during the session
Application of a variety of assessment techniques to gauge learning was one of the weaker areas as only
32% of the trainers scored 3 points and 5% scored 4. 55% did apply some assessment techniques but with
room for improvement. The remaining trainers did not assess learning during the sessions (Table 19).
Overall, the clear finding from the results of these four indicators means that greater emphasis needs to
be placed on how trainers are sensitised to the needs of different types of participants, and how their
own training approach needs to be flexible to accommodate difference. This relates to the earlier findings
about training in a learner-centred way and providing a good model to teachers to use in their own
classrooms. This concept is at the heart of the TEQS.
9%
73%
18%
0%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
16. Motivates less active participants by encouraging and assigning specific tasks to them
32%41%
14%5%
9%
0% 20% 40% 60% 80% 100%
1234
NA
17. Manages dominant people or participants who talk too much
without directlyoffending them
5%50%
41%5%
0%
0% 20% 40% 60% 80% 100%
1234
NA
18. Appreciates contribution of participants verbally and non-verbally
9%55%
32%5%
0%
0% 20% 40% 60% 80% 100%
1234
NA
19. Applies different assessment techniques during the session
26
20. Recaps the important elements and the responses and concludes the session effectively
Concluding and recapping important elements of the sessions successfully was not observed to be a strong
area among the trainers as only 38% scored 3 points and 5% scored 4. Almost 24% trainers did not recap
the sessions at all. The remaining (33%) showed inconsistency with this important aspect of the training,
signifying the need for further capacity building (Table 20).
Table 20 Table 21
21. Creates/sets up conducive-learning environment through participation, encouragement, recognition,
harmony, teamwork and feedback
As a positive outcome, a high number of trainers scored 3 points (64%) and 4 (5%) in creating a conducive
learning environment through encouragement and recognition, which shows that most of the trainers
understood the importance of positive interaction during delivery. Interestingly, this is in contract to the
findings for the preceding indicators where the practical approaches to inclusivity, flexibility and learner-
centred were less than optimal.
Summary of Findings: DELIVERY
Looking at the overall findings for the delivery topic, it is evident that trainers showed potential and scored
in line with the core expectations with 42% scoring 3 on the Likert scale on all indicators. Around 3% of
the trainers overall delivered exceptionally well scoring 4 and stood out as excellent trainers. The pilot
study indicates that from this group, around 45% of the trainers demonstrated strong level of competence
to deliver high quality training. These trainers should be deployed in future training courses organised by
PITE.
On the other hand, almost 43 % of the trainers delivered their training but with some inconsistencies on
most indicators, signifying either some gaps in their performance. These were the ones who tried to
perform in line with the training and instructions they had received but had not fully internalised the TOT
training. It is evident that with some more input and practice these trainers could strengthen their skills
24%
33%
38%
5%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
20. Recaps the important elements and the responses and concludes the
session effectively
9%23%
64%5%
0%
0% 20% 40% 60% 80% 100%
1234
NA
21. Creates/sets up conducive-learning environment through participation,
encouragement, recognition,teamwork and feedback
27
in order to provide good quality training. 11% of the trainers clearly demonstrated their lack of potential
and inability to deliver trainings scoring 1 on the Likert scale in almost all indicators.
The pilot study has demonstrated those areas of delivery which require further capacity building (probably
due to their conceptual and methodical complexity). These include understanding how to provide learner-
centred training; managing participants with greater insight; assessment of learning, and providing
opportunities for greater discussion and reflection in the training.
3.2.3. Section 3: COMMUNICATION
At the heart of effective and impactful training experiences lie the communication skills of the trainers.
Communication ranges from body language to eye contact to pitch and tone of voice and a confident
demeanour. Delivery and communication are closely linked, as a good communication style is at the heart
of effective training delivery. The communication section of the observation monitoring tool has 10
indicators.
Indicators 22 - 31
22. Projects self-confidence and maintains professional composure
23. Communicates in the required language with proper pronunciation
24. Communicates in clear and strong voice using pitch and volume appropriately
25. Maintains eye contact with appropriate expressions
A majority of the trainers displayed strong levels of performance on their projections of self-confidence
(59%), (Table 22) proper use of language (82% + 5%) (Table 23), and voice intonation (55% + 9%) during
the training sessions (Table 24). The area of further improvement where more than half the trainers either
did not manage this consistently or not at all was maintenance of eye contact with appropriate
expressions (Table 25). This needs to be considered against cultural norms in this context, although there
are ways of enhancing eye contact in a training situation without upsetting cultural sensitivities.
Table 22 Table 23
5%
36%
59%
0%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
22. Projects self-confidence and maintains professional composure
0%
14%
82%
5%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
23. Communicates in the required language with proper pronounciation
28
Table 24 Table 25
26. Demonstrates a positive and non-threatening body language and puts the participants at ease
27. Communicates ethically and without conveying any bias through verbal, non-verbal cues
A very high sensitivity to use of proper body language was displayed by the trainers (77% + 5%) during the
sessions (Table 26). Similarly, a large number of trainers demonstrated sensitivity in avoiding any kind of
bias through verbal and non-verbal cues (77% + 5%). This shows that the TOT successfully addressed these
aspects of training which are often significantly absent (Table 27).
Table 26 Table 27
28. Manages conflict effectively through discussion and mediation
29. Generates enthusiasm and energy throughout the lesson
30. Respects the views of the trainees
31. Exhibits sensitivity to students' personal cultural, gender differences and disabilities, responds
appropriately in a non-threatening, pro-active learning environment
0%
36%
55%
9%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
24. Communicates in clear and strong voice using pitch and volume
appropriately
9%
45%
41%
5%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
25. Maintains eye contact with appropriate expressions
5%
14%
77%
5%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
26. Demonstrates a positive and non-threatening body language and puts
the participants at ease
0%
14%
77%
5%
5%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
27. Communicates ethically and without conveying any bias through
verbal, non-verbal cues
29
The ability to maintain authority in the training environment and balance this with sensitivity and respect
is one of the more complex competencies that good trainers need to develop. In the important area of
conflict management 41% of trainers displayed some ability to resolve issues but with shortcomings, but
only 23% were able to manage difficult situations at a satisfactory level. In 36% of the cases it was marked
as NA, which means that either a conflict situation did not arise or the group was well managed by the
trainer (Table 28).
Table 28 Table 29
Around 59% of the trainers did not satisfactorily maintain energy levels consistently during the training
and 9% were not able to generate energy and enthusiasm as a trainer, showing that this requires more
attention in future TOTs. 32% successfully created a proactive learning environment for trainees
consistently (Table 29).
A majority of trainers (55%) positively demonstrated respect for participants’ views and contributions
while the rest showed room for further improvement in this regard (Table 30). Again, this is an indicator
that is at the heart of learner-centred approaches and consequently needs to be prioritised. As far as
responding appropriately towards trainees’ gender, personal and cultural differences and/or disabilities
36% cases were marked as not applicable perhaps due to the homogenous nature of the group, while in
the remaining instances 41% trainers performance well with a small number showing strong levels of
understanding and management of differences within the group (Table 31).
0%
41%
23%
0%
36%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
28. Manages conflict effectively through discussion and mediation
9%
59%
32%
0%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
29. Generates enthusiasm and energy throughout the lesson
30
Table 29 Table 31
Summary of Findings : COMMUNICATION
The TEQS lays emphasis on good communication skills for teacher educators. The overall collated results
showed a generally positive outcome as far as communication skills of the trainers are concerned. Around
55% of the trainers showed acceptable levels of skill a in language, intonation, self-confidence and positive
body language by scoring 3 points. An additional 3% demonstrated communication skills beyond
expectations.
However, 32% overall clearly found generating and maintaining energy and positive body language
challenging. Managing conflict is also one of the higher-level, more sophisticated skills and it is not
therefore surprising that less than half of the trainers were able to demonstrate acceptable skills in this
area. Only about 3% demonstrated poor communication skills and would not merit further inclusion in a
pool of teacher educators.
3.2.4. Section 4: TIME MANAGEMENT
Good time management by a trainer does not only mean that he or she was able to cover all the stated
content in the stipulated time but it also includes a good adjustment of time to accommodate discussion,
questions and clarifications without disrupting or overly stretching the schedule. The best of trainers can
ruin a good training by poorly managing the available time. The time management section of the OT
monitoring tool includes 4 indicators.
Indicators 32 - 35
32. Starts the training session on time
33. Remains focused on the topic to manage the time effectively during the training sessions
34. Avoids wastage of time by handling irrelevant discussions professionally
35. Teaching aids and other material is placed in sequence of activities to save time
0%
45%
55%
0%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
30. Respects the views of the trainees
0%
18%
41%
5%
36%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
31. Exhibits sensitivity to students’ personal, cultural, gender
and ability differences and responds appropriately
31
Even though the trainers scored well on indicators pertaining to time management, one of the most
disquieting findings was the inability of 63% of TEs to start the training session on time (even though
results of indicator 1 show that 73% reached the venue on time). The male trainers were significantly
worse at this (73%) than female trainers (18%). This was more surprising as day two of the training was
selected for observation to allow the trainers and trainees to settle into the schedule of the training (Table
32). While cultural factors indicate that such time-keeping is not prioritised (and presumably not well-
valued) it does interfere with the ability to manage a training programme effectively and efficiently. In
other contexts, time-keeping is viewed as a priority and an indicator of respect.
However, the TE’s focus on effective time management (Table 33 and 34) once the sessions started was
stronger, including remaining focused on the topic (50% + 5%) and the professional handling of irrelevant
issues (50% + 9%) which were dealt optimally in more than half the instances. However, a significant
number of the trainers struggled when viewed against these indicators. In terms of organising the training
materials to save time the results were almost half and half (Table 35).
Table 32 Table 33
Table 34 Table 35
45%
18%
18%
18%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
32. Starts the training session on time
18%
27%
50%
5%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
33. Remains focused on the topic to manage the time effectively during the
training sessions
5%
36%
50%
9%
0%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
34. Avoids wastage of time by handling irrelevant discussions
professionally
18%
36%
36%
5%
5%
0% 20% 40% 60% 80% 100%
1
2
3
4
NA
35. Teaching aids and other material is placed in sequence of activities
to save time
32
Summary of Findings: TIME MANAGEMENT
It is probable that the ability to manage time in a training situation can be developed with more practice
over time rather than through the explanation of its importance. Those trainers who struggle with this
should be encouraged to observe other trainers who have stronger skills in this area and then be given
the chance to reflect on what works and what does not as strategies for improving how to manage the
limited time available in a training session. The same points can be made in terms of how to manage time
in a classroom. Teachers need a great deal of support in understanding how to make purposeful use of
time with their students, hence the importance for them to view effective models of time management
by teacher educators.
3.3. Findings from the Focus Group Discussions
Focus Group Discussions (FGDs) were carried out to complement the data derived from the observational
study and provide a measure of triangulation in the findings. Three different groups were targeted:
• Participants of the CPD training
• Master Trainers; and
• The core team who developed the TEQS.
The premise was that the data from the FGDs will be used alongside the observational data to determine
the extent to which the principles of the TEQS had been implemented in a real training situation during
the pilot, and the extent to which the implementation of the TEQS had had a positive impact on
participants, trainers and senior officials of PITE and DCTE (in terms of strengthening the teacher educator
function).
FGD data were gathered around a limited and pre-determined set of questions, (Annex VIII) which had
been specifically designed to address the underlying concepts and principles of the TEQS. The data are
qualitative-dominant, like the observational study, and is useful for identifying perceptions and reactions
to the training experience carried out as part of the pilot.
A total of five FGDs were held in Haripur and Peshawar with selected trainers, trainees and the team
involved in the TEQS pilot. Each group contained 10-12 participants. A summary of the findings from the
FGDs is given below.
3.3.1. FGD 1: Master Trainers
For the master trainers the questions focused around the selection process and how in their view it was
different from the previous processes. Additionally, they were asked to reflect on their training delivery
experiences and the perceived impact of their training. Two FGDs were conducted with male and female
participants in Haripur and Peshawar.
33
The following diagrams represent a collated set of findings from the two FGDs under three different
categories: the selection process; training content and delivery; and application and impact of training.
Selection Process
SELECTION PROCESS
Unique & different process
Competitive process
resulting in better group of
trainers
Demonstration could be
longer & more comprehensive
Online application
process could be considered
Stronger, more relevant set of
criteria is a positive
development
Notification process needs
to be made more efficient
Process should be standard for
PITE
34
Training Content and Delivery
Application and Impact of Training
Training Content & Delivery
TOT was v useful & practical
TEQS well-explained
and v relevant
Qualities of a good
trainer v relevant
Good balance of
content and techniques
Lots of new techniques
and methods
Self-reflection was new
Excellent lead trainers
Application & Impact of
Training
Strong link between
training & teaching
methodology
Training provides good
model for better teaching
& learning
Better TOT resulted in
better participation in
CPD training
Lack of multi-media needs to
be addressed
35
3.3.2. FGD 2: Trainees/Participants of the CPD Training
Two FGDs were conducted with the trainees in Peshawar and Haripur for male and female participants.
The focus of these FGDs was to gather reflections on how they found the CPD training useful and why.
Additionally, they were asked to comment on their trainers and the overall training experience focusing
on what they felt was different and what techniques they found most beneficial for them.
The following diagrams represent a collated set of findings from the two FGDs under three different
categories: training content and delivery; reflections on the master trainers; and new learning.
Training Content and Delivery
Training Content & Delivery
Stimulating -brainstorming,
groups, discussion
Active & energetic sessions
Interesting new
methodology
Useful techniques
Good training created greater
confidence
36
Reflections on Master Trainers
New Learning
Reflections on Master
Trainers
Strongly positive
responses
Good command over knowledge & methodology
Calm, patient and
professional
Carefully selected trainers
Excellent example of
good master trainers and
better training
New Learning
Techniques which can be
applied -feedback,
questioning, discussion
Training, teaching
and facilitation
Observation forms, self-reflection
diaries etc.
Activities for use in the
classroom to enhance group
work and participation
37
3.3.3. FGD 3: TEQS Design Team (PITE/DCTE/RITE/DE&SE)
An FGD was organised for selected experts who had participated in the developed of the TEQS over more
than a year from late 2017. These experts had also been closely involved in the piloting process, So it was
important to get their reflections on the overall process and its application to their work as teacher
training designers, deliverers and managers. They were asked to comment on what had worked and what
had not and what needed further improvement.
A summary of their responses is given below in the diagram covering the following techniques: design
process; implementation and institutionalisation; and challenges going forward:
Design Process
Design Process
Participatory & consultative of all
relevant stakeholders
Strong ownership of the process and content
Need to select active & relevant
participants to develop TEQS
Merit-based selection of
design team of participants
needed
Content & methodology appropriate:
challenging and interesting
Need to communicate the TEQS to a wider audience to get more field level
perspectives
38
Implementation and Institutionalisation
Implementation
& Institutionalisation
TEQS represents a paradigm shift in TE
profiling
TEQS needs to become the official standard and help to create a career
path and a cadre of TEs
TEQS can produce quality trainers and
teachers if implemented
TEQS should become part of
performance management
system for trainers and teachers
TEQS needs to be regularly updated
and advanced
E&SED now has strong process for
selecting and developing a pool
of teacher educators
Policy shift in favour of TEQS would go a long
way to improving quality education in
KP
39
Challenges
In conclusion, the participants said that the results of the TEQS pilot would set the path for E&SED to
enforce its implementation. There need to be clear linkages between TEQS and JRF, SQMI and ESP as they
are all based on similar principles. To ensure sustainability the department should place specific focus on
implementation and institutionalisation of the TEQS and avoid the establishment of parallel reform
agendas without proper integration and coordination.
Challenges
Lack of professional specialists in
PITE and DCTE
Process for selection &
deployment of TEs is inefficient
Resources needed to
ensure application of standards at PITE/DCTE
How to maintain a pool of TEs and not
fall back into an ad hoc system
Winning the argument about the benefits of creating a TE
cadre
40
Chapter 4: Recommendations and Conclusions
The overall findings of the pilot study of the TEQS are encouraging.
The observational study reported in Chapter 3 shows generally satisfactory performance results in terms
of compliance of the selected trainers with the principles of the TEQS. In places there were examples of
exceptional performance and in others some examples of less than optimal performance, especially in
those areas that could be described as higher-order skills which necessarily take longer to acquire and
perfect.
If we look at the overall aggregated scores in all indicators then 40% of trainers scored well in line with
the standards at the desired level of effectiveness on almost all elements of the training (rated as a 3 on
the Likert scale). An additional 14% performed exceeded expectations, taking the combined score of
effective and exceptional trainers to 54%. This is a positive outcome of the process and provides
endorsement to the selection process as well as the TOT approach in terms of methodology and content.
These high quality trainers can be recommended for inclusion in a permanent pool of teacher educators
for PITE (Table 36).
Table 36
Of the remaining trainers, 14% performed poorly and did not manage to meet the standards in most
instances. These trainers are not recommended to be involved in future trainings as they did not indicate
any potential and performed weakly in most aspects of the training.
The remaining 33% of the trainers showed some potential but also demonstrated a need for further
capacity building to further refine their abilities and develop their skills so that they are in line with the
13.92%
32.61%39.84%
13.64%
0%
20%
40%
60%
80%
100%
1 2 3 4
Checklist for Session Observation of CPD training -Aggregated Score
Total
41
TEQS. These were the trainers who tried to match the requirements of the training but were not
consistent in their performance.
The aggregate score shows that the total number of potential trainers identified during the pilot with a
possibility to be included in the pool equals 86% (scores 4 + 3 + 2), with those scoring 2 requiring further
capacity development in the identified weaker areas.
When we look at the gender disaggregated scores from the TEQS pilot study we find no significant
differences in the performance levels of male and female trainers (Table 37). This is a positive
endorsement of the process for selecting trainers on merit rather than through some form of nomination
(which tends to favour the participation of men in the KP context). There is a slightly higher number of
men who scored lowest against all indicators and a slightly higher number of women who scored higher
against all indicators. The only stark difference was in time keeping where the majority of male trainers
were typically late to start their training.
Table 37
The results of the FGDs are consistent with these findings. The trainees were satisfied with the training
and commented favourably on the trainers’ performance. They were able to recount several examples of
learning, which they claimed would help them in their daily work as teachers or headteachers.
The trainers also endorsed the selection process, recognising the significance of the multi-step process
they had to go through to become part of the trainers’ group. They acknowledged the benefit they derived
from the training they received which in their view supported them to perform professionally, both in
terms of content knowledge, skills and professional competence.
16.58%
31.82%36.36%
15.24%11.14%
33.43%
43.45%
11.98%
0%
20%
40%
60%
80%
100%
1 2 3 4
Checklist for Session Observation of CPD training -Gender-wise score
Male Female
42
The results of the FGDs with PITE and DCTE experts show a strong attachment to and ownership of the
TEQS. All of them see the benefits that can be accrued through the implementations of the standards but
cited certain policy, resource and coordination issues that may challenge its implementation.
The principles and approach of the TEQS, however, seems to be endorsed and affirmed through its
application during the pilot study.
The TEQS were designed to identify and develop quality teacher educators who will be able to deliver
consistently good quality teacher training in KP and raise the standards for training and teaching at
primary and secondary levels.
4.1 Implications for Further Capacity Building
Despite the fact that there was an encouraging number of trainers who performed well according to the
TEQS indicators, there were still a number who performed below the expected requirements with some
inconsistencies in professional practice. Certain capacity gaps were found in all sections of the training
including preparedness, delivery, communication and time management. Future capacity building
sessions by PITE should focus more on these core areas and, in particular, the areas outlined below (many
of which represent higher-order skills which take more time to fully understand and put into practice):
• Developing a learner-centred approach
• Demonstrating flexibility in the training room according to the needs of the trainees
• The ability to involve participants in meaningful interactions through questions to and by the
participants and motivating the less active participants
• Adjustment/improvisation of training techniques to match the participants’ abilities
• Assessment of learning during training
• Effectively starting and concluding sessions
• Generating energy and positivity
• Conflict management during training, and
• Time management.
43
4.2 Recommendations
The development of the TEQS allowed the experts in the Department, DCTE, and PITE to think innovatively
and design strategies and revise processes to facilitate both the articulation of a unique set of quality
standards for KP and the subsequent the implementation of the standards. The results of the pilot study
show that all these modifications and additions that were recommended and implemented yielded
positive results.
The challenge now is to take these developments to scale and make it part of the regular functioning of
PITE processes. This will not be an easy task, as all these process variations require additional resources
both in terms of policy, strategy, resourcing and the management of human capital.
PITE runs multiple trainings in a year throughout the province, including a massive teacher induction
programme. One such programme, the CPD training, is running in 8 districts of KP with almost 2000+
trainers involved in conducting the quarterly Professional Development (PD) day, and around 400 trainers
to train the head teachers. The KP Government has now approved the expansion of the CPD model to an
additional 8 districts of the province, requiring an additional 2000+ trainers for teachers and 400 or so
trainers for head teachers.
These numbers are challenging in terms of the level of effort and resources required to allow for the
implementation of the TEQS. However, the clarity and consistency offered by the TEQS provides in terms
of the levels of quality needed to design and deliver good quality training at scale needs to be understood
by E&SED, DCTE and PITE as an opportunity to continue building on. The current system which is best
described as ad hoc and inconsistent simple will not provide the results that the Department in seeking in
terms of better teaching and learning across the Province.
The TEQS pilot study has identified a number of recommendations which the E&SED, DCTE and PITE need
to consider going forward to strengthen the design and delivery of teacher education in KP. These are
organised in terms of recommendations for the selection of teacher educators; the training of trainers to
prepare them to deliver training effectively; the actual delivery of training; and the monitoring of training
so that PITE and DCTE can be confident that quality standards are being achieved consistently.
4.2.1 Training Stage 1: Selection of Teacher Educators
The pilot introduced a three-step selection process as opposed to the single step process practised
currently. Each step was designed to refine the selection to ensure identification of the best candidates.
These steps included:
• Orientation of the DEOs to the standards and the selection requirement
• Shortlisting of CVs against pre-determined criteria, and
• Practical demonstration sessions of training ability scored through an observation tool.
44
The pilot study targeted a small number of trainees. The challenge going forward will be to upscale the
approach so that it can cope with the thousands of trainers routinely deployed by PITE for its various
training programmes. Specific recommendations are outlined below.
Recommendations for the Selection Stage of Teacher Educators:
Step1
Orientation of DEOs
• Orientation of DEOs should be continued as a key step in trainers’
selection
• For future requirements, for example CPD in 8 new districts, all the
relevant DEOs should be invited to PITE for a half-day session on the
principles of the TEQS and its implications for the selection of trainers
and other elements of the training.
• This model should be applied to all training and become part of PITE’s
SOPs.
• An additional benefit of this step would be the continuous
widespread dissemination and understanding of the TEQS and
ownership of the district management in the quality aspects of
teacher professional development.
• The DEOs should not just send nominations only but merit-based CVs
of the recommended candidates.
• The initial filtering of the CVs should be done at the DEOs office in
line with the requirements highlighted in the orientation session. An
IT platform can be used to store the results of filtering and enable
rapid sharing of results.
• The E&SED should facilitate this process by ensuring direct access of
PITE to the districts.
Step 2
Shortlisting of CVs
This is another important step for filtering the nominations. This step
prevents the engagement of irrelevant and unsuitable applicants and helps
to ensure a good balance of qualifications, skills and experience.
It is recommended that:
• PITE continue with this process as regular practice after the
nominations and the districts forward the long-list of CVs.
• The tool for shortlisting developed for the pilot should be used for
this step linked to IT platforms to store and share the results.
• The CV format developed for the pilot should be made a regular part
of the selection process. Using an IT platform to share the templates
45
online for candidates to fill and submit will facilitate the process and
speed up the selection process.
Step 3
Practical
Demonstrations of
Training Ability
This was the final step in the TEQS pilot for final shortlisting of the trainers
based on their actual performance. Though an important step it is the most
challenging to take to scale. Observing demonstrations of 24 plus candidates
was possible given the time frame but for PITE to organise demonstration
sessions for more than 2500 candidates for CPD alone would require a
solution involving other groups working hand in hand with PITE.
• As demonstration gives the best insight into a candidate’s potential
this step should continue in some form.
• Options could include:
o Inclusion of demonstration sessions in the TOT (with room for
rejecting some candidates during the TOT)
o Training district staff (DEOs and ASDEOs and possibly RITEs) in
conducting demonstration sessions (after PITE finalises the
shortlist of CVs) to join hands with PITE to select the most
suitable candidates.
o Capacity building of all experts conducting demonstrations
should be ensured to minimise the risk of selecting poor trainers.
4.2.2 Training Stage 2: Training of Trainers
For the TEQS pilot 2 days were added to build the capacity and capability the trainers and introducing
them to the concept of the teacher educator quality standards.
Recommendations for the Training of Trainers:
Additional days of
training
• PITE should continue to arrange for additional days of training for all
the future candidates for the multiple training courses it manages
throughout the year. This is important to strengthen skills and
competencies in core skills, especially the more complex higher-
order ones.
Resourcing • The Department, DCTE and PITE should advocate for additional
resources to facilitate the roll-out of a strengthened TOT approach.
46
TOT Content • The training of trainers on standards and facilitation techniques
should be spread over 3 instead of 2 days. This is because the 2-day
approach could not capture all the areas that require proficiency in
trainers, which was evident through some of the weaker areas
identified.
• Include practice and demonstration sessions during the TOT to
compensate for any deficiencies in utilising demonstration during
selection and ensure filtering of candidates at this stage,
• PITE should build up a pool of highly qualified Lead Trainer Teacher
Educators from around the province to conduct the TOTs to
complement staff at PITE who will not be able to carry out all the
tasks themselves at scale.
Development of a
permanent pool of
Teacher Educator
trainers
• PITE can use the three-day TOT to run annual courses for potential
master trainers to keep strengthening its pool of trainers.
• An online database of all trainers in the pool needs to be established
with full records of their skills and the training courses they have
designed and/or delivered.
• The pool needs to be managed and quality-controlled by PITE so that
the TEQS principles are maintained and operationalised.
• The pool is, to some extent, an interim measure. Its effectiveness can
be evaluated over time to assess the feasibility and desirability of
establishing a teacher educator cadre in KP. This would greatly
support the continued development and strengthening of the
teacher educator function in KP.
4.2.3 Training Stage 3: Delivery and Monitoring of Training
In many ways this is the most significant stage where appropriate training skills deliver good training to
teachers across the province. Monitoring is a vital aspect of training as it provides the kind of management
information that gives reassurance that training courses are being delivered according to the agreed
quality standards. The monitoring process for the training designed for the pilot served its purpose in
identifying potential trainers either for direct inclusion in the pool or for further building up to fit that
purpose. A more comprehensive framework for monitoring of future training will need to be agreed and
put in place.
47
Recommendations for the Delivery and Monitoring of Training:
Observation Tools • PITE should continue to use this tool for monitoring all training
and develop a database of all trainers recommended for the pool
based on observational data of skills, competencies and
behaviours.
Monitoring Processes • A strengthened standardised approach to monitoring (and
evaluation) of all training courses should be instituted at PITE
• PITE should ensure that monitoring becomes an essential and
regular part of every training programme.
• Staff at PITE and all Lead Trainers should be trained in the use of
the monitoring tools and how to report on the data gathered.
• An online form to enter and collate results is already developed
and can be used as part of the monitoring approach.
• The monitoring framework needs to be aligned with the M&E
framework in operation at PITE and DCTE and aligned to wider
quality assurance mechanisms that have been recommended for
E&SED.
4.3 Conclusions
Delivering quality education to each school child is one of the core purposes of an education system.
Quality in education is, however, a multidimensional concept which means that reaching this goal may
not be straightforward. It requires multiple, parallel inputs from all stakeholders with a well-
coordinated approach which links policy and strategy to sound management of the systems to create
the conditions for effective and efficient implementation.
The TEQS is an essential component of the quality chain, which requires concerted efforts for its sustained
institutionalisation in the system. An encouraging factor is the evidence of strong ownership of the TEQS
in the Department, Directorate, DCTE and PITE, which can guide the efforts for its continued
implementation alongside other quality-focused initiatives. This is due, in no small part, to the highly
participatory and consultative way in which the TEQS was designed over many months with
representatives of DCTE, PITE, the RITEs, DE&SE and the Planning Cell of the E&SED. The TEQS pilot study
confirms the value and applicability of the standards and their potential in achieving the purpose they
were designed for – better quality training to facilitate better quality teaching and learning in the
classroom.
48
The system already provides opportunities for the professional development of teachers through in-
service provision and the pre-service induction programme (managed by DCTE, PITE and RITEs). The
teacher educator quality standards are designed to strengthen the quality assurance mechanisms within
these organisations and bring new energy and strategic coherence to existing practices. As such the TEQS
strengthen the accountability of teacher education institutions in delivering their services.
For this purpose, DCTE and PITE need to continue this work - extending their own efforts and providing
strategic management of teachers in-service training and also assessing the feasibility of extending the
approach to the pre-service induction programme. Clear roles and responsibilities of all stakeholders for
every stage of the training process should be written and agreed in detailed SOPs.
Additional resources should be allocated for the special purpose of developing the capacity and capability
of teacher educators and developing, as a matter of priority, a pool of teacher educator trainers to serve
as a resource that PITE can draw upon and manage strategically and operationally.
The Teacher Educator Quality Standards open up the discussion on systemic and procedural reforms
required to make their regular implementation as business as usual a reality in KP. This includes
strengthening existing practices as well as looking at long-term sustainable solutions. Setting up a
permanent cadre for teacher educators within the system with a clear chain of command and career
ladder is one such longer-term solution to streamline the professional development landscape for
teachers and teacher educators in the province. A detailed feasibility study analysis is required to chart a
way forward with policy, strategy, implementation and cost implications and a phased action plan that
could be implemented.
49
Annexures
50
Annex I: Pilot Training Centres
Peshawar
1 Circle Office Badaber
2 Circle Office ChughalPura
3 City Circle Office
4 GPS Nahqi No.1
5 GPS KhazarKhwani.No.4 Circle Urmar
6 Circle Office Hayatabad
Haripur
1 Circle Office Ghazi (Sub-Division Ghazi)
2 Circle Office Pharallah
3 Circle Office Khan Pur
4 Circle Office Haripur
5 Circle Office Sari Saleh
6 Circle Office Kot Najeeb Ullah
51
Annex II: TEQS Pilot Monitoring Team
Haripur
Ms. Safia, PITE PSh
Ms. Samia, DCTE Abt
Ms.Sumera, PITE Psh
Ms. Badia, PITE PSh
Ms. Iffat, RITE (F) Swat
Ms. Zuhra, DCTE
Peshawar
Gohar Ali Khan Director DCTE
Zulfiqar Khan, DCTE Abt
Tariq Ali, PITE Psh
Ziaud Din, PITE Psh
Munsif, PITE PSh
S.Shah Hassan, RITE Psh
52
Annex III: Observer Code of Conduct
Dos and Don’ts for Observers
As an Observer you should:
• Dress professionally / properly.
• Follow the timings of the session. Be punctual, arrive before the start of the day and on time for
every session and leave when the session ends.
• Prepare a file for observation that contains copies of sessions, monitoring forms, and accessories.
Be organised.
• Read thoroughly the content of the sessions and monitoring form so that you are clear about
everything.
• Meet facilitator before the start of the session and inform about your role for repertoire.
• Ask the facilitator where she / he would like you to sit. Ideally, it would be good to sit at the back
and side of the room so that you can observe both the facilitator and participants without
disturbing the class.
• Observe the sessions carefully and answer all the questions of the observation form. Start filling
out checklist as session starts.
• Be attentive throughout the session.
• Observe until the end.
• Before ending the observation day, recheck file and finalize things.
• Maintain a neutral facial expression throughout the session.
• Fill out both forms and answer the two questions at the end of the form.
As an Observer you should not:
• Go unprepared.
• Leave the session.
• Distract the facilitator during the session, or join into the session by asking the facilitator or
participants any questions.
• Use your mobile phone for telephone calls, messages and emails while the session is in progress.
• Impose your ideas on participants or give any comments.
• Be biased.
• Sit at a place to interrupt the classroom activities (participants and trainers)
• Leave any items on the checklist try to fill in remarks, if possible.
• Walk in classroom while group work in progress.
• Give feedback on session.
53
Annex IV: Criteria for Short-listing the CVs of Trainers
CV Checklist for selection of TEs
Name of applicant Gender M/F
District Contact details
S.
No Indicator
Level of demonstration
3 2 1
1 Acad: qualification
2 Prof: qualification
3 Teaching experience more than 5 years 2 – 5 years less than 2 years
4 Training experience as a Trainer 6 Trainings 4 Trainings 2 Trainings
5 Training experience as a trainee
6 IT skills Yes No N.A
Note: Advance Qualification = 3
Pre-requisite Qualification = 2
Below Pre-requisite qualification = 1
54
Annex V: CV Template - Haripur & Peshawar
1 Name:
2 Father’s Name:
3 Date of Birth:
4 Marital Status
5 Address:
6 Contact No.
7. Academic/Professional Qualifications
No. Detail Subject Institution
1 Masters
2 B.Ed
3 M. Ed
3 M.Phil/MS
4 PhD
8. Professional Experience
No. Designation Department/Institute From To
1
2
9. Technical Experience
a) As a Trainer
No. Title of training Venue No. of Days
55
b) As a Trainee
No. Detail of training Venue No. of Days
10. IT Skills
No. Skill Level of Expertise
Basic Int Good Master
1 MS Word
2 MS Excel
3 MS Power Point
11. Languages Indicate competence on a scale of 1 to 5 (1 – excellent; 5 – basic)
No. Languages Speaking Reading Writing
1 English
2 Urdu
3 Pashto
4 Other
56
Annex VI: Training Demonstration Observation Tool
Checklist for Session Observation Date
Center Name
Name of
applicant
Gender
District Contact details
Qualifying score: 50%
S.No. Sub-Areas NA 1 2 3 4 Remarks
1 Introduces himself/herself at the beginning
2 Explains the objectives at the beginning of the
session
3 Uses resources effectively, including
technology and AV aids
4 Demonstrates good command over subject
matter knowledge
5 follows the structure and instructions of the
training module
6 Responds effectively and confidently to the
questions raised by participants
7 Projects self-confidence and maintains
professional composure
8 Communicates in the required language with
proper pronunciation
9 Uses a clear and strong voice using pitch and
volume appropriately
10 Maintains eye contact with appropriate
expressions
11 Communicates ethically and without
conveying any bias through verbal, non-verbal
cues
12 Summarizes key points and clarifies where
necessary to ensure that all participants are
following
13 Remains focused on the topic to manage the
time effectively during the training sessions
14 Is able to generate enthusiasm and energy
throughout the lesson
Total:
G. Total:
Additional Remarks:
57
Annex VII: Observation Tool for Monitoring of Training
The Observation Checklist for teacher educators is derived from the TEQS and is designed to be filled out
by an observer to determine the quality of teacher training in terms of competences, skills and personal
and professional behaviours. It can also be used to provide ongoing feedback and coaching to individuals
who provide teacher training. Furthermore, it can be used as a guidance document when designing or
revising professional development opportunities. The tool outlines a set of indicators of minimum quality
(i.e. quality that is acceptable) that should be present in all professional teacher educators. Shortcomings
provide the content for future capacity building of the trainers to further align their skills with the TEQS
and, of course, the needs of PITE and DCTE.
The observation tool used for the TEQS Pilot helped in answering the following key questions:
• Does the TE have required knowledge and skills to deliver training(s) effectively?
• Does the TE have the potential to be part of the teacher educators pool?
• What are the weak areas which can then form part of the future CD initiatives for the TE?
As such, the tool is designed to communicate clear performance standards to observers and the observed
that are aligned to the policy and standards framework. The observation tool is organised into four main
topic areas:
Topic 1:
Preparedness
Indicators in this section are designed to assess whether the TE is well prepared
for the training as per the instructions, guidelines and discussions during the TOT.
Preparedness spans punctuality, readiness in terms of materials and training
schedules along with arrangement of materials to be used during training.
Topic 2: Delivery
This is the most important aspect of an effective training programme. Delivery
refers to all elements of imparting the training content including setting the
ground rules, subject matter knowledge, effective use of training techniques,
engaging the participants through asking and encouraging questions and keeping
all trainees involved. Effectively delivery also encompasses:
• appreciation of work done by the trainees to keep them motivated
• assessing participants’ learning by engaging with them, and
• recapping the key content areas, summarising and checking.
Topic 3:
Communication
At the heart of effective training experiences lie the communication skills of the
trainers. Communication ranges from body language to eye contact to pitch and
tone of voice and a confident demeanour. Delivery and communication are
closely linked, as a good communication style is at the heart of effective training
delivery.
Topic 4: Time
Management
Good time management by a trainer does not only mean that he or she was able
to cover all the stated content in the stipulated time but it also includes a good
adjustment of discussion, questions and clarifications without disrupting or
overly stretching the schedule. It requires expert skills from the trainers to
58
understand when to cut a discussion short and when and where to create space
for it. A trainer who barely manages to complete the narrative of the content in
the designated time without room for interaction is not necessarily a good time
manager.
Checklist for Session Observation of CPD training
Date of visit:
Centre Name
EMIS code
Name of trainer Gender M/F
District Contact details
Name and designation of Observer
Address of observer
Start Time:
End time:
Expected number of trainees Head count of trainees at the time
of observation:
Training date:
Topic Sub-Areas Yes No NA Remarks
Pre Training
1.
Preparedness
Reached venue well in time before the start of
the session
Ensures availability of teaching aids as required in
the module
Training schedule is displayed
During the training
2.
Delivery Introduces himself/herself
Describes participation and ground rules
Explains the session objectives
NA 1 2 3 4
Uses a variety of training techniques to address
individual learning needs to ensure maximum
participation
Uses resources effectively, incl technology and AV
aids, to enhance trainees’ learning
59
Demonstrates good command over subject
matter
Manages contributions from all trainees by
involvement in discussion & activities
Follows the structure, instructions and flow
(activities) of the training module
Stays focused on the stated objectives
Conveys clear instructions
Applies appropriate questioning techniques to
engage all participants
Motivates participants to ask questions and
responds effectively & confidently
Motivates less active participants by encouraging
& assigning specific tasks
Manages dominant participants who talk too
much without directly offending them
Appreciates contribution of participants verbally
and nonverbally
Applies different assessment techniques
Recaps important elements and responses &
summarises the session
Sets up conducive learning environment through
participation, encouragement, recognition,
teamwork and feedback
3.
Communication
Projects self-confidence and maintains
professional composure
Communicates in the required language with
proper pronunciation
Communicates in clear and strong voice using pitch
and volume appropriately
Maintains eye contact
Demonstrates a positive and non-threatening body
language and puts the participants at ease
Communicates without conveying any bias
through verbal, non-verbal cues
60
Manages conflict effectively through discussion
and mediation
Generates enthusiasm and energy throughout the
lesson
Respects the views of the trainees
Exhibits sensitivity to students' personal, cultural,
gender and ability differences, responds
appropriately in a non-threatening learning
environment
4.
Time
Management
Starts the training session on time
Remains focused on the topic to manage the time
effectively during the sessions
Avoids wastage of time by handling irrelevant
discussions professionally
Teaching aids and other material is are used
appropriately
Post-training
• How far do you think the performance of the TE aligns with the requirements of the TEQS?
• What (if any) are the areas for further improvement in line with the TEWS for the TE to qualify as
a candidate for the pool of trainers associated with PITE?
61
Annex VIII: Focus Group Discussion Templates x 3
Master Trainers
A. Selection
QUESTIONS RESPONSES
1. In what ways did you find your selection as a Master Trainer for this CPD training
different from previous selection procedures?
2. What do you think were the principles underlying the selection procedure?
3. How appropriate was the selection procedure for the purposes of identifying and
preparing new Master Trainers?
4. What suggestions do you have to improve the selection procedure in the future?
B. Teacher Training
QUESTIONS RESPONSES
Training Content and Delivery
1. How appropriate do you think the training content and methodology were in
meeting the requirements of developing good Master Trainers? Please give
examples.
2. In what ways did the training you received help you to develop the skills and
techniques that characterise a good Master Trainer?
3. How did you find this ToT different from previous ToTs you have participated in?
Please explain.
Application and Impact of Training
1. How did the TOT help you to deliver your CPD training? Please give examples of
ideas, methods etc. you found useful.
2. How did the TOT influence the way you conducted your CPD training?
3. What challenges did you face in implementing the skills you observed while you
were undergoing the TOT? Please give examples.
4. What did you notice about the ways in which the skills and methods you used
helped the trainees to engage with the CPD training?
62
Training Participants
A. Training content and delivery
QUESTIONS RESPONSES
1. In what ways did you find this CPD training useful? Please give some examples.
2. What was it about the methodology of the CPD training that made the experience interesting/useful? Why?
3. What training methods and/or skills did you notice the trainer using to facilitate the session?
4. How appropriate do you think the training content and delivery was in meeting your needs as a teacher? Why?
5. Thinking about the CPD training, what facilitation skills do you think you would be able to use in your own teaching situation? Why?
6. To what extent did the CPD training illustrate you attended provide a model of good quality training. Please give examples.
63
TEQS Design Team (PITE/DCTE/RITE/DE&SE)
A. Design Process
QUESTIONS RESPONSES
1. How did you find the process of developing the TEQS in 2018? Please give details.
2. What worked well and what did not work so well? Please explain.
3. To what extent was the mix of people involved in the design process appropriate
to the nature of the work? Consider experience, skills and roles and
responsibilities.
4. In what ways have your ideas about what makes a good quality teacher educator
been influenced and/or changed by the experience of developing the TEQS? Please
explain.
B. Implementation and Institutionalisation
QUESTIONS RESPONSES
1. In what ways can the implementation of TEQS improve in-service teacher training
to produce quality teachers? Please give suggestions.
2. In what ways can the TEQS help to develop a sustainable pool of good quality
Master Trainers? Please explain.
3. What are the main challenges in implementing TEQS in PITE/DCTE? Please give
examples.
4. How can the TEQS be institutionalised in the work of the PITE/DCTE so that this
system is maintained? Please give your suggestions.
64
Annex IX: Pilot Implementation: Monitoring and Evaluation Framework
The KP-ESPSP programme has been promoting the idea of developing teacher educator standards in
KP since 2017. After a detailed process of developing the draft standards with representatives of DCTE,
PITE and DE&SE, the activity is now ready for piloting in the field.
The pilot of the Teacher Educator Quality Standards (TEQS) is a key component of assessing the extent
to which the work done to develop the draft standards fits the purpose of PITE. The training being
used for the pilot is the continuous professional development (CPD) training in two districts: Haripur
and Peshawar.
The objectives of the monitoring and evaluation of the TEQS pilot are as follows:
• To assess the effectiveness of the TEQS approach and principles,
• To keep activities aligned to the results targeted by the TEQS, and
• To develop a feedback loop which can provide insight to stakeholders on the fitness for purpose of the TEQS approach to enhancing the effectiveness of the teacher educator function.
With these objectives in mind, the M&E approach will address the implementation arrangements as
well as investigating the short-term impact of the TEQS approach on the teacher training activities
contained in the CPD programme. Therefore, part of the Kirkpatrick model can be used as the basis
for analysing evidence on the effectiveness of the TEQS in this pilot.
About the Kirkpatrick Model
The Kirkpatrick Model is recognised as a standard to evaluate the effectiveness of training. It
recommends the measurement of training value across four levels. Each level is important as it draws
attribution linkages between the various levels. Each advancing level requires more effort in
assessment and measurement but also provides more information pertinent to training effectiveness.
As far as the piloting of the draft standards developed by the TEQS activity, it is important to collect
monitoring data on the first two levels of performance (reactions and learning), given the relatively
short timeframe of the CDS. The other two levels (transfer and results) refer to longer-term impact
and change, which fall outside the scope of the pilot. The details of the four levels of the Kirkpatrick
Model and their measurement in relation to the pilot are outlined below:
65
1. Level I – Reactions This level measures how participants respond to the training experience using the TEQS approach. It
covers an assessment of how they feel about operationalising the TEQS approach and content; the
extent to which is was regarded as valuable was; how satisfied they were with the training approach
and methodology and how they viewed the instructor’s role. In relation to the TEQS pilot reactions-
level data can be derived from both the trainer trainers (the teacher educators) as well as from the
teachers who are the participants in the CPD programme.
1.1 Areas of Reaction
The implementation of the Kirkpatrick Model incorporates the reaction and perception of teacher
educators and teachers on the following dimensions:
• Relevance to the participants’ needs
o Professional Development
o Individual development
• Training Experience
o Methodology
o Delivery
o Environment
• Anticipated Gain
o Awareness
o Learning
o Practice
1.2 Tools to capture Reactions
For the TEQS pilot the reaction could be captured through the implementation of pre- and post-
training questionnaires completed by the trainers leading the training as well as by the teacher
educators overseeing the management of the process. Other elements of reaction could be
incorporated, including the teachers’ reaction (while under training), participation, interest, verbal
feedback and body language and these will be captured by training observers. The data from
participant responses collected at this level, together with observational data, will need to be analysed
and used to produce brief Summary Analysis Reports for each workshop, creating a short-term
feedback loop for the stakeholders of the pilot (PITE, teacher educators, and the KP-ESPSP
programme).
2. Level II: Learning
This level measures what the teacher educators have learned in terms of knowledge and professional
skills as a result of the pilot. To apply this method, it is important that specific learning objectives are
determined for the entire pilot. Teacher educator assessments can be created from some of the
questions contained in the pre-and post- training questionnaires to allow a judgment to be made
about what they have learned from the process as well as their assessment of the fitness for purpose
of the TEQS approach. It may also be appropriate to focus on any learning assessments of the
participants of the CPD training.
2.1 Areas of Learning
66
The learning assessment is expected to objectively measure any short-term change in terms of these
objectives. A number of measures can be used in different ways depending on these objectives.
2.2 Tools to Capture Learning The assessment at this level would use a pre- and post- assessment method for training, as indicated
above. A number of the questions in the pre-and post-training questionnaires target the area of
learning so that a “before and after” benchmark for this level of assessment can be identified.
Depending on the number of participants involved it may be necessary to use a sample of the total
rather than surveying all participants (if the team decide to incorporate the learning of the teachers
involved in the CPD training in addition to the learning of the teacher educators involved in the pilot).
An important tool to capture learning will be observational formats. These should be designed around
existing formats used by PITE for its induction and CPD programmes. These will need to be amended
and adapted to reflect the concepts and principles of the TEQS. Standardisation will need to take place
once the formats are finalised to ensure that all observers interpret what they observe in the same
way. A variety of numerical values can be applied to certain question areas using a standard four-point
Likert scale. Consideration also needs to be given to where notes can be incorporated to add depth
and reflection to the observational activity.
Consideration can also be given to carrying out focus group discussions with different groups included
in the TEQS pilot: the lead teacher educators overseeing the whole process in the field; the teacher
educators (trainers) running the training, and, if time and resources allow, a limited number of teacher
participant groups. Key informant interviews can also be incorporated with a limited number of key
people in the process. This will add important verification detail to the final report on the pilot study.
Control District(s)
It will be important to consider the practicality and value of incorporating a control district(s) as part
of the pilot. This can add some objectivity to the analysis of the fitness for purpose of the TEQS
approach. Incorporating a control adds time and complexity to the pilot, so the value added of having
a control needs to be considered carefully. The key answer it needs to give is: ‘will the control provide
us with data which otherwise we would not be able to capture and understand?’
If a control is incorporated it will be important for the M&E tools to be closely aligned between the
control and the pilot – in other words, ensuring that a core set of key question areas forms a part of
all the M&E tools. This is important in order to make a valid comparison between the TEQS pilot and
the control. In addition, in terms of process a Lead Teacher Educator will need to oversee a control
district(s) and work in tandem with the other Lead Teacher Educators to ensure that the approach to
assess delivery and short-term impact is the same.
67
Result Level Activity When Frequency
Responsibility Risks & Mitigation
Reactions Pre- and Post-Training Questionnaires and Observational data to capture the response and learning of training participants
Pre-Training Questionnaire administered
Either before training or at the start of training
1 per training
tbc but a balance of KP-ESPSP and PITE responsibility
Depending on the number of participants for each course/workshop a sample of participants can be taken and analysed for indicative purposes. It will be the responsibility of TEQS pilot observers on the ground to ensure this is carried out efficiently using a standardised system for reporting to limit the risk of inconsistent analysis.
Post-Training Questionnaire administered
At the end of training
1 per training tbc but a balance of KP-ESPSP and PITE responsibility
Analyse Data from Pre-and Post-Training Questionnaires
A day after training
1 analysis per training
Tbc but probably overseen by the TEQS Consultant
Reactions & Learning
Observation studies, FDGs and Interviews
Observation Studies using standardised tools (these should be based on existing PITE tools used in the CPD and induction programmes)
To be summarised and shared as soon as possible after training event
1 per training
Lead teacher educators and TEQS Consultant to finalise the design and focus of the observation formats. Lead teacher educators to use the formats and record the detail.
Qualitative aspects of the training experience will be captured through the Summary Analysis Reports (including the analysis of Pre-and Post-Training Questionnaire responses and observation data) to identify areas of response and learning.
Focus group discussions with Lead Teacher Educators, teacher educators (trainers) and teacher participants
To be carried out after the pilot training has been carried out
1 per group per training
TEQS to design the list of questions for the FDG. KP-ESPSP team to carry out the FDGs (following standardisation to ensure consistency)
The FDG data will need to be analysed and incorporated into the Summary Analysis Reports. The analysis should be tabulated against each of the question areas and structured around themes so that higher-level issues can be easily identified.
Key informant interviews Completed after the pilot is completed
As required Key informant interviews will target Director PITE, other senior figures in PITE, TEQS Consultant and KP-ESPSP team
Interview data will be incorporated in the Final Report on the pilot. The interview will be structured around a list of standard questions for consistency and analysed thematically.
Summary Analysis Reports using summarised data from pre- and post- training questionnaires and observational data will be produced according to an agreed schedule for each district, and submitted to KP-ESPSP for review and inclusion in reporting on the pilot. The TEQS pilot report will be the mechanism for collating data from all of the pilot areas into one overall assessment of the fitness for purpose of the TEQS approach.
68
