Perceptions of Delivering Training Competency among Pakistani Practitioners

Naseem Saeed Sherwani, Ph.D. nassherwani@gmail.com

Edgar Yoder,Ph.D. epy@psu.eduWilliam J. Rothwell, wjr9@psu.eduThe Pennsylvania State University,

University Park, PA, USA

Abstract

The research paper investigated the Pakistani Workplace Learning and Performance (WLP) practitioners’ perceptions of the evolving importance of delivering training in the workplace. By focusing on themes and the contents of designing, delivering and improving training, this research illuminated gaps in current knowledge and need for future improvements with respect to occupational expertise.  Looking at the context of the growing formal workforce in Pakistan offers insights into the demographic and technological shifts occurring in the economies of the developing world. Forty WLP practitioners volunteered to respond to a paper-pencil based survey using convenience and snow-balling sampling approaches. This study employed inferential statistics to identify the differences in perceptions of practitioners regarding the current and future importance of selected delivering training competencies. The results of this study suggested that degree of engagement in knowledge and actions areas in delivering training would be sufficient for predicting the development of a skillful workforce. Very little empirical research had been done to connect delivering training in knowledge and actions areas. Despite possible sampling bias, this research bridged the information gap by examining the relationship between perceptions of Pakistani practitioners regarding the current and future importance of delivering training competency.

Keywords- Delivering Training, Workplace Learning, Performance, Technology, Knowledge, Actions, competencies

Paper type: Research paper

Introduction

Delivering training is one of the most pronounced areas of interest in Workplace Learning and Performance (Bernthal, Colteryahn, Davis, Naughton, Rothwell and Wellins, 2004). Delivering training has the overarching goals of upgrading and retaining the existing global workforce. Especially in developing countries problems such as outdated skills have made majority of the workforce more vulnerable to economic challenges (International Labour Organization, 2003). Improving methods for delivering training may equip individuals with the skills that are in demand so that people have greater access to job opportunities. Normally, in the field of workplace learning and

performance, delivering training has been considered a simple task of delivering instructions. Looking at the other disciplines of Adult Learning, Instructional Methods, and Performance Systems, though, gives delivering training a new dimension of complexity. These knowledge areas have largely been neglected by the practitioners in workplaces outside the economies of developed nations. In particular, Pakistani managers, trainers, and executives have indicated that the most important skills to gain relate to delivering training (Labor and Manpower Division, 1998; 2008).

Although India has made impressive inroads into the global economy, its neighbor to the northeast, Pakistan, has occupied a more tenuous position in the world. Pakistan is a highly populated country of 155.8 million and ranks 139 th out of 179 countries of the world on the Index of Human Resource Development (HDI) by the United Nations (UNDP, 2008). The Human Development Index looks beyond gross domestic product to a broader range of well-being. However, economic development is necessary to address the pressing political and social concerns that have tarnished Pakistan’s image on the world stage. In order to improve the standard of living in Pakistan, the workforce must be developed for the jobs that are in demand.

The American Society for Training and Development (ASTD) has developed a competency model (Bernthal, et al 2004) that conveys important knowledge areas for delivering training. According to ASTD, delivering training is not just delivering instructions, but also encompasses a wide variety of knowledge areas (e.g., adult learning theories, instructional design theory and methods, delivery options such as online learning, classroom training and print media, learning technologies and support systems). Further, delivering training requires several key action areas to communicate effectively, (e.g., preparation for training delivery, alignments of learning solutions with course objectives and learner needs, conveying objectives, and facilitating learning). The expertise of delivering training needs to be generalized to the contexts outside of the United States of America and the developed world.

Today, organizations are continuously facing changing business environments, increasing global competition, and rapid advances in technology. All of those factors impact Pakistan’s economy which requires building a flexible and highly skilled workforce (Long and Smith, 2004). Thus, organizations that provide training and development opportunities to their employees are better able to develop employees’ know-how into a strategic asset.

Training delivering is different than common teaching principles because it is geared towards adult learning environments (Noe, 2002), instructional designs and various instructional methods (Cantor, 2002). Adult learning theories play a key role in delivering training instructions. Cercone (2008) cites in her research a leading scholar, Malcome Knowles (1970), who suggested that activities conducted prior to and during the first session could "greatly affect it" (p. 270). These activities include promotional materials and announcements, activities designed to assess learner needs prior to the event, physical arrangements, and the opening session, including greeting, learning activity overview, introductions, and treatment by the instructor. More recently, adult

educators are recognizing that these factors in the learning environment relate to psychological, social and cultural conditions and exert a powerful influence on the growth and development of learners (Hiemstra, 1991).

Due to globally expanding, knowledge-driven economies, diverse skills, such as the ability to train new or existing employees (Sloman, 2007) have become more crucial. A need exists to think deeply about the current training practices and training career anchors. A 2004 research study interviewed 540 managers, and a questionnaire measured the relative importance of individual career anchors. The study found that younger managers were more oriented towards their own skills and what they could contribute, while older managers were more inclined to be aware of the limitations of their roles in the organization. The study stressed that the difference between the younger and older managers’ perceptions of their careers is something which the trainer needs to consider. Also significant to the trainer was how similar male and female managers were in their perceptions of their career anchors (Kniveton, 2004).

Technology is ubiquitous in the field of workplace learning – in the context of delivering training and learning it cannot be ignored. Learners are exposed to the superior quality products of advance technologies such as ipods, web-based applications, and social networking tools in other fields around them. Therefore, current training delivery practices demands relevance with today's multidimensional digital age. Although scholars have investigated training studies, mostly in the health, business and manufacturing sectors, they do not address delivering training as a subset of workplace initiatives. Current research will highlight a specific area of expertise for WLP practitioners as specified by the ASTD.  This approach will bridge the knowledge gap of the importance of practitioners’ perceptions about various aspects of delivering training. The background research will help to determine that the theoretical framework of delivering training developed for the USA can also be applied to a developing economy.

Like other countries around the world, Pakistan is, on the one hand, experiencing extremely high unemployment, elevated costs of living, and high debt. On the other hand, as a developing country, Pakistan faces additional challenges. Whereas- countries such those in Europe and the US are experiencing an influx of immigrants seeking job opportunities (however scarce), in Pakistan_ the reverse is true. The mounting emigration of highly skilled individuals is creating skill shortages. Lack of sophisticated workplace training is contributing to slow economic development of the country inhibits the corporate decision makers in utilization of resources. This phenomenon indicates the importance of delivering training to overcome the skill shortages and to enhance the retention of workers. Insufficient research has focused on delivering training in Pakistan and other developing countries. The current research filled the gap in understanding the perceptions of Pakistani practitioners in terms of their current and predicted future preparedness for performing their jobs.

Literature Review

Delivering training can occur in several ways. Three of the most influential and widely applied forms (Wyrick, 2001) are: Instructor-led training, technology assisted training, and blended learning. Each has benefits and drawbacks. The first and most traditional form of training has the benefit of not requiring much technology but it has a hidden cost of lost productive man-hours. On the contrary, technology-assisted training helps to preserve employee productivity but requires Information Technology (IT) and technical know-how. The blended learning approach “utilizes e- learning and instructor-led training to address the diverse learning needs of all its trainees on a 24/7 basis” (Kapp and McKeague, 2002, p. 10). The following sections examine several studies in order to evaluate the attributes of several methods of delivering training. Ultimately, the examination discusses some of their shared characteristics in order to illuminate the complexity of delivering training, particularly in the context of the Pakistani workplace.

Effective training delivery leads to employee job satisfaction, motivation and commitment. A research study examined the responses of 134 employees and lower managers of five large Greek organizations after they completed a training program. The questions asked contained information about employee attitudes towards the training received, as well as their attitudes towards their employers. Results indicated a significant correlation between the employees’ perceived training effectiveness and their commitment, job satisfaction, and motivation (Sahinidis and Bouris, 2008). However, the study examined employee feelings, and it did not take into account their demographic characteristics, which may be important. Findings of this study for managers and especially for professionals are quite significant, given their roles in funding, designing and delivering training interventions. Apparently, training appear to be important to offering training programs to employees, but also the training program content must be perceived as effective and of value to those participating in it. Moreover, the study examined employee attitudes, which appear to be related to a greater or a lesser extent to organizational performance outcomes including, productivity, turnover, and absenteeism.

Organizations undertake learning and development interventions to improve employees’ performance. Delivering training is one of the most important interventions. The demand for professionals in learning and development, especially from smaller financial services, has increased due to downward twisting in the job market. Recruitment specialists want to recruit dedicated training professionals to develop their managers. Companies seek learning and development professionals who offer experience in delivering training in areas such as presentation skills, people management and recruitment and selection. Training professionals are also increasingly being expected to show added value and a strong business understanding in an organization (Bentley, 2006).

The delivery of training has a direct impact on the quality of learning that the employee retains. Suzy Siddons book draws attention of all kinds of trainers

So it is every trainer's responsibility is to provide appropriate training, to the right audience, with the right tools, and in an engaging way. Drawing on a wealth of

personal experience, Suzy Siddons provides trainers with practical advice on: - answering the fundamental questions - who, why and what am I training? - gathering vital pre-course information sets achievable objectives, fulfill expectations and avoid logistical glitches - establishing rapport, understanding group dynamics and opening a course successfully - accommodating learning preferences using a range of methods and ensuring your pace, presentation and feedback encourage a learning climate - handling training room crises and difficult customers (Siddons, 2003).

Moreover, delivering training is a multitasking phenomenon and trainers need to be fully equipped with the knowledge areas and actions needed to perform those skills 

Knowing more about additional skills like Neuro-linguistic programming (NLP) helps individuals and organizations, whether developing or delivering training and learning solutions. NLP focuses on helping clients to overcome their own self-perceived or subjective problems. It seeks to do this while respecting their own capabilities and wisdom to choose additional goals for the intervention as they learn more about their problems, and to modify and specify those goals further as a result of extended interaction with a therapist However, NLP is therapeutic, and its patterns have also been adapted for use outside of psychotherapy including business communication, management training, sales, sports and interpersonal influences As a personal effectiveness tool, learning to change or adapt limiting beliefs helps managing time more effectively. However, NLP is useful when it becomes part of everyday life and work -- and not just a toolkit to delve into imparting instructions (Yemm, 2006).

In addition to the design and delivery of training instructions itself, the implementation of training can also maximize the efficiency and productivity of employees. Partnerships among organizations lead to better training delivery. For example, Meridian Community College and Rush Health Systems are partners in delivering training focused on Rush's mission statement of hospital-wide commitment to excellence in service management. Rush and MCC have delivered customized classes in areas such as medical billing, leadership management, computer training, admissions administration, and system-wide customer service training (Willis, 2007).

Assessment of training approaches strengthens the program of developing a skilled workforce. For example, the US government successfully used the Assess-Train-Assess (ATA) approach for delivering training on the National Employer Training Programme (NETP), which planned to reduce the number of unskilled workers by offering free, flexible training. The effort could have been undermined by widespread use of unsuitable delivery methods. NETP aimed to cut the number of unskilled workers by 2.4 million by 2010. An earlier IES study established that only 15% of pilot projects for the NETP used ATA (Sheppard, 2005).

The quality in the delivery of programs by maintaining high standards is essential by following best practices. For example, Tanzania recognizes quality assured tertiary qualifications for training at both national and international levels. To ensure registration

processes and institutional accreditation for delivering training programmes, the National Council for Technical Education offers awards at appropriate levels. Tanzania National Council for Technical Education is one of the first regulatory bodies in Africa to introduce academic quality standards in tertiary technical institutions. The experience of setting standards and accreditation for delivering training in Tanzania is an example for other countries interested in similar initiatives for professional development. Accreditation and standards instill best practices in education and training to achieve expected results over a period of time. Finally, all key stakeholders make a joint effort to ensure quality in (Manyaga, 2008).

New developments in employee training with the latest technologies are beneficial to the learning organization. For example, by aligning training strategies with corporate goals, continuous learning, training of manufacturer-user, and designing and delivering training are more cost effective.  The American Transtech aligns its training with corporate goals.  The company trains its managers to hire their own personnel and encourages teamwork participation in its 2-day orientation where company policies and procedures are outlined. S. B. Thomas Inc. adopted 2 types of training in its continuous learning plan - social and technical task training. Its commitment to in-house training and motivation of all employees has proven effective. Goodyear Tire and Rubber Co. initiated a manufacturer-user program with its suppliers. It requires its suppliers to enter into a training agreement when awarding an equipment bid. C&P Telephone Co. implemented accelerated learning and found its costs of one course reduced by 42% (Derouen and Kleiner, 1994).

Delivering training works well to improve knowledge and skills in an environment that is conducive for learning such as social networking. For example, the Staff Development Unit of the University of Birmingham within Birmingham Heartlands Hospital delivered a training course on effective teaching skills. Initially, the course structure allowed many doctors as possible who had an interest in medical teaching could attend and become part of a network sharing. An initial introductory half-day was followed by 13 monthly one-hour lunchtime sessions. Fifty-six doctors attended at least one session and formed a mailing list to receive relevant documents. Of those returning feedback forms, all found the sessions enjoyable and felt they had gained knowledge. The majority felt they had gained in skills and many commented on the value of being able to discuss medical teaching issues with colleagues and the tutors. The network of colleagues continues to meet and is actively involved with developments in the universities undergraduate medical curriculum. This way of delivering training in medical teaching is practical, successful, and contributes to improving a hospital's learning environment (Rawer, Morton, McCulloch, Heyes, and Ryan, 1997).

Managing the learning environment plays a crucial role in delivering training by creating outreach learners. For example, John Matchett Ltd. conducts short training sessions in London sushi bars to change the way executives work and to boost their potential. Sushi-bar learning, from John Matchett Ltd, targets people who do not have time to attend traditional training courses, yet have to develop in order to achieve their goals. Similar sessions are a way of delivering training to people who would not

normally receive training. Short learning sessions are often run in the evening, and sometimes they are held during weekends to avoid interrupting the busy patterns of the work day (Anonymous, 2003).

Today, knowledge about learning management systems plays a supportive role for large audiences at diverse locations. For example, Marshall, and Swift/Boeckh, a US-based global player in the insurance world, is using a new learning-management system (LMS) to bring training to insurance agents, underwriters, claims adjusters and contractors across the US. Using Pathlore's LMS, the company delivers training via the Internet to its stakeholders in the insurance industry (Anonymous, 2003).

One alternative is electronic performance support systems (EPSS). Nguyen and Klein (2008) examined the effect of EPSS and training on user performance, time-on-task, and time-in-training. They found that participants receiving only EPSS and those receiving training and EPSS performed significantly better on a tax preparation procedure than participants who received only training. Training-only users also spent significantly more time completing the procedural task than their counterparts in other treatment groups, leading to a negative correlation between time-on-task and performance. The findings indicate that design, development and delivery of training and performance support have important implications for performance improvement of employees.

Interestingly, creating a successful learning environment is crucial for motivating learners, especially in an on-line situation. A research study focusing on the experiences of adult learners in a self-directed e-learning environment indicated the challenges they faced during their learning process. The twelve adult learners took self-directed e-learning courses in either academic (e.g., universities) or workplace settings. Results showed that learners found courses with a low degree of interactivity and lacking in application and integration of content to be motivationally challenging. In contrast, courses that provided learners with authentic and interactive learning activities, such as animations and simulations, a positive learning climate, and control over the pace and sequence of instruction proved to be motivating for the learner. Therefore, delivering training should create motivational design for self-directed e-learners (Kim, 2009).

Estimates suggest that over $51 billion is spent on formal training each year in the US (Nguyen, 2007). However, delivering training is not only costly but also requires a significant time away from actual responsibilities. Therefore, organizations have reduced their costs by adopting alternative training strategies. Kay Baldwin-Evans (2004) explained the factors responsible for a shift from classroom training to e-learning in restrictive economic climates, such necessities as reduced staffing means less time and resources available for employees to attend workplace classroom training. Tighter training budgets are expected to train more employees with fewer resources.

In this situation, e-learning proves to be an accepted method of delivering training to increase skills and knowledge of diverse and geographically dispersed

employees. E-learning delivers a significant return on investment through effectiveness and monetary terms. For the most part, organizations have seamlessly integrated e-learning into their training strategies along with all the other methods of delivering training. Even organizations that waited until e-learning was proven effective are now enjoying the many benefits that e-learning brings. However, SkillSoft conducted a qualitative investigation into the attitudes and views of the users by interviewing over 200 employees, across a range of organizations, in over 14 countries. Findings indicated that 93 percent of respondents enjoyed e-learning, 87 percent said that they put into practice skills and knowledge they gained, and almost 100 percent said that they would recommend e-learning to a friend or colleague. The large number of employees are learning in this way proves that e-learning as a delivering training method is here – a permanent fixture (Baldwin-Evans, 2004). Research, undertaken by Taylor Nelson Sofres, a leading market research company on behalf of SkillSoft International, indicated that companies embrace e-learning as a method of delivering training in critical management, business and professional skills (Anonymous, 2001). So e-learning as a method of delivering training has also gained significance for the organizations to seek learning and performance outcomes.

At the beginning of 2000, e-learning seemed poised to displace traditional training programs. Even though e-learning - delivering training over the Internet or corporate intranet - offered such great promise; however, many banks have not integrated e-learning into their training strategies due to the difficulties with the technology sector. Resultantly, banks tended to evaluate technologies and vendors more carefully than ever before in order to avoid the challenges associated with the applications and tools of e-learning. In addition, broader economic downturn during the same period has caused banks to scrutinize all non-interest expense items more vigilantly, especially those requiring an up-front investment, such as e-learning (Carlivati, 2002).

Many companies view their training budgets strictly as a cost center which is susceptible to cuts when the business needs to reign in expenses. Eugene Deeny’s research (2003) explores the benefits of training investments with a particular focus upon measuring the value of delivering training via an enterprise e-learning initiative. In addition to the many hard/measurable returns, the study explores the greater "benefit halo" of an e-learning initiative and offers a number of formulas for calculating the ROI and justifying the cost to senior managers. The study concludes with a case study detailing the experiences of Rockwell Automation when the company implemented an enterprise e-learning initiative to deliver much of its training to both employees and customers (Deeny, 2003).

Currently, companies increasing demand for "just-in-time”, "just-for-me” training has exalted to meet the need for cost-effective and performance-focused competitiveness. In order to meet changing company objectives, technological communication is more supportive for delivering training. In addition, harnessing in-house e-mail facilities creates better individual responsibility and empowerment for learning, particularly in cultures of distance learning. Therefore, e-mail training acts as a

catalyst for successful change-making by providing training with a minimum of delay and maximum of personal tailoring, while also helping to transcend the cultural divides between East and West (Gilleard, 1996).

Advances in computer technology facilitate innovative methods for delivering training in organizations. For example, the internet enables the delivery of computer-based training across time and distance. This medium, known as Web-based distance learning (WBDL), provides opportunities to develop human resources to support creating a competitive advantage for an organization. Principles of WBDL design measure the effectiveness of training delivered and build the relationship between WBDL and human resource development planning in organizations (Long and Smith, 2004).

Delivering training is performed in a combination of different methodologies such as web-based, classroom instructions, and on-the job training. Blended training methodologies have become the norm in large enterprises as a method of delivering training to large, diverse employees as well as have made the job description of instructors more complicated. In an interview, Keith Phillips, learning and development consultant talked about his training life in Europe, the Middle East and Africa. He shared the nature of his job tasks involved in delivering training. He works in a true blended learning environment so he does everything in a trainer's job description: training needs analyses, workshop development, creating e-learning courses, delivering training, and coaching and consulting (Anonymous, Personal communication, March 2006).

Blended training delivery methodologies empowers individual to achieve understanding of a given topic, become self-sufficient, improve job performance, and drive results that support business objectives. Knowledge of various training methodologies expands the traditional role of training by providing a robust set of tools that allow employees to obtain the information and instruction they independently and uniquely need, all within the daily flow of work. A blend of traditional and synchronous and asynchronous tools have given a new dimension to delivering training (Baldwin-Evans, 2004). Again, learners can personalize their training interests and needs, as well as how content is delivered. Personalization of learning leads to customize the relevant training contents for a better learning retention (Hartley and West, 2007).

Key knowledge areas, like adult learning, instructional design, and technology, from around the globe, have affected the work environment in Pakistan. Generally, technology is not embedded in the training delivery process to capitalize the role of learning and performance at workplace. Trainers must upgrade new knowledge areas comprising of changing skills and behaviors (Bartell, 2001). In a new knowledge- based economy, trainers’ roles become more challenging when developing more practices that ensure knowledge transfer and sharing between organization members. The literature suggests that diffusion of technology does not minimize trainer’s roles rather it enlarges them (Bartell, 2001). 

[H]e/she becomes a knowledge resource, facilitator, liaison, conduit consultant and architect for the learning experience. In other words, a [SIC] trainer’s expanded role is to maximize total learning in the organization (p. 358).

Research indicates factors responsible for the slow learning process are incompetence of instructors, traditional curriculum, low exposure to computers, and lack of facilities at the workplace (Haider, 1998).  In the 1990’s, the Netherlands Library Development Project in Pakistan funded the technology related courses for the librarians in cataloguing, classification and the use of computers (Mahmood, 1997).

Currently several issues are inhibiting Pakistani practitioners competing with the global market.  First, the workplace learning and performance disciplines are not taught as a part of degree programs at Masters, M. Phil or PhD. levels at higher academic institutions. A very few courses are taught as a part of Masters of Business Administration programs. Second, technology is not embedded in delivering instruction.  Due to the advent of technology, textbooks are quickly outdated and the technology that is on the cutting edge for practitioners and learners using the internet need to be updated continually. Third, designing relevant curricula or content development is an important element in delivering training (Siddons, 2003). Government sources reported “Curricula tool [is], mostly non-relevant in the present day requirements” (Economic Advisor’s Wing, 2000-01, p. 149).

In addition, official sources indicated that the education and training system of Pakistan encompasses low quality education, lack of qualified trainers, and lack of proper physical infrastructure. A report further stated that “Teachers lack training, dedication, motivation, and interest in their professions” (Economic Advisor’s Wing, 2000-01, p. 149).

The purpose of the current study was to discover differences of knowledge area and actions needed for training delivery among the practitioners in Pakistan. Forty participants were selected from Training and Development and related fields using convenience and snowballing sampling approaches.  The data collected was analyzed using inferential statistics to establish differences between current and future perceived effectiveness required to be successful on their jobs as compared to the standard set by the ASTD.

This research identifies the Pakistani Human Resource Development (HRD), Organization Development, Career Development, Training & Development (T and D), and Human Resource Management (HRM) practitioners’ perceptions of the current levels of importance and the future levels of importance for WLP competencies grouped within the delivering training category.

What do Pakistani practitioners believe are the most important competencies in delivering training in the present and near future for their work performance? This study points a baseline for further studies of WLP competencies in Pakistan. Furthermore, this research will add to current knowledge of delivering training particularly in the context of

a developing economy, and “enable(s) practitioners to effect positive, progressive and enduring change in organizations” (Bernthal et al., 2004). This research justifies the continued investigation of the importance of delivering training competency for Pakistani practitioners. 

Methodology

Research Questions

This research seeks to answer the following research questions:

RQ1. What was the demographic profile of the respondents?

RQ2. To what extent do the perceptions of Pakistani practitioners differ on the current and future importance regarding delivery of training (1) knowledge areas and (2) action items?

Participants

The target population for this study consisted of Pakistani Workplace Learning and Performance (WLP) practitioners who were over 18 years old, had post-secondary degrees but were not currently students. They each worked in one of the following Workplace Learning & Performance (WLP) disciplines: Training and Development, Organization Development, Career Development, Human Resource Management, or Human Resource Development, and related areas such as workers’ compensation, occupational health and safety. After recruiting the initial subjects to participate in the research study, the researcher asked for assistance from the subjects to help identify people with a similar Workplace Learning and Performance discipline.

Instrument

The survey instrument used to measure practitioner’s importance in delivering training has used in the ASTD 2004 Competency Model for Workplace Learning and Performance questionnaire developed by Bernthal, et al. (2004). That questionnaire was originally designed for WLP professionals in the United States. The instrument used a 5-point Likert-type response scale (1=Not important to 5=Essential) to evaluate self-reported technical competence called delivering training, which was defined by Bernthal, et al. (2004).

Delivering learning solutions (for example, courses, guided experience) in a manner that both engage the learner and produces desired outcomes; managing and responding to learner needs; ensuring that the learning solution is made available or delivered in a timely and effective manner (p. 69).

Theoretical Framework

The Delivering Training competencies shown in The ASTD’s Delivering Training survey are based on the International Board of Standards for Training, Performance and Instructions (IBSTPI) standards consists of two major sections: (a) Key knowledge areas and (b) Key actions or behaviors.

(a) Key knowledge areas include 14 items (see Table 2) (b) Key actions include 12 main items (see Table 3) had the following sub-items

Prepares for training delivery (three items) Aligns learning solutions with course objectives and learner needs (three

items) Conveys objectives (two items) Delivers various learning methodologies (two items) Facilitates learning (seven items) Encourages participation & builds learner motivation (six items) Establishes credibility as instructor (three items) Manages the learning environment (six items) Delivers constructive feedback (five items) Creates a positive learning climate (four items) Ensures learning outcomes (four items) Evaluates solutions (two items)

Competencies are defined per the 2004 ASTD competency study “Competencies are clusters of skills, knowledge, abilities and behaviors required for job success” (p. 51). This definition will be used to assess the self-perceived importance on levels of competency (Ogbeide, 2006; Drummond, 2005).

Data Collection

Dillman’s (2007) and Borg and Gall’s (1996) survey research methodologies were guidelines for the data collection process. The original questionnaire and the mailing list were sent, via email, to a colleague in Pakistan for administration of the survey. He administered these questionnaires by first class Pakistan Postal Services with a stamped, self-addressed return envelope. The questionnaires were sent to the participants with requests to return them, via an included self-addressed envelope within two weeks after receiving the survey packet.

Telephone calls to request participation in this study, as well as to check on the contact information were made to each prospective respondent before sending the questionnaire. In total 120 respondents were contacted by phone, mail and e-mails. During these correspondences, the purpose of the study, requests for cooperation, and identification of the researcher were shared with the prospective respondents. A questionnaire was sent only to those who agreed to participate in this study.

 In this study the random sampling technique was infeasible due to lack of financial support and time constraints, which become problematic due to the study’s focus on Pakistan and its developing economy (For similar justification see Norman and Streiner, 2008). Therefore, the criterion sampling technique (Patton, 2002) was applied in order to identify qualified, potential participants.  These participant-practitioners were first contacted by Skype phone to acquire their willingness to participate. They then received a questionnaire delivered by a colleague currently living in Pakistan. Furthermore, criterion sampling and snowballing, or chain sampling, were employed. The snowballing technique requested willing participants to provide additional contacts who would also become potential participants (Patton, 2002). The initial telephone contacts included a request for names and telephone numbers of other practitioners who met the criteria. These networked, potential participants were contacted by phone and surveys were mailed to them by a research colleague in Pakistan.

Research Design

The research design of this non-experimental, quantitative study is descriptive and correlational in nature (Black, 1999). This study seeks to examine the Pakistani practitioners’ perceptions regarding the current and future importance of competencies with regard to delivering training. Correlational research investigates the relationships between two or more variables (Kerlinger and Lee, 2000; Graziano and Raulin, 2000). However, this type of research typically does not establish causal relationships (Cohen, Cohen, West and Aiken, 2003). Using this methodology brings statistical reliability and validity to the practitioner’s assertions of the importance of knowledge areas and skills pertaining to delivering training. In other words, this correlational analysis seeks to examine whether or not a majority of the practitioners surveyed believe that delivering training occupies a fundamental place in building the future of Pakistan’s workforce development. Accordingly, this research does not need to establish a causal kind of relationship between the current importance and future importance of such competencies. Rather, comparing perceptions of the current with those of the future shows the statistical probability that these skills are perceived essential in the next five years. Examining perceptions of the current and future importance of delivering training recognizes the importance of knowledge areas and actions that need to be taken to improve training delivery.

Justification for Inferential statistics

Nonparametric test for two related samples was used to test for differences between paired scores because of limitations to make the assumptions required by the paired-samples t test. Procedures were used for testing ordinal variables. The Wilcoxon signed-ranks method test was used to compare paired medians from the same sample.

Data Analysis

The data were analyzed using SPSS 18.0 for Windows. Descriptive statistics and the Wilcoxon matched pairs signed rank test were used in the analysis (Huck, 2008, pp. 488-490).

Results

Profile of the Respondents

Participants were predominantly male (62.5%) and most participants held a masters degree (69.2%). At the time of the survey, 12 (30.0 percent) had more than 15 years of Learning and Performance (L& P) experience, 11 (27.5 percent) had 3-5 years of L& P experience, six (15.0 percent) had 11-15 years of L& P experience, five (12.5 percent) had less than 1 year of L& P experience, four (10.0 percent) had 1-2 years of L& P experience, and two (5.0 percent) had 6-10 years of L& P experience. The average age was 36.6 years (SD=10.7 years). According to respondents’ job titles, eight (20.0 percent) identified themselves as supervisors, seven (17.5 percent) were managers, six (15.0 percent) were executives, five (12.5 percent) were university professors, college instructors, or entry level professionals, while three (7.5 percent) identified themselves as directors, private consultants and team leaders.

The type of industries in which the participants worked included business schools and company management training, broadcasting and communication, finance and insurance, health care and social services, information and other services, management consulting services (including HR consulting), education, publication (including government), real estate and rental and leasing, software publishing and retail trade. Information in Table 1 describes the participants in greater detail.

[Insert Table 1 here]

Differences in Perceived Current and Future Importance of Knowledge Areas

Respondents’ perceptions regarding current and future importance of knowledge areas are summarized in Table 2. Generally across the 14 specific knowledge areas respondents perceive a current moderate level of importance. The median value for the 14 specific knowledge areas ranged from a low median value of 2.50 (instructional design theory and methods) to a high median of 3.00 for the remaining knowledge areas.

Future perceived level of importance for the 14 specific knowledge areas reflected an increase in each of the median values when compared to median values for current perceived level of importance. This increase was statistically significant (p

≤ .05) for all 14 knowledge areas as reflected in the alpha level of the Wilcoxon matched pairs signed rank test.

Respondents perceive a high level of importance for 13 of the 14 knowledge areas as reflected in the median values consistently being about 4. The one exception is for the cultural differences in learning styles knowledge area where the median is 3.5. The quartile information for the future perceived importance of the cultural differences in learning styles knowledge area reflects a fair amount of variability (Q1 = 2.25 and Q3 = 4.0).

[Insert Table 2 here]

Differences in Perceived Current and Future Importance of Action Items

Results summarized in Table 3 reveal statistically significant differences (p ≤ .05) in the ratings of current and future perceived importance for 11 of the 12 training and delivery action items. For all 11 action items where a significant difference existed the median value for future ratings of importance was higher than the median values for current ratings of importance. Manage the learning environment was the one training and delivery action item where there was no significant increase (p = .266) in the rating from current importance (median = 3.33) to future importance (median = 3.50).

The current perceived level of importance for the 12 training and delivery action items ranged from a low median value of 3.0 to a high of 3.5. This reflects a moderate level of importance. The future perceived level of importance ranged from a low median value of 3.5 to a high median value of 4.0 indicating a tendency toward a perceived high level of future importance.

[Insert Table 3 here]

Current research predicts that technology will be very important in delivering training instructions to make an efficient and productive workforce in the next five years. Knowledge areas like adult learning, instructional design and electronic performance systems are not recognized in Pakistan. Pakistani practitioners have identified strong perceptions about the knowledge areas identified by the ASTD for delivering training or instructions in their future jobs. The current research indicates that Pakistani practitioners need to embrace the knowledge areas identified by the ASTD for meeting client needs and delivering solutions.

All the values of the items, except “Manages the learning environment” of the delivering training survey indicate a reliability coefficient greater than 0.5 or higher. The scores from the items on the instrument are internally consistent (Table 2).

Conclusions and implications

Interestingly, all of the items for future importance are rated higher than those for current importance. In general, most of the items for current importance average around 3.0. Although the survey of delivering training was developed for the international practitioners, the findings indicate the survey was applicable for Pakistani practitioners. The Pakistani participants who reported current and future importance were equivalent with respect to generalization of delivering training expertise. For some items, they reported a highly significant mean gains determining future importance as compared to those in the current. Therefore, the conclusion is that in the future that the perceptions most of the action areas will be more important than is perceived currently. The findings are consistent with several previous studies that linked delivering training intervention with improved learning practices in a variety of fields (Long and Smith, 2004; Hartley and West, 2007; Gilleard, 1996; Anonymous, 2001, 2003a, & 2003b; Kim, 2009; Baldwin-Evans, 2004; Rawer et el., 1997).

Implications

The results of this study have practical implications for Pakistani instructors, academia, and researchers for delivering training instructions by providing information on the importance of existing and future workforce requirements. A well designed, and standardized training program will prepare the Pakistani workforce to embrace new roles, changing tasks, and emerging technological trends. Delivery of training within important knowledge areas will enhance the capacity to efficiently transfer the skills on the job. However, current research strongly supports that a series of structured training programs in different delivery formats, for instance, technology-embedded programs will improve employees’ capabilities to understand automation or workflow processes in the workplace. Research explains several benefits for individuals and organizations. For example learners used their computing skills in writing books, articles, delivering lectures, and providing consultancy services (Sharif and Mahmood, 2001).

Interestingly, Pakistan spends 3.39 percent as a percentage of annual development programs on education and training.  This percentage is very low even among the developing countries. Following the best practices organizations incorporating technology in the delivery training may reduce the costs per course by 42% (Derouen and Kleiner, 1994). Though the diffusion of technology is a slow process, it improves the learning environment (Rawer, Morton, McCulloch, Heyes, and Ryan, 1997), manages to reach the learners (Anonymous, 2003), and manages time more effectively (Yemm, 2006). Pakistani practitioners’ perceptions of various actions for delivering training can add value to businesses (Bentley, 2006), profitability in the private sector, service-delivery in the public sector and the achievement of shared objectives in the voluntary sector (Sloman, 2007).

Teaching of the WLP discipline can prepare potential Pakistani practitioners to deliver training using a variety of formats such as classroom, or technology assisted instruction.  The mix of these formats encompasses several benefits, like self-sufficiency for obtaining information and instruction for employees at the workplace (Baldwin-Evans, 2006). In addition, Pakistani employees can personalize their training needs for better and relevant learning retention (Hartley and West, 2007). As a result, effective training delivery will lead to employee job satisfaction, motivation and commitment on the one hand, and organizational outcomes including, increased productivity, reduced turnover and absenteeism on the other hand (Sahinidis and Bouris, 2008).

Furthermore, professional development activities like train-the-trainers programs for instructors, especially those who are new entrants in the WLP field, are highly recommended. The results of this study are useful in designing learning objectives, developing learning environment, and providing feedback to learners. In addition, academia needs to formulate the guidelines for developing curricula to prepare the potential workforce. Finally, researchers will gain insight into the current and future directions of competencies in Pakistan. Consequently, a dedicated, motivated, and engaged workforce equipped with 21st century skills will play a vital role to fill the gap between haves and have notes.

Recommendations 

Delivering training for preparing learners to be successful in their workplaces has been identified as important areas of expertise of the workplace training practitioners. In fact, improving competency of trainers will potentially change behaviors of employees and application of knowledge areas will bring more awareness in terms of decent work for all (Labor and Manpower Division, 2008) in the world of work. Based on the findings, the following recommendations for Pakistan’s future research and policy are offered:

Awareness of important knowledge areas that include adult learning and instructional design theory, methods and principles should be integrated into delivering training instruction by the instructors or trainers at the Pakistani workplace.

This study is based on a limited number of Pakistani workplace practitioners. To enhance generalizability and practical significance, future research is needed to examine the effects of delivering training in broader curricula, and a greater number of industrial and vocational training areas and WLP disciplines.

Research based on each of the subscale items can guide curriculum development, instructional planning, and evaluation activities in delivering training programs at the Pakistani workplace.

In summary, additional qualitative and quantitative studies are needed to examine the effects of delivering training on transfer of skills on the job. Further studies could provide a baseline to develop training instructions to improve employees’ 21 st century

skills. Delivering training has the potential to unleash the employees’ competencies to overcome the worst economic crises in the country (Nelson, 1990).

References

Anonymous. (2001).”New research proves that critical business skills can be

web delivered”.

Industrial and Commercial Training, 33(3), p.112.

Anonymous. (2003a). “Sushi-bar courses provide short, sharp learning”.

Industrial and Commercial Training, 35(6/7), pp. 312-313.

Anonymous. (2003b). “Training gets covered at US insurer. Industrial and

Commercial Training”, 35(6/7), pp. 313-314.

Baldwin-Evans, K. (2004). “Employees and e-learning: what do the end-users

think?”,

Industrial and Commercial Training, 36(6/7). pp. 269-274.

Baldwin-Evans. K. (2006). “Blended learning: The what, where, when and

how”. Training & Management Development Methods, 20(3), pp. 353-

367.

Bartell,S. M., (2001). “Training’s new role in learning organizations”,

Innovation in Education and Teaching International, 38(4). pp. 354-363.

Bentley, R. (2006, April). “From famine to feast”. Training & Coaching Today,

pp. 31-32.

Bernthal, P. R., Colteryahn, K., Davis. P., Naughton, J., & Rothwell, W. J.

(2004). “ASTD competency model mapping the future: New workplace

learning and performance competencies”. Alexandria, VA: ASTD.

Black, T. R., (1999). Doing quantitative research in the social sciences: an

integrated approach to research design, measurement and statistics.

Thousand Oaks, CA: SAGE Publications, Inc.

Borg, W. R., & Gall, M. D. (1996). “A new paradigm for leadership in

organization”. Human Resource Development Quarterly, 7(3), pp. 287-

293.

Cantor, J. A., (2002). “Delivering instruction to adult learners”. Revised

Edition. Toronto, Ontario: Wall & Emerson, Inc.,

Carlivati, P. A., (2002). “E-learning evolves”, American Bankers Association.

ABA Banking   Journal. 94(6), pp. 49-52.

Cercone, K. (2008). “Characteristics of adult learners with implications for

online learning design”. AACE Journal, 16(2). pp. 137-159.

Cohen, J., (1988). Statistical power analysis for the behavioral sciences. (2nd

ed.). Hilldale, NJ: Lawrence Erlbaum. 

Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple

regression/correlation analysis for the behavioral sciences. (3rd ed.). 

Mahwah, New Jersey: Lawrence Erlbaum Associates,

Deeny, E., (2003). “Calculating the real value of e-learning”. Industrial and

Commercial Training. 35(2/3), pp. 70-72.

Derouen, C, & Kleiner, B.  H., (1994, Mar/Apr). “New developments in

employee training”. Work Study. 43(2), pp. 13-16.

Dillman, D. A., (2007). Mail and internet survey: The tailored designed

method. (2nd ed.). Danvers, MA: John Wiley.  

Drummond, R. M. (2005). Developing core job competencies for payment

services roles and associating them with personality traits using the

Workplace Big Five Profile(TM). Capella University, 1-259. (AAT

3161743).

Economic Adviser’s Wing. (2000-2001). Pakistan Economic survey. Finance

Division. Islamabad. Government of Pakistan. Author.

Gilleard, J. (1996).  “Delivering training down the line”. Industrial and

Commercial Training, 28 (7), pp. 22 – 27.

Graziano, A. M., & Raulin, M. L. (2000). Research methods: A process of

inquiry with students. (4th ed). Boston: Allyn and Bacon.

Haider, S. J. (1998). “Educating future librarians in Pakistan: a library

educator’s perspective”.

              Education for Information, 16 (1), pp. 29–44.

Hartley, D., & West, K. (2007, Nov). “Taking it personally: Tailoring training

for more relevance”, T+D. 61(11), pp. 21-23.

Huck, S.W. (2008). Reading statistics and research. 5th edition. Boston:

PearsonEducation, Inc.

International Labour Organization (ILO). (2003). Learning and training for

work

in the knowledge society. Fourth item on the agenda. Report IV(1).

http://www.ilo.org/public/english/standards/relm/ilc/ilc91/rep-iv-1.htm.

Accessed on October 06, 2009.

Kapp, K. M., & McKeague, C. (2002). “Blended learning for compliance

training success”.

EduNeering, Inc. 1-12. Retrieved from

http://www.astd.org/NR/rdonlyres/CA5E2B24-AFAF-4464-B689-

3C1C78D7F279/0/blendedlearning.pdf. (Accessed 12 September 2009).

Kendall, S., Murfield, J., Dillon, J., & Wilkin, A. (2008). . “Education outside the

classroom: Research to identify what training is offered by initial

teacher training institutions”. National Foundation for Educational

Research. Research Report RR802.

Kerlinger, F. N., & Lee, H. B. (2000). Foundations of behavioral research, (4th

ed.). Harcourt College Publishers. Boston: Allyn and Bacon.

Kim, K-J. (2009). “Motivational challenges of adult learners in self-directed e-

learning”. Journal of Interactive Learning Research, 20(3). 317-335.

Kniveton, B. H., (2004). “Managerial career anchors in a changing business

environment”, Journal of European Industrial Training. 28(6/7), p. 564

Knowles, M. S. (1970). The modern practice of adult education; Andragogy

versus pedagogy Association Press.

Kotrilik, J. W., & Williams, H. A. (2003). “The incorporation of effect size in

information technology, learning and performance research”.

Information Technology, Learning and Performance Journal. 21 (1), pp.

1-6.

Labour & Manpower Division. (1998). Pakistan manpower commission report.

Author.

Labour & Manpower Division. (2008). A growth oriented national employment

policy. Government of Pakistan. Author.

Long, L. K., & Smith, R. D. (2004).  “The role of web-based distance learning

in HR development”. The Journal of Management Development. 23

(3/4), pp. 270-284.

Mahmood, K. (1997). “Information technology and library education in

Pakistan: Recent developments in curriculum”, Education for

Information, 15, pp. 197-205.

Manyaga, T. (2008). ”Standards to assure quality in tertiary education: The

case of Tanzania”. Quality Assurance in Education: An International

Perspective, 6 (2), pp. 164-180.

Nelson, J. M. (1990). Economic crisis and policy choice: the politics of

adjustment in the third world. Princeton, NJ: Princeton University Press.

Nguyen, F. (2007). The effect of an electronic performance support system

and training as performance interventions. Arizona State University.

Dissertation Abstracts International Section A: Humanities and Social

Sciences, 68(4-A). 1424.

Nguyen, F. & Klein, J. D. (2008). “The Effect of Performance Support and

Training as performance interventions”. Performance Improvement

Quarterly, 2(1), pp. 95-114.

Noe, R. (2002). Employee training and development. (2nd ed.). Columbus, OH:

McGraw-Hill Companies.

Norman, G. R., & Streiner, D. L. (2008). Biostatistics: The bare essentials.

Hamilton Ontario: BC Decker Inc.

Ogbeide, G-C. (2006). Employability skills and students' self-perceived

competence for careers in the hospitality industry. (Doctoral

dissertation), University of Missouri-Columbia. (AAT 3284791). 111

Olejnik, S., & Algina, J. (2000). “Measures of effect size for comparative

studies: Applications, interpretations, and limitations”, Contemporary

Educational Psychology, 25, pp. 241-286.

Oxford English Dictionary (2003). Neurolinguistic programming'. Retrieved June 23, 2009.

Patton, M. Q. (2002). Qualitative research and evaluation methods. (3rd ed.).

Thousand Oaks, CA: Sage Publications, Inc.

Rawer, H.,  Morton, A., McCulloch, R., Heyes, L., & Ryan, J. (1997).

“Delivering training in teaching skills to hospital doctors”. Medical

Teacher, 19 (3), pp. 209 – 211.

Sahinidis, A.  G., & Bouris, J. (2008). “Employee perceived training

effectiveness relationship to employee attitudes”. Journal of European

Industrial Training. 32 (1), pp. 63-76.

Sharif, A., & Mahmood, K. (2001). “Impact of computer training on

professional library

              activities in Pakistan”. Information Development; 17 (3), pp. 173-

178.

Sheppard, G., (2005, September 07). “Assessments jeopardize; Government

skills plan”. Training Magazine.  Retrieved September 26, 2009 from

accessmylibrary: http://www.accessmylibrary.com/article-1G1-

135700350/assessments-jeopardise-government-skills.html. (Accessed

12 September 2009).

Siddons, S. (2003). Training essentials: Delivering training. London. UK:

Chartered Institute of Personnel and Development (CIPD).  

Sloman, M., (2007). The changing world of the trainer. London, UK: CIPD

Enterprises Limited.

United Nations Development Programs (UNDP). (2008). Fighting change:

Human solidarity in a divided world. Human Development Report. New

York: Author.

Willis, J. H. (2007). “Rush health systems and meridian community college:

People serving people”, Community College Journal of Research and

Practice, 31 (6), pp. 475-478.

Wyrick, C. (2001). The ASTD Handbook of Training Design and Delivery: A

Comprehensive Guide to Creating and Delivering Training Programs--

Instructor-Led, Computer-Based, or Self-Directed, 54 (4), pp. 1037-40.

Yemm, G. (2006). “Can NLP help or harm your training and development?”

Training & Management Development Methods, 20 (3), pp. 407-416.

 Table 1. Profile of the Respondents

Variable Number Percent

Gender (n=40) Male Female

2515

62.537.5

Highest Education Level (n=40) Less than Bachelor Degree Bachelor Degree Master Degree Post Master Degree

53

275

12.57.567.512.5

Total years of Learning and Performance Experience (n=40) Less than 1 year 1 – 2 years 3 - 5 years 6 – 10 years 11 – 15 years More than 15 years

54

1126

12

12.510.027.55.015.030.0

Current Level in Organization (n=40) Executive Director Manager Supervisor Team Leader University Faculty Private Consultant Entry Level Professional

63783535

15.07.517.520.07.512.57.512.5

Type of Industry/Business (n=39)Agriculture, Forestry, Fishing or HuntingFinance & Insurance Information Services & Communications Business Schools & Management Training Education Software Development Health Care & Social Services Retail Trade Manufacturing Real Estate rental & leasing Other (private consulting, government

29644332223

5.123.115.410.310.37.77.75.15.15.17.7

Table 2. Summary Descriptive Statistics and Wilcoxon Signed Rank Test Results for Knowledge Areas Current Perceived and Future Perceived Importance (n = 40)

Training Delivery Action Item

Current Importance

Median1 (1st Q /3rd Q)2

Future Importance

Median1 (1st Q / 3rd Q)2

Wilcoxon Signed Rank Test p value

Key knowledge areas 3.27 (2.83 / 3.67) 4.23 (3.69 / 4.85) <.001Adult learning theories and techniques

3.00 (3.00 / 3.00) 4.00 (3.00 / 5.00) <.001

Instructional design theory & methods

2.50 (2.00 / 3.75) 4.00 (3.00 / 5.00) <.001

Various instructional methods (lectures, discussions, exercises)

3.00 (2.00 / 3.75) 4.00 (3.00 / 5.00) <.001

Various delivery options/media

3.00 (3.00 / 4.00) 4.00 (3.00 / 5.00) .002

Existing learning technologies and support systems

3.00 (3.00 / 4.00) 4.00 (3.00 / 5.00) <.001

Emerging technologies and support systems

3.00 (2.00 / 4.00) 4.00 (3.00 / 5.00) <.001

Presentation techniques and tools

3.00 (3.00 / 4.00) 4.00 (4.00 / 5.00) <.001

Organizational work environment and systems

3.00 (2.25 / 4.00) 4.00 (3.00 / 5.00) .001

Individual learning styles e.g. audio, visual

3.00 (2.00 / 4.00) 4.00 (3.00 / 5.00) .001

Cultural differences in learning styles

3.00 (2.00 / 3.00) 3.50 (2.25 / 4.00) <.001

Own personal learning preferences

3.00 (3.00 / 4.00) 4.00 (3.00 / 4.00) <.001

Tools for determining learning preferences

3.00 (3.00 / 4.00) 4.00 (3.00 . 5.00) <.001

Familiarity with content being taught

3.00 (3.00 / 4.00) 4.00 (4.00 / 5.00) <.001

Legal and ethical issues relevant for delivering training

3.00 (2.00 / 4.00) 4.00 (3.00 / 5.00) <.001

Note: 1. Response scale was 1=Not Important; 2= Slightly important; 3=Moderately Important;

4=Highly important; and 5 = Essential. 2. Q refers to quartile.

Table 3. Summary Descriptive Statistics and Wilcoxon Signed Rank Test Results for Action Items Current Perceived and Future Perceived Importance (n = 40)

Training Delivery Action Item

Current Importance

Median1 (1st Q /3rd Q)2

Future Importance

Median1 (1st Q / 3rd Q)2

Wilcoxon Signed Rank Test p value

Prepares for training delivery 3.50 (2.00 / 4.00) 3.83 (3.33 / 4.33) .023Aligns learning solutions with course objectives and learner needs 3.00 (2.00 / 3.67) 4.00 (3.67 / 4.33) <.001Conveys objectives 3.50 (2.50 / 4.00) 4.00 (3.50 / 4.38) .001Delivers using various learning methodologies

3.50 (2.13 / 3.50) 4.00 (3.50 / 4.38) <.001Facilitates learning 3.00 (2.75 / 4.00) 3.83 (3.50 / 4.33) .035Encourages participation and builds learner motivation

3.25 (2.67 / 3.67) 3.83 (3.33 / 4.33) <.001Establishes credibility as instructor

3.50 (3.00 / 4.00) 4.00 (3.33 / 4.67) .001

Manages the learning environment

3.33 (3.00 / 4.00) 3.50 (3.17 / 4.00) .266

Delivers constructive feedback

3.40 (2.65 / 3.75) 3.83 (3.33 / 4.33) .003

Creates a positive learning climate

3.50 (2.81 / 4.00) 4.00 (3.31 / 4.25) .001

Ensures learning outcomes 3.00 (2.31 / 3.44) 3.75 (3.25 / 4.25) <.001Evaluates solutions 3.00 (2.50 / 3.50) 4.00 (3.50 / 4.50) <.001Note: 1. Response scale was 1= Not Important; 2= Slightly important; 3= Moderately Important;

4= Highly important; and 5 = Essential. 2. Q refers to quartile.