Evaluating HRD Programs - हे Buddy Web viewEvaluating HRD Programs. Evaluating HRD Programs....
Transcript of Evaluating HRD Programs - हे Buddy Web viewEvaluating HRD Programs. Evaluating HRD Programs....
Evaluating HRD Programs
HUMAN RESOURCE DEVELOPMENT
EVALUATING HRD PROGRAMS
XAVIER INSTITUTE OF SOCIAL SERVICERANCHI
Submitted by: Submitted to:
Group: VI Prof. Sajeet LakraPersonnel Management-II2010-2012
Page 1
Evaluating HRD Programs
GROUP CONSTITUTION
Page 2
Members Roll No.
Ashutosh Sharma 06Aditi Gupta 15Kumar Pratik 28Divyendu Shekhar 38Utkarsh 52Shashank Shekhar Rai 66
Evaluating HRD Programs
ACKNOWLEDGEMENTSuccess of a project depends on the endeavor put behind it. Encouragement and patronization perfect an endeavor.We take this opportunity to express our heartfelt gratitude to our faculty Prof. Sajeet Lakra, for providing us with the opportunity of preparing the study material for the of the HRD curriculum and give presentation in the class. The group’s involvement and team work helped us understand the chapter in a lucid manner. The group’s cohesiveness and team work helped us to make this task a success. We thank all the team members and also all our colleagues for their unconditional support.Thank YouGroup VI
Page 3
Evaluating HRD Programs
CONTENTSS.No. Topics Pg. No.
1. Introduction 052. Purpose of HRD Evaluation 063. Principles of Evaluation 074. Importance of Training Evaluation 085. Benefits of Evaluation 106. Process of Training Evaluation 117. Models and Framework of Evaluation 158. Data Collection for HRD Evaluation 219. Linking Training and development
with Organization’s Business Strategy
22
10. Case Study 2611. Conclusion 2812. Bibliography 29
Page 4
Evaluating HRD Programs
1.INTRODUCTION TO HUMAN RESOURCE DEVELOPMENT EVALUATION PROCESS
A comprehensive and effective evaluation plan is a critical component of any successful training program. It should be structured to generate information of the impact of training on the reaction; on the amount of learning that has taken place; on the trainees behaviour; and its contribution to the job/ organization. Therefore evaluation of training is a measure of how well it has meet the need of its human resources.
Any attempt to obtain information (feedback) on the effect of training program and to assess the value of training in the light of that information for improving further training.”- Hamblin“The systematic collection of descriptive and judgmental information necessary to make effective training decisions related to the selection, adoption, value, and modification of various instructional activities.”“Any attempt to obtain information (feedback) on the effect of training program and to assess the value of training in the light of that information for improving further training.”
Page 5
Evaluating HRD Programs
Assessing the effectiveness of the training program in terms of the benefits to the trainees and the company must look into the following aspects:– process of collecting outcomes to determine if the training program was effective– from whom, what, when, and how information should be collected
2. PURPOSE OF EVALUATION
• Determining accomplishment of specific training objectives.
• Determining cost effectiveness of training. • Explain program failure, if it occurs.• Enhancement of the credibility of training and
development if tangible benefits to an organization are proved.
• Determine whether the program is meeting the intended objectives
• Identify strengths and weaknesses• Determine cost-benefit ratio• Identify who benefited most or least
Page 6
Evaluating HRD Programs
• Determine future participants• Provide information for improving HRD programs• Reinforce major points to be made• Gather marketing information• Determine if training program is appropriate• Establish management database
3. PRINCIPLES OF EVALUATION
• Clarity about the goals and purposes of evaluation.• Evaluation must be continuous.• Evaluation must be specific.• Evaluation must be based on objective methods
and standards.• Realistic target dates must be set for each phase
of the evaluation process.• Evaluation has to be accepted as a process than an
end product of training.
Page 7
Evaluating HRD Programs
4. IMPORTANCE OF TRAINING EVALUATION
An organization that invests in training programs for employees should also conduct some form of training evaluation. This can be done in a number of different ways. The purpose of evaluation is to determine whether or not the training achieved the desired objective. The objective might have been very precise, or extremely loose, but there would have been one. This is usually set as a result of assessing trainining needs. Evaluating training will help show how effective it was at meeting the objective. All organizations, regardless of size or type, should use some process of assessing the effectiveness of training undertaken by employees. Education in any form is an investment of time and money, which makes it important that the
Page 8
Evaluating HRD Programs
organization establishes, as far as it can, the return on that investment.
Evaluating training effectiveness is important because it sheds light on four aspects. They include:
How well the training program met the learner’s needs and objectives
What knowledge and skills it has imparted to learners
What desirable change it has brought in the learners’ performance
What organizational benefits it has yieldedHowever, unfortunately, in today’s organizations, this mechanism of training evaluation is in short supply. Organizations are unwilling to spend their resources for a comprehensive after-training evaluation. They end it by serving a ceremonious feedback form to learners and getting their opinions. But you must realize that it will not work. It will not make your training programs effective and yield the results you want.In this absence of a proactive follow-up, the following undesirable consequences will have to be faced:Lack of application of skills by learners:
You had analyzed the performance gaps in your employees and devised a suitable training program. At the end of the training program, your participants said that the training program was quite useful and they had learnt many useful skills. You are happy about that. But at the end of your annual auditing, you are horrified to find no progress in the performance in spite of the training. And your employees have not used the skills you intend them to after the training. This is quite
Page 9
Evaluating HRD Programs
possible. You can avert it by putting in place a proper evaluation mechanism.Inability to measure ROI:
Financial assessment of a training program is crucial in that it provides useful insights into the Returns on Investment (ROI). This assessment depends on evaluating how well the employees implemented the skills they have learnt in the training program.Inability to make future training programs contextual:
In today’s dynamic marketplace, the need for consistent skill up gradation never loses its importance. What was said to be an effective training program in the past may not be so now. You need to continuously adapt it to suit your current needs. This is possible when you know how effective your previous training program was.There are methods like Kirkpatrick’s Training Evaluation Model, Bloom’s Taxonomy of Learning Domains and Keller’s ARCS Model. Use the models suggested here depending on the available resources, size of the training program and your corporate culture. When you have complex and critical training programs, you need to choose elaborate tools and have a wider approach to evaluate each program at various stages. Only then can the evaluation become effective and accurate.
Page 10
Evaluating HRD Programs
5. BENEFITS OF TRAINING EVALUATION
Improved quality of training activities. Helps employees to monitor their own
improvement. Builds morale, by demonstrating an interest in staff
development. To maximize the training ROI (return on
investment). Helps to determine the form of future training
programs. Assists with identifying the effectiveness of
different forms of teaching (such as classroom based or web based).
Improved ability of the trainers to relate inputs to outputs
Better discrimination of training activities between those that are worthy of support and those that should be dropped
Better integration of training offered and on-the job development
Better co-operation between trainers and line-managers in the development of staff
Evidence of the contribution that training and development are making to the organization
Page 11
Evaluating HRD Programs
Reduction in preventable accidents measured Reduction in scrap/rework measured in cost of
labor and materials
6. PROCESS OF TRAINING EVALUATION
There are two considerations involved in evaluation:• Who is responsible for the validation and
evaluation processes?• What resources of time, people and money are
available for validation/evaluation purposes? Thus the process of evaluation involves 3 stages:• Before training• During training• After training
BEFORE TRAINING: The learner's skills and knowledge are assessed before the training program. During the start of training, candidates generally perceive it as a waste of resources because at most of the times candidates are unaware of the objectives and learning outcomes of the program. Once aware, they are asked to give their opinions on the methods used and whether those
Page 12
Evaluating HRD Programs
methods conform to the candidate’s preferences and learning style. DURING TRAINING: It is the phase at which instruction is started. This phase usually consists of short tests at regular intervals.AFTER TRAINING:
It is the phase when learner’s skills and knowledge are assessed again to measure the effectiveness of the training. This phase is designed to determine whether training has had the desired effect at individual department and organizational levels. There are various evaluation techniques for this phase.
The Training Evaluation should involve: senior management the trainer line management the training manager the trainee
SENIOR MANAGEMENT:
• Awareness of need & value of training • Involving Training Manager in senior management
meetings• Knowledge & support of training plans
Page 13
Evaluating HRD Programs
• Active participation in events• Evaluation to be done & regular summary reports• Policy & strategic decisions based on results &
return on investment data.THE TRAINER:• Provision of any necessary pre-program work • Identifying knowledge & skills level• Training & learning resources to enable the
learners to learn within objectives of program• Monitoring the learning as the program progresses• Assessment of & receipt of reports.
THE LINE MANAGER:• Work-needs & people identification• Involvement in training programme & evaluation
development• Pre-event preparation & holding briefing meetings• Giving ongoing & practical support to the training
programme• Reviewing the progress of learning implementation• Final review of implementation success and
assessment.
THE TRAINING MANAGER:Page 14
Evaluating HRD Programs
• Management of the training department & agreeing to the training needs
• Maintenance of interest & support in the planning & implementation of the programs including practical involvement where required
• The introduction & maintenance of evaluation systems
• Production of regular reports for senior management
• Contact with senior management• Liaison with the learners' line managers and
arrangement of learning implementation.
THE TRAINEE:• Plan & design the training programme• Contribute towards the evaluation process• Take interest & performing an active part in the
training programme or activity• Complete a personal action plan during & at the
end of the training• Support the evaluation processes.
Page 15
Evaluating HRD Programs
7. MODELS AND FRAMEWORK OF EVALUATION
• There are six frameworks for evaluation• Kirkpatrick• CIPP• Brinkerhoff• Kraiger, Ford & Salas
Page 16
Evaluating HRD Programs
• Holton• Philips
Two approaches are mainly used: Goal-based Approach-
The Kirkpatrick Model System-based Approach:
(i)CIPP Model (ii) TVS Approach (iii) IPO Approach
Kirkpatrick (1959) CIPP Model (1987) IPO Model (1990) TVS Model (1994)
1. Reaction: to gather data on participants reactions at the end of a training program
1. Context: obtaining information about the situation to decide on educational needs and to establish program objectives
1. Input: evaluation of system performance indicators such as trainee qualifications, availability of materials, appropriateness of training, etc.
1. Situation: collecting pre-training data to ascertain current levels of performance within the organization and defining a desirable
Page 17
Evaluating HRD Programs
level of future performance
2. Learning: to assess whether the learning objectives for the program are met
2. Input: identifying educational strategies most likely to achieve the desired result
2. Process: embraces planning, design, development, and delivery of training programs
2. Intervention: identifying the reason for the existence of the gap between the present and desirable performance to find out if training is the solution to the problem
3. Behavior: to assess whether job performance changes as a result of training
3. Process: assessing the implementation of the educational program
3. Output: Gathering data resulting from the training interventions
3. Impact: evaluating the difference between the pre- and post-training data
4. Results: to assess costs vs. benefits of training programs, i.e., organizational impact in terms of reduced costs, improved quality of work, increased quantity of work, etc.
4. Product: gathering information regarding the results of the educational intervention to interpret its worth and merit
4. Outcomes: longer-term results associated with improvement in the corporation’s bottom line- its profitability, competitiveness, etc.
4. Value: measuring differences in quality, productivity, service, or sales, all of which can be expressed in terms of dollars
GOAL-BASED SYSTEM-BASED
These models do not define the steps CIPP, IPO, and TVS) seem to be more
Page 18
Evaluating HRD Programs
necessary to achieve purposes and do not address the ways of utilizing results to improve training.
useful in terms of thinking about the overall context and situation but they may not provide sufficient granularity.
The difficulty for practitioners following such models is in selecting and implementing appropriate evaluation methods (quantitative, qualitative, or mixed).
Systems-based models may not represent the dynamic interactions between the design and the evaluation
of training.
Because of their apparent simplicity, trainers jump feet first into using such models without taking the time to assess their needs and resources or to determine how they’ll
apply the model.
Few of these models provide detailed descriptions of the processes involved in each steps. None provide tools for evaluation.
Many organizations do not use the entire model, and training ends up being evaluated only at the reaction, or at best, at the
learning level.
These models do not address the collaborative process of evaluation, that is, the different roles and responsibilities that people may play during an evaluation process
THE KIRKPATRICK MODEL:
Page 19
Evaluating HRD Programs
Level 1 : ReactionReaction evaluation is how the delegates felt about the training or learning experience. Evaluation tools and methods • 'Happy sheets', feedback forms.
Page 20
Evaluating HRD Programs
• Verbal reaction, post-training surveys or questionnaires
Relevance and practicability • Quick and very easy to obtain. • Not expensive to gather or to analyse
Level 2 : Learning Learning evaluation is the measurement of the increase in knowledge before and after.Evaluation tools and methods • Typically assessments or tests before and after the
training• Interview or observation can also be used
Relevance and practicability • Relatively simple to set up; clear-cut for
quantifiable skills• Less easy for complex learning
Level 3 : Behavior Behaviour evaluation is the extent of applied learning back on the job - implementation.Evaluation tools and methods
Page 21
Evaluating HRD Programs
• Observation and interview over time are required to assess change, relevance of change, and sustainability of change
Relevance and practicability • Measurement of behaviour change typically
requires cooperation and skill of line-managersLevel 4 : Results Results evaluation is the effect on the business or environment by the trainee.Evaluation tools and methods • Measures are already in place via normal
management systems and reporting - the challenge is to relate to the trainee
Relevance and practicability • Individually not difficult; unlike whole organisation
Process must attribute clear accountabilities
Page 22
Evaluating HRD Programs
8. DATA COLLECTION FOR HRD EVALUATIONThere are several techniques of data collection. They are as follows:
Observation Questionnaire Interview Written Test Simulation/ Performance test
Advantages and Disadvantages of Techniques :
METHODS ADVANTAGES DISADVANTAGES
1.Interview • Flexible• Opportunity
for clarification• Depth Possible• Personal
Contact
• High Reactive effects
• High cost• Face-to-Face
Threat Potential
• Trained Observers Needed
•
2.Questionnaire
• Low Cost• Anonymity
Possible• Respondent
sets pace
• Possibly Inaccurate Data
• Return Rate Beyond
Page 23
Evaluating HRD Programs
• Variety of Options
Control
Observation • Non- Threatening
• Excellent way to Measure Behaviour Change
• Reactive Effect
• Probable• Trained
Observers Needed
Written Test • Low Purchase Cost
• Readily Scored and Quickly Processed
• Easily Administered
• Wide Sampling Possible
• Possible low Relation to Job Performance
• Reliance on Norms may Distort Individual Performance
• Cultural BiasSimulation/ Performance Test
• Reliable• Objective• Close Relation
to Job Performance
• Time consuming
• Simulation often Difficult and Costly
Page 24
Evaluating HRD Programs
9. LINKING TRAINING AND DEVELOPMENT WITH BUSINESS STRATEGY
• Training and Development strategy is a comprehensive master plan which ensures that investment in learning is assessed and evaluated.
• Business Strategy is the determination of the long term goals and objectives of an organization, and allocation of resources necessary for carrying out these goals.
• Training functions lie at the heart of how businesses access and develop human resources required to maintain a sustainable competitive advantage over rivals.
• Training should result in creating skills that are appropriate for achieving the strategic objectives of a business.
The Factors governing the starting point for formulating an effective training and development strategy are as follows:
Page 25
Evaluating HRD Programs
• Understand the industry and the business’ unique offer and what gives it competitive advantage.
• Consider the changes predicted in the environment- the rate of growth/decline, the competition and degree of technological change.
• Assess the need to change and adapt to economic circumstances.
• Assess the level of knowledge and professional capability of staff now and that needed in the future.
• Understand the importance of customer service and reflect the community served- what sort of customers are served and the nature of their expectations.
Training and development strategy should also align with the cultural beliefs about what learning may achieve and the way it is regarded. Assumptions include: • All work is a process of continual learning and
improvement. This can be the starting point for creating a ‘learning organization’.
• Learning should be encouraged only where there is measurable benefit to the organization.
• Learning should be targeted on key performance skills.
• Learning may cover other aims than just a narrow definition of business benefit- it can support the
Page 26
Evaluating HRD Programs
organization’s duty to the community or wider social responsibility.
• Learning is a cost to the business and should be delivered effectively and efficiently.
• Learning and talent development is an investment for the future.
• Personal career development is critical for retention and engagement and should ne encouraged.
According to Rothwell & Kazanas the ten Approaches to link T&D strategy with Business Strategy are:
1. The Top Down Approach: The Organizational strategy determines and dictates the HRD offerings.
2. The Market Driven Approach: The HRD function or department identifies future learning on the basis of future market conditions confronting the organization.
3. The Career Planning Approach: The HRD effort helps individuals prepare for the future against the backdrop of organization’s strategic plan.
4. The Futuring Approach: The HRD function formally and directly assists top managers as they formulate the organizational strategy.
5. The Artificial Experience Approach: The HRD function simulates the conditions the organization may face in the future.
6. The Pulse Taking Approach: Since HRD function is at the forefront of communicating, assessing needs, design and develop instructions and
Page 27
Evaluating HRD Programs
evaluate results-they maintain a high profile. This unique position in organization’s communication chain is utilized to the best advantage in this approach. HRD people take pulse of organization regarding implementation of business strategy.
7. The Performance Diagnosis Approach: While assessing the training needs, HRD professionals often uncover performance problems or opportunities of strategic importance. They relay that information to the top management for their use in strategic planning.
8. The Education Approach: HRD practitioners try to link HRD with the organizational strategy by teaching people how to think strategically as they do their work. By applying strategic planning model people can anticipate the future problems and opportunities.
9. The Inter Personal Approach: It is informal and may involve socializing with strategists to know what they are seeking.
10. The Rifle Approach: It takes its name from its laser like focus on achieving pinpointed results. The HRD practitioners aim their efforts to the areas of greatest need or the business’ most pressing problem.
Page 28
Evaluating HRD Programs
10. TRAINING EVALUATION IN BRITANNIA
For each training programme conducted in Britannia, a training brochure is developed.The brochure developed consists of the following information: • Programme objectives: Need of the training and
what are the objectives that this training aims to achieve, what likely outcomes are expected to come out of impact of this training.
• Programme content: Topics being covered during the training.
• Methodology adopted. • Programme faculty. • Personnel to be covered. • Training methodology. • Training effectiveness criteria and scale. • The training effectiveness is measured by
measurement of the achievement of the objectives.
• A person gets nominated for a training programme which flows from the training needs of that individual
Page 29
Evaluating HRD Programs
• The programs are divided into three broad categories:
Functional Behavioural General/ Omnibus programmes
Functional programs:The outcome of the training is measured by comparing the data pre-training and post-training. A scale is developed for measuring the effectiveness of training based on the % achievement of the objectives.
Behavioural programs:The effectiveness of the training of this nature is measured annually. This is seen through the training need identification for the coming year for the employee. If the training need is repeated there, then the training provided is taken as ineffective. If the training need is repeated but with focus on a part of the need, then the training is partially effective. If not repeated, then the training is effective.
General programs:These are the training needs flowing directly from the organizational needs. Examples of these can be ISO 9000 training, ISO 14000 training and any awareness
Page 30
Evaluating HRD Programs
training. These are omnibus training programmes, which are run for a large number of employees.
The effectiveness of the training is measured by: Achievement of those organizational objectives
within the time lines. Number of audit issues raised on the areas covered
in the training. Any other such thing as defined in the training
brochure
11. CONCLUSIONEvaluation of HRD Program helps in gauging the degree to which a training (or other HRD program) achieves its intended purpose.In other words it measures the effectiveness of the HRD programThe importance of training programs must be understood by HRD professionals. Not every program needs to be evaluated to the same extent. The key is to have a well planned evaluation strategy that sets the stage for how and to what extent each program will be evaluated.
The value of the training evaluation forms is that they give immediate feedback and they are inexpensive to set up and process. However, it is possible for someone
Page 31
Evaluating HRD Programs
to leave a course having enjoyed it, and believing they learned a great deal. It is only when they come to apply this knowledge that they discover whether the training was as effective as they thought.A more thorough evaluation can involve a subsequent round of training evaluation forms, or an assessment interview, which are conducted a few days or weeks after the training. This allows the learner time to reflect on what they have learned and to have put at least some of it into practice. Their opinion of the training may have changed based on their experience in the workplace.Methods for the effective evaluation of training have been studied for some time. The Kirkpatrick model is now considered to be the industry standard, and is frequently used by HR departments when determining their own systems.Organizations that train but do not evaluate that training cannot be certain of its value, either to themselves or their employees.
12. BIBLIOGRAPHY
• Human Resource Development, Jon M. Werner & Randy L. Desimone, Pg. 151-201,Eighth Edition, Cengage Learning India Pvt. Ltd.
• Human Resource Development- Wikipedia, The Free encyclopedia http://en.wikipedia.org/wiki/Human_resource_development
Page 32
Evaluating HRD Programs
[Accessed on 27th November, 2011]
• http://managementhelp.org/trng_dev/evaluate/evaluate.htm
[Accessed on 27th November, 2011]
• http://www.ifets.info/journals/5_2/eseryel.html
[Accessed on 25th November, 2011]
Page 33