Computer based Simulation Technology as part of the AC · PDF fileComputer based Simulation...

1

1

Computer based Simulation Computer based Simulation

Technology as part of the AC and Technology as part of the AC and

DCDC

a global South African perspectivea global South African perspective2

AgendaAgenda

Introduction

Assessment Centers Evolution

First concept development of the EAC

Custom made CBST

Development Methodology

Research Results

Value of Technology

Implications for Assessment Practitioners

Demo

3

About usAbout us

Global management-

assessment, -training and

development organization

providing Assessment-,

Training-Technologies

through e-Learning and

Business Simulation

Solutions and Continuous

Education

4

Mentorship

Asia

2

5

Geographical focusGeographical focus

Botswana

Namibia

Lesotho

Netherlands

Switzerland

Nigeria

Sweden

South Africa

Zimbabwe

Vietnam

Cambodia

Sri Lanka

Nepal

Melbourne

Singapore

Bangalore

Johannesburg

Perth

Singapore

6

Computer based Business

Simulation Technology

in Assessment Centers

7 8

History of using

technology

in Assessment Centers

3

9

Before TechnologyBefore Technology

10

1989-19981989-1998

Discuss performance with Management/ direct manager

Give individual Feedback with Individual Development Plan

Write Feedback report

Integrate scores through consensus

Evaluate behavior against NORM

Measure behavior

Conduct 5 exercise • Case Study • Group Exercise • In basket• One on One interview• Presentation

Measured only Behavioral competencies

PROCESSTrained Observers and Administrators

EARLY DAYS

Traditional Assessment Centers

HISTORICAL TIME LINE AND DEVELOPMENT

Type writing poolZX Spectrum

DOSLotus NotesWin Word

Win 3.1 (WordExcel, Email etc)

AccuvisonVideo Vignettes

11

AC EvolutionAC Evolution

TraditionalAC

Behavior

’75-’92

Video based

BehaviorMultiple choice

’92-’96

Looking Glass

Behavior

’96-’99

SJT’s

BehaviorBIG 5Multiple choice

2000-2003

Technology BehaviorOrganizational Multiple choice and observation

2004-

12

1998-2004

4

13

E – Learning

Cyber teams

Technology/ paperless environments

E-mail, Cell phones, Skype, Blogging, Facebook, Mixit, GPS

Google and You tube

Why not shape assessment- events and - experiences to

fit this exciting world of Technology?

1998-2004

14

1998-20041998-2004

Aligned generic Behavioral competencies with client specific job competencies and used generic Organizational process competencies as per simulation

PROCESSDeveloped AC to measure both

Behavioral and Organizational competencies

Pre Assessment

Prior to a 18 month development program

USE OF BUSINESS SIMULATION TECHNOLOGY AS A COMPUTERISED INBASKET

Assessment Centers with Generic Leadership Business Simulations

Technology enhanced AC’s

15

Issues Issues

Functionally

Culture

Behavioral

Organizational

Technical

16

Definition of

Assessment Centers

Evaluation of Behavior

Based on multiple inputs

Specially developed simulations

Multiple trained observers and techniques

Judgements about the behavior is made

Judgements are then reported and discussed

Agreement among assessors on behavior of candidate.

MOSES 1975

5

17

Definition Experiential

Assessment

…a scientifically based approach to the identification of future managerial qualities important for the future managerial success at a particular level of management.

18

FunctionalityFunctionality

Exercises are all linked

within the

computerized

simulation but still

continue to observe

behavior in the

behavioral exercises

Your

19

Example 1Example 1

It’s Friday night, and you have already put in 45 hours this week.The owner asks you to work tomorrow night, promisingthat you will be given a large dinner party which shouldget you a big tip.

Which of the following would youmost likely say?

A. ‘‘I’d rather not, thanks.’’B. ‘‘Sure!’’C. ‘‘I’m kind of tired, but I’ll do it if you can’t findsomeone else.’’D. ‘‘I’ll be happy to do it if you can guarantee the tip

20

Example 2Example 2Client ; ….a contract worker as a ship bridge worker

He : Hi …. SILENCE …You : Good Morning. How can I help you?He : I am Jan I want to apply for a personal loan.

What will you do?

Explain the procedure : Choose the right procedure1 Need an ID2 If banking with XXX BANK need bank statement3 Ask for pay slip4 Go in to system and check if client will good for the amount5 If Not explain to him that it will not be bale to do6 If ratings is good explain to him the various option of a loan

a. 1.3.2.4.5.6.b. 4.3.1.2.5.6.c. 4.5.6.3.2.1. d. 1 2,3,4,5,6,

6

21

Capacity BuildingCapacity BuildingIntroduction to EACDiscussion ExerciseDiscussion NormDiscussion ScaleDiscussion CompetenciesDiscussion SimulationsCase StudyAnalyze reportClassify CompetenciesRate behaviorFeedback givenDo simulationRead through feedbackFacilitate discussionReflect on outcome and behavior Group DiscussionObserve discussionClassify CompetenciesRate behaviorFeedback givenOne on One Read instructionsDo One on One Classify CompetenciesRate behaviorFeedback givenIntegrationSim scores and Behavioral scoresIntegrate scoringFeedback reportPerformance Matrix Discussion with ManagementClosure

Trained two groups of Observers and Administrators within client company to Administrate and deliver the various EAC

Trained Observers and Administrators within ET to Administrate and deliver the EAC

Selected and earmarked a few that will have to be appointed into the roles of Observer/Administrator.

Ring fenced those who could not

22

TechnicallyTechnically

• Involvement from IT

• Computer rooms

• IT infrastructure

• Delivery methods

•

23

Senior Managers Experiential CenterManagers Experiential CenterFirst line mangers Experiential CenterProject Managers Experiential CenterSales Managers Experiential Center

2007-20092007-2009Experiential Assessment Centers

•AC imbedded with all or various technologies

•Computerized Business Simulation

•automated report writing capabilities••Rapid development

•With a proper LMS

•Web Enabled

•Low on Band width

•Easy to build

•No Programming needed

•Customizable for any client industry specific

•Should still be an AC 24

SIM Development Methodology

Computer based Simulation Technology

Computer based Simulation Technology

Perspective Custodians

Developed assessment

criteria

Business analysis Learning Outcomes

Compiled scenarios and story lineCustodians write feedback, score each scenario against competencies

Quality ChecksDevelop Multi Media Assemble

� assessment center

� measures of performance

• Balanced Scorecard

• Formative Assessment

• Summative Assessments

� learning topics if any

� research and investigation on topic

� theory behind it

� creation of the story line

� validation to start

� multimedia

� compilation

� test and quality check

7

25

Energy Sector

26

Financial Institution

2004-20072004-2007

27

Insurance Industry

28

Manufacturing Tire Industry

8

29

Toughest partToughest part

Locate Most Appropriate answerID the PRIMARY competencies per answer

ID the Secondary CompetenciesDO that per answer per scenarioScoring : 10 point scale • +5 to - 5• All answer Primary competencies• Secondary Competencies• Tertiary CompetenciesIf an answer is MOST AppropriateCompetencies are scored highest +4, +5If answer is in the middle AppropriateCompetencies are scored towards the middle +2, +3 or -2 , -3

If answer is LEAST AppropriateCompetencies are scored lowest -4 , -5

Get SME’s and behavioral team to score and agree on each answer to each scenario

30

2007-20092007-2009

31

ExercisesExercises

On line exercises

Off Line exercises

Case Study

In Basket

Group Discussion

One on One Interview

Presentation

32

ChallengesChallenges

Culture

Functionally

Behavioral

Organizational Process

Technical

Legal

9

33

KNOWING YOUR CHALLENGESKNOWING YOUR CHALLENGES

Cultural

34

FunctionalFunctional

Complete in your own words

35 36

10

37

TechnicalTechnical

Intranet vs. IntranetBandwidth� Edge� GPRS� 3G� HDLMS or HRISSCORMMultimedia� Flash� Squeeze

38

Server and back up GeneratorServer and back up Generator

39

LegalLegal

40

Conducting AC and DC

with CBST

11

41

ON LINE ASSESSMENT CENTER PROCESS FLOW

TK Manager

Experiential Center

E-mail to candidate Reminder e-mailScheduled with candidate

Manager access On-lineManager completes Case StudyManager e-mails case study report to ETCandidate completes In Basket

Database update ET compiles report Report e-mailed to TK

same day next day if neededsame day

One hour same day 3 hours scheduled day

When done with simulation Next business day Within two business days

Interactive exercises

Two hours

42

Conducting the EACConducting the EAC

Your

4343

Exercises

4444

CBST on line:

• Analysis Problem – Project Plan and Charter

-- Business Plan or

-- Sales plan

• Integrated Simulated In-Basket

Customized Simulations could include:

• Group Discussion

• One-on-One Client or Employee Interview

• Presentation

• Technical Processes

• Product knowledge

ExercisesExercises

12

4545

Managerial Clusters

&

Competencies

4646

ORGANISATIONAL CompetenciesBEHAVIORAL

� Provision of Guidance

� Managing Diversity

� Interpersonal Sensitivity

� Social Responsibility

� Interpersonal Influence

� Presentation Skills

� Decision Making & Business

Acumen

� Client Focus

� Strategic Approach

� Financial Management

� Risk Management

� Global Orientation

� Conceptual Thinking

� Innovation

� Energy

� Process Skills

Cluster A:

Achievement Motivation

Cluster B: Conceptualizing

Cluster C:

Entrepreneurial & Commercial Thinking

Cluster D:

Business Skills

Cluster E:

Leadership Skills

Custer F:

Managing Relationship and Building

Networks

trade activation methodology

4747

Rating Scale

4848

Rating ScaleRating Scale

1 Training and Development needed

2 Development needed

2.5 Development with potential

3 Rounding-off needed

4C Adequate (Needs counseling)

4 Adequate (Norm: effective

management practice)

5 More than adequate

13

4949

on Line scenarios

50

51 52

Feedback

14

53

Feedback

54

Feedback

5555

Feedback to Individual

Feedback to Management on

results

Talent Management --- assess the

performance of individual in the

workplace

FeedbackFeedback

5656

15

5757

Per Exercise:

Simulation

Per Exercise:

Simulation

5858

Cluster: Leadership

Skills

Cluster: Leadership

Skills

59

Talent SolutionTalent Solution

Succession Planning

Workplace Diversity and

Skills Plan

Performance

Management

Results

Assessment

Centre Reports

Talent

Management

Training

Assessment

Results

Equity Plans

Individual Development

Plans

60

Overall resultsOverall results

1NOT FIT

2WEAK MATCH

3MODERATE MATCH

4FIT

5FIT WITH HIGH POTENTIAL

RatingOVERALL MATCH

16

6161PERFORMANCE HighLow

High

0 . 0 10 . 0

P ot en t ia l

R an k in g

1

Potential

Ranking

Limited

Potential

Potential

High

Potential

Talent MatrixComparing Results

Talent MatrixComparing Results

6262

RecommendationsRecommendations

Integrated Approach

Talent Management

Development of IDP’s

Solutions focused on

Innovation

Leadership for business results

Strategy Alignment and implementationE.G Client relations vs. Client care

6363

Research

CORRELATION STUDIES

64

CORRELATION STUDIESCORRELATION STUDIES

Correlation between Behavior vs. Organizational

Correlation with Organizational Scorecard

Performance

Correlation between Technology and Behavior

Comparing results with Psychometric instruments

Correlation Pre and Post Assessment

Correlation between Technology : Psychometric

results and overall outcome

17

65

MethodologyMethodology

Assessment (two days)

Inclusive of all races, gender and ages

Middle Management to Senior Management

Ratings on all exercises

On-Line Assessment

Integration

Integration of Behavioural Assessments

Correlation with Psychometric Instruments

Actual correlations

66

Inter Correlation between

Technology and Behavior

Inter Correlation between


.68.87Interpersonal skills

.68.87Energy

Selection in

Simulation

Observed

.98.96Analytical thinking

.96.96Decision-making

Business Acumen

N = 403

67

Actual Correlation

On Line Assessment

Actual Correlation

On Line Assessment

Ranking correlation (n=403) (p≤.05)

Behavioral (Analysis, Group, One-on-One,

Presentation) : On-Line r = .85

Case Study and On-Line : Total Assessment

r = .89

68

Comparing resultsComparing results

Compare Data with OPQ (SHL instrument) previously done

Divide competencies in three categories

EXTREME - HIGH -MODERATE IMPACT ON ROLE

SHL to also identify EXTREME - HIGH MODERATE- impact competencies for the role separate from us

Compare scores on OPQ , SIM and Behavioral

Apples with Apples

18

69

Agreement on competencies

importance

Agreement on competencies

importance

.95Moderate job match competencies

correlation

.95High job match competencies

.98Extreme job match competencies

70

Inter correlation between


Inter correlation between


.68.87Interpersonal

skills

.68.87Energy

Selection in

Simulation

Observed

.98.96Analytical

thinking


Business

Acumen

71

Correlation between Technology

other psychometric results and

overall outcome

Correlation between Technology

other psychometric results and

overall outcome

.68.87Interpersonal skills

.68.87Energy

OPQEAC

.98.96Analytical thinking


Business Acumen

72

Client Projects

19

73

CLIENT EXPERIENCESCLIENT EXPERIENCES

SADEC, NEAD and ASIA specific regions

strategy to appoint quality leaders in each country

confirmation on already identified talent

enabler to advance on management development programs

right level with right competence – Assessment

assessment for respectively development and fit for purpose

leadership pipeline identification

enhance performance

The key challenges they are faced with were;

� Spending money correctly

� Saving on development costs

� Focused on individual development as competencies were assessed

� The right people cap turnover

� Matching personal competencies with organizational needs

74

COKE OVERALL COUNTRY RANKINGS (N=213)

0.0

20.0

40.0

60.0

80.0

100.0

120.0

Achievement

Motivation

Conceptual

Thinking

Business

Skills

Entrepreneurial

Commercial

Leading and

Working with

People

M anaging

Relations and

Building

Networks

Clusters

Percentile Sri Lanka

Nepal

Vietnam

Cambodia

South Africa

75

020406080100120

Percentile

Provision of

guidance

Diversity Interpersonal

Senstivity

Social

Responsibility

Competencies in cluster

COKE : Leading and working with People Cluster : N= 213

Sri Lanka

Nepal

Vietnam

Cambodia

South Africa

76

0.950.8610.655Maximum,

-0.199-0.453-0.056Minimum

0.1000.3140.152Average

Different

competencies,

different methods

(expected to be

the lowest)

Different

competencies,

same method

(would reflect

method bias)

Same competency,

different methods

(expected to be the

highest)

20

77

South Africa

78

01234567

Persentage of

N=213

Limited

Potential

Potential Potential

Plus

High

Potential

Potensial Scale

Coke Persentage Potential

Sri Lanka

Nepal

Vietnam

Cambodia

South Africa

79PERFORMANCE HighLow

High

COKE Talent Matrix Discussion Comparing Results

COKE Talent Matrix Discussion Comparing Results

80


quality leaders to execute their strategy

required real role clarity , that is the alignment of roles with the business


Assessment for respectively development and fit for purpose

experiential learning

pipeline identification

The key challenges many organizations and themselves were faced with were;

� Spending money correctly with pipeline promotions


� Development is now focused

� Focused on individual development and competencies were assessed

� Utilizing skills according the competencies

� Reduce the costs of the wrong person on a job

� The right people cap turnover

� People have a choice “Do I want this role or not?”

� Getting to know my self and my key preferences

� Matching personal competencies with role competencies

21

81

Service Advisor N = 403Service Advisor N = 403

82Local Marketing Orientation

Sales Orientation in the branch

Branch Sales and Service

Planning

SALES

PERSPECTIVE

Decision-making

Planning, Organising and

StrategisingADMIN SKILLS

Self DevelopmentPresentation Skills

CommunicationInterpersonal Influence

COMMUNICATION SKILLS

Development of peopleSensitivity

Performance ManagementDecisiveness

Mentoring

LEARNING AND

GROWTH

Task Structuring

Passion for service excellenceUtilisation & Development

LEADERSHIP SKILLS

Bank policies and procedures

INTERNAL

PROCESSES

Flexibility

Analytical Thinking (Branch

Managers)

Decision-making and Business

Acumen

Branch Resource ManagementAnalytical Thinking

DECISION-MAKING SKILLS

Business Acumen (Branch

Managers)Energy

Financial Management of the

branch

FINANCIAL

PERSPECTIVE

Innovation

PERFORMANCE

MOTIVATION

54321ORGANISATIONAL

COMPETENCIES54321

BEHAVIOURAL

COMPETENCIES

Correlation of the Behavioral and Organizational CompetenceCorrelation of the Behavioral and Organizational Competence

83 84

Service Advisor N = 403

22

85


86

.032Var(Error)

.264Var(person *

Competency *

Method)

.118Var(Competency *

Method)

.119Var(person * Method)

.007Var(person *

Competency)

.125Var(Method)

.061Var(Competency)

.043Var(person)

EstimateComponent


87


88


Relationship Manager's Role in

New challenges in Broker market

Position changed from Admin to Business partner


enabler to advance on management development program

confirmation on already identified talent

assessment for respectively development and fit for purpose

leadership pipeline identification

enhance performance

The key challenges they are faced with were;

� Spending money correctly during recruitment process


� Focused on individual development as competencies were

� To match the different Relationship Manager with the broker segments and business partners

World cup 2010 Project

23

8989

Insurance Industry (113)Insurance Industry (113)

0

20

40

60

80

100

120

1

90


0

0.5

1

1.5

2

2.5

3

RATING SCALE

Behavioral Organisat ional

CLUSTERS

SANTAM RELATIONSHIP MANAGER PERFORMANCE MATRIX N= 113

91

SANTAM RELATIONSHIP MANAGERS PERFORMANCE ON SCORECARD

PERSPECTIVES

00.51

1.52

2.53

3.54

4.5

Strategic

Relationship

Broker

Practice

Broker

Development:

Broker/

Practice

Cross- Up-

selling, Unit

Growth

Rentention

Profit

Self

Development

People

Development

Leadership

and

Teamwork

Marketing:

Business

Plan, Sales

Plan, Sales

Maintain

Brokers

Growing

Brokers

CLIENT

PERSPECTIVE

FINANCIAL PERSPECTIVE RESOURCES PERSPECTIVE INTERNAL PERSPECTIVE

Perspectives

Rating Scale


92


SANTAM RELATIONSHIP MANAGERS

BEHAVIORAL PERFORMANCE

0

0.5

1

1.5

2

2.5

3

3.5

INNOVATION

ENERGY

PROCESS

SKILLS

CONCEPTUAL

THINKING

DECISION

MAKING &

BUSINESS

STRATEGIC

RELATIONSHIP

CLIENT FOCUS

RISK

MANAGEMENT

GLOBAL

ORIENTATION

CHANGE

MANAGING

DIVERSITY

SOCIAL

RESPONSIBILITY

INTERPERSONAL

SENSITIVITY

INTERPERSONAL

INFLUENCE

PRESENTATION

ACHIEVEMENT MOTIVATIONCONCEPTUALISATION ENTREPRENEURIAL & COMMERCIAL LEADING & WORKING WITH PEOPLE MANAGING

RELATIONS &

BUILDING

NETWORKS

Behavioral Clusters

Scale

24

93

Value of CBST in AC and DC

94

Simulations add value in

the following manner:



Improve a assessee’s understanding

Facilitate communication and debate

Allow managers to prioritize

Improve understanding of an industry’s

dynamics

95





understanding of cause-and-effect

relationships

make more effective decisions

Allow managers to complete a what-if

analysis

demonstrate leadership strengths96

UniquenessUniqueness

25

97

The ValueThe Value

The most significant value can be described in three words:

experience through experience

98

Critical success factors

99

Critical success

factors

Critical success

factors

• Commitment and involvement

• Consequence management

• Coaching - willingness and involvement

• Sustainability of development

• Customized learning solutions that address business and

individual needs

• Relevance of learning

• Strategic alignment

100

Implication for us Implication for us Project Management Skills� Right project and project right� Stakeholders management� Scope creeping� Q T Budget vs. competitiveness

Technology Skills � E learning� Managing IT people� Technology changes� Virtual hosting

Research capabilities� Validity� Reliability � country and group norms

Process driven vs. better business results� Results between organizational scorecards and AC outcomes

26

101

Debate QuestionsDebate Questions

Are trade activation here to stay? Trade activation Organizational- vs. Behavioral-Competence measured

� Reliability of 95% ?

Ethical issues of technology assessment centers

� high performance teams

Process measurements vs. results measurement

Potential vs. Competence?

� Fairness vs. predicative validity and Team competence

102

In conclusion In conclusion

Technology helped us to simulates real world issues and global practices and processes

Aligning business workflow and process across the organization and countries

Establish best practices and learning

Testing of these practices and knowledge

Sharing of knowledge between member countries

Teaches Business Acumen

Create awareness of Big picture

103

cont..cont..

Pre course attendance

Determine the growth potential

Targeted development (individually)

Determine a risk profile of people

Assessment as well as learning through

experience

Supports the new organizational culture

Enable change management

Competitive advantage

104

DemoDemo

Computer based Simulation Technology as part of the AC · PDF fileComputer based Simulation...

Documents

Transcript of Computer based Simulation Technology as part of the AC · PDF fileComputer based Simulation...