Common Measureme nts the Tool Guy Gordon Assistant Deputy Minister Service Manitoba Manitoba Lois...
-
Upload
roxanne-cunningham -
Category
Documents
-
view
213 -
download
0
Transcript of Common Measureme nts the Tool Guy Gordon Assistant Deputy Minister Service Manitoba Manitoba Lois...
Common Measurements
the
Tool
Guy GordonAssistant Deputy Minister
Service Manitoba
Manitoba
Lois BainDirector, Quality Service
OPS Restructuring Secretariat,
Cabinet Office
Customer Feedback - CMT 2
Today’s Objectives
The importance of understanding customer needs
What is the Common Measurements Tool (CMT) What is it Who is using it Tools to help you to collect customer data Lessons Learned
New generations of the CMT Come join us on the journey!
Customer Feedback - CMT 3
Public OpinionHow Canadians View Customer Service (general)
DELIVERYTURN-OFF
PEOPLE TURN-OFF
PEOPLE EXCELLENCE
DELIVERYEXCELLENCE
Banks 70%
Large Retail 66%
Gov’t 40%
Small Retail 86%
Pharmacies 92%
*National Quality Institute, Nov. 1997
Source: National Quality Institute Survey 1997
Customer Feedback - CMT 4
Citizens Rate Recent and Specific Services Higher
Serv
ice
Qu
alit
y (0
-100
)
64
62
60
64
62
61
58
59
60
61
62
63
64
6553
47 47
57
50 51
Municipal Federal
0
10
20
40
50
60
1998
2000
Municipal Provincial Federal
0
Recent and Specific Experience
General
Provincial
Citizens First 2000
Customer Feedback - CMT 5
Redrawing Maps- Citizens First 2000 Citizen Satisfaction - Canada
Fire departmentsPublic librariesSupermarketsPoliceProvincial/Territorial parks, campsDepartment storesTelephone companiesCanada PostTaxisRevenue CanadaHospitalsBanksCable television companiesRoad maintenance
797772 6665 616159565452514940
SERVICES SATISFACTION4 of the top 5 services are Public Services
>>
>>
Customer Feedback - CMT 6
Citizen needs and expectations: (EKOS led Study)
Do citizens want to talk to government about what they want? YES: Citizens perceive an influence gap:
citizens have too little and big business and the media have too much.
Do citizens want to talk about service delivery? YES: Citizens believe it is important for the
government to engage citizens about: service delivery (82%), policy development (75%), governance (74%), and program design (73%).
Customer Feedback - CMT 7
The Cost of Quality...
“The Good”
““The Bad”The Bad” “The Ugly”
•people•material•overhead•etc.
•training•systems improvements
•gathering & using client feedback
Fixing errors BEFORE they reach your
clients
Fixing errors AFTER they reach your
clients
COST = Production + Prevention + Inspection & + Field Problem Costs Costs Correction Costs
Costs
(1X) (10X)(10X) (100X)
30 - 50% of costs intypical service organizations
Customer Feedback - CMT 8
Citizen-Centred Service Network
Citizens FirstErin Research Inc
1998 IPACGold Award
2000 CAPAMSilver Medal
Common Measurements Tool
FayeSchmit
A network of Federal/Provincial/Municipal civil servants dedicated to improving the quality of public service in Canada
Customer Feedback - CMT 9
Objectives of the CMT:
provide a ready-made tool for public service organizations
enable like-organizations to compare results
facilitate the sharing of information gained and lessons earned between organizations
enable organizations to build internal benchmarks
Customer Feedback - CMT 10
How high is up? The need for common measures
0 6Very Dissatisfied Very SatisfiedSatisfiedDissatisfied
1 4Poor ExcellentGood Very good
1 10Poor Fair Good
90% Satisfied and Very Satisfied
90% Satisfied and Very Satisfied
=
What does the CMT Measure?
Central core of standard questions and standard 5-pt measurement scales
Client feedback (not citizen level)
Five elements of the service experience in the five dimensions or areas of the service experience
Additional questions customized by the organization to met their needs
Customer Feedback - CMT 12
Elements of the Service Experience:
Client expectations
Client perceptions of service experience
Level of satisfaction
Level of importance
Priorities for Improvements
Customer Feedback - CMT 13
Understanding Service Gaps:
Definition of Service Gap:
The difference between client expectations and the client’s perception ofthe service experience.*
*Source: Zeithaml, Valerie. et al., 1990
Expect-ations
Service
Gap
Customer Feedback - CMT 14
BC Office Products Centre
Low High
Hig
h
Areas where priorities should be focussed
•Products arrived on time
•Prompt handling of customer complaints
Organizational strengths
•Courteous and helpful staff
•Same day delivery service
Lo
w
Low Priority
•Personal visits by customer service reps
Unnecessary strengths - possible overkill
•On-line electronic ordering
•Electronic billing
IMP
OR
TA
NC
E
SATISFACTION
Customer Feedback - CMT 15
Service Dimensions:
Responsiveness
Reliability
Access & Facilities
Communications
Costs
Customer Feedback - CMT 16
Service Dimensions:
Responsiveness
Reliability
Access & Facilities
Communications
Costs
• timeliness
• # of contacts to get service
• courtesy
• flexibility
• fairness
• competence
• etc.
Customer Feedback - CMT 17
Service Dimensions:
Responsiveness
Reliability
Access & Facilities
Communications
Costs
• outcome
• minimal error rate• adherence to policy & standards• etc.
Customer Feedback - CMT 18
Service Dimensions:
Responsiveness
Reliability
Access & Facilities
Communications
Costs
• convenient location
• telephone access
• hours of service
• variety of access modes
• etc.
Customer Feedback - CMT 19
Service Dimensions:
Responsiveness
Reliability
Access & Facilities
Communications
Costs
• availability of info
• official languages
• questions answered
• etc.
Customer Feedback - CMT 20
Service Dimensions:
Responsiveness
Reliability
Access & Facilities
Communications
Costs
• ease of payment
• reasonable cost
Customer Feedback - CMT 21
How Does the CMT Work?
Offers a consistent measurement system
Offers comprehensive item bank for selection
Offers descriptive and Likert types of questions
Requires customization by organization
Allows addition of questions by organizations
Customer Feedback - CMT 22
e-CMT Overview
Customer Feedback - CMT 23
e-CMT - How do I use it?
Step 1 The CMT Intranet web site Step 2 Design survey on-line Step 3 Administer survey Step 4 Load data Step 5 Analyze results
Customer Feedback - CMT 24
e-CMT - Two Parts
Survey Design&
Database(Internal Server)
Survey Analysis&
Reporting(Desktop Software)
Data and Surveys
Web
bas
ed s
up
po
rt
Customer Feedback - CMT 25
e-CMT on Manitoba Intranet
Customer Feedback - CMT 26
Design your survey on-line
Customer Feedback - CMT 27
CMT DEMO
Run DEMO here.
Customer Feedback - CMT 28
Customer Service Support
User Manual FAQ Help Files CD ROM Tutorial EDS Helpdesk
Customer Feedback - CMT 29
1. Surveying Guidebook:
Takes a surveyor through: planning constructing conducting analyzing and acting on surveys.
Customized for the CMT by the Province of Ontario with permission from the USA’s Environmental Protection Agency (EPA).
Customer Feedback - CMT 30
3. CMT Publications:
Client Satisfaction Surveying: Common Measurements Tool
Client Satisfaction Surveying: A Manager’s Guide
These (and other useful service publications) are available online at:
www.ccmd-ccg.gc.ca/publica/publi.html
(located under the service quality theme)
CommonMeasurementsTool
Faye Schmidt withfor the
ANAGEMENT DEVELOPMENT
Teresa Strickland
Customer Feedback - CMT 31
Training
Applied Research Methods Delivered by U of W 2-Day Training
Course Co-developed MB-
Canada Experimenting with
on-line Training
Customer Feedback - CMT 32
Who’s been using thee-CMT in Manitoba
Department Natural Resources Education & Training
STEP, Stevenson, Career Start
Family Services Regional Op’s
Labour Agriculture French Language Services SOA’s Auditor General
Department Natural Resources Education & Training
STEP, Stevenson, Career Start
Family Services Regional Op’s
Labour Agriculture French Language Services SOA’s Auditor General
Type of Survey paper
paper paper
paper & electronic
paper, telephone, in-person electronic electronic
kiosk
Type of Survey paper
paper paper
paper & electronic
paper, telephone, in-person electronic electronic
kiosk
Customer Feedback - CMT 33
e-CMT examples: Manitoba
Student Temporary Employment Services
• initial CMT survey identified “information” as largest value/satisfaction gap 2000
• focus groups confirmed application process as major difficulty for students (access, confirmation)
• STEP application put on-line 2001
• 94% of applications now received on line
• e-CMT telephone survey indicates 97% satisfied or very satisfied
• 2001 focus groups identified managers want requests via Intranet.
Customer Feedback - CMT 34
e-CMT examples (2)
Manitoba Child and Family Services• Regional operations
• Round 1 : CMT administered to direct clients via telephone interviews
• Survey confirmed clients generally satisfied but find access and program information as priorities for improvement
• Round 2 CMT delivered to “partner” agencies
• Survey revealed “partners” generally satisfied
• Managers indicate major benefit was to demonstrate and reinforce positive performance to staff
Customer Feedback - CMT 35
Other Jurisdictions
Government of Canada• Service Improvement
Initiative
Ontario B.C. PEI Quebec City of Winnipeg City of Toronto
Customer Feedback - CMT 36
CMT Implementation
Approximately 50 CMT Surveys completed Federal Government , Manitoba, Ontario, B.C., PEI
(Social Services to Internal Travel) Mail, Telephone, In-Person Interview, Internet Macro - Organizational - Business Planning Micro - Client Segments focusing on specific operational
issues.
Innovation/Custom Applications Employee Attitudes Salary Surveys/Competencies
Customer Feedback - CMT 37
Ministry of Labour Ontario
60
65
70
75
80
85
90
95
100
60 65 70 75 80 85 90 95 100
Percentage - Important/Very Important
Per
cen
tag
e -
Sat
isfie
d/V
ery
Sat
isfie
d
Time
Courtesy
Competence
Fair and Equitable
Informed
MEAN (S)
Opportunities – AREA FOR FOCUS
Exceed
Other Opportunities
Maintain
Customer Feedback - CMT 38
Lessons Learned
Project Management Jurisdictions have under estimated resources needed to support CMT/e-
CMT Slow roll-out based on pilot projects more successful Training is essential to ensure organizational readiness
Technical Customized solutions may be necessary “Firewall/Security” issues must be
dealt with immediately - internal servers Distribution of software should be centralized
Providing Customer Support is Crucial Survey design,administration, analysis e-CMT utilization
Customer Feedback - CMT 39
Lessons Learned (2)
Based on evaluation of CMT for use for
Internal Services (conducted for CCMD) CMT Questions need to be modified for
Internal Services CMT works best where dept’s have access to
technical experts re: survey design and administration
e-CMT correct way to go, but needs to be more flexible
Customer Feedback - CMT 40
CMT - Ongoing development
CMT Standards Board coordinating development of expanded CMT on-going revision of CMT based on feedback
received to date expand CMT question bank to include other
service delivery channels (telephone, mail, internet, kiosk)
Customer Feedback - CMT 41
CMT by Channel
Federal TBS and Manitoba testing new CMT questions designed to evaluate channel specific services: Telephone Service On-Line Services Internal Service Delivery
What drives ratings of government web sites?National results
1.Easy to find what I'm looking for
2 . Outcome: I got what I wanted
3. Visually appealing
4. Has the information I need
5. Pages load quickly
These five drivers account for 41% of the variance in citizens’ ratings of government sites.
Customer Feedback - CMT 42
Interest in ESD Evaluation is Growing
Post 9/11 fiscal environment ESD proponents are being challenged
to demonstrate “benefits” ESD project sponsors looking to
quantify benefits and success stories ESD project managers need
information re: success, maintenance, and future development of applications
Customer Feedback - CMT 43
ESD Performance Framework
CustomerSatisfactionMeasures
Application &Service RelevanceMeasures
ActivityMeasures
Custom orApplication-specificMeasures
Value Measures
Customer Feedback - CMT 44
Tools & Techniques
Focus Groups Interviews On-Line Survey The Common
Measurements Tool
Customer Feedback - CMT 45
Future of CMT
The International Institute for Citizen-Centred Service Affiliated with IPAC Lines of Business
• Establishing a CMT benchmarking centre • The development and marketing of the electronic
CMT• Intergovernmental research projects and learning
events• Establishing a best practices database re:service
delivery
Customer Feedback - CMT 46
CMT
Questions and Answers
…Thank you