© fedict 2005. All rights reserved User satisfaction & impact measurement Belgian practice...
-
Upload
hannah-franklin -
Category
Documents
-
view
215 -
download
0
description
Transcript of © fedict 2005. All rights reserved User satisfaction & impact measurement Belgian practice...
© fedict 2005. All rights reserved
User satisfaction & impact measurement
Belgian practice Description using the Template
Christine Mahieu, Business Analyst FEDICT
24 october 2008
© fedict 2005. All rights reserved 2
Measurement initiatives
2004 : Fed-eView / Administration • Measuring the degree of back-office development
of the federal administrations and their egov readiness 2005-2006 : Fed-eView / Citizen
• Understanding the needs of the citizens 2007 : eGov Measurement Framework Design
• Based on international best practices • Integrated instrument to continuously monitor and
benchmark the development of ICT and eGovernment in Belgium
• Partnerships are essential (Statistics department, regions, municipalities, EU, private partners)
2008 : Implementation of the framework• Consolidation of the dispersed existing indicators• Development of new indicators and collect data in
cooperation with other public and private actors
© fedict 2005. All rights reserved 3
Fed-eView Citizen A strategic decision support tool
To assess the citizen’s needs, perceptions and use of electronic government, covering also ICT usage and security requirements
To monitor trends and evolutions Methodology :
• Quantitative survey : Panels from both (internet user /non-user) populations of citizens were followed in several waves over 15 months in 2005-2006
• Qualitative survey : Focus groups (for citizens and for entreprises) provided a list of potential applications to be developed end 2006-begin 2007
© fedict 2005. All rights reserved 4
A. General properties Repeatability
Yes Independence of size
Maybe, not tested Estimated cost of using the methodology
About 125.000 euros for 3 waves of survey and 8 focus groups Prerequisites to carry out and use the measurement
eGov is a horizontal matter, an enabler. Satisfaction about egov services or about public service ?
Stability Yes, this is the primary objective of the methodology used
Accuracy Other surveys (like ICT Household survey - urostat) confirm the results
Conformity to WAI and W3C standards Not a requirement
Number and type of administrations using the method FEDICT and the Ministry of Economy/DG Statistics for some indicators
Multidimensionality • Quantitative survey : 2 types of citizens (internet users and non-users)• Qualitative survey : 2 types of users (citizens and entreprises)
© fedict 2005. All rights reserved 5
B. Intended use of measurements results
1. Who are the intended users?• Policy makers• Researchers • Public
2. Measurements what for • To assess current status and to monitor
trends and evolution• No measurement on impact
© fedict 2005. All rights reserved 6
C. Deployment properties
1. Who carries out the measurement?• Outsourced to a neutral third party (team of 3
people + research bureau) 2. How is the measurement carried out?
• Use of a ´mixed mode survey´with two random samples:
• An online panel representative for the Belgian Internet population interviewed in 3 waves via CAWI (panel members were invited by e-mail to participate)
• An offline panel representative for the Belgian internet non-users interviewed in 2 waves via CATI.
• Frequency is used as the criterion of internet use: a minimum of once a month use of the internet.
© fedict 2005. All rights reserved 7
C. Deployment properties
3. Measurement coverage: subset or exhaustive sampling ?
• The 2 panels constitute representative samples of the Belgian populations of internet users and internet non-users.
• Note: Belgian population is about 10 millions. Online panelInternet users
Telephone panelInternet non-users
Wave I N = 3324 N = 1421
Wave II N = 1480 N = 624
Wave III N = 1139 -
© fedict 2005. All rights reserved 8
5. Testing No specific test
8. Score calculation and statistics used Percentages Scale of satisfaction : from 0 (absolutely dissatisfied) to 10
(absolutely satisfied) Use of quadrant for combining indicators :
C. Deployment properties
© fedict 2005. All rights reserved 9
Comparisons, e.g. time series or needs and wants/actual perception Yes
C. Deployment properties
Possession of internet at home 2005
Possession of internet at home 2006
Use of internet 2006%
Yes No No 2.2
Yes Yes No 2.5
Yes No Yes .6
Yes Yes Yes 11.9
No No No 52.0
No Yes No 1.4
No No Yes 14.6
No Yes Yes 14.9
Total 100.0
© fedict 2005. All rights reserved 10
9. Reporting of the evaluation results The results are summarised in a public
paper in 4 languages (on ePractice) Access to raw data will be possible when
the eGov Monitor will be implemented
C. Deployment properties
© fedict 2005. All rights reserved 11
> Does the measurement lead to policy actions? • Partially : definition of citizen personalised one-
stop-shop and eBox• But no optimum use of the results
> Lessons learned • Use of panels
• Low-cost way of citizen’s feedback. • Opportunity to go in-depth in the subject• Not useful to measure impact of specific communication
campaigns• Decrease representativity in 15 months time
• Focus groups: selection of people not easy• Measuring the satisfaction of eGov service or of
public service ?
D. Experience from using the methodology
© fedict 2005. All rights reserved 12
Maintained in the context of the global measurement framework eGov Monitor
E. Maintenance of Measurement Method