Subjective and Objective Performance Measures The Case of Local County E-Government

 

Dr. James McQuiston, University of AkronDr. Aroon Manoharan, University of Massachusetts Boston

Midwest Public Affairs ConferenceColumbus OH, June 4, 2016

This paper seeks to examine the relationship between subjective and objective measures of performance using a sample of US county governments.

The research also determines the factors affecting both types of performance measures.

The objective measures were obtained from a content analysis of their official websites, while the subjective measures were based on a survey of the county IT administrators.

The research addresses an important question concerning the value and performance of governments, and the relationship between subjective and objective performance measures of government.

Research Focus

E-Government is “the use of information technology to support government operations, engage citizens, and provide government services via the

Internet”

What is E-Government?

Benefits of E-Government: increased government accountability to citizens greater public access to information more efficient, and effective government

Potential of E-Government: A major transformation of the way the government conducts

business Reversal of long-term loss of social capital

Transformation to E-Government

E-Government Paradigm • coordinated network building • external collaboration• customer services

Traditional Bureaucratic Paradigm• standardization • departmentalization• operational cost-efficiency

Stages of E-Government Development

Layne, K. and Lee, J. (2001). Developing full functional e-government: A four stage model. Government Information Quarterly. 18:122-136.

E-Government Success Measures

Two important measures of ‘Success’– - Hard measures – cost benefit analysis, benchmarks

- Soft measures – improved decision-making, citizen/customer satisfaction, employee productivity

Challenges to E-Government

Three types of e-government failures (Goldfinch, 2007) Project failure, system failure, user failure

Reasons for e-government failures (Sarantis, 2007)1. Goal definition of public organizations is usually vague, compared to the private

sector.2. The analysis of the e-government projects indicate that their structures are more

complex with interactions from the technical, managerial, political, social, cultural, and legal dimensions.

3. Planning for IT projects in the public sector is much more instable, largely due to the electoral process.

4. The management of government transformation projects in e-government cuts across agencies, and there are few best practices exclusively used for the public sector.

5. The legal and regulatory issues in the public sector come from legal and constitutional arrangements.

As governments incorporate various technologies, it is important to measure the performance of e-government and develop strategies to improve such performance.

Much of the studies on e-government performance tend to focus primarily on the official websites of cities and states, while some studies are based on survey of public administrators involved in the e-government development process.

However, would adopting both methods result in wide variances in performance measures for local governments?

Subjective and Objective Performance Measures

What is the relationship between objective and subjective measures of e-government performance?

What are the factors affecting both types of performance measures?

Does higher e-government performance lead to higher managerial satisfaction?

Research Questions

No. Dimension Features

1 E-information

Information on public employees, location of government offices, GIS mapping, FAQ (Frequently Asked Questions), job/position vacancies calendar of events, video clips/minutes of public meetings, targeted audience links

2 E-transactionUtility bills, fines/tickets, filing taxes, books/reports purchase, permit applications, event tickets licenses applications, e-procurement

3 E-participationOnline policy forums, bulletin boards, e-petitions, e-meetings, community newsletter, online survey polls, crime report, file complaints, feedback forms

E-Government Performance Measures

The highest possible e-government score for any one county is 72, while the highest possible score on each of the three dimensions is 24.

• Content analysis of official websites using a 24 point index • Measures coded on a scale of 0, 1, 2 or 3 based on the levels of sophistication

MethodologyDependent Variables

10

MethodologyIndependent Variables

- Survey sent to county Chief Information Officers/IT managers between March and May 2009.

- NACO, has a membership of over 2,000 counties representing over 80% of the national population, maintains a comprehensive database and periodically gathers and disseminates information on both counties and county officials. - The survey was administered via the Internet, which has the advantage of both cost and time savings over regular mail surveys involving significant printing and mailing resources.

- The survey was divided into two main sections: (institutional variables + contextual variables)

- The survey also included two questions assessing county managers’ success and satisfaction with

the implementation of e-government services.

- Questionnaires for the online survey were emailed through Survey Monkey to counties across the United States that have official websites.

- Overall survey response rate of 17.8%11

County Adoption of Official Websites

12

- Identification of counties with websites by accessing the corresponding links on the NACO website.

- The absence of county website confirmed through a query to the Google search engine.

- Overall, a total of 2,376 counties have official websites.

Percentage of Counties with Official Websites by Region

The states with the highest percentage of county website adoption (100%) are Arizona (15), California (58), Delaware (3), Florida (67), New Hampshire (10), New Jersey (21).

Difference GroupsGroup A – Respondents that had a greater actual than perceived e-government score (n=37).

Group B – Respondents that had an greater perceived than actual e-government score (n=145).

Regression Analysis Results – Actual Performance v Perceived Performance

Variables/Parameters Scale

Form of Government1 = Board of Commissioners (BC), 0 = non-BC

IT Budget 1 = Above $500,000, 0 = $500,00 and below

IT Contracting 1 = Yes, 0 = No

Stakeholder Support (Elected Officials)1 = Not supportive to 4 = Extremely supportive

Stakeholder Support (Top Administrators) 1 = Not supportive to 4 = Extremely supportive

Stakeholder Support (County Employees)1 = Not supportive to 4 = Extremely supportive

Organizational Size (No. of Functions)  

Neighboring Websites 1 = Yes, 0 = No

External Collaboration (Counties) 1 = Not that closely to4 = Extremely closely

External Collaboration (Nonprofits) 1 = Not that closely to 4 = Extremely closely

External Collaboration (Businesses) 1 = Not that closely to 4 = Extremely closely

Population US Census Data

Education US Census Data

F  Constant  

Adjusted R-Square  *p<0.10; **p<0.05; ***p<0.01

1

2

3

4

5

6

7

8

9

10

0% 2% 4% 6% 8% 10% 12% 14% 16% 18% 20%

Self-reported Success of

E-Government

Overall, how would you rate the success of your county in the area of e-government? (1 = “Not successful at all” to 10 = “Very Successful”)

County’s Self-Reported Success in Implementing E-Government

Research Question 1Correlation of Subjective Measures and Objective Measures of Performance

    E-Information E-Transaction E-Participation E-Government Self-Reported Success

E-InformationPC 1

 Sig. (2-tailed)

E-TransactionPC 0.224**

1 

Sig. (2-tailed) 0.002

E-Participation

PC 0.100 0.301**

1

 Sig. (2-tailed) 0.177 0

E-GovernmentPC 0.525** 0.913** 0.516**

1 

Sig. (2-tailed) 0 0 0

Self-Reported Success

PC 0.239** 0.237** 0.032 0.268**1 

Sig. (2-tailed) 0.001 0.001 0.664 0

**. Correlation is significant at the 0.01 level (2-tailed).*. Correlation is significant at the 0.05 level (2-tailed).

Research Question 2Factors Affecting Objective Measures of Performance

Performance – Actual Coefficient (s.e.)

(Constant) -8.395 (5.594) CountyYears .229 (.120)* IT Department .643 (.295)** IT Employed .397 (.117)** County Form Govt .061 (.194) EGovtSupport - County -.091 (.160) EGovtAdminSupport .213 (.164) EGovtEmploySupport .392 (.162)** EGovtCitizenSupport -.271 (.143)* ChampionEGovt -.197 (.201) ITBudget -.094 (.096) NeighboringCtyWebSite .078 (.091) ITWorkOutsource .076 (.098) PrivateNonfarmEstablish -1.115E-005 (.000) PopulationLog .228 (.153) HSEducation 1.243 (1.192) Satisfaction .018 (.551)

Research Question 2Factors Affecting Subjective Measures of Performance

Perceived – Group B Coefficient (s.e.)

Constant 5.781 (.60)***

ITWorkOutsource .228 (.14)

NumberSourcesCtygovt -.040 (.03)

ChampionEGovt -.582 (.32)*

ITEmployed .277 (.12)** *=p<.10, **=p<.05, ***=p<.01

Perceived – Group A Coefficient (s.e.)

Constant .612 (1.12)***

ChampionEGovt 1.036 (.35)***

ClosenessCountyCollab -.285 (.18)

CountyFormGovt .702 (.40)

CountyYears .370 (.24)*

EGovtEmploySupport -.770(.33)**

ITEmployed .444 (.11)***

EGovtAdminSupport .310 (.28) *=p<.10, **=p<.05, ***=p<.01

Research Question 2Factors Affecting the Difference between Objective and Subjective Performance

Difference Coefficient (s.e.)

Constant 15.65 (3.90)***

Population -.43 (.29)

NeighboringCtyWebSite -.31 (.20)

HighSchoolGrad -.03 (.03)

NumberSourcesCtygovt -.15 (.04)***

EGovtCountySupport -.41 (.29)

EGovtEmploySupport -.42 (.32)

PrivateNonfarmEstablish 4.60-5 (.00)*

ITBudget -.22 (.16) *=p<.10, **=p<.05, ***=p<.01

Satisfaction

Can the below measures be considered satisfaction measures or do these questions measure perceived performance?

    E-Information E-Transaction E-Participation E-Government

Resulted in saving of financial resources

PC 0.174* -0.013 -0.058 0.033

Sig. (2-tailed) 0.018 0.863 0.434 0.662

Increased government transparency and

accountability

PC 0.150* 0.150* 0.012 0.167*

Sig. (2-tailed) 0.043 0.044 0.869 0.025

Increased collaboration with other levels of government

PC 0.039 0.007 -0.096 -0.006

Sig. (2-tailed) 0.599 0.924 0.197 0.936

Enhanced the county’s ability to hire, manage

and oversee contractors

PC 0.200** 0.232** 0.024 0.249**

Sig. (2-tailed) 0.007 0.002 0.749 0.001

Increased the interactions between citizens and county government

PC 0.206** 0.134 -0.049 0.157*

Sig. (2-tailed) 0.005 0.071 0.508 0.034

**. Correlation is significant at the 0.01 level (2-tailed).*. Correlation is significant at the 0.05 level (2-tailed).

Research Question 3Performance v Satisfaction

Does a higher e-government performance lead to a higher managerial satisfaction?

Research Findings

Actual performance is impacted by shareholder support, the presence and size of IT department, and the experience that the county has had with ICT.

Perceived performance is influenced in both groups by the existence of an e-government champion, but the variable possesses different signs. IT size matters in perceived performance, while the county's experience with ICT and support levels of a workforce towards e-government has a varied effect depending on the sign of the difference variable.

Different variables matter in explaining the difference between perceived and actual levels of e-government success; the number of sources of county government and amount of private firms are significant.

Questions?

Thank you