10.1.1.4.655.pdf

7/28/2019 10.1.1.4.655.pdf

http://slidepdf.com/reader/full/10114655pdf 1/9

271Proc. 10 h Australasian Conference

on Information Systems, 1999

The Relationship Between Participation in Information Systems Planning

and Development and the Achievement of Performance Criteria in

Australian Commercial Organizations That Plan Strategically for

Information Systems

Donald J. Falconer and R. Alan Hodgett

School of Accounting and Information Systems

University of South Australia

Adelaide, Australia

[email protected]

Abstract

Senior management in three samples of Australian organizations were surveyed. Findings

relating to the involvement of various categories of participants in information systems

planning and development in Australian organizations are reported. Similarities were found

across different sizes of organizations in terms of patterns of staff involvement in planning

and development, the establishment of performance criteria and subsequent evaluation of

systems against these criteria. The study has identified associations between the involvement

of various categories of persons at different stages of planning and development and the

achievement of pre-established performance criteria.

Keywords

IS Strategic Planning, IS Policy, IS Planning Issues, IS Project Development Policies, ISPerformance Assessment, IS Performance Evaluation, User Involvement, User Participation

INTRODUCTION

Much has been written about who should be involved in the various stages of information systems

planning and development. Such planning and development was once left to information systems

professionals. The objective of this paper is to document the results of the project as they apply to

the relationship between categories of people involved in the planning and development of a typical

information systems project and the achievement of pre-established performance criteria for the

project. The content of this paper is part of a larger study exploring the planning, management and

use of information systems in Australian businesses. The overall study surveyed 1100 Australian

organizations in three samples chosen to represent the business demographics of Australia.

With the cost of new systems increasing and costly failures a business reality, organizations have had

to consider who should be involved and the processes of involvement. There has been much

research and publication in many countries related to this issue. Too often IS professional or other

managers involved in systems development do not recognize systems development as a change

process (Lederer and Nath 1991). Lack of user involvement can lead to systems that do not fully

meet user needs, resulting in redesign and even sabotage (Tripp 1991). Lack of top management

involvement and support can result in the development of systems that fail to meet the long term

needs of the organization. (Doll 1985, Rademacher 1989). As early as 1986 Ewing explained thatthe MIS leader of the future would need managerial skills and a business perspective. Earl (1989)

7/28/2019 10.1.1.4.655.pdf


272

suggested the development of hybrid managers who would combine business literacy with sound

IS/IT technical knowledge together with a third dimension of astuteness in the making of sound

decisions for the use of IT in business. Tait and Vessey (1988) related the level and quality of user

involvement in systems development to systems success. Jarvenpaa and Ives (1991) implicitly

accepted the view that senior executives should participate in information systems planning and

management and then explored the nature of appropriate participation. Yetton (1994) discussed a

"federal" model that organizations might adopt to broaden the technical base of business staff and at

the same time increase the business experience of its technical staff.

THE AUSTRALIAN CONTEXT

Australian businesses generally are smaller and employ fewer people than those in countries which

have been the subject of much research; for example, Lederer and Mendelow (1988) surveyed

organizations with up to 125,000 employees and Earl (1993) surveyed organizations having average

annual revenues of £4.5 billion. Even the largest Australian organizations are small by comparison to

these. The Australian Bureau of Statistics (1993) (ABS) reports on the Australian business sector ineight employment size groupings, the largest being those businesses employing more than 1000

people. In 1993 there were 26,060 organizations with 20 - 99 employees, 4218 with 100 - 499

employees and 1090 with more than 500 employees (op. cit.). Karpin (1995), in his report to the

Australian Government makes extensive reference to a general lack of management skills, including

the management of information technology, in Australian businesses. The report compares Australia

unfavourably in these areas with other developed countries.

THE PROJECT

This paper is part of a project undertaken as a large mail survey of Australian businesses based on astratified sample of members supplied by the ASCPA. A total of 1100 companies in three samples

of Australian companies were surveyed: 500 companies employing from 20 to 99 employees, 400

companies employing from 100 to 499 employees and 200 companies employing 500 or more.

These groups include Australian owned and managed companies as well as part or completely

foreign owned or controlled. The Australian Bureau of Statistics (1993) (ABS) reports on the

Australian business sector in eight employment size groupings, the largest being those businesses

employing more than 1000 people. In 1993 there were 26,060 companies with 20 - 99 employees,

4218 with 100 - 499 employees and 1090 with more than 500 employees (op. cit.). The ABS

(1993) includes statistics for a defined business unit titled "Management Units". Our survey used a

term "organisation" which was defined in the questionnaire and is a close surrogate for the ABSManagement Unit.

This paper addresses the following issues in relation to categories of people involved in the planning

and development of a typical information systems project and the achievement of pre-established

performance criteria for the project:

• The categories of people involved in each stage of systems planning and development in different

sized organizations.

• The establishment of performance evaluation criteria set for information systems projects.

• The evaluation of completed projects against the performance criteria.

• The percentage of projects that failed to meet established performance criteria.

• The sizes of projects, in monetary terms, considered to be failures.

7/28/2019 10.1.1.4.655.pdf


273

• The relationship between categories of people involved and the achievement of pre-established

production performance criteria.

RESULTS

The response rates for the survey for the three samples were 33.4%, 35.6% and 35.5%. This

compares favourably with the response rates achieved by many recently published surveys.(c.f. Steinbart and Nath 1992, Raymond et al. 1995 and Ward et al. 1996). Any sample should be

as representative as possible of the population surveyed. The sample responses fit well with the

Australian business population, except for over-representation of manufacturing and under

representation of community services. This is almost certainly explained by manufacturing industries

employing a higher percentage of accountants than do community services companies.

The qualified accountants to whom this survey was addressed were asked to give the title of the

positions currently held by them. The results are summarised in Table 1 and confirm that all

respondents held executive or senior management positions. Any survey must necessarily accept that

respondent bias will be reflected in the data gathered. The writers believed that because the

respondents held senior management positions they would have the necessary knowledge to

complete the questionnaire and that their answers would be sufficiently homogeneous to allow

analysis and reporting in summary form.

Table 1. Descriptions of Positions* Currently Held by Respondents

Description Large* Medium** Small***

Co's Co's Co's

(%) (%) (%)

Finance controller (Director) 34 42 25

Accounting manager 15 13 12

Divisional manager 9 4 5

Information systems manager 9 2 0

Management accountant 8 5 11

CEO 5 10 14

Director 3 6 9

Other managerial positions 17 18 24

*n = 66, ** n = 126, *** n = 153.

Ninety-four percent of large organizations, 90% of medium organizations and 74% of small

organizations asserted that they undertake strategic planning. The results presented in this paper

relate to those organizations which indicated that they undertake strategic planning.

Respondents were asked to report on one project completed in their organization in the last threeyears. No coaching was provided to respondents. They were free to select one system on which to

report from all systems completed in their organization in the last three years. The reasons for

selecting the systems reported were not requested and were not provided by any respondents. This

approach was used to enable data to be gathered as randomly as possible. It is possible that some

undetected systematic bias is present, but the authors have no means to detect this. It is considered

more likely that, given the compete freedom to choose, the data is sufficiently random for analysis.

To gauge approximate project size, respondents were asked for an estimate of the total project cost

and development time. As was expected, smaller organizations generally reported on smaller

projects, but large projects were well represented across all organization sizes with reported

development costs ranging from $50,000 to in excess of $1m. Total elapsed development time was

7/28/2019 10.1.1.4.655.pdf


274

generally in the range 6 to 24 months, with a shortest time of 3 months and with one project taking

up to 48 months.

Table 2. Participants in the various stages of information systems planning and

development

Participants Propos

ed

Planne

d

Design

ed

Approv

ed

Implemen

te d(%) (%) (%) (%) (%)

Large companiesBd of directors 0 0 0 14 0Chief executive 5 3 0 19 0Snr mgt group 29 29 0 43 12Org. planning group 14 45 0 12 26Financial accountant 0 0 43 12 17Mgt accountant 0 0 29 14 7In-house sys specialists 0 7 83 29 57Funct. Area mgr 2 0 69 43 43Funct. Area reps 0 2 36 0 0User/user group 45 43 93 60 53

Internal auditors 0 0 21 0 0Consultants 2 14 55 0 5External auditors 0 0 10 0 0Other 2 0 2 2 7

Medium companiesBd of directors 4 1 0 17 2Chief executive 14 2 0 29 7Snr mgt group 40 45 0 54 31Org. planning group 9 23 0 12 9Financial accountant 2 1 55 29 29Mgt accountant 0 0 29 10 9In-house sys specialists 1 5 56 31 56Funct. Area mgr 4 2 52 32 28Funct. Area reps 0 0 29 0 0User/user group 20 29 65 34 31Internal auditors 1 0 7 1 0

Consultants 1 21 42 0 4External auditors 0 0 5 0 0Other 4 2 4 1 2

Small companiesBd of directors 8 2 0 22 1Chief executive 20 14 4 38 9Snr mgt group 37 36 2 31 26Org. planning group 4 7 0 1 7Financial accountant 4 8 40 24 26Mgt accountant 0 1 28 10 14In-house sys specialists 1 2 29 9 20Funct. Area mgr 4 6 35 20 25Funct. Area reps 0 0 12 0 0User/user group 17 23 43 24 24Internal auditors 0 0 0 0 0Consultants 5 27 57 0 4

External auditors 0 0 2 0 0Other 2 2 0 0 1

*n = 42, **n = 82, ***n = 86

Respondents were asked to report on systems proposal and development processes in their

organizations based on a traditional development life cycle framework. The questions related to

involvement in the planning and development processes and to the establishment and use of

performance criteria. Table 2 shows the participation rates of a range of individuals and groups in

the various stages of information systems planning and development in regard to the systems project

reported on by survey respondents. The table shows the percentage of occurrences of each

participant type at each stage, for example, in large organizations, user groups were involved in the

approvals stage in 60% of cases.

7/28/2019 10.1.1.4.655.pdf


275

The patterns of involvement of participants between stages in planning and development were

significantly different, while the patterns of involvement of participants in each stage across the three

samples was confirmed statistically to be the same. Most systems were proposed by users/user

groups, organizational planning groups or senior management groups. There is an increased

representation by users and user groups in the sample of large organizations in the proposal stage.

There are a number of noticeably low percentages including in-house systems specialists, financial

accountants, consultants and internal auditors in this stage. The table shows fewer chief executives in

large organizations propose information systems project than do in medium and small organizations.

CEO proposals increase as the size of the organization decreases. Presumably chief executives in

smaller organizations adopt a more “hands on” approach to their systems. The reverse situation can

be noticed for organizational information systems planning groups, where it can be assumed that the

smaller the organization the less likely it is to have a formal organizational planning group. The

planning stage is quite similar, except for more involvement of consultants. Senior executives and the

CEO hand over to functional area managers, in-house systems specialists and user groups at the

design stage. The approvals stage sees fairly equal representation from all levels within the

organization. External entities, such as consultants and auditors do not, as we would expect, play a

part in the approvals stage. Implementation is the province of middle level managers, users and in-

house systems specialists. Few organizations relied on consultants at this stage.

Respondents were asked whether performance criteria were established for the new system. Eighty-

one percent of large, 72% of medium and 70% of small organizations indicated that they prepare

performance criteria for the project. They were not asked for details of these criteria. The results

are somewhat similar for all three organization sizes, although large organizations appear to more

often set performance criteria for new systems than the medium and small organizations. It is

surprising that any respondents replied in the negative to this question.

Respondents from organizations that did prepare performance criteria were asked whether the

completed system was evaluated against the performance criteria and how well the system met the

criteria. Sixty-seven percent of large, 67% of medium and 66% of small organizations indicated that

they evaluated the completed system against established performance criteria. The results are almost

identical for all three organizational size groupings. It is worth noting that about one third of

organizations in each group do not evaluate their new systems' performance even though some of

them set performance criteria. The survey design does not allow a conclusion that this result is

representative of the population of all new systems produced, but the authors believe it is persuasive

evidence that many organizations are neglecting this important step in systems development.

Finally, respondents who indicated that performance criteria were set were asked to provide anopinion as to how well the completed system met the pre-established performance criteria.

Respondents were asked to score their opinion using a Lickert Scale, choosing between: met all

criteria, met most, met some, met few or met none. Table 3 shows the responses.

Table 3. Indicators of how well the completed system met

the pre-established performance criteria for the project.

Percentages of Respondents

La rge* Medium** S ma ll ** *

Co's Co's Co's

Met criteria (%) (%) (%)

Met all criteria 21 17 13

Met most criteria 41 46 49

7/28/2019 10.1.1.4.655.pdf


276

Met some criteria 12 6 6

Met few criteria 0 1 0

Not eva luated 26 30 32

*n = 42, **n = 82, ***n = 86

Of all organizations which set performance criteria, only 21 percent of large organizations, 17

percent of medium and 13 percent of small organizations reported that their new systems met all

criteria and nearly half the respondents felt their new systems met most criteria. Some respondents

reported that their new systems had met only some or few performance criteria. No-one reported

that their system met no criteria. Once again, the results are very similar across all three sample

groups, which suggests that the size of the organization is not relevant when considering issues of

systems success.

An investigation was made to ascertain whether the size of the project was related to the success or

failure of the project. There was no relationship found, except that the smaller organizations tended

to report more on smaller projects. There were two projects costing in excess of $1m which were

reported as meeting few performance criteria.

Participants

Participated

in Planning

Process

Participated

in Design

Process

Participate

d

in

Approvals

Process

Managed the

Implementatio

n Process

Direct Relationship

Financial accountants á

Management accountants á á

Organizational planning group á

Board of directors á

Chief executive officer á

Senior management á

Inverse Relationship

Consultants â â

In-house systems specialists â â â

Users/User groups â

* á indicates a direct relationship â indicates an inverse relationship.

Figure 1. Indications of relationships* between certain categories of participants in

information systems planning and development processes and the achievement of pre-

established performance criteria.

As the patterns of involvement of participants in each stage across the three samples was the same,

the final analysis for this paper was made combining the responses from all three samples into a single

sample. The results were examined to determine whether a relationship exists between the

categories of people involved and the reported achievement of all or most pre-established

performance criteria or the achievement of some, few or no criteria. These two broad categories

were labeled “success” and “failure”. Percentages of categories of people involved in each planning

and development stage were prepared for both the “success” and “failure” groups. An examination

of the percentages revealed different patterns of involvement for each group. There was from 10 to

20 percent difference in the levels of involvement of some participants between the “success” and

“failure” groups. The pattern of these differences is reproduced in Figure 1.

7/28/2019 10.1.1.4.655.pdf


277

The upward arrows in Figure 1 represent a direct relationship and the downward arrows an inverse

relationship between the participants in a particular process and the “success” or "failure" of the

project. Cells without arrows indicate that there was no identifiable differences between the groups.

Relationships which showed a difference were subjected to chi-square tests for association for each

pair. A significant association was found in all bar two cases. The two pairs involving financial and

management accountants contained too little data to test for association. The indicators for these

two pairs have been left in Figure 1 for completeness.

SUMMARY

This survey has found similar patterns of involvement in planning and development between all three

organizational-size groups, the establishment and evaluation against systems performance criteria and

completion success rates. This result was unexpected. It was thought likely that organizational size

would be relevant in determining an organization’s approach to the planning and development of

information systems.

For a considerable percentage of new systems, performance criteria are not established. In cases

where they are established, the resulting system is not necessarily measured against the criteria . This

should be of concern to all involved in the planning and development of information systems.

Educators and trainers should note carefully that the messages delivered in management training are

still not being extensively put into practice.

It should also be of concern to all to learn that only about 20% of surveyed systems were considered

to have met all established criteria and that the majority met only most or some criteria. If this result

is representative of Australian organizations in general, then many organizations are failing to

maximise rewards for their investment in information technology and systems. There is an

opportunity here for researchers to inquire further, perhaps on an industry-by-industry basis, to helporganizations identify reasons for their systems failing and to learn from past lessons of others. All

business mangers should ensure that sound management practices are followed.

The identified associations between the involvement of various categories of persons at different

stages of planning and development and the meeting of pre-established performance criteria require

careful consideration and analysis. No reasons for or explanations of the relationships were

identified in this study. Perhaps the finding that the involvement of consultants at an early stage is

associated with less likelihood of a successful outcome is related to the contracting process. It might

also be related to the way in which organizations perceive and develop their relationships with

consultants. This finding indicates that further study into the consultant-client relationship in the

planning and design stages for new information systems will be of benefit to all parties. This study

offers the broad suggestion that organizations which leave the bulk of planning and development to

in-house systems specialists may be less likely to enjoy success with their new system than those

organizations which involve a range of their business staff. This finding supports the contentions of

much of the literature and the efforts of those organizations which seek to train and develop their in-

house information systems specialists in business matters and vice versa. Perhaps an explanation for

the inverse relationship between the increased involvement of users in the approval stage and likely

success might be that the approval process requires business managers with a more broadly-based

understanding of the business than might be held by those close to the operation of the system.

7/28/2019 10.1.1.4.655.pdf


278

REFERENCES

Australian Bureau of Statistics, (1993), Profiles of Australian Business 1992, Commonwealth of

Australia.

Doll, W.J. (1985) Avenues for top management involvement in successful MIS development, MIS

Quarterly, Spring, 17–35.

Earl, M.J. (1989) Management Strategies for Information Technology, Prentice Hall.

Earl, M.J. (1993), Experiences in strategic information systems planning", MIS Quarterly, Mar, 1–

24.

Ewing, T. (1986) The future of MIS, Information Week , Dec 1, 26–37.

Jarvenpaa, S.L. and Ives, B. (1991) Executive involvement and participation in the management of information technology, MIS Quarterly, June, 205–227.

Karpin, D.S. (Chairman), (1995), Enterprising Nation, Reviewing Australia’s Managers to Meet

the Challenges of the Asia-Pacific Century, Commonwealth Government of Australia.

Lederer, A. L. and Nath, R. (1991) Managing organizational issues in information systems

development, Journal of Systems Management, 42:11, 23–27.

Lederer, A.L. and Mendelow, A.L. (1988) Convincing top management of the strategic potential of

information systems, MIS Quarterly, Dec, 525–534.

Raymond, L., Paré, G. and Bergeron, F. (1995) Matching information technology and organizational

structure: an empirical study with implications for performance, European Journal of Information Systems, 4:1.

Steinbart, P.J. and Nath, R. (1992) Problems and issues in the management of data communications

networks: the experiences of American organisations, MIS Quarterly, March.

Tait, P. and Vessey, I. (1988) The effect of user involvement on system success: a contingency

approach, MIS Quarterly, March, 91–108.

Tripp, R.S. (1991) Managing the political and cultural aspects of large-scale MIS projects,

Information resources Management Journal . Fall, 2–13.

Ward, J., Taylor, P. and Bond, P. (1996) Evaluation and realisation of IS/IT benefits: an empirical

study of current practice, European Journal of Information Systems, 4, 214–225.

Yetton, P.W. (1994) False prophecies, successful practice and future directions in IT management,

TC8AUS IFIP Information Systems International Working Conference, Qld, May.

COPYRIGHT

Donald Falconer and R. Alan Hodgett © 1999. The authors assign to ACIS and educational and

non-profit institutions a non-exclusive licence to use this document for personal use and in courses of

instruction provided that the article is used in full and this copyright statement is reproduced. The

authors also grant a non-exclusive licence to ACIS to publish this document in full in the Conference

Papers and Proceedings. Those documents may be published on the World Wide Web, CD-ROM,

in printed form, and on mirror sites on the World Wide Web. Any other usage is prohibited withoutthe express permission of the authors.

7/28/2019 10.1.1.4.655.pdf


279

10.1.1.4.655.pdf

Documents

Transcript of 10.1.1.4.655.pdf