An Expectation-Perception Gap Analysis of Information Systems Failure

Linda, SL LAI Department ofInformation Systems, City University of Hong Kong,

Kowloon, Hong Kong

Abstract

This primary objective of this paper is to investigate new ways to tackle the old problem of organisation failure of information systems (IS) development. IS failure is defmed as a gap between what the users expect from an information system and how well these expectations are met by the perceived performance of the delivered system. Problems leading to this expectation-perception gap are identified and modelled as five interrelated discrepancies or gaps throughout the process of IS development. The antecedents pertaining to each gap and corrective measures to close the gaps are also suggested. Success in IS development could be achieved by closing the five interrelated gaps and thus closing the expectation-perception gap.

I Introduction

The subject of information systems grew out of computer science to fill a gap created by the failure of machine-code programmers to understand and solve user problems [1]. As pointed out by Stamper in 1973 [2]: 'On the one side, stand the technologists, most of whom have no idea of the complexity of organisations. On the other side, stand the managers and administrators who are unable to translate their problems into feasible demands upon technology'. It was an attempt to bridge technological solutions with organisational problems that started IS work. Some might say that despite the plethora of information systems development tools that have become available since 1973, little has changed in relation to bridging the gap between user's expectations of system capability and how well these expectations are met by the perceived usefulness of delivered information systems. Indeed the increasing rate of development of technology might even be fuelling higher expectations by users which information systems developers are fmding increasingly difficult to meet. Such a gap between stakeholders' expectation expressed in some ideal or standard and the perception of the delivered system is defmed as an IS failure [3].

2 A Gap Analysis of IS Failure

The expectation-perception gap or IS failure can be assessed by a comparison of • an IS user's expectation ofhislher requirements - stated or unstated, conscious

or merely sensed, technically operational or entirely subjective, and may be a

T. Wood-Harper et al. (eds.), Methodologies for Developing and Managing Emerging Technology Based Information Systems© Springer-Verlag London 1999

131

moving target in a dynamic organisational environment, and • the IS user's perception of the performance of the delivered system (based on

conscious and unconscious judgment) after he/she has some experience of using the system.

An expectation-perception discrepancy related to a single transaction leads to user dissatisfaction. Incidences of dissatisfaction over time results in users' negative feeling and eventual rejection of the delivered information system. A delivered information system may mismatch users' expectations in one or more important aspects [4]: it may fail to provide sufficient functionality, its performance may be inadequate; or it may not provide a good fit with the organisation's practices and procedures. It may fail in all these respects, yet still conform to the functional, performance and design specifications formally agreed between users and developers at the outset. Information systems, as a bridge between technological solutions and organisational problems, must be crossed from the side of organisational users. A systems project, no matter how expensive and elaborately designed, is still of little value if it does not perform as its users expect.

The expectation-perception failure of IS development can be understood and analysed by a gap model [5] as illustrated in Figure 1. The model features discrepancies or gaps that need to be closed in order to achieve success in IS development projects. Here, IS failure is defmed as a gap between user's expectation and perception of the performance of a delivered system. The expectation-perception gap (Gap 6) is in turn caused by five interrelated gaps (Gap 1 to Gap 5) throughout the process of Information system development (ISD).

2.1 The cognition gap (Gap 1)

The cognition gap (gap 1) is a difference between 'what the users need' and 'what the users think they need'. User expectations, are the de facto requirements against which the success of a delivered system will be judged. However, most IS professionals have a myth about user expectations. As the saying goes: 'Users don't know what they want. Users keep changing their minds.' System developers just wish users could have worked out ahead of time what they want and communicate the requirements to them in unambiguous writing. The identified requirements, should then be signed and sealed once and for all time. Unfortunately, such a positivistic view of requirements analysis is deemed to be too simplistic. Factors contributing to the cognition gap include: • Requirements analysis is not based on positivism. User needs which shape

user expectations, do not 'exist out there' and ready for the 'picking'. Users initiate an IS project when they experience a felt need for information, a holistic sense of something missing in the current system, an awareness that technologies have the kind of capability which might help. It is legitimate for users to have only a very fuzzy idea of their requirements. User requirements are not easily articulated as they are just mental models in users' minds.

132

• Users requirements are constantly changing. Systems users are embedded in a constantly changing organisational environment and situation which are often unpredictable, apparently illogical and incomprehensible. In order to deal with the uncertainly, most successful organisations have very flexible and responsive informal systems which grow and decay as required. Under such circumstances, it is virtually impossible for the users to produce a set of unalterable requirements specification.

• Users have limitation and bias in information processing. Due to the availability bias, users may elucidate only those organisational requirements related to problems that are current, frequent and easier to recall from. After initial requirements are stated, users, because of the confirmatory bias, may selectively detect only information to support the original statement. The idea of representative bias suggests that users tend to believe a small sample of transactions as being typical of the population from which it came.

Helping users to set their expectations of the new system at an appropriate level is the first and possibly most crucial step in achieving project success. IS developers must be aware of the cognition gap of users, and take measures to aid users to articulate what they want based on what they need now and possibly in the future.

2.2 The comprehension gap (Gap 2)

The comprehension gap (Gap 2) is a discrepancy between 'what the users need/want' and 'what the developers think the users need/want'. Information systems professionals, as pointed out by [6], 'have expanded their roles from product developers and operations managers to become service providers' Providing services that users perceive as excellent requires that the service providers know what users need and want. There are various obstacles that may inhibit IS professionals from gaining a comprehensive picture of users expectations: • Requirements analysis should be inside-looking-out. Despite a genuine

interest in providing effective services to users, many IS professionals miss the mark by thinking outside-in - they know what the users should need and deliver that - rather than inside-out. Systems developers, very often, overestimate their own capabilities, authority and power in relation to the users. They simply do not seek inputs from users to defme their professional services.

• Conflicting users requirements. The information systems developers' task in understanding users requirements is made more difficult by the fact that most IS projects have a surprising diversity of users (referred by some authors as stakeholders), each with different expectations. So, even if users are able to articulate their needs, it may still be the case that the identified requirements are mutually contradictory or jointly defme a much larger system than can be budgeted.

• Users requirements are culture bound. No one can understand a culture unless he/she is inside it, and once a person is inside a culture, he/she will be

133

unaware of it. If a systems analyst remains as an outsider of users' culture, he/she is unlikely to be able to interpret the culture bound requirements as insider members do, and so is likely to get the interpretation rejected. If a systems analyst acts as an insider, he/she will then take the culture bound requirements for granted without self-reflective questioning.

• Communication barriers between users and analysts. The importance of user-analyst relationships in systems development has been widely recognized. However, it is not always possible to maintain a continuous line of communication between users and developers due to various barriers (e.g., background, concerns & languages) between the two parties. Due to the lack of communication, developers have difficulties in comprehending users' information requirements.

Not knowing what the users expect is a serious concern of information systems developers. Being a bit wrong about users requirements can mean expending money, time and other resources on things that do not count to users. Being a bit wrong can even not surviving in a fiercely competitive market.

2.3 The expression gap (Gap 3)

The expression gap (Gap 3) is a difference between 'the developers' understanding of users' needs/wants' and 'the translation of developers' understanding into a requirements specification'. Understanding users' expectations is an essential step in delivering a successful information system, but once system analysts know what users expect, they face another critical challenge: using the understanding to elaborate the understanding into a formal requirements specification for subsequent· systems development. The process of translating requirements in minds to requirements on paper is by no means straightforward because of the following reasons: • The process of structuring needs is problematic. An IS developer's mental

constructs (e.g., perceptual processes, values, ethics, motives, prejudices, intellectual ability, experience, etc.) have an effect on the understanding he/she gained of the situation. It is, very often, those mental constructs rather than the expertise of a developer that determine the relevance and importance of elicited users' requirements. The task of determining feasibility, prioritizing identified users requirements, depends, among other things, on the personal characteristics of the systems developers.

• Modeling reality is problematic. When IS developers translate their understanding of business tasks into technical functions, they map human activities, objects, events into 'processes', 'data format' and 'data structures'. While this reduction is useful and inevitable, it should be noted that no model or modeling technique is capable of capturing the degree of complexity of an organisation'S requirements. Systems developers should not be so enthralled by today's graphic, narrative, and representational modeling aids that they lose sight of their mission and forget that the map is not the territory.

Systems analysts can be compared to translators. Translators convert documents written in one language into another while systems analysts convert

134

users requirements into systems requirements. Yet, no matter how good is a translation, it can never be perfect. There are always errors that occur in translations.

: .......... ~

: Gap 2

Needs for information services

Gap 1 (Cognition Gap)

Expectation of infonnation services

Gop 6 (Expectation-Perception Gap)

Perception of infonnation services

: Comprehension Gap

- .• - • - Utilization of information services

Gap 5 (Utility Gap)

Delivery of information services

Gap 4 (Delivery Gap)

Translation of understanding into requirements specification

Gap 3 (Expression Gap)

: .......... ~ Understanding of information needs

Figure 1: An expectation-perception gap analysis ofIS failure

2.4 The delivery gap (Gap 4)

I.S Users

I.S. Developers

The delivery gap (Gap 4) is a difference between 'what is specified' and 'what is delivered'. In some cases, IS professionals do have a fair understanding of users' expectations and do set appropriate specifications and still the delivered system falls short of what users expect. In order to deliver an information system that meets users' expectation, IS developers need to analyse the requirements specifications, detail the physical and logical design of the systems, and provide the software required and other technical services. Issues related to the systems construction process have long been a focus of IS research and studied exclusively under two disciplines - IS project management and software engineering.

135

• IS project management. IS project management is about the planning, organizing, directing and controlling of resources to maintain several necessary conditions of IS development so that the target systems can be delivered successfully. Three conditions that are generally used to evaluate the effectiveness of project management are 'performance, time and cost' [7]. It is not possible to achieve project success in IS development if anyone of the three dimensions fail.

• IS software engineering. Software engineering [8] refers to the application of formal and rigorous methods to ensure the quality of the software deliverables of an IS are kept within specified standards. IS software quality is a combination of desired attributes such as conformity, usability, reliability, efficiency, security, auditability, maintainability, reusability, flexibility and portability. Software quality can be maintained by ensuring that a whole range of technical and organisational issues are adequately addressed by IS developers.

The delivery gap indicates discrepancies in the ability of systems specialists to transform users' requirements specification into tangible IS product and service deliverables. IS professionals can narrow this gap by applying good project management and taking an engineering approach to software development.

2.5 The utility gap (Gap 5)

The utility gap (Gap 5) is a difference between 'what is delivered' and 'what is being used'. An information technology system is physically constructed by actors working in a given social context, it is also socially constructed by actors through the meanings they attach to it. The social and physical construction of a technology do not stop at the point of systems delivery, but also occurs at the post-implementation phase. The utility gap measures the extent to which the utilization of an operation system is consistent with the original design intent of its systems developers. This gap is a result of several phenomena: • Technologies are socially reconstructed. A technology can be appropriated

in diverse ways and have different meanings and effects for different users. Users could mediate technological effects, adapt systems to their needs, resist them or choose not to use them at all. The flexibility in use of a technology, according to [9], is influenced by characteristics of the material artifact (e.g., the specific hardware and software comprising the technology), characteristics of the human agents (e.g., experience, motivation) and characteristics of the use context (e.g., social relations, task assignment, resource allocations).

• Time-space discontinuity between the design & use of an IS. With many IS projects, the activities that constitute the technology are often separated in time and space from the activities that are constituted by the technology, with the former typically occurring in developer organisations, and the latter occurring in user sites. Due to the time-space discontinuity between the design & use of an information system, it is not surprising that IS users and

136

developers often have very different interpretations of the role and utility of the underlying technology.

• Lack of proper training. Users' perceptions of the performance of an information system are not based on the delivered functions, but on the functions that they can use. As [10] says: people don't pay for technology; they pay for what they get from technology. In this way, IS success do not build on how a technology is constructed, but rest on how much users use a new technology and how differently they use it. Inadequate and improper training is a key reason users reject and resist the implementation of new systems.

The existence of the utility gap suggests the effectiveness ofan information system can be diminished by users' inability to grasp the larger purpose and significance of the underlying technology. A 'technically successful' system can result in organisation chaos if its implementation phase is ineffectual and fail to meet user expectations.

3 Use of the gap model by IS practitioners

Figure 2 exhibits the various problematic issues and their relationships with the gaps that lead to failures in information systems development. In this extended model, as in the basic gap model (Figure 1), the gap between users' expectations and perceptions of an information system (Gap 6) results from five gaps (Gaps I through 5) during the process of IS project development. Each of the five gaps is in turn caused by various factors that are itemised in the left-hand column of Figure 2. In this way, we may argue that information systems failure (or users' expectation mismatch) results from one or more of the following reasons: • users' inability to cogitate their information needs; • developers' inability to comprehend users' information needs; • developers' inability to translate their perceived information needs of users

into requirement specifications; • developers' inability to transform specified needs for information provision

into systems deliverables; • users' inability to utilize the delivered systems to satisfy their information

needs The expectation-perception gap (Gap 6) is a function of gaps 1,2,3,4 and

5. Thus, the key to achieving success in information systems development is to keep Gap 6 closed by closing Gaps I through 5. IS practitioners need to monitor all the contributing factors and determine, perhaps through structured analysis, which factors are critical for achieving success in a particular project setting. Figure 3 depicts a logical process which IS practitioners can employ to enhance the chance of project success. The sequence of questions of the six boxes on the left side of the figure corresponds to the six gaps illustrated in Figures I and 2. Specifically, the process can begins with an understanding of the extent of Gap 6 and then successively searching for evidence of Gap 1 through 5, taking corrective measures wherever and whenever necessary.

UIen' ......... iiiIiiIIiua is' ' : I----J

aa ... ;~·.·· ... --j-----~

Ta ' 'eI"'~",''''' ........ ;

~=~"

Figure 2: An extended model of the expectation-perception gap

137

Gap'

138

r---------------~----~--------~ No ..

Figure 3: A logical process to analyse the expectation-perceptation gap

4 Implications of the gap model for ISD process

The expectation failure (or mismatch) leads to resources being spent on refming systems or worse delivering systems, which are simply not used by their host organisation. The most common explanation of this discrepancy is the reliance of IS developers on a product centred perspective for systems development which assumes that user needs can be defmed and that solutions to these needs can be engineered using an appropriate systems development methodology. However as organisations increasingly question their purpose and processes and as boundaries between organisations become increasingly fuzzy and vague, it is no longer possible to start with the notion that it is necessary to create or computerise an information system. IS development thus has to be seen as a continuous process which is led by the human activity system in the organisation which the information system will serve [11], [12]. Any and every information system can always be thought about as entailing a pair of system, one a system which is served (the organisational activity system), the other a system which does the serving (the information technology system) as shown in Figure 4 [13].

Whenever one system serves or supports another, it is a very basic principle that the necessary features of the system which serves can be worked out only on the basis of a prior account of the system served. This must be so because the nature of the system served will dictate what counts as 'service' and hence what functions the system which provides that service must contain. It follows that any method or methodology that seek to guide individual in making effective IS development must meet the following criteria:

139

• It should provide mechanisms to make sense and understand the behaviour of human activities which an information system is developed to serve;

• It should render a seamless transition between its resulting information requirements model and the process to design a technology-based system to satisfy those requirements.

Purposeful human actions: making & selling products,

delivering services & providing offerings etc

Served System

Real World A designed arrangement of hardware, software, data,

procedure & people

Serving System

Conceptual World

Figure 4: The served-server concept [13]

5 Conclusion

The notion of expectation-perception failure, as argued by [14], is comprehensive enough to encompass other systems failure concepts such as correspondence failure [15], process failure [16], interaction failure [17], etc. in the IS literature. It has significant implications on the development and management of technology-based information systems.

Several factors have been identified by the expectation-perception gap analysis for IS failure, with many factors relating to organisational and human aspects of IS development. Information systems development concerns 'an interplay of human, organisation and technical factors which cannot be easily separated' [18]. The main role of an information system is that of a support function, such systems do not exist for their own sake. Thus, the correct place to start an ISD project is not the information system but the organisation activities that an information system is to serve.

Bridging the expectation-perception gap also requires that IS professionals take a radical shift from being the proprietor of information systems and products to being service providers to end-users. [19] suggests that IS departments should be viewed as a service enterprise responsible for providing business solutions

140

rather than solely technical support, and individuals served by IS should be regarded as customers rather than users. In order to deliver systems and services that IS customers (users) perceive valuable, IS managers must become expert in determining and assessing users' expectations and perceptions.

The contribution of the expectation-perception gap model lies in providing a fresh look at the role of IS professionals and the current approaches to IS development.

References

1. Jayaratna N. Understanding and Evaluating Methodologies, NIMSAD - a systemic framework. McGraw-Hill, Maidenhead, 1994

2. Stamper RK. Information in business and administrative systems. Batsford, London, 1973

3. Lyytinen K. Expectation failure concept and systems analysts' view of information system failures: results of an exploratory study. Information & Management 1988; 14: 45-56

4. Shand RM. User manuals as project management tools: part I - theoretical background. IEEE Transactions on Professional Communication 1994; 37(2): 75-80

5. Zeithaml VA, Parasuraman A, Berry LL. Delivering Quality Service: Balancing Customer Perception and Expectation. The Free Press, NY, 1990

6. Pitt LF, Watson RT, Kavan CB. Service Quality: a measure of information systems effectiveness. MIS Quarterly 1995; 19(2): 173-187

7. Cleland 01, King WR. Systems Analysis and Project Management. McGraw-Hill, NY, 1983

8. Boehm BW. Software Engineering Economics. Prentice Hall, Englewood Cliffs, NJ, 1981

9. Orlikowski WJ. The duality of technology: rethinking the concept of technology in organisation. Orghanization Science 1992; 3(3): 398-427

10. Drucker P. Innovation and entrepreneurship: practice and principles. Harper & Row, NY, 1985

11. Checkland PB. Systems Thinking, Systems Practice. John Wiley & Sons, Chichester, 1981

12. Walsham G. Interpreting Information Systems in Organisations. John Wiley & Sons, Chichester, 1993

13. Winter MC, Brown DH, Checkland PB. A role for Soft Systems Methodology in information systems development. European Journal of Information Systems 1995; 1(4): 130-142

14. Lyytinen K, Hirschheim R. Information systems failures - A survey and classification of the empirical literature. Oxford Survey of Information Technology 1987; 4: 257-309

15. Alter S, Ginzberg MJ. Managing uncertainty in MIS implementation. Sloan Management Review 1978; 19: 23-31

16. Brooks FP. The Mythical Man-Month. Addison-Wesley, Reading, Mass, 1975

141

17. Lucas HC. Implementation: the key to successful information systems, Columbia University Press, NY, 1981

18. Walsham G, Symons V, Waema T. Information systms as social systems: implications for developing countries. Information Technology for Development 1988; 3(3): 189-204

19. Kettinger WJ, Lee CC. Exploring a 'gap' model of information services quality. Information Resources Management Journal 1995; 8: 5-16