DIAGNOSIS OF ORGANISATIONSIN DEVELOPMENT CO-OPERATIONGuidelines for application of the Staircase Model

Report for Sida 1997

Göran AnderssonPeter Winai

1. Introduction _____________________________________________________________ 1 1.1 The staircase model in brief______________________________________________________ 1

1.2 The need for diagnostic tools in organisational development___________________________ 3

1.3 Target groups _________________________________________________________________ 4

1.4 Situations in which the model can be applied _______________________________________ 4

1.5 Organisations on which the model can be applied ___________________________________ 5

1.6 Overview for the reader_________________________________________________________ 5

2. The elements of the model __________________________________________________ 8 2.1 Looking at the organisation in terms of input-output_________________________________ 8

2.2 Looking at the model stage by stage _______________________________________________ 9 Stage 1 Is there an organisation? Focus on output _____________________________________________ 10 Stage 2 Focus on output, efficiency and dependence ___________________________________________ 12 Stage 3 Focus on change and dependence ___________________________________________________ 13 Stage 4 Focus on client value _____________________________________________________________ 14

2.3 Dependence __________________________________________________________________ 16

2.4 Measuring output and change___________________________________________________ 19 Quantitative measures___________________________________________________________________ 20 Qualitative measures____________________________________________________________________ 21 Approach to measuring__________________________________________________________________ 23

2.5 Synthesis of the model _________________________________________________________ 24

2.6 Reconciling the Model and LFA _________________________________________________ 27

3. Approaching the organisation - the object of study _____________________________ 31 3.1 Planning a study ______________________________________________________________ 31

3.2 Application in practice_________________________________________________________ 32 A: Overview of the situation _____________________________________________________________ 32 B: Data collection ______________________________________________________________________ 37 C: Analysis ___________________________________________________________________________ 46 D: Diagnosis __________________________________________________________________________ 49

4. Applicability and usefulness - concluding observations _________________________ 51 4.1 How to improve the relevance? __________________________________________________ 51

4.2 How to improve precision? _____________________________________________________ 51

4.3 How to improve application in practice? __________________________________________ 52

4.4 Areas of development__________________________________________________________ 52

1

1. Introduction The objective of these guidelines is to provide a tool to facilitate and improve planning, monitoring and evaluation of institutional development co-operation.

The need for such a tool has grown as the demands on the aid agencies to show results have become stronger. In the field of institutional and organisational development the difficulties to describe achievements are particularly great. At the planning stage it is difficult to describe the current state of an organisation and the state which the assistance should help to reach. In the evaluation phase it is difficult to describe what stage the organisation actually has reached, without a tool of some kind.

The ambition is not to present a comprehensive discussion of organisational development issues in developing countries. Rather the aim is to provide an approach for diagnosing the state of an organisation. Therefore the booklet does not contain any extensive discussion of the characteristics of environment or internal management practices in government organisations in developing countries.

1.1 The staircase model in brief

The staircase model provides a frame for descriptive analysis. It consists of four stages of development. It should be used to describe, not to explain organisational development. However, the model also carries a normative element: It is an assumption that it is necessary for an organisation to reach one stage before the next. Reality is of course more complicated. The organisation may have reached a high degree of development in one respect, while in other respects it is still struggling with basic problems.

What is it that reflects the stage of development that an organisation has reached? The model identifies two main dimensions of measurement. One dimension covers the output of the organisation. Is anything at all produced and if so, in which quantities and with what quality? Has the output been produced with reasonable input of resources? Is the output relevant to the environment and have the desired effects or outcomes been achieved?

The second dimension covers what is accomplished within the organisation in terms of changes: In types of output produced, in production methods, in administration and management. Changes in these respects reflect the development ability and capacity of the organisation.1

1 In addition to these two dimensions a third one can be identified: The value of output in the client system . An

important condition for increased client value is active interaction with the client; thus it can be looked upon as a separate dimension. On the other hand, the client value represents quality of the output; in that respect it can be included in the output dimension.

2

The model identifies four development stages:

• Stage 1: There is an organisation, but with low and unpredictable output

At stage 1, an organisation has been established, but output is unpredictable and of low quality. Output is defined as the products or services produced by the organisation for use outside its borders. Output is measured in quantitative and qualitative terms.

• Stage 2: The organisation is able to deliver expected output with reasonable reliability and quality

Production is carried out within installed capacity and with available resources. However, the organisation lacks capability to respond to new demands if external conditions are changed or if key staff leave the organisation.

• Stage 3: The organisation carries out changes on its own

When the organisation has reached stage 3, it meets the performance standards which have been set when delivering output. It is also capable of making changes of its own, and actually executes such changes: In products, production methods and administrative/management systems.

The model is thus based on the assumption that the ability to develop is manifested in implemented changes and on-going development activities.

• Stage 4: The organisation works actively with its clients or customers

The organisation which has climbed to stage 4, works actively to increase the value of its products or services in the client/customer system. It knows the clients, has an on-going dialogue with them and allocates resources to facilitate utilisation of its output.

Stage 4

The organisation works actively with its clients

Stage 3 The organisation carries out changes on its own

Stage 2 The organi-sation delivers expected output

Stage 1 Low and unpre-dictable output

3

Figure 1. The staircase model for organisational diagnosis

Thus, as stated above, output and ability to change are the two basic dimensions used to describe the development stage of an organisation.

To facilitate diagnosis, measurements of these two dimensions have to be complemented by different ratio measures such as productivity, staff turn over, service level, resource allocation and resource consumption.

In development co-operation it is also necessary to find out how output and change have been brought about. To what extent is the organisation dependent on external financial and technical assistance? It may very well deliver expected output and carry out important changes, but with external support. If this is the case, the development level reached is unlikely to be sustainable if assistance is terminated.

The model does not explain the causes behind a certain level of development. Therefore it is not a sufficient tool for designing a development programme for an organisation. It is however instrumental in describing a factual situation, which can be used as a basis for problem analysis as well as identification and design of development measures.

1.2 The need for diagnostic tools in organisational development

Generally, a characteristic of organisational development situations is scarcity. Thus, we assume that we have to be economical when using the scarce resources. The resources have to be handled with care, in order to produce the best possible output. Investments made in production must not be in vain - in future the organisation should be able to produce without outside support. Gradually, it must become self-reliant. Adaption and internal development is necessary.

There are two main factors behind the development of the staircase model.

The first is connected with difficulties of the development co-operation agencies in objectively assessing the results of a project aiming at the development of an organisation. Which criteria should be used to assess the level of development achieved? How could organisational development be measured? Is the development level that has been reached sufficient and sustainable - i.e. will the organisation maintain and develop without further assistance?

The second factor is associated with the shift from budgetary control to result-oriented management of governmental activities. This shift has sharpened the demands on the development agencies to present results, i.e. to describe with fair objectivity what has actually been achieved by the assistance. Institutional and organisational development are areas where methods to describe results of development assistance have been exceptionally difficult to find.

Naturally, the purpose of result-based management is to improve effectiveness in operations. Thus, the availability of methods to describe results is a condition for the improvement of management and monitoring of development assistance efforts.

4

1.3 Target groups

The persons we are addressing in this booklet belong to the following broad categories involved in development co-operation:

• Staff of donor agencies - at head office and abroad, working with planning, monitoring and evaluation of organisational development projects;

• Representatives of the recipient country and its organisations, dealing with planning, project preparation, implementation and evaluation of development co-operation projects;

• The organisation which is subject to the support;

• Consultants - international and local, contracted by donors and recipients, working with project preparation, implementation and evaluation.

1.4 Situations in which the model can be applied

Planning

The model can be applied in several situations of planning, e.g.

1) When starting assistance e.g. to re-structure or improve efficiency in an already established organisation;

2) When continued assistance is being considered to an organisation which is established since long.

The model can then be used to: • establish baseline information which can be used for comparison at a later point in

time in order to record development of the organisation; • diagnose the stage of development.

Monitoring

If a baseline study has been carried out with the model, monitoring of the project is made possible by comparing new observations with earlier ones. It will be possible to substantiate that, e.g. ”no progress has been made with planning instruments”. Thus, the model observations can help focusing on specific items, e.g. the planning machinery.

Review

In review situations, e.g. annual project reviews, the model can be applied as a tool. Naturally, regular and repeated systematic observations in terms of the model will increase precision of the diagnosis.

Another review situation where the model can be utilised is when a holistic review is being made of the entire output from an organisation, including the specific output on which the project may be focused.

5

Evaluation

Evaluations of development co-operation projects are usually ex post analyses. Organisational development projects are often confined to a certain aspect of an organisation e.g. an accounting system, organisational structures, staff development and management training. The underlying rationale for such interventions is to improve the production of the organisation so that the output results in useful outcomes or effects for the clients of the organisation. The model may be used in evaluations to describe the state of affairs of the organisation as a basis for assessment of project impact.

1.5 Organisations on which the model can be applied

Primarily, the organisations we look at with the model at hand are governmental agencies and departments, as well as other organisations which are grant financed and/or controlled by government.

These organisations have a societal mission. They are established to work for the common good. There is an identifiable production and there are recipient clients. Part of the output may be ”sold”, which make the organisations partly financed by a ”market” and also affected by a business market culture. Such organisations, working in the interface between the public sector and a market are in fact becoming increasingly common. In these situations, the model helps focus whether the organisation fulfils the societal mission to produce an output with value for the recipient clients.

A basic prerequisite for using the model is that there is an organisation, at least at an embryonic stage. A common situation in development assistance is that a new organisation is being established or is in a state of fundamental restructuring. The assumption is that the main functions performed are long term and that there are no signs that the organisation is about to be abolished.

The easiest case to handle is the straight-forward relationship with one distinct client, to whom a simple output is delivered. The situation is getting more complex to analyse when the organisation produces a mix of different outputs. E.g. output may be mixed of clear-cut ”hard-ware” products, e.g. a building permit, and diffuse ”soft-ware” services, e.g. an information campaign on traffic safety. The complexity increases further when the organisation produces an output for other organisations and not for the ultimate clients. Examples are departmental head offices, policy making units and similar organisations with an indirect relationship with the ultimate clients.

It is less meaningful to apply the model in situations when e.g. privatisation and/or changed form of association for the organisation is being considered. The model is not geared to directly provide support for decisions in such situations. However, indirectly a diagnosis made with the concepts of the model may offer useful information.

1.6 Overview for the reader

In the next chapter, Chapter 2, the staircase model will be more thoroughly presented. The discussion starts with the picture of an organisation as an input-output-system.

6

The staircase model is based on this picture. The basic concepts of the model are introduced together with some other concepts commonly used for analysis of organisations. Then each of the four stages are described and characterised by a number of key questions. The possibilities to measure output and change are discussed. Towards the end of the chapter, a synthesis of the model is presented. Finally the model is related to the Logical Framework Approach.

In Chapter 3, the organisation to be studied is approached. The practical procedures of planning the study, getting an overview and analysing the organisation are discussed. A number of examples from case studies are presented to illustrate how things can be done in the different phases of the study. Also, some examples of how to summarise observations and diagnose organisational development are given.

In the last chapter - Chapter 4 - the applicability of the model is discussed: Its relevance, its precision and possibilities to improve application. See illustration on next page.

7

...IN PRACTICE THE MODEL...

When can the model be applied?

Why do we need the model?

What is the basic idea of the staircase model?

1. INTRODUCTION

The staircase The need for a tool

Applications

2. THE ELEMENTS

Input-Process-Output The stages of the model

Output and Change Dependence and sustainability

Synthesis LFA and the staircase

3. APPROACHING THE ORGANISATION

Planning the study Application in practice

Examples

4. APPLICABILITY

Relevance Precision Practice

How is dependency handled in the model?

Some observanda for practical use

Strengths and limitations of the model?

Checklists and loopholes? Delimitations of

the system under study?

Approach in theory?

Illustrations of practical use?

What information is contained in each stage?

Efficiency, effectiveness and the model?

How is the staircase related to LFA?

Which are the central concepts of the model?

Fig.2: Overview for the reader

8

2. The elements of the model In the first chapter we have presented the basic parts of the model, and the situations in which it is intended to be applied. In this chapter we will look at the model step by step and discuss its elements in more detail.

2.1 Looking at the organisation in terms of input-output

The model is based on a common picture of an organisation as an open system: The system receives input in terms of different resources and also in terms of items to be processed. The system then delivers an output to be consumed/ used by its clients in order to have certain desired outcomes or effects2. For the system to survive in a long term perspective i.e. to continue to receive inputs, it has to produce outputs that are relevant, of acceptable quantity and quality and cost to the client. The output must generate outcomes which are valued by the clients. How the organisation actually is producing the output is not of primary interest to the clients as long as it meets their requirements. The figure below captures the system and relationships with which the model is concerned.

Dependence Change Quantity/Quality

Efficiency

Cost-effectiveness

Effectiveness

OUTPUT OutcomeObjective INPUT PROCESS

Fig.3: The organisation as an input-output-system

2 Thus, the model observes both the client perspective and the societal effect perspective.

9

The production process, at least initially, is looked upon as a black box, i.e. the production process as such is not the main issue. What is in focus is the output which is produced, its quantity and quality. In order to survive and develop the organisation must be able to change. In order to observe change one must look into the black box. Ability to change is manifested inside the organisation but also in change of type of outputs produced. Changes of inputs also requires that the black box is opened. Such changes appear, for example, as quality changes regarding personnel competence. Technical and financial inputs provided by donors reflects the external dependence of the organisation.

The concepts that the model is focusing on are found in bold letters in figure 3.

To facilitate analysis and diagnosis different kinds of ratios may be utilised. One type of ratio is efficiency, which is generally expressed as the relation between output and input. Specific ratios which may be used are e.g. labour productivity, cost productivity or unit costs. However, the use of such measures in concrete situations usually raises substantial problems, because of complex output and absence of data. To be useful, the measures have to be expressed in the form of time series. Notwithstanding, efficiency measures are central to the staircase model. They provide the test needed to assess the organisation’s ability to economise with scarce resources.

Cost-effectiveness measures capture the relation between costs to produce output and the outcomes. A prerequisite is that outcomes can be expressed in sufficiently clear terms, e.g. the reduction of malaria incidence refers to a certain section of the population. Such measures can be used to compare alternative ways to achieve the same outcome.

Effectiveness as pictured in the figure above, is defined as the relation between the value of outcomes and the value that was intended and expressed as an objective. In practice such measures are very difficult to obtain in quantitative terms as it is very difficult to assign values to the outcomes.

2.2 Looking at the model stage by stage

How are the four stages of the model defined and what do they imply?

It should be observed that all variables of the model are relevant at each stage: Input of resources, output, change processes and client value3. The normative element of the model implies, however, that each stage is characterised by certain values of the variables. When following the model stage by stage we will focus the attention to the essential characteristics of each stage and by the end of the chapter all variables will have been commented upon.

3 A synthesis of the model , linking the variables to each stage, is made in section 2.5.

10

Stage 1 Is there an organisation? Focus on output

Almost always there is ”something”. There is an organisational embryo, a management, a mission to fulfil, some basic functions and staff for these basic functions.

Stage 2

Stage 1 Low and unpre-dictable output

Fig.4: Stage 1 in the staircase model

The condition ”organisation” could imply autonomy: I.e. the organisation can fulfil its mission and it can be managed independently of day-to-day disturbances from outside; also, it can employ personnel by itself. However, this is almost never the case. The organisation is part of the public sector and is seldom independent to act. Thus, stage 1, does not imply full organisational autonomy, but rather an entity with borders defined by its mission. The organisation is that entity which has to fulfil a certain mission, deliver a certain output.

It is seldom obvious from the beginning how the organisation to be studied should be defined. Government organisations are typically structured in hierarchical patterns. A department may consist of divisions and the divisions may be organised in sections. Some units are operative others are producing administrative services. The units may be organised according to function, type of service, geographical area or other criteria. How the organisation to be studied should be delimited has to be deliberately considered. The model can be applied at different levels e.g. department or division or section level.

A common situation of an organisation in developing countries seeking external assistance is that there may be parts of the organisation in operation, other parts may be set up but not functioning and some may only exist on the organogram.

What are the characteristics of an organisation operating at stage 1?

The model identifies output as the crucial variable. From the clients point of view the organisation is of no use if useful output is not produced. An organisation which is not capable of producing what it has been set up to produce is therefore in the terms of the model at a low level of development. The following set of questions can be used to measure output:

Are outputs defined?

11

A pre-requisite for production is naturally that the general terms of reference or mission of an organisation has been analysed to determine what it should produce. In a dynamic organisation at a high level of development this is a live question constantly given attention to. In an organisation at stage 1 there may be considerable uncertainty of what it should produce. In a build up situation the reason may be that the organisation has not had time to analyse its mission or it may lack the necessary competence. In a reconstruction situation lack of competence is a more likely explanation to the state of affairs.

In the individual organisation, it is important that the different kinds of output be identified. A common situation is that the organisation produces e.g. both standardised and ad-hoc services, guidelines, processed information etc. The output may have different degree of ”specificity”, a concept which will be explained below in section 2.4.

Are standards set?

If outputs have been defined, has the organisation set any standards regarding the quantities and qualities of the outputs? Standards regarding quantities and qualities may be set in production plans or business plans and expressed in terms of quantities to be produced within a certain time limit. Quality may be expressed in a variety of ways depending on the particular output.

Organisations at stage 1 are characterised by not having production plans or business plans formulated in a way that clarify what they intend to produce during the plan period. If plans exist they play no role in actual execution of the production.

Are standards monitored?

Monitoring requires collection of information about the actual quantity and quality produced. Existence of production statistics may indicate that monitoring is done. At stage 1 it is not likely that production statistics is available or used.

If standards are not monitored, it indicates that management is still at a low level, and there are reasons to suspect that output quality is low, i.e. that standards are not met, which brings us to the next question:

Are standards met?

Even if the organisation has not set any standards regarding quantity and quality or maintains any system of production statistics it may still produce an output. Characteristic of stage one organisations is that the output is low, uneven and unpredictable. Quality is low and may not even be considered an issue because of the overriding problem of getting anything done. A lot of obstacles are encountered which management is unable to master and which lead to a situation of low output, compared to installed capacity in terms of people, physical facilities and budget.

12

Stage 2 Focus on output, efficiency and dependence

When an organisation has reached stage 2, we expect output to be produced with fair regularity. Quality is still a problem. At stage 2 the problems of maintaining a regular production is still a major challenge of the organisation. Issues of quality, efficiency and change are emerging and getting attention. Competence and other resources are scarce to initiate and implement development and change.

Stage 3

Stage 2 The organi-sation delivers expected output

Stage 1

Figure 5. Stage 2 in the staircase model

We can repeat the questions introduced above. At stage 2 the following answers can be expected:

Are outputs defined?

The organisation is capable of describing the type of output it is delivering. Central overview might lack but at lower levels in the organisation outputs have been identified.

Are standards set?

Standards may be set regarding quantities through business plans or plans of operation. It should, however, be observed that performance standards may exist as generally accepted norms, although they may not be articulated or deliberately determined. Quality standards are mainly based on professional norms and tradition, but are rarely analysed and formulated explicitly.

Are standards monitored?

Production statistics is normally kept or data exist that can be compiled when needed.

Are standards met?

Production targets - quantity standards - are met with a reasonable variation. Regular production is maintained but is vulnerable to changes in external conditions.

Efficiency and dependency

13

Stage 2 organisations pay little attention to efficiency. Productivity measures or analysis of unit costs and similar data are hardly to be found.

The organisation is capable of producing its regular output without external support, financial or technical. Due to financial constraints and lack of adequate competence the organisation needs external support to embark on and implement development activities.

Stage 3 Focus on change and dependence

At stage 3, the organisation is capable of producing its output with fair predictability. Productivity and output quality are constantly observed and measures are taken to improve both productivity and quality.

In modern management and organisational literature, change is in focus. A distinction is often made between adjustment and tool development on one hand and more fundamental change of direction, in terms of dominating ideas, strategies and output on the other.

In a stable state changes are normally made within the institutional framework. Changes are made according to the rules of the game. However, a governmental organisation is subject to many external institutional changes. It may query those changes, and even challenge the existing rules of the game. Thus, another distinction can be made between change within given conditions e.g. a new organisational structure, as opposed to activities aiming at changing the externally given conditions, e.g. by transforming from agency to business company. In the latter case the organisation may increase its mandate to act on its own, without day-to-day political interference.

At stage 3 the organisation manages its own change. It is capable of defining its own development needs and it is not dependent on outside competence or resources to initiate and implement changes.

Stage 4

Stage 3 The organisation carries out changes on its own

Stage 2

Figure 6. Stage 3 in the staircase model

The ability to change manifests itself in implemented or on-going changes such as:

14

• New types of output, e.g. statistics in a new field or consulting which has not been offered before;

• Change of production methods, e.g. introduction of new, computerised equipment for report printing;

• Change of internal administrative systems and processes, e.g. a new financial management system;

• Change of ”soft” variables as leadership style or organisational culture, e.g. introduction of management by objectives, result orientation and decentralisation of powers.

Stage 4 Focus on client value

Also client value has got a scarcity aspect. Output should be utilised as good as possible. Utility for the customer should be optimised.

An output that is not utilised by the customer or client is wasted. An output which is not fully utilised by the client is also wasted to some extent.

The reasons why the output is not utilised may vary: The product is of poor quality, it is not adapted to the client needs or the client is not aware of or does not understand the qualities of the product. E.g. statistics produced is unreliable, irrelevant or difficult to understand. Different actions have to be taken to rectify the situation, but in all cases the dialogue between producer and client must be improved. In order to increase value of the output in the client system, the producer must keep himself informed of the developments of client needs, and the client has to be informed about how to utilise the output in the best way.

Stage 4

The organisation works actively with its clients

Stage 3

Figure 7. Stage 4 in the staircase model

Indications that an organisation has reached stage 4 are positive answers to the following type of questions: • Does the organisation know who its clients/customers are? • Does the organisation know if and how its output is consumed/utilised? • Does the organisation have personnel allocated for customer/client dialogue and

support? • Are there concrete manifestations of dialogue with the clients/customers? • Are there concrete manifestations of active involvement to increase value of output

in the client/customer system? • Are there concrete manifestations of client/customer impact on services provided?

15

Focus on client value corresponds to the development of business in later years during which a flow of literature on service management and customer orientation etc. has been published. Service management emphasises the service ingredient which is part of virtually all products, even if the product is a car or other hard-ware. Orientation towards the customer implies that all activities should be customer driven, i.e. legitimised by the value for the customer. Thus, a good relationship and open communications with the customer, it is argued, is a condition for quality, and it becomes more vital the more complicated the services are.

16

2.3 Dependence

The function of aid is to add external resources to an organisation. They may have the form of money, physical assets or technical assistance through experts and training. In most cases the aid will expand the capacity to produce - through added physical capacity, improved quality of human resources and improved organisation and methods.

Increased capacity of an organisation will require additional funds to sustain the operations at the new level. Only if the improvements are accompanied by sufficient productivity increases may the additional operating costs be offset by savings. There is a risk that the aid process creates a dependence on external resources in order to operate at the higher capacity level which the aid has helped to build, but which the Government is unable to sustain. In many countries this risk has turned into reality for a number of government institutions. The public service reforms of the last decade have been driven by an insight that this situation is unsustainable.

Public service institutions are highly dependent on Government policies and priorities, which are i.a. materialised in budgetary allocations. In these cases the dependency relationship is based on a legal foundation. In fact, we can distinguish between two kinds of dependency relationships - that with the Government (”internal”) and that with the external aid institutions. The institution has to adjust to the rules of the game set by the Government. The efficiency of the internal relationship may be questioned on the basis of degree of autonomy, resource allocation, policy directives etc., but the relationship as such is constituent of the situation. Aid relationships, on the other hand, are voluntary and temporary. They may be terminated or re-negotiated. The rational for entering an aid relationship is that it will end as soon as the purpose of the relationship has been achieved. The dependency created by entering the relationship is positive as it opens prospects for growth and development.

When agreeing on support to institutional development the assumption of the aid organisation and the recipient is that the support will terminate at some time in the future. The dependence created by the transfer of resources should eventually be reduced. In practical terms this implies that recurrent costs for the created capacity should be borne by the recipient at a level which secure reasonable efficient utilisation of the capacity. Necessary development expenditure should also be borne by the recipient. If sufficient recurrent funding is not forthcoming or little or no resources are allocated for development activities the positive dependence created at the start of the relationship may turn into a negative dependence characterised by decreasing effectiveness in use of both recipient and aid resources.

To what extent are external resources critical for achieving output and change? This question can be answered by assessing the dependence of external funding and competence in current operations and in development of the organisation. The figure below illustrates four typical situations that need to be assessed.

17

Financial Competence

Current Operations

A

To what extent is the organisation dependent on external financial support to sustain operations?

C

To what extent is the organisation dependent on external technical assistance to carry out operations?

Development Activities

B

To what extent is the organisation dependent on external financial support for development and change?

D

To what extent is the organisation dependent on external technical assistance for development and change?

Fig.8 Financial and competence dependence in current operations and development activities.

Financial dependence implies that the organisation uses external financial support to carry out its operations and development activities. The dependency may be measured as the relation between external financial support and own financing.

A. Financial dependence to sustain operations is manifested in direct budgetary support to meet operating costs of regular programmes e.g. salaries, purchase of training material, maintenance and renewal of equipment, travel expenses etc., or indirect support to similar expenses under the label of development expenditure. In many of the least developed countries this is a common situation caused by two processes. First, a weak financial situation of many Governments over a number of years has lead to a situation where the operations of public service organisations - ministries, departments, agencies - are underfunded in relation to pursued policies and installed capacity. The underfunding is manifested in very low salaries, insufficient funds for operations and maintenance and lack of funds for renewal of equipment and machinery. Second, donors have at the same time funded development programmes without realising the financial implications for the Governments when the programmes are transformed into regular operations. If donors terminate the support there is a high risk that operations can not be maintained at the level the assistance has helped to create.

A situation that may arise is that - in terms of the model - the organisation may slip back from a stage, where the organisation delivers expected output, towards stage 1 which is characterised by low and unpredictable output. If this happens the financial dependence is critical to sustainability.

18

An organisation able to carry out changes on its own would meet such a situation by restructuring and rationalisation measures in order to adjust to the new circumstances.

B. Dependence of external funding to carry out development and change activities is manifested in financial contributions to investments in human resource development, systems development, equipment and physical facilities. This is the model aid situation. The lack of own resources justifies external support. A common assumption is that the investments in organisational development will in due course result in productivity increases, increased budget allocation from Government to maintain the investments, or a combination of the two. The degree of dependence can be assessed on the basis of the relation between own and external financial input to development and change activities. What would happen if external support was terminated? If development and change activities would cease, then the organisation would be critically dependent on external finance for development and change.

C. Dependence on external technical assistance to carry out operations is manifested in the classical gap filling type of support. It is typically justified because no indigenous person meeting the requirements can be recruited to fill a critical line position in the organisation. The degree of dependence may be assessed on the basis of the relation between the number of existing managerial and professional posts and the number filled by external personnel and the length of service of external personnel.

External assistance of this kind was more common in the early days of development co-operation. Many governments have also own schemes through which expatriates are recruited and financed.

D. Dependence on technical assistance for development and change is manifested in the extent to which expatriate personnel on long and/ or short term assignments are instrumental in identifying problems, articulating visions, preparing projects, implementing project activities, monitoring and reporting progress and evaluating results.

To assess and measure dependency of external competence is very delicate. The mere fact that an organisation uses external expertise is in itself no sign of critical dependence of importance for its ability to produce, change and develop. Independence or self-reliance in terms of competence may be defined as follows:

• the organisation has the ability to define its knowledge needs - e.g. for a particular specialist - which can not be met internally, identify where the missing knowledge can be found and how it can be acquired and ability to utilise the knowledge acquired;

• the organisation has the ability to identify, formulate and solve problems and maintain this capability.

A move towards self-reliance should manifest itself in concrete changes where internal expertise play an increasing role in all phases of the change process and a corresponding reduced role in technical assistance. There is obviously no simple measurement that can be used to assess competence dependence. Different aspects

19

have to be assessed on the basis of which an informed judgement can be made. The difficulty in assessing the nature of dependency on external competence for development should not be underestimated. The fact that expatriate personnel are in place and seemingly dominating the development work, may point at a strong dependence which requires continued support. However, what would happen if the expatriate personnel were withdrawn? That is very difficult to assess. Would the work continue or not? What would happen if funds were made available for the institution instead of technical assistance? Would it be capable of substituting the expatriates for own staff or hire indigenous experts? Would it be capable of continuing the work at the same level of efficiency and quality?

In applying the dependency criteria on concrete cases one can expect to find that an organisation may operate at stage 2-4, but that it is critically dependent on financial and/or technical assistance. The implication is that the diagnosis has to be qualified, due to the specific type and degree of dependence observed.

2.4 Measuring output and change

The possibilities to measure output and change in concrete cases depend on many factors. The model is focused on organisations. The first question which has to be answered is whether the whole organisation or a part should be measured. There is no clear-cut answer to this question. Measuring the whole organisation provides an overview. It may result in a good picture of the general state of affairs. On the other hand, the difficulty to identify output that can be measured quantitatively and qualitatively increases, the more that is included in the analysis. At the same time important weaknesses/ strengths in the organisation’s ability to produce and change may be overlooked when the whole organisation is measured. This dilemma has to be handled in each particular case.

In order to draw conclusions about changes it is necessary to compare information gathered at different points in time4. Availability of information is then decisive. It may be possible to get information of the current situation but difficult to get access to historic information. In case the organisation is new, the diagnosis can only include a description of the current state and cannot be supported by information which reflects the dynamics of its development.

Another aspect of the problem of measurability is that organisations often lack institutionalised procedures for data collection and dissemination. This may render it impossible to obtain the type of data that the model requires or necessitate special arrangements for data collection.

Furthermore, in public service organisations it is seldom possible to summarise the status of the organisation in a single or a few measures. In this sense there is a major difference with the typical company operating on market conditions. In the business company productivity, profitability and turnover data capture essential aspects of the

4 It is of course possible to get an impression of changes that have taken place on the basis of information

gathered at one occasion. E.g. interviews are carried out with members of the organisation, and they are asked to give their own assessment of the changes. Theoretically this implies, however, that information from a previous point in time is (although indirectly) collected.

20

state of the organisation. In public service organisations such information is less meaningful. The difficulties to summarise are increased by the fact that many public service organisations5 have a complex mix of both high and low specificity outputs which do not lend themselves to meaningful aggregations. In the following we will discuss different approaches to these problems, combining quantitative and qualitative measures.

Quantitative measures

The possibility of obtaining meaningful measures of output is very dependent on the nature of the output. The better an output can be specified the easier it is to measure it quantitatively. If production statistics is available, it is a sign of measurability. The absence of production statistics may have different reasons: It may be due to practical difficulties to produce meaningful statistics, low appreciation of the need for production statistics and/ or inability to collect information in the organisation.

High specificity output is characterised by the possibility to delimit the output in discrete units. The output is standardised and is typically produced repetitively e.g. a monthly report on price statistics, a decision to grant a certain permit, number of outpatients attended per day, number of audit reports produced, number of graduated students etc.

Low specificity output has opposite characteristics. The output is unique i.e. it is neither standardised as to content or produced repetitively and it is difficult to delimit and describe in discrete units. Examples of low specificity output are strategic policy decisions, advisory services, issuance of guidelines and similar types of output. Outputs from policy making organisations are typically of low specificity whereas outputs from direct service delivery organisations are of high specificity.

In between the extremes there may be a whole range of outputs which are more or less possible to measure. In many cases indirect measures have to be used.

E.g. in service production, counting the number of outputs is seldom sufficient. The number of audit reports produced by a national audit office does only give a hint as to the state of the organisation. Qualitative measures are often necessary to make quantities meaningful. What do the reports actually contain, e.g. how reliable are the observations made?

High specificity output:

Standardised, discrete units, produced repetitively: Regular reports, permits issued, examinations etc.

Low specificity output:

Not standardised, units vary in shape and content: Policy decisions, professional advice, issuance of guidelines etc.

On the other hand, qualities like reliability or availability may be possible to capture in quantitative terms, e.g. fault rate per 100 telephone lines per year. The level of service is also a qualitative characteristic of a certain service and may also be

5 There are examples of companies with a combination of low and high specificity also in the ”private” sector,

e.g. insurance companies or private hospitals.

21

quantified e.g. a norm is set that an application for a certain permit should be processed by the responsible institution within 1 month.

Another indirect way of measuring output is to look at what is happening next, in terms of outcome, e.g. in terms of equal access of health services. A whole range of different outputs such as policy, regulation, advice and physical measures may be produced in order to achieve equal access. Instead of measuring each and every output one may establish measures that capture change in the outcome variable i.e. equal access to health services and use this measure as an indicator of the capability of the organisation to produce.

Quantitative measures may also be used as indicators of ability to change. Measures showing frequency of introduction of new services and abolishment of old, and frequency in change of quality of services are examples of such indicators: Statistical reports covering new areas, introduction of performance auditing, provision one-stop-shop service to clients etc. Improved values of different ratios can also be used as indicators of ability to change e.g. improved productivity, reduced staff turn over, reduced vacancy rate, reduced variance between planned and actual expenditure.

Qualitative measures

The concept of quality may be defined in many different ways. How good is the output? In measuring output quality one way is to differentiate between the two dimensions; objective and subjective quality.

Ratios

Figures of ratios and relationships are often useful techniques to express both quantitative and qualitative developments. Some of the measures mentioned in the text are ratios. Examples are:

Fault rates, e.g. per 100 telephone lines

Productivity measures: Output in relation to input of some kind

Relations between actual and planned conditions: E.g. qualified incumbents in relation to posts; Vacancy rates

Change rates: E.g. output changes between years

Objective quality may be defined by predetermined characteristics of a particular service. Does the output - e.g. a piece of advice or a decision - meet these characteristics? The objective quality would be measured as the relation between the pre-determined and actual characteristics. In the public service, objective quality is largely determined by laws and regulations and professional norms, e.g. building regulations or ethical norms in the nursing profession. Financial considerations play an important role in determining objective quality, e.g. teacher/pupil ratio. However, actual availability of resources, financial and human, play a decisive role in meeting the quality criteria.

Subjective quality is defined by the user of a service. How good is the output - e.g. a piece of advice - perceived to be? During the last 10 - 15 years the clients/customers assessment of particular services has become an important factor to consider in designing services. Many public institutions carry out regular surveys to measure

22

client satisfaction as a means to improve services. In terms of the model this type of activity would constitute a sign of an organisation operating at stage 4.

Measuring change is essentially a question of qualitative judgement. In most cases the measurement will consist of qualitative descriptions of a particular state based on available documentation, observations and interviews. The qualitative description may be supported by quantitative information. The descriptions may be made at different degrees of detail. It may be enough to observe in which areas changes have taken place and in which changes are on-going.

23

These observations may be explained further as to their content and the description may also include an assessment of the quality of the change. The rather trivial but fundamental problem lies in selection of aspects to be described. This will differ from one area to the other but it is essential to describe the state of affairs of the change area in such a way that meaningful comparisons can be made at a later point in time. Over a period of time the quality of a particular change should have effects on output quantity and quality, efficiency and effectiveness.

Approach to measuring

The various difficulties and circumstances that always must be expected wherever the model is to be applied, make it is necessary to adopt a flexible and pragmatic approach to the measuring problems.

The basic approach in establishing quantitative and qualitative measures is to use information which is readily available or which can be obtained with reasonable effort. If production statistics is not kept, it will normally not be feasible to collect such statistics during a diagnostic study. Similar conditions hold for other types of measures. If no standards are set and service levels not defined, these are observations which should be recorded. In most cases this would be sufficient information for the purposes of the study. The application of the model in a concrete case may reveal substantial weaknesses in existing information systems. This is in itself important information which can help in diagnosing the organisations level of development.

If information systems are very weak, and follow-up studies are intended to be carried out at a later point in time, it is necessary to identify the information base required, i.e. to specify the type of information that should be collected and how the collection should be made. The establishment of such information systems has to be made in co-operation with the organisation itself, be within the capability and capacity of the organisation to handle and be regarded as useful in the management of the organisation.

24

2.5 Synthesis of the model

The table below summarises the staircase model and its stages in terms of expected values of the crucial dimensions measured. Each stage is characterised by a certain mix of requirements and at each subsequent stage new requirements are added.

Variable Stage 1 Stage 2 Stage 3 Stage 4

Client value No sign No sign Emerging signs Signs

Change of - administration and management -production methods - output

No sign

Emerging signs

Signs

Signs

Output - quality - quantity

Low Unpredictable

Low - Medium Predictable

Medium - High Predictable

Medium - High Predictable

Competence and resources

External input crucial

Adequate for regular production, inadequate for change

Adequate for regular production and change

Adequate for regular production and change

Figure 9. Expected values of critical variables at the different stages of the model. ”Signs” refer to concrete manifestations in the organisation.

The expected values of the different variables at the different stages constitute the normative element of the model. By setting these norms a yardstick is obtained. Such a yardstick is necessary to enable the observer to record the development of an organisation from one time to another.

At stage 1 an organisation exists but it does not produce expected output. Output is not produced in all areas and may not even be defined. Outputs of recurring production cycles regularly fail to meet deadlines. Questions of quality are rarely raised and if raised pushed aside by the more important issue to deliver any output at all. Quality is typically low and quality standards are not set. Objective quality is below requirements implicit in regulations and professional norms.

The organisation may recognise development needs but lacks ability to articulate those. Initiative to change is taken outside the organisation.

The organisation may have identified its clients but their needs are not observed. It is occupied by its internal problems. The internal perspective dominates.

The organisation requires external financial and technical support to trigger off development.

25

At stage 2 the organisation produces outputs in most areas expected in its terms of reference. Quantity may vary between different areas in relation to installed capacity or production plans and between different points in time. Output from recurring production cycles normally meets deadlines, although delays may occur. Productivity increases and decreases without long term improvement.

Output produced is of acceptable or lower quality. The objective quality varies but seldom exceeds what regulations and professional norms require.

Business as usual is the prevailing norm. Manifest changes of output are rare. If observed they are typically initiated from outside. On-going change activities may have been initiated but have most likely been articulated with external assistance. Implementation of change activities is made with external financial and technical support.

The clients are defined but no active measures can be observed to communicate with the clients. The internal perspective still dominates.

Regular output is produced with own financial and human resources. Financial and competence constraints are severe and development work requires technical and financial support.

At stage 3 output is produced in all areas. Output quantity is predictable in relation to terms of reference and installed capacity. Productivity is monitored and generally improving. The output is of acceptable or higher quality. Deadlines are met. Objective quality meet regulatory and professional norms and typically represent the ” state of art”.

The organisation is constantly involved in a number of change activities: to redefine its role, to re-prioritise, to change procedures, methods and technology. It initiates changes on the basis of signals from the environment and active search for improvement possibilities. The organisation articulates its change ideas and defines the competence it needs for development and implementation.

The external perspective is gaining ground in the organisation. It has started to identify activities and measures to enhance client value.

The organisation is self-reliant in terms of finance and competence.

An organisation at stage 4 has the same characteristics as at stage 3 but the external perspective is dominating. The organisation puts strong emphasis on measures to enhance the value for the client of its output. Concrete activities to that end are part and parcel of the operations.

An organisation may very well meet the requirements of production and change of the different stages of the model but not the requirements of self-reliance. It is thus possible to arrive at the assessment that an organisation is operating at stage 2 or 3, but that production and change are critically dependent on external financial and technical inputs. In a diagnosis this has to be articulated. The stage identified has to be qualified on the basis of the degree of dependence observed. Such a qualification would give an assessment as to how likely it is that the organisation will at least stay

26

on the stage which it has reached.

27

2.6 Reconciling the Model and LFA

The staircase model is built on the same basic ”input - process - output - outcome” model as the Logical Framework Approach in project planning. Organisational development projects are typically concerned with the processes of an organisation. The interventions aim at improving output, production technology, organisational structures, systems and procedures, quality of management and staff etc., which will enhance the capability of the organisation to produce the right things in the right way.

The staircase model is applied at organisational level and encompasses the defined organisational entity as a whole, whereas development projects in most cases are confined to a certain area in an organisation or in a sector of society. A certain development project may focus on improving management capacity. The output from that project is intended to result in effects at organisational level. It will impact on the competence of staff and organisational processes and thereby also on the output (see fig. 10 below). Eventually it will also impact on the outcomes of these outputs for the clients.

PROJECT OUTPUT INPUT

ORGANISATIONINPUT OUTPUT

Fig.10 The project outputs impact on the organisation.

The distinction between project and organisation is necessary to have in mind, for a number of reasons: One is the obvious difference between the performance of the project and the performance of the organisation.

28

The staircase model puts an emphasis on what is to be built up to work more permanently, as opposed to the project or projects which should assist in the organisational building. In both cases we may be interested in internal processes as well as output. In figure 11 the different foci are illustrated.

Processes Outputs Outcomes in society

Organi-sation

1.

How do processes work? E.g. production flow or strategic development

3. What is the organisation’s output? (Quantity and quality) Has this output improved? Will improvements be sustainable?

5.

Which will be the long term effects in society?

Project

2. How do processes work? E.g. interaction with other assistance projects and transfer of knowledge to the organisation

4. What is the project output? (Quantity and quality) E.g. proposed or implemented changes of systems and procedures

Fig.11: Processes and outputs in organisations and projects

The processes of the project (box 2) concern activities carried out within the project, its interaction with other projects, the way transfer of knowledge to the organisation is handled. The outputs of the project (box 4) concern e.g. proposals on new systems and procedures of the organisation, staff of the organisation trained in certain skills and implementation of new organisational structures. In the long term or ultimately the rationale of the project is to impact on the society (box 5). This impact is an indirect outcome of the project and a direct outcome of the outputs produced by the organisation.

The focus of the staircase analysis is the organisation itself. The ultimate criterion of a long-lasting result of an organisational development project is of course the development of the organisation itself, and in a broader context whether it successfully fulfils its mission in society. Thus the staircase model will direct attention away from project output (box 4) to the transmission from project to organisation, and to a well-functioning organisation (box 1 and 3) and a sector of society developing in the right direction (box 5).

An organisation that has reached stage 4 of the model has the characteristics described in figure 9 above (see section 2.5). The model does not explicitly include measurements of outcomes or effects for the clients of the organisation. It is,

29

however, assumed that the desired outcomes are likely to occur when an organisation operates at stage 3 and 4.

In the LFA methodology desired outcome is usually described as the development objective to which a project is to contribute. This is natural and follows from what has been pointed out earlier: How output is produced is irrelevant to the client as long as the output meets the requirements. The client has no interest in looking inside the organisation - it is a ”black box”. However, organisational development projects are concerned with exactly that: The functions inside the organisation. A project may focus on administrative and managerial systems, staff development, production technology etc. Typically development projects deal with organisations at stage 1 and 2 and the purpose is to make the organisation function efficient and effective. The desired state - the development objective- concerns the organisation itself. An organisational development objective may therefore be seen as an intermediate objective in relation to the ultimate mission of the organisation.

The logical hierarchy of a project as defined by Sida in its application of the LFA methodology consists of the following levels:

Project Objective

Results (Outputs)

Activities

Development Objective

Inputs

Fig.12 Levels in the project hierarchy in Sida application of the LFA methodology

Organisational development projects will normally require an organisational development objective, between the development and projects objectives of the hierarchy above. The reason is that objectives serves the purpose of giving direction to what is to be done.

30

In defining objectives it is therefore important to determine the overall purpose of the intervention:

• If the purpose is to develop the organisation, it is logical to describe the desired state in terms of the staircase model i.e. ability to produce output, ability to change and degree of dependence.

• If the overall purpose is to create values for the clients of the organisation the development objective should be described in such terms i.e. in terms of outcomes.

The difference may seem sophisticated in view of the points made above - organisations are not ends in themselves. But the distinction is important because the logic of the first option is to concentrate project efforts on activities that maximise the capability of the organisation - ”you help the organisation to learn how to fish”, while the second option makes you to concentrate on efforts to maximise client values - ”you feed the people with fish”. It would make sense to design the logical hierarchy in organisational development projects as follows:

Overall development objective Expressed in terms of client value

Organisational development objective Expressed in terms of state of output, change and dependence - related to the stages of the staircase model

Project objective Expressed in terms of state of certain aspects of output and change- related to the stages of the staircase model

Project results/ outputs Expressed in terms of state of specific systems, services/outputs

Activities

Input

Fig.13: Project hierarchy in organisational development

31

3. Approaching the organisation - the object of study In this chapter we will go through the phases of a diagnostic study. First we will describe the planning phase and then, based on experiences from previous studies, we will illustrate how the model can be applied practically. Examples of tools for data collection and analysis will be given.

3.1 Planning a study

A characteristic of the public service is the hierarchical structures and the more or less strong linkages that exists between the different levels. Organisational entities placed in the centre or at the top of the hierarchy typically produce outputs for another level in the hierarchy which in turn may produce outputs for the ultimate clients of the services. But an organisation which produces output for a lower level may at the same time produce for the ultimate clients.

In defining the object of a study it is important to attempt to clarify the complexity of production. Least complex are organisations producing a limited range of services directly for ultimate consumers. A primary school is an example. The complexity increases when an organisation produces a range of outputs of different specificity for use both by intermediary organisations and ultimate clients. This is often the case with national ministries. An example is a Department of Education producing guidelines and curricula for the schools as well as directly taking part in education of groups of students - the ultimate clients.

The complexity of production should be considered when planning a study. In organisations with a complex production it is likely that considerable efforts will have to be spent on structuring of the production system and identification and definition of outputs. In these situations a pre-study should be made where the production system is identified and described and data availability, accessibility and quality are assessed. Based on such a pre-study, a decision on delimitation of the object of study can be made. Actual data collection, validation and analysis would constitute the main study and result in a final report containing base line data and diagnosis.

In structuring the production system, identification and definition of outputs a ”top down” or a ”bottom up” approach may be chosen. The top down approach attempts to identify the main types of outputs and based on the emerging structure individual outputs are identified. The advantage of this approach is that it helps to determine the level of detail of the study. A bottom up approach implies that each individual output is mapped and possible aggregates of outputs determined on this bases. The advantage is that a complete picture is arrived at while the disadvantage is that the approach may show to be too time consuming.

The planning considerations discussed above may also be seen as a question of striking a balance between breadth and depth of the observations and measures. If depth is chosen at the expense of breadth, difficulty to assess the whole organisation

32

arises. If breadth, on the other hand, is chosen at the expense of depth the problem of validity of observations arises. A strategy to handle this dilemma which may be rational is to prioritise breadth in an initial study and pursue detailed investigations if time so permits or in a main study.

In carrying out studies it is a great advantage if the study team has pre-understanding of the field of study and knowledge of the particular organisation or system to be investigated. To secure this the study team should preferably include staff of the studied organisation, subject matter expertise recruited externally or from the organisation and methods expertise.

3.2 Application in practice

In this section we describe the case study process as consisting of four phases. In practice, of course, the phases are seldom distinct. Data collection has probably started before we have an overview of the situation and the picture obtained in the overview will most likely have to be revised as we come to know more. Also, the diagnosis will develop during the study on the basis of data acquired.

In the frame some of the main points and strategic issues to pay attention to in the study are listed.

MAIN POINTS IN STAIRCASE STUDY A. Overview Basic facts - organisation and environment Basic output System delimitation Emphasis: Top-down Breadth Pre-study B. Data collection Data collection guide Attention to data quality Practical arrangements Emphasis: Bottom-up Depth Main study C. Analysis Quantitative measures - qualitative descriptions High/low specificity How much is covered by quantitative measures? Exclude measurement of specific output areas? D. Diagnosis Validation - two risk-reducing measures: Check with experts - do they support diagnosis? Diagnosis will be used as a baseline - later observations will relate to baseline

A: Overview of the situation

Firstly, it is important to get an overview of the situation, in order to create a first structure for measurement. Basic facts about the organisation and its institutional environment should now be collected. What is the size of the organisation in terms of personnel and budget? Which is the basic mission and tasks? Which are the basic types of output from the organisation? How does the organisation relate to other parts of the sector? How is it headed and monitored? How should the system be delimited?

Typically, in this step the approach should be broad, looking from the top down into the organisation.

33

Initial structuring (I): Bureau of Statistics

The National Bureau of Statistics is an easily identifiable unit. It is characterised by a special mandate to collect statistical data from different sources. Its activities are regulated by a special law and a governmental instruction for the organisation. In principle, the Bureau consists of a number of statistical production units and support units for administration.

STATISTICAL PUBLICATIONS

Business community

Private public

Governmental agencies and departments

UNITS

PRODUC-TION

TICAL

STATIS-

ADMIN SUP-PORT UNITS

CLIENTS

Input

Fig.14: The Bureau of Statistics as a delimited system

Thus it is an uncomplicated task to delimit the Bureau of Statistics in relation to its environment. Output can be identified mainly as statistical publications - ad-hoc and periodical. Clients are governmental institutions, private companies and NGO’s as well as the individual citizens.

34

Initial structuring (II): The Province Administration

This example is taken from a study of a Provincial Administration, which included analysis of two governmental departments, namely the Departments of Health and Education.

In this study, the whole Province Administration with its departments is identified as a system, producing different values for its citizens. The separate departments represent subsystems, each with their own production. The departments furnish their respective sectors, which we characterise as service delivery systems.

OutputOutput

Output

SOCIETY

SERVICE DELIVERY SYSTEM

DEPART-MENT

Fig.15: A department as an input-output system

Each department is the primary unit of analysis, in principle consisting of a Head Office and regional organisations.

This is the theoretical and principal definition of the department system. In practice, however, both of the departments studied contain units which would normally belong to the service delivery systems. The Health Department e.g. encompasses a referral hospital and the Education Department a number of technical colleges. Thus, the functional distinction between department and delivery system becomes somewhat blurred.

Each department produces policies, interpretations of policies and management of the delivery systems. Applied on the two departments studied:

• The Health Department produces output for the hospitals, clinics and other institutes, together constituting the health service delivery system.

Its production consists of advice and regulations regarding medical and technical matters, delivery to the hospitals of medicines, training and examination of nurses and reform programmes in different areas.

35

• The Education Department produces output for the schools and education activities, together constituting the education service delivery system.

Also in the Education Department, part of the production consists of advice and regulations, delivery of material to the schools (furniture, books etc.), training and examination of teachers, reform programmes etc.

In principle the departments do not produce output directly for the citizens - this is done by the service delivery systems. Again, this is the principle. However, in reality both departments actually do deliver services to people. Examples are the Hospital and the Technical Colleges mentioned above, and also part of the services delivered through some of the programmes which are directly aimed at the citizens: Oral health, occupational health, environmental health, adult education and pre-school education.

Areas of measurement: The Province Administration

Above, we have used the Province Administration to illustrate how a system can be delimited for analysis. The table below summarises the areas of measurement, which were focused in the same study. On the basis of this categorisation, more detailed questions were listed for the two departments.

Output I: Support and guidelines

Output II: Change of delivery system

Output III: Information from management systems

Ia) Output to delivery system e.g. - Advice - Regula-tions - Meetings - HRD/ Training - HRM/ Salaries - Provi-sioning of goods

Ib) Output to citizens e.g. - Provi-sioning of services that are normally provided by the delivery system

Change in direction of services New support systems

Monitoring and evaluation of delivery system

Change of management systems, procedures and practices within Department Client Value: Increase of value of output through dialogue with delivery system Dependencies: Financial and competence input from outside/foreign sources

Agency input from other governmental bodies

Fig.16: Summary of measurement concepts used in the data collection and analysis of the Health and Education Departments of the Province.

36

In the Province Administration study two main types of output were identified:

One type included services and common standards for the delivery system. Examples are advice, regulations, conferences and training, personnel and equipment. The Health and Education Departments both produce professional personnel for the delivery systems, e.g. through nursing and teacher training colleges. To a certain extent they also produce output for the citizens directly.

Another type of output concerned the change and development of the delivery systems: Changes in the direction of services and the providing of new support systems to the delivery systems are the two main types of output in this category. Characterising a department is the transformation of political intentions into proposals and decisions to make these political intentions reality.

To these two main types of output a third type was added:

Output from monitoring systems and evaluation activities. Examples are regular and ad-hoc reports.

In the department, like in other organisations, internal changes may concern structure and decision making, planning and financial management systems.

It was a striking fact that the new province administration, including the departments under study, had inherited personnel, routines and resources from the previous government. Also, services were still delivered from the previous site of the Province administration. One can imagine that in such a situation some of the staff were positive to the changes, some were openly against change and some were obstructing change through sophisticated means.

Changes in organisational culture and values are qualitative changes that require special methods and efforts to be measured. It was not possible within the limited scope of time to attend to these changes. However, in principle, internal value changes are perhaps the most important changes in an organisation working with fundamental political reforms.

Who are the clients of a departmental system? The primary clients were identified as personnel in the delivery system: Principals and teachers in the educational system and doctors and nurses in the health delivery system, as well as other kinds of staff in the two systems. Pupils, patients and their relatives were the secondary and ultimate clients of the departmental systems. The political level could be called the owners or principals of the departments, but they also represented the ultimate client, and in that respect they were looked upon as intermediate clients.

The value of output in the client system is dependent on the actions of the producer. Value can be enhanced through a dialogue between the Department and the primary client. The dialogue may concern questions regarding interpretations of policies and initiatives launched by the Department. It becomes manifest in different types of participatory measures such as involvement of primary clients in planning and budgeting, meetings and conferences, ad-hoc advice, manuals and generally in the attentive attitude of department officials. Active monitoring and evaluation is part of such an attentive attitude.

37

It was not possible to directly measure the creation/enhancement of value; however the manifestations of dialogue, as well as the existence of monitoring and evaluation were included in the observations.

External dependence was defined as dependence on input in terms of competence and financial resources from sources outside the Government. Input could be provisions from foreign donors, international agencies or NGO’s. Thus the grants from the National Government were not considered as external input.

B: Data collection

As has been pointed out earlier, the phases of the study may not be as distinct in practice, as may appear from the headlining. As the above example indicates, some information about output will be acquired already in this first phase.

In the Data collection step it is important to prepare a framework and guide for data collection. It is very useful to refer to this guide as the study goes along, and tick off questions as they are responded to. Also, the guide will help to show if things have been misunderstood (this is normal!) in the beginning. Normally the guide will be revised several times during the study.

In this step great attention has to be given to data quality. Data may initially look ”available”, but will sometimes show to be very difficult to get in practice, without extensive processing. One example is personnel statistics, which may be available for the total staff. However, the possibilities to break down data into gender, professional categories or management levels may be impossible within given time constraints. Another example is unit costs, which seem to measure the same aspects or phenomena, but which are based on different definitions.

Also, we must be careful not to collect data just because they are available. There is the measurement trap, meaning that data are collected and processed, because they are at hand, not because they are important. It may be obvious, but still: Being caught in the measurement trap is quite common. Which data are really interesting, relevant - and useful?

Essentially the question guide will reflect the main questions of the staircase model. Thus, what is needed is information on the organisation as such, on its output and external inputs, on internal changes and on client relations.

Are production plans and output statistics available? Are standards and targets set and is the fulfilment of these monitored?

Data on resource utilisation, personnel input and costs are normally crucial data.

Another main theme is internal changes: Both in production and in production processes, as well as in management systems.

External inputs including those from domestic public sector organisations and development assistance projects are important to get hold of.

38

Finally, different data that can contribute to a picture of how client relations develop, are important to acquire. A very important demand is to get data organised in such an order that repeated data collection will be possible.

Having constructed the data collection guide, the practical arrangements remain. We cannot expect to find all the people that we need to meet, nor that people will be available at any time. It is useful to develop relations with a few central sources, both to get references to other sources and to ”test” reliability of data.

39

As the picture of the department system in the previous section indicates, the delimitations of the Health and Education Departments may not appear self-evident. Also, output is complicated and mixed of services of high and low specificity. Below, two examples of data collection will be given. In both cases, the system borders were fairly easy to define. Also, the organisations had output of high specificity - statistics and telecommunication services.

Data Collection (I): Bureau of Statistics

The Bureau of Statistics was the first case to be analysed with the staircase model. Four crucial areas, which correspond to the areas mentioned above, were derived from the model: Output, change, input and activities to increase output value for clients. The four crucial areas were then split up into critical questions, as shown in the frames below.

The way to obtain answers was to collect planning documents, indicating plans for different products and product groups. The master source was the Plan of Operations. Different product groups were defined. For each product the planned date of delivery was registered. The planning date was compared with the actual date of delivery - i.e. when the product ( a document) actually appeared. Some of the products printed were not

listed in the planning documents but had to be traced through interviews.

The Bureau of Statistics framework Critical questions related to Output • Does the organisation produce an output? • Are any production performance standards

defined? • Does planned output appear? • Is planned output delayed? • Is there an awareness of quality matters? • Is output quality sufficient? • Is output quality systematically monitored?

Delays were calculated by comparing the delay time with the planned production time. Output quality was assessed by picking 15 publications from different areas. The publications were scanned through to find out whether quality aspects and methodology definitions were mentioned in the text. This indirect measure of quality was complemented by direct measurements. A fundamental aspect of quality is the reliability of statistics produced. E.g. it was found that response rates varied considerably between products and over time.

The Bureau of Statistics framework Critical questions related to Change and development of output and production methods • Has new output been introduced? • Has existing output been dropped? • Has existing output been changed? • Have new production methods been introduced? • Have existing production methods been changed? • How have changes been initiated and carried out?

Next step was to get a picture of how output and production methods had changed (see adjacent frame).

The changes of a number of products in different areas were followed up in interviews with the Bureau of Statistics management. It was found that the organisation had made some changes in output as well as in production methods. It

40

was also found that external dependence was prevalent, when initiating the changes. Both the introduction of new products and implemented changes in production methods were externally generated, in all cases studied.

Another type of change refers to administrative and management systems. The existence of such implemented and ongoing changes contributes to the general picture of the organisational status. The questions in the frame cover this area.

The information given in interviews indicated a low level of change. No systematic work was carried out to prepare for changes. The organisation was struggling to sustain the changes made. It was also found that all changes

in administrative and management systems during four years emanated from external sources.

The Bureau of Statistics framework Critical questions related to Change and development of administrative and management systems Have any changes been implemented or are there any ongoing activities aiming at developing or changing • formal structure? • production planning systems? • leadership methods? • incentive system? • personnel management? • financial management?

Another set of critical questions focused the composition of input and its sources.

One subset of questions concerned different kinds of resources: staff, facilities and funding. The other subset referred to the financing of inputs.

It was concluded in the analysis that the organisation to a large extent was dependent on external funding, and that this dependence had increased. Expansion had to a large extent been financed through external sources. Close to all transport, data processing and printing facilities had been provided through external funds. Both external and in-house training had also been largely externally financed. The exception was office accommodation, which had been government financed.

The Bureau of Statistics framework Critical questions related to Input of resources Has the organisation adequate • staff? • office accommodation? • transport facilities? • data processing facilities? • printing facilities? • recurrent funding?

How is input financed? How is expansion financed?

41

The Bureau of Statistics framework Critical questions related to Client value of output • Does the organisation actually know who its clients

are? • Does the organisation actually know if and how its

output is utilised? • Does the organisation have personnel allocated for

marketing, marketing research and client support? • Are there concrete manifestations of dialogue with

clients? • Are there concrete manifestations of active

involvement to increase value of output in client system?

• Are there concrete manifestations of client impact on production?

Finally, a set of critical questions related to client value of output were posed. These focused on how aware, competent and active the organisation was, with reference both to client needs and to the utilisation of output by the client.

The overall impression was that the organisation studied barely kept records of its clients. There were no specific resources or personnel to follow up client needs or utilisation of output. Neither were there any concrete manifestations of client dialogue.

Data Collection (II): Telecom Company

The Telecommunications Company represents a very different, but still clearly identifiable, kind of production. However, Telecom operates on commercial terms whereas the Statistical Bureau was grant financed. The question guide put together for the case of Telecom also contained five subset of questions. The main questions were the following (see next page):

42

Case Study Questions - Telecom Company Set 1 Output 1.1 What are the services produced? 1.2 Are performance standards set/defined? a) Quantity, b) Quality, c) Costs, d) Tariffs and e) Profitability targets 1.3 Are the performance standards that are set a) defined according to international norms and b) adjusted to local conditions? 1.4 Are performance standards monitored? 1.5 Are performance standards met?

Set 2 Change and development of output and production methods 2.1. Have new services been introduced? When? How? Have the changes been initiated, implemented? 2.2 Have new techniques/methods to produce services been introduced? When? How? Have the changes been initiated, implemented? 2.3 How have changes in 2.1 and 2.2 been initiated and carried out 2.4 By whom have changes been initiated? To what extent is change initiated and driven by internal and local staff members?

Set 3 Change and development of administration and management - type of changes - have the changes been initiated, implemented? 3.1 Change of framework, legislation, ”market status”? - outside/inside initiatives - change of formal structure? performance contract? - dialogue with Ministry? 3.2 Organisational structure within company? 3.3 Leadership philosophy/methods 3.4 Planning systems and procedures 3.5 Human resources management 3.6 Financial management 3.7 By whom have changes been initiated? To what extent is change initiated and driven by internal and local staff members? 3.8 Changes in organisational values and culture - manifested changes in priorities from production requirements to market and customer demands - manifested changes from production quality to customer value

Set 4 Input of resources - relations expressing e.g. personnel and financial input - financial key figures and standards defined by the company - personnel key figures and standards defined by the company - composition of staff

Set 5 Customer value of output 5. 1 Does the organisation know who its customers are? - Composition of customers today? - Potential composition of customers? - Focus/ prioritisation on certain customer categories? 5.2 Does the organisation know if and how its output is consumed/utilised? 5.3 Does the organisation have personnel allocated for marketing, marketing research and customer support? 5.4 Are there concrete manifestations of dialogue with customers? 5.5 Are there concrete manifestations of active involvement to increase value of output in the customer system? - How is customer benefit of the services provided increased? 5.6 Are there concrete manifestations of customer impact on services provided?

43

At a certain stage - if lucky - one will find that the same data will be repeated during the interviews and data collection. There will be a saturation of data. Repeated observations will add less and less to the study. One has to decide when to stop and allocate time and budget for other activities.

In the Telecom study, it would have been a great loss if data collection had ceased earlier. After about a week it looked as a stage of saturation had been reached. Then suddenly a source was hit, namely the financial information system. This information system showed e.g. to contain information about quality development and productivity. Thus: To balance between too little and too much data collection work is very delicate.

Data Collection (III): The Province Administration

In the Province Administration a number of areas of measurement were identified. Based on these areas, a checklist with more detailed items was compiled for each department. The checklist followed the same structure as the matrix in figure 16 above. The example on the next page is cut from the first part of the checklist for the Department of Education.6 The items concern

Output I: Support and guidelines provided to the education delivery system.

A specification of Output I is shown in figure 17.

Output II: Outputs concerning change and development of the education delivery system, including

Change of direction of services towards equity, Resource allocation, Racial and gender equity, New/revised curricula/syllabi, Planning and budgeting, Accounting system, and Management information systems (MIS), Human resources management and development (HRM, HRD) and Change/investment of physical assets

Output III: Use of management system for Monitoring/Evaluation

Dimensions of change: Management systems, procedures and practises within the Department, including

Operations planning/ Business planning, Financial management, Decision making/ delegation, HRM, HRD and Organisational structures.

6 A similar list was prepared for the Department of Health.

44

Output I: Provided to education delivery system

Output area Description of output Measure/Indicator Observation of value on indicator

Comments

I:1 Current Advice Oral/written 1) Quantitatively through measurement system if available 2) Directly, interviews with clients seeking advice 3) Indirectly, interviews with staff (and their managers) giving advice

I:2 Written guidelines Guidelines Printed leaflets, brochures, policy documents; Numbers, edition volumes; Quality - face value and assessment by others

I:3 Regulations Regulations systematically planned, organised and issued

No of new regulations - issued - changed in relation to plans

1:4 Meetings/conferences

Meetings/conferences organised for teachers, school boards etc; Fora for dialogue with professionals, admin.

No of meetings Planned meeting activities No of participants

I:5 PRESET Teachers of different types

No of teachers graduated annually

I:6 INSET Training of teachers and other staff (not in Dept)

Training days produced for different categories of staff; No of staff attending courses; No of courses organised

I.7 Examinations Examinations prepared and implemented by Dept

System: Subjects, levels Degree of own production by Dept

I:8 Posting of teachers and school staff Personnel Recruitment/appointment/ transfer

Posts filled Standards met (e.g. 40 pupils/teacher in Prim.School, 35 in Sec.School) Variation in rel. to standard per region or municipality

I.9 Payment of salaries Salaries paid Correct - no of complaints or corrections Timely - delays in days

Fig. 17: Checklist for data collection in Department of Education; Output I, Support and guidelines provided to the education delivery system.

As can be seen in the figure, the items were described and defined in terms of measures and indicators. These definitions were gradually revised as a consequence of the learning process. Also, new items were added and deleted during the data collection.

45

46

C: Analysis

Analysis refers to the process of resolving a thing into its elements. What is resolved here is the bulk of data, which is scrutinised, put together and compared, combining different ways of measuring. The analysis is concluded by organising the observations in terms of the model variables.

A basic problem is to describe and measure essentially qualitative situations in a fairly objective way. The measurements should ”stand for themselves”, without being accompanied by lengthy explanations. Quantitative measures can improve the observations and make comparison possible. To measure, one is however confined to use those quantitative data that are available. When carrying out a study, we cannot normally allocate a great deal of the time to collect raw data and construct new measures.

Basically, we have two typical measurement situations: Those of high specificity and those of low specificity. Situations with high specificity output normally give access to quantitative data, whereas the opposite is the case when output has low specificity.

The four-field matrix below gives a framework to discuss the different situations

Low specificity situations High specificity situations

Quantitative data available

Fill in with descriptions which come close to quantitative measures: • existence/non-existence of

items • frequencies of items • ratios of items • judgements

Qualitative data available

Fill in with qualitative interpretation of data

Fig.18. Specificity and measures - different analytical situations

47

Normally, many quantitative measures relate to fairly ”simple” and/or high specificity output. In such situations, the quantitative data should naturally be taken care of, and put into their context in the qualitative interpretation.

In opposite situations, when output is of low specificity, e.g. the output consists of planning documents, quantitative measures are not available and the first observations have to be qualitative, i.e. in terms of a text describing the state of affairs. Various approaches have already been mentioned, like

Zero/One: I.e. the existence or not of a specific phenomenon - process, system etc. which will tell about the development level of the organisation.

Frequencies and ratios: The number of activities e.g. meetings, inspections or complaints.

Judgements: Qualified assessment by different interviewees, e.g. ”agree or not” that the variable concerned has certain characteristics.

A specific example would be the situation where the output itself concerns change in the service delivery system. E.g. the output could be a new planning system to be implemented by the service delivery system. The first question would be: Is any output of this kind produced? Is there a new planning system? If not, further investigation into the quality of output is irrelevant. The next question would be: Is there any ongoing preparatory work? If yes, in that situation it is a matter of describing how far the preparatory work has come.

If the answer is yes and a new planning system is being implemented, it may become relevant to measure how much - how many units - of the delivery system that are actually using the new system.

In analysing an organisation we may come across specific output areas which may raise our curiosity. These may be islands of highly specialised production. However, measurements and assessment of output in such areas may require special efforts. It should then be considered whether they do necessarily form a part of the diagnostic exercise.

The following is an example of how observations may be summarised:

Summary of observations (I): Bureau of Statistics

”The observations concerning the year of the study show that the Bureau of Statistics delivers output with fair regularity. Delays are however frequent and there is uncertainty as to timely delivery. Some of the planned products do not appear and their delivery is postponed time after time. Seen over a longer period delivery of outputs has improved considerably. Until the period when development co-operation started, very few products were actually published.

Output quality varies. Our observation of the existence of quality discussions in publications indicate that low attention is paid to quality aspects. The uneven and sometimes low response rates indicate low quality of regular products.

48

On the dimension of change and development we have been able to record introduction of some new products as well as changes in production methods. On-going work to change production methods has been recorded. Few changes have been implemented concerning management and administration.

Compared to the situation ten years ago, the capacity in terms of quantity and quality of production factors has increased substantially. The existing capacity - in terms of staff, physical facilities and immaterial resources - is adequate for the production programme of regular statistics.

The evidence indicates a substantial dependence on external know-how and finance. All changes implemented and on-going changes registered have been and are brought about by external support, and in fact through external initiative. The upkeep of regular production is partially dependent on external finance and vulnerable, because of inadequate recurrent funding.”

In the summary, no direct reference is made to the issue of value of output for the client. Virtually no manifestations of active work with the clients were registered. Strict application of the model would have implied that these variables had been commented on. However, as will be seen from the diagnosis, the organisation was far from the stage of working actively with its clients. In the next section we will return to the diagnosis of the Bureau of Statistics in terms of the model. But first we will give another example of summarised observations from the Telecom case:

Summary of observations (II): Telecom Company

”Output:

Services are produced with fair predictability. The volume of services measured in terms of direct exchange lines has increased with about 38% over the last four years, but has not kept pace with the increase in expressed demand. The quality measured as availability and fault rate has remained about the same. The quality measured as service level has improved slightly, but still remains low in and international comparison. Productivity measured as lines per employee remains about the same as before.

Change in output:

Several new services have been introduced. Production technology is continuously modernised through increasing investments in the network.

Changes in administration and management

The rules of the game set by Government before incorporation have remained stable. The company was completely re-structured in 1993, one year after incorporation. Several minor changes have since been made within that organisational framework.

A number of policy documents have been introduced in the management process. Entirely new computerised finance management and customer systems have been developed and implemented. A performance appraisal system is underway.

49

As for human resources management an organisation has been established, systems and procedures introduced and several training programmes developed and implemented. Affirmative Action has improved the situation for disadvantaged groups.

A cultural change programme has been launched to re-inforce development towards a business oriented culture, already visible in the organisational structure and at conceptual level.

Focus on customer value

Several concrete manifestations exist, indicating active dialogue with customers.

Dependence on external resources and competence

Technical assistance has been instrumental in building of competence and in support of administrative and management development. Operations are managed and run entirely by own staff, except for the Managing Director. Telecom is profitable and able to generate surplus for investments, and to borrow on the market. The company is also self-reliant in competence in the sense that it is capable to identify and contract external expertise when required.”

In this case, like in the previous, the summary is built on the key variables of the model. The next phase is the diagnosis.

D: Diagnosis

The diagnosis is based on a comparison of the summarised observations and the staircase: Conclusions are drawn as to at what stage the observations fit into the staircase. Firstly we revert to:

Diagnosis (I): Bureau of Statistics

”The Bureau of Statistics is about to reach stage 2. The organisation delivers its own output with fair regularity. However, some planned products do not appear and their delivery is postponed time after time. Concurrently, the organisation is working with change and development as regards production methods and quality issues. It has implemented a number of changes over the past years.

Thus looking at the output dimension, it could be said that the Bureau is operating on the second stage, although its position is not yet convincingly firm. On the development dimension it is operating on stage 2 and 3. However, when applying the self-reliance perspective, it is quite evident that external support in terms of know-how and finance is crucial.

In terms of know-how a degree of self-reliance has been achieved as regards production of regular products of given design and quality. Self-reliance in development and change is however not evident. The manifestations and emerging signs of change and development are all initiated and brought about with external support.

50

Financially, the Bureau of Statistics is dependent on external resources to keep up the present production programme and to finance development activities.”

Another example of summarised observations and diagnosis is given below.

Diagnosis (II): Telecom Company

”When interpreting the observations in terms of the model, our conclusion is that Telecom is about to reach stage 4. However, this statement has to be qualified because of the level of aggregation applied as well as the relevance and validity of the measurements used.

First, the statement refers to the organisation as a whole and not to different functions or departments. When looking at individual parts, it would be possible to find departments at a lower level. It has e.g. been mentioned above that the various training functions still remain to be properly staffed. Also, the company is vulnerable to losses of key staff.

Strong synergies have been prevalent during the development process and may have balanced weaknesses in individual departments. These synergies have emanated from the strong political back-up for the creation of a modern business oriented organisation and an obviously strong leadership.

Secondly, this model is focused on concrete manifestations of output and change. The approach does not include in-depth analysis of internal organisational processes. Nor does it assess the organisation’ s ability to handle strategic issues for its long-term survival, such as technological and international market development. In-depth analysis would require instruments of another kind e.g. models for analysis of internal organisational processes. Assessment of strategic abilities would require specialist competence in those fields.

Thirdly, in measuring change we have basically identified type, direction and volume. We have not made in-depth analysis of details and qualities of the changes. This may influence the validity of our assessment. It should also be born in mind that most of the changes identified are in an early stage of implementation and their intended effects are yet difficult to assess. In a longer perspective one would expect that these changes should impact positively on performance.

The model does not specifically look into the environment of the organisation. However, for Telecom, environmental factors, mainly technological and international market developments, seem to pose great challenges for long term sustainability.”

A way to validate the diagnoses is to confront it with experts that should have insight in the organisation and the sector. If they think that the diagnosis is correct or fair, the diagnosis is strengthened.

It should also be remembered that the first diagnosis will be used as a baseline for later studies. That means that observations at different occasions will be related to each other. The new diagnosis will tell more when compared with the first observations and the diagnosis.

51

4. Applicability and usefulness - concluding observations The basic idea behind the staircase model is to provide a tool which can be used in designing and evaluating organisational development projects. The few case studies which have been carried out so far in order to test and develop the model indicate that it has the potential of becoming a useful tool. But further development and refinement of the model is needed. These guidelines will hopefully stimulate the use of the model. Only through practical applications will it be possible to improve its usefulness.

There are a number of issues that should be thought about when applying the model.

4.1 How to improve the relevance?

The model is descriptive and provides a normative framework for placing an organisation at a certain level of development. For donors and recipients it is highly relevant to have access to a tool that helps to define the situation at the start and also helps to determine what should be achieved. E.g. the project should help to take the organisation from stage 1 to stage 2 or to stage 3. Although it is possible to limit the use of the model only to describe a baseline situation, the relevance increases if the model is applied repeatedly during a project. In considering the use of the model it is therefore important to determine how it should be fitted into the decision making and reporting processes of a project.

4.2 How to improve precision? The model has not yet been used as a regular tool in planning, monitoring and evaluation of organisational development projects and its usefulness remains to be proven in practice. The usefulness will largely be determined by the precision of the diagnoses that are made. If they are impaired by too large margins of error they will be of little help.

The limitation to four steps makes the model handy and reduces complexity, which is often an advantage in the follow-up dialogue, especially when many projects are assessed. In other situations however, it may be a weakness that the steps are too few. Although placed at the same stage, organisations may still have many different characteristics. Development assistance is typically extended to organisations at stage 1 and 2 and there may be a need to divide these stages further in order to better discriminate between different levels.

A strength of the model is that it attempts to measure verifiable phenomena by recording quantity and quality of actual performance, manifestations of actual change and activities to increase client value. In this respect it measures objective realities which then form the basis for interpretation. It does not make any attempt to explain why things look the way they do. A weakness then, is that the interpretation of data leading to the diagnosis is not guided by explanatory information. The need for explanatory information when interpreting data should be further looked into.

52

The problem of interpretation increases at the upper part of the staircase. More observations have to be interpreted. In the case studies done the quality of changes have e.g. not been assessed. The criterion, ” carries out changes on its own” implies that the changes are of an acceptable quality. Assessments of the quality of change require in-depth studies. On the other hand it can be argued that such assessments are not required as the quality of changes will manifest itself in output and productivity measures at a later point in time. This argument also underlines the importance of follow-up studies.

4.3 How to improve application in practice?

Questions regarding the applicability may be split into two aspects. One has to do with the access to data, and the other with the psychological preparedness to be observed - and assessed.

The question of access to data is crucial for the application of the model. Data is not always available in the format required by the model. In the case studies carried out, the information searched for regarding the basic variables was essentially available without extensive efforts. However, in many cases it may be necessary to conduct special studies in order to retrieve relevant information, and also to invest in special measures to make relevant information retrievable for future use. Our observation is that the lower the development level of an organisation, the more likely it is that basic information e.g. regarding performance, is not available. It is also likely that performance has not even been defined.

Associated with the question of access to data is the need for specialist competence to search for and assess information. If specialists are needed, e.g. in the fields of statistics or telecommunications in the individual cases, the practical applicability of the model appears somewhat restricted. Our principal approach is that it should be possible to carry out the exercise without specialists in the team, but that some pre-understanding of the field is helpful. Also, it has to be taken into account that involvement of new people, e.g. specialists who may not be used to evaluation tasks or likewise, and are not acquainted with the model, would require training.

Naturally there has to be a psychological preparedness in the organisation concerned, to be measured and assessed. In our studies this has not been an obstacle. However, it should be kept in mind that it is not self-evident that the organisation is prepared to give out information not directly connected with a development co-operation project. As mentioned above, the object of the model is not the project but the organisation as a whole, and these two entities do not necessarily match. Certain information may be considered sensitive for political or commercial reasons.

4.4 Areas of development As has been repeatedly emphasised, the staircase model is a framework which can be further developed and given more details within the frames, in terms of both concepts and measurement techniques. These are some areas of development, linked to each stage of the staircase:

The concept of organisation

The first stage of the model states that there should be at least an organisational embryo, implying that there is an actor, a management and common objectives of

53

some kind. But an organisation may be more or less dependent on other organisations. A governmental agency may be part of a ministry, share its budget and staff. If the organisation is greatly dependent on bodies at higher levels, e.g. when recruiting personnel, the organisational development is affected accordingly. A question is:

To what extent does the management actually control its organisation?

The concept of output quality

The model itself does not state the content of quality in output and production. It uses the quality definitions available in the specific situation. However, it has become more and more obvious, also in developing countries, that ”quality” must include also common societal and social values, e.g. consumer safety and environmental care. Then a question to be answered is:

To what extent does the organisation fulfil demands of contribution to a sustainable development?

The concept of inside change

Change in itself is not necessarily good. The crucial issue when establishing whether the organisation has reached stage 3 is who it is that is orchestrating change. The question could be phrased in parallel to that regarding organisational autonomy above:

To what extent does the management actually govern its own change?

A way to establish the quality of change process is to study the organisation repeatedly. Development will show whether the changes have been for better or worse. In other words:

Have the changes carried out contributed to the long term development?

Another issue of internal development concerns the degree of commitment amongst staff. Motivation to participate is important. But there is a great difference between on one hand being ”motivated” by a good salary and on the other understanding the mission of the organisation and sharing the values on which that mission is based. The organisation with members committed to its mission have reached a higher degree of maturity. Thus a crucial question is:

To what extent do the individuals share the mission?

In management and organisation development we often make a sharp distinction between ”the inside and the outside”. In reality there is no such sharp distinction. Production costs have to be at level with those of competing organisations. Values inside the organisation have to be in harmony with the development of values in the society as a whole. E.g. does the organisation work for improvements in racial and gender equity? Or more generally:

Do inside changes reflect the strive for democratic development in society?

The concept of client value

54

It can be argued that client value is not a dimension of its own. Output quality reflects the value clients give the output, in its subjective or functional component. However, stage 4 of the model focuses the ways the organisation actually deals with its clients, in order for its output to be useful. Thus the model at this stage emphasises the external interaction with clients and the reflection in terms of internal attention to demands. So the ”staircase studies” made have been confined to measurements inside the organisation or its interface and not in the recipient client system. Client studies would contribute to the picture of organisation development, but do also require resources. A question which could be elaborated on is:

How is the organisation and its output perceived by the client?

The model in itself does not in any way exclude elaboration on the above questions. It is perfectly possible to shed light on both internal and external conditions within its framework. However, development of checklists and measurement tools for different application would definitely contribute. Such development is welcome!