ANALYTICAL AND NORMATIVE UNDERPINNINGS OF SOCIAL EVALUATION B. Essama-Nssah PREM Poverty Group...

ANALYTICAL AND NORMATIVE UNDERPINNINGS OF SOCIAL

EVALUATION

B. Essama-Nssah

PREM Poverty Group

September 05, 2000

2

Shape of Things to Come

• Structure of Evaluation

• Identification of Valuable Dimensions of the Living Standard

• Measurement Issues

• Ranking Social States

• Implications of the PRSP Principles for Evaluation.

3

Structure of Evaluation

• Evaluation versus Valuation– Evaluation relates to the assessment of relative

merit of an action, while valuation applies to the comparison of things.

• Fundamentals of Evaluation– Identification and measurement of valuable

dimensions of the object of evaluation.– Formulation of an overall rating on the basis of

some explicit or implicit yardstick.

4

Structure of Evaluation

What is Social Evaluation?

• Object: Extent to which prevailing social arrangements maintain or improve the living standard of the population.

• Identification and measurement of valuable dimensions of the living standard.

• Ranking of social states on the basis of the quality of life they support.

5

Identification of Valuable Dimensions of the Living Standard

Basic Approach• Specification of the valuable dimensions of the

living standard depends essentially on the underlying view about personal features that are deemed important in the realization of any life plan.

• Important distinction: “agency” and “well-being”

(Sen 1999).

6


Individual Agency

The “agency aspect” of a person focuses more on the individual as a doer, someone whose actions bring about change and whose achievements can be assessed on the basis of his or her own values and objectives.

7


Well-Being

The well-being dimension considers individual achievements and opportunities in the context of his or her personal advantage or self-interest.

8


Well-Being

Given the fundamental economic assumption that consumers purchase the best bundle of goods they can afford, the level of expenditure (or consumption) has emerged as a preferred indicator of the living standard within this approach.

9


International Development Goals• In the context of PRSP, the International

Development Goals set for the year 2015 relative to 1990 by the Development Assistance Committee (DAC) of the Organization for Economic Cooperation and Development (OECD) focus on the following dimensions of the living standard: income, health, education (with particular attention to gender equity), and environment.

10


International Development Goals• Reduction by half the incidence of extreme income-

poverty (proportion of people living on less than a dollar a day)

• Reduction of infant and child mortality by two-thirds.

• Achievement of universal enrollment in primary education

• Eliminating gender disparity in education (by 2005).

• Implementation national strategies in all countries by the year 2005 to reverse trends in loss of environmental

resources by 2015.

11


WDR 2000/2001

The above dimensions of the living standard represent the basic set of outcomes that a poverty-focused development strategy must seek. They are consistent with not only the fundamental vision of the Comprehensive Development Framework, but also with the three pillars of WDR 2000/01: (1) opportunity; (2) empowerment and (3) security.

12

Measurement Issues

• Outcomes

Result-orientation raises serious challenges in the assessment of outcomes for the purpose of impact evaluation, particularly in the areas of health, education and environment. This is not to say that the income dimension does not raise measurement issues of its own

13

Measurement Issues

Education

• The field of education has traditionally focused on the numbers of schools, the numbers of teachers, student enrollment, and repetition rates. At best, these statistics reveal the extent of educational opportunities.

14

Measurement Issues

Education

• For an informative evaluation of individual advantage and social progress, there is a need to measure the learning outcomes of education i.e. what is that people learn given the educational opportunities available to them. The content of learning achievements involves knowledge, skills, critical thinking, and values.

15

Measurement Issues

Education• National assessments based on representative

samples of student populations are emerging as preferred tools for the production of outcome indicators for educational systems.

• They provide information on cognitive and affective outcomes as well as contextual information on background variables that may affect student achievement.

16

Measurement Issues

Health

• Health status, the primary output of the health system, includes such dimensions as mortality, morbidity, nutritional status, management of fertility and disability.

• This is a result of a complex interaction between household behavior, household resources, and the community physical and social environment.

17

Measurement Issues

HealthIn assessing health outcomes, it is important to

recognize the importance of non-health sector inputs (such as the role of infrastructure), and to frame the assessment in terms of lifecycle, moving from pregnancy, through birth, infancy, childhood, the school years, adolescence, adulthood and aging, including the reproductive period.

18

Measurement Issues

Environment

Accounting for the environmental effects of development activities requires a sound understanding of ecological processes and the relationship between them and the socioeconomic system. Current understanding of these phenomena is quite limited, and this poses a serious challenge for evaluation

19

Measurement Issues

Environment• Ongoing efforts to value environmental assets

distinguish two broad categories of values, use and nonuse values.

• Use values are based on the benefits that people derive from the utilization of the assets, for instance the productivity value of good soils, the health benefits of clean water, and the protection of watersheds by woodlands.

20

Measurement Issues

Environment

• In addition, some of these resources have option values based on potential future use (e.g. biological diversity and medicine).

• Nonuse values relate to the intrinsic worth of the natural world beyond its value to human beings in terms of food, medicine, recreation, ecological services, etc.

21

Measurement Issues

Instrument

• In general, a household survey is the preferred instrument for measuring variables that are considered important for well-being.

22

Measurement Issues • Household Survey

• The relevant issues are:– Reliability of the sampling method.– Cost of operation.– Irreversibility: difficult to change mode of

operation or to use data for other purposes.– Confidentiality of some private information.– Comprehensiveness of data with respect to

household characteristics and activities (Deaton 1997).

23

Measurement Issues

Household Survey

• The design stage seems to focus more on the statistical dimensions of reliability. However, measurement issues associated with the survey instrument go beyond the design phase to include incentive issues that may arise at the implementation stage.

24

Measurement Issues

Household Survey

• Some types of households may be more likely to refuse to participate in the survey. In addition, even if a household decided to participate in the survey, there is no guarantee that it will reveal its private information truthfully.

25

Measurement IssuesHousehold Survey

• There are thus statistical (frame, probability of selection) of and socioeconomic dimensions to the reliability of household survey data. The socioeconomic dimension relates to the incentives that influence the household’s decision to participate and to reveal truthfully the necessary information. These aspects must all be taken into consideration in the design of household surveys.

26

Ranking Social States

• Accounting for Interactions– Desirable Properties of Social Arrangements.– Asymmetric Information– The Counterfactual

• Invoking Value Judgments– Dominance– Efficiency and Equity– Participation

27

Accounting for Interactions

• Evaluating public policy requires an assessment of the likely responses of the socioeconomic system to the policy (Dasgupta 2000). This is the basic idea behind the specification of the counterfactual, and highlights the fact that behavior is at the heart of the social interaction

28

Accounting for InteractionsDesirable Properties of Social Arrangements

• Social arrangements constitute the medium of interaction among socioeconomic agents.

• These arrangements (based on both explicit and implicit rules and values which affect individual behavior) ultimately determine the degree of access that people have to social, material and natural resources and their ability to transform such resources into well-being (Narayan 2000).

29


Properties of a social arrangement

• Contract as unit of analysis: Interpret any social arrangement as a contract i.e. a promise by one party (the agent) to take some action of socioeconomic value to the other (the principal), in return for some reciprocal action or reward (Dixit 1996).

30


Properties of Social Arrangements

• The living standard enjoyed by an individual may be interpreted as an outcome of an interaction between opportunities offered by society and the readiness and ability of the individual to exploit such opportunities.

• Thus any social policy may be viewed as a social contract between the individual and society.

31


Desirable Properties of Social Arrangement

Perfect Contract: complete and efficient.– Every contingency is anticipated.– All relevant information is equally shared by

participants– Risk is allocated to the party that can bear it at least

cost.– Each resource goes to the parties that value it most.

(Cooter and Ulen 2000).

32


Benchmark case in an agency relationship:

• All relevant information is common to all parties before and during the relationship.

• Result depends on level of agent’s effort (assumed verifiable) and on a random variable.

• Terms of contract set both the level of effort for the agent and associated compensation.

33


Benchmark case in an agency relationship:

• Solution: Seek the set of constrained Pareto optima that maximize the utility of the principal while the agent is held on a given level of utility, based on her or his opportunity cost (Salanié 1998).

34


Benchmark case in an agency relationship:• If the principal is risk-neutral and the agent risk-

averse, efficiency dictates that the agent receive a fixed wage.

• If instead the principal is risk-averse and the agent risk-neutral, then the optimal contract is a franchise in which the agent pays a fixed amount to the principal and receives the result of the relationship.

• When both participants are risk-averse, the contract will link the optimal distribution of risk between them to their relative degree of risk-aversion.

35


Asymmetric Information

• When one party possesses some relevant information that the other does not have, we face a situation of asymmetric information.

• Adverse selection: Principal ignores some important characteristics of the agent that affect the value of a contract (e.g. car insurance and driving habits).

36



• The key principle in solving an adverse selection problem is to design a menu of contracts such that the choice of a contract by an agent would reveal his type.

• When the actions of the agent are not observable, then a moral hazard problem arises.

37

Accounting for Interaction


• This case may be illustrated by the failures of some government program to provide social services to the rural poor because the central authority has no way of monitoring whether the services are actually provided

38


Asymmetric Information• One possible approach to dealing with the

above informational problems in the context of social service delivery is to resort to community-based delivery and monitoring (Stiglitz 2000). It is believed that community-based monitoring would be more effective than monitoring from central headquarters. Community-based provision may also be more effective because the providers share the concerns of the community and the group may invoke some form of social sanctions to enforce compliance

39

Accounting for Interaction

The Counterfactual

• The counterfactual describes the state of the world that would have prevailed had the activity (policy) not been carried out.

• Fundamentally, outcome evaluation for a given program of intervention involves a two step comparison.

40


The Counterfactual– First, the situation of the participants (the

people affected by the intervention) before the program is compared with their situation afterward to assess the extent of gains in key outcomes.

– Second, and here is where the counterfactual comes in, program participants are compared to an equivalent (control) group of non-participants to establish whether the gains of the participants exceed those of the control group (Weiss 1998).

41


The Counterfactual

(Importance of Interactions)• Example of health project (Hammer 1997).

Generically, a health project is any investment designed to improve people’s health. For a careful specification of the counterfactual, it is necessary to model explicitly the demand for, and non-governmental supply of services.

42


The Counterfactual

• The key idea here is to give full consideration to the reaction of the other actors in the health system, including: consumers, private providers, NGOs, and insurers. This way of specifying the counterfactual could reveal that the market outcome under appropriate taxes, subsidies or regulation would be better than public provision. –

43


The Counterfactual

( Health Project)

Note that, to be as comprehensive as possible, evaluation should take note of factors such

as travel time (to reach clinics), waiting time, and the courtesy of service providers.

44

Invoking Value Judgments• The judgement of the aggregate merit of a state of

affairs is based (explicitly or implicitly) on a rule for combining the different valuable aspects of the state.

• This step in the evaluative process requires aggregation over:– (1) heterogeneous components of the living standard

(income, health, education and so on), and

– (2) diverse individuals and socioeconomic groups. This raises therefore the fundamental issue of the weights to attach to each element of the aggregate.

45

Invoking Value Judgments

Dominance

• The comparison of policy outcomes may be based on the relation of dominance.

• According to this criterion, if a given state a provides more of a valued component than state b, and as much of all the components, then the dominance principle will rank a ahead of b.

46


DominanceWhen the distribution of an indicator of the living

standard is taken to represent a social state, the relation of dominance can be used to rank such distributions on the basis of various representations.

• If, for instance, the Lorenz curve for a distribution a lies everywhere above that for distribution b, a configuration known as Lorenz dominance, then

we conclude that the distribution of living standards is more equal under a than under b

47


• Dominance

• In the context of Atkinson (1970) theorem, Lorenz dominance is the necessary and sufficient condition for the reform to imply an unambiguous welfare improvement according to the class of all transfer-approving social welfare functions.

48


Dominance

• The TIP curve provides a graphical summary of incidence, intensity and inequality dimensions of aggregate poverty based on the distribution of poverty gaps (Jenkins and Lambert 1997).

49


Dominance

This curve is constructed in three steps:

(1) Rank individuals from poorest to richest.

(2) Form the cumulative sum of the absolute poverty gaps divided by population size.

(3) Plot the resulting cumulative sum of poverty gaps as a function of the cumulative population share.

50


Dominance

To illustrate the principle, consider the simple distribution x=(15, 25, 35, 45, 55) and choose z=30. The following table is a numerical representation of the corresponding TIP curve denoted J(p), where p represents the cumulative population share.

51


Table 3.1. Numerical representation of a TIP curve

Cumulative Population

Share

0.00 0.20 0.40 0.60 .80 1.00

Poverty gaps 0.00 15.00 5.00 0.00 0.00 0.00

TIP values 0.00 3.00 4.00 4.00 4.00 4.00

52


Figure 3.1. A TIP curve based on Table 3.1

53


Dominance

• TIP curves may be used to test for unanimous poverty comparisons across time, across regions and countries based on a wide class of poverty indices.

• Given a poverty line z, if a TIP curve a lies entirely above another TIP curve b, we say that a dominates b. In other words, there is more poverty in situation a than in situation b.

54

Invoking Value JudgmentsEfficiency and Equity

• Atkinson’s framework for inequality analysis illustrates how to factor efficiency and equity consideration in a social evaluation function.

• Taking income (or expenditure) as an indicator of the living standard, the approach is based on the following value judgments.

55


Efficiency and Equity• Households do not differ significantly in their non-

income characteristics (homogeneity).• The individual is the best judge of her or his well-

being (individualism).• income provides utility to income units.• more income is preferred to less.• It does not matter socially who receives which

income (symmetry).

56


• Overall well-being is equal to the average income utility across individuals (additivity and separability).

• Rich-to-poor income transfers are a good thing in the sense that they increase overall welfare (inequality aversion).

(Lambert 1993).

57


Efficiency and Equity• The bedrock concept in this framework is that of

equally distributed equivalent income.

• This represents the level of per capita income which, if enjoyed equally by every individual, would provide the same level of social welfare as the current distribution for some choice of utility function.

58

Invoking Value JudgmentsEfficiency and Equity

• The analytical expression of the equally distributed equivalent income is equal to the mean multiplied by one minus the Atkinson index of inequality.

• This abbreviated social evaluation function can be generalized by using other inequality indices such as the extended Gini coefficient.

59


Efficiency and Equity• The ordinary Gini is a member of a family of

indices represented by the extended Gini coefficient (Yitzhaki 1983).

• The normative content of this family of indices is based on such feelings as deprivation or altruism (Lambert 1993).

60


])1(,cov[)(

1

pxG

61


Algorithm for Extended Gini (Lerman and Yitzhaki 1984)

• Rank all observations in increasing order of the selected indicator of living standard (e.g. expenditure).

• Divide the rank of each observation by the total number of observations.

• Compute one minus this relative rank and raise the result to a power equal the aversion parameter minus one.

62


Algorithm for Extended Gini

• Compute the covariance between the “income” variable and the transformed rank obtained above.

• Multiply the covariance by the negative of the ratio of the aversion parameter and the mean of “income”.

63


Efficiency and Equity

• The central concern of equity is: How should a society or a group allocate its scarce resources or product among individuals with competing claims? Various theories of distributive justice provide different answers to this question.

64


Efficiency and Equity• Aristotle’s equity principle states that goods

should be divided according to each claimant’s contribution.

• Bentham’s classical utilitarianism, recommends that goods be distributed so as to maximize the total welfare of the claimants (the greatest good for the greatest number).

65


Efficiency and Equity• The difference principle of Rawls requires that

goods be distributed in such a way that the least well-off in society is made as well off as possible.

• According to the principle of equality of opportunity, a just society owes its members equal access to advantage, but the individual is responsible for turning this into actual advantage by application of effort (Roemer 1998).

66


Efficiency and Equity

• Roemer (1998) proposes an algorithm that implements the equal opportunity principle. Solutions of this mechanism are between utilitarian Rawlsian outcomes.

• This algorithm is also relevant for the design of budgetary policy.

67


Participation• In the end, the selection of evaluative weights boils

down to priority setting among the different dimensions of the quality of life and among diverse individuals or groups (within resource and institutional constraints).

• Even though the above principles offer some guidance on how to address distributive problems, It is important to derive explicit evaluative weights from a participatory approach based on open public debate involving all those who are concerned.

68

Implications of the PRSP Principles for Evaluation

Principles

• Country ownership

• Comprehensiveness

• Coherence

• Focus on outcomes

• Long-term perspective.

69

Implications of the PRSP Principles for Evaluation

Implications• Contextual approach to Evaluation

• Context: Collection of objects logically associated with the object of discourse.

• Hence:

– Make evaluation an integral part of development efforts.

– Focus on link between means and ends (outcomes and underlying social arrangements).

– Pay attention to interaction between the object of evaluation and its environment.

– Make evaluation participatory.

70

THE END.

ANALYTICAL AND NORMATIVE UNDERPINNINGS OF SOCIAL EVALUATION B. Essama-Nssah PREM Poverty Group...

Documents

Transcript of ANALYTICAL AND NORMATIVE UNDERPINNINGS OF SOCIAL EVALUATION B. Essama-Nssah PREM Poverty Group...