Criteria and methodology for Criteria and methodology for the evaluation of minority the evaluation of minority

policiespolicies

Informal International Consultative Meeting in the Area of Minority Issues, European Centre for Minority Issues, Flensburg, 16-18 September 2004

François GrinUniversity of Geneva & Education Research Unit

Structure of presentationStructure of presentation

I. Introduction: rationales for action

II. Three features of policy analysisIII. Applying policy analysis: six

« vignettes »IV. Conclusion: implications for

minority policy development

Combining different rationales Combining different rationales for action in minority policyfor action in minority policy

Principle-based: application of norms and standards, e.g. rights-based approach

Goal-oriented: selection and design of policies yielding the « best » results.

Goal-oriented: types I and Goal-oriented: types I and IIII

Type I: goal-oriented approaches give substance to principles (principles precede analytically-informed action)

Type II: Goal-oriented approaches shape principles (analysis precedes the setting of standards)

Different subcultures: the diversity cloverDifferent subcultures: the diversity clover

It is important to remember that different subcultures exist in the area of minority issues.

Four main families can be identified among professionals as well academics, centred respectively on the issues of « national minorities », « language minorities », « migrants » and « indigenous peoples ».

Interaction between these « families » and their respective « subcultures » remains limited, but needs to be developed in order to locate the most appropriate expertise and intellectual resources.

Policy v. political questionsPolicy v. political questions

What (exactly) should we do (once principles are by and large agreed upon)?

How should we do it? Choice and application of criteria Contribution of policy analysis in

three principles and six « vignettes »

Basics of policy analysisBasics of policy analysis

• Aims at creating knowledge about the consequences and performance of possible (ex ante) or existing (ex post) policies

• Stresses the cause-and-effect links between policy action (upstream) and results (downstream)

• Relies on the principle of comparison between competing options

Methodological Methodological implicationsimplications

Policy analysis therefore requires:

• Careful investigation of the cause-and-effect relationships through which policy decisions are related to results model building

• Identification and measurement of policy measures (« policy inputs »), intervening variables, direct policy effects (« outputs ») and results in terms of the processes actually targeted (« outcomes »).

Vignette #1: outcomes v. Vignette #1: outcomes v. outputsoutputs

• Outputs: direct effect of a policy• Outcome: final effects of a policy,

in terms of the variables one wishes to influence.

Policy measures produce outputs, which influence processes that result in outcomes

Vignette #2: on the nature Vignette #2: on the nature and role of analytical modelsand role of analytical models• A model is needed to connect the policy decision

« upstream » (A) to its direct « outputs » (B) through to the resulting «  outcomes » (C) that occur downstream, given certain intervening variables (D).

• B=f(A) and C=g(B, D), hence C=g[f(A), D]• Rather than a representation of reality, a model is a

tool to help us think about reality focus on causal links, with particular attention to necessary and sufficient conditions

• A model is « a metaphor whose implications have been spelled out ».

Vignette #3: dearth of targeted Vignette #3: dearth of targeted modelsmodels

Relatively little is known about the causal relationships between policy measures « upstream » and outcomes « downstream », when the outcome is (i) a defining feature of minorities, or (ii) a variable whose level is structurally correlated with minority identity or experience.

Need for model development example: the P-TOP (« policy-to-outcome path »).

Figure 3.1 THE POLICY-TO-OUTCOME PATH: A GRAPHICAL REPRES ENTATION

POLITICAL DEBATE

POLITICAL DECISION

CHOICE OF POLICY MEASURES

RML EDUCATION

RML PUBLIC SERVICES

DIRECT RML PROMOTION

CAPACITY DEVELOPMENT

OPPORTUNITY CREATION

RML US E BY SPEAKERS

RML VITALITY

FEEDBACK

FEEDBACK

PO

LIT

ICA

L L

EV

EL

PO

LIC

Y

IMPLE

ME

NTA

TIO

N

PO

LIC

Y

EF

FE

CT

S

LA

NG

UA

GE

BE

HA

VIO

UR

OU

TC

OM

ES

Source : Grin, 2002b

ATTITUDES IMPROVEMENT

Vignette #4: working backwardsVignette #4: working backwards

Identify the desired outcome (e.g.: minority language use)

Then analyse this outcome as the result of a process in which different variables operate

Then select, among these variables, those that can be influenced through policy measures.

Vignette #5: The role of Vignette #5: The role of indicatorsindicators

Indicator: measurement of a variable which is relevant to the policy-influenced causal relationship. Indicators must be:

Context-specific Non-circular Clearly located along a policy-to-outcome path

( (i) « systemic » (ii) « interrelated » (iii) [some] reponsive to policy)

Relatively easy to collect

Vignette # 6: criteria for good/best Vignette # 6: criteria for good/best practice/policypractice/policy

Processes Results

Resource allocation

Rule of law; accountabil

ity

Effectiveness

Cost-effectivene

ss

Resource distribution

Democracy Fairness

Defining and understanding criteriaDefining and understanding criteria

• Resource allocation (1): effectiveness (« making a difference »)

• Resource allocation (2): cost-effectiveness (« least-cost effect »)

• Resource distribution: fairness as result; democracy as process

• All four criteria can be measured through indicators

Implications [1]—we need Implications [1]—we need to:to:… distinguish clearly between the

outputs and the outcomes that can be associated with each option.

… develop a model linking the policy measure, its outputs, and the outcome; identify causal links, necessary and sufficient conditions, etc.

Implications [2]… and to:Implications [2]… and to:

… explicitly state the substance of « effective », « cost-effective », « fair » and « democratic » in the context of the policy/programme/project under consideration

… pick or design indicators for effectiveness, cost-effectiveness, fairness, and democracy, making sure that these indicators display the necessary properties