Active labour market measures and
entrepreneurship in Poland
Rafał Trzciński
Impact Evaluation Spring School
Hungary, 18.05.2011
case 1 – Evaluation of @lternative II case 1 – Evaluation of @lternative II projectproject• Objective of the project: preventing unemployment among
young people.• The territorial scope: 59 poviats of Poland (NUTS 4) with
unemployment rate above 20% and high unemployment among young people.
• Eligibility:• young unemployed (27 or younger), • registered at the labour office.
• Total number of beneficiaries: 5 657.• Budget: € 4 090 702.• Type of services: pre-training assistance (recruitment, needs
assessment, guidance); vocational training services linked with ECDL, both at basic and advanced level, as well as related areas; practical training (temporary employment/ on the job training organised under the agreements signed with employers; training allowances; vocational guidance and post training assistance; job-broking.
• Period of implementation: 2005-2006.
?
Selectionto the project
Self-selection to the project
Problem: High unemployment
rate among young people
@lternative IIproject
Efect: lower unemploy-
ment
Factor x1
Factor xn
SELECTION BIAS
Evaluation problemEvaluation problem
The selection bias problem in The selection bias problem in control/comparison group control/comparison group approachapproachTo estimate the impact of the
intervention we cannot simply compare beneficiaries (treated) with those who did not participate in the project (non-treated)
This is because of factors affecting both participation and outcomes.
If we don't control for those factors, we can overestimate or underestimate the impacts(picking the winners/ picking the losers).
DData used in the evaluationata used in the evaluationPULS System, which:•is used for services for the unemployed•is present in approximately 90% of Poviat Labour Offices in Poland (2006)•provides a wide range of data on each unemployed person (socio-demographics, employment characteristics, previous qualification improvement, skills etc. ),•includes a detailed history of unemployment and other activities on each person (registration in the office, deregistration, trainings, use of the benefits, etc.).
Data collectionData collection• We collected data from 55 of the 59
Poviat Labour Offices involved in the project.
• In total we managed to identify 5 065 participants of the @lternative II project (90% of all beneficiaries).
• Moreover we collected data on 126 633 persons (non-treated), which meet the formal conditions for eligibility for the project (registration in the labour office, age condition).
VariablesVariablesSocio-demographic characteristics Sex Age Marital status Single parenting Number of children Education Poviat
Employment characteristics Profession (ten categories) Number of days of work Number of professions (in total)Number of days being unemployed before participating in the projectNumber of days receiving the unemployment benefit before…Number of job offers during the one year period before…Number of days participating in subsidised workNumber of days of permanent unemployment (during the two years period before…)
Previous qualification improvementNumber of training courses, in which the person participated during the one year prior to participation in the projectTotal number of days spent on trainingHaving a work placement before participating in the project Motivation to find a job
Percentage of showing up in the Poviat Labour Office,Having the right to unemployment benefit
SkillsPossession of driving license (B category)
Back to the selection Back to the selection problem…problem…
Counterfactual action
Bearing in mind the Bearing in mind the assumptions…assumptions…
Conditional Independence Assumption
Population A
Population BCounterfactual
action
• We assume that if we can control for observable differences in characteristics between the beneficiares and non-treated population, the outcome (observable change) that would result in the absence of treatment (counterfactual action) is the same in both populations.
• Ergo, we assume that unobservables do not affect the outcomes!
Eligible non participants(N=126 633)
Control group(N=5 065)
ps= 0,6ps= 0,5
ps= 0,8ps= 0,1
ps= 0,2
ps= 0,3ps= 0,2
ps= 0,01
ps= 0,4
ps= 0,9
Beneficiaries(N=5 065)
ps= 0,8ps= 0,3
ps= 0,9ps= 0,4
ps= 0,1
Propensity score matching (1-1; nearest Propensity score matching (1-1; nearest neighbour)neighbour)
What we have achieved What we have achieved usingusing PSM?PSM?
ImpactImpact
Source: Ex-post evaluation of Phare 2003 Economic and Social Cohesion – Human Resources Development component, PAED, Warsaw 2007
ImpactImpact
Cost-benefit analysisCost-benefit analysis
• Objective of the project: encouraging business activities among unemployed people.• Beneficiaries: unemployed people (with
priority to young job-seekers).• Type of services: initial business training;
guidance on conducting economic activities; training allowance; relevant specialised training; coaching after setting up a business.• Time of implementation: 2004-2005.• Evaluation framework: the same
approach as in the @lternativa II exaple (the same methodology, source of data, analysis...).
(Counter)example 2 – (Counter)example 2 – Entrepreneurship promotion projectEntrepreneurship promotion project
Impact?Impact?
Source: Ex-post evaluation of Phare 2002 Economic and Social Cohesion – Human Resources Development component, PAED, Warsaw 2006
Lessons learned/points for the Lessons learned/points for the discussiondiscussion
What data we were lacking in both examples?◦ Missing covariates? (Are our assumptions plausible?)◦ Missing outcome variables?
What do we know and what we don't know after completing the evaluation (towards theory based impact evaluation)?
How we could modify the plan of the evaluaton to get more insight on impacts (targeting issue)?
What is the avaibility of systems such as PULS in other EU countries (looking for possibilities of implementing IE)?
What is the utility of data collected in public statistics? Do we need new data systems for IE or maybe we need to modify existing ones? (towards more systematic discussion on IE planning).
Thank you!!!
Top Related