Sims Hungarian Report (Eng)
description
Transcript of Sims Hungarian Report (Eng)
SIMS Social innovation and mutual learning
on micro-saving in Europe
Impact assessment report on the programme
implemented in Hungary
Country report
Isa ALDEHI, Léopold GILLES (CREDOC)
Aniko BERNAT (TARKI)
July 2013
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
2
1/ NATIONAL AND EU CONTEXT 3
2/ THE SIMS EXPERIMENT IN HUNGARY 7
2.1 RATIONALE BEHIND THE PROGRAMME 7
2.2 OBJECTIVES, TARGET GROUPS AND KEY ACTIONS 9
2.3 STAKEHOLDERS AND PARTNERS INVOLVED 12
2.4 RECRUITING BENEFICIARIES 13
EVALUATION METHODOLOGY 14
1/ TWO WAVES OF QUANTITATIVE SURVEY OF BENEFICIARIES AND OF A CONTROL
GROUP 14
2/ THE ANALYSES OF THE FOLLOW-UP DASHBOARD 16
3/ QUALITATIVE INTERVIEWS OF STAKEHOLDERS AND PARTICIPANTS OF THE
PROGRAMME 16
IMPACT OF THE PROGRAM ON BENEFICIARIES 17
1/ PROFILE OF THE BENEFICIARIES 17
2/ PARTICIPATING IN THE PROGRAMME 18
2.1 REASONS FOR JOINING THE PROGRAMME 19
2.2 CAUSES FOR DROPPING OUT 21
3/ IMPACT ON THE BEHAVIOUR, THE KNOWLEDGE AND THE OPINIONS OF THE
BENEFICIARIES 23
3.1 A POSITIVE IMPACT ON THE BEHAVIOUR OF PARTICIPANTS 23
3.1.1 The saving behaviours 23
3.1.2 Borrowing behaviour: a good repayment rate 26
3.1.3 Less overdraft situations for the beneficiaries of the programme 26
3.2 MORE BANK ACCOUNTS BUT LESS MONITORING 27
3.3 AN INCREASED CAUTION ON SAVINGS AND BORROWING 33
LESSONS FROM THE HUNGARIAN EXPERIMENT 36
1/ STRENGTHS 36
2/ WEAKNESSES 38
3/ KEY FACTORS OF SUCCESS 39
APPENDIX 2: INTERVIEW GUIDES 46
1/ INTERVIEW OF MAIN STAKEHOLDERS (PROJECT PILOTS, CONTRIBUTORS AND
PARTNERS) 46
2/ INTERVIEW OF BENEFICIARIES 50
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
3
Context of implementation of the project
In 2011, three coordinated pilot programmes were set up simultaneously in Belgium, France and
Hungary. These experiments, aimed at a low income audience, were designed to promote saving
through education and/or financial incentive to micro-saving. Micro-saving is part of the more
general context of micro-finance. It concerns encouraging people on low incomes to build savings
as a precaution or with a view to future investment, even though most often it involves small
amounts.
The SIMS (Social Innovation and mutual learning on Micro-Saving in Europe) project was
jointly funded by the European Commission as part of the PROGRESS 2010 programme, section 2
of which is intended to promote experiments in the social protection and social inclusion sphere.
This report gives the results of the national assessment of the project implemented in Hungary.
The assessment reports on the Belgian and French projects and an overall report and summary of
all results are also available.
The SIMS programme was rolled out in Hungary for 239 beneficiaries in 2012. Three different
saving programmes were implemented by Autonomia Foundation in rural villages amongst poor
households mainly belonging to the Roma minority. Two of the actions (CAF and Bank of Chance or
BOC) implied a collective saving programme, with a possibility to grant loans to some of the
participants. The BOC saving programme was more formalized as an account was opened for each
participant, and also included financial education and energy saving courses. The third action, IDA
(Individual Development Account), both includes a matched saving programme meant to finance
housing improvements, as well as financial education and energy saving trainings.
1/ National and EU context
Standard of living and risk of poverty and social exclusion
The unemployment rate rose continuously in Hungary in the last decade. The employment situation
improved on average of European Union between 2004 and 2008 then impaired after the 2008
economic crisis. Meanwhile, in Hungary, the unemployment rate raised regularly between 2003 and
2010.
Figure 1 Unemployment rate - annual average
(%)
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
European Union (27
countries) 9.1 9.3 9.0 8.3 7.2 7.1 9.0 9.7 9.7 10.5
Hungary 5.8 6.1 7.2 7.5 7.4 7.8 10.0 11.2 10.9 10.9
Source: Eurostat http://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=une_rt_m&lang=en
The standard of living in Hungary is lower than the average of the 27 countries of the European
Union. The median equivalised net income in Hungary represents only 33% of the average EU
income. But the part of the Hungarian population considered as being at risk of poverty on the
national standard is less important than in the EU, showing less inequalities of income.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
4
The indicator of severely materially deprivation takes in consideration the lack of at least four out
of nine items1. According to this indicator, the part of the population severely materially deprived is
much higher in Hungary than in EU (23% vs. 9%), confirming a low standard of living compared to
European standards. The part of the Hungarian population living in a very low work intensity
household is slightly higher in Hungary than in the EU as a whole.
In 2011, the risk of poverty or social exclusion2 concerns a little less than one third of the
Hungarian population, while less than a quarter of the EU population faces such a situation.
Figure 2 Population at risk of poverty or social exclusion in 2011
% of the total population
At risk
of
poverty
(1)
Severely
materially
deprived
Very low
work
intensity
(2)
Persons falling under at least one of the
three criteria (at risk of poverty, severely
materially deprived or in a very low work
intensity household)
EU-27 countries 16.9 8.8 10.0 24.2
Hungary 13.8 23.1 12.1 31.0
Source: Eurostat new release 171/2012. In 2011, 24% of the population were at risk of poverty or social
exclusion – based on EU-SILC http://epp.eurostat.ec.europa.eu/cache/ITY_PUBLIC/3-03122012-
AP/EN/3-03122012-AP-EN.PDF
(1) People at-risk-of-poverty are those living in a household with an equivalised disposable income below the risk-of-poverty threshold, which is set at 60% of the national median equivalised disposable income (after social transfers).
(2) People living in households with very low work intensity are those aged 0-59 who live in households where on average the adults (aged 18-59) worked less than 20% of their total work potential during the past year. Students are excluded.
Household debts and arrears in paying
In Europe, the increase in household debt has been the general trend for several years (with the
notable exception of Germany). Hungarian households had in the early 2000’s a very low debt-to-
income ratio, compared to the average of the Euro zone. This ratio has increased regularly in the
following ten years, and the household debt now represents almost two thirds of their income. The
trends observed in other Eastern Europe countries such as Poland, Slovakia or the Czech Republic
are similar to what is recorded in Hungary.
1 Severely materially deprived persons have living conditions constrained by a lack of resources and experience
at least four out of the nine following deprivation items: cannot afford 1) to pay rent/mortgage or utility bills on
time, 2) to keep home adequately warm, 3) to face unexpected expenses, 4) to eat meat, fish or a protein
equivalent every second day, 5) a one week holiday away from home, 6) a car, 7) a washing machine, 8) a
colour TV, or 9) a telephone (including mobile phone).
2 Persons falling under at least one of the three criteria (at risk of poverty, severely materially deprived or in a
very low work intensity household)
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
5
Figure 3 Gross debt-to-income ratio of households
(%)
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
Euro zone (17
countries) 75.1 77.1 79.6 83.0 87.7 91.6 94.1 95.1 97.6 99.7 99.0
Hungary 12.7 18.6 27.5 32.4 37.9 43.0 50.3 62.2 62.9 67.3 63.4
European Union (27 countries): data not available
Source of Data: Eurostat
http://epp.eurostat.ec.europa.eu/tgm/table.do?tab=table&language=en&pcode=tec00104
Gross debt-to-income ratio of households is defined as loans (ESA95 code: AF4), liabilities divided by
gross disposable income (B6G) with the latter being adjusted for the change in the net equity of
households in pension funds reserves (D8net).
If credit, by stimulating consumption, can contribute to economic growth, their subscription can be
problematic in case of unexpected financial difficulties. This is what happened with the current
economic and financial crisis, through rising unemployment, which led to a decline in income for
many European households. There is therefore an increase in the proportion of individuals unable
to pay their debts. The international study carried by Eurofound in 2010 on "Managing household
debt"3 shows that the risk of being in serious debt increased for people who live alone or with
young children, who are unemployed or have low income, for individuals in poor health (chronic
disease), as well as youth and migrants.
According to the Hungarian country report of the Eurofound research on household debt advisory
services in the European Union4, the main reason for widespread debt is linked with a rise in
housing costs, including energy costs. This rise in the recent years was out of proportion with the
rise of income. This happens in a general context where most of the housing stock is old, very
poorly insulated and with inefficient heating systems. Low income households are more likely to
live in bad condition either in decaying family homes or in prefab housing estates. In a survey
conducted by Tarki in 2010, 18% of Hungarian households are in arrears, most of them because of
housing-related costs. The 2009 EU-SILC5 data shows that 20% of the Hungarian population live in
a household that has been unable to pay one or more utility bills in the last 12 months.
Saving devices
The average saving rate in Hungary is slightly lower than the one observed in the European Union
as a whole. In the EU as well as in Hungary a slow a reduction of the savings rate has been the
general trend between 2001 and 2011.
3 Hans Dubois, Household debt advisory services in the European Union, Eurofound, 2012
4 Anikó Bernát (TÁRKI Social Research Institute) and Lea Köszeghy (Habitat for Humanity): Managing
household debt: Hungarian country report, Eurofound, 2011.
5 Statistics on Income and Living Conditions (EU-SILC)
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
6
Figure 4 National household saving rates in Europe
(%)
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
EU (27 countries) 12.2 12.2 12.0 11.6 11.1 11.0 10.8 11.2 13.2 11.7 11.1
Hungary 11.9 10.3 7.9 10.1 11.3 12.0 8.4 8.0 10.2 10.8 10.4
Source: Eurostat based on National accounts – GDP Gross Domestic Product
http://epp.eurostat.ec.europa.eu/tgm/table.do?tab=table&init=1&plugin=0&language=en&pcode=tsdec
240
The household saving rate is defined as gross household saving divided by gross disposable income, with
the latter being adjusted for the change in the net equity of households in pension funds reserves.
Financial inclusion as a way of combating social exclusion
The European Union has been implementing programmes and directives to promote inclusion and
financial education to help combat social exclusion.
In May 2013, the European Commission proposed a European directive to promote the access to a
basic payment account. This directive also reaffirms the link between financial inclusion and
social inclusion: “European citizens cannot play a full part in life in society if they do not have a
basic bank account”6.
The European Parliament7 and European Commission8 in 2007 and 2008 also advocated
developing financial education and especially the need to include it in school curriculums and
establish a financial education network bringing together both the public and private sectors, but
this remains at the cost of EU member countries. A group of financial education experts has also
been created by the Commission. Its brief is to publish best practices, harmonise teaching
programmes and methods and advise the European Commission on financial education questions.
There are currently no European arrangements to promote saving in the financial inclusion context
and few projects are dedicated to promoting it amongst low income groups. Although, in a
preventive approach to financial insecurity that can result in financial and social exclusion, saving is
a valuable tool in so far as it means medium/long management of a budget. Some NGOs acting in
the field find that in spite of opinions generally in favour of saving, there are numerous
impediments to opening a savings account – no confidence in the ability to save, no financial
incentive, no budget management skills. By combining an incentive to save and financial education,
the SIMS experiment is designed to offer a solution to remove these barriers and improve the
situation of low income groups.
6 Abstract from the 8 May 2013 European Commission press release.
7 European Parliament resolution number 37 dated 11 July 2007 on the 2005-2010 financial services policy –
White paper: “The European Parliament keenly anticipates the Commission’s initiatives on strengthening the
financial culture and invites it, and member countries, to double their efforts to put together specialist
programmes and web sites, preferably associating the companies concerned, but also exhorts it to make such
financial knowledge a component of basic education”.
8 Statement by the European Commission, 28 December 2008: “Consumers should have the benefit of
education in economic and financial matters as soon as possible, from school age. National authorities should
plan to make financial education a compulsory subject for school programmes”.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
7
2/ The SIMS experiment in Hungary
2.1 Rationale behind the programme
In Hungary, financial and savings education is scarce, especially for the target group of the
Hungarian SIMS. Only a few programmes were carried out in this field, and the Autonómia
Foundation played a major role in most of them (by carrying out the programme alone or with a
partner organization, such as Habitat for Humanity Hungary).
In the Hungarian villages targeted by the SIMS programme, inhabitants have difficulties to access
credit facilities, and are more likely to use a local ‘loan shark’9 if they need money.
Debt advisory services in Hungary
Due to the Hungarian Act on Social Administration and Benefits, local authorities have to provide
advice services to over-indebted households10. This obligation only concerns housing-related debts
(including utilities).
For the Eurofound research, researchers conducted interviews amongst debt advisory services in
Hungary. All people interviewed agreed that the demand for those services has increased in the
recent years. These services are compulsory in towns over 40 000 inhabitants. In smaller
communities, local authorities generally do not provide these services for two main reasons:
lack of resources;
lack of recognition of the issue.
A minority of the smaller towns and villages in Hungary manage to provide debt advisory services
to over-indebted households with the participation of NGOs. Some are part of the ‘Pilot Programme
for the Prevention of the Debt Trap’ and are funded by the European Social Fund.
Some examples of local programmes in Hungary
aimed at reducing debts or providing training on financial education
Habitat for Humanity Hungary – Financial training for low income households
“This programme is based on recognition that disadvantaged families lack the skills to plan their
family finances – and indeed lack awareness of the various loan products. By offering financial
education, Habitat for Hungary can help them make use of whatever income they have in a
sustainable way and can mitigate the risks caused by their financial illiteracy. Institutional partners
in the project include the Hungarian Red Cross, the Maltese Charity, the SOS Children’s Village and
various family crisis centers. Training takes place in groups and includes general financial education
for households, introducing such basic concepts as income, expenditure, savings, family budget
and monthly cash flow. It seeks to make families understand the importance of planning ahead and
keeping track of (or even being conscious of) their spending. The aim of the exercise is to give
9 A loan shark is a person lending funds illegally+ and/or at a very high rate of interest. Loan sharks tend to
target the most vulnerable people who have no access to regular loans via banks (as they are not regarded as
“good clients” with appropriate income). Using those services may lead poor people into a circle of debts from
which they can hardly break out. Apart from borrowing money from family, friends or neighbours, loan sharks
are often the only solution for many poor people.
10 Dubois, 2012 and Bernát, Köszeghy, 2011, op. cit.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
8
families a clear understanding of their own responsibility for their spending/saving habits, as well
as the basic tools to monitor and improve those habits. Such training sessions are also carried out
under the ‘Investment in Our Homes’ programme, based on the Individual Development Account
model and run together with the Autonómia Foundation.”
Hungarian Anti-Poverty Network – “Independent debt-prevention and debt-management
information system: This programme helps households in need of information and advice on their
financial problems. It aims to restore the financial balance and assist socially disadvantaged
households in debt with the management of their household budget. The programme was initiated
by the Hungarian Anti-Poverty Network (Magyar Szegénységellenes Hálózat). One innovative
component of the programme is the involvement of banking and social experts as counsellors. It
seeks to show why people fall into debt, to assess families’ resources, to inform them about
support options and to assist them in financial planning.”
“Kiútprogram” (“The way out programme”) funded primarily by the EU and supplemented by
funds from the Hungarian government and some major private companies like Raiffeisen Bank or
the Hungarian Telekom, aims at facilitating social mobility and integration of the Roma. The
objective of Kiútprogram is to enable people living in deep poverty, primarily the Roma, to become
self-employed by providing them with social support, financial services and information; and as a
result, help them improve their social status and make a living for themselves and their families.
The most important specific goal is to see the clients establish sustainable businesses that connect
into the regular flow of local economy. The most important tool of the financial services provided
by Kiútprogram is issuing group-based, unsecured microloans. In order to achieve these goals,
financial education is also a relevant element in the project. As this is also a pilot project, relatively
few participants were targeted.”
Source: Anikó Bernát (TÁRKI Social Research Institute) and Lea Köszeghy (Habitat for Humanity): Managing
household debt: Hungarian country report, Eurofound, 2011.
In spite of those local initiatives, in most of the areas, and especially in rural communities, the
population has no access to any of these services.
Hungary is part of the minority of European countries with no consumer bankruptcy law11.
Most of the existing services are not preventive; they tackle households already facing difficulties
in paying some of their bills.
The SIMS programme was created and implemented by the Autonómia Foundation. This
organization, created in 1990, has more than twenty years of experience. Its main aim is to
promote the development of civil society with a specific focus on poor rural regions. According to
their website, Autonómia “supports civil initiatives in which people mobilize the available local
resources to reach their goals. [Autonómia] partners with Roma and non-Roma civil organizations
in [its] development and grant giving activities amongst marginalized people12.” Autonómia
implements programmes targeting the poor which are subsidised by various Hungarian and other
11 A Consumer Bankruptcy is a court process that allows individuals whose debts are not connected to their
profesional activity to be relieved from their debts, generally after the sale of their goods.
12 http://autonomia.hu/en
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
9
European funds. Besides direct development work, the organization is active in training and
research.
2.2 Objectives, target groups and key actions
Objectives
The two main objectives of the programme are to promote saving and to provide financial
education. By encouraging savings, the experiment wishes to help poor households facing financial
crisis.
One of the aims of the programme is to help inhabitants avoiding the use of local loan sharks. Loan
sharks are informal or unregulated money lenders who usually lend money with a very high
interest rate.
This programme was mainly oriented towards the improvement of housing conditions, including the
opportunity to promote energy saving and to buy firewood for the winter season. This created
some difficulties. In the rural areas part of the firewood market is on black economy. The
administration of the programme had to insist to get bills as they were necessary to justify the
expenses, especially in the case of matching. Suddenly the prices went up and it was a difficult
negotiation to obtain a reasonable price.
For the programmes which had a collective dimension, the objective was to develop community
solidarity within villages.
Key actions
Three specific methods have been used for the SIMS project in Hungary:
IDA (Individual Development Account). This method comes from the United States of
America. Each participant defines at the beginning of the programme the total amount of
money they plan to save. The amount of the monthly saving is set every month13. A free
individual account is created for each of the participants at the local saving cooperative14. It
was generally possible to withdraw money twice from this account, under the condition that
the amount was put back on the account the next month. The programme lasted generally
from February to November 2012. At the end of this period, if participants have saved at
least for 8 months, the saving is matched : the amount saved is doubled. The money can
be used only for specific purposes related to housing. During the time of the programme
participants attend collective training courses on energy saving techniques and finance.
Bank of Chance. 6 to 10 members form a group. They meet regularly (often every month)
to save money for 6 months. An account is opened in a bank for each participant. After a
certain period of saving, some of the members can get a loan. The group agrees about who
will get the loans first. Loans have to be paid back with an interest rate (generally of 12%).
When the loans start to be paid back, other members can get a loan. In this model
Autonómia cooperates with Mikrohitel and NOBA. Mikrohitel provided the loan, NOBA
provided the internet access so that members can check their account. Each participant has
13 A minimum monthly saving amount of approximately 7€ was required in theory, but that minimum was
sometimes reduced ; an upper limit of 15 000 HUF a month (around 50 €) was also defined.
14 According to the Autonomia foundation, these cooperatives have been taken-over by the State after the end
of the programme (in July, 2013)
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
10
to attend the financial education courses organized by Autonómia Foundation. They also
have to attend training on energy efficiency.
CAF (Comunidades Autofinanciadas – Self-Funded Communities). This method was
first developed in Venezuela and widely spread in Latin America to facilitate access to
credit. No outer financial source is needed : the money comes from the shares participants
buy, which give them a right to borrow a certain amount of money (based on a multiplier of
3 to 4 depending on the groups) The money is kept in a safe box considered as a mini-bank
(no formal bank account is opened). The details of the organization are decided by each
group (value of share, interest rates15, default interest, roles of each member, possibilities
to borrow…). Each CAF group is different, their regulations vary (i.e. decision making:
some have to vote unanimously, some don’t). They have to follow how much money is in
the box and to decide who can get a loan first. Borrowing money from thr group requires a
link of trust and solidarity between members. The financial training (planning and
managing money) comes from participating in this experience.
Target population
The programme promotes saving amongst poor groups, especially the Roma since they are over-
represented amongst the poor in Hungary.
The programme was set to be implemented in nine rural settlements where more than 20% of
population is Roma. In the end, fourteen different locations were concerned by the SIMS project. In
two of those villages, several groups were operating: one CAF and two IDA programmes in one
village, and one CAF and one IDA programme in another one.
Figure 5 – Settlements where the programme was implemented
Source : Autonomia foundation
15 Usually varying from 1% to 3%
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
11
The project was aiming at reaching 90 to 120 people. In the end 239 people started the
programme, and amongst them 123 completed it.
The crucial role of mentors
The programme was implemented in the villages through the action of mentors who were the local
representatives of Autonómia. The mentors had many different tasks. First, they were in charge of
recruiting the participants. They also organized the meetings for the group (CAF and Bank of
Chance models) and visited regularly the IDA participants. They had to keep in touch with the
participants for all matters related to the programme: they had to check the participants’ bank
accounts, and to do the accounting for the savings and loans. In addition, mentors were also those
who could help the participants with financial difficulties by finding out solutions to prevent them
from dropping out. These solutions could come from the programme itself, such as delaying the
repayment, postponing savings or getting a loan, or from outsources such as local social services.
Mentors therefore were also responsible for dealing with crisis management on an individual basis.
Mentors had to send a monthly report to Autonómia to give an account of the functioning of the
group and of the events related to the programme. The Autonómia programme manager visited
them frequently. They had one day training before starting the programme and one meeting at the
end of the programme to collect their feedback.
As Autonómia has been dealing with such projects for years, they chose the mentors in the local
communities based on their own experience with them or based on recommendations. In most of
the cases, mentors worked in previous IDA or Bank of Chance programmes for Autonómia as
mentors, or were involved in other similar Autonómia projects. The social status of the mentors is
varied. Some mentors’ backgrounds are similar to the participants’, while in some cases the
mentors had a higher status. The core requirements for a mentor were to know the settlement and
the neighbouring area very well, preferably to live there or in the neighbourhood and to have many
and good contacts with the target group. Ability to understand the goal and all the details of the
programme was also taken in consideration.
Most of the mentors were women, but it was not a conscious decision of Autonómia when the
mentors were selected. In some cases Autonómia called upon their male contacts but when they
had other work and no free time, they recommended their wives or other women relatives who
also met the requirements.
The use of the money saved
Some groups decided in the first meeting how they would use their savings. Buying fire wood was
the most common goal, other groups were saving to buy a stove or to renovate their house. In
some bigger locations, several groups existed, each of them with a different aim. Some CAF groups
negotiated different individual purposes such as firewood, housing improvement and a washing
machine.
Training courses on financial education and energy saving
Five training courses on financial education and two on energy saving were organized for the IDA
and Bank of Chance groups
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
12
For the CAF model, the training came from direct participation in groups, without any outside
trainer.
Energy efficiency training courses were also organised as the energy expenses for housing play a
major part in the risk of over-indebtedness in Hungary. These poor households tend to be the most
wasteful and they could benefit from a change of attitude not only by renovating their houses but
also by having conscious energy saving activities. Those courses included elements about cheap
housing insulation, use of more efficient stoves and making of bio-briquettes from paper and
agricultural waste.
2.3 Stakeholders and partners involved
Ten different mentors were involved in the programme. One of them having worked only for four
days, this person is not taken into account in the statistics below16.
In average in 2012 each mentor spent 84 days working for the SIMS project. This represents the
equivalent of more than a third of a full time job for a year. Five mentors spent over 100 days.
If we consider that a full time job is 20 working days a month, the SIMS project created the
equivalent of three full-time jobs in 2012.
Most of the mentors were in charge of piloting the programme on one specific settlement, three of
them were in charge of two projects each, and one of the mentors was in charge of the six Bank of
Chance programme plus one of the IDA programmes (which was though not part of the SIMS pilot
experiment).
Mentors were paid 50 euros for the equivalent of a work day (eight hours of work), which
represents approximately twice the median salary in Hungary17.
In 2012, the permanent staff in Autonómia spent 172 days working on the project. This includes:
- 18 days for the SIMS programme director
- 51 days for the SIMS programme manager
- 36 days for two financial managers
- And 19 days for the financial education trainer.
Two partners from the cooperative world have been involved:
- NOBA, a peer to peer internet lending platform
- Mikrohitel (microcredit corporation)
The Open Society (Soros) Institution funded the matching for the IDA project and the loan capital
that was required for the Bank of Chance. The Levi’s foundation also founded the “self-
contribution” part of the budget.
Local authorities and social workers were also involved in the stage of recruiting participants or to
provide premises for the meetings.
16 These numbers come from calculations made by CRÉDOC on the basis of an Autonómia dashboard on the
management of their staff for the SIMS. Eight hours of work represent a day.
17 The last available Eurostat data on the median hourly earnings of employees working in a firm of at least ten
employees dates from 2010. For Hungary it represents 3.44€ per hour, that is to say 27.52€ for a work day of
8 hours. Cf. http://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=earn_ses_pub2s&lang=en
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
13
2.4 Recruiting beneficiaries
Participants were recruited by mentors who selected people amongst those most in need of the
programme in the villages. The mentors were not allowed to participate as beneficiaries in their
own group, but in some cases their relatives took part in the group. As some mentors were hardly
in a better socio-economic situation than the beneficiaries, they could join another of the SIMS
group.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
14
Evaluation methodology
The objective of the evaluation is to measure the impact of the experimentation on savings
behaviour and budgetary skills as well as on opinions and attitudes. It also analyses the
implementation processes to identify key factors of success and areas of improvements.
1/ Two waves of quantitative survey of beneficiaries and of a control group
The international evaluation aimed at creating – whenever it was possible - common tools for the
three national programmes. The questionnaire used for Hungary was in its broad lines identical to
those used in Belgium and in France.
Two waves of quantitative surveys were conducted, one at the beginning of the programme, in
February 2012, and the second wave in December 2012. The questionnaires were carried-out by
Autonómia mentors. The questions were identical in the two waves to identify the evolutions during
the programme.
In each settlement mentors recruited people who were not involved in the SIMS programme to
create a control group and compare their answers with those of the participants. Mentors were
asked to choose for the control group people whose profile was as close as possible as that of the
participants.
The response rate between wave 1 and wave 2 is rather high: 80% of the beneficiaries who
answered the first questionnaire also answered the questionnaire of the second wave. The
response rate of wave 2 of the control group is slightly lower (72%).
Figure 6 Number of participants in the quantitative survey
and response rate between wave 1 and wave 2
SIMS beneficiaries First wave Second wave Response rate
(W2/W1)
Beneficiaries 239
(123 completed
the programme)
137 110 80%
Control group - 88 63 72%
The response rate is rather similar in the three different programmes. Bank of Chance participants
answered slightly more to the second questionnaire than the participants of the two other
programmes (88% for BoC vs. 78% foc IDA and CAF).
The profile of people who answered wave 1 and 2 is quite similar.
The profile of the control group is also rather similar to that of the beneficiaries. However, the
control group counts a higher proportion of under 30, of childless couples and of unemployed
people than the participants, but the differences are low.
The analysis of the survey results meant to compare the answers between wave 1 and wave 2 for
the beneficiaries, and for the control group. When the trend of evolution is the same for the
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
15
beneficiaries and the control group, there is no reason to consider that the changes are due to the
programme.
Figure 7 Profile of the Hungarian participants to the SIMS project
who answered to the quantitative survey
Participants Control group
Wave 1 Waves 1 & 2 Wave 2
Base 137 110 63
GENDER Men 32 29 30
Women 68 71 70
AGE
Less than 30 28 25 30
30-39 32 35 30
40-49 20 23 21
50-59 16 14 13
60 and over 4 4 6
Average age 38 38 37
FAMILY
STATUS
Alone 18 17 19
Couple without children 26 25 22
Lone parent 14 13 11
Couple with children 42 45 46
Other 1 1 2
HOUSING Owner 74 73 75
Tenant / family / sharing 26 27 25
WORK STATUS
Full time worker 2 3 3
Part time worker 15 15 11
Retired 4 5 3
Unemployed, other 78 76 83
AVERAGE
INCOME wave1
In Hungarian forint (HUF) 100 616 106 188
In euros 335 356
AVERAGE
INCOME wave2
In Hungarian forint (HUF) 111 338 115 229
In euros 371 384
Source: Surveys waves 1 and 2
To avoid measuring the evolution which could be due to the change between respondents in wave
1 and those in wave 2, the results are computed on the population who answered both waves.
Sustainability is difficult to assess
It is still early to know if those programmes are going to change the saving behaviours of its
participants. Mentors were interviewed three to four months after the end of the programme. Some
of them noticed that a minority of participants went on saving; some CAF groups went on meeting.
But it seems that those more likely to continue savings are those who were already doing it before
the experiment. Most of the bank accounts open during the experiment seem to have been close to
avoid charges.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
16
Some of the mentors were asked by villagers if the programme was going to be continued. The
mentors have a key role in carrying out the programme. One of them noticed that in spite of the
strong commitment of the group participants, once he stopped being available, the group did not
carry on meeting without him. Sustainability would certainly demand a minimum budget for paying
the mentors and for maintaining travel costs for Autonómia to supervise the programmes. The
sustainability will relay on the possibility to get funding partners.
2/ The analyses of the follow-up dashboard
The dashboard provided information on the number of participants involved in each settlement and
on their saving and borrowing transfers.
The follow-up dashboards for each programme in the different locations were slightly different
depending on the type of programme implemented.
In the case of IDA, as the follow-up is individual, the information on the amount saved per person
and per month is available. For the two collective programmes, (Bank of Chance and CAF), the
number of savers and the amount saved every month is provided globally, but the data is not
available at an individual level. In some settlements the only available information is on the
amount saved or borrowed globally. The number of participants who got a loan is known for the
Bank of Chance programme, not for the CAF programme.
3/ Qualitative interviews of stakeholders and participants of the programme
In-depth interviews were conducted by Tarki, CREDOC’s Hungarian partner for the evaluation, in
the spring 2013 (March-April).
Three members of Autonómia staff were met; the managing director of Autonómia, the SIMS
programme officer and the SIMS programme manager.
Seven mentors were interviewed individually, 4 were in charge of CAF groups, 2 were in charge of
an IDA settlement, and 1 was in charge of all the Bank of Chance groups plus one IDA settlement.
In the evaluation project it was not planned to conduct in-depths interviews of participants. To
enrich the analysis, it was decided later on to do it. 4 groups of beneficiaries were interviewed. One
group consisted of individuals who followed the IDA model, one group participated in the CAF
model and 2 groups were involved in a Bank of Chance programme.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
17
Impact of the program on beneficiaries
1/ Profile of the beneficiaries
The quantitative survey (see figure 7 above) shows that the Hungarian participants were mainly
women (for more than two thirds of them). Different age groups were represented with an average
age of 38 years old. More than half of them lived with their children, mostly with a partner.
Three out of four participants were owners of their home. According to the interviews, those
houses are generally in poor condition.
A vast majority of the participants were unemployed (78% in wave 1), and when they had a job, it
was generally as part time workers. In that respect the programme reached its goal of targeting
low-income groups. In those villages, job opportunities are scarce. In some areas (for instance in
wine-growing regions) it may be possible to work as harvester but those jobs are irregular and
provide low salaries. To be able to get social benefits, it is often compulsory to participate in short
time subsidised jobs, often in gardening or cleaning, for public authorities.
Mentors often had the feeling that the situation worsened during the programme for these poor
families. The questionnaire, which provides information on the monthly income of the participants’
household, doesn’t confirm this feeling. On average, their income was 354 euros a month at the
beginning of the programme, and 371 euros at the end. This increase of 5% matches the yearly
inflation in 2012. The average income of the control group was slightly higher in wave 2, around
400 euros, but it went down to 384 euros in the second wave questionnaire.
In wave 1 as in wave 2, a vast majority of beneficiaries of the programme stated difficulties in
making ends meet. Members of the control group faced the same difficulties.
Figure 8 Difficulties for facing current expenses
Source: Questionnaire wave 1 and wave 2
People answering waves 1 and 2: beneficiaries 110 - control group 63
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
18
Those difficulties in making ends meet can lead to the unabilty to pay bills. More than half of the
participants have faced this situation during the programme (Figure 9). However, this situation is
more frequent for the control group than for the beneficiaries at the end of the programme: this
can be attributed to the programme as it offered the possibility to withdraw money or to get a
loan.
Figure 9 Inability to pay bills in the last three months
Source: Questionnaire wave 1 and wave 2
People answering waves 1 and 2: beneficiaries 110 - control group 63
The IDA participants have slightly more difficulties in paying their bills in wave 2 than in wave 1
(the part of participants mentioning those difficulties raised from 58% to 65%). For the other
participants, things slightly improved; the part of participants mentioning difficulties went from
50% to 40% for Bank of Chance and from 67% to 61% for CAF.
2/ Participating in the programme
In six settlements, Autonómia organized one or two meetings to try to start groups (5 CAF groups
and 1 BOC group), but didn’t manage to gather enough people to start, either by lack of trust or
because people felt too poor to be involved in a saving program. Autonómia also mentioned that
some of the IDA projects could not start because local banks refused to open accounts for the
participants, or due to the lack of cooperation of the banks employees. These groups are not part
of the “participants” population included in the impact assessment.
Finally, the number of people involved in the SIMS project varied depending on each specific
programme. The IDA participants were the most numerous, with 155 people who got involved in
this programme in 8 settlements. Bank of Chance and CAF had each around 40 participants (43 for
Bank of Chance in 7 settlements, and 41 for CAF in 4 settlements).
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
19
The dashboard shows that the proportion of persons who saved amongst those who applied to the
programme is very high for CAF (95% of participants saved at least one), and rather high for Bank
of Chance (77%). It is much lower for IDA (33% of participants saved regularly enough to receive
the matching, and 23% never saved).
Figure 10 Participants who were active during the programme
Source: dashboard – CREDOC calculations
In the IDA program, the rate of savers varies from 8% to 91% depending on the settlements.
Amongst those who gave up, some were part of an active settlement in the programme but they
never started saving, others saved for a very short time (generally one, sometimes two or three
months) then withdrew all their assets before the end of the programme.
The dashboard shows that the IDA program was implemented with flexibility in some of the
settlements. For instance, in principle, the amount saved per month was supposed to be identical
every month. However, when some participants skipped a month of saving, they could stay in the
programme by doubling their saving the next month. In some villages, part of the beneficiaries of
the IDA program did not always save the same amount but saved more when they could
(especially after the summer period when some participants certainly could earn more money
thanks to seasonal work in agriculture). A minority of the IDA participants withdrew money from
their account and then put it back the following month.
2.1 Reasons for joining the programme
According to the mentors, recruiting was not always easy because of the difficulty to convince
people that the programme was trustworthy. Some villagers had heard of frauds when the people’s
savings were stolen and they were suspicious. The mentors often had to explain the programme
over and over to convince the villagers. There was at the same time scepticism and a great desire
to believe in the programme. This is expressed by one of the IDA participants who was
interviewed. In this village participants didn’t hesitate to start the programme even if they
seriously doubted that the Foundation would double their savings – “but I started it, because I was
hoping”. They also met frequently in the village and helped to convince each other that Autonómia
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
20
would double the money or at least wouldn’t steal the amount they saved - “they held together: it
will succeed”. But they really believed it when they finally got the firewood.
One of the Bank of Chance groups which was interviewed had already participated in a previous
Autonómia programme. They already knew each other and the mentor and didn’t hesitate to
participate: “We thought it would be good, so we started it”. Their village is very poor so they felt
they had to take every chance: “It’s such a poor village, you can’t even leave”.
Having already participated in an Autonómia programme and knowing mentors and most
of the participants were factors favouring the commitment to the programme. As one of
the Bank of Chance group explained, they knew each other, some of them were from the same
families (uncles, cousins), and so it made it easier to cooperate. An IDA mentor explained that one
of the motivations was the matching. Participants knew that it would be very difficult to gather
enough money to buy the winter firewood. This purchase was the aim of this group. Usually those
people get loans from the saving cooperative but with high interest rates. Poor families who cannot
afford to buy wood sometimes have to steal it in the forest, with all the risk associated to this
illegal practice.
The strength of group dynamics
The main reason for following the programme was the financial help, but it was not the only
positive aspect. For the two programmes which involved group participation, the group dynamics
really helped to motivate the participants. The general opinion was that the atmosphere was
friendly, the meetings went well and participants were happy to meet and discuss. A CAF mentor
said that as most of the participants were housewives with children, the programme was a nice
opportunity to get outside their home, to learn new things and to socialize. The meetings were the
opportunity to discuss everyday problems. Other issues were discussed and from time to time the
meetings became a broader social event. Another mentor, also in charge of a CAF group, noticed
that the programme helped “getting people closer” and raising solidarity. Some people now visit
each other regularly, help out each other for baby-sitting, doing small jobs in the house or cutting
wood.
Some CAF group organized social events such as excursions, cooking sessions, or a cinema outing
with all the participants, thanks to the money provided by Autonómia, or sometimes with the
money left at the end of the programme18. These outings were a great success and were highly
appreciated by group members. They reinforced the positive aspect of group dynamics.
For the two programmes including the possibility to get a loan (CAF and Bank of Chance), the
groups had to discuss to know who could get the loan first. Most of the time, it did not create any
problems. One of the CAF group explain their experience on this issue: the CAF meetings went fine,
they were able to decide who can get money out. When they realized that too many members
needed loans “we discussed that to share the money equally”. In a Bank of Chance group which
was interviewed, one of the participants explained: “We didn’t care about who gets the money first
– there were no quarrel or things like this”. The group confirmed that the decision about who
should get the loan was always easy, the group decided and there was “no hard feelings”.
Most of the mentors confirmed that there was no tension in the groups when they were discussing
about who could get the loans before the others. One of the CAF mentors said she was surprised
18 This money could come from the interest rate for those repaying loans, or from little fines collected for those
who missed a meeting or training with no good reasons or those who were late in paying back the loan.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
21
and glad that they could work as a group. They lent money for those who needed it, they stuck to
the rules and no one left. This group met every week but when a participant needed money they
had to gather and vote whether she or he could have the money. According to the stakeholders, no
application was ever rejected, in spite of the need for unanimous vote required in this group for
each decision.
Different opinions on the training courses
Concerning financial education trainings, opinions on the training courses were not
unanimous. It is sometimes presented as a reason to stay in the programme, and in other cases
the interest of participants was more limited. Some of the participants and some mentors describe
the content as interesting but not always easy to use, sometimes not adapted to their needs. One
of the Bank of Chance group participant interviewee said: “You can save money if you don’t eat”
when it comes to “such poor people like us”. Some participants said they could not remember
properly the lectures. Similarly, in another group (CAF group), the trainings were judged enjoyable
to attend, but not really adapted to their needs. The group members do not believe that any bank
would lend them money as they are quite poor. This idea was shared by one of the IDA participants
who considers that they are not a target group for banks: “Do you think that they will give credit to
a public worker anywhere?”. Generally the trainings on banking credit services were not so
successful because participants found them difficult to understand and far from their everyday
needs. Trainings on saving practices were more successful. Autonómia Foundation plans to improve
the format of these trainings in the future to include more interactive role play and case studies
(such as “participatory budgeting19”).
Training courses on energy saving were considered as more practical and better received.
This was the case especially concerning courses related to insulation techniques and rocket stoves.
Some participants plan to use bio briquettes, others are more doubtful. In one of the groups, the
opinion was that they are expensive to produce and not as good as wood: “It doesn’t give you
warmth [while firewood] is warm like hell”. So they were not convinced.
2.2 Causes for dropping out
According to the in-depths interviews conducted with participants and mentors, one of the main
drop-out factors was the lack of trust in the programme, especially with IDA and the reality
of the matching at the end. Some people had heard of “pyramid game” societies that collected
money with the promise of very high interest rate and disappeared with the assets. One of the
targets of saving groups was to buy a new type of stoves which were built locally with Autonómia
support. Some of the participants were worried about the efficiency of these stoves and thought
there was a risk of seeing this stove blow up.
The other main reason to give up the program was financial difficulties. In the case of the IDA
programme, a minimum of 2 000 HUF (around 7€) a month was necessary for savings, and for
some of the participants this amount was too difficult to put aside. Some had cuts in their income,
often because of the loss of jobs, which made them unable to carry on their savings. Sometimes
19 Participatory budgeting is a process of democratic deliberation and decision-making, and a type of
participatory democracy, in which ordinary people decide how to allocate part of a municipal or public budget.
Participatory budgeting allows citizens to identify, discuss, and prioritize public spending projects, and gives
them the power to make real decisions about how money is spent (source : wikipedia)
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
22
participants faced unexpected expenses, due to health problems, and they had to withdraw their
asset. In case of financial difficulties, CAF and Bank of Chance could provide a loan to overcome
hard times, but if the situation lasted, participants could hardly pay back and had to leave the
programme. In those cases the mentors managed to get the money back in the next months
(sometimes they had to be “pushy” as one of the groups explained), but it was too late for the
participants to go on with the programme.
Some drop-outs were the result of religious issues. In one of the settlement, the religious leader
turned against the programme and convinced most of the participants to quit in spite of several
rounds of negotiations between this leader and Autonómia. In another settlement, participants
were belonging to two different churches and this created conflicts between them. The meetings
turned into theological debates and created hostility until it was not possible to maintain the group
together. The result was the end of the programme in the settlement.
In one of the settlements, the mentor took advantage of his position to start campaigning for his
election as the leader of the local minority, against another mentor in a neighbour village who was
also candidate for this position. The two CAF groups didn’t continue their activities.
Why a higher rate of drop outs for IDA?
Several factors may explain the higher rate of drop out in IDA:
- Contrary to the two other programmes, IDA is an individual programme. The only group
activities were the training courses. Group dynamics could not play as much its role of
motivation in the behaviour of participants.
- The amount put on the account had to be fixed at the beginning of the programme.
Because the matching was potentially quite interesting, some participants may have
chosen a too ambitious goal and did not manage to maintain their savings during the eight
months requirid to get the matching.
- The IDA programme could not give the opportunity to get a loan, and did not provide any
help in case of an unexpected need of cash during the programme.
- The rules were less flexible for IDA. In theory, the only way to get the money back before
the end was to stop participating.
Some IDA mentors negotiated with Autonómia to change the rule and have more flexibility: several
possibilities to withdraw money or to skip a month, changes in the amount saved according to the
household cash flow. When this was made possible, the proportion of participants staying until the
end in the programme was higher.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
23
3/ Impact on the behaviour, the knowledge and the opinions of the
beneficiaries
3.1 A positive impact on the behaviour of participants
3.1.1 The saving behaviours
IDA participants who received the matching managed to put aside 125 euros20 each on average in
2012 (the average amount is 51€ for those who saved at least once). Bank of Chance participants
saved 91 euros on average. The amount put in the CAF programme was much lower : participants
bought the equivalent of 13 euros of share on average per year, usually at the beginning of the
programme in order to get the group started.
In the three programmes the saving rate tended to slow down during the year. In the case of CAF
and Bank of Chance, this decrease is due to the replacement of a saving behaviour by a payback
behaviour after receiving a loan. In the case of IDA, this is due to the drop-outs of participants who
did not manage to maintain their saving behaviour.
Figure 11 Monthly saver rate*
Source: dashboard – CREDOC calculations
* For the CAF programme, the data from the dashboard did not provide the number of participants who put
money aside monthly but the number of share bought (each participant could buy zero, one or more share each
month. The figures for the CAF programme show the average number of shares bought for 100 CAF
participants.
20 This calculation takes into account the withdrawals made by the participants from there saving accounts. On
average, active participants placed 171 euros and withdrawed 46 euros so here we consider that they managed
to save 171-46 = 125 euros.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
24
The averaged monthly amount saved by participants who did a deposit on these months tends to
decline after a few months for Bank of Chance. It may be that the opportunity to get a loan after a
few months reduces the motivation in savings.
For the CAF programme, people bought shares within the first few months : this is when
participants in each group started building their collective asset.
For the IDA, the monthly saving tend to increase after the summer months and until the end of the
programme. This increase may be due to more income during the summer months by participants
who could find seasonal jobs and by the prospect of getting a matching at the end of the
programme.
Figure 12 Average monthly deposit amongst those who saved (IDA, BoC) / amongst
active participants (CAF)
Source: dashboard – CREDOC calculations
The habit of saving was very low when the programme started, less than 5% of the participants
had saved monthly in the past three months before wave 1, and a quarter of the beneficiaries had
saved monthly in the previous 3 months in wave 2. This can be seen as a direct effect of the
programme since the behaviour of the control group did not change. Also, it is important to bear in
mind that this results is underestimated since some of the participants were not saving at the end
of the programme because they received a loan and were paying it back.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
25
Figure 13 Monthly savings in the previous three months
Source: Questionnaire wave 1 and wave 2
People answering waves 1 and 2: beneficiaries 110 - control group 63
The survey results show that the proportion of savers has risen between waves, but the average
amount saved in the past three months for those who saved did not change much (from 68€ to
72€).
An indicator sums up the evolution of savings: we looked at people who saved more often at the
end of the programme (from no saving to saving at least occasionally, or from saving occasionally
to saving on a monthly basis). 37% of participants saved more often in wave 2 than in wave
1. This is quite more than in the control group where only 4% of them saved more in
wave 2 than in wave 1.
There are two main situations explaining the rise in the intensity of saving:
- People didn’t save in the months before wave 1 and said they saved either from time to
time or every month in the three months before wave 2. They represent 32% of
participants.
- People went from saving from time to time to saving every month (5% of participants).
Compared to those who did not change their saving behaviour, the persons who have increased
their saving behaviour are:
- more often women (87% vs. 65%)
- slightly older (average age 41 vs. 37).
Their average income is identical in both groups.
Those who did not increase their rhythm of saving represent 63% of the participants. Most of those
who did not change their behaviour are non-savers in wave 1 and in wave 2 (those non-savers
represent 49% of participants). Only 11% save less regularly in wave 2 than in wave 1.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
26
Different opinions in the interviews about the changes in behaviour
The interviews of some of the mentors reveal very positive opinions on the effect of the
programme on saving behaviours. One of the CAF mentors said she had the feeling that the
members were more aware of their spending habits. Another mentor thought the programme
changed the participants’ behaviour. She encouraged them to list expenditures and this was
discussed during the meetings. The mentor in charge of the Bank of Chance programme thought
that a majority of the participants tried to pay attention to their expenditures after the trainings.
Some CAF groups went on saving after the end of the programme.
Other mentors thought that those who went on saving were those who were already saving before
the programme.
3.1.2 Borrowing behaviour: a good repayment rate
As already said, The IDA programme did not open the possibility to get a loan. According to the
dashboard, the total amount of the money borrowed by the participants of the Bank of Chance
programme represents 5 490€. The total saving of the Bank of Chance participants represents 58%
of the amount lent21. 24 loans were allocated, with an average amount of 229€.
The CAF programme allocated 920€ in loans for all its participants : the total savings also represent
59% of the total loans. The number of loans is not stated in the dashboard.
The mentors interviewed in those two programmes mentioned that there was no difficulty
regarding the loan repayments. Even those who left the programme paid their loan back,
sometimes with a little delay.
3.1.3 Less overdraft situations for the beneficiaries of the programme
In spite of difficult financial situations, beneficiaries have managed to avoid having an overdrawn
account. These situations have decreased for them, while those situations have been more
frequent for the control group. This could be a positive aspect of the SIMS programme. The saving
behaviour, the matching for IDA or the possibility to get a loan for Bank of Chance or CAF
beneficiaries certainly helped participants to face their expenses during the SIMS programme.
21 When participants start paying back, the money from the loans can be lent to other beneficiaries of the
programme. This is why the amount lent is more important than the amount saved.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
27
Figure 14 Overdraft situations in the past three months
Source: Questionnaire wave 1 and wave 2
Note: only people with a bank account were asked this question.
People answering wave 1: beneficiaries 68 - control group 37
People answering wave 2: beneficiaries 78 - control group 40
3.2 More bank accounts but less monitoring
Around six people out of ten already had a current bank account when they were interviewed for
wave 1. After the programme, the part of the beneficiaries having a bank account increased (from
62% to 71%). There is no such a progression in the control group (Figure 15).
Figure 15 Possession of a bank account
Source: Questionnaire wave 1 and wave 2
People answering waves 1 and 2: beneficiaries 110 - control group 63
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
28
The proportion of bank account owners was extremely high for Bank of Chance participants at the
beginning of the programme: this is due to the fact that an account was opened for each
participant when entering the pilot. For the CAF programme where no bank system is used, the
proportion of bank account owner did not change during the experiment. The overall increase
comes from the IDA participants.
Figure 16 Possession of a bank account in each programme
Have a bank account Wave 1 Wave 2
IDA 48% 63%
Bank of Chance 97% 97%
CAF 53% 56%
Source: Questionnaire wave 1 and wave 2
Saving accounts less exceptional after the programme
When people had a bank account, they were asked about the possession of another account
specifically for savings. The proportion of households having a specific account for saving has
increased dramatically between wave 1 and wave 2 (from 6% to 48%), which is certainly a direct
effect of the programme. In the meantime, there is no change for the control group: only 3% had
a saving account on both waves.
Figure 17 Possession of another bank account for saving
Source: Questionnaire wave 1 and wave 2
Note: only people with a bank account were asked if they had another account for putting money aside.
People answering wave 1: beneficiaries 68 - control group 37
People answering wave 2: beneficiaries 78 - control group 40
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
29
Most of the people who consider thay have a specific account for putting money aside at the end of
the programme are Bank of Chance participants. Almost all of them have a precise idea of the
amount of money on their account and six out of ten know how much they could earn with their
saving account.
A good knowledge of the cost of running a bank account
Amongst those who have a bank account, the SIMS participants and the control group share a
good knowledge of how much the bank charges for running this account. This was already the case
at the first wave, where more than nine people out of ten knew the cost of running their account. It
is still the case in wave two.
Figure 18 Knowledge of the cost of their bank account
Source: Questionnaire wave 1 and wave 2
Note: only people with a bank account were asked this question.
People answering wave 1: beneficiaries 68 - control group 37
People answering wave 2: beneficiaries 78 - control group 40
A distant monitoring of the bank account
Participating to the SIMS programme did not lead to a closer monitoring of bank accounts. The
proportion of participants monitoring their bank account every week, which was already low, even
declined.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
30
Figure 19 Monitoring weekly the amount on the bank account
Source: Questionnaire wave 1 and wave 2
Note: only people with a bank account were asked this question.
People answering wave 1: beneficiaries 68 - control group 37
People answering wave 2: beneficiaries 78 - control group 40
Fewer participants check “every time” if there are errors on their bank account. This tendency of
decrease over time is shared by the beneficiaries of the SIMS programmes and the control group.
Figure 20 Checking errors on the bank statements
Source: Questionnaire wave 1 and wave 2
Note: only people with a bank account were asked this question.
People answering wave 1: beneficiaries 68 - control group 37
People answering wave 2: beneficiaries 78 - control group 40
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
31
This trend in checking less accurately the bank account on wave 2 than on wave 1 is not due to
new bank account owners: the participants who opened an account between wave 1 and wave 2 do
not monitor less closely their account than those who already had a bank account.
This could be due to a change in the understanding of what means a close monitoring after the
financial trainings and not to a real change in the behaviour. Participants could be more aware of
what checking a bank account really means.
No improvement on the planning and monitoring of the budget
During the first wave of questionnaires, less than four participants out of ten had a precise idea of
how much they were spending every month. This proportion did not change after the programme,
while it slightly increased for the control group.
Figure 21 Precise knowledge of the monthly expenditures
Source: Questionnaire wave 1 and wave 2
People answering wave 1: beneficiaries 68 - control group 37
People answering wave 2: beneficiaries 78 - control group 40
Another question confirms that on average SIMS beneficiaries did not improve the monitoring of
their expenses. The proportion of those who said that they tracked their expenses decreased
between both waves. Meanwhile the control group trend shows a slight improvement. Again, an
explanation of this result could be that the idea of what it means to track expenses has become
more precise after participating in the programme. This could explain why they don’t qualify their
behaviour as a real monitoring in wave 2.
This decrease is mainly due to the CAF participants: two out of three said they were tracking their
expenses on an ongoing basis in wave 1, only one quarter of them said they did so in wave 2.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
32
Figure 22 Regular tracking of expenses
Source: Questionnaire wave 1 and wave 2
People answering wave 1: beneficiaries 68 - control group 37
People answering wave 2: beneficiaries 78 - control group 40
Similarly, there is no improvement in the anticipation on the expenses and income. The proportion
of people preparing a budget is lower for beneficiaries as well as for the control group (see figure
23 below). This proportion dropped down even more for the CAF participants: half of them said
they were preparing a budget on a monthly basis in wave 1, and only one person (out of 16 who
answered this question on wave 2) said so.
Figure 23 Preparation of a budget
Source: Questionnaire wave 1 and wave 2
People answering: beneficiaries 110 - control group 63
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
33
3.3 An increased caution on savings and borrowing
Attitude towards borrowing: beneficiaries more aware of the risks
Beneficiaries of the programmes seem more careful after attending the programme and more
conscious of the potential risk of borrowing. The proportion of those agreeing that borrowing is not
a problem as long as you can repay the loan has decreased from 85% to 71% between the waves.
It is much more than what is observed in the control group.
Figure 24 Opinion on ‘borrowing is not a problem’
Source: Questionnaire wave 1 and wave 2
People answering: beneficiaries 110 - control group 63
More and more people agree that « when you have a bit of money, it's better to give it to your
family or close relations when they need it rather than saving it for yourself ». This increase can
be observed for the beneficiaries as well as for the control group. Family solidarity seems to
become more and more important when time goes and the economic crisis lasts.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
34
Figure 25 Opinion on ‘saving for oneself’
Source: Questionnaire wave 1 and wave 2
People answering: beneficiaries 110 - control group 63
On wave 1, the SIMS participants were more inclined to have a positive opinion on savings than
the control group, and their answers did not vary much between the waves. On wave 1,
beneficiaries were more likely than the control group to:
- Disagree about the opinion that there is no point in saving small amounts.
- Disagree about the opinion that saving brings no money
- Agree that putting money aside avoids the need to borrow
- Tend to think that banks refuse to lend money to low income households.
It is quite likely that people who entered the programme had a better opinion on savings than the
control group, which is formed with people of the same villages who did not wish to participate to
the SIMS experiment. On most of those issues, opinions did not vary much between the two
waves. It is possible that at the date of the first wave (January 2012) they were already convinced
by the mentors’ arguments about the interest of savings and alternative banking, as they had
already been approached by them to be convinced to enter the programme.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
35
Figure 265 Opinions on saving and credit
Beneficiaries Control group
Wave 1 Wave 2 Wave 1 Wave 2
There's no point saving small amounts Disagree 72% 77% 64% 65%
Saving brings in no money Disagree 84% 83% 67% 69%
Putting money aside avoids the need to borrow Agree 91% 85% 72% 85%
If you really want to save, you must put money aside on a
regular basis Agree 97% 93% 88% 95%
If you want to put money aside, you need a saving account
in addition to your current account Agree 65% 67% 52% 55%
One should avoid borrowing, it’s too risky Agree 88% 86% 81% 83%
Very few banks lend to people with small revenue Agree 71% 70% 43% 46%
Source: Questionnaire wave 1 and wave 2
People answering waves 1 and 2: beneficiaries 110 - control group 63
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
36
Lessons from the Hungarian experiment
The SIMS programme in Hungary aimed at enhancing saving behaviours amongst poor families in
rural settlements, with a high percentage of Roma population. In total, 239 participants were
involved in the programme. A vast majority of the Hungarian villagers where the experiment took
place have great difficulties in facing their current expenses. Three different methods were used :
two of the methods, Bank of Chance and CAF (Comunidades Autofinanciades - self-funded
communities) were conducted with groups and associated savings and the possibility to borrow
from the group or the bank. The third method, IDA (Individual Development Account), is individual.
After regular savings –at least for eight months during the year of the experiment –the participants
get a matching, doubling the amount saved.
Local mentors were recruited and trained by Autonómia to implement the programme. For IDA and
Bank of Chance financial education and energy efficiency courses were compulsory to attend. In
the CAF programmes, the training came from direct practice of dealing with the savings and taking
decisions about lending, deciding of interest rates and of fines in case of delay.
The evaluation is based on three types of data:
- A quantitative survey with a face-to-face questionnaire carried out by mentors in two
waves (February and December 2012) with 110 participants. The same questionnaire was
answered by a control group of 63 persons with comparable profile living in the same
villages as participants.
- A follow-up dashboard of the action, providing monthly information in each settlement and
each method on participants, savings, drop-outs and loans.
- Qualitative interviews with participants, mentors and Autonómia managers in spring 2013,
at the end of the programme.
As a whole, the interviews show a high level of satisfaction concerning the programme. The
programme is described as “friendly” and flexible enough to adapt to the changes of situations
such as improvement of deterioration of the capacity to save because of variation in the level of
income. The main reason of satisfaction is that participants managed to satisfy their needs:
firewood, stove, or material for improving their home. The mentor of an IDA programme confirmed
that the extra money provided at the end of the project was a major help.
1/ Strengths
Positive impact on saving behaviours
The objective of developing a saving behaviour was clearly reached: less than one person out of
twenty were saving in the previous months before the beginning of the programme, and a quarter
of them saved every month at the end of the programme. As a whole, 37% of participants have
increased their rhythm of savings, as opposed to only 4% in the control group. This increase in the
saving behaviour is a direct result of the programme as the percentage of savers in the control
group remained very low.
On average active participants saved 125 euros for the IDA programme , 91 euros for the Bank of
Chance programme and 13 euros for the CAF programme. In the IDA programme, participants who
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
37
saved for at least 8 months could double their amount with the matching. This created a strong
incentive in the amount saved, for those who did not drop out.
The proportion of savers is higher during the first few month of the programme and decrease
throughout the year : in the CAF and Bank of Chance methods, the first months are a time of
savings and the last months are a time of paying back the loans made possible by the previous
savings of the group.
Strong effects of the group dynamics
The drop-out rate was very low for the CAF programme (5%) and quite low as well for the Bank of
Chance programme (23%). It was much higher for IDA (67%) where most of the actions are
individual. The very low drop-out rates in both collective programmes show the power of the group
to keep the motivation at a high level. The interviews confirmed that speaking with other members
of the group was a way of reinforcing the trust in the programme and the motivation to go on.
Participating to those programmes enhanced the social integration and empowered the inhabitants.
It showed the participants that their situation could be improved and that savings were possible in
spite of their very hard socio-economic condition.
The CAF programme has a community strengthening function as all the decisions are made directly
by the group without any intermediaries. It provides a strong opportunity to discuss in order to
reach a group decision.
The interviews revealed several situations where the programme created strong links between
members who started socializing and helping each other (for babysitting, renovating the houses or
cutting wood…). The social events were very much appreciated as some of the participants are
housewives who do not always get an opportunity to go out. Through the objective of savings
those programmes developped the community and gave an opportunity to help each other.
When loans were distributed, the repayment rate was very high (close to 100% for most of the
CAF groups). This can be explained by the commitment of participants in the programme, who
knew that the possibilities for other members of the group to get a loan were depending on the
repayment of the first members who had been able to borrow money. This is another sign of the
strong solidarity amongst members developed by the programme.
The training courses also took part to the building of community dynamics. During the IDA
programmes, they were the only moments when the groups were together. According to the
interviews, most of the training courses were directly useful for participants, especially the parts
which were practical. One of the CAF mentors said that practicing money saving techniques in the
group was an efficient way of enhancing their knowledge and helped group empowerment. Another
mentor, who used the CAF programme earlier on, thinks that it teaches people to handle even
small amounts and has a strong community function.
Opinions vary on the impact of the training courses. Because of their lack of link with the banking
system, some of the participants in the financial training courses had the feeling that part of the
training was not adapted to their needs. There were also difficulties in understanding the
theoretical aspects of the training courses. The training courses could be better received if they had
been more practical and more adapted to the level of the audience.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
38
Less overdraft and more caution on borrowing
Another positive effect of the programme is that in spite of a hard economical context, more
beneficiaries have avoided overdraft at the end of the program. Meanwhile more persons from the
control group experienced an overdraft of their bank account. The percentage of participants who
were unable to pay some bills remained stable during the year 2012 while this situation worsened
for the control group. On those two aspects, the comparison between the control group and the
participants tends to show a positive effect of the programme. The matching in the case of IDA and
the possibility to get a loan for Bank of Chance and CAF participants certainly helped facing their
expenses.
The SIMS participants show more caution at the end of the programme concerning borrowing, the
proportion of those agreeing with the sentence “Borrowing is not a problem if you know you can
repay the loan” dropping down from 85% to 71% during the pilot programme. In the same period
this opinion was only very slightly reduced for the control group (from 88% to 84%).
2/ Weaknesses
A high drop-out rate for the individual programme
The IDA programme had a high rate of drop-outs. The lack of group dynamics certainly created a
less motivating environment. Other factors may have contributed to this higher drop-out rate. It is
possible that the prospect of doubling the assets by matching encouraged participants to save at a
level which was sometimes more than what they could assume for several months in a row. In
principle this programme was less flexible and participants had to continue saving the same
amount every month. It was sometimes difficult to withdraw money without quitting the
programme, which created difficulties when the participants had to face unseen expenses or an
unexpected shortage of income.
On a longer run, there is a risk that people who did not succeed in remaining in the programme wil
be discouraged to enter another saving programme by fear of another failure. The strength of the
programme is that it targets poor and vulnerable inhabitants. This is also one of its weaknesses as
it may be difficult for those households to put money aside. Those families are likely to give up in
case of income cuts or unexpected expenses.
No improvement in financial monitoring
More participants of the SIMS programme have one or more bank account at the end of the
programme. But their monitoring does not seem to have increased. If they quite often know the
cost of their bank account at the end of the programme, it was already the case at the beginning of
the experiment, and this knowledge is shared by the control group. SIMS participants have less
regular contact with their bank advisor; they do not monitor closely their bank account and did not
develop the habit of checking very regularly if there are errors on their bank account.
The knowledge of the amount spent every month is not more accurate at the end of the
programme: less than half of the participants know precisely how much they spend every month. A
minority only tracks their expenses on a regular basis. They are even less often preparing a budget
at the end of the experiment. On all these points their behaviour is not different from the control
group. It may take longer to really change the habits in this field. Introducing a habit of writing a
budget may be a difficult goal for poor people who are not using writing culture in their everyday
life.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
39
3/ Key factors of success
Trust in mentors from the local community
One of the key elements for the success of the programme was the ability to produce trust so that
villagers joined and remained in the programme. The mentors played a crucial role for creating and
maintaining this trust. Their first role was to ease the community anxiety. They were present for
re-motivating participants and also for trying to find solutions in case of financial difficulties. They
negotiated with Autonómia to create more flexibility in the programmes to avoid drop-outs (for
instance to make it possible to deposit different amounts in the IDA programme according to the
financial situation).
On the contrary, if mentors may encourage taking part in the programme when they manage to
win participants’ confidence, they may also block the recruitment should the opposite occurs. In
small settlements, rumours may spread quickly and dissuade people from taking part. This may
happen when mentors are not well known and when inhabitants don’t have previous experience
with the Autonómia foundation. The rivalries between mentors for political reasons and religious
disagreements between group members have contributed to break off some of the groups.
The power of groups
Belonging to a group helped reducing worries about the reliability of the experiments. Trust was
reinforced. The collective programmes had a much lower rate of drop-outs, and mentors insisted in
their value for developing community empowerment. This is why it seems preferable to favour
collective actions where the power of group dynamics is strong.
Targeting the method according to the public specification
The CAF programme has a lot of flexibility as the rules are defined by the group. It is faster and
more efficient to get loans as there is no intermediary. But it seems to be less incentive for
savings: the amounts collected are less important. Therefore the number and amounts of loans are
more limited, so the financial help provided by CAF is less important. The help to realise goals is
more limited than in the Bank of Chance. As one of the mentors said, “CAF is better for the poorer,
while Bank of Chance has a bigger perspective”.
Improving flexibility
Dropping out may occur when it is difficult to adjust to unexpected variations of income or
expenses. Poor villagers often have to face an unstable economic environment. This is why
flexibility in the duration of the programme, in the amounts saved from one month to another, and
in the possibility to withdraw money may help maintaining the participation.
Some mentors suggested that it would be better to adjust the timing of the programme to the
projects that are to be financed by savings / borrowings, mainly before the beginning of the winter
season, in particular buying wood or fixing the house.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
40
Appendix 1 : questionnaire
I - Your budget: your income and expenses
1. What are your current sources of income
(Several answers are possible)
Your salary
Social benefit
Casual jobs
Your family or friends
Other (specify) ___________
2. How much money do you have available for your household every month on average
(all sources of income including your salary, social benefits, help from your
relatives…)?
Please try to give an amount, or at least to give an estimate using the scale below.
_____________£
1500 £ and more
1000 to 1499 £
500 to 999 £
300 to 499 £
Less than 300£
3. How would you describe your current financial situation?
Very bad
Bad
Rather bad
Rather good
Good
Very good
4. How difficult would you say it is for you to face your current expenses every month ?
Very difficult
Rather difficult
Not really difficult
Not difficult at all
5. Do you know how much you spend approximately every month?
Yes, precisely (to the nearest £10 or £20)
Yes, more or less (to the nearest £50)
No, not precisely
No, I have no idea => go to Q7
6. How much do you spend every month on average (your expenses, for you and your
family)?
Please try to give an amount, or at least to give an estimate using the scale below.
_____________£
1500 £ and more
1000 to 1499 £
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
41
500 to 999 £
300 to 499 £
Less than 300£
7. Do you track your spending on an ongoing basis
(in a notebook, computer file, etc.)
Yes
No
8. Do you sometimes prepare a "budget" (to work out your incomings and outgoings)?
Every month
On certain occasions
Never
9. Have you been unable to pay certain bills on time over the past 3 months ?
Yes, every month
Yes, but not every month
Never
10. When you need to purchase something, what do you do?
You buy it without a second thought
You first check that you can afford it
You try and buy it on credit or pay for it in instalments
You put money aside to buy it later
II - Saving and borrowing money
11. How often have you put money aside over the past 3 months ?
Every month
Once in a while but not every month
Never => go to Q15
12. Do you know roughly how much you have put aside over the past 3 months?
Yes, precisely (to the nearest £10 or £20)
Yes, more or less (to the nearest £50)
No, not precisely
No, I have no idea => go to Q14
13. How much money have you managed to put aside over the past 3 months ?
Please try to give an amount, or at least to give an estimate using the scale below.
_____________£
Less than 50 £
50 to 100 £
100 to 200 £
More than 200£
14. Why have you put money aside?
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
42
(Several answers are possible)
In case of an incident or a difficulty
To be able to buy something later
To help out your family if need be
To give somebody a present (birthday, Christmas, etc.)
To finance a personal plan (e.g.: holidays, computer purchase, etc.) or business plan (e.g.:
start up a business)
For an important annual expense (insurance, tax, fuel…)
To cover replacing / fixing an equipment in your house (furniture, works…)
Other (specify) ___________
15. Have you borrowed money over the past 3 months (from a bank, a credit institution
or from someone in your family circle)?
(Several answers are possible)
Yes, from a bank
Yes, from a credit institution
Yes, someone in my family circle (family, friend, close relation, etc.)?
No => go to Q17
16. Why did you borrow money?
To be able to buy something later
To help out your family if need be
To give somebody a present (birthday, Christmas, etc.)
To finance a personal plan (e.g.: holidays, computer purchase, etc.) or business plan (e.g.:
start up a business)
For an important annual expense (insurance, tax, fuel…)
To cover replacing / fixing an equipment in your house (furniture, works…)
To make ends meet at the end of the month
To pay some bills
17. Over the past 3 months, have you bought something in a shop on credit or with a
credit card (with or without charges)?
Yes on credit
Yes with a credit card
No
III - Your bank account (this section applies if you have at least one personal bank
account)
18. Do you currently have a bank account?
Yes
No=> go to Q32
19. (If so) For how long have you had it?
Under a year
1 to 2 years
2 to 5 years
You have no idea
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
43
20. Do you use it on a regular basis?
Yes No
With a debit card
With a credit card
With a chequebook
21. How often have you received or deposited money on this account over the past 3
months?
Every month
Less often
Never
You don't know
22. Over the past 3 months, how often have you monitored your bank account to find out
how much you have left?
Every week
Once or twice a month
Less often
Never => go to Q24
23. How did you monitor your account?
With monthly bank statements (post or e-mail)
Online
With text messsages/e-mail alerts in case of a debit balance
24. Over the past 3 months, how often have you checked that your bank statements did
not contain errors?
Every time
Often
Seldom
Never
25. Do you know whether your bank charges you to run your account?
Yes it does
No it doesn't
You don't know
26. Has your account been overdrawn over the past 3 months (that is to say, you have
spent more than you had on your account)?
Yes, often (more than 6 times a year)
Yes sometimes (3 to 5 times a year)
Yes, but rarely (1 or 2 times a year)
No, never
27. Do you have another account to put money aside (savings or deposit accounts)?
Yes
No=> go to Q32
28. Do you know how much you have on this "savings" account?
Yes, precisely (to the nearest £10 or £20)
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
44
Yes, more or less (to the nearest £50)
No, not really
No, I have no idea
29. Do you know how much you earn with this "savings" account?
Yes I do
No I don't
30. Have you been in touch with an adviser at your bank over the 3 months (at the
branch, by telephone or e-mail)?
Yes, once
Yes, several times
No, not at all
IV - Your opinion on banks, credit and savings
31. About savings: how much do you agree with the following
phrases ? Totally
agree
Tend to
agree
Tend to
disagree
Not
agree at
all
Don ’t
know
There's no point saving small
amounts
Saving brings in no money
If you really want to save, you
must put money aside on a
regular basis
If you want to put money aside,
you need a saving account in
addition to your current account
Putting money aside avoids the
need to borrow
When you have a bit of money,
it's better to give it to your
family or close relations when
they need it rather than saving
it for yourself
32. About borrowing: how much do you agree with the following
phrases ? Totally
agree
Tend to
agree
Tend to
disagree
Not
agree
at all
Don ’t
know
One should avoid borrowing, it's
too risky
It is always difficult to know
how much one will owe every
month when borrowing
Borrowing is not a problem if
you know you can repay the
loan
Very few banks lend to people
with small revenue
All credit are more or less
equivalent, there's little point
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
45
spending too much time
choosing one
33. Finally, if you needed to borrow money, would you
Try and find the cheapest credit
Choose the first credit you are granted
34. In your view, what is the purpose of saving / putting money aside? (open-ended
question)
V - In conclusion
35. How old are you? ________
36. Are you
A woman
A man
37. Where do you live?
In an apartment / house you rent
In an apartment / house you own
With parents or friends
You flat-share/house-share
Other
38. What is your work status
Unemployed
Retired
Part-time worker
Full time worker
Other
39. What is your family status
Alone
Couple
Alone with kids (how many kids…)
Couple with kids
Other
40. Have you ever followed a course / training to learn how to better manage your bank
account or future purchases?
Yes
No
Thank you for answers
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
46
APPENDIX 2: INTERVIEW GUIDES
Qualitative evaluation – March 2013
1/ Interview of main stakeholders (project pilots, contributors and partners)
Bold font: the main topics to be covered during the interview
Normal font: specific prompting when necessary
General comment:
Always try to distinguish between what was planned / expected and what really happened /
unexpected results.
Collect data and documentation for deeper understanding of the situation concerning the social need at
the origin of the project (ie. Profile of people with excessive debt in Hungary…) + information about
the Roma minority people + additional information on the project (internal report/evaluation,
communication, articles, information on beneficiaries ie amount of money saved, how was the loan
used…)
BACKGROUND INFORMATION ON THE PROJECT
Description of the organization (Autonomia or its partners)
Background information on the organization (history, partners, financing)
Missions of the organization: what actions have been implemented before the SIMS project /
who are the usual beneficiaries / which territory do they usually cover?
Description of the interviewee and their involvement in the project
What is the role of the respondent in the organization he belongs to? What is his role for the
SIMS project?
How did he get involved into the SIMS project? When did he first hear about the project? What
did he think about it back then? Why did he decide to take part into the project?
How did it all started : description of the origin of the project
Which evidences led to the creation of the project? What issues/social need is the project
aiming to address? Why is the project targeting Roma minority people?
How the project was built? Who was involved?
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
47
Added value of the project: in which way is the project innovative?
Description of national and local programs concerning financial education, saving incitation and
debt prevention (detail of the national or local programs, their objectives, targets, pilots and
main results known).
Which programs were already implemented by Autonomia to help Roma / unprivileged people
and/or concerning financial education?
Were Autonomia and its partners on this project already working together or is it a new
partnership?
Is the SIMS project the continuation or the extension of a preexisting project?
DETAILED DESCRITPION OF THE PROJECT
Project objectives and expected results
Qualitative objectives
Quantitative objectives
Personal opinion of the interviewee concerning the objectives
Did the objectives change with time - compared to what was originally planned?
Beneficiaries: recruitment process and dedication to the project
Description of the target (at the origin of the project)
How were they recruited? Did the participant volunteer to take part into the program / have
they been chosen by the project pilot?
Were the participants selected in any way? Based on which criteria?
Number of participant: number expected and number of participants in reality.
How did the participants respond to the project (did they commit to the program, did they like
it?). What was the participation rate? The dropout rate? Why did people dropout?
Does the beneficiaries profile match what was expected? (in terms of social characteristic,
knowledge and behavior toward finance and saving).
Detailed description of the actions implemented
Overall organization of the project: who is piloting the program? Who is in charge of the
operational implementation?
Detailed description of all the actions (nature of the actions, content, frequency, location,
number of participants ...)
Description of the communication developed to mobilize the partners; to inform people
targeted by the program; communication / events related to the project to a larger public.
Project coordination: how do Autonomia and its partners coordinate their action (meetings,
exchanges, feedback)? How is the partnership working?
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
48
Description of the tools used for the program. Were they built specifically for the SIMS
project?
Has there been adaptation / changes with the actions implemented compared to what was
initially planned?
Ressources
Financial and technical ressources
Human resources (new recruitments? what qualification do they have? Were they trained for
the project?)
Role and involvement of the partners. Is there any organization that was identified as a
potential partner but did not respond positively? On the opposite, did Autonomia create new
partnerships that were not planned at the origin of the project?
OPINION AND REPORT ON THE PROJECT
Bilan
Strength and weakness of the program
Difficulties encountered
- Difficulties related to the target group: involvement, motivation, behavior.
- Organisational / financial / material / technical difficulties
- Difficulties related to the partnership.
Impact of the program on recipients :
- What is the global opinion of the interviewee concerning the impact of the program on
the beneficiaries? Did he get any feedback from the participants themselves?
- Impact of the program on beneficiaries behavior:
o Behavior concerning saving / credit ;
o Behavior concerning spending habits, budget management
o Do they effect on behavior last beyong the experimentation?
- Impact of the program on beneficiaries knowledge:
o Concerning the banking system;
o Concerning the way saving and credit work;
o Concerning advantages and disadvantages of saving/credit.
Impact of the project on the professional practices of operational actors and partners.
Impact on partnership: did Autonomia develop new partnership? Does it have an impact on
other projects?
Idea for improvement
The project today. Has it been sustained beyond the experiment? Is there any dissemination /
spin-off effect? Has there been any impact on public policies so far?
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
49
CONCERNING THE EVALUATION
Did the evaluation impact the definition or the implementation of the project in any way?
Dashboard: how was it created and filled in?
Description of the quantitative study :
- how was the questionnaire passed on?
- how was the control group selected: do they have the same profile as the participants?
- How were respondents followed up after the first wave ?
- Did they encounter any difficulty due to the evaluation?
Is there any additional data on the beneficiaries that we can collect ie amount of money saved,
frequency of payments, use of loans…
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
50
2/ Interview of beneficiaries
Presentation of the respondents - briefly
Age, work and family status, housing condition, financial situation, previous training.
How did it all started?
How did you first hear about the project? Did you know the Autonomia foundation? Did you
know the local mentor before the project? Why do you think he/she asked you to participate?
What did you think about the project when you first heard about it?
Did you decide right away to take part or was it difficult to decide? Why? / What attracted you
the most (was it the matching? the training? …)
Description of the program
What was the project about?
Description of the training, the group meetings (except for IDA), amount saved, loan and
usage… depending on the program.
During the program what did you save for? What kind of project (individual or collective)?
Did you know the other participants? How was the atmosphere in the group?
(CAF) Which rules did you establish for the group? Was it easy to decide? Were the rules easy
to follow? Did you make changes?
Did you or other group member drop out of the program? Why?
Opinion and impact
Overall, what do you think of the SIMS project: what’s the pros and cons for you?
Did it help you in anyway? How?
Impact on knowledge and habits.
Did you learn something? What do you remember?
Concerning:
- Saving: why is it important to save?
- And also: Credit (including with local lenders / risk taking), Budget management, Energy
efficiency in the house, Banking system, relation to the bank.
Did you change your habits? Why?
Concerning:
- Saving: do you save more since the program took place? More often? What for (for more
security in general? For specific projects)? Does is impact your financial situation?
- And also: Credit, Budget management, Energy efficiency in the house, Banking system,
relation to the bank.
SIMS programme –Hungary evaluation report - CRÉDOC - July, 2013
51
Would you say your financial situation is better than before the program?
Did a project/goal become a reality thanks to the SIMS program? Which one? What you would
have done without the program?
About individual and collective projects: have you started planning more your future since the
SIMS program? If so, why kind of project? Who do those projects involve (individual /
household / SIMS group / the entire village)? Do you have collective projects or common
events with the other participants? With people in the village? Is this more than before the
SIMS project? Are you still seeing the other participants to discuss about budget management /
saving / credit / everyday life? Are you still seeing the other participants to do saving together?
Or for anything else?
Ideas for improvement
Would you do the project again? Would you recommend it to your friends and family?
How would you improve the project? Why?