Review of Mellemfolkeligt Samvirke/ActionAid Denmark · the areas of local governance, youth...
Transcript of Review of Mellemfolkeligt Samvirke/ActionAid Denmark · the areas of local governance, youth...
Review of
Mellemfolkeligt Samvirke/ActionAid
Denmark
Date: November 2014
Technical Advisory Services Department,
Ministry of Foreign Affairs
2
Contents Executive Summary ......................................................................................................................................... i
Overall performance ................................................................................................................................... i
Capacity development .............................................................................................................................. iii
Danish Arab Partnership Programme ...................................................................................................... iv
Financial management and administration .............................................................................................. iv
Follow up to previous reviews and evaluations ........................................................................................ v
1. Introduction ................................................................................................................................................. 1
2.1 Brief Description of MS/AADK ............................................................................................................ 1
2.2 Background to the Review .................................................................................................................. 1
2.3 Approach and Methodology ................................................................................................................ 2
3. MS/AADK’s Strategic Framework .............................................................................................................. 3
3.1 MS/AADK’s Strategy between 2010 and 2012 .................................................................................. 3
3.2 MS/AADK’s new strategy: “Together Against Poverty” ..................................................................... 3
3.3 Relevance to Danida Strategies and Policies .................................................................................... 5
3.4. Attribution, contribution, results and efficiency. ................................................................................ 6
4. The MS/AADK Program under the Framework Agreement ..................................................................... 7
4.1 Activities and Partners under the Framework Agreement ................................................................. 7
4.2 Use of Framework agreement resources ........................................................................................... 7
5. MS/AADK’s MENA Engagement ............................................................................................................. 10
5.1 Relevance of the MENA Engagement .............................................................................................. 11
5.2 Activities and Partners in the MENA Engagement .......................................................................... 12
5.3 Size and Flow of Resources ............................................................................................................. 13
5.4 Analysis of Performance ................................................................................................................... 13
5.5 Issues to Consider Regarding Programing of the MENA Engagement .......................................... 14
5.6 DEMENA Pool ................................................................................................................................... 15
5.6.1 Positioning and Administration of the Pool............................................................................... 15
5.6.2 Performance of the DEMENA pool ........................................................................................... 16
5.6.3 Additional Challenges ............................................................................................................... 16
6. MS/AADK’s Performance in the Area of Capacity Development ........................................................... 18
6.1 MS/AADK’s Capacity Development Approach ................................................................................. 19
3
6.1.1 Analysis of T4C ......................................................................................................................... 19
6.1.2 Global Platforms ........................................................................................................................ 21
6.132 Analysis of P4C ........................................................................................................................ 22
6.1.3 Analysis of MS/AADK’s Sub-granting for Program Support .................................................... 24
6.2 Analysis of MS/AADK’s Capacity Development and AAI’s Theory of Change ............................... 25
6.3 Analysis of Synergies between Capacity Development Methods ................................................... 26
6.4 Analysis of the Performance of MS/AADK’s Capacity Development Approach ............................. 27
6.4.1 Capacity Development Outputs ................................................................................................ 27
6.4.2 Quality ........................................................................................................................................ 28
7. Monitoring and evaluation of programs ................................................................................................... 30
7.1 Reporting ........................................................................................................................................... 31
7.1.1. Annual report ............................................................................................................................ 31
7.1.2 Reporting under the Framework Agreement ............................................................................ 31
7.1.3 Reporting under the DAPP Agreements .................................................................................. 32
7.2 MS/AADK’s New Monitoring System ................................................................................................ 33
7.3 MS/AADK’s System for Evaluation ................................................................................................... 33
8. Analysis of MS/AADK’s financial management and administration ....................................................... 34
8.1 Financial management roles and capacity ....................................................................................... 34
8.2 Management of the Arab Regional Initiative .................................................................................... 35
8.3 Systems and procedures for fund transfer and management ........................................................ 35
8.3.1 Funds managed within MS/AADK’s own organisational structure ............................ 35
8.3.2 Funds transferred to and managed by local partner organisations .......................... 36
8.4 Auditing .............................................................................................................................................. 37
8.5 Anti-corruption ................................................................................................................................... 38
8.6 Human resource management, time registration and duty travel ................................................... 39
8.7 Capacity to generate own financing ................................................................................................. 39
9. Summary of recommendations ................................................................................................................ 42
Annex 1: Follow-up on Recommendations from Previous Review ............................................................ 46
Annex 2: MS/ AADK performance to 2010-12 strategic goals. .................................................................. 48
Annex 3: Current MS/AADK strategic objectives and indicators ................................................................ 49
Annex 4: MENA progress to objectives ....................................................................................................... 50
4
Abbreviations
AAI ActionAid International AA Country Office
This term is used as a common denominator for ActionAid affiliate and associate members.
ARI Arab Regional Initiative Danida Term used for Denmark’s development cooperation, which is an area of activity
under the Ministry of Foreign Affairs of Denmark DAPP Danish Arab Partnership Program DKK Danish Kronor CBO Community Based Organization Local NGO This term is used to denominate NGOs that are partners with the AA Country
Office LRP Local Rights Programs MENA Middle East and Northern Africa MFA Ministry of Foreign Affairs MoU Memorandum of Understanding MS/AADK Mellemfolkeligt Samvirke/ActionAid Denmark MS Mellemfolkeligt Samvirke MS-TCDC MS Training Centre for Development Co-operation NGO Non-governmental Organization P4C People for Change SDA Swedish Development Advisors AB T4C Training for Change
i
Executive Summary
The purpose of the review was to assess the performance of MS/AADK in capacity
development since the last Danida Review in 2009. The specific objectives were to assess:
- MS/AADK performance in developing the capacity of civil society organizations in the
South and in the Arab region
- The outcomes of this capacity development with regard to MS/AADK’s objectives and
agreements with MFA and with individual partners,
- MS/AADK engagement under the DAPP, including the administration of the
DEMENA Pool,
- MS/AADK’s organizational and financial capacity to operate its programs with MFA
framework agreement/DAPP funds,
- MS/AADK’s follow-up to the recommendations from the latest reviews and evaluation.
The review was undertaken from February to July 2014 by a team from the Technical Advisory
Services and consultants from Swedish Development Advisers. The Office for Humanitarian
Action, Civil Society and Personnel (HCP) and the Office responsible for the Middle East
(MENA) in the Ministry of Foreign Affairs participated with resource persons during the review.
Overall performance
Strategic alignment
The focus and approach of MS/AADK is aligned the Danish Strategy for Development
Assistance, the Policy for Danish Support to Civil Society and the Danish Arab Partnership
Programme. MS/AADK, in general, they administer the programmes in line with agreements
with Danida. MS/AADK’s programmes have a clear focus on HRBA and aim to build the
capacity of local NGOs in HRBA approaches, particularly in relation to citizens holding
governments to account.
MS/AADK is an affiliated member of Action Aid International (AAI) and is subject to AAI’s
global strategy. MS/AADK has a distinctive and relevant “footprint” in the work of AAI in line
with Danish development strategies and policies. The comparative advantage of MS/AADK is
formalised within the federation in the areas of democratic governance and capacity development
and youth.
MS/AADK, however, should strive towards implementing its HRBA approach in all aspects of
AAI’s work; this includes continued focus on local organisations as equal partners in
implementing programmes and not as service providers; helping National AA offices implement
transparent and democratic governance structures; and avoiding the unintended effects of being a
strong international organisation working with local organisations.
ii
Performance
The affiliation of MS/AADK with AAI has enabled it to make a distinctive and influential
impact within the federation, allowing it to ‘scale up’ its methodologies and best practises beyond
the scope of a traditional Danish NGO. However, membership of a federation also presents
some challenges to MS/AADK as a Danish NGO principally funded by Danida - most obviously
how to monitor and report on the use of Danida funds at country and programme level and on
the specific contribution of these funds to programme impacts.
MS/AADK’s progress towards stated objectives has been generally satisfactory although this
assessment is difficult given the difficulties in accessing adequate monitoring data on some of the
objectives. Progress toward objectives in the MENA programmes has been less than planned –
due to both the volatile context and organisational issues. MS/AADK should adopt a more
systematic, pro-active approach to risk management when operating in volatile contexts.
M&E framework
MS/AADK reporting during the period 2010 – 2012 suffered from a number of weaknesses.
There has been a systemic weakness in how MS/AADK has managed goal setting and Danida’s
annual review of this process. MS/AADK established new goals every year for the framework
funding without reporting against the goals for the previous year. MS/AADK’s most recent
Strategic Plan (2012-2017) establishes eight impact goals and the underlying Implementation
Plans establish output goals. MS/AADK has yet to identify outcome goals and to measure
progress against these. This led to a lack of consistency in MS/AADK’s reporting against goals
and targets, and made it difficult for Danida to carry out its oversight role in assessing whether
MS/AADK has achieved the targeted results.
Similarly, the current DAPP reporting arrangements do not allow MS/AADK or the MFA to
reliably assess progress of objectives. Not all the original indicators are used for annual reporting
to MFA and, as a result, the Annual Reports do not assess overall progress towards the
achievement of objectives. For example, quarterly reports from ARI country program
coordinators to MS/AADK capture all original indicators but the data is presented in different
formats and with different levels of detail which makes tracking of progress extremely difficult.
There is also some ‘double counting’ of activities or beneficiaries in the reporting of different
outputs and objectives.
The 2013 Annual Report indicates that MS/AADK has improved its monitoring and reporting
systems. MS/AADK has recently developed an impressive new system for the monitoring and
collection of data from several different sources. This system if focused on collecting data to
monitor outputs. However, there is room for improvement in the new M&E system to ensure
that MS/AADK can monitor and report output results in a reliable manner, but also to report on
outcomes. For example, MS/AADK should continue to work on clearer targets with partners; to
collect data to measure; to establish indicators for which data can be collected within reasonable
cost; and to implement a data collection system that collects outcome data on a regularly and
iii
consistently manner. MS/AADK should continue its dialogue with AAI on how to improve AAI
monitoring and reporting systems.
Capacity development
Approach
MS/AADK’s approach to and support for capacity development is in line with its own and
Danida’s strategic frameworks. It is important that MS/AADK preserves demand-led orientation
of its capacity development, since there are some indications that AAI country offices and
partner organisations may feel obliged to use MS/AADK training.
The MS/AADK approach to capacity development is heavily dependent on face-to-face training
for which it has a well-developed methodology. There is a yet-to-be-realized potentials for
increased synergies between the various MS/AADK capacity development approaches.
MS/AADK does not have a fully developed methodology for on-line training yet which would
be well suited to the MS/AADK target groups e.g. youth, and key to its leading role in capacity
development within AAI.
Performance
During interviews and surveys, it was clear that MS/AADK’s training methodology and
programme-led approach is highly appreciated by the participants. MS/AADK systematically
monitors the effectiveness of training through annual surveys. This measure how participants
have acquired and used the new skills acquired, but do not address the impact of the training on
their organisation and/or programmes. The measures used for monitoring step-down training,
for example, do not measure whether the overall or aggregated training targets are met.
MS/AADK should address these challenges in order to improve monitoring and reporting on
training outcomes.
The People4Change programme shows evidence of results for communities and beneficiaries in
the areas of local governance, youth mobilization and women’s rights. Organizations receiving
an Inspirator or Adviser report a high degree of satisfaction but it needs to be considered that
more support in preparation to host an Inspirator or Adviser is required to take full advantage of
the resource.
The management responsibilities for MS/AADK Global Platforms need to be clarified so that
MS/AADK can transfer management responsibility for the Global Platforms to the AAI country
programmes/affiliates more effectively. MS/AADK should continue to take responsibility for
the development and quality assurance of the concept of the Global Platforms within the AAI
federation.
MS/AADK’s coaching and mentoring at local rights programmes level was greatly appreciated by
partners. MS/AADK should consider including these activities in its capacity development
model and reporting.
iv
Danish Arab Partnership Programme
MENA programme
The objectives of the MENA programme are well aligned with Danish Arab Partnership
Programme and national youth policies’ objectives. Programme activities are designed through
appropriate consultation with stakeholders and are relevant to the target groups.
MS/AADK progress towards meeting the objectives has been mixed – in part due to a number
of factors beyond the control of MS/AADK e.g. popular protest in Egypt and escalating crisis in
Syria. Youth training targets are likely to be met while targets related to the DEMENA Pool and
institutional capacity development are likely not to be achieved. The review highlights a number
of suggestions to improve the performance of the MENA programme - for example, by
addressing the current focus on school and university students; defining roles and responsibilities
more clearly in partnership agreements; exploiting national and regional networks to facilitate
knowledge sharing and leverage other actors; considering how to explore both the supply and
demand side of work on local governance; and expanding the range of social accountability tools
in the support of youth mobilization.
DEMENA Pool
MS/AADK is well positioned to manage the DEMENA Pool given their experience and
expertise in youth and capacity development and presence in the region. After some initial
difficulties, progress has been made in improving the administration of the pool e.g. the selection
process. However, progress in implementation has been slow due to a low number of applicants.
The Pool would benefit from greater transparency in the selection processes and an M&E
format. The review recommends a number of measures to improve visibility, targeting and
criteria for the Pool support to expand its target groups (to Arab organisations especially) and
improve monitoring and reporting.
Financial management and administration
MS/AADK has relatively comprehensive systems and procedures for financial management and
administration. While the use of resources in MS/AADK was found to be in line with planning
and budgets, systemic weakness in financial systems have been identified, especially with regard
to the administration of the ARI. These weaknesses should have been reported more openly and
quickly to the MFA. At the time of the review, MS/AADK had taken some measures to rectify
the situation. Further efforts are warranted to ensure adequate control of funds transferred to and
managed by the organisation’s local offices, including the Regional MENA Office. Audit and
anti-corruption policies should be reviewed and consistently communicated within the
organisation as well as to external partners.
MS/AADK should continue to monitor and report on the allocation and use of framework
funds transferred to AAI country offices and ensures that AAI has a balanced approach to the
degree of funding channelled to local partners outside AAI. The review found that approximately
25% of framework funding is channelled to what it would consider as partner organisations.
v
MS/AADK will need to continue to ensure the use of anti-corruption clauses in all grant
agreements.
To date, AADK has been able to raise the required level of matching funds and is making
progress on diversifying their funding to a level where 20% of their funding is independent on
funding from Danida. The growth in contributions from individual donors is slower than
expected and MS/AADK will need to actively review their individual and institutional
fundraising strategies to achieve their ambitious targets.
Follow up to previous reviews and evaluations
The majority of the recommendations of the 2009 review have been addressed. However, these
recommendations have become less relevant since the affiliation of MS with AAI in 2010. The
review notes positively that MS/AADK commissioned a thorough evaluation of their practices in
2011 and is gradually implementing these recommendations.
1
1. Introduction
2.1 Brief Description of MS/AADK
Mellemfolkeligt Samvirke/ActionAid Denmark (hereinafter abbreviated to MS/AADK) is one of
the largest civil society organizations in Denmark and has for a long time received substantial
funding from Danida based on an “Agreement of Cooperation” (Samarbejdsaftale, latest from
1994).
MS/AADK is a membership-based organization with an elected Council of Members as its
supreme decision-making body. It meets once per year. In between meetings, MS/AADK’s
Board meets frequently, during the past three year they have met between 7 to 11 times.
MS/AADK was reorganized in 2012. The ARI programme was managed directly by the director
of capacity development at the time of review.1
In 2010, MS became an affiliate member of the federation ActionAid International (AAI). The
transformation to become a Danish NGO in an international federation has included substantial
work in terms of restructuring such as; closing of MS country offices, aligning with AAI policies
and strategies, assisting in developing AAI policies and strategies, designing and implementing
PM&E systems in collaboration with AAI, undertaking the mandate to develop governance,
capacity development and youth as programme areas in AAI, among others.
MS/AADK has an important role of delivering and developing a substantial part of AAI’s
capacity development. As such they deliver training through nine venues; the larger training
village, MS-TCDC, in Tanzania and seven urban and one rural training venue in six countries
(called Global Platforms (GP))2 specifically focusing on youth empowerment training3. Specific
training on HRBA and governance and course development activities carried out in the Global
Platforms has been financed by the MFA framework agreement. Of the total financing of the
GPs, about 20 and 22% was financed by Danida in 2011 and 2012 respectively4. In 2013 the
Global Platform funding model was changed into scholarships that fund each participant.
In August 2006 MS/AADK obtained the status as a Danida framework organization and now
they receive an annual budget directly allocated from the National Budget Act. The annual budget
under the current 4-year rolling plan is DKK 149 million. MS/AADK has been a partner to the
Danish Arab Partnership Programme (DAPP) since 2006. In 2013 MS/AADK was approved as
a strategic partner under DAPP receiving an additional grant of DKK 11 million for 2013-2014.
2.2 Background to the Review
In accordance with the Strategy for “Danish Support to Civil Society in Developing Countries”
the MFA undertakes thematic reviews of the framework organizations to enhance the
1 A new restructuring was announced shortly before the review and will be fully effective in September 2014. 2 El Salvador, Jordan, Kenya (urban and rural), Myanmar, Nepal, Tanzania. 3 In addition MS/AADK also operates one Global Platform in Denmark 4 Based on data provided by MS/AADK compared to Audited Financial statements for MS/AADK.
2
professional dialogue between Danida and these organizations. In March 2014 a Review Team
was formed to carry out this review.
The Review Team consisted of Rene Taus Hansen, Team leader, and Mette Bastholm, Technical
Advisory Services, Ministry of Foreign Affairs; Åsa Königson, Jonas Lövkrona and Kevin
Hempel from Swedish Development Advisers. Jens Kåre Rasmussen and Trine Storm from
Ministry of Foreign Affairs acted as resource persons and participated in the field work of the
review in Uganda and Tanzania, Jordan and Egypt, respectively.
The Review Team is thankful to all staff in MS/AADK for their collaboration in the preparation
of the review, especially those who assisted in organizing the field missions.
2.3 Approach and Methodology
The review covers four different agreements with the Ministry of Foreign Affairs; the framework
agreement and three separate contracts under the DAPP. The framework agreement and DAPP
contracts are different in terms of modalities; the framework agreement allows a greater degree of
freedom in terms of country selection and thematic area of MS/AADK whereas the DAPP
contracts defined to a higher degree which countries and what thematic areas the organization
should engage in. However, due to similarities in thematic areas and means of implementation it
has been decided to include both types of programs in the review.
The review is not tasked to undertake an assessment of the transition process of becoming a
member of the AAI federation. The review focuses on programme delivery and methodology
and whether the transition has allowed MS/AADK to undertake the work outlined in the
framework agreement and application, as modified.
The overall objective of the review is:
to assess MS’s professional performance in relation to capacity development of civil society
organizations in the South and in the Arab region
to assess the outcomes of this capacity development on the objectives stated by the
organization and in the agreements with MFA, at head office level as well as at the level of
the individual partners,
to assess the engagement under the DAPP, including the administration of the DEMENA
Pool,
to provide a general assessment of MS’s current organizational and financial capacity to
operate its programs with funding from the MFA framework agreement/DAPP,
to assess the organization’s follow-up of the recommendations from the latest reviews and
evaluation.
The selection of MS/AADK’s partners selected to visit was based on the need to observe and
collect data on project funded under the DAPP program. . Two countries were selected; Jordan,
which is the regional hub for MS/AADK and Egypt which has a number of activities applying
the various instruments which the review was tasked to assess. .Furthermore, MS-TCDC was
selected as it is MS/AADK’s largest capacity development hub and AA Uganda which is one of
the three main partners to MS/AADK in terms of funds sub-granted. The RT has assessed the
3
performance of MS/AADK against their own stated objectives and indicators. The RT has
assessed reports; validated information and performance during interviews with beneficiaries,
partner organizations and staff; and tried to merge performance indicators with the use of
financial and human resources.
The review has focused on performance assessment of the work undertaken under the
framework agreement and the Danish Arab Partnership Programme (DAPP). Capacity
assessment of the organization has only been done to assess whether MS/AADK has the
capacity to manage and implement the framework agreement.
3. MS/AADK’s Strategic Framework
3.1 MS/AADK’s Strategy between 2010 and 2012
Between 2010 and 2012, MS/AADK had a transitional strategy in place, “MS/AADK in
Transition”. It established MS/AADK’s overall vision, mission, values and approach. During the
transition period MS/AADK’s strategy was not directly linked to that of AAI.
A review of the documentation provided to Danida and the MS/AADK’s Board with regard to
the fulfilment of the prior strategy shows that MS/AADK has achieved three of the seven goals
assessed as part of this review. Details of results to strategic goals can be found in Annex 2. For
some of the goals, it is not possible to provide conclusive data on achievements.
3.2 MS/AADK’s new strategy: “Together Against Poverty”
During the first part of 2012, MS/AADK prepared a new strategy, “Together Against Poverty”
to direct its work between 2012-2017. This strategy is aligned with and support AAI’s strategy for
the same period of time. AAI’s strategy establishes 10 Key Change Promises that the entire AA
federation will work to fulfil. Most of the Key Change Promises are measureable. They cover
several themes within democratic governance: youth, women, mobilization of poor and local
governance. AAI’s Key Change Promises can be called impact objectives, i.e. medium to long
term objectives where several factors will contribute to the actual results. These are important to
the AA federation as these set the direction for all AA Country Offices to work towards,
establishing their own strategic plans with these overall Key Change Promises in mind.
The Key Change Promises are followed-up in the ActionAid’s Annual Report. The AAI 2013
Annual Report provides data on each of the Key Change Promises but one, and attempts to
measure progress.
MS/AADK has selected four of AAI’s Key Change Promises to contribute to, building on their
own experience in capacity development, democratic governance and youth work. The following
MS/AADK Strategic Objectives each contribute to one of AAI’s Key Change Promises:
Objective 1: AADK will contribute to holding governments and corporates to account, and
thereby improved public services for five million people living in poverty.
Objective 2: AADK will contribute to ActionAid winning significant victories in achieving fair
redistribution of resources.
4
Objective 3: AADK will contribute to ActionAid’s work with women and men affected by
disasters.
Objective 4: AADK will contribute to mobilizing 5 million young agents of change
In addition, MS/AADK established the following Strategic Objectives:
Objective 5: Together with AAI, AADK will develop a regional program in the Middle
East and North Africa.
Objective 6: AADK will support the capacity of ActionAid and ActionAid partners to
apply a Human Rights Based Approach.
Objective 7: AADK will have a significant impact on Danish and European policy
within our campaign areas, and we will strengthen cooperation between the Danish
government and international civil society in matters of common interest.
Objective 8: AADK will ensure that Danes have an arena for action as global citizens
and strengthen their engagement and involvement in campaigns aimed at fighting poverty
and promoting sustainable alternatives.
The Strategic Objectives will be measured using a set of indicators, see Annex 3.
In addition to the MS/AADK’s “Together Against Poverty”, five “Implementation Plans” have
been developed5. These documents establish sub-objectives for each of the respective areas,
output targets, and in some cases outcome targets and activities to be implemented. The
“Implementation Plans” cover the period 2012-2015. These are well formulated and clear plans
that allow for an understanding of what is to be achieved, by when and with what activities.
MS/AADK’s strategic framework for the period 2012-2017 has potential for further
improvement:
The strategic objectives are vague and do not help MS/AADK to plan its activities. The
objectives need to be more specific to clearly establish MS/AADK’s responsibilities.
The strategic objectives need to clearly indicate what is to be achieved and what a success
can be defined as.
There are also goals that are inputs i.e. “to establish AA in the MENA region” are a
means to achieve another goal i.e. help youth, women and girls to demand accountability
and justice.
The indicators MS/AADK has defined attempt to measure output i.e. Advisors placed,
or persons trained.
There is no baseline defined in “Together against Poverty” nor in the “Implementation
Plans”. Without a baseline value it is difficult to appraise an achievement both as to the
relevance of the objective and as to the actual accomplishment
The strategic objectives should be formulated to be Specific, Measureable, Achievable,
Relevant and Time-bound.
5 For respectively “Democratic Governance”, “Advocacy and Action”, “Empowerment” and “Resources”. An additional plan has been developed for internal membership – which has not been reviewed by the Review Team as this was outside the scope of the review.
5
Using a Results Based Management (RBM) approach6, the Key Change Promises represent an
Impact target for MS/AADK and MS/AADK’s own Strategic Objectives could be re-defined into
Outcomes with the “Implementation Plans” establishing the Output targets. MS/AADK has
established SMART output-level targets (in the ”Implementation Plans”) but also its Strategic
Objectives would benefit from being specific, measureable and based on a baseline.
As part of a large federation MS/AADK is expected to contribute to the Key Change Promises.
The Review Team does not expect MS/AADK to fully deliver upon the Key Change Promises,
but to establish a baseline and realistic outcome and output targets for the program(s) that
Danida funds. MS/AADK has a system for establishing such goals, together with their partner
AA Country Offices (through the Programme Objective Plans (POPs) and for monitoring and
reporting progress (see Chapter 7).
Findings
MS/AADK’s Strategic framework findings are:
Overall, MS/AADK’s Strategic Objectives are in line with MFA strategies and aim to
contribute to the MFA’s strategic goals.
Objectives are statements of intent and not goals.
Objectives are not specific and as such do not provide guidance on what activities are
important to achieve them.
There are also objectives which actually are input i.e. “to establish AA in the MENA
region” was a means to achieve another goal e.g. greater youth mobilization in the region.
There is no baseline defined.
The goals should be specific, measurable, attainable and time bound.
Recommendation
Recommendation is listed under the section on Monitoring and Evaluation.
3.3 Relevance to Danida Strategies and Policies
MS/AADK Framework Agreement should contribute to the following strategies; the “The Right
to a Better Life – Strategy for Denmark’s Development Cooperation”7 and “The Civil Society
Strategy8 – Strategy for Danish Support to Civil Society in Developing Countries”9.
MS/AADK’s Strategic Objectives are in line with five of eight strategic goals established in the
document: “The Civil Society Strategy – Strategy for Danish Support to Civil Society in
Developing Countries”.10 However, MS/AADK does not fully comply with the requirements of
”setting clear targets” and ”strengthening their results-orientation” (Strategic goal 1.2.7 of the
strategy).
6 An approach applied by a large number of donors (including Danida) and used to report results by NGOs world-wide. 7 The Danish Government. “The Right to a Better Life – Strategy for Denmark’s Development Cooperation”. 2012. 8 The strategy was replaced by a new policy by Danida, however the strategy has been used as reference point as this was in effect for the implementation period under review. 9 MFA. ”The Civil Society Strategy – Strategy for Danish Support to Civil Society in Developing Countries”. 2008. 10 MFA’s strategic goals 1.2.1, 1.2.2, 1.2.3, 1.2.4 and 1.2.6.
6
Finding
Overall, MS/AADK’s Strategic Objectives are in line with the MFA’s strategies and aim
to contribute to the MFA’s strategic goals.
3.4. Attribution, contribution, results and efficiency.
The decision by MS/AADK to affiliate with the global AAI federation poses a challenge
regarding attribution, contribution, results reporting and cost efficiency. While affiliation to a
global programme allows MS/AADK to have a much wider reach - 45 countries - than if MS had
operated alone, clear results reporting becomes more difficult because MS/AADK no longer has
full control of the programmes implemented.
MS/AADK support to AA country offices allows for priorities to be established by the AA
country programmes itself. This is comparable to budget support to an AA country programme,
with the corresponding challenges of attribution and contribution of the MS/AADK share of the
budget. MS/AADK should continue its efforts to enhance the monitoring capability of the
programmes and enter into a dialogue with AAI and country programmes on the balance
between use of the budget support at AA level or at local rights programme level and with local
partners, to ensure that the priorities are set by the country programme.
MS/AADK has a clear and substantial comparative advantage in capacity development in AAI,
specifically in training and placement of advisers and inspirators. The theory of change indicates
that the contributions of MS/AADK to the capacity of AAI and partners allow AAI to engage in
productive programmes which ultimately aim to achieve societal change. Capacity development
efforts are generally measured in number of people trained, placements of staff, and the
satisfaction with training and placements respectively. The challenge for MS/AADK is that
monitoring at the level of trainee satisfaction is a poor proxy indicator for the effect of the
support at the level of societal change. The current monitoring system, while very impressive, is
stretching the evidence for the theory of change to a degree that makes it very difficult to assess
the validity of the theory.
Assessment of effective use of resources in a global federation, with an intentional mingling of
funds, is very difficult. The framework guidelines state that results reporting and cost efficiency
are mandatory considerations. It is difficult to show the MS/AADK contribution to the overall
objectives, to assess cost efficiency and matching results with resources used. The Review Team
finds there is room for improvement in this area and that the organisational model of AAI,
challenges the need for result reporting and cost effectiveness analysis.
In the future it will be important for MS/AADK to be open about the challenges and dilemmas
faced, the rationale for the decisions made and the consequences of these. It is also important for
MS/AADK to continue the discussions in the wider AAI federation on the need for focussing
on the requirements of the Danish framework funding requirements.
7
4. The MS/AADK Program under the Framework Agreement
4.1 Activities and Partners under the Framework Agreement
AAI defines the methods to be used to develop capacity as a) training, b) on-line tools, and c)
people11. MS/AADK and AAI have signed a ”Delegation Agreement” whereby MS/AADK is
responsible for delivering courses, managing the “People” method of capacity development
within the Federation, and establishing 14 Global Platforms.
MS/AADK’s offer to the AAI (partly funded by Danida) consists of:
Training4Change (T4C) i.e. face-to-face training design, delivery and follow-up for training
on democratic governance, HRBA, youth, campaigns, institutional governance and middle
management within AAI.
People4Change (P4C) i.e. recruiting and placing Inspirators (short term professional staff –
up to 9 months) and Advisors (long-term professionals – up to 24 months) with a local NGO
or with an AA Country Office.
Program support to selected AA Country Offices in specific program areas: Governance,
Trade Justice and Land Rights. Funds are sub-granted to the AA Country in accordance with
“Partnership Agreements”12 that are based on the AA Country’s own strategic plans13.
Knowledge generation, exchange of experience, and development of methodologies and
performance management models.
MS/AADK’s target groups include support to local NGOs as well as AA Country offices that
carry out activities and work directly with end beneficiaries. MS/AADK has nine priority
partners within the AA Federation14.
AA country offices fall into three categories – affiliates (fully integrated, independent members of
the federation); associates (with registration in progress or completed); and country programs;
fully managed by AAI structures.
4.2 Use of Framework agreement resources
Danish framework funding can be applied very flexibly within the AAI federation supporting
MS/AADK priorities. The framework agreement does not specify standards for the amount to
be transferred to local organisations, nor does it specify the proportion of funding to be spent for
M&E, programme development etc.15 However, guidelines for framework agreements do state
that results should be documented; activities described using log-frame and /or a result based
management methodology; and efficiency and effectiveness in terms of use of resources.
The review has prepared an overview of the use of framework resources to assess the efficiency
and effectiveness of the implementation of programmes. However, lack of detailed financial data
11 Agreement for delegation of Capacity Development from the ActionAid Secretariat, on behalf of the Federation, to ActionAid Denmark, Revised January 2014. 12 Called ”Cooperation Agreements” prior to 2012. 13 In some cases called “Program Document”, in other, “Program Objective Plan” 14 Guatemala, Kenya, Mozambique, Nepal, Nicaragua, Tanzania, Uganda, Zambia and Zimbabwe 15 Apart from administration, information and specifically agreed activities such as UNV.
8
at the level of outputs and outcomes has made it difficult to make an assessment across the
review period. An analysis has been possible for 2013 only. The RT was unable to make an
assessment of cost efficiency of the various outputs but was able to illustrate the use of resources
allocated under the framework agreement. MS/AADK has a structured generic cost calculation
for scholarships, and cost of courses etc.16 the calculations are used to charge the framework
agreement e.g. according to number of scholarships awarded. However, it is not possible with the
current systems to calculate unit cost of scholarships and training days or other outputs. The
framework agreement is therefore charged according to the budgeted figures developed on the
basis of the cost calculation.
In terms of overall efficiency and effectiveness, the RT finds that the overall allocation of
MS/AADK resources is in line with the focus of the organisation and its mandates within the
federation. 84% of framework funding is used for programme activities; 38% for programme
countries (8% for P4C, 30% for country offices), 10% for People 4 Change, 16% for Training 4
Change and 20% for methodology development.
Overall use of framework funding in MS/AADK programme 2013. Percentages are shares of total framework funding. Compiled from multiple sources of financial documentation and reporting from MS/AADK
16 I.e. the cost of week course in a GP is approximately USD300 and applied across the organisation. MS/AADK informed the RT that while there are differences in cost structures in various countries, they have decided to apply a uniform rate.
9
MS/AADK reported the use of framework funding in 2013 to 21% on cost of development in
support of programmes; 61% on support to AA countries; and 18% on other support,
information and administration.
According to country programme financial reports, 10% of the frame is used in AAI or
MS/AADK country programmes (which the RT does not consider as local partners as they are
directly implemented by AAI or MS/AADK). The remaining 25% is used by AAI Affiliate or
Associate country offices which qualify as local organisations with local systems of governance.
The RT recognizes that financial transfers are made to local organizations outside AA. Therefore
this is not the sole criteria for whether end beneficiaries are serviced and the target group
supported. However, AADK should continue to focus on the balance between funds transferred
to partners and funds utilized within and by the AAI members in AA structures.
The AA Country Offices use a large part of their support to support local partners in the 9
countries, although due to different programme portfolios, proportions vary significantly across
countries. Overall 14% are used in non-AA organisations.
It is important to note that decisions on resource allocation in individual AA country
programmes are not done by MS/AADK, but by the AA country programme. The amount of
resources channelled from AA country programmes to non-AA partners is not directly17 under
the control of MS/AADK. Most funding in AA countries is allocated by Child Sponsorships,
which are restricted in terms of what the fund can be utilized for, i.e. 70% needs to go to the
community level. Danida funds do not have the same restriction18. Therefore, there is relatively
more funding available for empowerment at the local level while, for example, capacity
development, policy/campaign work, research and alliance building are under-resourced.
MS/AADK estimates that the 38% funding from MS/AADK represent approximately 1/3 of
the total funding to the country programmes in priority countries.
Across the period 2010 to 2013 the
support to the 9 priority countries
has amounted to 58% of the
framework allocation19. In 2013, 46%
of the frame is used for P4C, T4C
and methodology development. This
is a high proportion of the overall
frame, but the prioritisation should
be seen in the context of the global
mandate to undertake capacity
development, governance and youth portfolio on behalf of AAI. A good part of the framework
funding for T4C is used for methodology and course development. Most of the funding for the
17 Although MS/AADK can include these considerations in the negotiations with the country programmes. 18 MS/AADK does however specify that maximum 15% of the funding can be used for administration and programme management etc. at the level of AA country programme. 19 The 58% is not directly comparable with the diagram showing the situation for 2013 due to differences in compilation of numbers.
10
implementation of the courses is coming from the participants (organisations) themselves or
from other strategic partners. MS/AADK estimates that T4C only receives 40-45% of its total
resources from the framework funding.
RT team found that the AA federation allows MS/AADK to deploy its comparative advantage in
capacity development to a global programme; it would not have as bilateral NGO. In this context
the proportion of funding spent on methodology and development is justified.
Findings:
The overall balance of resource utilisation is comparable to the mandates and role of
MS/AADK in the federation.
The RT finds that MS/AADK has a clear and sizable “footprint” in the AA federation.
The thematic areas are relevant and MS/AADK methodology is implemented across the
federation and not only in the 9 priority countries of MS/AADK.
Detailed assessment of efficiency and effectiveness in terms of unit cost or cost in
relation to outputs and outcomes has not been possible with the data available.
25% of the framework funding is channelled to organisations which the RT categorizes as
local partners.
While MS/AADK is abiding by the financial reporting guidelines in terms of the
framework agreement, the current set-up does not allow management to make regular
assessment of effectiveness and efficiency in the implementation of results.
Recommendations:
1. MS/AADK should discuss with the wider AAI and with AA country programmes on the
balance between funding to partners outside and inside the AA federation.
2. MS/AADK should consider internal systems for assessment of overall efficiency and
effectiveness in its use of resources20, including actual cost of key outputs.
3. Clarification from HCP on the criteria for what constitutes a local partner should be
considered.
5. MS/AADK’s MENA Engagement MS/AADK has been active in the MENA region since the mid-1990s and a grant recipient under
the DAPP since 2006. Since 2011, MS/AADK has received three additional grants from the
MFA, the latest of which was granted against the background of becoming a strategic partner
under the DAPP.
Title of grant Grant amount Period
Youth Engaging in Change Processes in North Africa
9.6 million DKK July 2011 – 30 June 2013 (extended till March 2014)
Building Bridges for Youth Action – Part II
28.4 million DKK January 2012 – December 2014
Young Women and Men’s Action to address Injustice in the Arab Region
11.0 million DKK July 2013 – December 2014
20 MS/AADK is abiding by the financial reporting guidelines in terms of the framework agreement; the recommendation is aimed at internal systems for assessing whether resources are used efficiently and effectively.
11
These ongoing grants seek to support the implementation of AAI’s “Arab Regional Initiative
“(ARI). Following MS/AADK’s original MENA strategy issued in 2008, AAI has approved a
new strategy for activities across the MENA region in 2013 and mandated MS/AADK with the
management responsibility of the initiative.21
MS/AADK’s overall development objective for the DAPP is to enhance the capacities of and
opportunities for young people and CSOs/NGOs to participate in social and political change in
North Africa and the Middle East.22
The primary target group of the program are young people aged 15 – 30. This includes
adolescents (as young as 10-12 years old), youth with limited prior engagement, engaged youth
that belong to a CSO, as well as informal youth groups. In addition, the program also targets
NGOs/CBOs (and youth centers to a lesser extent) as intermediaries for promoting youth civic
engagement. Geographically, the activities are focused to selected cities and communities in
Jordan, Lebanon, and the occupied Palestinian territory (i.e. the Levant countries) as well as
Egypt (North Africa).23
5.1 Relevance of the MENA Engagement
The Danish Arab Partnership Program (DAPP) was established by the Danish MoFA in 2003
with a dual objective:
To promote reform and democratization processes in the Middle East and North Africa;
To improve dialogue, understanding and cooperation between Denmark and the MENA
region.
MS/AADK’s objectives for its MENA program are focused on youth, youth organizations, their
mobilization and how to strengthen cooperation and dialogue with youth from Denmark.
MS/AADK’s objectives remain fully relevant under the new DAPP strategic framework 2013-
2016, which defines “strengthening youth participation in public life” as a strategic objective
under its thematic area I “Promoting Human Rights, Human Freedoms, and Good Governance.”
The activities carried out by MS/AADK address the aims stated in the relevant national youth
policies in the three countries where it is active - by training youth to become active and
politically active citizens; by training CSOs; and by supporting campaigning thereby facilitating
active participation in the local democracy.
Findings:
Promoting youth civic and political engagement in MENA is important from a youth
development and governance perspective, and well aligned with Danida priorities under
the Danish Arab Partnership Program.
21 ActionAid (2013). Arab People’s Action to End Injustice and Poverty. Arab Regional Initiative Strategy Paper 2013-2017. 22 The specific wording varies across the three project grants. 23 To a lesser extent activities were also targeted at Syria, Tunisia, and Libya.
12
MS/AADK’s goals for its MENA program are consistent with countries’ policy
objectives regarding young people.
5.2 Activities and Partners in the MENA Engagement
While each of MS/AADK’s ongoing grants from the MFA has some specific elements, all of
MS/AADK’s work in the region can be considered to fall under the following types of activities:
Capacity building through the Global Platform in Jordan and strategic partners targeted a)
youth, and b) CSOs and youth groups, in the areas of campaigning, leadership; human
rights based approach, and governance and accountability. This also includes temporary
placements of professionals (6-24 months) in partner CSOs to strengthen their capacity.
Mobilization through providing “safe” spaces and platforms for youth; facilitating local
and national campaigns; young people’s interaction with local authorities; and
strengthening youth movements/networks e.g. through the Activista network or MENAct
meetings.
Dialogue and exchange including supporting partnerships between Danish and Arab civil
society organizations through the DEMENA Pool; facilitating volunteer and internship
placements for young Danes in the Arab region; and supporting youth from the Arab
region to come to participate in international trainings, campaigns or study visits.
To ensure the relevance of its activities, MS/AADK has put in place training needs assessments
with partner organizations and youth to determine which types of capacity development
interventions would be most useful to the target groups. This is in line with the HRBA whereby
the target group is part of the process to design and decide on what it needs.
Program proposals were developed in close coordination with the MENA regional director; in
consultation with staff in the regional office in Jordan; and program managers from Egypt and
Lebanon and local partners.24 MS/AADK’s approach has been informed by a detailed analysis of
key issues and the civil society landscape in target countries carried out when developing AAI’s
strategy for the Arab region.25
Findings
Overall, MS/AADK’s activities in the MENA region are relevant to the target group and
to the achievement of MS/AADK objectives.
The design of the projects/programs is based on a consultative process with management
and staff from country programs and dialogue with local partners.
24 In addition, a MoU between MS/AADK and the ActionAid office in Palestine was developed to outline mutual roles and responsibilities. 25 ActionAid (2011), Assessment for engaging in the MENA region: Analysis of Lebanon, Syria, Jordan and Egypt and MS/AADK (2012), Desk study for ActionAid International’s Strategy Development Seminar and regional initiative in the Middle East and North Africa (MENA).
13
5.3 Size and Flow of Resources
In the absence of an AA Country Office in the MENA region outside Palestine, the regional
office in Jordan is managed and almost exclusively funded by MS/AADK.26 These funds cover
administrative and staff costs as well as transfers to partner organizations for specific projects.
Project Total Budget (in DKK)
Total expenditure (by Dec 2013)
Youth Engaging in Change Processes in North Africa
9,600,000 6,573,709
Building Bridges for Youth Action – Part II 28,400,000 13,731,994
Young Women and Men’s Action to address Injustice in the Arab Region
11,000,000 0
Total 49,000,000 20,305,703
Of the 20.3 million DKK spent between 2011 and 2013, about 80% was used for program
activities, with slightly less than 20 % for program support, management and administration.
Program activities also include a coordination component, approximately 40% of program
activities are transferred to partners as sub-grants27 and another 40% of program activity
expenditures stem from AADK-implemented activities (e.g. training courses, workshops, etc.).
5.4 Analysis of Performance
The MENA program was appreciated by the beneficiaries interviewed by the Review Team,
especially with regard to the following aspects:
MS/AADK’s T4C approach, content and delivery are adapted to the local context.
MS/AADK’s P4C program. Partner organizations underlined high levels of satisfaction
with Inspirators and Advisors, valuing their professional contributions; the value added
for their organization; the personal relationships; and the element of intercultural
exchange.
MS/AADK’s approach to rally a variety of organizations within a defined geographic area
behind a common goal has proven to be a strong mechanism to foster youth engagement
and mobilization in the region where CSOs traditionally work in isolation.
MS/AADK’s dialogue in developing partnership agreements and tendency to support the
agenda and goals of the local partners rather than imposing its own agenda.
Annex 4 summarizes the achievement of objectives of MS/AADK’s MENA projects for which
data is available28. Progress towards the achievement of objectives is mixed. Targets for youth
training and youth-led initiatives are overall on track or have exceeded but other activities, such
as the DEMENA pool and the capacity building of CSOs in Egypt will likely miss their set
26 From 2013, additional funding has also been raised for the emergency programme by AA UK and AADK. 27 70 percent of sub-grants were composed of sub-grants to local partner organizations and 30 percent of sub-grants were allocated through the DEMENA pool. 28 A robust assessment of the achievements is difficult due to the weaknesses in the M&E and reporting structures
(see section on M&E challenges).
14
objectives. Moreover, the activities under the most recently approved frame agreement have not
been initiated.
The following key issues were identified by the RT as having affected performance:
During 2013, MS/AADK experienced high staff turnover within mostly mid- to senior-
level positions at the Regional Offices, including the Regional Director.
The decision to establish the Syria Emergency Response Program (ERP) drained
resources from DAPP activities.
Work in Egypt suffered from high levels of popular protest, several changes in
government, a deteriorating security situation, and a difficult operating environment for
foreign NGOs e.g. MS/AADK’s inability to formally register.
The escalation of the Syria crisis prevented MS/AADK initiating activities.
Findings:
Overall, AADK’s T4C and P4C activities in MENA are highly appreciated by
beneficiaries and partners. The coalition-building approach is a strong asset to the
program.
Progress towards the achievement of MS/AADK’s MENA objectives is mixed. Targets
for youth training are generally being exceeded. Targets related to the DEMENA pool,
institutional capacity building and engagement efforts are likely not to be achieved. .
5.5 Issues to Consider Regarding Programing of the MENA Engagement
A number of factors may affect MS/AADK’s engagement in the MENA region:
There is a bias towards better educated and already engaged youth participants e.g.
students in high-school or university or university graduates, which is conflicting with
MS/AADK’s strategic objectives to strengthen the voices of the most marginalized.
While it is may be strategically valuable to leverage the propensity of students and
graduates for engagement, there is also a risk of focusing on the “low-hanging fruits.”
MS/AADK Training of Trainers program may not lead to the assumed trickle-down
effects as training and mobilizing others is not necessarily the primary motivation for
some participants.
Partnerships are loosely defined. The roles and objectives for MS/AADK’s partnerships
in MENA have not been clearly articulated in the past. MS/AADK is aware of lack of
clarity in its partner engagements and is planning to revise its partnership approach.
There can be tensions between MS/AADK’s roles as donor, capacity building provider,
and activist organization in the engagement with local organizations. CEWLA in Egypt,
for example, thought they were obliged to accept MS/AADKs training for youth despite
having their own trainers. MS/AADK is too focused on its own methodology with few
strategic partners at national or regional level which could enhance their engagement
beyond local-level organizations (with the exception maybe of the collaboration with
15
UNFPA). Potential for knowledge sharing and leveraging with other stakeholders should
be considered.29
There is some uncertainty about how MS/AADK should engage local and national
authorities. MS/AADK has discontinued capacity development of duty bearers, based on
a recommendation by the most recent evaluation carried out by TANA Copenhagen. As
a result, there is a strategic vacuum if and how authorities should be engaged for best
results. This dilemma becomes obvious in the case of MS/AADK’s work in Zarqa,
Jordan, where partner CSOs strongly value the involvement of the municipality in its
local coalition which has helped them access public spaces; strengthen their voice with
authorities, and the credibility of the CSOs. In the MENA context of very centralized
countries, it could be argued involving national authorities when campaigning for local
issues to increase the legitimacy and chance of success of the initiative. MS/AADK
should continue to explore the possibility of engaging authorities, in the interest of
pursuing the objectives related to rights holders.
The lack of a training portfolio that includes social accountability tools such as
community monitoring, scorecards, and social audits may limit AADK’s ability to provide
a fuller spectrum of opportunities engaging the youth. Given that development partners
and governments increasingly incorporate social accountability and citizen engagement
components in their projects, a broader portfolio of tools might also enable MS/AADK
to mobilize more young people (youth) around specific projects.
5.6 DEMENA Pool
The DEMENA Youth Pool, established in 2006, is administered by MS/AADK as part of the
agreement with MFA on support to the MENA program (Building Bridges for Youth Action).
Grants are provided to three types of project - partner identification, pilot and partnership
projects. To date, 31 projects have been funded, approximately 50% of the total number of
applications received.
5.6.1 Positioning and Administration of the Pool
The comparative advantage of the DEMENA pool is its role as an incubator for less established
civil society organizations and for innovative initiatives. While other small grant pools exist in
Denmark e.g. the Danish Youth Council, the open nature and more risk-taking approach of the
DEMENA pool is considered a distinguishing factor.
The administration of grants is perceived as efficient and appreciated by the applying
organizations. AADK makes a significant investment in supporting prospective applicants,
including comments on drafts of proposals and budgets. Initially, MS/AADK faced some
administrative challenges with regard to the pool. These have been addressed, including the
development of procedures for receiving and screening applications and assisting applicants in
this process. Overall, the process is conducive and flexible to interested organizations e.g. pilot
projects can be submitted on a rolling basis, the processing of applications meet their deadlines
and there are no delays in disbursing funds.
29 MS/AADK informed the RT that it is present in regional fora and has initiated a synergy forum with Danish partners, but it is not seen to result in actual collaboration in joint programming or substantive collaboration.
16
5.6.2 Performance of the DEMENA pool
Implementation in the current grant period has been slow. By the end of 2013, only six projects
were funded, a relatively low number of beneficiaries, leaving an under-spend balance (see table
below).
Indicator Target (by end 2014)
Actual (by end 2013)
Number of new full partnership projects approved through the DEMENA pool
18 6
Number of Danish and Arab youth in dialogue and cross-cultural exchanges*
2,000 507
Grants committed (in DKK) 6.000,000 2.126,429 Data is based on reported numbers by the end of 2013. (*) Numbers are based on completed projects, ongoing programmes have not reported number of youth at time of review.
The main reason for the slow implementation progress is a low number of applications, partner
identification and pilot projects. All which could represent a pipeline for larger projects to be
funded. One reason for lack of applications is MS/AADK’s inability to effectively support
proposals from Arab organizations despite the fact that they are eligible to apply. Only one
project led by an Arab partner organization has been funded in the past. This can be attributed to
the language barriers (applications are accepted in English only) and the lack of structures to help
Arab organizations identify Danish partners. Earlier reviews of the DEMENA Pool have
recommended that MS/AADK’s regional office in Amman should be more involved in
mobilizing and supporting potential applicants in the MENA region, but no systematic efforts to
that end has been initiated. . Another reason is the low demand for small-grant pools in Denmark
in recent years.
MS/AADK has taken some measures to boost demand e.g. changing the selection criteria to
allow successful applicants to reapply as well as putting stronger outreach efforts in place.30 The
open nature of the fund can be seen as a disadvantage given the lack of a clearly defined target
group for the outreach efforts and capacity building of prospective grantees. There is also a
potential tension between supporting new partners and innovative projects and maintaining
relatively high quality standards.
5.6.3 Additional Challenges
It is difficult to assess whether the DEMENA pool’s core objectives are being achieved. The
Danida reporting templates are not necessarily linked to the overall objectives of the DEMENA
pool. For example, grantees are not asked to report on the key indicators of MS/AADK’s grant
agreements with the MFA regarding the DEMENA pool.31 Grantees are not required to report
on progress using the indicators defined in the project applications and the impacts reported rely
on anecdotes and general perceptions.
30 Outreach efforts include communication through embassies, information meetings, newsletters, promotion of the fund to a network of small Danish CSOs, and contacting Danish NGOs with Arab origin. 31 Based on final report template and example of completed final report.
17
While the current selection process is meeting the needs of the DEMENA pool, there is room to
improve the transparency of the selection process. Broad selection criteria32 are laid out in a
guidelines document available on the MS/AADK website33 but this does not provide a specific
list of the selection criteria being used to evaluate the proposals. While the criteria used to
evaluate DEMENA applications are meaningful34, these are not communicated to applicants.
More stringent selection criteria would increase the transparency of the selection process35, and
make the pool less vulnerable to changes among the members of the selection committee.
Linkages between the DEMENA Pool and MS/AADK’s work in Denmark should be
strengthened. MS/AADK is developing a new national strategy which could provide
opportunities to develop linkages for the DEMENA Pool to reach out to less organized youth
groups; provide capacity building support to prospective applicants; and find synergies between
issues relevant to local democracy and ethnic minorities in Denmark and the MENA region.
Findings
MS/AADK is well positioned to manage the DEMENA pool, being a leading
organizations on youth development in Denmark; having strong roots in promoting
intercultural dialogue; having an emerging presence in the MENA region through its
Arab Regional Initiative; and its experience in institutional capacity building.
MS/AADK has been proactive in making adjustments to the DEMENA pool and the
overall administration appears efficient and conducive to meeting the needs of applicant
organizations.
Implementation progress has been slow due to a lack of applications to the pool
M&E arrangements for supported projects are weak and selection criteria are not fully
transparent.
Recommendations:
4. MS/AADK needs to further explore opportunities to foster new applications to the
DEMENA pool. In addition to focusing on strengthening communication as already
carried out, MS/AADK might also want to explore:
Developing a clear communications strategy for the DEMENA Pool, including a
definition of objectives, target audiences, messages, tools and activities,
Targeting not only existing Danish CSOs but also universities (e.g. student
clubs/initiatives) from which innovative ideas may emerge.
32 These include key priorities to be met (i.e. dialogue, gender mainstreaming, communication of results) and specifies project requirements (e.g. eligible countries, targeted age group, focus on innovation). 33 ActionAid, DEMENA Youth Pool – Selection Criteria. 34 These include relevance and realism of proposed activities, target group, partner/management structure, innovation, likelihood of achieving results, budget and cost-effectiveness, sustainability, and communication. 35 There is some indication that unsuccessful organizations have found the feedback communicated to them to justify the panel’s decision as insufficient, potentially discouraging them from resubmitting future applications to the DEMENA pool. This finding can also be supported by the DEMENA pool meeting minutes available to the Review Team, which illustrate that the rejection of an applicant organization was justified with merely five short bullet points (meeting minutes serve as the basis for the feedback shared with applicant organizations).
18
Establishing a process that would allow Arab organizations to be supported in the
proposal development and partner identification stage, for instance by organizing a
separate call for proposals in Arabic through the regional country office and actively
supporting strong applicants in identifying a Danish partner organization.
Formalize a transparent list of evaluation criteria which can be included in the
guidelines for applicants to enhance the transparency of the selection process.
Feedback from the selection panel to successful and unsuccessful applicants should
be systematically structured according to those criteria.
5. Beyond organizing a periodic DEMENA conference, consider facilitating a network of
DEMENA grantees to enhance opportunities for exchange, partnerships, and follow-up
activities beyond the project duration.
6. Strengthen the DEMENA pool’s ability to demonstrate results through stronger
arrangements for Monitoring and Evaluation. This could include
Aligning project application and reporting formats, including with a view to ensure
that pre-defined indicators are used for measuring progress and results and that
information feeds into the key indicators reported under the DAPP agreements
Developing a guidance note on M&E to help applicants put in place relevant
indicators and other appropriate M&E arrangements
Providing a sample evaluation form to supported organizations that they can adapt to
their specific project as needed in order to help them collect relevant information
from their beneficiaries (e.g. satisfaction, changes in knowledge and attitudes, etc.)
6. MS/AADK’s Performance in the Area of Capacity Development MS/AADK’s ambition is that training should go beyond individual outcomes and focus on the
capacity development and empowerment of organisations. Capacity is defined by MS/AADK as
”abilities to achieve strategic change in a sustainable way”36. MS/AADK has two objectives37 for
its capacity development activities:
1. Develop a global training programme that contributes significantly to the capacity development of civil society organisations throughout the world. MS/AADK’s T4C department aims to train approximately 37,000 staff and partners through step-down training within the AAI, 80,000 individuals through on-line training, and directly train 10,000 individuals by 201538. Other outputs targets established are to develop a series of governance related courses, advocacy and campaign courses and youth training, delivered at 11-13 training centers in Africa, America, Asia and Europe.
2. Integrate the P4C program to improve organizational performance of MS/AADK’s target group i.e. transfer skills, knowledge, attitudes to AA Country offices and AAI partner organizations. MS/AADK aims to place 1,200 Inspirators and 300 Advisors during the period 2012-201739 and to expand its P4C offering into 60% of all Least Developed Countries that are part of AAI.
36 “The Six Step Pathway to Capacity Development”. 37 Called Organizational Objectives. 38 There are discrepancies regarding output targets between Implementation Plan A and C as well as discrepancies within Implementation Plan C and between the Implementation Plans and ”Together Against Poverty”. 39 Based on the information provided in meeting with P4C team on March 5 2014. There are two different Implementation Plans (Implementation Plan A and Implementation Plan C) that provide different output targets. Implementation Plan A states that MS/AADK expectes to place a total of 590 Inspirators and 90 Advisors by 2015
19
6.1 MS/AADK’s Capacity Development Approach
MS has adopted a focussed approach to capacity development and is targeted to the specific
needs of the programmes implemented by partners. This is a positive development. Partners and
trainees were very pleased with the applicability of the support given under the programme-led
capacity development model. The main components of MS/AADK’s Capacity Development
methodology that are funded by Danida are: T4C, P4C and Program support and
methodology/strategy development.
6.1.1 Analysis of T4C
MS/AADK provides:
A methodology for designing, delivering and evaluating training, the “Six-step Pathway
for Training Development”, that is generic and can be used for any face-to-face course.
Specific training on democratic governance, HRBA, youth, campaigns, institutional
governance and middle management.
Specific training through scholarship programs that are funded by certain donors (Danida
and Ford Foundation).
The “Six-step Pathway” is a comprehensive and easy-to-follow manual for developing and
delivering face-to-face training and appears to be well-known and applied at the training centers
visited. Training needs assessments are carried out by the training centers (MS-TCDC and AA
Countries). AAI has agreed that the “Pathway” model should be used for all AAI Secretariat-
commissioned training. So far, MS/AADK has only delivered one ‘Pathway’ designed on-line
course. It plans to add content to the “Pathway” relevant to on-line course development once the
organization has more has experience. Given MS/AADK’s ambitions to a) train more than
80,000 young people between 2012 and 201540 and b) to act as a global standard-setting
organization for capacity development for NGOs, the development of a modified or new
“Pathway” for on-line education should be a priority. It is the young people’s learning- and
communication-instrument and cost effective in reaching new countries and target populations.
During 2010-13, MS/AADK has been responsible for training on: Local Governance, HRBA,
Tax Justice, Youth and Disaster, Peace building and conflict management41. MS-TCDC has
delivered other courses on a commercial basis. Some courses offered by MS/AADK’s T4C e.g.
on governance, HRBA and Youth related courses, have been funded by Danida from 2013
onwards through scholarships.
Feedback from participants of the courses at MS-TCDC and in the MENA region was
overwhelmingly positive. MS/AADK’s interactive, peer-to-peer, action-oriented, and multi-
cultural training environment is highly appreciated by participants especially so in the MENA
region. MENA and MS-TCDC participants were all sponsored by MS/AADK’s scholarship
program to attend local governance, HRBA courses and youth oriented courses. The global
survey of training participants that MS/AADK has carried out in 2012 and 2013 also shows that
to support different stratgic objectives. Implementation Plan C states 600 Implementors and 150 Advisors to be placed by 2015. 40 Implementation Plan A. 41 MS/AADK/MS/AADK excel: T4C data 2012 and 2013.
20
the knowledge is used and has helped both the individual and the local NGOs. The step-down
training i.e. the participants training others in what they learned also appears to have worked as
60 % of the participants responding to the training participant survey in 2013 stated that they
have trained up to 50 persons.
The Danida-funded scholarships are only applicable to courses on governance, youth and HRBA.
Focus groups discussion carried out at MS-TCDC indicated that there is a large demand for
organizational development-type courses i.e. project management, fundraising, application-
writing etc. where MS-TCDC is seen as a good, but very expensive option. In the MENA region,
MS/AADK does not offer this type of courses at all.
MS/AADK reports aim to reach a total of 370,000 people with its training and subsequent step-
down training although the RT would highlight the challenge of how to assess the value of the
numbers reached. MS/AADK is attempting to measure capacity through its annual surveys of
training participants which aims to measure training outcomes, not only outputs. The survey
questions measure the participants’ opinion of whether she/he has applied the knowledge gained
through the training and whether the skills acquired have helped the organization. The strategic
and sustainable change aspects of capacity development are not measured. Independent,
observable evidence of training outcomes on an NGO may be difficult to gather through surveys.
Evidence gathered by AA Country offices could be cross-referenced against survey results to gain
a better understanding of the overall outcome of the training.
The Review Team has observed instances in both Africa and the MENA region, where local
NGOs as well as AA country offices were asked to accept training from MS/AADK, for
example, the relatively strong CSO partner CEWLA in Egypt described in the section on MENA.
AA Country offices can, at present, choose other training providers if they wish. There are on-
going negotiations between the AAI and MS/AADK for MS/AADK’s priority AA country
offices to be obliged to accept part of their training from MS/AADK training platforms. This
would appear to contradict the AAI HRBA approach, whereby AA Countries decide on their
own programs, and MS/AADK’s own demand-driven approach. It is important that
MS/AADK’s capacity development methods are a) demanded and b) competitive i.e. value for
money to the AA Country offices. The tying-in of AA Country offices to MS/AADK’s capacity
development methods should therefore not be undertaken.
If a MS/AADK training offering is not taken-up by the AA Country offices, it is important that
MS/AADK assesses why the offering is not needed/demanded so that MS/AADK can review
its theory of change in order to provide the type of activities/courses/support that allows the AA
Country to achieve its objectives and ultimately, AAI’s Key Change Promises.
Finding
MS/AADK’s “Six-step Pathway for Training Development” is a well-developed,
uniformly applied and logical tool for designing, delivering and monitoring face-to-face
training. MS/AADK has only recently started developing on-line training and does
therefore not have a method for developing such learning modalities.
21
Programme led-capacity development is a positive focus by MS/AADK and is assessed
to have increased the relevance of the capacity development efforts.
The methodology used by MS/AADK in training is highly appreciated by all participants
interviewed and validate the surveys undertaken by MS/AADK
MS/AADK does not measure capacity according to its definition “abilities to achieve
strategic change in a sustainable way”.
There are at times tensions between MS/AADK’s roles as donor, capacity building
provider, and activist organization in its engagement with local organizations. The Review
Team’s findings show that MS/AADK has, at times, been seen as insisting on its training
being used in the local AA Country Offices or local partners.
MS/AADK’s on-going negotiations to commission a fixed part of their total training
from AA country programmes each year appears to contradict AAI’s HRBA approach
and MS/AADK’s demand-driven mentality.
MS/AADK has begun to develop on-line courses. Given MS/AADK’s ambitious targets
(number of youth trained and global standard setting capacity developer of NGOs);
MS/AADK should put priority on developing methods for designing and offering on-
line training.
Recommendation
Recommendation related to monitoring outcomes of capacity development is included in section on Monitoring and
Evaluation.
6.1.2 Global Platforms
MS/AADK and AAI have the ambition to expand the number of Global Platforms from eight to
14 in order to contribute to AAI’s Key Change Promises 3 and 6, and MS/AADK’s Strategic
Objective 442. The RT acknowledges the value of a dedicated training concept, especially with
regard to youth engagement and innovation. All participants from GP programmes interviewed
emphasized that training content and methodology were highly relevant to their respective roles.
Some applied their skills in organizations, while others considered it vital in enhancing
opportunities for employment. MS/AADK emphasis on ensuring institutional anchoring, e.g. in
NGO’s, of individual training varied significantly.
The model of Global Platforms having separate structures in some countries, do not coincide
with the ambition having all Global Platforms fully aligned with AA Country Offices. The MOU
on the Global Platforms specifies shared management responsibility for a Global Platform
between the AA Country Office and MS/AADK. However, the AA Country Office is in effect
managing the Global Platform on behalf of MS/AADK and, for example, use the Global
Platform for some of their training needs. A financial surplus at country level will be shared only
if MS/AADK has a surplus globally. In summary, the AA Country Office, formally having a
shared role in the management of the Global Platform, has a financial and legal responsibility but
42 Source: MOU re Global Platforms between MS/AADK and the respective AA Country.
22
little control over the training concepts and methodology 43 44 and is unlikely to share any
financial surplus. MS/AADK intends to hand over full responsibility of the GP to the respective
AA countries45, but no criteria for the handover have been developed, nor has this intention been
formally communicated.
The RT found that MS/AADK’s value added is their ability to develop and safeguard concepts
rolled out in the Global Platforms, rather than to directly manage the platforms. The role of
concept development, quality assurance and mentorship of the Global Platform fits well with
MS/AADK’ commitment to delegated responsibility for capacity development.
Findings:
The Global Platforms have been assessed positively by training participants interviewed
by the Review Team.
MS/AADK is following up on the need for institutional anchoring of participants in the
GP training courses. However, the experiences varied significantly from skills being
transferred to an organizational set up, to being for the benefit of an individual only.
The value added of MS/AADK in the GP concept is predominantly the development of
concepts, and subsequent quality assurance of the methodology.
Recommendations:
7. MS/AADK should clarify criteria for handover of GP responsibility to the respective AA
organizations, and continue to ensure development and quality assurance of the concept
of the GP in line with the overall mandate under the federation.
6.132 Analysis of P4C
Between 2010 and 2013, P4C has placed 844 Inspirators and Advisors. During 2010-2013, 87 %
of all Inspirators were placed with local NGOs. 82 % of all Advisors were placed with an AA
affiliate or associate, supporting local NGO partners and the AA Country office. Local NGO
partners requesting Inspirators from the AA Country Offices who then apply for Inspirators
from MS/AADK. A recent survey of Inspirator recipients showed that 80 % had been part of a
needs assessment and/or participated in designing the ToR for the Inspirator46.
The concept of Inspirators is in line with MS/AADK’s capacity development approach where
the target audience is “those civil society organisations and movements who fight poverty and
demand democratic and rights based alternatives47”. Organizations that have received an
Inspirator report a very high level of satisfaction; emphasizing that Inspirators helped the
43 The RT has visited the GP in Tanzania and Jordan, of which only the Tanzania GP is relevant for the framework
agreement. Therefore the findings in this regard is based on a sample, However, documentation and MOU have
been studied for other GPs. 44 MS/AADK has informed the RT that the GP in Tanzania is closing in August 2014. 45 The concept is likely to be in the form of “Franchises” 46 Presentation by P4C team to Review Team, March 5, 2014. 47 MS/AADK/MS/AADK, “Programme Partnerships with ActionAid Denmark 2014 -2018”. Document not dated.
23
organizations achieve their objectives; and that they had interacted well with the organizations’
own beneficiaries in local communities48.
The Review Team has the following observations regarding the P4C component:
Recruitment processes were generally well managed.
The aspect of South-South cooperation was well received by the organizations and
Inspirators interviewed.
Inspirators still carry out a substantial amount of “gap filling” activities.
Many Inspirators’ expressed that the planning and preparation for their placements was
not sufficient e.g. the time gap between the needs assessment and actual placement is
sometimes too long which makes the need redundant.
The systems established for follow-up on work with Inspirators e.g. quarterly meetings
and debriefings, are not uniformly applied. This sometimes led to a less than optimal use
of the Inspirators.
None of the interviewed local NGOs receiving training through T4C were aware of the
Inspirator/Advisor, and therefore had not benefited.
The Inspirators were worried about the sustainability due to high staff turnover in small
local NGOs.
Inspirators and advisers have concerns with regard to their international health insurance,
preferring local health insurance due to its acceptance by local service providers.
Receiving organizations also need to be prepared and coached to receive an Inspirator in
order to gain full advantage of the facility.
From 2010-2013 none of the placement targets
were met for Inspirators although the numbers
increased substantially in 2013. .
This was due to a need for MS/AADK to ”sell”
the concept of Inspirators to all AA Countries,
and not only to MS/AADK’s priority countries.
During the same period, 50 % of Inspirators were
placed in non-MS/AADK priority countries.
There is a growing awareness of and demand for
Inspirators among AA Countries.
The AA Country offices identify the need for the
Advisors, and placements are administered by
MS/AADK (in contrast to the Inspirator
program). During 2010-2013, 64 Advisors were
placed; 82 % of whom were placed at AA
Country Offices, the vast majority from lower- or
middle-income countries.
48 Presentation by P4C team, March 5, 2014.
24
During 2010-2013, the placement of Advisors has fallen far short of the targets. In 2013, the
number of Advisors increased significantly to slightly below target, and in countries outside the
nine priority countries of MS/AADK. AAI is acknowledging the value of the model.
Findings:
The P4C program is fully in agreement with MS/AADK’s capacity development
approach. MS/AADK’s survey of recipients also shows a high degree of satisfaction and
the Review Team’s focus groups indicated that in many instances; MS/AADK’s theory of
change is working i.e. that by placing Inspirators local communities are strengthened and
end-beneficiaries are able to demand their rights to the authorities.
The Inspirators confirmed that their work had produced results at community and end-
beneficiary level in the areas of local governance, youth mobilization and women’s rights.
6.1.3 Analysis of MS/AADK’s Sub-granting for Program Support
MS/AADK’s sub-granting of the Danida Framework funds for program support to AA Country
offices is a substantial part of the overall Danida framework funding. In 2013, 38 %49 of the total
Danida budget was sub-granted to nine priority AA Country offices (see detailed analysis under
section 4.2 above).
Prior to 2014 AA Country Offices prepared a “Program Document” i.e. a two-year program,
objectives, budget and logframe with details of the local programme NGO partners. AA Country
offices carry out their own activities e.g. training, campaigns, research, networking as well as
support local NGOs funding. As of 2014, the AA Country offices carrying out work related to
the Key Change Promises prepare “Program Objective Plans”. The process of agreeing on a two-
year Partnership Agreement is AA Country led. MS/AADK is seen as a genuine partner,
supporting and helping the AA Country Offices to achieve their goals. During the planning
process, the AA Country offices identify ideas for the programs, and receive feedback from
MS/AADK. They are encouraged to set their own targets and implement the programs. Program
support is negotiated and agreed in Partnership Agreements. MS/AADK does not micro-manage
the implementation. MS/AADK is also seen as a service provider by the AA Country Offices,
mainly through the T4C program and MS-TCDC. The process is made easier by the fact that all
the AA countries are working towards a common set of Key Change Promises, although the
specific links between activities and how they contribute to Key Change Promises is a challenge
in AA, as outlined earlier in the report.
MS/AADK is generally seen as a valuable partner in enhancing the M&E and reporting systems
of AAI. The RT found that MS/AADK have more elaborated M&E systems compared to AAI
and are well positioned to support the wider AAI M&E development efforts.
Program support to AA Country Offices includes sub-grants to local NGOs, funding for
program activities e.g. workshops, training and program staff salaries. A maximum of 15 % to be
49 Off this 8% was utilized for the P4C programme
25
used for staff and administrative support costs. Training carried out by the AA Country offices is
for local communities, end-beneficiaries and networks of CBOs. The interaction between
program officers and local rights program staff and communities is very appreciated and- the
engagement of local rights officers with local communities contributes notably to local
organizations capacity.
The relationship between AA country offices and local partners varies in type and quality. Some
partners reported indirect, and non-intentional, pressure from the AA office in the pursuit of
KCPs. Some local partners, in Uganda and Jordan, reported having changed their own
organisational objectives to comply with AA KCP objectives.50 Several grant agreements
reviewed by the RT in both Uganda and Jordan between national AA office and local partner was
substantially service delivery contracts allowing the AA programme to perform against KCP
targets, rather than assisting a local organisation to pursue their own agenda.
Finding:
MS/AADK and priority AA Country offices appear to work in partnership where there is
ample freedom and independence. The partnerships are based on formal agreements that
are negotiated and adhered to and no micro-management by MS/AADK.
Coaching, mentoring and general assistance from programme officers at Local Rights
Programme Level is highly appreciated by recipients of AA support. MS/AADK should
consider including this as a deliverable in their capacity development monitoring.
Currently “invisible” as a tool for capacity development
6.2 Analysis of MS/AADK’s Capacity Development and AAI’s Theory of Change
Capacity development is not mentioned as a separate strategy or activity in AAI’s Strategy
although it ties in with AAI’s
theory of change. The
MS/AADK’s Strategy defines
capacity development as an
activity to achieve the overall
goals. The Review Team’s
understanding of AAI’s theory
of change and MS/AADK’s
contribution to this is
illustrated below.
The ToC is relevant and
appropriate although the
theory should, once the M&E
system is fully operational,
50 An isolated instance was observed of a national level NGO presenting reports to the respective AA office which had manipulated data in order to fulfil the targets (KCPs) set by AA rather than reflect the work done by the organization out of fear of not remaining as a partner. The instance is not assessed by the RT as requiring further action by AA.
26
Local partners in Egypt.
After initially experiencing problems
with delays in disbursement, an
Inspirator who did not speak the
language and little advisory support
from the AA office in Cairo, the Local
partner organisations in Egypt
received targeted support through
training, Inspirators & Advisor
support. The partner experienced
marked increase in capacity to engage
their communities and collaborate
with local authorities as a result. The
positive impact was attributed to the
combination and close interaction at
programme level.
continuously be tested to see if the methods are appropriate and contribute significantly to the
achievement of both MS/AADK’s Strategic Objectives and the AAI’s Key Change Promises.
Finding:
MS/AADK should regularly test the theory of change i.e. monitor and report against its
Strategic Objectives to verify or disprove if the methods employed are appropriate. This
is done by regular reporting of indicators against targets.
6.3 Analysis of Synergies between Capacity Development Methods
MS/AADK’s ambition is that its capacity development
methodologies be coordinated to have greatest effect e.g.
a local NGO receiving program support can receive
online or face-to-face training at MS/AADK’s training
platforms, and either an Inspirator or an Advisor to
support them in implementing the program or in
following up on the training they have received. Not all
organizations are in need of all aspects of capacity
development and the deployment of the modalities
needs to be based on a needs assessment.
The combination of learning methodologies would make
it possible to ensure that knowledge is transformed and
sustained into practice and/or systems at the local level,
although the prioritization of the various instruments are
not as coordinated as some organizations would wish.
The capacity development initiatives covered by the review were rarely combined. A review of
T4C statistics51 shows only one case of MS/AADK combining all three learning
methodologies.52.
The combination of capacity building methods has the potential to create synergies and a more
sustainable impact for local NGOs. A comparative evaluation, once sufficient samples are
available, of effects of combining capacity building methods with e.g. only training or receiving
an Inspirator, would be a useful test of MS/AADK’s ambition and theory of change.
At present, the planning and implementation of the three capacity development methods is
managed by different teams within the same team. In order to be able to combine the methods, a
common goal for the MS/AADK’s capacity building approach would need to be agreed. Sub-
goals for each of the methods would need to be mutually beneficial. The administrative, financial
and logistical aspects would then need to be coordinated between AA Country offices, training
centers and MS/AADK’s staff responsible for placements.
51 Analysis of T4C and P4C statistics for Uganda, Tanzania, Egypt and Jordan for 2013. 52 However, partners may have received training in other years than the year they benefitted from advisor/inspirator, multiyear cross reference was not undertaken in the review.
27
Finding:
At present, MS/AADK is rarely combining the Capacity Development methods at its
disposal. To succeed in this would require a rethinking of the organization and logistics of
MS/AADK’s Capacity Development approach and setting common goals and
measurement indicators for the three methods. This should be done together with the
AA Country offices that have the direct linkages to the local NGOs.
6.4 Analysis of the Performance of MS/AADK’s Capacity Development Approach
6.4.1 Capacity Development Outputs
During the period 2010-2013 MS/AADK have
significantly increased training activities from
62,000 training days in 2010 to 124,000 training
days in 2013.
In 2013 the largest amount of training was
delivered under the “Youth” program (43%),
which encompasses Global Contact53. 31% of all
training days were delivered on commercial terms
and included language training, seminars and
conferences. Governance, HRBA and tax justice
training represented 25% of all training activities.
During the period 2010-13, data on the type of
training days is difficult to collate as the
definitions have changed and there are a number
of inconsistencies in the data reported to
Danida54 as a consequence of an effort to improve the reporting on the deliverables. However,
governance is the main focus for training and has increased four-fold since 201055.
There is no consistently reported data on who is trained through MS/AADK’s training centres
e.g. type of NGO, AA staff or other, why a comparison or trend analysis is not possible to
prepare. AA staff represented about 16 % of all persons trained at MS-TCDC in 2010 and 2011.
MS-TCDC then mainly trained regional and international NGOs. CBOs represented only about
9% of all persons trained between 2010 and 2011. Statistics from 2012 and 2013 show that 14%
and 12 % of persons trained at all training centers were AA member staff. However, there is no
data on training of regional or nation NGOs or CBOs. The target audience is also the CBOs,
53 Global Contact is MS/AADK’s volunteer program. The mission is to promote a sense of “Global Citizenship”
against the backdrop of growing inequality in a globalised world. Global Contact is included in the People4Change
strategy, although organisationally it functions separately from the Inspirators/Advisers part of P4C. It is not
financed through the Danida Framework. 54 Inconsistencies in terms of what is measured, training days, training participants, training weeks as well as courses that have been re-classified. 55 From 700 persons trained in governance in 2010 to 2,910 persons trained in 2012. Source: Danida Framework Applications.
28
local authorities and the poor and marginalized. The Review Team’s observations are that the
persons trained are primarily university graduates, employees of regional or national NGOs.
The reporting56 against targets established for the period is unclear. It is a narrative as opposed to
measureable indicators and targets appear to change during the period of time. The RT analysis is
that MS/AADK:
- Achieved and exceeded targets57 for T4C and achieved targets in terms of number of
participants (governance and HRBA).
- Did not achieve targets for P4C.
- Achieved targets for their partnership approach with the AA Country offices.
- Did not report against set targets for Governance.
- Did not provide sufficient evidence to be able to conclude if targets have been achieved
for Youth Mobilization in Governance.
6.4.2 Quality
MS/AADK has during the review period endeavoured to collect qualitative data on the outputs
of the capacity development approach through surveys of beneficiaries, training participants,
partners receiving Advisors and Inspirators and impact assessments.
In 2010 and 2011, the MS/AADK M&E system was not robust enough to obtain and analyze
data on learning, behaviour and results of training58. The INTRAC Evaluation in 2011 attempted
to collate data from different sources in order to assess behavioural changes training had on
organizations. The INTRAC Evaluation concluded that 53 % of Governance Course participants
had stated that the course improved the way they delivered their work and 41 % stated to a lesser
extent. These studies further concluded a 90 % satisfaction level in the quality of training at MS-
TCDC, increased visibility of NGOs, improved research and NGOs involved in monitoring
government services.
P4C activities were assessed by carrying out Impact Evaluations in specific countries. The results
revealed a high relevance of P4C’s goals and a large amount of anecdotal positive contributions
from the P4C component on local NGOs.
In 2012 MS/AADK made a concerted effort to establish systems for monitoring both output
and outcome data in line with its “Six Step Pathway for Training Design”. The T4C surveys in
2012 and 2013 show that a majority of
the respondents stated that between 1
and 50 secondary beneficiaries
benefitted from the acquired
knowledge. In 2013 an increasing
percentage (22 % as compared to 14 %
in 2012) of people stated that more than
300 persons benefitted from the
training.
56 As presented in the Danida Framework Applications 2010, 2011, 2012 and 2013. 57 The targest mentioned here are those established in the Danida Framework Application 2012-2015. 58 INTRAC. Report on Training for Change. 2012.
29
One objective of MS/AADK’s training is “step-down training” i.e. when a participant trains his
or her NGO staff or other stakeholders. The T4C survey reports that step-down-training
occurred in majority of cases, with 70% and 60% of the respondents in 2012 and 2013
respectively, training up to 50 persons. However, the information from the survey is not
sufficiently detailed to allow an analysis of what type of knowledge is transferred and to whom.
The Review Team observed that in the MENA project the step-down training was not
necessarily related to MS/AADK’s social change objectives and it was not defined to what degree
the training was utilized. The Review Team finds that the measure for step down training in the
survey, while relevant, is not sufficient to measure whether overall targets are met for people
trained.
The quality of the P4C component is also measured through a survey, sent out exclusively to
NGOs where Inspirator/Advisor were present in 2013. The survey shows that the respondents
are very satisfied with the work of the Inspirators and Advisors. In 2013 and 2014, more than
70% of respondents stated that their skills were still relevant to the NGO. 91% stated that the
work of Inspirators and Advisors had a positive impact of the effectiveness of the organisation.
Inspirators and Advisers highlighted weak coordination between their placement and the
program support provided, resulting in lack of funds to implement activities and thereby often
rendering placement inefficiency. This was a bigger problem for Inspirators usually posted for
less than one fiscal year, compared to Advisers with a 24 month posting.59Nonetheless, the
Review Team also observed a high level of work engagement of the Inspirators their NGO
partners and the communities benefitting. The Inspirators and Advisers expressed that they
achieved substantial results compared to the inputs provided.
The Advisors interviewed by the Review Team were mainly positioned with the AA Country
office and their work was more in line with the ToRs e.g. coaching in the Activista networks,
mentoring NGOs, providing thematic training and developing networking tools
Findings
The surveys of T4C and P4C participants and beneficiaries are comprehensive and a
good measurement of whether the outputs have been achieved. They do not, however,
capture the extent to which organisational capacity has been improved.
MS/AADK’s new monitoring system shows that participants’ are applying the knowledge
gained through the training; that the skills acquired through Inspirators and Advisors
have helped the organizations; and that step down-training is taking place.
The Review Team finds that the measure for step down training in the survey, while
relevant, is not sufficient to measure whether the targets are met for people trained at an
overall level.
59 The Review Team takes note of the small facility available to Inspirators and Advisers whereby they can access small funds to implement their activities, but it could not compensate enough for the lack of resources due to little or no coordination between ToR and program budgets.
30
Recommendation
8. MS/AADK should articulate partnership strategy, clearly differentiating the different
types of potential local, national, and regional partners, this could include:
specifying criteria for engagement with (local) government authorities,
explore channels for increased use and reach of its training concepts and
methodologies, e.g. through partnerships with other INGOs, large local NGOs, and
appropriate public entities
targeting strategy for individuals for the different types of capacity building services
offered. In particular, consider how to be more inclusive of marginalized youth and
target change agents (TOT participants) to maximize impact.
7. Monitoring and evaluation of programs
During the period 2010-2012, MS/AADK monitored Program Support to priority AA Countries
and progress to Framework targets with the format established in their cooperation agreements
with the priority AA Country offices. This formed the basis of their reporting to the Council of
members, Danida, the MS/AADK Board and AAI.
AAI and MS/AADK are now rolling out a new M&E framework which consists of:
AAI’s “Global M&E Matrix” into which all AA members must submit data.
AA members’ own “Country Monitoring Framework”, which feeds into both AAI’s
monitoring system and that of MS/AADK.
MS/AADK’s internal monitoring of its activities in its “Change Monitoring Framework”.
MS/AADK’s internal project reporting to indicators common with the “Change
Monitoring Framework” and with the “Country Monitoring Framework”. This last
system also includes project specific indicators that each project manager chooses.
All AAI countries,
including MS/AADK,
report annually to the
AAI Secretariat on the
Key Change Promises
they select to help
achieve. MS/AADK
reports to the AAI
Secretariat against the
indicators established in
the “Global M&E
Matrix”. In addition,
MS/AADK’s priority
countries submit the
”Annual Reflection
Matrix” to MS/AADK as
31
it requires more detailed information from partners than requested by AAI. MS/AADK review
and collate this information along with data collected through surveys and training statistics and
this is included in the ”Change Monitoring Framework”. This will, from 2013, form the basis of
reporting to the AAI Secretariat, its Board, Council of Members and Danida. MS/AADK has
established 105 indicators to measure progress towards its eight Strategic Objectives. These are
both at output (e.g. number of training days,) as well as at outcome level (e.g. increased capacities
of trained persons). The following illustrates the Review Team’s understanding of how the data is
sent, where it is compiled and to whom it is reported.
MS/AADK did not report against the targets included in the” Danida Framework Application”
submitted in 2012, nor summarized the results of the previous transition period. In the most
recent ”Danida Framework Application” (submitted 2013), MS/AADK structured their activities
and achievements along six of MS/AADK’s eight Strategic Objectives and presented a narrative
description of activities and outputs. At the point of writing MS/AADK have not provided clear
and measurable outcome level reporting.
7.1 Reporting
7.1.1. Annual report
The MS/AADK 2013 “Annual Report” recounts in a narrative and quantitative manner against
each of the eight Strategic Objectives and Organizational Objectives. The report mainly presents
output results e.g. number of Advisors placed; few outcome results are presented. This is
understandable as it is the first report during the current strategy period (2012-2017). MS/AADK
has a total of 105 indicators to measure Strategic Objectives and only some of the indicators are
used to report on expected output results. MS/AADK should consider eliminating many
indicators considering the data gathering efforts needed and that only a few of these are reported
against.
7.1.2 Reporting under the Framework Agreement
Framework reporting has been reviewed up to 2012, and other reporting up to 2013. A number
of observations have been made:
MS/AADK does not report on achievement of all established targets/goals.
Goals for one period (e.g. 2012-2015) are changed year on year.
MS/AADK mainly reports on a programmatic level. The global reporting tends to focus
at input and activity level as opposed to overall result/outcome level.
The Strategic Objective (# 1) is the only one where the results of all country programs are
collated and reported on.
There is no clear summing-up or reporting of the results of the previous strategy “MS in
Transition”. The Review Team’s analysis shows that MS/AADK achieved three of the
eight goals for the period 2010-2012.
Two separate desk reviews of MS/AADK’s reporting to Danida covering 2011 and 2012
corroborate the same findings regarding lack of measureable indicators, weaknesses in reporting
on achievement of objectives, and lack of aggregated reporting.
32
Overall, the MS/AADK’s latest “Annual Report” (2013) shows that its ability to capture output
data and report against its Strategic Objectives has improved substantially since 2012.
7.1.3 Reporting under the DAPP Agreements
MS/AADK manages the three ongoing grants under the DAPP independently and reports
separately on each of these to the MFA. Discussions with the MFA about consolidating the
different grants have been unsuccessful due to the different nature of the funding streams within
the MFA. Reports to the MENA department are prepared according to MFA standard reporting
templates. Objectives, indicators, and targets are organized according to the logical framework of
the original project proposal. The current DAPP reporting arrangements do not allow
MS/AADK and the MFA to reliably assess progress to objectives. Key issues include:
Not all original indicators are used for annual reporting to MFA60 and, as a result, the
Annual Reports do not assess full progress towards the achievement of objectives.
While the quarterly reports from the ARI regional program coordinators to MS/AADK
capture all original indicators, the data is presented in different formats and with different
levels of detail which makes tracking of progress extremely difficult.
In quarterly and annual reports the same activities or beneficiaries are counted towards
the achievement of output and outcome targets under different objectives.61
There are inconsistencies between status reports and data collected through T4C. In part
this can be attributed to training carried out by local partners which is not captured by
T4C data. However, given the lack of differentiation between who was trained by
MS/AADK directly and who was trained through partners, it becomes impossible to
reconcile the different reporting formats.
Findings
The MS/AADK’s latest “Annual Report” shows that MS/AADK’s system to capture
output data to report against its Strategic Objectives has improved substantially compared
to 2012. MS/AADK should be able to improve its reporting also to Danida.
There appears to be a systemic weakness in the manner in which a) MS/AADK ha
managed the goal setting process and b) Danida’s annual review of this process.
MS/AADK has established new goals every year for the framework funding, without
reporting against the ones for the previous year. Danida has not objected, nor condoned
the practise.
MS/AADK’s target setting process and rolling-plans during the period 2010-2012 has led
to changing goals, lack of reporting against the goals and a lack of consistency.
MS/AADK’s reporting against targets is weak and does not allow Danida or other
stakeholders to assess if the targets have been achieved or not.
With changing goals it would appear very difficult for Danida to carry out its oversight
role and for Danida to assess if MS/AADK has achieved the targeted results.
60 MS/AADK states that this practice is accepted by the MFA. 61 For instance, in the 2013 annual report for North Africa, the same youth-led initiatives were counted towards the objective 1 as well as to objective 3.
33
Recommendation
9. Enhance monitoring system to ensure that outcomes are monitored and reported. Clear
targets with regard to outputs and outcomes should be established. Changes in targets
during implementation, and the rationale for changes, should be communicated clearly in
reporting.
7.2 MS/AADK’s New Monitoring System
MS/AADK has developed an impressive data collection system to ensure that it can collect data
from many sources - AA Country offices, training centers, training participants and NGOs
receiving Inspirators among others. The Review Team observed opportunities for further
improvement of the monitoring system:
Some AA Countries have poor systems for collecting data. It may be difficult to know
what data to collect if there is no baseline or systematic data collection methods/tools.
Reporting formats and levels of detail have, during the review period changed from year
to year, making comparisons and measurement against targets difficult.
The periodic reporting from MS/AADK’s Jordan office suffers from double-counting of
activities and beneficiaries under different program components.
There is inconsistency in the numbers of participants reported by the ARI Regional office
in periodic reports and P4C.
These challenges indicate a potential lack of guidance and training in the wider AAI context on
how to define the data to be collected and the reporting formats. The M&E new system may
need some time to “settle” and the above feedback could serve as input into the process of fine-
tuning the system.
Findings
MS/AADK has recently developed an impressive data collection system with several
methodologies, such as; own statistics, on surveys of training participants and managers
of Advisor and Inspirators and from the AA Countries.
MS/AADK’s new system for monitoring and collection of data is, at the time of the
Review, suffering from weaknesses that need to be addressed in order to ensure
MS/AADK’s ability to monitor and report against established results in a reliable
manner.
Recommendation:
10. MS/AADK and AAI need to clearly define how to measure outputs, outcomes and
impact and provide training in data collection and reporting methods to the AA
federation.
7.3 MS/AADK’s System for Evaluation
An M&E system should include regular evaluations in order to gain information on impacts and
results that may not be captured through the regular monitoring system. In 2011 MS/AADK
commissioned INTRAC to carry out a large evaluation of all its main activities. The evaluation
34
methodology included surveys, tracer studies, document reviews and interviews with partners,
staff and stakeholders.
MS/AADK should consider making a plan for evaluations i.e. what should be evaluated, with
what frequency. Lessons learned from evaluations have been widely shared in the past in
accordance with best practice. In order to ensure that evaluation findings continue to be shared;
the monitoring fulfilment of the recommendations by each cost center should be continued.
Finding
MS/AADK has carried out both cross-cutting and theme-specific evaluations during the
period 2010-2014 in accordance with best practice. MS/AADK should consider
developing a plan for evaluation and reviews.
8. Analysis of MS/AADK’s financial management and administration
8.1 Financial management roles and capacity
The responsibility for the organisation’s finances rests with the Council, Board and Secretary
General. The Board approves the consolidated budget and is answerable to the Council for the
Annual Financial Reports. In line with the AAI financial framework, the Board has established a
Finance and Audit Committee, which has an important role in making financial information
accessible to the Board and the Council.62 Nevertheless, evidence suggests that the Finance and
Audit Committee could potentially be more active in supporting the Board in providing financial
oversight, including by controlling the Annual Financial Reports prepared by the MS/AADK
Secretariat and following-up on audit findings and recommendations.
MS/AADK’s Finance Team has been operating at a reduced capacity during the past one year.
The position of Head of Finance was vacant for almost five months (until January 2014), and
both of the former Controllers also left the organisation in 2013. The capacity constraints facing
the Finance Team have coincided with a high level of staff-turnover at the Regional MENA
office, which in combination with other factors has resulted in considerable financial control
weaknesses.
Findings:
MS/AADK has a democratic organizational structure. Accountability is ensured through
a functioning internal governance system, detailed organizational charts, job profiles and
various guidelines and instructions. However, the quality of the financial oversight
exercised by the Board can be questioned.
The recent re-structuring of the MS/AADK Secretariat will potentially strengthen
important support functions as well as the strategic management capacity of the senior
leadership of the organisation. However, the capacity of the Finance Team remains weak,
especially in terms of financial monitoring of MS/AADK’s local offices and non-AA
partner organizations.
62 Interviews with members of the MS/AADK Board.
35
8.2 Management of the Arab Regional Initiative
During 2013 and early 2014 the Regional MENA Office experienced high staff turnover in most
mid- to senior-level positions. These changes severely affected administrative capacity and
implementation of DAPP-funded activities. The turnover in staff appears to have been linked
largely to the weak management capacity63 of the new regional director and the resulting tensions
between the Regional Director and other staff; below market-rate salaries64; and increasing work
pressure on staff.65
While the immediate management issues were addressed in a generally timely and appropriate
manner, longer-term administrative issues have experienced little progress over time. Many of the
issues raised in an action plan developed by the MS/AADK Knowledge Director in spring 2012
are still relevant today including, for example, scattered project documentation and the lack of a
functioning M&E system and PME manual.
MS/AADK maintains more regular communication with the MFA on the MENA Program than
on the activities funded under the framework agreement. Yet, the management and
administrative challenges facing the Arab Regional Initiative appear to have been insufficiently
communicated to the MFA.
Findings:
The Arab Regional Initiative has suffered from weakened administrative capacity and
slow implementation due to management issues and significant staff turnover during
2013 and early 2014.
The changes introduced by AADK to get the Arab Regional Initiative back on track
should provide the necessary foundation to speed up implementation and improve the
regional office’s systems and processes.
Communication and reporting arrangements between MS/AADK and the MFA have
been insufficient in capturing management and administrative issues experienced by the
Arab Regional Initiative.
8.3 Systems and procedures for fund transfer and management
8.3.1 Funds managed within MS/AADK’s own organisational structure
Procedures for monthly closing of accounts and internal financial reporting are in place but have
not been consistently implemented due to the capacity constraints of the Finance Team during
the past one year.66 This has affected MS/AADK’s ability to meet the Danish auditor’s findings
with regard to improving the financial reporting process, including the correlation between
bookkeeping and the preparation of the annual financial statement, although corrective action
63 Issues raised included micro-management, lack of consultation with others, and other approaches inconsistent with AADK’s culture and policies. 64 In response, MS/AADK has conducted a benchmark analysis of relevant salaries Jordan and revised its salary scales for 2014. 65 While ERP and other (DAPP-funded) activities are separate in the ARI organizational chart, there was a lot of overlap in practice. 66 Interviews with MS/AADK.
36
now has been taken.67 From 2014, MS/AADK plans to institute a new system with fourth-
monthly internal status reports, on top of the monthly financial reporting, focusing on
achievements, financial results, risks, learning and considerations for the next period.
Funds are advanced to MS-TCDC and the Global Platforms on a quarterly basis. The
administrative and financial capacity, including the existence of internal guidelines for
administration, varies from one Global Platform to the other. Significant shortcomings have been
found in the financial management of the Regional MENA Office, including the lack of a proper
accounting software, authorisation policies, attestation rules and bank and cash reconciliation
routines.68 MS/AADK is aware of the seriousness of the situation and is taking remedial action.
In addition to the appointment of a new Regional Director and accountant, an “emergency plan”
has been put in place as an interim measure for the period March-May 2014.69 This would have to
be followed by additional efforts to develop written instructions, checklists, templates, etc., and
the establishment of a system to monitor/audit MS/AADK offices abroad (including TCDC and
the Global Platforms) in relation to the MFA requirements and MS/AADK’s financial policies
and procedures in general.
8.3.2 Funds transferred to and managed by local partner organisations
The case of national AA organisations
A significant share of the funds provided under the framework agreement with MFA is sub-
granted to national AA organisations in Africa and Nepal, and regulated in two-year cooperation
agreements. Notably, the agreement refers to MFA’s administrative guidelines for Danish
framework organisations, which are also attached as an annex. MS/AADK has recently
developed a new standard agreement format that will be used from 2014.70
With regard to the framework funds that are transferred from the national AA organisations to
other NGOs and CBOs, it is noted that AA Uganda and AA Tanzania make use of a standard
agreement that defines the terms and conditions for the use of funds and the reporting
requirements. Both AA Uganda and AA Tanzania adheres to MS/AADK’s budgeting and
reporting cycle and funds transferred on time. They also have well-developed internal control
frameworks. Comprehensive financial policies and procedures are in place responding to global
AAI standards, the respective boards hold the organisations accountable to those standards, and
finance section staff members appear competent. However, AA organisations call for more
substantive and regular communication with the MS/AADK Finance Team.
MS/AADK partner organisations under the MENA programme
Partnerships with local NGOs are formalised in agreements between the Regional MENA Office
and the local organisations although not in a standardised manner. Some agreements are only
67 Deloitte, 2013. Action Aid Denmark Management Letter Financial Year 2012. 68 Interviews with Regional MENA Office staff. 69 MS/AADK, 2014. Arab Regional Initiative – Re-establishing sound financial management systems and procedures. Plan for the period March-May 2014. 70 Partnership Agreement 2014 between ActionAid International (Country) and ActionAid Denmark.
37
available in Arabic and/or do not clearly define disbursement arrangements and requirements on
reporting and auditing.
In general, the Regional MENA Office’s control of DAPP funds transferred to local partner
organisations appears weak – for example, there is a lack of a system for tracking reporting
deadlines and compliance with the terms and conditions for the use of funds; agreements do not
include reference to the MFA terms and conditions; and the management letter of the local
auditor71 does not indicate if records of local partner organisations have been reviewed as part of
the audit.
Findings:
Systems for financial management and administration at the MS/AADK Secretariat level
are relatively comprehensive and are being further strengthened with the help of the new
PME system. However, the reliability of the accounting and internal control system can
be questioned given the weaknesses found in the Regional MENA Office.
The Regional MENA Office has not adhered to MS/AADK’s financial policy and
procedures, which by extension has affected the organization’s ability to live up to MFA’s
requirements on accounting and financial control of the MENA programme.
The cooperation agreement between MS/AADK and national AA organizations
communicates the MFA requirements and conditions on Danish framework
organizations, and transfers the relevant accountability to the national AA organization.
While the field visits to Uganda and Tanzania indicate that national AA organizations
have systems and procedures to ensure responsible administration of framework funds,
there is a clearly expressed wish for closer communication and exchange with
MS/AADK’s Finance Team.
8.4 Auditing
MS/AADK’s financial policy and procedures specify that MS/AADK must comply with the AAI
guidelines on choice of auditors and procedures regarding audit. The external audit of framework
funds, including those remitted to AAI partners must be carried out by an internationally
recognised audit company. The final approval of the auditor is granted by the MS/AADK
Council at its annual meeting.
The selection of Danish auditor is based on a public tender and Deloitte have carried out the past
three audits of the expenditures in Denmark.72 However, Deloitte’s audit report for 2013 does
not include any reference to the local audit of the regional MENA office, as the local audit was
delayed and could therefore not be reflected. The local audit of the MENA programme has so far
been conducted by a Jordanian auditor selected by the Regional MENA Office. Some but not all
of the agreements between MS/AADK and local partners in the MENA region include
71 ActionAid Denmark (MENA Regional Office) Management Letter Points. 72 The instruction repeats the requirements presented in the MS/AADK financial policy and procedures manual and includes further details with regard to the planning and performance of the audit.
38
stipulations on auditing. Expenditures taking place at the level of the Global Platform in
Tanzania are not subject for any external audit.73
AAI’s internal audit department also conducts internal audits of all AAI members, including
MS/AADK. Each year a sample of countries is selected for such internal audits. AA Uganda and
AA Tanzania are subject for internal audits every two years. As earlier indicated, AA Uganda also
has its own internal audit department which carries out yearly compliance audits, including of
partner organisations, and has a system whereby issues arising from the internal and external
audits are compiled in an action plan.74 Notably, MS/AADK does not have a similar system for
its country offices and TCDC. The quality of existing follow-up of audit findings is also
questionable given that important recommendations included in the Danish auditor’s
management letter of 2011 were repeated in the one for 2012. However, that most of the
recommendations of the 2013 management letter have been marked as completed.
Findings:
The audit procedures of MS/AADK could be further improved. There is need to adopt
an audit policy that communicates the existing audit requirements in detail within and
outside the organization to ensure that external audits are performed by international
recognized audit companies in line with ISA at every level and that MS/AADK’s Finance
Team as well as the Danish auditor systematically reviews the findings of these audits and
follow-up on corrective action taken.
8.5 Anti-corruption
MS/AADK has an “Anti-Corruption Code of Conduct”75 and a whistle-blower mechanism that
includes procedures for reporting suspicion or evidence of breaches, guidelines for the report,
procedures for investigation of allegations, and provisions for the protection against false
allegations. The code of conduct establishes that allegations may be reported anonymously.
MS/AADK staff members are responsible for informing partner organisations about the code of
conduct. The standard agreement with AA partners include the MFA “anti-corruption clause”,
which conveys a zero-tolerance stance on corruption and requires that similar clauses are inserted
in project agreements with local partners. This is the case in the MoUs between the national AA
organisations and their local partners in Uganda and Tanzania. Some but not all of MS/AADK’s
agreements with local partners under the MENA programme refer to corruption, and none of
the agreements collected by the review team includes the MFA anti-corruption clause.76
The organisation reports that it has not received any reports of corruption during the past year,
and that there are no unsettled cases. MS/AADK’s auditors have neither identified any suspected
or actual fraud. Nevertheless, interviews with AA Uganda suggest that there have indeed been
73 Prior to 2013, the accounting records of the Global Platform in Tanzania were audited as part of the TCDC audit, but this practice has been discontinued. 74 Interviews with AA Uganda staff. 75 MS/AADK, 2010. Anti-Corruption Code of Conduct – For all MS employees worldwide 2008 (Whistle Blowing Policy). 76 There is also an “AAI Policy on Fraud and Corruption”, which is communicated to national AA organisations and is in overall conformity with the MS/AADK code of conduct.
39
incidents that could have given rise to suspicions of corruption and misuse of funds, but which
have not been reported on to MS/AADK.
Findings:
MS/AADK has an anti-corruption policy and whistle-blower mechanism. The MFA
“anti-corruption clause” is incorporated with the standard agreement between
MS/AADK and AA Country offices but is missing from the agreements with local
MENA partners.
While MS/AADK has not received any reports on suspected or proven irregularities
during the past year, the field studies show that there have been incidents that could have
given rise to suspicions. This suggests that MS/AADK’s anti-corruption policy has not
been sufficiently explained to partner organizations.
8.6 Human resource management, time registration and duty travel
Personnel policies and procedures follow the standards of the AAI and are available to all staff
on MS/AADK’s intranet. In addition, the financial policy and procedures manual sets out
principles and processes for employee related expenses. MS/AADK has a Human Resource
Team (HROD) supporting the rest of the organisation on human resource management issues.
The recruitment procedures are outlined in written instructions.77 Employees’ working conditions
and salaries are set out in an employment letter, based on the collective bargaining agreement
between the Ministry of Finance and the different labour unions. A principle has been adopted
that the conditions of work should be competitive on the Danish labour market but never be
leading amongst comparable organisations. A new staff structure with grades following the AAI
system has recently been introduced in the Regional MENA Office.
Findings:
MS/AADK has a comprehensive human resource management system that meets MFA’s
requirements. Personnel policies and procedures are established in writing and accessible
to all staff. Working hours are budgeted and recorded by project, and monitored with the
help of a web-based time tracking system. Regulations are in place to ensure that cost-
consciousness is observed in terms of the organisation of staff travel and accommodation
and the provision of allowances.
8.7 Capacity to generate own financing
Historically, the main share of MS/AADK’s funding has come from MFA. Although still
dependent on MFA funds, the funding base of MS/AADK has since increased and the
organisation has been able to meet the 10 percent requirement on matching funds.78 In 2012
77 MS/AADK, 2014. Recruitment process in MS/AADK. 78 MS/AADK, 2012. Sub strategy D: Resources.
40
MS/AADK contributed DKK 12.6 million in own financing, corresponding to slightly more
than 10 percent of the framework funds used for programme and projects support and P4C.79
From 2013, MS/AADK had to raise 20 percent in own financing, including 5 percent from
fundraising in Denmark and 15 percent from other sources. MS/AADK’s draft annual report for
2013 states that a total of DKK 152,4 million in framework funds were spent during the year.
MS/AADK contribution from fundraising in Denmark corresponded to 6 percent of this
amount, while the share of funding from other sources was 22 percent.80
MS/AADK’s main income-generating activities are conducted through Global Contact and T4C.
However, the net surplus generated by Global Contact and T4C decreased from DKK 6,1
million in 2012 to DKK 3,3 million in 2013.81 MS/AADK’s expectation is that the trend will be
reversed with the adoption of a more strategic and long-term approach to income generating
activities through these two mechanisms.
MS/AADK is also seeking to increase voluntary fundraising by enlisting more members and
increasing the number of other regular givers and campaign supporters. The target is that
MS/AADK should be able to raise DKK 15 million in unrestricted funds by 2015 and DKK 30
million by 2017.82 In 2013, MS/AADK reportedly received contributions from 12,500 individual
donors, up from 10,000 in 2012 and 8,250 in 2011. However, the number of individual donors is
increasing slower than expected and if the growth of unrestricted funds continues at the present
pace, MS/AADK will raise DKK 18 million from individual donors by 2016, which is far from
the 2017 target.83
Findings:
Although still heavily dependent on MFA funds, MS/AADK has diversified its funding
base in recent years and been able to raise the required level of matching funds.
A strategy is in place to increase all types of income, including from public funds,
voluntary fundraising, and income-generating activities. The targets set for income
generating activities and fundraising activities in Denmark are very ambitious.
MS/AADK has decided, after reviewing lessons learned from previous efforts, not to
focus further on raising funds from the Danish business sector.
Recommendations on Finance and Administration:
11. MS/AADK should strengthen the capacity of its Finance Team to carry out systematic
financial monitoring of local offices, projects and partner organizations abroad
12. MS/AADK must ensure that proper accounting systems, including customized
accounting software, are used at all levels of the organizational chain and be accompanied
by written instructions for budgeting, accounting, and financial reporting.
79 MS/AADK, 2013. Mellemfolkeligt Samvirkes årsberetning 2012. 80 MS/AADK, 2014. Draft annual report for 2013. 81 MS/AADK, 2014. Draft annual report for 2013. 82 The amount of unrestricted funds raised is not reported on in MS/AADK’s annual reports. 83 MS/AADK, 2014. Draft annual report for 2013.
41
13. MS/AADK should develop standard agreement templates for different types of
partnerships and partner organizations, clearly conveying MS/AADK’s own and MFA’s
requirements and conditions, including the requirement to report on suspected mal-
practice. All agreement templates must include the MFA “anti-corruption clause”.
14. MS/AADK should describe the procedures for external auditing in greater detail in
MS/AADK’s “Financial Policy and Procedures Manual”, and a system should be
established to ensure that local audit reports are systematically reviewed and followed-up
by the organization
42
9. Summary of recommendations
# Recommendation Responsible Suggested benchmark
(to be developed in consultation with HCP)
Comments
from HCP
1 MS/AADK should initiate a discussion in the wider AAI
and with AA country programmes on the balance between
funding to partners outside and inside the AA federation.
MS/AADK
2 MS/AADK should consider internal systems for
assessment of overall efficiency and effectiveness in its use
of resources84, including actual cost of key outputs.
MS/AADK
3 Clarification from HCP on the criteria for what constitutes
a local partner should be considered.
HCP
4 MS/AADK needs to further explore opportunities to
foster new applications to the DEMENA pool. In addition
to focusing on strengthening communication as already
carried out, MS/AADK might also want to explore:
Developing a clear communications strategy for the DEMENA Pool, including a definition of objectives, target audiences, messages, tools and activities,
Targeting not only existing Danish CSOs but also universities (e.g. student clubs/initiatives) from which innovative ideas may emerge.
Establishing a process that would allow Arab organizations to be supported in the proposal development and partner identification stage, for
MS/AADK
84 MS/AADK is abiding by the financial reporting guidelines in terms of the framework agreement; the recommendation is aimed at internal systems for assessing whether resources are used efficiently and effectively.
43
instance by organizing a separate call for proposals in Arabic through the regional country office and actively supporting strong applicants in identifying a Danish partner organization.
Formalize a transparent list of evaluation criteria which can be included in the guidelines for applicants to enhance the transparency of the selection process. Feedback from the selection panel to successful and unsuccessful applicants should be systematically structured according to those criteria.
5 Beyond organizing the periodic DEMENA conference,
MS/AAI should consider facilitating a network of
DEMENA grantees to enhance opportunities for exchange,
partnerships, and follow-up activities beyond the project
duration.
MS/AADK
6 Strengthen the DEMENA pool’s ability to demonstrate
results through stronger arrangements for Monitoring and
Evaluation. This could include
Aligning project application and reporting formats, including with a view to ensure that pre-defined indicators are used for measuring progress and results and that information feeds into the key indicators reported under the DAPP agreements
Developing a guidance note on M&E to help applicants put in place relevant indicators and other appropriate M&E arrangements
Providing a sample evaluation form to supported organizations that they can adapt to their specific project as needed in order to help them collect relevant information from their beneficiaries (e.g. satisfaction, changes in knowledge and attitudes, etc.)
MS/AADK
44
7 MS/AADK to clarify criteria for handover of GP
responsibility to the respective AA organizations in the
countries and continue to ensure the development and
quality assurance of the concept of the GP in line with the
overall mandate under the federation.
MS/AADK
8 MS/AADK should articulate partnership strategy, clearly
differentiating the different types of potential local,
national, and regional partners, this could include:
specifying criteria for engagement with (local) government authorities,
explore channels for increased use and reach of its training concepts and methodologies, e.g. through partnerships with other INGOs, large local NGOs, and appropriate public entities
targeting strategy for individuals for the different types of capacity building services offered. In particular, consider how to be more inclusive of marginalized youth and target change agents (TOT participants) to maximize impact.
MS/AADK
9 Enhance monitoring system to ensure that outcomes are
monitored and reported. Clear targets with regard to
outputs and outcomes should be established. Changes in
targets during implementation, and the rationale for
changes, should be communicated clearly in reporting.
MS/AADK
10 MS/AADK and AAI need to clearly define how to
measure outputs, outcomes and impact and provide
training in data collection and reporting methods to the AA
federation.
MS/AADK
11 MS/AADK should strengthen the capacity of its Finance
Team to carry out systematic financial monitoring of local
MS/AADK
45
offices, projects and partner organizations abroad
12 MS/AADK must ensure that proper accounting systems,
including customized accounting software, are used at all
levels of the organizational chain and be accompanied by
written instructions for budgeting, accounting, and financial
reporting.
MS/AADK
13 MS/AADK should develop standard agreement templates
for different types of partnerships and partner
organizations, clearly conveying MS/AADK’s own and
MFA’s requirements and conditions, including the
requirement to report on suspected mal-practice. All
agreement templates must include the MFA “anti-
corruption clause”.
MS/AADK
14 MS/AADK should describe the procedures for external
auditing in greater detail in MS/AADK’s “Financial Policy
and Procedures Manual”, and a system should be
established to ensure that local audit reports are
systematically reviewed and followed-up by the
organization
MS/AADK
46
Annex 1: Follow-up on Recommendations from Previous Review
The following table summarizes the conclusions and recommendations from the previous
Review (in 2009) and the Review Team’s observations as to the current state.
Conclusion/Recommendation Implemented
Governance program
1. MS/AADK have made significant progress in terms of
aligning country programs around fewer and more focused themes
re-conceptualizing its personnel assistance MS/AADK,
putting in place a sound programmatic and thematic guidance framework, including useful planning, monitoring and reflection tools at national level.
MS/AADK has during the past year supported AA Country offices to develop Programme Objective Plans (POPs) that link the local accountability work with the national and international level.
2. The Review Team acknowledges the magnitude of work that MS/AADK have undertaken with respect to programming focus and the development of underlying systems and formats for effective project cycle management, including monitoring work.
MS/AADK has continued this work within the framework of AAI. However, the current M&E system has a number of important weaknesses (see Chapter 8 above).
3. The focus of MS/AADK’s governance work comes across as somewhat biased towards the empowerment element of the governance approach. The country programs include initial experiences on promoting accountability, but work on this issue should be further expanded.
MS/AADK has not yet been able to measure the strategic and change aspects of capacity development, which means that the theory of change remains to be validated. In the MENA program, there is uncertainty about how MS/AADK should engage local and national authorities.
4. The Review Team’s discussions and observations highlighted a weakness in connecting core partner activities (at local level) to national level strategic partners, which is critical for advocacy.
Actions taken by MS/AADK:
country program documents “POP” developed by AA Countries together with local NGO partners
MS/AADK developed training material re local-national linkages
MS/AADK established a seed funding mechanism offering support to AA Country offices and partnering civil society organizations.
However, the Review Team’s field studies in Uganda and Tanzania suggest that the national-level impact of local empowerment work is limited. Within the MENA program, MS/AADK has few partners on the policy and national level. This affects MS AADK’s ability to disseminate lessons learned and leverage other stakeholders.
5. The Review Team noted limited concern of and attention to issues of replicability and scalability of activities in Africa.
Replicability and saleability improved through the
development of Communities of Practice throughout the AAI federation
47
consolidation of AAI’s Reflection-Action methodology
good practice storied from Inspiratory and Advisors.
Some aspects of MS/AADK’s approach could be improved such as:
Increasing synergies between P4C, T4C and the program support.
The lack of strong partners at the national and regional level means less reach,
There is an issue as to if relevant ‘step-down’ training is taking place.
6. There are reservations about past and current assessments of cost-effectiveness. This relates to past practice of Development Worker placements as well as some of the current activities that the team observed.
MS/AADK has made the following changes:
Inspirators work based at both national and LRP level.
The Advisor terms and conditions were revised in 2013 to align to AAI conditions, and a low-cost model was introduced.
However, there are concerns about local beneficiaries not being fully involved in the planning of placements and the substantial amount of ‘gap-filling’ activities that the Inspirators carry out, affecting the prospects for sustainability.
7. The Review Team has observed a more instrumental approach to partners who – for all practical purposes – increasingly are delivering outputs to MS/AADK Country Programming Strategies. The concern of the Review Team is whether MS/AADK will be able to ensure that fundamental features of their partnership approach can be maintained within the merged structure and the actual partnership status of national ActionAid organisations – that technically are at par with MS/AADK – vis-à-vis community based partners.
The current partners of AADK are the AA Country offices, which have full ownership and management responsibility of the programs funded by MS/AADK. In the MENA program, the partnership concept remains loosely defined, with some local partners primarily playing the role of service provider to MS/AADK. There is also evidence of tensions between MS/AADK’s role as donor, capacity building provider, and activist organization in its engagement with local organizations.
48
Annex 2: MS/ AADK performance to 2010-12 strategic goals.
MS’ Strategic Goals related to its work in AAI 2010-2012
Results by 2014
To promote and assist in enhancing democracy and participation within ActionAid and to assist in developing credible well rooted ActionAid members.
The reported results focus on strengthening in thematic areas of HRBA and gender and carrying out a stock taking analysis regarding internal governance. The reporting against this goal does not conclusively establish if MS has achieved this goal or not.
To become a ‘lead agency’ within the governance theme in ActionAid
Achieved in 201085.
To provide a world-wide training structure for civil society within the theme of governance
Achieved 2013 (with the new Delegation Agreement).
To play a coordinating and supportive role in developing ActionAid to become a strong international campaign organization
Not assessed as part of this evaluation
To test and prove relevant People4Change approaches and having these mainstreamed in ActionAid.
Reported to be well-established in AAI in 2011 based on evidence from Tanzania and Kenya86. Report also states that this initiative is beginning to be seen as an important part of the development of partner organizations by AAI. However, the uptake of the P4C program has been slow and none of the targets have been achieved during the period 2010-2012. How to test the relevance is therefore difficult as no target value was established.
To help further develop the Partnership Approach in development activities within AA
Not reported.
To assist in developing professional planning monitoring & evaluation systems and practices within AA.
Achieved in 2013 with the new PME framework Global M&E matrix
Support establishment of ActionAid in the MENA region.
Situation in 2014: MS/AADK country office exists in Jordan (not independent AA associate or affiliate). There is one AAI Country Office in Palestine but there are no AA associated or affiliated country offices in the MENA region.
85 Årsberetning 2010-2011 86 Årsberetning 2010-2011
49
Annex 3: Current MS/AADK strategic objectives and indicators
MS/AADK’s Strategic Objectives Example MS/AADK indicator
Objective 1: AADK will contribute to
holding governments and corporates to
account, and thereby improved public
services for five million people living in
poverty.
Number of people, particularly women & youth,
trained to hold public service providers to account
Number of Advisor placements to support partners to
hold public service providers to account
% of course participants who feel that skills and
knowledge acquired has made them perform better in
their work
Objective 2: AADK will contribute to
ActionAid winning significant victories in
achieving fair redistribution of resources.
Number of people, particularly women & youth,
trained in quality and gender responsive public
service financing and tax justice
Objective 3: AADK will contribute to
ActionAid’s work with women and men
affected by disasters
Number of people, particularly women & youth,
trained in accountability in emergency and disaster
responses
Number of advisor placements to support partners in
accountability in emergency and disaster responses
Number of Inspirator placements to support partners
in accountability in emergency and disaster responses
Objective 4: AADK will contribute to
mobilizing 5 million young agents of change
Number of youth trained
Number of Advisor placements
Number of inspiratory placements
Objective 5: Together with AAI, AADK will
develop a regional programme in the Middle
East and North Africa.
Number of people, particularly women, trained
Number of young people, particularly youth who have
increase capacity to act
Objective 6: AADK will support the
capacity of ActionAid and ActionAid
partners to apply a Human Rights Based
Approach.
Number of people trained
Objective 7: AADK will have a significant
impact on Danish and European policy
within our campaign areas, and we will
strengthen cooperation between the Danish
government and international civil society in
matters of common interest.
Number of volunteers engaged by the policy and
campaign team and Global Contact
Objective 8: AADK will ensure that Danes
have an arena for action as global citizens
and strengthen their engagement and
involvement in campaigns aimed at fighting
poverty and promoting sustainable
alternatives.
Number of financial supporters
Number of volunteers engaged by the policy and
campaign team and Global Contact
50
Annex 4: MENA progress to objectives
Objective Target Achievement by end 2013
Youth Engaging in Change Processes in North Africa
Objective 1: Develop a forum for youth to act together at policy development level)
6 Inspirators placed Inspirators placed: 4
Advisors placed: 1
3 of sub-fora created 0 sub-fora created
Objective 2: Develop capacities of young people to play an active role in social change
313 youth trained in toolbox courses 389 youth trained in toolbox courses (plus 669 received other training)
60 youth trained through TOT 35 TOT participants
More than 5 collaborations with youth centers
3 collaborations with youth centers
Objective 3: Support youth-led initiatives and CSOs in facilitating social change and democratic reforms
25-30 CSOs received capacity
building
15 CSOs received capacity building
10 CSOs and 10 youth groups
engaged in local campaigns.
11 CSOs and 9 youth groups engaged in local campaigns.
1,000 youth engaged 700 youth engaged
Building Bridges II
Objective 1: Increased
capacity of CSOs to
become advocates for
change
30 CSOs received trainings 27 CSOs received trainings
8 organizations of those trained have
promoted youth citizenship
education and civic engagement
13 organizations of those trained have promoted youth citizenship education and civic engagement
1,600 young people that have shown
civic engagement
514 young people that have shown civic engagement
Objective 2: Increased the competencies of and opportunities for young activists and youth groups to engage in change processes.
500 youth trained 695 youth trained (plus 570 adolescents trained in basic life skills)
15 CSOs and youth groups have
taken action to address local issues
12 CSOs and youth groups have taken action to address local issues
5 youth groups being engaged 24 youth groups being engaged
Objective 3: Strengthened knowledge of Denmark and the MENA region respectively and increased mutual understanding
18 new full partnership projects were
approved through the DEMENA
pool
6 new full partnership projects were approved
2,000 Danish and Arab youth in
dialogue and cross-cultural
exchanges
507 youth had participated in exchanges through the DEMENA pool
290 Danish participants to be reached through volunteerism, internships, or study tours, about 160 have been reached by the end of 2013
160 Danish participants reached by 2013
DAPP Frame-
agreement
No activities initiated.