Template Project Completion Review

37
1 Project Completion Review - Top Sheet Review Date: 04/12/2020 Title: Proyecto Ganadería Colombiana Sostenible Colombian Sustainable Cattle Ranching Project, also known as the Silvopastoral Systems (SPS) Programme Programme Code: Start Date: December 2012 End Date: January 2020 Summary of Programme Performance Year 2013-14 2014-15 2015-16 2016-17 2017-18 2018-19 Programme Score B A B A A A+ Risk Rating High Medium High High Medium Moderate The programme piloted silvopastoral systems (SPS), which combine tree planting and cattle ranching, on 4,100 Colombian farms to understand their effectiveness at mitigating climate change, restoring biodiversity and reducing poverty. The programme enabled over 38,000 hectares of degraded pasture to be converted to SPS. UK funding resulted in the mitigation of almost 1m tonnes of carbon dioxide equivalent. Per hectare farm productivity increased by 17% on average with some farmers benefitting from per hectare gains worth as much as US$523 in annual income. The programme showed some early signs of delivering a transformational change for the livestock sector by inspiring the Colombian Government to double the sustainable cattle ranching target in its 2018-2022 Development Plan. The successful delivery of climate, biodiversity and poverty reduction benefits also advanced BEIS and HMG priorities. Overall, the programme has been scored as A+: outputs moderately exceeded expectations. Compared to revised end of programme targets, the programme benefitted 100 more farms, enabled SPS implementation on 8% more land and supported environmentally-friendly production systems on 20% more land than expected. Although the target for greenhouse gas abatement was not quite reached, it is likely that this will be met (and exceeded) in the near future. Indicative results from a sample of farms indicate that maximum per hectare productivity gains may have been worth $70 more (annually) than expected. Expectations on leveraging private co-investment from farmers were not met because the UK business case did not foresee the partial trade-off between maximising poverty reduction by supporting small farmers and maximising financial leverage and climate mitigation outcomes by supported larger farms. Nevertheless, the programmes results clearly show that SPS are effective, feasible and attractive to farmers when support is provided, and applicable to a large proportion of Colombia’s cattle farms. The programme generated lessons that can guide the future upscaling of SPS in Colombia and beyond. Programme results showed that implementing SPS can deliver valuable productivity gains, but also demonstrated that upfront financial support is required to enable farmers to establish SPS areas. Analysis of SPS farms revealed that sustained public investment in technical assistance and capital incentives will be needed to attract the private investment required for widespread SPS upscaling. Financial Position Original Programme Value £15,000,000 Extensions/ amendments 2017: Programme extended by two years due to underspend. Additional funds of £258,478 provided to cover administration costs of World Bank oversight for additional two years. 2020: Costless (i.e. £0) extension to enable programme closure activities to be fully completed. Log-frame revisions (with dates) 1. Original version finalised in May 2014 2. June 2016: First revision. Language adjusted and end of programme targets for two indicators reduced. See 2015/16 Annual Review. 3. January 2018: Second revision following programme extension and restructure. Targets increased for several indicators. See 2017/18 Annual Review. Total programme spend £15,258,478 Follow up actions required following closure N/A

Transcript of Template Project Completion Review

Page 1: Template Project Completion Review

1

Project Completion Review - Top Sheet

Review Date: 04/12/2020

Title: Proyecto Ganadería Colombiana Sostenible – Colombian Sustainable Cattle Ranching Project, also known as the Silvopastoral Systems (SPS) Programme

Programme Code:

Start Date: December 2012

End Date: January 2020

Summary of Programme Performance

Year 2013-14 2014-15 2015-16 2016-17 2017-18 2018-19

Programme Score B A B A A A+

Risk Rating High Medium High High Medium Moderate

The programme piloted silvopastoral systems (SPS), which combine tree planting and cattle ranching, on 4,100 Colombian farms to understand their effectiveness at mitigating climate change, restoring biodiversity and reducing poverty. The programme enabled over 38,000 hectares of degraded pasture to be converted to SPS. UK funding resulted in the mitigation of almost 1m tonnes of carbon dioxide equivalent. Per hectare farm productivity increased by 17% on average with some farmers benefitting from per hectare gains worth as much as US$523 in annual income. The programme showed some early signs of delivering a transformational change for the livestock sector by inspiring the Colombian Government to double the sustainable cattle ranching target in its 2018-2022 Development Plan. The successful delivery of climate, biodiversity and poverty reduction benefits also advanced BEIS and HMG priorities.

Overall, the programme has been scored as A+: outputs moderately exceeded expectations. Compared to revised end of programme targets, the programme benefitted 100 more farms, enabled SPS implementation on 8% more land and supported environmentally-friendly production systems on 20% more land than expected. Although the target for greenhouse gas abatement was not quite reached, it is likely that this will be met (and exceeded) in the near future. Indicative results from a sample of farms indicate that maximum per hectare productivity gains may have been worth $70 more (annually) than expected. Expectations on leveraging private co-investment from farmers were not met because the UK business case did not foresee the partial trade-off between maximising poverty reduction by supporting small farmers and maximising financial leverage and climate mitigation outcomes by supported larger farms. Nevertheless, the programme’s results clearly show that SPS are effective, feasible and attractive to farmers when support is provided, and applicable to a large proportion of Colombia’s cattle farms.

The programme generated lessons that can guide the future upscaling of SPS in Colombia and beyond. Programme results showed that implementing SPS can deliver valuable productivity gains, but also demonstrated that upfront financial support is required to enable farmers to establish SPS areas. Analysis of SPS farms revealed that sustained public investment in technical assistance and capital incentives will be needed to attract the private investment required for widespread SPS upscaling.

Financial Position

Original Programme Value £15,000,000

Extensions/ amendments 2017: Programme extended by two years due to underspend. Additional funds of £258,478 provided to cover administration costs of World Bank oversight for additional two years. 2020: Costless (i.e. £0) extension to enable programme closure activities to be fully completed.

Log-frame revisions (with dates) 1. Original version finalised in May 2014 2. June 2016: First revision. Language adjusted and end of

programme targets for two indicators reduced. See 2015/16 Annual Review.

3. January 2018: Second revision following programme extension and restructure. Targets increased for several indicators. See 2017/18 Annual Review.

Total programme spend £15,258,478

Follow up actions required following closure N/A

Page 2: Template Project Completion Review

2

A. INTRODUCTION AND CONTEXT

DevTracker Link to Business Case: http://www.aidstream.org/files/documents/SPS-Business-Case.pdf

DevTracker Link to Log frame: https://science-and-innovation-network.s3.eu-west-2.amazonaws.com/BEIS+ICF/SPS/SPS+Results+Framework+-+Logframe.pdf

The Colombian Sustainable Cattle Ranching Project (also known as the ‘SPS' Programme’) was designed to pilot the implementation of silvopastoral agroforestry systems (SPS) on Colombian farms, in order to establish whether these approaches could contribute to the wider sustainable transition of the agriculture, forests and land use sector. SPS techniques integrate planted trees, shrubs and fodder crops into cattle pasture in ways that can preserve and increase tree cover, sequester carbon dioxide, improve biodiversity, enhance soil quality and increase livestock productivity.

The programme’s rationale theorized that the introduction of SPS techniques would increase the environmental and economic sustainability of cattle ranching in Colombia. Inefficient ranching on degraded land is a key reason that the livestock sector is responsible for 16% of Colombia’s national greenhouse gas (GHG) emissions. Extensive ranching on ever-degrading pasture is also a driver of tropical deforestation. SPS techniques reverse land degradation and biodiversity loss on degraded pasture, sequester carbon dioxide via restored tree cover, and enable higher intensity cattle farming on existing farmland thereby reducing the economic drive to convert forests into pasture.

The programme began with US$ 7m in funding from the Global Environment Facility (GEF). UK funding was added in 2012 and reached the ground in 2014. UK funding expanded the programme’s scope to include: the piloting of a new payment for ecosystem services (PES) scheme that rewarded farmers for the carbon capture resulting from SPS implementation; the extension of the programme into two ‘deforestation hotspot’ areas to test the link between introducing SPS and reducing deforestation; and the provision of technical assistance (TA) to farmers through an expanded network of demonstration farms. In 2017, the programme was extended until 2020 because of early implementation delays and an increase in the purchasing power of programme funds following depreciation of the Colombian Peso.

The Federación Colombiana de Ganaderos (Fedegan), the Colombian Cattle Ranchers Association, served as the lead executing agency, supported by NGO partners that provided technical inputs: Fundación Centro para la Investigación en Sistemas Sostenibles de Producción Agropecuaria (CIPAV), the Nature Conservancy (TNC), and Fondo Acción. The Fedegan-led, cross-organisation team that implemented the programme will hereafter be referred to as ‘the implementation team’. The World Bank performed a supervisory role providing overarching technical and operational oversight to the implementation team, alongside monitoring & evaluation and fund management functions.

The programme aimed to implement SPS on 35,500 hectares of degraded (i.e. open, treeless) pasture across 5 different eco-regions in Colombia, in order to gather evidence on its effectiveness as a sustainable land-use practice. In intervention areas, small and medium-sized farms were targeted, 47% of which were run by households living below the poverty line according to baseline surveys. Adopting an innovative quasi-experimental approach, the programme sought to demonstrate how the introduction of SPS could help reduce greenhouse gas (GHG) emissions and deforestation; restore soil quality, forest cover and biodiversity; and alleviate poverty through: i) increased milk and beef productivity, ii) alternative incomes from forestry products and iii) payments for ecosystem services. Overall, the programme aimed to test if and how SPS could be rolled out at scale, with a view to informing the design and delivery of larger scale SPS interventions in the future.

As a large-scale pilot intervention, the programme demonstrated that silvopastoral techniques are effective at mitigating climate change, feasible for implementation on degraded pasture in a variety of different ecosystems, and attractive to farmers when the appropriate capability building activities and financial incentives are provided. Upon the completion of its execution phase in January 2020, the programme’s package of support had enabled over 38,000 hectares of degraded cattle pasture to be converted to silvopastoral systems, thereby supporting enhanced livelihoods on 4,100 small and medium sized farms and mitigating over 1.5m tonnes of carbon dioxide equivalent (tCO2e) within the programme’s lifespan.

The programme’s successful delivery of climate, biodiversity and poverty reduction benefits in Colombia served BEIS and HMG objectives. The programme advanced the poverty-reduction objectives of UK Official Development Assistance, BEIS Departmental Objective 4.2 to “support clean growth and promote global action to tackle climate change”, and BEIS International Climate Finance objectives to support innovative techniques with the potential to drive sectoral transformation within a priority country partner.

Page 3: Template Project Completion Review

3

B: PERFORMANCE AND CONCLUSIONS

Overall, the programme has been scored an A+: outputs moderately exceeded expectations. Compared to end of programme targets, the programme benefitted more farmers than expected and supported SPS adoption and sustainable land use on a larger area of land than envisioned. Results confirm the climate and biodiversity benefits of the approaches, whilst indicative results from a sample of participating farms also suggest that the programme could have enhanced farmer incomes to a greater extent than expected in the UK business case. This indicates that the programme’s approach to supporting SPS adoption was technically effective, feasible to implement and attractive to beneficiaries.

Although there is less evidence of the programme catalysing a wider SPS transition by farmers and funders not directly involved in the programme, the programme generated important lessons that can guide the future upscaling of SPS in Colombia and beyond. The programme has showcased the potential of SPS conversion to deliver mid-term benefits and demonstrated that upfront financial support is required to enable poor farmers to cope with the resource intensity of establishing SPS. Although the productivity gains of SPS farming models were clearly established, analysis commissioned by the programme found that profitability gains for most types of SPS farm would not be sufficiently attractive for private investors because of the prohibitively high costs of local bank lending. As a result, private investment is unlikely to drive widespread SPS upscaling without further public investment.

Overall Outcome Assessment

The programme was successful in delivering outcomes closely linked to on-farm SPS implementation, but there is less evidence of the programme spurring wider uptake by non-participant farmers or inspiring additional public investment in SPS conversion. Five outcomes were expected in the programme’s Theory of Change1 (see Annex):

1. Reduced GHG emissions from cattle farming as a result of SPS adoption. 2. Increased productivity of cattle farming and increased incomes for farmers. 3. A wide range of environmental benefits generated, including for biodiversity, water and soil, and

increased climate resilience. 4. Wider adoption of SPS in Colombia and further afield as a result of the project demonstrating

benefits. 5. Better informed Government and REDD+ policy and legal support mechanisms for SPS.

The successful delivery of environmental and developmental outputs led to the achievement of outcomes 1-3. As demonstrated by reported progress against project-wide indicators, on-farm implementation of SPS directly sequestered carbon dioxide and led to a demonstrable reduction in off-farm deforestation in intervention areas. Per hectare milk productivity increased by 17% (on average) across participant farms, whilst in-depth evaluation of a subset of 101 farms revealed that SPS techniques had increased milk productivity by 24% (on average) and livestock carry capacity by 23% (on average) to increase farm incomes by as much as $523/ha/year2. This income gain exceeded the maximum estimated in the UK’s Business Case for funding the programme by around $70/ha/year3. As on-farm SPS adoption increased, the proxy measure for biodiversity – the Environmental Services Index – increased on farms, whilst a landscape evaluation of biodiversity identified increased bird and beetle populations and increased species mobility in intervention areas4. Analysis of soil quality and water retention also identified improvements and farmers recognised these improvements in perception surveys.

There is limited evidence that outcomes 4 and 5 have been achieved. The programme was instrumental in building the case for SPS in Colombia and Latin America and it can be credibly claimed that the programme influenced the doubling of the sustainable livestock target in the 2018-2022 Colombian

1 In the programme’s business case, it was agreed that the pre-existing results framework would continue to be used. As a result, the programme did not have a standard ICF logframe. 2 Productivity analysis taken from a probabilistic sample of 101 farms receiving support from the programme. Productivity data summarised in: Mainstreaming Sustainable Cattle Ranching Project : Business Case 3 The 2012 SPS business case assumed that SPS could increase farmer incomes by as much as £300/ha/year. Adjusted for inflation, and converted to USD, this is equivalent to $449/ha/year in 2019 USD. 4 The TNC/CIPAV biodiversity evaluation identified a 32% increase in bird populations and a 47% increase in beetle populations in monitored intervention areas, and determined that the enhanced landscape connectivity generated by SPS systems proved critical for the mobility of 65% of species monitored. Evaluation data summarised in: Project’s Implementation and Completion Report (ICR), prepared by the World Bank (forthcoming).

Page 4: Template Project Completion Review

4

National Development Plan5. However, at this stage the demonstration effects hypothesised by the Theory of Change have as yet failed to materialise fully for outcomes 4 and 5 (see Section H). Nevertheless, the legacy left by the programme increases the likelihood that these outcomes will be realised in the future. New ranchers have continued to visit demonstration farms and learn from the sustainable cattle ranching network following programme closure, whilst survey data indicates that ex-participant farmers are expanding SPS conversion on their farms and their efforts are being replicated by some of their neighbours6. The programme enhanced the evidence base and supported the creation of a series of publicly accessible knowledge products and SPS training modules that have since been mainstreamed into the main curriculum for extension professionally. Collectively, this suggests that the wider adoption of SPS in Colombia may yet materialise.

Output Score and Description

The programme has been scored: A+ – outputs moderately exceeded expectations. This score reflects above expectation achievement on 4 out of 6 project wide indicators and the scoring and weighting of the programme’s output components in the programme’s results framework. Two output components scored an A+ and two scored A. When factoring in impact weighting, the programme can reasonably be judged to score an A+.

The table below sets out progress against end-of-programme (EOP) targets for the project-wide indicators in the programme’s results framework (equivalent of the logframe). Progress against more detailed component-specific indicators is explained in section C. Final results were reported in December 2019, supplemented by analysis on emissions reductions from avoided deforestation in early 2020. Similar to past annual reviews, progress has been compared against the January 2020 EOP targets for each output.

Project-wide Indicator(s) Progress reported at June

2019

End of programme

progress

End of programme

target

1 Area under environmentally friendly cattle ranching production systems implemented in project areas (ha)

96,600 100,515 84,000

2 Land area where sustainable land management practices have been adopted as a result of the project (ha)

34,468 38,383 35,500

3 Increase in the production of milk per intervened hectare in participating farms

17% 17% 10%

4 Improved presence of globally important biodiversity in project areas, as measured by an increase in the Environmental Services Index (ESI) resulting from the adoption of environmentally-friendly SPS in participating farms in project areas, over baseline

1,228,909 1,410,875 1,522,000

5 Reduction in GHG emissions from avoided deforestation and forest degradation and increase in carbon sequestration at the farm-level through adoption of environment-friendly SPS in participating farms (tonnes CO2eq)

1,049,566* 1,565,026** 1,600,000

6 Number of cattle ranching farms benefitting from project instruments (technical assistance, PES, or support for establishment of on-farm nurseries)

4,100 4,100 4000

Notes: *The value of emissions reductions & carbon dioxide sequestered reported in June 2019 results does not include emissions reductions originating from avoided deforestation (KPI 8).**The figure at programme close does include emissions reductions from avoided deforestation. 1,565,026 tCO2e = 1,131,056 tCO2e (cumulative total of carbon dioxide sequestration attributed to the programme) + 433,970 tCO2e (avoided GHG emissions from deforestation/degradation attributed to the programme).

Following weaker progress in early years, the final 3 years of programme operation exhibited strong implementation rates that translated into good results ‘on the ground’. The programme benefitted more farmers than expected and encouraged sustainable practices on a larger area of land than envisioned in the business case. The programme enabled the conversion of 38,383 ha of cattle pasture to be converted to SPS, 8% more land than the revised EOP target. One hundred more farms benefitted than was expected and 20% more farmland transitioned to environmentally-friendly production systems than expected. The programme did not quite reach its EOP GHG abatement target, yet because a larger area was converted to SPS than expected and survey data indicates that most farmers intend to sustain SPS practices, it is likely that the GHG reduction target will be met (and exceeded) in the near future. Indeed, it is estimated that an additional 1.24m tCO2e will be sequestered from land converted to SPS by 2024 (see section D

5 Climate Change Compass Portfolio Evaluation 3: Support for Policy Change, June 2020 6 Data provided in correspondence from the World Bank programme team.

Page 5: Template Project Completion Review

5

and H for further detail on BEIS attribution of GHG abatement, estimates of future GHG abatement and comparison to the business case). The overriding message from these results is that silvopastoral techniques are effective, feasible and attractive to farmers when support is provided, and applicable for a large proportion of the c. 500,000 cattle farms operating across a third of Colombia’s total land area7.

Despite the programme’s good performance, it is important to note that EOP output targets were revised during the Mid-Term Review in 2016 and the timeframe for delivery was extended as a result of the 2017 programme extension (see the 2016-2018 Annual Reviews). Revisions to targets were made because of early delays in implementation, improved land-use baseline data, and improvements in the purchasing power of programme funds. As a result, EOP targets for indicators 2, 4 and 5 were reduced and the EOP target for indicator 1 was increased. Hence, whilst many EOP results exceed revised expectations, several fell short of the expectations of the business case and/or the first iteration of the results framework.

Summary of progress in implementing recommendations from last year’s annual review

• At programme close, 43 of 55 main demonstration farms and 12 TNC-supported demonstration farms remained operational. Most demonstration farms will continue to facilitate training and knowledge exchange, supported by the sustainable cattle ranching network of extension professionals.

• A methodology for reporting KPI 8 (avoided deforestation/degradation) was agreed in Autumn 2019 and results were collected as part of the 2020 ICF Results Collection exercise. KPI8 results demonstrated that deforestation had been avoided in intervention areas, thereby indicating a low likelihood that on-farm SPS implementation generated leakage (see Section D).

• Proactive knowledge dissemination and engagement continued until programme closure. Additional engagement included a session at the 2019 Colombian Cattle-ranching Congress, an event that brought together politicians (including the Colombian President) and sector stakeholders, where the programme team and British Embassy staff presented information about the programme.

• Since the last Annual Review, Fedegan has agreed to maintain the programme website following programme closure to enable continued access to SPS knowledge products.

Lessons and how these have been shared

The operation and results of the programme reveal several important lessons for SPS interventions, the potential to scale SPS, supporting farmers and delivering effective ICF programmes:

Lessons about SPS as a land management practice: As a programme designed to pilot and gather evidence on the effectiveness of silvopastoral systems, the programme has been the subject of considerable study and analysis for dissemination. A fundamental lesson from the programme is that SPS techniques are effective at sequestering carbon and reducing deforestation, and attractive to farmers. Future SPS programmes should factor in that implementation can be hindered by a lack of seed and planting inputs (see section H), severe weather events and increased labour requirements in contexts where labour is in low supply and alternative economic activities are more lucrative.

Lessons about trade-offs between objectives when funding is limited: As expanded on in section H, an unforeseen partial trade-off emerged during programme delivery between maximising the spatial extent of SPS conversion (i.e. maximising GHG mitigation objectives) and maximising poverty reduction. SPS land advances both climate and poverty reduction objectives on farms where it is adopted, but it is not possible to fully maximise both objectives in intervention areas when funding is constrained. This is because each objective is maximised by targeting a slightly different beneficiary group. Maximising spatial extent entails prioritising support for a smaller number of wealthier farmers with larger farms, whilst maximising poverty reduction entails prioritising support for a larger number of poorer farmers with small farms. In the future, UK business cases should give greater consideration to the synergies and trade-offs between multiple objectives, and the manner in which funding allocations and beneficiary selection should be optimised to balance partially competing objectives.

Lessons about financial incentives for smallholders: The need to redesign the programme’s PES-2 mechanism (see section H), indicate that, in isolation, PES mechanisms weighted in favour of ex-post payments are less suited to the constraints faced by small farmers. The programme shows that PES mechanisms can successfully incentivise environment stewardship on small farms, yet it should be recognised that prohibitive upfront implementation costs can still prevent change.

7 Data on ranching sector sourced from: Study on the Implementation and Expansion of Silvopastoral Systems for Colombian Cattle Ranchers

Page 6: Template Project Completion Review

6

Lessons about multi-partner ODA programme delivery: Although combining the comparative advantages of different delivery partners is important for delivering holistic outcomes, these arrangements create challenges. In early years, the programme was beset by problems resulting from the unaligned decision-making and administrative processes between delivery partners that delayed the procurement of inputs, the setting of baselines and early payments to farmers. These challenges were resolved, but future programmes should take more proactive measures to mitigate these risk factors. Additionally, low staff retention in the lead executing partner (Fedegan) undermined implementation in early years, indicating the importance of delivery partners providing salaries commensurate with prevailing market conditions.

Lessons about policy influence: The programme demonstrates the importance of providing evidence to inform host country policy. As explained in section H, the tangible results and evidence amassed by the programme built the confidence of the Colombian National Planning Department to raise sustainable cattle ranching targets in the 2018 National Development Plan8. Through its successful communication efforts and ability to provide technical support to the Colombian livestock NAMA preparation process, the programme also shows the benefits of targeted communication activities for policymakers and the benefits of being able to adapt and respond to requests of the host government.

Lessons and programme findings were shared during the programme’s lifespan and further opportunities for dissemination are being explored. As highlighted in Section C and H, programme findings were shared with a variety of stakeholders in Colombia and programme data was data was shared with several researchers and international organisations with interests in sustainable livestock. The World Bank had planned to hold a series of lesson-sharing events with regional and global stakeholders in 2020, but the COVID-19 pandemic derailed these plans. The BEIS programme team is actively considering opportunities to disseminate programme lessons, results and analysis across BEIS ICF and HMG, and with other countries who could benefit from adopting SPS approaches.

Prospects for scaling-up SPS in Colombia

Replicating the success of this programme on a larger scale will depend upon clear policy frameworks, the institutionalisation of SPS innovations, public-private partnerships, and a package of financial and technical assistance interventions in the market. In 2019, the World Bank team commissioned a study9 to investigate the scaling potential of SPS in Colombia and identify the factors required to realise this scaling potential. Drawing from programme results and economic analysis of the cash-flow of participant farms, the analysis found that:

• SPS methods positively affects 5 dimensions material to the profitability of cattle ranching for beef and dairy: increased carry capacity of land; increased birth rates; increased milk productivity; increased animal weight gain; and lower production costs.

• SPS focusing on increasing dairy production is the most cost effective and most profitable but possesses the least potential to transform land use.

• SPS focusing on beef production has greatest potential to generate ecological benefits from land use change, but implementation is expensive and likely to be deterred by the prohibitive cost of local bank lending (c. 17% interest).

• SPS tailored towards dairy production and animal fattening are most suitable for attracting private investment and can obtain a sufficiently level of profitability, even when factoring in the high cost of local bank lending.

• A return on investment in SPS can typically be realised in 5-7 years, so long term financing is required to support adoption.

The analysis identified several measures required to convert 1.6m hectares of land to SPS and iSPS, the spatial SPS target proposed for the Colombian livestock NAMA. The analysis found that adoption at this scale will require capital incentives and technical assistance to support adoption by farmers and make investment in SPS more attractive for private sector investors. Technical support and capital incentives10, provided as investment subsidies and long-term concessional loans due to their greater impact, were identified as the key ingredients in the recipe for increasing profitability and replication of SPS across Colombia (see Figure 1). Scaling SPS to the level outlined in the NAMA was estimated to require US$375m worth of capital incentives and US$95m worth of technical assistance over 5 years. Although costly, SPS

8 Government of Colombia, Plan Nacional de Desarrollo, 2018-2022: Pacto por Colombia, pacto por la equidad 9 Study on the Implementation and Expansion of Silvopastoral Systems for Colombian Cattle Ranchers 10 The Colombian Government (FINAGRO) credit line that was originally intended to be deployed alongside this programme is an example of such a capital incentive.

Page 7: Template Project Completion Review

7

adoption on this scale is estimated to increase incomes on 187,000 farms, leverage US$570m of private investment and sequester 6m tonnes of CO2 annually (see Figure 2).

Figure 1: Recipe for upscaling SPS in Colombia11

Figure 2: Estimated benefits of SPS/iSPS adoption on 1.6m hectares of land in Colombia (proposed livestock NAMA goal)12

11 Study on the Implementation and Expansion of Silvopastoral Systems for Colombian Cattle Ranchers 12 Study on the Implementation and Expansion of Silvopastoral Systems for Colombian Cattle Ranchers

Page 8: Template Project Completion Review

8

C: DETAILED OUTPUT SCORING

Output Title Component 1. Improving productivity in participating cattle ranching farms in project areas through SPS

Output number in Logframe N/A Output Score A+

Risk: Minor Impact weighting (%): 35%

Risk revised since last AR? No

Impact weighting % revised since last AR?

No

Indicators Progress, June 2019

End of programme

progress

End of programme

target

Area converted to iSPS in participating farms 4,388 4,633 4500

Increase in average stocking rate (cows/ha) in project areas 15% 15% 10%

Number of cattle ranching farmers sensitized on SPS and informed about availability of credit sources

21,294 21,294 18,500

Number of professionals and technicians trained on SPS establishment and management

654 654 550

Key Points

Accelerated rates of implementation in the final three years of implementation enabled the programme to exceed all revised EOP targets. On that basis this component has been scored an A+. Yet, it should be noted that EOP targets were lowered significantly in 2016 because the high input requirements and diminished resources of small farmers meant that converting land to iSPS proved more challenging than anticipated in the UK’s Business Case for funding the programme.

The purpose of this component was to improve productivity on livestock farms via the adoption of silvopastoral techniques, particularly iSPS where fodder crops are planted in addition to trees and shrubs. Activities in this component tested the relative advantages of different intervention strategies to incentivise farmers to adopt SPS, increased the capability of extension professionals to train farmers in SPS techniques, and used a variety of channels to sensitise farmers on the benefits of SPS. Of particular significance was the strategic alliance forged with the National Training Service (SENA), which saw SPS modules being mainstreamed into the technical curriculum for student extension professionals.

Rates of land conversion to iSPS were slow in early years due to unforeseen design and delivery challenges. The programme was originally designed to align its support with a credit line issued by the Colombian Financial Fund for the Agricultural Sector (FINAGRO), which would provide farmers with the upfront financing required to cover the otherwise prohibitively high implementation costs of iSPS. In practice, the FINAGRO credit line was not made available and in its absence the programme’s provision of TA and ex-post PES-2 (carbon) payments proved less effective at enabling iSPS adoption than envisioned. Conversion to iSPS was also hampered by a lack of appropriate seeds and planting inputs and administrative delays in procurement. Nevertheless, implementation barriers were resolved, and knowledge-sharing and outreach strategies improved later in the programme, enabling the implementation team to catch-up on iSPS roll out.

Although procurement challenges were resolved and the PES-2 mechanism was successful adapted (see Section H) in 2015-16, the EOP iSPS target was lowered from 10,000 to 4,500 hectares in 2017. Concurrently, EOP targets for farmers sensitised and professionals trained were increased from 5000 to 18,500 and 100 to 550, respectively. These revised targets were nevertheless ambitious and component 2 activities successfully enabled substantial iSPS uptake in the final three years of the programme.

Summary of responses to issues raised in previous annual reviews (where relevant)

• At programme close, 43 of 55 main demonstration farms and 12 TNC-supported demonstration farms remained operational. Most demonstration farms will continue to facilitate training and knowledge exchange, supported by the sustainable cattle ranching network of extension professionals.

Recommendations for future programmes

• Field days at demonstration farms were instrumental for educating ranchers and convincing stakeholders to prioritise SPS policies and practices. Future programme should emulate this approach.

Page 9: Template Project Completion Review

9

Output Title Component 2. Increasing connectivity and reducing land degradation through differentiated PES schemes

Output number in Logframe N/A Output Score A

Risk: Minor Impact weighting (%): 35%

Risk revised since last AR? No Impact weighting % revised since last AR?

No

Indicators Progress, June 2019

End of programme

progress

End of programme

target

Area under PES schemes in project areas (PES-1 and PES-2)

60,158 (PES-1) 3967 (PES-2)

60,158 (PES-1) 4240 (PES-2)

49,000 (PES-1) 4500 (PES-2)

Number of cattle ranching farms benefitting from a PES scheme (biodiversity or carbon)

1520 (PES-1) 1341 (PES-2)

1866 (PES-1) 1341 (PES-2)

1700 (PES-1)

1255 (PES-2)

Number of focal plant species used/conserved in cattle ranching farms (25 of which are globally important species)

50 50 50

Number of market-based / consumer initiatives designed, (including large-scale PES mechanisms), that could support the broader adoption of SPS by the end of the project

2 2 2

Key Points

After a slow start, positive progress was made against indicators tracking land area and farms benefitting from PES schemes. Land area under the PES-2 (carbon) scheme did not reach its EOP target despite exceeding the EOP target for cattle ranching farms benefitting. This is a product of an unforeseen trade-off between spatial extent and number of farms supported. All other indicators achieved or exceeded EOP targets. As a result, this output has been scored as A.

This component was designed to financially incentivise farmers to preserve and increase ecosystem connectivity by establishing biodiversity corridors and/or increase carbon sequestration by adopting iSPS. These incentives were provided through two short-term PES schemes. The PES-1 scheme paid farmers for measurable increases in biodiversity, whilst PES-2 paid farmers for carbon sequestration via iSPS. This component also supported the design of two long-term funding mechanisms with the potential to catalyse wider adoption of SPS in Colombia. The two mechanisms supported were the “Agua Vivo” water fund designed to increase tree cover within the watershed that supplies water to the city of Manizales, and the Marketing Board of Colombian Angus and Brangus Cattle Producers’ ‘green seal’ certification scheme that could pave the way for produce from SPS farms to be sold to consumers at a premium.

The PES-2 (carbon) mechanism, piloted from 2015, aimed to support iSPS implementation. As explained in sections B and H, the ex-post modality of PES payments was less effective than was envisioned. This resulted in modest early results for component 2 indicators. Yet, once the intervention strategy was revised following the Mid Term Review and the PES-2 mechanism was reconfigured to provide more ex ante support and payments based on hectares converted, results improved. These improvements, combined with modifications to the farmer enrolment processes, reduced the number of farmers withdrawing from the PES-2 scheme i.e. the withdrawal rate fell from 29% (first call) to 7% (fourth call).

On balance, the programme achieved good results on this component. Positive progress on PES-2 results is particularly notable because this scheme prioritised smaller farmers with less capacity to convert land to iSPS. The reason why the PES-2 area EOP target was not reached, whilst the EOP target for farms benefitting was exceeded, can be explained by the trade-off between maximising area converted and number of farmers supported that is discussed in sections B and H. Success remains notable, particularly because farms enrolled in PES-2 were smaller and more dispersed than expected in the business case, which served to increase the transaction costs of delivery.

Summary of responses to issues raised in previous annual reviews (where relevant)

• The finalisation of KPI 8 results demonstrated that off-farm deforestation had been avoided in intervention areas, thereby indicating a low likelihood of leakage from on-farm SPS implementation.

Recommendations for future programmes

• The limitations of payment by results should be considered in future interventions. Upfront resource costs mean that PES mechanisms can be ineffective without complementary support in advance.

Page 10: Template Project Completion Review

10

Output Title Component 3: Strengthening Sub-sector Institutions, Dissemination and Monitoring & Evaluation

Output number in Logframe N/A Output Score A+

Risk: Minor Impact weighting (%): 20%

Risk revised since last AR? No

Impact weighting % revised since last AR?

No

Indicators Progress, June 2019

End of programme

progress

End of programme

target

Number of strategic alliances established with key public and private, national and regional entities for the promotion of SPS in Colombia

11 11 10

M&E system established and providing timely and relevant information on project’s activities and results.

Yes Yes Yes

Communication Strategy implemented for different target audiences (mainly policy makers and farmers)

Yes Yes Yes

Information system in place for reporting farms adopting SPS, including those not directly participating in the Project.

Yes Yes Yes

Key Points

This component generated valuable data on the efficacy of SPS and farmer incentives. Programme results, combined with a series of additional studies, were collated by the World Bank and implementation teams effectively. Good progress was made to capture and disseminate findings to inspire further support for SPS. Moreover, there is evidence that programme data combined with dissemination activities had a meaningful impact on political will for SPS adoption in Colombia. All component reached their EOP targets, hence this output has been scored as A+.

This component was designed to gather and disseminate evidence of ‘what works’ not only to ensure successful implementation, but also to inspire the wider adoption of SPS. Using a variety of approaches, the implementation team tracked and assessed several metrics including: changes in land use and tree cover at the farm and landscape level; changes in livestock productivity (milk and beef); changes in the socioeconomic status of participants; changes in farm level flora and fauna biodiversity; and carbon emissions/ sequestration at project sites. Effective communications, tailored to policy makers, farmers and industry stakeholders were also deployed to positive effect, particularly in the programme’s final years.

The M&E system was slow to be established due to delays in agreeing the methodologies for setting baselines and verifying land use changes. These tasks are inherently complex, yet agreement should have been reached sooner because it effected the payment schedule for farmers under the PES-1 (biodiversity) scheme. Nevertheless, these issues were resolved, and apart from some delays in finalising the methodologies for reporting on ICF Key Performance Indicators (KPIs), the quality of M&E was good.

Overall, M&E and information dissemination activities became a key strength of the programme and were instrumental in reinforcing the evidence base for SPS in Colombia and the region. The programme delivered important analysis on the scaling potential of SPS in Colombia (see 2019 Annual Review), and programme results were used by other researchers to produce relevant knowledge (see section H). Additionally, communication activities garnered the programme high-profile media attention in Colombia and on the BBC (see 2019 Annual Review). Importantly, the findings of the Compass PE3 evaluation of ICF policy influence13 suggest that the programme’s outreach activities contributed to greater ambition on sustainable cattle ranching in Colombia (see section H).

Summary of responses to issues raised in previous annual reviews (where relevant)

• A methodology for reporting KPI 8 (avoided deforestation/degradation) was agreed in autumn 2019 and results were collected as part of the 2020 ICF results collection exercise.

• Proactive knowledge dissemination and engagement continued until programme close. Additional engagements included a high profile session at the 2019 Colombian Cattle ranching Congress, a national level event bringing together political leaders (including the Colombian President) and

13 Climate Change Compass Portfolio Evaluation 3: Support for Policy Change, June 2020

Page 11: Template Project Completion Review

11

important stakeholders in the cattle sector, where the programme team and British Embassy staff (including the British Ambassador to Colombia) presented information about the programme.

Recommendations for future programmes

• Methodological debates between delivery partners, exacerbated by sub-optimal decision-making processes, hampered the setting of baselines and measurement of land use changed. Future programmes should ensure proactive measures are taken to mitigate delivery risks emanating from these factors.

Page 12: Template Project Completion Review

12

Output Title Component 4: Project Management

Output number in Logframe N/A Output Score A

Risk: Minor Impact weighting (%): 10%

Risk revised since last AR? No

Impact weighting % revised since last AR?

No

Key Points

Improvements to programme management following the critical Mid-Term Review in 2016 were sustained until programme close. Enhanced programme management and improved collaboration between implementing partners saw strong implementation and positive on-the-ground results in line with, or exceeding, revised targets. As earlier programme management impediments were successfully rectified to engineer strong results, this output has been scored as A.

The 2016 Annual Review scored this component as “C – Outputs substantially did not meet expectation”. This was score was given because of fundamental delivery delays brought about misalignment between the delivery partners, inefficient procurement and payment processes, and delays to the establishment of an appropriate monitoring and evaluation system. This was exacerbated by low staff retention in the Fedegan implementing team. The 2016 determined that these issues should and could have been rectified sooner. Nevertheless, remedial actions, agreed during the Mid-Term Review and reinforced during programme extension in 2017, were successfully implemented and revised milestones were met. As a result, programme management improved considerably. This positive trend was maintained throughout the final years of the programme.

Reporting to BEIS: The Fedegan-led implementation team provided biannual reports to the World Bank. Although delays in the production of these reports was a feature in earlier years of the programme, these reports were of a good quality and were useful for assessing progress. Following the programme extension in 2017, the BEIS programme manager, the World Bank team and BEIS officials embedded in the British Embassy in Bogota consistently had regular meetings to discuss implementation progress. Programme results against ICF KPIs were reported as part of the results collection exercise at the beginning of each calendar year. The most recent inputs were of high quality and were provided in a timely manner.

Supervisory missions: The World Bank team conducted biannual supervisory missions to monitor progress, and a larger Mid-Term Review Mission was organised in 2016. BEIS officials, often the embedded officer in the British Embassy in Bogota, would feature in one of the biannual trips to validate progress, seek stakeholder views and contribute to the formation of upcoming priorities. The World Bank has also conducted technical visits to participate in dissemination events.

Summary of responses to issues raised in previous annual reviews (where relevant)

• Since the last Annual Review, Fedegan has agreed to maintain the programme website following programme closure to enable continued access to SPS knowledge products.

Recommendations for future programmes

• When delivering through multiple delivery partners, administrative processes within and between delivery partners need close alignment to facilitate delivery. Expectations concerning these processes, and evidence of a feasible and agreed, should be provided in the programme development and initiation phases.

• Staff retention on the implementing team was a significant challenge in the programme’s earlier years that required salaries to be increased to conform to prevailing market conditions. Appropriate salaries for implementing teams should be established from programme initiation to reduce the likelihood of disruptive staff turnover.

Page 13: Template Project Completion Review

13

D: VALUE FOR MONEY & FINANCIAL PERFORMANCE

Key cost drivers and performance

The main cost drivers for the programme were administration fees and the exchange rates between GBP, USD and the Colombian Peso. Following an institution-wide change in World Bank trust fund management fees, administration fees increased to 6.7% of UK funding as a result of the new parallel trust fund established as part of the 2017 programme extension. Currency exchange rates fluctuated over the duration of the programme affecting the value of encashments and disbursements, yet a persistently beneficial exchange rate between the USD and the Colombian Peso prevailed.

At the conclusion of the execution phase in January 2020, the World Bank had disbursed 99% of total resources to Fedegan (see Table 1) and Fedegan had executed 99% of BEIS funding to programme activities (see Table 2). Owing to a small underspend by Fedegan, political delays to the livestock NAMA formulation process and the impact of the COVID-19 pandemic, a cost-neutral extension to programme’s financial close date was arranged in early 2020 to enable the World Bank to oversee continued technical support for the NAMA process and undertake programme completion activities.

Table 1: Status of World Bank disbursements of donor funding to the programme at programme close

GEF BEIS TOTAL

Funding allocated ($) 7,000,000 20,700,000 27,700,000 Funding disbursed ($) 7,000,000 20,527,186 27,527,186 Funding outstanding ($) 0 172,814 172,814

Funding disbursed (%) 100% 99.2% 99.4%

Table 2: Status of Fedegan execution of project funding, by project component at programme close

Component BEIS funding allocated following

2019 restructure ($)

June 2019 amount executed ($) (BEIS)

Programme close amount executed

($) (BEIS)

Programme close % of BEIS funding

executed14

Component 1 9,314,543 8,354,101 9,152,579 98.3% Component 2 6,612,149 5,933,122 6,671,778 100.9% Component 3 2,890,311 1,865,963 2,909,914 100.7% Component 4 1,882,997 1,478,122 1,792,916 95.2%

Total 20,700,000 17,691,308 20,527,187 99.2%

VfM performance compared to the original VfM proposition in the business case

Economy: As a result of the 2017 fee increase, administration fees (6.7% of UK funding) were higher than envisioned in the business case (5% of UK funding). Nevertheless, these fees are proportionate considering the standardisation of cost recovery rates across all World Bank-managed trust funds, and their similarity to administration costs in other ICF forest programming (c.4-7% of UK funding).

Efficiency: As demonstrated by output scoring in sections B and C, the programme achieved or exceeded the vast majority of its revised EOP output targets thereby indicating good efficiency. Of key significance was the achievement of output targets for land area converted to SPS and iSPS because the fundamental climate, biodiversity and development outcomes derive from such conversion. Overall, having rectified slow delivery of outputs in early years, results indicate that the World Bank and implementation teams efficiently allocated programme resources for delivery.

Cost Effectiveness: The business case proposed four key measures by which to assess value for money:

1. Leverage ratio of ICF resources to private / public investment. Reporting of public finance mobilised ceased because programme participants were unable to access the Colombian Government’s FINAGRO credit line as had been originally envisaged in the business case. The programme is estimated to have leveraged £12.2m in private finance15, representing a leverage ratio of 1:0.74. This ratio fell short of business case expectations, where it was expected that £23m in private investment would be leveraged representing a leverage ratio of 1:1.81. The under-delivery of private finance leverage is likely explained by the strategic decision to prioritise support for smaller farmers over

14 The data in this table shows the amount executed ‘on the ground’ by Fedegan, as opposed to the amount disbursed by the World Bank to Fedegan. This explains the difference between the total BEIS budget executed in this table (85%) and the disbursement rate of 89% detailed in the text above. 15 In this case, private finance is understood to be the labour and capital that farmers ‘co-invest’ in SPS adoption.

Page 14: Template Project Completion Review

14

medium sized farmers to maximise poverty alleviation objectives. That is, an unforeseen trade-off emerged between supporting larger farmers with greater resources to ‘co-invest’ and smaller farmers with fewer resources. Minor changes in attribution rate as well as movements in the USD:GBP exchange rate also had a negative impact on figures reported for the leverage ratio.

2. Abatement costs in terms of £ per tonne of CO2e abated. UK-attributed abatement costs are sensitive to BEIS ICF estimation methods that have changed since the programme commenced. In order to ensure conservative benefit estimation, a 25% leakage factor was introduced for GHG abatement results across the forest and land use portfolio16. This leakage factor was applied retrospectively to GHG abatement results for the programme from the March 2019 results collection onwards. However, the academic literature indicates that leakage risks are lower for forest restoration projects than for forest conservation projects. Indeed, leakage risks are likely to approach zero for SPS interventions, which do not displace land-use activities and simultaneously increase long-term carbon storage and agricultural productivity on previously degraded land17. Therefore, it is reasonable to conclude that a lower leakage factor may be more appropriate for the mitigation benefits generated by the on-farm implementation of SPS.

Using figures for actual cumulative GHG emissions reductions within the programme’s lifespan and data from the 2020 results collection, UK-attributed abatement costs have been calculated at a maximum and a minimum leakage factor (see Table 3). Figures for UK-attributed abatement costs within the programme’s lifespan range between £16.68/tCO2e and £19.43/tCO2e, with the latter representing the most conservative figure calculated using the 25% leakage factor.

Table 3: Abatement costs for UK-attributed GHG emissions abatement within programme lifespan

Leakage factor applied* UK-attributed GHG emissions

abatement (tCO2e)

Abatement cost (£/tCO2e)

Maximum leakage factor (25%) 849,023.41 19.43

Minimum leakage factor (5%) 988,635.66 16.68 Note: *Varying leakage factors have only be applied to GHG abatement generated by the carbon sequestration resulting from the direct on-farm implementation of SPS/iSPS. Meanwhile, a 25% leakage factor is consistently applied in both scenarios to GHG abatement resulting from avoided deforestation in intervention areas

In the business case, the expected abatement cost was calculated on the assumption that CO2e sequestration from land converted to SPS would continue for 4 years after the programme closed. Assuming that all land converted to SPS at the time of programme close is maintained for an additional 4 years, and accounting for BEIS attribution and leakage factors, the estimated abatement cost would range between £8.83/tCO2e and £10.82/tCO2 depending on the leakage factor applied. This is based on estimates for total cumulative UK-attributed GHG abatement by 2024 ranging from 1.52m to 1.87m tCO2e (see Table 4).

Table 4: Estimates of UK-attributed GHG emissions abatement costs until 2024

Leakage factor applied* Estimate of UK-attributed GHG

emissions abatement (tCO2e)**

Estimate of abatement cost

(£/tCO2e)

Maximum leakage factor (25%) 1,524,275.63 10.82

Minimum leakage factor (5%) 1,867,816.51 8.83 Note: *Varying leakage factors have only be applied to GHG abatement generated by the carbon sequestration resulting from the direct on-farm implementation of SPS/iSPS. Meanwhile, a 25% leakage factor is consistently applied in both scenarios to GHG abatement resulting from avoided deforestation in intervention areas. **Projected GHG emissions abatement for 2020-2024 only include estimates of projected additional sequestration from on-farm implementation of SPS/iSPS and not projected abatement from avoided deforestation.

The conservative cost per tonne figures for abatement within the programme’s lifespan (£19.43/tCO2e) and abatement to 2024 (£10.82/tCO2e) are higher than the main estimates used in the business case (£8.12/tCO2e) and the 2017 programme extension (£13/tCO2e). However, these figures are not directly equivalent because neither the business case nor the programme extension estimates factored in any leakage risk. As a result, the cost estimate for abatement to 2024, with the minimum

16 The leakage factor accounts for the risk that programmes may unintentional displace GHG emissions outside their accounting boundaries. 17 For example: Schwarze R., Niles JO. and Olander J. 2002 Understanding and managing leakage in forest–based greenhouse–gas–mitigation projects Phil. Trans. R. Soc. A.3601685–1703; Lee, D., Skutsch, M. and Sandker, M. (2018). Challenges with measurement and accounting of the Plus in REDD+.

Page 15: Template Project Completion Review

15

leakage factor applied (£8.83/tCO2e), is a more equivalent figure and demonstrates that the programme delivered an abatement cost similar to the one expected in the business case.

3. Cost per hectare converted to Silvopastoral systems. The average per hectare cost of conversion to SPS delivered by the programme was £430. This is lower than the original business case estimate (£540), which was based on converting 28,000 hectares, whereas the programme successfully delivered an additional 10,000 hectares with the same funding volume.

4. Increase of income for small-scale farmers. The business case expected farm incomes to increase because SPS enables a higher livestock stocking rate and increased productivity per hectare. The programme delivered higher productivity gains than expected, generating a 17% increase in milk production per hectare, on average, compared to the 10% gains estimated in the business case.

Effectiveness: The programme largely delivered on EOP targets that were finalised in 2017. As a result, the programme demonstrated how SPS reduces the impact of cattle ranching on climate change and the environment. Although the headline figure for estimated cost per tonne of CO2e abated is higher than expected in the business case, when similar assumptions are used, the realised abatement cost is close to meeting expectations. All estimates of delivered abatement cost fall within the ICF standard portfolio range of £6-£37/tCO2e, whilst the lowest – yet credible – estimate (£8.83/tCO2e) falls close to the ICF portfolio’s weighted average of £8/tCO2e. This abatement cost estimates indicate good value for money. Indeed, because past SPS projects indicate that farmers will maintain SPS and pasture converted to SPS is capable of sequestering carbon dioxide for several years, the mitigation gains of SPS are likely to extend until 2030 and beyond.

The programme also had a significant impact on the livelihoods of small-scale farmers, which are closely linked to farm productivity. The programme successfully enabled average farm productivity to increase above the programme’s EOP target and the expectation of the business case (10%). Indeed, as highlighted in section B, the in-depth evaluation of a sample of farms was found to have increased farmer incomes by as much as $523/ha/year18, exceeding the business case estimate by $70/ha/year19. Additionally, small farms became the focus of the PES-2 scheme introduced with BEIS funding, for which indicators for number of participants surpassed EOP targets. Overall, the programme had a demonstrably positive impact on reducing poverty amongst the small-scale farmers that were the beneficiaries of the programme.

Assessment of whether the programme represented value for money

On balance, the programme represented good value for money. Following a slow start, funds were disbursed efficiently to enable the achievement of climate and poverty alleviation targets. Productivity enhancements were higher than expected and the extent of pasture converted to SPS was significantly higher than expected in the business case. As a result, in time, the programme can be expected to deliver carbon abatement at a similar or greater level than expected in the business case. The overall impact of achieving these targets is judged to represent good value for money.

Quality of financial management during programme

The quality of financial management during the programme was good, validating BEIS’s confidence in the World Bank’s ability to manage UK funds. The World Bank possessed strong procurement, transparency, risk and financial management policies and processes, and the World Bank team proved effective in managing financial arrangements with the executing partners. The implementation team provided the World Bank with financial information in biannual reports, and the World Bank published the programme’s financial performance data online twice a year as part of regular implementation status and results reports.

Date of last narrative financial report August 15, 2019

Date of last audited annual statement July 2, 2019. Final audit report due by 31/12/2020.

18 Productivity analysis taken from a representative sample of 101 farms receiving support from the programme. Productivity data summarised in: Mainstreaming Sustainable Cattle Ranching Project : Business Case 19 The business case assumed that SPS could increase farmer incomes by as much as £300/ha/year. Adjusted for inflation, and converted to USD, this is equivalent to $449/ha/year in 2019 USD.

Page 16: Template Project Completion Review

16

E: RISK

Overall risk rating: Moderate

Overview of programme risk over the life of the programme

Programme risk was rated as high to medium in the early and middle years of the programme’s operation, falling to risk ratings of medium/moderate in the final two annual reviews. The positive delivery trends identified in the final two annual reviews continued to programme close, thereby sustaining lower risk ratings. Growth in organisational capacity and capability in the delivery chain, combined with favourable exchange rate changes between the US dollar and the Colombian Peso, meant that revised EOP targets were nearly all achieved or exceeded. The increased purchasing power of programme funds also enabled extra communications and policy engagement activities to take place, thereby increasing the likelihood of the programme delivering its longer-term outcomes.

The programme piloted several innovative interventions that had not been tried in Colombia before. This manifested in high programme risk in the early years. Some of these risks materialised (discussed below) to impact rates of implementation yet their arrival provided important lessons for the design of PES mechanisms for farmers and the partial trade-offs between maximising different programme objectives.

Reputational risks emerged when the programme was the subject of critical media reporting in the UK. As explained in the 2017 extension assessment, media reports generally lacked detail and misrepresented the programme’s objectives. Robust responsive lines were prepared in response and coverage did not impact implementation. This negative reporting contrasted considerably with the positive press attention received by the programme in Colombia and the UK in its later years, and serves to highlight the notable changes in media reporting on climate change.

Quality of risk management over the life of the programme

Assessment of risk management over the life of the programme reveals some deficiencies in the early stages, followed by the implementation, albeit delayed, of successful mitigation measures. Several key implementation and operational risks were identified in the business case that materialised due to delayed mitigation responses. Specifically, administrative complexity between the implementing partners, technical complexity in establishing monitoring baselines, design challenges in the PES 2 (carbon) scheme, high staff turnover rates and difficulties procuring planting inputs were slow to be addressed. The materialisation of these risks resulted in lower than expected rates of implementation in the early years, thereby necessitating the programme extension in 2017. However, despite delayed deployment, mitigation measures were ultimately successful in reducing risk to enable the successful delivery of revised targets.

F: COMMERCIAL CONSIDERATIONS

Delivery against planned timeframe

Strong progress in delivering results in the later years of the programme enabled the delivery of the key outputs within the timeframe. Nevertheless, due to initial delivery challenges (see section E), a timeframe extension was required to enable the full implementation and disbursement of programme funds, particularly those directly related to the conversion of land to SPS. Additionally, a short extension to the Memorandum of Understanding between BEIS and the World Bank was required in 2020 to allow for policy engagement, evaluation and financial close activities to be finalised. These activities had been delayed due to political processes in Colombia and the COVID-19 pandemic.

Performance of partnership(s)

BEIS’s primary relationship with the World Bank functioned well. During the early stages of the programme, administrative process between the delivery partners were poorly aligned leading to coordination challenges and delays in procuring inputs for farmers. By the time of the 2017 programme extension, this misalignment had been rectified. Throughout the programme, the World Bank team were responsive and helpful and ultimately proved able to supervise and coordinate the programme’s executing partners.

A key lesson learned about the operation of the partnership was the high value added by having a dedicated BEIS ICF team embedded in the British Embassy in Bogota. This representation in Bogota enhanced oversight of the programme, facilitated coordination between delivery partners, invigorated programme-related engagement with government stakeholders and aided coordination between the programme and other HMG programming in Colombia.

Page 17: Template Project Completion Review

17

Asset disposal and value obtained by BEIS

Management of programme assets was overseen by the World Bank in their supervisory role, under the oversight of a BEIS programme manager. The implementation team, led by Fedegan, provided the Bank biannual reports detailing programme results and explanatory material, which were validated and reinforced by the World Bank’s biannual implementation support missions. The BEIS programme manager and World Bank manager maintained regular contact to monitor and control programme matters and assets, supplemented (in later years) by the oversight of BEIS ICF Forests and Land-Use Team members based in the British Embassy in Bogota.

Natural capital assets created by SPS implementation on farmers land will remain the property of Colombian ranchers. Knowledge assets developed to support the training of farmers have been mainstreamed into the extension agent training curriculum in many areas and will remain the property of those extension service providers. Knowledge assets (e.g. implementation manuals and guidance documents) produced for wider consumption will continue to be available on the programme website, which Fedegan has agreed to maintain following programme closure.

G: CONDITIONALITY

Partnership principles assessment

Not applicable.

H: MONITORING & EVALUATION

Evidence and evaluation

As a programme supporting the piloting of innovative land practices, the SPS programme set out to build the evidence base for SPS and test the effectiveness of different forms of intervention. Programme monitoring, supplemented by in-depth evaluation on a subset of farms, generated important evidence on the effectiveness of SPS for restoring biodiversity, sequestering carbon dioxide and increasing farm productivity20. A perception survey of 345 participant farmers found that respondents could identify the sustainability and yield improvements on the ground and were overwhelmingly ‘satisfied’ or ‘very satisfied’ to have participated21. Data from farms receiving support from the programme has been of considerable interest to academics and international organisations. For example, data and evidence from the programme featured in a UN Food & Agriculture Organisation review of the contribution of silvopastoral systems to delivering sustainable development goals22, whilst some academics have used the programme as an example for guiding the global forest and landscape restoration movement23. This external analysis has added further credibility to the programme’s own results monitoring, evaluations, and perception surveys that show the benefits of SPS and its attractiveness to farmers.

The programme’s Theory of Change (see Annex) was validated to a reasonable extent, with many assumptions between outputs and outcomes holding true, yet two theorised outcomes were not realised in full. As explained in section B, programme monitoring confirmed the delivery of outcomes 1-3 that were linked directly to on-farm SPS conversion, whilst evidence gathered for ICF KPI 8 reporting demonstrated a credible causal link between on-farm SPS implementation and reduced deforestation in intervention areas24.

The wider adoption of SPS in Colombia (outcomes 4) was not realised, largely because the Theory of Change assumed that wider adoption would be supported by a Colombian Government (FINAGRO) credit line that did not materialise in practice. Although the programme was re-engineered to compensate in

20 Productivity analysis taken from a representative sample of 101 farms receiving support from the programme. Productivity data summarised in: Study on the Implementation and Expansion of Silvopastoral Systems for Colombian Cattle Ranchers 21 Perception data will be summarised in the World Bank’s forthcoming Implementation Completion and Results Report 22 Chará J., Reyes E., Peri P., Otte J., Arce E., Schneider F. 2019. Silvopastoral Systems and their Contribution to Improved Resource Use and Sustainable Development Goals: Evidence from Latin America. 23 Calle, Alicia. "Partnering with cattle ranchers for forest landscape restoration." Ambio (2019): 1-12. 24 As only one intervention area proved to be in an active ‘deforestation hotspot’ (i.e. an area with high rates of deforestation, as identified by the Government of Colombia) the link between SPS implementation and reduced deforestation in areas strong drivers of deforestation would require further verification by additional interventions.

Page 18: Template Project Completion Review

18

intervention areas, a lack of credit to cover the upfront implementation costs incurred by farmers fundamentally impeded wider SPS adoption. Moreover, a later evaluation determined that the economics of small-scale cattle ranching meant that public financial support would be critical for upscaling SPS in Colombia25. Yet, there is some indicative evidence to suggest that wider adoption of SPS in Colombia could materialise if further support were provided. A survey of 345 participant farmers found that farmers believed that SPS was not implemented to scale farmers desired largely because of a lack of money, lack of labour and lack of inputs, exacerbated by bad weather26. Moreover, 35 percent of respondents reported that the sustainable practices they introduced on their farms were being replicated by neighbouring farmers who did not participate in the programme27. Hence, although the impact evaluation (see below) would ideally have provided a more rigorous assessment, there is indicative evidence to suggest that SPS has broader appeal and its wider adoption is inhibited by lack of resources rather than lack of motivation.

Although the Compass PE3 evaluation of ICF policy influence28 found evidence that the programme influenced Colombian agricultural policy, better informed policy and legal mechanisms to support SPS (outcome 5) were not fully delivered. The PE3 evaluation determined that the evidence amassed by the programme positively influenced the raising of Colombia’s sustainable cattle ranching targets from 72,000 ha (2018) to 147,000 ha (2022) in the 2018 National Development Plan29. Despite such evidence of the programme’s demonstration effects on policymakers, there are so far few signs that increased commitments have translated into government-funded mechanisms for SPS. Nevertheless, outcome 5 could be realised in the future. There are signs that SPS adoption will be supported by a REDD+ programme (the BioCarbon Fund Initiative for Sustainable Forest Landscapes) in the Orinoquía Region30, and the implementation team led the technical preparation of the Government of Colombia’s NAMA plan for the livestock sector, which is poised to guide government sectoral policy and planning that encourages further SPS adoption.

As revealed by the mid-term review, the design of the intervention needed be changed in several ways over the course of the programme. Firstly, the PES-2 (carbon) mechanism needed to be modified following low early uptake of iSPS. The PES-2 (carbon) mechanism was originally designed to provide a package of financial incentives for adopting iSPS worth up to US$650 per farmer. The allocation of payments was weighted in favour of ex-post performance payments rather than ex-ante payments. There was little uptake for this payment structure, because ex post payments could not solve the fundamental resource constraint farmers faced i.e. insufficient funds to establish SPS. As a result, the PES-2 mechanism was changed to base payments on hectares converted and increase the share of financial support provided ex-ante, which led to a significant rise in iSPS uptake. Secondly, low supplies of appropriate seeds to support conversion to SPS in intervention areas proved to be an unforeseen barrier to implementation. In response, the programme supported seed nurseries and the establishment of on-farm seed banks to secure seed supply.

Thirdly, the intervention design also did not account for a partial trade-off between maximising land converted to SPS and maximising poverty reduction. Maximising the former entailed prioritizing support for wealthier ranchers with larger farms and greater capacity to co-invest in the conversion of land to SPS, whilst maximising the latter entailed prioritising support for poorer ranchers with smaller farms and lower capacity to convert large proportions of their farms. The trade-off resulted from the increased 'transaction costs' of providing technical assistance services to a larger number of farms and the reduced ability of smaller farmers to dedicate labour to SPS establishment and leave pasture unused while SPS establishes itself. As a result, a policy decision had to be made about which objective to maximise and which beneficiary group to prioritise, wherein it was decided to prioritise support for the poorest ranchers to maximise poverty reduction objectives. This partial trade-off between maximising objectives in intervention areas was not fully recognised within the UK business case.

Impact evaluation: The World Bank commissioned a quasi-experimental impact evaluation for the programme to assess the average effect of the programme’s intervention strategies (aka ‘treatments’) on participant farms (aka ‘treatment farms’) via comparison with a matched group of farms in a control group (see the 2017, 2018 and 2019 annual reviews for further detail). Although a methodology was agreed and

25 Study on the Implementation and Expansion of Silvopastoral Systems for Colombian Cattle Ranchers 26 Perception data will be summarised in the World Bank’s forthcoming Implementation Completion and Results Report 27 Data provided in correspondence from the World Bank programme team 28 Climate Change Compass Portfolio Evaluation 3: Support for Policy Change, June 2020 29 Government of Colombia, Plan Nacional de Desarrollo, 2018-2022: Pacto por Colombia, pacto por la equidad 30 World Bank, 2020, Sustainable Cattle Ranching Pays off for Colombian Farmers

Page 19: Template Project Completion Review

19

a credible evaluation team was appointed, data challenges and difficult working arrangements exacerbated by the COVID-19 pandemic led to the delivery of an impact evaluation narrower in scope than envisioned. In particular, the impact evaluation has not provided clear evidence of the attractiveness of SPS practices to non-participant farmers, a factor that is important when considering the prospects for the wider adoption of SPS practices. Findings from the evaluation report are summarised in the World Bank’s forthcoming Implementation Completion and Results Report and will inform a broader report to be published by the World Bank technical team.

Notwithstanding its limitations, the impact evaluation found that the programme’s interventions enabled substantially more SPS adoption than would have occurred without support. By the end of 2020, treatment farms had collectively adopted a form of SPS on almost 9,200 hectares more land than the control group, and collectively established 5000km more living fences than the control group. The impact evaluation also indicated that providing technical assistance alone could result in substantial land use change, with the additional provision of PES payments generating a large additional impact for the implementation of iSPS only. Dispersed trees in pasture was the most popular form of SPS implemented on treatment farms, with many participants requiring only TA to enable adoption.

Monitoring progress throughout the programme

The evidence used for this project completion review was sourced from the 2020 ICF results collection exercise and the 2019 Annual Review. Throughout the operation of the programme, data for annual reviews and results collections was sourced from Fedegan’s biannual implementation reports, the World Bank’s aide memoires following monitoring missions, and discussions with the World Bank team and BEIS ICF staff embedded in the British Embassy in Bogota, Colombia.

The programme’s monitoring and evaluation system was fully operational and collected results throughout the programme’s lifespan. The implementation team regularly reported against the indicators in the results framework (equivalent to a logframe). As the programme was established as an extension of a pre-existing programme managed by the World Bank, it was agreed at business case stage to continue use of the pre-existing results framework but establish monitoring approaches for reporting against ICF KPIs. As of January 2020, the programme’s final reporting of progress against ICF KPIs stood as follows:

• ICF KPI 631 - Net Change in Greenhouse Gas Emissions (tCO2e) – tonnes of GHG emissions reduced or avoided. At programme close, the accumulated emissions reductions attributable to UK funding stood at 849,023 tCO2e (when the most conservative leakage factor of 25% is applied). This figure includes on-farm sequestration of carbon dioxide and attributed off-farm avoided deforestation in intervention areas. After early challenges in establishing consensus on agreed methodologies, results against KPI 6 were collected as part of annual ICF results collection exercises. In 2020, the KPI 8 methodology was also finalised, enabling GHG emissions reductions resulting from attributed avoided deforestation in intervention areas to be reported. The figure reported here is sourced from the 2020 results collection exercise. As discussed in section D, the amount of emissions reduced as a result of UK funding is estimated to grow after programme closure, but there are no plans to monitor or verify these future reductions.

• ICF KPI 8 - Number of hectares where deforestation and degradation have been avoided through ICF support (ha). Over the duration of the programme, 2,240 hectares of avoided deforestation was attributable to UK funding. Results against KPI 8 were reported for the first time in the 2020 results collection.

• KPI 12 - Volume of private finance mobilised for climate change purposes as a result of ICF funding (£). At programme close, the value of private finance mobilised attributable to the provision of ICF funding was £12,213,384. This is based on the amount co-invested in implementing SPS and iSPS by farmers receiving support from the programme (an average, per hectare figure), multiplied by the number of hectares converted to both land-use types. Results against KPI 12 were reported as part of results collection exercises, and this figure is sourced from the 2020 results collection exercise.

• KPI 15 - Transformational Change. A methodology for assessing Transformational Change has been developed and the most recent full scoring was set out in the 2019 Annual Review. The programme was assessed as scoring a: “2 - some early evidence suggests transformation likely”.

• KPI 17 - Hectares of land that have received sustainable land management practices as a result of ICF support. KPI 17 was introduced as a new ICF KPI in 2020. Although the programme did not

31 These refer to ICF-wide KPIs rather than the KPIs which are specific to this project, which are assessed within this review and referred to as ‘project development objectives’

Page 20: Template Project Completion Review

20

report against KPI17 during its operation, programme monitoring data of on-farm land-use can be used to report against this indicator retrospectively. Programme activities attributable to BEIS funding directly enabled sustainable land management practices on 27,831 hectares of land. This sustainable land management took the form of converting previously degraded land to SPS and iSPS. Programme activities likely contributed to sustainable land management on an additional area of on-farm secondary forest, yet direct attribution cannot be established with confidence.

Page 21: Template Project Completion Review

21

ANNEX: THEORY OF CHANGE

Page 22: Template Project Completion Review

Smart Guide The Programme Completion Review is the opportunity to reflect on the entire programme, its performance, achievements, lessons and how learning will be shared to inform future programming.

The Programme Completion Report assesses and rates outputs using the following rating scale. The Aid Management Platform (AMP) and the separate programme scoring calculation sheet will calculate the overall

output score taking account of the weightings and individual outputs scores

Description Scale

Outputs substantially exceeded expectation A++

Outputs moderately exceeded expectation A+

Outputs met expectation A

Outputs moderately did not meet expectation B

Outputs substantially did not meet expectation C

Teams should refer to the considerations below as a guide to completing the annual review template.

Summary Sheet

Complete the summary sheet with headline information on the programme and any follow up actions

Early closure is likely to be summed up by issues such as ‘poor performance by partners’ and/or ‘major changes in operating environment’

A: Introduction and Context

Briefly outline the programme, results achieved and contribution to the overall Operational Plan and BEIS’s international development objectives. Where the context supporting the intervention has changed from that outlined in the original programme documents explain what this will mean for UK support

B: Performance and conclusions

Outcome Assessment

Brief assessment of whether the programme achieved the Outcome

Overall Output Score and Description

Progress against the milestones and results achieved that were expected as at the time of this review.

Lessons Any key lessons you and your partners have learned from this programme Have assumptions changed since design? Would you do differently if re-designing this programme?

How will you and your partners share the lessons learned more widely in your team, across BEIS and externally

C: Detailed Output Scoring

Output

Set out the Output, Output Score

Output Score

Enter a rating using the rating scale A++ to C.

Impact Weighting (%)

Enter the %age number which cannot be less than 10%.

The figure here should match the Impact Weight currently shown on the logframe (and which will need to be entered on AMPas part of loading the Annual Review for approval).

Revised since last Annual Review (Y/N).

Risk Rating

Risk Rating: Minor/Moderate/Major/Severe

Enter: Minor, Moderate, Major or Severe

The Risk Rating here should match the Risk currently shown on the logframe (and which will need to be entered on AMP as part of loading the Annual Review for approval).

Where the Risk for this Output has been revised since the last review (or since inception, if this is the first review) or if the review identifies that it needs revision explain why, referring to section E Risk

Key points

Summary of response to issues raised in previous annual reviews (where relevant)

Page 23: Template Project Completion Review

Recommendations for future programmes

Repeat above for each Output.

D: Value for Money and Financial Performance

Key cost drivers and performance Consider the specific costs and cost drivers identified in the Business Case Have there been changes from those identified in previous reviews or at programme approval. If so, why?

VfM performance compared to the original VfM proposition in the business case. Performance against vfm measures and any trigger points that were identified to track through the programme If programme was closed >1 month early please describe in a few words why, what savings were made, as well as what money was spent.

Assessment of whether the programme represented value for money? Overall view on whether the programme was good value for money

Quality of Financial Management during programme Consider our best estimate of future costs against the current approved budget and forecasting profile Have narrative and financial reporting requirements been adhered to. Include details of last report Have auditing requirements been met. Include details of last report

E: Risk

Quality of risk management over the life of the programme.

How were risks managed, the degree to which they were realised and/or mitigated?

Did any risks materialised through the duration of the programme which were not foreseen? What where they and how was this managed?

What lessons have been learned?

F: Commercial Considerations

Delivery against planned timeframe. Y/N

Compare actual progress against the approved timescales in the Business Case. If timescales are off track provide an explanation including what this means for the cost of the programme and any remedial action.

Performance of partnership How well are formal partnerships/ contracts working Are we learning and applying lessons from partner experience Could BEIS be a more effective partner

Asset disposal and value obtained by BEIS

How were assets managed throughout the programme? How have they been (or will they be) disposed to get maximum value?

G: Conditionality

Partnership principles assessment

For programmes where we have decided to use the PPs for management and monitoring, PCRs should generally include an assessment of commitment to the PPs, including any concerns that have occurred over the year, the partner government’s response, and details of any response by us. Teams should refer to the BEIS guidance on reviewing projects.

H: Monitoring and Evaluation

Evidence and evaluation Changes in evidence and implications for the programme Where an evaluation is planned what progress has been made How is the Theory of Change and the assumptions used in the programme design working out in practice in this programme? Are modifications to the programme design required? Is there any new evidence available which challenges the programme design or rationale? How does the evidence from the implementation of this programme contribute to the wider evidence base? How is evidence disaggregated by sex and age, and by other variables?

Where an evaluation is planned set out what progress has been made.

Monitoring process throughout the programme Direct feedback you have had from stakeholders, including beneficiaries Monitoring activities throughout review period (field visits, reviews, engagement etc) The Annual Review and PCR process

Page 24: Template Project Completion Review

Project Completion Review – Supporting Information

Summary sheets from Colombian Sustainable Cattle Ranching Project annual reviews (2012 – 2019)

Page 25: Template Project Completion Review

Type of Review: Annual Review

Project Title: Silvopastoral systems for climate change mitigation and poverty alleviation in Colombia´s livestock sector. Review date:

Project Location: Colombia, South America

Project Timescale: 2012-2018

Current Reporting Period: December 2012-June 2014

Funding: (ICF Funding and possibly other sources)

International Climate Fund and Global Environment Facility

Project website (if available): www.fedegan.org.co/programas/ganaderia-colombiana-sostenible

Project leader’s name: Jane Ellaway

Review Summary

What are the key messages from this Review?

The project has been marked B overall Components 1 and 2 have been marked B, but some components have achieved above this, as noted below. The project has slipped by a year The project has incurred delays of a year, moving the project end date to January 2018. The delay has occurred in the design phase of the Additional Financing (AF) elements and the redesign of some aspects of the parent programme. In general, the Additional Financing elements have experienced the most delay this year. Despite the problems, the delay has allowed for a better quality product to be developed and for planning to take place (for example, conducting thorough environmental and social impact assessments, and building in time to test new payment for environmental services (PES) approaches before wider roll out). Difficulty disaggregating Additional Finance results It is difficult to separate out the results of the parent programme from the ICF’s Additional Finance as some of the ICF funds are going to extending or expanding the original project. There are some discrete elements such as the deforestation hotspots and the PES 2 farms which are easier to disaggregate and we will be consider how to do this as part of the next annual review. DECC will also look again at the attribution of project benefits. Some elements have proceeded despite delays Flexibility and resourcefulness have allowed the training of assessors (component 4) and the production of vegetal material (component 2.3) to proceed using a retroactive financing clause in the Administration Arrangement, despite delays to the wider project. This will allow the project to make up ground once the other elements are in place by initiating critical activities that are season-dependent without having to wait on for the Grant Agreement to be signed. Components 3 and 4 have been marked A in this annual review. Component 3 (“Strengthening Sub-sector Institutions, Dissemination and Monitoring & Evaluation”) is proving effective and early indications are that the project could deliver the anticipated benefits of dissemination and replication. The long term impacts will be contingent on the project itself being successful, which is not yet known.

Page 26: Template Project Completion Review

Component 4 (“Project management would support the overall Project coordination.”) is also proceeding well. The retroactive financing provision enabled the recruitment of the experts required for the Additional Financing to go ahead as scheduled. Project Wide Lessons

1. More pro-active and regular engagement from the ICF is required to enable the regular reporting to be conducted faster and more efficiently and to ensure DECC can participate more effectively in conversations about the project.

2. The project reporting would benefit from greater transparency from the World Bank, including more reflection and analysis of what has worked well and what has worked less well so it is clearer where issues have arisen and that lessons have been learnt.

3. Methodologies to disaggregate project results between GEF and ICF will need to be developed so that DECC can report on the ICF’s results.

4. Future reports will need to be clearer on what progress is being made on delivering the Additional Finance elements specifically. The Additional Finance is being used to extend some existing elements of the programme and the report should detail progress on using the Additional Finance to do this, distinct from continuing activities that were already being undertaken.

Legend on scoring

Description Scale

Outputs substantially exceeded expectation A++

Outputs moderately exceeded expectation A+

Outputs met expectation A

Outputs moderately did not meet expectation B

Outputs substantially did not meet expectation C

Page 27: Template Project Completion Review

Annual Review - Summary Sheet

Title: Proyecto Ganadería Colombiana Sostenible – Sustainable Cattle Ranching Project, Colombia

Programme Value: £15m Review Date: 22nd May 2015

Programme Code:

Start Date: June 2014 End Date: May 2015

Summary of Programme Performance

Year 2013-14 2014-15

Programme Score B A

Risk Rating

Summary of progress and lessons learnt since last review Building the foundation to deliver the project is progressing well. For example, the number of trainers trained has exceeded the target for this year, the production of the vegetal material has met need and some of the payments for environmental services have now occurred. This puts the project on a good footing to roll out all the elements over time and it is expected to have made good progress on preserving endangered areas. In contrast, the delivery of the land use changes such as rolling out SPS and iSPS is moving more slowly and the target number of hectares to be delivered this year will not be met by the June deadline. Fedegan believes it may be met by the end of the calendar year, but we expect the more likely scenario is that progress will be made, but the target will be missed. There has been no request to reduce the targets as Fedegan remains optimistic that they will eventually meet the project final target, now that initial and complex prerequisite activities have been done. Overall the project has been scored A – outputs meet expectation. While there are some significant indicators lagging behind such as land area converted, significant progress has been made this year against many other indicators and we anticipate this will put the project in a good place to rapidly implement changes on the ground. Lessons learnt take two forms: 1. Lessons about the delivery of this project and the impacts it will have on this project’s outputs 2. Lessons that could inform future plans to expand the roll out of sustainable cattle ranching

techniques beyond the life or scope of this project. 1. Lessons about the delivery of this project and the impacts it will have on this project’s

outputs a. Consideration needs to be given to the cost implications of the time overrun and what impact that

might have on the eventual deliverables of the project. For example, the vegetal material that has already been ordered cannot be planted out when the rainy season fails to materialise. Therefore costs may increase where trees need to be kept in the nursery for longer than expected. This is somewhat off-set by the higher likelihood of more mature material surviving once it has been planted out in comparison to younger plants. Material is only ordered by Fedegan when requested by farmers so there should be no additional cost for ordering new material over a more protracted period. The cost of retaining trained advisors however may scale linearly for each additional year they are needed to support farmers during a more protracted delivery period.

b. The cost of baselining and monitoring land use is currently prohibitively high in comparison to the payments made to farmers. Fedegan has taken a more pragmatic approach of self-declaration by the farmers, backed by checks by the technical advisers, but will be looking for other options too.

c. There is some initial evidence of interest by neighbours of the project farms following the on-site visits of the technical agents. This suggests the assumption that the project farms would have a

Page 28: Template Project Completion Review

demonstration effect could be correct, although it is too early to say whether this will lead to change in the neighbouring farms.

2. Lessons that could inform future plans to expand the roll out of sustainable cattle ranching

techniques beyond the life or scope of this project. a. Implementation of the project has been difficult and progress has not been as fast as expected. The

benefits of SPS are great, but so too are the challenges and these need to be addressed in any plans for nationwide roll-out.

b. The pilot has some characteristics which are suitable for a pilot project aiming to gather evidence about the use of Payment for Environmental Services (PES) and Silvopastoral Systems (SPS), but that would be prohibitively costly and time consuming for a national programme. Simplification will be essential to reduce the cost of implementation and the time it takes in future projects. This could include the land classification system which currently includes nine different land categories and the land use monitoring which is still too expensive for extensive use.

c. The primary difficulty the project has experienced when trying to sign up interested farmers is farmers’ lack of formal land rights and land history documentation. Creating systems to formalise land rights will be essential to ensuring all farmers can participate in a national programme and that it does not exacerbate land rights issues or divert resources to illicit activities.

Summary of recommendations for the next year

1. Consider reviewing the assumptions about project costs and project milestones to ensure targets are still realistic – The World Bank will be undertaking a review next year to carry this out.

2. Ensure everything is in place to report a full set of results in March 2016 – The World Bank M&E lead will be responsible for this.

3. DECC needs to engage with the WB’s in-depth review scheduled for next year to ensure we get the outputs we are interested in, including the cost-implications of reforming delivery versus the status quo

4. The March 2015 results collection included the following points which need to be followed up before the next results collection in March 2016. ‘A’ and ‘B’ are joint DECC/WB actions, ‘C’ is a WB action:

a. Monitoring activities under KPI-6 and KPI-8 have not yet been initiated and will formally start with the signing of a contract with TNC. We need to determine when we will be able to report on these.

b. “Attribution factors will be discussed during the next supervision planned in May 2015.” There was not time to cover this during the mission so we will need to pick this up in the next few months. The attribution may not be the same for all KPIs. For example, much of the public finance was brought in under GEF’s funding and so attributes to them, but the CO2 savings are more evenly split.

c. We are still lacking milestones for the KPI as these will be developed following the base lining study currently being completed.

Page 29: Template Project Completion Review

Annual Review - Summary Sheet

Title: Proyecto Ganadería Colombiana Sostenible – Sustainable Cattle Ranching Project, Colombia

Programme Value: £15m Review Date: 4 August 2016

Start Date: June 2015 End Date: June 2016

Summary of Programme Performance

Year 2013-14 2014-15 2015-16

Programme Score B A B

Risk Rating High Medium High

Summary of progress and lessons learnt since last review Overall the project has been scored B – outputs moderately did not meet expectation. The project is designed to gather evidence on whether introducing silvopastoral systems (SPS) can help reduce the deforestation caused by cattle ranching and alleviate poverty through the increase of milk and beef production, and the provision of alternative income through forestry products and payments for environmental services. The DECC32 project manager joined the World Bank’s Mid-Term Review mission in June 2016 to verify and gather evidence on progress so far; triangulate evidence; take the views of other stakeholders, including the Colombian Government which manages the other major donor funding from the Global Environment Facility; to assess lessons learned; and to agree an action plan for the remainder of the project life to January 2018. While the project is delivering some good and interesting results on the ground and the project’s concept remains highly relevant to the future development of Colombia’s agricultural sector and plans for forest protection, the project is at a critical moment in its life. A number of factors, within and without the project’s control, have contributed to slower than expected progress in implementation including:

• fragmented, complex administrative and decision-making processes between the institutions involved;

• the lack of an adequate information management system;

• slow recruitment of the project team and subsequent high turnover of staff;

• El Niño effects in 2014 which delayed planting of trees in some areas;

• the low investment capacity of small scale farmers, hampering the chances of achieving wide-scale adoption, especially in intensive silvopastoral systems (iSPS); and

• the wide geographical spread of properties leading to inefficiencies in the provision of technical assistance and project management.

While progress has been made since the last annual review, particularly the revision of the PES2 (payment for environmental services) scheme into a working incentive, it appears the strength of the project’s foundation measures may have been overestimated at the last review and some significant risks to overall delivery remain. The capacity of the project team remains understrength and unless working arrangements between the project partners improve markedly, then the project is unlikely to catch up, or even make adequate progress here on in, in contracting farmers, converting land to SPS and making payments. If the verification of land change uses, contracting and processing payments are not made more efficient then this will continue to undermine the reputation of the project and threaten the replicability of the project’s interventions. BEIS will continue closely monitoring progress. This has resulted in a downward trend in the World Bank’s evaluations of the project from a “Satisfactory” assessment in March 2015, to “Moderately Satisfactory” in December 2015, and to a rating of “Moderately Unsatisfactory”, in the recent June 2016 Mid-Term Review mission.

32 In July 2016, the Department of Energy and Climate Change (DECC) was merged with elements of the Department for Business, Innovation and Skills to form the new Department for Business, Energy and Industrial Strategy (BEIS). In this report, references to actions before the change will refer to DECC, future recommendations will refer to BEIS.

Page 30: Template Project Completion Review

The project administrators Federación Colombiana de Ganaderos (FEDEGAN) (the Colombian cattle ranchers federation) and the other institutions involved, need to accelerate delivery in order to achieve the project’s proposed development objectives. The World Bank has drawn up detailed action and milestone plans in agreement with the project team in order to quickly address the issues identified in the latest review mission. This includes revised contracting and payment processes to reduce minimum processing times from six to two months in each case and the urgent recruitment of contractors to develop an all-encompassing information management system for the project, beginning with a payments module. The mission also made several agreements with, and recommendations to, the project team including that:

• the project team be strengthened;

• the role of field technician be divided into different specialist roles;

• farmers be compensated for previous late payments;

• field staff be given access to information on the progress of payments to farmers in their area;

• field staff be better remunerated;

• resources currently allocated to Global Environment Facility (GEF) funded contracts (whose amount was overstated) be freed up to spend on other priorities;

• the project team raise its targets for recruiting technicians trained in silvopastoral systems (SPS) and that it work with the national technical training agency, Servicio Nacional de Aprendizaje (SENA), to ensure a supply of qualified professionals in this field ;

• twenty further demonstration farms are set up; and

• the project team quickly recruit new farmers to replace those lost to the project. These interventions should give the project team the capacity to significantly accelerate the conversion of land to SPS. A fully effective project team delivering the project’s interventions at scale, combined with a fully functioning monitoring and evaluation framework, which must now be urgently finalised, will allow BEIS to build on lessons learned so far and take a more considered view of the value for money and effectiveness of the project’s interventions and their potential for transformative change in climate-friendly agriculture. A further assessment of value for money and strategic merit will be undertaken following the World Bank’s next mission in November, to consider a one year extension to the project to allow time for more results to be delivered. However, should the changes agreed in the milestones and action plans not be carried through, then an extension should not be countenanced. Summary of recommendations for the next year

1. BEIS should continue to engage closely with the World Bank team over the coming months to monitor the project’s progress against the agreed milestones and action plans and in particular, to check progress on streamlining the contracting and payment processes and reducing the current backlog. This will be crucial for putting the project back on track.

2. The World Bank must ensure that the remaining elements of the monitoring and evaluation framework are in place as soon as possible and that are no gaps in the 2017 results collection.

3. An early assessment should be made by the project team of the effect of the reduced expectations around land conversion to iSPS as part of the assessment of the project’s overall contribution to reduced GHG emissions and avoided hectares of deforestation.

4. Should the project meet the requirements of the agreed milestones and action plans over the next six months, BEIS should review the case for a one year extension to the project and possible additional supervisory funding (this decision will be taken within the context or other portfolio priorities and will be based on ongoing performance).

5. The project implementation team should ensure that lessons learned on the effectiveness of the various interventions made are disseminated as part of the project’s communications strategy.

BEIS and British Embassy Bogota should continue to monitor and encourage Colombian Government engagement with the project to ensure lessons from the project are informing wider policy development,

particularly in any future bids for financial support through forest funds, the NAMA facility or Green Climate Fund.

Page 31: Template Project Completion Review

Annual Review - Summary Sheet

Title: Proyecto Ganadería Colombiana Sostenible – Sustainable Cattle Ranching Project, Colombia

Programme Value: £15m Review Date: September 2017

Start Date: June 2016 End Date: June 2017

Summary of Programme Performance

Year 2013-14 2014-15 2015-16 2016-17

Programme Score B A B A

Risk Rating High Medium High High

Summary of progress and lessons learnt since last review Overall the project has been scored an A: outputs met expectations. The project is designed to gather evidence on whether introducing silvopastoral agroforestry systems (SPS) can help reduce the deforestation caused by cattle ranching and alleviate poverty amongst small scale ranchers through the increase of milk and beef production, and the provision of alternative income through payments for providing ecosystem services such as preserving biodiversity and managing land in a way which sequesters carbon. SPS systems allow ranchers to sustainably intensify production on existing land, reducing the need for farmers to expand onto new land and thereby reducing pressure on adjacent forests. Deforestation is a major cause of greenhouse gas emissions in Colombia and across much of Latin America, and the intention is that the learning from this project could pave the way for a much wider transition to sustainable land management across Colombia and comparable regions. The BEIS project manager joined the World Bank’s mission in April 2017 to verify and gather evidence on progress so far; take the views of other stakeholders, including the Colombian Government which manages the other major donor funding from the Global Environment Facility; to assess lessons learned; and to assess the potential for an extension to the project. This annual review is based on this and other missions over the course of the last year and on the results reported in the June 2017 technical report from the lead implementing agency, FEDEGAN. The project’s original closing date is January 2018: noting this, this review has assessed and scored outputs by looking at the likelihood of meeting the end of project targets, based on the actual implementation progress at June 2017 and the trajectory of progress over the last year. The project team have made good progress over the past year and this assessment has shown that the project is currently delivering good value for money. The last annual review noted that the project was ‘at a critical moment in its life’ and a number of detailed recommendations were made, both through the annual review and through the project’s mid-term review. These recommendations have been taken forward and as a consequence the rate of implementation has improved. This has been accompanied by an increase in the rate at which project funds have been disbursed, from 35% (October 2016) to 62% (June 2017). The project team should be commended for the improvements seen over the past year, but it should also be noted that the project will still not be in a position to deliver against all of the revised output targets by the current project closing date. This largely reflects delays to implementation incurred in previous years which are detailed in previous reviews. The last annual review recommended that if the project team made sufficient progress over the next year and implemented the recommendations put forward in the mid-term review then BEIS should consider extending the programme to allow the project to meet its targets in full. The World Bank submitted a formal extension request to BEIS in July 2017. This review has found that sufficient progress has been made for this extension to be considered: we are therefore conducting a parallel but separate assessment of the case for extending the programme, which will be published separately to this review. A key driver for originally investing in the SPS programme was the value the programme has as a pilot and demonstration project. It was designed to test different approaches for driving land conversion to SPS and to gather further evidence regarding the contribution SPS can have on reducing greenhouse gas emissions and alleviating poverty amongst small scale cattle ranchers. It was also designed to build institutional capacity for implementing SPS in Colombia. This review has found that implementation has

Page 32: Template Project Completion Review

improved significantly over the last year and that as a result the programme would be in a good position to deliver this important work if extended. This would allow the project to realise its full value as a pilot, ultimately significantly increasing the likelihood that the lessons learnt from this programme are used to pave the way for a much greater transition to sustainable livestock ranching across Colombia and Latin America. Summary of recommendations for the next year 1. BEIS and the World Bank team should work with the project’s implementing agencies to ensure that

they recognise the value of the project’s work to disseminate learning, and to ensure that they are effectively recording activity undertaken in relation to this work. The project’s implementing team should also consider maximising the learning which can be drawn from this project by producing a specific report on land conversion to iSPS, describing the lessons that have been learnt and explaining how these could be applied to similar future interventions.

2. As part of the project’s end of programme assessment the project team and the World Bank should

produce a joint report setting out their experiences relating to the design and implementation of PES schemes. This should set out specific recommendations in PES design for the Colombian Government. We recommend that the team engage the Colombian Agricultural ministry with this work, and that consideration be given to producing these reports earlier if it becomes apparent that this would be beneficial to other initiatives seeking to replicate this approach (enabling the learning from this programme to more effectively drive wider transformational change).

3. BEIS should assess the World Bank’s extension request by October 2017, ensuring thorough due

diligence to ensure an extension represents good value for money.

4. The project implementation team should recruit a new project co-ordinator as soon as possible.

5. If BEIS approve an extension to the project, the World Bank and the project team should conduct an updated assessment of the resources required to fully deliver the outputs expected in the extension phase. This could form part of the risk assessment referenced in section E.

6. The World Bank, the project team and the Colombian Government should consider how the project’s

interventions can be used to deliver results in regional and national results-based payments programmes such as the BioCarbon Fund Initiative for Sustainable Forest Landscapes.

7. BEIS and the British Embassy in Bogota should consider how they can encourage greater

cooperation between the various initiatives which BEIS and the ICF fund in Colombia. 8. BEIS / British Embassy Bogota should continue to engage with the Colombian Ministry of Agriculture

as regards its intentions to pursue further sustainable cattle intensification programmes. 9. If the project is extended, an updated risk assessment should be conducted by the World Bank

identifying any further risks to implementation and to the potential of the project to deliver the intended long-term outcomes and impacts. Appropriate mitigation plans should be put in place.

Page 33: Template Project Completion Review

Annual Review - Summary Sheet

Title: Proyecto Ganadería Colombiana Sostenible – Sustainable Cattle Ranching Project, Colombia

Programme Value: £15,258,478 Review Date: September 2018

Start Date: June 2017 End Date: June 2018

Summary of Programme Performance

Year 2013-14 2014-15 2015-16 2016-17 2017-18

Programme Score B A B A A

Risk Rating High Medium High High Medium

Summary of progress and lessons learnt since last review Overall the project has been scored an A: outputs met expectations. The project is designed to promote a transition towards sustainable cattle ranching in Colombia by gathering evidence on whether introducing silvopastoral agroforestry systems (SPS), which combine tree planting with cattle ranching, can help reduce the deforestation caused by cattle ranching and alleviate poverty amongst small-medium scale ranchers. SPS systems allow ranchers to sustainably intensify production on existing land, reducing the need for farmers to expand onto new land and clear forests. Deforestation is a major cause of greenhouse gas emissions in Colombia and across much of Latin America, and this pilot project could pave the way for a much wider transition to sustainability in this sector. Last year’s annual review found that implementation of this project was rapidly accelerating but that delivery of outputs lagged behind targets in absolute terms. This year’s review has found that the recommendations made last year have been taken forward and that the acceleration in progress has now translated into good results ‘on the ground’, with the project’s outputs in-line with our expectations at this point. The project has now implemented silvopastoral systems on almost 27,000ha of land, with 80,000ha of land under more sustainable management practices, split across 3873 farms. As a result, this year’s review has found that the programme is currently delivering good value for money. The project has now been running for long enough to gather data about the impacts it is having in terms of avoiding deforestation, improving biodiversity and improving the livelihoods of participating ranchers, many of whom are very poor. Although full analysis of this data will not take place until the project’s independent impact evaluation in 2019, emerging results taken from a sample of 101 participating farms show that they have benefited from an average increase in the production of meat (23%), and milk (36%), whilst the cost of production has fallen by 19%. As these emerging results have become available, the project team have stepped up their work to disseminate these findings and learning, increasing the probability that this pilot programme will contribute to a wider transition towards sustainability in Colombia’s cattle ranching sector. This review explores progress over the period June 2017 to June 2018 in detail and provides a number of discrete recommendations to help ensure the full potential of this project is realized in the remaining period of implementation. Summary of recommendations for the next year

• The project needs to continue prioritizing technical assistance support to highly motivated farmers that show a high interest in undertaking land change transformations towards SPS, and downscale support to those ranchers that, despite dedicated TA support, are undertaking land changes at a very slow pace.

• It is important for the project to establish a mechanism to report the activities that are implemented by the project partners around the promotion of SPS systems outside the direct scope of the GCS project, as these activities represent important indirect outcomes of the project.

• The project has accumulated significant experience in (i) the provision of planting materials through in-situ farm production and commercial nurseries; and (ii) land conversion monitoring mechanisms. It

Page 34: Template Project Completion Review

is important to speed-up the preparation of learning notes on these two topics to be disseminated to external audiences. This recommendation was made during the supervision missions, but it has not yet been implemented by the project team.

• The project needs to continue strengthening the links between on-farm biodiversity monitoring (carried out by CIPAV) and landscape biodiversity monitoring (carried out by TNC).

• FEDEGAN needs to continue strengthening monitoring mechanisms to ensure that all the activities (and associated results) funded or co-funded by the project and implemented through the project partners are properly recorded and disseminated through the project’s communication media outlets.

• It is crucial that the project’s partners ensure the timely delivery of commitments linked to the preparation of the livestock NAMA. The project is leading the preparation of this strategic document (together with WRI), under the guidance and oversight of a NAMA committee led by MADR and MADS. Fedegan should appoint a member of the project team to assume responsibility for delivering this important task, which may require recruiting. This person should be appointed by 20th November 2018.

• BEIS and the World Bank should work together to ensure that high-quality monitoring and reporting for KPI 8 continues going forward.

• As this project draws to a close BEIS, the World Bank and the project team should seek to convene a multi-stakeholder discussion on the future of sustainable cattle ranching in Colombia.

• Over the past year the World Bank and the project’s implementing partners have made good progress towards scaling up work to communicate and disseminate the project’s emerging results. To maximize the impact of this project it is essential that this work continues; particular emphasis should be placed on engaging parties that could play a role in taking SPS to scale.

Page 35: Template Project Completion Review

Annual Review - Summary Sheet

Summary of Programme Performance Year 2013-14 2014-15 2015-16 2016-17 2017-18 2018-19 Programme Score B A B A A A+ Risk Rating High Medium High High Medium Moderate

Summary of progress and lessons learnt since last review

Overall, the programme has been scored an A+: outputs moderately exceeded expectations. Compared to end of programme targets, the programme is benefitting more farmers than expected and supporting a transition to sustainable production systems on a greater area of land. Results confirm the climate and biodiversity benefits of the approaches, whilst indicative results from a sample of farms also suggest that the programme could be enhancing farmer incomes to a greater extent than expected in the Business Case. Overall, this indicates that the programme’s approach is technically effective, feasible to implement and attractive to beneficiaries.

The programme is designed to promote a transition towards sustainable cattle ranching in Colombia. It pilots silvopastoral agroforestry systems (SPS), which combine tree planting with cattle ranching, on Colombian farms in order to gather evidence on whether the introduction of these approaches can help reduce the greenhouse gas (GHG) emissions generated by cattle ranching and reduce poverty amongst small-medium sized cattle ranchers. SPS allow ranchers to intensify production sustainably, on their existing farmland, thereby reducing the need for farmers to clear forest and expand onto new land. Deforestation is a major cause of GHG emissions in Colombia (and much of Latin America). This pilot programme is trialling an approach that can contribute to a wider transition to sustainability in the agriculture, forests and land use sector.

This year’s review finds that the positive implementation trends noted in the previous annual review have continued to translate into strong results ‘on the ground’ that are either meeting or exceeding expectations. The programme has so far enabled 4100 farms to adopt silvopastoral systems on more than 34,000ha of land. This has exceeded expectations, with one hundred more farms now benefitting than was expected at programme end and 97% of the end of programme land area target for SPS having already been achieved. In total, the programme is now contributing to more sustainable land management on over 96,000 ha of land, 15% more than was expected for programme end. As a result, this year’s review determines that the programme is continuing to deliver good value for money.

Full analysis of the programme’s impact will come in the independent impact evaluation due in April 2020. Indicative results on a sample of 101 farms indicate that SPS mitigates climate change, improves biodiversity and can increase farmer incomes by as much as $523 per hectare (ha)/per year. These indicative results confirm the programme’s theory of change and exceed expectations for income enhancement.

Over the past year, the programme’s implementation team have taken considerable efforts to disseminate the positive findings from the programme to a wide audience of ranchers, policy makers and journalists. The programme received considerable publicity in a leading Colombian business magazine and on BBC radio. The programme’s results have been noted favourable by senior political leaders in Colombia, and dedicated targets for sustainable cattle ranching – based on techniques pioneered by the programme – now feature in Colombia’s 2018-2022 National Development Plan. The extent to which the programme’s outreach activities may have contributed to the increased political attention on sustainable cattle ranching will likely be assessed as part of an upcoming Climate Change Compass evaluation.

This year also saw delivery partners look towards the programme’s legacy and the future of sustainable cattle ranching in Colombia. The programme commissioned important analysis into the ‘business case’ for SPS in Colombia and the potential for SPS to be scaled-up from this pilot programme.

Title: Proyecto Ganadería Colombiana Sostenible – Sustainable Cattle Ranching Project, Colombia

Programme Value (full life): £15,258,478

Review Date: February 2020

Review Period: July 2018 - June 2019

Programme Code: Programme Start Date: December 2012

Programme End Date: January 2020

Page 36: Template Project Completion Review

This review assesses the programme’s progress over the period July 2018 to June 2019 and makes recommendations to assist the programme maximise potential results before the implementation period concludes in January 2020. This will be the final annual review for the project – progress in June 2019-January 2020 will be captured in the project closure review (PCR).

Summary of progress in implementing recommendations from last year’s annual review

Recommendation Status

Prioritize technical assistance to farmers that demonstrate high interest in implementing SPS.

Achieved. Technical assistance through the fourth call prioritised municipalities with high rates of farmer interest, whilst technical assistance outside of this call prioritised farmers demonstrating high interest from previous calls.

Establish a mechanism to report activities undertaken by programme partners to promote SPS but are implemented outside the direct scope of the programme.

Rolled over. Information requests have been sent to executing partners, and the World Bank and Fedegan* expect to consolidate the information in time for the project completion report in April 2020.

Speed up preparation of learning notes on: i) production of planting materials from in-situ farm production and commercial nurseries, ii) land conversion monitoring mechanisms.

Rolled over. The implementation team is committed to preparing these learning notes. They are to be prepared for dissemination at the technical closing event scheduled for late April/early May 2020.

Continue strengthening the links between on-farm biodiversity monitoring (carried out by CIPAV*) and landscape biodiversity monitoring (carried out by TNC*).

Partially achieved. Integrated monitoring approaches have been established. CIPAV and TNC will finalise a series of biodiversity publications prior to programme closure.

Fedegan should continue strengthening monitoring mechanisms to ensure that all activities (and associated results) funded or co-funded by programme are properly recorded and disseminated.

Partially achieved. The World Bank and Fedegan will establish a repository of documents by programme close. In 2018/19, key publications were disseminated through regional events that gathered over 600 ranchers, as well as the programme’s website and social media channels. Further dissemination events will be organised prior to programme closure, including the National Livestock Congress in November 2019.

Ensure timely delivery of commitments linked to the preparation of Colombia’s Nationally Appropriate Mitigation Action (NAMA) for the livestock sector

Partially achieved. Despite sustained engagement and leadership of the preparation process, disagreement regarding the scope have delayed overall the overall livestock NAMA preparation process. The full livestock NAMA document is scheduled to be completed/published in April 2020, yet certain components will be concluded by programme closure including: emissions baseline, emission reduction scenarios, prioritization of mitigation options,

BEIS and the World Bank should work together to ensure that high-quality monitoring and reporting for ICF Key Performance Indicator 8 (KPI 8) continues going forward.

Partially achieved. The methodology for KPI 8 was anticipated to have been agreed in the past year, but debate regarding the trade-offs associated with different approaches for establishing a baseline caused delays. TNC have now finalised a proposal for the KPI 8 methodology, ready for agreement in the immediate future.

As this project draws to a close BEIS, the World Bank and the implementation team should seek to convene a multi-stakeholder discussion on the future of sustainable cattle ranching in Colombia.

Rolled over. The programme has engaged in several multi-stakeholder discussions on sustainable cattle ranching via its support for the preparation of the livestock NAMA and through the Roundtable on Sustainable livestock. Discussions have not progressed as much as envisioned. Technical discussions with relevant partners in Colombia are scheduled around the last round of dissemination events in Spring 2020.

Work to disseminate the programme’s results should continue, with particular emphasis placed on engaging parties that could play a role in taking SPS to scale.

Achieved. Good use of communication channels and demonstration farms have been made in order to showcase the benefits of SPS practices to ranchers, journalists and policymakers. Notably there was a 60-page feature in one of Colombia’s most prominent business magazines.

* Note on abbreviations: Fedegan - Federación Colombiana de Ganaderos (Colombian cattle ranchers association); CIPAV - Fundación Centro para la Investigación en Sistemas Sostenibles de Producción Agropecuaria; TNC - The Nature Conservancy.

Page 37: Template Project Completion Review

Summary of recommendations for the next year

• BEIS and the World Bank should work together to ensure that the methodology for reporting KPI 8 (avoided deforestation/degradation) is agreed by the end of October 2019, to enable high-quality reporting by the time of the project completion review. Once the methodology is finalised, planned spatial analysis should be undertaken by spring 2020 to enable assessment of whether leakage effects are occurring from programme interventions.

• Findings from analysis into the business case and business models for SPS, and findings from analysis into the links between SPS and climate resilience should be disseminated by the World Bank team to key stakeholders in government, the livestock sector and impact investors in early 2020.

• Until programme closure, the implementation team should continue its efforts to communicate programme results in order to maximise impact and reach. Communication activities should be tailored to target audiences and focus on communicating the programme’s lessons and multiple benefits to government stakeholders responsible for livestock sector planning.

• As this programme draws to a close in early 2020, the World Bank and implementation teams should:

o identify how existing information assets (e.g. website) will be maintained, and create a publicly accessible repository to enable sharing of the studies and tools created by the programme with external parties interested in adopting SPS or replicating the approaches pioneered by the programme.

o Identify a means for collecting farmers’ knowledge of local ecological dynamics to assist with species selection in future SPS initiatives.

• In collaboration with organisations in the Colombian livestock sector, the implementation team should identify ways to institutionalise the links between SPS-trained ranchers, and the links between demonstration and regular farms, within local/online networks that can continue to offer peer-to-peer support for SPS implementation following programme closure. By April 2020, the implementation team should have developed a set of actions detailing how these local knowledge networks will be maintained and advanced by local stakeholders. The positive legacy of knowledge embedded in upskilled farmers and demonstration farms should also feature strongly in all remaining communication activities and products relating to the programme.

• Once published in 2020, the World Bank should circulate the NAMA document to key stakeholders in the Colombian livestock sector, and disseminate lessons from the experience of its creation to other governments in the region because very few sectoral NAMAs have been created in this collaborative fashion.

• Once the programme closes, the World Bank team should ensure that the programme’s lessons, impacts and independent evaluation are disseminated to a wide audience by June 2020. Targeted stakeholders should include the World Bank’s agriculture, natural resources and climates practices; government stakeholders and companies within the beef and dairy values chains in Latin America; the Food and Agriculture Organisation of the United Nations; other donor countries; and policy/practitioner networks related to REDD+33 and forest and landscape restoration.

• BEIS, the World Bank and the implementation team should explore the value and feasibility of further UK support for SPS in Colombia in order to scale the lessons and technique piloted by this programme and support the Government of Colombia’s ambition to roll out SPS nationally. By May 2020, the World Bank team and executing partners should finalise a proposal for a future phase of support for SPS in Colombia.

33 REDD+: Reducing Emissions from Deforestation and forest Degradation, plus the sustainable management of forests, and the conservation and enhancement of forest carbon stocks.