Analytical Overview of Joint UN Gender Programme Portfolio · 3.6 Funding 31 3.7 Summary 32 4...
Transcript of Analytical Overview of Joint UN Gender Programme Portfolio · 3.6 Funding 31 3.7 Summary 32 4...
1
AnalyticalOverviewofJointUNGenderProgrammePortfolio
FinalReport
EvaluationOffice
2011
2
AnalyticalOverviewTeam:ThisstudywasmanagedbytheUNWomenEvaluationOffice.TheteamwasledbyCeciliaMLjungman,anindependentevaluationspecialist,withresearchsupportfromFlorenciaTateossianandcoordinatedbyIsabelSuárezfromtheUNWomenEvaluationOffice.
Disclaimer:Theviewsexpressedinthispublicationarethoseoftheauthor(s)anddonotnecessarilyrepresenttheviewsofUNWomen,theUnitedNationsoranyofitsaffiliatedorganization.
3
Acronyms 5ExecutiveSummary 71Introduction 13
1.1Approach 131.1.1QuantitativeData 131.1.2QualitativeInformation 14
1.2Limitations 151.3DefiningtheJPGPortfolio 161.4StructureoftheReport 17
2BackgroundtoandRationalefortheJGPEvaluation 182.1BriefBackgroundtoJointProgrammesasPartofUNReform 182.2EffectivenessofUNDevelopmentEffortofGEWE 192.3RationalefortheJGPEvaluation 20
2.3.1AjointandRights‐BasedEvaluation 213QuantitativeDeskAnalysisoftheJGPPortfolio 22
3.1EvolutionofJGPPortfoliooverTime 233.1.12001‐2005 243.1.22006‐2010 24
3.2Partners 253.2.1NumberofParticipatingUNPartners 253.2.2LeadAgency 27
3.3Geography 283.4ThematicArea 293.5PlannedTimeframe 303.6Funding 313.7Summary 32
4FindingsfromQualitativeDeskAnalysis 344.1SourcesofQualitativeInformation 34
4.1.1JGPEvaluations/ReviewstoDate 344.1.2OtherRelevantJPDocuments 354.1.3Stakeholders 36
4.2IssuesandInformationNeeds 364.2.1Qualityof“Jointness” 364.2.2JGPDesign 404.2.3EffectivenessinTermsofResults 414.2.4Sustainability,NationalLevelPartnerships,NationalOwnership&People‐CentredApproaches 434.2.5EfficiencyandOperationalEffectiveness 45
5EmergingPriorities&ReflectionsfortheFutureTermsofReference 475.1StrategicPriorities 47
5.1.1ResultsandAddedValueofJGPs 475.1.2Sustainability,NationalLevelPartnerships,OwnershipandPeople‐CentredApproaches 485.1.3Synergies 49
5.2Evaluability 495.2.1AssessingResults 505.2.2Assessing“Jointness” 50
Contents
4
5.2.3IntegratingHumanRights 505.2.4DataandTimeLapse 51
5.3AMixedMethodEvaluation 525.3.1DocumentationExamination 525.3.2CaseStudyApproach 525.3.3SurveyPossibilities 545.3.4TeamExpertise 55
Annexes Annex1:TermsofReference Annex2:JGPPortfolioTable Annex3:ListofJGPswithMissingProgrammeDocuments Annex4:ListofStakeholdersConsulted Annex5:ListofDocumentsConsulted Annex6:UNDGDefinitionsforFundManagementModalities
Contents
5
CEDAW ConventionontheEliminationofDiscriminationAgainstWomen
CEE‐CIS Central&EastEuropeandCommonwealthofIndependentStatesregion
CCA CommonCountryAssessments
CSO Civilsocietyorganisation
CRC ConventionontheRightsoftheChild
DAC DevelopmentAssistanceCommittee
DAO DeliveringasOne
DFID UKDepartmentforInternationalDevelopment
EC EuropeanCommission
ECLAC EconomicCommissionforLatinAmericaandtheCaribbean
ECSOC UnitedNationsEconomicandSocialCouncil
EVAW Eliminatingviolenceagainstwomen
FAO FoodandAgriculturalOrganization
GEWE Genderequality/Women’sEmpowerment
GRB Genderresponsivebudgeting
IFAD InternationalFundforAgriculturalDevelopment
IOM InternationalOrganizationforMigration
ILO InternationalLabourOrganization
JGPs JointGenderProgrammes
JPCs JointProgrammeCoordinators
LAC LatinAmericanandCaribbeanregion
MDGs
MDGFund
MillenniumDevelopmentGoals
MillenniumDevelopmentGoalAchievementFund
M&E Monitoringandevaluation
MDTF Multi‐donortrustfund
NGO Non‐governmentalorganisation
NORAD NorwegianAgencyforDevelopmentCooperation
NSGE‐DV NationalStrategyforGenderEqualityandDomesticViolence
OCHA OfficefortheCoordinationofHumanitarianAffairs
OSCE OrganizationforCooperationandSecurityinEurope
RBA RightsBasedApproach
RCOffice ResidentCoordinatorOffice
SIDA SwedishInternationalDevelopmentCooperationAgency
SMART Specific,measurable,attainable,realisticandtimely
Acronyms
6
TCPR TriennialComprehensivePolicyReview
UNAIDS JointUnitedNationsProgrammeonHIV/AIDS
UNCDF UnitedNationsCapitalDevelopmentFund
UNCT UnitedNationsCountryTeams
UNDAF UnitedNationsDevelopmentAssistanceFramework
UNDG UnitedNationsDevelopmentGroup
UNDG/DOCO UnitedNationsDevelopmentOperationsCoordinationOffice
UNDP UnitedNationsDevelopmentProgramme
UNEG UnitedNationsEvaluationGroup
UNESCO UnitedNationsEducational,ScientificandCulturalOrganization
UNFPA UnitedNationsPopulationFund
UN‐Habitat UnitedNationsHumanSettlementsProgramme
UNHCR OfficeoftheUNHighCommissionerforRefugees
UNICEF UnitedNationsChildren’sFund
UNIFEM UnitedNationsDevelopmentFundforWomen
UNFTEVAW UnitedNationsTrustFundinSupportofActionstoEliminateViolenceAgainstWomen
UNWomen UnitedNationsEntityforGenderEqualityandtheEmpowermentofWomen
WHO WorldHealthOrganization
WHR Women’shumanrights
Acronyms
7
ExecutiveSummary
STUDYBACKGROUNDANDAPPROACH
Thisreportisapre‐studytosupportthescopingprocessforthefutureEvaluationofJointGenderProgrammes(JPGs)intheUNsystem,whichwillevaluatetheUN’sjointprogrammesintheareaofgenderequality,women’srightsandwomen’sempowerment.ItprovidesthedraftersofthetermsofreferenceforthefutureJGPsevaluationwithqualitativeandquantitativeanalyticaloverviewoftheJGPsportfolioandreflectionsontheemergingstrategicpriorities,evaluabilityandmethodologicaloptions.AccordingtoUNDG’sGuidanceNoteonJointProgrammes(2003),ajointprogrammeinvolvestwoormoreUNorganisationsand(sub‐)nationalpartnersthathavejointlysignedaprogrammedocument.JGPshavebeendefinedasthosethathaveanexplicitobjectiveof:empoweringwomen;and/orpromotinggenderequalityatthestrategiclevel;and/orwomenand/orgirlsmayconstitutethemainbeneficiaries/programmepartners.Thus,jointprogrammesthatmaymainstreamequalitybetweenmenandwomen(whichintheoryarealljointprogrammes)buthaveotheroverallgoals,havenotbeenincludedintheportfolio.Theauthorshaveestablishedadatabasethatmapsoutthecharacteristicsof113JGPsfrom2001to2010accordingtoninemaincharacteristics.ThedataforJGPsthatwereinitiatedbefore2006areincomplete,butthedatafortheJGPsinitiatedbetween2006and2010generallyhavehighreliability.Themostsignificantchallengeforthestudywasobtainingreliabledatasincenoneoftheagencydatabaseshavesystemstoallowforstraightforwardsearchesofjointprogrammes.Theyfrequentlylackconsolidatedandsystematicinformation,containerrorsandarenotregularlyupdated.WhiletheteamhasusedinnovativesearchtacticsandanextensivenumberofhourstryingtoidentifyJGPs,itispossiblethatthedatabasedoesnotcontainall
existingJGPs.TheJGPdatabaseisfurthermorelimitedtoinformationprovidedbythesignedprogrammedocuments–thusanychangessincethesigningofprogrammedocumentsmaynothavebeencapturedbythedatabase.Thegatheringofqualitativedatafocusedonobtaininganoverviewofkeyissuesandinformationneeds.Thedatawereacquiredthroughconsultationsandinterviewswithover30keystakeholdersandtheanalysisof20evaluations/reviewsthatwerelocated.Whilesomeusefulinformationhasbeengleanedfromthese,ithasbeenrelativelylimitedsincethelevelofqualityisofteninconsistentandmostareweakonthejointaspectofjointprogramming.Toprovideananalyticaloverviewofthepolicyenvironmentthatunderpinstherationaleforthefutureevaluation,areviewofdozensofrelevantpolicydocuments,reportsandstrategieswasalsoconducted.KEYFINDINGS
Findingsfromthequantitativedeskanalysis:CharacteristicsoftheJGPsportfolioInthebeginningofthedecade,atmost,acoupleofJGPswereinitiatedeachyear.Thebudgetswerealsomodest,withamediansizeofUS$320,000.ThesecondpartofthedecadesawariseinthemedianbudgetedprogrammesizetoUS$2million,withadramaticriseinthetotalnumberofJGPsin2008and2009.ThiscanpartlybeexplainedbytheadditionofMDGFundresources.However,thelargestaveragesizeofJGPswasin2010(US$7million),whichwasaftertheMDGFundcontributionsweredistributed.From2006to2010,thetotalplannedvalueoftheJGPportfoliowasUS$463millionandthetotalfundedvalueatthetimeofsigningoftheprogrammesdocumentswasUS$274million.Twenty‐fourdifferentUNentitieshaveparticipatedinatleastoneJGP,withUNFPA,UNDP,formerUNIFEMandUNICEFparticipatinginover60JGPs
8
ExecutiveSummary
each.UNDP,UNFPAandformerUNIFEMwerealsobyfarthemostprevalentintheroleofleadagency.ThespecializedagenciesWHO,ILO,UNESCOandFAOarethesecondmostfrequentparticipants.ThemajorityofJGPsaremadeupofthreetofourparticipatingUNagencies,whileone‐thirdofJGPshavefiveormoreparticipatingUNagencies–somehaveover11.AfricahasthegreatestnumberofJGPsandaccountsforthelargestportion(55%)ofthetotalplannedfinancialvalueoftheJGPportfoliofrom2006to2010.TheAsia/PacificandtheLACregionsaccountfor14%eachofthetotalplannedfinancialvalueoftheJGPportfolio,butinLACtheindividualJGPsaremuchsmallerinsize.Multi‐sectoralJGPsarefew,buttheyhavelargebudgetsthataccountfor33%oftheaggregatedplannedfinancialvalueoftheJGPportfolio.IntermsofnumberofJGPs,theeliminatingviolenceagainstwomen(EVAW)thematicareaisthelargest–roughlyaccountingforjustlessthanone‐thirdofallJGPsandone‐thirdoftheaggregatedplannedfinancialvalueoftheentireJGPportfolio.JGPsinthegovernanceareaarealmostasnumerousasEVAWJGPs.However,theyhavemuchsmallerbudgetsthatamounttoonly13%oftheaggregatedplannedfinancialvalueoftheJGPportfolio–whichissimilartothevalueofthehealth(13%)andeconomicempowerment(9%)JGPs.Thenumberandvalueoftheeducation,traffickingandHIV/AIDSJGPsrepresentonlyafewpercenteachofthetotalbudget.OnlyfiveJGPsrepresentingfourthematicareashaveobjectiveswithaconflict‐relatedangle.CorefundsfromtheparticipatingUNagenciesarethemostimportantsourceoffunds–benefitting62%to72%oftheJGPsfrom2006to2010.TheaggregatedcorefundingfromUNagenciesisthelargestsourceoffundingoverall(overUS$98million).TheMDGFundisthelargestnon‐coresourceoffunding(aroundUS$90million).Othertrustfundsprovidefundingtoatleast16%oftheJGPsfrom2006to2010.Bilateral,OCED‐DACcountrycontributionstoJGPsattheinceptionoftheprogrammehavebeenmadebyAustralia,Austria,Belgium,Canada,Denmark,Ireland,Italy,Germany,theNetherlands,Norway,Spain,Sweden,
SwitzerlandandtheUK.Othernationalgovernmentshavealsoprovidedresources–financialorin‐kind–foratleast13JGPs.
Findingsfromthequalitativedeskanalysis:ConvergenceonstrategicprioritiesThisstudyhasdemonstratedthatthereisconsiderableconcurrenceamongstakeholders,evaluations/reviewsandpolicydocumentsregardingtheoverallprioritiesfortheevaluation.First,thereisacommonperspectiveontheuseoftheevaluation.WhileitwillbeusedtorenderjudgmentabouttheoverallmeritorworthofJGPs,theprincipleuseswillbetofacilitateimprovementsandgenerateknowledge.Theseusesshouldguidethescopeandapproachoftheevaluation.Second,theanalysisrevealsthattheprioritiesfortheevaluation’sstrategicscopeconvergeonthreeareas.Inrelationtotheseareas,thedatasuggestthateffectiveness,sustainabilityandpossibleimpactarethedominantevaluationcriteriatoassesstheJGPs.Relevanceissuesarelessprominentbutstillpertinent.EfficiencyandoperationaleffectivenessissuesweregenerallyconsideredlessimportantforlearningfromandimprovingJGPs.Whilethedatarevealsmanychallengesinthisarea,stakeholdersallagreedthatthesewerenotuniquetoJGPs,butcommontomanyormostjointprogrammes.Itwasaconcernthatthisevaluationmaintainsitsfocusontheeffectivenessofjointprogrammesandnotbetakenoverbysystemicoperationalefficiencyissuesthatrelatetoalljointprogrammesinallsectors.ThefirstareaofconvergencerelatestowhetherJGPsareeffectiveinproducingresultsandhow/whethercollaboratingtogetheraddsvaluetotheseresults.ThecallforinquiryintothisareacomesfromacombinationofUNpolicydirectives,UNWomen’smandate,thelackofevaluativeevidenceandseveraltypesofinformationneedsthatstakeholdersconsultedbythisstudyhaveexpressedthefollowing:• ThereareseveralGeneralAssemblyresolutions
thaturgetheUNdevelopmentsystemtoenhanceaccountabilityintheareaofgender
9
ExecutiveSummary
equalityandwomen’sempowerment.
• PolicydirectiveshaverecommendedthattheUNdevelopimprovedguidanceonthenature,qualityandeffectivenessofjointprogrammesinsupportofgenderequalityandtheempowermentofwomen.
• ThereisarelativepaucityofstrategiclevelassessmentsofspecificUNeffortstoaddresswomen’sempowerment,women’srightsandgenderequality.
• ThereisalackofevaluativeevidencerelatingtoJGPs.
• WithitsmandatebeingtoleadandcoordinatetheoveralleffortsoftheUNsystemtosupportthefullrealizationofwomen’srightsandopportunities(bypromotingcoherenceandactingasaglobalbrokerofknowledgeandexperience),UNWomenneedsevidencetoinformitspolicydevelopment.
• StakeholdersconsultedbythisstudyexpressastrongneedforinformationonthedegreeandnatureofcollaborationamongstparticipatingUNpartnersinJGPsandhowjointprogrammesaddvaluetogenderequality/women’sempowermentresults.
• StakeholdersprioritisetheanalysisofJGPdesignanddesignprocessesbecause(i)stakeholdersconsiderJGPdesignanddesignprocessesasdeterminantsofsuccessfulresultsand(ii)theJGPevaluations/reviewshaveidentifiedseveralJGPdesignproblems.
• Stakeholdersdesireinformationoneffectivenessinrelationtoseveraltypesofeffectsinclude:(i)genderequality,women’sempowermentandhumanrightsresults;(ii)capacitydevelopmentamongduty‐bearersandrights‐holders;(iii)advocacyeffects;(iv)processresultsfromanright‐basedapproachperspective;(v)intangibleeffects;(vi)synergeticeffectsand(vii)goodpractices.
ThisareaofconvergencecouldbecapturedbyanoverallaimofassessingtowhatextentandinwhatwayscollaboratinginaJGPhasenhancedthegenderequality/women’sempowermenteffectsachievedbytheparticipatingUNagenciesandtheirpartners.Thiswouldplaceeffectivenessintermsof
genderequality/women’sempowermentresultsinthecentreoftheevaluationandwouldlinkitwiththeconceptofcollaborationor“jointness”.Itwouldrequirethestudyofthenatureanddegreeofjointness(indesignprocesses,implementationprocesses,governance,resourcemobilisation,communication,knowledgemanagement)andwhatkindofcollaborationcontributedtobetterresults.TheevaluationwouldneedtoexaminethestrengthsandweaknessesofJGPsinrelationtoproducingeffects;howtheUNcouldimproveJGPssothattheyaremoreeffectiveinproducingresultsandwhethertherearegoodpracticestolearnfrom.ThesecondareawhereprioritiesconvergerelatestosustainabilityandhowtheJGPsinteractwithandsupportstakeholdersatthecountrylevel.Itinvolvesnationalownership,people‐centredapproachesandUNpartnershipswithgovernment.ThereareanumberofpolicyleveldocumentsthatareconcernedwiththeseaspectsandthequestionistowhatextentJGPsaresuccessfulataddressingthem.Discussionswithstakeholdersandanalysisoftheevaluations/reviewsalsorevealsimilarconcerns:• UNDG’sGuidanceNoteonJointProgramming
statesthatjointprogrammesaimtoenhancetheUNcontributioninthecurrentcontextofinternationaldevelopmentassistance,withafocusonself‐relianceandcapacitybuilding.
• TheAidEffectivenessprinciplesoftheParisDeclarationandtheAccraAgendaforActionplaceownershipatthecentreofdevelopmentco‐operation.
• TheHighLevelPanelonUNSystem‐wideCoherence’sreportDeliveringasOnestatedthatreformtoimprovethecoherenceoftheUNdevelopmentsystemmustbeunderpinnedbytheprinciplesofnationalownershipandpeople‐centredapproaches.
• UN’scommitmenttomainstreaminghumanrightsinallofitsdevelopmentworkrequiresanapproachofstrengtheningtheaccountabilityofduty‐bearersandsupportingrights‐holdersindemandingtheirrights.
• TheUNDGGuidanceNoteonJointProgrammesstatesthatjointprogrammesarespecificallyintendedtostrengthenhowtheUN
10
ExecutiveSummary
organizationsprogrammejointlywithgovernments.
• StakeholdersrequiremoreanalysisofwhatJGPsmeanforownershipandjointprocessesinthepartnershipsthattheagenciesenjoywithgovernmentsandcivilsociety.
• Theevaluations/reviewsofJGPsconcludedthatsustainabilityofJGPswaslowwhilestakeholdersrequireinformationandanalysisofhowsustainabilitycanbeimproved.
ThisareaofconvergencecouldbecapturedbyanoverallaimofassessingtowhatextentandinwhatwaysJGPshavecontributedtogovernmentsmeetingtheircommitmentstotheBeijingPlatformforActionandfulfilledtheirobligationstowardswomen’sandgirl’shumanrights;whilealsosupportingrights‐holdersdemandtheirrights.ThiswouldsettheUN’snationallevelpartnershipswithduty‐bearersandrights‐holdersattheheartoftheevaluation.Itwouldcovertheissuesofnationalownership,howeffectivelyandsustainablytheUNagenciesprogrammejointlywithgovernmentsandtheextenttowhichJGPapproachesarepeople‐centred.ThethirdandmuchsmallerareaofconvergencerelatestosynergiesbetweenJGPsandotherUNefforts:• ReformoftheUNdevelopmentsystemto
promoteeffectivenessandsustainabilityfocusesoncoherence,coordinationandcollaboration–notonlywithinprogrammesbutalsoamongprogrammes.
• ManystakeholdersshowaveryhighdegreeofinterestinunderstandingwhetherandhowaJGPinacountrycanresultinsynergeticeffectswithotherUNprogrammesatcountrylevel.TheywanttoknowwhetherJGPshaveaninfluenceontheUN’soverallgenderequalitymainstreamingefforts.
• MainstreaminggenderequalityintoallUNprogrammespresentssignificantchallengesformostagencies.JGPsareseenbymanystakeholdersasresourcesformainstreaming.
• MostoftheJGPevaluations/reviewsdonotreportonsynergiesandthefewthatdidfound
thereweremissedopportunitiestocreatethem.
ThisareaofconvergencecouldbecapturedbyanoverallaimofassessingtowhatextentandinwhatwaysJGPshavecontributedtoimprovedgenderequalitymainstreamingandwomen’sempowermentinotherUNprogrammesandeffortsatcountrylevel.ThefocusherewouldbeonsynergeticeffectswithotherUNefforts.ItwouldrequirestudyingtowhatextentJGPsaffectedincreasedcollaboration,coordinationandinformationexchangewithintheUNCTinrelationtogenderequalityandwomen’sempowerment.
EMERGINGPRIORITIES&REFLECTIONSFORTHEFUTUREEVALUATIONTERMSOFREFERENCE
Thestudyhasidentifiedevaluability,methodologicalandscopingissuestobetakenintoconsiderationduringthedraftingofthetermsofreferenceforthefutureEvaluationofJGPsintheUNsystem.Tobeginwith,programmesaddressinggenderequalityandwomen’sempowermentareinherentlydifficulttoevaluatesincetheyconcernchallengingandchangingcomplexsocietalnormsanddynamics.BecauseofthecomplexityandfluidityofdevelopmentprocessesandthefactthatJGPsusuallyhavesuboptimallogframes,indicatorsandmonitoringsystems,assessingeffectivenessanddeterminingcausalityinthecaseofintermediateoutcomeswillbechallengingfortheevaluators.Thereisadesiretoobtainknowledgeofarangeofeffects–synergeticeffects,intangibleeffectsandeffectsrelatedtocapacitydevelopment,humanrightsandempowerment.Identifying,analysingandassessingthesedifferenteffectswillrequireanumberofdifferenttechniquesandapproaches.
Basedonthefindingsofthequalitativeportfolioanalysis,thereportmakesanumberofrecommendationswiththeaimofhelpingtosupportimplementthefutureevaluationofJGPs:
11
ExecutiveSummary
Recommendation1:ThetermsofreferenceshouldstipulatethattheevaluationteamisrequiredtodemonstratehowitwillassessthedifferenttypesofJGPseffectsinitsmethodology.
Recommendation2:Thetermsofreferenceshouldrequestthattheevaluationteamconsiderwaystoassesstheaddedvalueof“jointness”initsmethodology.Evaluatingtowhatextent“jointness”enhancesresults,wouldideallyrequirethatcontrolprogrammesbeidentifiedsothatpairsofgenderequality/women’sempowermentprogrammes–onejointandtheother“single”–becompared.Unfortunately,itisunlikelythattwocomparableprogrammescanbefoundinthesamecountry,beingimplementedatthesametimeinthesamethematicarea.However,bydrawingontheknowledgeandexperienceofUNstaffandpartnerorganisations,itwouldbepossibletoreconstructhowasingleprogrammemighthavebeendifferentfromajointprogramme.Participatorytechniques–suchascollectiveanalysiscouldbeusefulinthisregard.
Recommendation3:Thetermsofreferenceshouldrequestthattheevaluationteampresenthowitwillintegratehumanrightsinitsmethodology.Itappearsthatasignificantproportionoftheportfoliomayhavemediumorhighevaluabilityforintegratinghumanrightsandgenderequality.Toaddresstheevaluabilitychallengesintermsofintegratinghumanrights,ahumanrights‐basedstakeholderanalysiswillbecritical.Aglobalevaluationspanning60countrieswillentaillimitationstotheamountofstakeholderparticipationthatispracticallypossible.Itwill,however,bepossibletoensureanacceptablelevelofparticipationinthecountrycasestudies.
Recommendation4:TheevaluationtimescopeshouldincludeJGPsfrom2006to2010toensuredatareliabilityandusefulness.Thedatareliabilityfortheprogrammesfrom2001to2006islowsincemorethanhalflackprogrammedocuments.MostoftheearlierJGPsaresmall‐scaleandthuslesslikelytoproduceeffectsthatareidentifiableyearslater.Institutionalmemoryforthisperiodmayalsobelow.Ontheotherhand,theauthorshaveobtainedprogrammedocumentsfornearly90%fortheJGPsthatwereinitiatedbetween2006and2010.TheJGPsthatwereinitiatedinthelatterhalfofthedecadearealsomorerelevanttolearnfromsincetheywereconceptualisedandimplementedinthecontextofanumberofinstitutionalchangesaffectingjointprogrammes(greaterharmonisationofoperationalpractices;furtheralignmentofUNDAFstonationalprocesses,thepilotingoftheDeliveringasOneinitiativeandthecreationoftheMDGFund).
Recommendation5:Theagenciesshouldensurethatthecountryofficessupplythenecessaryprogrammereportsanddatatocompletethedatabaseanddocumentrepository.Exceptfor20evaluations/reviews,theJGPdatabasecurrentlydoesnotcontainreportsrelatedtotheJGPs.Aconsiderableeffortwouldbeneededtoensurethatcountryofficessupplythenecessaryreportsanddatatocompletethedatabase.
Recommendation6:TheevaluationshouldincludeadeskreviewofthewholeJGPportfolioandanin‐depthportfolioanalysisofasizeableproportionoftheJGPportfolio.TheJGPdatabasethathasbeenestablishedbythestudyandtheanalysisthathasalreadybeenundertakenconstitutesignificantresourcesfor
12
ExecutiveSummary
theevaluationteam.Theseexistingproductswillallowtheteamto“hitthegroundrunning.”Nevertheless,whilethedatabasecontainsarangeofdatathatcanallowfurtheranalysisandcomparisons,tobefullyutilised,thedatabasewouldneedtobeupdatedtoincludeup‐to‐datefundinginformation.Itwouldbeimportantfortheevaluationteamtobeginbyreviewingall94programmedocumentsandthe20availableevaluations.Forbothpracticalandresourcereasons,itwouldmakesensetoundertakeamorein‐depthdeskstudyofaroundone‐quartertoone‐thirdoftheJGPs.TheevaluationteamwouldneedtoprovidecriteriaintheinceptionphaseonhowtoselecttheseJGPs.SomeofJGPsforwhichtherehavebeenreviews–suchastheAlbanianJGPsandsomeoftheMDGFundprogrammes–wouldconstitutegoodcandidatesforfurtherdeskstudy.
Recommendation7:Theevaluationshouldinclude4to6casestudiesthatinvolvecountryvisits.Theevaluationshouldalsoincludedesk‐levelcasestudies.Allcasestudiesshouldbecarefullychosenbypurposefulsamplingtakingintotoconsiderationthesamplingcriteriaprovidedbythisreport.ToobtainadeepenedunderstandingofwhatresultsJGPsareachievingandwhethercollaborationamongagenciesiscontributingtothis,itwillbecriticalthattheevaluationundertakescasestudies.Visitingfourcountriesislikelytoprovidethedatarequired,especiallyif(i)countrieswithmorethanoneJGPareprioritisedand,(ii)fieldstudiesofJGPsarecomplementedwithin‐depthdeskstudiesofotherJGPs.UNconcernstoensureregionalrepresentationforpoliticalreasonsmayrequirethatsixcasestudiesbeundertaken.Thecasestudieswouldmostappropriatelybeselectedbypurposefulsamplingtoensurethatanumberofvariablesarecoveredandthatthe
casesare"informationrich"andilluminative.WhilethesamplingshouldbebiasedtowardsJGPsthatareconsideredbystakeholderstobeinnovative,havedevelopedgoodpracticesand/oraresuccessful,thesamplingshouldalsoconsiderJGPsthathavestruggledtoproduceresults.Theprioritycriteriatoconsiderforsamplingincludeamixofdifferentthemes,agencies,arangeofnumbersofparticipatingagencies,countrieswithdifferenthumandevelopmentandgenderequalityindexes,regions,budgetsizesandlevelsofprogrammematurity.Conflict‐relatedJGPsshouldalsobeincludedinthesample.
Recommendation8:Thetermsofreferenceshouldcallforateamwithstrongskillsandin‐depthknowledgeandexperienceintherangeofrelevantareaslistedabove.Planningshouldtakeintoconsiderationthelead‐timethatbusyhighqualityconsultantsmayrequire.Importantqualitiesofthefutureevaluationteamincludeknowledgeandexperiencein:
Genderequality,women’sempowermentandwomen’srightsmovement
Developmentco‐operationprocessesandpolicies
TheUNdevelopmentsystem,theUN
reformprocessandUNdevelopmentprogrammes
Rights‐basedapproaches
Evaluationmethods,participatoryapproachesanddatacollection
Developingcountries,conflict‐affected
countries,regionalandcross‐regionalexperience
Assessingcapacitydevelopment
Theteamwillneedstronganalytical,writingandfacilitationskillsandarangeoflanguageskills.Theteamshouldrepresentdiversityandconsistofbothwomenandmen.Nationalconsultantsshouldbeincludedonthecasestudymissions.
13
Introduction
Thisreportisapre‐studyforthefutureevaluationofJointGenderProgrammes(JGPs)intheUNsystem,whichwillevaluatetheUN’sjointprogrammes(JPs)intheareaofGenderEquality,Women’sRightsandWomen’sEmpowerment(GEWE).Tosupportthescopingprocess,thisreportprovidesthedraftersofthetermsofreferenceforthefutureJPEevaluationwiththefollowing:
1. Ananalyticaloverviewofthepolicyenvironmentthatunderpinstherationaleforthefutureevaluation;
2. AnanalyticaloverviewoftheJGPportfolio–includingaquantitativeanalysisandananalyticaloverviewofkeyissues,concernsandinformationneedsthatareassociatedwithJGPs;
3. Reflectionsontheemergingstrategicpriorities,evaluabilityandmethodologicaloptions.
Toundertakethisstudy,twootherproductshavebeenproducedbytheteam.Theyare:
1. Arepositoryofrelevantdocumentation–programmedocuments,evaluations,reviews,policydocumentsandtools–thatrelatetotheJGPportfolio;
2. AdatabaseoutliningthecharacteristicsoftheJGPsaccordingtoninemaincategories.
Theremainderofthischapterprovidesinformationonthemethodologyused,itslimitationsandthedefinitionsappliedbythestudy.Thefinalsectionincludesanoverviewofthereport’sstructure.
APPROACH
QuantitativeDataIn2010,theEvaluationUnit(EU)oftheformerUNIFEMbegancollectinginformationonJGPsfrom
2001to2009inwhichUNIFEMwasaparticipatingagency.TheEUreliedoninformationgatheredfromtheUNIFEMonlineAnnualReporttrackingsystem,theUNDGdatabase,theMDG‐Fund(MDG‐F)database,theUNDG/DOCOdatabaseofResidentCoordinators(RCs)AnnualReportsanddirectfollowupandfeedbackfromfieldstaff.TheformerUNIFEMEUdevelopedafirstscanandadatabasewithalltheinformationgatheredandestablishedapreliminaryrepositoryofprogrammedocumentsandevaluationreports.InFebruary2011thecurrentstudyteamexpandedthesearchtotakeaccountofJGPswhichUNIFEMdidnotparticipateinandincludedallJGPsfrom2010.Thisconsistedofsearchinganumberofdatabases–theUNDG/DOCOdatabaseofRCsAnnualReports,theMDG‐Fdatabase,UNDGdatabase,UNFPA’sdatabaseofJGPsfrom2009‐2010,UNDP’sAtlasExecutiveSnapshot,theMulti‐DonorTrustFunddatabaseandcountryofficewebsites.WidersearchesontheInternetwerealsoundertaken.ThesearchesontheInternetandofthevariousdatabaseshelpedtheteamtouncoverpotentialJGPs.However,sincemuchoftheinformationwasproventobeunreliableorpartiallyincorrect(pleaseseesection0),itwasimportanttotriangulatedataandgivehighprioritytolocatingsignedprogrammedocuments.Contactwasmadewiththegenderexpertsand/orevaluationofficesofUNDP,UNICEF,UNESCO,UNFPA,WHO,FAOandUNESCOtoverifyandconfirmdataandobtainprogrammedocuments(whichseveralagencieswerehelpfulindoing).Contactwasalsomadeinsomecaseswithcountryofficestoverifyandconfirmdataandobtainprogrammedocuments.Allprogrammedocumentswerefiledelectronicallyaccordingtoregion.Atotalof113JGPswereeventuallyidentified.ThesewereenteredintoamoreelaboratedversionoftheinitialdatabasethatwasestablishedbytheEUin2010.Theupdateddatabasemapsoutthecharacteristicsof113JGPsfrom2001to2010
14
Introduction
accordingtoninemaincharacteristics:
1. Budget:ThedatabasecontainsboththeplannedbudgetofeachJGPandthefunded
budgetatthetimetheprogrammedocumentwassigned.Sincejustover50percentoftheJGPswerefullyfundedfrom
thestart,thesefiguresarethesameformanyJGPs.Thedifferencebetweenthesetwofigures(thefundinggap)differsfrom
JGPtoJGPbutisonaverage28percent.Itisimportanttonotethatunlesstheprogrammeisfullyfunded,bothfiguresare
indicative.TheteamhasnotcollecteddataonthecurrentfundingsituationoftheJGPs.
2. Country/region:TheregionsusedareLatin
AmericaandtheCaribbean,Sub‐SaharanAfrica,CentralandEasternEuropeandCommonwealthIndependentStates,Asia
andthePacificandtheArabStates.3. Theme:Thethematicareaswerederived
fromtheSecretaryGeneral’s2010reporton
theimplementationofBeijingPlatformforAction.TheyareEliminationofViolence
againstWomen(EVAW),Education,Health,Trafficking,EconomicEmpowerment,Governance,HIV/AIDSand“Integrated.”
ThelatterreferstoJGPsthathavemorethanonemainthematicareaofwork.
4. Timeframe:Thestartdate,enddateand
timeframeasstatedintheprogrammedocumenthavebeenentered.However,preparationtimeandtimeextensionshave
not.5. UNPartners:ApartfromUNfunds,
programmesandspecializedagencies,UN
missions(forinstancetoHaiti),regionaleconomiccommissions,RCOffices(whentheyhavecontributedfunding)andcertain
Secretariatoffices(OCHA)havebeenincluded.
6. OtherPartners:Thedatabasehascategories
formultilateralpartners(e.g.WorldBank,
InternationalOrganizationforMigration,regionaldevelopmentbanks,etc.);bilateralpartnersandnationalpartners.
7. LeadAgency:Thishasbeenenteredwhenagenciesmentionaleadagencyintheprogrammedocument.
8. FundManagementModality:Theseareparallel,pass‐through,pooledorcombinationofanyofthethreeformer–as
pertheUNDGGuidanceNoteonJointProgrammesfrom2003.
9. FundingSources:Fundingsourcesinclude
recipientgovernments;bilateraldonors;differenttypesoftrustfundsandcorefundingfromtheUNagenciesthemselves.
Theamountsprovidedbythedifferentsourceshavenotbeenentered.
ThedataforJGPsthatwereinitiatedbefore2006isincomplete–theteamhasnotbeenabletolocateprogrammedocumentsforoverhalfofthe19JGPsidentifiedfromthisperiod.RegardingtheJGPsinitiatedbetween2006and2010,thedataismuchmorereliable–only10%oftheJGPsinthedatabasefromthisperiodaremissingprogrammedocuments.ForJGPswithoutprogrammedocuments,somedatafromotherlessreliablesourcesofinformation(e.g.draftprogrammedocumentsandinformationfromdatabases)havebeententativelyenteredintothedatabase.ThelistofprogrammesforwhichtheEvaluationUnitisseekingtheprogrammedocumentsisincludedinAnnex3.
QualitativeInformationThegatheringofqualitativedatafocusedonobtaininganoverviewofkeyissuesandinformationneeds.Thedatawasacquiredthroughconsultationsandinterviewswithkeystakeholdersandareviewofdocumentation.Interviewswereheldwithover20peopleworkingatheadquarterslevelthatwereeithergenderexperts,evaluationspecialistsorengagedinonewayoranotherinJPs.Interviewsand/ordiscussionswere
15
Introduction
heldwithstaffrepresentingUNDP,UNICEF,UNFPA,UNWomen,WHO,MDTF,MDG‐F,UNDG/DOCOandtheUNTrustFundtoSupporttheEliminationofViolenceAgainstWomen.Furthermore,theteamconsultedwith11field‐basedstaffincludingResidentCo‐ordinators,UNWomenCountryRepresentativesandJointProgrammeCoordinators(JPCs).JointlytheyrepresentedAsia,Africa,ArabStates,EasternEuropeandLatinAmerica.Thedocumentationreviewwasrelativelycomprehensive.Ataminimumallprogrammedocumentsinthedatabasewereskimmedthrough–somewerestudiedinmoredetail.DozensofrelevantGeneralAssemblyresolutions,ECOSOCResolutions,reportsoftheSecretaryGeneral,reportstoexecutiveboards,MinisterialDeclarationsandtheTriennialComprehensivePolicyReview(TCPR)havebeenreviewed.EvaluationsandreviewshavebeensearchedforintheevaluationdatabasesofUNDP,UNEG,UNWomenandUNFPA.Inaddition,Internetsearchesforevaluationshavebeenundertaken.Whileonly20evaluations/reviewsthatspecificallycoveredJGPswerelocated,theteamalsostudiedsignificantevaluations/assessmentsregardinggendermainstreamingandGEWE,JPsandUNreform.Thedraftfindingsofthisstudywerepresentedanddiscussedinthreedifferentfora–i)theUNDGTaskTeamonGenderEquality;ii)representativesfromthematicandgeographicalsectionsofUNWomen;andiii)genderequalityand/orevaluationspecialistsUNDP,UNICEFandMDG‐FundaswellasrepresentativesfromSpanishAgencyforInternationalDevelopmentCo‐operationandtheGovernmentofNorway.
LIMITATIONS
Themostsignificantchallengeforthestudywasobtainingreliabledata.NoneoftheagencydatabaseshavesystemstoallowforastraightforwardsearchofJPs–letaloneJGPs.Theavailabledatabasesfrequentlylackconsolidatedandsystematicinformation,containerrorsandarenotupdatedonregularbasis.Thepoorandinconsistent
datarequireddiligentverificationandtriangulation.WhiletheteamhasusedinnovativesearchtacticsandspendhundredsofhourstryingtoidentifyJGPs,itispossiblethatthedatabasedoesnotcontainallJGPs.
Second,theJGPdatabaseislimitedtoinformationprovidedbythesignedprogrammedocuments.Sincetheprogrammesoftendevelopaftersigning–theymayhaveamassedmorefunds,enteredintopartnershipwithnewdonors,thetimeframeorfundmanagementmodalitymayhavechanged.Suchchangeswillnothavebeencapturedbythedatabase.Third,theJGPdatabasehasnotbeenfullyvalidatedbythecountryoffices.Whilesomeofficeshavebeencontactedindividuallyforinformation,giventheveryearlystageofthisevaluationprocess,itwasfeltthatitwouldbeprematuretoofficiallyenlisttheResidentCo‐ordinatorsystem.ThismeansitispossiblethattheremaybeafewJPsthattheteamhasnotyetfound.Fourth,consultationsheldwithstakeholderswererelativelylimitedandrepresentedjustUNstakeholders.Itwasbeyondthescopeofthestudytointerviewnationallevelpartners.Nevertheless,theinformationgatheredfromthestakeholdersconverges,providingarelativelyclearpictureofinformationneeds.
16
Introduction
Fifth,only20evaluations/reviewshavebeenundertakenofJGPs.Whilesomeusefulinformationhasbeengleanedfromthese,ithasbeenrelativelylimited.Thevastmajorityareactuallyreviewsormid‐termevaluations.Thequalityisofteninconsistentandmostareweakonthejointaspectofjointprogramming.
DefiningtheJGPPortfolio“JointGenderProgrammes”(JGPs)aretheintersectionofJPsandprogrammesthatpromotegenderequalityand/orwomen’sempowerment.WhatqualifiesasaJPandGEWEprogrammeisdiscussedinthefollowingsections.
JointProgramme
AccordingtoUNDG’sGuidanceNoteonJointProgrammes(2003),aJPinvolvestwoormoreUNorganisationsand(sub‐)nationalpartners.Theobjectives,strategy,workplanandrelatedbudgetformpartofaJPdocument,whichwillalsodetailrolesandresponsibilitiesofpartnersincoordinatingandmanagingthejointactivities.TheJPdocumentissignedbyallparticipatingorganisationsand(sub‐)nationalpartners.Acoordinationmechanismor“JointProgrammeSteeringCommittee”ultimatelygovernsaJP.ItincludesseniorpersonnelofallsignatoriestotheJPdocument,eachwithasimilarlevelofdecision‐makingauthority.ThefundmanagementmodalityoptionsforJPsarepooled,pass‐through,paralleloracombinationoftwooralloftheseoptions.Thus,aprogrammeinwhichoneagencydoesnotsigntheprogrammedocumentbutactsasasub‐contractor(“implementingagent”)foranother,doesnotrepresentaJP.Norarejointevents–suchasconferencesorcampaigns–recognisedasJPsinthisstudy.WhiletheUNDGdefinitionforJPsmakesitrelativelysimpletodistinguishwhetheraprogrammeisjoint
byanalysingtheprogrammedocument,howjointaprogrammeisinpracticecanonlybedeterminedbyanalysingeachJPmoreclosely.Thisisdiscussedfurtherinsection4.2.1.GenderEquality/Women’sEmpowerment(GEWE)Programmes
Topromotegenderequality,alljointUNprogrammesintheorymainstreamequalitybetweenmenandwomen.Becauseoftheabilitytoproducewide‐scaleresultsformenandwomeninawholerangeofdifferentsectors,effectivemainstreamingofgenderequalityinallUNeffortsundoubtedlyhasimmensepotentialinfulfillingtherightsofbothwomenandmen,enhancingtheirwellbeingandincreasingprosperity.However,thepersistinginequalitiesbetweenmenandwomenandboysandgirlscreateanunlevelplayingfieldthatunderminestherightsofwomenandgirls.Therefore,inadditiontothemainstreamingofgenderequality,thereisacompellingneed–asrecognisednotleastbyMDG3–toundertakeprogrammeswiththespecificobjectiveofempoweringwomenandgirls,promotingwomen’srightsandestablishinglegislative,policyandinstitutionalframeworksforgenderequality.Thus,JGPshavebeendefinedasthosethathaveanexplicitobjectiveofempoweringwomenand/orpromotinggenderequalityatthestrategicleveland/orwomenand/orgirlsmayconstitutethemainbeneficiaries/programmepartners.Thus,JPsthatmaymainstreamequalitybetweenmenandwomen(whichistheoryareallJPs)buthaveotheroverallgoals,havenotbeenincludedintheportfolio.ThethematicareasthattheJGPportfoliofallsintoincludeeducation,economicempowerment,governance,health,violenceagainstwomen,traffickingandHIV/AIDS.1Inaddition,someJGPsaddressmorethanonethematicareaandaredefinedas“multi‐sectoral”.Thethematicareasarebroadandsomecouldbefurtherbrokendowninto
1Thenamesofthesethematicareasarebasedonthe2010ReportoftheSecretaryGeneralentitled“TheReviewoftheImplementationoftheBeijing
PlatformforAction,theoutcomesofthetwenty‐thirdspecialsessionoftheGeneralAssemblyanditscontributiontoshapingagenderperspective
towardsthefullrealisationoftheMillenniumDevelopmentGoals”.
17
Introduction
sub‐themes.Therearealsofiveprogrammesthathaveaconflictangleintheirobjectives.However,thematicallytheyareagoodfitwiththecategoriesusedandhavethereforenotbeengroupedintoaseparatecategory.Todeterminewhichthemestheprogrammefallinto,theteamhasexaminedtheoverallobjectivesoftheprogrammes.Thus,ifthereareJPsthathavenotspeltoutitsgenderfocusintheobjectivesbutduringimplementationhasdevelopedastrongGEWEcharacter,itwouldnotbeincluded.Forcertainsectors,ithasbeenslightlychallengingtodistinguishwhetheraprogrammemainstreamsgenderequalityorfocusesdirectlyonempoweringwomenandgirls.Thishasbeenparticulartrueforsectorssuchaseducation,governanceandHIV/AIDS.Inthesecases,theobjectives,expectedoutcomesandlogicalframeworkshavebeenstudiedtodeterminetheextenttowhichtheprogrammespromotestrategic2genderequalityeffortsand/oraimtoempowerwomenandgirlsdirectly.InagreementwithUNFPAandWHO,allJPsinmaternal,sexualand/orreproductivehealththattheteamhasuncoveredhavebeenincludedintheJGPportfolio.Theseprogrammesmeetthecriterionofwomenand/orgirlsconstitutingtheprimarybeneficiaries.Intheory,theseprogrammesofferanopportunitytopromotetheempowermentandrightsofwomen.STRUCTUREOFTHEREPORTThereporthasfourchaptersinadditiontothisintroductorychapter.Chapter2providesabackgroundtoandrationaleforthefutureJGPevaluation.ItincludesabriefonJPsaspartofUNreformandasummaryoftheeffectivenessofpromotinggenderequalityandwomen’sempowermentwithintheUN.Chapter3rendersaquantitativeanalysisoftheJGPportfolio.Chapter4
analysestheissues,concernsandtheinformationneedsthathavebeenraisedbythedifferentsourcesofqualitativeinformation.Chapter5presentsemergingprioritiesforthefutureevaluationanddiscussesevaluabilityandmethodologicalconsiderations.
2 Promotinggenderequalityatthestrategiclevel,would,forinstance,betopromotegenderresponsivebudgetingorgenderequalityinlegislative
reformprocesses.Meanwhile,aJPthataimstoenhanceeconomicsecurityamongruralpoormenandwomenwouldbeconsideredaprogramme
thatmainstreamsgenderequalitywitheconomicsecurityasanoverallaim.
18
BackgroundtoandRationalefortheJGPEvaluation
ThischapterprovidesashortbackgroundtoUNreformandhowJPswereconceivedaspartofthisprocess.Asummaryoftheeffectivenessofpromotinggenderequalityandwomen’sempowermentwithintheUNdevelopmenteffortisalsobrieflyoutlinedinsection2.2.Byhighlightingrelevantpolicydirectivesandcommitments,thefinalsectionprovidestherationalefortheevaluation.
BRIEFBACKGROUNDTOJOINTPROGRAMMESASPARTOFUNREFORM
AftertheUNSecretaryGenerallaunchedtheUNreformprocessinJuly19973,theUNDevelopmentGroupwasformed(UNDG),CommonCountryAssessments(CCAs)andUNDevelopmentAssistanceFrameworks(UNDAFs)wereintroducedandpreparedforanumberofcountries.Fromthestart,itwashopedthatUNDAFswouldpromotejointprogrammingamongsttheUNagencies.TheJointNordicIndependentAssessmentoftheCCA/UNDAFprocessstudiedtheprogressofJPsthreeyearslater,in2001.ItfoundthatwhereattemptsofestablishingJPshadbeenmade,effortswereimpededbythedifferentadministrativesystemsofeachindividualorganisation.OtherbarriersfacedbyJPsincludedlackofclarityandguidanceonhowtoproceed.Thereportconcludedthat:
...untilharmonisationeffortsareunderwayandthevariousheadquartersgivethefieldthesolidbackingitneeds,collaborativeandjointprogrammingeffortsarelikelytofaceuphillstruggles,riskingthatatleastshort‐termbenefitsandoutcomesofjointprogramminginitiativeswillbeoffsetbyhighpreparatoryandmanagementcosts.4
TheSecretaryGeneral’sreport“Strengtheningthe
UnitedNations:anAgendaforFurtherChange”(A/57/387)in2002broughtgreaterimpetustothereformagenda.JointprogrammeswerespecificallymentionedasmeansoffurtherenhancingtheeffectivenessandefficiencyoftheUNsystem.UNDGwasrequestedtoprepareanimplementationplanby2003tostrengthentheeffectivenessoftheOrganisation’spresenceindevelopingcountries.Aspartofthiseffort,theUNDGGuidanceNoteonJointProgrammeswasdrafted,andinitsoriginalformisthemainguidancedocumentforJPstoday.TheGuidanceNotestatedthatthereformagenda:
“...callsforincreasedjointprogrammingandpoolingofresourcestofurtherenhancetheeffectivenessoftheUnitedNation’ssystemindevelopingcountries,andtoensurethesystem’scombinedresourcesareputtobestuse.ThesemeasuresareintendedtomaximizeUN’seffectiveness,reducetransactioncostsforgovernments,donors,andtheUN,andstrengthenhowtheUNorganizationsprogrammejointlywithgovernments.Theyalsoseektorespondtodonors’andprogrammecountries’concernstoenhancetheUNcontributioninthecurrentcontextofinternationaldevelopmentassistance,withafocusonself‐relianceandcapacitybuilding.”5
In2005,intheSecretaryGeneral’sreport“InLargerFreedom,”6greatersystemcoherence,result‐basedmanagementandstrongleadershipbytheResidentCo‐ordinatorsystemwasemphasised.Aspartofthereformeffort,theSGsubsequentlyestablishedtheHighLevelPanelonUNSystem‐wideCoherencetoexaminehowtheUNsystemcouldworkmorecoherentlyandeffectivelyacrosstheworldintheareasofdevelopment,humanitarianassistanceandenvironment.Inthemeantime,UNDGcommissionedan
3SecretaryGeneralReportRenewingtheUnitedNations:AProgramforReform:MeasuresandProposals1997.14Ljungman,Cetal.LayingtheKeystoneofUNDevelopmentReform:theJointNordicAssessmentoftheCCA/UNDAFProcess.COWIA/S2001.5UNDGGuidanceNoteonJointProgramming,2003.6UNSecretaryGeneral,Inlargerfreedom:towardsdevelopment,securityandhumanrightsforallReportoftheSecretaryGeneral,2005
19
BackgroundtoandRationalefortheJGPEvaluation
assessmenttoenhanceefficiencyandeffectivenessofJPs.Thereportwasfinalisedinthespringof2006.Itreviewed160JPsandundertookcasestudiesin14countriescovering21JPs.ItconcludedthatJPswerenotfullyexploitingtheirpotentialtomainstreamthematicpriorities–suchashumanrightsandgenderequality–thatarenotexclusivetoanyoneUNagency.Itfurthermorehighlighted“gender”anasanareathatcanbemoredeeplyintegratedintoimplementationwhenUNagenciesworktogether.ThereportalsoprovidedanumberofrecommendationsonhowtoimproveJPs(thereviewisfurtherdiscussedinsection4.1.2).Someoftherecommendationsfromthereportwereneverfullyactedupon–suchastherecommendationtoupdate,andcontinuallyupdate,theUNDGguidelinesforJPs.Norwasthesubsequentplantoconductafull‐scalejointevaluationofJPseverfulfilled–althoughadrafttermsofreferencewasprepared.Areasonmaybethatattentionshiftedasaresultofthetwonewrelateddevelopmentsdiscussedbelow.Laterthatyear,theHighLevelPanelissueditsreport“DeliveringasOne”whichraisedthebarandsetoutaprogrammeofreformthatfocusedonfourmainprinciples:OneLeader,OneBudget,OneProgrammeandOneOffice.In2007,eightdevelopingcountriesagreedtopilotDeliveringasOnetoincreasetheUNsystem’simpactthroughmorecoherentprogrammes,reducedtransactioncostsforgovernments,andloweroverheadcostsfortheUNsystem.WhileJPsremainafeatureofthisapproach,DeliveringasOneisamorefar‐reachingandcomprehensiveinitiative.Atthesametime,inDecember2006,theSpanishgovernmentestablishedtheMDG‐Fwith$US700milliontoimproveUNeffectivenessindevelopingcountries.OnerationalefortheFundwasthatfactthefundingstructuresandinstitutionalincentivesforjointUNinterventionswerenotwellenoughresourced–norespeciallyconduciveintheirdesign–toallowforcoherentsupportbytheUNforthe
MDGsatthecountrylevel.ThustheFundwasfromthestartspecificallygearedtosupportingJPs.OneoftheeightprogrammaticareasthatissupportedbytheFundisGEWE(the“GenderWindow”).TheFundcontributedtoincreasingthetotalnumberandaggregatefinancialsizeofnewJGPsin2008and2009.EFFECTIVENESSOFUNDEVELOPMENTEFFORTINGEWE
TheFourthWorldConferenceonWomenin1995signalledaclearcommitmenttointernationalnormsandstandardsofequalitybetweenmenandwomen.Itstipulatedthatmeasurestoprotectandpromotethehumanrightsofwomenandgirl‐childrenweretoconstituteanintegralpartofuniversalhumanrightsandmustunderlieallactions.Institutionsatalllevelsweretobereorientedtoexpediteimplementation.ThisrequiredthatGovernmentsandtheUNcommittedtopromotethe“mainstreaming”ofagenderperspectiveinpoliciesandprogrammes.Inthelastdecade,severalsignificantinitiativeshavebeenundertakentoassesstheprogressinthisarea.In2006Noradcommissionedanindependentreviewthatsynthesisedtheconclusionsofthegendermainstreamingevaluations.7ItincludedtheUNgendermainstreamingevaluationsofUNDP(2006),ILO(2005)andHabitat(2003)andconcluded:
Thefindingsintheevaluationsallpointinthesamedirection.Workoninstitutionalisingtheempowermentofwomenandgenderequalityhavehadlowpriority,therehavebeeninsufficientresourcestoimplementpoliciesandstrategies,thefocushasshiftedtootherareas,andthereisnosystematicreportingofresultsinthisarea.Themainstreamingstrategyhasbeenunsuccessful.
InthesameveintheHighLevelPanelonUNSystem‐wideCoherencewasappointedbytheSecretary
7Norad.LessonsfromEvaluationsofWomenandGenderEqualityinDevelopmentCooperation,SynthesisReport,2006.Italsoreviewedevaluationsof
theWorldBank,Sida,DfID,Norad,DACandEC.
20
BackgroundtoandRationalefortheJGPEvaluation
Generalin2006(asmentionedabove.)Afterfurtherdeliberationswithdifferentstakeholdersitcametotheconclusionthat:
WhiletheUNremainsakeyactorinsupportingcountriestoachievegenderequalityandwomen’sempowerment,thereisastrongsensethattheUNsystem’scontributionhasbeenincoherent,under‐resourcedandfragmented.
ThePanelstatedthatforhumanrightsanddevelopmenteffectivenessreasons,theUNneedstopursueGEWE“farmorevigorously.”Furthermore,itheldthatamuchstrongervoiceonwomen’sissuesisneeded“toensurethatGEWEaretakenseriouslythroughouttheUNsystem”.ItconcludedthatwhilethecommitmenttogenderequalityisandshouldremainthemandateoftheentireUNsystem,agenderentity—basedontheprinciplesofcoherenceandconsolidation—wouldneedtobecreatedtoadvancethiskeyUNagenda.Thisledtothecreation,in2011,ofUNWomen.CorporatelevelgendermainstreamingevaluationscontinuedtobeconductedbyUNagenciesafter2006,butasynthesisoftheirresultshavenotyetbeenundertaken.TheseincludeUNICEF(2007),WFP(2008),IFAD(2010)andtheUNSecretariat(2010)–withoneevaluationofFAOandUNHabitatthatarecurrentlyongoing.Theconclusionsofthefirstthreeoftheseevaluationsareslightlylessnegativethantheevaluationsfrom2000‐2006.Infact,theevaluatorsofeachevaluationclaimthattheorganisationisbetteratmainstreaminggenderequalitythanotherorganisationshavebeeninthepast.Nevertheless,gapsinmonitoringandevaluation,policycoherence,knowledgemanagementandleadershipwerehighlighted.Meanwhile,thefirsteverevaluationoftheUNSecretariat,conductedin2010bytheOfficeforInternalOversightServices,wasunabletodrawdefinitiveconclusionsabouttheoveralloutcomesofgendermainstreamingoritseffectivenessinadvancinggenderequalitybecausethelinkbetweenthestructures,processesandtheirresultswasweakormissing.Itdidconclude,however,thatlackofalignmentbetweenpolicyandpracticeposes“arisktothereputationoftheUN,whichhascommittedto
andhaspromotedgendermainstreamingasastrategyforachievinggenderequality.”Whatisnotablefromtheabovecorporate‐levelevaluationsinthelastdecade,isthatwhiletheyhaveassessedgendermainstreaminginUNagencies,lessemphasishasbeengiventoevaluatingtheresultsofspecificeffortstoaddresswomen’sempowerment,women’srightsandgenderequalityatthestrategiclevel.RATIONALEFORTHEJGPEVALUATON
Atthepolicylevel,therehavebeenseveraldirectivestoenhanceaccountabilityintheareaofgenderequalityandwomen’sempowerment.Forinstance,theGeneralAssemblytookresolutionsin2009and2010thatstate:
EncouragesincreasedeffortsbyGovernmentsandtheUnitedNationssystemtoenhanceaccountabilityfortheimplementationofcommitmentstogenderequalityandtheempowermentofwomenattheinternational,regionalandnationallevels,includingbyimprovedmonitoringandreportingonprogressinrelationtopolicies,strategies,resourceallocationsandprogrammes...
TheTriennialComprehensivePolicyReview(TCPR62/208)from2007recommendedthatintheefforttoimprovetheeffectivenessofadvancingnationalprioritiesandinternationalcommitments,theUNdevelopmentsystemshoulddevelopimprovedguidanceon“thenature,qualityandeffectivenessofJPsinsupportofgenderequalityandtheempowermentofwomen.”Inresponse,theGeneralAssemblyresolutionGA62/208thatrespondedtourgedtheUNorganisations“totakeacoherentandcoordinatedapproachintheirworkongender‐relatedissuesandtosharegoodpractices,toolsandmethodologies.”Reflectingthis,UNIFEM’sStrategicPlan2008‐2013underlinedtheorganisation’scommitmentto“generatingconcreteevidenceandknowledgeonthe“howto”ofgenderequality.”ThespecificcommitmentofUNWomentoevaluate
21
BackgroundtoandRationalefortheJGPEvaluation
JPsintheareaofGEWEemanateoriginallyfromformerUNIFEM’smandatetoengageinJPswhich“offerkeyentrypointstostimulatinggreateroverallUNeffectiveness,whilesimultaneouslygeneratingsignificantbenefitsforgenderequality.”ItalsostemsfromtheformerUNIFEM’scommitmenttoassessingtheconcretebenefitsthatemanatefromJGPs“bothintermsoftheirresultsinadvancinggenderequalityandinbuildinggenderequalitycapacityandcommitmentamongstUNpartners.”ThiscommitmentwasreinforcedbythecreationofUNWomenanditsmandatetoleadandcoordinatetheoveralleffortsoftheUNsystemtosupportthefullrealisationofwomen’srightsandopportunities.TogetherwiththeUNDG,UNWomenispreparingasystem‐widecoordinationstrategyongenderequalityinthefirsthalfof2011,withcleardeliverablesforUNWomenandtheUNSystem,topromotegreatercoherenceinlinewithexistingagencies’mandatesandpriorities.8GiventhatUNWomenisguidedbytheprinciplesofleadingandpromotingcoherenceinUNsystemworkongenderequality;9andactingasaglobalbrokerofknowledgeandexperience,aligningpracticewithnormativeguidance;anevaluationoftheJGPportfoliointheUNhasbecomeallthemorepertinent.
AJointandRights‐BasedEvaluationTakingintoconsiderationthecollaborativenatureofJPs;thecommonstakethatUNagencieshaveinthem;andthemutualeffortthatisrequiredtoimprovethem,itwouldberationalandadvantageousforanevaluationofJGPstobeundertakenjointlywiththeothermainUNagenciesinvolvedinJGPs.ThiswouldalsocoherewithGAresolution62/208andtheSGReportfrom2002,whichbothspecificallypromotethattheUNagenciesengageincollaborativeapproachessuchasjointevaluations.
TocoherewiththeUN’scommitmentsinceovera
decadetomainstreamhumanrightsinallofitsdevelopmentwork,anarticulatedhumanrightsperspectivethroughouttheevaluationwouldbeappropriate.Forseveralyears,theUNEGhasbeendevelopingandpilotingaguideforintegratingarights(andgenderequality)perspectiveinallUNevaluations.Itwillbepublishedinthecomingmonthsandcanbeappliedtothisevaluation.
8UNWomen:Visionand100‐DayActionPlan:ASummaryBriefing.9MichelleBachelet,“StatementtotheFirstRegularSessionoftheExecutiveBoard,UnitedNationsEntityforGenderEqualityandtheEmpowerment
ofWomen”,January24,2011.
22
QuantitativeDeskAnalysisoftheJGPPortfolio
The quantitative analysis is based on a database of the 113 JGPs from the period of 2001 to 2010 that the team has located. For each JGP, information relating to eight main parameters has been entered. These are:
1. Budget: Planned budged and funded budget as per signing of the programme document
2. Actors and Partners: UN, multilateral, donor and national level partners
3. Geography: Country, sub-region and region
4. Lead agency 5. Thematic area 6. Timeframe: Start date, end date and
duration 7. Fund management modality 8. Funding source
The database constitutes the most complete of its kind for JGPs, and perhaps for any kind of JP. The data pertaining to 94 programmes starting from 2006 onward is reliable in relation to the signed programme document for each JGP. However, since many
programmes develop as they are implemented, it is not necessarily accurate or up to date. Most importantly, the funded budget of a programme may increase as it mobilises resources during the implementation period. Likewise, the fund management modality and timeframe could change along the way. For the 19 programmes that started in 2001 to 2005, the data is much less reliable because the signed programme documents have not been located for more than half JGPs. It is even conceivable that some of these JGPs in this database were never initiated as JPs. Due to the unreliability of the data from before 2006, the tables in this report that rely on budget figures mainly use data from 2006 to 2010. It is important to note that the budget figures that are used are indicative – unless a programme is fully funded from the start (which is the case for 54% of all JGPs that started between 2006 and 2010), a JGP will in reality be better resourced after the programme has initiated, but may still have a funding gap. The team has not been able to gather data on actual level of funding in cases where there was a funding gap from the start.
Using the data collected on the 113 JGPs that were initiated between 2001 to 2010, the following sections provide a quantitative analysis. The areas covered include evolution of the JGP portfolio size over time; the number and type of participating UN agencies; regional differences; the thematic spread; the duration of the JGPs and the funding of the JGPs.
JGP Repository
The study team has developed a virtual repository containing al the documents reviewed during the analytical overview fo the JGP portfolio. This includes all the signed joint programme documents, relevant evaluations, guidelines, background documents and policy documents. The repository was built using the D-Groups platform at http://dgroups.org/library. The repository will be made available to the evaluation team and ultimately be incorporated in UN Women’s Knowledge Management systems.
23
QuantitativeDeskAnalysisoftheJGPPortfolio
Figure 3.1: Total number of JGPs Initiated in Each Year from 2001 to 2010
EVOLUATIONOFTHEJGPPORTFOLIOOVERTIME
Figure 3.1 above shows that while a few JGPs existed in the start of the decade, the number of JGPs have increased, but not linearly. In terms of numbers of JGPs, however, since JGPs can be as small as $US 26,000 or as large as $US 43 million, the number of JGPs is only half the story. This range of sizes makes it helpful to study the median size of JGPs. Figure 3.2 shows that JGPs between 2001 and 2005 were relatively small, but more or less in line with the size of other JPs of that period. In the second half of the decade, the median size grew to $US 2.1 million. The sections that follow analyse the evolution of the JGP portfolio from 2001 to 2005 and 2006 to 2010.
Median Size of Programme Budget
JGPs from 2001-2005 $US 320,000 (planned)
All JPs from 1999-2005 $US 300,000
JGPs from 2006-2010
$US 2,100,000 (planned)
All JPs from 2006-2010 No data
Figure 3.2: Median of JP and JGP Budget Sizes
24
QuantitativeDeskAnalysisoftheJGPPortfolio
2001‐2005
Before2005,therewereonlyfiveJGPs–threeofwhichpre‐datetheUNDGguidelinesforJPs–andallbutonehadplannedbudgetsthatwerearoundonetotwomilliondollarsinsize–whichisrelativelylargecomparedtotheJGPsthatfollowedin2005.However,theteamdidnothavethesignedprogrammedocumentstoverifythisinformation.In2005,therewasalargehikeinthenumberofJGPswith14beinginitiated.Nineofthesehadplannedbudgetsthatweresmallerthan$600,000–ofwhichfourweresmallerthan$130,000.ThedevelopmentoftheUNDGGuidanceNoteonJointProgrammescouldbeonereasonthatcountryofficeslaunchedmoreJPs.Furthermore,atthetimethattheGuidanceNotewasprepared,theUNagencieshadanewmeansoftransferringfundslegallyamongstoneanother.Consideringthelargenumberoffairlysmallprogrammes,thisincreasecouldrepresentafewprogrammesthatmoreresembleafundtransferfore.g.anactivityofcommoninterest.OneJGPfromthisperiodisthesupporttotheGenderFacilityforResearchandAdvocacyinChina,inwhichfiveUNagencieshavepartneredwithDFIDinaJGPfullyfundedat$US2.7million.Itisuniquesinceitrunsfora10year
period–withanenddatein2015.
2006‐2010
Thedatabasehastwobudgetfiguresformost10JGPsfromthisperiod–oneistheplannedbudgetfortheprogramme,theotheristhefundedbudgetatthetimeofsigningtheprogrammedocument.11Thedifferencebetweenthebudgetsisillustratedinthefigurebelow.Onaverage,thereisa28%differencebetweentheplannedandfundedbudgets(fundinggap)atinceptionforJGPsatcountryandregionallevel.However,regardlessofwhethertheplannedorinitiallyfundedbudgetfiguresareused,thetrendremainsthesameovertime:since2007,thetotalvalueofnewJGPseachyearhasbeenabove$US70million.Figure3.3showsapeakin2008inbothplannedandfundedbudgets.Around$62million(sevenJGPs)in2008and$24million(fourJGPs)ofthetotalvalueofnewJGPsin2009originatedfromtheMDG‐F.However,thepeakin2008cannotwhollybeexplainedbytheadditionofresourcesfromtheMDG‐Fsincetheincreasefrom2006is$US122million.Meanwhile,thevalueoftheJGPsthatwerestartedin2009areroughlyequivalenttothe2007levels,withtheextraadditionof$US24fromtheMDG‐F.
10Thedatabasecontainsaplannedbudgetfigureforallbut5.3%oftheJGPsfrom2006to2010anddoesnothavethestatusofthefundedbudgetfor14.9%oftheJGPs.Theaggregatedataforplannedbudgetsisthereforemorecomplete.Theplannedbudgetisfurthermorearguablyabetterreflectionofthesizeoftheprogrammeasitwasconceived.However,itisimportanttotreatthebudgetfiguresasindicative.Thereare5JGPsthathavenotbeenincludedinthegraphswithbudgetfiguresduetolackingdata.Thesearei)SaoTomeandPrincipe:Strengthenedcapacityofnationalandlocalinstitutionstoadvancegenderequality;ii)JPonRapidReductionofMaternalandNeonatalMortalityinthePhilippines;iii)GuineaBissau:Egalitédegenreetrenforcementdesmoyensd'actiondesfemmes;iv)Zambia:JointProgrammeonTrafficking;v)Comoros:Accélerationdelareductiondelamortalitématernelle,néonataleetinfantile.11In54%ofthecases,theJGPisfullyfundedfromthestartandthesetwobudgetfiguresarethesame.Thisisthecasefore.g.the12JGPsfundedbyMDG‐FandvirtuallyallJGPsintheLACregion.
Figure 3.3: Total Value of Planned Budgets and Funded Budgets* of JGPs Initiated Each Year from 2006 to 2010
*At the time of the signing of the Programme
25
QuantitativeDeskAnalysisoftheJGPPortfolio
Figure 3.4: Average Planned Financial Size of JGPs Initiated Each Year from 2006 to 2010
Figure3.4aboveshowsadifferentangleoftheevolutionbyprovidingtheaverageplannedfinancialsizeofJGPseachyear.Likethepreviousgraph,itdepictsasteepclimbinbetween2006and2007.However,fromthenonthefigurefollowsaverydifferentpattern.Whiletheaverageplannedfinancialsizein2007and2008weresimilar,therewasasteepfallin2009,onlytoberecoveredin2010,whichrepresentsthepeakinaveragesizeat$US7.04million.Theteamhasnotidentifiedanexplanationforthistrend.
PARTNERSSince2001,therehavebeenover24UNentitiesthathaveparticipatedinJGPs.AUNentityisdefinedasparticipatingwhenithassignedtheprogrammedocument.Itmostcasesaparticipatingentityalsocontributessomecorefunds.UNFPAfollowedbyUNDP,formerUNIFEMandUNICEFarebyfarthemostfrequentactors,participatinginover60JGPseach.WHOhasparticipatedin38,makingitamedium‐largeactor.Threespecialisedagencies–ILO,UNESCOandFAO–constitutethemedium‐smallbracket(participatingin10to21JGPs).InthisbrackettheyarejoinedbytwocomparativelylargeUNagencies–UNHCR,andWFP;andUNAIDS.AmongtheUNorganizationsthathaveparticipatedinfive
orlessJGPsareamixofsmalleragencies,UNmissions,andacoupleofUNregionaleconomiccommissions.
NumberofParticipatingUNPartnersThedatarevealsthat38percentoftheJGPsfrom2001to2010havethreeorfourpartners.Thesecondmostcommonsizeistwopartners,whichmakeup28percentoftheJGPs.TherearenineJGPsthathavenineormorepartners–threeofwhichhave11ormore.TheseincludeJGPsinKenya(13partners,plannedbudgetof$US56.5millionwithabudgetgapof$28million);Vietnam(12partners,plannedbudgetof$US4.7millionfullyfundedbytheMDG‐F);andUganda(11partners,plannedbudgetof$US24.6millionwithabudgetgapof$6.4million).ItwouldbelogicaliftheaverageandmediansizeoftheplannedbudgetsoftheJGPsgenerallygrowswiththenumberofUNpartners.ThisisgenerallytrueforJGPs.ProgrammeswithtwoUNpartnershaveamedianplannedbudgetsizeof$US678,500(theaveragebudgetsizeismorethandoubleduetosixprogrammesthatare$US3.3to10millioninsize).TheJGPsthathave11UNpartnersormorehaveanaverage/medianplannedbudgetthatisaround$US30million.However,whetheraJGPismadeupoffivetosixUN partners or seven to
26
QuantitativeDeskAnalysisoftheJGPPortfolio
Figure 3.5: Frequency of Participation of UN Organizations in JGPs 2001 to 2010
Figure 3.6: Number of Participating Agencies in JGPs 2001 to 2010
eightUNpartnersdoesnotmakeasubstantialdifferenceinthebudgetsize.Infact,themediansizeisvirtuallythesameforthesetwogroups.
ItwouldbeexpectedthatthenumberofJGPswitharelativelysmallplannedbudgetwoulddecreaseasthenumberofUNpartnersgrow.Indeed,suchapatternexistsbetweentwotosixUNpartners:38%oftheJGPswithtwoUNpartnersareworthlessthan$US350,000.Whenthenumberofpartnersisincreasedto three to four UN partners, there is a 12-point drop to 26%. There is another 12-
Average Median
2 Agencies $ 1,782,728* $ 678,500
3-4 Agencies $ 2,776,805 $ 1,099,000
5-6 Agencies $ 4,988,120 $ 3,638,888
7-8 Agencies $ 4,093,399 $ 3,640,222
9-10 Agencies
$ 8,811,441 $ 8,000,000
11+ Agencies $ 30,778,849 $ 31,106,657
Figure 3.7: Average and Median Size of Planned Financial Value of JGPs Per Number of Participating Agencies 2001 to 2010
27
QuantitativeDeskAnalysisoftheJGPPortfolio
Figure 3.9: Percentage of JGPs by Different UN Lead Agencies 2006 to 2010
point drop when a JGP consists of five to six agencies. JGPs that have nine or more partners are never under $US 350,000. However, an astounding 43% of JGPs with seven to eight UN partners are under $US 350,000.
LeadAgency
The2003GuidanceNoteforJointProgrammesdoesnotrecognisetheconceptof“leadagency”–anagencythatplaysthecentralcoordinatingroleintheprogramme.Inone‐thirdoftheJGPsfrom2006‐2010,thereiseithernoleadagencyspecifiedintheprogrammedocumentorthedataismissingonthisaspect.However,two‐thirdsoftheJGPsthatwereinitiatedbetween2006and2010havedesignatedaleadagencyintheprogrammedocument.AtleasteightdifferentagencieshaveservedasleadagenciesforJGPs.At24%UNFPAisthemostcommonleadagencyforJGPs.ItisfollowedbyformerUNIFEM(20%)andUNDP(16%).WHOactedasleadagency
Percentage of JGPs under $350,000
2 Agencies 38%
3-4 Agencies 26%
5-6 Agencies 14%
7-8 Agencies 43%
9-10 Agencies 0%
11+ Agencies 0%
intwoJGPs.UNCDF,UNICEF,ECLACandILOhaveeachleadoneJGP.
ThemostcommonmultilateralpartneroutsideoftheUNsystemistheInternationalOrganisationforMigration(IOM).TheIOMisinvolvedinnineJGPs,whichmeansitisamorecommonJGPpartnerthan12oftheUNpartnersincludedinFigure3.9.TheWorldBank,theAfricanDevelopmentBankandOSCEhaveeachparticipatedinoneJGP.
Figure 3.8: Percentage of JGPs with Planned Budgets Under US$ 350,000 per Number of Participating Agencies 2001 to
28
QuantitativeDeskAnalysisoftheJGPPortfolio
GEOGRAPHY
Sixty‐onecountrieshavehadatleastoneJGPintheyears2006to2010.SeventeencountrieshadtwoormoreJGPsduringthisperiod.ArgentinastandsoutbyhavinghadfourJGPsinthisperiod,butallwere12‐monthprogrammeswithrelativelysmallbudgets.Egypt,Morocco,NepalandthePhilippineseachhadthreeJGPs,whichalloverlappedintime.InallcasesbutNepal,thethematicareaofeachJGPwasdifferent–NepalhadtwodifferentEVAWprogrammes–onewithaconflict‐relatedfocus.TwelvecountrieshadtwoJGPs–witheachJGPscoveringadifferentthematicareafromtheother.Inhalfthecasesthetimeframesoverlapped.
WhenanalysingtheJGPportfoliofrom2006to2010bygeographicregion,AfricaleadswiththegreatestnumberofJGPs(29).TheaggregationofallofthevalueofplannedbudgetsinAfricaduringthisperiodgivesatotalof$US254million–whichismorethanfourtimesthatoftheAsia/PacificandLACregions.TheLACregionisrightbehindAfricawiththetotalnumberofJGPswith26–14ofwhichhavebeenimplementedinsixcountries.TheCEECISregionhasthesmallestnumberofJGPs,witheightintotal.TheArabStatesregionhasfivelessJGPsthanCEECIS,buttheplannedfinancialvalueamountstomorethanfourtimesthatoftheCEECISregion.Figure3.12belowillustratestheresultsofanalysisoftheregionsusingthenumberofJGPsasapercentageofall94JGPs,andtheplannedfinancialvalueofJGPasapercentageofthetotalplannedvalueof$US463million.AfricaaccountsforoverhalftheplannedfinancialvalueofJGPs(althoughthereisa55%gapbetweenthefundedbudgetatinceptionandtheplannedbudget).ThebudgetsfortheLACJGPswerealmostfullyfundedfromthestart(theonlyregionwherethisisthecase).Meanwhile,theaveragefundinggapintheCEE/CISandArabStatesregionsisaround20%.TheAsia‐Pacificregionhasanaveragefundinggapof39%atinception.
Region Numberof
JGPs
TotalValueof
PlannedBudget
MUSD
Africa 29 254
ArabStates 13 54
Asia&Pacific 17 66
CEECIS 8 13
LatinAmerica&Caribbean 26 66
Global 1 10
Figure 3.10: Countries with JGPs with More Than One JGP 2006 to 2010
Figure 3.11: Number of JGP and their Total Planned Value Per Region 2006 to 2010
No. Of JGPs Region Country Time Overlap of JGPs
No. Of Thematic areas
4 JGPs LAC Argentina 2 overlappe
d – all under 12 months
3
Arab States
Egypt Yes 3
Morocco Yes 3
Asia/ Pacific
Nepal Yes 2
3 JGPs
LAC
Philippines Uruguay
Yes Yes
3 1
Africa Comoros No 2 Eritrea Yes 2 Lesotho No 2
Liberia Yes 2 Arab States
Iraq Tunisia
Yes Yes
2 2
LAC Bolivia Yes 2
Ecuador No 2 Haiti Yes 2 Venezuela No 2 Asia/ Pacific
Bangladesh Yes 2
2 JGPs
Mongolia No 2
29
QuantitativeDeskAnalysisoftheJGPPortfolio
Figure 3.12: Number of JGPs and Planned Financial Value of JGPs per Region Respectively as a Percentage of the Numeric Size and a Percentage of the Financial Size of the JGP Portfolio
Funded Planned Difference %Difference
Africa 114,632,435 253,598,293 138,965,858 55
ArabStates 43,349,903 54,268,704 10,918,801 20
AsiaPacific 40,664,938 66,186,117 25,521,179 39
CEE/CIS 9,738,941 12,695,524.0
3
2,956,583 23
LAC 65,119,151 66,259,871 1,140,720 2
Global 1,030,000 9,830,000 8,800,000 90
THEMATICAREA
WithinJGPseightthematicareashavebeenidentified.Asdiscussedinsections00,theseincludeGovernance,HumanTrafficking,EconomicEmpowerment,EliminationofViolenceAgainstWomen(EVAW),Education,Health,HIV/AIDSandmulti‐sectoralprogrammes.WhileseveralJGPshaveelementsofmorethanonetheme,theJGPsthatareincludedinthelatterincludecleargoalsintwoormoreofthepreviousthematicareas.
EVAWisthemostcommonthematicarea(29JGPsor31%)andaccountsforasimilarchunkoftheoverallaggregatedplannedbudget(28%).Meanwhile,themulti‐sectoralJGPsaccountforonly
11%oftheJGPsbutbecausetheseareallverylargefinancially,theyrepresent33%ofthetotalaggregatedplannedbudgets.Thereisalsoalargediscrepancybetweenthepercentageofgovernanceprogrammes(29%)andthepercentageofthetotalfundsplannedforthesetypesofprogramme(13%).Thereare5JGPs(1inSierraLeoneand2JGPsinLiberiaandNepaleach)thatspecificallymention“conflict”inthetitleand/orobjectives.TheirthematicfocusesaretheareasEVAW(2),Health,GovernanceandEconomicEmpowerment.Inaddition,thereare8otherJGPsincountriescurrentlyorrecentlyaffectedbyviolentconflictthatdonothaveaconflictangleintheobjectives.
Figure 3.13: Difference Between Planned and Funded Financial Budgets of JGPs per Region
30
QuantitativeDeskAnalysisoftheJGPPortfolio
Figure 3.14: Number of JGPs and Planned Financial Value of JGPs per Thematic Area Respectively as a Percentage of the Numerical Size and a Percentage of the Financial Size of the JGP Portfolio
Figure 3.16: Number of JGPs from 2001 to 2010 by Duration in Months
Average Duration
All JPs from 1999-2005 26 months
JGPs from 2001-2005 25 months (36 months)*
JGPs from 2006-2010 30 months
All JPs from 2006-2010 No data
PLANNEDTIMEFRAME
TheaverageplannedtimeframeforJGPshasincreasedovertimefromaround25monthsto30months.SincemostJGPsseemtofaceimplementationdelays,itisprobablethattheactualdurationislonger–insomecases,considerablylonger.
Atleast17percentoftheprogrammesareplannedtolastlongerthanthreeyears,whileatleast14%are12monthsorlessinduration.
Figure 3.15: Average Duration of JPs and JGPs
31
QuantitativeDeskAnalysisoftheJGPPortfolio
Figure 3.17: Percentage of JGPs from
FUNDINGFrom2006to2010,thetotalplannedvalueoftheJGPportfoliowas$US463millionandthetotalvaluethatwasfundedatthetimeofsigningoftheprogrammesdocumentswas$US274million.Thesourcesoffundingincludecorefundsfromtheagencies,16differenttrustfunds,oneregionaldevelopmentbank,onemultilateralagency(IOM),13recipientgovernmentsand15donorgovernments.Moredatagatheringandresearchisneededtobeabletodetermineexactlywhatsourcecontributedwhatamounttothisportfolio.Nevertheless,someinterestingfactscanbegleanedfromthecurrentdata.
TheillustrationaboveshowsthatatleastmorethanhalfofallJGPswerefullyfundedfromthestart.Whilewedonothavefiguresfor16%oftheJGPs,weknowthatatleast28%facedafundinggapatinception.Between62%to72%oftheJGPsfrom2006to2010havebenefittedfromcorefundingfromtheparticipatingUNagencies.Atleast17%oftheJGPswerefullyfundedbycorefundsfromUNagencies.TheseJGPsrangedinfinancialsizefrom$US27,000tonearly$US5millioninsize,withanaveragesizeofabout$US1million.Atleastanother17%werenotfullyfundedtostartoffwith,butatthetimeofsigning,theirbudgetwasonlymadeofUNagency
corefunds.TheUNcorecontributionsfortheseJGPsrangedfrom$US140,000toover$US28million,withanaverageofabout$US3millionmakingupbetween22%to90%oftheplannedbudget.Intotal,UNagencieshavecontributedover$US98fromtheircoreresourcestotheJGPportfolio,makingthecombinedUNcorefundsthelargestsourceoffundingforJGPs.Inthe25caseswhereparticipatingUNagencieshavenotcontributedtotheJGPs,theyhaveinmostcasesbeenfullyfundedfromthestartbyatrustfund.TheMDG‐Fisthemostfrequentsinglenon‐coresourceoffundingwith14JGPsor15%being
fullyfundedbyit,amountingtoabout$US90million.TheUNTrustFundfortheEliminationofViolenceagainstWomenhassupportedsixJGPs.JGPshavealsoreceivedfinancialsupportfromOneUNfunds(4),theUNTrustFundforHumanSecurity(2),theUNPeace‐BuildingFund(2),theUNGlobalInitiativetoFightHumanTraffickingandtheUNDemocracyFund(1).Intotal,atleastnearly40%JGPshavereceivedsupportfromtrustfunds.Sincesomeofthestrongestbilateralsupportersofgenderequality(andthemultilateralsystem)prefertoprovidevoluntarycorefundstoagencies,andsincecorefundsmakeupasignificantpartoftheJGPportfolio,listingthedonorcountriesthathave
32
QuantitativeDeskAnalysisoftheJGPPortfolio
Figure 3.18NumberofJGPsandPlannedFinancialValueofJGPsperFundManagementModalityasaPercentageoftheNumericandFinancialSizeoftheJGPPortfolio
providednon‐coreresourcestoJGPsgivesanincompletepictureofhowsupportivedonorcountrieshavebeentoJGPs.Nevertheless,thecountriesthathaveprovidedfundstobetweenonetothreeJGPsattheprogramme’sinceptionincludeAustralia,Austria,Belgium,Denmark,Canada,Ireland,Italy,Germany,theNetherlands,Norway,Sweden,SwitzerlandandtheUK.Inaddition,theEuropeanCommissionhassupportedoneJGP.IncludingthesupportfromtheMDG‐Fund,Spainisthemostcommonnon‐coresupporterofJGP.Inadditiontothe14largeMDG‐fundedJGPs,ithassupportedsmallJGPsinLatinAmericanregion(3)andinEquatorialGuinea(1).Moredatawillneedtobegatheredtoestablishtheexactfinancialvalueofthenon‐coresupport,particularlysinceinmanycasesbilateralsupportwillhavebeengrantedaftertheprogrammedocumentsweresigned.
Nationalgovernmentshaveprovidedresources–financialorin‐kind–foratleast13JGPs.For10JGPs,thefundingsourcesareunknown.Therearethreemainfundmanagementmodalities12forJPs.Theseareparallel,pooledandpass‐through.Thesemodalitiescanalsobecombined.Thereareonly10JGPsthatusethepooledfundmanagementmodalityandtheyaretypicallysmall–40%areworthunder$US350,000.Aboutone‐thirdoftheJGPsaremanagedthroughparallelarrangementsandanotherthirdbythepass‐thoughfundingmodality.However,thetotalfinancialvalueofthe17%ofJGPsthataremanagedthroughacombinationofmodalitiesisgreaterthanthetotalfinancialvalueoftheJGPsmanagedbyeitherthepass‐throughorparallelfundmanagementmodalities.
SUMMARY
InthebeginningofthedecadetheJGPportfoliowassmall–withatmostacoupleofJGPsbeinginitiatedeachyear.Thebudgetswerealsomodest–themediansizewas$US320,000.In2005,therewasasurgeof14JGPs–probablyasaneffectoftheUNDG
JointProgrammeGuidanceNotehavingbeencirculatedtheyearbeforeandtheUNreformprocessmovingahead.Thesecondpartofthedecadesawariseinthemedianbudgetedprogrammesizeto$US2millionandthetotal
12PleaseseeAnnex7forUNDG’sdefinitionsoffundmanagementmodalities.
33
QuantitativeDeskAnalysisoftheJGPPortfolio
numberofJGPsrosedramaticallyin2008and2009.ThiscanpartlybeexplainedbytheadditionalofMDG‐Fresources.However,thelargestaveragesizeofJGPswasin2010($US7million),whichwasaftertheMDG‐Fcontributionsweremade.Twenty‐fourdifferentUNentitieshaveparticipatedinatleastoneJGPandUNFPA,UNDP,UNICEFandformerUNIFEMhavebeenthemostfrequentagencies,participatinginover60agencieseach.UNDP,UNFPAandformerUNIFEMwerealsobyfarthemostprevalentintheroleofleadagency.ThespecializedagenciesWHO,ILO,UNESCOandFAOarethesecondmostfrequentparticipants.ThemajorityofJGPsaremadeupofthreetofourparticipatingUNagencies.One‐thirdoftheJGPhavefiveormoreparticipatingUNagencies–somehaveover11.Generally,thesizeoftheplannedbudgetgrowswiththenumberofUNactors.However,asignificantnumberofJGPswithsevenoreightparticipatingUNagencieshaveaninexplicablysmallbudget(under$US350,000).AfricahasthegreatestnumberofJGPsandaccountsforthelargestportion(55%)ofthetotalplannedfinancialvalueoftheJGPportfoliofrom2006to2010.TheAsia/PacificandtheLACregionsaccountfor14%eachofthetotalplannedfinancialvalueoftheJGPportfolio,butinLACtheindividualJGPsaremuchsmallerinsize.Multi‐sectoralJGPsarefew,buttheyhavelargebudgetsthataccountfor33%oftheaggregatedplannedfinancialvalueoftheJGPportfolio.IntermsofnumberofJGPs,theEVAWthematicareaisthelargest–roughlyaccountingforjustlessthanone‐thirdofallJGPsandone‐thirdoftheaggregatedplannedfinancialvalueoftheJGPportfolio.JGPsinthegovernanceareaarealmostasnumerousasEVAWJGPs.However,theyhavemuchsmallerbudgetsthatamounttoonly13%oftheaggregatedplannedfinancialvalueoftheJGPportfolio–whichissimilartothevalueofthehealth(13%)andeconomicempowerment(9%)JGPs.Thenumberandvalueoftheeducation,traffickingandHIV/AIDSJGPsareonlyafewpercenteach.OnlyfiveJGPsrepresentingfourthematicareashaveobjectiveswithaconflict‐relatedangle.
CorefundsfromtheparticipatingUNagenciesarethemostimportantsourceoffunds–benefitting62%to72%oftheJGPsfrom2006to2010.Meanwhile,theMDG‐Fisthemostimportantsinglesourceofnon‐corefundsduring2006to2010(13%).Othertrustfunds–UNTrustFundfortheEliminationofViolenceagainstWomen,OneUNfunds,theUNTrustFundforHumanSecurity,theUNPeace‐BuildingFund,theUNGlobalInitiativetoFightHumanTraffickingandtheUNDemocracyFund–accountforthefundingto16%oftheJGPsfrom2006to2010.Bilateral,OCED‐DACcountrycontributionstoJGPsattheinceptionoftheprogrammehavebeenmadebyAustralia,Austria,Belgium,Canada,Denmark,Ireland,Italy,Germany,theNetherlands,Norway,Spain,Sweden,SwitzerlandandtheUK.Othernationalgovernmentshavealsoprovidedresources–financialorin‐kind–foratleast13JGPs.Asmanyas56%JGPshavebeenfullyfundedfromthestart.Thepass‐throughandparallelfundingmodalitiesarethemostcommon–amountingtoaboutathirdeachofthetotalnumberofJGPsfrom2006to2010.However,theparallelfundedJGPsaresmallerinfinancialsize–makinguponly16%oftheplannedfinancialvalueoftheJGPportfolio.PooledJGPsarelesscommon(11%)andfinanciallyverysmall–2%oftheplannedfinancialvalueoftheJGPportfolio.JGPsfundedthroughacombinationofmodalitiesmakeup17%oftheJGPsbut37%oftheaggregatefinancialvalue.
34
FindingsfromQualitativeDeskAnalysis
Thischapterdocumentsthequalitativefindingsoftheportfoliooverview.Itisbasedoninformationthathasbeengatheredfromevaluations/reviews,otherassessments,documentsandconsultationswithover20stakeholders.Thefirstsectiondiscussesthequalitativesources–stakeholderinformants,evaluations/reviewsandotherdocuments.Thesecondpartofthischapteroutlinestheissues,concernsandtheinformationneedsthathavebeenraisedbythedifferentsourcesofqualitativeinformation.
SOURCESOFQUALITATIVEINFORMATION
ThreesourcesforqualitativedatawereusedintheanalyticaloverviewoftheJGPportfolio.Eachisdiscussedinthesectionsthatfollow.
JGPEvaluations/ReviewstoDate
ThereappearstobeveryfewevaluationsandreviewsofJGPsfrom2001to2010.Theteamhasonlyuncoveredfourevaluationsand16reviews/mid‐termevaluations–includingthe11mid‐termevaluationsoftheJGPsthatarefundedbythe“genderwindow”oftheMDG‐F.Therearethreeassessmentscoveringprogrammesthatwereinitiatedintheperiod2001to2005.Theevaluationsandreviewsareofvaryingquality–manyofwhicharefoundtobelackinginquality.MostdonotanalysethejointaspectoftheJGPs.Sincemostarereviews,thereisconsiderablefocusontheprogrammingprocess.Althoughanumberoftheprogrammesanalysedbytheevaluations/reviewshaveahumanrightsperceptiveintheoverallgoalandoutcomes,theevaluations/reviewshavetendedtobebeenveryweakinassessingtherightsbasedapproach.
COUNTRY ASSESSMENT
TYPE
NAMEOFPROGRAMME TIMEFRAME THEME ASSESSED
1. INDIA REVIEW COORDINATEDHIV/AIDSANDSTDRESPONSE
THROUGHCAPACITY‐BUILDINGANDAWARENESS
(CHARCA)
2003‐07 HIV/AIDS 2006
2. DRC REVIEW LAPRÉVENTIONETLARÉPONSEAUXVIOLENCES
SEXUELLESFAITESAUXFEMMES,AUXJEUNESETAUXENFANTS
2005‐08 EVAW 2007
3. ANGOLA REVIEW JOINTGENDERPROGRAMME 2005‐08 MULTI‐
SECTORAL
4. MAURITANIA FINALEVALUATION
SUPPORTTOINVOLVEWOMENINDECISIONMAKING
PROCESS2006‐07 GOVERNANCE 2007
5. LAC
REGIONAL
FINAL
EVALUATION
ENGENDERINGBUDGETS 2006‐08 ECONOMIC
EMPOWERMENT
2008
6. ALBANIA MID‐TERM
EVALUATION
NSGE‐DV–ADVANCINGDEMOCRATICGOVERNANCE‐ 2008‐10 GOVERNANCE 2009
7. HAITI FINALEVALUATION
SUPPORTINGWOMEN’SPOLITICALPARTICIPATION– 2007‐08 GOVERNANCE 2009
8. BANGLADESH REVIEW JOINTPROGRAMMEONMATERNALANDNEONATAL
MORBIDITYREDUCTION‐REVIEW
2007‐10 HEALTH 2010
9. TIMORLESTE MID‐TERM
EVALUATIONSUPPORTINGGENDEREQUALITYANDWOMEN'SRIGHTSINTIMORLESTE–
2008‐11 GOVERNANCE 2010
Figure 4.1: Table of JGP Reviews and Evaluations
35
FindingsfromQualitativeDeskAnalysis
10. MOROCCO MID‐TERM
EVALUATION
FIGHTAGAINSTGENDER‐BASEDVIOLENCETHROUGHTHE
EMPOWERMENTOFWOMENANDGIRLS
2008‐10 EVAW 2010
11. JORDAN FINALEVALUATION
SUPPORTTOEFFECTIVEWOMEN'SPARTICIPATIONINPUBLICLIFEATTHELOCALMUNICIPALLEVEL
2008‐10 GOVERNANCE 2010
12. NAMIBIA MID‐TERM
EVALUATION
SETTINGTHINGSRIGHT‐TOWARDSEQUALITYAND
EQUITY‐
2009‐12 MULTI‐
SECTORAL
2010
13. BOLIVIA MID‐TERM
EVALUATION
PRODUCTIVEPATRIMONIALASSETSBUILDING&
CITIZENSHIPPROGRAMMEFORWOMENINEXTREME
POVERTY
2008‐11 ECONOMIC
EMPOWERMENT
2010
14. BRAZIL MID‐TERM
EVALUATION
INTER‐AGENCYBRAZIL‐PROGRAMMEFORTHE
PROMOTIONOFGENDERANDETHNIC‐RACIALEQUALITY
2008‐10 GOVERNANCE 2010
15. COLOMBIA MID‐TERM
EVALUATIONINTEGRALSTRATEGYFORTHEPREVENTIONANDAWARENESSOFGENDER‐BASEDVIOLENCE
2008‐11 EVAW 2010
16. GUATEMALA MID‐TERM
EVALUATION
FORTALECIENDOLAINSTITUCIONALIDADDELASMUJERES
ENGUATEMALA
2008‐11 GOVERNANCE 2010
17. NICARAGUA MID‐TERM
EVALUATIONPROMOTINGWOMEN’SPARTICIPATIONANDGENDER
RESPONSIVEBUDGETING2008‐11 ECONOMIC
EMPOWERMENT2010
18. OCCUPIED
PALESTINIANTERRITORIES
MID‐TERM
EVALUATION
GENDEREQUALITY‐SOCIAL,POLITICALANDECONOMIC 2008‐11 GOVERNANCE 2011
19. ETHIOPIA MID‐TERM
EVALUATIONLEAVENOWOMENBEHIND 2009‐12 INTEGRATED 2011
20. VIETNAM MID‐TERM
EVALUATION
GENDEREQUALITYINVIETNAM 2009‐11 GOVERNANCE 2011
OtherRelevantJPDocuments
Theteamtriedtolocateotherstudies,reportsandassessmentsthatcouldbeofrelevancetothisstudy.Thishasbeenchallengingandyieldedlimitedresults.13Nevertheless,thefollowingdocumentshavebeenused:• AsignificantassessmenteffortofJPswas
thecross‐sectoralreview14thatwascommissionedbyUNDGin2006toenhanceefficiencyandeffectivenessofJPs(pleaseseesection0).15Covering160JPsfrom1999
to2005andincludingcasestudyreportsfrom14countries,thisreportoffersthemostcomprehensiveandanalyticalassessmentoftheeffectivenessandefficiencyofJPstodate.
• In2004UNICEF’sExecutiveDirectorpreparedareporttotheExecutiveBoardconcerningUNICEFexperienceinjointprogramming.
• In2006,UNDP,UNICEFandUNFPApreparedareporttotheirexecutiveboardsontheimplementationexperienceofjoint
13OthershavefounddocumentationonJPstobedifficulttocomeby.The2009reviewoftheGenderProgrammeinAlbaniamentionedthatattemptsweremadetosearchforotherevaluationsonotherJPstobuildontheexperience,butlittlewasfound.14UNDG.EnhancingtheEffectivenessandEfficiencyofJointProgrammes:LessonsLearnedfromaUnitedNationsDevelopmentGroupReview.2006.15Inthelastchapter,thisstudyprovideddatathatallowedcomparisonsfordataonmediansizeofJPsandtheirduration.
36
FindingsfromQualitativeDeskAnalysis
programmingandJPsbyUNDP,UNFPAandUNICEFsince2004.
• AmongtheDeliveringasOneevaluationsundertakenin2010–CapeVerde,Albania,Mozambique,Vietnam,UruguayandRwanda–assessedGEWEeffortstosomedegree.
• Themid‐termevaluationsoftheMDG‐FcovereddozensJPsinarangeofsectors.TheFundhaspreparedaggregateanalysesforthreeothersectorsthatprovideanoverview.Meta‐analysesoftheevaluationsarecurrentlyunderway.Ameta‐evaluationofthe11JGPsresourcedbytheMDG‐Fisexpectedtobereadylaterthisyear.
Stakeholders
Asdiscussedinsection0,thisstudyhasidentifiedinformationneedsissuesbyconsultingwitharangeofstakeholdersfromwithintheUNsystem.TheseincludeHQstaffconcernedwithgenderequality,JPsorevaluations;andfieldlevelstaffincludingRCs,JPcoordinatorsandUNWomencountryrepresentatives.WhilethereisageneralcoherenceamongtheinterviewedstakeholdersonwhatinformationisneededandwhatthechallengesarethattheJPsface,theweightgiventodifferentissuesistypicallyafunctionofeachstakeholder’sspecificareaofworkandposition.Forarights‐basedstakeholderanalysisforthisevaluation,pleaseseeAnnex4.
ISSUESANDINFORMAITONNEEDS
Thissectiondocumentstheissuesandinformationneedsraisedbystakeholderswhowereconsultedforthisstudyandtheevaluations,reviewsandotherdocumentsthathavebeenexamined.Theissuesandneedshavebeendividedintofivethematicareas–thequalityornatureoftheJGP’s“jointness”;thedesignanddesignprocessesoftheJGPs;theeffectivenessofJGPsintermsofresults;
nationallevelpartnerships,nationalownershipandpeoplecentredapproachesofJGPsand,efficiencyanoperationaleffectivenessofJGPs.Theaimofthissectionistoprovideawell‐roundedandanalyticalaccountoftheissuesandinformationneeds.Whilethesourcesoftheinformationforthissectionareprovidedinthetext,thetextisnotnecessarilyorganisedbythedifferentsources(stakeholders,evaluations,reviews).Furthermore,althougheffortshavebeenmadetodrawuponasmanyevaluations/reviewsaspossible,youwillfindthatsomeevaluations/reviewsfeaturemorefrequentlyinthetext.Thisisbecausetheyhaveofferedmorereflectionsand/orraisedissuesandneedsmoreclearly.Meanwhile,someevaluations/reviewhaveofferednoorminimalinsightsforthisstudy.TheUNDG2006reviewofJPsisreferredtoinseveralsectionsasameansofillustratingconcurrencewithmorerecentassessments–whichsuggeststhatcertainissueshaveremainedimportantthroughoutthedecade.
QualityofJointness
ThedefinitionofaJP(renderedinsection 1.3)providesabasicsetofcriteriathatneedtobemetfortheprogrammetoberecognisedasaJP.Aspointedoutinanumberofevaluations,itisfairlyeasyforaprogrammetopassasjoint.However,stakeholdersandevaluationsholdthatJPsvaryconsiderablyintermsoftheir“jointness”andthereforedeterminingthedegreeofjointnessisapertinentquestion.AnumberofstakeholdersmentionedthatsomeJPscouldbejointinnameonly.Forinstance,itwaspointedoutthatsinceJPsofferoneofthefewmeansoftransferringfundslegallybetweenagencies,someJGPsmay,uponcloserinspection,bemoreofaninter‐agencyarrangementthanatrueJP:toboostitsdisbursementrateattheendoftheyear,anagencycouldjoinapooledJP.
37
FindingsfromQualitativeDeskAnalysis
JPsthatareactivelyjointinallpossibleways–design,governance,implementation,monitoring,reporting,co‐operationwithgovernmentactors,resourcemobilisationandadvocacy–appeartoberare.Theevaluations/reviewsrevealthatmostprogrammesseemtoliesomewhereinbetween“jointbyname”and“fullyandactivelyjoint”.The2006UNDGReviewheldthatwhilejoint“datagatheringanddisseminationexercisespresentanexcellentplatformtointroduceandbenefitfromawellcoordinatedjointprogramme”,onlytoacertaindegreehadUNcountryteamsintheperiod2001‐2005recognizedtheopportunitiesandperceivedbenefitsofworkingtogetherinthismanner.Thus,itconcluded,“jointprogrammeshaveyettooptimallyexploittheUN’spartnershippotential.”ThisconclusionstronglyresonateswiththemorerecentJGPreviewsandwithmanyofthestakeholders’perspectives,suggestingthattheUNdevelopmentefforthasyettoaddresstheissuesraisedbythe2006UNDGreview.SeveralevaluationspointoutthatJGPsoftenresemblealooselyconnectedsetofoutputs(e.g.programmesinVietnam,Guatemala,NamibiaandMauritania.)TheJGPthatappearstooperatewiththehighestdegreeofjointnessisinAlbania.ThereviewofthisJGPnotedthatworkingcollaborativelyandtheinternalcoordinationofagencyactivitiespermitsthefourUNagenciesinvolvedtohavemoresynergiesandagreatercombinedeffect.NotonlywasthecoordinationbetweenUNagenciesmuchimproved,butalsoitwasmoreoverhavingthesideeffectofimprovingcoordinationandcollaborationbetweenlineministries.Nevertheless,theReviewrecommendedthattheUNpartnersshouldconstantlyconsider“howtomakethesynergiesfunctionmosteffectively,andcreate
themostsnowballeffect.”Themid‐termevaluationoftheJGPinTimor‐Leste(MDG‐F)providedatooltoanalysethedegreeofjointnessbasedonthestructureoftheresultsframework.ThishasbeenadaptedandincludedinBox1.
AfundamentaladvantageofJPsisthat,intheory,theyreduceduplication.Withacoupleofexceptions,thiswasnotdiscussedinmostofthereview/evaluations.However,theUNDGJPReviewfrom2006foundevidenceofduplicationbeingavoided.Itstatedthat“featuresuniquetojointprogrammes,suchasjointneeds‐assessment,jointmonitoringandevaluation,collaborativedecision‐making,streamlinedgovernmentdialogueandorenhancedgovernmentparticipationinkeymulti‐agencydecisionmakingbodieshavefacilitatedareductioninduplicativeactivitiesacrossUNagenciesaswellasbetweenUNagenciesandtheirdevelopmentpartners.”FollowingthislogicandgiventhatsomeoftheotherreviewsandevaluationsmaintainedthatmanyofthesefeatureswerenotconsistentlypartoftheJGPs,someduplicativeeffortsmayindeedbehappening–althoughwecandeducethatduplicativeprogrammeshaveprobablybeenreduced. Somestakeholdersreflectedonthenatureof“jointness”intermsofthebalanceofrolesplayedbytheparticipatingUNagencies.OneindicatorofjointnesssuggestedbystakeholdersistheextentthatUNagencies participateasequalsintheprogramme.16NoneofthereviewsandevaluationsanalysethisaspectofthepartnershipsamongUNagenciesinJPs.
TherewasadivergenceinviewsconcerningthevalueofJPs.SomeheldthatJPsareonlyofvalueiftheyimproveefficiencyandeffectiveness.Others,whoregardedJPsasfundamentalforUNreform,sawJPsashavingamoreintrinsicvalue.Most
38
FindingsfromQualitativeDeskAnalysis
Box 1: Determining the Robustness of a Joint Programme Adapted from Mid-Term Evaluation of the MDG-F Joint Programme of “Supporting Gender Equality and Women’s Rights in Timor-Leste”
A Joint Programme involves two or more participating UN Agencies working together to achieve a common result. This common result could, however, be an outcome and/or an output (and even an impact!). One way of determining the robustness of jointness would be from the way a results framework is structured.
At the simplest level, each of at least two participating agencies is responsible for its own outcome (as illustrated by Model 1 below) that is linked to a common UNDAF outcome. Greater jointness is achieved in Model 2, in which at least two participating agencies are each responsible for the delivery of their respective outputs, although these lead to a common outcome.
Jointness is maximised in Model 3 in which at least two participating agencies are responsible for a common output. This means that while the agencies are each implementing their own activities in accordance with their respective expertise and mandates, they are working towards the achievement of a basic result – the output. It follows that the output cannot be achieved if the participating agencies do not complete their activities. While the three models can be considered as joint programmes, it is obvious that the first model is a weaker version compared to the second and the third.
Examples of Jointness: 2 Agencies, 2 Outcomes, 3 Models Result Model 1: Low jointness Model 2: Medium jointness Model 3: High jointness
Outcome 1 Output 1.1 Output 1.2
Agency 1 Agency 1 Agency 1
Agency 1 & 2 Agency 1 Agency 2
Agency 1 & 2 Agency 1 & 2 Agency 1 & 2
Outcome 2 Output 2.1 Output 2.2
Agency 2 Agency 2 Agency 2
Agency 1 & 2 Agency 1 Agency 2
Agency 1 & 2 Agency 1 & 2 Agency 1 & 2
stakeholdersgenerallyheldthatonceaprogrammewasformulatedasjoint,themorejointtheimplementationprocess,thebetter.However,therewerealsosomedifferencesinviewsastowhethermaximumjointnesswasanidealtoaimforinallinstances,orwhetherthecircumstancesgoverningeachsituationwoulddeterminetheappropriatelevelofjointness.Ausefulcontributionoftheevaluationshouldbetodeterminetowhatextentthelevelofjointnessleadstobetterresults.
TheUNDGJPReviewstatedthatJPsknowledgeandexpertisesharing,acommoncommitmentforresultsandamoreopendialogueamongUNcountryteammemberswereanimportantaspectofjointness.Anumberofstakeholdersheldthereweregapsinoverallknowledgemanagementof
JGPs.Untilthisstudy,therehasnotbeenonecomprehensivedatabasecontaininginformationofexistingJGPs.Asthisstudyestablished,dataonJGPshasbeenscatteredindifferentdatabasesornotenteredintoanydatabaseatall.Likewise,repositoriesforreports,evaluationsandreviewsconcerningJGPshavenotbeencreatedfortheJGPcommunity.Howknowledgeismanagedjointlyatcountryandregionallevelswouldbeusefulfortheevaluationtoexamine.
Finally,atopquestionamongstakeholderswastowhatdegreedoJGPsaddvaluetoresults:towhatdegreedoJGPsaddvaluetotheeffortsofstrengtheningtheaccountabilityofduty‐bearersandsupportingrights‐holdersindemandingtheirrights?Whataddedvaluedoesthejointprogramming
16TheJPsfinancedthroughpooledfundingarebydesignlessjointinmanagementterms,sinceoneagencytakesresponsibilityformanagement.Nevertheless,intheory,pooledprogrammesshouldbejointandequalintheirsteeringstructureandinthemanagementoftheknowledgegainedfromtheprogramme.
39
FindingsfromQualitativeDeskAnalysis
processbring?WhataddedvaluedoeseachUNparticipatingagencybring?TowhatdegreedoagenciesseetheirroleasaddingvaluetotheJGP?OnestakeholderheldthatwhenthereisnocommonleadershipinaJPandeachparticipatingUNagencyfollowsparallelplanswithitstraditionalcounterpart,thereisnoaddedvalueintermsofresultsandefficiencies–“ThustheJPrepresentsaclearfailure,whereon1+1+1isnot=3but‐2”.Insum,stakeholdersandevaluations/reviewsdirectlyorindirectlyidentifiedinformationneedsandanumberofquestionsaboutJGPsthatrelatetothedegreeandqualityofthecollaborationamongsttheparticipatingUNpartnersandtheaddedvaluejointnessbringstoresults.Theseincludequestionsconcerningconceptualisation,design,methodology,implementation,accountability,M&E,resourcemobilisation,results,knowledgemanagementandpartnershiprelations:1 Towhatextentwasthereasharedvisionfor
eachJGPamongtheparticipatingagencies?Towhatextentweretheconceptualisationprocessescollaborativeundertakings?Towhatextentdidtheparticipatingagenciesjointlyconductunderlyinganalyses?
2 Towhatextentdotheprogrammedesignsestablishcoherencebetweentherolesoftheagencies?TowhatextentaretheJGPswellthoughtthroughandbuiltonthestrengthsandareasofexpertiseofeachparticipatingagency?Forinstance,haveJGPscapitalisedontheaddedvalueofspecialisedagencies?HowwastheparticipationbythedifferentUNagenciesdetermined?Weretheretrade‐offsbetweencoherenceandinclusion?Towhatextentwereanypotentialrisksborneequallyamongtheparticipatingagencies?
3 Towhatextentweredifferencesamongparticipatingagenciesinmethodologyandapproach(prioritizationofareasandpopulations,methodologyforcommunitymobilization,modalityofdeliveryoftechnicalassistance)identifiedandresolved?
4 Towhatextentare/wereUNagencies
participatingasequalpartnersintheimplementationprocesses?Towhatextentwereimplementationplanssharedandsynchronizedamongtheparticipatingagencies?Towhatextentdoparticipatingagenciesimplementactivitiesjointly?(Oraretheyalooselyconnectedsetofactivities,withoutcoherence?)
5 TowhatextentistherejointaccountabilityoftheJGPs?WhatreportingsystemshavebeenusedbyJGPs?Towhatextenthavetheybeenjoint?
6 Towhatextentisthedesignandimplementationofmonitoringandevaluationeffortsundertakenjointly?
7 Towhatextenthaveagenciesjointlyaddressedknowledgemanagementneedsatcountry,regionalandHQlevels?
8 Towhatextenthasresourcemobilisationbeenjointlyundertaken,capitalisingoneachagency’scomparativeadvantagesinthisarea?
9 Towhatextentthelevelofjointnessleadstobetterresults?TowhatextentdoJGPsdelivercoherentandjointoutputsandoutcomesthatadduptosomethinggreater(suchasacombinedsynergeticeffects)thanaseriesofactivities?Towhatextenthaveadvocacyeffortsbeenundertakenjointlyandtowhatextenthas“jointness”contributedtoanysuccesses?
10 TowhatextentdopartnersperceivetheUNasoperatingdifferentlyundertheJGPs?Whatdopartnersconsiderasmainadvantages?
JGPDesign
TooptimallytakeadvantageoftheUN’spartnershippotential,aJPhastobewelldesigned.AsstatedintheUNDGJointProgrammeReview,aJP’sdesign“shouldreflectdeepercountry‐levelcommitmentandcoordinatedassistanceonascalethatcouldnotbeachievedthroughasingleagencyprojectorcollaborativeactivities.”TheJPsthatwerereviewedbythis2006study“almostunanimouslycalledforincreasededucation
40
FindingsfromQualitativeDeskAnalysis
ontheidentificationanddesignofJPsthroughenhancedUNDGguidanceandthroughtheRCsystematthecountrylevel.”ItfoundthatinsomeoftheearlierJPs,participatingagenciesevendesignedtheirpartoftheprogrammeinisolationoftheothers.Themorerecentevaluations/reviewsalsouncoveredseveralproblemswithdesign.AccordingtotheMDG‐F,theformulationphasehasbeenthemostcommonchallengeforallJPs,regardlessofsector.WhiletheMGD‐Fevaluationsgenerallyconfirmedthealignmentoftheprogrammegoalswithgovernmentpriorities,mosthadoverlyambitiousgoals;somedidnottakeintoconsiderationavailableexpertisenationallyandinternationally;somedesignprocessweretooshort,under‐resourcedortoolong(thegovernmentprioritiesinthemeantimechanged);somehadinadequatelogframes;somedidnottakethelocalsituation/developmentssufficientlyintoconsideration;andsomedidnotinvolvegovernmentpartnerssufficiently.Thereweregooddesignexamplestoo:theEthiopiaevaluationpraisedtheJGPforitsunderlyingdesignandconceptualisation.ItusedsimpleanddirectapproachestoaddressarangeofMDGGoals,therebymakingadeliberatecontributiontothepurposeoftheMDG‐F.WiththeexceptionoftheAlbanianreview,noneoftheevaluationsorreviewscommentedontheextenttheprogrammeshadbeendesignedfromarights‐basedstudy;orwhethertheprinciplesofarights‐basedapproachhadbeenappliedduringthedesignprocess.AlargemajorityofstakeholdersstronglyunderlinedtheimportanceofanalysingprogrammedesignanddesignprocessesofJGPs.BothstakeholdersandevaluatorsdeemedthatthedesignanddesignprocessweredeterminantsofhowjointlyaJPwillbeimplementedinrealityandhowsuccessfultheprogrammewillbeintermsofresults.Asputbyonestakeholder:“gettingitright,doesallstartwiththerightdesign...thekindsofassistancewearedeliveringthroughourjointprogrammesshouldbeintrinsicallydifferenttowhatwedidbefore.Ifitisn’t,theremaybesomethingwrongwiththe
underlyinglogicanddesignofthejointprogramme.”Therewasastrongdesireforgreaterinformationontheconceptualisationprocess:WhatwastheimpetusfortheJP?Dothecomplexityoftheissues,highambitionsandtheexpectedresultswarrantaJPapproach?Doexpectedresults;theintentionstoimproveintra‐UNcoordinationorotherfactorsdrivetheformulationofJGPs?Whatrolesdidthedifferentactorsplay?Whatanalyseswereused?,etc.Theyalsocalledforanalysesregardingtheextenttowhichvariousdesigncharacteristicscontributedtoresults.Questionsthatwereraisedthatrelatetodesign(someoverlapwiththeformer“jointnesssection”)includethefollowing:
1 TowhatextentaretheJGPs’goalsrights‐basedandcoherentwiththeirrespectiveUNDAF?Towhatextentarethegoalsrealistic,i.e.instepwiththeresources,capacitiesandsituationathand?TowhatextentwereJGPsdesignedtobebasedmoreontheavailabilityoffundsasopposedtoneeds?Towhatextentdidthelevelofcomplexity,theambitionsandtheexpectedresultswarrantajointprogrammingapproach?
2 Towhatextenthasthedesignprocessintegratedhumanrightsprinciples?TowhatextentdoJGPsdesignsandinterventionstrategiesencompassarights‐basedapproach?
3 TowhatextentareJGPinterventionstrategieswelladaptedtothesocio‐culturalcontext?WhatactionsdidJGPsenvisagedtorespondtoobstaclesthatmayarisefromthepoliticalandsocio‐culturalbackground?Wereriskassessmentsconductedinthedesignphaseandweremitigationstrategiesadopted?
4 WhatanalysesweretheJGPsbasedon?DidtheJGPsrelyonagenderandrights‐basedanalyses?TowhatextentwereJGPslinkedtoCRCandCEDAWconcludingobservations?Inconflicted‐affectedcountries,wereconflictassessmentsused?Werecapacityassessmentsundertaken?
5 Towhatextentiscapacitydevelopmentof
41
FindingsfromQualitativeDeskAnalysis
rights‐holdersandduty‐bearersacentraltenetofJGPdesign?TowhatextentdoJGPshavecapacitydevelopmentplans?
EffectivenessinTermsofResults
TheJGPevaluationsandreviewsreportsomeachievements,butmostoftheseassessmentshavenotbeenundertakenattheendorfinalstagesofaJGP,whenmosteffectsarebestassessed.Furthermore,thereislittleanalysisonhowthe“jointness”mayhaveaddedvaluetotheresults.Would,forinstance,fiveagenciesthatprogrammedseparatelyhaveledtosimilarresults?GivethegreatpaucityoffinalevaluationsofJGPs,stakeholdersgenerallygavetopprioritytothatthefutureevaluationprovidesinformationonresults.
Thediscussiononresultswithstakeholdersandtheanalysisofeffectsintheevaluation/reviewreportshavefocusedonseventypesofeffects.Thesearei)genderequality,women’sempowermentandhumanrightseffects,ii)capacity‐buildingeffectsamongduty‐bearersandrights‐holders,iii)advocacyeffects,iv)processresultsfromanRBAperspective,v)intangibleeffects,vi)synergeticeffects,andvii)goodpractices.Theseseventypesarediscussedbelow.
Tobeginwith,basedontheuniquelegitimacyofitsuniversalmembershipandonitsdiverserolesasastandard‐setter,capacity‐builderandadvocate;theUNhasauniqueroletoplayinpromotingGEWEindevelopingcountries.StakeholdersrosethequestiontowhatextentJGPshavecapitalisedonthistoproduceenhancedeffectsinrelationtogenderequality,women’sempowermentandwomen’shumanrights.Likewise,informationwassoughtregardingtheextenttowhichtheobjectivessetoutbytheBeijingPlatformforActionandMillenniumDeclarationarebetteraddressedwhenUNagenciescollaborateinaJGP.
CapacitydevelopmentisacorefunctionoftheUNdevelopmentagenciesandacentraltenetofachievingtheMDGs.Capacitydevelopmentisfurthermorevitaltothehumanrights‐basedapproach,whichinvolvesbuildingthecapacityof
duty‐bearerstofulfiltheirobligationstowardsrights‐holders,aswellasthatofrights‐holderstodemandtheirrightsfromduty‐bearers.The2003GuidanceNoteonJointProgrammesstatesthatJPsshouldhaveafocusoncapacitybuilding.Nevertheless,theUNDGJointProgrammeReviewfrom2006foundedthatofthe21JPsreviewed,onlynineoftheJPsexplicitlyaddressednationalcapacityneeds.TheJGPevaluations/reviewsrevealmixedresultsintheareaofcapacityenhancementandmonitoringofcapacitybuildingefforts.Forinstance,follow‐upoftraininginitiativeshasgenerallybeenpoor.Interviewedstakeholdersandtheevaluationssuggestthatabetterunderstandingofwhatresultshavebeenachievedincapacitydevelopmentisneeded.
Arights‐basedapproachentailsthatapersonisasubjectofhisorherrightsandanactiveparticipantinhisorherdevelopment.Thussincearights‐basedapproachaimstocontributetothepracticalityandactiveenjoymentofrights,therealisationofhumanrightsisbothanoutcomegoalandaprocessgoal.Withregardtotheformer,itisimportantforthefutureevaluationtoexaminetowhatextentrootcausestogenderinequalityhavebeenaddressedbyJGPs.ItfurthermorepertinenttoassesstheextenttowhichgloballyacceptednormsandstandardsintheareaofGEWEhavebeenperpetuatedwithsupportfromtheJGPsinlegislativework,statisticalworkandtrainingefforts.
Humanrightsprocessgoalsinvolveapplyingtheprinciplesofparticipation,equality,non‐discrimination,accountabilityandtheruleoflawthroughoutthedesign,implementation,monitoringandevaluationprocessesofJGPs.Whilesomeoftheevaluations/reviewscommenttosomeextentonresultsthatfurthertherealisationofhumanrights,noneoftheevaluations/reviewsassessthequalityoftherights‐basedapproachesorcommentontheattainmentofprocessgoalsassuch,withtheexceptionofthereviewoftheJGPinAlbania.
Aspointedoutbystakeholders,JGPscreatespaceforpoliticaldialogueandofferanexcellentplatformtojointlyadvocatewithonevoiceonthecrosscuttingissuesofGEWE.Thequestionis
42
FindingsfromQualitativeDeskAnalysis
whethertheseopportunitieshavebeenfullyexploitedandwhethertheyhaveyieldedresults.Theevaluationsdonotofferenoughclues.Whileafewmentionthatadvocacyeffortshavetakenplace,noneanalysewhetherjointeffortshaveenhancedadvocacyeffects,exceptfortheAlbaniaevaluationwhichstates:“Advocacyeffortshadbeenstrengthenedon(domesticviolence)thankstotheunifyingoftheeffortsofthefourUNagencies,increasedinformationexchangeleadingtostrongandclearadvocacymessages.”Bothstakeholdersandevaluation/reviewsconsideredintangibleeffectsofJGPs.TheUNDGReviewheldthat:
Knowledgeandexpertisesharing,acommoncommitmentforresults,andamoreopendialoguebetweenUNcountryteammemberswereoftennotedinthecasestudiesassignificantoutcomesofjointprogrammes.Thecasestudieshaverevealedavaluableinterplaybetweentheme/workinggroupdiscussionsandthedevelopmentandimplementationofjointprogrammes.Thecasestudieshavealsodemonstratedenhancedlearningbetweengovernmentministriesasaresultoftheirparticipationinjointprogrammemanagement.
ItfurthermorenotedthatattimesparticipatingagenciesinJParrangementfoundvalueinthepartneringprocessitself,believingthattheimprovedworkingrelationshipsandcollaborationamongtheUNCTgenerallyenhancestheUN’ssupportofnationalgovernmentobjectives.Likewise,someoftheMDG‐FevaluationsmentionthattheJPprocesshasboostedtrustamongagencies,improvedinter‐agencyknowledgeandenhancedcommunications.Somestakeholderswereinterestedinwhattypesofintangibleeffectswereachieved(improvedcommunication,bettercollaboration,moreinformation‐sharing,cross‐agencylearning,strongerUNspirit,accesstowidernetworks,etc.)andwhatbenefitstheybring.InthecontextofUNreform,areJGPscontributingtoanewcultureofcollaborationamongUNagencies?SomewonderedifJGPssometimesdisempoweredanagency,thuscreatinganoppositedynamic.
ReformoftheUNdevelopmentsystemtopromoteeffectivenessandsustainabilityfocusesoncoherence,co‐ordinationandcollaboration.Thisincludesnotonlywithin(joint)programmes,butalsoamongprogrammes.ManystakeholderswereextremelyinterestedinwhetheraJGPinacountrycanresultinsynergeticeffectswithotherUNprogrammesatcountrylevel,particularlythoseinothersectors.TheywantedtoknowwhetherJGPshaveaninfluenceongenderequalitymainstreaminginotherUNprogrammes.Furthermore,stakeholdersquestionwhetherJGPshaveaffectedincreasedcollaboration,coordinationandinformationexchangewithintheUNCTinrelationtoGEWE.TheAlbaniareviewnotedthatmoreattentioncouldbepaidtoensuringthattheotherJPsinthecountryintegrategenderequalityconcernsandoperatesinsynergywiththeJGP.Itsuggested,forexample,thatworkintheeducationsectoroninclusiveeducationcouldeasilycoverthespecificissuesrelatedtogirlshighlightedbyCEDAW.WorkonthesocialprotectionsystemneedtobeeffectivelyenmeshedwiththeJGP’sworkonprotectingwomenfromdomesticviolence.Stakeholders–particularlythosebasedincountryoffices–sawagreatneedforinformationongoodpracticestobecollected.Someoftheevaluations(e.g.OPT,NamibiaandEthiopiaJGPmid‐termevaluations)pointouttheexistenceofinnovativeapproaches.Wouldtheseinnovationshavebeenlikelyforasingleagencyprogramme?AlsoofinterestamongstakeholderswastheextenttowhichJGPshavehadpotentialforreplicationand/orscalingup.
1 TowhatextentandinwhatwaysareJPsaddingvalueandcontributingtotheobjectivessetbytheBeijingPlatformforAction?IsthereevidencethatJGPsaddressGEWEmoreeffectivelythansingleagencies?
2 TowhatextenthaveJGPscontributedtocapacitydevelopmentofbothrights‐holdersandduty‐bearers?HavetheJGPsbeenabletooffercontextuallyrelevantandhighquality
43
FindingsfromQualitativeDeskAnalysis
genderequalitymainstreamingtrainingtoseniorpolicymakers,focalpersonsandNGOexecutivestaff?
3 TowhatextenthastheUN’sroleinadvocatingforthenationalapplicationofinternationalnorms,standardsandactionsonhumanrightsandglobalissuesimpliedintensifiedeffortsinpolicyadvisoryservicesthroughJGPs?DoJGPsperpetuateinternationalnormsandstandardsinlegislativework,statisticalworkandtrainingefforts?Towhatextenthavehumanrightsprocessgoalsbeenachievedbyapplyingtheprinciplesofparticipation,equality,non‐discrimination,accountabilityandtheruleoflawthroughoutthedesign,implementation,monitoringandevaluationprocessesoftheJGPs?
4 What,ifany,jointadvocacyeffortshaveJGPsundertaken?Whathavebeentheeffects?
5 Towhatextenthaveintangibleeffectsbeenachieved?TowhatextenthaveJGPscontributedtointer‐agencynetworking,informalinformationexchange,aconstructiveteamspirit,aconsciousfeelingofbeingamemberofoneUNfamily,etc.amongtheUNagencies?TowhatextenthaveJGPsledtoimprovedcommunication,synergies,coordinationandcollaborationamongthedifferentnational‐levelimplementingpartnerorganisations(includingCSOs)andamonglineministries?TowhatextenthaveJGPsenhancedcommunicationbetweentheUNandgovernments?
6 TowhatextenthaveJGPsachievedsynergeticeffectswithotherUNprogrammesinothersectorsorareasofwork?
7 Whatgoodpracticeshavebeenidentified?TowhatextentdoJGPshavepotentialforreplicationandupscaling?TowhatextenthaveJGPsmadeeffortstopromotefutureupscaling?
Sustainability,NationalLevelPartnerships,NationalOwnership,&PeopleCentredApproaches
JointprogrammesdonotexclusivelybelongtotheUN–theyalsobelongtothenationalduty‐bearersandrights‐holders.TheHighLevelPanelreport“DeliveringasOne”statedthatreformtoimprovethecoherenceoftheUNsystemmustbeunderpinnedbytheprinciplesofnationalownershipandpeople‐centredapproaches:
Nationalsovereigntyandnationalownershipofdevelopmentplansmustremainthebedrockofeffectivedevelopment.Thesystemmustberealignedtoademand‐drivenapproachandtoprogrammesdeliveredasclosetobeneficiariesaspossible.
TheAidEffectivenessprinciplesoftheParisDeclaration(ownership,alignment,managementfordevelopmentresultsandmutualresponsibility)andtheAccraAgendaforActionfurtherplaceownershipatthecentreofdevelopmentco‐operation.Nationalownershipisconnectedwithsustainability.Withoutanadequatelevelofownership,aprogramme’ssustainabilityislikelytobelower.However,aspointedoutbyonestakeholder,sustainabilityisnotguaranteedbyownership.Thissectionwilldiscussinformationneedsrelatingtoownership,sustainability,theUNpartnershipwithgovernmentsandpeople‐centredapproaches.Thereviews/evaluationsprovideamixedpictureofnationalownership.Therearereviewsthatstatethatnationalownershipisbeing“consciouslyrecognizedandexercised”(TimorLeste,Colombia)andothers(Namibia,NicaraguaandBolivia)thatassessedthelevelofownershiptobegenerallyinsufficient.InNamibiatheJGPhascreatedstructuresthatrunparalleltothegovernmentstructuresandmostgovernmentpartnersfelttheyhadlimitedownershipandpowertoinfluencethekeydecisionsintheprogramme.OneoftheJGPreviewsprovidessomeinsightsonownership–aconceptthatitfoundwasoftenunderstoodtomeansomethingdifferentbydifferentstakeholders.Itcontemplatedownershipasfollows:
• Stakeholdersfeltthattherewasownership
44
FindingsfromQualitativeDeskAnalysis
becausegovernmentpartnershadbeeninvolvedandconsultedfromthebeginning.However,consultingandinvolvementdoesnotleadtoowning.Ontheotherhand,thereportconcludedthatjointdecision‐makingisanindicatorofjointownership.
• WhiletheJGPwasfirmlyembeddedingovernmentstrategiesandlegalframeworks,thereviewquestionedwhetherthisactuallyrepresentedgenuineownershipwhenbudgetswerenotallocatedaccordingly.
• Evenwheretherewassolidpoliticalwill,ownershipwastosomedegreelimitedbycapacityandresources.Poorlybackedgenderfocalpointsinministrieshadlittlemotivationandlimitedcapacity–particularlysincetheyhaveotherresponsibilitiesbesidesgenderequality.
Totakeownershiprequiresnotonlycommitmentbutalsosomedegreeofcapacity.InVietnam,theJGPwasdesignedwithoutadequateattentiontothecapacitiesneededtocarryitsobjectivesout.TheJGPinTimorLestefacedaconstraintinbalancingtheelementofnationalownershipwiththemanagementefficiencyoftheprogramme.TheissuesthatlendthemselvestobeingaddressedthroughJPsarehorizontal,complexandinter‐disciplinarylikeGEWE.However,thesearepreciselytheissueswheregovernmentsinthenorthandsouthfacesignificantchallenges.Thishasimplicationsforinterpretingnationalownershipandhowtoapproachcapacitybuilding.Asstatedbyonestakeholder,“Youneedtobuildcapacitynotonlyforimplementationofthecomponentsoftheprogrammeinlineministries,butalsothenationalpolicyandstrategydevelopmentthatneedstobringitalltogether...sothereisthisbiggerdimensionbehindeveryJP–irrespectiveofthesector–whichrelatestohowgovernmentsaddresscomplexpolicychallengesthatcutacrossthecabinettable.”TheevaluationsandreviewsgenerallyassessedtheJGPsasveryweakinrelationtofuturesustainability–althoughsomeevaluatorssawsomepotentialforsustainability(OPTandNamibia).Anumberofreviewssawaneedforaninstitutional
andfinancialsustainabilityplanaswellasexitstrategies.Onestakeholderbelievedthefutureevaluationcouldshinelightontheenablersanddisincentivesforsustainabilityandtoassesstowhatextentgovernmentownershipandleadershipmakeadifferenceforsustainability.Does,forinstance,leadershipfromagovernmentmakeadifferenceforsustainability?Afewstakeholdersfounditveryrelevanttostudyhowpeople‐centredtheJGPswere.Towhatextenthadcivilsocietyandrights‐holdersbeeninvolvedinthedesignandimplementationprocesses?DidtheJPimplementationapproachpromotecivilsociety?Or,asisfearedbyacoupleofstakeholders,doesitdetractcivilsocietyinvolvement?ThereviewoftheJGPsin,forinstance,OPTconcludedthatwhilenon‐governmentalactorswhereinvolvedintheprogramme,theyhadnotbeeninvolvedindecisionprocessesandthereforeameaningfulparticipationofnon‐governmentalinstitutionswasanelementthatcouldbefurtherimproved.Meanwhile,inNamibia,thereviewfoundthatwhilecivilsocietyactorshadbeenconsultedduringthedesignphase,theyhadbeenleftoutduringtheimplementationprocess.AccordingtotheUNDGGuidanceNote,JPsarespecificallyintendedtostrengthenhowtheUNorganizationsprogrammejointlywithgovernments.IftheJGPisstructuredsothatthegovernmenthasalessfragmentedinteractionwithitsUNpartners,isthisconducivetogreaterownership?Orisitpossiblethattheoppositeoccursinsomecases?OnestakeholderreportedthatsomegovernmentshaverealisedthatsinceJPspromotedcoordinationamonggovernmentagenciesandUNagenciesalike,itpavedthewayforeffectivenationalleadership.However,thiswasnotthecaseinmanycountries,andsomenationalstakeholdersmayevenpreferafragmentedapproach.AfewreviewscommentedonthatengagingwiththeUNagenciesinaJGPreducedtransactioncostsforgovernments,butthiswasnotconfirmedbyallreviews/evaluations.Theinformationneedsthattheevaluationcouldaddressinthisareaarelistedbelow:
45
FindingsfromQualitativeDeskAnalysis
1 Howstrongisgovernmentownership?TowhatextentaretheJGPsalignedwithgovernmentprioritiesandmeetingeffectivedemand?Towhatextentdogovernmentsexercisejointdecision‐makingwithUNparticipatingagenciesandprovideproactiveleadership?Towhatextenthavegovernmentsbeeninvolvedintheconceptualisationprocessandconsultedalongtheway?Towhatextenthavegovernmentshowncommitmentbyprovidingresources(financialand/orin‐kind)totheJGPs?
2 Towhatextentwerethecapacitiesofgovernmentandparticipatingnationalagenciescarefullyconsideredinrelationtotheirrespectiveabilitytocoordinate,manageandprovideinputs(cash,supplies,in‐kindortechnicalexpertise)?Towhatextenthaveoperatingcapacitiesbeencreatedand/orreinforcedinnationalpartners?TowhatextenthaveJGPsbeenfacedwithbalancingnationalownershipwiththemanagementefficiencyoftheprogramme?
3 TowhatextenthavethetargetpopulationstakenactiverolesinJGPdesignandimplementationprocesses?Whatrolehascivilsociety–inparticularwomen’smovements–playedintheplanning,design,implementationandmonitoringofJGPs?TowhatextentareJGPsconducive/unfavourablewithregardtoinvolvementofcivilsocietyactors?Towhatextenthavepublic/privatenationalresourcesand/orcounterpartsbeenmobilizedtocontributetoJGPs’objectiveandproduceresultsandimpacts?
4 TowhatextentdoJGPshaveexitstrategiesthataregearedtowardsustainablephase‐outofactivities?Arethenecessaryfoundations(leadership,commitment,capacitiesandresources)inplacetoensurethesustainabilityoftheresultsofJPs?
EfficiencyandOperationalEffectiveness
CentraltotheaimofpromotingJPswithinthecontextofUNreformwastoreducetransactioncostsforallpartners(i.e.improveefficiency)andincreaseeffectiveness.Stakeholdersandreviews/evaluationsoftencommentedonwhetherinJGPsinfactlowertransactioncosts.Whilemanyheldthattransactioncostsarelikelytobereducedinthelongrun,transactioncostsforindividualagenciesoftenappeartobehigherinaJP,particularlyinthebeginning.However,severalstakeholderspointedoutthatiffivesingleUNagencyprogrammesarecomparedwithoneJGPwithfiveUNagenciescollaboratingjointly,thetotalleveloftransactioncostsforthelatterwouldmostprobablybelessthanfortheformer.ThereisnodoubtthatmostJGPshavefacedanumberofchallengesthathaveaffectedefficiency,timelinessandoperationaleffectiveness.Belowaresomeofthechallengesthathavebeenraisedatleastthreetimesbythedifferentevaluations/reviewsand/orstakeholdersconsulted:
• InsufficientguidancetoUNCTs(atleaseinitially)onhowtoformulate,setupandimplementJPs(Therearenotmanymore
availabletools.)• Longformulationprocess–notleasebecause
ofinexperienceandinsufficientguidance–
affectedtimeliness.• Turnoverofstaffandtime‐consuming
recruitmentprocess.
• HighturnoverofJPCs.Thiswassometimesduetothedifficultroleheorshehasbeenfaced
withregardingcoordinatingthedifferentagencies.Insufficientguidanceandstatusvis‐à‐visthepartnerswerealsomentionedasa
problem.• Lackofharmonisedreportingrequirements,
systemsandproceduresamongtheUN
agencies.Thiscausedslowreleaseoffunds.• Lackofclarityregardingrolesinthe
programme–JPCs,programmemanagement
46
FindingsfromQualitativeDeskAnalysis
unit,thevariouscoordinationmechanisms,RCs.
• InsufficientleadershipfromRCsandtheheads
ofagencies.Sometimesnotallofthegovernanceandcoordinationstructures–particularlyatthestrategiclevel–have
functionedasplanned.• Sub‐standardlogframesandindicators.• Poorqualitymonitoringandevaluation
systems.• Accountabilityandreportingpracticesthatdid
notsufficientlycaptureeffectsoftheprogrammeasrequiredbyRBM.
Stakeholders all agreed that these challenges were not unique to JGPs, but common to many JPs, regardless of sector. Initial reviews of the mid-term evaluations of JPs supported by the MDG-Fund show a substantial level of similarities in relation to the challenges of achieving efficiency and operational effectiveness. The meta-evaluations currently being commissioned by the MDG-F are expected to collate these findings in each sector and present an overview. Perhaps one feature related to operational effectiveness that may be unique to JGPs is that because gender equality and women’s empowerment affords lower status, the UN often relies on its junior staff to work in this area. This was raised by a couple of stakeholders and one review which stated that the JGP “relied from the UN side to a large extent on junior staff, particularly interns and
UN Volunteers, among whom there is relatively high turnover, meaning a lack of consistency in dealings with the government” (Vietnam.) Stakeholders generally gave less priority to efficiency aspects and operational effectiveness for this evaluation, with the exception of a few JPCs who desired more good practice examples on efficient management of JPs. While these aspects were considered important, it was felt that a priority for this evaluation was to focus on results, national level partnerships, design of JGPs and their level of jointness. Furthermore, the issues of efficiency and operational effectiveness would be more relevant to address in a system-wide cross-sectoral study. It was a concern that the evaluation maintains its focus on the effectiveness of JGPs producing GEWE results and not be taken over by systemic efficiency issues that relate to all JPs in all sectors.
“Theprogrammeiscaughtinacycleofdelaysinreportingandfundsdisbursementresultinginrushedimplementation,rushedliquidationandworkoverloadforstaff.Therearedelaysinreportingandinscheduleofimplementation.”–2010MDGMid‐TermEvaluationofJGPinEthiopia
47
EmergingPriorities&ReflectionsfortheFutureTermsofReference
Basedontheanalysesofthepreviouschapters,thischapterpresentsthestrategicprioritiesthathaveemerged.Italsodiscussesevaluabilityandpresentsconsiderationsandrecommendationsforthedraftersofthetermsofreference.
STRATEGICPRIORITIES
Itisparticularlyimportantforalarge‐scale,strategicandjointevaluationlikethisonethatsignificanteffortsaremadetoensureclear,focusedandpurposefultermsofreferencetoguidetheevaluationteam.Thismeansthatprioritisationsneedtobemade.Fortunately,thisstudyhasdemonstratedthatthereisconsiderableconcurrenceregardingtheoverallprioritiesfortheevaluation.First,thereisacommonperspectiveontheuseoftheevaluation.WhileitwillbeusedtorenderjudgmentabouttheoverallmeritorworthofJGPs,theprincipleuseswillbetofacilitateimprovementsandgenerateknowledge.Theseusesshouldguidethescopeandapproachoftheevaluation.
Second,theanalysisrevealsthattheprioritiesfortheevaluation’sstrategicscopeconvergeonthreeareas.Inrelationtotheseareas,thedatasuggestthateffectiveness,sustainabilityandpossiblyimpactarethedominantevaluationcriteriatoassesstheJGPs.Relevanceissuesarelessprominentbutstillpertinent.Efficiencyissueswereconsideredlessimportant.Theareasofconvergencearediscussedinthesectionsthatfollow.
ResultsandAddedValueofJGPs
ThefirstareaofconvergencerelatestowhetherJGPsareeffectiveinproducingresultsand
how/whethercollaboratingtogetheraddsvaluetotheseresults.ThecallforinquiryintothisareacomesfromacombinationofUNpolicydirectives,UNWomen’smandate,thelackofevaluativeevidenceandseveraltypesofinformationneedsthatstakeholdersconsultedbythisstudyhaveexpressed:
• ThereareseveralGeneralAssemblyresolutionsmentionedinsection2thaturgetheUNdevelopmentsystemtoenhanceaccountabilityintheareaofgenderequalityandwomen’sempowerment.
• Thepolicydirectivesoutlinedinsection2haverecommendedthattheUNdevelopimprovedguidanceonthenature,qualityandeffectivenessofJPsinsupportofgenderequalityandtheempowermentofwomen.
• ThereisarelativepaucityofstrategiclevelassessmentsofspecificUNeffortstoaddresswomen’sempowerment,women’srightsandgenderequality.
• ThereisalackofevaluativeevidencerelatingtoJGPs.
• WithitsmandatebeingtoleadandcoordinatetheoveralleffortsoftheUNsystemtosupportthefullrealizationofwomen’srightsandopportunities(i.e.promotingcoherenceandactingasaglobalbrokerofknowledgeandexperience),UNWomenneedsevidencetoinformitspolicydevelopment.
• StakeholdersconsultedbythisstudyexpressastrongneedforinformationonthedegreeandnatureofcollaborationamongstparticipatingUNpartnersinJGPsandhowJPsaddvaluetoGEWEresults.
48
EmergingPriorities&ReflectionsfortheFutureTermsofReference
• StakeholdersprioritisetheanalysisofJGPdesignanddesignprocessesbecausei)stakeholdersconsiderJGPdesignanddesignprocessesasdeterminantsofsuccessfulresults,andii)theJGPevaluations/reviewshaveidentifiedseveralJGPdesignproblems.
• Stakeholdersdesireinformationoneffectivenessinrelationtoseveraltypesofeffectsincludingi)genderequality,women’sempowermentandhumanrightsresults;ii)capacitydevelopmentamongduty‐bearersandrights‐holders;iii)advocacyeffectsiv)processresultsfromanright‐basedapproachperspective;v)intangibleeffects;vi)synergeticeffects;and,vii)goodpractices.
ThisareaofconvergencecouldbecapturedbyanoverallaimofassessingtowhatextentandinwhatwayscollaboratinginaJGPhasenhancedtheGEWEeffectsachievedbytheparticipatingUNagenciesandtheirpartners.ThiswouldplaceeffectivenessintermsofGEWEresultsinthecentreoftheevaluationandwouldlinkitwiththeconceptofcollaborationor“jointness”.Itwouldrequirethestudyofthenatureanddegreeofjointness(indesignprocesses,implementationprocesses,governance,resourcemobilisation,communication,knowledgemanagement)andwhatkindofcollaborationcontributedtobetterresults.Itwouldalsoentailstudyingthedifferenttypesofeffects;thestrengthsandweaknessesofJGPsinrelationtoproducingeffects;howtheUNcouldimproveJGPssothattheyaremoreeffectiveinproducingresults;andwhethertherearegoodpracticestolearnfrom.
Sustainability,NationalLevelPartnerships,OwnershipandPeople‐centredApproaches
Thesecondareawhereprioritiesconverge
relatestosustainabilityandhowtheJGPsinteractwithandsupportstakeholdersatthecountrylevel.Itinvolvesnationalownership,people‐centredapproachesandUNpartnershipswithgovernment.ThereareanumberofpolicyleveldocumentsthatareconcernedwiththeseaspectsandthequestionistowhatextentJGPsaresuccessfulataddressingthem:
• UNDG’sGuidanceNoteonJointProgrammingstatesthatJPsareaimedtoenhancetheUNcontributioninthecurrentcontextofinternationaldevelopmentassistance,withafocusonself‐relianceandcapacitybuilding.
• TheAidEffectivenessprinciplesoftheParisDeclarationandtheAccraAgendaforActionplaceownershipatthecentreofdevelopmentcooperation.
• TheHighLevelPanelreport“DeliveringasOne”statedthatreformtoimprovethecoherenceoftheUNdevelopmentsystemmustbeunderpinnedbytheprinciplesofnationalownershipandpeople‐centredapproaches.
• UN’scommitmenttomainstreaminghumanrightsinallofitsdevelopmentworkrequiresanapproachofstrengtheningtheaccountabilityofduty‐bearersandsupportingrights‐holdersindemandingtheirrights.
• TheUNDGGuidanceNotestatesthatJPsarespecificallyintendedtostrengthenhowtheUNorganizationsprogrammejointlywithgovernments.
Atthesametime,stakeholders’informationneedsandthefindingsoftheJGPevaluations/reviewscallforinquiryintothisarea:
• StakeholdersrequiremoreanalysisofwhatJGPsmeanforownershipandjoint
49
EmergingPriorities&ReflectionsfortheFutureTermsofReference
processesinthepartnershipsthattheagenciesenjoywithgovernmentsandcivilsociety.
• Theevaluations/reviewsofJGPsconcludedthatsustainabilityofJGPswaslow.
• Stakeholdersrequireinformationandanalysisofhowsustainabilitycanbeimproved.
ThisareaofconvergencecouldbecapturedbyanoverallaimofassessingtowhatextentandinwhatwaysJGPshavecontributedtogovernmentsmeetingtheircommitmentstotheBeijingPlatformforActionandfulfilledtheirobligationstowardswomen’sandgirl’shumanrights,whilealsosupportingrights‐holdersdemandtheirrights.ThiswouldsettheUN’snationallevelpartnershipswithduty‐bearersandrights‐holdersattheheartoftheevaluation.Itwouldcovertheissuesofnationalownership,howeffectivelyandsustainablytheUNagenciesprogrammejointlywithgovernmentsandtheextenttowhichJGPapproachesarepeople‐centred.
Synergies
ThethirdandmuchsmallerareaofconvergencerelatestosynergiesbetweenJGPsandotherUNefforts:
• ReformoftheUNdevelopmentsystemtopromoteeffectivenessandsustainabilityfocusesoncoherence,coordinationandcollaboration–notonlywithinprogrammesbutalsoamongprogrammes.
• ManystakeholdersshowaveryhighdegreeofinterestinunderstandingwhetherandhowaJGPinacountrycanresultinsynergeticeffectswithotherUNprogrammesatcountrylevel.TheywanttoknowwhetherJGPshavean
influenceontheUN’soverallgenderequalitymainstreamingefforts.
• MainstreaminggenderequalityintoallUNprogrammespresentsmostsignificantchallengesformostagencies.ManystakeholdersseeJGPsasaresourceformainstreaming.
• MostoftheJGPevaluations/reviewsdonotreportonsynergiesandthefewthatdidfoundthereweremissedopportunitiestocreatethem.
ThisareaofconvergencecouldbecapturedbyanoverallaimofassessingtowhatextentandinwhatwaysJGPshavecontributedtoimprovedgenderequalitymainstreamingandwomen’sempowermentinotherUNprogrammesandeffortsatcountrylevel.ThefocusherewouldbeonsynergeticeffectswithotherUNefforts.ItwouldrequirestudyingtowhatextentJGPsaffectedincreasedcollaboration,coordinationandinformationexchangewithintheUNCTinrelationtoGEWE.
EVALUABILITY
Programmesaddressinggenderequalityandwomen’sempowermentareinherentlydifficulttoevaluatesincetheyconcernchallengingandchangingcomplexsocietalnormsanddynamics.Giventhattheevaluationscopeisglobal;involvesmultipleagencies;arangeofsectors/thematicareas;andspans10years,thesubjectareapresentschallengeswithregardtoevaluability.Lowevaluabilitynotonlyimpliesdifficultiesduringtheevaluationprocess,itmayincreasecostsbecausemoreeffortandresourcesareneededtogatherdata.Thissectionpresentssomeofthechallengestoevaluabilityandhowtheycanbeaddressedbytheevaluation.
AssessingResults
50
EmergingPriorities&ReflectionsfortheFutureTermsofReference
Aswithallprogrammesthataimatresultsthatconsistofsocialchange,determiningcausalityandattributionofresultsbeyonddirectoutputspresentsdifficulties.Tosaywithconfidencethataninterventioncausedchange,evidencehastobeproducedthatshowsthattheinterventionactuallycausedthechange(i.e.nothingwouldhavechangedintheabsenceoftheintervention–theinterventionwasnecessary),andtheinterventionwastheonlycauseofthechange(nothingelsewasneededtobringaboutthechange–theinterventionwassufficient.)
TheJGPevaluations/reviewsindicatethatJGPsusuallyhavesuboptimallogframes,indicatorsandmonitoringsystems.Thisislikelytopresentobstaclestoassessingeffectivenessanddeterminingcausalityinthecaseofintermediateoutcomes.However,becauseofthecomplexityandfluidityofdevelopmentprocesses,causalitymaybedifficulttodetermineeveninthebestofcircumstancewherethereisqualitybaselineinformationandrobustmonitoringframeworks.WhenitisnotpossibletoconcludewhatoutcomescanbedirectlylinkedtoaJGP,itwillbenecessaryfortheevaluationtodeterminewhetherthereisevidencethatsuggeststhataJGPiscontributingtoorhasthelikelihoodofattaininglonger‐termgoals;andwhethernecessarypreconditionsforsuccessfulresultsexists.
Anotherissuerelatedtoevaluatingresultsisthatthisstudyhasdocumentedthatthereisadesiretoobtainknowledgeofarangeofeffects–synergeticeffects,intangibleeffectsandeffectsrelatedtocapacitydevelopment,humanrightsandempowerment.Identifying,analysingandassessingthesedifferenteffectswillrequireanumberofdifferenttechniquesandapproaches.
Assessing“Jointness”
Evaluatingtowhatextent“jointness”enhancesresults,wouldideallyrequirethatcontrolprogrammesbeidentifiedsothatpairsofGEWEprogrammes–onejointandtheother“single”–becompared.Unfortunately,itisunlikelythattwocomparableprogrammescanbefoundinthesamecountry,beingimplementedatthesametimeinthesamethematicarea.However,bydrawingontheknowledgeandexperienceofUNstaffandpartnerorganisations,itwouldbepossibletoreconstructhowasingleprogrammemighthavebeendifferentfromaJP.Participatorytechniques–suchascollectiveanalysiscouldbeusefulinthisregard.
IntegratingHumanRights
FormostoftheJGPs,humanrightsand/orgenderequalityaretheprimaryfocusesoftheintervention.AlthoughitisbeyondthescopeofthisstudytodeterminetheevaluabilityofeachJGPindetail,basedontheinterventionlogicandgoalstructure,itappearsthatasignificantproportionoftheportfoliomayhavemediumorhighevaluabilityforintegratinghumanrightsandgenderequality.Nevertheless,someJGPs–suchasthosefromwithinthehealthsector–
Recommendation1:ThetermsofreferenceshouldstipulatethattheevaluationteampresentshowitwillassessthedifferenttypesofJGPseffectsinitsmethodology.
Recommendation2:Thetermsofreferenceshouldrequestthattheevaluationteamconsiderwaystoassesstheaddedvalueof“jointness”initsmethodology.
51
EmergingPriorities&ReflectionsfortheFutureTermsofReference
havenotconsistentlyconsideredhumanrightsdimensionsinthedesign,implementationandmonitoring.Inthesecasesitwillbemorechallengingtointegratehumanrightsandgenderequalityintheevaluationprocessandanalysis.
Toaddresstheevaluabilitychallengesintermsofintegratinghumanrights,ahumanrights‐basedstakeholderanalysiswillbecriticaltoensurethatstakeholderswhomaynothavebeenconsideredintheinterventionareincludedintheevaluation.Theevaluationteammayalsoneedtoseekinformantsanddocumentsthathaveusefulinformationonhumanrightsthatmaynothavebeencapturedbytheintervention(e.g.statisticsofficers,otherdevelopmentagencies,civilsociety,academia,etc.)Aglobalevaluationspanning60countrieswillentaillimitationstotheamountofstakeholderparticipationthatispracticallypossible.Itwill,however,bepossibletoensureanacceptablelevelofparticipationinthecountrycasestudies.Theevaluationteamislikelytorequiresupportfromthecountryofficestohelporganisemeaningfulparticipationofdutybearersand/orrightsholdersintheevaluationprocess.
DataandTimeLapse
Fundamentaltoanyevaluationistheavailabilityofdata.TheteamhasmadeasubstantialefforttoobtainasmuchdataaspossibleontheJGPportfolio,butsomedatagapsremain.Inparticular,thereisinsufficientdocumentationabouttheJGPsthatwereinitiatedbefore2006.
Justoverhalftheprogrammesfrom2001to2006lackprogrammedocuments.ItisevenpossiblethatsomeoftheseJPswereneverinitiatedorchangedintosomethingelse.
MostoftheearlierJGPsaresmall‐scale–80%areunder$600,000andmorethanonefifthareunder$140,000.Thetimelapssincethefirstpartofthedecadecreatesacoupleofimportantchallenges.First,smallprogrammesarelesslikelytoproduceeffectsthatareidentifiableyearslater.Second,thefactthatmostoftheseprogrammesendedyearsagoentailthattherewillbelessinstitutionalmemorytotapintofordata.
Ontheotherhand,theteamhasobtainedprogrammedocumentsfornearly90%oftheJGPsthatwereinitiatedbetween2006and2010.Afutureappealtothecountryofficesislikelytoyieldmoredocuments.
Since2001,therehavebeensomechangestotheinstitutionalenvironmentofJPs.First,fouroftheearlierJGPsweredesignedbeforethe2003GuidanceNoteonJointProgrammeswereissued.Second,from2007onwardsthefollowingchangestookplace:
• Harmonisationofaccountingstandards,businesspracticesandhumanresourcesmanagementaswellasfurtheralignmentoftheUNDAF;
• FurtheralignmentofUNDAFstonationalprocesses;
• TheMDG‐FundbeganfundingJPs;and• DeliveringasOnewaspilotedineight
countries.Tomaximisetheevaluation’sutility,itwillbeimportanttoassessJGPsthatwereconceptualisedandimplementedwithinthecontextofthesechanges.
Apositiveaspectoftimelapseisthatseveralof
Recommendation3:Thetermsofreferenceshouldrequestthattheevaluationteampresenthowitwillintegratehumanrightsinitsmethodology.
52
EmergingPriorities&ReflectionsfortheFutureTermsofReference
theJGPsarecomingorhavecometoaclose,makingiteasiertoassessendresults.Forevaluationsthatwereinitiatedin2006,theremayevenbeapossibilityofassessingmedium‐termeffects.
Takingintoconsiderationthechallengesandopportunitiesrelatedtothetimelapse,combinedwiththeissueofhigherdatareliabilityofthe2006to2010JGPs,focusingtheevaluationonJGPsinitiatedbetween2006and2010wouldbeofgreaterevaluativeworth.
AMixedMethodEvaluation
Forthisevaluationtobeconsidereduseful,itmustprovidecredibleandvaluablefindingsonhowtoimproveJGPsandhowtogenerateknowledge.ThiswouldinvolvegainingadeepenedunderstandingofJGPsbycapturingandcommunicatingtheir“stories”sothattheseillustratetheresultsandrelevantprocessesofJGPsforkeystakeholderswhomakedecisionsaboutJGPs.Suchinquirywouldrequireaqualitativeapproach.Thisimpliesthatdatacollectionisundertakenby,i)documentexamination,ii)evaluator’sobservationandmeasurement,iii)participatoryandcollectiveanalysis,andiv)interviews.Surveyscanalsobedesignedtogatherqualitativedata.Quantitativemethodswouldaddvaluetotheevaluationprocess.TheJGPdatabasethathasbeenestablishedbythestudyandtheanalysisthathasalreadybeenundertakenconstitutesignificantresourcesfortheevaluationteam.
Theseexistingproductswillallowtheteamto“hitthegroundrunning.”Nevertheless,whilethedatabasecontainsarangeofdatathatcanallowfurtheranalysisandcomparisons,tobefullyutilised,thedatabasewouldneedtobeupdatedtoincludeup‐to‐datefundinginformation.Theremainderofthischapterwillprovidesomereflectionsfortheevaluationapproachthatareworthconsidering for thedraftingof the termsofreference.
DocumentationExamination
Itwouldbeimportantfortheevaluationteamtobeginbyreviewingall94programmedocumentsandthe20evaluations.Forbothpracticalandresourcereasons,itwouldmakesensetoundertakeamorein‐depthdeskstudyofaroundone‐quartertoone‐thirdoftheJGPs.TheevaluationteamwouldneedtoprovidecriteriaintheinceptionphaseonhowtoselecttheseJGPs.SomeofJGPsforwhichtherehavebeenreviews–suchastheAlbanianJGPsandsomeoftheMDG‐Fprogrammes–wouldconstitutegoodcandidatesforfurtherdeskstudy.Exceptfor20evaluations/reviews,theJGPdatabasecurrentlydoesnotcontainreportsrelatedtotheJGPs.Aconsiderableeffortwouldbeneededtoensurethatcountryofficessupplythenecessaryreportsanddatatocompletethedatabase.
Recommendation4:TheevaluationtimescopeshouldincludeJGPsfrom2006to2010toensuredatareliabilityandusefulness.
Recommendation5:Theagenciesshouldensurethatthecountryofficessupplythenecessaryprogrammereportsanddatatocompletethedatabaseanddocumentrepository.
53
EmergingPriorities&ReflectionsfortheFutureTermsofReference
CaseStudyApproach
ToobtainadeepenedunderstandingofwhatresultsJGPsareachievingandwhethercollaborationamongagenciesiscontributingtothis,itwillbecriticalthattheevaluationundertakescasestudies.Visitingfourcountriesislikelytoprovidethedatarequired,especiallyif,i)countrieswithmorethanoneJGPareprioritised,andii)fieldstudiesofJGPsarecomplementedwithin‐depthdeskstudiesofotherJGPs.UNconcernstoensureregionalrepresentationforpoliticalreasonsmayrequirethatsixcasestudiesareundertaken.
Thecasestudieswouldmostappropriatelybeselectedbypurposefulsamplingtoensurethatanumberofvariablesarecoveredandthatthecasesare"informationrich"andilluminative.WhilethesamplingshouldbebiasedtowardsJGPsthatareconsideredbystakeholderstobeinnovative,havedevelopedgoodpracticesand/oraresuccessful;thesamplingshouldalsoconsiderJGPsthathavestruggledtoproduceresults.Thefollowingcriteriawouldbethemostimportanttoconsiderinthesamplingprocessofthedifferentcasestudies(boththein‐depthdeskstudiesandthefieldstudies):
1. Mixofdifferentthemes:Thereareeightthematicareas.Themostimportantonestostudyatcountrylevelwouldbemulti‐sectoralJGPs,EVAWJGPsandgovernanceJGPsbecauseoftheirprevalenceandfinancialvalue.HealthandeconomicempowermentJGPsshouldalsobestudied–preferableatcountrylevel.Education,traffickingandHIV/AIDSaremuchless
commonbutshouldatleastbeincludedforin‐depthdeskstudy.
2. Conflict‐relatedJGPs:Conditions,opportunitiesandissuesinconflict‐affectedcountriesoftendifferquiteconsiderablefromotherdevelopingcountries.WhilethestudyhaschosennottoconsiderJGPswithaconflictangleasthematicallyseparate,theUN’simportantroleincountriesaffectedbyviolentconflictwouldmakeitpertinenttostudyJGPswithandwithoutaconflictangleinconflict‐affectedcountries.
3. Mixofagencies:ThecountrycasestudyselectionneedstoensurethattheJGPschosenhaveUNFPA,UNDP,UNWomenandUNICEFasparticipatingagencies.ThiswillberelativelyeasyduetotheprevalenceofJGPswiththeseagenciesinvolved.ItisimportantthattheselectionalsoincludesJGPsinwhichspecializedagenciesparticipate.Ifthecasestudyselectionhasalreadyensuredthatthethematicareasarecovered,thenitwillautomaticallyensurearelativelygoodmixofthemostprevalentagencies.
4. MixofdifferentnumbersofparticipatingUNagencies:ToanalysethedegreeofjointnessandthecollaborationdynamicsinJGPs,theselectionwouldneedtoincludeJGPsthathavethemostcommonnumberofparticipatingagencies(two,three‐fourandfive‐six).ItwouldalsoneedtolookatthelargerJGPsseven‐eight,nine‐tenand/or11+.Ifpossible,theselectionshouldtrytoincludefinanciallylargeandfinanciallysmallJGPswithmanypartners.IncludingafewJGPswithmanyUNpartnerswillautomaticallyensurethattherewillbeamixofmanydifferentagenciesrepresentedintheselectionasrecommendedinpoint2above.
5. Amixofcountrieswithdifferenthumandevelopmentandgenderequalityindexes:Thelevelofcapacityinacountryandthe
Recommendation6:TheevaluationshouldincludeadeskreviewofthewholeJGPportfolioandanin‐depthportfolioanalysisofasizeableproportionoftheJGPportfolio.
54
EmergingPriorities&ReflectionsfortheFutureTermsofReference
extenttowhichgovernmentsprioritisegenderequalityandwomen’srightscanhaveimplicationsforthesuccessofaJGP.Itisthereforeimportanttoincludeamixofcountriesinrelationtothesequalities.
6. AtleasttwocountrieswithJPsthathaveexpressedgendermainstreamingstrategiesand/orMDG3components:IftheaimofassessingJGPsintermsoftheirsynergieswithotherUNprogrammes(seesection0)istobeincludedintheevaluationobjectives,thissamplingcriterionwouldbeimportant.
7. Mixofdifferentbudgetsizes:ThesamplewouldbenefitfromhavingabiastowardsJGPswithbiggerplannedbudgets.Thiswillpotentiallyofferagreaterrangeofeffectsandprocessestoexamine.ToexaminehowagenciesconductresourcemobilisationforJGPs,itwouldbeimportanttoincludeafewJGPsthatwerenotfullyfundedfromthestart.
8. Mixofdifferentregions:BecauseofthesubstantialvalueandhighnumberofJGPsinAfrica,thesamplewouldbenefitfromincludingatleasttwosub‐SaharanAfricancountries.
9. Mixofdifferentlevelsofprogrammematurity:Tobeabletoassessresults,theselectionofcasestudiesneedstoincludeolderJGPsthatarefinalisedornearcompletion.Atthesametime,itisimportanttoexaminenewerJGPsthathavebeenformulatedinmorerecentpolicyenvironmentsandwhichperhapsbuildonlessonsfromthepast.
Othercriteriathatwouldbeimportantinthesamplingprocessofcasestudiesincludethefollowing:
10. Leadagency:Totheextentpossible,theselectionshouldtrytoensureamixofdifferentagenciesperformingthelead
agencyrole.
11. Mixofdifferentnon‐corefundingsources:Theselectionshouldtrytoincludeamixofdifferentfundingsources–suchasdirectbilateralsupportandcontributionsfromtrustfunds.ItisimportanttonotethattheJGPdatabasedoesnotprovidethemostrecentinformationonthis,ascontributionsprovidedaftertheprogrammedocumentwassignedarenotincludedinit.
12. CountriesthathavemorethanoneJGP:TomaximisethenumberofJGPsthatwillbeexaminedatfieldlevel(andthusmeetingmoresamplingcriteria),itwouldmakesensetoprioritise–asrelevant–amongcountrieswithmorethanoneJGP.
13. Typeofexecution:ThecurrentdatabasedoesnotincludeinformationonwhetherJGPsareundertakenthroughnationalordirectexecution.However,ensuringthatthesamplehasamixofthiscouldprovideinterestingresultstocompare.
14. AtleastoneDeliveringasOnecountry:SincetheDeliveringasOnecountriesarepilotsfortheUNreformprocess,examiningJGPsinthiscontextcouldprovideimportantconclusions.
Intermsoffundmanagementmodalities,thepass‐through,parallelandcombinationmodalitieswillnotbedifficulttoachieveamixofsincetheytendtobemorenumerous.However,thereareonlyafewpooledJGPs–12intotal–andtheyaremostlyrelativelysmallinsize.Thedynamicsofjointnesswillbequitedifferentinapooledprogramme.Infact,dependingonwhatstrategicscopeischosen,leavingpooledJGPsspecificallyoutofthesamplingsetmaybeoptiontoconsideraftertheinceptionphase.Othercriteriaforsamplingmaycomeoutoftheinceptionphaseoftheevaluation.Forinstance,initialresearchoftheevaluationteamcoulduncoverinformationon
55
EmergingPriorities&ReflectionsfortheFutureTermsofReference
differentgovernanceandmanagementset‐upsthatmaybeimportanttocompare.
SurveyPossibilities
Tobecost‐effective,asurveyshouldbeeasytoadministerandresultsneedtobeeasilyanalysed–whichmeanselectronicquestionnaires.Shortquestionnaireswithclosedquestionsaremoreusefulsincetheytendtopromoteresponseratesandofferquantifiabledata.
Surveyingnationalpartnersusingclosedquestionsandquestionsthatarerankedbytheinterviewees’judgmentscouldtheoreticallyprovideinterestingdata.However,theneedtotranslatethequestionnairebeyondtheUNlanguages;theunevenelectronicconnectivityofnationalpartners;theriskofapoorresponserateandtheanswersthattrytoprovide“desired”answerswoulddisfavoursuchasurvey.
Whilestillaconsiderableundertaking,surveyingUNstaffismorestraightforward–staffcanallbereachedelectronically,thesurveycanbelimitedtothreelanguagesandpressurecanmoreeasilybeexertedtopromoteagoodresponserate.However,itiscurrentlydifficulttoseeasuitablesubjectareaforasurveyofUNstaff,whichincludesonlyclosedquestions.Ontheotherhand,dataoneffectscouldbeenrichedbyobtainingqualitativedatathroughopen‐endedquestions(combinedwithclosedquestions).Thiswouldinvolvegatheringstaff
viewsonandknowledgeofdifferentkindsofresultsachieved,includingintangibleresults,advocacyeffortsandsynergeticeffectsachievedwithotherUNprogrammes.
TeamExpertise
Thequalityofqualitativedatadependstoagreatextentonthemethodologicalskill,sensitivity,andintegrityoftheevaluator.Generatingusefulandcrediblequalitativefindingsthroughobservation,interviewing,contentanalysisandparticipatoryapproachesrequiresdiscipline,knowledge,experience,creativityandhardwork.Moreover,itreliesontheabilitytotakeininformationbyseeingthebigpicture,focusingontherelationshipsandconnectionsbetweenfacts,identifyingpatternsandbeingattunedtoseeingnewpossibilities.Otherimportantqualitiesofthefutureevaluationteamincludeknowledgeandexperienceinthefollowingareas:
1. Genderequality,women’sempowermentandwomen’srightsmovement
2. Developmentcooperationprocessesandpolicies
3. TheUNdevelopmentsystem,theUNreformprocess,UNdevelopmentprogrammes
4. Rights‐basedapproaches5. Evaluationmethods,participatory
approaches,datacollection6. Developingcountries,conflict‐affected
countries,regionalandcross‐regionalexperience
7. Assessingcapacitydevelopment
Theteamwillneedstronganalytical,writingandfacilitationskillsandarangeoflanguageskills(English,FrenchandSpanish.)Theteamshouldrepresentdiversityandconsistofbothwomen
Recommendation7:Theevaluationshouldincludefourtosixcasestudiesthatinvolvecountryvisits.Theevaluationshouldalsoincludedesk‐levelcasestudies.Purposefulsampling,takingintotoconsiderationtheabovesamplingcriteria,shouldbeusetocarefullychooseallcasestudies.
56
EmergingPriorities&ReflectionsfortheFutureTermsofReference
andmen.Nationalconsultantsshouldbeincludedonthecasestudymissions.Skilledandhighlyqualifiedconsultantsarebusyandoftenbookedupmonthsinadvance.Toensurethatconsultantswiththesequalitiesareavailable,theevaluationshouldbescheduledtoallowadequatelead‐time.
Recommendation8:Thetermsofreferenceshouldcallforateamwithstrongskillsandin‐depthknowledgeandexperienceintherangeofrelevantareaslistedabove.Planningshouldtakeintoconsiderationthelead‐timethatbusyhighqualityconsultantsmayrequire.