FCV Group’s Geo-Enabling initiative for Monitoring and Supervision (GEMS)

For further information, just type gems/ in your WBG intranet browser, see here for our intranet site or contact the GEMS team at the FCV Group (GEMS-team@worldbankg.org)

Building client capacity

in digital data collection and analysis with simple

and free open source toolsThe Core GEMS Team

GEMS: More transparency and accuracy (fewer fake cats) for better development decisions

For further information, just type gems/ in your WBG intranet browser, see here for our intranet site or contact the GEMS team at the FCV Group (GEMS-team@worldbankg.org)

The Master Plan: Systematically Getting Eyes on the Ground for real-time insights

Example: Fadama III project,

operating in Borno State, Northeast Nigeria

Refugee / IDP camp

Conflict Hotspot

Mafa Water Point Fully Functional

City of Mafa | Mafa LGA | Borno State | 3,560 beneficiaries

Site Inspection Protocol

Inspected: Funke Mgwale | June 6, 2017 | 9:20 am-Sample-

Observations:• The water point has been completed and is intact• Utilization by the local community is high

Northeast Nigeria:Joint UN-WB HDP Stabilization Mission

Combining the “Rocket Science,” “Pocket Science,” and “Socket Science”

Remote SensingThrough SatelliteImagery

The GEMS Method: Systematic Real-Time Field Data Collectionthrough Smartphones

‘Plugging In’Spatial Data From Various Sources

?

Let’s connect the cows –ugh– dots between the technology solutions

GEMS systematically builds client capacity in the creation of customized digital M&E systems. This is based on the use of cost-free open source tools and hand-held devices (smartphones/tablets) to collect granular geo-tagged data that automatically feeds into interactive dashboards and web maps.

The aim is to contribute to effective planning, real-time M&E, remote supervision, accountable TPM & geo-mapping of projects and provide Data Integrity along the project cycle.

The ‘Pocket Science’: Definition of the GEMS method

The used platform KoBoToolbox: • Combines all relevant data related to operations, components, and field activities. • Centralizes all collected data in one integrated system, connected to the M&E framework.• Provides data in various formats incl. tables, text, photos, videos, audio, and maps.• Allows for systematic recording of project indicators and survey responses to strengthen

project planning implementation, risk assessments, and coordination across operations.• Enables Bank Teams to directly access project data from PIUs in real-time.

Mirror your data in real-time across stakeholders

All-in-one cost-free open source tool: KoBoToolbox – what it does

https://www.youtube.com/watch?v=4PNtT51h3CQ

Cost-free, simple-to-use digital M&E tool: KoBoToolbox (developed by HHI)

• Fully cost-free accounts, app, server space

• Everything Open Source

• Very simple interface – easy to learn

• Data protected & flexible use of servers

• Quick and fully flexible form creation

• Ownership by clients & easy to scale

• Data and forms easy to share

• Large user Community (hum & dev)

• Ready to go – collect data from day 1

Simple enough and free for anyone to use

The growing Community behind KoBo Toolbox

The GEMS method is cheap, easy to implement, scalable & sustainable

Enter project data

-Sample-

Photo: Impression from geo-enabling training for the South Sudan Local Governance and Service Delivery Project PIU

• Simple and free technology (KoBo)

• Customized data collection forms

• No internet/networkconnection needed

• Automatic integration of data in centralized MIS or M&E system

• Automatic mapping of interventions

• Easy to scale across portfolio/partners

• No ICT experts needed

➢ Building Capacity with clients for sustainable use

Systematic Mapping & Creation of Databases on Sub-Projects at Activity Level

Over 60,000 sub-project locations in 20 countries mapped via GEMS

Example: Togo

Example: Madagascar

Example: DRC

Examples of GEMS Use in FCV Contexts

In Africa, Asia, and Latin America, GEMS provides real-time insights in insecure areas

In Niger, GEMS helps to plan and supervise infrastructure investments

for refugees, internally displaced people and host communities

Project Planning and supervision

In Uganda, a multisectoral nutrition project uses GEMS to monitor

deworming activities that are crucial to prevent stunting in school children

Project Progress Monitoring

In the DRC, detailed data on over 25,000 schools was collected in 3

months, changing the perspective of the Ministry of Education

Portfolio Mapping & Coordination

In South Sudan, remote health facilities have been mapped and

available services and medicine are regularly tracked by GEMS

Real-time Tracking of Service Delivery

Status quo of GEMS implementation:

• > 2,500 local staff & partners trained• Around 450 WBG projects covered• Implemented in 40+ countries• Various Partner agencies included

(AFD, EU, GIZ, IOM, KFW, UNDP, UNFPA, UNHCR, UNICEF, UN-OCHA, UNOPS, WFP, WHO, WWF)

• GEMS listed as supervision solution for the COVID-19 Strategic Preparedness and Response Program (SPRP)

• Operational M&E and remote supervision during preparation & implementation:

• Digital monitoring platforms across components, projects, programs, sectors and countries.

• Structured geo-mapping and recording of baseline data on service infrastructure and use.

• Remote supervision of activities (e.g. infrastructure construction, health trainings, SP activities).

• Real-time tracking of equipment stocks/delivery (e.g. through barcode recording).

• Real-time monitoring of compliance with environmental and social safeguards/ESF.

• Beneficiary or multi-stakeholder engagement (e.g. perception/ feedback surveys).

Leveraging GEMS for the COVID-19 Response

Example: Transmission Hotspot Tracking in Kinshasa

Example: Health Facility Database in Cameroon

‘Normal’ in-Country Training: 3 Days Interactive Hands-on Capacity Building

• Creation and administration of data platform

• M&E skills and digital questionnaire design

• ‘Train-the-trainers’ approach for field data collection

• Data mapping and analysis within the application

• Data export, spatial analysis, Excel analysis

➢ Interactive design customized to M&E needs of the projects and implementation environment

➢ System is fully owned and run by client after training and can be used for flexible needs and scaled across operations

For further information, just type gems/ in your WBG intranet browser, see here for our intranet site or contact the GEMS team at the FCV Group (GEMS-team@worldbankg.org)

1) Designate regional/CMU and GP/project-specific Focal Points (FPs)• Ideally 1 or more full-time GP/CMU FP and 1 part-time FP per project, to take ownership of the system• Can be existing staff or new hires (STCs). GP/CMU FPs can be embedded in GEMS team via WPA/charge code• Pre-existing IT/GIS skills not required for FPs, but very useful to take the system further for advanced analysis

2) Identify countries/sectors and projects for GEMS rollout • Ideally based on delivery priority (COVID-19 response), time-sensitivity, geographic footprint• Define what data should be collected, where, how, by whom – and how it feeds into decision-making

3) GEMS team creates a Portfolio Mapping Platform in collaboration with the GP/CMU/region• GEMS provides template, CMU decides what data to collect, GEMS creates platform• GP/CMU FP to take ownership of platform, with data viewing access provided flexibly (CD/CM, TTLs, gov, partners)

4) GEMS team delivers a remote “training-of-trainers” workshop with the FPs and other stakeholders• Training delivered remotely over 8-10 hours, split into 2-3 days with a day in between and “homework” assigned• Training to cover all the technical skills needed for FPs to create & administrate customized digital monitoring systems• All participants need to attend the entire training, as each bit builds upon the previous ones

5) GEMS team supports FPs in finding the best ways of deploying system based on local conditions and sector needs, incl.:• Field access in the client countries and specific options for local access by stakeholders• PIU capacities and ability to conduct remote trainings with PIUs; existing equipment in-country

6) System is deployed with FPs leading the content aspects and take full ownership of the system• GEMS team supports FPs with the technical issues, data architecture and customization of the system, as needed• Supervision/data analysis and follow-up is done by the FPs to make sure it feeds into GP/CMU/project decisions

Adjusted GEMS rollout amidst COVID-19 Restrictions: Specific Steps for GP/CMUs

Check our website (gems/) for general information, documents, follow-up

Upcoming GEMS Learning Sessions (click image to get on signup page)

BACKUP

Answers to the specific Questions of the Geospatial CoP (1)

1. Data formats: What happens with the data collected/analyzed/produced in your platform? Can you export it and use it in other platforms/software?

➢ All data is saved on a server of your choosing (HHI, UN-OCHA, UNHCR, or your own)

➢ Interactive dashboard, webmap, etc; Export as XLS, KML; connect to any platform (API)

2. Usage: How difficult is to use the tool? Do you need any technical or special skills to use it? what is the timeframe to implement it?

➢ Extremely easy to use and create your own customized platform. No tech skills needed.

➢ You can create a platform and digital form in minutes and collect data right away

3. Support: Are there support materials/training/technical support available (proprietary tool)? Is there a sizeable user community (open tools)?

➢ Very active online forum with helpdesk function, lots of tutorials, and the GEMS team ☺

➢ Huge user community in the humanitarian & development sector and academia

4. Availability: Is it currently available to WB staff? What about clients countries/governments?

5. Cost: Is it free, pay-as-you-go or a subscription-based system? Has the WBG already paid for it?

6. Licensing: Does the WBG currently have a license to use it? Can WB client countries utilize it as well?

➢ KoBoToolbox is available to anyone fully cost-free.

➢ There is no license, no fee, no premium version, no hidden costs. It is just fully free!

➢ How is that possible, you ask? It is because KoBo is funded by donors as a Global Public Good for better data.

Sorry, but I did not want to waste

this space ☺

Answers to the specific Questions of the Geospatial CoP (2)

7. Monitoring and Evaluation: Can you create an indicator (baseline) and track it overtime? How frequently can be done? How accurate?

➢ Yes, you can use KoBo itself to monitor the continuous development of indicators or physical changes (e.g. checking monthly construction progress via photos)

➢ Connection to baselines is better done outside of the tool – e.g. in Excel

8. WBG previous experience: Have the tool been used in a WB project before? If so, give us a quick example?

➢ Yes, the GEMS team has trained over 400 project teams and at least half of them regularly use the tool for M&E, portfolio mapping safeguards, etc.

➢ E.g.: in the DRC, all (50,000+) primary and secondary schools are mapped with KoBo

9. Trial: Can a Task Team Leader (TTL) try the tool? What is the best way to contact you for follow up questions?

➢ Of course ☺. Please type gems/ in your browser or email: gems-team@worldbank.org

Answers to the specific Questions of the Geospatial CoP (3)

Example Agenda for GP/CMU-specific Remote Trainings amid Travel Restrictions

Example Niger: Online Dashboard for Project Planning and Real-Time Analysis

MLI

NGEGIN

DRC

CAFCMR

HTI

KEN

SOM

PAK BGD

West AFRRegional Projects

TON

Ongoing project supportGEMS ICT Support provided for theCOVID-19 Response Requested project support

COL

MEX

HND

ETH

GTM

CIV

TJKKosovo

MMR

PNG

CHN

MDV

IRQLBN

DJI

MRT

LKA

ZMB

Pacific Islands

KAZ

IND

BFA

TCD

BDI

GEMS use cases include real-time project M&E, remote supervision, mapping of healthcare facilities, tracking of medication and PPE, beneficiary surveys, etc.

AFG

DOM

Geographic Extent of GEMS Support to COVID-19 response operations

NGATOG

GEMS provides ongoing ICT & M&E support to operations responding to COVID-19 in 20+ countries

Support extended for operational teams & clients across sectors, regions and country income levels

NB: This only covers specific COVID response support, with more projects being supported in other contexts

YEM

AZE

BRAAmazon Basin Regional Project

The boundaries, colors, denominations and any other information shown on this map do not imply, on the part of the World Bank Group, any judgement on the legal status of any territory, or any endorsement or acceptance of such boundaries.

Created by FCV Operational Support Unit (GTFOS)For questions on the map, contact: Bernhard Metz | bmetz@worldbank.org

Refugee & IDP campsData source: UNHCR

Camps & hosting sites

Major Conflict HotspotsData source UCDP | 2014-2016

Recorded conflict fatalities

Geo-Enabling initiative for Monitoring and Supervision (GEMS): Systematic Implementation in AFR

Mali

DRC

MauritaniaNiger

Chad

South Sudan

KenyaRwanda

Uganda

GEMS implementation in Africa (Apr, 2020)

• ~ 2,000 clients & field staff trained• > 450 WBG projects covered• Implemented in 26 AFR Countries• Various partner agencies included

Senegal

ROC

FCV’s GEMS initiative:

Building Capacity among Clients, Bank Teams and Partners to harness simple ICT Tools for Remote Supervision, M&E, TPM, Safeguards, & Portfolio Mapping-

To get eyes on the ground where we cannot always have boots on the ground.

8-time VPU Award Recipient

CdI CARGuinea

Burkina Faso

Cameroon Somalia

Nigeria

Madagascar

Burundi

TogoLiberia

Mozambique

Benin

Ethiopia

Zambia

Build for Sustainability

https://digitalprinciples.org/principles/

Use Open Standards, Open Data, Open Source, and Open Innovation

Creating a Public Good: Inclusion of Partners for more effective Collaboration

Be Collaborative

Design for Scale

All partners are invited to join!

Exit Questionnaire

Weekly Perception and Worksite Questionnaire

Current GEMS use case: “MARTA” - Social Protection Response to the Ebola Crisis

• The project covers over 100,000 beneficiaries in all the health zones affected by the 10th Ebola outbreak over an 18-month period.

• Registration, Entry, Exit, Perception and Worksite questionnaires developed and conducted with the GEMS tools.

• As of June 2020, around 80,000 surveys were administered & 1,5M data points collected.

10,127Surveys

94% Completion

7,253Perception

Surveys

296Worksite Surveys

568Pictures

• 99% proud of the work would recommend to others positive opinion from friends and family

• 95% had no problems during the project

• Perceptions on: recruitment process, choice of worksites/location, security, social cohesion, remuneration, issues related to the Ebola crisis, accidents.

• Real-time information from worksites for course correction - female participations, payments, GRM, conflicts, safeguards.

Registration Questionnaire

Entry Questionnaire for Beneficiaries

47,992Surveys

13,926Surveys

95% Completion

• Registration streamlined (5 to 2 days) andquality increased

• Provided baseline for impact evaluation• Database of vulnerable population

• Basic demographic/vulnerability profile• Household composition (indirect

beneficiaries)• Monitor compliance of:

- Contracts, code of conduct- Knowledge of health and safety

at work, GRM

Real-Time Supervision: Leveraging GEMS for interactive Portfolio Dashboards

Empowering Women & Youth with digital Skills

• More than half of GEMS team members and two thirds of in-country focal points are women.

• GEMS trainings encourage the participation of women and the proportion of female participants is continuously tracked.

• The technology and M&E skills taught by GEMS provide women and youth with new opportunities.

Closing crucial Data Gaps on Gender and Youth

• GEMS trainings build capacity in M&E on gender and demographic issues and raise client awareness on the importance of collecting disaggregated data

• The contents of GEMS surveys often focus on issues that are crucial for women’s and youth inclusion, such as livelihoods, jobs, maternal and children health, nutrition, agriculture, and water.

Leveraging GEMS for Gender and Youth Inclusion

Impressions from GEMS trainings in Nigeria and Uganda

Examples: Country-Wide and Project-specific supervision platforms

• Throughout Africa: Portfolio-Mapping and Monitoring Platforms

• Sahel: Cross-Agency Coordination and Portfolio Mapping Tool

• Niger: Project Planning, Monitoring and Evaluation Tool

• Azerbaijan: Project M&E and Citizen Engagement Tool

• Uganda: Real-Time M&E System to track diverse project components

• DRC: Structured mapping of all schools in the country (> 50,000)

Project Planning

Beneficiary Surveys

Baseline Data Collection

Results Framework Indicators

Impact Evaluation

Environmental and Social Safeguards Monitoring

Verifying TPM Activities“monitor-the-monitors”

Remote Project Supervision

Possible Uses of GEMS for Clients and the WBG / Partners

Project Teams & Clients WB Country Teams / Partners

Portfolio Mapping & Coordination

Citizen Feedback / Communications

Project Progress Monitoring

Real-time Tracking of Missions and Security IssuesReal-time Tracking of Service Delivery

Citizen Engagement / GRM

Icons provided by Freepik from www.flaticon.com

KoBo Toolbox: Flexible and Easy Questionnaire Creation

KoBo Toolbox: Easy Offline Data Collection

KoBo Toolbox: Easy Data Analysis Functions

1. Platform for effective project planning, prioritization, implementation, and M&E In close coordination with partners and stakeholders.

2. Verification of physical sites & implementation works and project progress in remote areasOne system can be used across the portfolio

3. Third Party Monitoring - Remote supervision of date, time, & locations of 3rd party activities “Monitor-the-Monitors” approach

4. Verification of interview and survey activities and automated recording of meta data Structured collection of all qualitative information and recording of date, time, locations

5. Recording detailed beneficiary data and interactions with beneficiariesSystematic methods for tracking community engagement

6. Structured analysis, monitoring, and geo-tagging of points and issues relevant for securityRemote tracking of security-related dynamics through open source data and partner input

Annex: Geo-enabling for Project M&E: Obvious Examples

Summary of the (potential) cost factors of implementation of GEMS

1. Software/systems: All the used software, mobile apps, accounts, and server space are open-source and fully cost-free.

2. Implementation trainings: GEMS works on a cost-recovery basis and costs solely relate to travel (once possible) and staff/STC time. As GEMS is all about building capacity and providing client ownership, the staff/STC time is typically limited to just 10 days each for 2 trainers. Typically, training costs can be split across charge codes (e.g. projects, CMU) lie:• Below USD 10,000 for fully remote support to WBG staff• Below USD 25,000 for in-depth client capacity-building that includes travel

3. Hardware: for collecting the data in the field, Android smartphones or tablets are needed. It is possible to use existing devices (incl. private ones) or to purchase relatively cheap phones (< USD 200).

4. Data collection: The potential costs of the PIU (or third parties) going to the field and collecting the data are project-specific. Yet, in most cases, paper-based data collection is already conducted and GEMS can thus replace or add to an existing approach.

Potential Risks and Risk Mitigation (1): Privacy and Data Protection

1. Data Protection: Potential risks related to data protection are inherent to digital data collection and storage. This can be of particular concern in FCV contexts when the collected data might contain information related to for instance conflict activities, sensitive survey responses, or personal information (PII) of victims or displaced people.

➢ The data platform is provided by the Harvard Humanitarian Initiative. The tool and servers comply with the latest industry standards in terms of data protection, incl. the EU’s General Data Protection Regulation (GDPR).

➢ Users retain full ownership of all submitted data, which is hosted on password-protected, encrypted accounts. The data is neither accessed by the platform provider, nor shared with any third parties.

➢ For additional data protection, the GEMS team also has an agreement with UNHCR to use their server, which enjoys diplomatic immunity, for PII or any data related to displacement.

2. Privacy: In addition to data protection in the technical sense, data privacy protocols are a main focus of GEMS.

➢ During the GEMS capacity-building workshops, the clients are also trained in best practices regarding data collection and privacy. This includes, among others, guidance in the following points:• The teams must adhere to the specific legal requirements of the respective country. • GPS coordinates must not be collected for displaced populations of less than 250 (UNHCR guidelines).• GPS coordinates of individual households are only collected in exceptionally and must never be shared publicly.• Before taking photos of people, enumerators must obtain their consent and have them sign a digital form.

➢ GEMS team provides very detailed guidance material on privacy and data protection (can be shared)➢ Client show generally positive uptake of best-practices for privacy and data protection during trainings

Potential Risks and Risk Mitigation (2): Sustainable use of GEMS

3. Limited Technological Capacity in FCV Contexts: FCS, often have low capacity in terms of digital infrastructure as well as ICT skills among government employees. Therefore, tools and methods that depend on sophisticated technology and steady network connectivity face limitations on the ground. In brief, simplicity is key for ICT in FCV!

➢ This is addressed by the design of the GEMS method and the used ICT tools. A central element of the method is to use ‘simple-enough,’ low-cost tools and procedures that are appropriate for low capacity environments.

➢ Technically, the GEMS method is based on the use of cost-free software and common smartphones that allow for offline field data collection (i.e., no internet or network connectivity are required in the field).

4. Sustainable Use and Scaling: A potential risk is that the tools and methods are not used efficiently and sustainably for the benefit of projects. This could be due to weak capacity, budget limitations, or a lack of incentives.

➢ The system provides incentives for sustainable use. The main aim of the GEMS methodology lies on building lasting capacity among the clients themselves to use the tools based on their specific needs.

➢ Project Task Teams and Country Teams on WBG-side are explicitly involved in the system setup and training. ➢ Dedicated GEMS focal points are designated within the projects and Country Teams to oversee the systematic and

sustainable implementation of the method. ➢ Regular follow up by GP and CMU staff with the project team creates incentives to properly run the system.

➢ GEMS is simple, low-cost & easy to scale, with no/limited need for procurement (solely smartphones, if needed).➢ However, local ownership by GPs/TTLs or CMUs and close coordination with clients is key!

Protection of Privacy and collected Data in KoBoToolbox

Server:• KoBo offers free access to a server (Amazon Web Services) with

unlimited space for every account (up to 10,000 submissions per month by default, more upon request – but always cost free).

• The server hosts the created questionnaires and collected data.• KoBo does not access or forward any data or questionnaires.

Data protection:• KoBo makes frequent backups of its entire database to safeguard

against data losses.• Accounts are protected by usernames and passwords and underlying

data is encrypted according to the latest standards.

Rights and data ownership:• The account users are the sole owners of any data and questionnaires

hosted on KoBo.• The account users are responsible for the collection and administration

of personal data according to national or international data protection legislation (see the EU’s General Data Protection Regulation – GDPR).

• Exclusively information (questionnaires and collected data) that has explicitly been made accessible for the public are available for internet users without the respective passwords. Data made public data may e.g. be included in Google searches, but access can be restricted any time.

You can also connect your own server to KoBoToolbox!

Project specific

uses

Beneficiary

surveys

Base data

collection

Individual Project

Accounts

Portfolio Mapping

Form

Regular M&E of

implementation

World Bank Admin

Account (CMU)

Impact Evaluation

Safeguards

tracking

TPM monitor-the-monitors

Regular Project

Supervision

Shared Account for

portfolio mapping

GEMS Data Architecture | Roles of Clients and & the WBG

WB CT Focal PointPMUs (M&E Experts)

Collect field data

CLIENTS (PIUs/PMUs)

Enumerators/TPM

• Prepare M&E forms• Analyze data & report to WB• Intervene if needed

WORLD BANKData Aggregation

• Administers WB forms• Analyze WB data• Link to PIU and TTLs

TTLs/CMU

Supervise / analyze data

Au

tom

atic

dat

a u

plo

ad

Can be shared Automatic sync.

For more information about GEMS, just type gems/ in your WBG intranet browser

or contact the GEMS team at:GEMS-team@worldbank.org

Don’t settle for tunnel vision! Get in touch with GEMS and check our website