Safe Water Project - PATHsites.path.org/water/files/2012/09/Cambodia-direct-and-retail...Safe Water...

DISTRIBUTION OF CERAMIC WATER

PURIFIERS THROUGH DIRECT SALES

AND RETAIL SALES PILOTS IN

CAMBODIA Results from an Evaluation of Safe Water Project Pilots

January, 2012

Prepared for:

PATH

Prepared by:

Abt Associates Inc.

This report was prepared by Slavea Chankova, Ramakrishnan Ganesan, Julie Williams, and Erica James

of Abt Associates for the Safe Water Project implemented by PATH with a grant from the Bill & Melinda Gates

Foundation.

Photo: PATH

Abt Associates is a mission-driven, global leader in research and program implementation in the fields of health, social and environmental policy, and international development. Known for its rigorous approach to solving complex challenges, Abt Associates was ranked as one of the top 25 global research firms in 2010. The employee-owned company has multiple offices in the U. S. and program offices in nearly 40 countries.

Abt Associates Inc. 4550 Montgomery Ave., Suite 800 Bethesda, Maryland 20814 USA www.abtassociates.com

http://www.abtassociates.com/

Table of Contents

List of Acronyms ............................................................................ 5

Acknowledgements ....................................................................... 6

Executive Summary ....................................................................... 3

1. Introduction ................................................................................ 9

2. Pilot Description ...................................................................... 10

2.1. Pilot Partners ......................................................................................... 10 2.2. Products Promoted through the Pilot............................................. 11 2.3. Pilot Geography .................................................................................... 12 2.4. Target Consumer Group ................................................................... 13 2.3. Direct Sales and Retail Sales Pilot Variants .................................... 13 2.4. Partner Roles and Responsibilities ................................................... 15

3. Evaluation Approach ............................................................... 16

3.1. Objectives of the evaluation .............................................................. 16 3.2. Evaluation methods .............................................................................. 17

3.2.1. Quantitative evaluation study ................................................. 17

3.2.2. Qualitative assessment ............................................................. 22

3.3.3. Assessment of commercial viability and implementation . 23

4. Evaluation Results .................................................................... 25

4.1. Background Characteristics of Surveyed Respondents and

Households ............................................................................................ 25 4.2. HWTS Knowledge, Perceptions, and Practices ............................ 32 4.3. Effect of Pilot Interventions on CWP Purchase and Use ........... 36

4.3.1. Product Awareness and Sources of Information ............... 36

3.2.1. Purchase and Use of CWP ...................................................... 42

4.3.3. Correct Use of CWP ............................................................... 48

4.3.4. Triggers and Barriers to Purchase and Use ..................... 49

7. Retail Sales Model: Price Sensitivity to Super Tunsai and

Feasibility of Implementing Coupon Schemes .......................... 56

8. Assessment of Commercial Viability: Direct Sales Model ... 59

8.1. Approach to Assessing Commercial Viability ................................ 59 8.2. Channel Efficiency ................................................................................. 60 8.3. Market Potential and Uptake of CWP............................................. 61 8.4. Channel Sustainability .......................................................................... 62 8.5. Commercial Viability for Hydrologic: Door-to-Door Sales ....... 62 8.6. Commercial Viability for Hydrologic: MFI Sales ............................ 63 8.7. Commercial Viability for VisionFund ............................................... 63

9. Partners’ Willingness to Continue with the Models .... 64

9.1. Hydrologic‟s interest in continuing the model .............................. 64 9.2. VisionFund‟s Interest in Continuing with the Model .................... 65

10. Comparison of Results from Cambodia Pilots and Other

SWP Pilots .................................................................................... 65

10.1 Comparison of MFI models in India and Cambodia .................... 66 10.2. Comparison of Direct Sales Models in India, Cambodia, and

Vietnam ................................................................................................... 69 10.3. Advantages of Cambodia Pilot Models ......................................... 70

11. Limitations of the Research .................................................. 72

12. Summary/Conclusions ........................................................... 73

13. Recommendations ................................................................. 75

Annex ............................................................................................ 76

List of Acronyms

CBO Community-based organization CWP Ceramic Water Purifier DHS Demographic and Health Survey FGD Focus Group Discussion GKB Germ-kill battery HUL Hindustan Unilever HWTS Household Water Treatment and Storage IDI In-depth Interviews NGO Non-governmental organization MFI Microfinance institution PATH Program for Appropriate Technology in Health PPP Purchasing power parity SWP Safe Water Project USD US dollar

Acknowledgements

Many individuals contributed to the design and implementation of the study, provided feedback on

the presentation of the study results, and contributed valuable comments on an earlier draft of this report. The author would like to acknowledge the contributions of Samantha Bastian, Benjamin Mandell, Lorelei Goodyear, Elizabeth Blanton, Tim Elliot, and Siri Wood, as well as the field team from IRL Cambodia who carried out the data collection.

ABT ASSOCIATES INC. 3

Executive Summary

Introduction

The PATH Safe Water Project (SWP) aims to develop sustainable distribution models for

ensuring access to Household Water Treatment and Storage (HWTS) products, together with promoting sustained use of those products over time by low-income populations. Pilot commercial models are implemented in several countries to assess and refine various HWTS products and business approaches.

In Cambodia, the SWP piloted two models for the distribution of a branded ceramic water

purifier (CWP) Tunsai and an improved-design version introduced through the pilot under the brand Super Tunsai. The models included a retail sales model using existing retailers at the community/village level, and a direct sales model. The direct sales model had two variants: a door-to-door sales pilot and a pilot implemented through a partnership with a microfinance institution (MFI) which provided loans for the purchase of the promoted brands of CWP. The retail sales model tested three different CWP price points through coupon-based discounts to consumers. PATH’s partners in the pilots were Hydrologic Social Enterprise, manufacturer and distributor of Tunsai, and VisionFund, a MFI operating in 17 provinces across Cambodia.

This report presents the results from an evaluation of the pilots conducted by Abt Associates in 2010-2011.

Evaluation approach

The evaluation used a mixed-methods approach which included:

Quantitative study using a population based survey of target consumers, with 3,293 households surveyed at baseline and 3,455 households surveyed at endline.

Qualitative (exploratory) study of target consumers and sales representatives, drawing on 12 focus-group discussions and 48 in-depth interviews with both consumers and CWP salesmen.

Assessment of commercial viability and implementation of the pilot models.

The quantitative study based on household survey data employed two evaluation designs:

1. A pre-post evaluation design (using baseline and endline survey data) was employed to compare changes in CWP uptake and use among MFI members.


2. A post-only design with intervention and control groups (using only endline survey data) was employed to evaluate and compare the impact on CWP uptake and use by the general population of the basic direct sales variant and the MFI sales variant vis-à-vis a control group of households in an area where neither of the two pilots was implemented.

These two evaluation designs are less rigorous than a randomized control trial design that would have assigned individual consumers or villages to control and pilot groups. A randomized design was not feasible in the context of the pilots, because the SWP concurrently tested the commercial viability and feasibility of various operational features of the models. The key limitations of the pre-post and post-only evaluation designs include the possibility that the results on impact of the pilots on key outcomes (CWP purchase and use) could be overstated. Findings from Consumer Research

The pilot interventions effectively targeted the SWP core target consumers group (the middle three country-specific wealth quintiles): within each group in the study area, 80 to 87 percent of households belonged to this core target group.

While there were a number of factors that limit the attribution of differences between groups to pilot activities alone, the evidence from this evaluation indicates that the direct sales and the MFI sales pilots had a substantial impact on CWP purchase among the general population. The impact of the MFI pilot among VisionFund members was of particularly high magnitude.

o CWP uptake was 7 percent across the direct sales area surveyed at baseline. At

endline, CWP purchase1 in the control group had remained at a similar level (7 percent) but reached 16 percent in the basic direct sales area and 21 percent in the MFI area.

o Among MFI members, 43 percent had purchased a CWP at endline compared to 6 percent at baseline.

The pilots were associated with a similar pattern of results for CWP use:

o At baseline, only 2 percent of households surveyed in the direct sales area reported using CWP at the time of the survey. Regular use of CWP (every day or almost every day) was 1-2 percent across the study groups.

o At endline, use of CWP was 5 percent in the control group, compared to 12 percent in the basic direct sales group and 17 percent in the MFI area group. The

1 Throughout this report, the terms „uptake‟ and „purchase‟ are used interchangeably when referring to CWP. Some

households obtained a CWP for free or at a subsidized price from NGOs but for the purposes of all analyses in this

report such households are included in „purchasers‟.


rates of regular use of CWP were nearly identical to the rates of current use in all groups.

The market potential for CWP in this population is high, as evidenced by the following:

o There was high felt need to treat water: about 60 percent across groups said it was absolutely necessary/good to treat. Although there were major differences in the predominant type of water source used during the dry and rainy season, the extent of perceived need for drinking water treatment was similar in both seasons.

o Nearly all households using a CWP had used boiling prior to purchasing the

device. Boiling was the predominant HWTS practice in all study groups with 60-80 percent of households across groups reporting boiling as a regularly used method.

Key motivating factors for purchase and use of Tunsai and Super Tunsai include the health benefits of CWP, time and money savings from its use compared to boiling, and the increased easy availability (in terms of quantity) of treated water. A key attractive feature for Super Tunsai was its look and design.

The main reason for not purchasing CWP was lack of felt need for the device. Price/affordability of CWP was also mentioned as a key reason for not purchasing. The CWP loan from VisionFund (used by nearly all MFI members who owned a CWP at endline) and the coupon discount in the retail pilot were mentioned as important drivers for purchase.

Among those who purchased a CWP but were not using it, a leading reason was breakage. Give-away was more prominent for Super Tunsai than other CWP brands, which indicates that the redesigned product might have aspirational/status symbol value.

There was a high level of recall of the promotional activities supported through the pilots. CWP demonstrations were particularly valued by consumers (according to data from the qualitative study).

The door to door salesmen were mentioned as a main source of messages on Tunsai/Super Tunsai at endline in pilot areas, while TV or radio was the most frequently cited source in the control area where half recalled messages on the promoted product through media. VisionFund meetings were an equally important source of messages in MFI areas, and the predominant source recalled by MFI members.


Commercial Viability of the Pilots

Feasibility of implementing coupon schemes

Examination of how the coupon discount schemes worked (through interviews with retailers and other pilot implementers) indicated that it is feasible to implement such a scheme, for example as a means to provide targeted subsidy to households in lower socio-economic groups. Such a scheme appears to have the potential to benefit the poor and yet not circumvent the market. However, implementing the scheme will require additional resources that will need to be supported.

Based on analysis of sales data and findings from the consumer research, we infer that Super Tunsai is price sensitive in the $17 - $22 price band, but further decreases in price may not increase demand in the long run.

Commercial viability of direct sales model Analysis of the findings regarding the overall market potential shows that:

There is high market potential for CWP in this population

Price/affordability of CWP is a significant barrier to purchase of CWP. However, installment payment plans appear to be able to overcome this barrier to a great degree as evidenced by the magnitude of uptake among MFI clients (including those who joined the MFI to avail of the CWP loan).

Direct sales models seem to be appropriate for the current product-market context. There was a high level of recall of the promotional activities supported through the pilots. CWP demonstrations were particularly valued by consumers.

Thus we conclude that CWPs and in particular Super Tunsai has high potential, and that direct sales models with consumer financing mechanisms can generate significant uptake among the population.

The results on indicators of overall commercial viability are as follows:

Analysis of average sales per direct salesperson and their revenue expectation showed that the MFI model was more efficient and the sales conversion rates yield income levels that meet or exceed the aspirations of the salespersons, which was not the case with the door-to-door sales.

Cost recovery through July 2011 for Hydrologic was at 47 percent in the door-to-door sales model and 95 percent in the MFI model. In both models, Hydrologic is able to recover the product costs, VAT and direct sales expenses (including product marketing). However, Hydrologic was not able to recover the administrative expenses or any mass media advertising that were expended through this period of pilot activity. Since the ‘as-is’ commercial viability of the MFI model was 95 percent during the period of the evaluation, we expect this model to become fully viable (100 percent or higher cost recovery) with marginal increase in scale of operations, increased efficiency of operations or improvements in sales productivity.


Based on parameters and assumptions as seen in the pilot, VisionFund achieved a 96 percent cost recovery through the July reporting period and was able to recover the product costs and direct sales expenses (including product marketing). However, VisionFund was not able to recover the administrative expenses at the pilot level through the July reporting period, as it was not able to spread these costs over enough loan volume in the early stages of the pilot.

Hydrologic representatives believe that the retail sales model is an essential strategy to long-term growth and sustainability of Hydrologic’s operations. However, Hydrologic believes that the retail sales operations model needs to be re-engineered for it to realize its potential.

Current focus of Hydrologic is in the direct sales models. Sales have been low in the door-to-door model, the returns to the salespersons and Hydrologic insufficient and hence this model is not considered attractive. Hydrologic is keen to continue the MFI model since it has yielded high sales, and feedback from staff involved in this model has been positive. It is keen to continue the partnership with VisionFund, with some changes to improve efficiencies.

Hydrologic intends to continue with the MFI model, with an emphasis on Super Tunsai. Extent of scale-up will depend on availability of working capital to support the expansion, and willingness of VisionFund to scale-up.

VisionFund considers its partnership with Hydrologic in the marketing of CWPs as a key growth strategy to expand its client base. In addition, safe water also aligned with VisionFund’s social objectives. Therefore, VisionFund is keen to continue and scale-up model. VisionFund has some reservations on the proposed models (for example the dedicated CSO model), and may require additional incentives / margins such as an access fee (payable to the MFI by the manufacturer) to continue with this partnership.

Comparison of the results from the Cambodia pilots to the results from all other SWP pilots (summarized in Annex Table A5) showed that:

The basic direct sales pilot in Cambodia outperformed all other SWP direct sales pilots in terms of effectiveness and commercial viability, but ranked lower in terms of potential for scale-up than some of the other direct sales pilots in which partners expressed a willingness to continue and scale up (albeit under certain conditions – such as external support for marketing costs - that might be difficult to achieve).

Similarly, the comparison of the SWP MFI pilots concluded that the Cambodia model was the strongest in terms of effectiveness, commercial viability and partners’ intention for scale-up. Our assessment ranked as second most promising the model piloted in Erode (an urban area in the state of Tamil Nadu, India) which offered PureIt, a tabletop durable filter, at a loan scheme of weekly installments of 43 Rupees ($0.95) paid over 50 weeks. The remaining MFI pilots in India had


substantially poorer performance in both observed and potential (‘could-be’) effectiveness, sustainability and scalability.

Recommendations

Changes to promotional and other pilot activities that partners could consider to

improve the pilot models include:

Defer mass-media communication promoting Super Tunsai CWPs until the MFI partnership has a wide scale of operations to make such communication cost effective. Until such time focus on village meetings as the primary promotion vehicle.

Highlight the CWP’s effectiveness in addressing the most frequent water issues/concerns: visible impurities and perceived germ contamination.

Addressing affordability issues by continuing to offer loans through VisionFund and similar partners, but also consider additional ways to make the products more affordable for segments of the population that cannot afford purchase at full price (e.g. through coupons offered through direct sales channels).

Improve consumer education through all channels (mass media, instructional pamphlets, flyers, and direct sales representatives) on the topics of CWP maintenance and cleaning, how Tunsai filters water, and what to do in case of malfunction/damage.

Increasing coverage of product demonstrations should be of particular focus. For example, consider changes to retailer incentives and retail sales model to encourage retailers to spend more time with interested customers and market CWP more effectively. Additionally, distributor agents could offer product demonstrations in/around stores to address lack of time/interest by retailers.


1. Introduction

The PATH Safe Water Project (SWP) aims to develop sustainable distribution models for

ensuring access to Household Water Treatment and Safe Storage (HWTS) products, together with promoting sustained use of those products over time by low-income populations. Pilot commercial models are implemented in several countries to assess and refine various HWTS products and business approaches. The primary objective of the pilots is to assess various distribution models for reaching lower income populations with products that are already commercially available and effective.

The SWP pilots seek to answer six key questions:

1. What is the uptake rate among target consumers of HWTS products made available

through commercial distribution channels and marketing?

2. What is the extent of consistent and correct use among target consumers of HWTS

products made available through commercial distribution channels and marketing?

3. What are the triggers and barriers to trial and use of HWTS products made available

through commercial distribution channels and marketing?

4. Can commercial partners earn a profit from sales to target consumers?

5. Will commercial partners continue and/or scale up the SWP pilot business model to

reach target consumers?

6. What other efforts/inputs/incentives are needed to stimulate supply and demand for

HWTS to target consumers?

In Cambodia, the SWP piloted two models for the distribution of a branded ceramic water

purifier (CWP): a retail sales model using existing retailers at the community/village level, and a direct sales model. The CWP is a water filter for use at the household level, effective in providing microbiologically safe water. The direct sales model had two variants: a door-to-door sales pilot and a pilot implemented through a partnership with a microfinance institution (MFI) which provided loans for the purchase of the promoted brands of CWP. The retail sales model tested three different CWP price points through coupon-based discounts to consumers.

Abt Associates conducted monitoring and evaluation of the SWP pilots. The evaluation approach included household surveys to measure uptake and use of promoted HWTS products, identifying motives and barriers to purchase and continued use through exploratory qualitative research, and assessment of the commercial viability of the models using sales and cost data and information from pilot project partners. This report presents the findings from the evaluation.


2. Pilot Description

2.1. Pilot Partners

For the SWP pilots in Cambodia, PATH partnered with Hydrologic Social Enterprise, a social-

commercial enterprise aimed at developing HWTS products, and VisionFund, a MFI operating in selected pilot areas. Hydrologic Social Enterprise

Hydrologic is a social-commercial enterprise aimed at developing HWTS products for

distribution in Cambodia which has been created through a partnership by Hagar International and International Development Enterprises (IDE). Hydrologic’s products build upon locally produced ceramic pot-style filters which remove harmful microorganisms from water by gravity filtration through porous ceramics, with typical flow rates of 2-3 liters per hour. The CWP filters significantly improve household water quality, eliminating up to 99.99% of E. coli. Water treatment at the household level through using ceramic filtration methods is considered as one of the most promising options for HWTS in developing countries. Use of CWPs in Cambodia is widespread and growing, with the involvement of local and international NGOs and government efforts that have been supported by UNICEF and other donors.

Hydrologic had been marketing the CWP through retail networks in Cambodia prior to the

pilot but sales had remained low. To increase consumer uptake of CWP and distribution efficiencies, PATH supported Hydrologic in developing a more attractive CWP, and in designing parts for a more compact shipping box. The pilot aimed to increase purchase and use of the original model of the CWP developed by Hydrologic and sold under the brand Tunsai, and of an improved version of the product introduced through the pilot under the brand Super Tunsai (described in greater detail in the following section).

VisionFund

VisionFund Cambodia is a Christian organization that focuses its ministry on the lives of Cambodia’s poorest families regardless of religious beliefs, gender or ethnic background. It is a microfinance organization (MFI) that aims to lift its clients and their wider communities out of poverty through the provision of micro credit and other complementary services. VisionFund Cambodia is owned and governed by VisionFund International, which operates in over 40 countries, and is supported by large commercial lenders and investors. VisionFund Cambodia is registered as a commercial financial service provider and holds a permanent microfinance license from the National Bank of Cambodia.


VisionFund Cambodia has three lending options: community bank loans for poor micro-entrepreneurs (ranging from $US 20 to $US 350), solidarity loans intended to meet greater customer needs (ranging from $US 350 to $US 1,000), and individual lending products for small and medium scale entrepreneurs ($US 1,000-10,000). About 83 percent of clients are in the community bank loan portfolio.

In 2010, VisionFund Cambodia had a loan portfolio of US$30 million and served more than

108,000 clients. Most of its clients live in suburban and remote areas of Phnom Penh capital and 17 provinces; the MFI plans to expand nationwide by 2013. At the end of 2010, repayment rate was at 99.5 percent. VisionFund Cambodia has a financial self-sufficiency of 105 percent and operational self-sufficiency of 111 percent. 2

2.2. Products Promoted through the Pilot

Hydrologic’s ceramic pot filters are enclosed in food-grade plastic containers, with a lid and a tap for dispensing filtered water. This ensures that treated water is safely and properly stored, preventing recontamination after treatment (which is a disadvantage with boiled water). The filters have only one moving part (the tap) and need no external energy source (such as UV lamps) or consumables. CWPs can become clogged if they are being used to treat particularly turbid water. In order to avoid clogging, Hydrologic recommends that extra turbid water be allowed to settle for a day or so in a storage container prior to filtration. The ceramic filter can easily be removed from the plastic enclosure for periodic cleaning.

Since 2001, International Development Enterprises has established a widely recognized

CWP brand in Cambodia with its Tunsai water purifier3 (Figure 1, left). The Tunsai water purifier continues to be distributed through NGO channels to the poorest Cambodians and other groups who cannot be served through ordinary commercial distribution channels.

Aiming to make its filters more appealing to a growing consumer base who demands durable household goods with a status symbol aspect (in addition to the products’ health benefits), Hydrologic collaborated with PATH to design a high-end, more visually aspirational CWP branded Super Tunsai (Figure 1, right). The development of the Super Tunsai built upon IDE’s long experience in social marketing in Cambodia, comprehensive market research by PATH (including extended user testing), and branding consultation from Cambodian and foreign experts. Early product testing showed an extremely enthusiastic response from Cambodian consumers in the Super Tunsai’s target demographic.

2 Source: VisionFund Annual Report 2010:

http://www.visionfund.com.kh/images/stories/files/Anual%20Report/VF%20annual%20rep%2004%20May%2011a.pdf 3 Tunsai means rabbit in Khmer, a creature popularly associated with wisdom and cunning.


Figure 1. Tunsai and Super Tunsai Ceramic Water Purifiers

2.3. Pilot Geography

The direct sales model was piloted in Kampong Speu province and retail sales model was

piloted in Kampong Cham province. Both provinces are located in central Cambodia and are close to the capital Pnom Penh (Figure 2).

Several variants of each model were tested through the pilot. Each variant had a

designated geographical area (consisting of entire districts or selected communes within a district) that did not overlap with the area for any of the other variants. One exception was an overlap between door-to-door sales and sales through VisionFund which occurred in some communes.4

4 The basic direct sales pilot variant covered 162 villages, the MFI variant covered 73 villages, and in 75 additional

villages both of these variants were implemented.


Figure 2. Location of the Safe Water Project Pilots

2.4. Target Consumer Group

The core target consumer group for the SWP project is households from the middle three

asset wealth quintiles in each country. A wealth index based on household assets – following the methodology used by the Demographic and Health Surveys - is used to place a household into one of the five country-specific relative wealth quintiles.

The SWP pilots in Cambodia covered all households in the geographical areas where pilot

activities took place, with the exception of the MFI direct sales variant which focused primarily on Vision Fund’s members.

The basic direct sales pilot covered an area with about 16,000 households. The MFI variant

had a catchment area of about 6,700 households (total population in the villages and communes covered by the pilot). The retail sales pilot variants covered about 33,000 households in total. The majority of the population in each pilot catchment area was rural.

2.3. Direct Sales and Retail Sales Pilot Variants

Direct Sales Model

Two direct sales variants were implemented in preselected areas of Kampong Speu province. In addition to the specific direct sales activities described below, both versions of the


CWP promoted through the pilot were available at full price through a limited number of retail outlets.

Basic Direct Sales: In this variant, product sales agents made door-to-door house calls to promote and sell both Tunsai and Super Tunsai at full price, R 52,000 (USD 13) and R 92,000 (USD 23) respectively. This variant was implemented from December 2010 through April 2011. Initially, only Tunsai was introduced through the pilot; Super Tunsai was added in February 2011.

Vision Fund Installment Scheme: In this variant, households were offered an installment scheme for CWP purchase in partnership with Vision Fund. Six-month loans with an 18-percent annual interest were offered by Vision Fund for purchase of Tunsai and Super Tunsai; clients could also purchase the products by paying in cash the full price. The primary focus of sales efforts were existing MFI clients but VisionFund offered the CWP loan product to non-clients as well. This pilot was implemented from February to July 2011. Pilot partners continued implementation after the end of the pilot.

There were 10 direct sales representatives, called Clean Water Experts, covering the basic direct sales and MFI pilot areas. All received a training developed by PATH as part of a larger tool kit for Hydrologic that included guidance on recruiting optimal sales personnel and using sales tools such as a highly visual flip-book that has been field tested and translated in Khmer to help salespeople.

Retail Sales Pilot Variants

The retail sales model was implemented from March to July 2011. Three retail sales variants were piloted. In each, the products were distributed through participating retail outlets. The objective of these variants was to test the price sensitivity of Super Tunsai and the feasibility of discount schemes through coupons. Since commercial enterprises like Hydrologic could be impaired when their product is given away for free through NGOs, the retail sales pilot aimed to test whether a coupon delivery scheme could be a viable way for NGOs to target subsidies for CWP purchases to the lowest income populations without undermining the commercial market.

Radio and TV advertising was conducted for three months in pilot areas (April-June, 2011),

and sales were through existing retailers at the community/village level.

Basic Retail Sales: In this pilot area, both CWP versions were available at full price from retailers.

Retail Sales with a Coupon Discount of R 20,000 for Super Tunsai: In this pilot area, Tunsai was sold at full price, while Super Tunsai was sold along with coupon offers for consumers for a R 20,000 (USD 5) discount.


Retail Sales with a Coupon Discount of R 40,000 for Super Tunsai: In this pilot area, Tunsai was sold at full price, while Super Tunsai was sold along with coupon offers for consumers for a R 40,000 (USD 10) discount. With the coupon, the consumer price for Tunsai and Super Tunsai was the same in this pilot variant.

Marketing and communication activities through mass media (such as TV, radio,

newspapers and magazines), local media (billboards, shop signs) and events were implemented as part of the pilots in both provinces.

2.4. Partner Roles and Responsibilities

In both the retail sales pilot and the door-to-door direct sales pilot, Hydrologic Social Enterprise, with support from PATH led the model implementation. In the MFI partnership model, VisionFund played the role of the CWP orders aggregator and financier. The specific roles of the partners are summarized in Table 2.

Table 2. Pilot Partners’ Roles in Implementation of the SWP Models

Door-to-door Direct Sales

Retail Sales MFI Partnership

Model Design Initiated by PATH, and finalized

through consensus with

Hydrologic

Initiated by PATH, and

finalized through consensus

with Hydrologic

Initiated by PATH, and

finalized through consensus

with Hydrologic and

VisionFund

Partnership facilitated by

PATH

Sales Pitch By Hydrologic‟s direct sales

representatives

Sales training and material

designed by PATH

By retailer

By Hydrologic‟s direct sales

representatives

Sales training and material

designed by PATH

VisionFund played facilitator

role in organizing the

meeting

Demand

Aggregation

Orders fulfilled by Hydrologic

sales representatives

Orders fulfilled by retailer Orders collected by

Hydrologic sales

representative.

Loan forms processed by

VisionFund

Financing Not applicable Not applicable Loan assessment, approval,

disbursement and collection

by VisionFund


3. Evaluation Approach

3.1. Objectives of the evaluation

The evaluation sought to provide information for the SWP’s six key questions (listed in the

introduction section) in the context of the pilots in Cambodia. In addition, the pilot in Cambodia sought to compare the effectiveness of the basic direct sales and MFI sales models in increasing uptake of CWP. The retail sales variants sought to explore the impact of different coupon discount schemes on CWP uptake.

The key research questions for the Cambodia pilots were:

1. What is the extent of uptake,5 correct, and consistent use among the target population of the CWP products promoted through each pilot variant?

2. What is the added impact of the direct sales pilot variants (basic direct sales and MFI sales) on uptake and use of CWP, compared with retail sales alone?

3. What is the impact of the coupon discount schemes on uptake of CWP?

4. What are the motives and barriers to purchase and use of CWP?

5. Can the commercial partners earn a profit from sales to target consumers?

6. Will commercial partners continue and/or scale up the SWP pilot business

models to reach target consumers?

7. What other efforts/inputs/incentives are needed to stimulate supply and demand for CWP to target consumers?

Additional topics of interest to pilot implementers that were addressed by the research

included an assessment of:

effect of promotional and marketing activities on CWP sales;

consumer preferences for Tunsai compared to Super Tunsai in the context of the offers available through the pilots;

5 Throughout this report, the terms „uptake‟ and „purchase‟ are used interchangeably when referring to CWP. Some

households obtained a CWP for free or at a subsidized price from NGOs but for the purposes of all analyses in this

report such households are included in „purchasers‟.


feasibility of using coupons as an NGO subsidy model.

As in all other SWP pilots, a baseline study was conducted among the population targeted by the pilot to inform the design of pilot activities, particularly the marketing campaign. The baseline study aimed to assess population characteristics, HWTS practices, and awareness and perception of CWP (product category and brands).

3.2. Evaluation methods

The key research questions and additional related topics considered in evaluating the set of

SWP pilots in Cambodia were addressed through a mixed-methods approach which included:

Quantitative study using a population based survey of target consumers;

Qualitative (exploratory) study of target consumers and sales representatives;

Assessment of commercial viability and implementation of the pilot models.

3.2.1. Quantitative evaluation study

Study design The quantitative study based on household survey data employed two evaluation designs:

1. A pre-post evaluation design (using baseline and endline survey data) was employed to

compare changes in CWP uptake and use among MFI members.

2. A post-only design with intervention and control groups (using only endline survey data) was employed to evaluate and compare the impact on CWP uptake and use by the general population of the basic direct sales variant and the MFI sales variant vis-à-vis a control group of households in an area where neither of the two pilots was implemented.

The post-only design to evaluate the impact of the basic direct sales variant was selected due to the research budget implications of the very large sample sizes required to implement a more rigorous design such as a pre-post or difference-of-differences design.6 However, the results of the baseline survey were used to assess the extent to which the control and basic direct sales area groups were comparable prior to the pilot;

6 The very high data collections costs estimated for these two evaluation designs prior to pilot implementation were

due to the fact that there were seven pilot variants and a control group, with numerous inter-group comparisons of

interest.


this in turn helped assess the relative strength/rigor of the post-only design in this context. The data informed pilot implementation and was also used to assess the extent of comparability of the basic direct sales group and the control group in terms of socio-demographics, drinking water sources, determinants of HWTS, and the key outcome indicators of CWP uptake and use.

A general population survey was not conducted in the MFI pilot area at baseline because sales through VisionFund had been expected to be among MFI members only (as described in greater detail in the following section). Thus, the measurement of the impact of this pilot variant could only rely on a post-only comparison with the control group.

While a randomized controlled trial design would be best suited to measure the effect of

specific pilot elements on the likelihood of CWP uptake and use by an individual household, this design could not be implemented in the context of the SWP pilots. Randomization of individual households or villages into pilot and control groups was not feasible due to the fact that the pilot was used concurrently to assess the implementation feasibility and commercial viability of the sales models. The latter goal required entire communes or districts to be covered by the same type of pilot intervention, ensuring that distribution of and sales activities reflect what a “real life” distribution model in a scaled up intervention would look like.

Data for the study was collected using a baseline and endline household surveys. The

surveys were cross-sectional, with different samples representative of each group selected in each survey round. The baseline survey was conducted in January and February 2011 (during the dry season) and the endline in October-November 2011 (during the wet season). Study sample

The sample size for the study is summarized in Table 3.

Table 3. Household Survey Sample

(number of households)

Control Group

Basic Direct Sales MFI Direct Sales

Pilot Area

“Try before you buy” Direct

Sales1

3 Retail Sales Areas

MFI Members

Baseline (Jan-Feb ’11)

470 (47 villages)

470 (47 villages)

n/a 470

(47 villages) 1,410

2

(119 villages) 473

(30 villages)

Endline (Oct-Nov ’11)

481 (32 villages)

1,005 (67 villages)

1,380 (58 villages)

n/a n/a 589 (57 villages)

Notes: 1 Sample from an area that was designated for a try-before-you-buy direct sales pilot which was planned prior to baseline

but was not implemented due to operational challenges. 2 Includes households from all three retail sales pilot variants.


The sampling design differed for the baseline an endline surveys. At baseline, a required sample of 470 households was determined for each of seven pilot

variant areas (non-overlapping geographical areas). This sample size was determined by following requirements:

Separate estimates were required for each pilot variant (including the control group, and each of the three separate retail sales groups). Uptake of CWP across pilot areas at baseline was estimated to be about 4% (prior to design of the study), based on sales data and other research by PATH;

Margin of error acceptable is +/- 2% points;

95% significance level. The sample size requirement accounted for an estimated design effect size of 1.2 and a 95%

expected response rate.

At endline, the required sample size in each group was determined to reflect modifications in evaluation goals resulting from the following changes in pilot implementation compared to plans at baseline:

CWP sales data from Hydrologics’ sales records in the three retail sales pilot areas suggested that a significant in magnitude increase in population level uptake of CWP between baseline and endline was unlikely. Therefore, pilot partners decided to exclude the retail sales pilot from the quantitative evaluation with household survey data.

VisionFund decided to offer the CWP installment plan to non-clients. As a result, a substantial portion of CWP sales through the MFI variant were to new VisionFund clients who joined the MFI to be able to access the CWP loan. These developments resulted in a new pilot target group that had not been included in the baseline survey – households living in VisionFund catchment areas. Measuring the uptake of CWP within this group and how it compared to the control group and to the basic direct sales group was of interest to pilot implementers. Therefore, a sample of households (general population survey) from MFI pilot areas was added to the endline survey.

As mentioned earlier, there was overlap of the MFI and the basic direct sales pilot activities in some areas. These areas were excluded from the endline survey.

A “try before you buy” variant was not implemented and this group was excluded from the endline survey.

As a result of these changes, and based on updated expectations of CWP uptake rates at

endline (according to sales records), the required sample increased in some of the four pilot groups to ensure the levels of precision of the endline survey estimates for CWP uptake that are shown in Table 4. These sample size requirements were higher than the requirements for the pre-post comparison within the MFI members group, and the post-only comparisons among


the control and two direct sales groups (i.e. the sample sizes in Table 4 allowed for 80% power at 95% significance level of CWP uptake differences in the inter-group comparisons).

Table 4. Sample Requirements Estimation for Endline Household Survey

Group Estimated CWP uptake rate

3

Acceptable margin of error

Sample size

Group 1: Population based survey in Control Group villages

4% +/- 2% 470

Group 2: Population based survey in in villages where only the Basic Direct Sales pilot variant was implemented

9% +/- 2% 1,000

Group 3: Population based survey in in villages where only the MFI pilot variant was implemented

1

16% +/- 2% 1,370

Group 4: MFI clients in villages where only the MFI pilot variant was implemented

1,2

50% +/- 5% 400

Notes: 1 The sample size estimates for groups 3 and 4 include a finite population correction for an estimated total population

of 6,744 households in group 3 and 1,679 households in group 4. 2 The sample size includes an estimated 342 MFI clients expected to be in the population based survey of MFI villages

(given the average MFI membership rates at village level), and a “booster sample” of 60 MFI members to be selected from villages not surveyed as part of the MFI area general population survey. 3 Estimate based on sales data as of July 2011.

In both survey rounds, a two-stage systematic random sampling process was used to select

the households. First, villages were selected using probability proportional to size sampling (whereby size was the number of households in each village). In each village, a fixed number of households were selected.

At baseline, the sample of MFI members was selected using sequential random selection of

groups, and all group members in selected groups were included in the study. At endline, the majority of MFI members were recruited for the study through the population survey in MFI area villages. A “booster sample” of 60 MFI members was selected from additional villages that were not surveyed as part of the MFI area general population survey to bring the total number of MFI members to the sample size requirement described in Table 3.

It should be noted that as a result of the sampling design for MFI members at endline, 38

percent (or 529 households) of the sample in the MFI area group are MFI members that are also part of the “MFI members” endline group. Conversely, 90 percent (these same 529 households) of the “MFI members” endline group are also part of the MFI area group.


The endline sample included both pre-pilot MFI members and MFI members that joined VisionFund to get a CWP loan.

Data collection instruments

In each of the sampled households, a structured questionnaire was administered to the person in charge of the kitchen (and thus the decision-maker on how the household’s drinking water was to be treated). The questionnaire covered background characteristics of the respondents and the households, general knowledge, attitudes, and practices related to HWTS, and intervention specific information. In addition, data collectors conducted observation of the CWP in households that claimed using the filter at time of the survey. An observation checklist – provided to the research team by PATH and partners - was used to ascertain correct usage.

Checklist for Correct Use of CWP

Observation checklist items

Verification of current use

1. There is water in CWP top chamber 2. There is water in CWP bottom chamber

Assembly 3. CWP filter is on level surface 4. CWP is kept off floor and elevated 5. The 3 components/parts rest evenly on each other

Storage

6. Water is served directly from the CWP 7. Lid for CWP bucket in place

Maintenance

8. Storage chamber is free of visible scum or scaling on inside

Analytic methods The main analytic methods employed in the study include comparison of means and

multivariate regression analyses using logistic regression models. All comparisons are adjusted for differences in socio-demographic and other variables that are likely to affect changes in the key outcome variables (the indicators of CWP uptake and use). The comparisons of means employ the analysis of variance (ANOVA) procedure.

Sample weights reflecting the sampling design and the probability of selection of each

household into the sample were computed and used in all analyses presented in this report.

The data analysis was conducted using the SAS software.


3.2.2. Qualitative assessment

A qualitative study was conducted at endline to assess the perceptions of and experience with pilot activities and promoted products among the target consumer audience, as well as retailers and direct sales representatives participating in the pilots. The study focused on extracting information regarding target consumers’ attitudes and practices in HWTS (particularly CWPs), as well as supplier-side barriers for CWP sales and uptake. The study was conducted in collaboration with SBK Research/Cambodia in October and November 2011, during the wet season.

The qualitative assessment included respondents from each of the pilot variant areas of the

direct sales and the retail sales models. The methodology used for data collection was a mix of Focus Group Discussions (FGD) with consumes, with approximately six respondents per group, and in-depth interviews (IDIs) with consumers and CWP direct sales representatives and retailers. Table 5 summarizes the study sample size.

Table 5. Sample Size for Qualitative Study

Segment IDIs FGDs

Consumers:

Consumers who purchased Tunsai 6

-

Consumers who purchased Super Tunsai 32

-

Consumers aware of CWP who had not purchased it -

12

Sales representatives

Retailers 5

-

Direct sales representatives 5

-

The selection of retailers and direct sales representatives ensured a mix of respondents who had relatively low and relatively high CWP sales. Respondents in the VisionFund pilot areas included both new members who had joined the MFI in order to avail of the CWP loan, and members who had been with VisionFund prior to the pilot. A more detailed description of the sample, as well as the study results is presented in a separate report produced by Abt Associates for the qualitative study.


Analysis of the data from the FGD and IDI transcripts was conducted by organizing the results around the key themes for the research. The NVIVO software was employed in the analyses.

3.3.3. Assessment of commercial viability and implementation

We used project cost and sales data and information from interviews with project partners. Along with survey research data, this information was used to answer research questions related to commercial viability of the direct sales models (basic direct sales and MFI partnership model). Specifically, we examined:

Channel efficiency: We examined the sales conversion over the pilot period with respect to the implementation challenges that came up to establish benchmarks for analysis of commercial viability.

Channel effectiveness: Examined market potential and achievements during the pilot including levels of and increases in awareness and uptake of Tunsai/SuperTunsai, the potential market size, and effectiveness of marketing activities.

Channel sustainability: We utilized project cost, sales, and revenue data collated by PATH in the form of a profit and loss statement. For Hydrologic, this was used to analyze operating costs and projected sales revenue to identify the level of commercial viability, using the framework shown in Table 6.

Table 6: Levels of Commercial Viability

Sustainability

Level

Component Cost Recovered

Product Distribution Marketing Overheads

& Profit

Level 1 No No No No

Level 2 Yes No No No

Level 3 Yes Yes No No

Level 4 Yes Yes Yes No

Level 5 Yes Yes Yes Yes

The levels of commercial viability depend on the cost components recovered. Level 1

indicates that none of the cost components – production, distribution, marketing or overheads and profit have been recovered. At Level 2 only production costs are recovered, at Level 3 production and distribution costs and at Level 4 production, distribution and marketing are recovered. Level 5 indicates that all the cost components are recovered by the project and that it is completely commercially viable.


We interviewed Hydrologic, VisionFund, and PATH representatives to understand how the project was implemented, whether it met their expectations, the challenges encountered, and the lessons learned. Through these interviews, we examined if the current design could be adopted by Hydrologic and VisionFund as is, and changes (if any) that the partners would recommend based on the pilot project’s learning.

Based on the ‘levels of commercial viability’ framework and information from interviews

with partners and supply chain actors about their experience with the pilot and intentions to continue and/or scale-up the pilot, we assessed the overall commercial viability and potential of the pilot.


4. Evaluation Results

This section presents and discusses the results from the three evaluation components:

Quantitative study using a population based survey of target consumers;

Qualitative (exploratory) study of target consumers and sales representatives;

Assessment of commercial viability and implementation of the pilot models. The results from the quantitative and qualitative research with consumers are presented

first, followed by the results from the commercial viability assessment. The presentation of the data from the household surveys in most sections is organized to

facilitate the main comparisons corresponding to the the evaluation design: (1) a post-only (endline) comparisons of the basic direct sales and MFI pilot general population groups vis-à-vis the control group; and (2) a pre-post (baseline vs. endline) comparison for MFI members. In all figures and tables where these two comparisons and made, the statistical significance of differences between groups is indicated as follows:

One star (*) indicates a difference significant at the 95% level (i.e. p<0.05).

Two stars (**) indicate a difference significant at the 99% level (i.e. p<0.01).

The star notations are placed next to the values for each of the pilot intervention groups, and refer to a comparison vis-à-vis the control group. Lack of a star notation indicates that the difference was not statistically significant at the 95% level (i.e. p≥0.05).

“N” in all table column/row headings refers to the sample size or denominator for the respective indicators.

4.1. Background Characteristics of Surveyed Respondents

Tables 7a and 7b present the background characteristics of the household survey

respondents and their households. About half of respondents were 40 years of age or older, and one in five did not have formal education. On average, households had five to six members, and about half had at least one child under five. There were some differences in the age, education, and household composition profiles of the three groups. Respondents in the control group sample were somewhat younger than in each of the two pilot groups, and a higher proportion of the control group than the basic direct sales group had primary or no formal education. The differences between groups in average respondent age and education are taken into account (controlled for) in the comparisons of key pilot outcomes, as they might affect these outcomes.


While the design of the survey was not designed to measure reliably the prevalence of diarrhea, we enquired about recent diarrhea episodes among household members to gain an overall idea of the potential magnitude of waterborne disease in study areas. About 4-6 percent of children under five years and 9 to 16 percent of adults in each area were reported to have had diarrhea in the two weeks preceding the survey (which took place during the dry season at baseline and during the wet season at endline). Diarrhea episodes were reported more frequently in the control group than in each of the pilot groups. The potential reasons for this difference could include differences in HWTS practices (which are explored later).

It should be noted that prevalence of diarrhea was not measured as an outcome indicator in

the SWP pilots, because the pilots focused on promotion and distribution of HWTS methods already proven to be effectively disinfecting drinking water, and because effective HWTS is only one of several key factors that influence diarrheal disease (e.g. hygiene and sanitation practices are other important determinants).

Table 7a. Background characteristics of survey respondents:

control vs. direct sales groups

Population Characteristic

Basic Direct Sales %

(Endline)

MFI Pilot Area %

(Endline)

Control Group % (Endline)

N=1,005 N=1,380 N=481

Respondent Age --- ** --- * ---

18-29 yrs 16.9

18.7

21.1

30-39 yrs 20.4

23.7

31.4

40-49 yrs 27.8

27.5

24.3

50 + yrs 34.9 30.1 23.2

Respondent Education --- * ---

---

No schooling / No formal schooling 22.0

27.8

22.4

Primary school (K1-6) 53.0

54.2

63.4

Lower secondary (G7-G9) 19.9

15.8

12.3

Upper secondary (G10-G12) or higher 5.1 2.1 1.9

Total household members 5.4 5.4 5.4

Household with children under 5 yrs 45.3 48.6 47.1

Any member of household had diarrhea in the last two weeks?

11.0 * 8.7 ** 15.8

Any child under 5 yrs had diarrhea in the last two weeks?

4.4 4.6 6.4

The socio-demographic profile of MFI members (Table 7b) was largely similar to the profile

of the other pilot groups, which reflects to a large extent the fact that the majority of MFI members (90 percent of the sample) are also part of the sample for the MFI pilot area group described in Table 7a. There were no significant differences in the age, education and household composition profile of MFI members in the baseline and endline samples. Nevertheless, we adjust pre-post comparisons of key pilot outcomes for differences in these characteristics.


The proportion of MFI member households reporting recent episodes of diarrhea among

both adults and children under five is significantly lower at endline than at baseline. This difference could be a function of a number of factors related to seasonal differences in households’ primary water sources and changes in HWTS practices (explored later). Of note, care should be taken in inferring diarrheal prevalence in a population on account of household recall at only one point in time (rigorous methods for measuring diarrhea prevalence were not in the scope of this study).

Table 7b. Background characteristics of survey respondents and households: MFI members

MFI Members

Baseline % Endline %

Population Characteristic N=473 N=589

Respondent Age

18-29 yrs 15.6 14.8 ---

30-39 yrs 25.6 24.5 ---

40-49 yrs 32.1 29.6 ---

50 + yrs 26.6 31.1 ---

Respondent Education

No schooling / No formal schooling 28.6 29.9 ---

Primary school (K1-6) 52.0 58.9 ---

Lower secondary (G7-G9) 15.2 9.5 ---

Upper secondary (G10-G12) or higher 4.2 1.8 ---

Total household members 5.4 5.6

Household with children under 5 yrs 51.0 51.4

Any member of household had diarrhea in the last two weeks?

22.3 10.2 **

Any child under 5 yrs had diarrhea in the last two weeks?

16.8 6.5 **

Figure 3 illustrates the wealth quintile distributions of the five sample groups. We computed a wealth index for the surveyed households based on reported ownership of 61 durable assets and housing conditions, and standardized the index quintiles in our survey based on the wealth quintiles distribution in the 2010 Cambodia Demographic and Health Survey (which uses the same components and methodology for constructing the wealth index). This allows for a direct comparison of each quintile in the pilot area with the corresponding quintile in the country and in Kampong Speu province.

Within each group in the study area at endline, 80 to 87 percent of households are in the

middle three quintiles, which constitute the core target group for the SWP. In Cambodia as a whole 60 percent of households belong to this wealth group. This indicates that the selection of the pilot areas and the coverage of pilot interventions effectively targeted the project’s core group. The proportion of study households that belong to each of the poorest and the richest


wealth quintiles is notably smaller than the share of households nationwide that belong to these two wealth quintiles.

The basic direct sales group has a wealthier profile than the control group: 60 percent of

households in the basic direct sales area are in the upper two (richest) quintiles compared to only 34 percent in the control group. The wealth distribution of households in the MFI pilot area is not significantly different than in the control group area.

Among MFI members, 52 percent of the baseline sample was in the richest two quintiles,

compared to 32 percent of the endline sample. This difference could be a function of changes in the profile of MFI clientele over time that are not related to the pilot, changes in clientele that could be related to the pilot (e.g. the wealth profile of those who joined VisionFund in order to avail of the CWP loan could be different than that of existing members), and the cross-sectional sample design (different samples were selected, although the sampling process was designed to ensure that the sample in each survey round was representative of the MFI members population in the study area7).

The inter-group comparisons of key pilot outcomes presented later are adjusted for

differences in wealth quintiles distribution of the groups.

Figure 3. Wealth quintile distribution of study sample

3

7 This could be due to the villages that were excluded from the endline survey sampling frame due to overlap of the

basic direct sales and MFI pilots (e.g. the overlap activities could have targeted richer villages).

Endline Baseline vs. Endline - MFI Members


Table 8 compares the wealth quintile distribution of the endline study sample to the general

population in Kampong Speu province (reported in the 2010 Cambodia DHS). The wealth profile of the province mirrors that for the country as a whole (i.e. approximately 20 percent of households in the province are in each quintile). The study sample, on the other hand, is concentrated in the middle three wealth quintiles, with only 6 percent of households in the poorest quintile and 9 percent in the richest quintile.

Table 8. Wealth quintile distribution of study sample compared to Kampong Speu province

DHS 2010 Data

SWP Study Sample (endline)

Kampong Speu Province

Study Areas in Kampong Speu Province

Wealth index

Poorest 21% 6%

Poorer 19% 18%

Middle 23% 32%

Richer 24% 35%

Richest 13% 9%

Total 100% 100%

As depicted in Figures 4a and 4b, TV is the leading source of media exposure with about 60

percent of respondents in each of the endline study groups reporting they watch TV almost every day. About a third of respondents reported listening to the radio almost daily. However, a substantial proportion of respondents rarely or never get exposure to each of these two media: in each group, about half listen to the radio less than once a week and one in five watches TV less than once a week or not at all. This implies that while radio and TV are opportune channels for advertising to many households, neither can be relied upon as the sole channel to reach effectively with promotional/advertising messages the vast majority of households in this area. Not shown in the figure is the finding that about 80 percent of the pilot population, on average, never reads the newspaper.


Figure 4a. Frequency of exposure to media: control vs. direct sales groups

The findings on media exposure are overall similar among MFI members (Figure 4b). There are notable differences between the baseline and endline samples in the frequency of both TV and radio exposure with the respondents in the endline sample reporting greater exposure to each of these channels.

Figure 4b. Frequency of exposure to media: MFI members


At endline (wet season), rainwater was the predominant source of drinking water for all groups (80 to 90 percent of households) and there were no substantial differences across groups in the overall distribution of households across types of sources (Figures 5a and 5b).

At baseline (dry season) surface water was the major source of drinking water in the direct

sales area, used by about 50 percent of households, followed by dug wells (about 20 percent). Rainwater was the source for only 9 percent of households at baseline.

Figure 5a. Main source of drinking water at endline:


This seasonal difference in type of water source is reflected in the pre-post comparison for MFI members (Figure 5b). Unsurprisingly, MFI members at endline used at similar rates the types of water sources used by households in the control and direct sales groups.


Figure 5b. Main sources of drinking water in current season at baseline (dry) and endline (wet): MFI members only

4.2. HWTS Knowledge, Perceptions, and Practices

Although there were major differences in the predominant type of water source used

during the dry and rainy season (as discussed in the previous section), the extent of perceived need for drinking water treatment was similar in both seasons.

At baseline (dry season), 84 percent of respondents in the direct sales areas rated the

overall quality of their drinking water as average or bad (versus good/excellent). This compares to about 70 percent of respondents across the endline (wet season) survey groups. Similarly, 90 percent of respondents stated at baseline that treating water at that time was absolutely necessary or good to do, and about 95 in each group surveyed at endline stated this level of treatment need. The qualitative study also found high levels of knowledge of health risks associated with drinking unsafe water.

In summary, even though water sources changed between seasons, the overall

perceived need for water treatment remained at a similar level. This finding indicates that the reliability of the pre-post comparisons of HWTS practices that our evaluation uses to assess the impact of the pilots may not be affected substantially by the fact that the baseline and endline surveys were conducted in different seasons.


Comparison of the evaluation study groups (Figure 6) does not reveal substantial

differences in the level of perceived need to treat water.

Figure 6. Perceived water quality and need for water treatment

Figures 7a and 7b summarize the most frequent issues with the quality of source water. Visible impurities and perceived germs contamination were the problems mentioned by most respondents as occurring at least once in the past week (about 70 to 80 percent across groups). Bad or unpleasant smell and taste of source water were also a problem with about half of respondents reporting these as issues with their drinking water in the week preceding the survey. Each of these four water source quality issues was reported more frequently in the control group than in the direct sales groups, though the differences were not large (Figure 7a).

The results from the qualitative study were aligned with these findings. The attributes of

“good” or “bad” water mentioned by study participants focus on visual clarity, taste and smell. Good water is characterized as “tasty”, transparent and not having any smell; whereas bad water was said to be “sour”, and to have unpleasant smell, particles, and scum.


Figure 7a. Drinking water issues: control vs. direct sales groups (% of respondents reporting this was an issue at least once in past week)

Figure 7b. Drinking water issues: MFI members


In each study group, 80 percent or more of households reported regularly treating their drinking water (defined as treating every day or almost every day), Figure 8a. Boiling was the predominant HWTS method, used by 73-78 percent in each group, followed by use of a CWP (explored in greater detail in the following section). Regular use of other HWTS methods (such as mineral pot filters, bottled water, settling and decanting) was very low.

A greater proportion of respondents in the control group reporting they did not treat

their water (20 percent) compared to the pilot groups (12 percent).

Figure 8a. Regular use of HWTS: control vs. direct sales groups (self-reported use daily/almost daily)

Among MFI members, between baseline and endline there was a significant decrease in boiling (from 76 to 57 percent), as well as in use of other treatment methods such as settling and decanting (Figure 8b). At the same time, use of CWP increased dramatically, from 3 to 38 percent. This switch from boiling to use of CWP is explored further in the following section.


Figure 8b. Regular use of HWTS: MFI members (self-reported use daily/almost daily)

76%

3% 3%

23%

57%

38%

1%

9%

Boil CWP Mineral Pot Filter Other

MFI Members - Baseline MFI Members-Endline

**

**

Annex A presents detailed data on HWTS rates of ever use and current use (i.e. at time of

the survey). The results for current use mirror the results on self-reported regular use discussed above. The overall pattern of HWTS rates of ever use compared across groups and methods are largely similar to the patterns seen for current and regular use.

4.3. Effect of Pilot Interventions on CWP Purchase and Use

This section focuses on the results related to the effects of the SWP pilot interventions,

including awareness and use of CWP (as a category and the brands promoted through the pilot), knowledge and perceptions of the products, and barriers to purchase and correct use.

4.3.1. Product Awareness and Sources of Information

Overall awareness of CWP as a category was 47 percent in the direct sales areas surveyed at

baseline. At endline, CWP awareness was 57 percent in the control group and significantly higher in each of the two direct sales groups (77 and 86 percent). Among MFI members, the levels of CWP awareness more than doubled, from 42 to 93 percent (Figure 9). These results


indicate that the targeted nature of CWP promotion activities through VisionFund meetings is very effective in rapidly increasing knowledge about the product.

Figure 9. Awareness of CWP (product category)

Exploring CWP awareness across brands (Table 9) shows that Tunsai/Super Tunsai was the

most widely recognized CWP brand at endline and that the results on category awareness largely reflect an increase in knowledge about the Tunsai/Super Tunisai brand.8 Thus the increase in CWP category awareness could be attributed to a great extent to the marketing and advertising activities conducted through the pilots (as these focused on the product brand).

Table 9. CWP brand awareness

Item

Basic Direct

Sales % (Endline)

MFI Pilot

Area % (Endline)

Control

Group % (Endline)

MFI Members Group


N=1,005 N=1,380 N=481 N=473 N=589

Brand Awareness

% of respondents who are aware of Tunsai CWP brand

41.90 * 21.70 ** 32.65 18.73 24.69

% of respondents who are aware of Super Tunsai CWP brand

25.20

58.36 ** 19.32 --- 72.30

% of respondents who are aware of Frog CWP brand

0.49

0.34 * 1.86 0.23 0.00

% of respondents who are aware of Cambodian Red Cross CWP brand

0.59 * 0.42 * 0.00 0.20 0.32

8 It is likely that many consumers did not differentiate Tunsai and Super Tunsai as two separate brands


This conclusion is supported further by the results on recall of messages on Tunsai and Super Tunsai in the six months preceding the endline survey (Figure 10): hearing or seeing messages on the brand was mentioned more frequently by respondents in the direct sales areas than in the control area. The difference in recall of Tunsai vs. Super Tunsai between the basic direct sales are and the MFI pilot area could reflect to some extent the fact that for the first two of the five months of the direct sales pilot only Tunsai was promoted by the pilot (with Super Tunsai introduced in the third month), whereas in the MFI pilot both brands were introduced from the beginning and promoted over the six months of pilot implementation.

Figure 10. Brands of CWPs on which seen/heard messages in past six months

(% of respondents)

The door to door salesmen were mentioned as a main source of messages on Tunsai/Super

Tunsai at endline in pilot areas (recalled by about 60 percent of respondents), while TV or radio was the most frequently cited source in the control area where half recalled messages on the promoted product through media (Figure 11a). VisionFund meetings were an equally important source of messages in MFI areas, and the predominant source recalled by MFI members (Figure 11b). Word of mouth is another important source of information on Tunsai/Super Tunsai: in all groups in the study population half of respondents mentioned friends, neighbors, or relatives discussing the products.

Non-governmental organizations (NGOs) and community based organizations (CBOs) were

not a major source of information on the products promoted through the pilots in the study areas. In each of the study groups NGOs/CBOs were mentioned by 10 percent or less as the first


source of information on CWP (i.e. source from which respondents first heard of the CWP product category).

Figure 11a. Source of messages on Tunsai/Super Tunsai heard/seen in past six months (at endline): control vs. direct sales groups

Figure 11b. Source of messages on Tunsai/Super Tunsai heard/seen in past six months (at endline): MFI members


These findings from the household survey were reiterated in the qualitative study where respondents indicated that their primary sources of information regarding Tunsai CWPs were the TV and radio commercials. Signage, displays at retailers and connections with VisionFund and Hydrologic staff were also important sources. Product demonstrations both live and via media, were seen as very useful in understanding how the product works and clearly stood out in respondents’ minds among all other promotional channels.

The findings in the household survey on knowledge about where one could purchase a CWP show that retailers (at shops or markets) are the most widely known source of availability of CWP in the general population across study groups, followed by door to door salesmen (Figure 12a); while VisionFund is the most frequently known source among MFI members (Figure 12b) and as known as retailers as CWP source in MFI pilot areas. Within each study group, NGOs and CBOs were cited by 10 percent or less as a source of availability of CWPs which indicates that such organizations were not significant players in the distribution of CWPs in the study areas.

At endline, about half of respondents in the control group knew at least one place where

CWP was available, compared to 73 percent in the basic direct sales group and 82 percent in the MFI area group; this proportion was highest among MFI members, 89 percent (which compared to only 28 percent at baseline).

Figure 12a. Knowledge about CWP source of availability:


One notable finding here is that about a third of MFI members did not mention VisionFund

as a place where they could obtain a CWP. This could relate to the finding that only half of respondents in the VisionFund members group reported that in the six months prior to the endline a member of their household had attended a meeting of the MFI where the Tunsai/SuperTunsai loan offer was promoted.


Figure 12b. Knowledge about CWP source of availability: MFI members

We explored the knowledge of the price of Tunsai and Super Tunsai among respondents

who knew of the products. About 20 percent of these respondents said they did not know the price of Tunsai and 10 percent did not know the price of Super Tunsai.

Table 10 summarizes the results among those who “knew” the price. Knowledge of the

approximate price of Super Tunsai was better in the MFI pilot than in the other groups, while knowledge of the price of Tunsai was better in the basic direct sales group. This could reflect in part the initial promotion of Tunsai only in the basic direct sales area (in the first two months of the pilot). While 10-20 percent of those who knew of the promoted products could not state the price, for each product less than 5 percent stated a price higher than the range shown in Table 10. This indicates that misinformation about the price of Tunsai/Super Tunsai whereby price is thought to be higher was not an issue in the study areas (which would have implications for affordability perceptions, discussed in the following section).


Table 10. Knowledge about price of Tunsai and Super Tunsai (among respondents who stated they know the product price)

Item

Basic Direct

Sales % (Endline)

MFI Pilot

Area % (Endline)

Control

Group % (Endline)

MFI Members Group

Baseline %

Endline %

% respondents who report the price of Tunsai Brand CWP to be between 40,000 and 60,000 Riel (actual price is R 52,000)

80.97 *

* 67.68 * 55.89 71.71 70.30

% respondents who report the price of Super Tunsai Brand CWP to be between 80,000 and 100,000 Riel (actual price is R 92,000)

64.21 81.44 * 74.82 n/a 77.36

3.2.1. Purchase and Use of CWP

At baseline, about 7 percent of households in the direct sales area had purchased a CWP9

and 2 percent reported using it at the time of the survey. Regular use of CWP (every day or almost every day) was very low, at 1.4 percent in the basic direct sales area and 1.1 percent in the control area; while 2.4 percent of MFI members reported regular use. The difference between the control and direct sales group was not statistically significant.

At endline, CWP purchase in the control group had remained at a similar level (7 percent)

but reached 16 percent in the basic direct sales area and 21 percent in the MFI area (Figure 13a).

Current use of CWP was 5 percent in the control group, compared to 12 percent in the basic

direct sales group and 17 percent in the MFI area group. These are rates of self-reported use by respondents but observation of the CWP confirmed that in nearly all cases the CWP was indeed in use: in 97 percent of households reporting current use, data collectors observed water in the bottom chamber of the device. The rates of regular use of CWP were nearly identical to the rates of current use in all groups.

The differences between the control and each of the pilot groups are statistically significant

for the indicators of CWP purchase, current and regular use. Additionally, the difference in rate of CWP use between the basic direct sales and MFI area group is statistically significant (p<0.05).

There was a high rate of regular CWP use among purchasers in all groups: 74 percent in the basic direct sales area, 81 percent in the MFI area, and 75 percent in the control area.

9 “Purchase” refers to purchasing a CWP at full or subsidized price or obtaining it for free (e.g. from an NGO) and

thus reflect CWP uptake.


Figure 13a. Purchase, current use, and regular use of CWP: control vs. direct sales groups

Among MFI members, the increase in CWP purchase and use was dramatic: while 6 percent had purchased a CWP at baseline, this was the case for 43 percent at endline (Figure 13b). The increase in reported current CWP use increased more than tenfold, from 3 to 39 percent. As with the other study groups, the rates of regular use mirrored the rates of current use and use among purchasers was very high (90 percent of those who purchased reported using the device).

Figure 13b. Purchase, current use, and regular use of CWP: MFI members


In each of the pilot groups, 7-10 percent stated that they were planning to purchase a CWP or were waiting for their VisionFund loan to be approved. Additionally, among MFI member purchasers, 44 percent reported that their first loan from VisionFund was for the purchase of Tunsai or Super Tunsai.

Purchase and use of CWP across wealth segments We explored CWP purchase rates within each Cambodia-specific wealth index quintile. In

each quintile, uptake of CWP was significantly higher in the basic direct sales and MFI pilot groups compared to the control group (Figure 14a). There was no clear gradient in uptake across quintiles in the control group, and this was also the case overall for the basic direct sales group. By contrast, in the MFI pilot area uptake increased from 12 percent in the poorest quintile to 18-19 percent in the second and third quintiles, to 24-26 percent in the two richest quintiles.

Exploring the difference in uptake (i.e. absolute difference in percentage point terms) we

see no clear gradient in the difference in uptake between the control group and the basic direct sales group across quintiles; the difference was within a relatively small range across quintiles (from 7 to 10 percentage points). This indicates that the impact of the basic direct sales pilot on CWP uptake was generally similar across quintiles.

The difference in uptake between the control group and the MFI pilot group had a relatively

consistent pattern of increase from 7 percentage points in the poorest quintile to 20 percentage points in the richest quintile, which indicates a stronger impact of the MFI pilot among richer segments of the population than among poorer segments.


Figure 14a. Purchase of CWP in each wealth quintile: control vs. direct sales groups

The difference in the wealth gradients of the two pilot variants are driven primarily by the

outcomes in the poorest and the richest quintiles: the effect of the pilots is the same in the poorest quintile, while in the richest quintile the effect of the MFI pilot is substantially larger compared to the effect of the basic direct sales pilot.

Among MFI members, the pattern in CWP uptake at endline across quintiles (Figure 14b) is

similar to that seen in the MFI area general population: uptake increases from 23 percent in the poorest quintile to 40-45 percent in the second and third quintiles, to 50-53 percent in the richest two quintiles. While there was no consistent gradient in the pre-post difference in CWP uptake across quintiles, the pattern is similar to that seen for the MFI variant in the general population: the effect of the pilot among MFI members is substantially stronger among the richest quintile than among the poorest quintile.


Figure 14b. Purchase of CWP in each wealth quintile:

MFI members

Lastly, we look at CWP uptake within the SWP core target group – the middle three country-specific wealth quintiles (Figure 15). In the control group and each of the two direct sales pilot groups, uptake of CWP within the SWP core target group is practically the same as uptake in all quintiles combined (which was shown in Figure 13a/b) which largely reflects the fact that the SWP core target group constitutes 80 to 87 percent of each study group.


Figure 15. Purchase of CWP within the SWP core target group (middle three wealth quintiles)

Distribution of CWP purchasers by wealth quintile We also explored the distribution of the CWP units purchased across wealth quintiles (i.e.

what proportion of all units were purchased by households in a given quintile), Figure 16. Overall, the pattern of differences across groups seen in Figure 16 is similar to the pattern of the population distribution across quintiles (shown in Figure 3). The majority of households who purchased a CWP were in the middle three quintiles (85% or more in each study group). A larger proportion of purchasers in the basic direct sales area (compared to the MFI area) were from the wealthiest two quintiles.


Figure 16. Distribution of CWP purchasers by quintile at endline

4.3.3. Correct Use of CWP

While the majority (74 percent or higher) of households who purchased a CWP were using it

at the time of the survey, observation of the device by the data collectors using the 8-item checklist described in Table 4 revealed negligible rates of fully correct use at endline: one percent of households in each of the control, basic direct sales, and MFI area groups; and about 3 percent of MFI member households. The most common problem was the most common problem was the presence of scum or scaling on the storage chamber (which could indicate improper or irregular cleaning) – in only 14 percent of households where a CWP was observed the storage chamber was free of visible scum or scaling on the inside. With the exception of this factor, correct use of the CWP among those who used at endline was very high.

Table 11. Components of correct use of CWP observed at endline

Components of Correct Use % of users

Lid for CWP bucket in place 100

Water in CWP filter (upper chamber) 83

Water in bottom chamber 97

CWP is kept off floor and elevated 96

Water is served directly from CWP 97


3 components of CWP rest evenly on each other 100

CWP filter placed on level surface 90

Storage chamber free of visible scum or scaling on inside 14

4.3.4. Triggers and Barriers to Purchase and Use

We explored the triggers and barriers to CWP purchase and use through both the

household survey and qualitative research. What were CWP users doing previously to treat their drinking water?

At baseline, about 70 percent of households in the direct sales study used boiling as a HWTS

method which indicated high “unmet need” (under the hypothesis that those who boil are the lower hanging fruit for CWP). This hypothesis was supported by the data from the endline survey which asked those reporting CWP use what they had done previously to treat their drinking water. Nearly all current CWP users had switched from boiling to CWP (96-100 percent in each group). Very few users reported they did not treat water at all before (less than 2 percent). In addition, the large increase in CWP use among MFI members (33 percentage points) was accompanied by a drop in boiling by 19 percentage points.

As discussed earlier in the section describing overall HWTS practices, boiling remained a prominent practice in all study groups at endline, with 60-80 percent of households across groups reporting boiling as a regularly used method. In addition, there was high felt need to treat water: about 60 percent across groups said it was absolutely necessary/good to treat. This high level of felt need for HWTS remained similar across seasons. These findings indicate that there is a large potential market for CWP in the Kampong Speu areas covered by the study.

Motivating factors for uptake and use of Tunsai/Super Tunsai

We used logistic regression analysis to explore the determinants of CWP uptake. Table 12

compares those who purchased (or obtained free) CWP with those who are aware of CWP but did not purchase/obtain one. To arrive at the main factors influencing uptake, we developed a parsimonious model10 of psycho-social determinants and socio-demographic factors using logistic regression. The logistic odds ratio indicates the strength of association between each factor and the dependent variable (CWP uptake); the higher the number, the stronger the correlation of this factor with the likelihood of uptake. An odds ratio of less than 1 indicates a negative influence/association.

10 Parsimonious means the model that contains the least number of factors without losing a significant explanatory

power.


The results in Table 12 indicate that the factors positively associated with uptake of CWP include: stronger belief in the importance of treating drinking household’s water, positive perceptions of CWP (particularly that it is an effective method, providing better water quality than other methods, and is affordable), membership of VisionFund and being in a higher wealth quintile.

Those who had first heard about CWP from an NGO/CBO, from VisionFund (community

bank group), in a shop, or from a door-to door salesperson, or from friends/relatives were significantly more likely to obtain a CWP than those who first learned about it from the TV or radio. Among these sources of initial information on CWP, NGOs/CBOs and VisionFund particularly stand out as strongly associated with CWP uptake.

Table 12. Determinants of CWP uptake

(results from multivariate logistic regression model)

N=2,403 (respondents who knew of CWP at endline)

Determinants Logistic

Odds Ratio Signif.

Respondent's opinion on the importance of treating their drinking water1

1.6 **

Method Perceptions2

CWP is overall a good method of treating water 2.2 **

Water treated by CWP cannot be boiled and used for making tea 1.2 **

CWP offers better water quality than other methods 2.0 ** I can afford to purchase CWP 2.7 ** Boiling is suitable for my household 1.7 ** Boiling is a smart choice for my household 0.7 *

Opinions on treating drinking water2

Belief that treating drinking water is a hassle 1.2 **

Belief that the drinking water of household needs to be treated 5.0 **

First source of info on CWP (ref: TV/Radio) --- ** At shop 9.3 Door-to-door salesman 7.0 * Community bank group / VisionFund meeting 11.0 ** Friends / neighbors / relatives 1.2 ** NGO / CBO 34.1 ** Saw CWP in public building or in someone's house 1.1 **

Member of VisionFund group 3.1 **

Wealth Index (ref: lowest / lower / middle) --- ** Upper 1.8 Highest 1.5

Among those aware of CWP (n=2403) Statistical significance: **p<.01 *p<.05 1 Item is on a 4 point scale where 1=Not at all required and 4=Absolutely necessary

2 Items are on a 5 point scale where 1=Strongly disagree and 5=Strongly agree


The qualitative study found that the promoted CWP products were associated with a number of positive attributes including effective, time- and cost-efficient treatment method, with capacity to provide sufficient amounts of treated drinking water. Users were keen to recommend the product to others, which is important in the context of the results from the household survey that showed word of mouth as one of the main sources of information on CWP.

Related to these positive perceptions among consumers are the key motivating factors for

use of Tunsai and Super Tunsai that emerged in the qualitative research with consumers. These factors included the known health benefits of the CWP, the time and money savings from its use compared to boiling, and the increased easy availability (in terms of quantity) of treated water. The cost savings on firewood and gas among those who otherwise boiled their water were one of the most prominent benefits mentioned in the qualitative study.

Comparison of User Perceptions of Tunsai vs. Super Tunsai In the qualitative study, one noted difference between the Tunsai and the Super Tunsai user

descriptions of the product is that the style, stance, and design of Super Tunsai was often mentioned as something the consumer liked about the product. There was no mention of this among users of the regular/original Tunsai.

The household questionnaire at endline also asked purchasers of Tunsai or Super Tunsai

about their perceptions of the differences between the two models, and also asked why they purchased the Tunsai versus the Super Tunsai, or vice-versa (depending on the model bought). The responses to these open-ended questions were then grouped/standardized to the extent possible during data entry. When asked about the differences between the two models, about half of the respondents stated that the Super Tunsai “looked nicer” than the Tunsai, and about a third stated that the Super Tunsai had a stand allowing for placing a water glass under the tap.

Among those who purchased the Tunsai, the most frequently cited reasons for purchasing

that model over the Super Tunsai were that only the Tunsai was available (about 70 percent), the Tunsai was less expensive (17 percent), or that they were not aware of the Super Tunsai (6 percent). Super Tunsai purchasers most often cited as their reason for choosing this version that only the Super Tunsai was available (69 percent), that it was better looking (16 percent), and that it had room to put one’s glass under (8 percent

These findings provide an overall indication that Super Tunsai is perceived as a more

aspirational product valued for its aesthetic look, and is also considered more functional in its design features (particularly the water glass stand under the tap which the original Tunsai model did not have).


Consumer perceptions of the pilots’ promotional and sales activities In the qualitative study, there was a high message recall on the following Tunsai/Super

Tunsai attributes: safe water, cost benefits (no boiling- gas/firewood costs), time efficient, and good investment. These messages map to the factors that stood out as positive attributes of the product among consumers, which indicates effectiveness of the messaging through the pilot on influencing consumer attitudes.

In the discussion of the mass media campaign in the qualitative study, respondents

mentioned that they found the mass media campaign to be appealing, in particular, the television ads. They found the TV ads appealing. Many of the respondents were homemakers and thus were able to see the ads frequently. Ads were described as being clear and easy to follow for the most part, and respondents particularly liked the way the ads explained how water was purified. A few respondents referenced the rabbit in the television ad as a positive attribute to the ad design.

The qualitative study revealed that consumers considered product demonstrations and instructional leaflets to be very helpful in understanding how to use and maintain the Tunsai CWP. The demos in particular were highly valued by consumers.

Consumers in retail sales pilot areas were generally impressed with the customer service

that they received from Tunsai retailers. Retailers were considered polite, knowledgeable, and helpful during sales transactions. Many were noted to have demonstrated the use of the CWP, distributed instructional leaflets, as well as provided additional customer support when necessary. However, a few consumers noted that although the retailers was polite, they were very preoccupied with their other customers and duties and were not able to dedicate 100 percent of their attention to the demonstration/explanation of the Tunsai CWP.

Similarly, consumers in the direct sales and MFI pilot areas were overall impressed with the

level of knowledge, marketing skills, and demeanor of the direct sales representatives. Vision Fund staff appeared to be knowledgeable and friendly during their conversations with consumers.

Barriers to CWP Purchase

In each of the study groups, the most frequently cited reason for not purchasing a CWP by respondents in the household survey who knew of the product was lack of felt need for the device, followed by not being able to buy CWP due to financial/affordability issue (Figure 17). Availability of CWP was a more prominent reason in the control group where one in five said they had not purchased it because it was not available/they did not know where they could buy it; by comparison, this reason was cited by only 5-7 percent in the pilot groups.


Figure 17a. Reasons for not purchasing CWP (among those aware of CWP): control vs. direct sales groups

Figure 17b. Reasons for not purchasing CWP (among those aware of CWP): MFI members


In the qualitative study, the price of the CWP emerged as leading barrier to purchase. Even those who purchased felt that others, in poorer socio-economic groups, would not be able to afford the product. Many who purchased Super Tunsai with coupons said that they would not have purchased without the discount; and VisionFund clients noted that the installment scheme was key for their decision to purchase the device.

The household survey also indicated that loans and installment plans were key for CWP

purchase in the MFI area and in the control area: 86 percent of those who had purchased a CWP at endline in the MFI area said they used a loan/installment plan for the purchase. While CWP was available for purchase cash down for MFI members, only 3 percent paid in full (i.e. did not use the VisionFund CWP loan).

Interestingly, loan/installment plans for CWP were used by 63 percent of purchasers in the

control area. This group is comprised of only eight respondents so reliable investigation of the loan source was not possible; of this small sample, some had purchased a CWP through a community bank group, while others from a salesperson who visited their community.

One in five CWP purchasers in the basic direct sales area also reported using a loan or

installment plan. Again, the number of respondents reporting use of loan/installments is very small; of them, the majority had purchased CWP through a salesperson who visited their house/community, while a few had purchased through a community bank group.

Reason for non-use of CWP among purchasers Table 13 summarizes the reasons for non-use among those who purchased CWP but were

not using it at endline. As discussed earlier, non-users comprised 25 percent or less of purchasers in each study group. We compare Super Tunsai to other CWP brands, to help further explore the relative benefits and drawbacks of the new product against pre-existing brands.

Breakage was the main reason cited for non-use in both CWP product groups, but was

less of an issue for Super Tunsai (42 percent) than for other brands (70 percent).11 This difference is likely due primarily to the longer duration of use for other CWP brands (since Super Tunsai came to the market through the pilot, while other brands were around much longer). But it could also indicate that Super Tunsai is sturdier or less prone to accidental breakage (e.g. less likely to be tipped over due to the design of the stand).

Give-away was a more prominent reason for non-use for Super Tunsai (26 percent)

compared to other brands (9 percent). The fact that Super Tunsai is a more popular gift item than other brands could indicate that consumers attached a higher aesthetic/status symbol value to it.

11 After accounting for give-away, breakage still remains higher among other brands (78 percent) than for Super

Tunsai (55 percent).


Close to a third of Super Tunsai purchasers not using the filter said the reason was that they believed the device to be “unsanitary”. Exploring this in the qualitative study data showed that several individuals commented that they felt purified water was not as clean as boiled water and as a result they chose not to purchase the filter. Others noted that certain aspects of the purifier, such as the clay component inside that serves as the filtering mechanism, could easily mix with the water, producing water that would be unsafe for drinking.

Table 13. Reason for non-use of CWP among purchasers of Super Tunsai vs. other CWP brands*

*Other brands include regular Tunsai Concerns about Tunsai/Super Tunsai The main concerns that emerged in the qualitative study were the long term durability/

functionality of the device, and questions about whether it would continue to produce safe water after long term use. Among the durability and quality concerns mentioned by respondents, the following stood out: the tap was perceived as fragile and easily breakable, the dripping speed of the water from the container was described as very slow, and the CWP lacked stability and could easily fall over.

There was some distrust in CWP water quality (primarily among non-users). Comparisons of

the effectiveness of boiling and CWP often favored boiling: some considered boiling a necessary first step in treatment (with boiled water then used in the CWP), others thought boiling was a better treatment compared to CWP or provided the same quality of purified water.


7. Retail Sales Model: Price Sensitivity to Super Tunsai and Feasibility of Implementing Coupon Schemes

The main objectives of the retail sales pilots were to:

Test the price sensitivity to Super Tunsai, i.e. the degree to which the uptake of Super Tunsai varied with the price at which it was available to consumers. Three different price points were operationalized through coupon schemes for the purposes of the pilot.

Determine the feasibility of implementing coupon schemes as a means to provide targeted subsidies to households of lower socio-economic stratum.

Price sensitivity of Super Tunsai would have been best determined by estimating the uptake

rates in areas where each price point was tested. However, due to the short duration of implementation and low overall sales volumes in the retail sales model, an endline survey was not conducted to estimate the uptake rates. As an alternative means to estimate the price sensitivity of Super Tunsai, we examine: (a) extent to which price is a barrier to purchase of Super Tunsai and (b) relative sales volumes in areas where each of the price points were tested.

Some of the findings summarized in this section of the report were already discussed in

more extensive detail earlier, but are noted here again in the context of examining the price sensitivity of Super Tunsai.

Feasibility of implementing coupon schemes as a means to provide targeted subsidies to

households of lower socio-economic stratum has been examined by collating feedback from Hydrologic and PATH representatives who designed and monitored the schemes that were piloted.12

Extent to which price is a barrier to purchase of Super Tunsai

We infer that price was a significant barrier to purchase of Super Tunsai from the following:

In the qualitative study, the price of the CWP emerged as leading barrier to purchase. Even those who purchased felt that others, in poorer socio-economic groups, would not be able to afford the product. Many who purchased Super Tunsai with coupons said that they would not have purchased without the discount; and VisionFund clients noted that the installment scheme was key for their decision to purchase the device.

The household survey also indicated that loans and installment plans were key for CWP purchase in the MFI area: 86 percent of those who had purchased a CWP at endline in the MFI area said they used a loan/installment plan for the purchase. While CWP was

12 Specifically, we interviewed Nop Thunvuth (PATH) and Sopee (Retail Sales Manager for Hydrologic in

Kampong Speu province).


available for purchase cash down for MFI members, only 3 percent paid in full (i.e. did not use the VisionFund CWP loan).

In each of the study groups, the second most frequently cited reason for not purchasing a CWP by respondents in the household survey who knew of the product was not being able to buy CWP due to financial/affordability issue (Figure 17).

Relative sales volumes in areas where different price points were tested

Here we examine the sales (in units) to understand the price sensitivity of Super Tunsai.

Table 15 shows the key characteristics of the pilot areas. While the number of participating retail outlets in each of the pilot areas was similar, the total number of households in the pilot area varied.

Table 15: Key characteristics of retail sales areas

Price of Super Tunsai Districts Number of

participating retailers Number of households

in the pilot area

$22 (full price) Kg Cham, Kg Siem 9 31,400

$17 ($5 off) Tboung Khmum 11 46,247

$12 ($10 off) Kang Meas, Ou

Reang Ov 10 20,971

Hence, in comparing the total sales (units) in each of the areas, we have factored these

differences (Table 16). After factoring these differences, we see that the sales in the area where Super Tunsai was sold at $17 was higher than the sales at full price by a factor of three to four. Further, the sales in areas where Super Tunsai was sold at $12 was higher than the sales at $17 by a factor of two – four. Thus, though the demand increases with decrease in price, the factor of increase in sales volume is lower at $12 compared to $17.

Table 16: Comparison of unit sales in retail sales area

Price of Super Tunsai Total Sales

(Units) Sales (units) per Retailer

Total Sales (Units) per 1,000 Households

$22 (full price) 30 3.3 1.0

$17 ($5 off) 131 11.9 2.8

$12 ($10 off) 206 20.6 9.8

Sales Factor

$17 / $22 4.4 3.6 3.0

$12 / $17 1.6 1.7 3.5

We also examined the monthly trends in sales in each of the pilot areas (Table 17).

Interestingly, the sales at $17 appear to come to the same levels as sales at $12 after about three to four months.


Table 17: Trends in unit sales of Super Tunsai in retail sales area

Price of Super Tunsai March '11 April '11 May '11 June '11 July '11 Total

$22 (full price) 1 0 9 14 6 30

$17 ($5 off) 5 9 38 24 55 131

$12 ($10 off) 40 39 40 40 47 206

Based on these, we infer that Super Tunsai is price sensitive in the $17 - $22 price band, but

further decreases in price may not increase demand in the long run.

Feasibility of implementing coupon schemes Hydrologic, with assistance from PATH, followed a rigorous retailer selection and

verification protocol to ensure that the discounts offered to customers through the coupon schemes benefited the customers. Towards this:

Only those retail outlets willing to participate in the scheme and adhere to the data collection requirements were offered this scheme.

Required data collection by the retailers included a fully filled-in form containing contact details of customers.

Hydrologic representatives conducted a 100% verification of all sales under the coupon scheme.

Interviews with Hydrologic and PATH representatives suggested that:

Gaining the support of retailers to participate in this scheme was not a challenge,

There were no instances of fraud – retailers consistently passed on the discount to the customers as intended.

The data collation and verification was resource intensive. Thus we conclude that it is feasible to implement such a scheme, for example as a means to

provide targeted subsidy to households in lower socio-economic groups. Such a scheme appears to have the potential to benefit the poor and yet not circumvent the market. However, implementing the scheme will require additional administrative (financial and human) resources that will need to be supported.


8. Assessment of Commercial Viability: Direct Sales Model

In this section, we describe the findings from the assessment of commercial viability of the

two direct sales variants – the MFI partnership variant and the door-to-door sales variant. Note that commercial viability of the retail sales model and its variants is not assessed since it is a well-established and commercially viable model for many products.

The assessment answers the following research questions:

1. Can the commercial partners earn a profit from sales to target consumers?

2. Will commercial partners continue and/or scale up the SWP pilot business model to reach target consumers?

3. What other efforts/inputs/incentives are needed to stimulate supply and demand for Tunsai and Super Tunsai to target consumers?

Some of the findings summarized in this section of the report were already discussed in more extensive detail earlier, but are noted here again in the context of assessing specific aspects of the model’s feasibility and commercial viability.

8.1. Approach to Assessing Commercial Viability

In order to determine the commercial viability of the model piloted through this partnership, we focus on two questions:

Can the direct sales variants be self-sustained and is it likely to be adopted by the key implementing actors?

How can uptake and continued use be increased? To answer these questions, we conducted analysis of data and information in the following

areas:

Channel efficiency: We examined the sales conversion over the pilot period and compared them against pre-pilot expectations, as well as feedback from the implementers at the end of the pilot.

Channel effectiveness: Examined market potential and achievements during the pilot including levels of and increases in awareness, uptake and use of Tunsai and Super Tunsai, the potential market size, and effectiveness of marketing activities.

Channel sustainability: Analyzed operating costs and projected sales revenue to identify the level of commercial viability of the pilot for the partners and supply chain actors’. The commercial viability of the pilot was assessed using the ‘levels of commercial viability’ framework and information from interviews with partners and supply chain actors about their willingness to continue and/or scale-up the pilot.


8.2. Channel Efficiency

Figure 18 shows the sales under the two variants of the direct sales model. In February 2011, Super Tunsai was launched in under the door-to-door variant and the MFI variant started with both Regular Tunsai and Super Tunsai. In April 2011, the door-to-door variant of the direct sales model was discontinued.

Figure 18: Sales in direct sales model

0

100

200

300

400

500

600

700

800

Dec Jan Feb Mar Apr May Jun Jul

Tota

l Filt

er

Sale

s

Door-to-Door Total Filter Sales MFI Total Filter Sales

As shown in Figure 18, the peak monthly sales in the door-to-door sales model was 400 units whereas tin the MFI model the peak monthly sales was more than 700 units. We note that only 4-5 sales representatives were deployed in the MFI model, thus making the difference even more stark. The average daily sales (units) by a sales representative in the door-to-door sales model was between 1.5 – 2.0 units. The corresponding figure in the MFI model was 6.0 – 8.0 units.

Prior to implementation of the pilots, Hydrologic representatives interviewed mentioned

that if the door-to-door sales yielded on average daily sales of two or more units per salesperson, they would consider it successful. The door-to-door model appears to have come close to meeting this benchmark, and the MFI model exceeded this benchmark.

Super Tunsai launched & MFI started

End of door-to-door sales


However, this pre-pilot benchmark may not have been realistic. At daily average sales of two units, a salesperson earns $40 a month through sales margins. This is well below salespersons’ expectations of earning at least $200 a month13. The MFI model, with average daily sales of 6-8 units, provides a sales person with $250 - $300 monthly revenue.

Thus, we conclude that the MFI model was more efficient and the sales conversion rates

yield income levels that meet or exceed the aspirations of the salespersons.

8.3. Market Potential and Uptake of CWP

The market potential for CWP and particularly the Super Tunsai in this population is high, as evidenced by the following:

There was high felt need to treat water: about 60 percent across groups said it was absolutely necessary/good to treat. Although there were major differences in the predominant type of water source used during the dry and rainy season, the extent of perceived need for drinking water treatment was similar in both seasons.

Nearly all households using a CWP had used boiling prior to purchasing the device. Boiling was the predominant HWTS practice in all study groups with 60-80 percent of households across groups reporting boiling as a regularly used method.

Super Tunsai is perceived as a more aspirational product valued for its aesthetic look, and is also considered more functional in its design features (particularly the water glass stand under the tap which the original Tunsai model did not have). Further, the uptake of Super Tunsai has been higher than Regular Tunsai at full price.

The pilot models were effective in realizing the potential market. The evaluation indicates

that the direct sales and the MFI sales pilots had a substantial impact on CWP purchase among the general population. The impact of the MFI pilot among VisionFund members was of particularly high magnitude.

o CWP uptake was 7 percent across the direct sales area surveyed at baseline. At endline, CWP purchase in the control group had remained at a similar level (7 percent) but reached 16 percent in the basic direct sales area and 21 percent in the MFI area.


Price/affordability of CWP is a significant barrier to purchase of CWP. However, installment

payment plans appear to be able to overcome this barrier to a great degree as evidenced by the magnitude of uptake among MFI clients.

Direct sales models seem to be appropriate for the current product-market context. There

was a high level of recall of the promotional activities supported through the pilots. CWP demonstrations were particularly valued by consumers.

13 Based on interviews conducted by Abt with the salespersons prior to the baseline survey.


Thus we conclude that CWPs and in particular Super Tunsai has high potential, and that

direct sales models with consumer financing mechanisms can generate significant uptake among the population.

8.4. Channel Sustainability

In order to determine the commercial viability of each of the partners, we utilized unit cost and sales data obtained from the partners as well as summarized profit and loss statements produced by PATH for the pilots.

The main assumptions regarding cost elements in the analyses are summarized in Table 19, and used in the analyses and conclusions we draw in the following sections.

Table19: Cost Assumption for analyses of commercial viability

of direct dales and MFI pilots

Cost element

Super

Tunsai

Cost

(Riel)

Tunsai

Cost

(Riel) Source / Assumption

Retail price per unit 92,000 52,000 Recommended retail

price

Wholesale price per unit 85,700 47,800 Hydrologic

Value Added Tax (VAT) per unit 9,200 5,200 Country norm (10% of

retail price)

Secondary freight per unit 5,200 5,200 PATH

Cost of goods sold per unit 52,878 30,912 PATH

Direct selling expenses per unit (reimbursement of gas,

base pay to direct sales representatives) 1,500 1,500 PATH

Monthly administrative expenses attributable to the pilot

model 18,000,000 Hydrologic

8.5. Commercial Viability for Hydrologic: Door-to-Door Sales

Table 20 summarizes the revenues and costs incurred by Hydrologic in the first two months of the direct sales pilot where only the door-to-door model was being implemented.

Based on this data, the cost recovery for Hydrologic is at 47 percent. Hydrologic is able to

recover the product costs, VAT and direct sales expenses. However, it is not able to recover the administrative expenses. Since direct sales expenses include cost of product marketing, and assuming that mass media advertising is not required for this model, Hydrologic is at Level 4 commercial viability.


Table 20: Hydrologic’s Revenues and Costs in Door-to-door Sales Model

Total Sales Units - Tunsai 627

Total Sales Units - Super Tunsai 0

Total Revenue 29,970,600

Expenses

VAT (2,997,060)

Product Cost (19,381,824)

Direct Sales Exp (5,830,850)

Administrative Expenses (35,823,925)

Total Expenses (64,033,659)

Net Margin (34,063,059)

8.6. Commercial Viability for Hydrologic: MFI Sales

Table 21 summarizes the revenues and costs incurred by Hydrologic from June – September 2011 in the direct sales pilot where only the MFI model was being implemented.

Table 21: Hydrologic’s revenues and costs in MFI sales model

Total Sales Units – Tunsai 20

Total Sales Units - Super Tunsai 2,536

Total Revenue 217,434,000

Expenses

VAT (21,743,420)

Product Cost (134,188,068)

Direct Sales Exp (4,920,000)

Administrative Expenses (71,647,850)

Total Expenses (232,499,338)

Net Margin (15,065,338)

Based on this data, the cost recovery for Hydrologic is at 95 percent. Hydrologic is able to recover the product costs, VAT and direct sales expenses. However, it is not able to recover the administrative expenses. Since direct sales expenses include cost of product marketing, and assuming that mass media advertising is not required for this model, Hydrologic is at Level 4 commercial viability.

8.7. Commercial Viability for VisionFund

For the analysis of commercial viability for VisionFund we assumed the following based on inputs provided by PATH:

Annual interest rate charged by the MFI (calculated on a reducing balance of outstanding principal): 20%

Cost of funds to the MFI (calculated on outstanding loan portfolio): 11%

Loan loss provision (calculated on loan amount): 2%


MFI overhead allocation (of outstanding loan portfolio): 18%

Loan recovery ratio: 100%

Based on these parameters and assumptions, VisionFund achieves a 96 percent cost recovery and level 4 commercial viability.

9. Partners’ Willingness to Continue with the Models

9.1. Hydrologic’s interest in continuing the model

Hydrologic representatives believe that the retail sales model is an essential strategy to long-term growth and sustainability of Hydrologic’s operations. However, Hydrologic believes that the retail sales operations model needs to be re-engineered for it to realize its potential. Some options being explored towards re-engineering the retail sales operations are:

Changing the retail margins for Tunsai / Super Tunsai so that it reflects the margin structure offered by other marketers of water purifiers,

Restructuring the compensation structure of Hydrologic’s sales representatives such that it has a higher variable component,

Exploring alternatives to current advertising and promotion mix such as events and demonstrations in congregation points near retail outlets,

Exploring the possibility of engaging with entrepreneurs linked to retail outlets who promote the CWP in congregation points / markets or though door-to-door sales.

Current focus of Hydrologic is in the direct sales models. Sales have been low in the door-to-door model, the returns to the salespersons and Hydrologic insufficient and hence this model is not considered attractive. Hydrologic is keen to continue the MFI model since it has yielded high sales, and feedback from staff involved in this model has been positive. It is keen to continue the partnership with VisionFund, with some changes to improve efficiencies. Changes that Hydrologic proposes are aimed at increasing the productivity (sales / person / day) of its sales persons. Specifically:

Encouraging VisionFund to appoint CSOs dedicated to processing and approving loans for CWP. Hydrologic’s learning from the pilot indicates that without this dedicated effort, the loan processing could be slow, and delayed approvals adversely affect salespersons’ productivity.

Alternatively, Hydrologic wishes to explore the feasibility of coordinating loan processing and forwarding with the Community Bank Management Committee directly


(with VisionFund’s support and approval). However, the details of such an option need to be developed further.

Hydrologic intends to continue with the MFI model, with an emphasis on Super Tunsai. Extent of scale-up will depend on availability of working capital to support expansion.

9.2. VisionFund’s Interest in Continuing with the Model

VisionFund considers its partnership with Hydrologic in the marketing of CWPs as a key growth strategy to expand its client base. In addition, safe water also aligned with VisionFund’s social objectives. Therefore, VisionFund is keen to continue and scale-up model. VisionFund has some reservations on the proposed models (for example the dedicated CSO model), and may require additional incentives / margins such as an access fee to continue with this partnership.

10. Comparison of Results from Cambodia Pilots and Other SWP Pilots

In this section we compare the results from the Cambodia SWP pilots to the results from

earlier SWP pilots conducted in India, Kenya, and Vietnam. Annex Table A5 summarizes the key features of all pilots. The detailed evaluation findings for each pilot, as well as descriptions of the pilot approaches, can be found in six individual pilot evaluation reports, a cross-country comparative report, and a report comparing MFI pilots in Cambodia and India – all reports produced by Abt Associates for PATH.

Here, we draw on these reports to provide a summary comparison of the results from the

pilots in Cambodia to the conclusions in the comparative reports that covered the pilots in the other three SWP countries. These comparative reports contain details on pilot locations, products, partners, pricing and promotion models, implementation challenges, specific results and resulting recommendations.

We organize the comparison of the Cambodia pilot results to the results from the other SWP pilots around the two key featured models implemented by the SWP: the MFI model and the direct sales model.


10.1 Comparison of MFI models in India and Cambodia

Table 22 summarizes the features of the MFI pilots that are compared in this section of the

report.

Table 22: Summary of Key Features of SWP MFI Pilots

Location

India Cambodia

Erode, Thirutani (r)

Kanchipuram (TN)

Nimach (MP)

Nagda (MP)

Nellikuduru (r) (AP)

Kampong Speu (r)

Product (manufacturer)

PureIt (HUL) PureIt (HUL) PureIt (HUL)

PureIt (HUL)

Aquasure Xtra (EFL)

Tunsai, Super Tunsai (HL)

MFI partner Spandana Spandana Spandana Spandana PSS VisionFund

Cost of Product Rs/ Riel ($ PPP)

Rs. 2,000 ($102)

Rs. 2,000 ($102)

Rs. 1,000 ($51)

Free Rs. 1,390 ($71) R 92,000 ($44) R 52,000 ($25)

Consumable & Bundling

None None None 2 GKB for Rs. 730

1 cartridge for Rs 350

No consumable

Loan Type Rs. 43 x 50 weeks

Rs. 80 x 25 weeks

Rs. 43 x 25 weeks

Rs. 39 x 17 weeks

Rs. 250 x 6 months

R 16,000 / 9,000 x 6 months

Manufacturer presence

Established Established New New Moderate Moderate

MFI presence Established Established New New Established New

Notes: (r) indicates rural location Manufacturer/MFI presence refers to presence in pilot areas before the pilot PPP = Purchasing power parity (price in international US dollars) Rs. = Indian Rupees HUL = Hindustan Unilever HL = Hydrologic GKB = germ-kill battery

The conclusion of the analysis comparing the SWP MFI pilots were that the Cambodia model

was the strongest in terms of effectiveness, commercial viability and partners’ intention for scale-up. Our assessment ranked as second most promising the model piloted in Erode (an urban area in the state of Tamil Nadu, India) which offered PureIt, a tabletop durable filter, at a loan scheme of weekly installments of 43 Rupees ($0.95) paid over 50 weeks. Our findings are summarized below. Effectiveness

Comparison of uptake (purchase) across the MFI pilots suggests that the MFI pilot in

Cambodia achieved similar rate of uptake to the best-performing MFI pilot in India (in Nagda, Madhya Pradesh) that offered a free PureIt filter bundled with two replacement germ-kill cartridges which were offered with a 17-week loan of 39 Rupees ($0.86). Uptake in the Erode pilot was also high (30 percent). In all other MFI pilots, uptake was below pre-implementation expectations, reaching about 7 percent or less.


However, rates of current use in Nagda and Erode were about half the rates seen in

Cambodia (21 vs. 39 percent). We found that many purchasers in Nagda were not using the product, with lapsing and re-sale prominent in this 100-percent cost defrayment model (suggesting that many who did not value or perceive need for the filter had purchased it). This pattern was not seen in the pilots in Cambodia or in Erode where use to purchase ratios were high (especially in Cambodia where 91 percent of purchasers used the filter).

After taking into consideration the relative feasibility of addressing the implementation challenges that were found to limit the effectiveness of each pilot (both in terms of purchase and use), we assessed the potential (“could-be”) effectiveness of the pilot in Cambodia to be substantially higher than that of the other MFI pilots, and the potential in Erode to be as high as that for the Nagda model. A key advantage of the model in Cambodia over the models in India is that the CWP does not require the regular replacement of a germ-kill battery that PureIt needs; some of the key implementation challenges in the PureIt pilots that were identified as constrains to effectiveness were related to resupply and cost of these batteries. However, further exploration of the breakage rates of CWP in the long term would be required to compare the long-term costs to consumers of CWP vs. PureIt.

The Cambodia MFI pilot was as effective in targeting the SWP core target group as the other

MFI pilots where similarly high proportions of MFI members belonged to the middle three country-specific wealth quintiles. Uptake within the core target group was similar in the Cambodia and the Nagda pilots, and in both sites close to the overall uptake among the entire target group (all quintiles). 14

Commercial viability

The level of commercial viability of the Cambodia pilot was lower than the commercial viability of the best-performing pilot in India – the model in Tamil Nadu state which reached greater than 100 percent cost recovery and commercial viability level 5 overall as well as for both the MFI and manufacturer partners.

In comparison with the cost-defrayment model in India (in Madhya Pradesh) the Cambodia

pilot had better commercial viability results for the filter manufacturer, but was outperformed for the MFI partner (as the Madhya Pradesh pilot reached more than full cost-recovery for the MFI).

The MFI pilot in Andhra Pradesh was the least successful among all MFI pilots due to low

loan recovery and low sales conversions (low efficiency due to implementation challenges and product problems).

14 Small sample sizes in the pilot in Erode did not allow for reliably comparing uptake in the core group there.


Our assessment of the ‘could-be’ commercial viability of the MFI pilots ranked the Cambodia model as high as the best-performing model from the India pilots (Tamil Nadu), and higher than the remaining two pilots. The rational for the ‘upgrade’ of the commercial viability of the Cambodia pilot was because of the following potential improvements that were assessed as highly feasible:

With improvements in operations, Hydrologic can break even.

If VisionFund treats part of its expenses as ‘market development’ cost (since this is their main strategy to attract new clients), then VisionFund will break even.

The ‘could-be’ commercial viability of the Andhra Pradesh model was assessed as

substantially better than its observed results given the potential for resolution of the issues that affected cost recovery in this pilot: with improvements in implementation and by overcoming the product issues, the CV of the model can improve substantially – to recoup distribution and marketing costs.

The ‘could-be’ commercial viability of the Madhya Pradesh pilot was assessed to remain at

the observed level: for the model to be at level 5 commercial viability, the continued use rates need to increase substantially, and we do not envisage this happening with marginal improvements in implementation based on the results from the evaluation of this pilot.

Partners’ intention to continue and scale up

The intention of pilot partners to scale up was assessed as highest for the Cambodia pilot,

and also high (albeit to a lesser extent) in the Tamil Nadu pilot (Rs 43 repayment each week for 50 weeks).

In India, a limitation of the assessment was that Spandana representatives were not

available for interviews, so their views are as reported by HUL. HUL is interested in continuing and scaling up the MFI model in Tamil Nadu, with modifications to ensure financing of GKK and re-supply of GKK through the MFI. However, since Spandana has been affected by MFI disturbances in India, they may continue only after the disturbances are resolved and their operations have stabilized. HUL was not interested in continuing the cost-defrayment model piloted in Madhya Pradesh.

Looking at the other MFI-manufacturer partnership in India (in Andhra Pradesh) we found

that EFL believes that a key challenge in sustainability of this model is the fixed remuneration to their sales representative. Hence, they are considering moving to a completely variable remuneration, i.e. a micro-entrepreneur model. If this is successful, they will scale it up. Further details of this were not available since EFL considered this business-sensitive. PSS was willing to continue this if revolving loan funds were available. Thus, the overall intention of partners to scale up this model was assessed as a ‘maybe’ and less promising than the Cambodia and Tamil Nadu models.


In summary, our comparison of the SWP MFI pilots concludes that the Cambodia model was

the strongest in terms of effectiveness, commercial viability and partners’ intention for scale-up.

10.2. Comparison of Direct Sales Models in India, Cambodia, and Vietnam

In this section, we compare the basic direct sales (door-to-door) model in Cambodia to three other direct sales pilots by the SWP which are described in Annex Table A5 and include:

Kenya pilot, where Chujio, a CWP product similar to regular Tunsai, was promoted and sold by SWAP, an NGO working with women’s self-help groups

Uttar Pradesh, India pilot which used bicycle entrepreneurs to promote and sell Aquatabs, a chlorine-based water disinfection tablet

Vietnam pilot, which used commune health workers to promote and sell Aquatabs to households in their catchment areas. This pilot had a basic and an enhanced version, with the latter including distribution of free safe water storage containers to households with children under five.

The basic direct sales pilot in Cambodia outperformed all other SWP direct sales pilots in

terms of effectiveness and commercial viability, but ranked overall lower on partners’ intention for scale-up. Effectiveness

Uptake in the Cambodia pilot was substantially higher than in the other direct sales pilots. The second best-performing direct sales pilot, the enhanced Aquatabs pilot in Vietnam, reached about 5 percent of ever use (the definition of uptake that the SWP adopted for FMCGs) while the remaining direct sales pilots reached less than 3 percent ever use. Similarly, the Cambodia pilot achieved substantially higher rates of current use (self-reported at the time of the endline survey). Uptake among the SWP core target group could not be measured reliably for the direct sales pilots other than the Cambodia model (due to the very low overall uptake). All SWP direct sales pilots were similarly effective in overall targeting of the SWP core target group, with about 80 percent of households in the pilot area for each model belonging to the middle three wealth quintiles.

The Cambodia pilot remained best-performing among the direct sales models when looking

at ‘could-be’ effectiveness in increasing uptake. While the ‘could-be’ effectiveness of the Kenya and Vietnam enhanced models was assessed as slightly higher than observed effectiveness, these two pilots are not expected to reach the levels of uptake potential that the Cambodia pilot could have.


Commercial viability

As discussed in earlier sections, the level of commercial viability observed for the Cambodia basic direct sales pilot was at level 4 for Hydrologic. All other direct sales models were at much lower levels of commercial viability (levels 1 or 2, and cost recovery less than 50 percent) and require financial support towards distribution and marketing costs as well as overheads to be able to continue. The ‘could-be’ viability of the direct sales models in Kenya, Vietnam, and India was assessed at level 2, so their potential for sustainability remains well below the potential of the model in Cambodia.

Partners’ intention to continue and scale up

As discussed earlier, the door-to-door model was not considered attractive by Hydrologic,

and they were planning to focus efforts on the retail sales and MFI models. In the Kenya and Vietnam basic model, all patterns have expressed an intention to continue and scale up the models (with a different CWP manufacturer replacing Chujio Ceramics in Kenya) but at certain conditions that could be challenging to achieve (e.g. some partners require additional support to cover distribution and marketing costs, or inventory carrying costs). Partners in the Uttar Pradesh Aquatabs pilot do not intend to scale up the model.

Thus, the potential for scalability of the Cambodia model is at a lower level relative to the

potential of the other directs sales models.

10.3. Advantages of Cambodia Pilot Models

In summary, the context and features of the Cambodia pilot that were advantageous,

compared to the other SWP pilots, for both observed and ‘could-be’ results include:

Already high perceived need for water treatment in pilot areas, which was a constrain for uptake in several pilot locations in India

High awareness of the product category at baseline, which contrasts with need of other pilots to build category awareness quickly and from very low levels (notably in Uttar Pradesh)

CWP has no consumable parts (such as germ-kill battery) that need to be replaced frequently. By contrast, the tabletop filters in the India pilots require a replacement of cartridges about every three to four months which requires a well-functioning cartridge distribution system and is a recurring cost for consumers who purchase the filters. In the FMCG models, distributors would need to reach all users on a regular basis to ensure consistent product availability for households.


Super Tunsai had a high perceived value-for-money and aspirational value relative to available HWTS alternatives

Partners’ flexibility in modifying the marketing approach to suit the consumer segment and channel. This is most evident in the contrasting approach of Chujio Ceramics and HUL to the SWP pilots: while HUL acted upon the early results from the pilots to modify their approach, Chujio was reluctant to address the price issue for the CWP (choosing instead to continue to rely on the high unit price it receives from the NGO market that then provides the free or highly subsidized product to the poorest segments of the population – i.e. those who are typically unable to pay for commercially viable products and are outside of the SWP target group ).


11. Limitations of the Research

There are several limitations of this evaluation that should be taken into account when

assessing the extent of internal validity of the estimated impact of the pilots in CWP uptake and use:

The pre-post comparison used for the MFI members group relies on a scenario of no interventions/events (between the baseline and the endline surveys) that could have affected the key outcome indicators other than the pilot interventions. As a result, it is possible that the estimates of pilot impact derived through the pre-post comparisons could be due in part to secular trends and/or interventions other than the pilot that may have influenced uptake of CWP. The lack of control group of MFI members to illustrate the counterfactual (“no pilot” scenario) limits the extent to which we can attribute to the pilot the impressive increase of CWP uptake.

The post-only evaluation design of the impact of the pilot variants on CWP uptake and use relies on:

o Similarity of the groups that are being compared in terms of baseline values of

the key outcome indicators (CWP uptake and use) and socio-demographic and other personal/household characteristics that could affect the key outcome indicators. The latter include unobservable characteristics which cannot be measured, and thus cannot be accounted for in the comparisons that we use to measure differences in key outcomes between groups. However, we control/adjust our estimates of pilot impact for differences in observable characteristics (through statistical analytic techniques). As a result, it is possible that the estimates of pilot impact derived through the post-only comparison of the MFI area group (for which we do not have baseline data) against the other two study groups could be due in part to differences that already existed between the groups a baseline. For example, it is possible that CWP uptake in the MFI area general population group was already high/close to the values measured for this group at endline.

o In the control and in the basic direct sales areas: same contents and intensity of interventions/events occurring between the baseline and the endline surveys that could have affected the key outcome indicators (other than the pilot interventions). For example, it could be the case that there were more intensive CWP promotion efforts or improvements in CWP availability through retailers in the basic direct sales area than in the control area; in such a scenario, the higher rate of CWP uptake in the basic direct sales area at endline would be attributable in part to these other interventions rather than solely to the pilot.


12. Summary/Conclusions

The following major conclusions can be drawn from the evaluation findings:

The pilot interventions effectively targeted the SWP core target consumers group (the middle three country-specific wealth quintiles): within each group in the study area, 80 to 87 percent of households belonged to this core target group.

While there were a number of factors that limit the attribution of differences between groups to pilot activities alone, the evidence from this evaluation indicates that the direct sales and the MFI sales pilots had a substantial impact on CWP purchase among the general population. The impact of the MFI pilot among VisionFund members was of particularly high magnitude.

o CWP uptake was 7 percent across the direct sales area surveyed at baseline. At

endline, CWP purchase in the control group had remained at a similar level (7 percent) but reached 16 percent in the basic direct sales area and 21 percent in the MFI area.


The pilots were associated with a similar pattern of results for CWP use:

o At baseline, only 2 percent of households surveyed in the direct sales area reported using CWP at the time of the survey. Regular use of CWP (every day or almost every day) was 1-2 percent across the study groups.

o At endline, use of CWP was 5 percent in the control group, compared to 12 percent in the basic direct sales group and 17 percent in the MFI area group. The rates of regular use of CWP were nearly identical to the rates of current use in all groups.

The market potential for CWP in this population is high, as evidenced by the following:

o There was high felt need to treat water: about 60 percent across groups said it was absolutely necessary/good to treat. Although there were major differences in the predominant type of water source used during the dry and rainy season, the extent of perceived need for drinking water treatment was similar in both seasons.


o Nearly all households using a CWP had used boiling prior to purchasing the device. Boiling was the predominant HWTS practice in all study groups with 60-80 percent of households across groups reporting boiling as a regularly used method.

Key motivating factors for purchase and use of Tunsai and Super Tunsai include the health benefits of CWP, time and money savings from its use compared to boiling, and the increased easy availability (in terms of quantity) of treated water.

The main reason for not purchasing CWP was lack of felt need for the device. Price/affordability of CWP was also mentioned as a key reason for not purchasing. The CWP loan from VisionFund (used by nearly all MFI members who owned a CWP at endline) and the coupon discount in the retail pilot were mentioned as important drivers for purchase.

Among those who purchased a CWP but were not using it, a leading reason was breakage. Give-away was more prominent for Super Tunsai than other CWP brands, which indicates that the redesigned product might have aspirational/status symbol value.

There was a high level of recall of the promotional activities supported through the pilots. CWP demonstrations were particularly valued by consumers.


13. Recommendations

The findings from the study point to several recommendations on changes to

promotional and other pilot activities that partners could consider to improve the pilot models.

These recommendations include:

Defer mass-media communication promoting Super Tunsai CWPs until the MFI partnership has a wide scale of operations to make such communication cost effective. Until such time focus on village meetings as the primary promotion vehicle.

Highlight the CWP’s effectiveness in addressing the most frequent water issues/concerns: visible impurities and perceived germ contamination.

Addressing affordability issues by continuing to offer loans through VisionFund and similar partners, but also consider additional ways to make the products more affordable for segments of the population that cannot afford purchase at full price (e.g. through coupons offered through direct sales channels).

Improve consumer education through all channels (mass media, instructional pamphlets, flyers, and direct sales representatives) on the topics of CWP maintenance and cleaning, how Tunsai filters water, and what to do in case of malfunction/damage.

Increasing coverage of product demonstrations should be of particular focus. For example, consider changes to retailer incentives and retail sales model to encourage retailers to spend more time with interested customers and market CWP more effectively. Additionally, distributor agents could offer product demonstrations in/around stores to address lack of time/interest by retailers.


Annex

Table A1. Current use of HWTS: control vs. direct sales groups

% of respondents who treated drinking water using any method in the preceding 24 hours


(Endline)

MFI Pilot Area %

(Endline)


N=1,005 N=1,380 N=481

Filter water using cloth, sieve, or net filter (%) 0.11

0.30

0.62

Allow the water to settle and decant (%) 4.61 ** 6.38 ** 1.86

Boil the water (%) 79.48

75.07

79.02

Chlorine/bleach tablet, powder or liquid (%) 0.00

0.07

0.21

Alum (%) 0.22

0.15

0.43

Ceramic Water Purifier CWP (%) 12.02 ** 17.07 ** 5.00

Mineral Pot Filter (%) 3.41

3.35

2.50

Ceramic candle filter (%) 0.00 ** 0.07

0.00

Biosand Filter (%) 0.19

2.59 * 0.22

Bottled Water (%) 10.50 ** 5.39 3.13

Table A2. Current use of HWTS: MFI members

% of respondents who treated drinking water using any method in the preceding 24 hours

MFI Members Group


N=473 N=589

Filter water using cloth, sieve,or net filter (%) 0.24 0.49

Allow the water to settle and decant (%) 14.49 5.82 *

Boil the water (%) 80.35 59.96 **

Chlorine/bleach tablet, powder or liquid (%) 0.23 0.00

Alum (%) 0.11 0.12

Ceramic Water Purifier CWP (%) 3.10 38.06

Mineral Pot Filter (%) 2.60 1.22

Ceramic candle filter (%) 0.22 0.00

Biosand Filter (%) 6.83 3.93

Bottled Water (%) 13.91 7.86


Table A3. Ever use of HWTS: control vs. direct sales groups

% Of Respondents Who Ever Treated Drinking Water Using Any Method


(Endline)

MFI Pilot Area %

(Endline)


N=1,005 N=1,380 N=481

Filter water using cloth, sieve,or net filter (%) 6.93

7.49

10.20

Allow the water to settle and decant (%) 20.14

20.69

24.93

Boil the water (%) 98.83

98.99

98.96

Chlorine/bleach tablet, powder or liquid (%) 2.22

0.74

1.67

Alum (%) 15.88

11.84

15.39

Ceramic Water Purifier CWP (%) 16.90 ** 21.43 ** 7.71

Mineral Pot Filter (%) 9.26

7.83

8.99

Ceramic candle filter (%) 0.10

0.07

0.00

Biosand Filter (%) 1.29

5.54 * 0.84

Table A4. Ever use of HWTS: MFI members

% Of Respondents Who Ever Treated Drinking Water Using Any Method

MFI Members Group


N=473 N=589

Filter water using cloth, sieve,or net filter (%) 7.43 6.91

Allow the water to settle and decant (%) 33.55 24.29

Boil the water (%) 98.05 98.33

Chlorine/bleach tablet, powder or liquid (%) 0.89 0.49

Alum (%) 11.50 11.34

Ceramic Water Purifier CWP (%) 6.86 43.24 **

Mineral Pot Filter (%) 7.04 2.52 **

Ceramic candle filter (%) 0.44 0.00

Biosand Filter (%) 10.77 9.89

Bottled Water (%) 86.94 84.12


Table A5. Summary of SWP Pilots in India, Kenya, and Vietnam Microfinance Institutions (MFI) Model Direct Sales Model

Model Description

Promotion and sales of durable water filters through MFIs with various MFI loan schemes and product price subsidies

Sales of an FCMG product by newly-recruited and trained bicycle entrepreneurs at weekly markets and door-to-door

Sales of a durable water filter by self-help group members already selling health products through a basket-of-goods approach at community group meetings and door-to-door

Sales of an FCMG product door-to-door by commune health station collaborators (public sector community health workers)

Product PureIt Classic durable water filter

PureIt Classic durable water filter

Aquasure Xtra durable water filter

Aquatabs water disinfection tablets

Chujio ceramic water pot filter

Aquatabs water disinfection tablets

Location Tamil Nadu, India

Madhya Pradesh, India

Andhra Pradesh, India

Uttar Pradesh, India

Nyanza and Western Provinces, Kenya

Can Tho Province, Vietnam

% of household

samples with intermediate or higher water contamination1

72% Not tested Not tested 45% 65% 0%

Population Strata Urban & rural areas

Rural areas Rural areas Rural areas Rural areas Rural areas

Product Partner Hindustan Unilever (HUL)

Hindustan Unilever (HUL)

Eureka Forbes Limited (EFL)

Medentech Chujio Ceramics Medentech

Channel Partner

Spandana Sphoorty Financial Limited, one of the largest MFIs in India

Spandana Sphoorty Financial Limited, one of the largest MFIs in India

Pragathi Sewa Sanstha (PSS), a second-tier MFI, through the umbrella organization ACCESS

MART, a rural marketing NGO

Safe Water and AIDS Project (SWAP), an NGO based on self-help group model

Zuellig Pharma (large pharm. distributor) and public health authorities incl. district medical centers and commune health centers

Partners’ market

presence in pilot area

Tamil Nadu is an established market for HUL and Spandana (2004 or earlier)

Madhya Pradesh is a new market for HUL and Spandana. Operations started in 2008-09

Warangal, Andhra Pradesh is an established market for EFL and PSS

None prior to the pilot

SWAP had well-established presence in the area

Zuellig and the public health authorities had well-established presence in the area

Product Price

Rs 2,000 ($45)

Two subsidized price points tested:

Rs 1,000 ($22) - 50% cost-defrayment for PureIt

Rs 730 ($16) for PureIt and two extra GKK – 100% cost-defrayment for PureIt

Rs 1,740 ($39) for Aquasure Xtra and one cartridge

Initially Rs. 0.5 ($0.01) per tablet, increased to Rs. 1 ($0.02) half-way through the pilot

Two price points tested, both subsidized

KSH 1,100 ($12.3)

KSH 700 ($7.8)

1,000 VND ($ 0.05) per tablet

Loan Scheme Two repayment plans tested (in different areas)

Rs 80 ($1.8) per week for 25

Weekly repayment amount similar in both areas

Rs 43 ($0.95) per week for 25 weeks

Rs 39 ($0.87) per

Monthly repayment plan Rs 250 ($5.60) per month for 6 months, plus upfront payment of Rs 250

n/a

Layaway plan with monthly installments over 3 months was introduced but implemented by very few vendors

n/a


weeks

Rs 43 ($0.95) per week for 50 weeks

week for 17 weeks

Pilot-specific

Questions

Which loan scheme generates higher uptake levels?

Is the level of uptake similar in urban and rural areas?

Are the uptake levels achieved in new markets broadly comparable to those achieved in established markets?

Does product cost-defrayment increase levels of uptake and consistent use?

Can the cost-defrayment be recouped through sales of the GKK?

Are the uptake levels achieved through a monthly repayment plan broadly comparable to those achieved through a weekly repayment plan?

Are the uptake levels achieved through the EFL-ACCESS-PSS partnership broadly comparable to those achieved through the HUL-Spandana partnership?

Determine if CWFs can be sold in a commercially sustainable fashion at scale through a direct sales model

Evaluate the impact of price on filter sales to determine the necessity of subsidies

Test whether local entrepreneurs can be motivated to distribute and promote Aquatabs to rural consumers sustainably.

Is there any added effect on usage of free water containers distributed through the pilot?

Does the cost of providing free storage containers justify providing them?

What effect did the incentive structure have on the sales of Aquatabs through HSCs?

Notes:

1. Data provided by PATH, based on water contamination testing of households‟ stored drinking water.

Intermediate or higher water contamination corresponds to household water samples having >10 thermotolerant

coliforms as defined by World Health Organization standards.

Safe Water Project - PATHsites.path.org/water/files/2012/09/Cambodia-direct-and-retail...Safe Water...

Documents

Transcript of Safe Water Project - PATHsites.path.org/water/files/2012/09/Cambodia-direct-and-retail...Safe Water...