Water Implementation Projects In Rural
North India: Evaluating
implementation “on the ground”
Katharine Owens, University of Hartford, Politics and Government
Marcia Hughes, University of Hartford, Center for Social Research
Overview of Presentation Project efforts to date Evaluation, implementation, and
collaboration Assumptions of this evaluation work and
inherent challenges of project efforts Evaluation model and methodology:
Contextual Interaction Theory Analyses, findings, and uses of the model Recommendations and conclusion
Projects in Abheypur, India University of Hartford student-faculty
service-learning teams
Professional and student chapters of Engineers without Borders
Interdisciplinary projects: engineering, social science, and education/outreach projects focusing on water.
The Projects January 2008: Installation of a solar powered well
pump and water storage system August 2008: Extension of the water supply to a poor
section of the village January 2009: Implementation of a rainwater-
harvesting system October 2010: Installation of a potable water supply
well and water storage tank for poorest community January 2010: Installation of a grey water drainage improvement project for the “Potter’s Village”
January 2011: Installation of a grey water drainage improvement project on main road of the village
October 2009 tank location
Abheypur, Haryana
2011 Soak Pit
• Solar panels• 2 wells• 4 tanks• Rainwater harvesting
• 2 tanks• Piped to girl’s school
• Well• Tanks
2010 Soak Pit
Right-of-Way
Private Farm
Gutter: 6” wide by 6” Deep
N S
W
Soak Pit
DrivewayBrick Wall
Gate
Continue Drain Across Road Drain Covered by Grate to Allow Vehicles & Carts to Pass
E
New Drain
No Gutter – Grade so swale takes water along side of road
About 3% grade from west side of road to soak pit
New Drain
Design for Soak Pit Implemented in 2011
Potters’ Village - Soak Pit Design Jan 2010
Forebay Soak Pit
Brick Weir (Width = 3”)
Greywater Inflow
Overflow to swale during for rain events4’
Forebay Depth = 1.5’Soak Pit Depth = 4’Forebay and Soak Pit covered with slate
4’ 4’
Stones and rocks to reduce water velocity
Two layers (6”) of bricks are used for soil structural support to keep soak pit open. Thus, excavated diameter is 1’ wider and longer than internal dimensions listed.
Implementation
When political ‘actors’ desire change in society, they develop and implement programs to address a perceived problem.
Researchers focus on the implementation stage of the policy process as a point where intentions and outcomes diverge
Two different processes: Developing a program/policy versus implementing a project/program/policy
Merging evaluation and implementation Program evaluators seek to quantify and
catalog the successes and failures of programs and policies.
Implementation policy analysts evaluates this stage in the process to better understand participants’ roles and any impediments to change.
We seek to infuse evaluation with implementation analysis methodology.
Merging the two Evaluation researchers find it’s not enough
to know whether outcomes are achieved, it’s important to understand how.
Using an implementation analysis tool in evaluation can answer questions about how people come together and work towards achieving a project (or fail to).
Scholars call “for responsive, contextual, flexible, adaptive, multidisciplinary, and mixed-methods approaches,” particularly when considering environmental programs and policies (Preskill, 2009, p. 99).
Collaboration is an imperative
(Gadja, 2004)
Being solo does not work for most of the problems that need addressing
Need shared efforts to achieve goals that would not otherwise be attainable working independently.
More and more, collaboration is becoming the method for addressing complex social problems
Understanding collaboration
Wostl et al. (2007) –
In managing water resources in particular, a paradigm shift is needed, one that recognizes the importance of
stakeholder involvement and collaboration.
Water: communal and imperative
Understanding collaboration
One goal: To develop communities of practice.
Would we recognize if we were not working collaboratively with our Indian partners?
Should we trust our own conceptions, or evaluate the process to confirm our goals of collaboration?
What does collaboration look like?The Practice of Collaboration Gajda and Koliba (2007) describe key
characteristics of interpersonal collaboration to be:
a shared purpose dialogue decision-making action, and evaluation (p. 29).
Water Implementation Projects in India: Inherent Challenges Working across cultures Across languages Across disciplines (engineers and social scientists) Across class (American middle-class and Indian
impoverished villagers) Caste distinctions remain present in the village Gender distinctions (American women in positions
of leadership, which may be atypical) There is short period to build trust among
partners, implement new projects, and manage and evaluate previous projects.
This evaluation work We work to accomplish project
implementation as well as outcome evaluation in India.
We find it important to engender true collaboration with India partners.
We use an implementation analysis theory to evaluate these water projects.
Contextual Interaction Theory finds project participants and other
stakeholders can influence implementation in critical ways.
analyzes collaboration between participants based on motivation, knowledge, and power, to shed light on how projects are implemented.
produces a prediction about interaction based on these core participant characteristics
Contextual interaction Theory allows the emphasis on implementation
(the how) as called for by evaluation researchers.
can provide evidence to those “on the ground” about their effectiveness in implementation.
allows large-n studies that enable researchers to find patterns in large data sets and enables multi-year studies.
Contextual Interaction Theory The motivation, knowledge, and power of
the implementer and target are the focus of the assessment.
Implementer: the student-faculty team, and a loose coalition including Navjoti, and the professional Engineers without Borders chapter
Target: local villagers, including those residing near the project and those taking an interest in the implementation of the project.
Research Questions How does motivation, knowledge, and
power among the different players affect implementation of the project?
What is the relationship between these characteristics among the people ‘on the ground’ and the outcomes of the implementation process?
How is the project plan or design implemented through collaboration processes?
Methods Faculty-student trip to Abheypur, India in
January 2010.
14 interviews using a semi-structured interview instrument.
Questions investigate the core elements of the theory (motivation, knowledge, and power)
We asked slightly different questions of the American and Indian counterparts as reflected their roles in the process.
Dependent variables Active cooperation Passive cooperation Forced cooperation Opposition Obstruction Joint learning No interaction Policy learning
Analysis[1] M scale=((proportion)-.50) *2, scale of -1.0 to +1.0[2] I and P scales= (proportion), scale of 0.00 to 1.00[3] n/a : participant did not answer questions on this concept.
Role MotivationM scale[1] Category Knowledge
K scale[2]
Power P scale
Implementer: Faculty-student team members
(+) 5 of 5 1.00 positive (+) 4 of 4 1.00 (+) 2 of 4 0.50
(+) 2 of 4 0.00 neutral (+) 3 of 6 0.50 (+) 0 of 2 0.00
(+) 2 of 2 1.00 positive (+) 4 of 6 0.67 (+) 0 of 2 0.00
(+) 1 of 2 0.00 neutral (+) 4 of 5 0.80 (+) 1 of 2 0.50
Implementer aggregate (+) 10 of 13 0.538462 positive (+) 15 of 21
0.7142857 (+)3 of 10 0.30
Target: Villagers
(+) 3 of 3 1.00 positive (+) 2 of 2 1.00 (+) 0 of 3 0.00
(+) 2 of 3 0.33 positive (+) 1 of 1 1.00 (+) 0 of 2 0.00
(+) 2 of 2 1.00 positive (+) 2 of 2 1.00 (+) 0 of 2 0.00
(+) 3 of 3 1.00 positive (+) 2 of 2 1.00 (+) 0 of 1 0.00
(+) 1 of 2 0.00 neutral n/a[3] n/a (+) 0 of 1 0.00
(+) 1 of 2 0.00 neutral (+) 1 of 1 1.00 (+) 0 of 2 0.00
(+) 2 of 2 1.00 positive n/a n/a n/a n/a
Target aggregate (+) 14 of 17 0.647059 positive (+) 8 of 8 1.00 (+) 0 of 11 0.00
Qualitative assessment
Implementers have some doubts about the work, their role, and the role of the target participants in the process.
"…but you know sometimes I wonder maybe there might be actually better things we could be doing with our time."
"I think [with] this project we weren't communicating with them before we got there, they didn't know our plan.”
New understandings of caste subgroups and inter and intra group dynamics
Analysis What is the potential for other scenarios to
emerge?
The strong imbalance of power does not come into consideration when both actor groups are in favor of a project.
But how might this impact implementation if one actor was no longer in favor of implementation?
Recommendations (February 2010) Community-driven projects based on a
formalized decision-making process. establish a shared purpose/role among EWB
interdisciplinary team Always consider the
caste/subcaste/gender/poverty issues and related power/resources, knowledge, and motivation among different groups
Recommendations (February 2010) Make collaboration a goal
Are all stakeholders involved? Is there a shared purpose and understanding of
the project? Is there equal decision-making? Has the
community been involved and/or surveyed for their feedback?
Is there local leadership for the project (i.e., in the village)?
Expect and plan for disagreement and conflict.
Recommendations (February 2010) Streamline the collaborative process
Develop project ideas with community leaders and Navjyoti.
Navjyoti can survey/scan community for input/feedback on project ideas.
Navjyoti can serve as link between EWB and village and district leadership (i.e., for communicating project design development, and planning).
Fine tune plan and develop budget-tasks and responsibilities (prior to starting project).
Recommendations (February 2010) To make previous projects sustainable
Monitoring Planning modifications
Monitor systems to ensure proper use and satisfaction with system(s), community leadership, and overall quality of water
Conclusions It is both a goal and an obligation to work
in a collaborative way with local groups in India.
Using the theory we can gain insight into not only the outcome of implementation (cooperation), but also provide context for how and why participants are cooperating.