Amulya Project Briefs final

Joy Amulya Project Case Examples page 1

JOY AMULYA Project Briefs The following are short descriptions of projects in which I combined research and learning methodologies to support innovation. The majority of my projects have been in the social innovation space, though I have also worked with partners in technology, education, and engineering. The approach I use is general to any arena where new ideas and solutions must take into account what people on the front lines of the endeavor know from doing the work. The “what-‐and-‐how” in a field of work – what people know from doing the work and how they do it – comprises the practice of that field. These examples show that when an inquiry process operates close to the world of practice, it leads to insights that transform thinking and action. Case 1. Advancing Innovation in Community Programs for Children Impacted by HIV/AIDS ............................. 3 Case 2. Advancing Innovation in Laptop Technology .......................................................................................... 5 Case 3. Advancing Innovation in Financial Services for the Poor ........................................................................ 7 Case 4. Advancing Innovation in Multiracial Community Democracy ................................................................ 9


(back of cover page)


Case 1

Advancing Innovation in Community Programs for Children Impacted by HIV/AIDS Problem A staggering number of children living in Namibia’s populous northern regions (which were among the most disadvantaged during the apartheid era) have been made vulnerable by the HIV/AIDS epidemic and as a result, do not complete elementary school. Elementary school completion has a crucial role in the ecosystem of survival in this part of the world – socially as well as economically. Although schools are the primary social institutions in the community, they are not equipped to help children overcome problems such as lack of adequate food, clothing, and nurturing. Grants were made by the Basic Education System (BES) Project (funded by USAID and operated by the Academy of Education Development in collaboration with the Namibian Ministry of Education) to establish grassroots community-‐school projects in 80 villages in northern Namibia to help an estimated 12,000 school-‐age children. The goal of the grants program was to promote sustainable collaborations between community leaders and schools to develop innovative approaches for helping children impacted by HIV/AIDS successfully complete primary school. BES staff needed a system for supporting continuous quality improvement (QI) and routine innovation at each site, and for capturing and sharing the innovations that proved most effective. The QI system also needed to monitor outcomes across projects and comply with national and international quality standards. Outputs • A comprehensive project report summarizing the development of the QI system, data collection

tools and procedures, and analysis guidelines (available at http://www.learningforinnovation.com/OVC_qual_impr_monitoring.pdf).

Approach As the lead researcher hired to develop the QI system, I worked on the ground in Namibia with community stakeholders, BES project staff, and in-‐country USAID stakeholders. After reviewing national and international quality standards for monitoring child health programs and adapting them to apply to grassroots community projects, I proposed a three-‐part collaborative process for designing the QI system: (1) site visits to learn from the experiences of projects in the early stages of startup; (2) using findings from the field visits to develop and pilot test a set of tools for improving program quality, promoting routine innovation by local teams, and gathering data on outcomes; and (3) training community outreach workers in the goal of the tools and how to put them into practice in the 80 project sites. Throughout the design process, I facilitated dialogue and analysis sessions among community project teams and BES field managers. These sessions played a critical role in aligning the QI system to local needs and contexts, as well as creating buy-‐in by the local teams and the BES field staff responsible for implementing the system. Site visits. With two BES field managers, I conducted site visits to five villages that had begun implementing projects funded through the grants program. The site visits were aimed at understanding and observing the issues experienced by the community members and school leaders during the startup period. I was particularly interested in helping project teams articulate the challenges they had encountered so far, and in looking for opportunities for how these small-‐scale startup projects could leverage a more comprehensive, community-‐wide strategy for children impacted by HIV/AIDS. We learned that there were often other resources in the community (for example, NGOs providing psychological support, HIV testing, microloans, etc.) that could be linked into the support project through a referral system. The site visits were also an opportunity to coach BES field managers in asking


the local project teams non-‐threatening, thought-‐provoking questions about their experiences and discussing potential solutions to the challenges they were facing. Develop and pilot test tools. Out of the site visits, I identified five key quality areas and created a set of tools for reviewing and improving each project within those areas. For example, a core monitoring tool captured data on program operating practices (e.g., record-‐keeping, how services were delivered), sustainability (e.g., volunteer recruitment and retention), and educational indicators (attendance, grades). Information from the core monitoring tool was fed into a quality improvement tool, aimed at facilitating discussions with local stakeholders about strengths and weaknesses in one quality area at a time (for example, asking how the volunteer community members operating the program were retained, part of the Sustainability area). A monthly workplan tool captured the needs for improvement identified in these discussions and the action plan agreed upon for addressing them (for example, allowing access to the project’s sewing machines to the volunteers who sewed school uniforms for HIV-‐affected children, so they could generate income for themselves). These tools were revised based on feedback from key stakeholders, and then pilot tested in five sites. The pilot tests gave the field management team first-‐hand experience in using the tools and allowed them to make their own revisions to the process. To create a periodic synthesis of data at the level of the children who were receiving support through the community-‐school projects (vs. at the level of the project), I developed the information requirements for a Learner Profile dashboard. The Learner Profile consisted of key educational, health, social and emotional wellbeing indicators, collected on 180 students from across the 80 project sites. Training BES community outreach workers in use of the QI tools. In the final stage of my in-‐country time, I facilitated BES staff in designing and implementing a 2-‐day workshop to train community outreach workers in understanding and using the QI tools. This included how to facilitate discussions about what was working, what was challenging, and what might help. The workshop was itself a participatory process in which community members, outreach workers, and field managers practiced using the tools during role-‐playing activities, followed by debriefs and recommendations for refinements. The workshops emphasized using the tools to promote project review, problem solving, and innovation over time in each project site. Key Insights and Challenges • A significant insight came during a site visit when our team was asking what we thought were lots of

potentially annoying questions about project operating issues, volunteer issues, etc. One of the community members – who, like most others, had never been involved in a project of this sort – asked “How can we keep having these conversations? They’re so useful to our thinking.” That shattered our assumption that the QI system would be seen as a necessary evil with no value proposition for the local site teams. We realized that a question-‐based format was experienced as highly supportive and might promote more rapid innovation, quality improvement, and eventual sustainability. The questions-‐and-‐discussion approach became core design principle.

• During site visit observations, we discovered that it was hard for projects not to single out the children receiving support from the project. Given that a key goal was to build self-‐esteem and confidence, we knew it was critical for each project not to exacerbate the stigma from having a family member with HIV/AIDS. When we asked project teams how they handled this challenge, many told us they had a hard time balancing the need to limit services to the target population with the need to avoid singling out individual children. However, a few sites had found innovative ways to avoid stigmatizing the children receiving support. We decided to include a question on the quality improvement tool asking projects about their efforts to avoid singling out program beneficiaries. This ensured that there would be a problem-‐solving process around this issue as part of the QI system. It also ensured that the innovations for avoiding stigma would be captured and could therefore be spread to other sites via the community outreach teams conducting the QI process.


Case 2

Advancing Innovation in Laptop Technology Problem HP Laboratories had developed a prototype dual-‐display form factor for a laptop computer, featuring a large touch-‐screen display mounted in place of the track pad. The display could toggle between a track pad mode and a touch-‐sensitive mode, allowing users to jot notes, draw simple diagrams, and make other kinds of annotations/scribbles using a stylus. Previous research had suggested users prefer scribbling notes on paper during common business and educational activities compared to any other existing technology solutions (e.g., Tablet PC). The HP Bangalore-‐based technology design team believed this was because the available technology was not easy or natural enough, and that if a better solution existed on a laptop, users would prefer a technology solution to jotting notes on paper. I was hired to lead the Bangalore design team through a process of designing and conducting an experimental research study to assess user preference for the new dual-‐display form factor for jotting notes or quick diagrams during laptop-‐based business tasks, as compared to stopping and scribbling on paper. Outputs • A step-‐by-‐step Powerpoint document to guide the team through a process of articulating goals,

required types of evidence, research questions, and methods for the experimental research study. Decisions made during the process were captured directly into the document. The result was a complete set of design specifications and rationale for the study, allowing the team to see how their insights as designers were critical in the research design.

• A similar Powerpoint document guiding the team through the results of the data analysis. This document facilitated data synthesis and ended up capturing the insights and further research questions generated by the results.

• A report co-‐authored with the technology design team, published online as an HP technical report (available at http://www.hpl.hp.com/techreports/2009/HPL-‐2009-‐81.html).

Approach Working on-‐site at HP Labs in Bangalore, I created a process for guiding the HP team through making key decisions about research methods and designing the experimental tasks for the study. I used an approach of outlining the steps involved in research design in a Powerpoint document, which showed how the team’s hypotheses and questions about user preference for the new dual-‐display form factor were needed to drive each methodological decision. These research design sessions also led the team through developing a set of experimental tasks that best approximated real-‐life situations experienced by business users. Through guided brainstorming sessions, the team created a large list of possible tasks for testing, then piloted a smaller set of them with 15 users. Five tasks were chosen for use in the experimental sessions. These hands-‐on sessions had the secondary purpose of transferring practical knowledge about quantitative research methods to the technology team and showing them how their insights as designers were critical to the development of a successful study. Following the design process, I coached the lead HP researcher on how to implement protocols for stratified random sampling (n=201), participant recruitment, and the experimental sessions themselves. During the 90-‐minute experimental sessions, participants were asked to carry out a task under two conditions: (1) using the prototype dual-‐display form factor on a standard laptop computer and (2) jotting notes and scribbles on paper while using a standard laptop (without the new form factor). The order of the two task conditions was randomly assigned and balanced across the sample. Participants


were asked to rate the ease of use, satisfaction, comfort, how well they could express themselves, and their overall experience following each task condition. They were then asked to compare their experience across the two task conditions. They also rated how realistic each task was, how frequently they encountered a similar situation in their work, and the extent to which they preferred one or the other form factor for that particular task. Demographic information, level and type of computer usage, and feedback on the overall experience during the experiment were also collected. The analysis was based on assessing the differences in each participant’s ratings during the two task conditions. The results showed that participants consistently gave higher ratings to the new form factor on all five tasks. This finding was backed up by the head-‐to-‐head preference ratings. The results were presented in a discussion session to guide the technology development team to further synthesize the data and generate actionable insights. This session also engaged the team in determining the most effective ways to disseminate the findings to: (1) inform further design and development of the prototype (e.g., cursor control, user interface, application support) and (2) summarize the study and its results for the global business unit (GBU) management team to inform them in whether to integrate the new form factor into future laptop designs. I provided extensive support to the team in writing the report for the GBU management team, and co-‐authored the HP technical report as well as a submission to the international conference on Human Computer Interaction (HCI). Key Insights and Challenges • The biggest challenge in designing the user preference study was developing realistic tasks that

closely approximated real business computing activities and could be adapted to the two experimental conditions (laptop with dual-‐display form factor vs. standard laptop using paper to make notes and scribbles). The tasks had to cover a range of activities that typical business users encounter and be compelling without taking too long. While challenging, the brainstorming and fine-‐tuning discussions were also a rich ground for connecting the ideas and experiences of the technology team with the concepts and practices of experimental research design.

• The team was intrigued by a pattern in user preference ratings for different dimensions of user experience. Ratings on likability, usefulness, and ability to express oneself favored the dual-‐display form factor compared to ratings for “natural,” “comfortable,” and “easy.” This makes sense, given that the new form factor was unfamiliar. Even so, users preferred it over working on a standard laptop and jotting notes and scribbles on paper.

• The HP technology design team did not have background or experience in quantitative research methods, which slowed down the initial pace of designing the study. However, through the process of explaining the concepts of experimental research design in terms of the team’s questions and knowledgebase about the new technology, I found that they came up the learning curve fairly quickly and were strong collaborators in designing the study. This partnership also meant that team members were highly engaged and empowered in the analysis and interpretation of results.


Case 3 Advancing Innovation in Financial Services for the Poor

Problem The Bill and Melinda Gates Foundation wanted to take stock of the ways in which its investment in CGAP (Consultative Group to Assist the Poor) had and had not promoted innovation in mobile banking as a transformational business model and an effective way to give very low income people in developing countries access to financial services. CGAP is a consortium of over 30 stakeholders focused on expanding access to financial services for the poor. Its Technology Program has supported the development of a variety of projects with microfinance institutions, banks, mobile network operators, and payment system providers targeting millions of very low income, unbanked people in Colombia, Ecuador, India, Kenya, Maldives, Mongolia, Pakistan, Philippines, and South Africa. The research project needed to document and analyze CGAP’s impact as a knowledge generator, disseminator, thought leader, and effective intermediary learning partner. Outputs • Project report with findings and recommendations (proprietary) • Follow-‐up one-‐day workshop on improving CGAP’s impact as a learning partner to the Foundation

and as a facilitator of knowledge generation and diffusion among its project partners Approach As a member of a four-‐person research team, I was responsible for designing and carrying out a content analysis of CGAP’s publications and blogs over the four-‐year period since they were started (2006-‐2009). The analysis used a coding system to describe and quantify the kinds of knowledge that were disseminated through these vehicles. Examples of the kinds of knowledge coded were: “a factor or condition that facilitates the implementation of branchless banking”; “a lesson learned or key finding”; “a model or component.” In addition, responses to the blog were categorized as a proxy for looking at the blogs’ role in catalyzing discussion in the field. The goal was to examine at a detailed level how CGAP concretely carried out thought leadership around innovation in this field – whether by describing innovative models, breaking them down into component parts, framing particular lessons in the design or implementation process, discussing facilitative conditions, etc. I also analyzed the degree to which CGAP used the blogs and publications to draw lessons across the projects they were developing. This analysis examined CGAP’s leadership in describing innovations (or aspects of innovations) that were transferable across contexts vs. those that were specific to a particular context. Finally, I looked at the range and frequency of the projects that were described in the blogs and publications as a means of understanding whether CGAP was drawing on the full extent of its “learning laboratory” – represented by the projects it had helped develop – to carry out its thought leadership strategy. To examine CGAP’s procedures for capturing innovations and lessons from its project partners, I analyzed the reports CGAP received from its partners, as well as what was shared about those innovations and lessons in CGAP’s quarterly reports to the Gates Foundation. Key Insights and Challenges • Although part of a larger analysis (other members of the research team conducted interviews with

key branchless banking stakeholders around the world), my analysis of the blogs and publications brought into view a specific weakness in CGAP’s approach to catalyzing innovation. CGAP did a great job of describing specific components and models of branchless banking (such as pricing and fees, mobile technologies, user interface), usually specific to a particular context. The weakness I discovered was that the publications and blogs provided information about how branchless banking


was being implemented without going the extra mile to analyze lessons learned, how that learning might be transferred to other contexts, or other implications for practice. The insight was that an innovation leader needs to find concrete ways to generate and disseminate promising models and practices from one place to another. In a field like branchless banking, the ecosystem varies considerably from one place to the next, involving the complex dynamics of regulatory infrastructure, cultural economic patterns, etc. CGAP had a unique and critical vantage point because of its involvement in a rich array of projects in different contexts, but it was not yet intentional about mining and sharing the learning from that vantage point.

• In light of the insight described above, the challenge in reporting the results was to make a clear distinction between describing specific innovative models, on one hand, and generating learning by looking across those models and offering guidance that could be put into practice. It helped that the Gates Foundation program staff had been dissatisfied about what they were able to learn from CGAP. In turn, CGAP leaders were frustrated with what the Foundation expected from CGAP as a learning partner. I was asked to facilitate a follow-‐up session with CGAP’s Technology Team about how to become a more effective intermediary learning partner for Gates, and a more effective innovation leader for the field. During this session, I took the CGAP team through experiential exercises that demonstrated the distinction between providing information from specific project examples vs. crystallizing the learning coming out of those projects.


Case 4 Advancing Innovation in Multiracial Community Democracy

Problem Rockefeller Foundation (RF) wanted to capture the insights and impacts from the work of five community organizations using highly innovative approaches to addressing urgent social problems by creating multiracial and multi-‐ethnic systems change solutions. These organizations were funded through RF’s Race, Policy, and Democracy program, and located in regions with very different racial and ethnic dynamics and histories (Jackson, MS; Los Angeles, CA; Boston, MA; Greensboro, NC; Austin, TX). The RF program officer believed that a standard program evaluation approach would not be able to capture the depth of learning and knowledge that had been acquired by these organizations. Although they had learned a great deal about how to effectively bring different groups together to work on common problems, ranging from voting rights and education reform to economic justice and equitable urban planning, the staff at these organizations lacked the tools and techniques for gaining visibility on what they did that was effective – and why. Their understanding about building multiracial democratic solutions was tacit, “know-‐how” gained through their day-‐to-‐day experiences of doing the work. I was contacted by RF to develop a way for these highly effective social change organizations to capture and organize what they had learned about getting people to work effectively across racial and ethnic lines. The project also needed to deliver a story-‐based product distilling key insights and providing recommendations to funders and practitioners. Outputs • A report using stories and images to present the key insights and recommendations to the target

audience of philanthropists and community organizations (available at http://www.learningforinnovation.com/Vital_Difference_fullreport.pdf).

• A DVD using interviews with participants and footage from the learning sessions to describe the process, the learning methodology, and the importance of the findings for the field.

• A toolkit guiding community organizations in the use of the learning methodology, Critical Moments Analysis.

Approach As part of a two-‐person team from MIT’s Center for Reflective Community Practice, I adapted an experience-‐based learning method I had previously developed for use by individual practitioners. Called Critical Moments Analysis, the method was aimed at surfacing the tacit knowledge accumulated over time through day-‐to-‐day work experiences. I modified this five-‐step process to allow the varied perspectives of staff with different roles in the work to be integrated and used to create a collective analysis of the work they had done together. I co-‐facilitated design sessions that introduced each organization to the learning method by having them use it to develop their own learning agenda for the project. The learning agenda consisted of the organization’s research questions, a list of projects and/or time periods to be the “dataset” for addressing those research questions, and the logistics for capturing the learning session (audio/video). Each learning session began with organizational leaders and staff making a list of the turning points and other “critical moments” from their work (successes, challenges, dilemmas, and other memorable points). They then narrated “the story” of each moment, bringing in a range of perspectives from those involved to describe what happened, what was significant and why, how they responded, and what did and didn’t work well. We also facilitated the staff in the analysis of each critical moment, articulating key questions and lessons arising from the stories associated with that moment.


After the learning sessions, we brought all five organizations together to share insights and findings from their individual learning sessions. In addition to exploring each organization’s “learning case,” the collective session was aimed at developing field-‐level findings to inform policy and practice for promoting racially inclusive democratic processes at the community level. Key Challenges and Insights • Racial differences are an important driver of innovation. The common theme running through the

work of all five grantees was that when they created innovative ways to engage community members from diverse racial and ethnic groups in problem-‐solving, they created new kinds of solutions that ended up being better for everyone. Specific learning points for building racially inclusive community processes are illustrated with stories from the field in the Vital Difference report, for example: “Racial identity is an important and effective tool for building participation” and “Inventive coalitions bring new possibilities for social change.”

• Initially, a big challenge was shifting the mindset of participating organizational staff away from “what the funder wants from us” to embracing an inquiry process that would allow them to use what they had learned to make their work better. Most of the organizations went through this shift when they were introduced to Critical Moments Analysis and began to see how much they could learn by analyzing their own stories. Several of the organizations ended up embedding the Critical Moments process in their regular staff or project team meetings.

• A question that recurred in the early stages of the project was whether harvesting the knowledge from practitioners in the field would contribute in meaningful ways to the body of knowledge already accessible to the philanthropic community (for example, from race theorists, community development research, etc.). In writing the final analysis report, I found that the insights and approaches that originated in the action of community practitioners added significantly to existing theory and research. Moreover, the process of accessing and analyzing critical moments proved to be a promising approach for capturing, diffusing, and promoting social innovations within and beyond the group of organizations we worked with. The findings were adaptable to any community context in which there are efforts to bring people together across race lines to work toward their goals for a better life.

• A very interesting challenge was to convince the MIT intellectual property counsel to give participating organizations, not the university, control over the materials generated through the research process, in order to be consistent with the democratic principles of the project. Through many months of discussions, a legal framework that could be applied to this project without compromising the university’s interests was developed and put into place through a set of forms and procedures.

Amulya Project Briefs final

Documents

Transcript of Amulya Project Briefs final