Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3...

12
Learning & Evaluation Framework // Page 1 Learning & Evaluation Framework

Transcript of Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3...

Page 1: Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3 Introduction In 2016, Astraea’s program team set out to create a system for learning

Learning & Evaluation Framework // Page 1

Learning & Evaluation Framework

Page 2: Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3 Introduction In 2016, Astraea’s program team set out to create a system for learning
Page 3: Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3 Introduction In 2016, Astraea’s program team set out to create a system for learning

Learning & Evaluation Framework // Page 3

Introduction

In 2016, Astraea’s program team set out to create a system for learning and evaluation, as a means to enable us to more holistically understand, communicate about, and learn from the contributions we are making as a public foundation to LGBTQI movements for social, racial, economic and gender justice. Specifically, we sought to:

• Articulate how our work advances social change (in a particular theme or geography, or overall)

• Learn from and with our grantee partners’ contributions to social change over time

• Capture the value of how we work with partners and movements

• Integrate learning and reflection throughout our programmatic work

• Uphold our accountability to our diverse stakeholders, including the movements we support.

The following table of outcomes and indicators builds on the program team’s work, over a period of several months and with support from an external consultant, to clarify our philosophy and practice of change (including our programmatic vision, values, lineage, beliefs, and core strategies), and to identify the goals, outcomes, and indicators that guide our work and shape our decisions. For a more complete exploration of our philosophy and practice of change—which was itself informed by feedback and recommendations from colleagues on Astraea’s staff and board, as well as our grantee partners, funders, and peers—please see our Theory of Change document.

As part of the process of clarifying our Theory of Change, Astraea program staff articulated four overarching program goals, organized according to the four institutional pillars that guide our work:

• Goal 1: Our grantmaking to organizations supports frontline LGBTQI activists to make autonomous decisions, set their own social change agendas, respond to evolving conditions, care for themselves and their communities, build power, and influence wider movements. Our grantmaking to artists supports them to tell stories and create art that challenges stereotypes, expands understanding, sparks joy, and inspires action.

• Goal 2: As an expression of solidarity, we accompany our grantees by providing access to the resources, expertise, networks, skills, tools, and funding they

need to build their resilience and their individual and collective power, and to deepen and extend their impact.

• Goal 3: We strengthen LGBTQI artists and activists’ ability to challenge norms and assumptions, shape their own narratives, build power and visibility, and shift critical conversations relevant to their lives through access to funding, networks, skills, communications and media tools, and digital technologies.

• Goal 4: We aim to transform philanthropy to be responsive, accessible, accountable, inclusive and transparent and to fund LGBTQI issues, communities, and rights with a race, class and gender justice lens, as well as a bodily diversity and autonomy lens, which will drive more and better funding to communities most impacted by discrimination, violence and institutional oppression.

Although our goals are ambitious, we believe that over time, our collective efforts should be able to generate four categories of changes:

• Outcome Level One: In our grantee partners’ external impact

• Outcome Level Two: In our grantee partners’ internal capacities

• Outcome Level Three: In LGBTQI and allied social justice movements

• Outcome Level Four: In the philanthropic sector

The following outcomes and indicators are organized into the four categories listed above, with a degree of overlap from one category to the next. Each outcome level includes several possible outcomes, and a set of corresponding indicators. We came up with several changes that our grantmaking, capacity-building and leadership development, media and communications work, and philanthropic advocacy will contribute to over time at each outcome level, and for each of those changes, a set of possible indications that progress is occurring.

The outcomes and indicators below were selected based on our own experiences; what we have learned from four decades of work to support and serve activists and movements working to spark change and bring about justice in a variety of geographic, political, social, and cultural contexts; and our peers’ and allies’ thinking and research about how change happens. They are

Page 4: Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3 Introduction In 2016, Astraea’s program team set out to create a system for learning

Page 4 // Learning & Evaluation Framework

also deeply informed by our core values – feminism, intersectionality, responsiveness, inclusion, collectivity, and courage – and our beliefs about how change happens, which are summarized below:

1. Social change is messy, unpredictable, and non-linear. What initially might seem like a setback might in fact be an indication of progress—only time will tell.

2. True social change is collective. While some may seek out ‘movement leaders’ and individual ‘change-makers,’ we honor the reality that no single actor or organization can make lasting change alone, and change runs deeper and lasts longer when it is brought about by collective action. Movements that are founded on shared decision-making, transformative leadership, and deep base-building are less likely to exclude certain groups or individuals, or make decisions that leave entire communities behind.

3. We all have a role in the change process, but the deepest and most lasting change is led by those who are directly impacted by the conditions and problems that need addressing. Support must be aligned around their agendas and priorities.

4. Social change is complex: it has many dimensions, and progress is necessary in all of them. For example, changes in laws and policies are necessary, but not sufficient, to affect people’s lives. Changes in awareness, cultural norms, and access to services and resources are also critical.

5. Much organizing, particularly led by people who are targeted by the state, is intentionally underground, and as a result, can appear invisible to funders or to more mainstream progressive groups. We recognize the role that stealth organizing plays in bringing about change, and the challenges that can cause in a sector where funding and recognition often depend on broad visibility.

6. Healing is inseparable from wider processes of liberation and social justice. Beyond taking action, grassroots organizers need time to build collective safety, well-being, trust, and community, to reflect and connect, and to build power to confront and transform the generational trauma, institutional oppression, and violence in their lives.

7. Deep social change takes a long time, and requires deep faith, strategy, practice, and patience. As we have learned, those invested in supporting such change are committed to the struggle for the long haul.

8. Social change involves strategic decisions. To cope with the lack of alignment between transformative visions and imperfect realities, we sometimes have

to make difficult decisions in order to build the power and progress we need to get closer to realizing our goals. Such decisions should always be transparent, participatory, and guided by our deepest values.

9. Liberation requires radical change in how power and resources are shared. Organizations seeking to challenge oppressive norms and systems must have strategies for building power and organizing their communities and movements, and the role of social justice funders is to ensure that they have the resources they need to do their work.

The beliefs listed above not only guided our selection of the most meaningful and relevant outcomes and indicators, they also shape our thinking about and approach to evaluation. As we move forward with putting a new system for learning and evaluation into practice, we will be guided by the following principles:

• Focus on contribution rather than attribution. Although it is important to lift up and celebrate our victories, as well as the victories of the organizations and movements we support, we must always keep in mind that no one actor is responsible for change. Change is both complex and collective, and the efforts of both visible and invisible partners and predecessors make each victory possible—our people have been resisting, organizing, and fighting throughout history, and the success of contemporary movements builds on a legacy of largely invisible, uncelebrated efforts. In funding and organizing climates that often incentivize and reward individuals and organizations that take credit for collective victories, we recognize that our efforts—and the efforts of the organizations and movements we support—are part of a complex ecosystem, and that change is more lasting and sustainable when it is the result of collective action. For these reasons, we will aim not to take credit for work in ways that don’t recognize these realities.

• Recognize our role as a grantmaker. The question of credit and contribution is particularly relevant for us as a public foundation, since it is not uncommon in the funding world for organizations that provide funding, training, and a variety of other monetary and non-monetary resources to claim responsibility for the changes that result from the work their resources supported. At Astraea, we seek to avoid this practice, while recognizing and exploring the important role that resources—whether that means funds, skills, access to technologies, or anything else—play in bringing about change in a global capitalist context. We are clear at Astraea that any changes sparked by organizations receiving resources from us are not changes brought about by Astraea, but rather changes to which we hope we have contributed in some way. We seek to highlight this nuance in our internal conversations

Page 5: Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3 Introduction In 2016, Astraea’s program team set out to create a system for learning

Learning & Evaluation Framework // Page 5

and public materials, including our communications and development materials. Our approach to learning and evaluation seeks to center the difference between what Astraea does and what our support helps enable, and to challenge the conflation of the two.

• Be accountable to movements, and trust organizations to determine their own goals and priorities. In selecting outcomes and indicators (particularly at the level of our grantee partners’ external impact), we sought to lift up what we have learned from grantee partners about how change tends to unfold across diverse contexts without privileging certain issues, strategies, or approaches over others. We believe that movements and organizations—not funders—should determine their own goals and priorities, which is why we don’t place restrictions on our grants. This, of course, poses challenges for a grantmaker seeking to capture and aggregate the impact of their funding—since our own funders often ask us to frame our work around more specific outcomes than we ask our grantee partners to identify. In developing these outcomes and indicators, we have done our best to strike a balance between what our own funding requires, and the flexibility we know movements and organizations need in order to thrive. We are also committed to making use of the information we gather, and will seek to avoid asking others to spend time and effort gathering unnecessary or excessive information.

• Emphasize process as well as results. Funders and mainstream organizations often privilege certain kinds of changes over others, emphasizing discrete moments of change (such as policy changes or legal victories) over the hard, often less visible work that makes such moments possible, or ensures that they are translated into changes in lived realities. Similarly, victories are often celebrated without examining lasting damage that was done along the way, or people who were left out of the change process. The outcomes and indicators we chose to include emphasize the importance of process as well as results, and highlight the importance of well-being, sustainability, inclusion, and ongoing efforts to bring values and practices into closer alignment.

• Recognize courage without claiming exclusivity. Astraea’s history, and the history of many of our grantee partners and the movements they lead and belong to, are filled with “firsts.” We recognize the importance of naming these acts of courage and boldness, as well as the bravery required to stand up for what is right before others are ready to join in the struggle. But we avoid characterizing these efforts as solitary, or describing ourselves as the “only” organization advancing a particular kind of work, since we cannot know the full extent of what is happening

or has happened around us, and we recognize that the landscape for our collective work is constantly changing.

• Learning is critical, but measurement can be tricky. The outcomes and indicators below are somewhat aspirational, and many of them are challenging to measure. We pledge to do our best to measure them within the bounds of our capacities and resources, and to be responsive to the capacities and resources of our grantee partners as they strive to evaluate and learn from their own work. This is an ongoing process, and we are committed to reviewing and adjusting our guidelines and expectations as needed, based on feedback from the organizations and movements we support.

• Demystify learning and evaluation. Learning and evaluation are topics that many organizations—including Astraea—approach with trepidation. Since normative research and evaluation practices often feel extractive, punitive, and/or opaque, many of us rightfully bring negative associations to the practice of evaluating our work. In developing and sharing this framework, Astraea’s goal is to reclaim the value of learning and evaluation as a means to remain accountable across our relationships, draw as much learning as possible from our work, and help our grantee partners, peers, and funders approach learning and evaluation with a deeper sense of relevance and meaning. As a result, we have chosen to use traditional evaluation terminology such as ‘outcomes’ and ‘indicators’ in the framework that follows, since these are the terms most commonly used in the evaluation world, and we want to equip our grantee partners to navigate that world with more ease.

Page 6: Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3 Introduction In 2016, Astraea’s program team set out to create a system for learning

Page 6 // Learning & Evaluation Framework

Page 7: Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3 Introduction In 2016, Astraea’s program team set out to create a system for learning

Learning & Evaluation Framework // Page 7

Table of Outcomes and Indicators, By Outcome Level

Outcome Level 1: Grantee partners’ external impact

Outcome 1.1: Stronger capacity among grantee partner community members to effect change

• Growth in number of members, leaders, volunteers, or other key constituents

• Increase in engagement and public representation of members, base, or constituents

• Diversification of membership and other constituencies, particularly among directly affected people, including greater public representation and leadership of directly affected people

• Stronger mobilization capacity

• Members and/or constituents have greater access to wellness, healing practices, and/or integrated protection strategies

Outcome 1.2: Changes in public understanding of LGBTQI issues and communities and/or new narratives by and about historically oppressed people within LGBTQI communities

• Increase in positive or neutral coverage of LGBTQI issues, people, and/or grantee partners in new or traditional media (including representations of LGBTQI people in TV, film, and other mainstream media)

• Development of new media, art, or other creative or cultural materials by and for historically oppressed people within LGBTQI communities that challenge stereotypes and expand understanding of LGBTQI people’s lived realities

• Expansion of online or offline reach (via online presence and platforms, social media, contact lists)

• Increase in public visibility and authority of grantee partners and/or constituents

• Shifts in public conversations as a result of advocacy and communications messages or materials, media attention, etc. developed by or centering the voices and experiences of historically oppressed people within LGBTQI communities

Outcome 1.3: Changes in or incremental progress toward laws, institutional practices, and policies that will benefit grantee partners’ constituencies

• Proposal, amendment, or passage of positive laws or policies that advance protection, limit harm, and/or increase equality for LGBTQI people

• Mitigation, defeat, or challenges to negative laws or policies that would undermine LGBTQI rights

• New or stronger relationships with policymakers, lawyers, members of the media, public officials, or other key advocacy allies

• Escalation of demonstrations or interventions for accountability in state (including police) actions

• Development of a new evidence base for policy change via documentation of rights violations of LGBTQI people, research, campaigns, reports, public hearings, or other methods

• Constituents elected or appointed to meaningful or influential roles in public office or on state advisory bodies

• Ensuring meaningful implementation and enforcement of laws, policies, and other legal protections

Outcome 1.4: Stronger relationships, alliances, and partnerships across a wide range of social justice issues, movements, and identities

• New or stronger relationships, alliances, and partnerships

• Increase in collaborative actions and campaigns

• Increase in solidarity actions by and for grantee partners and/or their members or constituents

Page 8: Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3 Introduction In 2016, Astraea’s program team set out to create a system for learning

Page 8 // Learning & Evaluation Framework

Outcome Level 2: Grantee partners’ internal capacities

Outcome 2.1: Deeper organizational resiliency and/or strengthened capacity in areas of self-identified importance

• Increased ability to weather external crises and internal challenges

• More shared understanding and clearer articulation of goals, vision, and strategies

• Strengthened capacity to articulate concrete change(s) sparked or sustained by the work

• Strategies revised and refined based on what worked and what didn’t

• New or deeper capacity and/or knowledge in areas of self-identified importance

• Deeper capacity to shift narratives, influence public conversations, and/or leverage media attention

• Stronger systems and infrastructure to sustain the work and support the people carrying it out

• Increase in representation of directly affected people among organizational leaders or decision-makers, including staff and board

• Organizational messages, materials, and practices reflect a commitment to collective leadership

Outcome 2.2: Increased resource base and stronger capacity to mobilize resources

• Increase in external funding (grants from foundations, governments, or other sources)

• Increase in community funding (grassroots fundraising from individuals, contributions from founders or members, etc.)

• Diversification of funding and income sources (including social enterprises or other income-generating activities)

• Increase in unrestricted resources and/or multi-year support

• Stronger capacity to manage funds, develop budgets, and report on finances

• Deeper and more critical understanding of the funding landscape and stronger capacity to make informed decisions about where and how to seek funds within it

• Strengthened capacity to receive funds (obtaining or making progress toward legal registration/

status, establishing a bank account, establishing a relationship with a fiscal host, identifying a new fiscal host, or improving terms with an existing fiscal host)

• Dedicated staff for development or resource mobilization

• New or stronger relationships with funders, greater access to funding spaces, increase in submissions/applications to other funders

Outcome 2.3: Increased access to a range of movement and advocacy spaces to build, connect, strategize, network, set priorities, and/or influence policies, processes, and decisions

• Increased access to key movement or advocacy spaces (local, national, regional, or global) among a diversity of organizational staff and/or members (beyond top leadership)

• Increase in diversity of participants in key movement and advocacy spaces (especially directly affected people)

• New skills, relationships, and collaborations built through participation in collective spaces

• Training received or knowledge built in relevant areas among a diversity of organizational staff and/or members (beyond top leadership)

Outcome 2.4: Deepened use of healing justice, holistic security, resiliency and/or survival practices that center the collective safety and emotional, physical, spiritual, environmental and mental well-being of communities

• Sustainability and safety practices to address the impact of violence and trauma, including interpersonal, systemic and generational violence, integrated into organizing strategies

• Cultural and holistic practices relevant to the community, including ancestral based traditions, affirmed and integrated into organizing strategies

• Strengthened leadership and organizing skills that incorporate cultural and political practices and analyses that explore safety, security, and well-being as integral to movement-building and collective survival

• Development of collective care strategies to address burnout, PTSD, secondary trauma and emotional and spiritual exhaustion experienced by activists/organizers

• Increased knowledge of and use of holistic security practices, including attention to digital and physical

Page 9: Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3 Introduction In 2016, Astraea’s program team set out to create a system for learning

Learning & Evaluation Framework // Page 9

security

• Deeper engagement in organizational analysis, including political education that addresses the pathologization of LGBTQI people and communities and cultivates new practices that meaningfully improve access to quality and dignified care (such as providing direct access to holistic practitioners and challenging harmful ideologies of well-being)

• Expansion of safety, security and well-being strategies and structures that are led and defined for, by and about communities who are marginalized, objectified, policed and kept under surveillance by state, medical and scientific institutions and structures

Outcome 2.5: Adoption of more intersectional agendas and practices

• Space created for, by, and about historically oppressed communities to address their own collective desires and needs (for example, a venue for political dialogue, political education, and building greater alignment around action to address internal oppressions)

• More attention to power dynamics, diversity of knowledge and practices, and the range of LGBTQI identities and experiences in programmatic work (including media, messages, materials, services, campaigns, and actions)

• More attention to power dynamics, diversity of knowledge and practices, and the range of LGBTQI identities and experiences in internal organization practices and culture

• Increase in representation of directly affected people among organizational leaders or decision-makers (especially staff and board)

• New partnerships and alliances built or collaborative actions taken with allied social justice organizations and movements.

Outcome Level 3: Movement-level shifts

Outcome 3.1: LGBTQI movement agendas, actions, and spaces are more inclusive and intersectional

• Effective recognition, visibility, and/or leadership of LBQ women, trans people, and intersex people in LGBTQI movements, with an emphasis on collective leadership

1 SOGIESC :Sexual Orientation ,Gender Identity and Expression ,and Sex Characteristics

• Effective recognition, visibility, and/or leadership of people facing forms of discrimination that are not primarily based on sexuality, gender, or sex characteristics within LGBTQI movements (e.g. people of color in the U.S. or Dalit and oppressed caste people in South Asia)

• Deeper awareness and recognition of the wide spectrum of lived experience within LGBTQI communities in media, messages, materials, and platforms developed by coalitions and campaigns for LGBTQI rights

• Increased support for activists and organizations who represent marginalized identities within LGBTQI movements to bring their whole selves to the table

• Decrease in movement compromises or ‘wins’ that benefit privileged people at the expense of historically oppressed people within LGBTQI communities

• Stronger alignment between identities of movement leadership and movement constituencies

• Messages and practices reflect collective understandings of leadership and affirm a diversity of skills, perspectives, and trajectories, rather than lifting up individual ‘changemakers’

Outcome 3.2: Allied social justice movement agendas, spaces, and actions are more LGBTQI-inclusive

• Effective representation, visibility, and/or leadership of LGBTQI people in allied social justice movements (e.g. migrant justice, climate justice, racial justice)

• Media, messages, materials, and platforms developed by social justice coalitions and campaigns mentions LGBTQI people and/or includes a SOGIESC1 analysis

• Allied social justice movements acknowledge and affirm the gender identities, gender expression, sex characteristics, and sexual orientations of LGBTQI participants

• Decrease in movement compromises or ‘wins’ that marginalize or exclude LGBTQI people or rights

Outcome 3.3: Greater leadership and representation of historically oppressed communities and most impacted people within LGBTQI and social justice movements

• Meaningful participation and inclusion of self-led groups in LGBTQI and allied movements and coalitions

• Meaningful participation and direct representation of

Page 10: Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3 Introduction In 2016, Astraea’s program team set out to create a system for learning

Page 10 // Learning & Evaluation Framework

people from historically oppressed communities in local, national, regional, and global LGBTQI movement and advocacy spaces

• Self-led organizations that represent the priorities and realities of historically oppressed LGBTQI people are shaping and influencing movement agendas

• Internal and external understandings of leadership and progress of most impacted people are more reflective of collective leadership and less focused on individual leaders or ‘movement stars’

Outcome Level 4: Shifts in philanthropy

Outcome 4.1: Increased recognition and support for activist-led, activist-informed, or community-led grantmaking in LGBT philanthropy

New activist-led, activist-informed, and community-led funds and foundations established (including emerging funds and foundations incubated, supported, or hosted by Astraea)

Increase in percentage of total funding for LGBTQI rights controlled by activist-led, activist-informed, and community-led funds and foundations

Activists enjoy greater access to philanthropic spaces, institutions, and decision-making processes

Outcome 4.2: LGBT and women’s rights funders have increased and improved funding across the spectrum of people’s lived

experiences of gender, sexuality and bodily diversity

Explicit mention of and/or support for the work and the rights of trans people, gender nonconforming people, intersex people, and/or LBQ women in materials, messages, programs, practices, or budgets developed by women’s rights funders (new or established)

Explicit mention of and/or support for the work and the rights of trans people, gender nonconforming people, intersex people, and/or LBQ women in materials, messages, programs, practices, or budgets developed by LGBT funders (new or established)

Increase in funding explicitly dedicated to the work and the rights of trans people, gender nonconforming people, intersex people, and/or LBQ women (disaggregated by population)

Outcome 4.3: LGBT funders have increased and improved funding to LGBTQI groups working to advance racial, social, and economic justice

Increase in amount/percentage of LGBT funding for LGBTQI groups centering the priorities and leadership of working-class LGBTQI people

Increase in amount/percentage of LGBT funding for people of color and migrants (US)

Increase in amount/percentage of LGBT funding for anti-criminalization and migrant justice work (US)

Increase in amount/percentage of LGBT funding for people from oppressed ethnicity/race/caste backgrounds respective to their regional contexts and people who face heightened levels of policing and criminalization, including sex workers (Global)

Outcome 4.4: LGBT funding policies and practices have become more aligned with grassroots priorities and realities, as well as social justice values

Increase in funding for LGBTQI groups that is not explicitly tied to outcomes related to legal and policy change

Increase in long-term, multi-year funding available to LGBTQI groups

Increase in core, flexible, and/or general operating support available for LGBTQI groups of any size

Increase in funding for LGBTQI groups to do grassroots or community organizing

Increase in funding for regional LGBTQI coalitions and networks in the Global South to do regional and global work (rather than funding Global North organizations to do work on LGBTQI rights in the Global South)

Increase in funding available for unregistered LGBTQI groups

Sustained and/or increased funding in places where civil society spaces are shrinking and funding regulations are increasing (only track in places that are geographic priorities for Astraea)

Funders’ materials, messages, and practices reflect more collective understandings of leadership and recognize the harmful impact of lifting up individuals as ‘movement stars’

Page 11: Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3 Introduction In 2016, Astraea’s program team set out to create a system for learning

Founded in 1977, the Astraea Lesbian Foundation for Justice is one of the oldest women’s funds in the world and the only fund dedicated to LBTQI rights globally. Through grantmaking, capacity building, philanthropic advocacy, and media and communications, we support brilliant and brave grassroots activists and artists who challenge oppression and seed social change. In our 40 year history, we are proud to have granted over $40 million to LBTQI activists in artists.

Page 12: Learning & Evaluation Framework · 2019-02-04 · Learning & Evaluation Framework // Page 3 Introduction In 2016, Astraea’s program team set out to create a system for learning

116 East 16th Street, 7th Floor New York, NY 10003

P: 1.212.529.8021 F: 1.212.982.3321 [email protected]

www.astraeafoundation.org