Equity as a Leading Principle

This is Part One of our five-part series titled, “Equity and Evaluation: Models of How Equity Can and Does Impact Evaluation.” The subsequent four parts will be released over the next two months (updated February 11, 2019).

The philanthropic and nonprofit fields, and organizations and individuals serving those fields, have shown an increasing focus on equity over the last few years [1]. Conversations have moved from murmurings to deeper engagement by institutions like Grantmakers for Effective Organizations, the Annie E. Casey Foundation, and the Equitable Evaluation Initiative, which have explicitly published principles on applying an equity lens to grantmaking and evaluation. 

At TCC Group, we have also taken a more proactive approach to making equity an internal value, including the establishment of a Diversity Committee that works on policies and practices related to our firm’s own work on diversity, equity, and inclusion. We’ve instituted a robust learning agenda for staff across the firm and strengthened our evaluation work by participating in the American Evaluation Association’s Graduate Education Diversity Initiative (GEDI). This win-win initiative is committed to fostering a more diverse field of evaluators, and participating allows us to provide hands-on experiences for evaluators from diverse backgrounds. It also helps us gain critical insight into how those being actively trained in Culturally Responsive Evaluation (CRE) perceive our evaluation practices [2].

Upon absorbing the wealth of information shared by our colleagues, we realized we’ve had varying experiences in how we’ve incorporated an equity lens (or lack thereof) into our work [3]. We’ve decided to share some of our experiences with our peers in the field in order to spur conversation, disseminate our lessons learned, and encourage others to talk about these issues. One of our biggest challenges in navigating equity issues in evaluation is how unprepared people can be to talk candidly. We hope that by sharing our experiences, we can make this conversation more explicit.

While we absolutely don’t pretend to have all the answers, or indeed a lot of them, we do think that our learnings will help guide others trying to embed the principles of equitable evaluation into their work [4].

Here are five scenarios around equity in which we’ve had to clarify our role and form an appropriate response:

1. Equity as a leading principle – How does an evaluator explicitly embed equity into their lens of evaluation?
2. Equity as a capacity – How does an evaluator assess the uptick of equitable practice when it is delivered via capacity-building trainings?
3. Equity as an afterthought – How does an evaluator focus on equity when it has been added as a new outcome years after the initiative has been in place?
4. Equity as a point of tension – How does an evaluator raise issues related to equity that are influencing the work when key players do not have a clear equity lens?
5. Equity as tokenism – How does an evaluator work within a situation when they are asked to favor certain staff over others because of their ethnic backgrounds or other demographic considerations?

Each scenario stems from actual experiences with clients, though we present them here without identifying information in order to place the focus on the concepts.

Scenario One

Equity as a leading principle – How does an evaluator explicitly embed equity into their lens of evaluation?

The Situation

TCC Group worked with a foundation that had an explicit focus on equity and expected itself—and its consultants and grantees—to use equity as a leading principle. As a way to implement this principle, the funder often sought to make sure that grassroots and values-driven organizations received the necessary funds to gain access to larger community conversations or actions.

The Evaluator’s Role

Our role was to evaluate the impact of the work that had been awarded over the last five years. Part of impact was defined by how effectively the foundation supported these grassroots and values-driven organizations in gaining access to conversations and contributing to decision-making.

What We Did Well

Our team used a mixed-methods approach in our work. This allowed us to build a more iterative process with multiple levels of reflection around the work and the impact of values (including equity) on that work. Using two methods—interviews and surveys—gave equal weight to qualitative feedback that may not have been reflected in the more quantitative survey.

We also focused on embedding participant ownership. In this particular project, we held space in the report to share stories of success directly from participants (i.e., not necessarily filtered through the lens or voice of the evaluator). We gleaned these stories from our raw interview notes and shared them with grantees for their review and approval. This allowed for grantees to define success in their own ways and explicitly sign off on how their work and impact were portrayed in the report. Transferring some of the ownership to participants also established a greater sense of transparency between the funder and the grantee, which is one way to create a more equitable dynamic between the groups.

Finally, because equity was an explicit value guiding the evaluation, we focused on measuring it explicitly. We dove deep into the data available to understand what types of organizations were being funded and what opportunities existed to support other organizations. This allowed us to raise the profile of some equity-focused organizations in a way that these organizations may not have been able to do on their own.

Where We Can Improve

While including grantee success stories is a good start, we’d like to challenge ourselves to include more participatory feedback from the actual grantees representing these grassroots organizations. This can happen by forming a stakeholder advisory group or by giving grantees an opportunity to weigh in on the evaluation design. We feel this would increase the likelihood that our evaluation design and final products would best answer the highest-priority questions for all stakeholders.

We’d also like to be more purposeful about building the evaluation relationship before diving into data collection. With a limited project budget, it’s easy to default to faster data collection methods (e.g., surveys) that might provide less nuanced data. We’ve learned from previous experience that grassroots organizations tend to value face-to-face meetings, building relationships, and gaining a full understanding of what the evaluation work and how it will be used. Ideally, we could have spent more time building these relationships with grantees to develop trust before jumping straight into asking for data. Furthermore, we did not pilot the tools we used for language to ensure that the ways the funder, the evaluators, and the grantees were talking about equity and outcomes were similar.

Lessons Learned

Funders who lead with equity may have a difficult time hearing feedback on how their intentions may be missing the mark. Funders who feel they are already on the morally correct side of leading with values may be hesitant to hear feedback that grantees see flaws in their approach. In these instances, we believe it’s the evaluator’s duty to ask questions about which groups are being excluded and highlight the value of developing more inclusive criteria than what the funder created originally.

Equity-based funders may have an interest in a more participatory evaluation but it may not necessarily be in line with their budgets. Regardless, it’s worth offering a range of participatory activities to make sure some level of client voice is included.

Want to Learn More?

We’ve found these additional resources to be useful:

• Equitable Evaluation Initiative
• UNICEF’s Guide to Equity in Evaluation (hosted on BetterEvaluation)
• Race Equity Tool’s Clearinghouse
• Public Policy Associates Considerations for Using a Culturally Responsive and Racial Equity Lens

[1] The Annie E. Casey Foundation defines equity as “the state, quality or ideal of being just, impartial and fair.” The concept of equity is synonymous with fairness and justice. It is helpful to think of equity as not simply a desired state of affairs or a lofty value. To be achieved and sustained, equity needs to be thought of as a structural and systemic concept.”

[2] The GEDI (Graduate Education Diversity Initiative) program is run by the American Evaluation Association and meant to introduce graduate students from underrepresented populations to the evaluation field. Associated activities include a site placement (where interns work two days a week) and additional activities linked to evaluation curricula, such as attending conferences.

[3] As a definition for equity, TCC tends to use the definition provided by Annie E. Casey (above). We haven’t fully defined equity lens for our own practice, but see it as emphasizing and bringing a focus on equity into our work as evaluators.

[4] The three articulated principles by the Equitable Evaluation Initiative are: 1) evaluation and evaluative work should be in service of equity; 2) evaluation work can and should answer critical questions about ways historical and structure decisions have contributed to the condition to be addressed, effect of a strategy on different populations and underlying systemic drivers of inequity, and ways in which cultural context is tangled up in both the structural conditions and the change initiative itself; 3) evaluative work should be designed and implemented commensurate with the values underlying equity work (e.g., multi-culturally valid and oriented toward participant ownership). Learn more about the Equitable Evaluation principles.