Equity as a Capacity

This is Part Two of our five-part series titled, “Equity and Evaluation: Models of How Equity Can and Does Impact Evaluation.” Read Part One: Equity as a Leading Principle. The subsequent three parts will be released over the next two months (updated February 25, 2019).

Scenario Two

Equity as a capacity – How does an evaluator assess the uptick of equity when it is delivered via capacity-building trainings?

The Situation

TCC Group was engaged in a multi-year, multi-site evaluation on behalf of a foundation. In addition to providing operating funds for grantees to engage in their work, the foundation also placed a heavy emphasis on capacity-building and technical assistance activities, as well as a strong emphasis internally on equity. This led to a situation where each site eventually received explicit trainings meant to improve their understanding of issues related to equity and how they could better use an equity lens in their work supporting economic and family development for people experiencing poverty.

The Evaluator’s Role

TCC’s role was to evaluate the efficacy, relevance, and impact of the technical assistance provided. While we were interested in understanding what technical assistance ideas stuck and ultimately entered the culture of each site, we were also interested in how each site responded to the idea being introduced by the funder rather than organically from the work taking place.

What We Did Well

A mixed-methods approach worked well in this scenario. Using a survey that asked respondents to rate their own adoption of values year-over-year as well as through in-person interviews, TCC discerned a true uptick around issues of equity, and how values varied by individuals. We also workshopped our data collection tools with grantees to ensure we were asking about equity in a way that resonated. One site, for example, always showed a relatively high commitment to equity based on survey data, but interviews helped us determine that it largely addressed equity in a superficial manner.

We also defined equity at both the value level and the operations level. We measured not just the level of buy-in and conversation about equity, but also the specific changes in organizational practices and procedures. We then verified these organizational changes with staff to understand how comprehensively these changes took place in reality.

Where We Can Improve

Ideally, evaluators would have the access to speak directly with program participants about their experiences. There’s not necessarily a correlation between program staff believing they have embodied equity into their work and the clients’ experiences . Because of the complicated structure of this particular evaluation, we weren’t able to talk to any clients directly to understand what values they felt were driving the program work. This led to a staff-driven perspective rather than an understanding of how the adoption of an equity lens was perceived (if at all) by participants.

Lessons Learned

• Be explicit about equity. Use a clear, unified definition that applies to all types of stakeholders. Arriving at a clear definition may require some facilitation between the groups of stakeholders to determine a point of common language that considers the various perspectives.
• Ask stakeholders at all levels about their experiences with equity and what has changed since the implementation of the program. Rather than just including leadership, include line staff and people receiving direct services themselves in data collection, which can provide a truer understanding of how equity is—or isn’t—being adopted into the culture and processes of work.

Want to Learn More?

We’ve found these additional resources to be useful:

• Equitable Evaluation Initiative
• UNICEF’s Guide to Equity in Evaluation (hosted on BetterEvaluation)
• Race Equity Tool’s Clearinghouse
• Public Policy Associates Considerations for Using a Culturally Responsive and Racial Equity Lens