Storytelling Souvenirs

What TCC Group’s Evaluation Team Brought Back from The American Evaluation Association’s Evaluation 2023 Conference

When our plane touched down on the runway at Philadelphia International Airport, the TCC Group Evaluation and Learning team’s suitcases were just a little weightier than when we departed. That’s because from October 9th-14th, we attended the American Evaluation Association (AEA)’s Evaluation 2023 Conference with 2,000 enthusiastic evaluators.

The theme of the conference was The Power of Story, and we brought home new insights on how we could leverage storytelling in the work we do for our clients. Taking a cue from AEA, we’re harnessing the Power of Story to share some insights and takeaways from our encounters that we are excited to bring back to our work, over the course of two stories.

Rose’s Story: Putting Descartes before the Report

I sighed and struggled as I used pushpins to hang up my 4 x 4 printed informational sign in the exhibit hall of the Indianapolis JW Marriott. Minutes later, evaluators buzzed around me, talking amongst themselves about the hundreds of presentation posters like mine that surrounded them. Six-inch tall text at the top of my poster asked passers-by “DO YOUR EVALUATION FINDINGS LAND WITH A THUD?” It focused on using a cartesian grid system to lay out findings in a way that would maximize impact – Putting Descartes before the Report. Evaluators from around the U.S. and the world stopped by to tell me about their travails in laying out evaluation findings in a way that their audience could absorb and then run with.

One evaluator for a children’s literacy organization described her funders’ recent pushes for her findings to be presented in a compelling way – preferably in the format of a PowerPoint deck rather than a Word document. The problem was that the findings were so meaty that the PowerPoint slides themselves ended up becoming cluttered with dense text, almost negating the point of switching the format from Word. How was she supposed to lay out her robust findings in a way that actually landed with her audience? Many at that poster session had come to similar realizations: the way we as evaluators choose to visualize findings affects audience uptake.

I walked away from my conversations in the exhibit hall enthusiastic about my responsibility to ensure that the findings that I produce register – and better yet – are actually used. This requires thinking from a design and storytelling standpoint and engaging in the following practices:

1) Putting the why at the forefront of any deliverable. Findings will begin by telling the audience why it’s extremely important to care about this content, and then they will dive into the content itself.

2) Respecting the time of my audience and those we gather data from, working hard so the findings don’t get consigned to a folder that sits on a shelf.

3) Emphasizing key points in service of the why. Designing deliverables with plenty of white space throughout allows them to rest their eyes and allows for the emphasis of key points, all in service of delivering a meaningful message.

Kathleen’s Story: Following the Stewardship Example

Even though it was 8:15 AM and I was perched on a moderately comfortable hotel conference room chair, I was invigorated. Five hundred evaluators sat around me and heard Tanaya Winder– author, singer/songwriter, speaker, and founder of As/Us: A Space for Women of the World, a literary magazine publishing works by BIPOC women charge us with being stewards of other peoples’ stories. I thought about this charge throughout every subsequent AEA session I led and attended. Stewardship is the secure and responsible management of something, often something that never “belonged” to you in the first place. For evaluators like us, that translates to thoughtfulness and intentionality about how and with whom we communicate our findings. I came away from the conference with a renewed focus on three key aspects of storytelling for my evaluation practice: matching storytelling strategies with appropriate audiences, communicating evaluation findings for the purpose of learning, and effectively sharing findings to communicate impact.

Ten months into a twelve-month project, as she polished and refined the remaining details in her drafted evaluation report, an evaluator asked her client whom the final report would be shared with. Thinking about how we communicate evaluation findings often comes toward the end of a project- but this is too late. Before we conduct our first interview, send a survey invitation, or even develop data collection tools, we should be thinking about how and with whom we will share what we learn. Conversations with clients or evaluation partners about who the audiences will be, and what the best methods for reaching those audiences are, should happen at the beginning of the engagement, in the planning stage.

One year after an evaluation project wrapped, an evaluator asked program staff how they had incorporated learnings from an evaluation project into their work. Though it was encouraging to hear one staff member describe one tweak they had made, it was disheartening to see so few able to cite examples of actually using the findings. One key purpose for sharing findings is to promote learning and decision-making, so we want to communicate findings in ways that spur meaning-making and action. To do this, it can be helpful to focus attention on key findings and critical decision points using methods such as sharing data in simple graphics and charts, hosting data gallery walks, and facilitating meaning-making sessions.

An evaluator we know followed the directions of her organization’s board to a T. After rigorous analysis, she poured her findings into a long, text-heavy final report, exactly as was requested of her. But when the report’s content never came up in conversation again, she started to wonder whether anyone had internalized it – or if they even read it.  A long and text-heavy final report, while sometimes the most appropriate format, is not the only option. Infographics distill the main findings into bite-sized and visually pleasing chunks. Standalone executive summaries, including visual elements, can play the same role. Case studies or vignettes are often effective ways to elevate success stories. Participants in a roundtable discussion that I led during the conference shared creative techniques they have used to communicate impact, including poetry slams and artwork created by community members who were part of an evaluation, podcasts, Instagram stories, and documentary films.

To be good stewards of the data people trust us with, it’s important to consider who our audience is, and how our storytelling method might best promote learning and decision-making and communicate impact for those audiences. As Tanaya Winder finished her remarks, I packed away my notebook, stood up from the JW Marriott conference room chair, and set out determined to become a responsible steward of data and practice effective storytelling in my evaluation work with the ideas I’d bring home from the 2023 AEA Conference.

The stories and souvenirs we brought back from AEA will continue to drive the way we learn and iterate on sharing findings, as we ensure that the data can be used meaningfully.  To share your stories with us, reach out here! We look forward to sharing more of our takeaways from AEA in the coming months.