2015 American Evaluation Association Annual International Conference

TCC Group will be presenting throughout the AEA Conference. Their presentations are listed below.

Wednesday November 11

3:00-4:30pm: 2374: How do YOU define a leader?

Foundations and nonprofits all around are looking for ways to measure the effectiveness of leadership development programs. But what constitutes a leader? What attributes should be measured to determine if someone has developed leadership? What are the considerations for deciding which attributes should be measured, and does this vary by sector?  Does it vary for those in formal vs. informal leadership roles? What is the role of an evaluator in helping to answer these questions? This session will place you as a fly on the wall listening in on how both large foundations and small nonprofits have grappled with these same questions and what this evaluation consultant has taken away from her experience in working with them.  

Presenters

• Lisa Frantzen, Senior Evaluation Consultant, TCC Group (Session Leader)

Thursday, November 12

3:00-4:00pm: PTTDE1: Frameworks and Methods – Learnings from the Implementation of Program

In conventional project management, it is often assumed that project evaluation should take a starting point in the “Iron Triangle” (goals are met, on time, within budget). However, an increasing amount of literature suggests that this perspective on project success is too narrow and should include broader notions of value. Many advocate a systems-theory approach that includes emergent and learning-oriented perspectives on project dynamics and multiple perspectives on success. When it comes to evaluation of projects, however, most of these ideas in the project literature lack specificity, and few take advantage of TBE.

Presenters

• Lisa Frantzen, TCC Group (Chair)
• Peter Dahler-Larsen, University of Copenhagen
• Sarah Strydom, Rape Crisis

4:45-6:15pm: 2383: Putting the “Impact” in Impact Investing: A Stakeholder-Focused Approach the Evaluating the Investing Logic Chair

The field of impact investing continues to represent a growing market, with more investors seeking to put their money behind ventures that will make a positive impact in the social sphere. As such, evaluators should have frameworks in place in order to evaluate these ventures. Because of the large amount and seemingly dense nature of data associated with impact investing, it is important to make the evaluations of impact investing manageable and meaningful. For that reason, this ignite presentation will lay out a stakeholder-focused logic chain for the evaluation of impact investing. This logic chain thoroughly lays out the key components of these ventures with the goal that stakeholders can be sure their impact investment truly does demonstrate impact.    

Presenters

• Rose Kowalski, Evaluation Consultant, TCC Group (Session Leader)

Friday, November 13

8:00-9:00am: 2393: Program Theory and Theory-Driven Evaluation Business Meeting: What Constitutes Exemplary Program Theory Development and Use?

Most contemporary evaluation approaches incorporate aspects of program theory techniques into their prescriptions for high-quality evaluation practice. Despite the ubiquity of program theory in evaluation practice, there is little agreement among evaluation scholars and practitioners about what constitutes exemplary program theory development and use. In this panel, thought leaders in program theory principles will share their latest thinking on program theory concepts and propose new agendas for program theory research and practice. The panel also includes evaluation practitioners, who will draw on their experience to provide insights into the mechanisms through which program theory techniques drive organizational change and propose new ways for practitioners to leverage technological tools in their program theory services. Audience members will come away with an understanding of the latest thinking and research on program theory as well as powerful new techniques for improving their evaluation practice.

Presenters

• Katrina L Bledsoe, DeBruce Foundation (Chair)
• Jennifer C Greene, University of Illinois at Urbana-Champaign
• John Gargani, Gargani + Company Inc.
• Charles Gasper, TCC Group
• Matthew Galen, Claremont Graduate University

1:45-3:15pm: 2063: Advocacy as a Team Game: Evaluating Multi-Stakeholder Advocacy Efforts

In today’s complex advocacy environment, it is rare that a single organization can pursue goals on its own. Rather, organizations generally work together, sometimes in very coordinated campaigns and sometimes in less formal networks or movements. In such an environment evaluators have to both assess advocacy outcomes and address the particular roles and contributions of multiple partners working together on some activities and in parallel on others. This session will present some of the most recent thinking from evaluators doing this work on the ground. It will include discussions of processes to consider for contribution analysis, how to assess the way organizations rely on each other, and how to use a field-building perspective to evaluate multi-stakeholder advocacy initiatives.

Presenters

• David Devlin-Foltz, The Aspen Institute (Chair)
• Jared Raynor, TCC Group
• Carlyn Orians, ORS Impact
• Jewlya Lynn, Spark Policy Institute
• Sue Hoechstetter, Alliance for Justice

1:45-3:15pm: 2407: Developing Shared Measurements to Assess Health and Well-being in a Community-based Health Improvement Initiative

New Brunswick, New Jersey’s city-wide initiative Healthier New Brunswick (HNB) adopts a cross-sector, community-based approach to understanding and building targeted interventions to address the range of pressing health needs across its urban environment.  Established by four entities—New Brunswick Tomorrow, Rutgers Robert Wood Johnson Medical School, Johnson & Johnson, and the City of New Brunswick—HNB plays an important role in connecting and mobilizing health and wellness efforts in the city.

This panel, comprised of representatives from three of the key partners of the initiative as well as the evaluation consultant, will share their experiences in developing a shared measurement system for the multi-sectoral initiative. Speakers will share how core health and well-being metrics were developed for New Brunswick, how stakeholder groups and community engagement factored into the process, and the critical lessons that were learned along the way.

Presenters

• Lisa Frantzen, Senior Evaluation Consultant, TCC Group (Chair)
• Shaun Mickus, Executive Director, Corporate Contributions & Community Relations, Johnson & Johnson
• Dr. Eric Jahn, Senior Associate Dean for Community Health, Rutgers Robert Wood Johnson Medical School
• Jaymie Santiago, Director, Program Operations, New Brunswick Tomorrow

1:45-3:15pm: 2299: Theory-Driven Evaluation Science Exemplars: Stories from the Field

This panel features evaluation practitioners sharing theory-driven evaluation science (TDES) successes and lessons learned. Three practitioners will each share a story of what, how, and why he/she used TDES and the impact of using TDES. Kirk Knestis, CEO with Hezel Associates (Syracuse, NY), will describe his team’s work with a complex National Science Foundation STEM learning research & development project. Charles Gasper will tell the story of evaluating a complex Public Media collaborative. And, Veronica Smith, Principal with data2insight, will share how TDES contributed to non-profit organizational changes in attitudes about the use of evaluation to inform elementary math program development. Session participants will take away tips, resources, and practices that they can apply in their work. For those new to TDES, this is an opportunity to better understand this evaluation approach by ‘seeing’ exemplars in action.

Presenters

• Karen Jackson, Custom Evaluation Services (Chair)
• Kirk Knestis, Hezel Associates, LLC
• Charles Gasper, TCC Group
• Veronica S. Smith, MSEE, data2insight LLC

3:30-4:15pm: 2272: Strategic Litigation: Advocacy Evaluation’s Latest Frontier

Using the judiciary as a means of redress related to critical societal issues is a fundamental element of our democracy. While most advocacy (and advocacy evaluation) focuses on legislative and executive advocacy, savvy advocates include advocacy through the courts (i.e. judicial advocacy) as another tool in their toolkit.  This session will lay out a framework that will allow evaluators to better understand judicial advocacy (also known as strategic litigation, impact litigation, and public interest litigation), recognize key evaluation questions related to judicial advocacy, and be equipped with a set of indicators and examples related to the subject.  Presenters will include real-life examples of strategic litigation evaluations touching many issues (including the death penalty, education, immigration, human rights, and others). The session will also present to the AEA community the results of a field-wide examination of strategic litigation through an evaluation lens.

Presenters

• Tanya Beer, Center for Evaluation Innovation (Chair)
• Jared Raynor, TCC Group
• Deepti Sood, TCC Group
• Kay Sherwood, Independent Evaluator

Saturday, November 14

7:00-7:45am: 2290: From Flexible to Formal Campaign Infrastructure: the Implication for Evaluators

Campaigns have varying levels of infrastructure. While some may be highly structured with a solid governing body, others may be much less formal and have no clear point-of-contact. TCC Group has evaluated many campaigns that have had various structures around campaign manager roles, what the governing body looks like, etc. This panel will explore how evaluators can stay relevant in campaigns with different structures. TCC’s evaluators will share five models reflecting the variety of campaign infrastructure, from more ad-hoc to more highly formalized. For each model, we will discuss implications for evaluation and our lessons learned from working with the different models. There will also be time for discussion so other evaluators working on campaigns can have the opportunity to discuss how our models fit into their practice.

• Katherine Locke, TCC Group (Chair)
• Deepti Sood, TCC Group

7:00-7:45am: 2323: Navigating Evaluation Detours Mid-Route: What to do when a Major Transition Throws Key Stakeholders Off-Track

Even when an evaluator has engaged in thoughtful planning and analysis throughout an evaluation project, there is always the opportunity for some things to go wrong. Roadblocks like low sample sizes and errors in data collection are predictable and somewhat easy to control for. However, other blocks are almost entirely outside of an evaluator’s control, even in a well-planned evaluation. One of these blocks occurs when clients, leadership, or other key stakeholders undergo major transitions in mid-evaluation. These transitions can include layoffs, resignations, or major changes in organizational mission and vision. The goals of this think tank are to first define common transitions and then to catalogue practical tips for predicting for overcoming those obstacles. Participants will leave the session with concrete strategies for how an evaluation can go on in the face of major stakeholder transitions.

Presenters

• Rose Kowalski, Evaluation Consultant, TCC Group (Session Facilitator)
• Lisa Frantzen, Senior Evaluation Consultant, TCC Group

Host

American Evaluation Association