4 Key Stages for Measuring Your Media Outcomes

What can media accomplish? What can it affect? These are questions funders often ask. Restating those questions from an evaluation perspective translates to: how can one measure the impact of media? To understand how to measure media, we must first understand the strategy behind media.

Media:

• Draws attention to issues.
• Improves understanding of issues and possible solutions.
• Inspires people to care.
• Motivates people to do something.
• Connects people with solutions.

I previously spoke to these, diagramming the flow of media effect, where people move through various different states from awareness to action. Looking at the model, it is easy to think that one piece of content can affect change.
However, it is rare that just one exposure to one piece of content affects change; rather it is the sum of multiple experiences – a subset of which is content. Savvy media professionals take into account the environmental factors in the development and distribution of their content, building on what is available, constructing multiple pieces of content for distribution, and developing a campaign for that distribution to affect their target audiences.

During the 2016 Olympic Games in Rio, NBC aired several commercials from Proctor and Gamble (P&G) in which athletes’ moms were featured as those who helped develop the inner strength of their children. The connection was to P&G’s products and the company’s role as a “proud sponsor of Moms” – a continuation of P&G’s commercials from the 2014 Sochi Winter Olympics and the 2012 London Olympics. This long-running campaign seeks to tie greatness to the support of moms; Proctor and Gamble’s products in turn provide needed support for a mom’s work. In this example, P&G is targeting parents – specifically moms of children who dream – a clear target audience. The content is tied to the Olympics, where P&G can spotlight the greatness of youth affected by the support of moms. They effectively connect how moms might feel about their relationship with their children and their hope for success with the excitement and emotions of the Games. Finally, their content ran concurrent to the Games on various platforms that moms might watch, including television and the Internet.

These campaigns often consist of a series of content that have various components, which include:

• Background awareness and education content that makes people aware of the issue and possible solutions.
• Strong, seminal pieces that build on the background content to bring the issue and possible solutions to the forefront.

The background content tends to be short pieces, enough to convey a thought or idea and little more. The seminal pieces tend to be longer, perhaps in the form of a documentary, news piece, or town-hall meeting. They build on the content already presented, bringing attention to those issues that have been rattling around in the background.

When and What to Measure

There is no magic formula for measurement, but rather it must be tied to the work and intended consequences. The work is really defined by four key stages: design, development, distribution, and reflection. Each stage has opportunities for measurement and can engage various associated methods.

Design

What We Do: In this stage, before content is produced, the project managers, production staff, and other engaged individuals (possibly funders or key informants) consider what changes they want to affect in the audience and how they will construct their work. The focus is on building a draft campaign plan and an initial discussion of the types of content and topics.

What We Measure: Measurement here focuses on understanding what the audience currently knows, their attitudes, and likelihood of engaging in the issues of the campaign. Various methods can be used to collect the needed information. Often surveys are employed, but other more traditional methods include interviews and focus groups of audience representatives.

Development

What We Do: In the development stage, the production staff is considering and putting together content for distribution. The ideal content will depend on the baseline understanding of the audience from the design phase—using known industry methods for development of content for specific platforms—and the desired changes; multiple pieces of content are created, targeting the various levels in the media flow. Content may include background pieces that build a knowledge base or seminal pieces that pull information together in a more dramatic manner.

What We Measure: Measurement of the preliminary content often focuses on each piece’s individual effects with audience representatives viewing the material and providing feedback. Data collection can be in the form of pre-/post-tests (i.e., having people state their views or thoughts before and then after being exposed to the content), interviews, and focus groups. Rarely are the audience representatives exposed to multiple pieces of content with the goal of assessing the cumulative effects; that typically occurs with the actual audience in the next stage. The focus of this stage is quality of content and whether it has the expected individual effect.

Distribution

What We Do: In this stage, the developed content is shared with the audiences. It is here that we focus on:

• Dosage – How much content is being exposed to the audience?
• Engagement – How involved is the audience with the content?
• Effect – Are the audience members moving through the various points of the flow?

Media professionals are now concerned with learning whether the audience is experiencing enough of the content to affect change.

Audience impact can be happening at various levels within the flow. It is here where adjustments can be made to the amount and how the content is shared. Tracking where the audience is within the flow and providing feedback to the distributors, helping them identify what content should be disseminated. It is also at this stage where it might be decided that new content needs to be developed due to the responses of the audience, sending the process back to the development phase.

What We Measure: Here measurement reflects the interest to tie the content to possible changes in the audience. Often engaged in surveys, the more general audience is asked how much of the content they have experienced and where they sit in the flow. The audience is asked this on a periodic basis, allowing for the assessment of trends, moving from a baseline of having no exposure to the content to the completion of the campaign.

Reflection

What We Do: Occurring at the completion of the campaign, the funders and developers of the content ask whether the campaign had its intended impact. If the processes of evaluation, data collection, and analysis have occurred in each of the previous stages, the probability of having a good outcome is pretty high at this point. It is at this stage where the media professionals and key stakeholders look at the results of the work and have an opportunity to learn from the overall experience, informing future campaigns.

What We Measure: Depending on what has been collected throughout the campaign, trends in audience knowledge, views, and activities might be considered. More likely, these are measured at the end of the campaign and compared to the audience’s baseline. More interested parties might dig deeper into the data, assessing what factors within and external to the campaign drove these changes.

Why is this Important?

Evaluation at all of these stages is helpful and in some cases critical to the success of the campaigns. The most often conducted assessment, and frankly most critical, is the baseline and post campaign comparison, assessing the effectiveness of the campaign. However, this assumes that the media developed will have the expected impact. This can have disastrous consequences. As seen in research on the effect of the well-known “This is Your Brain On Drugs” campaign, where funds went to media to promote the dangers of drug use. Rather than having the intended impact of driving youth away from drug use, there is evidence that the effect was opposite. By engaging in better testing in design and periodic assessments during the distribution phase, the funders and content developers could have caught the issues earlier and adjusted the content to better affect the audience.

AT&T’s “It can wait” campaign—an effort to discourage and reduce texting while driving—is an example of an attempt to use evaluation in a more proactive manner. Measuring changes in attitudes, changes in behavior, as well as changes in the uptake of the content, this campaign stepped beyond the pre-test/post-test structure. While the longer-term outcomes appear to not yet be affected and may not ever be affected by AT&T’s campaign, AT&T is conducting surveys, focus groups as well as using social media analytics and data available from states to assess whether there is an effect.

However, the process of capturing this data, including a representative population to initially assess the content in the design phase, is expensive and often not funded. Instead, both the funder and proxies within the organization developing the content are often used.

Recommendations

For you and your organization, when you consider or implement media campaigns, it is important that you do as many of the following:

If you engage in these behaviors, you will have the information you need to increase the likelihood that your media campaign will be successful.

In the next series of blog posts, we will go more deeply into each of the steps. We will discuss measurement and how evaluation can provide you with the information you need to observe your impact, identify opportunities, and improve the chances of affecting change.