CDC’s anti-smoking campaign last year included stunning vignettes offered as tips to smokers from actual former smokers. My favorite spokesperson was Terrie. She suggested smokers make a recording with their own voice for their grandchildren, maybe “sing them a lullaby, while you still can.” She said this with assistance from an artificial voice box while applying her wig.
Even more stunning, for a measurement geek like me, is an impressive paper in the September 2013 edition of The Lancet that describes CDC’s effectiveness measures for the campaign—Tips from Former Smokers. The bottom line measure was that over 100,000 people quit smoking, but the paper reveals many other measures that actually directed the campaign and made it a success.
A useful way to understand CDC’s accomplishment is to explore what they measured, and then look at how they conducted those measures. A framework I like to use in thinking about what to measure is: activities, reactions, retention, and results.
1. Activities: The number of communication activities and their cost.
2. Reactions: The communication noticed by the stakeholder and how they reacted to it.
3. Retention: The knowledge and feelings the stakeholders take away.
4. Results: The behaviors influenced by the communication, and the resulting business outputs and outcomes.
|Screenshot of the CDC's campaign website.|
CDC’s study maps perfectly to these four categories. Then they used focus groups, surveys and existing data to measure within this framework.
The $54 million CDC campaign was the first federally funded, national antismoking campaign. Using existing data about local markets allowed CDC to target their ad placements so carefully that they expected to reach four out of five smokers. Always try to leverage existing data as CDC did in crafting their tactical plans. TV accounted for three-fourths of their tactics, with radio, billboards, internet (banners, displays, Facebook, etc.) and print mainly in supporting roles.
In addition to tracking their exact number of spots they placed they also calculated the number of free placements they received by news organizations covering the campaign.
|Striking poster from the CDC campaign.|
Focus groups were the right measurement tool because they provide understanding. The dialogue nature of focus groups allows the researcher to probe and explore. Measurement before communicating, as CDC accomplished, is called formative research—it helps us adjust our communication plan before launching it.
Did the new message work in reaching people? 78% smokers recalled seeing at least one Tips advertisement on television during the three-month campaign. To determine this, and most of the measures in this study, CDC conducted a survey of a sample of nearly 6,000 smokers and non-smokers before and after the campaign to compare the difference. Since they collected a statistically reliable sample, they were able to generalize their findings to the full U.S. population.
The focus group findings allowed CDC to reset their communication goals—the targets for retained knowledge and feelings. Instead of focusing on the length of life, they now showed how smoking degrades the quality of life and how to access support to quit. For the feelings side, they now targeted empathy for and connection with former smokers. While CDC could have measure this intermediate step of retained knowledge and feelings, they instead cut right to the litmus test—results.
As with any good communication plan, CDC’s targeted behaviors that would yield outputs that would in turn yield outcomes. They targeted three major behaviors:
1. Peer influence: Getting friends and family to talk about the dangers of smoking.
2. Seek support: Getting people to seek resources to help them quit.
3. Attempts to quit: Getting people to actually attempt to stop smoking.
Why these three behaviors? Once again, existing data showed the way. CDC reviewed existing studies of different approaches to influencing people to quit long term, and these three behaviors surfaced as effective. CDC then used their survey to measure whether their campaign influenced these behaviors. Comparing the pre- and post-survey data, they found:
- Peer influence: Recommendations by non-smokers to quit doubled.
- Seek support: There was a dramatic increase in the use of cessation services. For example, calls to 1-800-QUIT-NOW increased 132%.
- Attempts to quit: 12% increase in number of attempts.
The targeted output from these three behaviors was people who quit smoking. Comparing their pre- and post-survey data, they found that the campaign influenced at least 100,000 people to quit smoking.
The outcome CDC targeted was an increase in quality-adjusted life years at a reasonable cost. Quality-adjusted life years is the number of years of life added by an intervention. Each year in perfect health counts for one full year, with partial years added for people in various states of poor health.
CDC’s outcome was almost ½ million life-years added to the US population. Dividing the $54 million cost of the campaign (all costs) by this figure means it cost less than $200 per life-year. Another useful benchmark—the cost represented less than 3 days of the $8 billion the tobacco industry spends annually on promotion and marketing.
What’s the value of your communication efforts? CDC’s approach to measurement represents a gold standard—a level of sophistication that takes time to develop. The trick is to get started with just one or two measures, then slowly build your capacity over time.
Approved for Public Release; Distribution Unlimited. 13-4571
©2014-The MITRE Corporation
* MITRE is a not-for-profit organization that operates research and development centers for sponsoring federal agencies. Jeff is also a Senior Lecturer in Organizational Communication at Northeastern University.
This discussion is brought to you by the Federal Communicators Network. FCN members are government employees managing U.S. government communications. We've published lots more on this topic, including Analytics Is Your Job and Live Tweeting Government Events: DOs and DON'Ts.