Monday, December 30, 2013

CDC Sets a Gold Standard for Communication Measurement in Government

Posted by: Jeff Brooke, Principal Consultant, Organizational Communication & Change Management, The MITRE Corp.* - December 30, 2013

CDC’s anti-smoking campaign last year included stunning vignettes offered as tips to smokers from actual former smokers. My favorite spokesperson was Terrie. She suggested smokers make a recording with their own voice for their grandchildren, maybe “sing them a lullaby, while you still can.” She said this with assistance from an artificial voice box while applying her wig. 

Even more stunning, for a measurement geek like me, is an impressive paper in the September 2013 edition of The Lancet that describes CDC’s effectiveness measures for the campaign—Tips from Former Smokers.  The bottom line measure was that over 100,000 people quit smoking, but the paper reveals many other measures that actually directed the campaign and made it a success. 

A useful way to understand CDC’s accomplishment is to explore what they measured, and then look at how they conducted those measures. A framework I like to use in thinking about what to measure is: activities, reactions, retention, and results. 

1.    Activities:
The number of communication activities and their cost.
2.    Reactions: The communication noticed by the stakeholder and how they reacted to it.
3.    Retention: The knowledge and feelings the stakeholders take away.
4.    Results: The behaviors influenced by the communication, and the resulting business outputs and outcomes. 
Screenshot of the Tips From Former Smokers website. Visible text includes the CDC's logo, with text, "Centers for Disease Control and Prevention: CDC 24/7: Saving Lives, Protecting People." A header above rotating photos is cut off but reads, "TIPS FROM FORMER SMOKERS: In memory of Nathan Moose, Tips campaign participa... public health hero, who dies of illness caused by seco... smoke on October 17, 2013. He was 54... Learn more ab..." Below, there is a photo of Terrie, mentioned in the blog post.
Screenshot of the CDC's campaign website.

CDC’s study maps perfectly to these four categories. Then they used focus groups, surveys and existing data to measure within this framework.

1. Activities

The $54 million CDC campaign was the first federally funded, national antismoking campaign. Using existing data about local markets allowed CDC to target their ad placements so carefully that they expected to reach four out of five smokers.  Always try to leverage existing data as CDC did in crafting their tactical plans. TV accounted for three-fourths of their tactics, with radio, billboards, internet (banners, displays, Facebook, etc.) and print mainly in supporting roles.

In addition to tracking their exact number of spots they placed they also calculated the number of free placements they received by news organizations covering the campaign.

2. Reactions

Poster reads, "A TIP FROM A FORMER SMOKER. RECORD YOUR VOICE FOR LOVED ONES WHILE YOU STILL CAN. Terrie, Age 52, North Carolina." There is a additional text below, but it is too small to read.
Striking poster from the CDC campaign.
The campaign developers assumed that hard-hitting images of death would motivate smokers to try to quit. Fortunately, they tested this assumption using focus groups of smokers—and proved themselves wrong. Smokers indicated that death is an abstract concept for many and simply didn’t make an impression. But the focus group participants also indicated that the diseases from smoking—long before death—could be quite meaningful. So CDC shifted gears and developed the campaign you’ve probably seen.

Focus groups were the right measurement tool because they provide understanding. The dialogue nature of focus groups allows the researcher to probe and explore.  Measurement before communicating, as CDC accomplished, is called formative research—it helps us adjust our communication plan before launching it.

Did the new message work in reaching people? 78% smokers recalled seeing at least one Tips advertisement on television during the three-month campaign. To determine this, and most of the measures in this study, CDC conducted a survey of a sample of nearly 6,000 smokers and non-smokers before and after the campaign to compare the difference. Since they collected a statistically reliable sample, they were able to generalize their findings to the full U.S. population.

3. Retention

The focus group findings allowed CDC to reset their communication goals—the targets for retained knowledge and feelings. Instead of focusing on the length of life, they now showed how smoking degrades the quality of life and how to access support to quit. For the feelings side, they now targeted empathy for and connection with former smokers. While CDC could have measure this intermediate step of retained knowledge and feelings, they instead cut right to the litmus test—results.

4. Results

As with any good communication plan, CDC’s targeted behaviors that would yield outputs that would in turn yield outcomes. They targeted three major behaviors:

1.    Peer influence: Getting friends and family to talk about the dangers of smoking. 
2.    Seek support: Getting people to seek resources to help them quit.
3.    Attempts to quit: Getting people to actually attempt to stop smoking.

Why these three behaviors? Once again, existing data showed the way. CDC reviewed existing studies of different approaches to influencing people to quit long term, and these three behaviors surfaced as effective. CDC then used their survey to measure whether their campaign influenced these behaviors.  Comparing the pre- and post-survey data, they found:

  • Peer influence: Recommendations by non-smokers to quit doubled.
  • Seek support: There was a dramatic increase in the use of cessation services. For example, calls to 1-800-QUIT-NOW increased 132%.
  • Attempts to quit: 12% increase in number of attempts.

The targeted output from these three behaviors was people who quit smoking. Comparing their pre- and post-survey data, they found that the campaign influenced at least 100,000 people to quit smoking.

The outcome CDC targeted was an increase in quality-adjusted life years at a reasonable cost. Quality-adjusted life years is the number of years of life added by an intervention. Each year in perfect health counts for one full year, with partial years added for people in various states of poor health.

CDC’s outcome was almost ½ million life-years added to the US population. Dividing the $54 million cost of the campaign (all costs) by this figure means it cost less than $200 per life-year. Another useful benchmark—the cost represented less than 3 days of the $8 billion the tobacco industry spends annually on promotion and marketing.

What’s the value of your communication efforts? CDC’s approach to measurement represents a gold standard—a level of sophistication that takes time to develop. The trick is to get started with just one or two measures, then slowly build your capacity over time.

Approved for Public Release; Distribution Unlimited. 13-4571
©2014-The MITRE Corporation

* MITRE is a not-for-profit organization that operates research and development centers for sponsoring federal agencies. Jeff is also a Senior Lecturer in Organizational Communication at Northeastern University.

This discussion is brought to you by the Federal Communicators Network. FCN members are government employees managing U.S. government communications. We've published lots more on this topic, including Analytics Is Your Job and Live Tweeting Government Events: DOs and DON'Ts.

No comments:

Post a Comment

Please leave a comment!