text:
Why Should You Use This Book?
Overview: The Health Communication Process
Stage 1: Planning and Strategy Development
Stage 2: Developing and Pretesting Concepts, Messages, and Materials
Stage 3: Implementing the Program
Stage 4: Assessing Effectiveness and Making Refinements
Communication Research Methods
Appendix A: Communication Planning Forms and Samples
Appendix B: Selected Planning Frameworks, Social Science Theories, and Models of Change*
Appendix C: Information Sources
Appendix D: Selected Readings and Resources
Appendix E: Glossary
The National Cancer Institute will no longer print hard copy books or compact discs of Making Health Communication Programs Work, also known as the Pink Book. Full-length HTML and PDF versions of this publication can be printed via the Page Option tools on this page. (04/02/2008)
This book is a revision of the original Making Health Communication Programs Work, first printed in 1989, which the Office of Cancer Communications (OCC, now the Office of Communications) of the National Cancer Institute (NCI) developed to guide communication program planning. During the 25 years that NCI has been involved in health communication, ongoing evaluation of our communication programs has affirmed the value of using specific communication strategies to promote health and prevent disease. Research and practice continue to expand our understanding of the principles, theories, and techniques that provide a sound foundation for successful health communication programs. The purpose of this revision is to update communication planning guidelines to account for the advances in knowledge and technology that have occurred during the past decade.
To prepare this update, NCI solicited ideas and information from various health communication program planners and experts (see Acknowledgments). Their contributions ranged from reviewing and commenting on existing text to providing real-life examples to illustrate key concepts. In addition, the Centers for Disease Control and Prevention (CDC) provided extensive input as part of the agency's partnership with NCI.
Although communicating effectively about health is an exacting task, those who have the earlier version of this publication know that it is possible. We hope the ideas and information in this revision will help new health communication programs start soundly and mature programs work even better.
Why Should You Use This Book?
The planning steps in this book can help make any communication program work, regardless of size, topic, geographic span, intended audience, or budget. (Intended audience is the term this book uses to convey what other publications may refer to as a target audience.) The key is reading all the steps and adapting those relevant to your program at a level of effort appropriate to the program's scope. The tips and sidebars throughout the book suggest ways to tailor the process to your various communication needs.
If you have limited funding, you might
- Work with partners who can add their resources to your own
- Conduct activities on a smaller scale
- Use volunteer assistance
- Seek out existing information and approaches developed by programs that have addressed similar issues to reduce developmental costs
Don't let budget constraints keep you from setting objectives, learning about your intended audience, or pretesting. Neglecting any of these steps could limit your program's effectiveness before it starts.
This book describes a practical approach for planning and implementing health communication efforts; it offers guidelines, not hard and fast rules.Your situation may not permit or require each step outlined in the following chapters, but we hope you will consider each guideline and decide carefully whether it applies to your situation.
Questions to Ask and Answer
- Can communication help us achieve all or some of our aims?
- How can health communication fit into our program?
- What theories, models, and practices should we use to plan our communication program?
- What types of evaluation should we include?
The Role of Health Communication in Disease Prevention and Control
There are numerous definitions of health communication. The National Cancer Institute and the Centers for Disease Control and Prevention use the following:
The study and use of communication strategies to inform and influence individual and community decisions that enhance health.
Use the principles of effective health communication to plan and create initiatives at all levels, from one brochure or Web site to a complete communication campaign. Successful health communication programs involve more than the production of messages and materials. They use research-based strategies to shape the products and determine the channels that deliver them to the right intended audiences.
Since this book first appeared in 1989, the discipline of health communication has grown and matured. As research has continued to validate and define the effectiveness of health communication, this book has become a widely accepted tool for promoting public health. Healthy People 2010, the U.S. Department of Health and Human Services' stated health objectives for the nation, contains separate objectives for health communication for the first time. Meanwhile, the availability of new technologies is expanding access to health information and raising questions about equality of access, accuracy of information, and how to use the new tools most effectively.
What Health Communication Can and Cannot Do
Understanding what health communication can and cannot do is critical to communicating successfully. Health communication is one tool for promoting or improving health. Changes in health care services, technology, regulations, and policy are often also necessary to completely address a health problem.
Communication alone can:
- Increase the intended audience's knowledge and awareness of a health issue, problem, or solution
- Influence perceptions, beliefs, and attitudes that may change social norms
- Prompt action
- Demonstrate or illustrate healthy skills
- Reinforce knowledge, attitudes, or behavior
- Show the benefit of behavior change
- Advocate a position on a health issue or policy
- Increase demand or support for health services
- Refute myths and misconceptions
- Strengthen organizational relationships
Communication combined with other strategies can:
- Cause sustained change in which an individual adopts and maintains a new health behavior or an organization adopts and maintains a new policy direction
- Overcome barriers/systemic problems, such as insufficient access to care
Communication cannot:
- Compensate for inadequate health care or access to health care services
- Produce sustained change in complex health behaviors without the support of a larger program for change, including components addressing health care services, technology, and changes in regulations and policy
- Be equally effective in addressing all issues or relaying all messages because the topic or suggested behavior change may be complex, because the intended audience may have preconceptions about the topic or message sender, or because the topic may be controversial
Communication Can Affect Multiple Types of Change
Health communication programs can affect change among individuals and also in organizations, communities, and society as a whole:
- Individuals—The interpersonal level is the most fundamental level of health-related communication because individual behavior affects health status. Communication can affect individuals' awareness, knowledge, attitudes, self-efficacy, skills, and commitment to behavior change. Activities directed at other intended audiences for change may also affect individual change, such as involving patients in their own care.
- Groups—The informal groups to which people belong and the community settings they frequent can have a significant impact on their health. Examples include relationships between customers and employees at a salon or restaurant, exercisers who go to the same gym, students and parents in a school setting, employees at a worksite, and patients and health professionals at a clinic. Activities aimed at this level can take advantage of these informal settings.
- Organizations—Organizations are groups with defined structures, such as associations, clubs, or civic groups. This category can also include businesses, government agencies, and health insurers. Organizations can carry health messages to their constituents, provide support for health communication programs, and make policy changes that encourage individual change.
- Communities—Community opinion leaders and policymakers can be effective allies in influencing change in policies, products, and services that can hinder or support people's actions. By influencing communities, health communication programs can promote increased awareness of an issue, changes in attitudes and beliefs, and group or institutional support for desirable behaviors. In addition, communication can advocate policy or structural changes in the community (e.g., sidewalks) that encourage healthy behavior.
- Society—Society as a whole influences individual behavior by affecting norms and values, attitudes and opinions, laws and policies, and by creating physical, economic, cultural, and information environments. Health communication programs aimed at the societal level can change individual attitudes or behavior and thus change social norms. Efforts to reduce drunk driving, for example, have changed individual and societal attitudes, behaviors, and policies through multiple forms of intervention, including communication.
Multistrategy health communication programs can address one or all of the above.
Communication Programs Can Include Multiple Methods of Influence
Health communicators can use a wide range of methods to design programs to fit specific circumstances. These methods include:
- Media literacy—teaches intended audiences (often youth) to deconstruct media messages so they can identify the sponsor's motives; also teaches communicators how to compose messages attuned to the intended audience's point of view
- Media advocacy—seeks to change the social and political environment in which decisions that affect health and health resources are made by influencing the mass media's selection of topics and by shaping the debate about those topics
- Public relations—promotes the inclusion of messages about a health issue or behavior in the mass media
- Advertising—places paid or public service messages in the media or in public spaces to increase awareness of and support for a product, service, or behavior
- Education entertainment—seeks to embed health-promoting messages and storylines into entertainment and news programs or to eliminate messages that counter health messages; can also include seeking entertainment industry support for a health issue
- Individual and group instruction—influences, counsels, and provides skills to support desirable behaviors
- Partnership development—increases support for a program or issue by harnessing the influence, credibility, and resources of profit, nonprofit, or governmental organizations
Characteristics of Effective Health Communication Campaigns
Certain attributes can make health communication campaigns more effective. Use the guidelines in this section to plan your campaign. Define the communication campaign goal effectively:- Identify the larger goal
- Determine which part of the larger goal could be met by a communication campaign
- Describe the specific objectives of the campaign; integrate these into a campaign plan
- Identify the group to whom you want to communicate your message
- Consider identifying subgroups to whom you could tailor your message
- Learn as much as possible about the intended audience; add information about beliefs, current actions, and social and physical environment to demographic information
Create messages effectively:
- Brainstorm messages that fit with the communication campaign goal and the intended audience(s)
- Identify channels and sources that are considered credible and influential by the intended audience(s)
- Consider the best times to reach the audience(s) and prepare messages accordingly
- Select a few messages and plan to pretest them
Pretest and revise messages and materials effectively:
- Select pretesting methods that fit the campaign's budget and timeline
- Pretest messages and materials with people who share the attributes of the intended audience(s)
- Take the time to revise messages and materials based upon pretesting findings
Implement the campaign effectively:
- Follow the plans you developed at the beginning of the campaign
- Communicate with partners and the media as necessary to ensure the campaign runs smoothly
- Begin evaluating the campaign plan and processes as soon as the campaign is implemented
Theories Guide Action to Increase Mammography Use
Fox Chase Cancer Center, in cooperation with area managed care organizations, designed a program that was based on key elements of the health belief model to encourage women to have regular mammograms. Selected women received educational materials explaining that virtually all women are at risk for breast cancer, regardless of the absence of symptoms, and that risk increases with age (susceptibility). The materials stressed that early detection brings not only the best chance of cure but also the widest range of treatment choices (benefit). Women received a letter stating their physician#8217;s support (cue to action) and a coupon for a free mammogram (to overcome the cost barrier). Those who did not have a mammogram within 90 days received different forms of reminders (cues to action). In the most intensive reminder, a telephone counselor called selected women to review their perceptions about susceptibility, benefits, and barriers. Program evaluation showed that mammography use increased substantially.
The Fox Chase program also applied social learning theory in developing interventions to encourage physician support of mammography and to improve clinical breast examinations (CBEs). The planners examined the environmental and situational factors that might affect physician behavior and tried to change the low expectations of physicians about the benefits of breast screening. The interventions included observational learning by watching an expert perform a CBE, an opportunity to increase self-efficacy by practicing CBE with the instructor, and the use of a feedback report and CME credits to reinforce physician skills.
In taking a community approach to change, a UCLA mammography program used a diffusion of innovations model. Community analysis showed that women who were early adopters (leaders) already had a heightened awareness of the value of mammography. To reach middle adopters, the program mobilized the social influence of the early adopters by using volunteers who had breast cancer to provide mammography information. The program also provided highly individualized educational strategies linked to social interaction approaches to reach late adopters. A social marketing framework influenced the program's planning approach, and media materials incorporated the health belief model to promote individual behavior change.
Communication programs can take advantage of the strengths of each of the above by using multiple methods. A program to decrease tobacco use among youth, for example, could include:
- Paid advertising to ensure that youth are exposed to on-target, unfiltered motivational messages
- Media advocacy to support regulatory or policy changes to limit access to tobacco
- Public relations to support anti-tobacco attitudes
- Media literacy instruction in schools to reduce the influence of the tobacco industry
- Entertainment education and advocacy to decrease the depiction of tobacco use in movies
- Partnerships with commercial enterprises (such as retail chains popular among youth) to spread the anti-smoking message
Using multiple methods increases the need for careful planning and program management to ensure that all efforts are integrated and consistently support program goals and objectives.
Planning Frameworks, Theories, and Models of Change
Sound health communication development should draw upon theories and models that offer different perspectives on the intended audiences and on the steps that can influence their change. No single theory dominates health communication because health problems, populations, cultures, and contexts vary. Many programs achieve the greatest impact by combining theories to address a problem. The approach to health communication we use in this book is based on the social marketing framework. (See Appendix B for an overview of some other relevant theoretical models.) Social marketing concentrates on tailoring programs to serve a defined group and is most successful when it is implemented as a systematic, continuous process that is driven at every step by decision-based research, which is used as feedback to adjust the program.*
National Objectives for Research and Evaluation
The Health Communication chapter of Healthy People 2010, the nationwide health promotion and disease prevention agenda, identifies increasing the proportion of health communication activities that include research and evaluation as one of six objectives for the field for the next decade (objective 11-3). This objective focuses attention on the need to make research and evaluation integral parts of initial program design. Research and evaluation are used to systematically obtain the information needed to refine the design, development, implementation, adoption, redesign, and overall quality of a communication intervention.
Why Use Theories and Models?
Although theories cannot substitute for effective planning and research, they offer many benefits for the design of health communication programs. At each stage of the process outlined in this book, theories and models can help answer key questions, such as:
- Why a problem exists
- Whom to select
- What you need to know about the population/intended audience before taking action
- How to reach people and make an impact
- Which strategies are most likely to cause change
Reviewing theories and models can suggest factors to consider as you formulate your objectives and approach, and can help you determine whether specific ideas are likely to work. Theories and models can guide message and materials development, and are also useful when you decide what to evaluate and how to design evaluation tools.
How Market Research and Evaluation Fit Into Communication Programs
Conducting market research is vital to identifying and understanding intended audiences and developing messages and strategies that will motivate action. Evaluations conducted before, throughout, and after implementation provide data on which to base conclusions about success or failure and help to improve current and future communication programs.
Evaluation should be built in from the start, not tacked on to the end of a program. Integrating evaluation throughout planning and implementation ensures that you:
- Tailor messages, materials, and activities to your intended audience
- Include evaluation mechanisms (e.g., include feedback forms with a community guide)
- Define appropriate, meaningful, achievable, and time-specific program objectives
Evaluating your program's communication efforts enables you to:
- Understand what is and is not working, and why
- Improve the effort while it is under way and improve future efforts
- Demonstrate the value of the program to interested parties such as partners, funding agencies, and the public
- Help program staff see how its work affects the intended audiences
In this book, we address appropriate evaluation activities for each stage; see the Communication Research Methods section for a description of the different types of research and evaluation that support each stage of the health communication process. See Appendix A for sample forms and instruments.
* From Theory at a Glance: A Guide for Health Promotion Practice (NIH Publication No. 97-3896), by the National Cancer Institute, 1995. Bethesda, MD. In the public domain.
Selected Readings
- Andreasen, A. (1995). Marketing social change: Changing behavior to promote health, social development, and the environment. San Francisco: Jossey-Bass.
- Backer, T. E., Rogers, E. M., & Sopory, P. (1992). Designing health communication campaigns: What works. Thousand Oaks, CA: Sage.
- Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.
- Centers for Disease Control and Prevention. (2001). CDCynergy 2001 [CD-ROM]. Atlanta.
- Glanz, K., Lewis, F. M., & Rimer, B. K. (Eds.). (1997). Health behavior and health education: Theory, research, and practice (2nd ed.). San Francisco: Jossey-Bass.
- Glanz, K., & Rimer, B. K. (1995). Theory at a glance: A guide for health promotion practice (NIH Publication No. 97-3896). Bethesda, MD: National Cancer Institute.
- Goldberg, M. E., Fishbein, M. F., & Middlestadt, S. E. (Eds.). (1997). Social marketing: Theoretical and practical perspectives. Mahwah, NJ: Erlbaum.
- Green, L.W., & Kreuter, M.W. (1999). Health promotion planning: An educational and ecological approach (3rd ed.). Mountain View, CA: Mayfield.
- Maibach, E., & Parrott, R. L. (Eds.). (1995). Designing health messages: Approaches from communication theory and public health practice. Thousand Oaks, CA: Sage.
- National Cancer Institute. (1993). A picture of health (NIH Publication No. 94-3604). Rockville, MD: U.S. Department of Health and Human Services.
- Rimer, B. K. (1995). Audiences and messages for breast and cervical cancer screenings. Wellness Perspectives: Research, Theory, and Practice, 11(2), 13−39.
- Siegel, M., & Doner, L. (1998). Marketing public health: Strategies to promote social change. Gaithersburg, MD: Aspen.
- U.S. Department of Health and Human Services. (2000). Healthy people 2010 (2nd Ed.; in two volumes: Understanding and improving health and Objectives for improving health.). Washington, DC: U.S. Government Printing Office.
Overview: The Health Communication Process
The Stages of the Health Communication Process
For a communication program to be successful, it must be based on an understanding of the needs and perceptions of the intended audience. In this book, we incorporate tips on how to learn about the intended audience's needs and perceptions in each of the program stages. Remember, these needs and perceptions may change as the project progresses, so be prepared to make changes to the communication program as you proceed. To help with planning and developing a health communication program, we have divided the process into four stages: Planning and Strategy Development; Developing and Pretesting Concepts, Messages, and Materials; Implementing the Program; and Assessing Effectiveness and Making Refinements. The stages constitute a circular process in which the last stage feeds back into the first as you work through a continuous loop of planning, implementation, and improvement.
Use this book to produce and implement a plan for a communication program. The final plan will include the following components:
- General description of the program, including intended audiences, goals, and objectives
- Market research plans
- Message and materials development and pretesting plans
- Materials production, distribution, and promotion plans
- Partnership plans
- Process evaluation plan
- Outcome evaluation plan
- Task and time table
- Budget
Because this process is not linear, do not expect to complete a stage and then move to the next, never to go back.You will be exploring opportunities, researching issues, and refining plans and approaches as your organization implements the program. This ongoing, iterative process characterizes a successful communication program.
To help work through program planning and development, we suggest many steps within each stage.You may not find all of the steps suggested in each stage feasible for your program, or even necessary. As you plan, carefully examine available resources and what you want to accomplish with the program and then apply the steps that are appropriate for you. However, if you carefully follow the steps described in each stage of the process, your work in the next phase may be more productive.
Each of the four stages is described here; they are described in more detail in the subsequent sections of this book.
Stage 1: Planning and Strategy Development
In this book, all planning is discussed within the Planning and Strategy Development section, but the concepts you learn there apply across the life cycle of a communication program. During Stage 1 you create the plan that will provide the foundation for your program. By the end of Stage 1, you will have:
- Identified how your organization can use communication effectively to address a health problem
- Identified intended audiences
- Used consumer research to craft a communication strategy and objectives
- Drafted communication plans, including activities, partnerships, and baseline surveys for outcome evaluation
Planning is crucial for the success of any health communication program, and doing careful work now will help you avoid having to make expensive alterations when the program is under way.
Stage 2: Developing and Pretesting Concepts, Messages, and Materials
In Stage 2, you will develop message concepts and explore them with the intended audience using qualitative research methods. By the end of Stage 2, you will have:
- Developed relevant, meaningful messages
- Planned activities and drafted materials
- Pretested the messages and materials with intended-audience members
Getting feedback from intended audiences when developing messages and materials is crucial for the success of every communication program. Learning now what messages are effective with the intended audiences will help you avoid producing ineffective materials.
Stage 3: Implementing the Program
In Stage 3, you will introduce the fully developed program to the intended audience. By the end of Stage 3, you will have:
- Begun program implementation, maintaining promotion, distribution, and other activities through all channels
- Tracked intended-audience exposure and reaction to the program and determined whether adjustments were needed (process evaluation)
- Periodically reviewed all program components and made revisions when necessary
Completing process evaluations and making adjustments are integral to implementing the program and will ensure that program resources are always being used effectively.
Stage 4: Assessing Effectiveness and Making Refinements
In Stage 4, you will assess the program using the outcome evaluation methods you planned in Stage 1. By the end of Stage 4, you will have:
- Assessed your health communication program
- Identified refinements that would increase the effectiveness of future program iterations
Because program planning is a recurring process, you will likely conduct planning, management, and evaluation activities described in Stages 1–4 throughout the life of the program.
Stage 1: Planning and Strategy Development
Questions to Ask and Answer
- What health problem are we addressing?
- What is occurring versus what should be occurring?
- Whom does the problem affect, and how?
- What role can communication play in addressing the problem?
- How and by whom is the problem being addressed? Are other communication programs being planned or implemented? (Look outside of your own organization.)
- What approach or combination of approaches can best influence the problem? (Communication? Changes in policies, products, or services? All of these?)
- What other organizations have similar goals and might be willing to work on this problem?
- What measurable, reasonable objectives will we use to define success?
- What types of partnerships would help achieve the objectives?
- Who are our intended audiences? How will we learn about them?
- What actions should we encourage our intended audiences to take?
- What settings, channels, and activities are most appropriate for reaching our intended audiences and the goals of our communication objectives? (Interpersonal, organizational, mass, or computer-related media? Community? A combination?)
- How can the channels be used most effectively?
- How will we measure progress? What baseline information will we use to conduct our outcome evaluation?
Why Planning Is Important
The planning you do now will provide the foundation for your entire health communication program. It will enable your program to produce meaningful results instead of just boxes of materials. Effective planning will help you:
- Understand the health issue you are addressing
- Determine appropriate roles for health communication
- Identify the approaches necessary to bring about or support the desired changes
- Establish a logical program development process
- Create a communication program that supports clearly defined objectives
- Set priorities
- Assign responsibilities
- Assess progress
- Avert disasters
Under the pressure of deadlines and demands, it is normal to think, "I don't have time to plan; I have to get started NOW." However, following a strategic planning process will save you time. Because you will define program objectives and then tailor your program's activities to meet those objectives, planning will ensure that you don't spend time doing unnecessary work. Program objectives are generally broader than communication objectives, described in step 2 on page 20, and specify the outcomes that you expect your entire program to achieve. Many of the planning activities suggested in this chapter can be completed simultaneously. Even if your program is part of a broader health promotion effort that has an overall plan, a plan specific to the communication component is necessary.
Planning Steps
This chapter is intended to help you design a program plan. The health communication planning process includes the following six steps explained in this chapter:
- Assess the health issue or problem and identify all the components of a possible solution (e.g., communication as well as changes in policy, products, or services).
- Define communication objectives.
- Define and learn about intended audiences.
- Explore settings, channels, and activities best suited to reach intended audiences.
- Identify potential partners and develop partnering plans.
- Develop a communication strategy for each intended audience; draft a communication plan.
To complete this process, use the Communication Program Plan template in Appendix A to help ensure that you don't miss any key points.
1. Assess the Health Issue/Problem and Identify All Components of a Solution
The more you understand about an issue or health problem, the better you can plan a communication program that will address it successfully. The purpose of this initial data collection is to describe the health problem or issue, who is affected, and what is occurring versus what should be occurring. Doing this will allow you to consider how communication might help address the issue or problem. In this step, review and gather data both on the problem and on what is being done about it.
Review Available Data
To collect available data, first check for sources of information in your agency or organization. Identify gaps and then seek outside sources of information. Sources and availability of information will vary by issue. The types of information you should (ideally) have at this stage include descriptions of:
- The problem or issue
- The incidence or prevalence of the health problem
- Who is affected (the potential intended audience), including age, sex, ethnicity, economic situation, educational or reading level, place of work and residence, and causative or preventive behaviors. Be sure to include more information than just basic demographics
- The effects of the health problem on individuals and communities (state, workplace, region, etc.)
- Possible causes and preventive measures
- Possible solutions, treatments, or remedies
To find this information, search these common data sources:
- Libraries (for journal articles and texts)
- Health-related resources on the Internet
- Sources of health statistics (a local hospital, a state health department, the National Center for Health Statistics on the CDC Web site)
- Administrative databases covering relevant populations
- Government agencies, universities, and voluntary and health professional organizations
- Clearinghouses
- Community service agencies (for related service-use data)
- Corporations, trade associations, and foundations
- Polling companies (for intended audience knowledge and attitudes)
- Depositories of polling information (e.g., the Roper Center)
- Chambers of commerce
- Advertising agencies, newspapers, and radio and television stations (for media-use data, buying and consumption patterns)
Both published and unpublished reports may be available from these sources. A number of federal health information clearinghouses and Web sites also provide information, products, materials, and sources of further assistance for specific health subjects. A helpful first step in planning may be to contact the appropriate Web sites and the health department to obtain information on the health issue your program is addressing. See Appendix C, Information Sources, for listings of additional sources of information, including Internet resources.
Identify Existing Activities and Gaps
Find out what other organizations are doing to address the problem, through communication and other approaches, such as advocating for policy or technological changes. Contact these organizations to discuss:
- What they have learned
- What information or advice they may have to help you plan
- What else is needed (what gaps exist in types of change needed, media or activities available, intended audiences served to date, messages and materials directed at different stages of intended audience behavior change)
- Opportunities for cooperative ventures
Gather New Data as Needed
You may find that the data you have gathered does not give enough insight into the health problem, its resolution, or knowledge about those who are affected in order to proceed. In other instances, you may have enough information to define the problem, know who is affected, and identify the steps that can resolve it, but other important information about the affected populations may be unavailable or outdated. To conduct primary research to gather more information, see the Communication Research Methods section.
Sometimes it is impossible to find sufficient information about the problem. This may be because the health problem has not yet been well defined. In this case, you might decide that a communication program is an inappropriate response to that particular problem until more becomes known.
Identify All Components of a Solution
Adequately addressing a health problem often requires a combination of the following approaches:
- Communication (to the general public, patients, health care providers, policymakers—whoever needs to make or facilitate a change)
- Policy change (e.g., new laws, regulations, or operating procedures)
- Technological change (e.g., a new or redesigned product, drug, service, or treatment; or changing delivery of existing products, drugs, services, or treatments)
Yet all too often we rely on health communication alone and set unrealistic expectations for what it can accomplish. It is vitally important to identify all of the components necessary to bring about the desired change and then to carefully consider which of these components is being—or can be—addressed. For example, consider a woman who needs a mammogram. The mammogram graphic shows some of the problems that may occur and potential solutions for each. Solutions that communication programs can help develop are highlighted.
The goal of a communication campaign is not always to teach or to influence behavior; it can also begin the process of changing a policy to increase health and wellness. This might mean getting community leaders excited about a new "rails to trails"project or working to bring up the issue of a lack of low-income housing. In each case, the final goal (i.e., helping people exercise by increasing the number of walking/biking trails, making sure that everyone in the community has a safe place to live by assigning more apartments in newly built housing to low-income residents) is more than a communication campaign can accomplish. However, the initial goal (gaining the support of decision-makers who can change current policy) can be met.
One of the most popular and effective ways to build support for policy change is to work with the media. Use the following questions to help plan your message:
- What is the problem you are highlighting?
- Is there a solution to it? If so, what is it?
- Whose support do you need to gain to make the solution possible?
- What do you need to do or say to get the attention of those who can make the solution happen?
Once you have developed your message, create a media list that includes organizations, such as newspapers and television stations; individuals, such as reporters, editors, and producers; and other contacts. Keep this list updated as you communicate your message and work to change policy. The following are a few methods to use:
- News releases
- Interviews
- Letters to the editor
- Media conferences
Media strategies are not the only way to build support for policy change. Also consider attending and speaking at local meetings, approaching issue decision-makers either in person or by letter, or working with and educating community members who are affected.
Determine Whether Health Communication Is Appropriate for the Problem and Your Organization
Create a map that diagrams the components of a problem and the steps necessary to solve it (as in the mammogram graphic) to help you determine a possible role for health communication. In some cases, health communication alone may accomplish little or nothing without policy, technological, or infrastructure changes (e.g., successfully increasing physical activity of employees in the workplace might require employer policy changes to allow for longer breaks or infrastructure changes such as new walking paths). In some instances, effective solutions may not yet exist for a communication program to support. For example, no treatment may exist for an illness, or a solution may require services that are not yet available. In these cases, decide either to wait until other program elements are in place or to develop communication strategies directed to policymakers instead of consumers or patients.
If you determine that health communication is appropriate, ask the following questions to consider whether your organization is best suited to carry it out:
- Does the organization have (or can it acquire) the necessary expertise and resources?
- Does the organization have the necessary authority or mandate?
- Will the organization be duplicating efforts of others?
- How much time does the organization have to address this issue?
- What, if anything, can be accomplished in that time?
2. Define Communication Objectives
Defining communication objectives will help you set priorities among possible communication activities and determine the message and content you will use for each. Once you have defined and circulated the communication objectives, they serve as a kind of contract or agreement about the purpose of your communication, and they establish what outcomes should be measured.
It is important to create achievable objectives. Many communication efforts are said to fail only because the original objectives were wildly unreasonable. For example, it is generally impossible to achieve a change of 100 percent. If you plan to specify a numerical goal for a particular objective, an epidemiologist or statistician can help you determine recent rates of change related to the issue so that you have some guidance for deciding how much change you think your program can achieve. (Remember that commercial marketers often consider a 2 to 3 percent increase in sales to be a great success.) Fear of failure should not keep you from setting measurable objectives.Without them, there is no way to show your program has succeeded or is even making progress along the way, which could reduce support for the program among your supervisors, funding agencies, and partners.
Because objectives articulate what the communication effort is intended to do, they should be:
- Supportive of the health program's goals
- Reasonable and realistic (achievable)
- Specific to the change desired, the period during which change should take place
- Measurable, to allow you to track progress toward desired results
- Prioritized, to direct the allocation of resources
One can imagine how the process of change occurs: A woman sees some public service announcements (PSAs) and a local TV health reporter's feature telling her about the "symptomless disease"—hypertension. She checks her blood pressure in a newly accessible shopping mall machine, and the results suggest a problem. She tells her spouse, who has also seen the ads, and he encourages her to have it checked. She goes to a physician who confirms the presence of hypertension and encourages her to change her diet and return for monitoring.
The physician has become more sensitive to the issue because of a recent article in the Journal of the American Medical Association, some recommendations from a specialist society, and a conversation with a drug retailer as well as informal conversations with colleagues and exposure to television discussion of the issue. Meanwhile, the patient talks with friends at work or family members about her experience. They also become concerned and go to have their own pressure checked. She returns for another checkup and her pressure is still elevated although she has reduced her salt intake. The physician decides to treat her with medication. The patient is ready to comply because all the sources around her--personal, professional, and media--are telling her that she should.
This program is effective not because of a PSA or a specific program of physician education. It is successful because the National High Blood Pressure Education Program has changed the professional and public environment as a whole around the issue of hypertension.
Be Reasonable
Objectives describe the intermediate steps that must be taken to accomplish broader goals; they describe the desired outcome, but not the steps involved in attaining it (you'll design strategies and tactics for getting there later). Develop reasonable communication objectives by looking at the health program's goal and asking, "What can communication feasibly contribute to attaining this goal, given what we know about the type of changes the intended audiences can and will make?"
Communication efforts alone cannot achieve all objectives. Appropriate purposes for communication include:
- Creating a supportive environment for a change (societal or organizational) by influencing attitudes, beliefs, or policies
- Contributing to a broader behavior change initiative by offering messages that motivate, persuade, or enable behavior change within a specific intended audience
Raising awareness or increasing knowledge among individuals or the organizations that reach them is also feasible; however, do not assume that accomplishing such an objective will lead to behavior change. For example, it is unreasonable to expect communication to cause a sustained change of complex behaviors or compensate for a lack of health care services, products, or resources.
The ability and willingness of the intended audience to make certain changes also affect the reasonableness of various communication objectives. Keep this in mind as you define the intended audiences in planning step 2. Your objectives will be reasonable for a particular intended audience only if audience members both can make a particular behavior change and are willing to do so.
By 2005, the number of women (over age 50; Washington, DC, residents; income under $45,000) who say they get annual screening mammograms will have increased by 25 percent.
By the end of our campaign, more than 50 percent of students at South Salem High School will report having increased the number of servings of fruits and vegetables they eat (on most days) by one.
Be Realistic
Once your program has developed reasonable communication objectives, determine which of them are realistic, given your available resources, by answering these questions:
- Which objectives cover the areas that most need to reach the program goal?
- What communication activities will contribute the most to addressing these needs?
- What resources are available? Include:
- Staff and other human resources—committee members, associates from other programs, volunteers, and others who have the requisite skills and time
- Overhead resources such as computer time, mailing costs, and printing
- Services available from another source, such as educational materials available free or at cost and the effort by other organizations willing to help
- Information about the issue, the intended audience, the community, and media structures, or about available educational materials
- Budget available to fund the program
- Time (weeks, months, or years available to complete the program)
- What supportive factors exist (e.g., community activities, other organizations' interests, positive community attitudes)?
- What barriers exist (e.g., obstacles to approval, absence of funding, sensitivity of an issue, intended audience constraints)?
- Which objectives would best use the resources your program has identified and best fit within the identified constraints?
Your answers to the last question should become your priority objectives. Sometimes you may feel so constrained by a lack of funds that proceeding appears impossible. An honest assessment may lead you to conclude that a productive communication effort is not possible. However, creative use of the resources already identified may enable you to develop a communication program that can make valuable contributions.
Planning Terms
The overall health improvement that an organization or agency strives to create (e.g., more eligible cancer patients will take part in cancer clinical trials, or more Americans will avoid fatal heart attacks). A communication program should be designed to support and contribute to achieving this specific desired improvement.
Communication Objectives
The specific communication outcomes you aim to produce in support of the overall goal (e.g., by 2005, 75 percent of Americans will know that participating in cancer research studies may be an option for them; or by 2005, 50 percent of rural adults over age 40 will know the warning signs for a heart attack and what to do if they occur). Objectives should be attainable, measurable, and time specific.
The overall approaches the program takes. Strategies derive from and contribute to achieving defined goals and objectives. They should be based on knowledge about effective communication, the intended audience's needs and characteristics, and your program'fs capabilities, timelines, and resources. (See planning step 6 for more information on developing a communication strategy and evaluation plan.)
3. Define and Learn About Intended Audiences
In this step, determine whom you want to reach based on decisions made in the previous two steps.
Begin by identifying intended populations for a program based on the epidemiology of the problem (who is most affected? at risk?) and other factors contributing to the problem. Intended populations are often defined very broadly, using just a few descriptors (e.g., women over age 50). Intended audiences are carved from these broad population groups and defined more narrowly based on characteristics such as attitudes, demographics, geographic region, or patterns of behavior. Examples might include physically inactive adolescents, heavy smokers with low education and income levels who are fatalistic about health issues, or urban African-American men with hypertension who live in the South. Because the intended audience's ability and willingness to make a behavior change affects the extent to which communication objectives are reasonable and realistic, it is most efficient to select intended audiences and develop communication objectives (plaanning steps 2 and 3) in tandem.
Goals and Objectives: Healthy People 2010
Healthy People 2010, the Nation's prevention agenda for the next decade, is designed to achieve two overarching goals: 1) increase the quality and years of healthy life, and 2) eliminate health disparities. For the first time, the Health Communication chapter of Healthy People 2010 includes objectives to improve the quality of health communication interventions, the skills of health professionals, the reach and quality of interactive communication media, and the health literacy of people with inadequate or marginal literacy skills. Meeting these communication objectives will contribute to the achievement of the overarching goals. Some communication efforts that could contribute to the achievement of these goals include the following:
- Interventions to improve the communication skills of health care providers and patients
- Assistance for people searching for and using health information
- Education for consumers and patients about important health topics and relevant risks, preventive measures, and ways to access the health care system
See www.health.gov/healthypeople to learn more.
Move From Intended Population to Specific Intended Audiences
Defining subgroups of a population according to common characteristics is called segmentation. Segmentation can help you develop messages, materials, and activities that are relevant to the intended audience's current behavior and specific needs, preferences, beliefs, cultural attitudes, knowledge, and reading habits. It also helps you identify the best channels for reaching each group, because populations also differ in factors such as access to information, the information sources they find reliable, and how they prefer to learn.
Increase your program's effectiveness by developing strategies that are attuned to the needs and wants of different intended audience segments. In fact, given the diversity of the general public, trying to reach everyone with one message or strategy may result in an approach that does not effectively reach those most able or ready to change. Be aware, though, that moving from a mass-market strategy to a differentiated strategy will add economic and staff resource costs for each additional segment. Segment a population into specific intended audiences using the following characteristics to define them:
- Behavioral—health-related activities or choices, degree of readiness to change a behavior, information-seeking behavior, media use, and lifestyle characteristics
- Cultural—language proficiency and language preferences, religion, ethnicity, generational status, family structure, degree of acculturation, and lifestyle factors (e.g., special foods, activities)
- Demographic—occupation, income, educational attainment, family situation, and places of residence and work
- Physical—sex, age, type and degree of exposure to health risks, medical condition, disorders and illnesses, and family health history
- Psychographic—attitudes, outlook on life and health, self-image, opinions, beliefs, values, self-efficacy, life stage, and personality traits
The key to success is to segment the intended population on characteristics relevant to the health behavior to be changed. A logical starting point is the behavior itself: When possible, compare those who engage in the desired behavior with those who do not and identify the determinants of their behavior. Many planners simply rely on demographic, physical, or cultural segmentations. However, people who share these characteristics can be very different in terms of health behavior. For example, consider two 55-year-old African-American women. They work together in the same department. They have the same amount of schooling and comparable household incomes. They live next door to each other, attend the same church, and often invite each other's family over for meals. They enjoy the same television shows, listen to the same radio stations, and often discuss articles that they both read in the paper. Neither has a family history of breast cancer, and both had children before age 30.Yet one woman goes for annual mammograms and the other has never had one. A demographic, physical, or cultural segmentation would group these women together, yet one is a member of the intended audience for health communications about mammography and the other is not.
Databases Help NCI Identify and Communicate With Intended Audience Segments
To help identify and understand its intended audiences, NCI's Office of Communications (OC) uses a unique database that combines health behavior information with geographic, demographic, and lifestyle data. OC uses this information to create Consumer Health Profiles that give a portrait of the intended audience segments most in need of cancer prevention and detection messages. Consumer Health Profiles describe:
- Which populations within a region most need cancer education and outreach and where these populations live, including maps (e.g., which areas of a state have the lowest cancer screening rates)
- How to reach these populations, based on factors such as media habits and knowledge, attitudes, and beliefs about cancer
Consumer Health Profiles are useful not only in locating an intended audience but also in understanding people better. NCI's Cancer Information Service and its partners have used the profiles to plan media buys and direct mailings to increase the number of women participating in low-cost mammography screening programs. For more information, contact CIS's Partnership Program at 1-800-4-CANCER or the Office of Communications at 301-496-6667.
Select Intended Audiences
Once you have identified intended audience segments, begin to set priorities and select the intended audiences (e.g., those segments with whom you will communicate). As you select your intended audiences, distinguish among the audiences your program will address. Primary intended audiences are those you want to affect in some way; you may have one or several primary intended audiences. If you have more than one, set priorities among them to help order your planning and allocate resources. Secondary intended audiences, or gateway audiences, are those with influence on the primary intended audiences or those who must do something to help cause the change in the primary intended audiences. These intended audiences might need different kinds of messages and tools to make the desired change.
Select intended audiences by answering the following questions for each segment:
- What is a reasonable and realistic communication objective for this intended audience? In other words, what behavior change can the intended audience make, and how willing is this group to make that change? Sometimes an intended audience can't make a behavior change—or can't make it easily—until a policy change is instituted or a new or improved product is developed. If your program cannot provide the necessary policy or technological changes, perhaps another intended audience would be a better choice. (See Appendix B for a description of relevant theories and models of behavior change that may help you answer this question.)
- Will achieving that communication objective with this intended audience adequately contribute to attaining the health program goal? (See planning step 2.) intended audience size factors prominently in the answer to this question. It is important to choose a segment or segments large enough that changes in their behavior will make a worthwhile contribution to your program's goal. If your program's goal is population-wide improvement, asking a larger intended audience to make a small change may get you closer to the goal and require fewer resources than helping a small group make a very large change.
- To what extent would members of this segment benefit from the communication? Some segments may already engage in the desired behavior or may be close to it (e.g., eating four servings of fruits and vegetables each day, but not five).
- How well can available resources and channels reach this segment? If you must rely on mass communication (e.g., mass media, public events), yet one-on-one skill modeling is needed to help this segment make a behavior change, your program's resources will be wasted.
- For secondary intended audiences, to what extent does this audience influence the primary intended audiences?
- To what extent will we be able to measure progress? See the Communication Research Methods section for a discussion of measurement considerations.
Answering these questions will also help you determine who will not be members of an intended audience. Ruling out intended audience segments will allow you to make decisions regarding message development and dissemination more easily and will help ensure that all program resources are spent productively. Two examples of intended audiences are 1) teens who smoke, and 2) women over age 50 who are not having regular mammograms.
Learn More About the Intended Audiences
You probably need to know more about the intended audiences than you learned from the initial research. Sometimes planners conduct consumer research on all potential intended audiences to help them set objectives, complete intended audience segmentation, and set priorities. At other times, they define and set priorities among intended audiences based on initial research and then conduct more intensive research with selected intended audiences. The approach often depends upon the amount of existing secondary research and the resources available to conduct primary research.
To learn about an intended audience, find answers to the following questions:
- What does the intended audience already know about the topic? Do intended audience members have any misconceptions?
- What are the intended audience members'relevant attitudes, beliefs, and perceptions of barriers to change?
- How "ready" is the intended audience to change? (Based upon the stages of change model—see Appendix B for a description.)
- What benefit do intended audience members already associate with making the behavior change?
- What social, cultural, and economic factors will affect program development and delivery?
- When and where (times, places, states of mind) can the intended audience best be reached?
- What communication channels (e.g., mass media, organization meetings, Internet sites) reach this intended audience? Which do its members prefer? Find credible? (Look to the census for this information.)
- Do certain individuals (or gatekeepers) either have particular influence with this intended audience or control access to it? What is their degree of influence?
- What are the intended audience's preferences in terms of learning styles, appeals, language, and tone?
See the Communication Research Methods section to learn ways to gather information about intended audiences.
4. Explore Settings, Channels, and Activities to Reach Intended Audiences
In this step, begin to think about the best ways to reach the intended audiences.
To reach intended audiences effectively and efficiently, first identify the settings (times, places, and states of mind) in which they are most receptive to and able to act upon the message. Next, identify the channels through which your program's message can be delivered and the activities that can be used to deliver it. In making these decisions, weigh what will best:
- Reach the intended audience
- Deliver the message
Explore Settings
To identify possible settings for reaching the intended audience, think of the following:
- Places where your program can reach the intended audience (e.g., at home, at school or work, in the car, on the bus or train, at a community event, in the local health care provider's office or clinic)
- Times when intended audience members may be most attentive and open to your program's communication effort
- Places where they can act upon the message
- Places or situations in which they will find the message most credible
Sometimes a given setting may be a good place to reach the intended audience but not a good place to deliver the message. For example, a movie theater slide might be a great way to reach the intended audience, but if the message is "call this number to sign up for this health program," people may not be receptive to (or able to act upon) the message—and they are unlikely to recall the message or the number later, when they can act on it. In contrast, if you reach people while they are preparing dinner--or in the grocery store—with a message to increase fruit and vegetable consumption, they are likely to be receptive to and able to act upon the message.
Explore Channels and Activities
Message delivery channels have changed significantly in the decade since this book first appeared. Today, channels are more numerous, are often more narrowly focused on an intended audience, and represent changes that have occurred in health care delivery, the mass media, and society. Consider the following channels:
- Interpersonal
- Group
- Organizational and community
- Mass media
- Interactive digital media
Interpersonal Channels
Interpersonal channels (e.g., physicians, friends, family members, counselors, parents, clergy, and coaches of the intended audiences) put health messages in a familiar context. These channels are more likely to be trusted and influential than media sources. Developing messages, materials, and links into interpersonal channels may require time; however, these channels are among the most effective, especially for affecting attitudes, skills, and behavior/behavioral intent. Influence through interpersonal contacts may work best when the individual is already familiar with the message, for example, from hearing it through mass media exposure. (Similarly, mass media are most effective at changing behavior when they are supplemented with interpersonal channels.)
Group Channels
Group channels (e.g., brown bag lunches at work, classroom activities, Sunday school discussions, neighborhood gatherings, and club meetings) can help your program more easily reach more of the intended audience, retaining some of the influence of interpersonal channels. Health messages can be designed for groups with specific things in common, such as workplace, school, church, club affiliations, or favorite activities, and these channels add the benefits of group discussion and affirmation of the messages. As with interpersonal channels, working through group channels can require significant levels of effort. Influence through group channels is more effective when groups are already familiar with the message through interpersonal channels or the others described here.
Doctor to Patient: The Interpersonal Channel
Interpersonal channels have shown great success in delivering credible messages that produce desired results. When the one-to-one message comes from the doctor, people are especially likely to listen. Good communication between health care providers and individuals is so important to achieving positive health outcomes that the Health Communication chapter of Healthy People 2010 includes an objective to "increase the proportion of persons who report that their health care providers have satisfactory communication skills" (objective 11-6). In addition, the chapter on cancer includes an objective to "increase the proportion of physicians and dentists who counsel their at-risk patients about tobacco-use cessation, physical activity, and cancer screening" (objective 3-10).
Examples of the results of physician-patient communication include:
- Doctor-patient communication has been associated with improved recovery from surgery, shortened hospital stays, lower blood pressure and blood sugar, and better health status.
- People who quit smoking in response to physician advice are more likely to make repeated attempts to quit and are more likely to remain off cigarettes.
- Women in a national survey said a major reason they never had a mammogram was, "My doctor never recommended one." When a Massachusetts program increased the number of physicians who recommended mammography, screening rates also rose.
- Most people in a national survey said their preferred source of information about prescription medicines is their physician. When patients and physicians communicate, compliance improves.
Organizational and Community Channels
Organizations and community groups, such as advocacy groups, can disseminate materials, include your program's messages in their newsletters and other materials, hold events, and offer instruction related to the message. Their involvement also can lend their credibility to your program's efforts. Organizational/community channels can offer support for action and are two-way, allowing discussion and clarification, enhancing motivation, and reinforcing action.
Mass Media Channels
Mass media channels (e.g., radio, network and cable television, magazines, direct mail, billboards, transit cards, newspapers) offer many opportunities for message dissemination, including mentions in news programs, entertainment programming ("entertainment education"), public affairs programs, "magazine" and talk shows (including radio audience call-ins), live remote broadcasts, editorials (television, radio, newspapers, magazines), health and political columns in newspapers and magazines, posters, brochures, advertising, and public service campaigns.You may decide to use a variety of formats and media channels, always choosing from among those most likely to reach the intended audiences.
Mass media campaigns are a tried-and-true communication approach. They have been conducted on topics ranging from general health to specific diseases, from prevention to treatment. Overall, research has demonstrated the effectiveness of mass media approaches in:
- Raising awareness
- Stimulating the intended audience to seek information and services
- Increasing knowledge
- Changing attitudes and even achieving some change (usually) in self-reported behavioral intentions and behaviors
However, behavior change is usually associated with long-term, multiple−intervention campaigns rather than with one-time communication−only programs.
Interactive Digital Media Channels
Interactive digital media channels (e.g., Internet Web sites, bulletin boards, newsgroups, chat rooms, CD-ROMs, kiosks) are an evolving phenomenon and are useful channels that should have even greater reach in the future. These media allow communicators to deliver highly tailored messages to and receive feedback from the intended audience. These channels are capable of producing both mass communication and interpersonal interaction. Use these media to:
- Send individual messages via electronic mail
- Post program messages (such as information about health-related campaigns) on Internet sites that large numbers of computer users access
- Create and display advertisements
- Survey and gather information from computer users
- Engage intended audiences in personalized, interactive activities
- Exchange ideas with peers and partners
Using interactive digital media is not without challenges. If you choose to do so, consider credibility and access issues.
Internet and Multimedia Channels
CD-ROMs—Computer disks that can contain an enormous amount of information, including sound and video clips and interactive devices.
Chat rooms—Places on the Internet where users hold live typed conversations. The "chats" typically involve a general topic. To begin chatting, users need chat software, most of which can be downloaded from the Internet for free.
Electronic mail (e-mail)—A technology that allows users to send and receive messages to one or more individuals on a computer via the Internet.
Interactive television—Technologies that allow television viewers to access new dimensions of information (e.g., link to Web sites, order materials, view additional background information, play interactive games) through their television during related TV programming.
Intranets—Electronic information sources with limited access (e.g., Web sites available only to members of an organization or employees of a company). Intranets can be used to send an online newsletter with instant distribution or provide instant messages or links to sources of information within an organization.
Kiosks—Displays containing a computer programmed with related information. Users can follow simple instructions to access personally tailored information of interest and, in some cases, print out what they find. A relatively common health application is placing kiosks in pharmacies to provide information about medicines.
Mailing lists (listservs)—E-mail−based discussions on a specific topic. All the subscribers to a list can elect to receive a copy of every message sent to the list, or they may receive a regular "digest" disseminated via e-mail.
Newsgroups—Collections of e-mail messages on related topics. The major difference between newsgroups and listservs is that the newsgroup host does not disseminate all the messages the host sends or receives to all subscribers. In addition, subscribers need special software to read the messages. Many Web browsers, such as Internet Explorer, contain this software. Some newsgroups are regulated (the messages are screened for appropriateness to the topic before they are posted).
Web sites—Documents on the World Wide Web that provide information from an organization (or individual) and provide links to other sources of Internet information. Web sites give users access to text, graphics, sound, video, and databases. A Web site can consist of one Web page or thousands of Web pages. Many health-related organizations have their own Web sites.
Credibility. Anyone can put information on the Internet, and it may or may not be accurate. Thus it is important to demonstrate the credibility of your organization when you use this channel to disseminate health information. This will help ensure that users trust the information they receive.
To improve the quality of health information on the Internet, Healthy People 2010 includes an objective to increase the proportion of health-related Web sites that disclose information that can be used to assess the site's quality (objective 11-4). To improve quality, health Web sites should disclose the following information:
- The identity of the developers and sponsors of the site, how to contact them, and information about any potential conflicts of interest or biases
- The explicit purpose of the site, including any commercial purposes and advertising
- The original sources of the content on the site
- How the privacy and confidentiality of any personal information collected from users is protected
- How the site is evaluated
- How content is updated
Access. The average computer user is affluent and well educated. Although access to this medium is increasing, it is definitely not universal; television and radio are better choices to reach a larger intended audience. The U.S. Department of Commerce issues reports on the "digital divide," the gap between those with access to computers and the Internet and those without. Healthy People 2010 includes an objective to increase from 26 to 80 the percentage of households with access to the Internet so that individuals will be able to get the information and services they need to address their health concerns (objective 11-1).
Type of Channel | Activities | Pros | Cons |
Interpersonal Channels |
Organizational and Community Channels |
Mass Media Channels Newspapers |
Radio |
Television |
Internet |
Weigh Pros and Cons
As illustrated in the table Communication Channels and Activities: Pros and Cons, each type of channel—and activity used within that channel—has benefits and drawbacks.Weigh the pros and cons by considering the following factors:
- Intended audiences you want to reach:
- Will the channel and activity reach and influence the intended audiences (e.g., individuals, informal social groups, organizations, society)?
- Are the channel and activity acceptable to and trusted by the intended audiences, and can they influence attitudes?
- Your message:
- Is the channel appropriate for conveying information at the desired level of simplicity or complexity?
- If skills need to be modeled, can the channel model and demonstrate specific behaviors?
- Channel reach:
- How many people will be exposed to the message?
- Can the channel meet intended audience interaction needs?
- Can the channel allow the intended audience to control the pace of information delivery?
- Cost and accessibility:
- Does your program have the resources to use the channel and the activity?
- Activities and materials:
- Is the channel appropriate for the activity or material you plan to produce? (Decisions about activities and channels are interrelated and should be made in tandem. See Stage 2 for a list of possible materials for health programs and a discussion of decision factors.)
- Will the channel and activity reinforce messages and activities you plan through other routes to increase overall exposure among the intended audiences?
Best Choice: Using Multiple Channels to Reach Intended Audiences
Using several different channels increases the likelihood of reaching more of the intended audiences. It also can increase repetition of the message, improving the chance that intended audiences will be exposed to it often enough to absorb and act upon it. For these reasons, a combination of channels has been found most effective in producing desired results, including behavior change.
For example, Center for Substance Abuse Prevention (CSAP) communication grantees have combined channels in unique ways that reflect their communities. One grantee used posters in community facilities, placed radio spots, and distributed brochures through community sites and requests by radio listeners. Another used a satellite network to show videos, made small group presentations through organizations, and worked with schools to promote at-home activities. Yet another promoted its message through a music and visual arts training program that resulted in a live performance and television broadcast of the program's art and musical creations.
5. Identify Potential Partners
Working with other organizations can be a cost-effective way to enhance the credibility and reach of your program's messages. Many public health institutions seek out partner organizations to reach particular intended audiences.
The benefits to your program of forming partnerships can include:
- Access to an intended audience
- More credibility for your message or program because the intended audiences consider the potential partner to be a trusted source
- An increase in the number of messages your program will be able to share with intended audiences
- Additional resources, either tangible or intangible (e.g., volunteers)
- Added expertise (e.g., training capabilities)
- Expanded support for your organization's priority activities
- Cosponsorship of events and activities
You may partner with one or a few organizations to accomplish specific projects. Some communication initiatives may call for tapping into or assembling a coalition of organizations with a shared goal. In some cases, you may need to assemble many organizations that support particular recommendations or policies. At other times, you may want the organizations to play an active role in developing and implementing communication activities.
To encourage selected groups to partner with your organization, consider the benefits you can offer, such as:
- Added credibility
- Access to your organization's data
- Assurance of message accuracy
- Liaison with other partners
Decide Whether You Want Partners
Although working with partners can be essential to achieving communication objectives, there are also drawbacks that you should recognize and prepare to address.Working with other organizations can:
- Be time consuming—Identifying potential partners, persuading them to work with your program, gaining internal approvals, and coordinating planning and training all take time.
- Require altering the program—Every organization has different priorities and perspectives, and partners may want to make minor or major program changes to accommodate their own structure or needs.
- Result in loss of ownership and control of the program—Other organizations may change the time schedule, functions, or even the messages, and take credit for the program.
Decide how much flexibility you would be willing to allow a partner in the program without violating the integrity of your program, its direction, and your own agency's procedures. If you decide to partner with other organizations, consider which:
- Would best reach the intended audiences
- Might have the greatest influence and credibility with the intended audiences
- Will be easiest to persuade to work with you (e.g., organizations in which you know a contact person)
- Would require less support from you (e.g., fewer resources)
Develop Partnering Plans
Think about the roles potential partners might play in your program and use the suggestions below to help identify specific roles for partners:
- Supplemental printing, promotion, and distribution of materials
- Sponsorship of publicity and promotion
- Purchase of advertising space/time
- Creation of advertising about your organization's priority themes and messages
- Underwriting of communication materials or program development with your organization
See Appendix A for a partnership plan form.
Working With For-Profit Partners
The National Cancer Institute uses these guidelines when considering commercial partners.
- The National Cancer Institute will not consider any collaboration that endorses a specific commercial product, service, or enterprise.
- The National Cancer Institute name and logo may be used only in conjunction with approved projects and only with the written permission of NCI. NCI retains the right to review all copy (e.g., advertising, publicity, or for any other intended use) prior to approval of the use of the NCI name and logo.
- The National Cancer Institute will formally review each proposal for partnership.
- No company will have an exclusive right to use the NCI name and logo, messages, or materials.
- Confidentiality cannot be guaranteed for any collaboration with a federal program.
Criteria for Reviewing Corporations Prior to Partnership Negotiations
- Company is not directly owned by a tobacco company and is not involved in producing, marketing, or promoting tobacco products.
- Company does not have any products, services, or promotional messages that conflict with NCI policies or programs (e.g., the company does not market known carcinogens or market some other product that NCI would not consider medically or scientifically acceptable).
- Company is not currently in negotiation for a grant or contract with NCI.
- Company does not have any unresolved conflicts or disputes with NCI or NIH.
- Establishing a partnership with this company will not create tensions/conflicts with another NCI partner or federal program.
- Company or institution satisfactorily conforms with standards of health or medical care.
- There is evidence that the company would be interested in becoming a partner with NCI.
Working With Partners
The staff person responsible for working with partners should be:
- A good manager who is able to balance all program components
- A team player who is able to work with other organizations
- Diplomatic and willing to negotiate
- Willing to share credit for success
Developing and Maintaining Coalitions
Community coalitions have become an important force in health promotion. Coalitions have all of the advantages of partnerships plus another benefit. Because they harness the resources and commitment of multiple organizations, the attention those organizations pay to an issue is institutionalized for long-term action. The strongest potential partners may be interested in joining coalitions.
Coalitions often grow from informal partnerships or advisory bodies created around special projects. Experience in working together lays the groundwork for a long-term association.
Use the following guidelines to create a successful coalition:
- Formalize the relationship to create greater commitment. Formal arrangements include written memoranda of understanding, by-laws, mission statements, or regular reminders of the coalition's purpose and progress.
- Make sure that the responsibilities of each organization and its staff are clear. In particular, staff members need to know whether to take direction from the coalition chairperson or from the agency that pays their salary.
- Structure aspects of the coalition's operation. Elect officers. Form standing committees. Have regularly scheduled meetings with written agenda and minutes. Expect and support action, not just discussion, at these meetings. Circulate action items resulting from meetings among coalition members. Establish communication channels and use them frequently.
- Ensure the involvement of representatives who show leadership characteristics, such as the ability to obtain resources, problem−solve, and promote collaboration and equality among members. Members with political knowledge, administrative or communication skills, or access to the media and decision-makers are also valuable.
- Create and reinforce positive expectations by providing information on the coalition's progress. Optimism and success sustain member interest.
- Formalize accountability and develop criteria for judging whether coalition members are honoring their commitments.
- Be flexible. Losing prospective partners can limit a program's effectiveness.
- Provide training to help members complete their tasks. For example, coalition members may need training in how to be effective advocates for your program's issues.
- Give members a stake in the coalition and an active role in decision-making.
- Seek external resources to augment member resources.
- Evaluate the effectiveness of the coalition periodically and make necessary changes. This should include process evaluation of the coalition's functioning and assessment of the coalition's impact on the health problem being addressed.
Steps for Involving Partners in the Program
- Choose organizations, agencies, or individuals (e.g., physicians) that can bring the resources, expertise, or credibility your program needs.
- Consider which roles partners might play to best support the program.
- Involve representatives of the organizations you want to work with as early as appropriate in program planning.
- Give partners the program rationale, strategies, and messages (in ready-to-use form). Remember that strategic planning, creative messages, and quality production are the most difficult aspects of a communication program to develop and may be the most valuable product you can offer to a community organization.
- Give partners advance notice so that they can build their part of the program into their schedule, and negotiate what will be expected of them.
- Allow partners to personalize and adapt program materials to fit their circumstances and give them a feeling of ownership, but don't let them stray from the strategy.
- Ask partners what they need to implement their part of the program. Beyond the question of funding, consider other assistance, training, information, or tools that would enable them to function successfully.
- Provide partners with new local/regional/national contacts or linkages that they will perceive as valuable for their ongoing activities.
- Give partners an appropriate amount of work. Give them a series of small, tangible, short-term responsibilities, as well as a feedback/tracking mechanism.
- Gently remind partners that they are responsible for their activities; help them complete tasks, but don't complete tasks for them.
- Assess progress through the feedback/tracking mechanism and help make adjustments to respond to the organization's needs and to keep the program on track.
- Provide moral support by frequently saying "thank you" and by providing other rewards (e.g., letters or certificates of appreciation).
- Give partners a final report of what was accomplished and meet to discuss follow-up activities and resources they might find useful. Make sure that they feel they are a part of the program's success.
- Share one final, tremendous "Thank you for a job well done."
6. Develop a Communication Strategy; Draft Communication and Evaluation Plans
At this point your program has:
- Defined intended audiences and the actions you want their members to take (communication objectives)
- Explored the settings, channels, and activities that can be used to reach them
- Identified potential partners
- Developed partnering plans
In this step, you will use this information as the basis for developing a communication strategy and drafting communication and evaluation plans.
Develop a Communication Strategy Statement (Creative Brief)
In this context, a strategy is a communication approach your program plans to take with a specific intended audience; while you may develop many different communication materials and use a variety of activities, the strategies are guiding principles for all program products and activities. A communication strategy includes everything you need to know to communicate with the intended audience. It defines the intended audience, describes the action its members should take, tells how they will benefit (from their perspective, not necessarily from a public health perspective), and how you can reach them. A communication strategy is:
- Based on knowledge of the intended audience's wants, needs, values, and accessibility
- Guided by general communication research as well as theories and models of behavior
- Tempered by the realities of available resources and deadlines
Developing the strategy statement provides a good test of whether your program has enough information to begin developing messages. It also gives you an opportunity to obtain management and partner buy-in for the approach.You may be tempted to skip this step, but do not. Having an approved strategy statement will save time and effort later. The statement provides both a foundation and boundaries for all the materials you produce and all the activities you conduct.
The communication strategy statement is sometimes called a creative brief because it is used to brief the creative team. In addition, sharing the strategy statement with management and partners allows you to make sure there is support for your program's approach before resources are expended and makes easier the approvals and cooperation you may need later.
For each of the intended audiences, write a creative brief (see Appendix A for a template to use) that includes the following:
- A definition and description of the intended audience (intended-audience profile). Think of one person in the intended audience and describe him or her, rather than describing the group. The information you gathered in planning step 3 should provide the basis for this section.
- A description of the action the intended audience members should take as a result of exposure to the communication. The action is the change the communication objective specifies (planning step 2). If you haven't already done so, now is the time to find out if intended audience members are willing and able to take the action—and to identify the current behavior that you want to change. Knowing what an intended audience currently does—and why it does it—will provide important insights into the behavior change process and can be used to develop communications that demonstrate replacing the old behavior with the new one.
- A list of any obstacles to taking action. Common obstacles include intended audience beliefs, social norms, time or peer pressures, costs, ingrained habits, misinformation, and product inaccessibility. The "map" you created in planning step 1 should identify many of the obstacles, particularly those related to product inaccessibility (e.g., a woman can't get to a mammography location, a worker has no access to fruits or vegetables at break times, a condom isn't available at the time of intercourse). The additional information you gathered about the intended audience in planning step 3 should also help you identify obstacles.
- The consumer-perceived benefit of taking the action. Many theories and models of behavior change include the idea that people change their behavior because they expect to receive some benefit (e.g., gain in time, money, enjoyment, potential gain in stature among peers) that outweighs the personal cost of the behavior change. Short-term, high-probability personal benefits generally are more effective than long-term population benefits (e.g., "stop smoking to smell better and be more attractive" rather than "stop smoking to reduce your risk of developing lung cancer").
- A description of the support that will make the benefit, and its ability to attain it, credible to the intended audience. Support can be provided through hard data, peer testimonials about success or satisfaction, demonstrations of how to perform the action, or statements from organizations the intended audience finds credible. Tailor the particular supports you use to the concerns intended audience members have about the action. For example, if they are worried they can't do it, a demonstration may be warranted; if they question why they should take the action or whether it will have the promised health benefit, hard data or statements from credible organizations may be in order; if they don't believe they need to take the action (e.g., they deny being in the intended audience), a peer testimonial can be compelling.
- The settings, channels, and activities that will reach intended audience members—particularly when they will be receptive to or able to act upon the message. This information should come from the work you did in planning step 4.
- The image your program plans to convey through the tone, look, and feel of messages and materials. The goal should be to convey an image that 1) convinces intended audience members that the communication is for them, and 2) is culturally appropriate. Image is conveyed largely through executional details. Printed materials convey image through typeface, layout, visuals, color, language, and paper stock used.Web materials convey image through design, typeface, color, layout, and ease of use. Audio materials convey image through voices, language, and music; in addition to these details, video materials convey image through visuals, characteristics of the actors (including their clothing and accessories), camera angles, and editing.Work with the creative team to develop the image you select.
Sample Strategy Statement
Draft Creative Brief Used for NCI 5 A Day for Better Health Program Concept Development and Testing (June 2001)
Intended Audiences—African-American and Latino adults (men and women) with primary responsibility for shopping and food preparation who have children under the age of 13 and have household incomes of $25,000 to $50,000 who believe they and their families should eat more fruits and vegetables.
Objective(s)—1) to believe that increasing their fruit and vegetable (f/v) intake is possible (provides a "can do" self-efficacy element), and 2) to increase their f/v intake (gets at the behavioral element, which is the ultimate goal; provides the "do it" element).
- Low salience/competition with everyday concerns and priorities
- Storage
- Low self-efficacy
- Children's reaction/sensory
- Cost, convenience, freshness (cost, both out of pocket and perishability, is a top barrier among African Americans)
- Not filling and don't taste good (Latino issues)
- Safety (safety/pesticide issues are top barriers for eating vegetables for Latinos)
- Lack of planning time
- Preparation time (preparation time is a "top" barrier for eating vegetables among African Americans)
- Lack of familiarity with certain fruits and vegetables
- Competition with other food products; i.e., fast foods
- Nutritional concerns about frozen/canned; e.g., canned are high in sodium; and for Latinos, "not as healthy" misconception
- Lack of confidence in ability to get fruits and vegetables outside the home (survey data, Latinos)
Key Promise
- If we eat more fruits and vegetables every day, I'll feel good knowing we're setting a good example for our children and investing in a healthy future.
- If my family members eat more fruits and vegetables, they will function at their best and protect their health as well as feel more energetic, help control their weight, and cleanse their system.
- If we eat more fruits and vegetables every day, my family will stay healthy while reducing its risk of cancer or heart disease in the future.
- If we have plenty of fruits and vegetables available for our family, we will be setting a good example and teaching our children good, lifelong eating habits.
Support Statements
- Fruit and vegetable consumption helps people feel good, look good, and maintain their weight.
- Fruit and vegetable consumption provides generous amounts of fiber and promotes digestive health.
- Obesity and childhood diabetes are epidemic.
- Fruit and vegetable consumption reduces the risk of heart disease/cancer.
Tone—Urgency without fear; positive...
- TV PSAs
- Radio PSAs/Live announcer scripts
- Posters
- Newspaper ads
- Billboard and Metro Transit ads
- Earned Media ("Do Yourself a Flavor"')—African-American/Latino women can send Graham Kerr recipes that he can select and feature
- Web site
- Articles in women's magazines
Other Channels/Intermediaries
- Produce for Better Health
- State coordinators
- Traveling home from work
- Mother's Day
- Community outreach (health fairs, in-store events such as taste tests or demonstrations), grassroots programs
Creative Considerations
- Adaptable to local needs (state health profiles)
- Focus on more than one fruit or vegetable (equal opportunity)
- Appetite appeal, culturally appropriate, cross-cultural concepts that can be tailored in execution
- Focus groups that include men and women and low-income ($25,000 or less in San Antonio) participants at each site, which may drive creative to accommodate low literacy; this is a good thing if we have both English- and Spanish-speaking people in the intended audience. (Note: Creative executions should be intended audience specific, while the concepts will cross over and work for both ethnic groups.)
The information in Appendix B and the information you learned about your intended audience in planning step 3 are the foundation for strategy development. Use this information to prepare a document similar to the NCI sample strategy statement provided here. At first, you may have question marks next to some items, or lists of possibilities for actions, benefits, support, or image.You can fill in the answers, narrow down the list, and get overall reactions to the strategy by conducting research with the intended audience. (See the Communication Research Methods section for suggestions on how to obtain this input.) Developing the communication strategy is usually an iterative process; as you learn more about one element, other elements will likely need to be adjusted.
The communication strategy provides all program staff—including writers, creative staff, and evaluators—with the same direction for developing all messages and materials. In a cooperative program with partner organizations, the strategy statement can also help all players communicate consistent themes and take similar action. Some organizations choose to produce report-length strategy statements that contain additional information, such as background on the health problem being addressed, extensive intended audience profiles, and situation analyses.
Once your program has decided on a communication strategy, all program elements should be compatible with it. This means every program task should contribute to reaching the established objectives and be designed to reach the identified intended audiences; all messages and materials should incorporate the benefits and other information from the strategy statement.
As you learn more about the intended audiences and their perceptions, you may need to alter or refine the strategy statement. However, it should be changed only to reflect information that will strengthen your program's ability to reach the communication objectives. Do not alter your strategy simply to accommodate a great idea that doesn't match the objectives.
Draft Communication Plan
All of the elements of your planning should be recorded in a communication plan that will become your "blueprint." It should be used to:
- Explain the plans within your agency and with others
- Support and justify budget requests
- Provide a record of where your program began
- Show the program's planned evolution over time
Include the following sections in the plan:
- Communication strategy
- Partnering plans
- Message and materials development and testing plans
- Implementation plans, including plans for distribution, promotion, and process evaluation
- Outcome evaluation plans
- Tasks and timeline
A template for a communication plan that includes all of these sections is in Appendix A. During Stage 1, prepare initial drafts of all plan sections except distribution and promotion. Realize that some of the sections, such as implementation and process evaluation (see Stage 3), may not be as detailed as others at this point.
Draft Outcome Evaluation Plans
Outcome evaluation is used to assess the degree to which the communication objectives are achieved. Conducting useful outcome evaluation can be challenging because of the following constraints:
- Many standard evaluation approaches assume a direct cause-and-effect relationship between the stimulus (your program's communication) and the intended audience's response to it. However, it can be impossible to isolate the effects of a particular communication activity, or even the effect of a communication program on a specific intended audience, because change does not often occur as a result of just one specific activity.
- Communication programs generally occur in a real-world setting, where there are many other influences on the intended audiences. Other activities (and often other organizations) may be addressing the same problem. Attributing change to program activities may be very difficult.
- Communication objectives can be reasonable but not measurable because of reasons such as:
- The change is too small for available methodologies to detect (e.g., a 2 percent increase over the course of a year in the number of women age 50 or over who get a mammogram would have important public health benefits, but would not be detectable by a survey with a typical 3.1 percent margin of error).
- The change is difficult to measure validly or reliably (e.g., self-reports of behavior are often unreliable).
Plan Outcome Evaluation Activities
Before you begin to plan for outcome evaluation, review Stage 4 for descriptions of common methodologies. As you plan, keep the following tips in mind:
- Ensure that the evaluation design is appropriate for the particular communication activity. Experimental designs, in which a treatment group (people exposed to the communication) is compared to a control group (people not exposed to the communication), are the gold standard of outcome evaluation. However, they often cannot be used to assess communication activities, largely because untreated control groups may not exist, particularly for national-, state-, or community-based efforts. Even if people are not exposed to your program's communication, they are likely to be exposed to some communication on the same topic. In these situations, appropriate designs include comparisons between cross-sectional studies (such as independent surveys taken at different points in time), panel studies (the same people are interviewed or observed multiple times), and time series analyses (comparisons between projections of what would have happened without the intervention versus what did happen). However, each is appropriate in different situations; seek the advice of an evaluation expert before selecting a design.
- Consider how the communication activity is expected to work and the time period in which it is expected to work. Then make sure it is evaluated in accordance with those expectations. For example, if you expect people to need at least five to eight exposures to the message before they will take action, make sure that you allow sufficient implementation time to achieve the intended level of exposure. If you expect people to take action immediately after exposure, then the outcome measurement should take place soon after that. Conversely, if you don't expect to see effects for at least a year, outcomes shouldn't be measured until then. Communication programs are often deemed "failures" because they don't reach people with sufficient repetition to work—either because they are inadequately funded or because everything runs late and they are not in place long enough before outcomes are measured. (Use process evaluation to track the level of intensity and the duration of message exposure to learn why expected outcomes did or did not occur.)
- Consider what level of evidence is acceptable for your outcome evaluation purpose (e.g., to report back to management or funding agencies).
- Consider what baseline measures you have available or can collect and how to track changes related to desired outcomes (e.g., how, and how often, data will be collected).
- Ensure that you measure change against the communication objectives and not against your program's goal. For example, if the communication objective is to increase the percentage of women age 50 or older who ask their doctor about a mammogram, you would measure how many women asked their doctor about a mammogram, not how many women got a mammogram.
- Ensure that progress toward outcomes is captured. For example, if you expect people to think about changing a behavior, and perhaps try changing it a few times before making and sustaining the change, make sure the evaluation can capture these intermediate outcomes. If the objective is to increase the percent of people engaging in moderate exercise on most days of the week, it would be important to determine 1) people's current behavior, and 2) whether they have thought about increasing their amount of activity, taken steps to increase it, or increased it some weeks but not consistently.
Appendix A contains an outcome evaluation form. Although you may not be ready to complete final evaluation planning now, it is important to put together a general plan so that your program can collect any necessary baseline data before implementation begins, build any needed evaluation mechanisms into the program, and ensure that evaluation resources are allocated. To get started, do the following:
- Read Stage 4 (Assessing Effectiveness and Making Refinements) and look at the table at the beginning of the Communication Research Methods chapter.What you learn about evaluation may affect what you choose to do with the program.
- Take another look at these sections after your initial communication plan is complete to be sure the evaluation activities will be appropriate and valuable.
- Involve an evaluation expert familiar with evaluating communication programs during initial planning. His or her advice can help prevent time-consuming fixes later by ensuring you develop a program that can be validly evaluated (e.g., making sure data collection mechanisms are in place, making sure baseline data are collected for comparison later).
Create a Timetable
Finally, produce a time schedule for development, implementation, and evaluation. The schedule should include every task you can think of from the time you write the plan until the time you intend to complete the program. The more tasks you build into the timetable now, the more likel you will remember to assign the work and keep on schedule. Also, detailing the tasks will make it easier to decide what resources will be required. If you forget important intermediate steps, your program's costs and schedule might change.
The timetable should be considered a flexible management tool. Review and update it regularly (e.g., once a month) so that it can function dually to manage and track progress. Many managers believe computer-based tools are especially useful for this task. Project management computer software contains schedule forms that you can fill in and monitor on the computer and print out for staff and others involved.
Common Myths and Misconceptions About Planning
Myth: Our program can't afford to conduct intended audience research.
Fact: Your program can't afford not to conduct intended audience research. Without it, you do not know for sure whom to select, where to reach them, what to ask them to do, or how to ask it. The information you need to develop effective communication may be relatively inexpensive or free. Resources include literature searches, information available free from government health agencies, and advisory groups and representatives of the intended audience. For example, the National Cancer Institute's 5 A Day for Better Health Program media campaign used primary research to identify its intended audiences, actions they would be willing to take, and benefits they would find compelling, but used existing marketing databases to obtain a great deal of information about their lifestyles, interests, outlook, and media habits. The cost for analyzing the databases was substantially less than it would have been to conduct and analyze additional primary research.
Myth: Market research isn't relevant for a health program.
Fact: Health program planners can use the methodologies and types of information normally associated with market research in many ways, including:
- Understanding why individuals would choose or not choose to undertake a new behavior, other preferences regarding the behavior, how communicators should talk about the behavior (tone and language), and where individuals seek out or receive information
- Creating a multidimensional portrait of the intended audience for communication planning; knowing only health-related factors limits understanding of the whole person your program is trying to influence and does not provide guidance on when and where to reach the person or what to say to persuade the intended audience
- Supporting strategy development for policy initiatives by helping to describe opinion leaders, policymakers, and their constituents
Myth: We don't have time for planning. Our boss (or funding agency or partner organization) wants us to get started right away.
Fact: Making health communication programs work requires planning, but planning need not be a long-term, time-consuming activity. Nor should all the activities suggested in this section be conducted en masse, before any other actions are taken. Planning is easiest and best done bit by bit—related to and just in time for the programmatic tasks it governs. For example, you need certain kinds of information about the intended audiences in order to define them, select them, and set objectives.You need different information to guide message development; gather each type as you need it.
One person should not be doing all planning tasks. Divide responsibilities for individual tasks such as managing market research or drafting the strategic plan and have the whole planning team (or manager) reconcile and revise to create the final plan.
Myth: If we work with partner organizations that represent the intended audience, we'll have access to all the channels we need.
Fact: Channels you access through a partner organization may be very useful, but they may miss intended audience segments the organization does not represent, and they may not be the most credible or effective way to influence the intended audience. Using additional channels will help reinforce your program's messages and enhance the likelihood that the intended audience will recall them.
Myth: It's best to use the channels we're comfortable with and have used before.
Fact: Selecting the right channels is as important to success as developing effective materials or having a sound strategy. If the intended audience never sees/hears the message, doesn't believe it because it comes from an unrespected source, or doesn't attend to it because it comes from a noninfluential source, the time and money your program spends developing the message will be wasted. While you may well make good use of channels where you have previous contacts, determine whether these channels alone will reach and influence intended audiences before relying solely on them.
Myth: If we use only one channel, we should use mass media.
Fact: In the past, many programs may have concentrated on mass media, particularly public service announcements. Today, however, many other effective channels exist and relying on mass media alone may not achieve some communication program goals. Although it may take time, effort, and possibly outside expertise to learn about and use new channels, the potential rewards make this a good investment in your program's future and in your organization's long-term skill/knowledge base.
Myth: Using interactive digital media requires major technical capabilities we don't have, and we can't keep up.
Fact: Using interactive digital media effectively does require professional expertise in product design--just as professional expertise is needed to create other types of communication vehicles and develop effective evaluations. Consultants from other branches of your organization, universities, a volunteer pool, or commercial firms can help. Some advertising/creative firms are beginning to develop expertise in these media, or you can use experts to advise you on the electronic end while you take care of the communication aspects.
These media are changing rapidly, just as the computer field as a whole. However, if you determine that interactive digital channels will be very effective in reaching the intended audience, networking with peers, and conducting program research, the investment may pay off.
Myth: If you don't have interactive digital media in your program, you are missing out on today's hottest communication opportunity and will look very out of date and low tech to your peers.
Fact: Remember, interactive digital media are just other channels. The same steps discussed in this guide still apply, and good communication principles and skills are still paramount. The key selection factor is how well these media will reach the intended audience and how suited they are to carrying the message. The intended audience may not have sufficient access to computers or have the skills or attitudes/interest to seek information through them. And, despite their potential, much remains to be learned about their best uses and how intended audiences respond to and interact with them.
Selected Readings
American Cancer Society. (2001). Cancer facts and figures. Atlanta.
American Public Health Association. (2000). APHA media advocacy manual 2000. Washington, DC.
Andreasen, A. (1988). Cheap but good marketing research. Homewood, IL: Dow Jones-Irvin.
Andreasen, A. (1995). Marketing social change: Changing behavior to promote health, social development, and the environment. San Francisco: Jossey-Bass.
Calvert, P. (Ed.). (1996). The communicator's handbook: Tools, techniques, and technology (3rd ed.). Gainesville, FL: Maupin House.
Center for Substance Abuse Prevention. (1994). Following specific guidelines will help you assess cultural competence in program design, application, and management [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1998). Evaluating the results of communication programs [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention Communications Cooperative Agreements. (1996). Bridging the gap for people with disabilities. Rockville, MD: U.S. Department of Health and Human Services.
Centers for Disease Control and Prevention. (1996). The prevention marketing initiative: Applying prevention marketing (CDC Publication No. D905). Atlanta.
Centers for Disease Control and Prevention. (2000). Beyond the brochure (CDC Publication No. PDF-821K). Atlanta.
Davis, J. (Ed.). (2001). Health and medicine on the Internet: An annual guide to the World Wide Web for health care professionals. Los Angeles: Practice Management Information Corporation.
The Dialog Corporation. (2001). Packaged facts. Available: http://library.dialog.com/sourcebook/researchline/pf.html.
Eng, T. R., & Gustafson, D. H. (Eds.). (1999). Wired for health and well-being: The emergence of interactive health communication. Washington, DC: U.S. Department of Health and Human Services, U.S. Government Printing Office.
Glanz, K., Lewis, F. M., & Rimer, B. K. (Eds.). (1997). Health behavior and health education: Theory, research, and practice (2nd ed.). San Francisco: Jossey-Bass.
Glanz, K., & Rimer, B. K. (1995). Theory at a glance: A guide for health promotion practice (NIH Publication No. 97-3896). Bethesda, MD: National Cancer Institute.
Goldberg, M. E., Fishbein, M. F., & Middlestadt, S. E. (Eds.). (1997). Social marketing: Theoretical and practical perspectives. Mahwah, NJ: Erlbaum.
Jernigan, D. H., & Wright, P. A. (1996). Media advocacy: Lessons from community experiences. Journal of Public Health Policy, 17, 306–330.
Maibach, E., Maxfield, A., Ladin, K., & Slater, M. (1996). Translating health psychology into effective health communication. Journal of Health Psychology, 1, 261–277.
Matiella, A. C. (Ed.). (1990). Getting the word out: A practical guide to AIDS materials development. Santa Cruz, CA: Network Publications.
Miller, J., & Pifer, L. K. (1993). Public understanding of biomedical science in the United States, 1993: A report to the National Institutes of Health. Chicago: Chicago Academy of Science.
National Cancer Institute. (1996). Cancer rates and risks (4th ed.; NIH Publication No. 96-691). Bethesda, MD: Surveillance, Epidemiology, and End Results (SEER) Program.
Schooler, C., Chaffee, S. H., Flora, J. A., & Roser, C. (1998). Health campaign channels: Tradeoffs among reach, specificity, and impact. Human Communication Research, 24, 410–432.
Slater, M. D. (1996). Theory and method in health audience segmentation. Journal of Health Communication, 1, 267–283.
U.S. Department of Health and Human Services. (2000). Healthy people 2010 (2nd ed.; in two volumes: Understanding and improving health and Objectives for improving health).Washington, DC: U.S. Government Printing Office.
University of Toronto. (1999). Overview of health communication campaigns. Toronto, Canada: Health Communication Unit, Centre for Health Promotion, University of Toronto.
Wallach, L., & Dorfman, L. (1996). Media advocacy: A strategy for advancing policy and promoting health. Health Education Quarterly, 23(3), 293–317.
Wallach, L., Dorfman, L., Jernigan, D., & Themba, M. (1993). Media advocacy and public health: Power of prevention. Thousand Oaks, CA: Sage.
Stage 2: Developing and Pretesting Concepts, Messages, and Materials
Questions to Ask and Answer
- What materials will fit our strategy, appeal to our intended audience, and adequately convey our message? How can we make the materials as effective as possible?
- Do we need to create new materials? What types?
- How do we develop culturally appropriate messages and materials?
- How do we develop effective materials for low-literacy intended audiences?
- How can we make sure the materials will be used?
- When and how should we pretest our materials?
- How can we keep pretesting costs down?
- What should we do with pretest results?
- How can we get the best results from creative and research professionals? From reviewers?
Why Developing and Pretesting Messages and Materials Are Important
Developing and pretesting messages and materials are important because they allow you to learn early in the program which messages will be most effective with the intended audiences. Knowing this will save your program time and money by ensuring that you do not go through the entire development process with an ineffective message. Positive results from pretesting can also give you early buy-in from your organization.
See Appendix B for descriptions of theories and models that suggest important audience factors to consider when creating messages so they are both acceptable and persuasive to the intended audience.
Steps in Developing and Pretesting Messages and Materials
In Stage 1, you created a communication strategy statement. In Stage 2, you will use the strategy you developed as a guide to:
- Review existing materials
- Develop and test message concepts
- Decide what materials to develop
- Develop messages and materials
- Pretest messages and materials
1. Review Existing Materials
Message and materials development and production can be time-consuming and costly. Because this process is creative and has tangible results, it is frequently the key developmental step for a health communication program. Before you begin developing and producing new materials, however, determine whether creating them is necessary.
You may have discovered existing communication materials (booklets, leaflets, posters, public service announcements, videotapes) while gathering data to plan the program. If not, look now.You may find materials at the following sources:
- Health departments (in your state or other states)
- University or public libraries
- Voluntary organizations
- Health professional associations
- Community-based coalitions
- Clearinghouses, Web sites, and telephone information services relevant to the issue
- Materials produced by the National Institutes of Health, the Centers for Disease Control and Prevention, or other agencies in the U.S. Department of Health and Human Services
- Healthfinder®, the Federal gateway to health information, to identify relevant Federal clearinghouses and other Federal information sources ( www.healthfinder.gov)
See Appendix C for additional contact information for these suggested sources.
If you find materials related to the health issue, decide whether they are appropriate for your program, either as they are or with modifications. Using the communication strategy statement as a guide, consider the following questions:
- Are the messages accurate, current, complete, and relevant?
- Are the materials appropriate for the intended audience in format, style, cultural considerations, and readability level? If not, could they be modified to be appropriate?
- Are the materials likely to meet the communication objectives?
Pretesting, discussed later in Stage 2, can help you answer these questions. Check with the group that originally produced the materials to learn about:
- Results of any pretesting
- Effectiveness of the materials to date
- Whether the group has advice or recommendations related to your program's needs
If you are considering using existing materials, also ask the original producer these questions:
- Are they available?
- Could your organization receive permission to use the materials? Modify them? Note: Materials produced by the Federal Government are not copyrighted and may be used freely.
- Are they affordable?
- How have they been used?
- How have they been received?
See Appendix A for a sample form that you can modify to help you conduct a materials review.You also may want to test promising materials with the intended audience (see step 2). If the materials prove to be inappropriate, you will have gained valuable information for modifying them or developing new materials.
NCI Adapts Pain Brochures for Readers with Low Literacy Skills
When a panel of experts set new guidelines for cancer pain control, NCI wanted to share its practical recommendations with patients. A sophisticated handbook was already available for well-educated readers, and NCI staff searched for existing materials that might work for readers with lower literacy skills. They found several local clinic and cancer center publications that could potentially fill this need, but each was at too high a reading level, lacked appealing graphics, and needed to be updated to reflect panel recommendations. NCI decided to adapt the existing pieces and worked with the original producers and specialists to develop easy-to-read materials and graphics. The revised brochure was reviewed in draft by partnering clinics and health professionals and pretested using focus groups and nurse-administered patient questionnaires.
One of the objectives included in Healthy People 2010 is to "improve the health literacy of persons with inadequate or marginal literacy skills" (objective 11-2). NCI's initiative to adapt a brochure from previous materials illustrates how increases in health literacy will come not only with improvements in individuals' skills, but also with improvements in the materials used to communicate critical information to patients and consumers.
2. Develop and Test Message Concepts
The communication strategy statement and the other planning you did in Stage 1 form the basis for developing message concepts. Message concepts are messages in rough form and represent ways of presenting the information to the intended audiences. These may include statements only or statements and visuals. Do not develop the actual messages at this point. (If you create two or more concepts for each message, you will be able to explore which alternative works best.)
In this step, you will learn about the components that go into developing and testing message concepts, including working with creative professionals, creating culturally appropriate communication concepts, choosing a type of appeal, and testing concepts.
To develop, pretest, and eventually produce messages and materials, assemble a team of creative professionals, market research experts, and others. See the Communication Research Methods section for tips on working with market research experts.
NCI's Cancer Research Awareness Initiative: From Message Concepts to Final Message
In 1996, the NCI's Office of Communications (OC), then the Office of Cancer Communications, launched the Cancer Research Awareness Initiative to increase the public's understanding of the process of medical discoveries and the relevance of discoveries to people's lives. OC's concept development and message testing for this initiative included the following activities.
Three values of medical research were selected for concept development:
- Progress (e.g., we are achieving breakthroughs)
- Benefits (e.g., prevention, detection, and treatment research are benefiting all of us)
- Hope (e.g., we are hopeful that today's research will yield tomorrow's breakthroughs)
Based on these values, the following message concepts were developed and explored in focus groups with intended audience members:
- Research has led to real progress in the detection, diagnosis, treatment, and prevention of cancer
- Everyone benefits from cancer research in some fashion
- Cancer research is conducted in universities and medical schools across the country
- Cancer research gives hope
- At the broadest level, research priorities are determined by societal problems and concerns; at the project level, research priorities are driven primarily by past research successes and current opportunities
The following messages were crafted after listening to intended audience members' reactions and their language and ideas about the importance of medical research:
- Cancer Research: Discovering Answers for All of Us
- Cancer Research: Because Cancer Touches Us All
- Cancer Research: Discovering More Answers Every Day
- Cancer Research: Because Lives Depend on It
- Cancer Research: Only Research Cures Cancer
Mall-intercept interviews were conducted to pretest them. Based on responses from the intended audience in these interviews, message D was selected as the program theme.
Working With Creative Professionals
Developing a communication campaign usually involves working with creative professionals, either within your organization or on a contract basis. In either case, managing the relationship effectively is critical to getting the creative materials you want:
- Get to know and feel comfortable with the people who will be working on the project. If you are considering
a contract with an advertising agency, public relations firm, or consulting firm, interview the professionals who will staff your effort (not just the agency representatives who solicit your business) and review samples of their specific work (not just the agency's). Write into the contract who will work on the project. - Be a good client. Use the creative brief to lay out the communication strategy (developed in Stage 1) and make
sure the team understands the brief and that it must be followed. Think about what you want before you discuss the assignment and show the creative team examples of other materials that worked well or didn't and explain why. If you say, "I don't know what I want; you're the creative one," you lose a valuable opportunity to give creative professionals the fundamental direction they want and need. This does not mean asking for a blue brochure; it means helping members of the creative staff understand the objectives and concerns and what you've learned about the intended audience so that they can use their expertise to suggest effective approaches. Discuss sensitive issues, key content points, and other aspects that you want to see conveyed in the messages and materials, based on your knowledge and expertise. - Agree at the outset to what pretesting and approvals will be required, when they will occur, and how long they will take.
- Discuss the theoretical grounding of the communication effort and help creative professionals understand and apply health communication theory to messages and materials development. Brainstorm with them about how the theory might shape the messages and materials and evaluate works in progress with this perspective in mind.
- Involve the creative team in concept exploration and pretesting. Ask its members what questions they would like addressed and make sure they can observe (not participate in) concept exploration sessions. Listening to the intended audience can help them craft messages and materials that use language and ideas that the audience will identify with.
- Assess draft messages and materials against the creative brief and what you know about an intended audience member's point of view. If the intended audience is urban teens at high risk of pregnancy, and you are a middle-aged suburbanite, recognize that the materials most likely to be effective with the intended audience may not appeal to you at all.
- Trust the team's professional expertise, provided that the material is consistent with your program's strategy and the intended audience's culture. While you have a key role to play in ensuring the appropriateness and accuracy of substantive content and in maintaining the program's strategic focus, developing the team's insights and commitment will keep the team involved.
Developing Culturally Appropriate Communications
Culture encompasses the values, norms, symbols, ways of living, traditions, history, and institutions shared by a group of people. Culture affects how people perceive and respond to health messages and materials, and it is intertwined in health behaviors and attitudes. Often, an individual is influenced by more than one culture; for example, teenagers are influenced by their individual family cultures as well as the norms, values, and symbols that comprise teen culture in their locale.
To develop effective health communications, you must understand key aspects of the cultures influencing the intended audience and build that understanding into the communication strategy. Messages must take into account cultural norms in terms of what is asked (e.g., don't ask people to make a behavior change that would violate cultural norms), what benefit is promised in exchange (in some cultures, community is most important; in others, individual benefit is), and what image is portrayed. The symbols, metaphors, visuals (including clothing, jewelry, and hairstyles), types of actors, language, and music used in materials all convey culture.
While it is important to acknowledge and understand the cultures within an intended audience, developing separate messages and materials for each cultural group is not always necessary or even advisable. For example, when print materials for a state program for low-income people depicted people of only one race, some intended audience members who were of that race felt singled out and said the materials suggested that only members of their racial group were poor. Careful intended audience research can help your program identify messages and images that resonate across groups–or identify situations in which different messages or images are likely to work best.
According to a Center for Substance Abuse Prevention Technical Assistance Bulletin, culturally sensitive communications:
- Acknowledge culture as a predominant force in shaping behaviors, values, and institutions
- Understand and reflect the diversity within cultures. In designing messages that are culturally appropriate, the following dimensions are important:
- Primary cultural factors linked to race, ethnicity, language, nationality, and religion
- Secondary cultural factors linked to age, gender, sexual orientation, educational level, occupation, income level, and acculturation to mainstream
- Reflect and respect the attitudes and values of the intended audience; some examples of attitudes and values that are interrelated with culture include:
- Whether the individual or the community is of primary importance
- Accepted roles of men, women, and children
- Preferred family structure (nuclear or extended)
- Relative importance of folk wisdom, life experience, and value of common sense compared with formal education and advanced degrees
- Ways that wealth is measured (material goods, personal relationships)
- Relative value put on different age groups (youth versus elders)
- Whether people are more comfortable with traditions or open to new ways
- Favorite and forbidden foods
- Manner of dress and adornment
- Body language, particularly whether touching or proximity is permitted in specific situations
- Are based on concepts and materials developed for and with the involvement of the intended audience. (Substituting culturally specific images, spokespeople, language, or other executional detail is not sufficient unless the messages have been tested and found to resonate with the intended audience. Formative research with audience members takes on added importance when planners and designers have different cultural backgrounds than the intended audience does.)
- Refer to cultural groups using terms that members of the group prefer (e.g., many people resent the term "minority" or "nonwhite." Preferred terms are often based on nationality, such as Japanese or Lakota.)
- Use the language of the intended audience, carefully developed and tested with the involvement of the audience
Identifying Messages that Resonate Across Cultures
As part of an effort to design messages that are meaningful and appealing to women in different ethnic groups and to older women, NCI's Office of Communications conducted separate focus groups with African-American, American-Indian, Asian, Caucasian, and Latina women.
The groups tested 10 motivational messages about mammography. Once participants had individually selected the motivational messages they found most and least persuasive, the moderator led them in a more detailed discussion of each message's strengths and weaknesses. Throughout the discussion, the moderator probed participants' knowledge, attitudes, and behaviors concerning breast cancer and mammography, sometimes exploring underlying motivations and barriers.
Across focus groups, the following message elements were viewed most positively:
- Breast cancer can develop at any time
- All women are at risk—even those age 65 and older, or those without a family history
- Mammograms can detect breast cancer early
- Early detection can save lives
The least persuasive messages made explicit reference to issues that were considered turnoffs, fear and age. Participants were uncomfortable with messages that specified age and, in some cases, gender. Many said that cancer was a risk for all people (some pointed out that men can get breast cancer), stating that older women (i.e., over 40) should not be singled out. The notion of a mammogram being able to "save your life" was persuasive not only because it was positive but also because it did not distinguish between age groups. In general, messages that seemed to tell women what to think were deemed offensive, while messages that were phrased as explanation or encouragement were more effective.
Choosing the Type of Appeal
To capture the intended audience's attention, you can scare people, tug at their hearts, make them laugh, make them feel good, or give them straight facts.What will work best? The answer generally depends on the intended audience's preferences, what your program is asking people to do, and how you plan to use the appeal in asking them to do it.
Positive emotional appeals show the benefits intended audience members will gain when they take the action portrayed in the message. Research has shown that, in general, messages that present a major benefit but do not address any drawbacks tend to be most appropriate when intended audience members are already in favor of an idea or practice. In contrast, messages that present a major benefit and directly address any major drawbacks work best when people are not favorably predisposed.
Humorous appeals can work for simple messages, especially if most competing communication is not humorous. The humor should be appropriate for the health issue and convey the main message; otherwise, people tend to remember the joke but not the message. Also, humorous messages can become irritating if repeated too frequently.
Threat (or fear) appeals have been shown to be effective with two groups. Research has shown that such appeals tend to be more effective with "copers" (people who are not anxious by nature) and "sensation seekers" (certain youth), and when exposure to the message is voluntary (picking up a brochure rather than mandatory attendance at a substance abuse prevention program). Research has also shown that, to be effective, a threat appeal should include:
- A compelling threat of physical or social harm
- Evidence that the intended audience is personally vulnerable to the threat
- Solutions that are both easy to perform (i.e., intended audience members believe they have the ability to take the action) and effective (i.e., taking the action will eliminate the threat)
In general, however, the effectiveness of threat appeals is widely debated.
The most appropriate type of appeal may differ from this general guidance, depending upon gender, age, ethnicity, severity of the problem, and the intended audience's relationship to the problem. For more information, please consult the following sources under Selected Readings at the end of this section: Backer, Rogers, and Sopory (1992); Goldberg, Fishbein, and Middlestadt (1997); Kotler and Roberto (1989); Maibach and Parrott (1995); Palmgreen et al. (1995); Siegel and Doner (1998).
Choosing Messages for Young Sensation Seekers
Research has found that some youth have a preference for novel experiences and stimuli. Called "sensation seekers," members of this group have four subcategories that represent degrees of the characteristic:
- Thrill- and adventure-seeking (e.g., parachuting and scuba diving)
- Experience-seeking (e.g., nonconforming lifestyle and musical tastes, drugs, unconventional friends)
- Disinhibition (sensation through social stimulation; e.g., parties, social drinking, a variety of sex partners)
- Boredom susceptibility (restlessness when things are the same for too long)
Some health communicators working on drug abuse prevention programs have found that focusing on sensation seekers with messages that appeal to this aspect of their personalities can be effective in promoting attention to and recall of the message and in affecting factors such as behavioral intention and attitudes.
For example, a University of Kentucky program designed for adolescents a creative high-sensation television PSA that focused on the importance of alternatives to substance use for meeting sensation needs. The PSA, titled "Common," featured heavy metal music and quick-action cuts of high-sensation activities. "Wasted," which had the highest sensation value, also had heavy metal music and displayed the words "wasted," "blasted," "stoned," and "fried." Voice-over and illustrative footage accompanied each word (e.g., "with drugs you can get fried" had footage of a monk's self-immolation). It closed with the words "without drugs you can still get high" and offered examples of high-sensation alternatives.
Concept Testing
Once you have defined intended audiences and communication strategies and have developed message concepts, testing the concepts with intended audiences can help you decide on message appeals (e.g., feararousing versus factual), spokespersons (e.g., a scientist, public official, or member of the intended audience), and language (determined by listening to research participants' language). Testing is especially important if the program deals with a new issue, because it will help you understand where the issue fits within the larger context of the intended audience's life and perceptions.
Concept testing will help save your program time and money because it will identify which messages work best with intended audiences. Use concept testing to identify:
- Which concept has the strongest appeal and potential for effect
- Confusing terms or concepts
- Language used by the intended audience
- Weaker concepts that should be eliminated
- New concepts
Concepts can be presented in a number of ways. The key is to convey the major characteristics of the appeal along with the action your program wants intended audience members to take and the benefit they will receive as a result. Focus groups or in-depth interviews are most appropriate for concept testing because they permit discovery of:
- How an intended audience thinks about an issue
- How its members react to different appeals or aspects of a message concept
- Why they react that way
Message concept tests often ask participants to rank a group of concepts from most to least compelling and then to explain their rankings. Participants then discuss benefits and problems associated with each concept. Health communicators often use a sentence or brief paragraph to describe a concept to participants. For example, the following are two "don'smoke" concepts for teens:
- Smoking harms your appearance.
- Cigarette advertisers have created a myth that smoking makes a person more attractive. They're lying.
While both concepts address attractiveness, the first concept uses it as the focal point of a negative appeal (to avoid becoming less attractive, don't smoke), whereas the second concept uses a factual approach and a different benefit—avoid being manipulated by the tobacco industry—designed to appeal to teens' strong desire not to be manipulated.
In each of the concepts above, both the action the intended audience members should take and the benefit are implied, not stated. This approach works in situations where the desired behavior is obvious. In other situations, the behavior or the benefit will need to be mentioned, as in the following examples:
- Mammograms detect breast cancer long before a lump can be felt. And finding it early can save your life
- If you're concerned about getting breast cancer, getting a mammogram may give you peace of mind
More detail on the structure of a concept test is provided later in this section under Conducting Concept Tests and Materials Pretests and in the Communication Research Methods section.
Cancer Risk Message Concept Development
To provide cancer risk information to the public in ways that it could be readily understood and used, NCI conducted a series of focus groups to learn what the groups thought of different methods for communicating about risks. The following insights from the groups underscored the importance of considering both word usage and presentation methods when developing message concepts and materials:
- Participants said that they want cancer risk messages to give them hope for preventing
- cancer and that risk information is less threatening when written in optimistic terms.
- When faced with "bad news" about cancer risks, they said that they look for why it does not apply to them.
- They wanted risk messages to address key questions such as "How serious is the risk?" and "What can be done to reduce or avoid the risk?" as well as explain how and where to get additional information.
- Word choice also influences how information is perceived; "risk" raises alarm, while "chance" minimizes it.
- Use of vague or unfamiliar terms (including "fourfold," "relative risk," "lifetime risk") gives people reason to discount the information.
- Combining brief text and visuals (such as charts, graphs) can increase attention and understanding.
- Statistical risk information was difficult for many participants to understand; percentages were more understandable than ratios, but in either case accompanying explanations of the seriousness of the risk were needed.
- Participants were interested in "the complete picture"—that is, what is known and what is not yet known about a risk, and what it means for "human beings."
- The source of risk information colors credibility, with participants saying that they are less likely to trust the media or a source with a business interest and more likely to trust risk information supplied by a physician or medical journal.
3. Decide What Materials to Develop
Once you have message concepts that are effective with the intended audience, determine the material formats (e.g., brochure, videotape) that will best suit your program by evaluating:
- The nature of the message (e.g., its complexity, sensitivity, style)
- The function of the message (e.g., to call attention to an issue or to teach a new skill)
- The activities and channels selected during Stage 1 (e.g., whether you will be most likely to reach the intended audience through a school, library, physician, the media, or a combination of these)
- The budget and other available resources
Most important: Make sure that your program's decisions about materials fit with the activities and channels you selected and will contribute to reaching the communication objectives developed in Stage 1.
If your program has to develop new materials, doing so will probably represent a major expenditure. Make sure to choose formats that your program can afford; don't allot so much of your budget to materials production that you can't afford sufficient quantities, distribution promotion support, or process evaluation. The list of materials options below will give you ideas of possible approaches. Don't stop there; use your knowledge of the intended audience to combine, adapt, and devise new ways to get the message across. Explore your program's preliminary decisions about materials with the intended audience or partners, perhaps in conjunction with getting reactions to message concepts.
Health Communication Materials Options
Interpersonal Channels
- Fact sheet with a list of questions for patients to ask health care providers
- Physician pads for patient counseling (similar to prescription pads, but used to provide information)
- Slides and a script to assist presenters
- How-to booklets and talking points for discussions in private homes or within the family
- Videos to trigger discussion
- Telephone information service scripts and responses
- Tailored communications, such as letters or personalized newsletters (see Tailored Communications)
Organizational Channels
- Newsletters
- Educational programs (in-person, audiovisual, computerized, print)
- Speeches
- Tailored letters to members
- In-house radio or video broadcasts
- Kiosks/displays
- Buttons, refrigerator magnets, or other giveaways
- Add-ons to regular communication (e.g., messages handed out with paychecks or organization notices)
- Event banners, flyers, stickers, buttons
- Conference exhibits, presentation slides
- Spokesperson training materials
Community Channels
- Displays for beauty and barber shops, pharmacies, grocery stores, airports, libraries, transit, and other public venues
- Posters
- Inserts with bills, in or on shopping bag
- Community newsletters
- Health fair exhibits and handouts
- Letters, e-mail to organize community response
- Kiosks in shopping malls, post offices
- Handouts for meetings with community leaders, lawmakers
- Spokesperson training materials
Mass Media Channels
- Audio or video news releases or B-roll
- Cartoons/comics
- Direct-mail letters, brochures
- Photonovellas
- Magazine and newspaper articles
- Newspaper inserts
- Media kits
- Music news releases/music videos
- Op-eds or letters to the editor
- Posters
- Radio, TV, print advertisements (paid or public service)
- Radio or TV programming
Interactive Digital Media Channels
- Ads on Web sites
- E-mail messages
- CD-ROMs
- Interactive quiz or game on Web site
4. Develop Messages and Materials
The following guidelines will help your program develop materials that intended audiences understand, accept, and use:
- Ensure the message is accurate:
- Scientific accuracy is vital to producing desired outcomes and to your program's credibility. This is particularly important because of rapid changes in advice and information for many health topics; often, what is accurate one year is no longer current the next. For this reason, experts on the health topic should always review your program's messages. However, you may have to work with them to get them to accept language that is simple enough for a nonscientific intended audience to understand. At times it may be helpful to share pretest results with them or get them to sit in on research with the intended audience so that they are exposed to the intended audience's level of knowledge and scientific sophistication.
Patches: An Innovative Format to Reach Girls
The Department of Health and Human Services, Substance Abuse and Mental Health Services Administration's Center for Substance Abuse Prevention's Girl Power! campaign partnered with the Girl Scouts of the USA to develop a Girl Power! Girl Scout patch program. Girl Power!, a national public health education program, addresses a wide range of issues affecting adolescent girls. The overall goal of Girl Power! is to delay and reduce the use of alcohol, tobacco, and illicit drugs among girls ages 9–14. This program also addresses related issues such as physical activity, nutrition, and mental health. Girl Power! and the Girl Scouts cooperate in an ongoing relationship. Through the Girl Scouts, the Center for Substance Abuse Prevention's Girl Power! campaign materials are distributed to over 2.8 million girls around the country. To see other formats both groups use to reach girls, visit www.girlpower.gov or jfg.girlscouts.org.
- Be consistent:
- All messages in all materials and activities should reinforce one another and follow the communication strategy. No matter how creative, compelling, or wonderful a message is, if it does not fit the strategy statement, objectives, and identified intended audiences, throw it out. Don't compete with your own campaign for attention.
- Recognize inconsistencies between the message and what the intended audience may have heard from other sources due to controversies among scientists, government agencies, and advocacy groups. The best way to determine whether and how to address such inconsistencies is to ask the intended audience what impact the disparities have on them and what they need from your program to make decisions and take the desired action.
- Use the same graphic identity in all campaign elements. In print materials, use the same or compatible colors, types of illustrations, and typefaces throughout the campaign. If there is a logo or theme, use it in all materials. Graphics and messages should reinforce each other, not send different signals.
Presentation Options for Advertising and Audiovisual Materials
Animation—Use for young children and as an eye-catching approach for adults (older children may consider some animation silly or "babyish"). Use animation to demonstrate desired actions or to address abstract subjects (e.g., respite care), sensitive subjects (like AIDS), or several disparate intended audiences (e.g., different ethnic groups) at once.
Demonstrations—Use an audiovisual format to demonstrate the desired health behavior, especially if your program must teach skills.
Slice of life—Use a dramatization within an "everyday" or familiar setting to help the intended audience relate to the message. A simple story is easy to remember; you might choose to present the health problem and show the solution. This style can be both credible and memorable, but it may not work with all intended audiences.
Testimonials—Use a credible presenter to lend credibility to the message. A recognizable spokesperson may be attention grabbing. The most credible and relevant presenter may vary for different intended audiences and may be an intended audience representative, an authority (e.g., a physician), or a celebrity connected with the health issue.
- Be clear:
- Keep it simple. Clear messages for lay intended audiences contain as few technical/scientific/bureaucratic terms as possible
and eliminate information that the audience does not need in order to make necessary decisions or take desired actions (such as overly detailed background about disease physiology, research uncertainties, or background about your organization). Readability tests (see the Communication Research Methods section for instructions and limitations) can help determine the reading level required to understand material and can help writers be conscientious about the careful selection of words and phrases. - Prominently feature the action you want the intended audience to take. Give people an explicit recommendation of what
you want them to do as a result of the message. Don't assume they will figure it out for themselves. For example, pretesting showed that a brochure about taking part in AIDS research studies was very well received by a grateful and emotional intended audience. However, no one recognized that the purpose of the booklet was to encourage people to consider participating in research studies and to talk about it with their health care providers. Instead, almost everyone thought the purpose was to give them hope and comfort. The booklet was revised to be more straightforward without compromising the hopeful tone intended audiences liked. - Demonstrate the health behavior or skills (if appropriate).
- Keep it simple. Clear messages for lay intended audiences contain as few technical/scientific/bureaucratic terms as possible
- Be relevant to the intended audience:
- Include incentives for the intended audience to take the recommended steps, beyond the health benefits they will receive. Audience research will help you understand what the intended audience might value. Incentives may be psychological (you'll feel more in control); altruistic (you'll help others by participating in a research study); economic (not smoking saves money); social (exercise is a fun group activity, where you can make friends, see friends); family-driven (do it for them); or ego-driven (sun exposure causes wrinkles).
- Choose a presentation style appropriate to the intended audience's norms and expectations; people must be able to see themselves in what your program presents. For example, intended audience preferences can help determine whether to take a rational or emotional approach, a serious or light tone. Use a light, humorous approach if appropriate, but pretest to be sure that it works and doesn't offend the intended audience. Responses to humor vary greatly.
- Keep regional differences in mind. A lush, green park will not look like the Southwest and an urban clinic will not be familiar to people who live in a rural area. The organization of the health care system and the way medical and social services agencies work together also differ from region to region.
- Use intended audience experience when creating materials. The intended audience will learn new information more easily when you build from the familiar to the unfamiliar (e.g., "When you have a headache that won't go away, you take an aspirin to relieve it. But that is not the best way to control cancer pain. Cancer pain medicines work best if you take them before your pain becomes severe and keep taking them on the schedule your doctors advise.")
- Create the message to match the readiness of the intended audience to make a change. The Stages of Change model (see Appendix B) describes five stages that people pass through in making behavior changes: precontemplation, contemplation, preparation, action, and maintenance. Messages created to match readiness to change start where the intended audience starts. For example, it is unrealistic to expect an intended audience that has never heard of the problem to immediately make changes. A realistic outcome is beginning to raise awareness of the issue (precontemplation) and helping move the intended audience to a consideration of the change (contemplation).
- Be credible:
- Use celebrity spokespeople selectively. Choose celebrity spokespeople who are directly associated with the message (e.g., using an athlete to promote exercise or cancer survivor to promote early detection) and who practice the desired health habit. Check with the intended audience about the suitability of the celebrity. (For example, while a famous television personality was well liked by members of the intended audience for a physical activity promotion program, they said she was not relevant to them because she could afford a personal trainer to make exercise easier.) Realize that while celebrities can help gain attention for the message, they may also compete with the message for attention or be unappealing to some intended audiences. In addition, your program will probably need to build its schedule around theirs, and celebrity involvement can cause production delays and extra costs. Caution: A network may not use TV PSAs featuring a rival network's star.
- Be sure that the person who presents the message is seen as a credible source of information, whether as an authority, celebrity, or intended audience representative.You may also want to partner with organizations that are credible with the intended audience and emphasize their involvement. If the message involves health services and health care coverage, it is particularly important to have a source that the intended audience believes is unbiased, i.e., does not have a vested (profit-driven) interest in people taking the recommended action. For example, consumers may dismiss health practices an employer or government agency suggests they take if they perceive the motivation is just to save the organization money.
Tips for Developing TV Ads
- Keep messages short and simple—just one or two key points.
- Use language and style appropriate for the intended audience.
- Repeat the main message as many times as possible.
- Recommend a specific action.
- Demonstrate the health problem, behavior, or skill (if relevant).
- Provide new, accurate, straightforward information.
- Be sure the message, language, and style are considered relevant by the intended audience.
- Be sure that the message presenter is seen as a credible source of information, whether an authority, celebrity, or intended audience representative.
- Select an appropriate approach (e.g., testimonial, demonstration, or slice-of-life format).
- Be sure every word works.
- Use a memorable slogan, theme, music, or sound effects to aid recall.
- Check for consistency with campaign messages in other media formats.
- Use positive rather than negative appeals.
- Emphasize the solution as well as the problem.
- Use a light, humorous approach, if appropriate, but pretest to be sure that it works and doesn 't offend the intended audience.
- Avoid high degrees of fear arousal, unless the fear is easily resolved and the message is carefully tested.
- Use only a few characters.
- Make the message understandable from the visual portrayal alone.
- Superimpose text on the screen to reinforce the oral message's main point.
- Identify the main issue in the first 10 seconds in an attention-getting way.
- Use 30-second spots to present and repeat the complete message; use 10-second spots only for reminders.
- If the action is to call, show the phone number on the screen for at least 5 seconds, and reinforce orally.
- Summarize or repeat the main point/message at the close.
- Be appealing:
- Produce variations of materials to appeal to specific intended audience segments. For example, NCI used several different covers on a mammography booklet to appeal to different cultures. Similarly, audio materials can be produced using culturally specific voices or music. Of course, changing executional detail to appeal to specific intended audience segments is appropriate only if the underlying communication strategy and messages have been tested with those audiences.
- Get the intended audience's attention. Given the number of health and other messages intended audiences receive, yours must stand out to be noticed. The best way to command attention will differ among intended audiences. It can be useful to know what has interested them before, but concept testing and message testing will help ensure that your program's approach will grab their interest.
- Produce high quality materials. If you feel you have to skimp on production, choose a simpler way of presenting the message. Producing poor quality materials wastes funds and can damage your program's credibility and your own.
- Entertain while you educate when using mass media. Whether you are pitching a news story or producing an ad, remember that the mass media are viewed as a source of information and entertainment, not education. Therefore, if the complete message is too complicated, or simply not considered interesting enough for use by the media, redesign the message so that it is more appealing to media professionals and their perceptions about what their intended audience wants.Working with media professionals will help ensure that your program's messages are interesting as well as accurate and may help you obtain greater exposure for the program.
Tailored Communications
Tailored communications are much more refined than communications created for a particular intended audience or segment of the population. If you can tailor each message for each individual, it is likely to be even more effective. Tailored communications are produced for each person based upon what is known about the individual. Obtain this information from health plan data, surveys, medical records, and other sources. This information permits matching people with messages appropriate for them. The following are two examples of tailored communication:
- A series of letters (e.g., to give smokers information about and support for quitting) from which paragraphs are selected to send to each individual based on past behavior—on barriers to, for example, quitting, or on other unique combinations of characteristics
- A cover letter, referring readers to the pages of an enclosed booklet that most relate to their interests and concerns
With the help of a computer and ordinary word processing software, individual letters can be tailored within a mass mailing, brochures can be printed on demand for a specific patient, or telephone counselors can appropriately focus their assistance and follow up with messages tied to the advice they gave.
When personal data are not available, interactive computer programs can prompt individuals to input key personal characteristics that direct the software to prepare individually tailored messages immediately (e.g., on a Web site or at a kiosk in a public location). NCI's 5 A Day Web site (www.5aday.gov/) allows users to chart their fruit and vegetable consumption and exercise and offers positive reinforcement in response to the behaviors they report.
Evaluation has shown that tailored communications can, in some circumstances, increase message effects, although much research remains to be done on how and why.
To use tailored communications, review the planning information from Stage 1 and determine:
- The important individual characteristics that affect a particular health behavior (i.e., differences in readiness to change behavior, or perceived benefits of or barriers to changing the behavior)
- Appropriate messages tailored to address individual characteristics (these messages may be identified or confirmed through primary research with each group, or through collaboration with a behavior change specialist or a subject matter expert, such as a genetic counselor or physician)
- Create a "library" of messages tied to each relevant characteristic, such as gender or risk factors ("as a woman who began smoking in the last five years...")
- Use word processing software with a mail merge feature that will match variations in personal characteristics with appropriate messages and produce the materials in the desired format
Tailoring is not always possible or necessary. However, think about using some of the principles of personalizing messages when appropriate and when funding is available. Find more information about tailored communications in the Selected Readings at the end of this section or use NCI's quick-start tutorial on the Internet at http://dino.nci.nih.gov/public/glassman/TailoringGuide.
Tailored Communications: Healthy Birthdays
In two 1999 projects, researchers designed tailored birthday cards and newsletters to increase breast and cervical cancer screening and smoking cessation, primarily among low-income African Americans.
In one study, the cards and newsletters were individually tailored based upon ethnicity, gender, and the individual's readiness to change according to the Stages of Change model. Smokers received either:
- Provider prompting
- Tailored cards and newsletters
- Tailored cards and newsletters and telephone counseling
The tailored cards and newsletters showed a highly significant quitting effect. Thirty-three percent of smokers who received only the tailored cards and letters quit smoking compared to thirteen percent of smokers who received provider prompting and nineteen percent of smokers who received the tailored cards and newsletters and telephone counseling.
In the other study, people who received the tailored birthday cards and newsletters showed higher rates of Pap tests and general cancer screening.
Image created by Bernard Glassman and Barbara K. Rimer. All rights reserved.
Developing Effective Print Materials for Low-Literacy Intended Audiences
Many intended audiences have a hard time understanding health materials written in technical language. This is especially true of intended audiences with low literacy skills. Present the message in a more easily understood way to these intended audiences by making specific choices about writing style, vocabulary, typography, layout, graphics, and color. These choices affect whether the message is read and how well intended audiences with low literacy skills understand it. A great deal of health information and promotion is organized around the use of print materials, often written far beyond the literacy skills of intended audiences. Differences in the ability to read and understand materials related to personal health appear to contribute to health disparities. The link between literacy and health disparities underlies the Healthy People 2010 objective to "improve the health literacy of persons with inadequate or marginal literacy skills" (objective 11-2).
A common misconception is that low-literacy materials are synonymous with low-reading-level materials. That is, if you avoid polysyllabic words and long sentences, you've met the need. In fact, low literacy encompasses more than reading level, and meeting this need requires that you complete the same planning and research steps and adhere to the same fundamental communication principles that you would use for any other health communication material. The difference is that certain aspects of the process must be done with particular rigor. The following list includes pointers for designing materials for low-literacy intended audiences:
- Include only the information needed to convey the behavioral objective and support the intended audience in attaining it. Strictly limit content because poor readers struggle with every word, often reading letter by letter. Keep the piece short and focused, and let the communication strategy statement guide you in answering the question, "From the many possible information points, which ones will predispose and enable the reader to take the desired action?" The biggest challenge will be excluding concepts and content points that fall outside of the category of "information the reader must know." Do not include information just because it may interest the reader or because you are trying to promote your organization's work.
- Organize topics in the order the reader will use them. Less skilled readers have particular difficulty connecting topics and processing the flow of an argument.
- Present the most important points first and last. Studies show that intended audiences with limited literacy skills remember these best.
- Group information into chunks, with a clear, ordered format. Use steps (1, 2, 3), chronology (by time of day), or topical arrangement (main heading, subheadings), depending on how the person will use the information.
- Respect the intended audience. This is especially critical when designing low-literacy materials. Examples abound of well-intentioned materials that talk down to readers or have childlike or simple cartoon illustrations. The low-literacy population encompasses people of different ages, genders, cultures, and socioeconomic status, including highly intelligent adults with significant life experience who just cannot read very skillfully.
- Follow these guidelines.
- Use short sentences and paragraphs.
- Write in the active voice.
- Clarify concepts with examples.
- Avoid jargon, technical terms, abbreviations, and acronyms.
- Include a glossary if necessary (but define key words within the sentence).
- Give the reader an action step he or she can take right away (e.g., call your clinic, send in a request); this tends to improve retention of information and encourages the reader to begin practicing the desired behaviors immediately.
- Use graphics and design to make the reader's job easier and to increase comprehension and recall; make sure they support, rather than compete with, the text.
- Don't assume that pictorial signs, symbols, and charts are more effective than words for low-literacy intended audiences. Some experts suggest that "universal" symbols, such as a stop sign, an arrow, or a big black "X," usually test well. Don't confuse this intended audience with large, busy matrices—for example, functionally illiterate individuals have trouble using a bus schedule.
- Avoid using all capital letters; they are more difficult for everyone to read, particularly so for less skilled readers.
- Use captioned illustrations that are relevant to the subject matter and model the desired behavior.
- Use headings and subheadings to convey a message and help reinforce the flow and content.
- Use bullets and other graphic devices to highlight key messages and to avoid large blocks of print.
- Avoid right-justified margins.
- Pretest all materials with the intended audience. This is absolutely crucial with low-literacy intended audiences.Writers and communication specialists are highly literate by definition. It is impossible for a person who reads well and has a good vocabulary to guess what people without those skills will understand. For example, an FDA brochure on food safety used the key message, "Keep hot foods hot, keep cold foods cold." Pretesting showed that low-literacy readers had no idea what they were supposed to do based on this message, nor did they understand what foods fell into the hot and cold categories.
A final note: You will find that most intended audiences of any reading level prefer well-produced materials that follow these guidelines.
Developing Effective Web Sites
A Web site should be graphically appealing and provide information about health issues in an informative manner. Some organizations begin by creating sites that primarily provide information to their stakeholders, employees, or members. To extend outreach, create an additional section in the site to appeal to the intended audience. For consumers, you might call this section the Help Center and provide a place for users to receive information about a particular health topic, participate in online surveys, or download your organization's consumer information. Many sites contain useful public health information and resources, but too often this information is buried within the site. Keep visitors interested in the site by making it easy to navigate.
To ensure that users will find the site well designed and easy to use, pretest the site as you would any other materials. Usability testing, which tests the site to see how well it helps users meet their goals, is crucial to creating an effective site. The best time to do this testing is as you are developing the site, not after it's completed. If the site is not yet running on a computer, test using paper or poster board mock-ups of pages. Conduct usability testing by having people who represent the intended audience actually sit down and use the site to complete tasks, either by themselves or in pairs. Observe how they interact with the site and ask specific questions once they have completed the tasks.
Their experiences and responses will allow you to improve the site before it is used. If you make major modifications to the site after usability testing, test again before the site goes live. For more information on usability testing, see www.usability.gov. Remember, your well-designed and attractive site is useless unless people know it exists. Therefore, consider launching a Web site by conducting both traditional and online media outreach. Online outreach can include alerting search engines such as Google or Yahoo about the site as well as selecting publications that specialize in online issues or exist only online.
Characteristics of Well-Designed Web Sites
- Compliant with W3C accessibility guidelines ( www.w3.org/wai), which ensure access to the Web by everyone regardless of disability, and, for government publications, Section 508 guidelines (www.usability.gov/accessibility)
- Clean and consistent design (e.g., a simple background, legible type, a few carefully selected colors)
- A search engine (program that helps users find information) and a link to the search engine on all Web pages
- Fast display of graphics and text
- Clear and consistent navigation elements (to make it easy for users to move to and from different information sources)
- Interactivity and fun elements
- Short/concise pages (e.g., a Web page should not exceed 250 words on average)
- Compatibility with major browsers and earlier versions of major browsers (e.g., Netscape, Microsoft Internet Explorer)
- Mechanisms to track site usage and invite user response
5. Pretest Messages and Materials
Although working with advisory groups and gatekeepers can add useful input for developing intended audience-appropriate materials, only testing with members of the intended audience will tell you what their reactions might be. It is always better to conduct research with intended audience members than to do without it, especially with audiences that you do not know well (e.g., cultural groups other than your own) or whose education or skill levels are different from yours (e.g., audiences with limited literacy skills). When resources are an issue, use lower cost methods and money-saving strategies to keep costs down (see sidebar, Keeping Pretest Costs Down).
Pretest preproduction draft materials. Testing at this stage permits you to identify flaws before spending money on final production. To test materials in draft form, use a facsimile version of a poster or pamphlet, a video version of a television PSA, or a prototype of text materials like a booklet. Test these materials with members of the intended audience to accomplish the following:
- Assess comprehensibility—Does the intended audience understand the message?
- Identify strong and weak points—What parts of the materials are doing their job best—for example, attract attention, inform, or motivate to act? What parts are not doing their jobs?
- Determine personal relevance—Does the intended audience identify with the materials?
- Gauge confusing, sensitive, or controversial elements—Does the treatment of particular topics make the intended audience uncomfortable?
A Review Process Adds Value
As you test materials in rough form with the intended audience, it is also valuable to obtain gatekeeper or other reviewer comments. Health communicators often ask gatekeepers (e.g., public service directors, physicians, teachers, partner organization leaders) to review materials both to get input from people close to the intended audience and to increase the likelihood that the gatekeepers will use the materials with the audience. If the graphics style or illustrations depart from what gatekeepers or other reviewers expect, focus on these issues when testing draft products with the intended audience. Use favorable responses from the intended audience to persuade gatekeepers to accept your program's approach. Gatekeeper review should not be used as a substitute for pretesting materials with members of the intended audience.
Many organizations have established review procedures that can seem like hurdles to timely production of materials. However, reviewers can potentially add value, and review by some experts can be imperative for producing accurate, accepted communication materials. Reviewers will help create accurate documents for pretesting. After pretesting, reviewers can help synthesize the results and help with revising the document. The following tips will help you structure a value-added review process:
- Choose reviewers carefully. Reviewers should have relevant knowledge to contribute and be conscientious enough not to delay the process. Make sure there is a range of expertise represented among the reviewers, including subject experts, communication specialists, intended audience experts/representatives, and those who understand your organization and your partners' policies and priorities. In some cases, reviewers with other skills or viewpoints may be important, such as legal professionals, law enforcement officials, social workers, school personnel, or clergy.
- Explain exactly what you want reviewers to do and not to do. One organization's national campaign materials were delayed when a top administrator was given camera-ready layouts for his final approval, and he wrote comments on them. Reviewers are commonly handed materials without guidance; they may not understand the purpose or the context of the material and have no way of knowing what you want from them. Often this results in reviewers' simply tinkering with words or proofreading the copy, delaying the process without adding value. Instead, give them a list of questions to answer or other guidance.
- Never skip technical review by an expert, who may catch concerns or inaccuracies others miss. For example, the facts you present may be accurate but not reflect new information that only an expert would know.
- Incorporate as many comments as possible.You may need to get clarification on some comments or to talk with reviewers whose comments seem off base.You may find that you can compromise, but remember you are not required to act on all the comments you receive.
- Allow sufficient time in the development process to deal with reviewers' comments. This includes giving reviewers enough time to do a thorough job and giving creative professionals enough time to make revisions thoughtfully and carefully.
Prefinished Materials: Formats for Pretesting
Print (e.g., paper-based)—It is best to test a complete prototype of the final material—for example, the text, layout, typeface, and visuals—that is planned for the final piece. If the test is conducted prior to commissioning original photography or artwork, stock photography or drawings can be used. For longer pieces (e.g., booklets), try to avoid testing text in manuscript form. Word-processing software makes it very easy to put the text into the intended layout, and testing a prototype instead of a manuscript will allow you to more accurately assess comprehensibility rather than assuming it will improve once the text is laid out.
Audio (e.g., radio)—Even if you will have a script rather than a final recording, it is best to produce and test a "scratch tape" so that every pretest respondent hears and reacts to the same thing.
Video (e.g., PSAs)—Storyboards (line drawings showing key scenes, like a comic strip), even if they are videotaped with a voice-over or soundtrack, are too rudimentary for testing to be able to predict reactions to the final product. More useful formats are:
- Animatics (a series of detailed drawings, filmed in rapid succession and using camera zooms and pans to give the illusion of motion)
- Photomatics (similar to animatics, but using a series of photos)
- Rough live action (filmed footage, with costs controlled by using nonunion talent and simplified sets)
- "Ripomatics" (adaptations or alterations of existing footage to create a new video product)
Interactive (e.g., Web, CD-ROM)—The site or program should be complete enough to allow basic functionality and design to be assessed (e.g., does the site or program include the information intended audience members want? Is it organized in such a way that they can easily find it?).
Providing Guidance to Reviewers
NCI sends memos to reviewers with each set of materials. The memo outlines:
- Questions to answer, specific points to check, or issues for reviewer to address
- The purpose, objectives, and intended audience for the materials
- A history of how and why the product was developed and who has already contributed to or reviewed it
For example, the reviewers for this book received instructions and worksheets to help focus their input, and not all reviewers were asked to comment on all topics; they were expected instead to focus on sections related to their expertise.
Conducting Concept Tests and Materials Pretests
To plan and conduct concept tests and materials pretesting, complete the following steps:
- Determine test objectives.
- Choose methods.
- Secure vendors, facilities, and moderators or interviewers (if required).
- Identify, screen, and recruit respondents.
- Draft test instruments (discussion guides, questionnaires).
- Conduct pretesting.
- Analyze results.
- Make the best use of results.
Key aspects of each step are presented below; see the Communication Research Methods section for more detail.
Determine Test Objectives
The first step in planning pretesting is to formulate research objectives. Write specific objectives to provide a clear understanding of what you want to learn from whom. Pretesting can also help to answer questions about alternate ways to present information, questionable inclusions or depictions, and reviewer conflicts regarding content, format, and appearance.
Develop a description of which segments of the intended audience you want to include and exclude in testing. Do you want to include individuals who have attempted a particular health behavior and succeeded? Failed? Ever thought of trying? Consider excluding individuals whose professions may make them too experienced with the topic or the method (e.g., health professionals, market researchers, employees of advertising agencies, or public relations firms). In some instances, exclude consumers whose past experience makes them too knowledgeable. For example, sometimes people with a chronic disease know too much about the condition to objectively assess materials designed for patients recently diagnosed.
Don't rely on these participants for creative guidance. They are expert consumers, and it is important to learn their preferences. They are not, however, communication or creative professionals, and their ideas for substantive message or materials changes are likely to be off the mark.
Benefits and Limitations of Pretesting
Examples of benefits derived from pretests include:
- Assess attention. Central location intercept interviews conducted by the National Heart, Lung, and Blood Institute showed that using symbols/analogies was a promising way to gain attention and to convey a message. However, the message itself must be fairly simple, or the use of an analogy becomes a complicating liability. For example, using a freeze frame and reverse action to "reverse" the risk of high blood pressure was attention getting, but no analogies for multiple risk factor messages proved straightforward enough to work successfully.
- Assess comprehensibility. When developing the USDA food pyramid, extensive pretesting revealed that the pyramid shape conveyed key concepts more clearly than a bowl or other shapes. In addition, planners learned that representing fats, oils, and sugars as a bottle of salad dressing, a can of soda, and a bowl of sugar created widespread misunderstanding. Substituting a stick of butter, a droplet of oil, and a spoonful of sugar improved comprehension.
- Assess motivation. Research conducted for the National Bone Health Campaign, Powerful Bones, Powerful Girls™, explored campaign concepts designed to increase calcium consumption and weight-bearing physical activity among 9- to 12-year-old girls. Girls identified strength—in the form of self-confidence and physical strength—as a key benefit of bone-healthy behaviors. The campaign theme, "Powerful Bones, Powerful Girls," was perceived as motivating, as were examples set in real-world settings and during social activities.
- Assess recall. In a test of a booklet on lung cancer, patients could recall on average 2 out of 12 ideas presented; half the patients could recall none. Too many technical terms, the density of concepts, and too little differentiation between diagnostic and treatment procedures inhibited intended audience recall. Another problem was the inclusion of concepts unimportant to readers. Although they wanted more information on outcomes and treatment, they had no interest in a description of the disease.
- Determine personal relevance. When the National Bone Health Campaign research team explored the credibility of a spokesperson among girls 9-12 years old, the intended audience wanted to hear from "a girl like me." Girls said they wanted to hear the message from a female who was strong, bold, confident, active, healthy, and popular. This guided the development of Carla, a cartoon spokes-character whose name means strong. Carla speaks to girls as a peer working to build powerful bones. In the pretesting of a Web site featuring Carla, girls from varied backgrounds described her as "powerful," "fun," "busy like girls I know," and "someone who feels good about herself"—someone they'd like to know and be like.
- Gauge cultural appropriateness. Pretesting and revising were critical to developing the "¡Mírame! Look at Me!" curriculum to help South Texas Hispanic youth ages 9–13 avoid alcohol, tobacco, and drug use. The research found that this intended audience learned best with active learning strategies and interpersonal interaction, which influenced curriculum format revision. Testing also found that the language used (typical of South Texas conversations, 85 percent English, 15 percent Spanish) was appropriate and easily understood and that using intended audience representatives telling their own stories ensured credibility and close audience attention.
- Identify strong and weak points. Pretests of an NCI booklet, "Cancer Research Studies with Patients: What You Need to Know," found that patients and family members were enthusiastic about the format, comprehensibility, approach (enabling patients to take an active role in decisions about their cancer care), and relevance of the book to patients' needs and situations. Participants' comments led to suggestions for strengthening sections on treatment costs and insurance and adding specific questions to the list patients should ask before agreeing to take part in a research study. In addition, pretests provided useful feedback that allowed NCI to improve graphics portraying complex concepts such as randomization and control groups.
- Identify sensitive or controversial elements. Would using vernacular language to discuss diarrhea in a booklet on chemotherapy for a low-literacy intended audience be preferable to a more technical approach? Pretest results showed that the intended audience found the vernacular offensive and preferred the technical discussion, even though it had a higher reading level.
- A pretest is only as objective as the person designing and interpreting the study.
- Pretesting cannot absolutely predict or guarantee learning, persuasion, behavior change, or other measures of communication effectiveness.
- Pretesting is not statistically precise. It will not reveal that booklet A is 2.5 percent better than booklet B. (Presumably, pretests of such precision could be applied, but the cost of obtaining such data would be high, and the findings may be no more useful than those from more affordable approaches.)
- Pretesting is not a substitute for experienced judgment. Rather, it can provide additional information to help guide sound decisions.
- Pretesting does not guarantee success. Good planning, thorough concept exploration, and sound pretesting can be negated by mistakes in final production or in program implementation. The message in a television PSA on cancer treatment, for instance, may pretest well, but then be flawed by an execution that uses an actress who seems too happy to be awaiting the results of a biopsy report. Similarly, leaflet copy that pretests well may be rendered ineffective by a poor layout, hard-to-read type, or inappropriate illustrations.
Choose Methods
A variety of research methods can and should be used to explore message concepts and test messages and materials. Which you choose depends on the research questions you want to have answered, the nature of the materials, the intended audience, and the amount of time and resources available for pretesting.
The following is a list of methods to use to test materials:
- Concept testing with intended-audience members
- Focus groups (face-to-face or telephone)
- In-depth interviews
- Pretesting with intended-audience members
- Self-administered surveys/questionnaires (by mail, handout, or computer)
- Interviewer-administered surveys/questionnaires (by telephone, through central-location
intercepts, or other face-to-face scenarios) - Theater testing (large groups respond to messages via questionnaire or electronic)
- Observational studies (e.g., observing behaviors of shoppers in a store or patients waiting
in a clinic)
- Other assessment methods
- Readability assessments
- Expert/gatekeeper review
Sometimes, using several methods in combination will help overcome the limitations of individual methods. For example, readability testing should be used as a first step in assessing draft manuscripts, followed by self-administered questionnaires or interviews or, for some long documents, a combination of the two with intended audience respondents. Central-location intercept interviews for short audio, video, or print materials permit contact with larger numbers of intended audience respondents, which is especially useful prior to final production of materials. See the Communication Research Methods section for definitions of commonly used pretesting methods and descriptions of the pros and cons and common uses of each.
Secure Vendors, Facilities, and Moderators or Interviewers
Some research methods require securing appropriate facilities. If you are using commercial facilities (e.g., for focus groups, central-location intercept interviews, or theater tests), the American Marketing Association's New York Chapter's Green Book provides an extensive directory of vendors, indexed by geographic locations and services offered. The Communication Research Methods section provides tips for working with vendors and for securing alternative facilities if location or budget precludes using a vendor. The following are vendor services your program may need:
- Teleconference services to set up telephone focus groups. Most allow observers to listen without being heard, and some provide remote viewing programs to allow the moderator to see a list of participants (with a symbol next to the name of the one currently speaking) or notes sent by a technician from observers listening to the call. Some teleconference services can recruit participants; with others, recruit or contract with a recruiter separately.
- Focus group facilities to recruit participants that also offer audio and video recording equipment as well as one-way mirrors with observation rooms for viewing both focus groups and in-depth interviews. These are usually available only in larger metropolitan areas.
- Commercial facilities to conduct central*#8211;location intercepts at shopping malls.You can also use other facilities or public locations that attract a large number of pedestrians (e.g., other stores, train stations, university campuses, large medical centers); usually permission is required to interview people in these types of locations.
Keeping Pretest Costs Down
Use the following tips to keep pretest costs down:
- Borrow questions from other pretests when possible.
- Work with partner organizations to recruit participants and conduct tests (e.g., an African-American church, patient educators, a clinic).
- Pay for and use transcripts when you conduct focus groups so that you can review the results and make your own decisions rather than paying an analyst to do so.
- When testing with a large number of respondents:
- Keep the questions short and to the point.
- Use as many close-ended or multiple choice questions as possible (for easier tabulation analysis).
- Develop codes in advance for quantifying responses to open–ended questions.
- Avoid overtesting (test to answer questions, not to gather the view of a large number of respondents).
If you are conducting focus groups or in–depth interviews, identify trained, experienced moderators or interviewers. If your organization has no experience in such studies, consider hiring a good, experienced moderator or interviewer to conduct the project or to train internal staff to develop in–house skills. Local advertising agencies, the American Marketing Association's Focus Group Directory, the Qualitative Research Consultants Association, or other health communication program managers may be of assistance in identifying a good moderator. Before choosing a moderator or interviewer, ask for and check references.
Identify, Screen, and Recruit Respondents
Use the intended audience description developed in Stage 1 to draft questions for recruiters to use to identify participants. See Appendix A for a sample form to use to screen potential participants.
Draft Test Instruments (Discussion Guides, Questionnaires)
Involve the creative team in developing the discussion guides or questionnaires. The team will often have specific issues or questions. Assess how much the intended audience likes each set of material, because research has found this to be a leading indicator of success. Other specific questions should be used to identify strengths and weaknesses in rough messages and materials. See Appendix A for a sample focus group moderator's guide and an intercept questionnaire.
Conduct Testing
If possible, have other team members, such as creative professionals, content experts, partners, and gatekeepers (including decision-makers who control your program), observe at least some of the focus groups or interviews. Creative professionals' observations are particularly important during concept exploration, because they often spot comments or trends important for creative development. Hearing the responses improves team members' understanding of intended audience reactions and can illustrate, more vividly than any report, the need for simple language or the power of a particular creative approach.
Analyze Results
To analyze and communicate the results of materials testing, write a report outlining the process and the findings. The report should include the following sections:
- Background—Who was tested? Why? How? What did you hope to learn? (Describe each in detail.)
- Highlights—Summarize the main points that emerged from testing that answer the research questions.
- Findings—Present a complete report of findings.Where appropriate, describe participants' reactions, quote participants, and use examples from the test documents to support the findings.
- Conclusions—Describe patterns that emerged or significant differences observed between groups. (If no patterns are apparent, more research may be needed.)
- Recommendations—Suggest revisions for materials or planned approaches based on findings and conclusions.
- Appendixes—Include copies of test instruments used, such as discussion guides, screening questions, and questionnaires.
Decide who should prepare the report. Ideally, choose a team member with a background in market research and health communication, and have that person include input from the creative team on recommendations. If such a person is not available, whoever conducted the test may be able to prepare a basic report on the findings but, depending on his or her background, this person may not be able to draft useful recommendations. Before relying on vendors to write reports, ask to see samples of reports they've written for other clients.
One problem that can arise in pretesting concerns interpretation of respondent reactions to a sensitive or emotional subject such as breast cancer or AIDS. Respondents may become unusually rational when reacting to such pretest materials and cover up their true concerns, feelings, and behavior. Therefore, the pretester must examine and interpret responses carefully.
The following are some tips for using pretest results:
- Involve the creative professionals who designed the materials in recommending creative solutions or revisions, rather than expecting researchers to make recommendations that creative professionals may later legitimately reject.
- Consider pretesting again if your program has made major changes to a message or product based on the original pretest conclusions.You may have addressed the right problems, but you can't be sure the new solutions will be effective without intended audience testing. The key is allowing time for this possibility in the initial schedule.
Make the Best Use of Results
Pretesting findings can be used to solve problems, plan programs, develop materials, or refine materials or messages. It is important to avoid misuse of market research results. Perhaps the most common error is to overgeneralize. Qualitative, diagnostic pretest methods should not be used to estimate broad-scale results. If 50 of the 100 respondents in an intercept test do not understand portions of a pamphlet, it does not necessarily mean that 50 percent of the total intended audience will be confused. The lack of understanding among those pretest respondents suggests, however, that the pamphlet may need to be revised to improve its comprehensibility. Remember that pretesting is indicative, not predictive.
Planning for Production, Distribution, Promotion, and Process Evaluation
Once you have decided which materials to produce, determine how many copies of each will be needed and develop a production schedule. To write a realistic production schedule, review or revise the following three components of the communication plan:
- Distribution channels you plan to use (including specific organizations, companies, sites, etc.), how many materials you plan to disseminate through each, and how many intended audience members you propose to reach.
- Promotion plans, which describe how you intend to promote use of the materials.
- Process evaluation plans, which describe the way the use of materials will be monitored (e.g., using bounceback cards to identify how respondents are using the materials). Go to the process evaluation section of Stage 3 for more details.
Distribution and promotion sections of the program plan will define both production costs and the quantity of materials to produce. It is a good idea to get input on the distribution, promotion, and evaluation sections of the program plan from partners and others involved with implementation.
Common Myths and Misconceptions About Materials Pretesting
Myth: I don't have the time or money.
Fact: Pretesting needs to be planned as an integral step in the materials development process from the beginning. Include time and resources for pretesting and for any changes you might need to make as a result in the project budget and timeline. Otherwise, your program may not have the funds, and your boss may see the time for pretesting and alterations in materials as a delay in production rather than evidence of careful planning and development.
Myth: My boss won't support pretesting.
Fact: Use the information in this guide and in the Selected Readings to convince your boss that you need to pretest. Beautiful materials and an elegant design can't guarantee that the intended audience will pay attention to, understand, or relate to the messages. It's cheaper to find out whether the materials might work before they are produced than to have to start over later or, worse, to have an unsuccessful program. Once you have pretested, be sure to explain to your superiors (in a report) how it worked and what resulted. Build a case for their acceptance of future pretesting. Using quotes from the intended audience or anecdotes to illustrate the findings can make the report more interesting and memorable.
Myth: I can tell the difference between good and bad materials, so I don't need to pretest.
Fact: Many people have said this, only to find out they can be wrong.Your training and experience are essential credentials, but are you sure you can react objectively to materials you have created or are responsible for? Can you really assume the role of someone who is different from you (if you are not representative of the intended audience) and see your materials through his or her eyes? Can you defend your decision, without objective evidence, to those who may disagree?
Myth: Our artist/producer says that pretesting can't be used to judge creativity.
Fact: Graphics staff, artists, and creative writers may be sensitive to criticism from "nonprofessionals," including the intended audience. Explaining the purpose of pretesting and involving them in the pretesting may help them understand and appreciate the process. Explain that you are testing all elements of the communication and not just their work. By testing alternative concepts or executions, you can provide the creative staff with direction without telling them their work "failed."
Selected Readings
American Marketing Association, New York Chapter. (2001/2002). Green Book: International directory of marketing research companies and services. New York.
Backer, T. E., Rogers, E. M., & Sopory, P. (1992). Designing health communication campaigns: What works. Thousand Oaks, CA: Sage.
Campbell, M. K., DeVellis, B. M., Strecher, V. J., Ammerman, A. S., DeVellis, R. F., & Sandler, R. S. (1994). The impact of message tailoring on dietary behavior change for disease prevention in primary care settings. American Journal of Public Health, 84, 783–787.
Center for Substance Abuse Prevention. (1994). Careful concept development paves the way to effective prevention materials [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). Conducting focus groups with young children requires special considerations and techniques [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). Following specific guidelines will help you assess cultural competence in program design, application, and management [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). A key step in developing prevention materials is to obtain expert and gatekeeper reviews [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). Pretesting is essential:You can choose from various methods [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). You can manage focus groups effectively for maximum impact [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). You can prepare easy-to-read materials [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). You can use communications principles to create culturally sensitive and effective prevention materials [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.
Centers for Disease Control and Prevention. (1994). Listening to your audience: Using focus groups to plan breast and cervical cancer public education programs (CDC Publication No. PDF-245K). Denver: AMC Cancer Research Center.
Centers for Disease Control and Prevention. (2000). Beyond the brochure (CDC Publication No. PDF-821K). Atlanta.
Debus, M. (1988). Methodological review: A handbook for excellence in focus group research. Washington, DC: Academy for Educational Development.
Goldberg, M. E., Fishbein, M. F., & Middlestadt, S. E. (Eds.). (1997). Social marketing: Theoretical and practical perspectives. Mahwah, NJ: Erlbaum.
Jones, J. P. (Ed.). (1998). How advertising works: The role of research. Thousand Oaks, CA: Sage.
Kotler, P., & Roberto, E. L. (1989). Social marketing: Strategies for changing public behavior. New York: Free Press.
Lipkus, I. M., Lyna, P. R., & Rimer, B. K. (1999). Using tailored interventions to enhance smoking cessation among African Americans at the community health center. Nicotine and Tobacco Research, 1(1), 77–85.
Maibach, E., & Parrott, R. L. (Eds.). (1995). Designing health messages: Approaches from communication theory and public health practice. Thousand Oaks, CA: Sage.
Merton, R. K. (1987). Focused interviews and focus groups: Continuities and discontinuities. Public Opinion Quarterly, 51, 550–556.
Merton, R. K., Riske, M., & Kendall, P. L. (1996). The focused interview (2nd ed.). New York: Free Press.
Morgan, D. L., & Krueger, R. A. (1998). The focus group kit. Thousand Oaks, CA: Sage.
National Cancer Institute. (1994). Clear and simple: Developing effective print materials for low-literate readers (NIH Publication No. 95-3594). Bethesda, MD.
National Cancer Institute. (1998). How the public perceives, processes, and interprets risk information: Findings from focus group research with the general public. Bethesda, MD: U.S. Department of Health and Human Services.
National Cancer Institute. (2000). Multiethnic focus groups to test motivational messages on mammography and breast cancer. Bethesda, MD.
Palmgreen, P., et al. (1995). Reaching at-risk populations in a mass media drug abuse prevention campaign: Sensation seeking as a targeting variable. Drugs and Society, 8(3), 29–45.
Rice, M., & Valdivia, L. (1991). A simple guide for design, use, and evaluation for education materials. Health Education Quarterly, 18(1), 79–85.
Rimer, B. K., & Glassman, B. (1998). Tailoring communications for primary care settings. Methods of Information in Medicine, 37(3), 171–177.
Selden, C. R., Zorn, M., Ratzan, S., & Parker, R. M. (2000). Health literacy, January 1990 through October 1999. Bethesda, MD: National Library of Medicine.
Siegel, M., & Doner, L. (1998). Marketing public health: Strategies to promote social change. Gaithersburg, MD: Aspen.
Strecher, V. J., Kreuter, M.W., DenBoer, C. H., Kobrin, S. C., Hospers, H. J., & Skinner, C. S. (1994). The effects of computer-tailored smoking cessation messages in family practice settings. Journal of Family Practice, 39, 262–270.
U.S. Department of Health and Human Services. (2000). Healthy people 2010 (2nd ed.; in two volumes: Understanding and improving health. and Objectives for improving health).Washington, DC: U.S. Government Printing Office.
Wells, W. D. (Ed.). (1997). Measuring advertising effectiveness. Mahwah, NJ: Erlbaum.
Stage 3: Implementing the Program
Questions to Ask and Answer
- How should we launch the program?
- Should we use a kickoff event?
- How should we develop and sustain media coverage? Partner involvement? Audience interest?
- How should we manage a press conference?
- How should we work with the media during a crisis?
- How can we ensure that our program operates according to plan?
- How can we use process evaluation?
- How can we find out whether we are reaching the intended audience with our information?
- How can we find out whether they are responding favorably to our message and materials?
- Are we maintaining good relationships with our partners?
Preparing to Implement Your Program
Before you launch the program, plan for distribution, promotion, and process evaluation. Make sure you also develop a launch plan, produce sufficient quantities of materials, and prepare your staff for the work ahead.
Program Launch
You may choose to launch your program quietly, starting activities on a limited basis in one geographic area or with just one partner to test the program. Using a limited approach will permit you to make adjustments before you fully commit your resources. This can be particularly useful for a large-scale program or for programs using a new technique or involving a new intended audience. Or, you may choose to launch with a kickoff event.
- Are our partners prepared for the launch?
- Have we invited reviewers, gatekeepers, and others who have been involved in program development?
- Have we prepared (or trained, if necessary) our staff and spokespeople?
- Are program-related services (e.g., a hotline, screening facilities) in place?
- Do we have a list of the media outlets we need to contact?
- Are all of our promotional materials ready?
- Do we have enough materials to start the program (e.g., PSAs and media kits) and respond to inquiries (e.g., leaflets for the public)?
- Are reordering mechanisms in place?
- Do we have mechanisms in place to track our program's progress and to identify potential problems?
- Are health professionals and other service providers in the community aware of our program and prepared to respond if their clients ask about it?
Kickoff Event
A kickoff event can create broader awareness of the program and promote community involvement. Kickoff events are an excellent way to develop relationships with people who may be willing to get involved in the program. Scheduling an event also creates a deadline, which will help your program avoid unnecessary lag time or protracted preparations.
To begin with a kickoff event, you might:
- Plan an event to celebrate the start of the program.
- Tie the kickoff to newsworthy happenings, such as the Great American Smokeout, Talk About Prescriptions Month, or the announcement of the results of a major study.
- Tie into community events, such as sporting events, church activities, shopping mall promotions, or holiday happenings.
- Work with partner organizations to fund events that the intended audience already participates in and that have broad media interest.
To enhance media coverage of your kickoff event:
- Create a news "hook" or angle that makes the event newsworthy
- Use a checklist to track preevent, event, and postevent activities. These may include room arrangements, speakers, expenditures, media kits, refreshments, transportation, equipment, and follow-up actions.
- Inform the media of your event in a timely way. Ask about their schedules, if possible, to avoid holding an event that conflicts with other media activities. Conflicts might prevent you from getting media coverage.
- Don't forget to include specialized media, such as community newspapers, cable TV stations, radio, health-related publications (the trade press), foreign-language publications or broadcast media, Internet "zines" and Web sites, and organization publications. These media may have a greater incentive to use your story than general newspapers or regular TV stations, and they can ensure an audience at a press conference if the mainstream media don't show up.
- Launch activities in multiple locales on the same date to make them more newsworthy.
- Create media kits to facilitate accurate reporting of the issue.
- Invite spokespeople who support your program and who may attract media interest.
- Hold the event in a location that is connected to your message and involves members of the intended audience. This might be a youth center for a program aimed at teenagers, a grocery store for a program about nutrition, or a neighborhood where screening will be offered. Make sure the location has sufficient space for the media and their equipment.
Holding a Press Conference
One effective kickoff event is holding a press conference.Your health communication program launch is unlikely to get much media attention if you just set up a press conference. News media have many opportunities to attend events and at the same time are finding it easier to get information electronically. This means that you must stand out to attract media to your event. Tying the program's launch to important health news can help. Such news could include announcing the results of a recent health study, releasing new statistics on your topic, or announcing the start of a comprehensive or multiorganization health program of which your program is a part. Even more attractive is announcing such news plus having representatives of the intended audience or other individuals tell compelling personal stories.
The following are tips for planning and conducting a successful press conference.
Be realistic about the media you invite. Local press people and those with whom you already have a relationship are more likely to attend than representatives from national newspapers or TV stations. Don't forget the health-related trade press, which often needs news and will help bolster attendance. Give reporters three to four days' notice. If yours is a major story, call wire services to have the event put on their daybooks. Remind the staff person responsible for contacting the media to call reporters the day before the event to pitch its importance.
Decide who will announce which aspects of your news. In general, select people for their recognizable names. If they are not familiar with your program, you can brief them, provide materials, and have knowledgeable people on hand to answer questions your main speakers cannot. Don't overlook the opportunity for personal testimony by patients, physicians, and family members. You may want to pretape their statements at their homes or in the hospital (you will usually need a written release from patients). Remember that all speakers, including patients, will need to be available during the day of the press conference to answer press inquiries and be interviewed.
The conventional wisdom is to hold press conferences between Tuesday mid-morning and Thursday afternoon. However, it is important to know the schedule of the media you've invited. Reporters from afternoon newspapers have morning deadlines and may not be able to attend a late morning or early afternoon conference. Try to limit your press conference to 15 minutes or less, including plenty of time for questions. Emphasize to speakers the need to keep their remarks short and simple, to avoid scientific jargon, and to focus on the action you want your intended audience to take.
Assign a staff person to arrange a suitable room and any equipment you may need, including a "mult box" that allows the media to connect their equipment to the public address system for better sound quality. The staff person should also arrange for parking, tell people where to park, and put up signs to help reporters find your room.You should also plan to have another person who has media experience on hand to distribute media kits, show media representatives where they can set up their equipment, answer questions, and point out who is available for follow-up interviews.
Media Kits
At a minimum, include:
- A press conference agenda
- A press release
- Local, state, and national statistics about your issue
- Background information
- Biographies of your speakers
- Reproducible copies of charts or graphs used in your speakers' presentations
- Copies of other program materials or reports
- Name and telephone number of someone who can be called to answer questions or verify information
To increase the usefulness of your kit to television reporters, include broadcast–quality stock video (B–roll). If you have invited non–English–language media representatives to attend, provide materials in their language (have the materials prepared and reviewed by people fluent in the language).
Follow Up
Deliver your new release or press kit in person to key reporters who didn't attend the press conference. Explain why your news is important.
Maximizing Media Coverage of Your Program
To maximize media coverage, be sure to:
- Know what different publications, stations, and shows typically cover, and which staff, editors, and reporters are responsible for what. Giving your story to the right outlets and the right people shows your understanding of their work and can improve the likelihood of coverage.
- Understand your media market. Some media, such as those in the Washington, DC, area, see themselves as providers of regional or national rather than local coverage, which makes it more difficult for local stories and issues to receive attention. In similar situations, pitch your story with a regional or national slant to increase the possibility of coverage.
- Respond quickly to requests for information. If you are able to give answers or statements within an hour of a request, media outlets will continue to call.
- Provide information the media can use.
- Be honest about your issue, your organization, what you know, what you can do for the media, and what you want from them. If you don't know the answer to a question they ask, tell them so and offer to get the answer quickly.
- Work personally with the media to help them understand your issue; just sending them news releases or PSAs is much less effective. Provide background information so that when a story breaks they have accurate facts on hand.
- Ask for something the media can give besides coverage. For instance, they may be able to provide data about their audience to help you decide which media to use, or help produce broadcast segments or PSAs, or cosponsor an event.
Maintaining Media Relations After Launch
Support your program's messages by encouraging media to cover your program after launch. To get continuing coverage of your program, you must develop an ongoing relationship with the media. These steps can help ensure continuing media coverage.
- Make a list of key media contacts, establish relationships with them early, and nurture the relationships throughout the life of your program. Many people can help you identify media contacts. Begin within your own organization. Ask whether staff or volunteers have media contacts or know media figures such as owners of newspapers or broadcast stations. Outside your organization, talk with partners; people you know at media outlets, public relations/advertising firms, and on the public relations staff of business firms; members of professional associations (such as chapters of the Public Relations Society of America); and people in public relations or marketing programs at local universities. Other sources include reference books in your local library that list local and national media contact information. Update your media lists regularly; using an incorrect name or title can mean the media won't cover your message.
- Develop a plan for periodic media coverage of your program and make your program newsworthy. Your plan should include your program's objectives, the messages you want to communicate to the media (including why your program or message deserves coverage), any promotional activities you plan to sponsor, and a schedule for media contact (when it will occur and who will initiate it).
- Identify and train media spokespeople. It is a good idea to select no more than three spokespeople. Be sure that all of them are providing the same information about your program by giving them written briefing points. The media usually prefer spokespersons with authority in your organization. The person who regularly handles media relations may not have that status. Some spokespersons will be savvy about working with the media and need only a briefing on your program. Others may need training on how to give interviews, respond to media queries during crisis or "bad news" situations, or how to be effective on TV or radio.
- Track media coverage. This includes coverage of issues generated by your media relations efforts as well as coverage that occurs independently. Monitoring all types of coverage can provide important process evaluation data. It will enable you to identify and take steps to correct misstatements and errors, determine the impact of your media activities and whether changes are needed, identify other media representatives interested in your issue, and find out whether your organization is being overlooked. Media coverage can be measured in terms of quantity (how much space did a story get and how often are stories published?); prominence (does it appear on the front page or not?); slant (is coverage positive or negative?); accuracy of content; and type of story (is the story an editorial or hard news?).
- Capitalize on breaking news. When something happens that is related to your program, call news outlets and offer them an expert opinion. If a negative event occurs, take the opportunity to explain how the changes advocated by your organization could help prevent similar problems in the future. For example, when the story about traces of poisonous substances in Chilean grapes received widespread coverage, tobacco control activists used the event to point out that larger amounts of those same substances are found in a single cigarette.
In 2001, the National Cancer Institute kicked off a large national clinical trial on prostate cancer using an extensive media launch, including national, local, and Spanish-language press releases as well as a professionally produced video news release (VNR). The launch employed extensive outreach to news outlets designed to make the media aware of the clinical trial. The hope was that these outlets would then report on the trial and that eligible participants would be encouraged to volunteer. The following process evaluation activities were used to measure the effectiveness of the media push in distributing the launch messages:
- Monitoring use of the VNR through Nielsen's SIGMA® encoding
- Tracking calls about the trial to the Cancer Information Service's toll-free information line (1-800-4-CANCER), which was promoted in the media materials
- Observing print coverage of the trial through LexisNexis
- Monitoring traffic to NCI Web sites describing the trial
- Surveying each study site about its specific media efforts
- Monitoring the total accrual rate to the trial
Findings showed that the full-fledged media push produced nearly 1,000 print and broadcast hits in the month following the launch and that the messages were successful in reaching the intended audiences. Men in the correct age range, for example, called the 1-800 information line in numbers far exceeding their usual number. The data also indicated that the media were effective in both reaching and motivating potential minority participants. Calls from minority men to the toll-free number were substantially higher than for previous trials and roughly equivalent to their proportion in the population.
What Do the Media Like?
- Stories with audience appeal
- Issues that stimulate debate, controversy, or conflict
- Stories that create higher ratings and larger audiences
- Fresh angles or twists on issues that will attract public interest
- Accurate background information
What Do the Media Dislike?
- Covering old topics
- Duplicating stories reported by competitors
- Reporting inaccuracies or an incomplete picture
- Receiving numerous calls when on a deadline
- People who persist when a story idea is rejected
- Organizations that believe their story is interesting simply because it is theirs or that convey the attitude that the importance of the story is obvious
Working with the Media During a Crisis Situation
If a crisis develops related to your health topic, your organization, or your program, the media will likely contact you. The following suggestions will help you work effectively with the media in these situations:
Be Prepared
- Identify a spokesperson to handle media inquiries.
- Train the spokesperson to handle routine inquiries, interviews, media appearances, and crises. A professional media consultant can help with this training.
Take Control
- Stay calm. Show your staff and the public that you are on top of the problem and are taking steps to resolve it.
- Respond quickly. Help reporters who call you meet their deadlines and call them if no one calls you.
- Tell the truth. Admitting mistakes and taking responsibility for them is important for your credibility. Crisis situations can work to your advantage by showing your ability to take charge under difficult circumstances.
- Be well informed. Get the facts you need to understand the situation and develop a response.When talking to reporters, focus on the main message you want to send.
- Track incoming calls. Keeping a record of who called, from where, why, and how the information you gave them will be used will give you a list of names to call if new information becomes available (and provide a good resource for the future).
- Say, "I don't know" when you cannot answer a reporter's question. Promise to get the answer quickly and follow through.
- Consider preparing a short statement with comments from your organization's leadership.
Managing Implementation: Monitoring and Problem Solving
Managing a health communication program is much like managing any other project. Key activities include monitoring activities, staff, and budget; problem solving; process evaluation; measuring intended audience satisfaction; and revising plans and operations.
Your communication plan should indicate how and when resources will be needed, when specific events will occur, and at what points you will assess your efforts. Once implementation is under way, however, contingencies may arise. Periodically, assess whether:
- Activities are being completed at scheduled times
- Your intended audiences are being reached
- Certain activities or materials are more successful than others
- Certain aspects of the program need to be altered or eliminated
- Your expenditures are within budget
You can often correct problems quickly if you can identify them. For example, if you ask the public to call you for more information, you should provide a simple form (electronic or manual) for telephone operators to use to record the questions asked and the answers given. Frequently review responses to identify inquiry patterns, to be sure that correct or adequate information is being given, and to find out whether more or different information may be needed.
Process Evaluation
Process evaluation takes place during implementation and monitors the functioning of program components. It includes assessment of whether messages are being delivered appropriately, effectively, and efficiently; whether materials are being distributed to the right people and in the right quantities; whether the intended program activities are occurring; and other measures of how well the program is working. Use process evaluation to track the following:
- The functioning and quality of your program
- Partner/coalition involvement
- The effectiveness of publicity, promotion, and other outreach efforts
- Media response
- intended audience participation, inquiries, and other responses
- Adherence to schedule
- Expenditures and adherence to budget
- Contractor activities:
- Are seasoned professionals doing the creative and managerial work?
- Is the contractor devoting enough time and money to your project?
- Are deadlines being met?
- Are performance and deliverables in keeping with the contract?
- Are the hours billed reasonable for the work performed?
- Are there problems in the relationship?
- Amount of time
given to your message by radio and television stations and what the estimated size/demographics of the audiences are - Print coverage and estimated readership
- Quantities of educational materials distributed
- Number of speeches and presentations given
- Number of special events
- Size of audiences at presentations and events
- Number of telephone, mail, and e-mail inquiries (how people heard of the program, what they asked)
- Number of people visiting Web sites or Internet services
- Number of organizations, businesses, or media outlets participating in the program
- Response to presentations (measured by completed participant feedback forms)
- Number of publications requested and distributed
- Demographic or other characteristics of the responding audience (to find out whether the intended audience responded)
The following are examples of ways to gather the information needed for process evaluation:
- Use activity tracking forms.
- Monitor the volume of public inquiries and requests for information.
- Ask callers what prompted their call.
- Use clipping services to gauge media coverage.
- Gather feedback cards from or make follow-up phone calls to television and radio stations.
- Review telephone responses for accuracy.
- Follow up with teachers, physicians, or other gatekeepers to check their preparedness and
interest. - Gather regular status reports from staff, contractors, and partners.
- Meet in person or by telephone with partners to review your program's progress.
- Track traffic to project Web sites.
- Review publication requests and distribution.
Measuring Audience Satisfaction
Audience satisfaction surveys are an important tool for both process and outcome evaluations of health communication programs. Use surveys to help you identify the following:
- The characteristics of those you reached—how well do they match the characteristics of your
intended audiences? - How the intended audience reacted to your materials and services—were materials easy to
understand? Useful? - How the intended audience used your materials—were materials read? Passed on to others?
Saved? Were events attended?
This information will help you determine whether you are reaching your intended audiences, whether your materials or activities need to be revised, and whether your materials are being used as you intended. Some managers also use these surveys to learn what information intended audiences would like to receive in the future. See Appendix A for an example of a user survey. The survey example, the Breast Cancer Risk Assessment Tool Evaluation from NCI, includes a summary of the responses as an example of the range of information you can glean from surveys like these.
Making Adjustments
The implementation stage will not always proceed as you expect. Materials may be delayed at the printer, a major news story may preempt your publicity (or focus additional attention on your issue), or a new priority may delay community participation. A periodic review of your planned tasks and time schedules will help you revise any plans that might be affected by unexpected events or delays. There is nothing wrong with altering your plans to fit a changed situation. In fact, you may risk damaging your chances of success if you are not willing to be flexible.
Maintaining Partnerships
If your organization has partnered with others, you will need to work to maintain a good relationship between your organization and your partners. Frequent two-way communication is essential to productive partnerships. If partners hear from you only when you need something or you hear from partners only if problems arise, the relationship will suffer.
To keep partners involved:
- Periodically call to find out how your partners' work is progressing. Offer to help when appropriate, congratulate them on their accomplishments, and show an interest in them that mirrors the interest you hope they take in your program.
- Involve them whenever it is reasonable (and they are interested) in your activities, such as special events or process evaluation.
- Give them regular updates on the program. Some organizations formalize this task by sending out newsletters or reports. Others handle it informally through calls, meetings, or letters. Tell partners about any changes in program activities that may impact their organization.
- Give them credit in your news releases and other publicity. If you generate a story that mentions them, send them a copy.
- Share new materials and information (e.g., about breaking stories relevant to their organization).
- Notify them of program results, whether positive or negative.
- Share feedback from your process evaluation.
- Explore opportunities for further collaboration.
Continue to Consider New Opportunities For Partnering
You may have planned to use one or several media, interpersonal, organizational, and community channels for your program. Before you implemented your program, you likely partnered with appropriate organizations. Once your program is under way, however, other organizations may want to get involved, or you may find that you need a new partner to reach a certain segment of your intended audience. Partners that have fulfilled objectives may step aside.
Common Myths and Misconceptions About Program Implementation
Myth: People need the information we are providing, so we will have a large number of requests for our materials.
Fact: "If we print it they will come" holds true only if you are printing money. For most programs, effective promotion is critical to getting materials into the hands of those who need them. Disseminating printed products is as challenging, and as important, as developing them.
Myth: Working with the media is more trouble than it is worth. They take statements out of context, don't bother to check facts, and care only about sensationalism.
Fact: Although is possible to have bad experiences with the media, if you pay attention to your work with them, you can help foster positive outcomes. Learn which reporters are responsible journalists and develop relationships with them. Then provide accurate background materials and offer to check their stories for technical correctness. Respect their need for stories of interest to the public and try to help them by thinking of positive, attention-getting angles for your program. If your topic is controversial, you may want to provide media training for your spokespeople to help avoid giving statements that could be misinterpreted or taken out of context.Work with the media to correct inaccuracies.
Myth: When the press conference is over, we can relax.
Fact: The longevity of news can be measured in days—if not minutes. Single channels or even the entire mass media are unlikely to reach all intended audiences, and one event or activity is not enough. Just as health messages need repetition and reinforcement, so do promotions.Your job isn't over until the warehouse is empty (unless you create enough demand for a second printing) or until you have met all of your objectives.
Myth: We have partners, so we don't have to worry as much about implementation. They'll do a lot of the work for us.
Fact: Partners can help with implementation, but they are unlikely to reduce your workload. Developing and maintaining partnerships is itself very labor intensive, and your role in leading, coordinating, and monitoring program operations is essential.
Myth: If a program is not an immediate success, it is a failure.
Fact: All change takes time. Even the best–planned health communication programs have to make midcourse corrections. The information you get from process evaluation can help you recognize where improvements are needed and get back on track. Each time you learn more about your intended audience, your own processes, and the barriers you face, you increase your likelihood of success.
Myth: It is unseemly for a government/nonprofit health program to spend money to promote itself.
Fact: By promoting your program, you are promoting your issue and your message. The more people who know about what you have to offer, the more the community will benefit. Having your organization recognized lends credibility to your program and will help you recruit supporters, partners, volunteers, and funding sources when you need them. As long as you keep the focus on your communication objectives and not on your office (or yourself!), program promotion is a legitimate, integral aspect of health communication.
Selected Readings
Agency for International Development, Academy for Educational Development. (1992). Results and realities: A decade of experience in communication for child survival. Washington, DC.
Arkin, E. B., Romano, R. M., Van Nevel, J. P., & McKenna, W. (1993). Effect of the mass media in promoting calls to the Cancer Information Service. In The Cancer Information Service: A 15-year history of service and research (Monograph of the Journal of the National Cancer Institute, No. 14).
Center for Substance Abuse Prevention. (1994). Following specific guidelines will help you assess cultural competence in program design, application, and management. [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). You can increase your media coverage [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Dignan, M. B., & Carr, P. A. (1992). Program planning for health education and health promotion (2nd ed.). Philadelphia: Lea & Febiger.
Jernigan, D. H., & Wright, P. A. (1996). Media advocacy: Lessons from community experiences. Journal of Public Health Policy, 17, 306–330.
Wallach, L., Dorfman, L., Jernigan, D., & Themba, M. (1993). Media advocacy and public health: Power of prevention. Thousand Oaks, CA: Sage.
Wallach, L., Woodruff, K., Dorfman, L., & Diaz, I. (1999). News for a change: An advocate's guide to working with the media. Thousand Oaks, CA: Sage.
Stage 4: Assessing Effectiveness and Making Refinements
Questions to Ask and Answer
- How can we use outcome evaluation to assess the effectiveness of our program?
- How do we decide what outcome evaluation methods to use?
- How should we use our evaluation results?<</li>
- How can we determine to what degree we have achieved our communication objectives?
- How can we make our communication program more effective?/li>
In Stage 3, you decided how to use process evaluation to monitor and adjust your communication activities to meet objectives. In Stage 4, you will use the outcome evaluation plan developed in Stage 1 to identify what changes (e.g., in knowledge, attitudes, or behavior) did or did not occur as a result of the program. Together, the progress and outcome evaluations will tell you how the program is functioning and why. (If you combine information from the two types of evaluation, be sure that you focus on the same aspects of the program, even though you look at them from different perspectives.) This section will help you revise your plans and conduct outcome evaluation.You should begin planning assessment activities either before or soon after you launch the program.
Why Outcome Evaluation Is Important
Outcome evaluation is important because it shows how well the program has met its communication objectives and what you might change or improve to make it more effective. Learning how well the program has met its communication objectives is vital for:
- Justifying the program to management
- Providing evidence of success or the need for additional resources
- Increasing organizational understanding of and support for health communication
- Encouraging ongoing cooperative ventures with other organizations
Revising the Outcome Evaluation Plan
During Stage 1, you identified evaluation methods and drafted an outcome evaluation plan. At that time, you should have collected any necessary baseline data. The first step in Stage 4 is to review that plan to ensure it still fits your program. A number of factors will influence how your communication program's outcomes should be evaluated, including the type of communication program, the communication objectives, budget, and timing. The outcome evaluation needs to capture intermediate outcomes and to measure the outcomes specified in the communication objectives. Doing so can allow you to show progress toward the objectives even if the objectives are not met.
Examples of Effectiveness Measures for Health Communication Programs
A public survey conducted before and after NCI's 5 A Day campaign found that knowledge of the message (a person should eat 5 or more servings of fruits and vegetables each day for good health) increased by 27 percentage points.
In 1988, the U.S. Surgeon General sent a pamphlet designed to influence attitudes on AIDS to every U.S. household. An evaluation conducted in Connecticut showed no change in attitude between residents who read the pamphlet and those who did not.
The Pawtucket Heart Health Program evaluated a weight-loss awareness program conducted at worksites. More than 600 people enrolled, and they lost an average of 3.5 pounds each compared with their preprogram weight.
Consider the following questions to assess the Stage 1 outcome evaluation plan and to be sure the evaluation will give you the information you need:
- What are the communication objectives?
What should the members of the intended audience think, feel, or do as a result of the health communication plan in contrast to what they thought, felt, or did before? How can these changes be measured? - How do you expect change to occur?
Will it be slow or rapid? What measurable intermediate outcomes (steps toward the desired behavior) are likely to take place before the behavior change can occur? The behavior change map you created in Stage 1 should provide the answers to these questions. - How long will the program last? What kinds of changes can we expect in that time period (e.g., attitudinal, awareness, behavior, policy changes)? Sometimes, programs will not be in place long enough for objectives to be met when outcomes are measured (e.g., outcomes measured yearly over a 5-year program). To help ensure that you identify important indicators of change, decide which changes could reasonably occur from year to year.
- Which outcome evaluation methods can capture the scope of the change that is likely to occur?
Many outcome evaluation measures are relatively crude, which means that a large percentage of the intended audience (sometimes an unrealistically large percentage) must make a change before it can be measured. If this is the case, the evaluation is said to "lack statistical power." For example, a public survey of 1,000 people has a margin of error of about 3 percent. In other words, if 50 percent of the survey respondents said they engage in a particular behavior, in all likelihood somewhere between 47 percent and 53 percent of the population represented by the respondents actually engages in the behavior. Therefore, you can conclude that a statistically significant change has occurred only if there is a change of 5 or more percentage points. It may be unreasonable to expect such a large change, and budgetary constraints may force you to measure outcomes by surveying the general population when your intended audience is only a small proportion of the population. - Which aspects of the outcome evaluation plan best fit with your organization's priorities?
Only rarely does a communication program have adequate resources to evaluate all activities. You may have to illustrate your program's contribution to organizational priorities to ensure continued funding. If this is the case, it may be wise to evaluate those aspects most likely to contribute to the organization's mission (assuming that those are also the ones most likely to result in measurable changes).
Quantitative research is used to gather objective information by asking a large number of people a set of identical questions. Results are expressed in numerical terms (e.g., 35 percent are aware of X and 65 percent are not). If the respondents are a representative random sample, quantitative data can be used to draw conclusions about an intended audience as a whole. Quantitative research is useful for measuring the extent to which a knowledge set, attitude, or behavior is prevalent in an intended audience.
Qualitative research is used to gather reactions and impressions from small numbers of intended audience members, usually by engaging them in discussion. Results are subjective and are not described numerically or used to make generalizations about the intended audience. Qualitative research is useful for understanding why people react the way they do and for understanding additional ideas, issues, and concerns.
Quantitative research methods are usually used for outcome evaluation because they provide the numerical data necessary to assess progress toward objectives. When evaluating outcomes, qualitative research methods are used to help interpret quantitative data and shed light on why particular outcomes were (or were not) achieved. See the Communication Research Methods section for detailed explanations of quantitative and qualitative research methods and the circumstances under which you should use each.
Conducting Outcome Evaluation
Conduct outcome evaluation by following these steps:
- Determine what information the evaluation must provide.
- Define the data to collect.
- Decide on data collection methods.
- Develop and pretest data collection instruments.
- Collect data.
- Process data.
- Analyze data to answer the evaluation questions.
- Write an evaluation report.
- Disseminate the evaluation report.
Evaluation Constraints
Every program planner faces limitations when conducting an outcome evaluation. You may need to adjust your evaluation to accommodate constraints such as the following:
- Limited funds
- Limited staff time or expertise
- Length of time allotted to the program and its evaluation
- Organizational restrictions on hiring consultants or contractors
- Policies that limit your ability to collect information from the public
- Difficulty in defining the program's objectives or in establishing consensus on them
- Difficulty in isolating program effects from other influences on the intended audience in "real world" situations
- Management perceptions of the evaluation's value
These constraints may make the ideal evaluation impossible. If you must compromise your evaluation's design, data collection, or analysis to fit limitations, decide whether the compromises will make the evaluation results invalid. If your program faces severe constraints, do a small-scale evaluation well rather than a large-scale evaluation poorly. Realize that it is not sensible to conduct an evaluation if it is not powerful enough to detect a statistically significant change.
See a description of each step below.
1. Determine What Information the Evaluation Must Provide
An easy way to do this is to think about the decisions you will make based on the evaluation report. What questions do you need to answer to make those decisions?
2. Define the Data You Need to Collect
Determine what you can and should measure to assess progress on meeting objectives. Use the following questions as a guide:
- Did knowledge of the issue increase among the intended audience (e.g., understanding how to choose foods low in fat or high in fiber, knowing reasons not to smoke)?
- Did behavioral intentions of the intended audience change (e.g., intending to use a peer pressure resistance skill, intending to buy more vegetables)?
- Did intended audience members take steps leading to the behavior change (e.g., purchasing a sunscreen, calling for health information, signing up for an exercise class)?
- Did awareness of the campaign message, name, or logo increase among intended audience members?
- Were policies initiated or other institutional actions taken (e.g., putting healthy snacks in vending machines, improving school nutrition curricula)?
3. Decide on Data Collection Methods
The sidebar Outcome Evaluation Designs describes some common outcome evaluation designs, the situations in which they are appropriate, and their major limitations. (See the Communication Research Methods section for more information.) Complex, multifaceted programs often employ a range of methods so that each activity is evaluated appropriately. For example, a program that includes a mass media component to reach parents and a school-based component to reach students might use independent cross-sectional studies to evaluate the mass media component and a randomized or quasi-experimental design to evaluate the school-based component.
The following limitations can make evaluation of your communication program difficult:
- Lack of measurement precision (e.g., available data collection mechanisms cannot adequately capture
change or cannot capture small changes). Population surveys may not be able to identify the small number of people making a change. Self-reported measures of behavior change may not be accurate. - Inability to conclusively establish that the communication activity caused the observed effect.
Experimental designs, in which people are randomly assigned to either receive an intervention or not, allow you to assume that your program causes the only differences observed between the group exposed to the program and the control group. Outcome evaluations with experimental designs that run more than a few weeks, however, often wind up with contaminated control groups, either because people in the group receiving the intervention move to the control group, or because people in the control group receive messages from another source that are the same as or similar to those from your program.
The more complex your evaluation design is, the more you will need expert assistance to conduct your evaluation and interpret your results. The expert can also help you write questions that produce objective results. (It's easy to develop questions that inadvertently produce overly positive results.) If you do not have an evaluator on staff, seek help to decide what type of evaluation will best serve your program. Sources include university faculty and graduate students (for data collection and analysis), local businesses (for staff and computer time), state and local health agencies, and consultants and organizations with evaluation expertise.
Outcome Evaluation Designs Appropriate for Specific Communication Programs Programs Not Delivered to the Entire Population of the Intended Audience | |
Evaluation Design | Major Limitations |
Randomized experiment. Members of the intended audience are randomly assigned to either be exposed to the program (intervention group) or not (control group). Usually, the same series of questions is asked pre- and postintervention (a pretest and posttest); posttest differences between the two groups show change the program has caused. |
Quasi-experiment. Members of the intended audience are split into control and intervention groups based simply upon who is exposed to the program and who is not. |
Before-and-after studies. Information is collected before and after intervention from the same members of the intended audience to identify change from one time to another. |
Independent cross-sectional studies. Information is collected before and after intervention, but it is collected from different intended audience members each time. |
Panel studies. Information is collected at multiple times from the same members of the intended audience. When intended audience members are differentially exposed to the program, this design helps evaluators sort out the effects of different aspects of the program or different levels of exposure. |
Time series analysis. Pre- and postintervention measures are collected multiple times from members of the intended audience. Evaluators use the preintervention data points to project what would have happened without the intervention and then compare the projection to what did happen using the postintervention data points. |
NCI's Cancer Information Service
Customer satisfaction surveys are one means of gathering data about a program's effects. Surveys of telephone callers to NCI's Cancer Information Service (CIS) have shown that:
- Eight out of 10 callers say they take positive steps to improve their health after talking with CIS staff.
- Seventy percent of those who call about symptoms say the CIS information was helpful in their decision to see a doctor.
- Fifty-five percent of those who call about treatment say they use CIS information to make a treatment decision.
- Two-thirds of callers who are considering participation in a research study talk with a doctor after calling the CIS.
The Right Turns Only Program
Right Turns Only is a video-based drug education series produced by the Prince George's County, Maryland, school system. The effects of this series (including collateral print material) on student knowledge, attitudes, and behavioral intentions were tested among approximately 1,000 seventh grade students.
Twelve schools were assigned to one of four groups: three intervention groups and one control group. One intervention group received only the video-based education, a second received both the video-based and a traditional drug education curriculum, a third received only the traditional curriculum, and the control group received no drug abuse prevention education. All interventions were completed within a 3-week period.
The six outcomes measured included: 1) knowledge of substance abuse terminology, 2) ability to assess advertisements critically, 3) perception of family, 4) conflict resolution, 5) self-efficacy in peer relationships, and 6) behavioral intentions related to substance use/abuse prevention.
Changes were measured using data from questionnaires completed by students before and after the interventions. The data were analyzed to identify differences based on gender, race, grades (self-reported), and teacher. Groups that received drug education scored higher than the control group on all posttest measures except self-efficacy. On two of the six measures, the group receiving the combination of the video series and traditional curriculum scored significantly higher than other groups.
The evaluation demonstrated that instructional videos (particularly when used in conjunction with print materials and teacher guidance) could be an effective tool for delivering drug education in the classroom.
In 1995 the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) developed a feedback mechanism for the promotion of its kit, "Feet Can Last a Lifetime," that was designed to reduce the number of lower extremity amputations in people with diabetes. The first printing of the kit included a feedback form for health care providers to comment on the materials. Based on the feedback, NIDDK revised the kit in 1997. The new kit's contents were then pretested extensively with practitioners for technical accuracy, usefulness, and clarity. The original kit was developed primarily for providers; based upon evaluation results, the revised kit also includes materials for patients. These include an easy-to-read brochure; a fact sheet with "foot care tips" and a "to do" list that contains steps for patients to follow to take care of their feet; and camera-ready, laminated tip sheets for providers to reproduce and give to patients.
4. Develop and Pretest Data Collection Instruments
Most outcome evaluation methods involve collecting data about participants through observation, a questionnaire, or another method. Instruments may include tally sheets for counting public inquiries, survey questionnaires, interview guides. Select a method that allows you to best answer your evaluation questions based upon your access to your intended audience and your resources. To develop your data collection instruments—or to select and adapt existing ones—ask yourself the following questions:
Which Data?
The data you collect should be directly related to your evaluation questions. Although this seems obvious, it is important to check your data collection instruments against the questions your evaluation must answer. These checks will keep you focused on the information you need to know and ensure that you include the right measures. For example, if members of your intended audience must know more about a topic before behavior change can take place, make sure you ask knowledge-related questions in your evaluation.
From Whom?
You will need to decide how many members of each group you need data from in order to have a sufficiently powerful evaluation to assess change. Make sure you have adequate resources to collect information from that many people. Realize that you may also need a variety of data collection instruments and methods for the different groups from whom you need information.
Before you decide how to collect your data, you must assess your resources. Do you have access to, or can you train, skilled interviewers? Must you rely on self-reports from participants?
Also consider how comfortable the participants will be with the methods you choose to collect data. Will they be willing and able to fill out forms? Will they be willing to provide personal information to interviewers? Will the interviews and responses need to be translated?
When you evaluate communication programs, you form a set of assumptions about what should happen, to whom, and with what results. Recognize that these assumptions and expectations may vary, depending on the cultural norms and values of your intended audiences.
You may need to vary your methods of gathering information and interpreting results. Depending on the culture from which you are gathering information, people may react in different ways:
- They may think it is inappropriate to speak out in a group, such as a focus group, or to provide negative answers. (This does not mean that you should not use focus groups within these cultures; observance of nonverbal cues may be more revealing than oral communication.)
- They may be reluctant to provide information to a person from a different culture or over the telephone.
- They may lack familiarity with printed questionnaires or have a limited ability to read English.
Remember that the culture of the evaluator your program uses can inadvertently affect the objectivity of your evaluation. When possible, try to use culturally competent evaluators when you examine program activities. If your program cuts across cultures and you adapt your evaluation methods to fit different groups, you may find it difficult to compare results across groups. This type of evaluation is more complicated, and if you plan to conduct one, enroll the help of an expert evaluator.
5. Collect Data
Collect postprogram data. You should have collected baseline data during planning in Stage 1, before your program began, to use for comparison with postprogram data.
6. Process Data
Put the data into usable form for analysis. This may mean organizing the data to give to professional evaluators or entering the data into an evaluation software package.
7. Analyze the Data to Answer the Evaluation Questions
Use statistical techniques as appropriate to discover significant relationships.Your program might consider involving university–based evaluators, providing them with an opportunity for publication and your program with expertise.
8. Write an Evaluation Report
A report outlining what you did and why you did it, as well as what worked and what should be altered in the future, provides a solid base from which to plan future evaluations.Your program evaluation report explains how your program was effective in achieving its communication objectives and serves as a record of what you learned from both your program's achievements and shortcomings. Be sure to include any questionnaires or other instruments in the report so that you can find them later.
See Appendix A for a sample evaluation report. As you prepare your report, you will need someone with appropriate statistical expertise to analyze the outcome evaluation data. Also be sure to work closely with your evaluators to interpret the data and develop recommendations based on them.
Writing an evaluation report will bring your organization the following additional benefits:
- You will be able to apply what you've learned to future projects. Frequently, other programs are getting under way when evaluation of an earlier effort concludes, and program planners don't have time to digest what has been learned and incorporate it into future projects. A program evaluation report helps to ensure that what has been learned will get careful consideration.
- You will show your accountability to employers, partners, and funding agencies. Your program's evaluation report showcases the program's accomplishments. Even if some aspects of the program need to be modified based on evaluation results, identifying problems and addressing them shows partners and funding agencies that you are focused on results and intend to get the most benefit from their time and money.
- You will be able to give evidence of your program and methods' effectiveness. If you want other organizations to use your materials or program, you need to demonstrate their value. An evaluation report offers proof that the materials and your program were carefully developed and tested. This evidence will help you explain why your materials or program may be better than others, or what benefits an organization could gain from using its time and resources to implement your program.
- You will provide a formal record that will help others. A comprehensive evaluation report captures the institutional memory of what was tried in the past and why, which partners had strong skills or experience in specific areas, and what problems were encountered. Everything you learned when evaluating your program will be helpful to you or others planning programs in the future.
NCI's CIS used an evaluation report, "Making a Difference," to show its partners, the research community, NCI/CIS leadership, and the media that its programs are effective. The document both quantified CIS results (e.g., making 100,000 referrals a year to research studies, providing information on breast cancer to 76,000 callers in 1996, providing information that increased fruit and vegetable consumption among callers) and put a human face on the calling public. Quotations from callers and leaders in the cancer community illustrated the personal impact of the service on people's lives and health.
The report was written in lay language and used pullouts and simple charts to explain statistics. Ideas for using the report with regional partners, the media, and community leaders were included with the copies sent to each CIS office. To maximize opportunities for using the report, CIS has also made it available on computer disk and as a PowerPoint® slide presentation.
Consider the Users
Before you write your evaluation, consider who will read or use it. Write your report for that audience. As you did when planning your program components in Stage 1, analyze your audiences for your report before you begin to compose. To analyze your audience, ask yourself the following questions:- Who are the audiences for this evaluation report?
- Public health program administrators
- Evaluators, epidemiologists, researchers
- Funding agencies
- Policymakers
- Partner organizations
- Project staff
- The public
- The media
- How much information will your audience want?
- The complete report
- An executive summary
- Selected sections of the report
- How will your audience use the information in your report?
- To refine a program or policy
- To evaluate your program's performance
- To inform others
- To support advocacy efforts
- To plan future programs
Consider the Format
Decide the most appropriate way to present information in the report to your audience. Consider the following formats:
- Concise, including hard-hitting findings and recommendations
- General, including an overview written for the public at the ninth-grade level
- Scientific, including a methodology section, detailed discussion, and references
- Visual, including more charts and graphics than words
- Case studies, including other storytelling methods
Selected Elements to Include
Depending on your chosen audience and format, include the following sections:- Program results/findings
- Evaluation methods
- Program chronology/history
- Theoretical basis for program
- Implications
- Recommendations
- Barriers, reasons for unmet objectives
9. Disseminate the Evaluation Report
Ask selected stakeholders and key individuals to review the evaluation report before it is released so that they can identify concerns that might compromise its impact. When the report is ready for release, consider developing a dissemination strategy for the report, just as you did for your program products, so the intended audiences you've chosen will read it. Don't go to the hard work of writing the report only to file it away.
Letting others know about the program results and continuing needs may prompt them to share similar experiences, lessons, new ideas, or potential resources that you could use to refine the program. In fact, feedback from those who have read the evaluation report or learned about your findings through conference presentations or journal coverage can be valuable for refining the program and developing new programs.You may want to develop a formal mechanism for obtaining feedback from peer or partner audiences. If you use university–based evaluators, the mechanism may be their publication of findings.
If appropriate, use the evaluation report to get recognition of the program's accomplishments. Health communication programs can enhance their credibility with employers, funding agencies, partners, and the community by receiving awards from groups that recognize health programs, such as the American Medical Writers Association, the Society for Technical Communication, the American Public Health Association, and the National Association of Government Communicators. A variety of other opportunities exist, such as topic–specific awards (e.g., awards for consumer information on medications from the U.S. Food and Drug Administration) and awards for specific types of products (e.g., the International Communication Association's awards for the top three papers of the year). Another way to get recognition is to publish articles about the program in professional journals or give a presentation or workshop at an organization meeting or conference.
Refining Your Health Communication Program
The health communication planning process is circular. The end of Stage 4 is not the end of the process but the step that takes you back to Stage 1. Review the evaluation report and consider the following to help you identify areas of the program that should be changed, deleted, or augmented:
- Goals and objectives:
- Have your goals and objectives shifted as you've conducted the program? If so, revise the original goals and objectives to meet the new situation.
- Are there objectives the program is not meeting? Why? What are the barriers you're encountering?
- Has the program met all of your objectives, or does it seem not to be working at all? Consider ending the program.
- Where additional effort may be needed:
- Is there new health information that should be incorporated into the program's messages or design?
- Are there strategies or activities that did not succeed? Review why they didn't work and determine what can be done to correct any problems.
- Implications of success:
- Which objectives have been met, and by what successful activities?
- Should successful communication activities be continued and strengthened because they appear to work well or should they be considered successful and completed?
- Can successful communication activities be expanded to apply to other audiences or situations?
- Costs and results of different activities:
- What were the costs (including staff time) and results of different aspects of the program?
- Do some activities appear to work as well as, but cost less than, others?
- Accountability:
- Is there evidence of program effectiveness and of a continued need to persuade your organization to continue the program?
- Have you shared the results of your activities with the leadership of your organization?
- Have you shared results with partners?
- Do the assessment results show a need for new activities that would require partnerships with additional organizations?
Once you have answered the questions above and decided what needs to be done to improve the program, use the planning guidelines in Stage 1 to help determine new strategies, define expanded or different intended audiences, and rewrite/revise your communication program plan to accommodate new approaches, new tasks, and new timelines. Review information from the other stages as you plan the next phase of program activities.
Common Myths and Misconceptions About Evaluation
Myth: We can't afford an evaluation.
Fact: Rarely does anyone have access to adequate resources for an ideal health communication program, much less an ideal evaluation. Nevertheless, including evaluation as a part of your work yields the practical benefit of telling you how well your program is working and what needs to be changed. With a little creative thinking, some form of useful evaluation can be included in almost any budget.
Myth: Evaluation is too complicated. No one here knows how to do it.
Fact: Many sources of help are available for designing an evaluation. Several pertinent texts are included in the selected readings at the end of this section. If your organization does not have employees with the necessary skills, find help at a nearby university or from someone related to your program (e.g., a board member, a volunteer, or someone from a partner organization). Also, contact an appropriate clearinghouse or Federal agency and ask for evaluation reports on similar programs to use as models. If the program has enough resources, hire a consultant with experience in health communication evaluation. Contact other communication program managers for recommendations.
Myth: Evaluation takes too long.
Fact: Although large, complicated outcome evaluation studies take time to design and analyze (and may require a sufficient time lapse for changes in attitudes or behavior to become clear), other types of evaluation can be conducted in a few weeks or months, or even as little as a day. A well-planned evaluation can proceed in tandem with program development and implementation activities. Often, evaluation seems excessively time-consuming only because it is left until the end of the program.
Myth: Program evaluation is too risky. What if it shows our funding source (or boss) that we haven't succeeded?
Fact: A greater problem is having no results at all. A well-designed evaluation will help you measure and understand the results (e.g., if an attitude or a perception did not change, why not?). This information can direct future initiatives and help the public health community learn more about how to communicate effectively. The report should focus on what you have learned from completing the program evaluation.
Myth: We affected only 30 percent of our intended audience. Our program is a failure.
Fact: Affecting 30 percent of the intended audience is a major accomplishment; it looks like a failure only if your program's objectives were set unrealistically high. Remember to report your results in the context of what health communication programs can be expected to accomplish. If you think the program has affected a smaller proportion of the intended audience than you wanted, consult with experts (program planning, communication, or behavioral) before setting objectives for future programs.
Myth: If our program is working, we should see results very soon.
Fact: Results will vary depending on the program, the issue, and the intended audience. Don't expect instant results; creating and sustaining change in attitudes and particularly in behavior or behavioral intentions often takes time and commitment. Your program may show shorter term, activity-related results when you conduct your process evaluation; these changes in knowledge, information seeking, and skills may occur sooner than more complex behavioral changes.
Selected Readings
Academy for Educational Development. (1995). A tool box for building health communication capacity. Washington, DC.
Agency for Toxic Substances and Disease Registry. (1994). Guidelines for planning and evaluating environmental health education programs. Atlanta.
Center for Substance Abuse Prevention. (1998). Evaluating the results of communication programs [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.
Flay, B. R., & Cook, T. D. (1989). Three models for evaluating prevention campaigns with a mass media component. In R. E. Rice & C. K. Atkin (Eds.), Public communication campaigns (2nd ed.). Thousand Oaks, CA: Sage.
Flay, B. R., Kessler, R. C., & Utts, J. M. (1991). Evaluating media campaigns. In S. L. Coyle, R. F. Boruch, & C. F. Turner (Eds.), Evaluating AIDS prevention programs. Washington, DC: National Academy Press.
Morra, M. E. (Ed.). (1998). The impact and value of the Cancer Information Service: A model for health communication. Journal of Health Communication, 3(3) Suppl.
Muraskin, L. D. (1993). Understanding evaluation: The way to better prevention programs. Washington, DC: U.S. Department of Education.
Rice, R. E., & Atkin, C. K. (2000). Public communication campaigns (3rd ed.). Thousand Oaks, CA: Sage.
Rossi, P. H., Freeman, H. E., & Lipsey, M.W. (1998). Evaluation: A systematic approach (6th ed.). Thousand Oaks, CA: Sage.
Siegel, M., & Doner, L. (1998). Marketing public health: Strategies to promote social change. Gaithersburg, MD: Aspen.
Windsor, R.W., Baranowski, T. B., Clark, N. C., & Cutter, G. C. (1994). Evaluation of health promotion, health education and disease prevention programs (2nd ed.). Mountain View, CA: Mayfield.
Communication Research Methods
Types of Communication Research
Research into intended audiences' culture, lifestyle, behaviors and motivations, interests, and needs is a key component to a health communication program's success. This section describes communication research methods commonly used throughout program planning. See the chart Types of Research and Evaluation for more detail about research conducted in each of the stages of health communication program planning.
Most programs use more than one research method. For example, conducting exploratory focus groups with an intended audience at the start of program planning can orient you to the types of approaches, messages, and channels that are most likely to be successful with a particular group. In some cases, focus groups might be augmented with in-depth interviews to learn more about intended audience members' motivations. Later, messages and materials might be pretested approximate how an individual would encounter them in "real life." Theater-style testing also approximates reality, using a simulated television-viewing environment. Clearly, some methods are better suited to specific purposes than others. Using multiple methods can help ensure that you get an accurate picture of your intended audience members and their likely responses to your program.
Differences Between Qualitative and Quantitative Research Methods
There are two basic types of research you might conduct with intended audiences: qualitative and quantitative. You will use methods from one of these two types depending upon what you want to learn. See the sidebar below, Qualitative Versus Quantitative Methods, for common distinctions between qualitative and quantitative research.
- Provides depth of understanding
- Asks "Why?"
- Studies motivations
- Is subjective; probes individual reactions to discover underlying motivations
- Enables discovery
- Is exploratory
- Allows insights into behavior and trends
- Interprets
- Measures level of occurrence
- Asks "How many?" and "How often?"
- Studies actions
- Is objective; asks questions without revealing a point of view
- Provides proof
- Is definitive
- Measures levels of actions and trends
- Describes
In this section, you will learn when to use each type of research, how to conduct research with members of your intended audience, and how you can use the data you collect to inform your project. Qualitative, quasi-quantitative, and quantitative research methods are discussed separately.
Qualitative Research
Use qualitative research methods when:
- You are planning a communication program and developing materials for it
- When the goal of your research is to explore a topic or idea
- When the goal of your research is to gain insights into an intended audience's lifestyle, culture, motivations, behaviors, and preferences
Conduct qualitative research by:
- Selecting a small group of people chosen for particular characteristics
- Convening a discussion (i.e., a focus group or in-depth interview) or observing individuals' behaviors in schools, malls, supermarkets, etc.
- Keeping the discussion fairly unstructured, so that participants are free to make any response and are not required to choose from a list of possible responses
- Choosing which question to ask next based on your participants' previous responses
Qualitative research results cannot be:
- Quantified or subjected to statistical analysis
- Projected to the population from which the respondents were drawn because participants are not selected randomly (to be representative of the population as a whole) and because not all participants are asked precisely the same questions
Quantitative Research
Use quantitative research methods when:
- You are planning a communication program (e.g., to measure the prevalence of a particular behavior) or assessing a program already in place
- The goal of your research is measurement of particular variables Conduct quantitative research by:
- Selecting a large group or groups of people
- Using a structured questionnaire containing predominantly forced-choice or closed-ended questions
Quantitative research results can be:
- Analyzed using statistical techniques
- Considered representative of the population from which the respondents were drawn if each person in the population had an equal chance of being included
Qualitative Research Methods
Use qualitative research methods during the following parts of your program:
- Stage 1—to find out more about your intended audiences and to learn what the priorities and approach should be for trying to influence their awareness, knowledge, attitudes, intentions, or behaviors
- Stage 2—to determine whether your materials communicate the intended messages effectively and persuasively
- Stage 3—to understand why the program is or isn't working as expected
- Stage 4—to learn more about what worked and what didn't, and why certain outcomes occurred
Types of Research and Evaluation | ||
Stage | Type of Research/Evaluation | Benefits |
Stage 1 | Consumer Research, Market Research | |
Planning and Strategy Development | Provides information on the problem, intended audiences, and barriers to and opportunities for change | Answers questions such as: What dimensions of the health problem do we need to address? How should the population be segmented and which intended audience should be targeted? What are the best ways of reaching the intended audience? What benefits would be credible and appealing to the intended audience? What images should be conveyed? What barriers need to be overcome? What actions can the intended audience take? |
Stage 2 | Pretesting | |
Developing and Pretesting Concepts, Messages, and Materials | Assesses reactions to proposed messages or materials | Tests and refines messages and materials prior to production |
Pilot or Field Testing | ||
Assesses program activities in limited areas and/or time periods | Tests and refines activities prior to full-scale implementation | |
Stage 3 | Process Evaluation | |
Implementing the Program | Documents and assesses implementation; quantifies what was done; when, where, and how it was done; and who was reached | Identifies areas for improvement as implementation proceeds Documents progress of implementation |
Stage 4 | Outcome Evaluation | |
Assessing Effectiveness and Making Refinements | Measures whether, and to what extent, a program or activity had the planned effects | Documents the extent of the campaign's success or failure Documents success to support replication Determines any need to improve the existing program or future efforts |
Impact Evaluation | ||
Measures whether, and to what extent, a program contributed to long-term goals | Is not often used for health communication activities (improving health status usually requires multifaceted approaches—e.g., communication plus changes in health care service delivery and relevant policies—and it is generally not possible to isolate a particular communication program's contribution to achieving longer-term goals) |
Focus groups and in-depth interviews are the most common methods used in qualitative communication research. However, there are many innovative methods, some described here, that can help you learn about an audience. Because the methodologies for each are very similar, they are discussed together here, using instructions for focus groups as a guide.
About Focus Groups
Working from a discussion guide, a skilled moderator facilitates a 1- to 2-hour discussion among 6 to 10 participants, which can be conducted either in person or by telephone (ideally in person). The moderator keeps the session on track while participants talk freely and spontaneously. As new topics related to the material emerge, the moderator asks additional questions to learn more.
Common Uses
- Developing a communication strategy:
- Learning about feelings, motivators, and past experiences related to a health topic
- Exploring the feasibility of various potential actions (from the intended audience's viewpoint)
- Identifying barriers to those actions
- Exploring what benefits the intended-audience members find compelling and what results they expect from taking a particular action
- Learning about the intended-audience's use of settings, channels, and activities
- Capturing the language used by the intended audience to discuss issues
- Exploring reactions to message concepts (concept testing):
- Identifying concepts that do or do not resonate and understanding why
- Triggering the creative thinking of communication professionals
- Illustrating to others how the intended audience thinks and talks about a health issue
- Developing hypotheses (or broad issues) for quantitative studies and identifying the range of responses that should be included in closed-ended questionnaires
- Providing insights into the results of quantitative studies by obtaining in-depth information from individuals typical of the intended audience to help understand why individuals responded in certain ways
- Group interaction can help elicit in-depth thought and discussion.
- Group interaction can help with brainstorming because respondents can build off one another's ideas.
- Moderators have considerable opportunity to probe responses.
- Focus groups yield richer data than surveys about the complexities of an intended audience's thinking and behavior.
- Findings are not generalizable to the population.
- Focus groups can be labor intensive and expensive, especially if sessions are conducted in multiple locations.
- Group responses do not necessarily reflect individuals'9 opinions because some individuals in the group may dominate the discussion or may influence others' opinions.
- Each person is limited to about 10 minutes of talking.
Working With Market Research Professionals
You may need to hire or contract with two kinds of market research professionals as you design, conduct, and analyze your concept and materials testing:
1. Someone to design the research and data instruments (e.g., questionnaires, discussion guides, screeners), to analyze the results, and to prepare a report
2. A vendor to handle the fieldwork (i.e., recruiting and hosting focus groups; administering telephone, mail, or in-person surveys)
Ideally, these professionals will have a background in health communication or, if not, a background in marketing or advertising research. You can get the best service from these professionals by:
- Providing clear research objectives and appropriate background information, including the creative brief.
- Learning enough about common communication research methods to understand their strengths and limitations, so that you don't ask for more than a given method can deliver (e.g., asking, "What percentage of the American public does that represent?" when a focus group study was conducted).
- Letting market researchers' expertise guide your selection of methods. Rather than saying, "We want to focus test this," explain your research objectives, timing, budgetary constraints, and any additional factors (such as the need for a publication to be tested with people from a wide range of cultures). Then let the experts propose methods to you and explain their rationale.
- Being realistic about timelines, quantity of information, materials to be tested at one time, and the level of "proof" you need. Pretesting is diagnostic; it can provide guidance on what needs to be improved, but it can't tell you how successful something will be. Other factors, such as the final production of your message, the number of people who see it, the frequency with which it is seen, and the presence of competing messages will all influence your message's success.
- Recognizing that there are inherent differences between testing advertising and other commercial communication materials versus testing health communication materials, even if the channel and activity (e.g., a television spot) are the same. Individuals trained in commercial concept development and copy testing will be able to draw on their commercial experience for selecting the appropriate methodology. However, they often have little experience assessing reactions to complex health messages; they are more familiar with assessing efforts to direct an existing behavior toward use of a particular product brand than with assessing efforts to completely change a behavior.
Sometimes, one individual or organization can play both roles; at other times, you may have internal staff, a consultant, or staff at a health communication firm to handle the first role but contract externally for the second. The American Marketing Association's Green Book lists suppliers and services geographically throughout the United States. Other sources include the Marketing Research Association, the Association of Public Opinion Researchers, the Qualitative Research Consultants Association, and faculty at university departments of marketing, communication, health education, psychology, and sociology.
About In-Depth Interviews
The process, benefits, and drawbacks of in-depth interviews are similar to those of focus groups, except that the interviewer speaks with one person at a time. In-depth interviews can take place at a central facility or at the participant's home or place of business. As with focus groups, when individual interviews cannot be conducted in person, they can be conducted by phone or over the Internet. Although these interviews are more time intensive, one of their key benefits is that each respondent is isolated from other respondents and therefore not influenced by what others say.
How to Design and Conduct a Focus Group or In-Depth Interview Study
To design and conduct a qualitative research study, complete the following steps.
Plan the Study
Determine the following:
- What you want to learn. Determine the objectives of your study at the outset, and then check to make sure that the moderator's/interviewer's guide includes lines of questioning that will provide the answers.You may also use the objectives to help analyze the results of the discussions and to organize the focus group or in-depth interview report.
- When you need to have that information.
- How you will apply what you learn. It is important to decide how you will use your focus group or in-depth interview results before you conduct your study.
- Your budget.
- Your criteria for who should participate.
Select people who are: - Typical of your intended audience (the same behavioral, demographic, and psychographic characteristics).You may want to conduct separate groups with "doers" (those who already engage in the desired behavior) and "nondoers" (those who do not) to help identify what actions the doers take, and why, so that those approaches can be explored with the nondoers.
- Not experts. Exclude market researchers and advertising professionals (because of their familiarity with the methodology) and those who have, or might be perceived by other group members as having, expertise in the subject matter (e.g., exclude health professionals from focus groups when the topic is a health issue).
- Relative newcomers to focus groups or interviews, so that their reactions will be spontaneous. This will help you avoid "professional" respondents (i.e., those who have participated in many previous focus groups or individual interviews before) who may lead or monopolize the discussion.
- The number of groups you will convene.
- Divide participants into different focus groups based on their gender, race, age, level of formal education, or any other variable likely to hinder their freedom of expression (e.g., teenage girls will be more comfortable discussing sexual activity if teenage boys or college-age women are not in the group).
- Conduct a minimum of two focus groups with each intended audience segment (e.g., if you are conducting separate groups with men and women, you will need at least four groups–two with men, two with women). If intended audience perceptions vary or the audience feedback is unclear, you may want to conduct additional groups with each segment, especially if you revise the discussion guide to more fully explore unresolved issues.
- If you are using in-depth interviews, conduct approximately 10 interviews per intended audience segment. If common themes do not emerge or the interviews, especially if you revise the interview guide in between interviews.
Choose the Location
You can convene focus group discussions or in-depth interviews in a variety of ways:
- Commercial focus group facilities can recruit participants for you (for both focus groups and interviews) and offer audio and video recording equipment as well as observation rooms with one-way mirrors for viewing. However, these facilities are usually available only in larger metropolitan areas.
- Teleconference services can set up telephone focus groups for you. Most allow observers to listen without being heard, and some provide remote viewing programs to allow the moderator to see a list of participant names (with a symbol next to the one currently speaking) or notes sent in by a technician from observers listening to the call. Some teleconference services can recruit participants; with others, you will have to recruit participants or contract with a recruiter separately.
- You can also conduct focus groups or in-depth interviews in meeting rooms at churches, office buildings, or other locations. If an observation room with a one-way mirror is not available, staff may still listen in through speakers hooked up in a nearby room or by audiotaping or videotaping the session. In some cases, one or two quiet observers may be allowed in the room to take notes.
See the sidebar Pros and Cons of Different Formats below for the advantages and disadvantages of different formats for focus group and in-depth interview research.
Pros and Cons of Different Formats
- Face-to-Face
Moderator/interviewer and participants are in one room, usually around a table; observers (members of the research team) are behind a one-way mirror.
Pros- Can assess body language If videotaped, can share with others who couldn't attend
- Have participants' undivided attention
- Responders lose some anonymity
- Higher travel expenses due to multiple locales
- Usually excludes people in rural areas or small towns
- Telephone
Moderator/interviewer and participants are on a conference call; observers listen in
Pros- More convenient for participants and observers
- Can easily include people in rural areas, in small towns, and who are homebound
- For professional groups, may be easier to gain participation because it is less likely participants will know each other
- Relative anonymity may result in more frank discussion of sensitive issues
- Can't assess nonverbal reactions
- More difficult to get reactions to visuals (although they can be sent ahead of time)
- Participants can be distracted by their surroundings
- Internet Chat Sessions
Moderator and participants "chat"; observers watch
Pros- Complete record of session instantly available
- Relative anonymity may result in more frank discussion of sensitive issues
- Only useful for participants comfortable with this mode of communication
- Relatively slow pace limits topics that can be covered
- No way to assess if participants meet recruitment criteria
- Can't assess body language or tone of voice
- More difficult to get reaction to visuals
Draft a Recruitment Screener
A recruitment screener is a short questionnaire that is administered to potential participants, typically by telephone, to ensure that they meet the criteria you developed during step 1. Ideally, the screener should help you exclude participants who are already familiar with the specific subject of the sessions, or who know each other. If participants know the subject in advance, they may formulate ideas or may study to become more knowledgeable about the subject than the typical intended audience member. If participants know each other, they may speak less freely. See Appendix A for a sample screener.
Recruit Participants
Recruit participants by telephone one to three weeks in advance of the sessions. You can identify potential participants in different ways depending upon the type of people you are seeking and the resources you have available. Identify members of the public through focus group facility databases, by running an ad in a local publication, by working with community organizations, or by using the phone book (although the latter will be extremely time consuming if you have stringent recruitment criteria). Identify professionals through a relevant association or mailing list service or through a focus group facility's recruiting databases. Depending on your budget and internal resources, you may choose to recruit in one of the following ways:
- Use your own organization's staff to recruit participants.
- Hire a focus group facility or independent recruiter (to identify such facilities, consult a directory such as the American Marketing Association's Green Book or the Marketing Research Association's Blue Book). If you have many facilities and recruiters to choose from, consider getting recommendations from local corporations or organizations that conduct qualitative research.
- Get help from a university marketing research or advertising class.
- Work through gatekeepers such as teachers (for students), health care providers (for patients, physicians, or nurses), religious institutions or community organizations (a small donation may encourage them to participate), and instructors of English as a second language.
Regardless of how the recruiting is done, ensure that the screener is followed carefully so that only individuals who qualify for participation will be included.
Contracting With Commercial FacilitiesBefore you contract with a commercial facility to conduct in-depth interviews or focus groups, prepare a specification sheet detailing all of the services you need and, if you will be asking the facility to recruit, a profile of your intended audience. Vendors will use this information to estimate the cost of the project and to develop bids.
Decide whom to approach by using the following checklist to determine which vendors will fit your needs. Each vendor should provide the following information:
- Descriptions of past projects
- Descriptions of, or a list of, clients
- Location of facility (Is the facility conveniently located? Is parking available? Is the facility accessible by public transportation? If not, does vendor provide transportation assistance, such as taxi money or van service? What does this add to cost?)
- Diagram of table/seating arrangement (What size and shape are tables? Rooms?)
- Description of the size and features of observation rooms
- Details about audio and video recording arrangements and costs
- Details about food arrangements for participants and clients
- Description of moderator services
- Description of recruitment methods and geographic coverage
- Recommendations for participant incentives
- Reasonable rates for the services they will provide (ask for nonprofit rates, if appropriate)
Getting People to Show Up
To ensure that enough people show up, offer an incentive (usually money) and recruit two or three more people than you actually need. If all show up, select those who best fit the screening criteria, thank the extra participants, give them the agreed-upon incentive, and dismiss them. Other ways to increase participation include:
- Scheduling sessions at times that are convenient for your potential participants (e.g., at lunch or after work)
- Choosing a safe and convenient site with easy parking
- Providing transportation (or reimbursement for agreed-upon transportation costs)
- Arranging for childcare, if necessary
- Letting them know you will be serving refreshments
Recruiting Patients and Their Families
Recruiting patients and their families requires special consideration. Contact clinics, hospitals, or local HMOs for help and make adequate plans to ensure that the participants and their family members are not inconvenienced or upset. Some organizations may require institutional review board (IRB) approval of your research. Gaining IRB approval is often a long process, so be sure you check with the organization early in the planning stage of your study to find out whether you will need IRB approval.
Recruiting for Telephone Interviews
If you are recruiting for in-depth interviews to be conducted on the telephone, create a spreadsheet that includes spaces for the following information about each potential participant:
- Time zone in which the person is located
- Date, time, and telephone number at which he or she should be called for the interview
- Disposition of each call (e.g., scheduled an interview, no answer, busy, refused)
Develop a Moderator's Guide
The moderator's guide tells the moderator what information you want from the groups and helps him or her keep the discussion on track and on time. It is only a guide, however. During the focus groups, experienced moderators flow with the conversation, asking questions in the prescribed language and sequence when possible but sometimes deviating from the guide to avoid awkward transitions or unnecessary back-and-forth between topics.
Before you draft the moderator's guide, answer the following questions:
- What do we want to learn from the focus group?
- How will we apply what we learn?
- What tools (e.g., descriptive information, message concepts, or other draft creative work) will we need to provide?
Then, write questions for the guide that relate to the purposes you have identified. Make most questions open-ended so that participants can provide more in-depth responses than just "yes" or "no," but make sure the questions are not leading. This will help you get answers that reflect participants' true feelings and not what they think you would like to hear. The amount of time and depth of questions devoted to each issue should reflect the value of the issue to the research. See Appendix A for an example of a moderator's guide.
Do not include questions for group discussion when you need individual responses. However, you can have the moderator give self-administered questionnaires to each participant to be completed prior to conducting a focus group, or participants can be asked to individually rank items on paper—such as potential actions, benefits, or message concepts—during a group to obtain both individual and group reactions.
Working With Community Organizations to Conduct Focus GroupsOnce you have identified potentially cooperative community groups (see the following sidebar for a list of groups you might approach), contact an official within each group (e.g., the president or program director) to request cooperation. You may make these initial contacts by telephone and follow up with a formal written request that includes the following:
- Description of your agency or organization
- Description of the material/topic to be discussed and its purpose
- Details regarding the participants to be recruited and how you will protect their confidentiality
- Outline of the activities involved<</li>
- Incentives you are offering the organization and the participants
- Detailed explanation of why the organization official should not reveal details about the nature of the discussion to participants in advance, unless the organization is to recruit participants
- If and how you will share the information learned
Once you have an agreement with a community organization, decide how you will recruit participants. One possibility is to conduct your research as part of the group's regularly scheduled meeting.
The advantages of this approach are:- Little extra effort is required to recruit participants.
- You may need to provide only minimal or no incentives.
- The group's regular and familiar meeting place can be used.
- You have little control over the number of people who will come or the composition of the group.
- It is difficult to place a 1- to 2-hour focus group on the agenda of a regularly scheduled meeting.
- Many organizations set their calendars months ahead of time (it may be difficult to schedule the focus group within a reasonable time frame).
An alternative is to recruit the group's members to a special meeting. Schedule this meeting immediately before or after the group's regular meeting to make it most convenient for the participants. If you use this alternative, contact members in advance on behalf of the group and ask them to participate. A person from the community group can also ask others to participate. To ensure that participants attend and stay through the whole meeting, let them know in advance that you will be providing refreshments (assuming that you are doing so).
The advantages of recruiting participants to a special meeting are:
- It provides an opportunity to screen participants on relevant characteristics and to eliminate market researchers or other experts who should not participate.
- It helps reduce participant fatigue because the entire meeting will be devoted to your research.
If you recruit participants yourself, you will have more control over what the participants are told about the focus group, and you will be able to screen potential participants to make certain that they fit special intended audience characteristics. However, recruiting participants takes a significant amount of time, and it is possible that people would be more likely to participate if asked by someone whom they know.
If the member organization recruits participants, it is essential that you provide the recruiter with detailed instructions for carrying out the task. These instructions must include a written description of the focus group topic, which should be read to potential participants verbatim, and a questionnaire to screen participants on relevant criteria.
Conduct the Focus Groups
Focus groups typically begin with the moderator welcoming participants and briefing them on the process (e.g., all opinions welcome—there are no right or wrong answers; the presence of audio- and videotaping and observers; the importance of speaking one at a time; confidentiality). Participants introduce themselves to the group by first name, usually including some information relevant to the topic of discussion (e.g., number of years with glaucoma, amount/type of insulin used each day). Next, the moderator asks a few simple "ice-breaker" questions to help participants get used to the group process and to reduce participant anxiety. This also helps the moderator develop rapport with the participants.
Continuing to follow the moderator's guide, the moderator manages the group and ensures that all topics are covered without overtly directing the discussion. Participants are encouraged to express their views and even disagree with each other about the discussion topics. The moderator does not simply accept what participants say but probes to learn more about participants' underlying thinking and attitudes. The moderator also seeks out opinions from all participants so that all are heard and a few do not dominate the discussion.
Near the end of the discussion, the moderator will often give participants an activity or simply excuse him- or herself from the room for a moment to check with the observers and obtain any additional questions. Alternatively or additionally, notes can be sent in to the moderator while the group is in process if the observers would like different questions asked or other changes made to the group.
One advantage of focus group methodology is that the moderator's guide, and any materials presented, can be revised between groups if necessary.
Analyze Results
The easiest and most thorough way to analyze focus groups is by reviewing transcripts, although groups can also be analyzed (albeit less thoroughly) by reviewing notes taken during the discussion. In many analyses, the goal is to look for general trends and agreement on issues. At the same time, it is important to note divergent opinions. Don't ignore individual comments that raise interesting ideas or concerns such as lack of cultural sensitivity or difficulty in comprehension. In some instances, the goal is to capture the range of opinions about an issue, rather than to look for evidence of agreement or consensus.
Avoid counting or quantifying types of responses (e.g., "75 percent of participants preferred concept A"). Attempting to quantify the results—or suggesting in other ways that they represent the opinions of the intended audience as a whole—is inappropriate for qualitative research.
Quasi-Quantitative Research Methods: Pretesting Messages and Materials
Some commonly used communication research methods, such as central-location intercept interviews and theater tests, are best termed quasi-quantitative. While these methods are used in situations in which the goal is measurement and typically involve a questionnaire with mostly forced-choice questions, the results cannot be projected to the population as a whole (as with true quantitative surveys) because of the way in which participants are selected. For centrallocation intercept interviews, the only people who have a chance to participate are those who go to the location where the interviews are being held and who go there during the times they are conducted; this is not a truly representative sample of the intended audience. For theater tests, the only people who have a chance to participate are those who are recruited for the test, and recruitment does not follow a truly representative sampling design.
Quasi-quantitative methods are most often used during Stage 2 to pretest messages and materials. If your intended audience is geographically dispersed or it is difficult for them to get to a central facility, you can use telephone interviews and send participants any materials in advance. This type of pretest typically resembles an in-depth interviewing project in price and number of interviews, although there may be more closed-ended questions and the question sequence may be adhered to more closely.
The Moderator's Role
The moderator does not need to be an expert in the subject of your research but must have experience facilitating group discussions. A good moderator builds rapport and trust and probes, without reacting to or influencing, participants' opinions. The moderator must be able to lead the discussion and not be led by the group. He or she must emphasize that there are no right or wrong answers to the questions that are posed. A good moderator understands the process of eliciting comments, keeps the discussion on track, and figures out other ways of approaching a topic if the first way is unproductive. Good moderators understand what you are looking for and what you need to do with the information, and they are able to probe and guide the discussion accordingly. Go over the guide with the moderator to point out any topics or concerns you want emphasized or discussed in depth. By the end of the focus group or interview, the moderator should ensure that all agreed-upon topics are covered sufficiently.
If your organization plans to conduct focus groups regularly, consider hiring a skilled, experienced moderator to train your internal staff to moderate focus groups. Use local advertising agencies, the American Marketing Association's Focus Group Directory, or the Qualitative Research Consultants Association to identify a good moderator.
Using Software to Analyze Qualitative Data
If you have conducted a large number of focus groups or interviews with many intended audience subsets-and are interested in analyzing results by different cultural groups, age groups, or economic groups within the overall respondent population-you may want to use computer software to do a comparative analysis of your results. If your results are from only a few groups, however, computer analysis will be too time consuming to benefit your program. Be sure to supplement computer analysis with "human" analysis, since the strength of qualitative research is that it can uncover unexpected human reactions that software cannot properly capture or weigh.
Before you decide to use computer software to analyze your qualitative data, assess the following advantages and drawbacks of this type of analysis.
- The ability to highlight sections of the transcript that are important to the project and to eliminate "noise" or sections of the transcript that are not important to answering your research questions (of course, a wordprocessor's cut-and-paste functions can also accomplish this).
- The ability to quickly access and compare information on one topic or questions across several transcripts.
- You must tape all interviews
- It costs time and money to transcribe focus group sessions and in-depth interviews.
- Coding the transcripts is both time intensive and expensive. If more than one person will be coding, you will need to train the coders, periodically assess intercoder reliability, and retrain as necessary.
- While all comments on a particular topic can be gathered, they are taken out of context in the process. Sarcasm and other tonal characterizations may be lost.
- Analysis by software will help you organize information, but will also produce overwhelming amounts of paper.
If you decide to use software to analyze your qualitative data, follow these steps:- Transcribe the focus group discussions or interviews into an electronic format that can be read by the analysis software. If you decide to use qualitative data analysis software, check the requirements of the package you choose.
- Develop and apply the codes you will use to organize the information in the transcripts. A code is a word or number that represents a research objective, research question, theory, or idea you are testing. The codes you develop will be unique to your research. For example, if your first research question is to find out how many vaccine shots a parent is willing to give his or her infant, you might code all lines of your transcript that include comments on that question as "A,"' signifying that they correspond to your first research question. Applying the code requires that you review the transcripts and use a mouse to indicate on the computer screen all the lines of the transcript that pertain to that code word.
- Generate reports using the codes you developed. To do this, you instruct the program to search for a particular code or a combination of codes. The program then presents a single report showing all the lines of the transcript bearing the requested codes.
For a complete review of qualitative data analysis software, refer to Computer Programs for Qualitative Data Analysis: A Software Sourcebook, by Eben A. Weitzman and Matthew B. Miles, 1995, Thousand Oaks, CA: Sage. A new edition of this book is due out in 2002.
Central-Location Intercept Interviews
Central-location intercept interviews consist of stationing interviewers at a point frequented by individuals from your intended audience and asking the individuals to participate in a study. If they agree, they are asked specific screening questions to see whether they fit the study criteria. If so, the interviewer takes them to the interviewing station (a quiet spot at a shopping mall or other site), shows the pretest materials, and then administers the pretest questionnaire.
For intercept interviews to be effective, you must obtain results from a minimum of 60 to 100 respondents from each intended audience segment you want to test.
- You can connect with harder-to-reach respondents in locations convenient and comfortable for them.
- The interviews can be conducted quickly. (The interview should be no longer than 15 to 20 minutes.)
- The interviews are a cost-effective means of gathering data in a relatively short time.
- If you choose an appropriate location, you will increase your chance to interview respondents who are among your intended audience.
- You must train interviewers.
- Your results are not representative or generalizable.
- Intercept interviews are not appropriate for sensitive issues or potentially threatening questions.
- Intercept interviews do not allow you to probe easily for additional information.
Central-location intercept interviews should not be used if respondents must be interviewed in depth or on emotional or sensitive subjects. The intercept approach also may not be suitable if respondents are likely to be resistant to being interviewed on the spot. In cases in which central-location intercepts will not work well, schedule interviews with respondents instead.
Estimated Costs of Focus Groups and In-Depth Interviews, 2002 | ||
These estimated costs are included to suggest how you should budget for focus groups and in-depth interviews if you are using commercial research firms. Your actual costs will vary depending upon your geographic location, the intended audience to be recruited, and the amount of time donated by staff, companies, and participants. Be sure you do not jeopardize the quality of your results with a budget that is too small. The focus group cost estimate in the table assumes that you conduct two groups, each composed of 10 members of the general public. A group size of 6 to 8 is sometimes preferred, because it is easier to engage all participants in the conversation. This estimate is also based on the assumption that each group session is 2 hours long, is conducted in English, and includes audiotapes. Staff travel, food for participants, and videotaping are not included. The in-depth interview estimate assumes a total of ten 30-minute interviews conducted in English and audiotaped. | ||
2 Focus Groups | 10 In-Depth Interviews | |
Develop screener | $ 920 - 1,380 | $ 920 - 1,380 |
Develop discussion guide | $ 920 - 1,840 | $ 920 - 1,840 |
Recruit | $1,725 - 2,875 | $ 860 - 1,725 |
Rent facility | $ 705 - 1,380 | $ 0 - 1,150 |
Provide respondent incentives | $ 690 - 1,380 | $ 0 - 575 |
Compensate moderator or interviewer to conduct | $1,000 - 2,200 | $ 500 - 1,000 |
Analyze and report results | $1,840 - 2,760 | $1,840 - 2,760 |
Total | $7,800 - 13,815* | $5,040 - 10,430** |
*Add an additional $400-600 for transcribing focus group audiotapes (optional). **Add an additional $300-400 for transcribing in-depth interview audiotapes (optional). |
Estimated Costs of Central-Location Intercept Interviews, 2002 | |
The central-location intercept interview cost estimate below assumes that you question 100 respondents from the general population for 15 to 20 minutes each. | |
Develop questionnaire | $ 460 - 4,025 |
Produce questionnaire, schedule facility, phones, and mail | $ 400 - 600 |
Screen/conduct interviews | $1,955 - 4,025 |
Provide respondent incentives | $ 345 - 575 |
Code/enter data/tabulate | $ 805 - 1,495 |
Analyze and report results | $1,495 - 4,025 |
Total | $5,460 - 14,745 |
Questionnaire Contents
Unlike focus groups or in-depth interviews, the questionnaire used in central-location intercept pretesting is highly structured and contains primarily multiple choice or closedended questions to permit quick response. Open–ended questions, which allow free–flowing answers, should be kept to a minimum because they take too much time for the respondent to answer and for the interviewer to record. Questions that assess the intended audience's comprehension and perceptions of the pretest materials form the core of the questionnaire. A few additional questions, tailored to the specific item or items being tested ("Do you prefer this picture—or this one?"), may also be included to meet your program planners' particular needs. The questionnaire should be pretested before it is used in the field. See Appendix A for a sample questionnaire.
Interview Setup
A number of market research companies throughout the country conduct central–location intercept interviews in shopping malls.You can also conduct these interviews in clinic waiting rooms, religious institutions, Social Security offices, schools, work sites, train stations, and other locations frequented by members of your intended audience. Be sure to obtain permission well in advance of the time you want to set up interviewing stations in these locations.
If you are using a market research company to conduct the interviews, you will need to provide screening criteria, test materials, and the questionnaire. In some cases, market research companies have offices in shopping malls, and you can watch the testing through a one-way mirror.
Participant Recruitment
If you or someone in your organization is recruiting the participants, you will need to develop a script and provide training in approaching members of the intended audience. For example, if you are recruiting participants in a clinic waiting room, the interviewer should be familiar with the screening criteria (e.g., women under 60 years of age) and approach only those people who appear to fit the criteria. When, after screening, individuals do not qualify to participate, the interviewer should thank them for their time and indicate that this study is not the right fit for them but that their willingness to participate is appreciated. If they do qualify, the interviewer can bring them to a designated location (e.g., another room or corner of the waiting room) and proceed with the study.
University and college departments of marketing, communication, or health education may be able to provide interviewer training or trained student interviewers. Pretesting is an excellent real–world project for a faculty member to adopt as a class project or for a master's student to adopt as a thesis project. However, this approach may mean that it takes longer to accomplish the research, and you could compromise the quality of the results if the individuals are not experienced in this type of research.
Examples of Community Organizations to Contact for Help Recruiting Participants
- American Legion
- B'nai B'rith Women
- Business groups
- Jaycees
- Junior League
- Knights of Columbus
- League of Women Voters
- Lions Club
- National Council of Negro Women
- National Urban League
- Parent Teacher Associations
- Religious organizations
- Rotary Club
- Schools
- Senior citizen centers
- Unions
- Veterans of Foreign Wars
Theater-Style Tests
Theater testing is often used in the commercial arena to test advertisements for products and services. Theater testing can also be used to test the effectiveness of PSAs. In this methodology, participants are invited to a central location to respond to a pilot for a new television show; in the midst of viewing the TV pilot, they are shown your PSA or advertisement along with other ads. Participants complete a questionnaire following the presentation, first answering questions about the show and then answering questions about how effectively your message was communicated to them and what their overall reactions were.
Common Uses
Theater–style tests are most commonly used to test TV advertisements and PSAs. For theater–style tests to be effective, you must obtain results from 50 to 100 respondents from each segment you want to test.
- You can obtain responses from a large number of respondents at the same time.
- Theater–style tests more closely replicate what goes on in someone's home when they are watching TV, so you can accurately judge people's reactions to your message.
- Theater–style tests can be cost–effective if you use donated facilities and equipment.
- It is expensive to rent a facility and equipment (if necessary).
- Your results are not representative of the general population.
General Format
Individuals typical of your intended audience are invited to a conveniently located meeting room. The room should be set up for screening a television program. Participants should not be told the real purpose of the session, only that their reactions to a television program are being sought.
At the session, participants watch a television program. The program can be any entertaining, nonhealth video approximately 15 to 30 minutes in length. The videotape is interrupted about halfway through by a sequence of four commercials. Your message should be inserted between the second and third commercials. See Appendix A for a description of how to create a roughcut video for theater-testing your message.
At the end of the program, participants receive a questionnaire and answer questions designed to gauge their reactions, first to the program and then to the advertisements. Finally, your ad is played again and participants complete several questions about your ad. The majority of these questions should be closed-ended to enable an easy and accurate summary of participant responses.
In more sophisticated theater testing, participants use automated intended audience response systems to answer questions. Participants are provided with a small device that has response keys. Once a question is asked, they push a key to respond and the data are automatically tabulated.You have instant access to the numbers using this system. In addition, questions can be instantly added or deleted from the questionnaire based on the previous responses. Using an automated system is much more costly than using a standard paper-and-pencil questionnaire.
Other Media You Can Test
This methodology can also be used to test videos by asking participants to view a series of videos in which yours has been included. Examples of videos that might be tested include a 15- to 30-minute breast cancer awareness video that will be played in a clinic or a "how-to" video on administering epinephrine. These testing sessions will, of course, last longer than those testing ads. Participants evaluate the videos as described above.
Print advertisements can also be tested using a variation of this methodology. Several ads, including yours, are inserted into a magazine. Participants are given an adequate amount of time to read through the article, which includes your ad and others. After reading the article, participants receive a questionnaire and answer questions designed to gauge their reactions, first to the article and then to the advertisements. Finally, your ad is displayed alone and participants complete several additional questions.
Designing and Conducting a Theater-Style Pretest
The process for conducting a theater–style test includes the following steps:
- Planning the pretest
- Developing the questionnaire
- Recruiting respondents
- Preparing for the pretest
- Conducting the pretest
- Analyzing the pretest
You may find step 2 also useful for central–location intercept interviews.
Plan the Pretest
- The purpose of the study (e.g., what do you want to learn?)
- When you need to produce results
- What your budget is
- The type and number of people who should participate in the pretest
- The locations where the pretest will be conducted
To conduct theater testing, you must have a large enough space to accommodate all of your participants at the same time. You must also ensure that you have several video monitors so that all participants can adequately view the program. Space constraints may be overcome by seeking out low-cost facilities such as a school auditorium or church hall.You may be able to borrow the audiovisual equipment from these facilities as well. You can also rent space, such as a hotel ballroom, if you want to test a large number of people. Hotels often rent audiovisual equipment as well. Reserve facilities and equipment well in advance of your pretest.
Develop the Questionnaire
To gather useful information from the pretest, you must carefully construct the questionnaire. See the sidebar Components Used in Most Questionnaires on the next page for general guidelines. Once you have written your questionnaire, be sure to test and revise it before you use it with a large number of respondents.
Recruit Respondents
Participants may be recruited through a market research facility or through local community organizations. In either case, you will need to provide an incentive for participants. If using a market research facility, you will also incur recruiting expenses. If you are working with a community organization, you may choose to make a donation.
Develop questionnaire | $ 460 – 2,760 |
Produce questionnaire | $ 400 – 600 |
Recruit | $ 0 – 5,750 |
Rent audiovisual equipment | $ 0 – 2,300 |
Conduct theater test | $ 0 – 920 |
Provide respondent incentives | $ 285 – 2,875 |
Code/enter data/tabulate | $ 920 – 3,680 |
Analyze and report results | $1,840 – 3,680 |
Total | $3,905 – $22,565+ |
* Estimates assume 50 participants. They exclude facility rental costs. The costs of large facilities (e.g., hotel ballrooms) vary widely by geographic location. Check with local facilities for approximate costs. |
Components Used in Most Questionnaires
Although the following components should be used in most questionnaires, these descriptions are specific to a theater-style pretest.
Program Questions
Program questions elicit general intended audience reactions to the program viewed. Use the questions in Part I of the sample Theater–Style Pretest Questionnaire in Appendix A as the first page of your questionnaire.
Recall, Main Idea, and General Reaction Questions
These three standard questions that assess your message's ability to attract attention, convey its main point, and create a positive response should appear on the second page of your questionnaire. Use the questions in Part II of the sample questionnaire in Appendix A.
These standard questions should be incorporated into the pretest for several reasons. The questions address the most important indicators of a message's potential effectiveness: 1) whether it attracts intended audience attention (recall), 2) whether it communicates your main point (main idea), and 3) what respondents thought and felt when they viewed the ad (reaction).
If you test many ads and always use the same questions, you can develop a database of results to allow you to assess the relative strength of various ads.
Specific intended audience Reaction Questions
These provide answers to specific questions you have about your message.
Develop questions that address specific concerns you have about your message. For example, suppose your message asks viewers to call a toll-free number for more information. You may want to include a question that asks, "What action, if any, does the message ask you to take?" A related question may be, "Did the telephone number appear on the screen long enough for you to write it down (or remember it)?"
It is best to develop one or more questions addressing each characteristic of your message. The list below includes various characteristics commonly found in messages. Note the characteristics that apply to your test message and then develop questions that focus on those characteristics.
- Use of music (with or without lyrics)
- Use of famous spokesperson
- Use of telephone number/Web site address
- Request for a particular action
- Instructions for adopting a specific health behavior
- Presentation of technical or medical information
- Presentation of new information
- Promotion of a sponsoring organization or event
- People intended to be typical of the intended audience
- Use of a voice-over announcer
- Presentation of controversial or unpleasant information
See Part III of the sample in Appendix A for examples of questions you can use for each message characteristic listed above. These questions are just examples and should be adapted to your needs. Remember that the objective of pretesting is to uncover any problems with your ad before final production.
Demographic Questions
These questions help to record the characteristics of the participants (e.g., their sex, age, level of education, health status).
Once you have written your questionnaire, be sure to test it before you use it with a large number of respondents. After any necessary revisions, you are ready to make copies for the pretest participants. You will need a cover page that instructs participants not to open their questionnaires until they are asked to do so by the meeting host. Place a cover sheet between each part of the questionnaire and instruct participants not to continue until they are asked to do so by the meeting host.
Prepare for the Pretest
Rehearse the testing session at your own office to anticipate and avoid any problems before actual pretesting. Review the following questions to be sure that your session will go as smoothly as possible:- Is the pretesting videotape ready for use? Are the video and audio portions clear?
- Is the videotape equipment—recorder and television monitors—functioning properly?
- Is the facility set up? Is the room reserved? Are there enough chairs? Are extra chairs available in case more people show up than you expect? Do you need another monitor so that everyone will be able to see the program? Is the heat or air conditioning working properly? Do you know where the light switches are? If a microphone is needed, is it set up and functioning properly?
- Have you made all the necessary staffing arrangements? Are your assistants coming to the session? Do they have transportation and directions for getting there?
- Have you made enough copies of your pretest questionnaire (including some extras)? Is each questionnaire complete (with no pages missing)? Do you have pencils for participants? Will they need clipboards or pads?
- Has participant recruitment taken place as scheduled? Did you call and remind participants to attend? Do they have transportation and directions?
- Has the moderator rehearsed?
Conduct the Pretest
The procedures to follow during the pretest session are relatively simple. The keys to a successful testing session are to:- Be friendly and courteous to participants from the moment they arrive until they leave (remember to say "thank you").
- Keep calm and cool-headed throughout the session.
- Anticipate problems in advance (conducting a rehearsal to make sure that both equipment and timing work is a good idea).
The test session should take no more than 1 hour and 15 minutes if you are organized and well prepared. Follow the steps below to conduct your test:
- Encourage participants to take a seat as they arrive. Close the doors no later than 10 minutes after the
scheduled starting time. - When everyone is seated, introduce yourself by your name only (assuming you are the host). Do not
tell participants the name of your organization during the session because it might bias their responses to your test ad. - Thank participants for coming and assure them that the evening should be enjoyable and that they will have
a chance to give their views to the producers of "new" television program material. Discourage participants from talking to one another during the session. Tell them you are interested in their own individual views and that there are no right or wrong answers. Also, encourage them to write their answers clearly in the space provided on the questionnaire. - After your introductory remarks, have your assistants hand out the questionnaires (see Appendix A for a sample), pencils, and clipboards (if needed). Instruct the participants not to open the questionnaire until you ask them to do so. Turn on the video recorder and monitor to begin the test session.
- Be attentive and watch for any problems with the sound or picture on the monitor. Be sure that the equipment is functioning properly throughout the program.
- Be prepared to stop the recorder when the television program has ended. Introduce the questions, and thank the participants for their help so far. Ask them to open their questionnaires and complete the questions on the first page.
- When the participants have finished Part I of the questionnaire, tell them that you would like to gather their reactions to the messages/PSAs that were shown during the program. Have them turn to Part II and instruct them to fill out the questions about the messages.When they have completed these questions, tell them that you want to obtain their reactions to one particular message in the series of messages they viewed.
- Start the video. (Note: To avoid an awkward pause in the session's pace, be sure there is not too much lead tape before the message starts.) After your PSA/ad has been replayed, ask participants to turn to the next page of the questionnaire and complete the remaining questions. Encourage them to answer every question and to avoid giving more than one answer, except when this option is indicated on the questionnaire.
- Circulate through the room to monitor progress and to be sure participants are not discussing their responses. Collect the questionnaires as participants finish.
- Thank participants for their cooperation. If you have incentives or token gifts, distribute them to participants as they leave. If you have provided a donation to a group in lieu of payment to participants, mention that you hope the group will find the donation helpful.
Analyze the Pretest
Analyze the questionnaires in two steps. First, tabulate or count how many participants gave each possible response to each question. Next, look for patterns in the responses to both closed– and open–ended questions. The patterns will help you draw conclusions about the effectiveness of your message. See Appendix A for detailed instructions on tabulating closed- and open-ended questions and for a table of average ratings to help interpret standard question responses.At this point, look at the overall results:
- What did you learn from the pretest?
- Did your message receive a favorable and appropriate intended audience reaction?
- Did your message fulfill its communication objectives?
- What are your message's strengths? Weaknesses?
- Did answers to any particular question stand out?
Use your answers to these questions to decide whether your message is both effective and appropriate and whether you need to revise your message prior to program implementation.
Diaries and Activity Logs
Other tools you can use to evaluate your program are diaries and activity logs. If you plan to use these tools to gauge the quality of program planning or execution, be sure to start keeping the diaries and activity logs as soon as you begin program planning. For each activity, request information in a specific format from program managers or participants. This information may cover issues such as the quality of program components or track how your intended audience uses the components.
Common Uses
- Track program implementation
- Assess effectiveness of program implementation
- Monitor whether planned activities are being conducted on schedule and within budget
- Learn what questions program participants had
- Learn what technical assistance was needed by program staff
- Track intended audience exposure to program components
- Allow respondents flexibility in their responses
- Enable researchers to observe behavior over time, rather than only once
- Require considerable effort on respondents' parts (for this reason, consider offering incentives for completion of the diaries/logs)
- Require staff able to code voluminous and challenging incoming data
- Are not appropriate for respondents who have low literacy skills or who have poor writing skills or penmanship
Instituting Diary/Activity Log Use
Steps in instituting the keeping of diaries and activity logs are:
- Planning the study
- Identifying who will participate
- Developing and pretesting the form you will use
- Collecting data
- Analyzing results
Follow the steps below to institute the keeping of diaries and activity logs.
Plan the Study
Determine the following:
- What you want to learn
- How much information you need to collect
- How you will apply what you learn
- When you need the information
- What your budget is
- Your criteria for who should participate
Identify Who Will Participate
The sample you select depends on the goals of your study. If you are focusing on program implementation, you will want the diaries/logs to be completed by program staff (e.g., nurses in a clinic). In this case, you may have some control over the quality of responses you receive.
When planning the study, you must obtain permission from a manager or supervisor on site for staff to complete the diaries/logs during the study.You should provide an estimate of the amount of time and effort participation will entail (e.g., 15 minutes per day, 1 hour per day). Share drafts of the diaries/logs and get input from the supervisor prior to the study. This will help to ensure cooperation during the study.
Before the study begins, you should train staff to complete the diaries/logs. Even if it seems obvious to you, it is essential that you explain exactly what you want recorded in the diary/log. (See the sample log in Appendix A.) In addition, you should provide detailed, written instructions for future reference. These instructions can be used in lieu of training if you cannot physically get to the study site.
If you are focusing on participant experience with a program, you will want the diaries/logs to be completed by people who were exposed to program components. In this case, you will have much less control over the quality and quantity of responses.
Obtaining cooperation from participants may also be more difficult in this situation. For example, people attending an educational program on nutrition might be recruited to complete a diary of what they eat for a week and send it back to the researchers.You will likely need to provide an incentive (e.g., a gift certificate upon receipt of the completed diary), and you may also need to remind participants to send back the diaries at the end of the study period.
Develop and Pretest the Form You Will Use
Once you have identified what you want to learn and who will complete the diaries/logs, you must create a user-friendly document to collect the necessary information.Create questions. Write questions that are specific to your study objectives.
Examples of the types of information you might collect include:
- For a toll-free hotline activity log:
- Date?
- What is the gender of the caller?
- Where did the caller get the number?
- Did the caller request any educational materials?
- What questions did the caller ask?
- For a health education program diary:
- Date?
- Which module was used?
- Which of the suggested activities were completed?
- How long did it take to complete the suggested activities?
- Would you use these activities again?
- Why or why not?
- For a clinic observation activity log:
- Date?
- How many people passed by the display containing the new breast cancer information?
- How many stopped to look at the information?
- How many took a brochure?
Pretest the diary/log. Once you have created the draft diary/log, you must pretest it with individuals who represent your intended audience. Describe the scenario for them before the pretest. For example, in the case of a hotline, you might say, "You are an operator on a hotline. People will be calling in, and you will need to fill out this activity log as you complete each call." Sit together with them and ask them to read each question aloud and tell you what they think they are supposed to do. Do not correct them if they do not say what you intended. This probably means that your diary/log is unclear. Continue through the entire diary/log and then ask them if there was anything that they found confusing or unclear. Pretest the diary/log with everyone as planned before you make any changes.
Revise the diary/log. Revise questions that people found confusing during the pretest. If a question was confusing only to one person, use your judgment to decide whether to change the question. Ask yourself whether there is something you can easily fix that would have helped that one person understand the question (e.g., providing an example). If so, you may be able to make a simple change or addition to clarify the question. Also consider whether this respondent found many of the questions confusing while other respondents had no problem with them. If this is the case, you may not want to make changes.You will have to decide on a case-by-case basis. If you make substantial changes to the diary/log, you should conduct another pretest before finalizing the form.
Collect Data
Produce diaries/logs in sufficient quantities so that respondents have several extra forms in case they make errors or need more space. Deliver the diaries/logs to respondents, along with detailed written instructions, prior to training (if applicable) or at least 1 week before the study begins. If you are asking program participants rather than program staff to complete diaries/logs for you, you will have to distribute the materials on site. Give respondents a fixed time frame to complete the diaries/logs and provide them with a means (envelope/postage) to return the data to you. If your study is longer than a week or two, you may want to ask respondents to ship the first week of data to you so that you can review the logs for accuracy and completeness and even begin to tally some of the information.
Analyze Results
In the planning phase, you determined what you wanted to learn from the study. Now you will have the chance to look through the diaries/logs to answer these questions. Diaries generally contain qualitative information (e.g., how food choices were made that day, evaluation of programs completed). Activity logs may contain several types of information—quantitative information you can tabulate easily (e.g., how many people called a hotline each day, whether people picked up a brochure) as well as qualitative information (e.g., reasons that students liked or participated in an activity).
Analyzing qualitative responses. The best way to analyze qualitative information is to read through the information, searching for similarities and differences between diaries. You will need to consider all of the questions that you determined were important in the planning phase. Once you have reviewed several diaries, you should be able to pull out general themes or patterns from the information. The best way to analyze these themes is to develop categories for the responses. For example, if you want to know why teachers thought their students liked or disliked a certain educational module in your program, you might group responses into categories such as "challenging," "fun," "too much work," "boring." Continue reading through the remaining diaries and see how many responses fall into these categories. As you go along, you may come up with additional categories or decide to collapse several categories together.You can certainly make inferences (e.g., "Teachers liked the module becau...") about diary information, but resist the temptation to quantify this information.
Analyzing quantitative responses. The easiest way to analyze these types of responses is to create a coding sheet for each quantitative question. Use a separate sheet for each question, writing the question at the top and creating columns for each possible response. For example, for a question about how many people picked up particular brochures, you could create columns for the following categories: 0, 1–5, 6–10, 11–15, 16–20, >20.
Use the following procedure to record the responses:
- Take the first activity log and record the response by making a check mark in the appropriate column.
- Repeat this procedure for every questionnaire.
- Tally the total number of check marks in each column and then calculate the percentage of participants who gave each type of response.
Quantitative Research Methods
Use quantitative research methods during the following parts of your program:
- Stage 1—to obtain information on prevalence of relevant knowledge, attitudes, behaviors, and behavioral intentions
- Stage 3—to monitor usage of materials and intended audience awareness of the communication program and its various tactics
- Stage 4—to measure progress toward objectives
Two different quantitative research methods, surveying and readability testing, can be used.
Surveys are characterized by large numbers of respondents (100 or more) and questionnaires that contain predominantly forced-choice (closed-ended) questions.
Common Uses
Used in planning and assessment to obtain baseline and tracking information on knowledge, attitudes, behaviors, and behavioral intentions
- Provides generalizable results (to generalize to a broader population, you must have a statistically valid random sample)
- Can be anonymous (useful for sensitive topics)
- Can incorporate visual material (e.g., can pretest prototype materials)
- Limited ability to probe answers
- Potential bias from possible respondent self-selection
Most surveys are custom studies that are designed to answer a specific set of research questions. Some surveys, however, are omnibus studies, in which you add questions about your topic to an already existing survey. A number of national and local public opinion polls offer this option.
Pros and Cons of Various Survey Formats
Pros- Can be used to cost–effectively access difficult–to–reach populations (e.g., the homebound, rural populations)
- Respondents can answer questions when most convenient for them
- Not appropriate for respondents with limited literacy skills
- Low response rate diminishes value of results
- May require extensive/ expensive followup by mail or telephone to increase response rate
- Respondents may return incomplete questionnaires
- Can be difficult to read responses>
- May take long time to receive sufficient numbers of responses
- Postage may be expensive if sample is large, questionnaire is long, or multiple reminder cards are needed
- Telephone (With interviewer using paper-and-pencil questionnaires)
Pros- Appropriate for those with limited literacy skills
- Results in more complete responses because interviewer fills out questionnaires
- Can control question sequence
- Potential respondents without phones cannot participate
- Respondents may hang up if they believe the survey is part of a solicitation call or they don't want to take time to participate
- Using computer-assisted telephone interviewing (CATI)
Pros- Useful for complex questionnaires because "skip patterns" can be programmed in
- Data entry is eliminated
- Requires CATI software and computers
- Requires extensive interviewer training
- Requires time to program questionnaire into CATI
- In Person (Interviewer-administered)
Pros- Face-to-face persuasion tactics can be used to increase response rates
- Can be used with those with limited literacy skills
- Useful with difficult-to-reach populations (e.g., homeless, rural) or when intended audience cannot be sampled using other data collection methods
- Interviewer can clarify questions for respondents
- Results in more complete responses because interviewer fills out questionnaires
- More expensive than self–administered or telephone data collection
- Not appropriate for sensitive, threatening, or controversial questions (respondents may not answer as truthfully in person)
- Written, self-administered
(Respondents are asked to complete survey at a location frequented by the target population (e.g., during a conference, in a classroom, after viewing an exhibit at a health fair)
Pros- Can connect with harder-to-reach respondents in locations convenient and comfortable for them
- Can be conducted quickly
- Cost-effective means of gathering data in relatively short time
- Can result in increased number of respondents within intended audience if appropriate location chosen
- Must be able to reach respondents in person at a central location or a gathering
- Respondents must have adequate literacy skills and be self-motivated to respond
- Computerized, self–administered
(Questionnaire is displayed on computer screen and respondents key in their answers)
Pros- Useful for complex questionnaires because computerized "skip patterns" can be used
- Can control question sequence
- Eliminates data entry and provides quick summary or analysis of results
- Not appropriate for intended audiences with limited literacy skills or those unfamiliar or uncomfortable with computers
- Requires expensive technical equipment that may not be readily available or may be cumbersome in many settings
- Internet (Computerized, self–administered Questionnaire is displayed on respondent's computer screen via a Web site)
Pros- Useful for complex questionnaires because computerized "skip patterns" can be used
- Can control question sequence
- Eliminates data entry and provides a quick summary or analysis of results
- Not appropriate for audiences with limited literacy skills or those unfamiliar with or uncomfortable with computers
- Respondents must have access to the Internet and be somewhat familiar with using computers
- No way to confirm the validity of information provided by respondents
Follow these steps to conduct a survey:
- Plan the study
- Determine how the sample will be obtained and contacted
- Develop and pretest the questionnaire
- Collect the data
- Analyze results
Sampling size and composition, questionnaire design, and analysis of quantitative data are complex topics beyond the scope of this book. If you are planning a quantitative study, see the reference list at the end of this book for additional information.
Additional Research Methods
Gatekeeper Reviews
Public and patient education materials are often routed to their intended audiences through health professionals or other individuals or organizations that can communicate with these audiences for you. These intermediaries act as gatekeepers, controlling the distribution channels that reach your intended audiences. Their approval or disapproval of materials can be a critical factor in your program's success. If they do not like a poster or a booklet or do not believe it to be credible or scientifically accurate, it may never reach your intended audience.
Common Uses
Gatekeeper review of rough materials is important and should be considered part of the pretesting process, although it is not a substitute for pretesting materials with intended audience members. Neither is it a substitute for obtaining clearances or expert review for technical accuracy; these should be completed before pretesting is undertaken. Sometimes, telling gatekeepers that technical experts have reviewed the material for accuracy will reassure them and may speed their approval of your material.
The methodology you should use for gatekeeper review depends upon your available resources, time, and budget. Common methods include:
- Self-administered questionnaires—
Participants are sent the materials and the questionnaire at the same time. - Interviewer-administered questionnaires—
Typically, an appointment for the interview is scheduled with the gatekeeper, and the materials are sent for review prior to the interview.
Develop questionnaires that ask about overall reactions to the materials and for assessment of the information's appropriateness and usefulness.
In some cases, you might not use a formal questionnaire (especially if you don't think the reviewer will take the time to fill it out) but will instead schedule a telephone conversation or a meeting about the materials. If you are not using a questionnaire, consider in advance what kind of questions you want to ask in the meeting or interview and determine whether you need formal approval of the materials. A discussion with gatekeepers (e.g., a television public service director, the executive director of a medical society) at this point can also be used to solicit their involvement in a variety of ways that extend beyond materials development.
Readability Testing*
Readability formulas often are used to assess the reading level of materials. Fry, Flesch, FOG, and SMOG are among the most commonly used. Applying these formulas is a simple process that can be done manually or by using a computer software program. Each method takes only a few minutes.
Typically, readability formulas measure the difficulty of the vocabulary used and the average sentence length. In addition, computer software programs analyze a document's grammar, style, word usage, and punctuation, and assign a reading level. These formulas, however, do not measure the reader's level of comprehension.
Readability software programs are available at computer stores. Some software programs, such as Microsoft Word, include a readability-testing function. (Note: Mention of software products does not constitute an endorsement by the National Cancer Institute.)
Researchers James Pichert and Peggy Elam suggest three principles for using readability formulas effectively:
- Use readability formulas only in concert with other means of assessing the effectiveness of the material.
- Use a formula only when the text's intended readers are similar to those on whom the formula was validated.
- Do not write a text with readability formulas in mind.
Before you choose a readability testing method, decide on an appropriate reading level for the materials you've written. Then use readability testing to determine whether your text corresponds to the reading level you want.
The term reading level refers to the number of years of education required for a reader to understand a written passage. Some experts suggest aiming for a level that is two to five grades lower than the highest average grade level of your intended audience to account for a probable decline in reading skills over time. Others note that a third- to fifth-grade level is frequently appropriate for low-literacy readers. Keep publications as simple as possible to increase reader comprehension of the material.
Readability Testing Methods
You can test readability easily using a formula such as Fry, Flesch, FOG, or SMOG. These tests can be done quickly to indicate any problems with the drafted text. They do not involve the intended audience.
To calculate the SMOG reading grade level of a written sample, begin with the entire written work that is being assessed, and follow these four steps:
- Count off 10 consecutive sentences each near the beginning, in the middle, and near the end of the text.
- From this sample of 30 sentences, circle all of the words containing three or more syllables (polysyllabic), including repetitions of the same word, and total the number of words circled.
- Estimate the square root of the total number of polysyllabic words counted. Do this by finding the nearest perfect square and taking its square root.
- Finally, add a constant of three to the square root. The resulting number is the SMOG grade or the reading grade level that a person must have reached to fully understand the text being assessed.
A few additional guidelines will help to clarify these instructions:
- A sentence is defined as a string of words punctuated with a period, an exclamation point, or a question mark.
- Hyphenated words are considered one word.
- Numbers that are written out should also be considered, and if in numeric form in the text, they should be pronounced to determine whether they are polysyllabic.
- Proper nouns, if polysyllabic, should be counted, too.
- Abbreviations should be read as unabbreviated to determine whether they are polysyllabic.
Not all pamphlets, fact sheets, or other printed materials contain 30 sentences. To test a text that has fewer than 30 sentences:
- Count all of the polysyllabic words in the text.
- Count the number of sentences.
- Find the average number of polysyllabic words per sentence as follows:
Average = Total # of polysyllabic words
Total # of sentences - Subtract the total number of sentences from 30.
- Multiply that number by the average.
- Add that figure to the total number of polysyllabic words.
- Find the square root and add a constant of 3.
Perhaps the quickest way to administer the SMOG test is by using the SMOG conversion table. Simply count the number of polysyllabic words in 30 sentences (chains of 10 each from the beginning, middle, and end of the text) and look up the approximate grade level on the chart. See the sidebar Example Using the SMOG Readability Formula on the next page for an example of how to use the SMOG Readability Formula and the SMOG conversion table. In the sidebar, each of the 3 sets of 10 sentences is marked with brackets.
Readability Testing With the Intended Audience*
Other methods of evaluating reading levels and comprehension include having your intended audience pretest your materials. The WRAT or the Cloze technique can be used to do this. These types of testing are useful when you suspect that the intended audience may encounter difficulties with the material. Including pretest participants who have the same characteristics as the lowliteracy intended audience you are trying to reach is critical to the validity of your pretest results. Recruiting participants through groups or settings that include people with limited literacy skills is a logical starting point. But the only way to be sure your pretest volunteers read at the same level as your intended audience is to test their reading skills.
The Wide Range Achievement Test (WRAT) is used to measure reading levels, and the Cloze technique is used to measure comprehension. To avoid offending or causing discomfort to those whose reading ability you are testing, you can integrate a WRAT or a Cloze test into the pretest interview. For example, in a recent pretest conducted by the National Cancer Institute, the interviewers introduced the WRAT test as the last part of the pretest. They stated, "Thank you for helping with the questions on the chemotherapy booklet.We need your help with one last part—a word list. This will take only a few minutes. The word list will help us understand how difficult the words are in the chemotherapy booklet." This integrated approach spared participants the pressure or potential embarrassment of "failing a reading test."
The WRAT is based on word recognition and does not measure comprehension or vocabulary. The WRAT is an efficient way to determine reading levels and takes only a short time to administer.
The WRAT involves listening to the participant read from a prepared list of words, arranged in increasing order of difficulty. Pronouncing the word correctly shows that the reader recognizes the word. The WRAT focuses on recognition because, at the most basic level, if a person does not recognize a word, comprehension is impossible.
The test is over after the reader mispronounces 10 words. The test administrator notes the level at which the last mispronunciation occurred. The "stop" level equates to a grade level of reading skills.You can compare this level with the reading level of your intended audience to see if your pretest readers are a representative match for that audience.
The Cloze technique measures the reader's ability to comprehend a written passage. Because it requires readers to process information, it may take up to 30 minutes to administer.
In a Cloze test, text appears with every fifth word omitted. The reader tries to fill in the blanks. This task demonstrates how well he or she understands the text. The reader's ability to supply the correct word also reflects his or her familiarity with sentence structure.
While packaged Cloze tests are available, Leonard and Cecilia Doak's Teaching Patients with Low Literacy Skills explains how to make up and score a Cloze test yourself, based on the materials you are pretesting. The book also discusses use of the WRAT to assess reading levels.
Example Using the SMOG Readability Formula
Benign Prostate Hyperplasia (BPH)
[Benign prostatic hyperplasia (BPH) is an enlarged prostate. Benign means noncancerous and hyperplasia means excessive growth of tissue. BPH is the result of small, noncancerous growths inside the prostate. It is not known what causes these growths, but they may be related to hormone changes that occur with aging. By age 60, more than half of all American men have microscopic signs of BPH, and by age 70, more than 40 percent will have enlargement that can be felt on physical examination.
The prostate normally starts out about the size of a walnut. By the time a man is age 40, the prostate may already have grown to the size of an apricot; by the age of 60, it may be as big as a lemon.
BPH, which usually does not affect sexual function, is a troublemaker because the prostate, as it enlarges, presses against the bladder and the urethra, blocking the flow of urine.
A man with BPH may find it difficult to initiate a urine stream or to maintain more than a dribble. He also may need to urinate frequently, or he may have a sudden, powerful urge to urinate.] Many men are forced to get up several times a night; others have an annoying feeling that the bladder is never completely empty.
Straining to empty the bladder can make matters worse; the bladder stretches, the bladder wall thickens and loses its elasticity, and the bladder muscles become less efficient. The pool of urine that collects in the bladder can foster urinary tract infections, and trying to force a urine stream can produce back pressure that eventually damages the kidneys. The kidneys are where urine is formed as waste products are filtered from the blood.
BPH sometimes leads to problems. [For instance, a completely blocked urethra is a medical emergency requiring immediate catheterization, a procedure in which a tube called a catheter is inserted through the penis into the bladder to allow urine to escape. Other serious potential complications of BPH include bladder stones and bleeding.
Diagnosing BPH
A detailed medical history focusing on the urinary tract—kidneys, ureters (the pair of tubes that carry urine from the kidneys to the bladder), the bladder, and the urethra— allows the doctor to identify symptoms and to evaluate the possibility of infection or other urinary problems.
The initial medical workup typically includes a physical exam called a digital rectal examination (DRE), a urinalysis to check for infection or bleeding, and a blood test to measure kidney function. Some physicians may also check the amount of prostate-specific antigen (PSA) using a PSA test to help rule out the likelihood of cancer. PSA is a protein that is produced by the cells of the prostate gland.
In addition, other tests may help a urologist—a doctor who specializes in disorders of the urinary tract and the male reproductive tract—to determine if BPH has affected the bladder or kidneys. These include tests that measure the speed of urine flow, pressure in the bladder during urination, and how much urine is left in the bladder after urinating.
Some other tests that are widely used, according to an expert panel sponsored by the United States Public Health Service (USPHS) practice guidelines, are expensive, sometimes risky, and, for most men, unnecessary. These include cystoscopy, in which the doctor inserts a viewing tube up the urethra to get a direct look at the bladder; an x-ray called a urogram, in which urine is made visible on an x-ray after dye is injected into a vein; and ultrasound, which obtains images of the kidneys and bladder after a probe is placed on the abdomen.]
Treating BPH
About half of men with BPH develop symptoms serious enough to warrant treatment. BPH cannot be cured, but its symptoms can be relieved by surgery or by drugs in many cases.
BPH does not necessarily grow worse. According to one review, mild to moderate symptoms worsened in only about 20 percent of the cases. They improved (without any specific treatment) in another 20 percent and stayed about the same in the rest.
[Men whose symptoms are mild enough often opt for an approach called watchful waiting. This means that they report for regular checkups and have further treatment only if and when their symptoms become too bothersome.
The USPHS Clinical Practice Guidelines call watchful waiting "an appropriate treatment strategy for the majority of patients." Men who choose watchful waiting should have regular, perhaps annual, checkups, including DREs and laboratory tests.
For those who choose watchful waiting, a number of simple steps may help to reduce bothersome symptoms. These include limiting fluid intake in the evening, especially beverages containing alcohol or caffeine, which can trigger the urge to urinate and can interfere with sleep; taking time to empty the bladder completely; and not allowing long intervals to pass without urinating.
Men monitoring prostate conditions should also be aware that certain medications they are taking for other ailments may make their symptoms worse. These include some over-the-counter cough and cold remedies, prescribed tranquilizers, antidepressants, and drugs to control high blood pressure. Switching to a different prescription may help.
Watchful waiting, of course, is not always enough for BPH, and surgery or drug therapy may be required.]
Readability Test Calculations | |
Total Number of Polysyllabic Words | = 104 |
Nearest Perfect Square | = 100 |
Square Root | = 10 |
Constant | = 3 |
SMOG Reading Grade Level | = 13 |
We have calculated the reading grade level for this example. Compare your results to ours, and then check both with the SMOG conversion table: | |
SMOG Conversion Table* | |
Total Polysyllabic Word Counts | Approximate Grade Level (± 1.5 Grades) |
0 — 2 | 4 |
3 — 6 | 5 |
7 — 12 | 6 |
13 — 20 | 7 |
21 — 30 | 8 |
31 — 42 | 9 |
43 — 56 | 10 |
57 — 72 | 11 |
73 — 90 | 12 |
91 — 110 | 13 |
111 — 132 | 14 |
133 — 156 | 15 |
157 — 182 | 16 |
183 — 210 | 17 |
211 — 240 | 18 |
* Developed by Harold C. McGraw, Office of Educational Research, Baltimore County Schools, Towson, Maryland. |
* Adapted from Clear and Simple: Developing Effective Print Materials for Low-Literate Readers (NIH Publication No. 95-3594), by the National Cancer Institute, 1994. Bethesda, MD. In the public domain.
Appendix A: Communication Planning Forms and Samples
The following is a listing of forms and samples that appear in this appendix.
Stage 1: Planning and Strategy Development
Stage 2: Developing and Pretesting Concepts, Messages, and Materials
Stage 3: Implementing the Program
- Breast Cancer Materials User Survey
- Breast Cancer Risk Assessment Tool Evaluation: Summary of Responses
Communication Research Methods
Please also see Stage 2 forms listed above.
- Producing Rough-Cut Video for Pretesting/Theater Testing
- Sample Theater-Style Pretest Questionnaire
- Sample Script for Hosting a Theater-Style Test
- Tabulating Responses to Closed- and Open-Ended Questions
- Average Ratings for Commercial Ads to Help Interpret Standard Pretesting Questions
Appendix B: Selected Planning Frameworks, Social Science Theories, and Models of Change*
* From Theory at a Glance: A Guide for Health Promotion Practice (NIH Publication No. 97-3896), by the National Cancer Institute, 1995. Bethesda, MD. In the public domain.
Planning Systems/Frameworks
Once health communications planners identify a health problem, they can use a planning framework such as the two described below: social marketing and PRECEDE-PROCEED. These planning systems can help identify the social science theories most appropriate for understanding the problem or situation. Thus, planners use the theories and models described below within the construct of a planning framework.
Using planning systems like social marketing and PRECEDE-PROCEED increases the odds of program success by examining health and behavior at multiple levels. Planning system perspectives emphasize changing people, their environment, or both.
Social Marketing
Social marketing has been defined as "the application of commercial marketing technologies to the analysis, planning, execution, and evaluation of programs designed to influence the voluntary behavior of target audiences in order to improve their personal welfare and that of their society" (Andreason, 1995). This definition encompasses several key aspects of the social marketing approach; it is seen as:
- A key benefit to individuals and society; not focused on profit and organizational benefits as commercial marketing practices are
- A focus on behavior, not awareness or attitude change
- An approach centered on the target audience's having a primary role in the process
Social marketing practices are based on commercial marketing practices that make the consumer the central focus for planning and conducting a program. The program's components address:
- Price—what the consumer must give up in order to receive the program's benefits (these costs may be intangible [e.g., changes in beliefs or habits] or tangible [e.g., money, time, or travel])
- Product—what the program is trying to change within the intended audience and what the audience stands to gain
- Promotion—how the exchange is communicated (e.g., appeals used)
- Place—what channels the program uses to reach the intended audience (e.g., mass media, community, or interpersonal)
The formulation of price, product, promotion, and place evolves from research with the consumers to determine what benefits and costs they would consider acceptable and how they might be reached. Lessons learned from social marketing stress the importance of understanding the intended audiences and designing strategies based on their wants and needs rather than what good health practice directs that they should do.
For Further Reading
Andreason, A. (1995). Marketing social change: Changing behavior to promote health, social development, and the environment. San Francisco: Jossey-Bass.
Kotler, P., & Roberto, E. L. (1989). Social marketing: Strategies for changing public behavior. New York: Free Press.
Lefebvre, R. C., & Rochlin, L. (1997). Social marketing. In K. Glanz, F. M. Lewis, & B. K. Rimer (Eds.), Health behavior and health education: Theory, research, and practice (2nd ed.). San Francisco: Jossey-Bass.
The PRECEDE-PROCEED framework is an approach to planning that examines the factors contributing to behavior change. These include:
- Predisposing factors—the individual's knowledge, attitudes, behavior, beliefs, and values before intervention that affect willingness to change
- Enabling factors—factors in the environment or community of an individual that facilitate or present obstacles to change
- Reinforcing factors—the positive or negative effects of adopting the behavior (including social support) that influence continuing the behavior
These factors require that individuals be considered in the context of their community and social structures, and not in isolation, when planning communication or health education strategies.
PRECEDE-PROCEED Framework
For Further Reading
Green, L.W., & Kreuter, M.W. (1999). Health promotion planning: An educational and ecological approach (3rd ed.). Mountain View, CA: Mayfield.
Green, L.W., & Ottoson, J. M. (1999). Community and population health (8th ed.). New York: McGraw-Hill.
Selected Social Science Theories, Models, and Constructs
Individual Level
Behavioral Intentions
Studies of behavioral intentions suggest that the likelihood of intended audiences' adopting a desired behavior can be predicted by assessing (and subsequently trying to change or influence) their attitudes toward and perceptions of benefits of the behavior, along with how they think that their peers will view their behavior. Research by Fishbein and Ajzen supports the idea that individuals' and society's (perceived) attitudes are an important predecessor to action. Therefore, an important step toward influencing behavior is a preliminary assessment of intended audience attitudes, and subsequent tracking is necessary to identify any attitudinal changes.
For Further Reading
Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.
Communications for Persuasion
William McGuire has described the steps an individual must be persuaded to pass through to assimilate a desired behavior. These steps are:
- Exposure to the message
- Attention to the message
- Interest in or personal relevance of the message
- Understanding of the message
- Personalizing the behavior to fit one's life
- Accepting the change
- Remembering the message and continuing to agree with it
- Being able to think of it
- Making decisions based on bringing the message to mind
- Behaving as decided
- Receiving positive reinforcement for behavior
- Accepting the behavior into one's life
To communicate the message successfully, five communication components all must work:
- Credibility of the message source
- Message design
- Delivery channel
- Intended Audience
- Intended behavior
Paying attention to McGuire's steps helps ensure that a communication program plan addresses all the factors that determine whether a message is received and absorbed, that the program is staged over time to address intended audience needs as they differ over time, and that progress is being made toward behavior change.
For Further Reading
McGuire, W. J. (1984). Public communication as a strategy for inducing health-promoting behavioral change. Preventive Medicine, 13(3), 299−313.
Stages of Change
The basic premise of the stages-of-change construct, the central construct of the transtheoretical model, is that behavior change is a process and not an event and that individuals are at varying levels of motivation, or readiness, to change. People at different points in the process of change can benefit from different interventions, matched to their stage at that time.
By knowing an individual's current stage, you can help set realistic program goals.You can also tailor messages, strategies, and programs to the appropriate stage.
Five distinct stages are identified in the stages-of-change construct:
- Precontemplation
- Contemplation
- Decision/determination
- Action
- Maintenance
It is important to note that this is a circular, not a linear, model. People don't go through the stages and "graduate"; they can enter and exit at any point, and often recycle.
For Further Reading
Prochaska, J. O., & Velicer, W. F. (1997). The transtheoretical model of health behavior change. American Journal of Health Promotion, 12(1), 38–48.
Health Belief Model
The health belief model (HBM) was originally designed to explain why people did not participate in programs to prevent or detect diseases. The core components of HBM include:
- Perceived susceptibility—the subjective perception of risk of developing a particular health condition
- Perceived severity—feelings about the seriousness of the consequences of developing a specific health problem
- Perceived benefits—beliefs about the effectiveness of various actions that might reduce susceptibility and severity (taken together, perceived susceptibility and severity are labeled "threat")
- Perceived barriers—potential negative aspects of taking specific actions
- Cues to action—bodily or environmental events that trigger action
More recently, HBM has been amended to include the notion of self-efficacy as another predictor of health behaviors'especially more complex ones in which lifestyle changes must be maintained over time. A wide variety of demographic, social, psychological, and structural variables may also impact people's perceptions and, indirectly, their health-related behaviors. Some of the more important variables include educational attainment, age, gender, socioeconomic status, and prior knowledge.
For Further Reading
Janz, N. K., & Becker, M. H. (1984). The health belief model: A decade later. Health Education Quarterly, 11, 1–47.
Strecher, V. J., & Rosenstock, I. M. (1997). The health belief model. In K. Glanz, F. M. Lewis, & B. K. Rimer (Eds.), Health behavior and health education: Theory, research, and practice (2nd ed.). San Francisco: Jossey-Bass.
Consumer Information Processing Model
The consumer information processing (CIP) model was not developed specifically to study health-related behavior, nor to be applied in health promotion programs, but it has many useful applications in the health arena. Information is a common tool for health education and is often an essential foundation for health decisions.
Information can increase or decrease people's anxiety, depending on their information preferences and how much and what kind of information they are given. Also, illness and its treatments can interfere with information processing. By understanding the key concepts and processes of CIP, health educators can examine why people use or fail to use health information and design informational strategies to better chances for success. CIP theory reflects a combination of rational and motivational ideas. The use of information is an intellectual process; however, motivation drives the search for information and how much attention people pay to it. The central assumptions of CIP are that 1) individuals are limited in how much information they can process, and 2) to increase the usability of information, they combine bits of information into "chunks" and create decision rules, known as heuristics, to make choices faster and more easily. According to basic CIP concepts, before people will use health information, it must be 1) available, 2) seen as useful and new, and 3) processable, or format-friendly.
For Further Reading
Bettman, J. R. (1979). An information processing theory of consumer choice. Reading, MA: Addison-Wesley.
Interpersonal Level
Social Cognitive Theory
Social cognitive theory (SCT) explains behavior in terms of triadic reciprocality ("reciprocal determinism") in which behavior, cognitive and other interpersonal factors, and environmental events all operate as interacting determinants of one another. SCT describes behavior as dynamically determined and fluid, influenced by both personal factors and the environment. Changes in any of these three factors are hypothesized to render changes in the others.
One of the key concepts in SCT is the environmental variable: observational learning. In contrast to earlier behavioral theories, SCT views the environment as not just a variable that reinforces or punishes behaviors, but one that also provides a milieu where an individual can watch the actions of others and learn the consequences of those behaviors. Processes governing observational learning include:
- Attention—gaining and maintaining attention
- Retention—being remembered
- Reproduction—reproducing the observed behavior
- Motivation—being stimulated to produce the behavior
Other core components of SCT include:
- Self-efficacy—judgment of one's capability to accomplish a certain level of performance
- Outcome expectation—judgment of the likely consequence such behavior will produce
- Outcome expectancies—the value placed on the consequences of the behavior
- Emotional coping responses—strategies used to deal with emotional stimuli, including psychological defenses (denial, repression), cognitive techniques such as problem restructuring, and stress management
- Enactive learning—learning from the consequences of one's actions (versus observational learning)
- Rule learning—generating and regulating behavioral patterns, most often achieved through vicarious processes and capabilities (versus direct experience)
- Self-regulatory capability—much of behavior is motivated and regulated by internal standards and people's self-evaluative reactions to their own actions
For Further Reading
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.
Lefebvre, R. C. (2000). Theories and models in social marketing. In P. N. Bloom & G. T. Gundlach (Eds.), Handbook of marketing and society. Thousand Oaks, CA: Sage.
Organization/Community/Societal Level
Organizational Change Theory
Organizations are complex and layered social systems, composed of resources, members, roles, exchanges, and unique cultures. Thus, organizational change can best be promoted by working at multiple levels within the organization. Understanding organizational change is important in promoting health to help establish policies and environments that support healthy practices and create the capacity to solve new problems.While there are many theories of organization behavior, two are especially promising in public health interventions: stage theory and organizational development (OD) theory.
Stage theory is based on the idea that organizations pass through a series of steps or stages as they change. By recognizing those stages, strategies to promote change can be matched to various points in the process of change. An abbreviated version of stage theory involves four stages:
- Problem definition (awareness)
- Initiation of action (adoption)
- Implementation
- Institutionalization
OD theory grew out of the recognition that organizational structures and processes influence worker behavior and motivation. OD theory concerns the identification of problems that impede an organization's functioning, rather than the introduction of a specific type of change. Human relations and quality of work-life factors are often the targets of OD problem diagnosis, action planning, interventions, and evaluation. A typical OD strategy involves process consultation, in which an outside specialist helps identify problems and facilitates the planning of change strategies.
Stage theory and OD theory have the greatest potential to produce health-enhancing change in organizations when they are combined. That is, OD strategies can be used at various stages as they are warranted. Simultaneously, the stages signal the need to involve organization members and decision-makers at various points in the process.
For Further Reading
Beyer, J. M., & Trice, H. M. (1978). Implementing change: Alcoholism policies in work organizations. New York: Free Press.
Porras, J. I., & Roberston, P. J. (1987). Organization development theory: A typology and evaluation. In R.W.Woodman & W. A. Pasmore (Eds.), Research in organizational change and development (Vol. 1). Greenwich, CT: JAI Press.
Community Organization Theory
Community organization theory has its roots in theories of social networks and support. It emphasizes active participation and developing communities that can better evaluate and solve health and social problems. Community organization is the process by which community groups are helped to identify common problems or goals, mobilize resources, and develop and implement strategies for reaching their goals. It has roots in several theoretical perspectives: the ecological perspective, social systems perspective, social networks, and social support. It is also consistent with social learning theory (SLT) and can be successfully used along with SLT-based strategies. Community organization is composed of several alternative change models:
- Locality development (also called community development) uses a broad cross-section of people in the community to identify and solve its own problems. It stresses consensus development, capacity building, and a strong task orientation; outside practitioners help to coordinate and enable the community to successfully address its concerns.
- Social planning uses tasks and goals, and addresses substantive problem solving, with expert practitioners providing technical assistance to benefit community consumers.
- Social action aims to increase the problem-solving ability of the community and to achieve concrete changes to redress social injustice that is identified by a disadvantaged or oppressed group.
Although community organization does not use a single unified model, several key concepts are central to the various approaches. The process of empowerment is intended to stimulate problem solving and activate community members. Community competence is an approximate community-level equivalent of self-efficacy plus behavioral capability, which are the confidence and skills to solve problems effectively. Participation and relevance go together: They involve citizen activation and a collective sense of readiness for change. Issue selection concerns identifying "winnable battles" as a focus for action, and critical consciousness stresses the active search for root causes of problems.
Social action approaches to community organizing go beyond the traditional notion of geographic and political boundaries. Communities of people who share common health problems have coalesced to attract attention to and to obtain power to address their needs—including health services, antidiscrimination policies, and more research funding. Foremost among these groups presently are AIDS activists.Women's health advocates have also used social action to pressure powerful institutions to address their problems; breast cancer is now a focus for action and advocacy among breast cancer survivors and their relatives. They have used media advocacy as a powerful tool in their efforts.
Media advocacy is the strategic use of mass media as a resource for advancing a social or public policy initiative. It is an important, and often essential, part of social action and advocacy campaigns because the media focus public concern and spur public action. The core components of media advocacy are developing an understanding of how an issue relates to prevailing public opinions and values and designing messages that frame the issues so as to maximize their impact and attract powerful and broad public support.
For Further Reading
Rothman, J., & Tropman, J. E. (1987). Models of community organization and macro practice: Their mixing and phasing. In F. M. Cox, J. L. Ehrlich, J. Rothman, & J. E. Tropman (Eds.), Strategies of community organization (4th ed.). Itasca, IL: Peacock.
Diffusion of Innovations Theory
Diffusion of innovations theory addresses how new ideas, products, and social practices spread within a society or from one society to another. The challenge of diffusion requires approaches that differ from those focused solely on individuals or small groups. It involves paying attention to the innovation (a new idea, product, practice, or technology) as well as to communication channels and social systems (networks with members, norms, and social structures).
A focus on the characteristics of innovations can improve the chances that they will be adopted and hence diffused. It also has implications for how an innovation is positioned to maximize its appeal. Some of the most important characteristics of innovations are their:
- Relative advantage—is it better than what was there before?
- Compatibility—fit with intended audience
- Complexity—ease of use
- Trialability—can it be tried out first?
- Observability—visibility of results
Communication channels are another important component of diffusion of innovations theory. Diffusion theories view communication as a two-way process rather than one of merely "persuading" an intended audience to take action. The two-step flow of communication—in which opinion leaders mediate the impact of mass media—emphasizes the value of social networks (or interpersonal channels) over and above mass media for adoption decisions.
For Further Reading
Green, L.W., Gottlieb, N. H., & Parcel, G. S. (1987). Diffusion theory extended and applied. In W. B.Ward (Ed.), Advances in health education and promotion. Greenwich, CT: JAI Press.
Rogers, E. M. (1983). Diffusion of innovations (3rd ed.). New York: Free Press.
Appendix C: Information Sources
National Sources of Health-Related Information
Agency for Healthcare Research and Quality (AHRQ)
Publication Clearinghouse
P.O. Box 8547
Silver Spring, MD 20907-8547
American Cancer Society (ACS)
1599 Clifton Road, NE
Atlanta, GA 30329
Cancer Information Service
National Cancer Institute
6116 Executive Boulevard, SMC 8322
Room 3036A
Bethesda, MD 20892
Center for Substance Abuse Prevention
National Clearinghouse for Alcohol and Drug Information
P.O. Box 2345
Rockville, MD 20847-2345
Centers for Disease Control and Prevention (CDC)
Centers for Medicare and Medicaid Services
7500 Security Boulevard
Baltimore, MD 21244-1850
1-800-638-6833 (Medicare)
202-690-6145 (Press Office)
Health Promotion Online—Canada
Partnerships and Marketing Division
Health Promotion and Programs Branch
Health Canada
55 St. Clair Avenue East, 4th Floor
Toronto, Ontario
National Center for Chronic Disease Prevention and Health Promotion
Division of Cancer Prevention and Control
4770 Buford Highway, NE
Atlanta, GA 30341-3724
770-488-4727 (fax)
National Center for Health Statistics
6525 Belcrest Road
Hyattsville, MD 20782-2003
Roper Center for Public Opinion Research
341 Mansfield Road
Unit 1164
University of Connecticut
Storrs, CT 06269-1164
U.S. Department of Health and Human Services
200 Independence Avenue, SW
Washington, DC 20201
Internet Resources
Combined Health Information Database
Community Toolbox
Health Education Professional Resources
(To locate Federal Clearinghouse and other Federal information sources)
American Journal for Health Promotion
1660 Cass Lake Road
Suite 104
Keego Harbor, MI 48320
248-682-1212 (fax)
American Journal of Health Behavior
American Academy of Health Behavior
P.O. Box 4593
Star City, WV 26504-4593
American Journal of Public Health
American Public Health Association
800 I Street, NW
Washington, DC 20001
202-777-2534 (fax)
Health Education and Behavior
Society for Public Health Education
1015 15th Street, NW
Suite 410
Washington, DC 20005
202-408-9815 (fax)
email@example.com (e-mail)
Health Education Research
Oxford University Press
2001 Evans Road
Cary, NC 27513
Journal of Health Communication
Department of International Public Health
School of Public Health and Health Services
The George Washington University
2175 K Street, NW
Suite 820
Washington, DC 20037
Journal of the American Medical Association
515 North State Street
Chicago, IL 60610
Morbidity and Mortality Weekly Report (MMWR)
Centers for Disease Control and Prevention
1600 Clifton Road
Atlanta, GA 30333
The New England Journal of Medicine
10 Shattuck Street
Boston, MA 02115-6094
617-734-4457 (fax)
Social Marketing Quarterly
Best Start Social Marketing
4809 East Busch Boulevard
Suite 104
Tampa, FL 33617
813-971-2280 (fax)
Appendix D: Selected Readings and Resources
Academy for Educational Development. (1995). A tool box for building health communication capacity. Washington, DC.
Adler, E. (1993). Everyone's guide to successful publications: How to produce powerful brochures, newsletters, flyers, and business communications, start to finish. Berkeley, CA: PeachPit.
Agency for International Development, Academy for Educational Development. (1992). Results and realities: A decade of experience in communication for child survival. Washington, DC.
Agency for Toxic Substances and Disease Registry. (1994). Guidelines for planning and evaluating environmental health education programs. Atlanta.
American Cancer Society. (2001). Cancer facts and figures. Atlanta.
American Marketing Association, New York Chapter. (2001). The focus group directory. New York.
American Marketing Association, New York Chapter. (2001/2002). Green Book: International directory of marketing research companies and services. New York.
American Public Health Association. (2000). APHA media advocacy manual 2000. Washington, DC.
Andreasen, A. (1988). Cheap but good marketing research. Homewood, IL: Dow Jones-Irvin.
Andreasen, A. (1995). Marketing social change: Changing behavior to promote health, social development, and the environment. San Francisco: Jossey-Bass.
Arkin, E. B., Romano, R. M., Van Nevel, J. P., & McKenna, W. (1993). Effect of the mass media in promoting calls to the Cancer Information Service. In The Cancer Information Service: A 15-year history of service and research (Monograph of the Journal of the National Cancer Institute, No. 14).
Backer, T. E., Rogers, E. M., & Sopory, P. (1992). Designing health communication campaigns: What works. Thousand Oaks, CA: Sage.
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.
Bettman, J. R. (1979). An information processing theory of consumer choice. Reading, MA: Addison-Wesley.
Calvert, P. (Ed.). (1996). The communicator's handbook: Tools, techniques, and technology (3rd ed.). Gainesville, FL: Maupin House.
Campbell, M. K., DeVellis, B. M., Strecher, V. J., Ammerman, A. S., DeVellis, R. F., & Sandler, R. S. (1994). The impact of message tailoring on dietary behavior change for disease prevention in primary care settings. American Journal of Public Health, 84, 783–787.
Center for Substance Abuse Prevention. (1994). Careful concept development paves the way to effective prevention materials [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). Conducting focus groups with young children requires special considerations and techniques [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). Following specific guidelines will help you assess cultural competence in program design, application, and management [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). A key step in developing prevention materials is to obtain expert and gatekeeper reviews [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). Pretesting is essential: You can choose from various methods [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). You can increase your media coverage [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). You can manage focus groups effectively for maximum impact [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). You can prepare easy-to-read materials [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1994). You can use communications principles to create culturally sensitive and effective prevention materials [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention. (1998). Evaluating the results of communication programs [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.
Center for Substance Abuse Prevention Communications Cooperative Agreements. (1996). Bridging the gap for people with disabilities. Rockville, MD: U.S. Department of Health and Human Services.
Centers for Disease Control and Prevention. (1994). Listening to your audience: Using focus groups to plan breast and cervical cancer public education programs (CDC Publication No. PDF-245K). Denver: AMC Cancer Research Center.
Centers for Disease Control and Prevention. (1994). The prevention marketing initiative community kit: It's your move, prevent AIDS (CDC Publication No. D738). Atlanta.
Centers for Disease Control and Prevention. (1996). The prevention marketing initiative: Applying prevention marketing (CDC Publication No. D905). Atlanta.
Centers for Disease Control and Prevention. (2000). Beyond the brochure (CDC Publication No. PDF-821K). Atlanta.
Centers for Disease Control and Prevention. (2001). CDCynergy 2001 [CD-ROM]. Atlanta.
Coyle, S. L., Boruch, R. F., & Turner, C. F. (Eds.). (1991). Evaluating AIDS prevention programs: Expanded edition. Washington, DC: National Academy Press.
Davis, J. (Ed.). (2001). Health and medicine on the Internet: An annual guide to the World Wide Web for health care professionals. Los Angeles: Practice Management Information Corporation.
Debus, M. (1988). Methodological review: A handbook for excellence in focus group research. Washington, DC: Academy for Educational Development.
The Dialog Corporation. (2001). Packaged facts. Available: http://library.dialog.com/sourcebook/researchline/pf.html.
Dignan, M. B., & Carr, P. A. (1992). Program planning for health education and health promotion (2nd ed.). Philadelphia: Lea & Febiger.
Eng, T. R., & Gustafson, D. H. (Eds.). (1999). Wired for health and well-being: The emergence of interactive health communication. Washington, DC: U.S. Department of Health and Human Services, U.S. Government Printing Office.
Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.
Flay, B. R., & Cook, T. D. (1989). Three models for evaluating prevention campaigns with a mass media component. In R. E. Rice & C. K. Atkin (Eds.), Public communication campaigns (2nd ed.). Thousand Oaks, CA: Sage.
Flay, B. R., Kessler, R. C., & Utts, J. M. (1991). Evaluating media campaigns. In S. L. Coyle, R. F. Boruch, & C. F. Turner (Eds.), Evaluating AIDS prevention programs. Washington, DC: National Academy Press.
Glanz, K., Lewis, F. M., & Rimer, B. K. (Eds.). (1997). Health behavior and health education: Theory, research, and practice (2nd ed.). San Francisco: Jossey-Bass.
Glanz, K., & Rimer, B. K. (1995). Theory at a glance: A guide for health promotion practice (NIH Publication No. 97-3896). Bethesda, MD: National Cancer Institute.
Glassman, B. & Rimer, B. K. (1999). Is there a use for tailored print communications in cancer risk communications? (Monograph of the Journal of the National Cancer Institute, No. 25).
Goldberg, M. E., Fishbein, M. F., & Middlestadt, S. E. (Eds.). (1997). Social marketing: Theoretical and practical perspectives. Mahwah, NJ: Erlbaum.
Green, L.W., Gottlieb, N. H., & Parcel, G. S. (1987). Diffusion theory extended and applied. In W. B.Ward (Ed.), Advances in health education and promotion. Greenwich, CT: JAI Press.
Green, L.W., & Kreuter, M.W. (1999). Health promotion planning: An educational and ecological approach (3rd ed.). Mountain View, CA: Mayfield.
Green, L.W., & Ottoson, J. M. (1999). Community and population health (8th ed.). New York: McGraw-Hill.
Ipsos-ASI, Inc. Market research database of diagnostic norms. Norwalk, CT. Information: www.ipsos-asi.com.
Janz, N. K., & Becker, M. H. (1984). The health belief model: A decade later. Health Education Quarterly, 11, 1–47.
Jernigan, D. H., & Wright, P. A. (1996). Media advocacy: Lessons from community experiences. Journal of Public Health Policy, 17, 306–330.
Jones, J. P. (Ed.). (1998). How advertising works: The role of research. Thousand Oaks, CA: Sage.
Kotler, P., & Roberto, E. L. (1989). Social marketing: Strategies for changing public behavior. New York: Free Press.
Lefebvre, R. C. (2000). Theories and models in social marketing. In P. N. Bloom & G. T. Gundlach (Eds.), Handbook of marketing and society. Thousand Oaks, CA: Sage.
Lefebvre, R. C., & Rochlin, L. (1997). Social marketing. In K. Glanz, F. M. Lewis, & B. K. Rimer (Eds.), Health behavior and health education: Theory, research, and practice (2nd ed.). San Francisco: Jossey-Bass.
Lipkus, I. M., Lyna, P. R., & Rimer, B. K. (1999). Using tailored interventions to enhance smoking cessation among African Americans at the community health center. Nicotine and Tobacco Research, 1(1), 77–85.
Maibach, E., Maxfield, A., Ladin, K., & Slater, M. (1996). Translating health psychology into effective health communication. Journal of Health Psychology, 1, 261–277.
Maibach, E., & Parrott, R. L. (Eds.). (1995). Designing health messages: Approaches from communication theory and public health practice. Thousand Oaks, CA: Sage.
Marketing Research Association. (2001). Blue book 2001. Available: www.bluebook.org/scripts/bluebook/index.efm.
Matiella, A. C. (Ed.). (1990). Getting the word out: A practical guide to AIDS materials development. Santa Cruz, CA: Network Publications.
McGuire, W. J. (1984). Public communication as a strategy for inducing health-promoting behavioral change. Preventive Medicine, 13(3), 299–313.
Merton, R. K. (1987). Focused interviews and focus groups: Continuities and discontinuities. Public Opinion Quarterly, 51, 550–556.
Merton, R. K., Riske, M., & Kendall, P. L. (1996). The focused interview (2nd ed.). New York: Free Press.
Miller, J., & Pifer, L. K. (1993). Public understanding of biomedical science in the United States, 1993: A report to the National Institutes of Health. Chicago: Chicago Academy of Science.
Morgan, D. L., & Krueger, R. A. (1998). The focus group kit. Thousand Oaks, CA: Sage.
Morra, M. E. (Ed.). (1998). The impact and value of the Cancer Information Service: A model for health communication. Journal of Health Communication, 3(3) Suppl.
Muraskin, L. D. (1993). Understanding evaluation: The way to better prevention programs. Washington, DC: U.S. Department of Education.
National Cancer Institute. (1993). A picture of health (NIH Publication No. 94-3604). Rockville, MD: U.S. Department of Health and Human Services.
National Cancer Institute. (1994). Clear and simple: Developing effective print materials for low-literate readers (NIH Publication No. 95-3594). Rockville, MD: U.S. Department of Health and Human Services.
National Cancer Institute. (1996). Cancer rates and risks (4th ed.; NIH Publication No. 96-691). Bethesda, MD: Surveillance, Epidemiology, and End Results (SEER) Program.
National Cancer Institute. (1998) How the public perceives, processes, and interprets risk information: Findings from focus group research with the general public. Rockville, MD: U.S. Department of Health and Human Services.
National Cancer Institute. (1998). Media strategies for smoking control guidelines. Bethesda, MD: U.S. Department of Health and Human Services.
National Cancer Institute. (2000). Multi-ethnic focus groups to test motivational messages on mammography and breast cancer. Bethesda, MD.
Palmgreen, P., et al. (1995). Reaching at-risk populations in a mass media drug abuse prevention campaign: Sensation seeking as a targeting variable. Drugs and Society, 8(3), 29–45.
Porras, J. I., & Roberston, P. J. (1987). Organization development theory: A typology and evaluation. In R.W.Woodman & W. A. Pasmore (Eds.), Research in organizational change and development (Vol. 1). Greenwich, CT: JAI Press.
Prochaska, J. O., & Velicer, W. F. (1997). The transtheoretical model of health behavior change. American Journal of Health Promotion, 12(1), 38–48.
Rice, M., & Valdivia, L. (1991). A simple guide for design, use, and evaluation for education materials. Health Education Quarterly, 18(1), 79–85.
Rice, R. E., & Atkin, C. K. (2000). Public communication campaigns (3rd ed.). Thousand Oaks, CA: Sage.
Rimer, B. K. (1995). Audiences and messages for breast and cervical cancer screenings. Wellness Perspectives: Research, Theory, and Practice, 11(2), 13–39.
Rimer, B. K., & Glassman, B. (1998). Tailoring communications for primary care settings. Methods of Information in Medicine, 37(3), 171-177.
Rogers, E. M. (1983). Diffusion of innovations (3rd ed.). New York: Free Press.
Rossi, P. H., Freeman, H. E., & Lipsey, M.W. (1998). Evaluation: A systematic approach (6th ed.). Thousand Oaks, CA: Sage.
Rothman, J., & Tropman, J. E. (1987). Models of community organization and macro practice: Their mixing and phasing. In F. M. Cox, J. L. Ehrlich, J. Rothman, & J. E. Tropman (Eds.), Strategies of community organization (4th ed.). Itasca, IL: Peacock.
Schooler, C., Chaffee, S. H., Flora, J. A., & Roser, C. (1998). Health campaign channels: Tradeoffs among reach, specificity, and impact. Human Communication Research, 24, 410–432.
Selden, C. R., Zorn, M., Ratzan, S., & Parker, R. M. (2000). Health literacy, January 1990 through October 1999. Bethesda, MD: National Library of Medicine.
Siegel, M., & Doner, L. (1998). Marketing public health: Strategies to promote social change. Gaithersburg, MD: Aspen.
Slater, M. D. (1996). Theory and method in health audience segmentation. Journal of Health Communication, 1, 267–283.
Strecher, V. J., Kreuter, M.W., DenBoer, C. H., Kobrin, S. C., Hospers, H. J., & Skinner, C. S. (1994). The effects of computer-tailored smoking cessation messages in family practice settings. Journal of Family Practice, 39, 262–270.
Strecher, V. J., & Rosenstock, I. M. (1997). The health belief model. In K. Glanz, F. M. Lewis, & B. K. Rimer (Eds.), Health behavior and health education: Theory, research, and practice (2nd ed.). San Francisco: Jossey-Bass.
U.S. Department of Health and Human Services. (2000). Healthy people 2010 (2nd ed.; in two volumes: Understanding and improving health and Objectives for improving health.). Washington, DC: U.S. Government Printing Office.
University of Kansas. (2002). Community Toolbox, Community Workstation. Available at http://ctb.lsi.ukans.edu/tools/CWS/socialmarketing/outline.htm.
University of Toronto. (1999). Overview of health communication campaigns. Toronto, Canada: Health Communication Unit, Centre for Health Promotion, University of Toronto.
Wallach, L., & Dorfman, L. (1996). Media advocacy: A strategy for advancing policy and promoting health. Health Education Quarterly, 23(3), 293–317.
Wallach, L., Dorfman, L., Jernigan, D., & Themba, M. (1993). Media advocacy and public health: Power of prevention. Thousand Oaks, CA: Sage.
Wallach, L., Woodruff, K., Dorfman, L., & Diaz, I. (1999). News for a change: An advocate's guide to working with the media. Thousand Oaks, CA: Sage.
Weitzman, E. A., & Miles, M. B. (1995). Computer programs for qualitative data analysis: A software sourcebook. Thousand Oaks, CA: Sage.
Wells, W. D. (Ed.). (1997). Measuring advertising effectiveness. Mahwah, NJ: Erlbaum.
Windsor, R.W., Baranowski, T. B., Clark, N. C., & Cutter, G. C. (1994). Evaluation of health promotion, health education and disease prevention programs (2nd ed.). Mountain View, CA: Mayfield.
Appendix E: Glossary
Attention. A pretesting measure used to describe a message's ability to attract listener or viewer attention; this is often measured as "recall" of a message or image.
Attitudes. An individual's predispositions toward an issue, person, or group, which influence his or her response to be positive or negative, favorable or unfavorable.
Baseline study. The collection and analysis of data regarding an intended audience or situation prior to intervention.
Bounceback card. A short questionnaire, often on a business-reply postcard, that is distributed with materials to collect process evaluation data.
Central-location intercept interviews. A method used for pretesting messages and materials. It involves "intercepting" potential intended audience members at a highly trafficked location (such as a shopping mall), asking them a few questions to see if they fit the intended audience's characteristics, showing them the messages or materials, and then administering a questionnaire of predominantly closed-ended questions. Because respondents form a convenience sample, the results cannot be projected to the population. Also called mall intercept interviews.
Channel. The route of message delivery (e.g., mass media channels include television, radio, newspapers, magazines; interpersonal channels include health professional to patient; community channels include community events, such as health fairs or sporting events).
Closed-ended questions. Questions that provide respondents with a list of possible answers from which to choose; also called multiple choice, forced-choice, or fixed-choice questions.
Communication concepts. See message concepts.
Communication objectives. The specific outcomes you expect exposure to your communications will produce in support of the program's overall goal.
Communication strategy. A statement that describes:
- The intended audience members
- The settings, channels, and activities that should be used to reach them
- The image that program communications should convey
- The action intended audience members should take as a result of exposure to your communication
- A compelling benefit they will receive by taking the action
- Support that convinces them they will experience the benefit
Communication strategy statement. A written document containing the communication strategy, which may be supplemented with additional information such as background on the health problem, the goals the communication program is designed to help attain, or more thorough intended audience profiles. This document provides the direction and consistency for all program messages and materials.
Comprehension. A pretesting measure used to determine whether messages are clearly understood.
Consumer panel. A research study in which the buying behavior and other characteristics of a group of consumers are studied over time. Data can be collected through periodic questionnaires, consumer diaries, UPC scanners, or a combination of techniques. Because information is collected at multiple times, changes in behavior over time can be examined and panel members can be recontacted and asked additional questions that are specific to a particular health problem or communication effort. Consumer panel data are subject to the same limitations of any panel study (e.g., sample possibly being misrepresentative due to selection bias, difficulty in ensuring participation over time, and inability to control the drop-out rate).
Convenience samples. Samples that consist of respondents who are typical of the intended audience and who are easily accessible; results cannot be projected to the entire population being studied.
Creative brief. A short (one- to two-page) version of the communication strategy statement, used to guide development of materials and activities. The short creative brief is sometimes used in place of the longer communication strategy statement, especially if the program is not very complex.
Diagnostic information. The results of pretesting research that indicate the strengths and weaknesses in messages and materials.
Education entertainment. A form of health communication in which educational content and information is intentionally incorporated into an entertainment format (e.g., songs, comics, nonnews television or radio programming, movies).
Environmental factors. Factors that are external to an individual but can influence the individual's behavior (e.g., policies, access to services, geography, physical features such as sidewalks and parks).
Focus group. A qualitative research technique in which an experienced moderator guides about 8 to 10 participants through a discussion of a selected topic, allowing them to talk freely and spontaneously. Focus groups are often used to identify previously unknown issues or concerns or to explore reactions to potential actions, benefits, or concepts during the planning and development stages.
Formative evaluation. Evaluative research conducted during program development. May include state-of-the art reviews, pretesting messages and materials, and pilot testing a program on a small scale before full implementation.
Frequency. The average number of times an audience is exposed to a specific media message.
Gatekeeper. An organization or individual you must work with before you can reach an intended audience (e.g., an organization, a schoolteacher, a television public service director).
Geodemographic databases. Customized computer programs that combine many variables—such as demographic, lifestyle, behavior, and geographic information—from different surveys into one analysis.
Goal. The overall health improvement an organization or agency strives to create.
Health belief model. A conceptual framework of health behavior stating that health behavior is a function of both knowledge and motivation. Specifically, the model emphasizes the role of perceived vulnerability to a condition, perceived severity of the condition, perceived benefits of the recommended action, perceived barriers to the advised action, cues to action, and selfefficacy in terms of one's ability to take action.
Impact evaluation. A type of research designed to identify whether and to what extent a program contributed to accomplishing its stated goals (here, more global than outcome evaluation).
In-depth interviews. A type of qualitative research in which a trained interviewer guides an individual through a discussion of a selected topic, allowing the person to talk freely and spontaneously. This technique is often used to identify previously unknown issues or concerns, or to explore reactions to potential actions, benefits, or concepts during the planning and development stages.
Intended audience. The audience selected for program messages and materials (see segmentation). The primary intended audience consists of those individuals the program is designed to affect. The secondary intended audience is the group (or groups) that can help reach or influence the primary audience. (intended audience is also referred to as "target audience.")
Intended population. A broad definition of the audience for a program. The intended population is defined by the epidemiology of the problem and factors contributing to it (e.g., women ages 40 and over for a mammography screening program).
Internet. A global network connecting millions of computers all over the world, allowing for the exchange of information.
Low literacy. A limited ability to use printed and written information to function in society, to achieve one's goals, and to develop one's knowledge and potential.
Media advocacy. The strategic use of mass media to reframe issues, shape public discussion, or build support for a policy, point of view, or environmental change.
Media literacy. Having the skills to deconstruct media messages to identify the sponsor's motives and to construct or compose media messages representing the intended audience's point of view. This is often taught to youth so they can evaluate the media messages directed toward them.
Message concepts. Brief statements, sometimes accompanied by visuals, that present key aspects of the communication strategy (e.g., action to be taken, benefit promised in exchange, support for the benefit) to the intended audience. Message concepts often differ in terms of the type of appeal used; one may be factual, one may be emotional, and one may demonstrate the action to be taken.
Objectives. See communication objectives.
Omnibus survey. A national survey conducted by a research organization that includes questions on varied topics for various sponsoring organizations. This method of survey research allows multiple organizations to add questions to construct one questionnaire, reducing survey costs to participating organizations.
Open-ended questions. Questions that allow an individual to respond freely in his or her own words, in contrast to closed-ended or fixed-choice questions.
Outcome evaluation. Research designed to assess the extent to which a program achieved its objectives.
Over-recruiting. Recruiting more respondents than required to compensate for expected "no-shows."
Polysyllabic words. Words that contain three or more syllables.
Pretesting. A type of formative evaluation that involves systematically gathering intended audience reactions to messages and materials before the messages and materials are produced in final form.
Primary intended audience. See intended audience.
Probe. A technique used primarily in qualitative research (e.g., focus groups, in-depth interviews) to solicit additional information about a question or issue. Probes should be neutral (e.g., "What else can you tell me about _____?"), not directive ("Do you think the pamphlet was suggesting that you take a particular step, such as changing your diet?").
Process evaluation. Research conducted to document and study the functioning of different components of program implementation; includes assessments of whether materials are being distributed to the right people and in what quantities, whether and to what extent program activities are occurring, and other measures of how and how well the program is working.
Program objectives. The specific outcomes that you expect your entire program to achieve. These will be broader than communication objectives, but must also specify outcomes.
PSA. A public service announcement; an advertisement that a mass media outlet (e.g., magazine, newspaper, radio station, television station, Web site, outdoor venue) prints or broadcasts without charging the sponsoring organization.
Public relations. Marketing activities designed to raise the public's awareness about a product, service, individual, or issue; management of an organization's public image that helps the public understand the organization and its products.
Qualitative research. Subjective research that involves obtaining reactions and impressions from small numbers of people by engaging them in discussions. The information gathered should not be described in numerical terms, and generalizations about the intended audience cannot be made. Qualitative research is useful for exploring reactions and uncovering additional ideas, issues, or concerns.
Quantitative research. Research designed to gather objective information by asking a large number of people identical (and predominantly closed-ended) questions. Results are expressed in numerical terms (e.g., 35 percent are aware of X and 65 percent are not), and, if the respondents are a representative random sample, quantitative data can be used to draw conclusions about the intended audience as a whole. Quantitative research is useful for measuring the extent to which knowledge, attitudes, or behaviors are prevalent in an intended audience.
Random sample. A sample of respondents selected from an intended population in which every member of the population had an equal chance of being included.
Reach. The number of people or households exposed to a specific media message during a specific period of time.
Readability testing. Using a formula to predict the approximate reading level (usually expressed in grades) a person must have achieved in order to understand written material.
Recall. In pretesting, a measure that describes the extent to which respondents remember seeing or hearing a message that was shown in a competitive media environment—usually centers on recall of the main idea, not the verbatim message.
Search engine. A mechanism for finding Web sites or documents contained on Web sites. To make sure others can find your site, you can register it with popular search engines (e.g., Yahoo!) by providing a description of your site and a few keywords.
Secondary intended audience. See intended audience.
Segmentation. Subdividing an overall population into homogeneous subsets in order to better describe and understand a group, predict behavior, and tailor messages and programs to match specific interests, needs, or other group characteristics. Segments may be demographic (e.g., age, sex, education, life cycle), geographic (e.g., Southeastern U.S., rural, north side of town), or psychographic (e.g., personality, lifestyle, usage patterns, risk factors, benefits sought), or they may be based on a combination of these factors.
Self-administered questionnaires. Questionnaires that are filled out by respondents themselves (rather than by an interviewer).
Settings. Times, places, and states of mind during which an intended audience is attentive and open to a message and finds it credible.
Social cognitive theory. A theory of human behavior that stresses the dynamic interrelationships among people, their behavior, and their environment. While the environment shapes, maintains, and constrains behavior, people are not passive in the process; they can create and change their environments. A basic premise of the theory is that people learn not only through their own experiences but also by observing the actions of others and the results of those actions. Social learning theory was the precursor to social cognitive theory.
Social marketing. The application and adaptation of commercial marketing concepts to the planning, development, implementation, and evaluation of programs that are designed to bring about behavior change to improve the welfare of individuals or their society. Social marketing emphasizes thorough market research to identify and understand the intended audience and what is preventing them from adopting a certain health behavior, and to then develop, monitor, and constantly adjust a program to stimulate appropriate behavior change. Social marketing programs can address any or all of the traditional marketing mix variables—product, price, place, or promotion.
Stages-of-change model. A theoretical framework that explains behavior change as a process rather than as an event. The model identifies individuals at various stages of readiness to attempt, to make, and to sustain a behavior change. The stages are precontemplation, contemplation, decision/determination, action, and maintenance.
Strategy. The overall approaches a program takes.
Tailored communication. Messages crafted for and delivered to each individual based on individual needs, interests, and circumstances.
Target audience. See intended audience.
Web site. A location on the World Wide Web containing documents or files. Each site is owned and managed by an individual, company, or organization.
World Wide Web. A part of the Internet designed to facilitate navigation of the network through graphic user interfaces and hypertext links.
Many health communication experts contributed to the revision of this book. For their invaluable input, we would like to thank:
- Elaine Bratic Arkin Health Communications Consultant
- Cynthia Bauer, Ph.D. U.S. Department of Health and Human Services
- John Burklow Office of Communication and Public Liaison National Institutes of Health
- Lynne Doner Health Communications Consultant
- Timothy Edgar, Ph.D. Westat
- Brian R. Flay University of Illinois at Chicago
- Vicki S. Freimuth, Ph.D. Office of Communication Centers for Disease Control and Prevention
- Joanne Gallivan, M.S., R.D. National Institute of Diabetes and Digestive and Kidney Diseases
- Karen Glanz, Ph.D., M.P.H. Cancer Research Center of Hawaii
- Bernard Glassman, M.A.T. Special Expert in Informatics National Cancer Institute
- Susan Hager Hager Sharp
- Jane Lewis, Dr.P.H. UMDNJ, School of Public Health
- Terry Long National Heart, Lung, and Blood Institute
- Kathleen Loughrey, M.S., R.D. Health Communications Consultant
- Susan K. Maloney, M.H.S. Health Communications Consultant
- Joy R. Mara, M.A. Joy R. Mara Communications
- John McGrath National Institute of Child Health and Human Development
- Diane Miller, M.P.A. National Institute on Alcohol Abuse and Alcoholism
- Ken Rabin, Ph.D. Ruder Finn Healthcare, Inc.
- Scott C. Ratzan, M.D., M.P.A. Journal of Health Communication U.S. Agency for International Development
- Barbara K. Rimer, Dr.P.H. Division of Cancer Control and Population Sciences National Cancer Institute
- Victor J. Strecher, Ph.D., M.P.H. University of Michigan
- Tim L. Tinker, Dr.P.H., M.P.H. Widmeyer Communications
We would especially like to thank Elaine Bratic Arkin, author of the original book, whose knowledge of health communication program planning made this revision possible, as well as Lynne Doner, whose broad-based consumer research and evaluation expertise has enhanced the book's content and quality. Both have provided hours of review and consultation, and we are grateful to them for their contributions.
Thanks to the staff of the Office of Communications, particularly Nelvis Castro, Ellen Eisner, and Anne Lubenow. And thanks to Christine Theisen, who coordinated the revisions to the original text.
This document was revised in coordination with the Centers for Disease Control and Prevention during development of CDCynergy—a program-planning tool on CD-ROM.