Generic Copy Test of Food Health Claims in Advertising
Principal Author - Dennis Murphy
Contributing Authors - Theodore H. Hoppock, Michelle K. Rusk
A Joint Staff Report of the Bureaus of Economics and Consumer Protection, Federal Trade Commission
November 1998
Executive Summary
I. Introduction
The attached memorandum presents the results of a large scale advertising copy test project that was conducted jointly by the Division of Advertising Practices and the Bureau of Economics of the Federal Trade Commission. This research explored several issues related to the Commission's 1994 Enforcement Policy Statement on Food Advertising ("Statement"). That document describes the manner in which the Commission's approach to advertising relates to the FDA's regulations governing the use of nutrient content descriptors and health claims in food labeling.
The guidance provided in the Commission's Statement closely adheres to the FDA definitions for nutrient content descriptors and to regulations concerning unqualified health claims. Under limited conditions, however, the Statement allows food advertisers to make in a qualified manner three categories of health claims that are not permitted in labeling. The research project reported here tested consumer reactions to a series of print advertisements that presented these three classes of health claims in either an unqualified or qualified manner. The purpose of the consumer research was to determine which of the various types of disclosures and warnings that appeared in the qualified ads would communicate most effectively information concerning the nutrient profile and health attributes of the advertised products.
Approximately 1,700 consumers participated in the project, which was conducted in shopping malls in 12 geographically dispersed cities across the country in January of 1996. The research utilized 29 test ads for five fictitious food products, with 60 respondents assigned to each of the 29 test cells. Respondents were screened to ensure they were users of the type of food product featured. After reading the ads, respondents were interviewed using a survey instrument that began with very general open-ended queries, and gradually narrowed to a key close-ended question relevant to the main focus of the experiment. The interview ended with a self-administered questionnaire that measured respondents' interest in and knowledge of nutrition issues.
In considering the results of this copy test, it is important to recognize that subtle changes in the wording or placement of claims and qualifying disclosures could have a significant impact on how consumers interpret an advertisement. The fictitious nature of the test ads and their rather plain format may also have affected consumer interpretation. In addition, consumer reactions may have been influenced by the choice of foods and the particular nutrient and health benefits featured in the test ads. Further, the research measures consumer interpretation of nutrient and health information in advertising, and the results may or may not have any bearing on how consumers interpret such information in labeling. Consumers may, for example, view a nutrition profile on a food label very differently from similar information presented in the context of an advertisement. Finally, although analysis of consumer responses to the initial open-ended questioning does not suggest that the various qualifying disclosures and warnings impeded the communication of truthful information concerning the positive nutrient characteristics of the featured products, the study was not designed to provide definitive conclusions on this issue.
II. Halo Effect Component
A. Overview
The first component of the research concerned claims for foods that contain high levels of a beneficial nutrient, such as fiber or calcium, but that also contain high levels of a nutrient (such as sodium, saturated fat, or cholesterol) that in sufficient quantity can increase the risk of a diet related disease. The FDA labeling regulations prohibit all health claims for these products. The Commission's Statement advises that advertisers may promote the health benefits of a food that has both positive and negative elements in its nutrient profile, but cautions that the advertisement may also need to disclose the presence, and possibly the significance, of the risk-increasing nutrient. According to the Statement, disclosure would be needed if an unqualified health claim for the beneficial nutrient would cast a halo over the entire product and lead a substantial number of consumers to assume that the product does not present any health risk.
The "Halo Effect" component of the project used ads for a fictitious minestrone soup ("Fresh and Hearty") and a sliced swiss cheese product ("Matterhorn") that, respectively, were high in fiber and calcium, but also high in either sodium or saturated fat. Two major issues were of interest. The first was whether or not the unqualified health claims for fiber or calcium would in fact lower respondents' perceptions of the level of sodium or saturated fat in the advertised products. The second issue was which, if any, of a series of disclosures would prove most effective in communicating the level of risk-increasing nutrients in the soup and cheese.
In order to test for the existence of a halo effect from an unqualified health claim for fiber or calcium, two groups of respondents were shown a soup or cheese control ad that contained no health or nutrient content information of any kind.(1) Two other groups saw ads with an unqualified health claim relevant to either fiber or calcium.(2) It was hoped that a comparison of the survey results between the two types of ads would reveal whether the health claims led respondents to view the overall nutrient profile of the advertised products in a more favorable light.
Additional groups of consumers were shown ads containing various qualifying disclosures intended to communicate the high levels of the risk-increasing nutrients in the advertised products. The sequence of ads included (1) an absolute disclosure of the quantity of sodium or saturated stated fat per serving stated in the appropriate metric measurement (grams or milligrams), (2) a relative disclosure that expressed the sodium or saturated fat content both in absolute terms and as a percentage of the Daily Recommended Value, (3) a hybrid quantitative and verbal disclosure that combined the relative disclosure with a warning that diets high in the risk-increasing nutrient could increase the risk of high blood pressure or heart disease, and (4) a strictly verbal disclosure that introduced the dietary warning with an advisory that the advertised product was high in the risk-increasing nutrient. The latter disclosure was evaluated in a follow-up test after it was determined that the wording of the hybrid disclosure had confused a large number of respondents in the initial copy test.
B. Results
The results of the Halo Effect component provided very clear and interesting evidence concerning the effectiveness of the various remedy disclosures. The attempt to test the primary halo effect hypothesis, however, was compromised by low response rates in key test cells. Because neither the control nor health claim ads contained any information concerning sodium or saturated fat content, from only one-third to one-half of the consumers in these test cells were willing to choose a response category when asked to rate the sodium or saturated fat content of the advertised products on a five-point scale (ranging from low to high). Among those responding, there were no statistically significant differences in the mean sodium or saturated fat content ratings between the control and health claim groups, and these limited results therefore do not provide any evidence of a halo effect from the health claim. In all cases, the mean ratings were between "somewhat low" and "neither high nor low," which indicates that, for whatever reason, respondents on average seriously underestimated the level of risk-increasing nutrients in these products.
Results from the remedial disclosure cells showed clearly that an absolute nutrient content disclosure expressed in grams or milligrams did not on net improve respondents understanding of the advertised products' saturated fat or sodium content. The performance of the relative disclosure, which expressed the level of these nutrients both in absolute terms and as a percentage of the Daily Recommended Value, was at least as poor. A closer examination of the results revealed that the remedies had operated in a highly bifurcated fashion. A sizeable minority of respondents interpreted the absolute disclosure correctly to indicate that the advertised products were high or somewhat high in the problem nutrient. Almost as many respondents, however, completely misinterpreted these disclosures to indicate that the products were low or somewhat low in sodium or saturated fat. Further, for both the soup and cheese ads, the additional information presented in the relative disclosure increased the proportion of incorrect responses to the point where the plurality of respondents (about 40%) thought the products were low or somewhat low in the two nutrients.
The third disclosure, which added to the relative statement an advisory warning of the health consequences of high dietary intake of sodium or saturated fat, exacerbated respondents' confusion. Almost half of the viewers of this disclosure apparently misconstrued the dietary warning as a favorable commentary on the quantity of sodium or saturated fat in the advertised products. This reflected a basic design defect in the disclosure, which incorporated quantitative disclosures that failed to communicate clearly that the soup or cheese was in fact high in the risk-increasing nutrients that were discussed in the verbal dietary warning.
The fourth disclosure, which stated directly that the soup or cheese product was high in sodium or saturated fat and then warned of the associated health consequences, provided a stark contrast to the poor performance of the quantitative and hybrid disclosures. About 85% of respondents seeing the cheese warning placed the saturated fat content of the product in the two highest categories, and 70% chose the highest rating of "high." Only about 5% indicated that the cheese was low or somewhat low in saturated fat. Although the corresponding sodium warning in the soup ad did not improve the results as dramatically, about two-thirds of respondents stated that Fresh and Hearty was high or somewhat high in sodium, with most ratings falling in the highest category.(3) About one-fourth of the respondents rated the soup as low or somewhat low in sodium, despite the contradicting message in the warning.
III. Substitution Claim Component
A. Overview
The second category of health claims examined in the copy test project pertained to food products that are relatively low in a risk-increasing nutrient, such as sodium, but not sufficiently low to meet the absolute standard for "low" that FDA prescribes for the relevant health claim in labeling, such as a claim linking low dietary intake of sodium to a reduction in the risk of hypertension. The FTC Statement advises that health claims may appear in advertising for products that are relatively, though not necessarily absolutely, low in a risk increasing nutrient if the advertisement (1) describes a dietary substitution between two similar products that will substantially reduce intake of the nutrient, and (2) is qualified sufficiently to communicate to consumers the limited nature of the product's nutrient advantage over other products mentioned in the ad.
The "Substitution Claim" segment of the research project tested advertisements for a fictitious brand of packaged sliced turkey ("Bradley Right Slices") that contained half the sodium of other leading brands, but which did not qualify as a low sodium food under FDA definitions. All of the advertisements stated that:
New Bradley Right Slices deliver the taste of oven roasted turkey with less than ½ the sodium of other leading brands! And that's important news. Because diets high in sodium can increase the risk of high blood pressure and heart disease!
The first advertisement, which contained no additional information concerning the sodium content of any of the products mentioned in the ad, tested whether consumers would misinterpret the sodium comparison and health claim to mean that Right Slices was a true low sodium food. The remaining ads incorporated different types of disclosures intended to place the sodium content of Right Slices in proper perspective. These included the absolute and relative quantitative expressions used in the Halo Effect segment, a relative disclosure that also included an explicit verbal advisory that Right Slices was not a low sodium food, and a purely verbal advisory that was incorporated in the main text of the ad.(4)
B. Results
The first object of inquiry in the Substitution Claim component was determining whether a substantial number of consumers would misconstrue the unqualified comparative ad to mean that Right Slices was a low sodium food. The results revealed that a sizeable minority of consumers in this cell did consider Right Slices to be low in sodium. About 30% of respondents volunteered this interpretation of the ad in response to the initial open-ended question, which simply asked for the main points of the ad. Almost half of the respondents, however, characterized the sodium content of Right Slices in strictly relative terms, such as "less sodium" or "half the sodium." In the relevant close-ended question, which asked readers of the ad to rate sodium content on a five-point scale, approximately 40 percent of respondents selected the "low in sodium" category.
The two quantitative remedy disclosures, which presented the sodium content of Right Slices in milligrams per serving and as a percentage of the Daily Recommended Value, replicated the weak performance observed in the Halo Effect soup ads. The mean sodium ratings in these cells did not increase over the rating obtained in the unqualified ad cell.
In the Substitution Effect segment, however, the two verbal advisories also failed to have any significant impact on respondents' mean rating of sodium content. It is possible that consumers have greater difficulty reconciling positive and negative information when it relates to the same nutrient. The Halo Effect verbal disclosure for soup informed consumers that the food was high in fiber, but also high in a separate problem nutrient--sodium. The verbal advisories used in the turkey ads attempted to present both positive and negative information about the product's sodium profile.(5) This possibly conflicting message may have confused consumers or simply have failed to register, which would suggest that even stronger or more prominent disclosures than those tested here may be necessary to communicate nutrient information of this type to those who otherwise would be misled..
An attempt was made to determine more rigorously whether respondents who rated Right Slices as low in sodium really thought the product was a true low sodium food, in the sense that it could meet an official absolute standard for "low," or whether these respondents merely considered Right Slices as low in sodium relative to similar products. Specifically, respondents were asked to compare the sodium content of Right Slices with another pictured turkey product that was labeled as "low sodium." Although difficulties in administering the question reduced the reliability of the responses, the results were at least consistent with the hypothesis that a substantial minority of respondents did in fact view Right Slices as a true low sodium food.
IV. Strength of Science Component
A. Overview
The final portion of the research addressed health claim subject areas that have not yet achieved the level of scientific support that FDA considers appropriate for health claims in labeling. The FDA regulations specify that no diet-disease relationship can be discussed in labeling unless FDA has determined that the relationship is supported by "significant scientific agreement." To date, eleven categories of health claims have been approved for labeling under this standard. These include claims linking diets low in saturated fat to a reduced risk of heart disease, diets high in fiber to a reduced risk of cancer, and diets high in calcium in a reduced risk of osteoporosis.
The FTC Food Policy Statement provides that any unqualified health claim in advertising should meet FDA's significant scientific agreement standard. Certain other claims, however, may appear in advertising provided that the ad is qualified to disclose clearly the extent of scientific support for the claim, and there is not a larger body of evidence in that area that contradicts the claim. The "Strength of Science" component of the research project attempted to determine what types of disclosures would best communicate to consumers limitations in the scope of scientific support for health claims that have not yet been approved for labeling. Two subject areas were selected for the test--diets low in trans fatty acids and a reduced risk of heart disease, and the use of antioxidant vitamin supplements and a reduced risk of cancer.
The fictitious products selected for the test ads were a margarine ("Better Blend") that was free of trans fatty acids, and a vitamin supplement ("ACE") that contained a full complement of antioxidant vitamins. To provide a benchmark for gauging the effect of the qualifying disclosures, one group of consumers was shown an ad that claimed the health benefits of the margarine or vitamin supplement were a proven scientific fact.(6) Additional groups of consumers saw ads that contained a series of increasingly qualified disclosures concerning the level of scientific support for the alleged health benefits.
The first level of qualification for the vitamin product was limited to a simple advisory that scientists would not be sure about the cancer reduction benefits of antioxidant supplements until "longer term research" was completed. The corresponding disclosure for the margarine ad stated that scientists would not know "how big a problem" trans fatty acids were until further research was conducted.
In the second level qualification, the vitamin ad disclosed that some recent studies had failed to find a cancer reduction benefit for antioxidant supplements. The margarine ad disclosed that there was an ongoing scientific debate concerning the amount by which trans fatty acids raise cholesterol. For both ads, the second level of qualification emphasized the incomplete status of research by cautioning readers that "It's too early to tell for sure."
The third and strongest qualification, limited to ACE vitamins, warned that a recent study had reported that high doses of an antioxidant may actually have increased the risk of cancer among smokers. This was a reference to a 1994 clinical study of the health effects of beta carotene supplementation among smokers in Finland.
The Strength of Science segment of the research also included a real print ad for an actual antioxidant vitamin supplement ("Nature Made"). Although the text of this ad was considerably longer than that of fictitious test ads, the general subject matter was very similar, and the ad cautioned that existing studies had not get proven the health benefits of antioxidant supplements. The Nature Made ad provided an opportunity to test whether respondents would regard the fictitious ads used elsewhere in the test as less credible or informative than a professionally produced ad for an actual product.
After viewing the various test ads, respondents were asked (following several more general questions) to indicate how sure scientists were about the heart benefits of a reduced intake of trans fatty acids or the cancer reduction benefits of antioxidant vitamin supplements. The principal object of the experiment was to determine which level of qualification was needed to reduce the certainty ratings below those registered by viewers of the unqualified "proof" claim ads for Better Blend and ACE.
B. Results
The results from the strength of science research revealed that the strong disclaimers included in the second level of qualification (such as explicit references to inconsistent study results or ongoing scientific debate) can have a significant impact on consumer perceptions of the level of proof underlying a health claim. For both the margarine and vitamin ads, respondents in these cells gave significantly lower ratings of scientific certainty than did viewers of the unqualified "proof" claims. The more mildly qualified claims, however, did not lower mean certainty ratings significantly. For the vitamin ads, the most strongly worded warning concerning the apparently perverse effect of antioxidant supplements on cancer rates among smokers did not provide any additional impact on certainty ratings over that obtained from the second level of qualification. This warning did, however, significantly lower respondents' interest in purchasing ACE vitamins.
The research results also revealed that even the respondents in the two "proof" cells were reluctant to assign very high levels of certainty to the science supporting the antioxidant-cancer or trans fatty acids-heart disease relationships. In both cases, the mean response to the question that asked how sure scientists were about these areas was slightly below "somewhat sure." Only 27% of the vitamin respondents and 31% of the margarine participants chose the "very sure" rating. These ratings fell to 12% and 18%, respectively, in the vitamin and cheese Qualified ads. For the Highly Qualified ads, the mean ratings fell by about one rating category to slightly below "neither sure nor unsure." Only one vitamin respondent in this cell rated scientists as "very sure." Six respondents (10%) gave this response in the cheese cell with the second level of qualification.
No serious discrepancies were found in the pattern of questionnaire responses between the real Nature Made ad and the fictitious ads used in the other cells. Most importantly, respondents did not find the ad for Nature Made more believable than the other ads. The mean certainty rating generated by the real ad fell between that of the vitamin "proof" ad and the vitamin ad with the first level of qualification, suggesting that the disclaimers in the Nature Made ad may have been somewhat lost in the lengthy text.
V. Major Conclusions and Areas for Future Research
Results from both the Halo Effect and the Substitution Claim components of the copy test indicate that, at the time of the test in early 1996, consumers had not yet become familiar with the quantitative metrics that food labels employ to communicate nutrient content information. Most participants in the test failed to interpret such measurements correctly, whether they were expressed as absolute numbers of grams or milligrams, or as percentages of the relevant Daily Recommended Values. Further, the Halo Effect findings suggest that most consumers who are not acquainted with these measurements will construe quantitative information about risk-increasing nutrients in a positive light and receive a message that is the direct opposite of that intended. At this point in time, it appears that advertising disclosures concerning high levels of risk-increasing nutrients will be most effective if framed in plain English. Future research could explore profitably whether such disclosures could be limited to a simple nutrient content descriptor, such as "high in sodium," or whether it is necessary to adopt the more complete health advisory format used in the Halo Effect segment of the copy test. Such research also might explore more fully whether the successful communication of the negative nutrient characteristics of a food product interferes in any way with consumer understanding of the positive elements of that product's nutrient profile.
In contrast to the other two components of the research, the Substitution Claim segment failed to yield any disclosure format that could correct misimpressions about limitations in the nutrient advantage and health benefits of the advertised product. A sizeable minority of respondents continued to view the turkey slices as low in sodium despite direct verbal advisories to the contrary. It would appear that many consumers do not distinguish between "lower" and "low" claims, at least for the type of product tested. Additional research might test even more prominent disclosures and include other groups of products, such as desserts or snack foods, to determine whether consumers are more likely to place "lower" claims in perspective when the advertised products are not considered generally healthy.
The Strength of Science research suggests that it is possible to communicate limitations in the level of scientific support for diet-disease relationships that have not yet attained significant scientific agreement. It could not be determined, however, whether the relatively low level of scientific certainty that respondents attached to all of the tested ads reflected an inherent skepticism of health claims in advertising, or whether this skepticism was specific to the two tested subject areas and products. Additional research that focused on a well established and familiar diet-disease relationship, such as saturated fat and heart disease, could reveal the level of scientific certainty that consumers assign to an advertised health claim in this area, and thereby provide a valuable benchmark for interpreting the results of this study.
1. See pp. 9a-9b of the main text for reproductions of these ads.
2. Reproductions of all of the ads used in the Halo Effect component may be found in Appendix A of the attached memorandum.
3. The verbal disclosure did not appear to interfere substantially with respondent perception of the positive nutrient content messages in the ads. Respondent playback of the soup's fiber content or the cheese's calcium content was virtually identical for the verbal disclosure ad as for the ads without remedy disclosures. The proportion of respondents volunteering a specific positive health claim as a main point of the ad did decline slightly in both the soup and cheese verbal disclosure ads, although no more so than in the other remedy ads. It should be noted, however, that the study did not explore this issue in a focused manner using close-ended questions that could test more explicitly for any trade-off between effective communications of both positive and negative information in print ads.
4. See pp. 33a-33e of the attached memorandum for reproductions of all of the ads used in the Substitution Claim component.
5. Since none of the tested remedy disclosures had any significant effect on respondent perceptions of the sodium content of the turkey slices, there was no need to analyze whether the various advisories had chilled useful information concerning the true sodium advantage of Right Slices.
6. See pp. 53a-53j for reproductions of all of the ads used in the Strength of Science component.