|
|
||||||||
Original Research Communications |
1 From the National Cancer Institute, Bethesda, MD, and Information Management Services Inc, Silver Spring, MD.
2 An earlier version of this article was presented at the Third International Conference on Dietary Assessment Methods in Arnhem, Netherlands, May 1998.
3 Address reprint requests to FE Thompson, Risk Factor Surveillance and Methodology Branch, Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, EPN 4016, 6130 Executive Boulevard, MSC 7344, Bethesda, MD 20892-7344. E-mail: ft5h{at}nih.gov.
| ABSTRACT |
|---|
|
|
|---|
Objective: Our objective was to evaluate the performance of 2 brief dietary assessment instruments, a 7-item standard screener and a new 16-item screener, and a complete food-frequency questionnaire (FFQ) in measuring total fruit and vegetable consumption.
Design: About 800 men and women from the National Institutes of HealthAARP Diet and Health Study completed an FFQ, 1 of the 2 screeners, and two 24-h dietary recalls. Fruit and vegetable intakes as measured by each screener and the FFQ were compared with estimated true usual intake by using a measurement-error model.
Results: Median daily servings of fruit and vegetables were underestimated by both screeners. The estimated agreement between true intake and the screener was higher for the new screener than for the standard screener and was higher for women than for men. The estimated agreement between true intake and the FFQ was higher than that for both screeners. Attenuation coefficients for the FFQ and screeners were comparable.
Conclusions: For estimating median intakes of fruit and vegetables and the prevalence of recommended intakes being met, the use of screeners without appropriate adjustment is suboptimal. For estimating relative risks in the relations between fruit and vegetable intake and disease, screeners and this FFQ are similar in performance.
Key Words: Diet epidemiologic methods surveillance validation study data collection questionnaires screener food-frequency questionnaire epidemiology nutrition nutritional epidemiology NIH-AARP Diet and Health Study
| INTRODUCTION |
|---|
|
|
|---|
These needs for data have stimulated great interest in how best to measure fruit and vegetable intakes (7). In the United States, 24-h recall data are used to monitor progress toward the Healthy People 2000 national fruit and vegetable intake objective (7). However, sometimes researchers have more limited purposes or resources, and they need brief dietary instruments that are easy to administer and process and that adequately satisfy a particular purpose.
After the 1991 5 a Day Baseline Survey (8), National Cancer Institute researchers and National 5 a Day Program grantees developed a brief dietary assessment tool, or screener, to measure total fruit and vegetable consumption. The results are used to track changes in fruit and vegetable intake and to measure program and intervention effectiveness (9). This standard screener includes 7 questions that ask how often fruit and vegetables were consumed in the past month (Table 1
).
|
|
| SUBJECTS AND METHODS |
|---|
|
|
|---|
2 adjacent pages of the FFQ skipped; >10 items with multiple frequency responses marked; <10 items with any reported consumption; or energy intake too high (>4200 kcal or 17573 kJ for men or >3500 kcal or 14644 kJ for women) or too low (<800 kcal or 3347 kJ for men or <600 kcal or 2510 kJ for women). Medical conditions such as end-stage renal disease, self-reported poor health, or history of cancer were also used as exclusion criteria.
The baseline FFQ data were used to assess individuals' intakes of 4 dietary variables of interest: percentage of energy from fat, dietary fiber intake, servings of fruit and vegetables, and servings of red meat. For each of these variables, 5 strata were defined for each sex and individuals were classified into these strata. To increase the sample's variability for these 4 dietary measures, extreme intakes (strata 1 and 5) on any of the 4 variables were sampled at the highest rate, proportionately fewer individuals were sampled from strata 2 and 4, and even fewer were sampled from stratum 3. Two additional strata were defined as high alcohol use (
90 g alcohol/d on baseline FFQ) by sex. Within each stratum, individuals were randomly chosen to participate in the Calibration Study. Of the 2053 members of the Calibration Study, 1986 provided two 24-h dietary recalls (described below) and the rest provided one. All 2053 members of the Calibration Study were mailed a second FFQ in October 1996. A total of 1415 FFQs were returned, but 52 FFQs were later excluded because of questionable data quality on the basis of the criteria described above.
By June 1997, 80 of the original 2053 persons had withdrawn from further participation in the Calibration Study. The remaining 1973 individuals were randomly divided into 3 groups and were mailed the standard fruit and vegetable screener, the new fruit and vegetable screener, or a new screener designed to assess percentage of energy from fat (not reported here).
For the standard screener, 2 of those mailed were undeliverable and 455 of the 657 mailed were returned (response rate of 69%). For the new screener, which was mailed to a different subgroup, 2 of those mailed were undeliverable and 454 of the 656 mailed were returned (response rate of 69%).
For these analyses, we excluded participants in the high-alcohol-use strata of the calibration sample (n = 11 in the standard-screener group and n = 9 in the new-screener group). We also excluded from further analyses participants with extreme fruit and vegetable values on either 24-h recall (
30 servings of fruit and vegetables on either day; n = 2 in each screener group) because we did not want our results to be influenced unduly by extreme values. Finally, we excluded individuals with poor-quality data on the screeners, as described below (6 and 5 subjects in the standard- and new-screener groups, respectively).
Of the 436 participants in the standard-screener group, 47% were men, 34% were aged 5059 y, and 66% were aged
60 y. Of the 438 participants in the new-screener group, 49% were men, 32% were aged 5059 y, and 68% were aged
60 y.
Data collection for the NIHAARP Diet and Health Study was approved by the National Cancer Institute Institutional Review Board and the Westat Inc Special Studies Institutional Review Board. Informed consent was obtained from all participants.
Data collection and processing and variable creation
Collection and coding of 24-h dietary recalls
The 24-h dietary recalls were administered by trained interviewers over the telephone from February through August 1996. The second recall was obtained an average of 24 d (median of 20 d) after the first recall. A set of measuring guides was sent to participants before the interview; 94% of the participants reported that they used the guides for the interview. Participants were not told in advance when they would be interviewed. Because diets differ by day of the week (10, 11), participant interviews were scheduled to reflect the distribution of weekdays and weekend days. Of the 1986 participants who reported 2 d of intake, 55% reported about 2 weekdays, 37% reported about 1 weekday and 1 weekend day, and 8% reported about 2 weekend days. Participants were asked to report all foods and beverages consumed on the day before the interview, for the entire 24 h from midnight to midnight. Interviewers used a Food Probe List containing standardized probes specific to the various foods in each of >100 food categories. The probes addressed such information as preparation method, type of fat used, brand, additions to the food, dilution, and quantity. Data were coded by using the FOOD INTAKE ANALYSIS SYSTEM, version 2.3 (12). This system uses food codes comparable with those used by the US Department of Agriculture (USDA) and includes extensive error-checking procedures for identifying potentially erroneous data. In addition, data checks were performed on records with extremely high values for fat, total energy, and total fruit and vegetable intakes and corrections were made to the data when warranted.
Conversion of intake data to servings of fruit and vegetables
National dietary guidance (1315) was used to define which foods count as fruit and vegetables and to quantify a serving for all the instruments. For fruit, a serving is defined as a whole fruit (eg, medium apple), three-fourths cup (178 mL) fruit juice, or one-half cup (120 mL) cut-up fruit. For vegetables, a serving is defined as 1 cup (240 mL) raw leafy vegetables (eg, lettuce), one-half cup other vegetables, or three-fourths cup vegetable juice. Data released from the USDA Continuing Survey of Food Intakes by Individuals (CSFII) for 19941996 (16) made it possible to translate food intake data coded with USDA codes into servings from the food guide pyramid groups. The data files provide the number of servings from each food guide pyramid group per 100 g of the food code, for each of >5000 food codes. With this system, the various components of food mixtures can be assigned to their appropriate food guide pyramid group.
Most of the food codes used to code the NIHAARP 24-h dietary recalls corresponded to codes in the 19941996 CSFII; this was the case for 3271 of 3759 food codes reported. When there was no matching 19941996 CSFII code, a similar database developed for the USDA 19891991 CSFII (17) was used to assess servings of fruit and vegetables (140 food codes). A small number of food codes (348) did not match codes in either USDA database. For these, a food code from the 19941996 CSFII database that best matched the food with regard to relative composition of fruit, vegetables, meat, and starch was chosen. After all the food codes reported on the NIHAARP 24-h dietary recalls had been matched to CSFII food codes, the total number of servings of fruit and vegetables from each food item was computed and these were summed across all foods to give an estimate of the total fruit and vegetable servings consumed by each individual. This total reflected all sources of fruit and vegetables, including fried vegetables such as French fries, miscellaneous sources such as potato chips, and small amounts such as those found in yogurt or desserts.
For the screener instruments, various assumptions were made to estimate the total servings of fruit and vegetables. On the standard screener, the amount of food consumed each time is not asked; instead, we assumed one serving as defined by the food guide pyramid. For the new screener, in the sections asking about intake by time of day (Table 2
), individuals were asked to report both the frequency and the number of portions per occurrence. They were instructed that a portion is one-half cup fruit, one whole fruit, or one-half cup vegetables. Servings were assigned to the 3 portion categories as follows:
1 portion = 1 serving, 2 portions = 2 servings,
3 portions = 3 servings. Responses to these questions were also used to estimate the total number of servings for orange and grapefruit juice, other 100% fruit juices, French fries or fried potatoes, and baked, boiled, or mashed potatoes, for which the number of portions was not asked.
For the FFQ, 19941996 CSFII survey data were used to construct a database that included, for each food asked about on the FFQ, a mean number of servings of fruit and vegetables for each of 6 different strata, defined by sex and portion size (small, medium, or large) (18). For each individual, the total daily number of servings of fruit and vegetables was computed by multiplying that individual's reported frequency by the number of fruit and vegetable servings assigned for his or her stratum (defined by sex and his or her reported portion size) for each food on the FFQ, summing across all foods, and standardizing to reflect daily average intake.
Missing responses on both of the screeners were examined and the following decisions were made. For the standard screener, the case was excluded from analysis if there was no frequency response on either the "other vegetables" question or the "fruit" question. When the frequency response for any other question was missing, it was assumed that the respondent never eats that food, and a 0 was assigned. For the new screener, a 0 was assigned when both the frequency and the corresponding portion question were not answered. When the frequency question was answered and the portion question was not, the portion most frequently reported by that respondent for that same type of food (fruit or vegetable) was used as the default value. For both screeners, if any question had multiple responses or entirely missing data, the case was excluded from analysis.
Analytic methods
The variable of interest for all analyses was true usual intake of fruit and vegetables expressed in daily servings. We used the measurement-error model developed by Freedman et al (19), which assumes that measurement error in a test instrument may include systematic bias as well as within-person random error, but that error in the criterion instrument (such as a 24-h recall) includes only within-person variation. The measurement-error model is written as follows:
![]() |
![]() |
Ti is the true usual intake of fruit and vegetables for the ith individual,
ß0 is the intercept and ß1 is the slope of the linear regression of Qi on Ti,
ei is random within-person error,
Fij is the jth repeat criterion measurement (24-h recalls) for person i,
uij is within-person random error for person i, repeat measurement j, assumed to be independent of Ti and ei. Because the repeat 24-h recalls were reasonably separated in time in this case, it is further assumed that random error uij and uij' corresponding to 2 repeat measurements for the same person are independent.
There are many parameters of interest when evaluating how a test instrument performs relative to true dietary intake. For each instrument comparison, we estimated the mean and median, slope (ß1), intercept (ß0), correlation coefficient between test instrument and true intake (R), and attenuation coefficient (
). The parameters ß1, ß0, and R reflect validity, which is whether the test instrument measures what it is supposed to measure. If the test instrument is unbiased, ie, if E(Qi|Ti) = Ti , the regression of measured on true intake would show a straight line at a 45° angle (ß1 = 1.0), intercepting at 0 (ß0 = 0). The squared correlation coefficient (R2) for the correlation between reported and true intake indicates the proportion of variation in intake measured by the screener that is explained by true intake. The attenuation coefficient (
= R2/ß1) is the amount by which the relative risk between the exposure (ie, fruit and vegetable intake) and some disease would be distorted because of measurement error in the test instrument. Although, in general,
can take on any value, it typically ranges from 0 to 1 for dietary variables. A
close to 1 indicates minimum attenuation, whereas a
close to 0 indicates maximum attenuation.
The variable "servings of fruit and vegetables" had a nonnormal distribution for all instruments, whereas the square root of that variable was approximately normally distributed. We therefore used a square-root transformation of the data, redefining Qi, Ti, and Fij in the measurement-error model to refer to their square-root-transformed values. We then calculated maximum-likelihood estimates of the parameters ß0, ß1,
e2,
u2, u T, and
T2 under the assumption of normality. From these parameter estimates, we calculated estimates of the percentiles of the distributions of Q and T and squared them to get estimates of the percentiles of the distribution of the untransformed variables.
| RESULTS |
|---|
|
|
|---|
|
|
|
The interquartile ranges for both screeners were smaller than were those for true intakes. For example, for men in the standard-screener group, the interquartile range for true intake was estimated as 3.6, compared with 2.6 for the screener. In the new-screener group, the interquartile range for true intake was estimated as 4.2, compared with 3.4 for the screener.
The percentages of individuals who met the Healthy People 2000 objective of eating
5 servings of fruit and vegetables daily were estimated by using the same model (Table 4
). In the standard-screener group, the estimated true prevalence among men was 73%, compared with 24% on the basis of the screener results. In the new-screener group, the estimated true prevalence among men was 70%, compared with 42% on the basis of the screener results. Among women, the estimated true prevalence and prevalence from the standard screener were 59% and 36%, respectively; the estimated true prevalence and prevalence from the new screener were 64% and 50%, respectively. Both screeners substantially underestimated the prevalence of the Healthy People 2000 fruit and vegetable objective being met, but the discrepancy was smaller for the new screener than for the standard screener.
|
|
|
|
Comparison of FFQ and screeners with estimated true intakes
We examined the performance of a complete FFQ relative to the screeners in estimating usual intake of fruit and vegetables (the relation between the FFQ and 24-h recalls for the entire calibration sample will be examined in a future publication).
Among men in the standard-screener group, the median estimated true intake was 6.6 servings, compared with the FFQ estimate of 6.6 servings and the screener estimate of 3.7 servings (Table 3
). For men in the new-screener group, the median estimated true intake was 6.5 servings, compared with the FFQ estimate of 6.3 servings and the screener estimate of 4.4 servings. Among women, the median estimated true intakes were intermediate between the FFQ and screener estimates in both screener groups. Among both men and women, median intakes of fruit and vegetables were estimated most closely by the complete FFQ and were underestimated by both screeners, although much less so for women than for men. Median intakes from the new screener were closer to true intake than were those from the standard screener. The FFQ was better than either screener at estimating the prevalence of the Healthy People 2000 fruit and vegetable objective being met (Table 4
).
Measures reflective of validity (ie, the slope and the correlation coefficient) and the attenuation coefficients were generally similar for the FFQ and screener for each comparison by screener type and sex (Table 5
). Only one comparison between the FFQ and a screener showed a significant difference: among men, the slope of 0.81 for the FFQ was significantly better than was the slope of 0.49 for the standard screener (P = 0.0009). The estimated attenuation coefficients indicate that there would be substantial attenuation in calculating the relative risk of a disease on the basis of dietary exposure data from any of these instruments.
| DISCUSSION |
|---|
|
|
|---|
A second novel feature of the present study is the use of an advanced measurement-error model to estimate a variety of parameters useful in understanding the performance of instruments that measure fruit and vegetable intake. The usual comparisons of means and correlations between reported and true intakes were supplemented with estimates of slopes in the regressions of reported intakes on true intakes and attenuation coefficients resulting from measurement error, allowing a fuller understanding of each instrument's utility in epidemiologic research.
The measurement-error-model approach used here uses 24-h recalls (2 for 95% of the samples and 1 for 5% of the samples) as criterion data. There is no requirement that the two 24-h recalls accurately estimate the true usual intake of the individual. Rather, the criterion instrument may involve error of its own, but the assumption is that this error has no systematic bias. Biomarker studies that used doubly labeled water and urinary nitrogen indicated systematic underreporting, at least at the population level, of energy and protein intakes when data were collected with dietary records and dietary recalls (25, 26). One study indicated that intakes of fruit and vegetables may also be underreported (27). If this were also true in the present study, the mean true intake would be underestimated and the differences between the screeners and the estimated true intake would be greater. The ranking of the instruments that we evaluated would remain the same, however. If the amount of underreporting on the 24-h recalls were similar across individuals, the other parameters of interest (slope, correlation coefficient, and attenuation coefficient) would be unaffected.
For surveillance and monitoring purposes, we are most interested in estimates of the mean or median true usual intake and the prevalence of consumption of recommended amounts of fruit and vegetables. In this study, both screeners underestimated median intake and the percentage of individuals consuming recommended amounts, but results from the new screener were closer to true intakes than were those from the standard screener. The median intake from a screener as a percentage of estimated true intake ranged from 56% (standard screener for men) to 88% (new screener for women). This finding is consistent with those of other, recent studies that found lower estimates of fruit and vegetable intakes from screeners relative to multiple 24-h recalls in adolescents (28) and 3-d food records in adult, African American church members in rural North Carolina (29). These studies taken together indicate that the utility of the standard fruit and vegetable screeners used in the National 5 a Day Program and in the Behavioral Risk Factor Surveillance System should be re-evaluated. For estimating median total fruit and vegetable consumption from all sources, these standard screeners may be suboptimal. However, it is possible that estimates from these screeners could be improved by analytic adjustments developed in calibration studies relating screener data to data from food records or recalls, ideally in the population of interest. Our data do not allow for evaluation of the use of either screener for tracking intakes over time or for targeting populations for program intervention.
The new screener was designed to divide a complex reporting task into smaller, more manageable tasks. Thus, foods that are broadly defined and frequently consumed in different ways and at different times of the day (ie, the food groups fruit and other vegetables) were asked about in 3 different time periods. Recently, we conducted cognitive testing that confirmed that many individuals, when asked to report their usual intakes of fruit and other vegetables, do so by recalling their typical day, from morning to bedtime. The improved performance of the new screener compared with the standard screener in this study supports our theory and cognitive findings.
Further analyses were conducted to explore the sources of the discrepancies between the screener results and true intakes. It is unlikely that certain miscellaneous uses of fruit and vegetables (eg, potato chips and condiments) would be considered by respondents when reporting on either screener. These miscellaneous uses contributed <2% to the total fruit and vegetables reported on the 24-h recalls. The dried beans group is not clearly specified in either screener, but it too accounted for little (2%) of the total consumption of fruit and vegetables reported on the 24-h recalls. Respondents may not have fully considered the use of fruit and vegetables in mixtures, such as soups, casseroles, and pasta dishes, when answering the screener questions; these uses accounted for >13% of total fruit and vegetables reported on the 24-h recalls. In addition, intake of other vegetables was underreported on the standard screener relative to the number of servings of these vegetables reported on the 24-h recalls. These findings indicate opportunities to refine the wording of questions on screeners to be developed in the future.
The FFQ used in this study performed better than did both screeners in estimating median intakes. Its better performance is most likely a result of 2 factors: the number of questions asked (frequency and portion size of 65 individual foods that contributed to the total FFQ estimate of fruit and vegetable intake) and the database used to quantify all responses into recommended servings of fruit and vegetables.
For epidemiologic studies examining the relation between fruit and vegetable intake and some outcome measure, both screeners and the complete FFQ are similar in performance, on the basis of these analyses in this particular study. Relative risk estimates made on the basis of either screener or the FFQ would be attenuated. For example, for men, a true relative risk of 4.0 would be observed as 2.2 when data from the standard screener were used, 2.1 when data from the new screener were used, and 1.9 when data from a complete FFQ were used. For women, a true relative risk of 4.0 would be observed as 2.1 when using data from the standard screener, 1.8 when using data from the new screener, and 1.61.7 when using data from a complete FFQ.
We conclude that for surveillance purposes of estimating median intakes of fruit and vegetables, both screeners substantially underestimated true intakes and thus the prevalence of meeting recommended intakes. If screeners are to be used for these purposes, appropriate adjustment procedures need to be developed and tested. For examination of relative risk estimates between total fruit and vegetable intake and some disease, both screeners performed nearly as well as a much longer FFQ in this sample. However, if individual components of usual fruit and vegetable intake are to be examined, a complete FFQ that provides more specific intake information would be needed.
More cognitive research is needed to understand how individuals conceptualize their intake of fruit and vegetables and how questions can be worded so that they correspond to these concepts. New approaches or, alternatively, more precise questions and refined language to make existing tools easier to complete, are needed. The resultant instruments should be tested in different populations, using accepted criterion data, and in comparison with another screener instrument.
| ACKNOWLEDGMENTS |
|---|
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
S. G. Chan, S. C. Ho, N. Kreiger, G. Darlington, E. M. Adlaf, K. F. So, and P. Y. Y. Chong Validation of a Food Frequency Questionnaire for Assessing Dietary Soy Isoflavone Intake among Midlife Chinese Women in Hong Kong J. Nutr., March 1, 2008; 138(3): 567 - 573. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. W. Greene, K. Resnicow, F. E. Thompson, K. E. Peterson, T. G. Hurley, J. R. Hebert, D. J. Toobert, G. C. Williams, D. L. Elliot, T. Goldman Sher, et al. Correspondence of the NCI Fruit and Vegetable Screener to Repeat 24-H Recalls and Serum Carotenoids in Behavioral Intervention Trials J. Nutr., January 1, 2008; 138(1): 200S - 204S. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. K. Campbell, K. Resnicow, C. Carr, T. Wang, and A. Williams Process Evaluation of an Effective Church-Based Diet Intervention: Body & Soul Health Educ Behav, December 1, 2007; 34(6): 864 - 880. [Abstract] [PDF] |
||||
![]() |
C. S. Levesque, G. C. Williams, D. Elliot, M. A. Pickering, B. Bodenhamer, and P. J. Finley Validating the theoretical structure of the Treatment Self-Regulation Questionnaire (TSRQ) across three different health behaviors Health Educ. Res., October 1, 2007; 22(5): 691 - 702. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. J. Thompson, J. Heimendinger, A. Diker, C. O'Neill, A. Haegele, B. Meinecke, P. Wolfe, S. Sedlacek, Z. Zhu, and W. Jiang Dietary Botanical Diversity Affects the Reduction of Oxidative Biomarkers in Women due to High Vegetable and Fruit Intake J. Nutr., August 1, 2006; 136(8): 2207 - 2212. [Abstract] [Full Text] [PDF] |
||||
![]() |
W. T. Hey, K. S. Calderon, and H. Carroll Use of Body-Mind-Spirit Dimensions for the Development of a Wellness Behavior and Characteristic Inventory for College Students Health Promot Pract, January 1, 2006; 7(1): 125 - 133. [Abstract] [PDF] |
||||
![]() |
H.-C. Hung, K. J. Joshipura, R. Jiang, F. B. Hu, D. Hunter, S. A. Smith-Warner, G. A. Colditz, B. Rosner, D. Spiegelman, and W. C. Willett Fruit and Vegetable Intake and Risk of Major Chronic Disease J Natl Cancer Inst, November 3, 2004; 96(21): 1577 - 1584. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. P. Bogers, P. van Assema, A. D. M. Kester, K. R. Westerterp, and P. C. Dagnelie Reproducibility, Validity, and Responsiveness to Change of a Short Questionnaire for Measuring Fruit and Vegetable Intake Am. J. Epidemiol., May 1, 2004; 159(9): 900 - 909. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. P. Bogers, P. C. Dagnelie, K. R. Westerterp, A. D. M. Kester, J. D. van Klaveren, A. Bast, and P. A. van den Brandt Using a Correction Factor to Correct for Overreporting in a Food-Frequency Questionnaire Does Not Improve Biomarker-Assessed Validity of Estimates for Fruit and Vegetable Consumption J. Nutr., April 1, 2003; 133(4): 1213 - 1219. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. M. Krebs-Smith and L. S. Kantor Choose a Variety of Fruits and Vegetables Daily: Understanding the Complexities J. Nutr., February 1, 2001; 131(2): 487S - 501. [Abstract] [Full Text] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |