| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Supplements |
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONSTUDY DESIGN AND PLANNINGCOLLECTION OF DATA ON...DATA ANALYSISSUMMARY AND RECOMMENDATIONSREFERENCES |
|---|
Key Words: Bias • diet • epidemiologic methods • measurement error • nutrition assessment
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONSTUDY DESIGN AND PLANNINGCOLLECTION OF DATA ON...DATA ANALYSISSUMMARY AND RECOMMENDATIONSREFERENCES |
|---|
Evaluation of observational studies and the evidence drawn from them includes consideration of the study design, sample selection, and type of measurements of disease and exposure. The execution of observational studies in epidemiology is a lengthy process that begins with the design and planning of the study then proceeds to data collection, analysis, and interpretation of results. Each part of the process consists of many steps. At the completion of the study, the inferences drawn can be no stronger than the weakest part of the process. To evaluate epidemiologic evidence relating food and nutrient exposures to a specific health outcome, many different aspects of this process must be considered.
|
STUDY DESIGN AND PLANNING |
|---|
|
TOPABSTRACTINTRODUCTIONSTUDY DESIGN AND PLANNINGCOLLECTION OF DATA ON...DATA ANALYSISSUMMARY AND RECOMMENDATIONSREFERENCES |
|---|
Another consideration is whether the study is designed to examine a specific outcome and to what extent data are collected on other risk factors for that outcome. Secondary analyses of data collected for other purposes may not include the appropriate covariates for the outcome of interest. For example, secondary analyses of dietary intake and cancer mortality from a study of cardiovascular disease may be limited by a lack of information about cancer-related risk factors.
In other studies, data on total dietary intake may be collected without a clearly specified prior hypothesis. These studies allow the examination of a wider range of foods and nutrients although detailed information on factors relevant to the study of a specific nutrient may not be available. Multiple analyses of data from such studies may reveal statistically significant results by chance alone. In general, a study that was designed to measure a specific exposure and a specific outcome and that includes data on other appropriate risk factors can provide stronger evidence for or against a particular association.
|
COLLECTION OF DATA ON DIETARY INTAKE |
|---|
|
TOPABSTRACTINTRODUCTIONSTUDY DESIGN AND PLANNINGCOLLECTION OF DATA ON...DATA ANALYSISSUMMARY AND RECOMMENDATIONSREFERENCES |
|---|
Recall and questionnaire methods such as 24-h recalls or food-frequency questionnaires do not require record keeping but do depend on the ability of the respondent to both remember and characterize dietary intake accurately. Respondents may omit or incompletely specify foods ( 5 – 7 ), report foods that were not actually consumed ( 5 , 7 ), or estimate portion sizes incorrectly ( 6 , 8 ).
Food-frequency questionnaires also require respondents to remember and characterize food intake accurately and present more complex cognitive tasks than are required in 24-h dietary recalls. Respondents are presented with a short, prespecified list of foods or groups of foods and are then asked to generalize in terms of the portion sizes and frequency of foods consumed. Numerous cognitive difficulties arise with this method ( 9 , 10 ); cognitive interviewing of respondents completing a food-frequency questionnaire has been described as a "sobering experience" ( 10 ). Research has shown that portion size estimates are relatively insensitive to the specification of exact amounts ( 7 , 11 ). With some semiquantitative food-frequency instruments, such as the Block questionnaire ( 12 ), respondents may be asked to indicate small, medium or large portion sizes, but the portion sizes used for calculations are based on those from 24-h recall data from other people of the same sex and age. In other semiquantitative food-frequency instruments, such as the Nurses' Health Study questionnaire, portion sizes are prespecified and respondents are expected to be able to modify and adjust their frequency responses accordingly ( 13 ). Less is known about frequency estimation than about portion-size estimation. For energy and macronutrients, differences in reported frequencies of consumption appear to be the main source of differences in ranking between food-frequency questionnaires and multiple-day reference methods ( 14 , 15 ).
|
DATA ANALYSIS |
|---|
|
TOPABSTRACTINTRODUCTIONSTUDY DESIGN AND PLANNINGCOLLECTION OF DATA ON...DATA ANALYSISSUMMARY AND RECOMMENDATIONSREFERENCES |
|---|
Changes over time in the content of prepared and processed foods, variations in recipes and cooking practices, and geographic and environmental variability may all contribute to a difference between the actual nutrient value of a specific food and the value calculated from a nutrient database. For example, as reported by Heaney ( 19 ), Charles ( 20 ) performed a chemical analysis of calcium in foods consumed in a series of metabolic balance studies in which the exact quantities of every food consumed were known with high accuracy; the calculated calcium intakes derived from food tables reflected <70% of the actual variability in calcium intake among subjects. Furthermore, if a food is incompletely specified it may be difficult to ascertain the correct nutrient content to use; the rapid introduction of new food products makes it difficult to maintain a completely updated nutrient database. Several studies have compared the results of calculating nutrient values for dietary data by using different nutrient databases ( 21 – 24 ). These studies show fairly good agreement in mean nutrient values for a group but some dramatic differences for individual diets.
Information on some nutrients may be sparse or inaccurate. One example of this is the trans fatty acid content of foods, for which limited nutrient database data are available ( 25 ). Another example is selenium content, which varies considerably with the geographic origin of foods and thus cannot be estimated accurately from nutrient databases ( 26 ). Sodium intake is also difficult to estimate accurately from dietary data ( 18 ), in part because of the variability in the amount of sodium used in cooking and processing procedures. We must remember that we are often not getting information on all nutrients, and it is often tempting to interpret dietary data only in terms of the nutrients available from the database. In addition, foods include many constituents that are not considered nutrients, including chemical compounds found naturally in raw foods, contaminants, and constituents produced during processing and cooking.
Additional issues arise if foods are grouped according to precoded groupings ( 27 , 28 ). This is generally done for food-frequency questionnaires but can also be applied to other methods. In effect, a food-frequency questionnaire, or any precoded questionnaire, uses a highly abbreviated nutrient database in which all foods within a category are assigned exactly the same nutrient content. The nutrient values for a food grouping are generally derived from the mean or median values for a usual mixture of foods within that group eaten by a reference population ( 3 ). This introduces a possibility for bias to the extent that the amount of a specified nutrient differs among the foods within the group. If the study emphasizes a certain specified nutrient, then foods can be grouped according to their content of the specified nutrient to minimize this source of bias. However, if multiple nutrients are of interest, it is difficult to form food groupings that will be reasonably homogeneous for all the nutrients being used. The food groupings are based on mean intake over the population, but a given individual may well select only specified items within the group. To the extent that an individual selects foods within the grouping with different mean contents for a nutrient than the specified value for that grouping, the estimates of nutrient intake for that individual for that nutrient will be biased. This effect is likely to be different for different nutrients. Thus, the use of food groupings may introduce not only error within one nutrient, but differences in error between nutrients. The use of food groupings may thus affect agreement of nutrient estimates with a reference method that does not use food groupings ( 3 ).
Not all foods or food groups can be included in a precoded questionnaire such as a food-frequency questionnaire. The selection of foods and food groups for a food-frequency questionnaire is generally based on items that account for a high proportion of the total diet or of the variability between individuals. However, it should be remembered that the coverage of the diet can vary among individual respondents. The questionnaire may cover less of the total diet of some individuals than others. Once dietary data are collected and translated into nutrients, many more issues arise within the analytic process that need to be considered both in the analysis and interpretation of data.
Energy intake and energy adjustment
Studies using the doubly labeled water method showed that use of self-reported dietary data from records and recalls often resulted in underreporting of energy intake (
29
–
32
). Population estimates suggest that reported amounts are insufficient to sustain energy balance (
29
,
33
,
34
). Therefore, it is reasonable to assume that dietary intakes are not being reported completely in many cases. Although dietary underreporting is more pronounced in people with higher body weights, it appears to occur to some degree in people with lower body weights as well (
29
–
32
).
An approach that has received considerable attention is that of energy adjustment. In the original version of this method, a linear regression of the nutrient on energy intake is performed, and the residuals from the regression are added back to the expected value at the mean energy intake to create an energy-adjusted value for the nutrient, which is then used in the analysis ( 35 ). The resulting value does not represent a corrected value for the absolute intake of the nutrient, but rather the intake of the nutrient relative to total energy intake. An approach more familiar to nutritionists that accomplishes similar results is to divide the amount of the nutrient by total energy intake to calculate, for example, milligrams of calcium per megajoule of energy.
There are 2 somewhat different justifications for energy adjustment, one to adjust for individual differences in energy expenditure, the other to compensate for errors in energy intake reporting. Individual differences in energy intake may reflect differences in body size, metabolic rate, or physical activity level ( 36 ). Intake of macronutrients such as fat may be expressed appropriately as a percentage of total energy intake. However, for many other nutrients, the absolute value may be of greater interest.
Many reports on energy adjustment address the issue of how to interpret the results of analyses that use different methods of energy adjustment, assuming that energy intake and macronutrient intake were both measured correctly (
11
,
37
–
39
). However, one of the goals of energy adjustment is to overcome problems of variable errors in the overall level of reporting (
35
). Such errors arise when some individuals overreport and other individuals underreport their intake. Reported energy intakes are often more variable in food-frequency data than in record and recall data from the same individuals (
14
,
35
). Several food-frequency questionnaires, including those used in the Nurses' Health Study and the Health Professionals Follow-Up Study, show a correlation of 
0.4 with a reference method for energy intake, implying that much of the variance in calculated energy intake may be error variation (
14
,
40
,
41
). Extreme overreporting or underreporting of energy intake generally leads to similar extremes in reporting of other nutrients (
15
). Thus, a large reported intake of a given nutrient can be the result of overreporting total dietary intake rather than true high consumption of the nutrient. In this case, the absolute amount of the nutrient is in error and cannot be corrected, but the amount of the nutrient relative to energy intake can be calculated.
If we assume that all dietary elements are reported proportionally, then energy adjustment should compensate for errors in the overall level of reporting. Several types of evidence suggest, however, that this assumption may not hold. For example, relative validation study results for fat intake from white women in the University of Michigan Food Frequency Study were almost identical to those from the Nurses' Health Study questionnaire ( 13 , 15 ). The unadjusted correlation with a reference method (multiple days of dietary intake) for fat intake was 0.39 for questionnaires in both studies, and the correlations for energy-adjusted fat intake between the questionnaire and the reference method were 0.60 for the Michigan study and 0.53 for the Nurses' Health Study. Analyses of the Michigan data for macronutrients showed that almost 40% of white women (as well as 32% of white men, 47% of black men, and 52% of black women) had misreported fat intake by 20% more or less than they had misreported protein and carbohydrate intake, with the predominant tendency being to underreport fat intake relative to protein and carbohydrate intake ( 15 ). If each individual had over- or underreported fat to exactly the same degree as they did for the other macronutrients, then energy adjustment would have removed any differences in ranking by macronutrient intake. However, the effect of disproportionate misreporting is that energy adjustment does not remove all the differences between methods. This can be seen from the correlation coefficients for both the Michigan study and the Nurses' Health Study, in which considerable differences between methods remained even after energy adjustment. Energy adjustment removes differences in methods resulting from overall differences in reported energy intake but cannot remove differences resulting from disproportionate reporting.
Many validation studies calculated the correlations of nutrient estimates from a food-frequency questionnaire and a reference method both with and without energy adjustment ( 13 , 40 , 42 , 43 ). Any differences that remain between methods after energy adjustment must be due to disproportionate reporting between methods. Thus, correlation coefficients between a food-frequency questionnaire and reference data that show considerably less than perfect correlations for energy-adjusted nutrients indicate the presence of disproportionate reporting between the 2 methods. Validation studies agree in showing disproportionate reporting on a food-frequency questionnaire compared with reference methods in the sense that the correlations of energy-adjusted nutrients between methods are considerably less than perfect ( 13 , 40 , 42 ). Not infrequently, some of the correlations for energy-adjusted nutrients are actually lower than the correlations for nutrients that have not been energy-adjusted . For example, in the Nurses' Health Study validation study, the correlation with the reference method for carbohydrate intake was 0.53 before energy adjustment and 0.45 after energy adjustment ( 13 ).
We know little about differential misreporting on records and recalls. If foods that are omitted have little nutrient value, then energy adjustment will tend to exaggerate the nutrient content of the diet.
Thus, it appears that energy adjustment is most useful when the following conditions both hold true: 1 ) The exposure of interest is not the absolute amount of the nutrient but the amount relative to total energy intake. This may be the case for macronutrients but is less likely to be the case for micronutrients. 2 ) Energy over- or underreporting is proportionally the same for all foods, so that the correct proportion of the nutrient to energy intake is reported. As reviewed above, most evidence suggests that food-frequency questionnaires differ from multiple-day food records in the proportionality of reporting, so that the second condition may not hold true for these questionnaires; less is known about other dietary methods.
Compensating for misclassification and measurement error
Energy adjustment, as noted above, is one approach to correcting for a specific type of measurement error, that of overall of energy intake. However, there are numerous other sources of measurement error in dietary assessment (
18
). Many approaches to correcting for measurement error make 2 potentially incorrect assumptions: that measurement error is random and that random measurement error biases estimates only toward the null value. Thus, it may be stated, erroneously, that the finding of a weak association by using a dietary method with considerable measurement error indicates that there is actually a much stronger association because the measurement error in the method would bias the findings toward the null value. However, this inference is questionable both because the measurement error is often not random and because the effects of even random error may include differential misclassification and bias away from the null value. It is sometimes assumed that increasing sample size will minimize the effects of measurement error. However if biases are systematic rather than random, then increasing sample size will have little effect on the bias.
One concern when categorizing dietary intake is that differential misclassification, in which the probability of being misclassified according to dietary intake is different in subjects with and without the disease under study, will occur. When the error probabilities are different in subjects with and without the disease under study, estimates of the association between diet and the outcome can be biased in either direction, either toward or away from the null. It is sometimes thought that differential misclassification cannot occur in prospective studies, in which dietary data are collected before the onset of the disease under study. However, whenever dietary intake is grouped into quantiles, differential misclassification is likely to occur even if measurement error is random and the study is prospective and the effects of this differential misclassification are not easily predictable ( 44 ). Correction methods that assume nondifferential misclassification will in general give erroneous results when applied to quantiles of dietary intake.
Measurement error is not necessarily random or normally distributed around the true value. Several studies showed that dietary data tend to have a flat-slope syndrome, in which larger portions of food are underestimated and smaller portions are overestimated ( 4 , 45 , 46 ). This is a form of systematic or scaling bias for which it is difficult to correct ( 47 ). A similar problem occurs with body mass index calculated from self-reported weight and height ( 48 ). Because the flat-slope syndrome tends to reduce variability and compress the distribution, it is difficult to recover the true value. Also, the reduced variability in reported exposure may tend to exaggerate the risk estimates because the variation in risk of the outcome occurs over a smaller reported exposure range.
Prentice ( 49 ) examined the potential effect of measurement errors in fat intake and the bias resulting from the association of such errors with body size for both 4-d food records and food-frequency questionnaire data and concluded that measurement error biases were such that dietary self-report instruments were potentially inadequate for epidemiologic studies of dietary fat intake and disease risk. Prentice used a flexible measurement error model that allowed for measurement error parameters to depend on body mass index and also allowed for a random underreporting quantity. This model suggested that the potential effects of measurement error were large enough to reduce a strong relation of fat intake with postmenopausal breast cancer of the magnitude seen from the international correlational analyses to a weak or nonexistent association.
The high level of error in dietary data appears to be more pronounced in shorter methods such as single 24-h recalls and food-frequency questionnaires. Clayton and Gill ( 50 ) pointed out many of the issues and the difficulty in carrying out statistical analyses that attempt to allow for measurement error effects. Because we do not really know the source or nature of these errors, we cannot be sure that errors are not associated with characteristics that also predict the outcome under study. Any association of measurement error with the outcome under study could lead to misleading or biased results.
Another aspect of measurement error issues is related to confounding, in which a third factor is associated with both the outcome and the exposure under study. One potential source of confounding is ethnic differences. Although only a few studies have addressed this issue, it appears that there may be some systematic differences between groups in their responses to different types of dietary assessments ( 14 , 51 ). Dietary assessment methods used for studies that compare ethnic groups may have differential measurement error in the 2 groups being studied, which will tend to confound the results.
If some foods are reported more accurately than others, then nutrients occurring in the former will be reported more accurately than those in the latter. Validation studies of food-frequency questionnaires suggest that there may be considerable differences in the accuracy of reporting of individual foods because the correlation coefficients between the food-frequency questionnaire and the reference method tend to differ considerably for different nutrients. A striking example is provided by a study conducted as part of the Iowa Women's Health Study, in which the Nurses' Health Study instrument was administered to a subgroup and the findings were compared with dietary records ( 42 ). Some nutrients, such as iron, showed a negative correlation with the reference method. This suggests that the instrument was not appropriate for measuring iron intake in this population. This study and many others also suggest that different nutrients are estimated with different levels of accuracy, which can affect analyses in several ways. First, in general, a nutrient that is more accurately measured may be more likely to show a consistent association with a certain outcome than will a nutrient that is less accurately measured, even if both nutrients might have similar associations with the outcome if measured accurately. Second, and perhaps more important, analyses of one nutrient are often controlled or adjusted for intake of a second nutrient, which is treated as a potential confounding factor. If the second nutrient is measured with error, then results are unpredictable. If the errors in the 2 nutrients are independent, this will result in incomplete control for confounding. However, if the errors in the 2 nutrients are correlated, as is likely to occur, then bias may occur in either direction; thus, the results of the analysis will be extremely difficult to interpret. This source of bias in the estimate of the association of a nutrient with a given outcome is often not considered.
|
SUMMARY AND RECOMMENDATIONS |
|---|
|
TOPABSTRACTINTRODUCTIONSTUDY DESIGN AND PLANNINGCOLLECTION OF DATA ON...DATA ANALYSISSUMMARY AND RECOMMENDATIONSREFERENCES |
|---|
Data on nutrient intake from dietary data are prone to several types of error. Studies using the doubly labeled water method suggest that energy intake is frequently underreported, which leads in turn to underreporting of other nutrients. However, the degree of underreporting appears to vary from person to person and also by nutrient. The degree and type of error in nutrient data can lead to many analytic problems and potentially to bias either toward or away from the null value. Methods of statistical correction and adjustment for error, including energy adjustment, cannot necessarily compensate completely for sources of bias in dietary data, partly because the assumptions about the type of errors may not be valid. Additional research should be conducted to identify sources of error in dietary data and to improve dietary assessment methods. Published research using reported dietary data should include a discussion of potential sources of error in such data and their effect on the results.
Because of the multiple sources of error in dietary assessment and the many measurement and analytic issues that arise, observational studies that rely on self-reports of dietary intake are not always well suited to making precise estimates of the effect of a single food or nutrient on a specific outcome. Observational epidemiologic studies of diet and outcome using self-reported dietary data may often serve better to generate hypotheses than to provide conclusive findings. Although randomized clinical trials of the effects of specific nutrients or dietary modifications are not always feasible, such trials provide more definitive results and should in general be considered more valid than observational studies using self-reported dietary intake. Well-designed observational epidemiologic studies using self-reported dietary intake data can provide valuable information to support or challenge hypotheses derived from clinical or laboratory data and suggest further directions for investigation.
|
FOOTNOTES |
|---|
2 Reprints not available. Address correspondence to KM Flegal, National Center for Health Statistics, 6525 Belcrest Road, Room 900, Hyattsville, MD 20782.
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONSTUDY DESIGN AND PLANNINGCOLLECTION OF DATA ON...DATA ANALYSISSUMMARY AND RECOMMENDATIONSREFERENCES |
|---|
This article has been cited by other articles:
|
|
 |
|
  D. Taubert, R. Roesen, and E. Schomig Effect of Cocoa and Tea Intake on Blood Pressure: A Meta-analysis Arch Intern Med, April 9, 2007; 167(7): 626 - 634. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. R Trumbo and K. C Ellwood Lutein and zeaxanthin intakes and risk of age-related macular degeneration and cataracts: an evaluation using the Food and Drug Administration's evidence-based review system for health claims. Am. J. Clinical Nutrition, November 1, 2006; 84(5): 971 - 974. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. H Hyun, E. Barrett-Connor, and D. B Milne Zinc intakes and plasma concentrations in men with osteoporosis: the Rancho Bernardo Study Am. J. Clinical Nutrition, September 1, 2004; 80(3): 715 - 721. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. Byers, B. Lyle, and W. Participants Summary statement Am. J. Clinical Nutrition, June 1, 1999; 69 (6): 1365S - 1367S. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
