|
|
||||||||
COMMENTARY |
1 From the Division of Population Health and Information, Alberta Cancer Board, Calgary, Canada
2 CMF was supported by a Canadian Institutes of Health Research New Investigator Award and an Alberta Heritage Foundation for Medical Research Health Scholar Award. 3 Address reprint requests and correspondence to HK Neilson, Division of Population Health and Information, Alberta Cancer Board, 1331-29 St NW, Calgary T2N 4N2, Canada. E-mail: heathnei{at}cancerboard.ab.ca.
| ABSTRACT |
|---|
|
|
|---|
Key Words: Energy expenditure motor activity physical activity metabolic equivalents questionnaires retrospective studies doubly labeled water validation studies epidemiologic methods adults
| INTRODUCTION |
|---|
|
|
|---|
Recently, higher levels of AEE have been reported to decrease the risk of all-cause mortality in elderly people (4) and blood pressure in younger adults (5). Higher levels of energy expenditure from nonexercise activities may also prevent weight gain (6, 7). However, owing to the inherent difficulties in assessing the duration, frequency, and intensity of all types of activities undertaken by free-living participants in large population studies (8, 9), the amount of AEE required for disease prevention and health promotion remains unclear.
In the continued absence of inexpensive, readily available, relatively noninvasive, valid and reliable technology for measuring AEE in large numbers of free-living humans, researchers may, by necessity, rely on estimates of AEE derived from physical activity questionnaires (PAQs; Figure 1
). Although a number of PAQs have been designed to capture various activity parameters, many have shown limited reliability and validity (12). Moreover, it is not entirely clear whether or not any are valid for estimating AEE at the individual level or even at the group level.
|
| SUBJECTS AND METHODS |
|---|
|
|
|---|
We included studies that aimed to 1) validate a PAQ against DLW, or 2) predict DLW-derived energy expenditure using a PAQ. We defined PAQs as instruments requiring retrospective activity recall beyond 24 h. The search was limited to human studies published in full text, in English, and with no restrictions on publication year. All subjects needed to be adults (aged
19 y) studied under free-living conditions. We excluded studies that were exclusive to athletes, pregnant or lactating women, or individuals with acute or chronic disease. We also excluded studies of PAQs designed for, and studied in, one ethnic minority subgroup because we were interested in PAQs that could, in theory, be applied more broadly to the general adult population.
Appraisal of study design
Using criteria described by Rennie and Wareham (43), we summarized study design characteristics that could affect the quality of PAQ validation studies. To appraise study designs, we extracted the following information from each article: mean age and body mass index (BMI; in kg/m2) of the respondents, sample size, length of DLW phase (defined as the number of days between DLW administration and the last day of urine collection), mode of PAQ administration, and the timing of each PAQ relative to its corresponding DLW phase.
Appraisal of questionnaire design
We summarized PAQ attributes to assess face validity for estimating usual AEE. We extracted information on the types and parameters of activities ascertained, time periods recalled, PAQ format, and the outcome summary measures. If this information was not provided for a particular PAQ, we consulted additional publications to complete our data collection. Physical activity duration, frequency, and intensity were classified as "self-reported parameters" only if respondents were asked explicitly to report them. Although a variety of activities were queried across PAQs, we limited our discussion to major activity types, defined as occupational, household, and leisure-time. Using criteria described previously in the literature (8, 38, 44, 45), we classified the format of each PAQ as global, recall, or quantitative.
Summary of analytic results
We quantified PAQ criterion validity in terms of the magnitude of agreement and correlation between DLW and PAQ estimates of TEE (TEEDLW, TEEPAQ) or AEE (AEEDLW, AEEPAQ). Mean differences (eg, AEEPAQ – AEEDLW) and Pearson's or Spearman's correlation coefficients (eg, AEEPAQ versus AEEDLW) were reported as group measures of agreement in kcal/d. We chose these statistics because no single measure is without limitation (46, 47) and because they are often used to estimate the validity of physical activity assessment tools (48, 49). We also reported SEM differences, P values for correlation coefficients, and 95% limits of agreement (95% LOA), where 95% LOA = mean difference [(ie, AEEPAQ – AEEDLW or TEEPAQ – TEEDLW) ± 2 x SD of the mean difference; 50)] as a measure of between-subject variability. If these statistics were not reported by authors, or were not reported as kcal/d, we derived them if enough data were provided in the article.
| RESULTS |
|---|
|
|
|---|
|
30), whereas 13 (51, 52, 54, 55, 58, 61, 65-71) included generally overweight subjects (ie, mean BMI of 25 to <30), but only 2 (69, 70) did so intentionally.
PAQ administration
Most studies (12 of 20) used interviewer-administered PAQs (53, 54, 56-60, 62, 63, 66, 67, 70, 71), whereas only 5 used self-administered PAQs (51, 55, 61, 68, 69). For 3 studies, the mode was not reported (52, 64, 65).
Only 4 of 36 comparisons (57, 59, 60, 62, 71) involved PAQs that covered exactly the same period of activity as the DLW phase, with the 4 periods ranging from 7 to 14 d (Table 1
). Ten of 36 comparisons (in 7 publications) (52, 53, 55, 56, 62, 67, 69) involved past year PAQs, with corresponding DLW phases ranging from 10 to 14 d. Twenty-one of the 36 comparisons presented in Table 1
(in 10 publications: 51, 53-55, 57, 59, 60, 62, 64, 68, 71) involved PAQs administered at the end of the DLW phase, when the period of recall would have included the period of DLW measurement. The timing of PAQ administration was not reported for 3 of 36 comparisons (52, 56, 58).
Questionnaire design
Within the 20 publications included in our review, we identified 23 distinct PAQs (Table 2
). Eleven of the 23 PAQs (51, 55, 58, 61, 62, 68, 69, 71, 76, 77) covered all major types of activity (ie, occupational, household, and leisure time) to varying degrees, and 3 others (18, 60, 70) included all types of activity in terms of intensity. Regarding the time period recalled, 6 of 23 PAQs inquired about activity over the past year (32, 52, 62, 69, 75, 77). One other PAQ asked about occupational activities over the past year and leisure activities over the past month (55), whereas another PAQ asked about sports and leisure activities over the past year and other activities as they are usually performed (73). Of 23 PAQs, we classified 7 as quantitative (15, 51, 55, 68, 69, 75, 77), 13 as recall (18, 32, 52, 60-63, 70-73, 76), and 2 as global (74, 78). We were unable to classify the format of one PAQ (58) because of lack of information. Fifteen of 23 PAQs were used to derive estimates of AEE or TEE (15, 18, 51, 52, 55, 60-62, 68-70, 73, 75, 77), and all were deemed to be recall or quantitative PAQs. Across all PAQs, intensity was assigned using information provided in the Compendium of Physical Activities (10, 11) or another approach (15, 18, 25, 61, 64, 73, 75, 77, 83, 84).
|
Analytic results
To appraise criterion validity, summary statistics are presented and arranged according to PAQ outcome measure and period of recall in Table 3
. Across all 20 publications, a total of 10 comparisons were made between DLW and either a unitless PAQ score (53, 62, 63, 65) or the duration of activity derived from a PAQ (58, 71). Because the 10 PAQs involved in these comparisons estimated neither AEE nor TEE, we excluded these 10 comparisons from our appraisal, which left only 26 comparisons (16 publications) in Table 3
.
|
10% (55, 59, 60, 62, 64, 66, 67, 70). In terms of kcal/d, the magnitude of the mean difference (AEEPAQ – AEEDLW) ranged from –10 kcal/d (Yale PAQ; 67) to 952 kcal/d (7-d MLTQ; 51). The mean difference (TEEPAQ – TEEDLW) ranged in magnitude from 17 kcal/d (7-d Physical Activity Recall; 66) to 1589 kcal/d (Harvard/College Alumni; 61). Twelve of 26 comparisons (52, 53, 56, 57, 61, 62, 67, 68, 70) showed negative mean differences (ie, group underreporting), 13 (51, 53-55, 61, 62, 64, 66, 69) showed positive mean differences (ie, group overreporting), and one (59) showed positive and negative differences (mean difference = 130 kcal/d, n = 34; mean difference = –160 kcal/d, n = 37; 60). In a comparison of AEEPAQ with AEEDLW, 3 separate evaluations of the past year Minnesota Leisure Time PAQ resulted in underestimation of AEE at the group level, with mean differences of –208 kcal/d (–37%) (56), –313 kcal/d (–39%) (53), and, in the third study, –487 kcal/d (–56%) and –752 kcal/d (–62%) for females and males, respectively (67). Conversely, 5 evaluations of TEEPAQ from the 7-d Physical Activity Recall questionnaire (53, 54, 61, 64, 66) showed group overreporting, with mean differences ranging from 17 kcal/d (0.7%) (66) to 989 kcal/d (31%) (54). However, the same questionnaire resulted in group underreporting in 2 (57, 70) of 3 studies (51, 57, 70) that compared this PAQ with AEEDLW.
In 8 comparisons, the direction of bias [ie, AEEPAQ – AEEDLW (51) or TEEPAQ – TEEDLW (54, 55, 59, 61)] became more positive with increasing mean AEE [ie, average of AEEPAQ and AEEDLW (51)] or TEE [ie, average of TEEPAQ and TEEDLW (54, 55, 59, 61)]. In other words, there was a positive trend between the individual differences and the means. Conversely, a slight negative trend, in which the direction of bias became more negative with increasing AEE, was reported for one comparison (70). Two other comparisons also showed negative trends (57, 67); however, in these instances the differences (AEEPAQ– AEEDLW)were plotted against AEEDLW rather than as the average of the 2 measures, which is recommended (86).
To assess validity further, we examined correlation coefficients reported for 19 comparisons in Table 3
(51, 53-56, 61, 62, 68-70). Coefficients ranged from 0.05 (7-d MLTQ in 80 middle-aged females; 51) to 0.83 (past year MLTQ in 13 elderly males and females; 56). Five coefficients were notably >0.60 (55, 56, 62, 68).
We also evaluated PAQ validity by considering mean differences and correlations, simultaneously, for 19 of the 26 comparisons in Table 3
. Of these, only 3 comparisons resulted in mean percentage differences of
10% and a correlation of
0.60 [modified Stanford Usual Activity 7-d index (62), TCHS (62), and Tecumseh Occupational Activity and past month MLTQ (55)]. All 3 comparisons were conducted in middle-aged men with BMIs ranging from values indicating normal weight to overweight, on average. Two of these PAQs—the Tecumseh Occupational Activity and past month MLTQ (55) and the TCHS PAQ (62)—also showed the greatest potential for capturing AEE in our earlier appraisal of questionnaire design (Table 2
). The former PAQ was self-administered (55), whereas the latter was administered in a face-to-face interview (62).
Individual level agreement
Inconsistent reporting precluded us from comparing individual level results across most studies and, thus, individual agreement was not summarized for this review. Only 7 of the 20 publications were explicit in reporting individual level agreement. Results were expressed as the proportions of positive and negative differences (ie, AEEPAQ – AEEDLW; 51), as the number of reporters within a certain percentage of their DLW estimate [within 10% (55, 70); TEEPAQ
5%, 5–10%, 10–20%, or >20% of TEEDLW (54)], or as tables of individual results (52, 56, 64).
We judged between-subject variability in terms of the 95% LOA for 22 of the 26 comparisons in Table 3
. The widths of 95% LOA (ie, upper limit minus lower limit) for the mean difference (AEEPAQ – AEEDLW) spanned from 817 kcal/d (past year MLTQ (56) to 4096 kcal/d (7-d MLTQ; 51). For the mean difference (TEEPAQ – TEEDLW) the widths of 95% LOA ranged from 1133 kcal/d (Tecumseh Occupational Activity and past year MLTQ; 69) to 17 948 kcal/d (Cross-Cultural Activity Participation Study 4-wk recall; 61).
| DISCUSSION |
|---|
|
|
|---|
One common feature of the PAQs in this review was their inclusion of exercise, defined as planned, structured, and repetitive bodily movements intended to improve or maintain one or more components of physical fitness (87). In this review, however, our interest applied more broadly to AEE, or the energy expended from all exercise and nonexercise activities, both volitional and nonvolitional. We also wanted to explore PAQ validity in the context of population-based, etiologic studies of chronic disease. For this reason we were interested in "usual" AEE; in other words, relatively stable patterns of activity that, if prolonged, could contribute to disease risk.
Only 4 of the 23 PAQs in our review contained all of the basic design elements required for estimating usual AEE (Table 2
): the QAPSE PAQ, TCHS PAQ, Tecumseh Occupational Activity and past month MLTQ, and the Tecumseh Occupational Activity and past year MLTQ. In addition to queries regarding the "major" activity types, these PAQs also inquired about personal care, climbing stairs, walking, transportation, or sedentary activities. In theory, each activity could contribute to PAQ validity for estimating AEE. For instance, lower intensity activities (88) and posture during sedentary activities can influence AEE (7, 89, 90). Although all 4 PAQs inquired about sedentary activity, the Tecumseh Occupational Activity and past month MLTQ contrasted sitting versus standing activities, inquired about sleeping, and asked about "general activities" such as childcare, reading, and watching television (55). Incidentally, this PAQ also had higher criterion validity (Table 3
) and was quantitative in format. In contrast, 3 separate evaluations of the past year Minnesota Leisure Time PAQ showed underestimation of AEE by 37% to 62% at the group level (53, 56, 67). Although we are uncertain why underestimation occurred, it is possible that the omission of key activities (eg, occupational) from this PAQ contributed to its discrepancies with DLW.
Face validity is important not only when interpreting PAQ validation studies, but also in etiologic research. Recently, the issue of PAQ validity was raised (91) when significant inverse associations were found between all-cause mortality and AEE estimated from DLW (4). Self-reported stair climbing and work for pay were more likely to be reported by participants with a higher AEE. In contrast, the proportion of individuals who self-reported high-intensity exercise, walking for exercise, or walking for reasons other than for exercise did not change significantly across AEE tertiles. It was speculated that self-report errors may be to blame for the latter (91). We propose, however, that these findings may have arisen in part because these activities were not significant contributors to AEE in this population. Before discrediting self-reporting, therefore, we will need to understand the contributions of various types of activity to AEE. Only then can we determine whether or not PAQs have sufficient face validity for testing AEE-related hypotheses.
Before drawing any conclusions about PAQ validity, the quality of each validation study must be taken into account. First, the generalizability of study findings in this review was limited by tightly constrained study conditions and small sample sizes. Because the mode of PAQ administration (76, 92) and respondent characteristics (51, 70, 93) could affect PAQ validity, it may be that PAQs that appear to be valid (or not valid) under the conditions in which they were studied would perform differently in other contexts. Presumably, in some cases, small sample sizes were due to the prohibitive monetary cost of DLW experiments at the time the studies were undertaken. Regardless, the decision to "scale down" validation studies clearly comes at the expense of an inability to generalize results to other populations and provides less precise estimates of PAQ validity. Correlation coefficients, for example, become less precise when based on smaller samples because of an increased SE [ie, SE = [(1 – r2)/(n – 2)]1/2) (47, 94). Unfortunately, SEs were rarely provided in the articles, whereas P values (Ho:
= 0) were common. It should be noted that less than half of the correlation coefficients in Table 3
differed significantly from zero, but this result may have arisen simply as a consequence of smaller sample sizes, at least in some instances. Even statistically significant correlations may be imprecise if based on very few observations. Thus, all correlations in Table 3
should be interpreted with caution.
Even with sufficient sample sizes, the limitations of correlation coefficients are well documented (47, 49, 50, 94), with the literature emphasizing 2 potential pitfalls. First, correlations depend on the degree of between-subject variability in a given study population, so an acceptable correlation found in one PAQ validation study may not apply to groups with a different range of energy expenditure levels (49). Second, correlations are measures of association as opposed to agreement. A method with known systematic bias can correlate quite strongly with an unbiased reference measure (47, 49, 50), thereby masking a lack of agreement between measures. Despite these limitations, correlation coefficients have thus far been the most commonly used statistics in the PAQ validation study literature (49). We recommend, as have others (47, 49, 94), that PAQ validity not be judged solely on the basis of correlations, but rather on several statistical methods that would each compensate for the other's unique limitations.
Unfortunately, we found that important details of PAQ validation studies were sometimes omitted from the published literature, which made it difficult to generalize findings to other populations. In this review we noted missing information on the mode of PAQ administration (52, 64, 65), the BMI of validation study participants (53, 63), the timing of PAQs relative to DLW phases (52, 56, 58), and the length of DLW phase (61). Thus, authors are encouraged to be more specific when reporting on PAQ validation studies.
Bland-Altman plots, although not always presented by authors, were more informative. Some plots showed mean differences (ie AEEPAQ – AEEDLW or TEEPAQ – TEEDLW) or group level bias that increased with the level of energy expenditure, typically in the positive direction. These positive trends may imply that populations with higher levels of AEE or TEE tend to overreport their activities in PAQs. Although this tendency was apparent in some groups, it is noteworthy that not every individual with a high level of energy expenditure would overreport their activities. The group level, proportional bias we observed suggests systematic error in physical activity reporting that could be corrected, perhaps using regression calibration techniques (95), or with a modified PAQ design based on the determinants of misreporting, once they are better understood.
Alternatively, it is possible that the positive trends observed in several Bland-Altman plots were the result of random error. In a recent simulation study (49), a spurious, positive trend resulted in a Bland-Altman plot when one hypothetical unbiased measure of physical activity (M) was compared with a second, unbiased reference measure (R). The authors created the trend by simulating greater random error in measure M, thereby violating the assumption of equal variances between M and R which is inherent to Bland-Altman plots (86). This explanation seems plausible in the context of a PAQ validation study, in which DLW is a very objective, precise method with less random error than PAQs. Judging from Bland-Altman plots alone, it is unclear how much of the positive trend was attributable to random and systematic error, respectively. Regardless, this potential weakness of the Bland-Altman method could have important implications for PAQ validation research.
Across studies we observed an overall tendency to report agreement at the group level as opposed to the individual level. Individual validity cannot be inferred from group level validity (96) and in fact serves a different purpose. In this review, the mean differences across studies indicated that neither under- nor overreporting of activity was more common at the group level. This finding differs from dietary research in which underreporting is universally more common (96), which implies that different factors may be involved in misreporting of activities and diet, respectively. Because only a few articles reported results at the individual level, we were unable to evaluate fully the degree of individual level validity for many of the PAQs. However, the 95% LOA proposed by Bland and Altman (50) allowed for speculation by examining between-subject variability. The 95% LOA for the mean difference (AEEPAQ– AEEDLW) from one PAQ, for example, ranged from –464 to 645 kcal/d (Yale PAQ; 53). By definition (48, 97), this result implies that there is a 95% probability that an individual from the same population would underestimate AEE by no more than 464 kcal/d and overestimate AEE by no more than 645 kcal/d. If this level of error was not acceptable for practical purposes, the PAQ could not be used as a surrogate for AEEDLW. In fact, no 95% limit in our review was within 100 kcal/d (10% if AEE = 1000 kcal/d) of the mean difference of AEEPAQ – AEEDLW and or within 250 kcal/d (10% if TEE = 2500 kcal/d) of TEEPAQ – TEEDLW, which suggests that the PAQs in this review may be of limited use for estimating individual AEE.
A related matter for concern is the validity of DLW for individuals. At the group level, the DLW method is widely accepted as the gold standard for estimating free-living energy expenditure in adults (9, 98-100). In a review on DLW validity (101), the percentage error in TEE estimation averaged
2% or 8%, depending on the equation used to calculate DLW results. This level of error is acceptable for evaluating PAQ validity at the group level, as we have assumed. For individuals, however, the precision of DLW (ie, the SD of individual percentage errors) in the same review article (101) was 8–9%, which meant that some individual estimates will deviate substantially from the average. Furthermore, in comparisons based on AEE, DLW estimates must be derived by subtraction (ie, AEE = TEE –RMR –TEF), which could introduce more error if RMR is based on prediction equations. In a review of prediction equations (35), the proportion of healthy adults with valid predicted RMR (ie, within 10% RMR measured by calorimetry) ranged from 45% to 81% in nonobese individuals and from 38% to 70% in obese subjects. Under a worst-case scenario, therefore, a PAQ validation study using one of these equations to predict AEE would inevitably suggest disagreement, which may be wrongly attributed to the PAQ. Moreover, statistical measures of validity normally deemed "moderate" may be, under this scenario, as high as could be expected. Although an important consideration, of the 20 publications we reviewed, only 6 compared PAQs with DLW-derived AEE (10 PAQ comparisons in Table 3
; 51, 53, 56, 57, 67, 70), and only one of those used an RMR prediction equation to derive AEE from DLW (51). Thus, for 34 of the 36 PAQ-to-DLW comparisons in our review [2 PAQs in Adams et al (51)] RMR-related error of this magnitude was probably not an issue. Otherwise, it is possible that some individual differences reflected in the 95% LOA may have arisen in part from DLW-related error.
Concurrency between PAQ and DLW administration is another factor to consider when evaluating PAQ validity (43). Ideally, in validation studies, the criterion measure and PAQ should observe the same time frame of reference. In this review, however, the majority of method comparisons (32 of 36) did not cover exactly the same time periods. Free-living energy expenditure, when assessed using objective measures, is known to vary over days of the week (102), over weeks (103), and over seasons (104, 105) in adults. In one study of habitual activity, Levin et al (104) periodically measured physical activity patterns in 77 adults in the United States over 1 y. On the basis of intraindividual variability, they determined that six 48-h accelerometer sessions were needed to achieve 80% reliability in estimating mean annual physical activity in MET (metabolic equivalents)-minutes per day (104). Although analytic error does contribute to estimates of within-subject variability (102, 106), so does actual change in activity levels over time. The latter might partly explain some of the differences we observed between PAQ and DLW estimates of energy expenditure. In our review, 10 of 36 DLW comparisons involved past year PAQs and single 1- or 2-wk DLW phases. An alternative approach would have been to repeat DLW phases to coincide better with PAQs, but none of the 10 studies reported to have done this.
A final consideration for interpreting any PAQ-to-DLW comparison is the potential for error in converting self-reported physical activity into units of energy expenditure. In large epidemiologic studies, it is usually not feasible to measure energy costs of activities for each individual. Hence, the Compendium of Physical Activities (10, 11) has become a widely accepted extrapolation tool. The Compendium provides a convenient 5-digit coding scheme that can be used to classify activities according to rate of energy expenditure or METs. By definition, 1 MET is approximately equal to 1 kcalh–1 · kg body wt–1 (11). Clearly, an assumed body weight or RMR will rarely reflect that of a given individual. One important limitation of the Compendium, therefore, is the reliance on group averages that may not apply to individuals (1, 10, 11, 107, 108).
In conclusion, despite the numerous validation studies already published, the validity of PAQs for AEE estimation remains unclear. Weaknesses in the design and reporting of studies, combined with a paucity of information on the original intent of many PAQ designers, mean that it is difficult to draw any firm conclusions about the validity of existing PAQs for the assessment of usual AEE in large population-based studies. Nevertheless, our review highlights some important considerations for scrutinizing PAQ validation studies. First, there is a need to consider each PAQ's design and its expected level of agreement with DLW, which measures all activities (freely available PAQs would facilitate this; 109). Furthermore, if a PAQ is to be used to estimate individual AEE, then its validity must be supported by the appropriate statistical analyses. Results on individual level validity were generally lacking across the articles we reviewed. We speculate that some discrepancies found previously between PAQ and DLW estimates could have resulted, in part, because PAQs did not include key activities relating to AEE or, possibly, because the PAQ period of recall and DLW phase did not coincide. Issues related to small sample size, use of correlation coefficients, and conversion of self-reported activity into energy expenditure, all continue to be problematic. Future research and development efforts should address these issues to clarify the true validity of PAQs in this context.
| ACKNOWLEDGMENTS |
|---|
The authors' responsibilities were as follows—HKN: conducted the review and wrote the first draft of the manuscript; IC and PJR: helped write subsequent drafts of the manuscript; IC, PJR, and HKN: conceptualized the review and interpreted the results; and CMF: critically appraised the manuscript. None of the authors had a conflict of interest with respect to this manuscript.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
A. Luke, L. R Dugas, K. Ebersole, R. A Durazo-Arvizu, G. Cao, D. A Schoeller, A. Adeyemo, W. R Brieger, and R. S Cooper Energy expenditure does not predict weight change in either Nigerian or African American women Am. J. Clinical Nutrition, January 1, 2009; 89(1): 169 - 176. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. M. Ulrich and R. S. Holmes Shedding Light on Colorectal Cancer Prognosis: Vitamin D and Beyond J. Clin. Oncol., June 20, 2008; 26(18): 2937 - 2939. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |