|
|
||||||||
Original Research Communications |
| ABSTRACT |
|---|
|
|
|---|
Objective: This study examined the reliability of the portable hemoglobinometer (PHM) system with use of capillary blood and the implications of errors of the magnitude found for the classification of anemia status in individuals and population groups. The precision and accuracy of the method with use of venous blood were also tested.
Design: Three empirical data sets were used to measure reliability, precision, and accuracy of the PHM system [2 from Honduras (n = 87 and 141); 1 from Bangladesh (n = 73)]. Simulation data were used to assess the implications of errors for screening individuals for anemia and to estimate anemia prevalence.
Results: High within-subject variability (unreliability) was identified when capillary blood from the left hand was compared with that from the right hand (CV: 6.3%) and when measurements were taken on 4 consecutive days (CV: 7.0%). Reliability was only 69% and 50%, respectively. Precision and accuracy, however, were very high (concordance coefficients of 0.99 and 0.98 and CV < 1%).
Conclusions: The simulation data showed that errors of the magnitude found due to unreliability can lead to misclassification of anemia status in individuals and small biases in anemia prevalence estimates. We recommend replicate sampling to reduce the influence of unreliability in the use of the PHM system with capillary blood.
Key Words: Nutrition assessment hemoglobin error reliability precision accuracy capillary blood hemoglobinometer Honduras Bangladesh anemia
| INTRODUCTION |
|---|
|
|
|---|
Less well documented is the reliability of the PHM system, meaning the extent to which a measurement in a given individual is reproducible over time (5) or at a single time across different sites. One study with only 6 subjects reported large variations in the hemoglobin concentration of capillary blood samples obtained from different fingers of the same individual at the same time (6). Within-person CVs ranged from 3% to 7%, substantially higher than those reported by Statland et al (7) for hemoglobin measured in venous blood samples and analyzed by standard methods (2.5% for hour-to-hour variation and 2.6% for day-to-day variation). In another study (8), venous blood samples drawn from the same individuals on 2 different occasions and assessed by using an electronic counter showed within-person variances of 28.16 (g/L)2 in males and 23.23 (g/L)2 in females, corresponding to CVs of
3.5%.
If capillary blood sampling with the PHM method is to continue to be popular in field research in developing countries, the reliability of the approach must be confirmed. Failure to control for even relatively small unreliability will widen the distribution of hemoglobin values and may result in biases in estimates of anemia prevalence. Failure to control for unreliability may also bias associations with risk factors and estimates of response to interventions toward the null, thereby underestimating true effects.
This paper reports the use of data from 2 studies to measure unreliability of hemoglobin measurements with capillary blood in the PHM system and the use of data from a third study to estimate precision and accuracy of the method with use of venous blood. A simulation model was used to test the implications of errors of the magnitude found in our empirical data for screening individuals for anemia and estimating anemia prevalence.
| SUBJECTS AND METHODS |
|---|
|
|
|---|
|
Study B: central Bangladesh (near sea level)
Study B was a study of the effect on micronutrient status of the adoption of 3 different packages of agricultural innovations that was conducted in the administrative regions of Saturia, Mymensingh, and Jessore. Seventy-three female subjects from 7 different villages (3 in Saturia, 2 in Mymensingh, and 2 in Jessore) were asked to contribute blood on 4 consecutive days; 1 woman was absent on the last day and therefore contributed only 3 measurements, and on 1 other occasion there was an interval of 2 d between consecutive measurements. Samples, from the left hand of right-handed women and from the right hand of left-handed women, were taken in the morning in Mymensingh and in the afternoon in Jessore and Saturia. Consecutive samples were not necessarily from the same finger. Blood sampling and analysis of hemoglobin by the PHM method was performed by specially trained doctors using the same method described for study A. Verbal consent was obtained from the women for this procedure after a careful explanation of its objectives and risks.
Study C: San Pedro Sula, Honduras (near sea level)
In this study of anemia in exclusively breast-fed infants, a sample of 141 infants aged 4 mo had 2 mL venous blood sampled from whichever arm had the most accessible vein. The infants came from poor families in the urban area of San Pedro Sula, the second largest city in Honduras. The blood was collected by a trained phlebotomist into a Monovette tube (Sarstedt, Newton, NC) containing EDTA as an anticoagulant. Immediately afterward, one drop of whole blood was transferred from the collection tube onto a small piece of parafilm, drawn by capillary action into 2 separate microcuvettes, and analyzed in the PHM by a specially trained laboratory assistant. At the same time, 0.5 mL whole blood was transferred to a glass tube and analyzed in duplicate within the hour in a hematology analyzer (Cell-Dyn 601; Abbott Diagnostics, Mount View, CA). Written, informed consent for the study procedures was obtained from the mothers of the infants within 1 wk of delivery.
In all 3 studies described above, special care was taken to ensure high-quality data from the PHM readings. The PHMs were checked each day by using the special microcuvettes provided by the manufacturer and sampling was conducted exactly as recommended by the manufacturer. In particular, each full cuvette was carefully checked for air bubbles and the plastic containers of unused cuvettes were immediately resealed after use to avoid degradation from heat, humidity, or both.
Simulation model
In addition to the 3 empirical data sets, we present results from simulated data with 20000 observations and 2 variables: "true" hemoglobin status and apparent hemoglobin status in the presence of instrument imprecision and biological variability. The parameters for the simulation data were derived from the 3 empirical data sets. We assumed that true hemoglobin values were normally distributed about their mean with an SD of 10 g/L and that the within-subject variability was also normally distributed and uncorrelated with the true value. The mean of the true values was allowed to take 3 different values (110, 120, and 130 g/L) and the within-person SD was allowed to vary from 4 g/L (CV of
3%) to 8 g/L (CV of 67%). The between-person SD of the true mean values was set at 10 g/L, roughly the average of the SDs found in our 3 empirical data sets. The simulation data were used to evaluate the effects of different levels of error on the sensitivity and specificity of a diagnosis of low hemoglobin (<120 g/L), as well as the predicitive values of a positive (<120 g/L) and negative (
120 g/L) screening result. We also assessed the magnitude of biases in anemia prevalence that would result from these levels of error.
Data analysis
Reliability is defined in this paper as the extent to which a measurement in a given individual is reproducible over time (5) or at a single time across different sites. Unreliability (1 - reliability) is assumed to be synonymous with within-subject variability and is the sum of imprecision (error due to measurement) and undependability (error due to short-term within-individual random fluctuations). Accuracy is defined as lack of error due to the instrument and refers to the comparison of the method with a gold standard.
Unreliability was estimated with data from studies A and B. Study A measured unreliability due to differences in hemoglobin concentration at different body sites and study B measured unreliability due to changes over time. Site-to-site unreliability was assessed by taking blood from the left and right hands of the same person within a period of 5 min and short-term temporal unreliability was assessed by taking blood from the same individual on 4 consecutive days. Within-subject SDs were estimated by using a one-way random-effects analysis of variance model with no covariates (9). The intraclass correlation coefficient (10) provided a direct measure of reliability (1 - unreliability). The within-subject CV (%) was calculated as follows:
|
| (1) |
± 2 SDs of the differences) (11). Precision was assessed by contrasting repeat measures obtained by using the same analytic method and blood. The means of the paired differences, the limits of agreement, and the within-pair CVs were estimated (9, 11). Pearson correlation coefficients between pairs of readings are also presented for comparability with the published literature. A more appropriate measure of correspondencealso presentedis the concordance coefficient, which, in addition to measuring the agreement between 2 readings, also measures the departure from the 45° line through the origin (11, 12).
Accuracy was assessed by using the same statistical approach, but by comparing the PHM method with the standard laboratory method based on the same (venous) blood samples. All analyses were conducted by using STATA (version 5.0; STATA Corp, College Station, TX).
| RESULTS |
|---|
|
|
|---|
|
20 g/L. The estimated within-subject SD was 8.3 g/L (not shown). Similarly, in study B, in which finger-prick blood samples were taken from the same individuals on 4 consecutive days, reliability was just 49.7%. The within-subject CV was 7.0% and the estimated within-subject SD was 8.2 g/L (not shown).
|
The likely effects of different levels of error on the sensitivity and specificity of a diagnosis of low hemoglobin (<120 g/L) with use of the simulation data set described in the Methods section are given in Table 4
. The predictive values of a positive (<120 g/L) and negative (
120 g/L) result with use of the capillary sample are also shown, as are the estimated population prevalences of low hemoglobin concentrations in each case. This analysis showed that when the population mean hemoglobin concentration is equal to the cutoff point for diagnosis of anemia (120 g/L), the true prevalence of low hemoglobin is equal to 50% and use of a single capillary sample provides an unbiased estimate of this quantity, irrespective of the level of within-subject error. At the level of individuals, an adequately sensitive and specific (
80%) diagnosis of low hemoglobin is obtained even with within-subject error as high as 6.0 g/L (SD, between sampling sites). This scenario corresponds with a within-person CV of 5%. With a level of error of 8 g/L (CV: 6.7%), sensitivity, specificity, and predictive values are all in the range of 7580%.
|
| DISCUSSION |
|---|
|
|
|---|
Our studies in the western region of Honduras and in Bangladesh (studies A and B) showed large within-subject variation in the hemoglobin concentration of blood sampled from fingertip capillaries. In the first case, this variation reflected differences in the site of sampling the hemoglobin concentration of capillary blood (left or right hand), whereas in the second case it reflected short-term, within-subject temporal variation over a 4-d period (and possibly the site of sampling also). The CVs estimated from these data6.3% in study A and 7.0% in study Bare strikingly consistent with the findings of Boulton et al (6), although only 6 individuals were investigated in that study. On the other hand, studies quantifying the within-person variability of hemoglobin concentrations of venous blood (7, 8) have found lower within-subject variability (2.23.5%).
We believe that it is biologically plausible that capillary blood would be more variable in hemoglobin concentration than venous blood because inclusion of extracellular fluid would decrease the concentration of components present in the red cell fraction, and the amount of extracellular fluid present in finger-prick samples is likely to be very sensitive to the technician's handling of the patient's finger. Although technicians were rigorously trained to avoid "milking" the finger in all of our studies, some contamination with extracellular fluid seems inevitable in any programmatic application of the PHM or capillary blood sampling method. More research is needed to establish whether within-subject CVs in capillary blood hemoglobin of the order of 67% represent the true norm for biological variation or whether further standardization of the sampling method can reduce this variability. Our analysis of study C suggested that imprecision (instrument error) can be ruled out as a significant component of the error due to unreliability (sum of error due to undependability + imprecision).
Our simulation model suggested that estimates of the prevalence of low hemoglobin are unbiased when the true prevalence is 50%, regardless of the degree of unreliability of the method. Bairagi (15) used a theoretical approach to show that the absolute size of the bias increases until the diagnostic cutoff is equal to the population mean ± 1 SD (equivalent to true prevalences of 16% or 84%) and then falls again as the difference between the cutoff and the population mean becomes even greater (true prevalences <16% or >84%). We have shown that the levels of unreliability found in our 2 data sets would result in absolute biases in the prevalence of low hemoglobin values of no more than 6% if the mean population hemoglobin were 1 SD above or below the diagnostic cutoff. A bias of this magnitude will often be acceptable; when it is not, it may be possible to assess the scale of error in a small subsample and use existing methods to correct the prevalence estimate (16).
We also examined the likely consequences of error in hemoglobin concentrations in capillary blood on the screening for anemia and found that if the population values were symmetrically distributed around the cutoff value for anemia, a single fingertip sample would provide an adequately sensitive and specific (
80%) diagnosis of low hemoglobin concentration, even if the level of error was at the top of the range considered (CV from
3% to 67%). However, if the population mean were 1 SD below the cutoff point, specificity (and the predictive value of a negative, or high, result) would be jeopardized, whereas with the population mean 1 SD above the mean, sensitivity and the predictive value of a positive (low) result would be similarly affected. As explained by Himes (17), the reliability of measures subject to within-subject error can be increased by using the mean of several replicate measures instead. Indeed, the Spearman-Brown Prophesy formula (10) allows investigators to calculate how many replicates would be needed to achieve a desired level of reliability. Our data from western Honduras (study A) indicated that no less than 4 replicates would be required to estimate individual hemoglobin concentrations with 90% reliability, whereas our Bangladesh data (study B), which included within-week temporal variation, suggested that 9 replicates would be required. Although we found double sampling to be well accepted at our western Honduras study site, we doubt that >2 replicates would be feasible in most circumstances.
In conclusion, we believe that the PHM method with use of capillary blood samples will continue to be a useful screening tool for anemia, but that unreliability poses a significant problem whenever individual-level estimates are required. In these circumstances, measures such as replicate sampling should be used to reduce the effect of within-subject error, and even then, residual unreliability should be assumed.
| ACKNOWLEDGMENTS |
|---|
| FOOTNOTES |
|---|
2 The ideas and opinions presented here are the responsibility of the authors and should in no way be attributed to the International Fund for Agricultural Development.
3 Supported by the International Fund for Agricultural Development (TA grant 301-IFPRI).
4 Reprints not available. Address correspondence to SS Morris, IFPRI-FCND, 2033 K Street NW, Washington, DC, 20006-1002. E-mail: s.morris{at}cgiar.org.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
T. W. McDade and B. Shell-Duncan Whole Blood Collected on Filter Paper Provides a Minimally Invasive Method for Assessing Human Transferrin Receptor Level J. Nutr., December 1, 2002; 132(12): 3760 - 3763. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. W Solomons and K. Schumann Lessons learned in iron intervention trials Am. J. Clinical Nutrition, September 1, 2002; 76(3): 691 - 693. [Full Text] [PDF] |
||||
![]() |
A. Bhargava, H. E. Bouis, and N. S. Scrimshaw Dietary Intakes and Socioeconomic Factors Are Associated with the Hemoglobin Concentration of Bangladeshi Women J. Nutr., March 1, 2001; 131(3): 758 - 764. [Abstract] [Full Text] |
||||
![]() |
P. R Pehrsson, P. B Moser-Veillon, L. S Sims, C. W Suitor, and E. Russek-Cohen Postpartum iron status in nonlactating participants and nonparticipants in the Special Supplemental Nutrition Program for Women, Infants, and Children Am. J. Clinical Nutrition, January 1, 2001; 73(1): 86 - 92. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. Zavaleta, L. E Caulfield, and T. Garcia Changes in iron status during pregnancy in Peruvian women receiving prenatal iron and folic acid supplements with or without zinc Am. J. Clinical Nutrition, April 1, 2000; 71(4): 956 - 961. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |