Remarks, EPA Science Meeting
April 12, 2001
Ron Roizen, Ph.D.send comments to ron@roizen.com
Slightly revised post-meeting. I thank Marc Stifelman for pointing out a mistake in the draft I presented concerning the probabilistic/nonprobabilistic character of state-level blood-lead survey data reported in MMWR ("Blood," 2000).
I'm going to talk a little about the draft Human Health Risk Assessment (or HHRA) completed last year for the Coeur d'Alene River Basin, specifically focusing on child blood lead levels.
The EPA has an action-initiating standard with respect to childhood blood lead -- it is: does the average child in a given community have a 5% or higher probability of possessing a blood lead level at or above 10 micrograms per deciliter? That probability standard for an individual child translates into an equivalent population measure -- namely, do 5% or more of a given community's population of children manifest blood levels at or above 10 micrograms?
I have just four points I want to make:
1. First, thoughtful students of childhood blood lead are going to disagree over whether these two standards represent good social or public health policy.
Back in the mid-1970s (according to Lynette Stokes' study), mean child blood levels were over 50 micrograms per deciliter and maximum measured blood levels were around 150 micrograms. So the area has come a long way in a quarter-century if the standard now being applied is 5% of the population at or above 10 micrograms.
The CDC lowered its blood-lead "level of concern" from 25 to 10 micrograms in 1991. The new standard was articulated in the fourth revision of a CDC document titled Preventing Lead Poisoning in Young Children. As it happens, that 1991 document's text actually encouraged inaction respecting child blood levels between 10 and 14 micrograms. I quote the three reasons the document gave:
First, particularly at low blood lead levels, laboratory measurements may have some inaccuracy and imprecision, so a blood lead level in this range may, in fact, be below 10 µg/dL. Secondly, effective environmental and medical interventions for children with blood lead levels in this range have not yet been identified and evaluated. Finally, the sheer numbers of children in this range would preclude effective case management and would detract from the individualized follow up required by children who have higher blood lead levels. (CDC, 1991)
I don't know whether blood measurement and low-blood-level therapeutics have greatly improved over the past 10 years, but it is apparent from recent survey studies that the CDC document's concern about the "sheer numbers" of children at the 10+ microgram level remains almost as true today as it was in 1991.A recent report in MMWR ("Blood," 2000) provided state-level survey data on the proportion of children measured at or above the 10 microgram level in 1998 (see Fig. 1).
Keeping in mind that the article's text cautions both that the measurement approach used may tend to marginally overestimate blood lead levels and that these data were not derived from probability samples,1 the rates reported are nevertheless striking.
Among the 19 states reporting: for the state as a whole, Michigan shows about 15% of children above the 10 microgram level, and Ohio and Wisconsin report about 12-13% of children above the same standard. Fully thirteen of the 19 states as a whole -- that's about two-thirds of the reporting states -- report that their proportions of children with 10+ micrograms meet or exceed the 5%-of-the-child-population triggering level. County-level reports in these states often exceeded the state's average level, as Fig. 1 clearly shows. Clearly, the equitable application of the EPA's "5%+ of children" standard to these data would launch remedial actions in many of the nation's states and counties. Hence, a policy standard set at the 5%+/10 microgram level is a virtual invitation to discretionary application.
2. My second point is that we don't possess a good empirical picture of the proportion of Silver Valley children at or above the 10 microgram level. The HHRA report notes that survey data have been generated from 4 surveys in the Basin, which collected 98 cases in 1996, 26 cases in 1997, 128 in 1998, and 272 in 1999 -- totaling 524 cases in all.
Interesting and useful as these survey data may be for case-finding and follow-up purposes, their catch-as-catch-can approach to sampling renders them almost useless for the purpose of estimating the proportion of the child population at or above the 10 microgram standard.
These survey data were subject to some notable potential biases, too: for instance, self-selection by parents who were more concerned about lead risk may have brought in for testing more children with higher blood levels; the $40 per test inducement offered in the big 1999 survey may have biased its results toward lower-income families -- and we know that child blood levels are correlated with that variable. A seasonal bias is also present, since all surveys were taken in summer.
The surveys also have a notable problem concerning repeat measures of the same children. The HHRA reports that 11 of the 26 cases collected in 1997 were children who were tested in the 1996 survey as well. But the report does not offer the repeater rate for the 1998 and 1999 surveys. If the high rate of repeating in the 1997 survey were characteristic of the later surveys, then twice-measured (or even three-times or four-times measured) cases would comprise a significant segment of the 524 total N. Regardless what the repeater rate was, the fact that the HHRA report often employed the 524 cases in descriptive or relational analyses shows that data analysts were happy to describe and analyze observations rather than unduplicated persons. Hence projections of one or another descriptive frequency using the 524 cases are not population rates at all but instead rates for observations.
Potential biases and duplicated-person measures pale in significance, however, next to the singular fact that these data were simply not collected by means of a probability sample.
The paramount importance of probability sampling was driven home to the American public a long time ago -- in the 1936 presidential race between Alf Landon and Franklin
D. Roosevelt. A catch-as-catch-can poll conducted by the Literary Digest, which collected responses from over two million respondents, confidently predicted that Landon would beat FDR by a substantial margin. A young George Gallup -- who employed a much, much smaller sample, but one carefully constructed to be statistically representative of the U.S. population -- predicted FDR would win. And, of course, Gallup turned out to be right, and pollsters thereafter recognized that the size of a sample was much less important than the sample's probabilistic design.
3. My third point is that the HHRA draft report only appears to avoid the weaknesses of this survey data by using a simulation model -- the IEUBK or Integrated Environmental Uptake Biokinetic model -- to estimate child blood levels in our communities. Given the problems I've mentioned above about the available survey data, we might breath a sigh of relief that a model is going to do the blood lead estimating instead.
But the model has its own estimation problems and its estimates fall short of compelling. Some of its drawbacks were obviously sensed by the HHRA author's themselves -- a fact evidenced, for example, in their close attention to sources of what they termed "uncertainties" as well as in the development and use of alternative versions of the model, which often made quite divergent predictions of blood levels in a given locale.
The scientific literature also has its concerns about the IEUBK. For example, In 1999, two Polish epidemiologists (Biesiada and Hubicki, 1999) in effect confirmed the Texas prediction by showing that the IEUBK model performed better estimating the population mean than estimating the 10 µg/dL fraction, where the model overestimated the 10+ proportion of the population by a factor of two. Susan Griffin and colleagues (1999) published an article in 1999 showing, in part, how sensitive was the IEUBK model's 10+ microgram estimate to variations in the value for the Geometric Standard Deviation in blood levels it employed.2
So, even with the IEUBK's help, we're not quite out of the woods yet in terms of knowing the proportion of children at or above the 10 micrograms in our communities.
4. My fourth point takes us back to the multiple predictions made by various versions of the IEUBK model -- for instance, from the DEFAULT version and the so-called BOX version. As I noted previously, these versions were quite capable of making quite divergent predictions about population child blood levels. For instance, for Mullan's children the DEFAULT version said 48% were above the 10 microgram standard and the BOX version said 18%.
How to decide which is right, or closer to right, then? When I got to subsection 6.7.5 of the HHRA report I was surprised to see that its authors appeared to have data with which to evaluate and select which model prediction was better. The text said, for example, "East of Wallace, the baseline Box Model is a better predictor of observed mean blood lead levels" (see Fig. 3). I wondered where the analysts were getting their fresh and useable data. But when I examined this text more closely I realized that the "observed" data to which it referred were nothing more or less than the original catch-as-catch-can data derived from the four case-finding surveys!
Fig. 3. Passage from Section 6.7.5 of Draft HHRA East of Wallace, the baseline Box Model is a better predictor of observed mean blood lead levels. In these areas, the EPA Default baseline model significantly over-predicts both observed concentrations and the percent of children to experience excess absorption. In the community mode, both models predict more than 5% of 0-84 month old children will exceed the 10 µg/dl criteria in Mullan, Wallace, and Burke/Nine Mile. The EPA Default Model predicts 40% to 50% exceedance in these areas, the Box Model predicts 15% to 20% above the criteria. Observed exceedance in these areas ranged from 10% to 22%.
What does that mean? It seems to me that we have a kind of scientific shaggy-dog story here: It took enormous labor and forbearance for the analyst-authors to complete the HHRA draft report -- with that document's elaborate description and defense of the IEUBK model and variants. Yet, in the end the arbiter employed for selecting alternative estimates (derived from model variants) turned out to be the same old substandard, non-probability data we began with. And so the old data, not the model, in effect had the hammer -- i.e., in the end determined which model variant and its predictions or estimates would be regarded as preferable. This strikes me as a kind of prediction shell game -- and I am interested to hear what comments our colleagues on the EPA side of this meeting regarding this virtual analytical or logical sleight-of-hand.
What shall we conclude and what shall we do about our condition of ignorance about child blood levels in the Silver Valley? The obvious answer, it would seem, is that we should conduct a quality probability based survey or even a full census of children. Are probability surveys difficult to do? -- yes, but they're by no means impossible. An ongoing national survey called NHANES (National Health and Nutrition Examination Surveys) collects child blood levels on a probabilistic basis from thousands of children -- and thus may offer guidelines as to how such data may be efficiently collected. Here in the Basin we have fewer than 1,200 children under 9 in total. Hence, it is by no means unrealistic to imagine that we could get a good picture of childhood blood levels -- and end our uncertainty -- by doing a defensible, probability-based survey study.
Thank you very much.
NOTES:1 An "Editorial Note" attached to this MMWR report includes the following paragraph describing some of the limitations of the state-level data reported:
The findings in this report are subject to at least four limitations. First, the small NHANES 1999 sample does not permit observing risks in specific subgroups or geographic areas, but it provides a nationally representative estimate of BLLs in children. The CBLS data set provides local information but is limited to children who receive clinical or diagnostic blood lead testing. Second, because CDC guidelines recommend the use of blood lead data and census data to target screening efforts in populations at increased risk for lead exposure, the proportion of children with elevated BLLs is higher in CBLS data than would be expected in NHANES 1999. Third, the guidelines for testing children vary by state, and adherence to the guidelines varies by health-care provider. Finally, CBLS data include samples collected by fingerstick, which can slightly over-estimate the blood lead result, and venous samples and results obtained by different laboratories. Despite these differences, the temporal trends in BLLs are similar between the CBLS and NHANES data sets.2 I note in passing that though the HHRA draft report's bibliography runs to more than twenty pages of citations, the three somewhat critically oriented articles I've cited here do not appear therein (i.e., Carroll and Galindo [1998], Biesiada and Hubicki, [1999], & Griffin et al. [1999]).
REFERENCES:Biesiada M, Hubicki L, "Blood lead levels in children: epidemiology vs. simulations," Eur J Epidemiol 15(5):485-91, (May) 1999.
"Blood Lead Levels in Young Children ---United States and Selected States, 1996--1999," MMWR -- Morbidity and Mortality Weekly Report 49(50);1133-7, (December 22) 2000. (Available at: http://www.cdc.gov/mmwr/preview/mmwrhtml/mm4950a3.htm .)
Carroll RJ, Galindo CD, "Measurement error, biases, and the validation of complex models for blood lead levels in children," Environ Health Perspect 106 Suppl 6:1535-9, (Dec) 1998 (abstract only available).
(CDC, 1991) U.S. Department of Health and Human Services, Public Health Service, Centers for Disease Control, Preventing Lead Poisoning in Young Children, Publication date: 10/01/1991 -- see
http://aepo-xdv-www.epo.cdc.gov/wonder/prevguid/p0000029/p0000029.asp
and search for "First, particularly".Griffin S, Marcus A, Schulz T, Walker S, "Calculating the interindividual geometric standard deviation for use in the integrated exposure uptake biokinetic model for lead in children,"
Environ Health Perspect 107(6):481-7, (Jun) 1999.Stokes, L, "Study Indicates Childhood Lead Exposure May Result in Health Effects 20 Years Later," at http://www.atsdr.cdc.gov/HEC/hsph73-1.html. (See also, Stokes L, Letz R, Gerr F, Kolczak M, McNeill FE, Chettle DR, Kaye WE, "Neurotoxicity in young adults 20 years after childhood exposure to lead: the Bunker Hill experience," Occup Environ Med 55(8):507-16, (Aug) 1998 (abstract only available).