|
|||||||
|
|
|
|
|
|
|
|
 |
|
COMMENT BY:
The Salt Institute
Dear Mr. Charney: Public Law 105-277 adopted and signed late last year requires "Federal awarding agencies to ensure that all data produced under an award will be made available to the public through the procedures established under the Freedom of Information Act...." The Salt Institute writes in support of the objectives of the proposed revisions to Section .36 of OMB Circular A110, as published in the Federal Register on February 4, 1999 (64 Fed. Reg. 5684) to provide the framework for federal agencies to implement the expanded "access to data" contemplated in the statute. The Salt Institute is the trade association representing the nation's salt producers and a frequent participant in regulatory rulemakings by federal agencies. We support sound, current science as the basis for federal policies, particularly in the areas of health and the environment in which we have the greatest experience. Science is a process of creative destruction: current understanding is expected to be replaced with deeper insights as the process unfolds. Thus, what we "know" today is an hypothesis to be bombarded with data, re-evaluated in light of newer findings and revised as necessary. Science not only encourages this process; it is absolutely essential to the advance of science. That is why prominent medical journals and public health agencies and advocacy groups support greater openness in the scientific process. In order to move this process ahead, it is necessary for scientists to share the relevant portions of their research with their fellow scientists who can assess methods and analysis. This is done in a very limited manner through the peer review process of medical journals. Scientists have crafted procedures which allow access to data while preserving the absolute confidentiality of patient data and the rights of the original investigators to full credit for the research. The OMB rule contemplates retaining these procedures which is absolutely essential to the FOIA process. Not only is allowing independent expert access to scientific data a professional imperative, but federal statutes and rules reinforce the concept that unless the data have been subjected to the rigors of such analyses they lack the power and credibility to sustain federal policy. Furthermore, studies which the federal government relies upon as the basis for policy-making which are also paid for by federal taxpayers are doubly obligated to make their data available for independent analysis and confirmation. It is insufficient protection for both the regulated parties and federal taxpayers that the editors of a medical journal may pronounce the validity and meaning of the research results. While in practice few studies would require such extensive, independent analysis, the availability of such an objective review would supplement the peer review process of medical journals in promoting presentation and interpretations of research findings which fairly characterize the methods and data. Improved "access to data" would enhance the quality of federal decision-making. Others are more expert in the details of the necessary procedural safeguards which must be incorporated into the rule. We write to share our experience as illustrative of the great need to move forward in broadening access to these data. Over the past decade, the Salt Institute has attempted to obtain independent expert analyses of the Intersalt Study, a multi-national study examining the relationship of dietary sodium to population blood pressures. Our experience shows why expanded access will enhance the quality of federal policymaking. For purposes of this example, then, the key federal agencies, all part of the Department of Health and Human Services, are the National Institutes of Health (specifically, the National Heart, Lung and Blood Institute), the Public Health Service and the Food and Drug Administration. The NIH has examined its procedures in recent years. In 1997, it adopted Guidelines for the Conduct of Research in the Intramural Research Programs at NIH (Jan. 1998) which declared the expectation that "results of research will be carefully recorded" and "...data, including the primary experimental results, should be retained.. .to allow analysis and repetition by others of published material from those data." In 1998, the NIH Working Group on Research Tools recommended' that: "NIH should promote free dissemination of research tools without legal agreements whenever possible, especially when the prospect of commercial gain is remote. "NIH should develop and disseminate guidelines for recipients of NIH funds as to reasonable terms ... both importing of research tools owned by other institutions and exporting of research tools created with NIH funds. "NIH should review its policies with regard to dissemination of research tools ... and revise and strengthen those policies. "...NIH should consider ... replacing current 'encouragements' to recipients to disseminate unique research resources with stronger requirements to the same effect...." The Public Health Service Policy Statement declares: "It is the policy of PHS to make available to the public the results and accomplishments of the activities that it funds. Restricted availability of unique resources upon which further studies are dependent can impeded the advancement of research and delivery of medical care. n2 The Food and Drug Administration policy declares: "The Food and Drug Administration will make the fullest possible disclosure of records to the public.... " Specifically: "The results of all testing or research conducted by or with funds provided by the Food and Drug Administration.. .are available for public disclosure when the final report is complete and accepted by the responsible Food and Drug Administration official... .Access to all raw data, slides, worksheets, and other similar working materials shall be provided at the same time that the final report is disclosed. " Congress also weighed in on the subject, including in the Nutrition Labeling and Education Act (NLEA) the standard that FDA rules be based on "all publicly available scientific evidence."5 With this as prologue, let us review the Salt Institute's experience with regard to the Intersalt Study. In 1980, the federal government adopted a public health nutrition policy of universal sodium restriction (salt is the primary source of dietary sodium, hence our interest). Critics were quick to cite the paucity of evidence to support the government's policy, so the National Heart, Lung and Blood Institute (NHLBI) agreed to co-fund an international epidemiological study to examine whether population sodium excretion (a reliable marker for dietary sodium intake) is related to population blood pressure. Populations with lower blood pressures have a lower incidence of cardiovascular events like heart attacks. Earlier, controversial studies had produced inconsistent results including some studies purporting to show such an association; upon those studies the government relied in setting its policy. The Intersalt Study was to be the definitive study to settle the question; the National Heart, Lung and Blood Institute alone provided principal investigator Dr. Jeremiah Stamler of Northwestern University with more than $1.3 million6 to conduct the study. Intersalt studied 10,079 individuals in 52 study centers in 32 countries. When reported in 1988 in the British Medical Journal,7 the authors - all committed advocates of sodium restriction - reported a weak positive relationship. They confessed, however, that when they excluded the four study centers whose subjects were remote primitive tribes without ready access to salt, which they did in the initial article, noting how different were the results of these four centers from the other 48, the sodium-blood pressure association lost statistical significance. The authors also reported a post-hoc analysis showing that if this "snapshot" study were re-conceptualized as a longitudinal cohort study, that sodium excretion was strongly correlated with a rise-with-age in blood pressure. The accompanying BMJ editorial concluded about Intersalt: "Salt has only small importance in hypertension." Critics disputed the validity of the rise-with-age analysis. Nevertheless, the federal Department of Health and Human Services relied upon the authors' characterization of the data for Surgeon General's recommendations on nutrition, for the national health objectives in Healthy People 2000, for NLEA sodium labeling regulations and, with USDA, for the 1990 Dietary Guidelines. The Salt Institute, belatedly recognizing how the Intersalt Study was being (ab)used, filed an FOIA seeking to have an independent expert analyze the data. We were told that NHLBI uses two types of awards mechanisms: contracts and grants; contracts were subject to FOJA, but support for the Intersalt Study was (surprise!) in the form of grants. Thus, although the agency funded the study, NHLBI said had "no right" to obtain the data. This, despite the fact that the lead NHLBI researcher was a co-author of the paper, and had direct access to those data. We asked the lead investigator, Dr. Stamler, for access; he refused. We contacted the NIH Office of Scientific Research Integrity; there we were told if we would allege scientific fraud, they would conduct an investigation. While we might have quarreled with the authors' interpretations, we were not alleging fraud, so that was a dead-end. Frustrated at these rebuffs, we had our attorney contact Dr. Stamler suggesting he had an obligation to make the data available for independent analysis. We never heard back from Dr. Stamler - but we DID hear back from his attorneys who told us: "Dr. Stamler and his INTERSALT colleagues and Northwestern University Medical School and abroad will not and cannot release the data which the Institute has requested."8 Thus began a couple more years of back-and-forth exchanges among the lawyers which culminated six years after initial publication with an offer by Dr. Stamler's attorneys that the Intersalt Consortium would run their data tapes and answer a single question for us. I had to sign a confidentiality agreement which we were able to negotiate allowed us to share the data with federal health agencies and public health nutrition panels providing we advised them to maintain the data in confidence. We received the data in mid-December, about a month before the only public hearing by the Dietary Guidelines Advisory Committee. Using expert statisticians at two universities, both of whom also signed confidentiality agreements, we were able to understand that the rise-with-age conclusions depended on a particular statistical maneuver driving all the vertical axis intercepts through zero. I reported the analysis to the Dietary Guidelines Advisory Committee and requested that they hold the data in confidence as I had promised to do. They did more they refused to consider the results because they had not been published! So, the authors' decision not to publish relevant data meant that government policy-making bodies would not consider it. I wrote to the editor of the British Medical Journal explaining the situation and suggesting that the BMJ, as the original publisher, owed a duty to its readers to revisit the situation. They agreed. They shared by letter with the Intersalt authors and invited them to respond. They did; they also prepared an entirely new analysis with further statistical modeling. These were published in May 1996. This produced another exchange of expert commentary in the form of letters-to-the-editor which were also published later." The entire episode was summarized in a lengthy investigative report published last August in Science magazine, the journal of the prestigious American Academy for the Advancement of Sciences." Since then, the NHLBI has been forced to have a Workshop on Sodium and Blood Pressure (to relieve the pressure for a truly independent review). At that Workshop, expert statisticians from Harvard and the University of California-Berkeley both criticized the Intersalt authors' manipulations. The damage was done, however. The 1995 Dietary Guidelines for Americans maintained the same policy thrust. Food labels can inform consumers that reducing dietary sodium "may" reduce their blood pressure. Upon leaving his post as head of research for the British Department of Health at the first of this year, Dr. John W. Swales, past president of the International Society of Hypertension, published in The Lancet a commentary on the sodium-hypertension issue decrying the "pseudoscience" being advocated by the loudest of the advocates of universal sodium restriction.'2 Yet "pseudo-science" is what has been relied upon in making U.S. public health nutrition policy regarding dietary sodium.. .because there have been inappropriate constraints on "access to data." This rulemaking offers a great opportunity to right this wrong. Researchers should know that their studies are truly open to independent, expert validation. Government agencies should know that the studies they rely on are credible and reproducible and that parties who may wish to address the substance of the science offered to support government policies will have the access necessary to conduct analyses to verify the authors' conclusions. We recognize the legitimacy of many of the issues being raised with regard to patient privacy, the proprietary interests of the researchers, the administrative costs of maintaining these databases and likely more. We believe all these issues either have or can be addressed without sacrificing the type of access which we embrace and recommend. We also know these legitimate concerns have been addressed for many years. Back in 1986, a "Sounding Board" article in the New England Journal of Medicine hit the issue right on: "We recommend that when a government-initiated study leads to changes in health policy or regulations, data supporting such decision should be shared with all interested parties. Access to these data should not depend on whether records are in the hands of the government or a private research contractor, or whether a products-liability lawsuit has been filed. Although peer review by journals provides a check on the appropriateness and quality of research methods and analyses, that process applies only to materials selected by the authors for presentation to the journal. It is not a substitute for having access to all the data." 3. The American Academy for the Advancement of Science requires authors of papers accepted for publication in its journal Science that: "Any materials and methods necessary to verify the conclusions of the experiments reported must be made available to other investigators under appropriate conditions. "Archival data sets... should be deposited with the appropriate data bank...." 4. The Salt Institute's particular struggle to gain public access for the Intersalt data prompted a discussion of this issue as well. BMJ Deputy Editor Tony Delamothe, in an editorial published concurrent with the Salt Institute's letter addressed the rhetorical question: "Whose data are they anyway?" and answered: "Raw data from research on patients should be available, anonymised, to whoever wants them. " He argued that the interests of the research subjects, rather than being protected by granting the investigators proprietary rights to preserve the data in secret, are advanced by broad access allowing them to be the springboard to yet further studies. He summarized the benefits, quoting from Dr. Davey-Smith:
o "Using existing data to answer questions not directly addressed by the primary researchers is an efficient use of resources A year and a half later, the BMJ revisited the subject in a letter by Dr. Keith Rennolls, a prominent London mathematician.'7 Rennolls explained: "Accepted scientific practice demands that results should be reproducible by independent scientists. Financial considerations mean that it is not realistic to reproduce data comparable to the Intersalt study, but the conflict between the Intersalt Cooperative Research Group and the Salt Institute focuses on the reproducibility and validity of the statistical analysis of the Intersalt data. "...Some of my qualms are:
o "the criss crossing of population trends mentioned by Hanneman, and described by Malcolm Law in his commentary as 'bizarre' and 'implausible' is seen to occur if the trend lines are plotted Rennolls concludes: "For the medical-scientific community to achieve a shared understanding it is necessary that confirmatory secondary analyses be carried out by other researchers. For these reasons, and the powerful ethical arguments made in Tony Delamothe's editorial, the sharing of medical research data is long overdue. A good model is provided by the Economic and Social Research Council's data archive, which ensures use only by bona fide researchers and that the primary researchers are appropriately acknowledged." This is not an issue of only historical interest. There are studies right now that beg for independent analysis since NHLBI is blocking access to yet-newer databases. OMB can strike a blow for stronger science with a carefully crafted rule allowing expanded "access to data." Sincerely
Richard L. Hanneman |
|
