II.9.5 Definition of "Published"

Criticism: Authors of the primary publications from data will not have the opportunity to participate in or respond to criticisms or possible refutations of their analyses.

Response: Such authors always have the right and opportunity to respond. This is part of the scientific process. One of the overarching principles or goals of federal data access policy should be to create a collegial environment where criticism is allowed and good science is advanced. It may be advantageous to invite (not require) primary investigators to the table when the data become accessible. The proposed policy provides the opportunity for discourse between investigators, promoting understanding of methods of data collection and original analyses, and possibly leading to new ideas for further analysis.

Criticism: Compulsory, premature disclosure could alter the behavior of study participants, thereby jeopardizing research into questions that require longitudinal evidence-gathering to answer.

Response: If scientists are concerned that release of the data will potentially impact the behavior of participants remaining in the study, then the scientists should not release the data for publication.

Moreover, this criticism poses an unlikely scenario. In a large longitudinal study, study participants, as well as the general public, have always had access to preliminary published reports. Access to the raw data on which these published reports are based conceivably could influence the behavior of subjects with the ability and interest in accessing and manipulating the data, but this would be a very small minority. The screen/coded or redacted data in raw form would not contain results or conclusions which might alter participant behavior.

Criticism: What about a 5-year follow-up versus a 10-year? Is CRE suggesting that the results of the 5-year study not be published?

Response: No. Confidentiality protections will also ensure against any individual study participant discovering his or her treatment status in a study of blinded design, therefore clinical trial subjects could not knowingly affect study results in any direction.

Criticism: The release of incomplete data sets, drafts of study reports, internal memos, private documents, or other incomplete or preliminary analyses will result in misunderstanding, misinterpretation and/or misrepresentation of work performed by primary investigators.

Response: The new data disclosure requirements would motivate scientists who might otherwise be less careful in their work than they ought to be. This would lead to an overall increase in the quality of scientific information that is published.

Primary investigators would also be encouraged to provide detailed methods of published analyses along with the raw data, which would guard against misinterpretation of the primary analyses of the data. Reanalyses based on misunderstandings, misinterpretations and/or misrepresentations of the work of primary investigators would be subject to the same amount of criticism as the primary work. Thus, if a reanalysis is based on flawed assumptions, misrepresentation, or incorrect or inappropriate methodology, the reanalysis will be unable to withstand the rigid scrutiny of peer reviews, other researchers or journal editorial staffs. In this way, data access would further promote fundamental scientific principles. Indeed, it is apt to make the original researcher more responsible.