A revised approach in the biomedical sciences for reporting research should set a strong precedent for researchers, publishers, and regulators around the U.S. who are committed to improving the science that guides public health decisions.
Scientists today are more prolific than ever. The sheer body of research published every year can be overwhelming—from breakthroughs in innovation, to new health and safety studies, to novel solutions to improve environmental health. If there is one thing that should ground them all in the public trust, it should be a commitment to adhering to an established sound scientific process.
Unfortunately, many of the scientific studies we read about in the news were not quite ready for prime time. Forbes contributor Geoffrey Kabat wrote in a November 15 piece that often it is the “anomalous, and almost certainly wrong, results” that can get the most attention. As Kabat points out, these inconsistent results can have serious ramifications for public health:
This phenomenon leads to an enormous waste of resources, which comes at the expense of research that might actually lead to saving lives. It also confuses the public and leads people to mistrust science generally.
Many in the scientific community agree, and at least one group of editors has already begun to lead a return to the fundamental principles of science to restore public confidence.
Journals stand up for science
“Reproducibility, rigor, transparency, and independent verification are cornerstones of the scientific method,” Editor-in-Chief Marcia McNutt made clear at the opening of a November 7 editorial published in the journal Science.
According to McNutt, a swath of editors from more than 30 major journals, together with funding agencies and other scientific leaders met at the American Association for the Advancement of Science headquarters earlier this year to tackle the reproducibility issue plaguing so many preclinical studies of late. Below is a list of guidelines they agreed to as first big step toward improving the integrity of the biomedical sciences:
- Journals should make it clear to authors the policies for statistical analysis and how they review the statistical accuracy of work under consideration
- Any imposed page limits should not discourage reproducibility (in other words, researchers should be given the time and space to show their work)
- Researchers should use a checklist to ensure the reporting of important experimental parameters: standards used, number and type of replicates, statistics, method of randomization, whether experimenters were blind to the conduct of the experiment, how the sample size was determined, and what criteria were used to include or exclude any data
- Journals should recommend that data be placed in public repositories, where available, and linked to the paper to ensure proper attribution
- Journals should ensure that all datasets relied upon for conclusions in the paper are made available upon request, where ethically appropriate, by editors and reviewers; in addition, journals should encourage the release of data for sharing after publication
- Once a journal publishes a paper, it should assume the obligation to consider publication of a refutation of that paper, subject to its usual standards of quality
- Journals should consider establishing best practices guidelines for images and biological material use.
Valuable lessons for improving chemical assessments
Because of their fundamental nature, the guidelines have tremendous potential to help the public beyond their use by journals and beyond the biomedical science field. For example, many of the guidelines proposed in McNutt’s editorial are consistent with the recommendations that have been recently offered to improve how federal programs are evaluating chemicals.
For example, there are many parallels when it comes to improving the review and transparency of the research used in chemical assessment programs. In several cases, these concepts are in line with ACC’s principles for enhancing federal risk assessment, especially when it comes to evaluating data and selecting studies used in assessments and providing full disclosure of underlying data and key information used to develop assessments.
Fully implementing these improvements to chemical assessments will provide regulators, the public, and industry with more accurate and useful information to help guide better decisions for protecting human health and the environment.
Call to action
Now that leaders in the biomedical sciences have acknowledged that their journals may not always meet scientific standards of reproducibility, rigor, transparency and independent verification, it’s time that researchers and journals that examine chemicals do the same.
The American Association for the Advancement of Science (AAAS) pledge should serve as a wake-up call to researchers and journals across the toxicological sciences to step up their game by renewing their commitment to the scientific method. And regulators can help facilitate this “return to the science” by rigorously reviewing existing published studies to ensure they meet these core tenets of the scientific method and calling on journals to publish more robust, independently verified studies going forward.
As McNutt concludes, “The hope is that these guidelines will not be viewed as onerous, but as part of the quality control that justifies the public trust in science.”
