Scientific rigor and reproducibility have become new buzzwords, often being floated in discussions in labs, department meetings, and scientific conferences. As public confidence in science has fallen in the United States, the need for increased transparency has risen. The majority of scientists now see this issue as a significant concern, if not an outright crisis. (1) The causes for both the fall in public opinion and some of the high-profile examples that have precipitated the current situation are well beyond this article. This post will focus on three parts: first, to state what rigor and reproducibility is and what it means for trainees; second, to identify several resources that trainees will find useful; and third, to highlight the upcoming 3-part Trainee Symposium on Rigor and Reproducibility at the 2018 Experimental Biology meeting in San Diego, CA (2). Links and resources that trainees may find helpful will also be shared through the APS Trainee social media Facebook and Twitter throughout the month of March.
The NIH has made increasing transparency and rigor in science one of its top priorities. They define scientific rigor as, “the strict application of the scientific method to ensure robust and unbiased experimental design, methodology, analysis, interpretation and reporting of results. This includes full transparency in reporting experimental details so that others may reproduce and extend the findings. Investigators should apply the elements of rigor that are appropriate for their science.” (2) Simply put, this means that science should be written and presented in a way that is clear and unambiguous so that others may find similar results should they repeat the study themselves. However, the NIH definition is less than helpful in making recommendations to scientists and trainees as to what steps would make their science more transparent. Luckily, there are resources available.
In results reported in a 2016 survey, poor experimental design and flawed statistical analysis were the leading causes of irreproducibility. (1) Fortunately, most academic institutions now offer courses and seminars on these topics at no-cost or low-cost to science trainees. Many institutions also have resources such as statisticians that are available for consultation. Additionally, there are many resources available online and in-person, some of which are detailed in this Nature article (4) and an associated Nature blog article (5). There are also many resources available from societies such as the American Physiological Society, including slides and videos from past seminars and symposiums on improving scientific rigor. You can visit their toolbox for reproducibility here (6).
Finally, the APS Trainee Advisory Committee will be hosting a 3-session symposium at the 2018 Experimental Biology meeting in San Diego, CA. These three sessions, each held from 7:00-8:00 am in Room 25A of the San Diego Convention Center across 3 consecutive days (21-23 April) will feature 4 speakers who will speak directly to the trainees needs and roles in developing rigorous and reproducible science. The speakers will be discussing the role of trainees in scientific rigor, obtaining research funding, experimental design, publishing results, and much more. For more information about the symposium, you can visit the APS website here or on the EB app prior to the meeting. Here are the speakers that you can look forward to listening to during the 3 sessions:
Sunday, 21 April
- Enhancing the Value of Research Findings: Ongoing Activities at NIH and Beyond
Shai Silberberg, Ph.D.,NINDS/NIH
Monday, 22 April
- Building Bridges: Learning to Work Effectively with Regulatory Committees
Bill Yates, Ph.D., University of Pittsburgh School of Medicine
- Practical Applications of Rigor and Reproducibility in the Laboratory
Sean Stocker, Ph.D.,University of Pittsburgh School of Medicine
Tuesday, 23 April
- Publishing Reproducible Research: Ensuring that Editors, Reviewers, and Readers Have Confidence in your Findings
Kim Barrett, Ph.D.,University of California San Diego School of Medicine
The need to have rigorous and reproducible research is only going to increase. Trainees have the potential to play an important role in the way we publicly discuss science. While trainees may have to seek out and maybe even create some of the resources they need to develop the next generation of transparent science, resources are already available at their institutions and from sources like the APS. Hopefully we’ll see you at the Trainee Symposium at EB in San Diego this April!
||Ryan Downey, PhD is an Assistant Professor in the Department of Pharmacology & Physiology at Georgetown University. As part of those duties, he is the Associate Program Director for the Master of Science in Physiology and a Team Leader for the Special Master’s Program in Physiology. He teaches cardiovascular and neural physiology across several graduate level courses. He received his Ph.D. in Integrative Biology from UT Southwestern Medical Center. His research interests are in the sympathetic control of cardiovascular function during exercise and in improving science pedagogy. When he’s not working, he is a certified scuba instructor and participates in triathlons.
- Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533, 452–454.
- 2018 Trainee Symposium — “Do it Again: How to Achieve Rigorously Reproducible Research”. American Physiological Society. Accessed 26 February 2018. http://www.the-aps.org/rigorously-reproducible-research
- Frequently asked questions — Rigor and Transparency. National Institutes of Health Office of Extramural Research. Last revised: 1 February 2016. Accessed 26 February 2018. https://grants.nih.gov/reproducibility/faqs.htm#4828
- Baker, M. (2016). Reproducibility: Seek out stronger science. Nature, 537, 703–704.
- Seeking out stronger science: An incomplete, non-systematic list of resources. Naturejobs — Naturejobs blog. Last revised: 28 September 2016. Accessed 26 February 2018. http://blogs.nature.com/naturejobs/2016/09/28/training-resources-for-experimental-design-and-analysis/
- Reproducibility in Research. American Physiological Society. Accessed 26 February 2018. http://www.the-aps.org/mm/SciencePolicy/Agency-Policy/Reproducibility