Think about Research Transparency now

Transparency is essential to trust and efficiency in research.

In light of SARS-CoV-2 and COVID-19, there has been increasing discussion about the importance of past and evolving research related to the virus and disease. Ensuring researchers have access to past data and findings that may provide insights is urgent and important, and fortunately is getting some attention with efforts like LitCovid, an open-resource literature hub for COVID scholarship. Despite this encouraging development, it is important that we don’t lose sight of the importance of research transparency in reinforcing the goals of efficiency, efficacy, and long-term trust in biomedical research. Increasing research transparency achieves these goals by

• Decreasing inefficiencies in funding research, and
• Surfacing hidden findings.

Some background

When scholarly research is done, it generally goes through some specific steps at a high level: 1) a research idea is proposed for funding, 2) once support is awarded, the project is conducted as described in the proposal, and 3) once results are found, the outcomes are reviewed by peers and shared more broadly. Often these steps are not really visible to others until outcomes are shared, usually in the form of a scholarly journal article. Though even then, not all can read the articles because the journals require a subscription, a privilege afforded to relatively few. And even when articles can be read, there can be challenges in fully benefiting from the valuable research that has been done.

You may wonder why this matters. After all, one may expect that other researchers generally can access this research and use it to advance our collective body of knowledge. This is not always the case, however, for several varied and sometimes complicated reasons. And even when others can read the research results, there are other challenges that limit the benefits. These issues are particularly concerning because research is often funded by public money obtained through taxes and other government revenue. Today I’ll just focus on one part of this puzzle, the question of research transparency.

Decreasing inefficiencies in research

One goal of research transparency is to decrease inefficiencies in research. The most expensive type of inefficiency is when similar research is funded and/or performed several times without knowing that the same or a similar study has already been done. A related situation is when a researcher knows that a study has been done, but is unable to get access to or use the data that supports the findings. At times it is difficult to contact researchers that have conducted studies because they are now doing research at a different location, or because they are in a region where contact is difficult. And, finding or accessing the data that was used for research can be difficult, or even impossible even when the researcher can be found; it is expensive to maintain raw research data, and the incentives for doing so are generally not well aligned with how researchers generally are assessed and rewarded.

There are several ideas for mitigating some of these challenges. They include things like

• Have funders share more information about what they are funding to help avoid unintended duplication in studies.
• Require and financially support data management plans, making it easier for researchers to maintain data that they collect during studies.

Surfacing hidden findings

Given the importance of both positive and negative evidence when making logical proofs, I find it curious that in research publishing is that it is very uncommon to see an article that highlights evidence that disproves what a researcher was expecting. From reading this literature, one might be tempted to come to the conclusion that researchers have a hypothesis about what will address a condition or a situation, and it is just a matter of conducting the study to prove what the researcher thought. Of course, this is not how things always go! Though, one generally doesn’t see an article that says, “I thought this was going to happen, but when I tested it, I was wrong.” Often these types are results are called “negative results.” What is surprising to me is how often important journals that are rejecting these articles, so researchers have challenges getting the results published if they wanted to. Imagine if there was a finding that a particular treatment did not provide benefits in treating COVID-19, but this finding was not included in the scholarly body of knowledge because it represents something that doesn’t work.

If these “negative results” are not published, it can add to the conditions outlined above that cause inefficiencies in research. After all, if you didn’t know that a study was funded and conducted, and then the results from the study are never published because a negative result was found, how is one going to know that they shouldn’t fund a similar study in the future?

Some ideas for mitigating these challenges include things like

• Change the language that we use to describe these types of results from “negative results” to “null findings.”
• Run experiments to explore the true interest in these types of publications. CMBRT did such an experiment in collaboration with Neurology Journal, the most widely read and highly cited peer-reviewed neurology journal. With this “Null Hypothesis” issue, they found that the articles had similar citations by other researchers and mentions in the press as those with positive results.
• Obtain pre-study publication commitments based on the study design rather than the results - if the study sounds interesting, a publisher will agree to publish the results regardless of the outcomes.

While all of these ideas are theoretically possible to put in place, it turns out that they are quite hard to do in practice. Most solutions are systemic in nature - it takes many different players, processes, and efforts working in tandem to make them a reality, and even then, the impact of such a change likely will not be felt for some time. As a result, it will take a level of tenacity and vision to see through the necessary adjustments. Though, through collaboration, this action can position research and scholars to be even more strongly prepared to address the next medical challenge.

About the BioMedical Transparency Summit

The thoughts and observations for this post were developed in January 2020, when I attended the BioMedical Transparency Summit (#BMTS20) hosted by the Center for Biomedical Research Transparency (@CBMRT_org). Held at the beautiful National Academy of Sciences in Washington, DC, this event included a who’s who of people who are working to reinforce efficiency, efficacy, and trust in BioMedical Research.