Should-I-publish-negative-results-or-does-this-ruin-my-career-in-science-featured-image-with-url

Should I publish negative results, or does this ruin my career?

Scientists often produce negative results. All experiments were done correctly – but there was no difference between test and control. They get conflicting advice from supervisors and ethicists. Some say that publishing negative results is a waste of resources and ruins their scientific careers. Others say that ‘not publishing negative results is unethical’ and promotes the reproducibility crisis. What should young scientists do in such a situation?

Negative results are frustrating and demotivating

Sad person representing the bad feelings of getting negative results

In my experience as a supervisor, the biggest problem with negative findings is the demotivating effect on young scientists.

During a typical PhD, most early-stage researchers experience an emotional dip anyway – after approximately two years.  

This is half the duration of a standard PhD contract. It is a recurring pattern I have seen often.

The emotional dip is substantially worsened when the students are ambitious and work particularly hard to accomplish a great project, but do not obtain statistically significant results.

Stopping a long-term project may hurt and/or relieve

Suppose a costly project does not deliver promising results. In that case, the supervisor may conclude that the project has to be abandoned without publishing the data and alter the research plans accordingly.

In many cases, such a decision will push the young researchers into a vortex of dramatic feelings. They may perceive themselves as a failure and feel mistreated.

In addition, they are probably worried about their future. On the other hand, it may also relieve the student because they may have felt for a long time that their project is going nowhere.

These mixed feelings do not make the situation easier. The supervisor is responsible for developing a new project and emotionally supporting the PhD student in this challenging situation.

“Publishing negative results is a waste of resources”

Many scientists believe that following up on and publishing negative results wastes resources of their research teams.

Indeed, this statement has some truth because the costs and research effort in a particular study may be high compared to the final output parameters (such as the impact factor of the journal, citations, etc.) reached at the end.

Negative results are often published with a low impact factor because editors hate boring data

Publishing is a big business – in particular, open-access publishing. 

Dollar sign representing that publishing is a big business

Editors play a pivotal role in the perception of the value of negative results. To increase the visibility of their journals, editors want the exciting stuff: new mechanisms, unexpected findings, and dramatic effects (“the paralyzed could walk again”), which increase citations, clicks, shares, and press coverage.

Unfortunately, negative results are often very boring. As a result, it is often difficult to publish negative results in an appropriate journal.

After several rejections, it ends up in a journal with a much lower impact factor than expected at the beginning of the experiments.

As a result, negative studies are considered less prestigious by many scientists.

Several peer-reviewed journals specifically focus on negative findings (see below). Unsurpringly, their impact factors and citation scores are rather low.

You might falsely hope that open access platforms  or OA journals might be more open to well-performed studies with negative results. In my experience, the editors of OA journals are as selective with negative studies as all the other scientific journals.

Unexpected negative data can be very interesting – but are often costly

High-impact journals may be interested in negative studies when they destroy a long-held paradigm or when a new method is used to show that most previous studies are flawed.

Following up on a negative story is always considered a risk because the research group may invest substantial resources (time, money, energy) without improving the quality of the paper and the final impact factor.

Reviewers may ask for multiple additional controls to ensure that the negative results are not just an effect of technical mistakes. Therefore, the opportunity costs are high (“Doing this study means you are not following up on potentially more promising data”).

“Publishing negative results ruins your career in science”

The words "your future" in a toilet representing a ruined scientific career

Many supervisors are convinced that publishing negative results will ruin the career of their PhD students as well as their own. They will spend a lot of resources on the wrong project, publish with a low impact factor, and consequently get less future funding.

Young scientists may feel that investing in a negative study will dramatically reduce their chances on the academic job market and may even force them to give up and pursue a different career.

You may correctly argue that high-impact factor publications are not absolutely necessary to become a professor and might not necessarily increase job opportunities in the industry or public sector.

Read more here:  Do I need Nature or Science papers for a successful career in science?

However, impact factors are still broadly used to evaluate the performance of single scientists, departments, and institutions.

Read more here: 

10 simple strategies to increase the impact factor of your publication)

Which bibliometric data are relevant for a research career?

Thus, a responsible supervisor will always aim for journals with a high impact factor and will tend to abandon projects without exciting results. Unfortunately, this behavior – as understandable as it is – may be one of the biggest problems in science.

“Not publishing negative results is unethical”

Many colleagues do not find abandoning projects without promising results a big deal.

Crossed fingers representing unethical handling of negative findings

“Fail faster” is a common strategy that means screening for dramatic effects (for example, a treatment, a drug, a genetic intervention, etc.) and leaving the less dramatic results untouched.

The big problem is that this knowledge is lost because all these experiments disappear, and many other scientists may repeat the same or a similar experiment because these results are not documented or publicly available.

In these instances, accessing research data from similar experiments that were never published could save much time and financial resources.

The scientific literature is awash with positive findings, which naturally paints a skewed perspective on the existing scientific knowledge. The publication of null results or negative data would add much-needed balance and also provide valuable information for systematic reviews.

As a consequence, a lot of time, effort, and taxpayer money is wasted due to unnecessary repetitions because negative or less-than-dramatic findings are unreported.

If you care for scientific progress, publication of negative results is a must. Negative studies may challenge existing paradigms and enhance progress by stopping further investment in scientifically barren topics, decreasing animal use in experiments, and focusing research in more fruitful areas (Boorman et al., 2015).

Publication bias and the reproducibility crisis

More than 70% of researchers in a Nature’s survey of 1 576 researchers have tried and failed to reproduce another scientist’s experiments (Baker et al., 2016). This so-called reproducibility crisis has many reasons (see reviews in Jarvis & Williams, 2016Begley & Ioannidis, 2015).

One important reason for the lack of reproducibility problems is positive-results bias, a particular form of publication biasPositive-results bias is just a fancy term for the tendency described above: when authors are more likely to submit, or editors to accept, positive results than negative or inconclusive results (Sacket et al., 1979). 

Briefly, publishing positive results only and filing away the negative findings produces a skewed view of reality, results in unnecessary repeats of experiments already done, and wastes a lot of taxpayer and industry money. It may lead to detrimental therapies and many frustrated scientists, among other unwanted outcomes. 

Reproducibility issues may even contribute to a growing skepticism regarding the integrity and relevance of all biomedical research (Jarvis & Williams, 2016).

It is not the task of young scientists to publish negative results.

After many years of struggling with this question, I came to the conclusion that it is *not* the task of young scientists to publish negative research outcomes – in the current system.

They still have to pursue their career, and – as explained above – publishing negative results may negatively affect their careers.

Controversial advice: Leave it to the old guys!

Old scientist representing the duty of senior scientists to publish negative findings
@sven hendrix / midjourney

The arguments listed above bring every scientist to a problematic situation.

Young scientists seem to have to choose between their careers and ethical behavior. Supervisors of research groups must give good advice, otherwise they leave the young scientists alone with their decisions.

However, there are no widely accepted and widely known procedures to handle negative outcomes that are not used for publication (see some suggestions below). Therefore, currently, the best advice is:

Do not publish negative results as a young scientist.

Leave it to the senior scientists who already have successful careers and can afford to publish negative findings for the sake of good science!

It is essential to note that I do *not* suggest selective reporting.

Selective reporting is a special case of reporting bias, resulting in the incomplete publication of analyses performed in a study, leading to the over- or underestimation of treatment effects or harms.

Selective reporting is scientific misconduct. In contrast, I advise young scientists not to waste their time, grant money, and energy on studies with negative results.

In the meantime, let us work on better procedures and rules to avoid “scientific waste” and substantially reduce unnecessary repeats of the same unpublished experiments.

How do you behave ethically and handle negative results correctly?

My advice to leave the publication of negative results to the senior scientists (who already have a successful career) comes with a price. We accept the publication bias for the sake of young scientist’s careers. This promotes the reproducibility crisis.

Senior scientists cannot be trusted to take over their responsibility to publish negative results because they are not rewarded. Even worse, they may get punished by their institutions and funding agencies, which are strongly focused on scientific output, mostly quantified in impact factors, citations, and grant money.

In other words,  the academic community gets what it incentivizes. There are many ideas to improve the current science system (for example, Begley & Ioannidis, 2015), but most initiatives are still at an embryonic stage.

To my knowledge, there are no internationally accepted rules on handling negative results. A non-representative survey among 10 young professors revealed that only one had a rough idea of where to store unpublished data that may interest other scientists. Below, you will find three suggestions on how to handle the problem of negative results at the level of the scientific community:

Three strategies to improve science

The scientific community, led by international organizations like the World Health Organization and the International Committee of Medical Journal Editors, scientific societies, and national institutes of health, should emphasize the importance of negative results in fostering actual scientific progress.

There should be international guidelines for publishing negative results, emphasizing their value and importance in scientific research.

Here are three strategies to improve the current practice in the near future:

1. Registration of all studies *before* they start

Registration of clinical trials is a widely recognized tool for facilitating complete public reporting (Zarin & Tse, 2008Williams et al, 2010) and to counter the conflicts of interests, for example, of pharmaceutical companies. 

Registration of any type of study (pre-clinical, observational, etc.) would dramatically increase the administrative load for researchers, institutions, and funders. However, it seems to be a necessary step to cure the entire research community of positive-results bias.

For clinical trials, there is a list of international registries in the Cochrane handbookSome of these registries already allow the registration of non-clinical studies (observational, pre-clinical, etc.). 

For social sciences and other types of research, the Open Science Framework (https://osf.io) offers the possibility to pre-register studies.

2. Saving inconclusive data in publicly accessible repositories to make them available to other scientists

Since publishing inconclusive data may be tedious and considered “a waste of resources” (see above), it should be effortless and a standard procedure to save the data in publicly accessible repositories. 

To guarantee independence, the international scientific community should finance these through scientific societies and/or national and international funders.

Some repositories already exist – see here a list of repositories on the Nature website.

Another example is arxiv.org,  a repository for documents and papers rather than data, including unaccepted or unsubmitted manuscripts, PowerPoint presentations, etc. Unfortunately, repositories are often costly, and researchers are not incentivized to use them.

3. Funders must oblige scientists to pre-register their studies and to make all data available (for example, in a publicly accessible repository or a journal of the funder)

Finally, funders should take responsibility for providing a standardized procedure to pre-register all funded studies and oblige the researchers to publish all their negative findings or deposit them in a public repository. Some funders have already started their own academic journals to publish these negative findings.

A possible incentive may be to freeze the last tranche of the funds (e.g., 25%) until data are made publicly available. Without a doubt, there must be very flexible regulations because some studies are published many years after the funding period has finished.

Final advice

The final advice is simple:

As a young researcher:

  • Do *not* publish negative results. Leave it to the senior scientists!

As a senior scientist:

  • Take responsibility and publish negative findings for the sake of science, even with a lower impact factor, but do not force the young scientists to do this.

All scientists:

  • Make negative findings known in reviews (which can be published with a high impact factor) and scientific talks.
  • Push your institution, your scientific society, and/or your funders to provide a public repository for negative results and to make it easy to make negative results publicly available without the hassle of a full publication.
  • Save your negative results or inconclusive data in a public repository to make them available.
  • Help to improve the system to incentivize the publication of negative findings and replication studies – for example, talk to funders or higher education and health politicians – many listen to scientists to improve the current procedures.

Please comment below if you have any other ideas to improve the current system.

Frequently asked questions (FAQ)

How to publish negative results

There is no magic bullet. In my personal opinion, you have to find a journal with the best fitting scope and target audience for your study. You have to convince the editor that the journal’s readers will profit from learning about these negative results. During the review process, you have to convince the editor and the reviewers that this article will prevent other scientists from perfoming useless experiments.

Are there journals that specifically publish negative research results?

Yes, some of the most notable journals that publish negative research results include:

Acknowledgments

Thanks to Noémie Aubert Bonn, who provided the information about registries and repositories and helpful comments. I have used AI systems, including Grammarly, Google Bard, and ChatGPT, to enhance the English and comprehensiveness of this article. This post may contain affiliate links, meaning I get a small commission if you decide to purchase through my link. Thus, you support smartsciencecareer at no cost to you!

Recommended reading

The following articles may also interest you:

Similar Posts

15 Comments

  1. Negative results may be just as valuable as confirmatory results or novel results. You should try to publish what you have and get as many publications as you can in order to keep your career going forward. In the future no one will know, care or remember if the results were negative or positive. The problem may be getting a high profile journal to accept a negative results paper. That depends totally on the accidental choice of referee. The chances are high that no one will notice. I have a hard time believing any of these correspondents have ever written anything.

  2. From a philosophical standpoint I don’t agree that negative data is worth much… It just proves that you cannot replicate previous observations and there could be many reasons for this (you suck at your work, hidden parameters, etc.). However I also agree that positive findings are only truly worth something when they can be reproduced by others and when there exist a common agreement on their true objectivity. Most science today deals with problems that are immensely difficult to measure and capture. One very real fear would then be that the big groups with lots of know-how and equipment monopolizes on this “true objectivity”. However still, negative results just shows that you cannot do what others apparently have done.

    1. It’s not about negative data is worth much or not in comparison with positive results. It’s about whether it’s worth or not. Documenting the experiment helps the researcher to improve and obtain feedback for another researcher. It may not benefit the public, but it benefits the experimenter as they will learn from unintended mistakes. Science goes ahead because of the experimenter.

      1. I agree. However, when giving advice to young scientists it is important to consider also the consequences for their careers. Therefore, I think it is unfortunately a much more complex discussion.

  3. Your final recommendations in the conclusions about publishing (or not) negative results is spot on, particularly the advice about saving negative results institutionally in a repository so there are available.

  4. Please, my sides hurt so much from laughing. Get a few years under your belt in this industry and you will see how much money is made by pushing absolute BS!

    It is, without question, the main reason for pharma’s R&D collapse, the people with the money got wise. Pump and dumps can only carry you so far.

  5. I totally disagree with you. Negative results must be published (and need to be encouraged if they are found) more than positive results and let me explain you why.

    1. Currently we have a reproducibility crisis as you said and you encourage young students to not publish their results if they are negative.. what the hell??

    2. Do you know what a meta-analysis is? Negative results are particularly neccesary to them so without them the meta-analysis would be flawed.

    3. The results of a particular study doesn’t make it good or bad, but the questions, the design and the statistical analysis are the ones that make a study worthy or not.

    4. Negative results together with positive results show which path must be followed in the future research. If you only see the positive results you have a huge bias.

    1. Hi,
      thanks a lot for your comment – in an ideal world I would agree with you – but under the current circumstances I can not advise “early stage” (= young) scientists to publish negative results because their career will suffer. I am sincerely convinced that it is the job of the old and established scientists. This is the fine distinction I make here. I have – against my own advice – published a few negative studies and I am unsuccessfully trying to publish one right now – and everything I state in this post is confirmed again and again.

  6. Sometimes posting negative results may lead to a positive outcome like others will that you are being honest and trustworthy. Also, make these negative results as your motivation and as a part of your success. Great post!

  7. Great article for encouraging and advising young researchers. Publishing negative data is definitely relevant but a real hellish journey for the (young) researcher who has to do this.

  8. Just happen to see this post pass by. I recently published negative findings (on negative selection, oh irony) in Nature Genetics (https://www.nature.com/articles/s41588-019-0532-6), so negative results should not necessarily imply a publication in a low impact journal at all. If you’re really convinced about the importance of your findings, go for it!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.