Publication Bias In Psychology: What It Is And Why It Causes Problems
Psychology, specifically its research side, has been in crisis for a few years, which does not help its credibility at all. The problem is not only found in the problems when replicating classic experiments, but also when publishing new articles.
The big problem is that there seems to be a prominent publication bias in psychology That is, it seems that the publication of articles is based more on aspects such as how interesting they may seem to the general public rather than the results and scientifically relevant information they offer to the world.
Today we are going to try to understand how serious the problem is, what it implies, how this conclusion was reached and if it is something exclusive to the behavioral sciences or there are others that are also at the same crossroads.
In recent years, various psychology researchers have warned about the lack of replication studies within the field, which has suggested the possibility that there were a publication bias in the behavioral sciences. Although this was something that was coming, it was not until the end of the 2000s and the beginning of the following decade that there was evidence that psychological research had problems, which could mean the loss of valuable information for the advancement of this great , although precarious, science.
One of the first suspicions of the problem was what happened with Daryl Bem’s 2011 experiment. The experiment itself was simple:
It consisted of a sample made up of volunteers who were shown 48 words. They were then asked to write down as many words as they could remember. Once this was done, they had a practice session, in which they were given a subset of those 48 previously shown words and asked to write them down. The initial hypothesis was that some participants would remember better those words that they were then made to practice.
Following the publication of this work, three other research teams, separately, attempted to replicate the results seen in Bem’s work. Although they essentially followed the same procedure as the original work, they did not obtain similar results. This, although it would allow some conclusions to be drawn, was reason enough for the three research groups to have serious problems in publishing their results.
Firstly, since it is a replica of a previous work, It gave the feeling that scientific journals were interested in something new, original, not a “mere copy” of something previous. Added to this was the fact that the results of these three new experiments, not being positive, were seen more as studies that were methodologically poorly done and that this would explain the obtaining of bad results rather than thinking that, perhaps, the new data represented a new advance for science.
In psychology, studies that confirm their hypotheses and, therefore, obtain more or less clear positive results, seem to end up behaving like rumors. They are easily disseminated by the community, sometimes without even consulting the original source from which they come or without carefully reflecting on the conclusions and discussions made by the author himself or by critics of that work.
When attempts to replicate previous studies that had positive results fail, these replications systematically remain unpublished. This means that, despite having carried out an experiment that confirms that a classic one was not replicable for any reason or reason, as it is not of interest to the magazines, the authors themselves avoid publishing it, and in this way it is not recorded in the literature. This means that what is technically a myth continues to be spread as a scientific fact.
On the other hand, there are the habits ingrained by the research community, ways of proceeding that are quite criticizable although they are so generalized that a blind eye is often turned: modifying experimental designs in a way that guarantees positive results, deciding the size of the sample after checking whether the results are significant, select previous studies that confirm the hypothesis of the current study, omitting or ignoring, casually, those that refute it.
Although the behaviors that we have just exposed are criticizable but, as far as possible, understandable (although not necessarily tolerable), there are cases of manipulation of the study data to ensure that they end up being published so that one can speak openly of fraud and a total lack of scruples and professional ethics.
One of the most savagely embarrassing cases in the history of psychology is the case of Diederik Stapel whose fraud is considered to be of biblical proportions: he came to invent all the data of some of his experiments, that is, speaking clearly, like someone writing a fiction novel, this man invented research.
This not only implies a lack of scruples and a scientific ethic that is conspicuous by its absence, but also the total lack of empathy towards those who used their data in subsequent research, causing these studies to have a fictitious component to a greater or lesser extent.
Studies that have highlighted this bias
In 2014, Kühberger, Fritz and Scherndl analyzed nearly 1,000 articles published in psychology since 2007, selected at random. The analysis overwhelmingly revealed evident publication bias in the field of behavioral science.
According to these researchers, theoretically, the effect size and the number of people participating in the studies should be independent of each other; however, their analysis revealed that there is a strong negative correlation between these two variables based on the selected studies. This means that studies with smaller samples have larger effect sizes than studies with larger samples.
In that same analysis it was also shown that The number of studies published with positive results was greater than the number of studies with negative results, the proportion being approximately 3:1. This indicates that it is the statistical significance of the results that determines whether the study will be published rather than whether it really represents any type of benefit for science.
But apparently it is not only psychology that suffers from this type of bias towards positive results. In fact, It could be said that this is a widespread phenomenon in all sciences, although psychology and psychiatry would be the most likely to report positive results, leaving aside studies with negative or moderate results. These data have been observed through a review carried out by sociologist Daniele Fanelli from the University of Edinburgh. He reviewed nearly 4,600 studies and saw that between 1990 and 2007, the proportion of positive results rose by more than 22%.
There is a mistaken belief that a negative replica invalidates the original result. That a research has carried out the same experimental procedure with different results does not mean that the new research is methodologically poorly done nor that the results of the original work have been exaggerated. There are many reasons and factors that can cause the results to not be the same, and all of them allow us to have a better knowledge of reality, which, ultimately, is the objective of any science.
The new replicas should not be seen as a harsh criticism of the original works, nor as a simple “copy and paste” of an original work only with a different sample. It is thanks to these replicas that there is a greater understanding of a previously investigated phenomenon, and it allows us to find conditions in which the phenomenon is not replicated or does not occur in the same way. When the factors that determine the appearance or not of the phenomenon are understood, better theories can be developed.
Prevent publication bias
Solving the situation in which psychology and the sciences in general find themselves is difficult, but this does not necessarily mean that the bias has to worsen or become chronic. So that all useful data can be shared with the scientific community requires the effort of all researchers and greater tolerance on the part of journals towards studies with negative results, some authors have proposed a series of measures that could contribute to ending the situation.
