Why So Many Retractions are Happening in the First Place
Not all retractions are caused by malicious intent. Retraction Watch organizes their data into different “reasons,” ranging from the uncommunicative author not responding to potential concerns, conflict of interest, contamination of samples, plagiarism, to manipulation of data and images as well other ethical violations.
Of all retractions in the database, honest mistakes are around 40%. In these cases, the scientist recognizes an error and retracts own paper. And though systemic changes would hopefully decrease the need for retraction, a quarter of the less honorable articles are written by just 2% of the authors. There were 22,763 total retractions at the time the data was obtained mid-November, and with sometimes multiple reasons leading to a retraction, there were 50,477 reasons for retraction in the data, divided among 98 categories.
A shocking amount of the papers retracted had no reason listed, with “little or no information” logged in its reason column at 16% of all listed reasons. This was then followed by “Investigation by Journal/Publisher” at 8.9%, “Withdrawal” at 4.7%,”Breach of Policy by Author” at 4.6% , and “Duplication of Article” at 4.2% rounding out the top five reasons.
“Fake Peer Review” made up 1.8%, or 903 counts, of all listed reasons. Similar specific reasons such as misconduct, hoaxes, plagiarism, fraud, falsification of data and others contributed to around one fifth of all instances a reason was cited. In a realm where truth and knowledge are the ideal goal, this is far from negligible, even actively working against it.
Notably there have been 75 retractions with their reason listed as “Doing the right thing.” Commonly listed with reasons alluding to errors made, this reason seems to be one of the more honorable retraction reasons, rather than someone “doing the right thing” after first doing the wrong thing.
We also see in the last 20 years that after a steep increase in retractions in 2010 and 2011, retractions have continued at a steady rate and will likely continue to. Reasons the rate may increase could be either due to better snuffing out of errors or an actual increase in these actions. Decrease in the rate could be due to the opposite, and similar to disease, it can be hard to know exactly how much it has permeated a population and over and underreporting has similar explanations.
The draw toward fraud and misconduct may be seductive, there are cases like Hwang which clearly demonstrate a draw toward money and fame, but they can also be desperate. Their livelihood and their families may depend on publications with a high impact; they may be required to publish research to continue to practice medicine or be considered for promotion as has been the case in China.
The Search for Truth, Honor and the Spirit of Science
A common desire for those going into science is a search for the truth of the people, world, and universe around us — to contribute to a greater understanding, some to a greater good.
This becomes problematic when some scientists value their Truth over the scientific process they need to take to arrive at the actual truth.
Political scientist Michael LaCour published some positive data — emotionally and statistically — on the impact of a gay person conducting a door to door survey about opinions on gay marriage compared to a straight person doing the same. He published this data in 2014 as a graduate student at the University of California Los Angeles, saying that meeting a person impacted by the issue made them more likely to support it. The findings were strong and clear.
And clearly shared a suspiciously similar distribution to the Cooperative Campaign Analysis Project (CCAP) survey data with which researchers David Brookman and Joshua Kalla were very familiar.
LaCour had apparently taken the older survey data and added random noise to the numbers and used that. He falsified the numbers and fabricated anecdotes, producing an elaborate fraud of a study. This clear need for control over his data got his paper retracted and lost him a shiny new job at Princeton.
After exploring many instances of fraud, Stuart Ritchie, author of “Science Fictions: Exposing Fraud, Bias, Negligence and Hype in Science,” concluded that people perform this scale of science misconduct to control what the studies said- whether that was to formulate a well-performing article like LaCour, or to “prove” some strongly held belief.
The impact of these falsifications and instances of misconduct is bad for science as an institution and the search for knowledge. In a tragic situation, the connection to a researcher who went on to commit major fraud led to the suicide of a great scientist and mentor.
In 2014 at the RIKEN Institute in Japan, more amazing stem cell research was being done. The RIKEN group had developed a revolutionary technique for producing pluripotent stem cells, which can develop into any tissue cell of the body, called stimulus triggered acquisition of pluripotency(STAP), and published in Nature. Lead Haruko Obokata quickly shot to Hwang’s levels of fame in Japan.
Shockingly fast, others noticed manipulated and duplicated images in the papers with miniscule changes such as just being flipped. And since STAP was such a simple procedure compared to earlier methods of pluripotency induction, replication here would be fast and simple. After everyone’s results came in negative, Obokata and those she worked with went to Nature to have it retracted. Few months later, Obokata resigned.
But that wasn’t the end of the direct impact her decisions had.
Feeling a responsibility for not having caught Obokata’s actions before they reached the media-heights they did, co-author Yoshiki Sasai committed suicide in the RIKEN building.
Unfortunately, once an article gains traction and coverage, some people are going to hold onto the “information” in them even once retracted. At its smallest impact, this will lead to a harmless urban legend, at its largest, it causes public health crises.
Lasting Impacts
As science shows, people like to experiment, manipulating variables sometimes just to learn what might happen. This is what happened in March 2020 where Matan Shelomi, entomology assistant professor at National Taiwan University, had an article titled “Cyllage City COVID-19 outbreak linked to Zubat consumption” published in the American Journal of Biomedical Science & Research. At first glance, perhaps the title sounds legitimate and the journal title seems like a respectable name- but Shelomi proved otherwise on both accounts.
In November 2020, Shelomi wrote an article for The Scientist, Opinion: Using Pokémon to Detect Scientific Misinformation, which highlighted how exactly they pulled off writing a fake article about a fictional creature in a fictional city without it being stopped at any point in the publishing process- all because of the route they took to publish in predatory journals. Shelomi says these journals “appear legitimate, but practice no peer review, no editing, not even a reality check,” and concludes “predatory journals are just expensive blogs, no more reliable as sources of scientific information than a celebrity’s Twitter feed, yet unfortunately equally trusted in some circles.”
As of November 1, the American Journal of Biomedical Science told Shelomi that the article would be removed, not because it was made up of blatantly false information, but because Shelomi hasn’t paid the publication fees that these kinds of predatory journals operate on.
So perhaps we can be thankful that one more piece of fake science about COVID-19 isn’t floating out there, but as it turns out, it has already been cited by other researchers who didn’t bother reading past the titles just like the predatory journal editors. Because of this, the article can live on, in the way that retracted articles sometimes do: unknowingly spread by those who may not know it had been marked as such.
False science lives on, marked as retracted to varying degrees of visibility, but remaining online in the form of references and citations made by unknowing readers and researchers. A recent analysis found that83% of citations of articles post-retraction were positive in nature, indicating a lack of recognition of the fact that it has been retracted.
Definitely most notable in the world of vaccines and an extreme example of information living far past its retraction date, Andrew Wakefield performed an experiment on a shockingly small sample of people which was influenced by wanting to produce a replacement MMR vaccine. Smearing the popular and common vaccine would make his own more likely to be successful. And now, even after widely retracted in 2010, arguments inspired by his research are used to argue against vaccination. In the UK, between 5 and 6 years after publishing, their MMR vaccination rates dropped from 91% (1997) to 80% (2003-2004). Fortunately around 2013, a positive trend began, showing an increase in vaccination rates, back to around 90%.
However, we’re still seeing measles and mumps outbreaks as a result of children not being vaccinated following Wakefield’s misguided research.
The impacts of scientific misconduct are deeply personal yet widely reaching. To get a better handle on the causes of fraud and harmful practices, and to prevent them through changes in policies and incentivization would mean a more trustworthy body of science. A closer vision to that of what science is meant to be in its “pure” and idealistic form.
Next Steps and Solutions
One worthwhile tool which has been happening for years now, is a sort of forensic sleuthing work to catch papers which fall under some form of misconduct. A household name in visual forensics in microbiology journals, Elisabeth Bik, the founder of Microbiome Digest and a scientific integrity consultant, is a force to be reckoned with in the effort to hold editors and scientists accountable.
Often, Bik’s work doesn’t result in swift and strong actions from editors who she’s notified. Whether a lab is protecting an author from an investigation, or an editor not feeling a sense of urgency, many papers with manipulated or duplicated images remain up in journals, but she continues the fight to hold these journals to a higher standard.
In addition to her constant visual forensics work in individual articles, she has helped uncover paper mills — where scientists can purchase ready-made papers to publish, often in an attempt to do the publishing portion of “publish or perish” or meet a certain quota to maintain a position.
While Bik and others work to uncover fraud, there are changes being proposed to the system itself which aim to lower the amount of scientific misconduct we’re seeing reflected in the Retraction Watch data.
A first priority is to reflect on past transgressions and learn what not to do. Lancet, after dealing with COVID-19 research retractions over questions of data quality and integrity. In response to this widely covered hydroxychloroquine retraction, The Lancet announced that more than just one author has access to the data and the ability to verify its authenticity. Data-sharing statements outlining what data is being released, as well as when, how, and to whom it will be shared to ensure transparency. Additionally, they’re making sure that more reviewers are data-literate and including more data scientists on review teams for large-data work. By being more intentional and transparent and explicit with concerns surrounding the quality of data, they hope to prevent the hydroxychloroquine situation from happening again.
The strongest and most positive changes will come with modifications to the peer review system. Logically, an immediate response could be to incentivize the important work in science and science publication which are currently cast aside, or not given the time and attention they deserve.
A study from 2013, published in the journal Research Policy, tested whether incentivization would increase or decrease quality of peer review. Though an incentive might lead more to agree to take part in being a reviewer, the study concluded that material incentive compromises the reviewer’s moral behavior and that without material incentive, interactions between editors, reviewers, and authors were judged as more cooperative and trustworthy.
Controlling for hostile review is something which needs to be done for the sake of quality science. A study published in the journal Accountability in Research earlier this year shows clear, cooperative and professional review and conversation will lead to higher quality and more relevant science to a given journal. Further, the opposite has been shown to harm diversity in the form of unprofessional comments based on competition or any minority identifier.
Specific techniques to prevent this discouraging and unprofessional hostile reviewing include prepping both submitting authors and reviewers with the goals and mission statement of the journal, to make sure it is first relevant and then focus the reviewers thoughts on what matters in terms of science and what they want to publish. This is a form of action easily taken to ensure proper professional behavior as well as better science.
A more novel review modification is one which Cell Press has decided to begin to use: parallel review. These are the publishers responsible for journals such as Cell, EBioMedicine, and Trends in Pharmacological Sciences. Simply put, an author can submit an article to be considered for any or all of Cell Press’ publications. After this point, a community editor determines if the article matches well with the potential journals they indicated and then the normal review process begins before the author(s) get a letter outlining possible decisions they can make, for example: revising and resubmitting if wanting to be in journal A, accepted to journal B, etc.
The appeal of parallel, also called community, review is that authors can more easily and more quickly be considered for publication in more places. Traditionally this process will happen with submissions one at a time, not overlapping, taking a very long amount of time. Perhaps by adopting this type of review at more publications would take stress off of multiple actors in this situation and allow for researchers to not rush through experiments and make sure their work doesn’t have errors which may lead to retraction.
The final peer review change suggested to increase quality of research and decrease the effects of a bias for positive results involves registered reports. The Retraction Watch data shows this bias for experiments which prove something did happen (positive results) can very easily and logically lead to researchers falsifying or fabricating data.
Registered reports involve peer review before the research is complete. This method puts more value into the rigor of the questions being asked and the experimental design being proposed. A paper can be conditionally accepted to a journal, and then as long as it was completed as outlined and there aren’t any errors in the work, it will be published once complete. This conditional acceptance has been shown to greatly increase publication of negative results which are just as important as positive results.
The Retraction Watch Database gives us a lot of insight as to the whos, whats, whens, wheres, and whys of retractions, a proxy of what is going wrong in science. It can be easy to see the individual bad stories as anomalies, especially with the percentage of it all being so small. But these instances point to problems that can be solved with systemic approaches to alter peer review to be its most effective. All hope is not lost; with these and other solutions we may not have thought of yet, science may be showing some cracks — but it’s not beyond repair.
