Reviewers are more likely to approve a manuscript if their own work is cited in subsequent versions than reviewers who are not cited, according to an analysis of 18,400 articles from four open access publications. The study, which is yet to be peer reviewed, was posted online as a preprint earlier this month.
The study was inspired by anecdotes from authors who cited articles only because reviewers asked them to, says study author Adrian Barnett, who researches peer review and metaresearch at Queensland University of Technology in Brisbane, Australia. Sometimes, these requests are fine, he says. But if reviewers ask for too many citations or the reason to cite their work is not justified, the peer-review process can become transactional, says Barnett. Citations increase a researcher’s h-index, a metric reflecting the impact of their publications.
These are the most-cited research papers of all time
Requesting unnecessary or unjustified requests for citations, sometimes called coercive citation, is generally considered poor practice. Balazs Aczel, a psychologist who studies metascience at Eötvös Loránd University in Budapest, says that the latest work isn’t the first to investigate reviewers asking for citations, but that the number of peer reviews included and level of analysis is novel. A barrier to studying the practice is a lack of data sharing from publishers, he says.
Approve, reject, reservations
The preprint considered articles from four publishing platforms — F1000Research, Wellcome Open Research, Gates Open Research and Open Research Europe — that make all versions of their articles publicly available, as well as reviewer comments. The publishers ask reviewers to approve articles, reject or approve them with reservations. Reviewers are also asked to explain why they ask authors to cite their own work. Of 37,000 reviews — at least two people reviewed each article — 54% of reviewers approved articles with no changes and rejected 8%. Almost 5,000 reviewed articles cited a reviewer and roughly 2,300 reviews requested a citation from a reviewer.
The analysis found that reviewers who were cited were more likely to approve the article after the first review than reviewers who were not cited.
But reviewers who suggested that their own research be cited were about half as likely to approve the article than reject it or express reservations. In more than 400 reviews in which the reviewer was not cited in version 1 of the article and requested a citation in their review, 92% of reviewers who were cited in version 2 recommended approval compared with 76% for reviewers who were not cited.
When a reviewer rejects a paper, they and the authors know that the reviewer is probably going to evaluate any revised versions of the article, says Barnett, so authors might opt for the path of least resistance and include the citation to get their paper accepted.
Reviewer comments
Barnett also analysed 2,700 reviewer comments and identified the 100 most frequently used words. He found that reviewers who requested citation were more likely to use words such as ‘need’ or ‘please’ in their comments when they rejected an article, which he says suggests that coercive language was used.
Jan Feld, a metascience researcher at the Victoria University of Wellington, New Zealand, is not convinced that such language is a sign of coercion. “That seems like a bit of a stretch,” he says. There are other explanations for reviewers rejecting an article than the author refusing to cite their work. He doesn’t doubt that reviewers request citations that are not warranted, but they can recommend citations, including of their own work, to address issues they’ve identified. But even after those recommendations, “if the paper has not improved or I still have concerns, I cannot recommend publication”, he adds.