How do I peer-review a scientific article?—a personal perspective

Introduction

The contribution of peer-reviewers is invaluable in scholarly publishing, science and medicine. Peer-review, also known as also known as “refereeing”, is a hallmark of the vast majority of scientific journals and represents the cornerstone for assessing the quality of potential scientific publications, since it is aimed to identify drawbacks or inaccuracies that may flaw the outcome or the presentation of scientific research (1). This voluntary and usually free activity is especially vital for biomedical sciences, because the publication of biased or incorrect information may seriously jeopardize patient safety, thus guiding the clinical decision making towards inappropriate diagnostic or therapeutic actions (2).

On the other hand, the activity of refereeing scientific articles may also be of value for the reviewer, for a variety of reasons including knowledge improvement on specific topics due to the possibility of reading articles before the information is published, may give valuable ideas for future studies on the same or other topics, may help improving you own writing skill, and is also a meaningful activity that can be included in the scientific curriculum. Although some generic rules for performing an accurate peer-review have been identified by many scientific journals, evidence exists that this activity not always ensures the quality of published biomedical research (3). Therefore, the purpose of this article is to present a personal and arbitrary perspective, accumulated after a 25-year experience (4), on how a scientific article should be peer-reviewed.

Limit peer-review to topics in line with your expertise

Throughout my career, I have (hopefully) accumulated a good background in the fields of clinical biochemistry, laboratory medicine and hemostasis. Therefore, my peer-review activity is actually limited to these specific areas of science and medicine. Nevertheless, during the past 3 months I have been repeatedly invited to peer-review scientific articles dealing with social sciences, astrophysics, thermal engineering, plant biology, fishery and even about worldwide economy. Besides highlighting that the credibility of these journals is probably null, the editors have a large liability since their activity can be defined as a clear misconduct when randomly assigning manuscripts to referees with no expertise on the topic. I have obviously declined to peer-review these articles and I really hope that other colleagues, whose background is also quite different from the topic of the articles, have also done so. I have brought this simple but paradigmatic example just for emphasizing that competency is the very first aspect that should guide the decision to accept or decline an invitation to peer-review a scientific article. Therefore, whenever you feel that the manuscript falls outside your competence or knowledge, you are ethically obligated to decline peer-review. As also endorsed by the Council of Science Editors (CSE) (5), peer-reviewers do not actually need to have an expertise covering all the different aspects of the article, but the assignment should only be accepted when the expertise is enough for providing authoritative assessment.

Check potential conflicts of interest

Although some journals mandatorily ask the reviewers to disclose potential conflicts of interest with the article or with its authors, this is not routine practice. Nevertheless, even if this is not clearly entailed by the journal, you should be fair enough to check potential conflicts of interest on your own before accepting the assignment. Conflict of interest disclosure is a broad enterprise, which can be actually summarized as the existence of interests that may impair your objectivity, and should hence lead to mandatory declining peer-review when (I) a direct relationship (personal or professional) exists with the authors, thus preventing positive bias in referee’s comments; (II) you have a negative opinion on, or you had previous disagreements with, the authors, which may then induce a negative bias in your peer-review; (III) the referee is engaged in similar or overlapping studies, so that there may be a propensity to (even unconsciously) underrate the outcome; (IV) there is a commercial relationship with companies whose drugs, devices or reagents have been tested or used in the study. I will never tell the source, even under torture, but time ago I was asked (for an error of the editorial office, hopefully) to peer-review one article that I had authored. This is, clearly, the greatest possible conflict of interest.

Personal beliefs diverging from the topic of the article may also be seen as potential conflicts of interest when the referee may not be able to keep them within an acceptable level of “interference”. As earlier discussed, usually referees are not paid for peer-reviewing articles and thereby there is no obligation to accept the assignment. Do not expect that the Editor will better treat your future submissions just because you have peer-reviewed some articles. This is totally unreasonable.

Check your availability and time

One of the worst aspects in scientific publishing is submitting an article to a peer-review journal and then waiting ages to receive the comments of the referees. This is frustrating, but may also have a dramatic impact on the chance of publishing the research. Original articles, whilst focusing on very innovative topics, may become old or even obsolete in few months, even in few weeks. The referee should hence always consider this aspect when accepting the assignment, since it is unfair to keep the article under revision for months, and it is even more unfair when the referee deliberately does so for delaying the publication of the article (see previous paragraph). When the referee finally submits the recommendations, many articles on the same topic may have been published by different authors. Whenever I accept to peer-review an article, my deadline never exceeds 3 to 5 days, whilst whenever I expect that I could not be able to peer-review the article within one week, I prefer to decline the assignment. Although the deadline for refereeing articles is quite heterogeneous among the various scientific journals (i.e., from 1 to 4 weeks), once you have established that the deadline fits your ongoing (or future) activities, then you must honour the commitment you made. Do not accept to peer-review an article if you are just about to leave for your holidays or you are not planning to work for quite a long time. The decision to accept or decline an assignment will also be influenced by the size and complexity of the article. You should hence consider that it may take quite a different time (and effort) to peer-review a short letter to the editor or a large meta-analysis. Importantly, impersonation or involvement of other scientists during the peer-review activity shall be seen as a severe misconduct.

Identify the innovative value of the article

Once you have finally accepted the assignment, checking how much the specific topic has been investigated in the recent scientific literature and whether or not the argument fit the scope of the journal are advisable practices. When the referee has a very good knowledge about the topic, then there is no need to search information elsewhere. However, when the referee is not completely familiar with the topic, or else there are some innovative aspects that are partially obscure, it is advisable to verify the volume and type of previously available information using reliable sources. This can be easily done by accessing some biomedical platforms such as PubMed, Google Scholar, Scopus and Web of Science (6), by entering the keywords used by the authors or representative terms captured from the title or the abstract of the manuscript. The first two biomedical search engines are free and cover a large number of scientific publications. Therefore, when institutional or personal subscriptions to Scopus and Web of Science are unavailable, a simple search in PubMed and Google Scholar will be sufficient. The simple number of publications retrievable with an electronic search should not necessarily guide your conclusions about the novelty of the article, since many differences may exist regarding the study population, the sample size, the analytical techniques, the endpoints. Nevertheless, it occasionally happens that all these aspects are quite similar, or virtually identical, to those contained in previously published articles. In such case, it is actually worthless to undertake a thoughtful revision of the manuscript, since it is unlikely that the conclusions of the study will contribute to improve the current scientific knowledge, and it may hence be advisable to limit your comments to a simple sentence stating that the novelty of the article is too low to recommend acceptance, or that the topic does not fit the scope of the journal.

Although inherently arbitrary, I also tend to use biomedical search engines for checking the number and type of previous publications by the same team of authors, provided that the article is not anonymized. This will give you advices about competence and reputation of the authors, and is a virtually unavoidable practice when you are invited to peer-review guidelines, recommendations or position papers. Notably, by checking PubMed, I have been also able to identify a number of duplicate (or very similar) articles, which cannot be always detected using a plagiarism check software (7). Some authors are getting smart; they submit duplicate articles with substantial word changes, but whose contents are totally overlapping with those of previous publications (8).

The comments

I usually read the article twice. The first reading is aimed to reach a general opinion about novelty, quality and practical implications. I do not typically write any comment during the first read. The second reading, often on a different day, is instead finalized to more accurately identify drawbacks or weaknesses.

The quality assessment of an article must be rigorous and meet a number of predefined criteria. Most of these have been discussed in a previous article, dealing with personal suggestions about writing scientific articles (9). Briefly, a good peer-review activity entails checking that (I) the title is appropriate; (II) the authors’ list really mirrors the individual contribution; (III) the abstract is focused on data and conclusions; (IV) the introduction clearly defines the main aspects of the topic being investigated and explains the aim of the study; (V) the materials and methods section exhaustively describes study population, sample size, analytical techniques, statistical tests, informed consent and ethical approval; (VI) the result section contains relevant findings without replicating data already shown in tables and figures; (VII) the discussion does not repeat data previously reported in results, tables or figures, appropriately discusses the findings according to current knowledge or existing literature, conclusions are supported by biological explanation, and study limitations are clearly highlighted; (VIII) the reference list fulfils journal’s guidelines, is appropriate and does not include many self-citations.

The referee should also carefully check that the article contains all necessary information for guaranteeing study reproducibility. A final scrutiny of article layout may also be advisable, focusing on style, presence of typos and unexplained abbreviations. When the first two aspects are poor, it may be advisable to suggest that the article should be reviewed by an English-native speaker, whilst the presence of many unexplained abbreviations needs to be highlighted, since these may not be understood by the readers. Although some publishers advocate that time should not be spent to polish grammar or spelling, not all journals carefully revise the original text before publication. Therefore, I prefer to highlight at least the major stylistic issues encountered during my readings, so that these can be fixed by the authors while resubmitting their manuscript.

Importantly, the referee must not use peer-review activity as an unfair means for boosting bibliometric indices, e.g., by asking to add citations to your previous articles, especially when these citations are completely unwarranted. When peer-review is blind, the referee should avoid using expressions that may lead the authors to identify referee’s identity.

Write your comments clearly

The worst aspect that challenges article revision according to the comments of reviewers is being unable to understand what reviewers are asking. It is not so rare to read comments like “I do not agree with your study design”, “a statement on page 5 is questionable” or “the statistics should be broadened”. Occasionally, the comments are written in such a bad English that the authors cannot even understand what the referee means. This makes article revision virtually unfeasible, or else the authors may introduce changes in the manuscript that are not really necessary. As a rule of thumb, I always write my comments indicating both page and line numbers or, when these are unavailable, I specifically indicate to the part of the manuscript needing revision, e.g., reporting the full sentence or the paragraph between brackets (e.g., I found a problem in the sentence: “…”), and I classify the potential caveats in "major" and "minor". Then, I read my comments almost twice, to be sure that what I have written can be clearly understood by the authors. I always structure my comments in numbers or dot points, since this helps authors’ reply.

Regarding the specific comments that you are willing to make about the article, disagreement is allowed, and often advisable, as long as its source is clearly disclosed and supported by objective data. It is not fair to judge a manuscript only guided by impressions. As previously mentioned, it is actually meaningless for both the editor of the journal and the authors to read a comment like “a statement on page 5 is questionable”, without such statement being explained. Therefore, whenever I do not agree with some parts of a manuscript, I always accompany my comments with reference to previous studies and clear explanations about what I think is a drawback, so that my note can be no longer considered personal or subjective. This will also help the editor taking a sounder decision when reading your comments and prevent embarrassing replies by the authors.

Be fair with the authors

It occasionally happens to receive weird, provocative and even offensive comments by the reviewers. The activity of peer-reviewing has nothing to do with a fight club. The reviewer is not engaged in a battle with the authors, but is only asked to provide expert advice to the Editor of the journal, who is the one and only responsible for the final decision. Therefore, even when the topic, the findings or the conclusions are strongly against your personal beliefs, you will need to express your disagreement with a fair and balanced approach, by constructively emphasizing the negative aspects or expressing an unbiased judgement about the strengths of the article. When communicating opinions about what is needed for improving the quality of the manuscript, the verb “must” should only be used when changes are absolutely necessary, otherwise the verb “should” seems more appropriate.

Weight revision according to the “impact” of the journal

One foremost issue that should guide your comments is the overall “impact” of the journal. It is not the same to accept an assignment to refereeing an article for a “top”, high impact factor journal, or for a local magazine. This aspect is often under-recognized by some reviewers and may also cause problems to the editors. As discussed elsewhere (10), a small sample size study, decently written, may still be suitable for publication in a non-indexed journal, whilst it is absolutely unfitted for high-impact factor journals. On the contrary, it is not so infrequent to submit an article to a local journal and then receiving the same comments as it had been submitted to Nature or to the New England Journal of Medicine.

The final recommendation

According to journal, once the peer-review process has been concluded, there may be a number of available options to summarize your final recommendations. These can be typically classified in “accept”, “minor revision”, “major revision” or “reject”. There may be other options (e.g., “resubmit as a short communication”, “transform in a letter to the editor”, “reject and resubmit”, “transfer to another journal”, etc.) but, more or less, their significance and consequences are overlapping. The final recommendation should hence be based on some essential and universally accepted criteria. Table 1 summarizes a series of questions that you should answer before deciding as to whether the article needs to be rejected, can be somehow improved by the authors after (minor or major) revision, or can be immediately accepted. You should find a good balance between each “yes”, “partially” or “no” answers that you have given to these questions. This approach is also sometimes available in the website of scientific journals, and is meant to help you (and the Editor) to summarize your previous thoughts. Importantly, your recommendations should be in accordance with the comments you have previously written. It occasionally happens to receive six pages of comments by a referee, which are then synthesized as “minor revision” or, even more ironically, to read a few number of minor issues which are then accompanied by the recommendation to “reject” the manuscript. Constructive criticism should also be expressed when recommending rejection, since this may help the authors improving the work for future submissions to other journals.

Table 1 Conventional criteria guiding the final recommendation
Full table

You should finally bear in mind that the definitive decision about the fortune of the manuscript will only be made by the editor, and will be weighted against his/her personal view and the comments of other referees (it is likely that the manuscript has been assigned to at least another referee). Therefore, you should not get upset or offended if your recommendation will then be reversed by the editorial office.

Confidentiality

According to the CSE (5), maintaining the confidentiality of peer-review entails “not sharing, discussing with third parties, or disclosing information from the reviewed paper”. Moreover, peer-reviewers are not allowed to retain copies of the article and are not allowed to use the knowledge of its content for purposes not pertaining to peer-review. Whatever deviation to this practice is seen as a serious misconduct.

Conclusions

As for a general assumption, no single and validated approach exists to peer-review scientific articles. Nevertheless, some simple concepts gathered after years of experience, may help performing this vital activity according to objective and fair rules (Table 2). More or less like writing scientific articles, the activity of refereeing is an ongoing learning. The more you experience, the more you learn. Therefore, I really hope that some notions reported in this dissertation may be a guide or a help, especially for young scientists who are willing to be engaged in peer-reviewing scientific articles.

Table 2 Basic notions for peer-review of scientific articles
Full table

Acknowledgements

None.

Footnote

Conflicts of Interest: The author has no conflicts of interest to declare.

References

1. Mayden KD. Peer review: publication's gold standard. J Adv Pract Oncol 2012;3:117-22. [PubMed]
2. Gasparyan AY, Kitas GD. Best peer reviewers and the quality of peer review in biomedical journals. Croat Med J 2012;53:386-9. [Crossref] [PubMed]
3. Jefferson T, Rudin M, Brodney Folse S, et al. Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database Syst Rev 2007.MR000016. [PubMed]
4. Zhong JS. Living legend in clinical biochemistry-Prof. Giuseppe Lippi: we should have strength to stick to things we do. Ann Transl Med 2016;4:322. [Crossref] [PubMed]
5. Council of Science Editors. Reviewer roles and responsibilities. Available online: https://www.councilscienceeditors.org/resource-library/editorial-policies/white-paper-on-publication-ethics/2-3-reviewer-roles-and-responsibilities/
6. Lippi G, Favalor EJ, Simundic AM. Biomedical research platforms and their influence on article submissions and journal rankings: an update. Biochem Med (Zagreb) 2012;22:7-14. [Crossref] [PubMed]
7. Eichmann K. Routine check for plagiarism. Cell Mol Life Sci 2015;72:4013. [Crossref] [PubMed]
8. Lippi G, Favaloro EJ. Detection of duplicates and redundancies. A major responsibility of peer-reviewers? Clin Chem Lab Med 2008;46:1796-7. [Crossref] [PubMed]
9. Lippi G. How do I write a scientific article?-a personal perspective. Ann Transl Med 2017;5:416. [Crossref] [PubMed]
10. Shokraneh F, Ilghami R, Masoomi R, et al. How to select a journal to submit and publish your biomedical paper? Bioimpacts 2012;2:61-8. [PubMed]