The Peer Review Process and Celebrating Journal Reviewers

The Journal of Pediatric and Adolescent Gynecology (JPAG) and other scientific journals could not publish high-quality scientific reports without the behind-the-scenes work of individuals who contribute considerable time and effort to reviewing manuscripts. These peer reviewers receive little recognition beyond the Editors' heartfelt gratitude and the yearly publication of a list of the names of all of the individuals who provided reviews during the past year. Does anyone look at this list of names? I'm not sure, as JPAG hasn't tracked this, but I will confess that, I DO look at this list for other journals, to find the names of recognized leaders in the field as an indicator of the quality of the journal.

In this issue of JPAG, Gina Sucato and Cynthia Holland-Hall have written a review on “Reviewing manuscripts: A systematic approach”, that I encourage you to read. If you are a reviewer, for JPAG or other journals, you will find helpful suggestions, as well as answers to questions that you may have wondered about, particularly if you, like me, didn't really have any formal instruction in how to perform a peer review. If you are a new reviewer, this review on reviewing provides “everything you always wanted to know” about reviewing. The review is broadly applicable to other fields and other journals beyond JPAG, and merits your recommendation to colleagues. If you are not a peer reviewer or contributor to the scientific literature, but instead, a clinician who looks to scholarly publications to guide your clinical practice, this review will still be of interest in providing reassurances that what you read in JPAG has been rigorously reviewed by experts in the field of pediatric and adolescent gynecology.

Early in my academic career, I was quite flattered by the honor of an invitation to provide peer review. I began to do reviews, and got better at it over time without the benefit of reading a how-to review; one of the benefits of reviewing is the opportunity to read the reviews of other reviewers, and to learn on-the-job. It remains an honor to be invited to review a colleague's manuscript, but there are a number of practical considerations that I have learned are important to consider before agreeing to provide a review. As a prospective reviewer, I initially ask myself, “Am I the right person to do this review”—i.e., is this in my area of expertise; if not, it is relatively easy to decline to do the review. But even that process isn't always simple, as it is helpful to the editor who is requesting the review if I can suggest other possible reviewers, particularly if I have some acquaintance with the subspecialty area and know who are the experts in that area of focus. If I conclude that the manuscript is in my wheelhouse of expertise, I am typically excited by the prospects of reading the manuscript and learning about new ideas that might prompt new thinking about a traditional topic or suggest future lines of scholarly investigation. Reviewing also provides the opportunity to stay up-to-date on cutting-edge research. However, the third question that I need to think about before accepting an invitation is whether I can realistically complete the review within the required period of time—typically 2-3 weeks. This is really essential, as the peer review process hinges on the timely review of manuscripts. Sucato and Holland-Hall cite studies indicating that the mean time required for a reviewer to write a review is 3 hours, with more than half taking <5 hours to review. Given my current academic responsibilities, and my responsibilities as JPAG Editor-in-Chief (EIC), it is becoming less often that I can make that commitment to other journals.

One of the aspects of reviewing for a journal is that it is an opportunity to provide mentorship to other scholars in the field. Being a good reviewer is less like being a critic, and more like being a mentor. The suggestions from good peer reviewers comment on the strengths of a manuscript, as well as provide feedback and suggest opportunities for improvement.

While the editorial peer review process may seem opaque to scholars who submit manuscripts, I would like for the editorial process at JPAG to be more transparent for authors. At last year's NASPAG Annual Clinical and Research Meeting, JPAG's Elsevier publisher, Andrea Boccelli and Gina Sucato presented a workshop that became the foundation and impetus for my invitation to Gina to write a review on the topic of reviewing. Andrea provided a flowchart for a journal's editorial process, given her experience with Elsevier with journals in general, and I've modified the chart a bit more to reflect the specifics of the process with JPAG Figure 1.

When an article is submitted through the online submission portal of Editorial Manager, the article is first reviewed by David Newcombe, JPAG's Managing Editor. David confirms that the submission contains all of the necessary elements required by the journal—the conflict of interest statement, contact information for the corresponding author, etc. The submission then comes to me as Editor-in-Chief. I read it over, making an initial assessment as to whether it merits review. If the submission is not really focused on pediatric and adolescent gynecology—for example, if the subjects include adult women, or adolescent girls and boys—I may decide on a “desk reject” as “out-of-scope” for the journal and journal readers. This quick decision and return of the manuscript to the authors precludes sending a manuscript to Associate Editors and Reviewers that is not likely to ultimately be published in JPAG. This also allows the authors to resubmit to another more appropriate journal in a timely manner. Frequently I will seek input from one of the Associate Editors or Deputy Editor, asking their opinions about the suitability of the submission. The Editorial Manager web site facilitates such discussions about submitted manuscripts. Other situations that may prompt a discussion include questions about ethical issues or the scientific novelty or importance of a submission. When there is agreement at the beginning of the submission process that an article clearly does not meet JPAG's standards for sending a submission for review, a desk reject may also be issued for this reason. However, most submission are deemed to deserve a careful peer review.

The next step in the process is for me to send the submission to one of the Associate Editors, who handles original submissions, or the Deputy Editor, who handles case reports. This assignment depends on the Associate Editor's areas of expertise and experience. Each of those individuals selects 2-3 reviewers, based on their areas of expertise, and these experts are invited to review, in what is, for JPAG, a single blinded review process in which the reviewers remain anonymous, but authors' names are disclosed. Typically, not every invited peer reviewer is able to agree to review, so the Associate Editors then need to find alternate reviewers. The submission of reviews through the Editorial Manager web site is tracked, reminder letters to reviewers are sent automatically, and invited reviewers who do not respond within a reasonable period of time are sent letters excusing them from reviewing. An online review form is completed by each reviewer, as well as a narrative Comments to the Authors section, with detailed assessment and review, and confidential Comments to the Editor Figure 2.

When an appropriate number of reviews has been returned, the Associate Editor collates and reconciles the reviewers' recommendations, and provides a recommendation to me as to whether the decision on the manuscript should be “Accepted, no revisions required”—a truly rare event, Rejected, or most commonly, “Major revisions required” or “Minor revisions required”. The reviewers' comments are sent back to the authors with a decision letter and a time frame for resubmission. With resubmission, the authors are asked to submit a detailed response addressing every concern from every reviewer. The manuscript then comes back to the original Associate Editor, who then may conclude that the authors have addressed all of the reviewers concerns (if they are relatively minor), or if major revisions are required, the original reviewers will typically be asked to look at the manuscript to see if the authors have satisfactorily addressed all feedback.

Currently, the timeline for the Editorial Process at JPAG from manuscript submission to first decision is averaging less than one month. The acceptance rate for submissions is currently running in the mid 30 percent range. The number of submissions to JPAG has increased over previous years' submissions, and the journal is thus able to be more selective, with a higher quality of the manuscripts selected for publication. As I have noted previously in my editorials, I've experienced a steep learning curve as the Editor-in-Chief, “el Jefe”. I continue to learn more about this job.

One example or something that I had not known about previously occurred recently, when I received a document which the publisher termed the FORM report—providing the Fate of Rejected Manuscripts from JPAG. This document provided a list of manuscripts that had been rejected by JPAG, but which were subsequently published by other journals—including which journal, the number of citations that the manuscript has received, and how these citations compare with expected citations of similar articles. The data were from the years prior to my coming on board as JPAG EIC, but nevertheless, I found it edifying. How are we as an editorial board and journal editors doing in making decisions about good science manuscripts that merit publication? While this document provides one measure of an answer, as editors, we welcome your thoughts and comments.