Understanding the Role of Peer Review

Peer review is the cornerstone of scholarly publishing. Some reviewers offer gentle, yet unhelpful feedback. Others may be harsh but give insightful comment. Striking the right balance is key. Let me share my approach on being the 'just right' peer reviewer The are 2 parts Part 1: What to pay attention to (per section) Part 2: Scripts on how to critique politely ----------- Part 1: 📝1️⃣ Abstract: • Is it a short, clear summary of the aims, key methods, important findings, and conclusions? • Can it stand alone? • Does it contain unnecessary information? 🚪2️⃣ Introduction: Study Premise: Is it talking about something new on something old? • Does it summarize the current state of the topic? • Does it address the limitations of current state in this field? • Does it explain why this study was necessary? • Are the aims clear? 🧩3️⃣ Methods: • Study design: right to answer the question? • Population: unbiased? • Data source and collection: clearly defined? • Outcome: accurate, clinically meaningful? • Variables: well justified? • Statistical analysis: right method, sufficient power? • Study robustness: sensitivity analysis, data management. • Ethical concerns addressed? 🎯4️⃣ Results: • Are results presented clearly, accurately, and in order? • Easy to understand? • Tables make sense? • Measures of uncertainty (standard errors/P values) included? 9/16: 📈6️⃣ Figures: • Easy to understand? • Figure legends make sense? • Titles, axis clear? 🌐7️⃣ Discussion: The interpretation. • Did they compare the findings with current literature? • Is there a research argument? (claim + evidence) • Limitations/strengths addressed? • Future direction? 📚8️⃣ References: • Key references missing? • Do the authors cite secondary sources (narrative review papers) instead of the original paper? ------------ Part 2: 🗣️ How do you give your critique politely? Use these scripts. Interesting/useful research question, BUT weak method: - The study premise is strong, but the approach is underdeveloped." Robust research method, BUT the research question is not interesting/useful: -"The research method is robust and well thought out, but the study premise is weak." Bad writing: -"While the study/ research appears to be strong, the writing is difficult to follow. I recommend the authors work with a copyeditor to improve the flow/clarity and readability of the text" Results section do not make sense: -"The data reported in {page x/table y} should be expanded and clarified." Wrong interpretation/ wrong conclusion: -"The authors stated that {***}, but the data does not fully support this conclusion. We can only conclude that {***}. Poor Discussion section -"The authors {did not/fails to} address how their findings relate to the literature in this field." Copy this post into a word document and save it as a template. Use this every time you have to review a paper. If you are the receiver of peer review - you can also use this to decode what the reviewer is saying.😉

𝑴𝒂𝒏𝒂𝒈𝒊𝒏𝒈 𝑹𝒆𝒗𝒊𝒔𝒊𝒐𝒏𝒔 𝒂𝒏𝒅 𝑹𝒆𝒔𝒑𝒐𝒏𝒅𝒊𝒏𝒈 𝒕𝒐 𝑹𝒆𝒗𝒊𝒆𝒘𝒆𝒓 𝑪𝒐𝒎𝒎𝒆𝒏𝒕𝒔 The revision process and responding to reviewer comments are crucial for getting your manuscript published. Here are best practices to guide you through this critical stage of academic publishing: 1️⃣ Understand the Feedback ☞ Read Thoroughly Carefully read the decision letter and reviewer comments. Ensure you fully understand the feedback before starting revisions. Categorize Comments ➣ Organize comments into categories (e.g., major revisions, minor revisions, suggestions) 2️⃣ Stay Organized ➣Create a Response Document Draft a separate document to address each reviewer's comment systematically. ➣ Track Changes Use the track changes feature in your word processor to clearly show your edits in response to the feedback. 3️⃣ Address All Comments ➣ Direct Responses Respond to each comment directly and clearly. If a comment is unclear, seek clarification from the editor. ➣ Detailed Explanations Provide detailed explanations for the changes you made. If you disagree with a comment, respectfully explain your reasoning. 4️⃣ Revise the Manuscript ➣Incorporate Changes Make the necessary changes in your manuscript based on the feedback. Focus on clarity, consistency, and accuracy. ➣Improve Overall Quality Use this opportunity to improve the overall quality of your manuscript. 5️⃣ Maintain Professionalism ➣Respectful Tone Maintain a professional and respectful tone in your responses, even if you disagree with the feedback. ➣Positive Attitude Approach the revision process with a positive attitude. 6️⃣ Clarify and Justify ➣Clear Justifications When changes are made, clearly justify why they were necessary. Provide evidence or references to support your modifications. ➣Address Contradictory Feedback If reviewers provide contradictory comments, address each individually and explain how you resolved the differences. 7️⃣ Be Timely ➣Meet Deadlines Adhere to the timeline provided by the journal for submitting your revised manuscript. If you need more time, communicate with the editor in advance. 8️⃣ Final Check ➣Proofread Carefully proofread your revised manuscript to ensure it is free of errors and inconsistencies. ➣Consistency Ensure that all changes are consistent throughout the manuscript. 𝗪𝗵𝗮𝘁 𝘀𝘁𝗿𝗮𝘁𝗲𝗴𝗶𝗲𝘀 𝗵𝗮𝘃𝗲 𝘆𝗼𝘂 𝗳𝗼𝘂𝗻𝗱 𝗲𝗳𝗳𝗲𝗰𝘁𝗶𝘃𝗲 𝗶𝗻 𝗺𝗮𝗻𝗮𝗴𝗶𝗻𝗴 𝗿𝗲𝘃𝗶𝘀𝗶𝗼𝗻𝘀 𝗮𝗻𝗱 𝗿𝗲𝘀𝗽𝗼𝗻𝗱𝗶𝗻𝗴 𝘁𝗼 𝗿𝗲𝘃𝗶𝗲𝘄𝗲𝗿 𝗰𝗼𝗺𝗺𝗲𝗻𝘁𝘀❓ 🔄 𝗦𝗵𝗮𝗿𝗲 𝘁𝗵𝗶𝘀 𝗴𝘂𝗶𝗱𝗲 𝘁𝗼 𝗵𝗲𝗹𝗽 𝗳𝗲𝗹𝗹𝗼𝘄 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵𝗲𝗿𝘀❗ #AcademicWriting #ManuscriptRevisions #PeerReview #PublishingTips

On reviewing at a top journal (part one). While I have written extensively about reviewing, I spent some time reading Suprateek Sarker and his team's latest editorial on writing constructive reviews at Information Systems Research - An INFORMS Journal. The editorial is so long that I'm breaking my comments into two posts. I like this editorial because it involves Edgar Whitley, who taught one of the first reviewer workshops that I ever attended. So, I knew a bit about the approach, and I learned a lot from the author's team's more recent and advanced thinking on reviews. Why is peer review important? (1) Significance of Peer Review: Peer review upholds the credibility of research, distinguishing scholarly articles in leading journals from those in sources of varied reliability. It assures readers that published work aligns with academic norms, contributing meaningfully to the discipline. (2) Quality Reviews in ISR: With ISR's growth and commitment to inclusive excellence, there's a rising need for reviewers adept at evaluating diverse research traditions. Quality reviews help uphold the journal's standards, ensuring impactful contributions. (3) Two Dimensions of Review: Gupta's (2018) framework outlines 'q-quality' (the idea's novelty) and 'r-quality' (execution). Effective reviews enhance both, providing constructive feedback to clarify contributions and guide methodical improvement. What do reviewers and editors do? (4) Roles of Reviewers and Editors: Reviewers serve as 'diamond cutters,' refining papers, while editors act as stewards, balancing constructive feedback and critical assessments. This ensures minimized Type I and Type II errors (accepting or rejecting papers mistakenly). (5) Ethical Conduct in Reviews: Reviewers must maintain high ethical standards, manage biases, avoid unethical behavior, and disclose conflicts of interest. Transparency and honesty, even about one's limitations, are key to upholding integrity. How should reviews be written? (6) Constructive Approach: Reviews should be collegial and developmental, focusing on how to advance the work constructively. Critiques must be specific, avoiding overly judgmental or dismissive tones. (7) Structure of Reviews: The authors suggest that we begin with a brief paper overview, provide a broad reaction, and move to detailed suggestions, then conclude with a summary to aid authors and editors in understanding the main points and potential improvements. While the themes should familiar with senior faculty, the comments are helpful for early career scholars writing their first reviews. The takeaway? Be clear, be fair, make your review accessible. You can find the editorial here: https://lnkd.in/eNV43M5b More tomorrow! #reviewing

✍🏽 Are you an early career researcher serving as a journal peer reviewer for the first time? Or planning to soon? Here are a few of the questions I think about in each section as I am peer-reviewing journal manuscripts: 1️⃣ Abstract • Is it clear what was studied and why? • Are the main methods and results clear? • Are the conclusions supported by the study results? • Does it contribute meaningfully to the scientific literature? 2️⃣ Intro • Is it reasonable (e.g., evidence-based, logical)? • Any major bodies of evidence/literature missing? 3️⃣ Methods • Are methods technically sound and justified? • Are methods sufficiently detailed for someone to reproduce? • Are methods reasonable for the study objective or hypothesis? 4️⃣ Results • Are results appropriate? • Any major results missing? 5️⃣ Discussion • Is the discussion balanced/unbiased? • Any major topics or literature missing? • Are study limitations sufficiently discussed? • Are results discussed in the context of prior literature? • Are interpretations reasonable and supported by the results? 6️⃣ Conclusion • Are conclusions supported by results? • Are conclusions over-generalized or over-stated? ❌ For many early career researchers, you are used to getting red-ink-soaked manuscript pages back from your advisor. Or maybe you've been put thru the ringer before by Reviewer 2—we all have! So, it may seem natural to provide similar levels of edits during peer-review. Don't. RESIST THIS URGE! ☝🏽 Remember: you are not a co-author on the paper, so it's not the same level of revisions you would do internally on our own paper and when your own name is on the document. 📈 I largely look at scientific manuscripts as progress reports. As a research community, we gradually and collectively build up the evidence and understanding in our fields. ⚡️ To close, here are 3 more bonus tips to keep in mind while you review: 🧘🏽 Try not to be too nitpicky about individual sentences or wording. The goal of peer review isn't to correct or improve every minor little thing. And no paper can comprehensively cover every tangential topic and reference. Nor is it going to be written in your preferred style or be perfectly clear (that's ok and normal). 👍🏽 If you suggest major changes or provide major critiques, then explain why (including references if helpful) and, when possible, suggest constructive ways for authors to address your feedback. 👏🏽 Reviewing manuscripts isn't just about critiquing the negatives! It's ok (and appreciated by authors!) when you point out positive things: great figures or tables, clear writing, clever study designs, well-explained interpretations, etc. 🎯 TL/DR: Focus on evaluating whether the research is technically sound and the manuscript is sufficiently rigorous (not to be confused with comprehensive or perfect). You can hold research to a high standard while being constructive and reasonable. Don't become Reviewer 2 or confuse yourself with being a co-author.

When they said, "Just revise and resubmit (your proposal), you will be fine," I am certain that they didn't mean this ⬇ 🙃 Revising and resubmitting a rejected proposal requires careful planning, consideration, and improvement of various aspects. Here are some tips. 1. Review feedback thoroughly: Start by thoroughly reviewing the feedback provided by the reviewers. Understand their suggestions, as this will be the basis for the revisions. 2. Address reviewer comments: Read each comment and criticism thoroughly from the reviewers (not in the R&R but internally to plan). Think about how you will incorporate their suggestions or why you choose not to if you believe your original approach is sound. 3. Improve clarity and organization: Ensure that the proposal is clear and well-organized. Make your goals, objectives, activities, methods, and significance of the project explicit and easy to understand. Include a TOC and/or LM. 4. (If NSF) Strengthen the Intellectual Merit and Broader Impacts: Emphasize the IM and BI of your project. Explain how your project advances scientific knowledge and benefits society. 5. Update data: Include new preliminary data or results that strengthen your case. This can demonstrate the progress or the feasibility of your project. 6. Highlight collaborations: If applicable, highlight any new collaborations or partnerships that have developed since your initial submission. 7. Revisit budget and timeline: Review them to ensure they are realistic and well-justified. 8. Seek external input: Consider seeking feedback from colleagues, mentors, or peers who were not involved in the initial submission. Fresh perspectives can be valuable. 9. Follow guidelines: Ensure that you follow all solicitation guidelines and formatting requirements to the letter. This includes adherence to page limits, font sizes, and other formatting rules. 10. Resubmit strategically: Timing is important. Don't rush to resubmit immediately after receiving feedback. Take the time needed to plan and make substantial improvements. 11. Write a stronger project summary: Pay attention to your project summary or abstract. It should provide a clear and compelling overview of your project. 12. Stay positive: Remember that rejection is common in the world of grant funding. Be persistent, maintain a positive attitude, and keep refining your proposal with each resubmission. 13. Maybe consider a different program: If your proposal is repeatedly rejected, it may be worth exploring whether another program might be a better fit. 14. Consult w/Program Officers: Reach out to the POs for guidance. Attend office hours/webinars. 15. Peer review: Consider having your revised proposal reviewed by colleagues/mentors who have experience with similar grants. Yes, the resubmission process can be challenging and time-consuming, but it's also a good opportunity to strengthen the proposal and increase the chances for success. 0 submission = 0 success.

Don't agree with a journal reviewer's comment? Here's how I recommend proceeding: ✅ Take Time To Process The Feedback ➡ As a PhD student, I struggled with emotionally reacting to feedback. Over time, I learned that I need a number of days to process reviewer comments before deciding on my course of action. ✅ Ask Why There Is A Disagreement ➡ Whenever I encounter a comment that I don't agree with, I ask myself, ‘Why do I disagree?’ Do I disagree because it would be a hassle to make the revision? Is my ego getting in the way, making me unwilling to admit that my thinking is flawed? Does my disagreement stem from the reviewer not fully grasping the issue they’re raising? ✅ Get External Guidance ➡ If I'm struggling to determine whether my disagreement is rational, I find it helpful to ask trusted mentors for their advice on how to best proceed with a comment. Senior researchers have likely encountered difficult reviewers in the past, so they'll have useful guidance based on their experiences. ✅ Provide A Rationale For Disagreeing ➡ If I disagree with a comment, I have found it helpful to: (a) thank the reviewer for their feedback; (b) acknowledge that I have thought critically about their comment; and (c) provide a strong rationale with supporting references for why I disagree. ✅ Determine If There Is Middle Ground ➡ Where possible, I like to determine whether there’s a middle ground where I can note their feedback in the manuscript without making substantive changes to my study. For instance, can I incorporate their comment as a footnote or in the limitations section? 💡 Key Takeaway As an early career researcher, I felt that I had to make every single change noted by reviewers to publish my manuscript. What I’ve learned since then is that it’s okay to disagree with a reviewer’s comment. At the end of the day, this is your paper, so you should feel comfortable with what you are putting out into the literature. When you do disagree, make sure to be respectful in your response by thanking the reviewer for their comment, acknowledging that you thought deeply about their feedback, and providing a strong rationale for why you disagree. P.S. What did I miss? How would you recommend disagreeing with a reviewer's comment? ➖ ➖ ➖ ➖ I'm Joseph Rios, the founder of Grad Student Academy. Follow me if you enjoyed this content. I write about graduate school and professional development issues for PhD students. My mission is to unveil the hidden PhD curriculum to all, not just the privileged few. Learn the strategies I used to go from being rejected by nearly every PhD program to: ✅ obtaining a top-notch industry job ✅ landing a tenure-track faculty position ✅ becoming an award-winning researcher