Scientific Peer Review

Rigorously evaluate scientific work for quality, validity, and reproducibility.

When to Use

• Self-reviewing manuscript before submission (REVIEW phase)
• Evaluating methodology and experimental design
• Checking statistical analyses and reporting
• Assessing reproducibility and data availability
• Reviewing others' manuscripts for journals
• Evaluating grant proposals
• Quality checking your own work during ANALYSIS phase

Review Workflow

1. INITIAL SCAN      → Overall impression, scope, significance
2. SECTION REVIEW    → Detailed evaluation of each section
3. METHODOLOGY       → Rigor, assumptions, controls
4. STATISTICS        → Appropriate tests, effect sizes, reporting
5. REPRODUCIBILITY   → Data, code, materials availability
6. FIGURES/TABLES    → Clarity, integrity, accessibility
7. ETHICS            → Approvals, consent, conflicts
8. WRITING           → Clarity, organization, accuracy
9. SYNTHESIZE        → Major/minor issues, recommendation

---

Stage 1: Initial Assessment

Quick Questions (5 minutes)

1. What is the central research question?
2. What are the main findings?
3. Is the work scientifically sound?
4. Are there any immediate major flaws?
5. Is it appropriate for the intended venue?

Initial Summary Template

## Initial Assessment

**Research Question**: [One sentence summary]

**Main Findings**: [2-3 key results]

**Initial Impression**: [Sound/Concerning/Major issues]

**Significance**: [Novel contribution to field?]

---

Stage 2: Section-by-Section Review

Abstract & Title

Check	Question	Status
Accuracy	Does abstract reflect the actual study?	☐
Clarity	Is the title specific and informative?	☐
Completeness	Are key findings summarized?	☐
Accessibility	Understandable to broad audience?	☐

Introduction

Check	Question	Status
Context	Is background adequate and current?	☐
Rationale	Is the research question justified?	☐
Novelty	Is originality clearly stated?	☐
Literature	Are relevant papers cited?	☐
Objectives	Are aims/hypotheses clear?	☐

Methods

Check	Question	Status
Reproducibility	Can another researcher replicate this?	☐
Rigor	Are methods appropriate for the question?	☐
Detail	Protocols, reagents, parameters described?	☐
Ethics	Approvals and consent documented?	☐
Statistics	Methods described and justified?	☐
Controls	Appropriate controls included?	☐

Critical Details to Verify:

• Sample sizes and power calculations
• Randomization and blinding
• Inclusion/exclusion criteria
• Software versions
• Statistical tests and corrections

Results

Check	Question	Status
Presentation	Logical and clear?	☐
Figures	Appropriate, clear, labeled?	☐
Statistics	Effect sizes, CIs, p-values?	☐
Objectivity	Results without interpretation?	☐
Completeness	Negative results included?	☐

Common Issues:

• Selective reporting
• Inappropriate statistical tests
• Missing error bars
• Over-fitting
• Batch effects or confounders
• Missing controls

Discussion

Check	Question	Status
Interpretation	Conclusions supported by data?	☐
Limitations	Acknowledged and discussed?	☐
Context	Placed appropriately in literature?	☐
Speculation	Distinguished from data-supported claims?	☐
Significance	Implications clearly stated?	☐

Red Flags:

• Overstated conclusions
• Ignoring contradictory evidence
• Causal claims from correlational data
• Mechanistic claims without evidence

---

Stage 3: Methodological Rigor

Statistical Assessment

Check	Question	Status
Assumptions	Are statistical assumptions met?	☐
Effect sizes	Reported alongside p-values?	☐
Multiple testing	Correction applied?	☐
Confidence intervals	Provided?	☐
Sample size	Justified with power analysis?	☐
Missing data	Handled appropriately?	☐
Exploratory vs confirmatory	Clearly distinguished?	☐

Experimental Design

Check	Question	Status
Controls	Appropriate and adequate?	☐
Replication	Biological and technical?	☐
Confounders	Identified and controlled?	☐
Randomization	Properly implemented?	☐
Blinding	Adequate for the study?	☐

---

Stage 4: Reproducibility Assessment

Data Availability

Check	Question	Status
Raw data	Deposited in repository?	☐
Accession numbers	Provided for databases?	☐
Restrictions	Justified (e.g., privacy)?	☐
Formats	Standard and accessible?	☐

Code and Materials

Check	Question	Status
Analysis code	Available (GitHub, Zenodo)?	☐
Protocols	Detailed enough to reproduce?	☐
Materials	Available or recreatable?	☐

Reporting Standards

Check adherence to discipline-specific guidelines:

Study Type	Guideline	Status
Randomized trial	CONSORT	☐
Observational	STROBE	☐
Systematic review	PRISMA	☐
Diagnostic study	STARD	☐
Animal research	ARRIVE	☐
Case report	CARE	☐

---

Stage 5: Figure and Table Review

Quality Checks

Check	Question	Status
Resolution	High quality?	☐
Labels	All axes/columns labeled with units?	☐
Error bars	Defined (SD, SEM, CI)?	☐
Statistics	Significance markers explained?	☐
Color	Colorblind-friendly?	☐
Scale bars	Included for images?	☐

Integrity Checks

Check	Question	Status
Manipulation	Any signs of image manipulation?	☐
Splicing	Gels/blots appropriately presented?	☐
Representative	Images truly representative?	☐
Complete	All conditions shown?	☐

---

Stage 6: Writing Quality

Structure and Organization

Check	Question	Status
Logic	Manuscript logically organized?	☐
Flow	Sections flow coherently?	☐
Transitions	Clear between ideas?	☐
Narrative	Compelling and clear?	☐

Writing Quality

Check	Question	Status
Clarity	Language clear and precise?	☐
Jargon	Minimized and defined?	☐
Grammar	Correct throughout?	☐
Concise	No unnecessary complexity?	☐

---

Structuring the Review Report

Summary Statement (1-2 paragraphs)

## Summary

[Brief synopsis of the research]

**Recommendation**: [Accept / Minor revisions / Major revisions / Reject]

**Key Strengths**:
1. [Strength 1]
2. [Strength 2]
3. [Strength 3]

**Key Weaknesses**:
1. [Weakness 1]
2. [Weakness 2]

**Bottom Line**: [Overall assessment of significance and soundness]

Major Comments

Issues that significantly impact validity or interpretability:

## Major Comments

1. **[Issue Title]**
   - *Problem*: [Clear statement of the issue]
   - *Why it matters*: [Impact on conclusions]
   - *Suggestion*: [How to address it]

2. **[Issue Title]**
   ...

Major issues typically include:

• Fundamental methodological flaws
• Inappropriate statistical analyses
• Unsupported conclusions
• Missing critical controls
• Reproducibility concerns

Minor Comments

Less critical issues that would improve the manuscript:

## Minor Comments

1. [Page/Figure X]: [Issue and suggestion]
2. [Methods section]: [Missing detail]
3. [Figure 2]: [Clarity improvement]

---

Review Tone Guidelines

Do ✓

• Be constructive and specific
• Acknowledge strengths
• Provide actionable suggestions
• Focus on the science
• Be thorough but proportionate

Don't ✗

• Use dismissive language
• Make personal attacks
• Be vague or sarcastic
• Request unnecessary experiments
• Impose personal preferences as requirements

---

Self-Review Checklist (Before Submission)

Use this during your REVIEW phase:

Methodology

• Methods are reproducible
• Controls are appropriate and documented
• Statistical methods are justified
• Sample sizes are adequate

Results

• All results support conclusions
• Effect sizes are reported
• Negative results are included
• Figures are clear and accessible

Reproducibility

• Data will be available
• Code is documented and available
• Protocols are detailed
• Reporting guidelines followed

Writing

• Abstract accurately summarizes the work
• Conclusions are supported by data
• Limitations are acknowledged
• References are current and complete

---

Integration with RA Workflow

REVIEW Phase Activities

1. Run self-review using this checklist
2. Document issues in tasks.md
3. Address each issue systematically
4. Re-review until checklist passes
5. Update .research/logs/activity.md

Pre-Submission Verification

Before calling a manuscript complete:

• Self-review completed
• All major issues addressed
• Figures meet journal requirements
• Data/code deposited
• Reporting checklist complete
• Cover letter prepared