PR Comment Skill

This skill posts automated progress comments to GitHub Pull Requests during the PR review workflow. Comments are self-contained with collapsible Review Session details, providing rich context to maintainers and contributors.

⚠️ Self-Contained Rule: All content in PR comments must be self-contained. Never reference local files like CustomAgentLogsTmp/ - GitHub users cannot access your local filesystem.

✨ Key Features:

Single Unified Comment: ONE comment per PR/Issue containing ALL sections (PR Review, Try-Fix, Write-Tests, Verify-Tests)
Section-Based Updates: Each script updates only its section, preserving others
Duplicate Prevention: Finds existing  comment and updates it
File-Based DryRun Preview: Use -DryRun to preview changes in a local file before posting
Simple Interface: Just provide PR number - script handles everything else

Comment Architecture

Unified AI Summary Comment

Most scripts post to the same single comment identified by . Each script updates its own section:

<!-- AI Summary -->

## 🤖 AI Summary

<!-- SECTION:PR-REVIEW -->
... PR review phases ...
<!-- /SECTION:PR-REVIEW -->

<!-- SECTION:TRY-FIX -->
... try-fix attempts ...
<!-- /SECTION:TRY-FIX -->

<!-- SECTION:WRITE-TESTS -->
... write-tests attempts ...
<!-- /SECTION:WRITE-TESTS -->

<!-- SECTION:VERIFY-TESTS -->
... test verification results ...
<!-- /SECTION:VERIFY-TESTS -->

Behavior:

First script to run creates the comment
Subsequent scripts find the existing comment and update/add their section
Sections are independent - updating one preserves others

Separate PR Finalization Comment

The post-pr-finalize-comment.ps1 script posts a separate comment identified by . This comment contains three sections:

Title: Shows the current vs recommended PR title
Description: Shows description assessment, missing elements, and recommended description
Code Review: Shows code review findings (critical issues, suggestions, positive observations)

If an existing finalize comment exists, it will be replaced with the updated sections. This keeps finalization reviews distinct from automated analysis.

⚠️ Important Requirements for PR Finalize Comments:

When TitleStatus is NeedsUpdate, always provide -RecommendedTitle
When DescriptionStatus is NeedsUpdate or NeedsRewrite, always provide -RecommendedDescription with the full suggested description text
The script will warn if these are missing but won't fail

Section Scripts

AI Summary Sections (Unified Comment)

Section	Script	Location
`PR-REVIEW`	`post-ai-summary-comment.ps1`	`.github/skills/ai-summary-comment/scripts/`
`TRY-FIX`	`post-try-fix-comment.ps1`	`.github/skills/ai-summary-comment/scripts/`
`WRITE-TESTS`	`post-write-tests-comment.ps1`	`.github/skills/ai-summary-comment/scripts/`
`VERIFY-TESTS`	`post-verify-tests-comment.ps1`	`.github/skills/ai-summary-comment/scripts/`

Separate Comments

Comment	Script	Marker
PR Finalization	`post-pr-finalize-comment.ps1`	`<!-- PR-FINALIZE-COMMENT -->`

Supported Phases

Phase	Description	When to Post	What This Enables Next
`pre-flight`	Context gathering complete	After documenting issue, files, and discussion	Tests Phase: Agent can now verify/create test files that reproduce the bug
`tests`	Test analysis complete	After identifying test files and coverage	Gate Phase: Agent can run tests to verify they catch the bug
`gate`	Test validation complete	After running tests and verifying bug reproduction	Fix Phase: Agent can explore alternative fixes (tests proven to catch bug)
`fix`	Solution comparison complete	After comparing PR fix with alternatives	Report Phase: Agent can finalize recommendation based on fix comparison
`report`	Final analysis complete	After generating comprehensive review	PR Decision: Maintainers can approve/merge or request changes based on full analysis

Usage

Simplest: Just provide PR number (auto-loads from phase files)

# Auto-loads from CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/*/content.md
pwsh .github/skills/ai-summary-comment/scripts/post-ai-summary-comment.ps1 -PRNumber 27246

Parameters

Parameter	Required	Description	Example
`PRNumber`	Yes	Pull request number	`12345`
`DryRun`	No	Preview changes in local file instead of posting to GitHub	`-DryRun`
`PreviewFile`	No	Path to local preview file for DryRun mode (default: `CustomAgentLogsTmp/PRState/{PRNumber}/ai-summary-comment-preview.md`)	`-PreviewFile ./preview.md`
`SkipValidation`	No	Skip validation checks (not recommended)	`-SkipValidation`

DryRun Preview Workflow

Use -DryRun to preview the combined comment before posting to GitHub. Each script updates the same preview file, mirroring how the actual GitHub comment is updated.

# Step 1: Run verify-tests script (creates preview file)
pwsh .github/skills/ai-summary-comment/scripts/post-verify-tests-comment.ps1 -PRNumber 32891 -DryRun

# Step 2: Run try-fix script (updates same preview file)
pwsh .github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 -IssueNumber 32891 -DryRun

# Step 3: Review the combined preview
open CustomAgentLogsTmp/PRState/32891/ai-summary-comment-preview.md

# Step 4: Post for real (remove -DryRun)
pwsh .github/skills/ai-summary-comment/scripts/post-verify-tests-comment.ps1 -PRNumber 32891
pwsh .github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 -IssueNumber 32891

Key behavior: The preview file exactly matches what will be posted to GitHub. Multiple scripts accumulate their sections in the same file.

Section Ordering

Sections appear in the unified comment in this order (based on which scripts run first):

VERIFY-TESTS - Test verification results
TRY-FIX - Alternative fix exploration attempts
WRITE-TESTS - Test writing attempts
PR-REVIEW - PR review phases

Each section is wrapped with markers like  and .

Cleanup

To reset the preview file for a fresh start:

rm CustomAgentLogsTmp/PRState/{PRNumber}/ai-summary-comment-preview.md

Prerequisites

Scripts require GitHub CLI authentication:

gh auth status  # Verify authentication before running

Comment Format

Comments are formatted with:

Phase badge (🔍 Pre-Flight, 🚦 Gate, 🔧 Fix, 📋 Report)
Status indicator (✅ Completed, ⚠️ Issues Found)
Expandable review sessions (each session is a collapsible section)
What's Next (what phase happens next)

Review Session Tracking

When the same PR is reviewed multiple times (e.g., after new commits), the script updates the single aggregated review comment and adds a new expandable section for each commit-based review session.

Example Output

## 🔍 Pre-Flight: Context Gathering Complete

✅ **Status**: Phase completed successfully

### Summary
- **Issue**: #33356 - CollectionView crash on iOS
- **Platforms Affected**: iOS, MacCatalyst
- **Files Changed**: 2 implementation files, 1 test file
- **Discussion**: 3 key reviewer comments identified

### Key Findings
- Crash occurs when scrolling rapidly with large datasets
- Existing PR adds null check in ItemsViewController
- Test coverage includes iOS device test

### Next Steps
→ **Phase 2: Gate** - Verifying tests catch the bug

---
*Posted by PR Agent @ 2026-01-17 14:23:45 UTC*

Script Files

post-ai-summary-comment.ps1 - Posts or updates the aggregated PR agent review comment
post-try-fix-comment.ps1 - Posts or updates try-fix attempts comment

Try-Fix Comment Script

The post-try-fix-comment.ps1 script updates the  section of the unified AI Summary comment. It aggregates all try-fix attempts into collapsible sections. Works for both issues and PRs (GitHub treats PR comments as issue comments).

✨ Auto-Loading from CustomAgentLogsTmp: The script automatically discovers and aggregates ALL attempt directories from CustomAgentLogsTmp/PRState/{IssueNumber}/PRAgent/try-fix/.

Usage

Simplest: Provide attempt directory

# All parameters auto-loaded from directory structure
pwsh .github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 `
    -TryFixDir CustomAgentLogsTmp/PRState/27246/PRAgent/try-fix/attempt-1

Or just provide issue number

# Auto-discovers and posts latest attempt from CustomAgentLogsTmp/PRState/27246/PRAgent/try-fix/
pwsh .github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 -IssueNumber 27246

Manual parameters

pwsh .github/skills/ai-summary-comment/scripts/post-try-fix-comment.ps1 `
    -IssueNumber 19806 `
    -AttemptNumber 1 `
    -Approach "LayoutExtensions Width Constraint" `
    -RootCause "ComputeFrame only constrains width for Fill alignment" `
    -FilesChanged "| File | Changes |`n|------|---------|`n| LayoutExtensions.cs | +17/-3 |" `
    -Status "Compiles" `
    -CodeSnippet "else if (!hasExplicitWidth) { ... }" `
    -Analysis "Core project compiles successfully"

Parameters

Parameter	Required	Description
`TryFixDir`	No*	Path to try-fix attempt directory (auto-loads all parameters)
`IssueNumber`	No*	Issue or PR number to post comment on
`AttemptNumber`	No*	Attempt number (1, 2, 3, etc.) - auto-detected from TryFixDir
`Approach`	No*	Brief description of fix approach
`RootCause`	No	Description of root cause identified
`FilesChanged`	No*	Markdown table of files changed - auto-generated from diff
`Status`	No*	"Compiles", "Pass", or "Fail" - loaded from result.txt
`CodeSnippet`	No	Code snippet showing the fix - loaded from fix.diff
`Analysis`	No	Analysis of why it worked/failed - loaded from analysis.md
`DryRun`	No	Print comment instead of posting

*When using -TryFixDir, all marked parameters are auto-loaded from files in the directory.

Expected Directory Structure

CustomAgentLogsTmp/PRState/{IssueNumber}/PRAgent/try-fix/
├── attempt-1/
│   ├── approach.md      # Brief description of the approach (required)
│   ├── result.txt       # "Pass", "Fail", or "Compiles" (required)
│   ├── fix.diff         # Git diff of the fix (optional)
│   └── analysis.md      # Detailed analysis (optional)
├── attempt-2/
│   └── ...
└── attempt-3/
    └── ...

Comment Format

## 🔧 Try-Fix Attempts for Issue #XXXXX

<!-- TRY-FIX-COMMENT -->

<details>
<summary>📊 <strong>Expand Full Details</strong></summary>

**Issue:** [#XXXXX](link)

---

<details>
<summary><strong>🔧 Attempt #1: Approach Name</strong> ✅ Status</summary>
... attempt details ...
</details>

---

*This fix was developed independently.*

</details>

Key Behaviors

First attempt creates new comment with  marker
Subsequent attempts edit the same comment (no new comments)
Outer wrapper shows "📊 Expand Full Details" - keeps PR page clean
Each attempt is a nested collapsible section inside the wrapper

Verify-Tests Comment Script

The post-verify-tests-comment.ps1 script updates the  section of the unified AI Summary comment. It documents test verification results (whether tests fail without fix and pass with fix).

✨ Auto-Loading from CustomAgentLogsTmp: The script automatically loads verification results from CustomAgentLogsTmp/PRState/{PRNumber}/verify-tests-fail/verification-report.md.

Usage

Simplest: Provide PR number

# Auto-loads from CustomAgentLogsTmp/PRState/{PRNumber}/verify-tests-fail/
pwsh .github/skills/ai-summary-comment/scripts/post-verify-tests-comment.ps1 -PRNumber 32891

With explicit report file

pwsh .github/skills/ai-summary-comment/scripts/post-verify-tests-comment.ps1 `
    -PRNumber 32891 `
    -ReportFile CustomAgentLogsTmp/PRState/32891/PRAgent/gate/verify-tests-fail/verification-report.md

Parameters

Parameter	Required	Description
`PRNumber`	Yes	Pull request number
`ReportFile`	No	Path to verification report (auto-discovered if not provided)
`Status`	No	"Passed" or "Failed" - auto-detected from report
`Platform`	No	Platform tested (ios, android, etc.) - auto-detected from report
`Mode`	No	"FailureOnly" or "FullVerification" - auto-detected from report
`DryRun`	No	Preview changes in local file instead of posting
`PreviewFile`	No	Path to local preview file for DryRun mode

Expected Directory Structure

CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/gate/verify-tests-fail/
├── verification-report.md   # Full verification report (required)
├── verification-log.txt     # Detailed log (optional)
├── test-without-fix.log     # Test output without fix (optional)
└── test-with-fix.log        # Test output with fix (optional)

Write-Tests Comment Script

The post-write-tests-comment.ps1 script updates the  section of the unified AI Summary comment. It documents test writing attempts for an issue.

✨ Auto-Loading from CustomAgentLogsTmp: The script can automatically load test details from the write-tests output directory structure.

Usage

Simplest: Provide test directory

# All parameters auto-loaded from directory structure
pwsh .github/skills/ai-summary-comment/scripts/post-write-tests-comment.ps1 `
    -TestDir CustomAgentLogsTmp/PRState/27246/write-tests/attempt-1

Or just provide issue number

# Auto-discovers and posts latest attempt from CustomAgentLogsTmp/PRState/27246/write-tests/
pwsh .github/skills/ai-summary-comment/scripts/post-write-tests-comment.ps1 -IssueNumber 27246

Manual parameters

pwsh .github/skills/ai-summary-comment/scripts/post-write-tests-comment.ps1 `
    -IssueNumber 33331 `
    -AttemptNumber 1 `
    -TestDescription "Verifies Picker.IsOpen property changes correctly" `
    -HostAppFile "src/Controls/tests/TestCases.HostApp/Issues/Issue33331.cs" `
    -TestFile "src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue33331.cs" `
    -TestMethod "PickerIsOpenPropertyChanges" `
    -Category "Picker" `
    -VerificationStatus "Verified"

Parameters

Parameter	Required	Description
`TestDir`	No*	Path to write-tests attempt directory (auto-loads all parameters)
`IssueNumber`	No*	Issue or PR number to post comment on
`AttemptNumber`	No*	Attempt number (1, 2, 3, etc.) - auto-detected from TestDir
`TestDescription`	No*	Brief description of what the test verifies
`HostAppFile`	No*	Path to the HostApp test page file
`TestFile`	No*	Path to the NUnit test file
`TestMethod`	No*	Name of the test method
`Category`	No*	UITestCategories category used
`VerificationStatus`	No*	"Verified", "Failed", or "Unverified" - loaded from result.txt
`Platforms`	No	Platforms the test runs on (default: "All")
`Notes`	No	Additional notes - loaded from notes.md
`DryRun`	No	Print comment instead of posting

*When using -TestDir, all marked parameters are auto-loaded from files in the directory.

Expected Directory Structure

CustomAgentLogsTmp/PRState/{IssueNumber}/write-tests/
├── attempt-1/
│   ├── description.md   # Brief test description (required)
│   ├── test-info.json   # {HostAppFile, TestFile, TestMethod, Category} (required)
│   ├── result.txt       # "Verified", "Pass", "Failed", or "Unverified" (required)
│   └── notes.md         # Additional notes (optional)
├── attempt-2/
│   └── ...
└── attempt-3/
    └── ...

test-info.json Format

{
    "HostAppFile": "src/Controls/tests/TestCases.HostApp/Issues/Issue27246.cs",
    "TestFile": "src/Controls/tests/TestCases.Shared.Tests/Tests/Issues/Issue27246.cs",
    "TestMethod": "ScrollToFirstItemWithHeader",
    "Category": "CollectionView"
}

PR Finalize Comment Script

The post-pr-finalize-comment.ps1 script posts a separate comment (not part of the unified AI Summary) specifically for PR finalization reviews. It provides structured feedback on the PR title, description, and code review findings.

Usage

Simplest: Just provide PR number (auto-loads from summary file)

# Auto-loads from CustomAgentLogsTmp/PRState/{PRNumber}/pr-finalize/pr-finalize-summary.md
pwsh .github/skills/ai-summary-comment/scripts/post-pr-finalize-comment.ps1 -PRNumber 33892

Full manual parameters (recommended for best results)

pwsh .github/skills/ai-summary-comment/scripts/post-pr-finalize-comment.ps1 `
    -PRNumber 33892 `
    -TitleStatus "NeedsUpdate" `
    -CurrentTitle "Fix 32650 Image Orientation" `
    -RecommendedTitle "[iOS][Android] MediaPicker: Fix image orientation when RotateImage=true" `
    -TitleIssues "- Missing platform tags
- Doesn't describe the behavior fix" `
    -DescriptionStatus "NeedsUpdate" `
    -DescriptionAssessment "The current description is minimal and missing:
- ❌ Missing NOTE block for testing artifacts
- ❌ No root cause analysis
- ❌ No technical details" `
    -MissingElements "Add the NOTE block, root cause, and technical details." `
    -RecommendedDescription "> [!NOTE]
> Are you waiting for this PR? Test it: [Testing PR Builds](link)

### Root Cause
...description...

### Description of Change
...details..." `
    -CodeReviewStatus "IssuesFound" `
    -CodeReviewFindings "### 🔴 Critical Issues
**1. Broken indentation**
- File: \`src/file.cs\`
- Problem: Inconsistent tabs/spaces

### 🟡 Suggestions
1. Consider disposing Matrix object

### ✅ Looks Good
- Proper cleanup in finally block"

Parameters

Parameter	Required	Description
`PRNumber`	Yes*	Pull request number
`SummaryFile`	No	Path to pr-finalize-summary.md (auto-discovered)
`TitleStatus`	No*	`Good` or `NeedsUpdate`
`CurrentTitle`	No*	Current PR title (fetched from GitHub if not provided)
`RecommendedTitle`	No	Required if TitleStatus is NeedsUpdate
`TitleIssues`	No	List of issues with current title
`DescriptionStatus`	No*	`Excellent`, `Good`, `NeedsUpdate`, or `NeedsRewrite`
`DescriptionAssessment`	Yes	Assessment of description quality
`MissingElements`	No	What's missing from the description
`RecommendedDescription`	No	Required if DescriptionStatus is NeedsUpdate/NeedsRewrite
`CodeReviewStatus`	No	`Passed`, `IssuesFound`, or `Skipped`
`CodeReviewFindings`	No	Markdown content for code review section
`DryRun`	No	Preview instead of posting

*At least PRNumber or SummaryFile required. Script auto-detects values when possible.

⚠️ Common Mistakes to Avoid

Missing RecommendedTitle when TitleStatus is NeedsUpdate
- The script will warn but still post - always provide a recommended title
Missing RecommendedDescription when DescriptionStatus is NeedsUpdate
- Users need to see what the description SHOULD look like
Code review findings not starting with proper headers
- Always structure with ### 🔴 Critical Issues, ### 🟡 Suggestions, ### ✅ Looks Good
Auto-parsing from summary file getting confused
- When in doubt, provide explicit parameters instead of relying on auto-parsing

Expected Directory Structure

The PR agent writes phase output files that comment scripts auto-load:

CustomAgentLogsTmp/PRState/{PRNumber}/
├── PRAgent/
│   ├── pre-flight/
│   │   └── content.md              # Phase 1 output (auto-loaded by post-ai-summary-comment.ps1)
│   ├── gate/
│   │   ├── content.md              # Phase 2 output (auto-loaded by post-ai-summary-comment.ps1)
│   │   └── verify-tests-fail/     # Script output from verify-tests-fail.ps1
│   │       ├── verification-report.md
│   │       ├── verification-log.txt
│   │       ├── test-without-fix.log
│   │       └── test-with-fix.log
│   ├── try-fix/
│   │   ├── content.md              # Phase 3 summary (auto-loaded by post-ai-summary-comment.ps1)
│   │   ├── attempt-1/             # Individual attempt outputs
│   │   │   ├── approach.md
│   │   │   ├── result.txt
│   │   │   ├── fix.diff
│   │   │   └── analysis.md
│   │   └── attempt-2/
│   │       └── ...
│   └── report/
│       └── content.md              # Phase 4 output (auto-loaded by post-ai-summary-comment.ps1)
├── pr-finalize/
│   ├── pr-finalize-summary.md
│   ├── recommended-description.md
│   └── code-review.md
└── copilot-logs/
    ├── process-*.log
    └── session-*.md

Auto-Loading Behavior

When post-ai-summary-comment.ps1 is called, it auto-discovers phase files:

Checks CustomAgentLogsTmp/PRState/{PRNumber}/PRAgent/*/content.md
Loads all available phase content files
Builds the comment structure from the loaded files
Posts/updates the unified AI Summary comment

This eliminates the need to pass large content strings as parameters.

Technical Details

Comments identified by HTML marker 
Existing comments are updated (not duplicated) when posting again
Review sessions grouped by commit SHA
Uses gh api for create/update operations