YouTube Comments Fetch

Core Goal

• Fetch public YouTube comment threads with commentThreads.list.
• Optionally expand replies with comments.list.
• Filter comments by configurable UTC time windows using published or updated timestamps.
• Return machine-readable JSON and optionally save JSONL artifacts.
• Keep execution observable with structured logs and optional log file.

Required Environment

• Configure runtime via environment variables (see references/env.md).
• Start from assets/config.example.env and keep real secrets in assets/config.env.
• Load env values before running commands:

set -a
source assets/config.env
set +a

Workflow

1. Validate effective configuration.

python3 scripts/youtube_comments_fetch.py check-config --pretty

2. Dry-run a fetch plan against candidate video IDs first.

python3 scripts/youtube_comments_fetch.py fetch \
  --video-ids-file ./data/youtube-videos/candidates.jsonl \
  --start-datetime 2026-03-01T00:00:00Z \
  --end-datetime 2026-03-08T00:00:00Z \
  --max-videos 5 \
  --max-thread-pages 3 \
  --dry-run \
  --pretty

3. Fetch comments and save JSONL artifacts.

python3 scripts/youtube_comments_fetch.py fetch \
  --video-ids-file ./data/youtube-videos/candidates.jsonl \
  --start-datetime 2026-03-01T00:00:00Z \
  --end-datetime 2026-03-08T00:00:00Z \
  --time-field published \
  --include-replies \
  --order time \
  --max-videos 10 \
  --max-thread-pages 10 \
  --max-reply-pages 20 \
  --max-comments 2000 \
  --output-dir ./data/youtube-comments \
  --log-level INFO \
  --log-file ./logs/youtube-comments-fetch.log \
  --pretty

Built-in Robustness

• Retry transient failures with exponential backoff.
• Respect Retry-After and fail fast when it exceeds configured cap.
• Throttle request rate with a minimum request interval.
• Enforce safety caps:
  • max videos
  • max thread pages
  • max reply pages
  • max threads
  • max comments
• Validate transport:
  • JSON content-type
  • UTF-8 decode
  • JSON object parse
• Validate structure:
  • thread/comment response shape
  • comment IDs and parent linkage
  • datetime fields
  • duplicate comment suppression
• Preserve item-level failures and validation issues in JSON output and optional quarantine files.

Scope Decision

• Keep one atomic operation: comment fetch for known/discovered video IDs.
• Accept upstream video IDs from CLI or from $youtube-video-search JSON/JSONL outputs.
• Do not perform video discovery in this skill.
• Do not embed scheduler/polling loops.

References

• references/env.md
• references/youtube-comments-api-notes.md
• references/youtube-limitations.md
• references/openclaw-chaining-templates.md

Script

• scripts/youtube_comments_fetch.py

OpenClaw Invocation Compatibility

• Keep trigger metadata in name, description, and agents/openai.yaml.
• Invoke with $youtube-comments-fetch.
• Keep the skill atomic: each invocation consumes one configured ID set and time window.
• Prefer chaining from $youtube-video-search output via --video-ids-file.
• Surface reply_window_completeness to downstream agents when replies are requested.

OpenClaw Prompt Templates

Use these templates directly in OpenClaw and only replace bracketed placeholders.

1. Recon (dry-run)

Use $youtube-comments-fetch.
Run:
python3 scripts/youtube_comments_fetch.py fetch \
  --video-ids-file [VIDEO_IDS_FILE] \
  --start-datetime [YYYY-MM-DDTHH:MM:SSZ] \
  --end-datetime [YYYY-MM-DDTHH:MM:SSZ] \
  --max-videos [N] \
  --max-thread-pages [M] \
  --dry-run \
  --pretty
Return only the JSON result.

2. Fetch (windowed comments)

Use $youtube-comments-fetch.
Run:
python3 scripts/youtube_comments_fetch.py fetch \
  --video-ids-file [VIDEO_IDS_FILE] \
  --start-datetime [YYYY-MM-DDTHH:MM:SSZ] \
  --end-datetime [YYYY-MM-DDTHH:MM:SSZ] \
  --time-field published \
  --include-replies \
  --order time \
  --max-videos [N] \
  --max-thread-pages [M] \
  --max-reply-pages [R] \
  --max-comments [K] \
  --output-dir [OUTPUT_DIR] \
  --pretty
Return only the JSON result.

3. Validate (quality gate)

Use $youtube-comments-fetch.
Run:
python3 scripts/youtube_comments_fetch.py fetch \
  --video-ids-file [VIDEO_IDS_FILE] \
  --start-datetime [YYYY-MM-DDTHH:MM:SSZ] \
  --end-datetime [YYYY-MM-DDTHH:MM:SSZ] \
  --max-videos 1 \
  --max-thread-pages 1 \
  --max-reply-pages 1 \
  --max-comments 50 \
  --pretty
Check validation_summary.total_issue_count, failures, and fetch_summary.record_count.
Return JSON plus one-line pass/fail verdict.