scrape
Installation
Summary
Fetch clean markdown, HTML, JSON, or screenshots from one or more URLs via Bright Data CLI.
- Supports single URLs, batched lists with parallelism, and paginated crawls; outputs markdown, HTML, JSON, or PNG screenshots
- Requires Bright Data CLI installation and authentication; skill verifies setup and guides login if needed
- Includes built-in block-page detection and escalation to browser automation for JavaScript-heavy or login-gated content
- Hands off to
data-feedsfor platform-specific extraction (Amazon, LinkedIn, TikTok, Instagram, YouTube, Reddit) and tosearchwhen URLs must be discovered first
SKILL.md
Bright Data — Scrape
Get clean content (markdown, HTML, JSON, screenshot) from one or more URLs via the Bright Data CLI. This skill owns the "fetch raw or lightly-structured content" job. For platform-specific structured data (Amazon, LinkedIn, TikTok, etc.), stop and use data-feeds instead — you'll get clean JSON without selector logic.
Setup gate (run first)
Before any scrape, verify the CLI is installed and authenticated:
if ! command -v bdata >/dev/null 2>&1; then
echo "bdata CLI not installed — see bright-data-best-practices/references/cli-setup.md"
elif ! bdata zones >/dev/null 2>&1; then
echo "bdata not authenticated — run: bdata login (or: bdata login --device for SSH)"
fi
If either check fails, halt and route the user to skills/bright-data-best-practices/references/cli-setup.md. Do not attempt the legacy curl fallback silently — ask the user first.