desearch-crawl
Crawl Webpage By Desearch
Extract content from any webpage URL. Returns clean text or raw HTML.
Quick Start
- Get an API key from https://console.desearch.ai
- Set environment variable:
export DESEARCH_API_KEY='your-key-here'
Usage
# Crawl a webpage (returns clean text by default)
scripts/desearch.py crawl "https://en.wikipedia.org/wiki/Artificial_intelligence"
# Get raw HTML
scripts/desearch.py crawl "https://example.com" --crawl-format html
More from desearch-ai/desearch-openclaw-skills
desearch-web-search
Search the web and get real-time SERP-style results with titles, URLs, and snippets. Use this for general web queries when you need current links and information from across the internet.
201desearch-x-search
Search X (Twitter) in real time. Find posts by keyword, user, or hashtag. Get a user's timeline, replies, retweeters, or fetch specific posts by ID or URL. Supports advanced filters like date range, language, engagement thresholds, and media type.
20desearch-ai-search
AI-powered search that aggregates and summarizes results from multiple sources including web, X/Twitter, Reddit, Hacker News, YouTube, ArXiv, and Wikipedia. Use this when you need a synthesized answer or curated links from across the internet and social platforms.
8