ClawFetch.ai Web Intelligence API
Web intelligence API for AI agents. Stealth rendering, 17 structured extractors (Twitter, GitHub, CoinGecko, NPM, PyPI, crates.io, Hacker News, YouTube, Wikipedia, Product Hunt, IMDb, Amazon, Crunchbase, SEC EDGAR, Zillow, Redfin, news), domain search, and multi-source research. Gasless USDC payments on Base via standard X402 protocol. Use when your agent needs web data from JS-heavy or Cloudflare-protected sites, structured data from supported sites, domain availability checks, or multi-source topic research.
ClawFetch.ai — Web Intelligence for AI Agents
Base URL: https://api.clawfetch.ai
Setup Guide (For Agents)
Step 1: Create a Base Wallet
You need an Ethereum-compatible wallet. Generate one using ethers.js:
import { ethers } from 'ethers';
// Generate a new wallet
const wallet = ethers.Wallet.createRandom();
console.log('Address:', wallet.address);
console.log('Private Key:', wallet.privateKey);
// Save the private key securely!
Or install ethers if you don't have it:
npm install ethers
Step 2: Get USDC on Base
Ask your human to send a small amount of USDC to your wallet address on the Base network (Chain ID 8453).
- Even $0.50 covers hundreds of API calls (cheapest endpoint is $0.001)
- No ETH needed! ClawFetch uses gasless X402 payments — you only need USDC
- USDC contract on Base:
0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913
Step 3: Install the X402 Client SDK
npm install @x402/fetch @x402/evm
Step 4: Make Paid Requests
Use the @x402/fetch wrapper — it handles the entire payment flow automatically:
import { wrapFetch } from '@x402/fetch';
import { ExactEvmScheme } from '@x402/evm/exact/client';
import { ethers } from 'ethers';
// Set up your wallet
const provider = new ethers.JsonRpcProvider('https://mainnet.base.org');
const wallet = new ethers.Wallet('YOUR_PRIVATE_KEY', provider);
// Create an x402-aware fetch client
const x402Fetch = wrapFetch(fetch, {
schemes: [new ExactEvmScheme(wallet)],
});
// Now use it like normal fetch — payments happen automatically!
const response = await x402Fetch('https://api.clawfetch.ai/render', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ url: 'https://example.com' }),
});
const data = await response.json();
console.log(data);
That's it! The x402 SDK handles:
- Detecting the 402 payment requirement
- Signing a gasless EIP-3009 authorization (no ETH needed)
- Retrying the request with the payment signature
- Returning the API response
Alternative: Manual Flow (No SDK)
If you can't use the SDK, you can manually follow the X402 flow:
# 1. Hit endpoint → get 402 with PAYMENT-REQUIRED header (base64 JSON)
curl -i -X POST https://api.clawfetch.ai/fetch \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com"}'
# 2. Decode the PAYMENT-REQUIRED header to get payment details
# 3. Sign an EIP-3009 TransferWithAuthorization message
# 4. Base64-encode the signed payload
# 5. Resend with PAYMENT-SIGNATURE header
curl -X POST https://api.clawfetch.ai/fetch \
-H "Content-Type: application/json" \
-H "PAYMENT-SIGNATURE: <base64-encoded-signed-payload>" \
-d '{"url": "https://example.com"}'
Recommended: Use the SDK. The manual flow requires understanding EIP-3009 and EIP-712 typed data signing. The SDK does all of this for you.
Endpoints
POST /fetch — $0.001 (0.1¢)
Basic HTTP fetch → clean text. No JS rendering. Fast, cheap, works for static pages and APIs.
{"url": "https://docs.example.com", "maxChars": 5000}
POST /render — $0.005 (0.5¢)
Full Puppeteer render with stealth anti-detection. Handles Cloudflare, bot checks, JS-heavy SPAs.
{"url": "https://app.uniswap.org", "maxChars": 3000, "waitFor": ".swap-page"}
POST /extract — $0.008 (0.8¢)
Structured JSON extraction via site-specific parsers. Returns typed fields, not raw text.
{"url": "https://github.com/openclaw/openclaw"}
Response:
{"type": "github", "name": "openclaw", "stars": "2.4k", "forks": "312", "language": "TypeScript"}
POST /domains/check — $0.003 (0.3¢)
Check domain availability via WHOIS. Returns registrar, expiry, and creation dates for taken domains.
{"name": "myproject", "tlds": ["com", "ai", "io", "dev"]}
Or explicit list:
{"domains": ["foo.com", "bar.ai", "baz.io"]}
POST /domains/suggest — $0.003 (0.3¢)
Generate domain name ideas from keywords + check availability.
{"keywords": ["agent", "fetch"], "tlds": ["com", "ai"], "maxCheck": 20}
POST /parse — $0.005 (0.5¢)
Convert office documents (docx, pptx, xlsx, pdf, odt, ods, odp, rtf, epub, csv, doc, ppt) to clean GitHub-Flavored Markdown. Pass a document URL or base64 bytes. Deterministic local Rust converter — no OCR for scanned/image-only PDFs.
{"url": "https://example.com/quarterly-report.docx"}
Or inline bytes (max ~15MB):
{"base64": "<base64 document bytes>", "filename": "report.docx"}
POST /company: $0.02 (2¢)
Company / lead enrichment from a domain. Returns name, description, logo, favicon, language, social profiles (X/Twitter, LinkedIn, GitHub, YouTube, Facebook, Instagram, Telegram, Discord, Crunchbase), contact emails, phones, schema.org postal addresses and Organization data, detected tech stack (with evidence), and key page URLs (about, careers, pricing, contact, docs, blog, privacy, terms). Deterministic parse of the live homepage plus up to 2 same-site pages. No LLM, no stale database.
{"domain": "vercel.com"}
POST /research — $0.02 (2¢)
Multi-source topic research. Searches the web, fetches top results, returns structured summaries.
{"query": "Casper Network DeFi ecosystem", "maxResults": 5}
GET /health — Free
Service status, browser pool stats, cache stats, available extractors.
GET /extractors — $0.001 (0.1¢)
List all structured extractors with supported domains and output fields.
Available Extractors (17)
- twitter — x.com, twitter.com → author, text, likes, reposts, views, media
- coingecko — coingecko.com → name, price, marketCap, volume, rank
- github — github.com → name, stars, forks, language, readme
- news — any public article → title, author, date, body, source
- npm — npmjs.com → name, version, downloads, dependencies
- pypi — pypi.org → name, version, author, license, requires_python
- crates — crates.io → name, version, downloads, features
- hackernews — news.ycombinator.com → title, author, score, comments, url
- youtube — youtube.com, youtu.be → title, channel, duration, views
- wikipedia — wikipedia.org → title, summary, sections, references
- producthunt — producthunt.com → name, tagline, votes, topics, makers
- edgar — sec.gov → company profile, CIK, ticker, filings, XBRL financials
- imdb — imdb.com → title, rating, cast, plot, genres, runtime
- amazon — amazon.com → title, price, rating, reviews, features, BSR
- crunchbase — crunchbase.com → name, funding, founders, categories
- zillow — zillow.com → address, price, Zestimate, beds, baths, sqft
- redfin — redfin.com → address, price, estimate, beds, baths, sqft
Payment Details (X402 Protocol)
| Detail | Value |
|---|---|
| Protocol | X402 (standard, gasless) |
| Scheme | exact (EIP-3009 TransferWithAuthorization) |
| Network | Base mainnet (eip155:8453) |
| Token | USDC (0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913) |
| Receiving Wallet | 0xc2afB91833f4dE19542A025872C6085cC29185eB |
| Gas Required | None — the facilitator pays gas |
| Client Needs | USDC on Base + ability to sign EIP-712 messages |
Important: This is standard X402 — the same protocol used across the Coinbase ecosystem. If you already have an x402 client set up for other services, it works here too.
Cost Estimates
| Use Case | Endpoint | Cost | $1 Gets You |
|---|---|---|---|
| Scrape a page | /fetch | $0.001 | 1,000 pages |
| Render JS page | /render | $0.005 | 200 pages |
| Extract tweet data | /extract | $0.008 | 125 extractions |
| Parse a document | /parse | $0.005 | 200 documents |
| Check domain names | /domains/check | $0.003 | 333 checks |
| Enrich a company | /company | $0.02 | 50 company profiles |
| Research a topic | /research | $0.02 | 50 researches |
When to Use ClawFetch
web_fetchfails on Cloudflare/JS-rendered pages → use/render- Need structured data (tweet metrics, repo stats, token prices) → use
/extract - Building a project and need a domain name → use
/domains/checkor/domains/suggest - Need multi-source research in one call → use
/research - Need a company profile / lead enrichment from a domain → use
/company - Hitting rate limits on direct scraping → ClawFetch caches and pools browsers
Error Handling
400— Missing required fields (check request body)402— Payment required (use x402 SDK or follow manual flow)500— Server error (retry after a moment)- Cached responses return
"cached": true— you're not double-charged
Troubleshooting
"I don't have USDC" — Ask your human to send $0.50-$1 USDC on Base to your wallet. That covers hundreds of calls.
"I don't have ETH for gas" — You don't need any! X402 uses gasless EIP-3009 signatures. The facilitator handles gas.
"How do I check my USDC balance?"
import { ethers } from 'ethers';
const provider = new ethers.JsonRpcProvider('https://mainnet.base.org');
const usdc = new ethers.Contract(
'0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913',
['function balanceOf(address) view returns (uint256)'],
provider
);
const balance = await usdc.balanceOf('YOUR_WALLET_ADDRESS');
console.log('USDC:', ethers.formatUnits(balance, 6));
- Downloads
- 187All time
- Security audit
- Review
- Last updated
- 3w ago
- Current version
- v2.0.0
- License
- MIT-0
