Advanced web crawling and content extraction tool with multiple extraction modes

Web Crawl Skill

Advanced web content extraction with multiple modes and intelligent content detection.

When to Use

Use this skill when:

  • User asks to "研究" / "深度研究" a topic
  • User wants to "抓取" / "提取" content from websites
  • Need to analyze multiple web pages systematically
  • Current web_fetch output is insufficient

Extraction Modes

ModeUse Case
textClean plain text
markdownFormatted Markdown (recommended)
linksExtract all links
structuredJSON metadata + content
fullMarkdown + links combined

Tools Available

  • web_crawl - Extract content from a single URL
  • parallel_crawl - Extract from multiple URLs in parallel
  • research_topic - Multi-step research with search + crawl

Example Usage

text
User: "研究一下 OpenManus-Max 项目"
→ Use research_topic tool with query="OpenManus-Max GitHub features"