AITeamGuide Hub · Agent
Crawl4AI
An open-source crawler that turns websites into clean Markdown for AI agents and RAG, with a self-hosted server and a hosted option.
Agent★ 86k starsArchived on GitHubApache-2.0ResearchSoftware development
Key facts
- Type
- Agent
- Jobs it fits
- Scrape websites; Feed web pages to an AI
- Fields
- Research, Software development
- Works with
- Python, Self-hosted
- GitHub
- unclecode/crawl4ai
- Licence
- Apache-2.0
- Stars
- 85,980 as of October 21, 2026
- Last updated
- October 21, 2026 (last push to GitHub)
- Author
- @unclecode
- Submitted by
- @aiteamguide
- Added to the Hub
- October 21, 2026
Install / Use it
Apache-2.0
Install
pip install -U crawl4aiSafety: Runs code from a third-party repo. Review the source and the permissions before installing.
Commands and configs are taken from the project's README and re-wrapped in each app's documented format. If they differ, the README wins.
Prefer a ready-made product?
Commercial agents for the same job, with real prices: Browser Use, all research agents, all browser & computer-use agents.
More from the Hub
-
Skill
Superpowers
A skills library and working method for coding agents: brainstorm, plan, test first, then build.
Software development★ 303k
-
Skill
Anthropic Agent Skills
Anthropic's public library of Agent Skills, including Word, Excel, PowerPoint and PDF skills and examples for building your own.
Personal productivity · Software development★ 183k
-
Agent
OpenCode
An open-source coding agent for the terminal and desktop that works with many model providers.
Software development★ 220k
-
Agent
MarkItDown
Microsoft's tool that converts PDFs, Word, PowerPoint, Excel and other files into Markdown an AI can read.
Personal productivity · Software development★ 191k
Summary written by AITeamGuide in our own words. GitHub numbers come from the public GitHub API and refresh daily. Maintainer and want a correction or removal? Tell us. Got something similar? Submit it to the Hub.