Skip to content
The Repo DudeGOOD FINDS. WITH RECEIPTS.

THE DUDE’S FIELD GUIDE

Open-source Browser & scraping

Browser and scraping tools automate web interactions or extract structured information. Choose between full browser automation and lighter document parsing based on the target page.

How to choose

Check JavaScript rendering, session handling and rate controls. Respect site access rules and avoid collecting information you do not need.

Filter and compare 25 repos →

Repositories to explore

Ordered by observed stars. This is a directory sample, not a best-to-worst verdict. Individual briefs show observation dates and source links.

Browser & scraping comparison
Project and purposeCode licenseSetupObserved stars
firecrawl/firecrawl

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

AGPL-3.0Developer library202,5892026-11-17
browser-use/browser-use

Connect an AI agent to a web browser.

MITDeveloper tool121,3092026-11-17
D4Vinci/Scrapling

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: and follow here for daily tips and tricks:

BSD-3-ClauseDeveloper library103,9152026-11-17
microsoft/playwright

Make browser tasks repeatable.

Apache-2.0Developer tool100,4272026-11-17
puppeteer/puppeteer

Control a browser with JavaScript.

Apache-2.0Developer tool95,8532026-11-17
unclecode/crawl4ai

Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers…

Apache-2.0Self-hosted service87,4802026-11-17
scrapy/scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

BSD-3-ClauseDeveloper library65,4502026-11-17
JCodesMore/ai-website-cloner-template

Clone any website with one command using AI coding agents

MITSee setup guide39,3682026-11-17
dgtlmoon/changedetection.io

Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, and website defacement monitoring—…

Apache-2.0See setup guide35,8672026-11-17
SeleniumHQ/selenium

Automate browser tests across your stack.

Apache-2.0Developer tool34,5942026-11-17
CloakHQ/CloakBrowser

Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 50+ tests passed.

MITSee setup guide32,9542026-11-17
apify/crawlee

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other…

Apache-2.0Developer library26,4822026-11-17
browserbase/stagehand

The SDK to extract data and interact with any site on the web. Get started with Claude Code, Codex, Eve, Mastra, and more.

MITDeveloper library26,0832026-11-17
lexmount/moli

Best headless browser for AI agents. Lite, Fast, High-Compatibility. Built in Rust

Apache-2.0See setup guide23,9212026-11-17
Evil0ctal/Douyin_TikTok_Download_API

🚀 自托管的抖音、TikTok、B 站数据采集与无水印视频下载 API,支持 AI agent 通过 MCP 调用与离线语音转文字,Docker 一键部署。| Self-hosted TikTok, Douyin & Bilibili scraper and no-watermark video downloader — async REST API, MCP server for AI agents,…

Apache-2.0MCP server21,4682026-11-17
getmaxun/maxun

Turn any website into a structured API. Extract, automate, search and monitor the web.

AGPL-3.0See setup guide18,4982026-11-17
MODSetter/SurfSense

Air-gapped, privacy-focused, open-source NotebookLM alternative. AI agent that researches, transforms, and edits your documents. Join our Discord:

Apache-2.0See setup guide16,8932026-11-17
yusufkaraaslan/Skill_Seekers

Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection

MITSee setup guide15,3292026-11-17
nanobrowser/nanobrowser

Open-source AI agent for web automation. Use your real, logged-in browser to automate daily tasks.

Apache-2.0See setup guide14,3342026-11-17
seleniumbase/SeleniumBase

📊 Blazing fast Python framework for web crawling, testing, scraping, and stealth. Supports pytest. Bypasses bot-detection with CDP Mode. Compatible with Playwright.

MITDeveloper library13,2662026-11-17
jo-inc/camofox-browser

Stealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.

MITSee setup guide12,3532026-11-17
jhy/jsoup

jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.

MITSee setup guide11,4182026-11-17
NopeCHALLC/nopecha-extension

Automated CAPTCHA solver for your browser. Works with Selenium, Puppeteer, Playwright, and more.

MITSee setup guide11,1342026-11-17
pinchtab/pinchtab

High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.

MITSee setup guide10,5882026-11-17
apify/crawlee-python

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand.…

Apache-2.0Developer library9,6772026-11-17

Is open source the same as free to run?

No. The code license and operating costs are separate. Check model terms, hardware, hosting and paid services in each project’s documentation.

Read our sourcing and ranking methodology →