Skip to content
The Repo DudeGOOD FINDS. WITH RECEIPTS.

THE DUDE’S BRIEF

scrapy

scrapy/scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.

Repository & setup guide

What is scrapy used for?

Scrapy, a fast high-level web crawling & scraping framework for Python.

Summary of project documentation; not a hands-on benchmark.

Requirements and tradeoffs

Check the setup guide for supported systems and dependencies. Open-source code does not guarantee free hosting, model access, or third-party APIs.

Code license
BSD-3-Clause
Setup type
Developer library
Primary language
Python
GitHub stars
65,450 Observed 2026-11-17
Last code push
2026-11-16
License checked
2026-11-17

Model weights, datasets, dependencies, hosting and paid APIs can have separate terms or costs. Stars show attention, not quality, safety or earnings.

Where should I start?

Use the official repository and setup guide to verify prerequisites and installation steps. Review the code license before adoption. This page reflects a dated catalog observation; check the source for changes.

Metadata observed 2026-11-17. Brief reviewed 2026-11-17. Published by The Repo Dude.

Community conversations

Selected source links are not a sentiment score. How we select and label evidence

More in Browser & scraping

Related by category; these projects may serve different use cases.

firecrawl

firecrawl/firecrawl

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

AGPL-3.0 · TypeScript · 202,589 stars

Browser Use

browser-use/browser-use

Connect an AI agent to a web browser.

MIT · Python · 121,309 stars

Scrapling

D4Vinci/Scrapling

🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: and follow here for daily tips and tricks:

BSD-3-Clause · Python · 103,915 stars

Playwright

microsoft/playwright

Make browser tasks repeatable.

Apache-2.0 · TypeScript · 100,427 stars