GitPlanet
All Git Users → scrapy-plugins

12 open source projects by scrapy-plugins

[ Open user page on Github ]
1. Scrapy Splash
Scrapy+Splash for JavaScript integration
✭ 2,666
pythonscrapyheadless-browsers
2. Scrapy Deltafetch
Scrapy spider middleware to ignore requests to pages containing items seen in previous crawls
✭ 219
python
3. Scrapy Magicfields
Scrapy middleware to add extra fields to items, like timestamp, response fields, spider attributes etc.
✭ 48
python
4. Scrapy Djangoitem
Scrapy extension to write scraped items using Django models
✭ 471
python
5. Scrapy Crawlera
Crawlera middleware for Scrapy
✭ 281
pythonpluginproxycrawlerscrapyscraping
6. Scrapy Jsonrpc
Scrapy extension to control spiders using JSON-RPC
✭ 264
python
7. scrapy-querycleaner
Scrapy spider middleware to clean up query parameters in request URLs
✭ 21
python
8. scrapy-monkeylearn
A Scrapy pipeline to categorize items using MonkeyLearn
✭ 37
python
9. scrapy-zyte-smartproxy
Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
✭ 317
pythonplugincrawlerproxyscrapingscrapycrawler-detection
10. scrapy-jsonschema
Scrapy schema validation pipeline and Item builder using JSON Schema
✭ 42
python
11. scrapy-streaming
No description, website, or topics provided.
✭ 17
python
12. scrapy-pagestorage
A scrapy extension to store requests and responses information in storage service
✭ 24
python
1-12 of 12 user projects