Top 7 scrapyd open source projects

admin ui for scrapy/open source scrapinghub

✭ 2,562

python HTML javascript CSS dashboard spider scrapy scrapyd scrapy-ui scrapyd-ui scrapyd-dashboard

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

✭ 2,601

python Vue javascript SCSS HTML shell vue docker vuejs django dashboard spider distributed scrapy scrapyd webspider gerapy

Scrapydweb

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO 👉

ARGUS

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

✭ 68

python Jupyter Notebook Batchfile scraping crawling scrapy webscraping scrapyd webcrawling

SpiderManager

爬虫管理平台

✭ 27

python django scrapy scrapyd

scrapy-admin

A django admin site for scrapy

✭ 44

python HTML crawler spider scrapy scrapyd

logparser

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

✭ 70

python visualization scrapy log-parser scrapyd log-parsing log-analyse scrapy-log-analysis scrapyd-log-analysis

1-7 of 7 scrapyd projects