All Categories → No Category → distributed-spider

Top 1 distributed-spider open source projects

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

✭ 38

python redis crawler kafka spider rabbitmq scraping crawling scrapy distributed-spider redisbloom rabbitmq-pipeline

1-1 of 1 distributed-spider projects