749 Open source projects that are alternatives of or similar to Reptile

ScrapingOutsourcing专注分享爬虫代码 尽量每周更新一个

收集各种免费的 Python 爬虫项目

日常爬虫

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

python爬虫小项目【持续更新】【笔趣阁小说下载、Tweet数据抓取、天气查询、网易云音乐逆向、天天基金网查询、微博数据抓取（生成cookie）、有道翻译逆向、企查查免登陆爬虫、大众点评svg加密破解、B站用户爬虫、拉钩免登录爬虫、自如租房字体加密、知乎问答

Creating Scrapy scrapers via the Django admin interface

Dynamic configurable crawl (动态可配置化爬虫)

163music spider by scrapy.

🎊 Design and implement of lightweight crawler framework.

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

🔧 🔩 🔨 收集整理了爬虫相关的工具、模拟登陆技术、代理IP、scrapy模板代码等内容。

JSpider会每周更新至少一个网站的JS解密方式，欢迎 Star，交流微信：13298307816

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO 👉

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

爬取百度贴吧、TapTap、appstore、微博官方博主上的游戏评论(基于redis_scrapy)，过滤器采用了bloomfilter。

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

NScrapy is a .net core corss platform Distributed Spider Framework which provide an easy way to write your own Spider

A web search engine built with Python which uses TF-IDF and PageRank to sort search results.

An open source webapp for scraping: towards a public service for webscraping

show full stack technology applications : Scrapy + webservice[restful] + websocket + VueJS + MongoDB

基于Scrapy框架的豆瓣电影爬虫

今日头条科技新闻接口爬虫

Analysis of the characteristics of different countries

Generating spiders dynamically to crawl and check those free proxy ip on the internet with scrapy.

Seeker - another job board aggregator.

A Facebook crawler

Simple but useful Python web scraping tutorial code.

💐Marmot | Web Crawler/HTTP protocol Download Package 🐭

基于python语言开发的千万级别搜索引擎

Free proxy server, continuously crawling and providing proxies, based on Tornado and Scrapy. 免费代理服务器，基于Tornado和Scrapy，在本地搭建属于自己的代理池

招聘网数据爬虫

admin ui for scrapy/open source scrapinghub

Iota is a web scraper which can find all of the images and links/suburls on a webpage

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

爬虫学习仓库，适合零基础的人学习，对新手比较友好

利昂图书馆预约系统自动预约&签到程序。支持包括中国人民大学、北京师范大学、济南大学、哈尔滨工业大学等在内的38所高校的图书馆系统

微博话题关键词,个人微博采集, 微博博文一键删除 selenium获取cookie,requests处理

百度贴吧生日爬虫，可抓取贴吧内吧友生日，并且在对应日期自动发送祝福

10 photo website spiders, 10 个国外图库的 scrapy 爬虫代码

免费 IP 代理池。Scrapy 爬虫框架插件

A collection of awesome web scaper, crawler.

A multi-thread crawler framework with many builtin image crawlers provided.

A python web scraper for public email lists.

A django admin site for scrapy

PTT 圖片下載器 (抓取整個看板的圖片，並用文章標題作為資料夾的名稱 ) (使用Scrapy)

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

百度贴吧爬虫(基于scrapy和mysql)

Uno Crawler por https://douban.com

A set of spiders and scrapers to extract location information from places that post their location on the internet.

V2EX爬虫

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

🍻 bilibili video (including bangumi) and danmaku downloader | B站视频（含番剧）、弹幕下载器

python爬虫

在线地址: http://119.23.223.90:8000

all kinds of scrapy demo

😩Tool For Taobao/Tmall| 儿时玩具已经过时

一个基于Scrapy的数据采集爬虫代码库

🎊 Design and implement of lightweight crawler framework.

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章

Word2vec 千人千面 个性化搜索 + Scrapy2.3.0(爬取数据) + ElasticSearch7.9.1(存储数据并提供对外Restful API) + Django3.1.1 搜索