3180 Open source projects that are alternatives of or similar to Awesome Python Primer

A Laravel wrapper for CrawlerDetect - the web crawler detection library

爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer

Crawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

Continuous scalable web crawler built on top of Flink and crawler-commons

dynamic crawler for web vulnerability scanner

No description or website provided.

一个轻量级、快速、多线程、多管道、灵活配置的网络爬虫。

🍰 一个可视化的爬虫平台

Serverless Python

📖 A collection of pure POSIX sh alternatives to external processes.

京东口罩库存监控爬虫(非selenium)，扫码登录、查价、加购、下单、秒杀

一个灵活、友好的爬虫框架

[Unmaintained] A simple and clean video/music/image downloader 👾

Dark Web OSINT Tool

zhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目

golang实现的爬虫框架，使用者只需关心页面规则，提供web管理界面。基于colly开发。

🌭💦 91porn爬虫在线无限制API接口（永久有效，口令每日更新） 及 在线web预览

知乎模拟登录，支持提取验证码和保存 Cookies

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion

爬蟲極簡教學（fetch, parse, search, multiprocessing, API）- PTT 為例

🍻 bilibili video (including bangumi) and danmaku downloader | B站视频（含番剧）、弹幕下载器

各种App、小程序、网站的请求签名或加密算法。 现已有：自如、小红书、蛋壳公寓、luckin coffee(瑞幸咖啡)、bangkokair(曼谷航空)

OSINT Swiss Army Knife

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

Enseñando pensamiento computacional a partir de python one-liners

A framework for creating semi-automatic web content extractors

Web crawler.

Incredibly fast crawler designed for OSINT.

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

A Facebook crawler

A collection of awesome web crawler,spider in different languages

A distributed web crawler framework.（分布式爬虫框架XXL-CRAWLER）

Distributed crawler powered by Headless Chrome

API of DouYin for Humans used to Crawl Popular Videos and Musics

A lightweight Django project - because Django can be nearly as simple as a microframework

Code repository for Node.js Design Patterns Second Edition, published by Packt

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章

网上搜集的自学python语言的资料集合,包括整套代码和讲义集合，这是至今为止所开放网上能够查找到的最新视频教程，网上找不到其他最新的python整套视频了,. 具体的无加密的mp4视频教程和讲义集合可以在更新的Readme文件中找到，下载直接打开就能播放，项目从零基础的Python教程到深度学习，总共30章节，其中包含Python基础中的飞机大战项目，WSGI项目，Flask新经资讯项目， Django的电商项目(本应该的美多商城项目因为使用的是Vue技术，所以替换为Django天天生鲜项目)等等，希望能够帮助大家。资源搜集劳神费力，能帮到你的话是我的福分,望大家多多支持,喜欢本仓库的话，记得Star哦。

一个超级轻量的百度图片爬虫

Create beautiful Javascript charts with minimal code

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

📖 For those who wanna learn Bash

A fun project to store my learning from each book that I read.

The official Heroku buildpack for Python apps.

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

Gospider - Fast web spider written in Go

Word2vec 千人千面 个性化搜索 + Scrapy2.3.0(爬取数据) + ElasticSearch7.9.1(存储数据并提供对外Restful API) + Django3.1.1 搜索

Seeker - another job board aggregator.

A high performance web crawler in Elixir.

Scraping scripts for various websites.

📕TILed is medium to share your day to day learning with the world

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

A bunch of links to blog posts, articles, videos, etc for learning Rust

[DEPRECATED] Simple, flexible, delightful web crawler/spider package

《Django开发从入门到实战》书籍各章节源代码

🔍 简单的搜索引擎, django 框架

💐 Full Amazon Automatic Download

Universal scrapping tool, which allows you to extract data using multiple environments