824 Open source projects that are alternatives of or similar to Arachnid

Tracking PG&E outages

基于Scrapy的QQ音乐爬虫(QQ Music Spider)，爬取歌曲信息、歌词、精彩评论等，并且分享了QQ音乐中排名前6400名的内地和港台歌手的49万+的音乐语料

Incredibly fast crawler designed for OSINT.

🍿爬虫代理IP池(proxy pool) python🍟一个还ok的IP代理池

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

htmlSQL is a experimental PHP library which allows you to access HTML values by an SQL like syntax.

💐 Full Amazon Automatic Download

Automatically add Accelerated Mobile Pages (AMP Project) functionality on your WordPress.

Google Search SERP Scraper

A curated list of awesome puppeteer resources.

Python Lambda Chrome Automation (naming pending)

A python3 crawler for crawling Pixiv ranking top and any illustrator all artworks

（已跑路）百度贴吧吧务管理工具，自动扫描帖子并处理违规帖

Find web directories without bruteforce

Polite, slim and concurrent web crawler.

A GUI client of schannel powered by therecipe/qt and golang

获取免费socks/https/http代理的网站集合

那些年，我爬过的北科。一个由浅入深的定向爬虫教程。

Headless Chrome For Java （Java 爬虫）

Web Crawler written in C#

《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题 ；HTTP协议介绍； Requests使用 ；解析器Xpath介绍； MongoDB与MySQL； 多线程爬虫； Scrapy介绍 ；Scrapy-redis介绍； 使用docker部署； 使用nomad管理docker集群； 使用EFK查询docker日志

[DEPRECATED] Simple, flexible, delightful web crawler/spider package

JS逆向研究

Crawler for leboncoin.fr

A Moodle downloader that downloads course content fast from Moodle (eg. lecture pdfs)

Search for and retrieve US Patent and Trademark Office Patent Data

Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok

用严肃的数据来回答“什么样的企业会到什么样的大学招聘”？

APIs for loginning some websites by using requests.

Google SEO研究及流量变现

Pythonic HTML Parsing for Humans™

Bringing Edge to your Web Performance ✨💥

Sample of using proxies to crawl baidu search results.

Seo Manager Package for Laravel ( with Localization )

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

Fetches PubMed article IDs (PMIDs) from email inbox, then crawls PubMed, Google Scholar and Sci-Hub for respective PDF files.

Viewers for statistics and dashboarding of Domain Search Engine data

Crawl websites for accessibility issues from the command line.

DownZemAll! is a download manager for Windows, MacOS and Linux

Simple CORPORA list crawler

🎬 电影资源爬虫,电影图片抓取脚本,Flask|Nginx|wsgi

爬取amazon/bestbuy/costco/6pm 的商品详情

📋 A simple website launch checklist to keep track of the most important enrichment possibilities for a website.

Site maintenance SEO PSR-15 middleware

Golang爬虫 爬取豆瓣电影Top250

[ Closed ] 🎨 image collector, which supports custom acquisition source configuration and is compatible with MacOS and Windows operating systems.

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

Android应用市场网络爬虫

🔥 Nextjs Headless WordPress

Scraping scripts for various websites.

Anime-dl is a command-line program to download anime from CrunchyRoll and Funimation.

🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

🧭 Declarative, asynchronous routing for React.

An Open Source Search Engine

A high performance skeleton starter for GatsbyJS that focuses on SEO/Social features/development environment.

Download comics novels 小说漫画下载工具 小説漫画のダウンローダ 小說漫畫下載:腾讯漫画 大角虫漫画 有妖气 知音漫客 咪咕 SF漫画 哦漫画 看漫画 漫画柜 汗汗酷漫 動漫伊甸園 快看漫画 微博动漫 733动漫网 大古漫画网 漫画DB 無限動漫 動漫狂 卡推漫画 动漫之家 动漫屋 古风漫画网 36漫画网 亲亲漫画网 乙女漫画 comico webtoons 咚漫 ニコニコ静画 ComicWalker ヤングエースUP モアイ pixivコミック サイコミ;アルファポリス カクヨム ハーメルン 小説家になろう 起点中文网 八一中文网 顶点小说 落霞小说网 努努书坊 笔趣阁→epub.

Vue.js Tutorial: A Prerendered, SEO-Friendly Example

A Jekyll plugin to add metadata tags for search engines and social networks to better index and display your site's content.

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

疫情数据爬虫，2019新型冠状病毒数据仓库，轨迹数据，同乘数据，报道