1299 Open source projects that are alternatives of or similar to Ruiji.net

Plugin framework for PuppeteerSharp

News, full-text, and article metadata extraction in Python 3. Advanced docs:

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

This repository is no longer maintained.

A web crawler (for bug hunting) that gathers more than you can imagine.

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题 ；HTTP协议介绍； Requests使用 ；解析器Xpath介绍； MongoDB与MySQL； 多线程爬虫； Scrapy介绍 ；Scrapy-redis介绍； 使用docker部署； 使用nomad管理docker集群； 使用EFK查询docker日志

码云仓库链接:AJay13/ECommerceCrawlers Github 仓库链接:DropsDevopsOrg/ECommerceCrawlers 项目展示平台链接:http://wechat.doonsec.com

scrapy专利爬虫（停止维护）

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

Araignée LeBonCoin avec Scrapy et ElasticSearch

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

A scrapy spider that scraps times and prices from Voyages Sncf. It uses scrapyrt to provide an API interface.

No description or website provided.

Az összes számlád még egy helyen :)

我的爬虫合集

PTT 網路版爬蟲

An R web crawler and scraper

Update Version of weibo_terminator, This is Workflow Version aim at Get Job Done!

Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy

High performance crawler for Nodejs

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion

swiss army knife for hackers

OSINT Swiss Army Knife

快速、简洁且强大的PHP爬虫框架

Crawler for Nintendo Switch eShop

LinkedIn Scraper (currently working 2020)

A collection of awesome web crawler,spider in different languages

[Unmaintained] A simple and clean video/music/image downloader 👾

Github 仓库及用户分析爬虫

Simple but useful Python web scraping tutorial code.

This repository contains all the code I use in my YouTube tutorials.

基于搜狗微信搜索的微信公众号爬虫接口

Life is short, *****.

Redis-based components for Scrapy.

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

Google play scraper for Python inspired by <facundoolano/google-play-scraper>

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

A simple distributed crawler for zhihu && data analysis

Run Lighthouse in CI, as a web service, using Docker. Pass/Fail GH pull requests.

Collection of China illegal cases about web crawler 本项目用来整理所有中国大陆爬虫开发者涉诉与违规相关的新闻、资料与法律法规。致力于帮助在中国大陆工作的爬虫行业从业者了解我国相关法律，避免触碰数据合规红线。 [AD]中文知识图谱门户

Use Tor for your C# HTTP clients. Tor + Privoxy = ❤️

golang light-weight image crawler

A magic html parser

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

A cross platform UI crawler which scans view trees then generate and execute UI test cases.

네이버 뉴스 수집을 위한 도구

A Tumblr Backup Application

A .NET Plugin Framework.

A useful wrapper around Google Lighthouse CLI

🌈Wenshu_Spider-Scrapy框架爬取中国裁判文书网案件数据(2019-1-9最新版)

An open source implementation of Sid Meier's Civilization.

simple multi-level scraper json input/output for Cheerio

用 node.js 爬你自己的 leetcode 解题源码

A Light-ORM for accesing WMI

Sample Application DDD, Reactive Microservices, CQRS Event Sourcing Powered by DERMAYON LIBRARY

🍿爬虫代理IP池(proxy pool) python🍟一个还ok的IP代理池

疫情数据爬虫，2019新型冠状病毒数据仓库，轨迹数据，同乘数据，报道