1170 Open source projects that are alternatives of or similar to Geziyor

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

Crawly, a high-level web crawling & scraping framework for Elixir.

Elegant Scraper and Crawler Framework for Golang

Declarative web scraping

OSINT Swiss Army Knife

A collection of awesome web crawler,spider in different languages

Free Web Scraping Tool with Java

自学入门 Python 优质中文资源索引，包含 书籍 / 文档 / 视频，适用于 爬虫 / Web / 数据分析 / 机器学习 方向

🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

A high performance web crawler in Elixir.

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer

Scraping scripts for various websites.

High performance crawler for Nodejs

Distributed crawler powered by Headless Chrome

[Unmaintained] A simple and clean video/music/image downloader 👾

A Facebook crawler

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

A web crawler (for bug hunting) that gathers more than you can imagine.

Universal scrapping tool, which allows you to extract data using multiple environments

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

No description or website provided.

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

Creating Scrapy scrapers via the Django admin interface

快速、简洁且强大的PHP爬虫框架

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion

Html网页正文提取

python crawler spider

Python的基础练习代码与各种爬虫代码

Crawl BookCorpus

LinkedIn Scraper (currently working 2020)

A framework for creating semi-automatic web content extractors

Crawler for Nintendo Switch eShop

Powerful web scraping framework for Crystal

🍻 bilibili video (including bangumi) and danmaku downloader | B站视频（含番剧）、弹幕下载器

Extract structured data from web sites. Web sites scraping.

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

The Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.

NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

带你了解一下Golang的市场行情

A distributed web crawler framework.（分布式爬虫框架XXL-CRAWLER）

API of DouYin for Humans used to Crawl Popular Videos and Musics

一个超级轻量的百度图片爬虫

✂️ High performance, multi-threaded image scraper

各种App、小程序、网站的请求签名或加密算法。 现已有：自如、小红书、蛋壳公寓、luckin coffee(瑞幸咖啡)、bangkokair(曼谷航空)

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

Simple but useful Python web scraping tutorial code.

A multi-thread crawler framework with many builtin image crawlers provided.

HTTP API for Scrapy spiders

Gospider - Fast web spider written in Go

Some classic web crawler projects.一些经典的爬虫

zhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目

Dark Web OSINT Tool

A python web scraper for public email lists.

Search for and retrieve US Patent and Trademark Office Patent Data

Fetches PubMed article IDs (PMIDs) from email inbox, then crawls PubMed, Google Scholar and Sci-Hub for respective PDF files.