1028 Open source projects that are alternatives of or similar to Weibo_terminator_workflow

🍭 epub generator for (light)novels (轻) 小说 epub 生成器，支持站点：轻之国度、轻小说文库

Golang pkg to quickly return a preview of a webpage (title/description/images)

Tổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt

Simple Weibo Scraper

Headless Chrome For Java （Java 爬虫）

Crawler for Nintendo Switch eShop

A high performance web crawler in Elixir.

[Unmaintained] A simple and clean video/music/image downloader 👾

This repository contains all the code I use in my YouTube tutorials.

Crawl instagram photos, posts and videos for download.

我的爬虫合集

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion

LinkedIn Scraper (currently working 2020)

Social media (Weibo) comments analyzing toolbox in Chinese 微博评论分析工具, 实现功能: 1.微博评论数据爬取; 2.分词与关键词提取; 3.词云与词频统计; 4.情感分析; 5.主题聚类

HTTP API for Scrapy spiders

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

A collection of awesome web crawler,spider in different languages

News, full-text, and article metadata extraction in Python 3. Advanced docs:

This repository is no longer maintained.

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

data resource untuk NLP bahasa indonesia

Universal scrapping tool, which allows you to extract data using multiple environments

crawler framework, distributed crawler extractor

快速、简洁且强大的PHP爬虫框架

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

OSINT Swiss Army Knife

An extremely fast web scraper that parses megabytes of invalid HTML in a blink of an eye. PHP5.3+, no dependencies.

Crawl BookCorpus

Crawly, a high-level web crawling & scraping framework for Elixir.

Declarative web scraping

An R web crawler and scraper

No description or website provided.

Distributed crawler powered by Headless Chrome

Az összes számlád még egy helyen :)

A Facebook crawler

Fetches PubMed article IDs (PMIDs) from email inbox, then crawls PubMed, Google Scholar and Sci-Hub for respective PDF files.

Scraping scripts for various websites.

Python爬虫，京东自动登录，在线抢购商品

Elegant Scraper and Crawler Framework for Golang

Node.js scraper to get data from Google Play

A web crawler (for bug hunting) that gathers more than you can imagine.

Google play scraper for Python inspired by <facundoolano/google-play-scraper>

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

Alternative for VK Audio API

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok

pip安装的天眼查爬虫API，指定的单个/多个企业工商信息一键保存为Excel/JSON格式。A Battery-included Scraper API of Tianyancha, the best Chinese business data and investigation platform.

👾 Fast and simple video download library and CLI tool written in Go

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

Be nice on the web

High performance crawler for Nodejs

The Engagement Meter calculates and shows engagement levels of an audience participating in a meeting

读书笔记《自己动手写网络爬虫》，自己敲的代码。主要记录了网络爬虫的基本实现，网页去重的算法，网页指纹算法，文本信息挖掘

A fast tool to fetch URLs from HTML attributes by crawl-in.