832 Open source projects that are alternatives of or similar to weibo-scraper

无cookie版微博爬虫，可以连续爬取一个或多个新浪微博用户信息、用户微博及其微博评论转发。

A web crawler (for bug hunting) that gathers more than you can imagine.

Scraping scripts for various websites.

APIs for loginning some websites by using requests.

A Facebook crawler

A high performance web crawler in Elixir.

Golang pkg to quickly return a preview of a webpage (title/description/images)

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

Headless Chrome For Java （Java 爬虫）

Alternative for VK Audio API

Elegant Scraper and Crawler Framework for Golang

Crawler for Nintendo Switch eShop

Declarative web scraping

No description or website provided.

Distributed crawler powered by Headless Chrome

Az összes számlád még egy helyen :)

爬虫案例合集。包括但不限于《淘宝、京东、天猫、豆瓣、抖音、快手、微博、微信、阿里、头条、pdd、优酷、爱奇艺、携程、12306、58、搜狐、百度指数、维普万方、Zlibraty、Oalib、小说、招标网、采购网、小红书》

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

Fetches PubMed article IDs (PMIDs) from email inbox, then crawls PubMed, Google Scholar and Sci-Hub for respective PDF files.

This repository contains all the code I use in my YouTube tutorials.

根据关键词抓取微博数据，再生成词云

High performance crawler for Nodejs

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.

👾 Fast and simple video download library and CLI tool written in Go

Be nice on the web

LinkedIn Scraper (currently working 2020)

Crawl BookCorpus

A collection of awesome web crawler,spider in different languages

Crawly, a high-level web crawling & scraping framework for Elixir.

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

HTTP API for Scrapy spiders

[Unmaintained] A simple and clean video/music/image downloader 👾

Social media (Weibo) comments analyzing toolbox in Chinese 微博评论分析工具, 实现功能: 1.微博评论数据爬取; 2.分词与关键词提取; 3.词云与词频统计; 4.情感分析; 5.主题聚类

Tổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt

新浪微博爬虫，用python爬取新浪微博数据，并下载微博图片和微博视频

Python爬虫，京东自动登录，在线抢购商品

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

新浪微博相册大图多线程爬虫。

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

Node.js scraper to get data from Google Play

OSINT Swiss Army Knife

Google play scraper for Python inspired by <facundoolano/google-play-scraper>

This repository is no longer maintained.

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

News, full-text, and article metadata extraction in Python 3. Advanced docs:

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

Crawl instagram photos, posts and videos for download.

🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

微博超级话题爬虫，微博词频统计+情感分析+简单分类，新增肺炎超话爬取数据

crawler framework, distributed crawler extractor

Universal scrapping tool, which allows you to extract data using multiple environments

免登录下载微博图片 爬虫 Download Weibo Images without Logging-in

Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok

快速、简洁且强大的PHP爬虫框架

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion

Multithreading download all HD photos / pictures from someone's Sina Weibo album.

pip安装的天眼查爬虫API，指定的单个/多个企业工商信息一键保存为Excel/JSON格式。A Battery-included Scraper API of Tianyancha, the best Chinese business data and investigation platform.

A Fast and Powerful Scraping and Web Crawling Framework.

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.