495 Open source projects that are alternatives of or similar to Weibopicdownloader

Secret and/ credential patterns used for gf.

知乎模拟登录，支持提取验证码和保存 Cookies

A lite distributed Java spider framework :-)

🌭💦 91porn爬虫在线API接口（永久有效） 及 在线web预览

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

The DomCrawler component eases DOM navigation for HTML and XML documents.

新闻抓取（微信、微博、头条...）

Android 本地网络小说爬虫，基于jsoup及xpath

Automatically constructing corpus for automatic speech recognition from YouTube videos

A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.

2019新型冠状病毒疫情实时爬虫及API | COVID-19/2019-nCoV Realtime Infection Crawler and API

Every web site provides APIs.

File system crawler, disk space usage, file search engine and file system analytics powered by Elasticsearch

Social OAuth Authentication for Laravel 5. drivers: facebook, github, google, linkedin, weibo, qq, wechat and douban

腾讯新闻、知乎话题、微博粉丝，Tumblr爬虫、斗鱼弹幕、妹子图爬虫、分布式设计等

Crawl GitHub APIs and store the discovered orgs, repos, commits, ...

tumblr解析网站

微信公众号爬虫，公众号历史文章，文章评论，文章阅读及在看数据，可视化web页面，可部署于Windows服务器。基于Python3之flask/mysql/redis/mitmproxy/pywin32等实现，高效微信爬虫，微信公众号爬虫，历史文章，文章评论，数据更新。

dynamic server-side rendering using headless Chrome to effortlessly solve the SEO problem for modern javascript websites

golang实现的爬虫框架，使用者只需关心页面规则，提供web管理界面。基于colly开发。

An online tool (crawler) to analyze users performance in online judges (coding competition websites). Supported OJ: POJ, HDU, ZOJ, HYSBZ, CodeForces, UVA, ICPC Live Archive, FZU, SPOJ, Timus (URAL), LeetCode_CN, CSU, LibreOJ, 洛谷, 牛客OJ, Lutece (UESTC), AtCoder, AIZU, CodeChef, El Judge, BNUOJ, Codewars, UOJ, NBUT, 51Nod, DMOJ, VJudge

爬蟲極簡教學（fetch, parse, search, multiprocessing, API）- PTT 為例

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework

Download comics novels 小说漫画下载工具 小説漫画のダウンローダ 小說漫畫下載:腾讯漫画 大角虫漫画 有妖气 知音漫客 咪咕 SF漫画 哦漫画 看漫画 漫画柜 汗汗酷漫 動漫伊甸園 快看漫画 微博动漫 733动漫网 大古漫画网 漫画DB 無限動漫 動漫狂 卡推漫画 动漫之家 动漫屋 古风漫画网 36漫画网 亲亲漫画网 乙女漫画 comico webtoons 咚漫 ニコニコ静画 ComicWalker ヤングエースUP モアイ pixivコミック サイコミ;アルファポリス カクヨム ハーメルン 小説家になろう 起点中文网 八一中文网 顶点小说 落霞小说网 努努书坊 笔趣阁→epub.

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

简单、易用、高效 一个有态度的开源.Net Http请求框架!可以用制作爬虫，api请求等等。

Easily create XML sitemaps for your website.

It makes a preview from an URL, grabbing all the information such as title, relevant texts and images.

Web Application Security Scanner Framework

PhotoBrowser is a light weight photo browser, like the wechat, weibo image viewer.

💬 一个轻量化的留言板 / 记事本 / 社交系统 / 博客。人类的本质是……咕咕咕？

POOPAK - TOR Hidden Service Crawler

🔧 🔩 🔨 收集整理了爬虫相关的工具、模拟登陆技术、代理IP、scrapy模板代码等内容。

Enjoy driving on a Javascriptive (originally Pythonic) way to Japanese AV!

This is a Multi-thread crawler for Tumblr.

It covers the blockade principle of most anti-climbing strategies and corresponding solutions.👽👽👽👽（涵盖了大部分的反爬策略的封锁原理以及对应的解决方案。）

🍭 epub generator for (light)novels (轻) 小说 epub 生成器，支持站点：轻之国度、轻小说文库

Flutter仿微博客户端, 包含首页、视频、发现、消息(仿微博聊界面)及个人中心模块

Recursive and multi-threaded broken link checker

Some classic web crawler projects.一些经典的爬虫

由Python编写的全异步实现的动漫之家(dmzj)漫画批量下载器（爬虫）

As you can see, a kuaishou crawler

Web Crawler written in C#

python requests + Django+ nodejs koa+ mysql to crawl eastmoney fund and stock data,for data analysis and visualiaztion .

Golang pkg to quickly return a preview of a webpage (title/description/images)

🇰🇷 Realtime integrated information analysis service

👾 Fast and simple video download library and CLI tool written in Go

Source code for the IndieWeb search engine.

Python爬虫，京东自动登录，在线抢购商品

微博分享、微信分享、qq分享，微信支付、支付宝支付 qq登录、微信登录、支付宝登录，直接引用官方提供api 安全省心

🔥 Shadowsocks 账号爬虫

python crawler spider

直接通过链家 API 抓取数据的极速爬虫，宇宙最快~~ 🚀

MonkeyKing helps you to post messages to Chinese Social Networks.

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.

Crawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites

Simple news aggregator displaying top stories in real time

네이버 뉴스 수집을 위한 도구

news-please - an integrated web crawler and information extractor for news that just works.