398 Open source projects that are alternatives of or similar to Mmjpg

Easily create XML sitemaps for your website.

🍭 epub generator for (light)novels (轻) 小说 epub 生成器，支持站点：轻之国度、轻小说文库

Android 本地网络小说爬虫，基于jsoup及xpath

Crawlera middleware for Scrapy

🇰🇷 Realtime integrated information analysis service

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

磁力網站U3C3介紹以及域名更新

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion

我的爬虫合集

The fastest dork scanner written in Go.

golang实现的爬虫框架，使用者只需关心页面规则，提供web管理界面。基于colly开发。

🐛一只登录正方教务管理系统，爬取数据的小爬虫

🌭💦 91porn爬虫在线无限制API接口（永久有效，口令每日更新） 及 在线web预览

📰 A responsive interface of Hacker News with summaries and thumbnails.

台灣股票即時爬蟲。Taiwan Stock Exchange Real Time Crawler

Web Application Security Scanner Framework

台灣上市上櫃股票爬蟲 Taiwan Stock Exchange Crawler

This is a Multi-thread crawler for Tumblr.

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

Recursive and multi-threaded broken link checker

A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.

python requests + Django+ nodejs koa+ mysql to crawl eastmoney fund and stock data,for data analysis and visualiaztion .

swiss army knife for hackers

Source code for the IndieWeb search engine.

Crawl GitHub APIs and store the discovered orgs, repos, commits, ...

一个灵活、友好的爬虫框架

Crawl a website and take screenshots

今日头条机器人，支持用户登陆、关注、取消关注、获取关注粉丝、发文、发悟空问答、点赞、评论、采集各种类型新闻讯息等，使用今日头条网页版API实现

爬蟲極簡教學（fetch, parse, search, multiprocessing, API）- PTT 為例

小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对

DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework

知乎模拟登录，支持提取验证码和保存 Cookies

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

🎵 将可下载的网易云音乐的缓存文件转换为 MP3 文件

An R web crawler and scraper

🌭💦 91porn爬虫在线API接口（永久有效） 及 在线web预览

line-bot-tutorial use python flask

A non API python program to crawl public photos, posts or followers

Update Version of weibo_terminator, This is Workflow Version aim at Get Job Done!

The DomCrawler component eases DOM navigation for HTML and XML documents.

The simple, easy to use command line web crawler.

各种App、小程序、网站的请求签名或加密算法。 现已有：自如、小红书、蛋壳公寓、luckin coffee(瑞幸咖啡)、bangkokair(曼谷航空)

A fast tool to fetch URLs from HTML attributes by crawl-in.

Intelligent proxy pool for Humans™ (Maintainer needed)

模拟登录各类网站，操作 API 完成各种不可描述的事情

Crawler (Bot) searching for credential leaks on different paste sites.

Simple Weibo Scraper

Every web site provides APIs.

Just a crawler based on tg-cli for Telegram. Deprecated by now, please use telegram-export.

A search engine that doesn't track you.

读书笔记《自己动手写网络爬虫》，自己敲的代码。主要记录了网络爬虫的基本实现，网页去重的算法，网页指纹算法，文本信息挖掘

An extremely fast web scraper that parses megabytes of invalid HTML in a blink of an eye. PHP5.3+, no dependencies.

A fluent and functional approach to querying HTML

Download videos from pornhub.

Simple yet powerful automation stuffs.

🍻 bilibili video (including bangumi) and danmaku downloader | B站视频（含番剧）、弹幕下载器

Script that crawls meta data from ICLR OpenReview webpage. Tutorials on installing and using Selenium and ChromeDriver on Ubuntu.

a reliable high-level web crawling & scraping framework for Node.js.

快速、简洁且强大的PHP爬虫框架

微信公众号爬虫，公众号历史文章，文章评论，文章阅读及在看数据，可视化web页面，可部署于Windows服务器。基于Python3之flask/mysql/redis/mitmproxy/pywin32等实现，高效微信爬虫，微信公众号爬虫，历史文章，文章评论，数据更新。