398 Open source projects that are alternatives of or similar to Scan T

🌭💦 91porn爬虫在线API接口（永久有效） 及 在线web预览

An extremely fast web scraper that parses megabytes of invalid HTML in a blink of an eye. PHP5.3+, no dependencies.

🎵 将可下载的网易云音乐的缓存文件转换为 MP3 文件

今日头条机器人，支持用户登陆、关注、取消关注、获取关注粉丝、发文、发悟空问答、点赞、评论、采集各种类型新闻讯息等，使用今日头条网页版API实现

DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework

👩 美女写真套图爬虫（一）

Intelligent proxy pool for Humans™ (Maintainer needed)

Social media (Weibo) comments analyzing toolbox in Chinese 微博评论分析工具, 实现功能: 1.微博评论数据爬取; 2.分词与关键词提取; 3.词云与词频统计; 4.情感分析; 5.主题聚类

一个灵活、友好的爬虫框架

小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对

Download videos from pornhub.

An R web crawler and scraper

Core part of NewPipe

知乎模拟登录，支持提取验证码和保存 Cookies

Crawly, a high-level web crawling & scraping framework for Elixir.

The DomCrawler component eases DOM navigation for HTML and XML documents.

各种App、小程序、网站的请求签名或加密算法。 现已有：自如、小红书、蛋壳公寓、luckin coffee(瑞幸咖啡)、bangkokair(曼谷航空)

Every web site provides APIs.

Crawler for Nintendo Switch eShop

Simple yet powerful automation stuffs.

A search engine that doesn't track you.

爬蟲極簡教學（fetch, parse, search, multiprocessing, API）- PTT 為例

本项目不再维护，详情可加群了解 https://t.me/onecomicbook

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

A non API python program to crawl public photos, posts or followers

swiss army knife for hackers

Web Application Security Scanner Framework

Open-source Enterprise Grade Search Engine Software

快速、简洁且强大的PHP爬虫框架

Crawl BookCorpus

🌭💦 91porn爬虫在线无限制API接口（永久有效，口令每日更新） 及 在线web预览

OSINT Swiss Army Knife

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

A framework for creating semi-automatic web content extractors

台灣上市上櫃股票爬蟲 Taiwan Stock Exchange Crawler

🍻 bilibili video (including bangumi) and danmaku downloader | B站视频（含番剧）、弹幕下载器

Android 本地网络小说爬虫，基于jsoup及xpath

Html网页正文提取

A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.

Script that crawls meta data from ICLR OpenReview webpage. Tutorials on installing and using Selenium and ChromeDriver on Ubuntu.

The fastest dork scanner written in Go.

🐨实时监控1000家中国企业的新闻动态

Crawl GitHub APIs and store the discovered orgs, repos, commits, ...

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

微信公众号爬虫，公众号历史文章，文章评论，文章阅读及在看数据，可视化web页面，可部署于Windows服务器。基于Python3之flask/mysql/redis/mitmproxy/pywin32等实现，高效微信爬虫，微信公众号爬虫，历史文章，文章评论，数据更新。

爬取菜鸟教程网站并转PDF__python_crawer_by_chrome

golang实现的爬虫框架，使用者只需关心页面规则，提供web管理界面。基于colly开发。

a reliable high-level web crawling & scraping framework for Node.js.

Crawlera middleware for Scrapy

LinkedIn Scraper (currently working 2020)

📰 A responsive interface of Hacker News with summaries and thumbnails.

台灣股票即時爬蟲。Taiwan Stock Exchange Real Time Crawler

Easily create XML sitemaps for your website.

Script that crawls meta data from ICLR OpenReview webpage. Tutorials on installing and using Selenium and ChromeDriver on Ubuntu.

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion

A collection of awesome web crawler,spider in different languages

Declarative web scraping

Python的基础练习代码与各种爬虫代码

What do people have in their dotfiles?

Crawler (Bot) searching for credential leaks on different paste sites.