405 Open source projects that are alternatives of or similar to Scylla

Az összes számlád még egy helyen :)

🌀 crawl bilibili user info and video info for data analysis | BiliBili爬虫

🍭 epub generator for (light)novels (轻) 小说 epub 生成器，支持站点：轻之国度、轻小说文库

🐛一只登录正方教务管理系统，爬取数据的小爬虫

We will try to find your visible basic footprint from social media as much as possible - 😤 more sites is comming soon

磁力網站U3C3介紹以及域名更新

Easy way to brute-force web directory.

Crawlera middleware for Scrapy

PTT 網路版爬蟲

我的爬虫合集

Source code for the IndieWeb search engine.

Auto Crawler Ptt Beauty Image Use Python Schedule

Web Application Security Scanner Framework

No description or website provided.

golang实现的爬虫框架，使用者只需关心页面规则，提供web管理界面。基于colly开发。

🍰 一个可视化的爬虫平台

This is a Multi-thread crawler for Tumblr.

Crawling CV conference papers with Python.

Crawl GitHub APIs and store the discovered orgs, repos, commits, ...

Fetching hotel data from TripAdvisor.

Recursive and multi-threaded broken link checker

Practical pinned gist which show your latest medium status 📌

📰 A responsive interface of Hacker News with summaries and thumbnails.

SSE 50 index options crawler 上证50期权数据爬虫

python requests + Django+ nodejs koa+ mysql to crawl eastmoney fund and stock data,for data analysis and visualiaztion .

读书笔记《自己动手写网络爬虫》，自己敲的代码。主要记录了网络爬虫的基本实现，网页去重的算法，网页指纹算法，文本信息挖掘

A simple and flexible web crawler framework for java.

An R web crawler and scraper

A fluent and functional approach to querying HTML

一个灵活、友好的爬虫框架

Crawl a website and take screenshots

line-bot-tutorial use python flask

TumblTwo, an Improved Fork of TumblOne, a Tumblr Downloader.

An extremely fast web scraper that parses megabytes of invalid HTML in a blink of an eye. PHP5.3+, no dependencies.

一个轻量级、快速、多线程、多管道、灵活配置的网络爬虫。

Update Version of weibo_terminator, This is Workflow Version aim at Get Job Done!

Videodl: A lightweight video downloader written by pure python.

爬蟲極簡教學（fetch, parse, search, multiprocessing, API）- PTT 為例

tw.pycon.org/2017/events/talk/314386410792550475/

The simple, easy to use command line web crawler.

无cookie版微博爬虫，可以连续爬取一个或多个新浪微博用户信息、用户微博及其微博评论转发。

Every web site provides APIs.

🐤️ Lost Ark wait notifier

A fast tool to fetch URLs from HTML attributes by crawl-in.

🤖 Prerendering for JavaScript powered websites. Great solution for PWAs (Progressive Web Apps), SPAs (Single Page Applications), and other websites based on top of front-end JavaScript frameworks

DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework

A Python DNS crawler to find identical domain names under different TLDs.

模拟登录各类网站，操作 API 完成各种不可描述的事情

Google search results crawler, get google search results that you need - php

Simple yet powerful automation stuffs.

High performance crawler for Nodejs

Simple Weibo Scraper

Continuous scalable web crawler built on top of Flink and crawler-commons

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

Just a crawler based on tg-cli for Telegram. Deprecated by now, please use telegram-export.

A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.

The fastest dork scanner written in Go.

微信公众号爬虫，公众号历史文章，文章评论，文章阅读及在看数据，可视化web页面，可部署于Windows服务器。基于Python3之flask/mysql/redis/mitmproxy/pywin32等实现，高效微信爬虫，微信公众号爬虫，历史文章，文章评论，数据更新。

Easily create XML sitemaps for your website.

🇰🇷 Realtime integrated information analysis service