876 Open source projects that are alternatives of or similar to flink-crawler

Web crawler.

📰 A responsive interface of Hacker News with summaries and thumbnails.

知乎模拟登录，支持提取验证码和保存 Cookies

爬蟲極簡教學（fetch, parse, search, multiprocessing, API）- PTT 為例

A web crawler for Go

dynamic crawler for web vulnerability scanner

🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

京东口罩库存监控爬虫(非selenium)，扫码登录、查价、加购、下单、秒杀

A web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.

Simple but useful Python web scraping tutorial code.

A multi-thread crawler framework with many builtin image crawlers provided.

A Facebook crawler

基于C#.NET+PhantomJS+Sellenium的高级网络爬虫程序。可执行Javascript代码、触发各类事件、操纵页面Dom结构。

web spider built by puppeteer, support task-queue and task-scheduling by decorators，support nedb / mongodb, support data visualization; 基于puppeteer的web爬虫框架，提供灵活的任务队列管理调度方案，提供便捷的数据保存方案（nedb/mongodb），提供数据可视化和用户交互的实现方案

直接通过链家 API 抓取数据的极速爬虫，宇宙最快~~ 🚀

项目已经移动至：https://github.com/BaiduSpider/BaiduSpider ！！ 一个爬取百度搜索结果的爬虫，目前支持百度网页搜索，百度图片搜索，百度知道搜索，百度视频搜索，百度资讯搜索，百度文库搜索，百度经验搜索和百度百科搜索。

Cross Platform C# Web crawler framework, headless browser, parallel crawler. Please star this project! +1.

大数据生态解决方案数据平台：基于大数据、数据平台、微服务、机器学习、商城、自动化运维、DevOps、容器部署平台、数据平台采集、数据平台存储、数据平台计算、数据平台开发、数据平台应用搭建的大数据解决方案。

Easy way to brute-force web directory.

Apache Nutch is an extensible and scalable web crawler

A simple but powerful web crawler library for .NET

ScrapingOutsourcing专注分享爬虫代码 尽量每周更新一个

多线程知乎用户爬虫，基于python3

A Laravel wrapper for CrawlerDetect - the web crawler detection library

Google search results crawler, get google search results that you need

An interactive Command-Line Interface Build in NodeJS for downloading a single or multiple images to disk from URL

利昂图书馆预约系统自动预约&签到程序。支持包括中国人民大学、北京师范大学、济南大学、哈尔滨工业大学等在内的38所高校的图书馆系统

A web spider for shodan.io without using the Developer API.

Easily crawl news portals or blog sites using Storm Crawler.

正方教务管理系统学生版的自动登录、选课、信息获取

FofaMap是一款基于Python3开发的跨平台FOFA数据采集器，支持网站图标查询、批量查询和自定义查询FOFA数据，能够根据查询结果自动去重并生成对应的Excel表格。另外春节特别版还可以调用Nuclei对目标进行漏洞扫描，让你在挖洞路上快人一步。

团队分享学习、复盘笔记资料共享。Java、Scala、Flink...

带图形界面的哔哩哔哩漫画下载工具

动漫之家漫画站电脑版原图爬虫

A crawler based on Phantom. Allows discovery of dynamic content and supports custom scrapers.

百度贴吧生日爬虫，可抓取贴吧内吧友生日，并且在对应日期自动发送祝福

A bash script to spider a site, follow links, and fetch urls (with built-in filtering) into a generated text file.

download document from book118 for free

项目已经过时！无法应用在改版后的网站上。

扫描“微信读书”已购图书并下载本地PDF的爬虫

PTT 表特版爬蟲圖片下載器

No description or website provided.

利用墨墨背单词的分享功能拿每日20个的单词上限奖励（多线程

Flink Forward China 2018 第一届记录，视频记录 | 文档记录 | 不仅仅是流计算 | More than streaming

Get movie info from douban(豆瓣) and display in your terminal

客户端爬虫安卓端demo

some small project and some articles

Get Aliexpress product details in JSON

爬虫管理系统，支持集群，弹性伸缩。支持运行feapder、scrapy、selenium、playwright等各种框架及脚本

flink 流处理源码分析

서울시 공무원의 업무추진비를 분석하여 진짜 맛집 찾기 프로젝트

Example setup to demonstrate Prometheus integration of Apache Flink

A DHT Spider allows you to sniff the torrents and magnets.You can download them directly.

An open source webapp for scraping: towards a public service for webscraping

🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音|TikTok数据爬取工具，支持API调用，在线批量解析及下载。

日常爬虫

千万级图片爬虫、视频爬虫 [开源版本] Image Spider

Multi docker container images for main Big Data Tools. (Hadoop, Spark, Kafka, HBase, Cassandra, Zookeeper, Zeppelin, Drill, Flink, Hive, Hue, Mesos, ... )

Command Line Tool to download torrents

DataFibers Data Service