765 Open source projects that are alternatives of or similar to Crawler

CoCrawler is a versatile web crawler built using modern tools and concurrency.

让基于 Guzzle 的项目完美无缝兼容 Swoole 协程，支持：Guzzle、Elasticsearch client——来自宇润 PHP 全家桶

This repository is no longer maintained.

Weave loom fibers into your Clojure

[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only.

data resource untuk NLP bahasa indonesia

🚀 Complete Java - A to Z ║ 📚 Notes and Programs of all Important Concepts of Java - OOPS, Data Structures, Algorithms, Design Patterns & Development + Kotlin + Android 🔥

简单、易用、高效 一个有态度的开源.Net Http请求框架!可以用制作爬虫，api请求等等。

Determine if a page may be crawled from robots.txt, robots meta tags and robot headers

All in one tool for Information Gathering, Vulnerability Scanning and Crawling. A must have tool for all penetration testers

Write "Concurrency" section of "A Tour of Go" in Haskell

An Instagram bot developed using the Selenium Framework

Implementation of blocking (IO-Bound) cancellable java.util.concurrent.CompletionStage and related extensions to java.util.concurrent.ExecutorService-s

대량의 뉴스 데이터를 수집하기 위해 만들어진 뉴스 크롤러입니다.

dynamic server-side rendering using headless Chrome to effortlessly solve the SEO problem for modern javascript websites

🌈 Code samples of advanced features of Http4s in combination with some features of Fs2 not often seen.

RECIPE : high-performance, concurrent indexes for persistent memory (SOSP 2019)

Plasma Programming Language

A lite distributed Java spider framework :-)

Workflow based REST framework code generator

Google play scraper for Python inspired by <facundoolano/google-play-scraper>

Digger is a powerful and flexible web crawler implemented by pure golang

Enjoy driving on a Javascriptive (originally Pythonic) way to Japanese AV!

A list of fancy questions I've been asked during the interviews I had. Some of them I ask when interviewing people.

Execution primitives for C++

Go concurrent-safe, goroutine-safe, thread-safe queue

😱Full Automatic Amazon Distributed Spider | 亚马逊分布式四国际站采集选款产品|账号admin,密码adminadmin

A complete set of primitives for concurrency and reactive programming on Swift

An Open Source Search Engine

Multithreading download all HD photos / pictures from someone's Sina Weibo album.

Concurrent wrappers for native Ruby IO & Sockets.

由Python编写的全异步实现的动漫之家(dmzj)漫画批量下载器（爬虫）

一个获取知乎用户主页信息的多线程Python爬虫程序。

Dynamic meta tags in your AngularJS single page application

Python3 script to continuously download all images/webms of multiple 4chan thread simultaneously - without installation

🔥 Shadowsocks 账号爬虫

Website Cloner - Utilizes powerful Go routines to clone websites to your computer within seconds.

一些爬虫的代码

News, full-text, and article metadata extraction in Python 3. Advanced docs:

This repository contains all the code I use in my YouTube tutorials.

Collection of Swift playgrounds used in my posts: From functional aspects of Swift to C interoperability.

Ignareo the Carillon, a web spider template of ultimate concurrency built for leprechauns. Carillons as the best web spiders; Long live the golden years of leprechauns!

The Floyd programming language

中国大陆大学列表爬虫

The C++ Standard Library for Parallelism and Concurrency

Touhou Project random music video generator/player, crawling image and video from websites to generate MV.

MM131网站图片爬取 🚨

Declarative concurrency in Scala - The implementation of the chemical machine

A lightweight REST client that gives you full control of your resources

📢 Ptt 文章通知機器人！Notify Ptt Article in Realtime

微博超级话题爬虫，微博词频统计+情感分析+简单分类，新增肺炎超话爬取数据

[not yet functional] Multi-threaded minecraft. Performance over correctness. What could go wrong?

As you can see, a kuaishou crawler

Go process used to crawl websites

基于注解实现并行地依赖注入（数据聚合），可以看做 Spring Async 注解的升级版

根据关键词抓取微博数据，再生成词云

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

2019新型冠状病毒疫情实时爬虫及API | COVID-19/2019-nCoV Realtime Infection Crawler and API

A library for encapsulating asynchronous operations and managing concurrency for Vue and Composition API.

To crawl all csgo skins from website.