398 Open source projects that are alternatives of or similar to Leboncoin Crawler

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

Simple but useful Python web scraping tutorial code.

Elasticsearch File System Crawler (FS Crawler)

🐾 Creeper - The Next Generation Crawler Framework (Go)

A Facebook crawler

Scraping scripts for various websites.

A multi-thread crawler framework with many builtin image crawlers provided.

Simple CORPORA list crawler

Fess is very powerful and easily deployable Enterprise Search Server.

Python脚本。模拟登录知乎， 爬虫，操作excel，微信公众号，远程开机

A high performance web crawler in Elixir.

带你了解一下Golang的市场行情

massive SQL injection vulnerability scanner

✌️ Python3 BitTorrent DHT crawler

Crawl websites for accessibility issues from the command line.

HTTP API for Scrapy spiders

A Tumblr Blog Backup Application

一个超级轻量的百度图片爬虫

Crawler used to crawl papers

NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

zhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目

基于搜狗微信搜索的微信公众号爬虫接口

爬取amazon/bestbuy/costco/6pm 的商品详情

The Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.

Life is short, *****.

Gospider - Fast web spider written in Go

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

[ Closed ] 🎨 image collector, which supports custom acquisition source configuration and is compatible with MacOS and Windows operating systems.

A tool for pixiv.net. 人人可用的P站爬虫

Fetches PubMed article IDs (PMIDs) from email inbox, then crawls PubMed, Google Scholar and Sci-Hub for respective PDF files.

A simple and flexible web crawler that follows the robots.txt policies and crawl delays.

Android应用市场网络爬虫

基金投资管理回测引擎

头条号爬虫案例

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

基于appium的app自动遍历工具

京东商品价格监控：监控用户设定商品价格，降价邮件/微信提醒。技术：Python爬虫/IP代理池/JS接口爬取/Selenium页面爬取

Sina stock options crawler with CSV output 新浪上证ETF期权数据爬虫

🎓 中国大学MOOC、学堂在线、网易云课堂、好大学在线、爱课程 MOOC 课程下载。

👧 美女写真套图爬虫（二）

Free Web Scraping Tool with Java

Google, Naver multiprocess image web crawler (Selenium)

API of DouYin for Humans used to Crawl Popular Videos and Musics

The Last Web Recon Tool You'll Need

A search application to explore, discover and share online files

Distributed Image Search Engine Crawler

A distributed web crawler framework.（分布式爬虫框架XXL-CRAWLER）

Crawls a website, gets PageSpeed Insights data for each page, and exports an HTML report.

Distributed crawler powered by Headless Chrome

Scrapy as a Web Crawler for Azure Search Samples

Redis-based components for Scrapy.

Dark Web OSINT Tool

支援 PTT 還有 PTT2 的 PTT API

获取最新可备案域名列表爬虫

📝 quickly crawl the information (e.g. followers, tags etc...) of an instagram profile.

🐞简单轻便的Java爬虫框架，只要会一点简单的正则表达式和简单的css选择器就能轻松的采集数据。

用严肃的数据来回答“什么样的企业会到什么样的大学招聘”？

Tor website crawler (specific for Alphabay at the time)

Implements the crawler package into Symfony

[Unmaintained] A simple and clean video/music/image downloader 👾