398 Open source projects that are alternatives of or similar to Auto Lighthouse

Crawl websites for accessibility issues from the command line.

massive SQL injection vulnerability scanner

The fast website crawler

Crawler used to crawl papers

zhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目

🔍 简单的搜索引擎, django 框架

爬取amazon/bestbuy/costco/6pm 的商品详情

新浪微博爬虫，用python爬取新浪微博数据，并下载微博图片和微博视频

A Tumblr Blog Backup Application

news-please - an integrated web crawler and information extractor for news that just works.

用严肃的数据来回答“什么样的企业会到什么样的大学招聘”？

Life is short, *****.

Discover hidden deepweb pages

Tor website crawler (specific for Alphabay at the time)

Tổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt

Simple CORPORA list crawler

Find web directories without bruteforce

Implements the crawler package into Symfony

Leetcode Contest Ranking Searcher

Scraping scripts for various websites.

File system crawler, disk space usage, file search engine and file system analytics powered by Elasticsearch

Elasticsearch File System Crawler (FS Crawler)

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

Python脚本。模拟登录知乎， 爬虫，操作excel，微信公众号，远程开机

抖音爬虫. 通过手机代理爬取用户的作品和用户的喜欢

Google, Naver multiprocess image web crawler (Selenium)

📝 quickly crawl the information (e.g. followers, tags etc...) of an instagram profile.

💐 Full Amazon Automatic Download

头条号爬虫案例

基于NodeJS的基金数据爬虫，爬取的数据存于github的@nullpointer/fund-data。

Scrapy as a Web Crawler for Azure Search Samples

Rust Web Crawler saving pages on Redis

Fetches PubMed article IDs (PMIDs) from email inbox, then crawls PubMed, Google Scholar and Sci-Hub for respective PDF files.

哔咔漫画收藏夹下载程序

Sina stock options crawler with CSV output 新浪上证ETF期权数据爬虫

A python3 crawler for crawling Pixiv ranking top and any illustrator all artworks

Distributed Image Search Engine Crawler

A Strong, Fast and Flexible Pixiv Client based on .NET Core and WPF

获取最新可备案域名列表爬虫

A GUI client of schannel powered by therecipe/qt and golang

[ Closed ] 🎨 image collector, which supports custom acquisition source configuration and is compatible with MacOS and Windows operating systems.

Golang爬虫 爬取汽车之家 二手车产品库

Android应用市场网络爬虫

那些年，我爬过的北科。一个由浅入深的定向爬虫教程。

基于appium的app自动遍历工具

Incredibly fast crawler designed for OSINT.

👧 美女写真套图爬虫（二）

Web Crawler written in C#

The Last Web Recon Tool You'll Need

This package is a complete tool for creating a large dataset of images (specially designed -but not only- for machine learning enthusiasts). It can crawl the web, download images, rename / resize / covert the images and merge folders..

Crawls a website, gets PageSpeed Insights data for each page, and exports an HTML report.

[DEPRECATED] Simple, flexible, delightful web crawler/spider package

Dark Web OSINT Tool

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

Crawler for leboncoin.fr

A powerful dynamic crawler for web vulnerability scanners

自学入门 Python 优质中文资源索引，包含 书籍 / 文档 / 视频，适用于 爬虫 / Web / 数据分析 / 机器学习 方向

Get the lyrics for the song currently playing on Spotify

vulnx 🕷️ is an intelligent bot auto shell injector that detect vulnerabilities in multiple types of cms { `wordpress , joomla , drupal , prestashop .. `}

🐞简单轻便的Java爬虫框架，只要会一点简单的正则表达式和简单的css选择器就能轻松的采集数据。