499 Open source projects that are alternatives of or similar to Strong Web Crawler

Crawl some picture for fun

京东口罩库存监控爬虫(非selenium)，扫码登录、查价、加购、下单、秒杀

抖音爬虫，tiktok crawler，抖音数据采集接口，抖音视频去水印，百分百成功，不需要服务器，不需要代理 IP。

golang light-weight image crawler

A Laravel wrapper for CrawlerDetect - the web crawler detection library

Alternative for VK Audio API

🍿爬虫代理IP池(proxy pool) python🍟一个还ok的IP代理池

A lightweight commandline tool for visual regression testing of UI components.

Gorecon is a All in one Reconnaissance Tool , a.k.a swiss knife for Reconnaissance , A tool that every pentester/bughunter might wanna consider into their arsenal

Headless Chrome For Java （Java 爬虫）

Determine if a page may be crawled from robots.txt, robots meta tags and robot headers

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

Capture screenshot of a website and return it as a stream

根据关键词抓取微博数据，再生成词云

Get (almost) original messages from google group archives. Your data is yours.

Dynamic meta tags in your AngularJS single page application

Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok

An API for your Kindle data

😱Full Automatic Amazon Distributed Spider | 亚马逊分布式四国际站采集选款产品|账号admin,密码adminadmin

📢 Ptt 文章通知機器人！Notify Ptt Article in Realtime

Easy to use lightweight web crawler（易用的轻量化网络爬虫）

CoCrawler is a versatile web crawler built using modern tools and concurrency.

Crawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites

💐Marmot | Web Crawler/HTTP protocol Download Package 🐭

简单、易用、高效 一个有态度的开源.Net Http请求框架!可以用制作爬虫，api请求等等。

Website style analyzer for designers. This is the service part that does the analyzation etc.

Enjoy driving on a Javascriptive (originally Pythonic) way to Japanese AV!

基于 Selenium 的知乎关键词爬虫

由Python编写的全异步实现的动漫之家(dmzj)漫画批量下载器（爬虫）

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.

A Composer Package which installs the PhantomJS binary (Linux, Windows, Mac) into /bin of your project.

链家网和贝壳网房价爬虫，采集北京上海广州深圳等21个中国主要城市的房价数据（小区，二手房，出租房，新房），稳定可靠快速！支持csv,MySQL, MongoDB,Excel, json存储，支持Python2和3，图表展示数据，注释丰富 ，点星支持，仅供学习参考，请勿用于商业用途，后果自负。

Official Algolia Plugin for Netlify. Index your website to Algolia when deploying your project to Netlify with the Algolia Crawler

Google play scraper for Python inspired by <facundoolano/google-play-scraper>

Collection of China illegal cases about web crawler 本项目用来整理所有中国大陆爬虫开发者涉诉与违规相关的新闻、资料与法律法规。致力于帮助在中国大陆工作的爬虫行业从业者了解我国相关法律，避免触碰数据合规红线。 [AD]中文知识图谱门户

To crawl all csgo skins from website.

crawler framework, distributed crawler extractor

A cross platform UI crawler which scans view trees then generate and execute UI test cases.

An Instagram bot developed using the Selenium Framework

An Open Source Search Engine

[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only.

pip安装的天眼查爬虫API，指定的单个/多个企业工商信息一键保存为Excel/JSON格式。A Battery-included Scraper API of Tianyancha, the best Chinese business data and investigation platform.

Crawl instagram photos, posts and videos for download.

대량의 뉴스 데이터를 수집하기 위해 만들어진 뉴스 크롤러입니다.

一个获取知乎用户主页信息的多线程Python爬虫程序。

네이버 뉴스 수집을 위한 도구

This repository is no longer maintained.

Python3 script to continuously download all images/webms of multiple 4chan thread simultaneously - without installation

本仓库收集整理爬虫相关资源，开发语言以Java为主

dynamic crawler for web vulnerability scanner

Elegant Scraper and Crawler Framework for Golang

用 node.js 爬你自己的 leetcode 解题源码

Website Cloner - Utilizes powerful Go routines to clone websites to your computer within seconds.

News, full-text, and article metadata extraction in Python 3. Advanced docs:

疫情数据爬虫，2019新型冠状病毒数据仓库，轨迹数据，同乘数据，报道

All in one tool for Information Gathering, Vulnerability Scanning and Crawling. A must have tool for all penetration testers

Norconex HTTP Collector is a flexible web crawler for collecting, parsing, and manipulating data from the Internet (or Intranet) to various data repositories such as search engines.

Simple news aggregator displaying top stories in real time

MM131网站图片爬取 🚨