3055 Open source projects that are alternatives of or similar to Linkedin Profile Scraper

A curated list of awesome puppeteer resources.

A framework for creating semi-automatic web content extractors

Python爬虫代理IP池(proxy pool)

Crawler for Nintendo Switch eShop

operative framework is a OSINT investigation framework, you can interact with multiple targets, execute multiple modules, create links with target, export rapport to PDF file, add note to target or results, interact with RESTFul API, write your own modules.

Alternative for VK Audio API

带你了解一下Golang的市场行情

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

Unofficial MyAnimeList PHP+REST API which provides functions other than the official API

Node.js body parsing middleware

The Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.

API of DouYin for Humans used to Crawl Popular Videos and Musics

NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

JS逆向研究

A search application to explore, discover and share online files

一个超级轻量的百度图片爬虫

A multi-thread crawler framework with many builtin image crawlers provided.

🐾 Creeper - The Next Generation Crawler Framework (Go)

A distributed web crawler framework.（分布式爬虫框架XXL-CRAWLER）

✂️ High performance, multi-threaded image scraper

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

一款分布式爬虫平台，帮助你更好的管理和开发爬虫。 内置一套爬虫定义规则（模版），可使用模版快速定义爬虫，也可当作框架手动开发爬虫。(兴趣使然的项目，用的不爽了就更新)

zhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目

Dark Web OSINT Tool

A python web scraper for public email lists.

Fetches PubMed article IDs (PMIDs) from email inbox, then crawls PubMed, Google Scholar and Sci-Hub for respective PDF files.

Search for and retrieve US Patent and Trademark Office Patent Data

Rust Web Crawler saving pages on Redis

news-please - an integrated web crawler and information extractor for news that just works.

💐 Full Amazon Automatic Download

Gospider - Fast web spider written in Go

[DEPRECATED] Simple, flexible, delightful web crawler/spider package

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

Incredibly fast crawler designed for OSINT.

Tổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt

Beanbun 是用 PHP 编写的多进程网络爬虫框架，具有良好的开放性、高可扩展性，基于 Workerman。

Tool to fetch Instagram's public content.

Golang爬虫 爬取汽车之家 二手车产品库

Web crawling framework based on asyncio.

python crawler spider

Python爬虫，京东自动登录，在线抢购商品

Python的基础练习代码与各种爬虫代码

Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.

📙 中华新华字典数据库。包括歇后语，成语，词语，汉字。

🕸 Modular, multithreaded, puppeteer-based crawler

Golang pkg to quickly return a preview of a webpage (title/description/images)

Linkedin Learning videos downloader

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

The web browser that's built for scraping.

Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them in a file.

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

Crawl some picture for fun

Java 網路資料爬蟲包

抖音 SDK，数据采集，爬虫抓取不是梦

Async Python 3.6+ web scraping micro-framework based on asyncio

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

GOPA, a spider written in Go.（NOTE: this project moved to https://github.com/infinitbyte/gopa ）

OWASP D4N155 - Intelligent and dynamic wordlist using OSINT

A scalable web crawler framework for Java.