3055 Open source projects that are alternatives of or similar to Linkedin Profile Scraper

A Scraper made 100% in Python using BeautifulSoup and Tor. It can be used to scrape both normal and onion links. Happy Scraping :)

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Supported databases: SQLite, MySQL, PostgreSQL. Supported headless clients: Puppeteer, Playwright, Cheerio, JSdom.

Golang爬虫 爬取汽车之家 二手车产品库

The web browser that's built for scraping.

Az összes számlád még egy helyen :)

Python爬虫代理IP池(proxy pool)

A simple and flexible web crawler framework for java.

A fast tool to fetch URLs from HTML attributes by crawl-in.

我的爬虫合集

The simple, easy to use command line web crawler.

Simple Weibo Scraper

An R web crawler and scraper

🕸 Modular, multithreaded, puppeteer-based crawler

Beanbun 是用 PHP 编写的多进程网络爬虫框架，具有良好的开放性、高可扩展性，基于 Workerman。

Crawl some picture for fun

An interactive Command-Line Interface Build in NodeJS for downloading a single or multiple images to disk from URL

golang实现的爬虫框架，使用者只需关心页面规则，提供web管理界面。基于colly开发。

An extremely fast web scraper that parses megabytes of invalid HTML in a blink of an eye. PHP5.3+, no dependencies.

爬蟲極簡教學（fetch, parse, search, multiprocessing, API）- PTT 為例

Scrapy Extension for monitoring spiders execution.

Python爬虫，京东自动登录，在线抢购商品

python crawler spider

Golang pkg to quickly return a preview of a webpage (title/description/images)

Tổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt

Some classic web crawler projects.一些经典的爬虫

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对

🌭💦 91porn爬虫在线无限制API接口（永久有效，口令每日更新） 及 在线web预览

各种App、小程序、网站的请求签名或加密算法。 现已有：自如、小红书、蛋壳公寓、luckin coffee(瑞幸咖啡)、bangkokair(曼谷航空)

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them in a file.

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

GOPA, a spider written in Go.（NOTE: this project moved to https://github.com/infinitbyte/gopa ）

Java 網路資料爬蟲包

抖音 SDK，数据采集，爬虫抓取不是梦

ScrapingOutsourcing专注分享爬虫代码 尽量每周更新一个

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

Node.js body parsing middleware

Crawl BookCorpus

OWASP D4N155 - Intelligent and dynamic wordlist using OSINT

NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

A search application to explore, discover and share online files

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

Async Python 3.6+ web scraping micro-framework based on asyncio

🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent

SEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.

爬取北大法宝网http://www.pkulaw.cn/Case/

zhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目

BaiduSpider，一个爬取百度搜索结果的爬虫，目前支持百度网页搜索，百度图片搜索，百度知道搜索，百度视频搜索，百度资讯搜索，百度文库搜索，百度经验搜索和百度百科搜索。

Golang爬虫 爬取豆瓣电影Top250

Search for and retrieve US Patent and Trademark Office Patent Data

Dark Web OSINT Tool

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

Node.js scraper to get data from Google Play

B站用户爬虫 好耶~是爬虫

一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

获取免费socks/https/http代理的网站集合

APIs for loginning some websites by using requests.