1540 Open source projects that are alternatives of or similar to Fbcrawl

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

Python爬虫代理IP池(proxy pool)

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

知乎模拟登录，支持提取验证码和保存 Cookies

A multi-thread crawler framework with many builtin image crawlers provided.

Creating Scrapy scrapers via the Django admin interface

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

A collection of awesome web crawler,spider in different languages

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

Crawly, a high-level web crawling & scraping framework for Elixir.

HTTP API for Scrapy spiders

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

[DEPRECATED] Simple, flexible, delightful web crawler/spider package

免费 IP 代理池。Scrapy 爬虫框架插件

🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

快速、简洁且强大的PHP爬虫框架

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

A web crawler (for bug hunting) that gathers more than you can imagine.

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

ScrapingOutsourcing专注分享爬虫代码 尽量每周更新一个

爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

An open source webapp for scraping: towards a public service for webscraping

A python web scraper for public email lists.

A high performance web crawler in Elixir.

crawler framework, distributed crawler extractor

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

Elegant Scraper and Crawler Framework for Golang

Scraping scripts for various websites.

💐Marmot | Web Crawler/HTTP protocol Download Package 🐭

High performance crawler for Nodejs

OSINT Swiss Army Knife

A framework for creating semi-automatic web content extractors

PTT 圖片下載器 (抓取整個看板的圖片，並用文章標題作為資料夾的名稱 ) (使用Scrapy)

Crawler for Nintendo Switch eShop

🐛一只登录正方教务管理系统，爬取数据的小爬虫

A collection of tools to (semi-)automatically collect and analyze data from online discussions on Facebook groups and pages.

Generating spiders dynamically to crawl and check those free proxy ip on the internet with scrapy.

我的爬虫合集

Simple Weibo Scraper

A fast tool to fetch URLs from HTML attributes by crawl-in.

🍭 epub generator for (light)novels (轻) 小说 epub 生成器，支持站点：轻之国度、轻小说文库

No description or website provided.

Uno Crawler por https://douban.com

Scrape a Facebook profile and turn it into a JSON file

百度贴吧爬虫(基于scrapy和mysql)

LinkedIn Scraper (currently working 2020)

磁力網站U3C3介紹以及域名更新

Python的基础练习代码与各种爬虫代码

一个基于webmagic框架二次开发的java爬虫框架实战，已实现能爬取腾讯，搜狐，今日头条（单独集成功能）等资讯内容，配合elasticsearch框架用法，实现了自动爬虫，已投入线上生产使用。

🔧 🔩 🔨 收集整理了爬虫相关的工具、模拟登陆技术、代理IP、scrapy模板代码等内容。

An R web crawler and scraper

Crawl BookCorpus

A set of spiders and scrapers to extract location information from places that post their location on the internet.

Html网页正文提取