1738 Open source projects that are alternatives of or similar to Antch

Elegant Scraper and Crawler Framework for Golang

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

Scrapy, a fast high-level web crawling & scraping framework for Python.

一个灵活、友好的爬虫框架

No description or website provided.

Declarative web scraping

Simple but useful Python web scraping tutorial code.

Continuous scalable web crawler built on top of Flink and crawler-commons

Crawly, a high-level web crawling & scraping framework for Elixir.

The simple, easy to use command line web crawler.

Distributed crawler powered by Headless Chrome

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

[Unmaintained] A simple and clean video/music/image downloader 👾

A scalable web crawler framework for Java.

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

Easy way to brute-force web directory.

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

Crawlera middleware for Scrapy

Apify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

A framework for creating semi-automatic web content extractors

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

A collection of awesome web crawler,spider in different languages

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Crawl sites for RSS, Atom, and JSON feeds.

a reliable high-level web crawling & scraping framework for Node.js.

🐾 Creeper - The Next Generation Crawler Framework (Go)

Rust Web Crawler saving pages on Redis

An interactive Command-Line Interface Build in NodeJS for downloading a single or multiple images to disk from URL

Apache Nutch is an extensible and scalable web crawler

Scrapy Extension for monitoring spiders execution.

Screen scraping and web crawling framework

Extract structured data from web sites. Web sites scraping.

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

Python framework to scrape Pastebin pastes and analyze them

OWASP D4N155 - Intelligent and dynamic wordlist using OSINT

HTTP API for Scrapy spiders

Free Web Scraping Tool with Java

自学入门 Python 优质中文资源索引，包含 书籍 / 文档 / 视频，适用于 爬虫 / Web / 数据分析 / 机器学习 方向

A simple but powerful web crawler library for .NET

A simple distributed crawler for zhihu && data analysis

Powerful web scraping framework for Crystal

Cross Platform C# Web crawler framework, headless browser, parallel crawler. Please star this project! +1.

🐝 Web vertical crawler framework for fun

Parser and database to index the terpene profile of different strains of Cannabis from online databases

Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them in a file.

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

News, full-text, and article metadata extraction in Python 3. Advanced docs:

Go process used to crawl websites

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

简单易用的Python爬虫框架，QQ交流群：597510560

An Instagram bot developed using the Selenium Framework

A curated list of awesome puppeteer resources.

A web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.

Proxy pool. Finds and checks proxies with rest api for querying results. Can find over 25k proxies in under 5 minutes.

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

蓝天采集器是一款免费的数据采集发布爬虫软件，采用php+mysql开发，可部署在云服务器，几乎能采集所有类型的网页，无缝对接各类CMS建站程序，免登录实时发布数据，全自动无需人工干预！是网页大数据采集软件中完全跨平台的云端爬虫系统

Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.