antchfx / Antch
Licence: mit
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
Stars: ✭ 198
Projects that are alternatives of or similar to Antch
Scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
Stars: ✭ 42,343 (+21285.35%)
Mutual labels: crawler, scraping, crawling, framework
Colly
Elegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+7745.96%)
Mutual labels: crawler, scraping, crawling, framework
Gopa
[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
Stars: ✭ 277 (+39.9%)
Mutual labels: crawler, scraping, crawling, web-crawler
Webmagic
A scalable web crawler framework for Java.
Stars: ✭ 10,186 (+5044.44%)
Mutual labels: crawler, scraping, framework
Dotnetcrawler
DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c
Stars: ✭ 100 (-49.49%)
Mutual labels: crawler, scraping, crawling
bots-zoo
No description or website provided.
Stars: ✭ 59 (-70.2%)
Mutual labels: crawler, scraping, crawling
Spidy
The simple, easy to use command line web crawler.
Stars: ✭ 257 (+29.8%)
Mutual labels: crawler, crawling, web-crawler
Crawly
Crawly, a high-level web crawling & scraping framework for Elixir.
Stars: ✭ 440 (+122.22%)
Mutual labels: crawler, scraping, crawling
Linkedin Profile Scraper
🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (-13.64%)
Mutual labels: crawler, scraping, crawling
flink-crawler
Continuous scalable web crawler built on top of Flink and crawler-commons
Stars: ✭ 48 (-75.76%)
Mutual labels: crawler, web-crawler, crawling
Easy Scraping Tutorial
Simple but useful Python web scraping tutorial code.
Stars: ✭ 583 (+194.44%)
Mutual labels: crawler, scraping, crawling
Headless Chrome Crawler
Distributed crawler powered by Headless Chrome
Stars: ✭ 5,129 (+2490.4%)
Mutual labels: crawler, scraping, crawling
Lulu
[Unmaintained] A simple and clean video/music/image downloader 👾
Stars: ✭ 789 (+298.48%)
Mutual labels: crawler, scraping, crawling
D4n155
OWASP D4N155 - Intelligent and dynamic wordlist using OSINT
Stars: ✭ 105 (-46.97%)
Mutual labels: crawler, scraping
Skycaiji
蓝天采集器是一款免费的数据采集发布爬虫软件，采用php+mysql开发，可部署在云服务器，几乎能采集所有类型的网页，无缝对接各类CMS建站程序，免登录实时发布数据，全自动无需人工干预！是网页大数据采集软件中完全跨平台的云端爬虫系统
Stars: ✭ 1,514 (+664.65%)
Mutual labels: crawler, crawling
Awesome Puppeteer
A curated list of awesome puppeteer resources.
Stars: ✭ 1,728 (+772.73%)
Mutual labels: scraping, crawling
Squidwarc
Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head
Stars: ✭ 125 (-36.87%)
Mutual labels: crawler, crawling
Antch
Antch, inspired by Scrapy. If you're familiar with scrapy, you can quickly get started.
Antch is a fast, powerful and extensible web crawling & scraping framework for Go, used to crawl websites and extract structured data from their pages.
Get Started
Follow the Getting Started instructions to start your first spider.
Features
- Polite, highly concurrent web crawler.
- Powerful and customizable HTTP middleware.
- Item data pipeline for the web spider.
- Built-in proxy support (HTTP, HTTPS, SOCKS5).
- Built-in XPath query support for HTML/XML documents.
- Easy to use and integrate with your project.
Examples
BingWallpaper - Bing daily wallpaper.
Documentation
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].