1671 Open source projects that are alternatives of or similar to Webmagic

Scrapy, a fast high-level web crawling & scraping framework for Python.

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

Elegant Scraper and Crawler Framework for Golang

一个灵活、友好的爬虫框架

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

No description or website provided.

OWASP D4N155 - Intelligent and dynamic wordlist using OSINT

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

Transistor, a Python web scraping framework for intelligent use cases.

Distributed crawler powered by Headless Chrome

🐾 Creeper - The Next Generation Crawler Framework (Go)

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

Universal scrapping tool, which allows you to extract data using multiple environments

A framework for creating semi-automatic web content extractors

Crawlera middleware for Scrapy

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Declarative web scraping

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

[Unmaintained] A simple and clean video/music/image downloader 👾

🐝 Web vertical crawler framework for fun

A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

Crawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites

Simple but useful Python web scraping tutorial code.

Free Web Scraping Tool with Java

Crawly, a high-level web crawling & scraping framework for Elixir.

自学入门 Python 优质中文资源索引，包含 书籍 / 文档 / 视频，适用于 爬虫 / Web / 数据分析 / 机器学习 方向

Python framework to scrape Pastebin pastes and analyze them

A simple Go framework for building GitHub Apps

A reactive frontend framework for JavaScript

🌟 FastSitePHP 🌟 A Modern Open Source Framework for building High Performance Websites and API’s with PHP

一个开发友好、功能完备的开源微信商城框架

Rapid Application Development framework for Joomla!™ 3 and 4

Laravel package to collect your bank statements history. Currently support for parsing statements history from BCA, Mandiri, BNI, and MUAMALAT e-banking websites.

An extensible event-driven application framework in haskell

A web framework comprised of reusable, independent components

All the power of Pug, Sass, Coffeescript and WebPack in your WordPress theme. Stop writing themes like it's 1998.

A web crawler (for bug hunting) that gathers more than you can imagine.

爬虫, http代理, 模拟登陆!

A curated list of awesome things related to Seed

Titanium toolchain with superpower and wrapper around bugs.

Async Python 3.6+ web scraping micro-framework based on asyncio

Mount Everest Application Framework

A vaporwave CSS framework

Important! A new UI Kit version for Bootstrap 5 is available. Access the latest free version via the link below.

Lightweight Promises for Swift & Obj-C

In this project we will discuss working Selenium with cucumber

ASP.NET Boilerplate - Web Application Framework

A minimalist CSS framework.

Python scraper for Language Pods such as Japanesepod101.com 👹 🗾 🍣 Compatible with Japanese, Chinese, French, German, Italian, Korean, Portuguese, Russian, Spanish and many more! ✨

Java 網路資料爬蟲包

A strongly-typed genetic programming framework for Python

Offensive Web Testing Framework (OWTF), is a framework which tries to unite great tools and make pen testing more efficient http://owtf.org https://twitter.com/owtfp

Supervised domain-agnostic prediction framework for probabilistic modelling

golang template for gin framework!

SARL Agent-Oriented Programming Language http://www.sarl.io

Extensible Red Team Framework

Your high performance web application C framework

抖音 SDK，数据采集，爬虫抓取不是梦