1591 Open source projects that are alternatives of or similar to Awesome Web Scraping

🔥 A cross-platform build utility based on Lua

Pipeline module for parallel real-time data processing for machine learning models development and production purposes.

INFINI-GATEWAY(极限网关), a high performance and lightweight gateway written in golang, for elasticsearch and his friends.

You can use this act to monitor any page's content and get a notification when content changes.

Local development against a remote Kubernetes or OpenShift cluster

Cookiecutter template for data scientists working with Docker containers

🌎 🏨 Scrape Booking.com 🏨 🌎

Forward proxy plugin for the Caddy web server

Socks5 Proxy with Go Lang. support USER_ID/PASSWORD. able to bypass HTTPS(SNI) censorship

Proxy server for adding push to your API

🧮 SOCKS5/4/4a 🌾 validating proxy pool and upstream SOCKS5 server for 🤽 LOLXDsoRANDum connections 🎋

Transfer 10Gbps http traffic over 1Gbps networks :)

gathertool是golang脚本化开发库，目的是提高对应场景程序开发的效率；轻量级爬虫库，接口测试&压力测试库，DB操作库等。

代理IP 采集程序

A cross-platform suite of tools for building and deploying software for iOS and other platforms.

A single node.js script to automatically inject user/password to http proxy server via a local forwarder

Tableau scraper python library. R and Python scripts to scrape data from Tableau viz

A simple and lightweight JavaScript data processing tool. Live demo:

mfsBSD

Web scraping Page Objects core library

Apify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.

YAStack: User-space network-stack based on DPDK, FreeBSD TCP/IP Stack, EnvoyProxy

Fast, mature, secure DoH server proxy written in Rust (doh-proxy).

Crawlzone is a fast asynchronous internet crawling framework for PHP.

虚拟host解决方案，轻松实现两套host环境

一款功能强大的漏洞扫描器，子域名爆破使用aioDNS，asyncio异步快速扫描，覆盖目标全方位资产进行批量漏洞扫描，中间件信息收集，自动收集ip代理，探测Waf信息时自动使用来保护本机真实Ip，在本机Ip被Waf杀死后，自动切换代理Ip进行扫描，Waf信息收集(国内外100+款waf信息)包括安全狗，云锁，阿里云，云盾，腾讯云等，提供部分已知waf bypass 方案，中间件漏洞检测(Thinkphp,weblogic等 CVE-2018-5955,CVE-2018-12613,CVE-2018-11759等)，支持SQL注入, XSS, 命令执行,文件包含, ssrf 漏洞扫描, 支持自定义漏洞邮箱推送功能

Awesome service mesh - https://www.servicemesher.com/awesome-servicemesh/

No description or website provided.

MyBB CAPTCHA Solver using Convolutional Neural Network in Keras

Makefile & supporting patches/scripts to build a bare metal ARM EABI toolchain.

Basic setup with random user agents and IP addresses for Python Scrapy Framework.

French translation of the Python documentation.

Make bootable Linux disk image abusing Docker

My personal study of blockchain related technology.

MockServer enables easy mocking of any system you integrate with via HTTP or HTTPS with clients written in Java, JavaScript and Ruby. MockServer also includes a proxy that introspects all proxied traffic including encrypted SSL traffic and supports Port Forwarding, Web Proxying (i.e. HTTP proxy), HTTPS Tunneling Proxying (using HTTP CONNECT) and…

A lightweight data processing framework for Apache Spark

Simple Golang SMTP relay/proxy server

✏️ A collection of React Native snippets for Sublime Text and Atom

High performance HTTP and reverse proxy server based on Node.js. 基于 Node.js 的高性能 HTTP 及反向代理服务器，类似nginx。

Public git conversion mirror of OpenBSD's official cvs ports repository. Pull requests not accepted - send diffs to the [email protected] mailing list.

C# Proxy html2canvas (aspx)

Selenium powered Python script to automate searching for vulnerable web apps.

A HTTP/S forwarding proxy in Go

Tool for port forwarding & intranet proxy

Community maintained packages for OpenWrt. Documentation for submitting pull requests is in CONTRIBUTING.md

[Python] Scraps comics and manga from various websites and creates cbz files from them

Example Makefile for Go projects

👾~ music, eternal ~ 👾

Project for learning Node.js internals

MTProxy autoinstaller for Ubuntu

📘 Making Data, the DataMade Way

High Performance HTTP Sidecar Load Balancer

PREVIEW - SQL databases in Bonobo, using sqlalchemy

Go实现的Trojan代理，支持多路复用/路由功能/CDN中转/Shadowsocks混淆插件，多平台，无依赖。A Trojan proxy written in Go. An unidentifiable mechanism that helps you bypass GFW. https://p4gefau1t.github.io/trojan-go/

Reverse proxy with simple routing configuration and override behaviour

CAPTCHA solving using Convolutional Neural Networks

⚡️一个轻巧的网络混淆代理🌏

3proxy - tiny free proxy server

Text Mining for Indonesian Online News Articles About Corona