139 Open source projects that are alternatives of or similar to pumba

[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library

Server written in PHP, provides a Javascript API for in the browser

Angular module to detect OS / Browser / Device

Easily crawl news portals or blog sites using Storm Crawler.

A webview integrated w/ native features to help create most advanced hybrid applications.

Sawmill is a JSON transformation Java library

😎 Google chrome browser extension

Aplus Framework HTTP Library

A dataset for textual analysis on arguably the best written comedy television show ever.

Get your lists of User-Agent Strings here

A precise user agent parser and device detector written in TypeScript

cdp4j - Chrome DevTools Protocol for Java

This snippet illustrates how to reconstruct the legacy navigator.userAgent string value from the modern navigator.userAgentData values.

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

The Universal Device Detection library will parse any User Agent and detect the browser, operating system, device used (desktop, tablet, mobile, tv, cars, console, etc.), brand and model.

네이버 뉴스 수집을 위한 도구

Easily generate correct user-agent strings for popular browsers

📅🇨🇳 中国法定节假日数据 自动每日抓取国务院公告

The Universal Device Detection library will parse any User Agent and detect the browser, operating system, device used (desktop, tablet, mobile, tv, cars, console, etc.), brand and model.

Download a large list of files concurrently

A distributed in-memory key-value storage for billions of small objects.

News, full-text, and article metadata extraction in Python 3. Advanced docs:

Browser sniffing gone too far — A useragent parser library for PHP

Crawler for linguistic corpora

Build a small, 3 domain internet using Github pages and Wikipedia and construct a crawler to crawl, render, and index.

A curated list of awesome puppeteer resources.

Proxying Python Requests

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

ProxyCrawl Python library for scraping and crawling

Download DIG to run on your laptop or server.

A User-agent analyze project.

Python crawling tutorial

Deprecated: Use the @encapsule/arccore package that includes the graph library

A Scrapy Spider for downloading PDF files from a webpage.

Browser sniffing gone too far — A useragent parser library for JavaScript

HTTP API for Scrapy spiders

Browser extension to spoof timezone, geolocation, locale and user agent.

Distributed crawler powered by Headless Chrome

🐛 Composable User Agent Detection using Ramda

Declarative web scraping

The complete web scraping toolkit for PHP.

Crawly, a high-level web crawling & scraping framework for Elixir.

A fake data generator written in and for Go

a reliable high-level web crawling & scraping framework for Node.js.

🕷 Automatically detect changes made to the official Telegram sites, clients and servers.

一个灵活、友好的爬虫框架

HTTP User Agent parser for the Go programming language.

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

PHP library to find podcasts

The simple, easy to use command line web crawler.

A JavaScript library for generating random user agents with data that's updated daily.

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

Golang module to detect bots and crawlers via the user agent

A command line tool and library for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET and TFTP. libcurl offers a myriad of powerful features

Haro is a modern immutable DataStore

📱React User Agent Component, Hook, and HOC. SSR-ready, full UT, using new React Context and Hooks API

User-Agent parser for Clojure

A Scrapy extension to log items coverage when the spider shuts down

SimFin's open source PDF crawler

📃 The main project repository