5822 Open source projects that are alternatives of or similar to Skrape.it

An extremely fast web scraper that parses megabytes of invalid HTML in a blink of an eye. PHP5.3+, no dependencies.

The fast & forgiving HTML and XML parser

Paco is a Java based framework for non-blocking and highly parallelized Dom testing.

jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.

Declarative web scraping

📱 Wrapper Appium Framework in Java which supports Automation of Mobile and Tablet apps.

A library for setting up Java objects as test data.

🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent

News, full-text, and article metadata extraction in Python 3. Advanced docs:

This repository is no longer maintained.

NSoup is a .NET port of the jsoup (http://jsoup.org) HTML parser and sanitizer originally written in Java

The Android SDK for the Parse Platform

Docker + Jest integration testing for Node.js

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

OS-level test automation

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

Moved to https://gitlab.com/craigbarnes/lua-gumbo

Mutation testing for Scala. Work in progress...

This repository contains all the code I use in my YouTube tutorials.

A web crawler (for bug hunting) that gathers more than you can imagine.

Simple and fast HTML and XML parser

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

ANTLR v4 grammar-based test generator

Crawl instagram photos, posts and videos for download.

Fuzzinator Random Testing Framework

A cross platform UI crawler which scans view trees then generate and execute UI test cases.

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

Straightforward HTML parser for JavaScript

Golang pkg to quickly return a preview of a webpage (title/description/images)

Java 網路資料爬蟲包

A simple but powerful web crawler library for .NET

Curated list of popular Java and Kotlin frameworks, libraries and tools related to software testing, quality assurance and adjacent processes automation.

Python爬虫，京东自动登录，在线抢购商品

Floki is a simple HTML parser that enables search for nodes using CSS selectors.

Node.js scraper to get data from Google Play

Zebrunner is a Test Automation Management Tool

Scrapy, a fast high-level web crawling & scraping framework for Python.

Google play scraper for Python inspired by <facundoolano/google-play-scraper>

A scraper for https://bandcamp.com

sub-component of Genie that parse the device output into structured datastructure

Kotlin/Java library and cli tool for scraping posts and media from various sources with neither authorization nor full page rendering (Facebook, Instagram, Twitter, Youtube, Tiktok, Telegram, Twitch, Reddit, 9GAG, Pinterest, Flickr, Tumblr, IFunny, VK, Pikabu)

Demeter is a tool for scraping the calibre web ui

Html Agility Pack (HAP) is a free and open-source HTML parser written in C# to read/write DOM and supports plain XPATH or XSLT. It is a .NET code library that allows you to parse "out of the web" HTML files.

Alternative for VK Audio API

Minimize HTML

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

⚡️ Render HTML5 as VDOM, with Components as Custom Elements!

A magic html parser

An android client for the MifosX platform

Headless Chrome For Java （Java 爬虫）

🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

Elegant Scraper and Crawler Framework for Golang

Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok

openQA web-frontend, scheduler and tools.

pip安装的天眼查爬虫API，指定的单个/多个企业工商信息一键保存为Excel/JSON格式。A Battery-included Scraper API of Tianyancha, the best Chinese business data and investigation platform.

Universal scrapping tool, which allows you to extract data using multiple environments

👾 Fast and simple video download library and CLI tool written in Go

Single code base framework to test android and iOS app using appium (v6.1.0), maven, testng,java. Option to start appium server programmatically.

The JavaScript SDK for the Parse Platform