5822 Open source projects that are alternatives of or similar to Skrape.it

👾 Fast and simple video download library and CLI tool written in Go

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

🇰🇷 Realtime integrated information analysis service

Scraper example built on Scala, Akka and Jsoup

我的爬虫合集

📝 HTML to DOM parser.

An R web crawler and scraper

pip安装的天眼查爬虫API，指定的单个/多个企业工商信息一键保存为Excel/JSON格式。A Battery-included Scraper API of Tianyancha, the best Chinese business data and investigation platform.

Web Application Security Scanner Framework

Arquillian provides a component model for integration tests, which includes dependency injection and container life cycle management. Instead of managing a runtime in your test, Arquillian brings your test to the runtime.

Android 本地网络小说爬虫，基于jsoup及xpath

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion

Elegant Scraper and Crawler Framework for Golang

本仓库收集整理爬虫相关资源，开发语言以Java为主

Crawl BookCorpus

Crawly, a high-level web crawling & scraping framework for Elixir.

OSINT Swiss Army Knife

Distributed crawler powered by Headless Chrome

Fast, flexible, and lean implementation of core jQuery designed specifically for the server.

Parse and output TODOs and FIXMEs from comments in your files

🔮 A Node.js scraper for humans.

Write e2e tests for CLI apps with ease

Apifier is a very simple HTML parser written in Python based on CSS selectors

Fetches PubMed article IDs (PMIDs) from email inbox, then crawls PubMed, Google Scholar and Sci-Hub for respective PDF files.

[Unmaintained] A simple and clean video/music/image downloader 👾

Tổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

Kotlin/Java library and cli tool for scraping posts and media from various sources with neither authorization nor full page rendering (Facebook, Instagram, Twitter, Youtube, Tiktok, Telegram, Twitch, Reddit, 9GAG, Pinterest, Flickr, Tumblr, IFunny, VK, Pikabu)

Scoopi Web Scraper is a heavy duty tool to extract data from HTML pages.

Straightforward HTML parser for JavaScript

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

A simple but powerful web crawler library for .NET

Docker + Jest integration testing for Node.js

Curated list of popular Java and Kotlin frameworks, libraries and tools related to software testing, quality assurance and adjacent processes automation.

A library for setting up Java objects as test data.

Scrapy, a fast high-level web crawling & scraping framework for Python.

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

Zebrunner is a Test Automation Management Tool

Mutation testing for Scala. Work in progress...

A scraper for https://bandcamp.com

Moved to https://gitlab.com/craigbarnes/lua-gumbo

sub-component of Genie that parse the device output into structured datastructure

NSoup is a .NET port of the jsoup (http://jsoup.org) HTML parser and sanitizer originally written in Java

An android client for the MifosX platform

Universal scrapping tool, which allows you to extract data using multiple environments

Alternative for VK Audio API

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

⚡️ Render HTML5 as VDOM, with Components as Custom Elements!

Simple and fast HTML and XML parser

A magic html parser

A cross platform UI crawler which scans view trees then generate and execute UI test cases.

openQA web-frontend, scheduler and tools.

Demeter is a tool for scraping the calibre web ui

PySys System Test Framework

A web app for API test automation

Golang pkg to quickly return a preview of a webpage (title/description/images)

Html Agility Pack (HAP) is a free and open-source HTML parser written in C# to read/write DOM and supports plain XPATH or XSLT. It is a .NET code library that allows you to parse "out of the web" HTML files.

🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok

crawler framework, distributed crawler extractor