595 Open source projects that are alternatives of or similar to PDAP-Scrapers

Scrapers for many e-SAJ systems

...Just a place for me to share my various YT-DLP & related bash scripts.

Scraper for downloading the entire ebooks repository of project Gutenberg

An easiest crawling and scraping module for NestJS

Scrapes VK user's photos

A scraper for videos that are trending on YouTube (https://www.youtube.com/feed/trending)

Node.js based scraper using headless chrome

Automated Tinder bot and scraper using selenium in python.

Page metadata as a service

Fast concurrent image scraper

Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!

Scrape everything you can from MyAnimeList.net

Search and download mangas from the command line

A Python class that scrapes ESRI Rest Endpoints and exports data to a geodatabase

At present contains scraped data from around 1500 problems present on the site. More to follow....

Scripts to maintain German law git repository

code, labs and lectures for the course

DorkScout - Golang tool to automate google dork scan against the entiere internet or specific targets

❄️ Extract links, ids, and names from a youtube playlist

proc(ess)-that - easy extendable ETL tool for Node.js. Written in TypeScript.

CubETL - Framework and tool for data ETL (Extract, Transform and Load) in Python (PERSONAL PROJECT / SELDOM MAINTAINED)

Continuously search imageboards threads for images/webms and download them

Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.

大数据采集,抽取平台

A PHP Scraping Class

Treats CSV and JSON files as SQL tables, and exports SQL SELECTs back to CSV or JSON.

csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.

web scraping extension

Configurable data bridge for permanent ETL jobs

图片爬取下载工具，极速爬取下载 站酷https://www.zcool.com.cn/, CNU 视觉 http://www.cnu.cc/ 设计师/用户 上传的 图片/照片/插画。

Extract rich metadata from URLs

Powerful RDF Knowledge Graph Generation with [R2]RML Mappings

京东App自动登录，在线商品自动下单

Aviation charts

ETL jobs for Firefox Telemetry

Grabs all of the audio files from all of the Blinkist books

A data transformation package for deep learning with Autonomio, Keras and TensorFlow.

[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library

StackExchange websites to ZIM scraper

Live import all your Google Sheets to your data warehouse

No description or website provided.

Essential PHP library that scrapes HTTP(S) and UDP trackers for torrent information.

Download subreddit comments

Analyse all kinds of data for a TV series

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

Proof of concept for a security issue (in my opinion) that I found in accounts.google.com

💾 YouTube video metadata archiver written in Golang

Yet Another ETL in PHP

There was no free Dictionary API on the web when I wanted one for my friend, so I created one.

BETL. Meta data driven ETL generation using T-SQL

在线爬虫，online web scraper

Web scraping using python, requests and selenium

The Move to Islandora Kit is an extensible PHP command-line tool for converting source content and metadata into packages suitable for importing into Islandora (or other digital repository and preservations systems).

CVparser is software for parsing or extracting data out of CV/resumes.

A simple web scraper built around the JavaFX WebEngine

bqloader is a simple ETL framework to load data from Cloud Storage into BigQuery.

A monadic web scraping library

EtlFlow is an ecosystem of functional libraries in Scala based on ZIO for writing various different tasks, jobs on GCP and AWS.

Open bank data for Leumi bank and Leumi card credit card

An Elasticsearch client tailored to data science workflows.