34 Open source projects that are alternatives of or similar to node-warc

📇 Tools to Work with the Web Archive Ecosystem in R

Read Web ARChive (WARC) files in PHP.

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

🐋 One-Click User Instigated Preservation

A robust web archive analytics toolkit

🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...

A list of things related to software, literature, and other content for 🕣 Memento

A social media open post web archiving tool

A server to collect & archive websites that also supports video downloads

A Memento Aggregator CLI and Server in Go

A small tool which uses the CommonCrawl URL Index to download documents with certain file types or mime-types. This is used for mass-testing of frameworks like Apache POI and Apache Tika

cUrl-like utility for fetching a resource (in this case we will run JS and return after network is idle) - great for JS heavy apps

Recover lost websites from the Web Infrastructure

A dockerized, queued high fidelity web archiver based on Squidwarc

Navigator for Web Archive

A service that provides archive-aware oEmbed-compatible embeddable surrogates (social cards, thumbnails, etc.) for archived web pages (mementos).

Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.

⚙️ A Rust library for reading and writing WARC files

A Tool To Push Web Resources Into Web Archives

Archiveror will help you preserve the webpages you love. 💾

🐋 Web Archiving Integration Layer: One-Click User Instigated Preservation

Streaming WARC/ARC library for fast web archive IO

Chrome extension to "Create WARC files from any webpage"

Social Feed Manager user interface application.

An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.

Serverless Web Archive Replay directly in the browser

Collect and revisit web pages.

A High-Fidelity Web Archiving Extension for Chrome and Chromium based browsers!

Core Python Web Archiving Toolkit for replay and recording of web archives

Webrecorder Player for Desktop (OSX/Windows/Linux). (Built with Electron + Webrecorder)

InterPlanetary Wayback: A distributed and persistent archive replay system using IPFS

Indelible links

🏌🏼‍E2E testing for dApps using Puppeteer + MetaMask

Evaluate JavaScript on a URL through headless Chrome browser.