All Projects → killercup → Static Filez

killercup / Static Filez

Licence: other
Build compressed archives for static files and serve them over HTTP

Programming Languages

rust
11053 projects

Projects that are alternatives of or similar to Static Filez

fimfarchive
Preserves stories from Fimfiction
Stars: ✭ 15 (-54.55%)
Mutual labels:  archiving
Wal E
Continuous Archiving for Postgres
Stars: ✭ 3,313 (+9939.39%)
Mutual labels:  archiving
Pgbackrest
Reliable PostgreSQL Backup & Restore
Stars: ✭ 766 (+2221.21%)
Mutual labels:  archiving
earkweb
E-ARK Web is a software for the creation and management of archival information packages, and it supports full-text search for individual files contained in them.
Stars: ✭ 18 (-45.45%)
Mutual labels:  archiving
compose-dump
Dump and restore Docker Compose-projects
Stars: ✭ 14 (-57.58%)
Mutual labels:  archiving
Bareos
Main repository with the code for the libraries and daemons
Stars: ✭ 651 (+1872.73%)
Mutual labels:  archiving
i7n-pdfhtml
pdfHTML is an iText 7 add-on for C# (.NET) that allows you to easily convert HTML and CSS into standards compliant PDFs that are accessible, searchable and usable for indexing.
Stars: ✭ 111 (+236.36%)
Mutual labels:  archiving
Warc
Golang WARC (Web ARChive) Library
Stars: ✭ 25 (-24.24%)
Mutual labels:  archiving
Nb
CLI and local web plain text note‑taking, bookmarking, and archiving with linking, tagging, filtering, search, Git versioning & syncing, Pandoc conversion, + more, in a single portable script.
Stars: ✭ 3,846 (+11554.55%)
Mutual labels:  archiving
Itext7 Dotnet
iText 7 for .NET is the .NET version of the iText 7 library, formerly known as iTextSharp, which it replaces. iText 7 represents the next level of SDKs for developers that want to take advantage of the benefits PDF can bring. Equipped with a better document engine, high and low-level programming capabilities and the ability to create, edit and enhance PDF documents, iText 7 can be a boon to nearly every workflow.
Stars: ✭ 698 (+2015.15%)
Mutual labels:  archiving
storytracker
Tools for tracking stories on news homepages
Stars: ✭ 47 (+42.42%)
Mutual labels:  archiving
pastpages.org
The news homepage archive
Stars: ✭ 81 (+145.45%)
Mutual labels:  archiving
Linkace
Your self-hosted bookmark archive. Free and open source.
Stars: ✭ 657 (+1890.91%)
Mutual labels:  archiving
archivers-harvesting-tools
ARCHIVED--Collection of scripts and code snippets for data harvesting after generating the zip starter
Stars: ✭ 31 (-6.06%)
Mutual labels:  archiving
Django Urlarchivefield
A custom Django model field that automatically archives a URL
Stars: ✭ 5 (-84.85%)
Mutual labels:  archiving
paperless-ng
A supercharged version of paperless: scan, index and archive all your physical documents
Stars: ✭ 4,840 (+14566.67%)
Mutual labels:  archiving
Pg probackup
Backup and recovery manager for PostgreSQL
Stars: ✭ 383 (+1060.61%)
Mutual labels:  archiving
Crocoite
Web archiving using Google Chrome
Stars: ✭ 30 (-9.09%)
Mutual labels:  archiving
Itext7
iText 7 for Java represents the next level of SDKs for developers that want to take advantage of the benefits PDF can bring. Equipped with a better document engine, high and low-level programming capabilities and the ability to create, edit and enhance PDF documents, iText 7 can be a boon to nearly every workflow.
Stars: ✭ 913 (+2666.67%)
Mutual labels:  archiving
Grab Site
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
Stars: ✭ 680 (+1960.61%)
Mutual labels:  archiving

Static FileZ

Build compressed archives for static files and serve them over HTTP

Build Status crates.io docs

What and Why

Say you want to store a huge number of very small files that you are only viewing in a browser. For example: You are using rustdoc to render the documentation of a library. Without much work you'll end up with about 100k files that are about 10kB each. As it turns out, this number of small files is very annoying for any kind of file system performance: Best case: making copies/backups is slow. Worst case: You're using an anti virus software and it takes ages.

Except for convenience when implementing software, and people being used to having folders of files they can look into, there is little reason to store these files individually. Indeed, it will save much space and time to store files like these in compressed form in one continuous archive. All that is needed to make this work is some well-designed and discoverable software.

static-filez is a prototype for that piece of software.

Installation

Download a binary

There are pre-compiled binaries that you can find on the releases page.

Alternatively, install it like this:

curl -LSfs https://japaric.github.io/trust/install.sh | sh -s -- --git killercup/static-filez

Compile manually

Make sure you have Rust and Cargo installed. Then, run cargo install --git https://github.com/killercup/static-filez.

Alternatively, clone this repository and run cargo install --path ..

Usage

  1. Build an archive (and index) from a directory: static-filez build target/doc/ ./docs.archive
  2. Start a HTTP server that serves the files in the archive: static-filez serve -p 3000 docs.archive
  3. Open a browser and see your files: http://127.0.0.1:3000/regex/ (regex is an example for a great documentation page you should read)

Architecture

Currently, static-filez will generate two files: An .index file, and an .archive file.

The index is a specialized data structure that maps paths to their content in the archive.

The archive file contains the (compressed) content of your files. The server is implemented in a way that it can serve the compressed content directly, with no need to ever look at the (potentially much larger) original decompressed data. (This works by using the HTTP Content-Encoding header, if you are curios.)

You can read more about the structure of the files in this issue, or, of course, the source.

License

Licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].