All Projects → nanmu42 → limitio

nanmu42 / limitio

Licence: MIT license
Golang io.Reader and io.Writer but with limits

Programming Languages

go
31211 projects - #10 most used programming language

Labels

Projects that are alternatives of or similar to limitio

Boson
A C++14 framework for asynchronous I/O, cooperative multitasking and green threads scheduling
Stars: ✭ 154 (+113.89%)
Mutual labels:  io
Luv
Cross-platform asynchronous I/O and system calls
Stars: ✭ 203 (+181.94%)
Mutual labels:  io
Lasio
Python library for reading and writing well data using Log ASCII Standard (LAS) files
Stars: ✭ 234 (+225%)
Mutual labels:  io
Kotlin Retry
A higher-order function for retrying operations that may fail.
Stars: ✭ 159 (+120.83%)
Mutual labels:  io
React Globe
Create beautiful and interactive React + ThreeJS globe visualizations with ease.
Stars: ✭ 167 (+131.94%)
Mutual labels:  io
Shrine Materialdesign2
implementation of Material Design 2 Shrine project
Stars: ✭ 215 (+198.61%)
Mutual labels:  io
Geojsonio
Convert many data formats to & from GeoJSON & TopoJSON
Stars: ✭ 132 (+83.33%)
Mutual labels:  io
GeoArrays.jl
Simple geographical raster interaction built on top of ArchGDAL, GDAL and CoordinateTransformations
Stars: ✭ 42 (-41.67%)
Mutual labels:  io
Gulpio
Binary storage format for deep learning on videos.
Stars: ✭ 178 (+147.22%)
Mutual labels:  io
Newandroidarchitecture Component Github
Sample project based on the new Android Component Architecture
Stars: ✭ 229 (+218.06%)
Mutual labels:  io
Mpejs
Next generation MIDI for the web
Stars: ✭ 161 (+123.61%)
Mutual labels:  io
Lightio
LightIO is a userland implemented green thread library for ruby
Stars: ✭ 165 (+129.17%)
Mutual labels:  io
Aiofile
Real asynchronous file operations with asyncio support.
Stars: ✭ 214 (+197.22%)
Mutual labels:  io
Ofxgpio
Library C++ for raspberrypi and orangepi, GPIO interfaces compatible with openframeworks.
Stars: ✭ 155 (+115.28%)
Mutual labels:  io
google-globe-trends
Create beautiful and interactive Google Trends globe visualizations with ease
Stars: ✭ 33 (-54.17%)
Mutual labels:  io
Async Io
Concurrent wrappers for native Ruby IO & Sockets.
Stars: ✭ 138 (+91.67%)
Mutual labels:  io
Byte Stream
A non-blocking stream abstraction for PHP based on Amp.
Stars: ✭ 208 (+188.89%)
Mutual labels:  io
GDAL.jl
Thin Julia wrapper for GDAL - Geospatial Data Abstraction Library
Stars: ✭ 78 (+8.33%)
Mutual labels:  io
bob
Bob is a free signal-processing and machine learning toolbox originally developed by the Biometrics group at Idiap Research Institute, in Switzerland. - Mirrored from https://gitlab.idiap.ch/bob/bob
Stars: ✭ 38 (-47.22%)
Mutual labels:  io
Codejam
Set of handy reusable .NET components that can simplify your daily work and save your time when you copy and paste your favorite helper methods and classes from one project to another
Stars: ✭ 217 (+201.39%)
Mutual labels:  io

LimitIO

GoDoc Build status codecov Go Report Card

io.Reader and io.Writer with limit.

go get github.com/nanmu42/limitio

Rationale and Usage

There are times when a limited reader or writer comes in handy.

  1. wrap upstream so that reading is metered and limited:
// request is an incoming http.Request
request.Body = limitio.NewReadCloser(c.Request.Body, maxRequestBodySize, false)

// deal with the body now with easy mind. It's maximum size is assured.

Yes, io.LimitReader works the same way, but throws EOF on exceeding limit, which is confusing.

LimitIO provides error that can be identified.

decoder := json.NewDecoder(request.Body)
err := decoder.Decode(&myStruct)
if err != nil {
    if errors.Is(err, limitio.ErrThresholdExceeded) {
        // oops, we reached the limit
    }

    err = fmt.Errorf("other error happened: %w", err)
    return
}
  1. wrap downstream so that writing is metered and limited(or instead, just pretending writing):
// request is an incoming http.Request.
// Say, we want to record its body somewhere in the middleware,
// but feeling uneasy since its body might be HUGE, which may
// result in OOM and a successful DDOS...

var reqBuf bytes.buffer

// a limited writer comes to rescue!
// `true` means after reaching `RequestBodyMaxLength`,
// `limitedReqBuf` will start pretending writing so that
// io.TeeReader continues working while reqBuf stays unmodified.
limitedReqBuf := limitio.NewWriter(&reqBuf, RequestBodyMaxLength, true)

request.Body = &readCloser{
    Reader: io.TeeReader(request.Body, limitedReqBuf), 
    Closer: c.Request.Body,
}

LimitIO provides Reader, Writer and their Closer versions, for details, see docs.

Status: Stable

LimitIO is well battle tested under production environment.

APIs are subjected to change in backward compatible way during 1.x releases.

License

MIT License

Copyright (c) 2021 LI Zhennan

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].