All Projects → zulhilmizainuddin → multipart-download

zulhilmizainuddin / multipart-download

Licence: MIT License
Speed up download of a single file with multiple HTTP GET connections running in parallel

Programming Languages

typescript
32286 projects

Projects that are alternatives of or similar to multipart-download

hent
A small utility to fetch remote files into buffers
Stars: ✭ 23 (-20.69%)
Mutual labels:  download, get
Pget
The fastest file download client
Stars: ✭ 724 (+2396.55%)
Mutual labels:  download, parallel
laravel-parallel
Laravel parallel
Stars: ✭ 41 (+41.38%)
Mutual labels:  parallel
DownloadFile
Typecho文件下载插件,使下载的文件保持上传时的文件名。
Stars: ✭ 12 (-58.62%)
Mutual labels:  download
bookmate downloader
Download books as epub from bookmate.com
Stars: ✭ 38 (+31.03%)
Mutual labels:  download
cruise
User space POSIX-like file system in main memory
Stars: ✭ 27 (-6.9%)
Mutual labels:  parallel
ezpq
an easy parallel queueing system
Stars: ✭ 42 (+44.83%)
Mutual labels:  parallel
dicom-dimse-native
node js native addon for dimse services
Stars: ✭ 33 (+13.79%)
Mutual labels:  get
split-tests
Split test files in Jest and Cypress into parallel CI jobs
Stars: ✭ 22 (-24.14%)
Mutual labels:  parallel
hack parallel
The core parallel and shared memory library used by Hack, Flow, and Pyre
Stars: ✭ 39 (+34.48%)
Mutual labels:  parallel
image-downloader
A Nodejs module for downloading image to disk from a given URL
Stars: ✭ 61 (+110.34%)
Mutual labels:  download
TidalSwift
Tidal Music Streaming Client & Library written in Swift
Stars: ✭ 45 (+55.17%)
Mutual labels:  download
crypto-candlesticks
Download candlestick data fast & easy for analysis
Stars: ✭ 26 (-10.34%)
Mutual labels:  download
tbslas
A parallel, fast solver for the scalar advection-diffusion and the incompressible Navier-Stokes equations based on semi-Lagrangian/Volume-Integral method.
Stars: ✭ 21 (-27.59%)
Mutual labels:  parallel
http
Aplus Framework HTTP Library
Stars: ✭ 113 (+289.66%)
Mutual labels:  download
SeisGo
A ready-to-go Python toolbox for seismic data analysis
Stars: ✭ 23 (-20.69%)
Mutual labels:  download
Parallel.GAMIT
Python wrapper to parallelize GAMIT executions
Stars: ✭ 22 (-24.14%)
Mutual labels:  parallel
swarmci
Swarm CI - Docker Swarm-based CI system or enhancement to existing systems.
Stars: ✭ 48 (+65.52%)
Mutual labels:  parallel
jobflow
runs stuff in parallel (like GNU parallel, but much faster and memory-efficient)
Stars: ✭ 67 (+131.03%)
Mutual labels:  parallel
youtube-dl-php
A quick and easy YouTube downloader/parser written in PHP.
Stars: ✭ 49 (+68.97%)
Mutual labels:  download

multipart-download Build Status

NPM

Speed up download of a single file with multiple HTTP GET connections running in parallel

Class: MultipartDownload

MultipartDownload is an EventEmitter.

start(url[, options])

  • url <string> Url of file to be downloaded
  • options <StartOptions> Download options (Optional)
    • numOfConnections <number> Number of HTTP GET connections to use for performing the download (Optional)
    • writeToBuffer <boolean> Store downloaded data to buffer (Optional)
    • saveDirectory <string> Directory to save the downloaded file (Optional)
    • fileName <string> Set name of the downloaded file (Optional)
    • headers <Object> Set custom HTTP headers (Optional)

Starts the download operation from the url.

Multiple HTTP GET connections will only be used if the target server supports partial requests. If the target server does not support partial requests, only a single HTTP GET connection will be used regardless of what the numOfConnections is set to.

If the numOfConnections parameter is not provided, a single connection will be used.

If the writeToBuffer parameter is set to true, the downloaded file will be written into a buffer.

If the saveDirectory parameter is provided, the downloaded file will be saved to the saveDirectory.

If the fileName parameter is provided, the downloaded file will be renamed to fileName. If the fileName parameter is not provided, the downloaded file will maintain its original file name.

If the headers parameter is provided, the headers will be included in the HTTP request.

Event: 'error'

  • err <Error> Emitted error

Event: 'data'

  • data <string> | <Buffer> Chunk of data received
  • offset <number> Offset for the chunk of data received

The file being downloaded can be manually constructed and manipulated using the data and offset received.

Event: 'end'

  • output <string> Downloaded file buffer or downloaded file saved path

output is the buffer of the downloaded file if the writeToBuffer parameter is set to true.

output is the location of the saved file if the saveDirectory parameter is provided.

output will be null if writeToBuffer is not set to true or saveDirectory parameter is not provided.

start(url[, numOfConnections, saveDirectory]) DEPRECATED and REMOVED in v1.0.0

Example

Download without writing to buffer or saving to file

const MultipartDownload = require('multipart-download');

new MultipartDownload()
  .start('https://homepages.cae.wisc.edu/~ece533/images/cat.png', {
    numOfConnections: 5
  })
  .on('error', (err) => {
    // handle error here
  })
  .on('data', (data, offset) => {
    // manipulate data here
  })
  .on('end', () => {

  });

Download and write to buffer

const MultipartDownload = require('multipart-download');

new MultipartDownload()
  .start('https://homepages.cae.wisc.edu/~ece533/images/cat.png', {
    numOfConnections: 5,
    writeToBuffer: true
  })
  .on('error', (err) => {
    // handle error here
  })
  .on('data', (data, offset) => {
    // manipulate data here
  })
  .on('end', (buffer) => {
    console.log(`Downloaded file buffer: ${buffer}`);
  });

Download and save to file

const os = require('os');

const MultipartDownload = require('multipart-download');

new MultipartDownload()
  .start('https://homepages.cae.wisc.edu/~ece533/images/cat.png', {
    numOfConnections: 5,
    saveDirectory: os.tmpdir(),
    fileName: 'kitty.png'
  })
  .on('error', (err) => {
    // handle error here
  })
  .on('data', (data, offset) => {
    // manipulate data here
  })
  .on('end', (filePath) => {
    console.log(`Downloaded file path: ${filePath}`);
  });
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].