All Projects → scottstensland → Websockets Streaming Audio

scottstensland / Websockets Streaming Audio

Licence: mit
Stream audio to a Web Audio API enabled browser from Node.js server side using Web Worker and Web Socket

Programming Languages

javascript
184084 projects - #8 most used programming language

Projects that are alternatives of or similar to Websockets Streaming Audio

Lgwebosremote
Command line webOS remote for LGTVs
Stars: ✭ 211 (-8.26%)
Mutual labels:  websockets
Web Worker Proxy
A better way of working with web workers
Stars: ✭ 218 (-5.22%)
Mutual labels:  web-worker
Postfacto
Self-hosted retro tool aimed at helping remote teams
Stars: ✭ 224 (-2.61%)
Mutual labels:  websockets
Wasm Worker
Move a WebAssembly module into its own thread
Stars: ✭ 215 (-6.52%)
Mutual labels:  web-worker
Node Red Contrib Uibuilder
Easily create data-driven web UI's for Node-RED using any (or no) front-end library. VueJS and bootstrap-vue included but change as desired.
Stars: ✭ 215 (-6.52%)
Mutual labels:  websockets
Angular2 Websocket
Websocket wrapper for angular2 based on angular-websocket
Stars: ✭ 218 (-5.22%)
Mutual labels:  websockets
Async Tungstenite
Async binding for Tungstenite, the Lightweight stream-based WebSocket implementation
Stars: ✭ 207 (-10%)
Mutual labels:  websockets
Nutz Book Project
做个平台
Stars: ✭ 228 (-0.87%)
Mutual labels:  websockets
Pont
An online board game in Rust and WebAssembly
Stars: ✭ 218 (-5.22%)
Mutual labels:  websockets
React Worker Image
React component to fetch image resources via web workers 🤖🤖
Stars: ✭ 226 (-1.74%)
Mutual labels:  web-worker
Isomorphic Ws
Isomorphic implementation of WebSocket (https://www.npmjs.com/package/ws)
Stars: ✭ 215 (-6.52%)
Mutual labels:  websockets
Webwire Go
A transport independent asynchronous duplex messaging library for Go
Stars: ✭ 216 (-6.09%)
Mutual labels:  websockets
Digital Restaurant
DDD. Event sourcing. CQRS. REST. Modular. Microservices. Kotlin. Spring. Axon platform. Apache Kafka. RabbitMQ
Stars: ✭ 222 (-3.48%)
Mutual labels:  websockets
Arduinowebsockets
A library for writing modern websockets applications with Arduino (ESP8266 and ESP32)
Stars: ✭ 213 (-7.39%)
Mutual labels:  websockets
Koa Websocket
Light wrapper around Koa providing a websocket middleware handler that is koa-route compatible.
Stars: ✭ 224 (-2.61%)
Mutual labels:  websockets
Aleph
Asynchronous communication for Clojure
Stars: ✭ 2,389 (+938.7%)
Mutual labels:  websockets
Jsdom Worker
👷‍♀️ Use Web Workers in Jest / JSDOM 🌈
Stars: ✭ 218 (-5.22%)
Mutual labels:  web-worker
Websocket Client
WebSocket client for Python
Stars: ✭ 2,810 (+1121.74%)
Mutual labels:  websockets
Django Sockpuppet
Build reactive applications with the django tooling you already know and love.
Stars: ✭ 225 (-2.17%)
Mutual labels:  websockets
Web Socket
Laravel library for asynchronously serving WebSockets.
Stars: ✭ 225 (-2.17%)
Mutual labels:  websockets

websockets-streaming-audio

NPM

stream audio to a Web Audio API enabled browser from Node.js server using Web Worker and Web Socket

plan is to make this modular enough to get added to your process as simple API calls - using Angularjs ;-)

I have now introduced a Web Worker to handle all server side calls - this fixed glitches of early versions suffered due to the single threaded browser

basic architecture :

Browser   <-->   Web Worker   <-->   Web Socket   <-->   Node.js

state transition sequence

  • start mode 1 then cycle between mode 2 and mode 3 until stream is done

mode 1

  • fill up browser audio buffer queue by requesting server side to start streaming the media file to return floating point typed array buffers via Web Worker using Web Sockets
  • to minimize audio play startup lag avoid populating Web Worker audio buffer queue
  • transition to mode 2 when browser queue is full

mode 2

  • launch Web Audio event loop if not already running
  • browser consumes audio buffers from browser buffer queue
  • browser avoids any interaction with Web Worker or server side
  • Web Worker is told to replenish its own audio buffer queue by requesting buffers from Node.js server side using Web Socket
  • transition to mode 3 when browser audio buffer queue gets too low

mode 3

  • browser seamlessly continues to render audio by consuming browser buffer queue
  • Web Worker does not interact with Node.js server side in this mode (critical to avoid interruption of rendered audio)
  • Web Worker begins this mode with a full buffer queue replinished during mode 2
  • Web Audio API event loop callback drives the browser to request Web Worker to send typed array audio buffers taken from the WW buffer queue which refills the browser buffer queue at twice the browser Web Audio consumption rate
  • transition to mode 2 when browser audio buffer queue becomes full

Installation

visit nodejs.org and install node.js

see project npm site at

https://www.npmjs.org/package/websockets-streaming-audio

Clone this repository to your local machine:

npm install websockets-streaming-audio

Change directory into the project folder to install the upstream modules:

cd node_modules/websockets-streaming-audio
npm install

Launch the nodejs app:

npm start

NODE_ENV=dev npm start

NODE_ENV=production npm start

Using a Web Audio API savvy browser (ff/chrome), point it at URL :

http://localhost:8888

then click one of the stream buttons, after a song has played click reload before hitting another stream button - I am still learning front ends ;-)

... OR ignore above and just see this app deployed live on heroku :

https://fierce-waters-00266.herokuapp.com

Current Limitations

  • server side source media parser I wrote only handles WAV format, however now that I am using Web Worker, I could transition to a compressed format. The bloated ogg decoder enabled using emscripten may go in soon - dunno
  • only streams audio from server side to browser - not other direction - I do plan to enable streaming microphone audio back to server side (or other browser originated audio : synthesized or uploaded file)
  • please click reload in between each stream button hit until I teach myself Angularjs ;-)))
  • on mobile it works OK using Android lollipop with mozilla nightly 38.0a1 (2015-02-01) and above - anything earlier fails to correctly implement web sockets inside web workers ... and even firefox ver 38 is a bit shady ;-) ... I do not use iOS so dunno how that phone handles this system

Lessions Learned

  • to stream audio using Web Audio API you must offload your Web Socket calls to a Web Worker layer or else you will hear audible glitches when the single threaded browser is suddenly interrupted away from babysitting the audio rendering committments just to service responses received from the server side as sent using Web Socket
  • a stepping stone to above point : the browser wants its own buffer queue, do not short change the design by relying on a Web Worker buffer queue to directly supply Web Audio event loop callback demands for time critical fresh audio buffers or else responses back from the server will interrupt the audio playback
  • create two buffer queues : browser based as well as Web Worker based, this assures the real time sensitive audio rendering done by Web Audio API is never interrupted by responses received back from server side

Feel free to contact me via the github Issues forum if you have any questions! :-) ... my twitter name is same as my github name

Bitdeli Badge

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].