mozilla / Rainbow

Introduction

Rainbow is an early prototype that provides video and audio capture capabilities to web pages. It is currently distributed as a highly experimental Firefox addon.

Rainbow uses a bunch of different backends depending on the platform to capture audio and video. The ogg, theora and vorbis libraries from xiph are used to encode them. WebM support is on our radar.

Mac: libvidcap for Video, portaudio for Audio Windows: DirectShow for Video, and WinMM for Audio Linux: Video not supported, ALSA for Audio

We would ultimately not want to depend on any external libraries. QTKit backends for Video and Audio on the Mac, as well as a V4L backend for Video on Linux are under development.

Quick testing

We have setup test pages for Rainbow on http://mozilla.github.com/rainbow/ which you can visit after you install the add-on. The domain has already been added to the default preference 'extensions.rainbow.allowedDomains', but if you are upgrading from an older version, you should receive a permissions dialog.

API

The JS API offers the capability to record multiplexed audio and video files to disk:

window.navigator.media.beginSession(params, ctx, observer);

where 'params' is a JS Object, which may contain the following properties. The types and default values for each property are in brackets:

{ audio:(boolean, true), video:(boolean, true), width:(int, 640), height:(int, 480), rate:(int, 22050), channels:(int, 1), quality:(float, 0.4) }

and 'ctx' is the 2D context of a canvas on which a live preview (if video was requested) will be drawn. 'ctx' may be null in case you do not want a preview.

'observer' is a JS function that will be called whenever a significant event occurs. It will take two arguments, and you will usually want to switch on the first one, which is the event type:

function onStateChange(type, data) { switch (type) { case "session-began": window.navigator.media.beginRecord(); break; case "record-began": break; case "record-ended": break; case "record-finished": alert("got file " + data); break; case "session-ended": break; } }

While beginSession only sets up Rainbow in 'preview' mode, which means the canvas will be painted as usual, you must explicitly call beginRecord() to start the encoding process. Calling endRecord() will result in the 'record-ended' and 'record-finished' events being passed to your observer, and the latter's 'data' argument will be set to a DOMFile.

Please see content/example.html for sample usage of the API.

Rainbow also offers support for recording from a canvas as source (instead of a webcam). The canvas will be sampled the rate of 30 frames per second. You can do this by specifying the 'source' property of the params object to be 'true'. This will treat the 'ctx' argument you pass to the beginSession function as the source of video rather than the destination of a live preview.

Please see content/example_canvas.html for an example of such usage.

Temporary Files

Rainbow does not correctly clean up files generated by it. You must manually remove recorded OGG files from your disk. These files may be found in:

Mac: ~/Library/Caches/TemporaryItems/ Windows: C:\Users<user>\AppData\Local\Temp
Linux: /tmp

We are working on an API to ensure that these files get cleaned up once the web application that requested their creation is done with them.

Common Errors

"Error: window.navigator.service is undefined"

This can mean one of two things:

• You tried to access the API from a webpage hosted on a restricted domain. By default, Rainbow will only allow access to web pages served from localhost, but you may change that by adding more domains to the 'extensions.rainbow.allowedDomains' preference through about:config. The value is simply a JSON array of domains.

• The binary component itself failed to load. This may because you are using an older version of Firefox, or that you are running the nightly on an unsupported platform (see below). If this is the case the error will be preceded by something like:

"Error: Components.classes['@labs.mozilla.com/media/recorder;1'] is undefined"

Platform Support

On Linux, we do not support video recording.

On Windows, some DV cameras may result in garbled video.

On Mac (Snow Leopard), 64-bit versions of Minefield are not supported. You may run your Minefield nightly in 32-bit mode like so: $ arch -i386 /Applications/Minefield.app/Contents/MacOS/firefox-bin

Building

If you wish to build the binary components yourself, fetch a recent build of the Gecko2 SDK from: http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/latest-trunk/ After extracting, set the MOZSDKDIR environment variable to point to it and simply run `make'.

Precompiled versions of the dependencies are already included for convenience, however if you wish to include custom versions of libvidcap, libportaudio, libogg, libvorbis or libtheora; place them in the appropriate directory in lib/ and the build system will automatically link with them.

You may set debug=1 to create a version of the XPCOM component with debugging symbols.