Proof-of-concept: Web based Jamulus client

Posted at https://github.com/jamulussoftware/jamulus/discussions/2039.

Video

First, here’s a demo:

https://user-images.githubusercontent.com/193136/134050768-e919104a-652d-4543-a025-60facf870013.mp4

Quick facts

• Web Audio API can provide low latency audio access to audio output with base latency of 5.33 ms (buffer size of 128 samples) under latencyHint: 'interactive' mode.
• WebRTC can create a low-latency UDP-like data channel under ordered: false, maxRetransmits: 0 settings. We also get end-to-end encryption for free.
• libopus can be compiled into WebAssembly and used inside an AudioWorklet.
• As of writing, all of the above is supported in the latest version of Safari on iOS.

How it works

There are 3 main components:

1. Jamulus server.

2. WebRTC-UDP gateway.

   • Source code: https://github.com/dtinth/rtcjam
   • It should be run on the same server as the Jamulus server. It does nothing more than providing browser access to Jamulus server over a different transport layer.
   • Since it will run on a publicly-accessible server, there will be no need to use STUN/TURN solutions.
   • Once a WebRTC connection is established, all data will be forwarded to localhost:22124 (and vice versa).

3. Web-based Jamulus client.

   • Source code: https://github.com/dtinth/webjam
   • This web-based client speaks the usual Jamulus protocol. However the transport layer is WebRTC instead of plain UDP.
   • An AudioWorklet uses libopus compiled to WebAssembly to decode Opus frames and feeds it to the speaker.

Limitations of the POC

• Right now it can only receive audio, as the encoder has not been implemented yet. The PoC is hardcoded to send silent Opus frames to the server.
• The PoC is hardcoded to use Stereo High Quality mode with 16 frames of jitter buffer. Client name is hardcoded to WebJam.
• Memory and CPU usage is very inefficient — a lot of clones and byte-by-byte operations in JavaScript. Safari on my iPad Pro (2015) crashes (page is forced to reload) after 1 minute of usage :joy:. But I believe this can be improved by putting more things in WebAssembly, e.g. Jamulus protocol handler, jitter buffer, etc. There is a tutorial on using WebAssembly with AudioWorklet on Google Developers
• The gateway server currently has no security mechanism at all; it accepts and forwards incoming UDP packets from any host (not just the target server).

Note: I have no intention to develop this any further. All this is just to satisfy my curiosity. However, I think it can be a good starting point to expand Jamulus to the web platform if anyone wants to pursue this goal.