Writing High Performance JavaScript with Web Workers

As someone who comes from a traditional desktop development background, I have severely underestimated the capabilities of modern JavaScript, especially its computational performance. The JavaScript interpreters have been exceedingly well optimized, there is threading in the form of Web Workers and high performance graphics with WebGL.

Here are some general guidelines for writing high performance code in JavaScript (or pretty much any language, to be honest).

  • Use concurrent operations wherever possible.
  • Don't starve the CPU.
  • Use Transferable objects.

I know it may sound obvious, but the first step towards writing high performance code is to figure out which parts can run concurrently. This is especially important when you are first designing the structure of your app as it is possible to design in bottlenecks if you are not considering performance from the beginning.

Once you have your concurrency figured out, you want to implement it with maximum efficiency, ensuring you are using all the available CPU cores whenever possible. As the core count grows on new processors, you want to make sure you can take advantage of it. Ideally, you want an equal amount of work distributed to each core so that each will finish in the same amount of time. This lets the CPU dedicate all of its processing power to the computation.

Finally, use Transferable objects to send and receive data from Web Workers. Time spent transferring data is time not spent doing computation. Without Transferables, the data has to be serialized, which just wastes precious CPU time. This is especially important if there is a lot of data, like an image, to transfer.

As an example of all these principles put to use, I'll show you how I handled the fractal computation in my Mandlebrot fractal generator.


Divide the Work

This is the stage where you have to decide how to break apart the computation. For clarity, each chunk of work will be called a single operation. It often takes a lot of testing and tuning to distribute the operations evenly across all the workers. If you feed the Web Workers too small an operation, you can spend more time managing the workers and combining the results then you do actual computation.

A fractal is very easy to compute on multiple threads because each pixel is calculated in isolation (although there is an optimization for which this isn't true). The problem, however, is that different pixels can take orders of magnitude longer than others to calculate, making it difficult to evenly distribute the work across all the cores.

At one extreme, we could divide the operations into calculating just one pixel each. This ensures that the work is evenly distributed, but wastes an incredible amount of time transferring data between the main logic and the workers. The other extreme is to divide the screen into the same number of sections as there are CPU cores. This would eliminate the wasted time transferring data, but some sections would finish much faster than others, leaving the CPU starved of work.

You have to do some trial and error to find the happy medium between the two extremes. I found that using 150px x 150px squares rarely starved the CPU and didn't spend much time managing the workers.


Process the Work

With the work divided, it is time to feed the Web Workers the operations in an efficient manner. To do this, we have an operation queue that is loaded with all the operations that we divided earlier. It is simply a matter of the Web Workers concurrently processing each operation until the queue is empty. The design of Web Workers prevents this from being a potential race condition as Web Workers cannot read from the queue, as you may have done in a traditionally threaded application. Instead, the queue is managing the Web Workers by sending one operation to each Web Worker and waiting for them to return a result before sending them the next operation.

This lets us encapsulate all the Web Worker code into one OperationQueue module, so the main logic only has to add an operation to the queue and wait for the callback when it finishes. It doesn't care if the operation is executed on the main thread, on a separate thread, on a server or a combination of all three.


Operation Queue

Let's take a look at the OperationQueue that I created. It is extremely easy to use, having only two public methods. You can add an operation to the queue and terminate an operation that is in progress.

Function Description
addOperation( message, callback, transferable )
  • message is the object sent to the Web Worker.
  • callback is a function that takes a single argument, the result from the Web Worker.
  • transferable is an optional array of Transferable objects that are in the message.
Returns the operation ID.
terminateOperation( id ) Terminates the operation with the given id.


//
// Operation Queue
//
var operationQueue = (function (workerJS) {
    var _operationQueue = [];
    var _availableWorkers = [];
    var _currentWorkers = [];
    var _id = 0;
    var _workerJS = workerJS;

    // Initialize the _availableWorkers array based on the number of cores.
    // If that is unavailable, it defaults to 4.
    for(var i=0; i<(navigator.hardwareConcurrency || 4); i++) {
        _availableWorkers.push(new Worker(_workerJS));
    }

    // Add an operation to the queue and immediately start processing.
    // Returns the operationID.
    function _addOperation(message, callback, transferable) {
        _id++;

        _operationQueue.push({ message: message,
                              callback: callback,
                                    id: _id,
                          transferable: transferable });

        _nextOperation();

        return _id;
    }

    // Terminates an operation, either in progress or in the queue.
    // If it is in progress, we have to terminate the Web Worker and
    // create a new one.
    function _terminateOperation(id) {
        for(var i=_currentWorkers.length-1; i>=0; i--) {
            if( _currentWorkers[i].operation.id === id) {
                _currentWorkers[i].worker.terminate();
                _currentWorkers.splice(i, 1);

                _availableWorkers.push(new Worker(_workerJS));
            }
        }

        for(var i=_operationQueue.length-1; i>=0; i--) {
            if( _operationQueue[i].id === id ) {
                _operationQueue.splice(i, 1);
            }
        }

        _nextOperation();
    }

    // Feed all the available workers operations if there are any
    // in the queue.
    function _nextOperation() {
        while( _availableWorkers.length > 0 && _operationQueue.length > 0) {
            var worker = _availableWorkers.pop();
            var operation = _operationQueue.shift();
            _currentWorkers.push({worker: worker, operation: operation});

            var msg = {id: operation.id, message: operation.message};
            worker.postMessage(msg, operation.transferable);
            worker.onmessage = _workerFinished;
        }
    }

    // Callback for the Web Worker. Uses the id to forward the results
    // to the object that added the operation.
    function _workerFinished( e ) {
        var id = e.data.id;
        var worker;
        var operation;
        var callback;
        var workerOpIndex = _currentWorkers.findIndex(
                                function(item) {
                                    return item.operation.id === id;
                                } );

        if( workerOpIndex != -1 ) {
            worker = _currentWorkers[workerOpIndex].worker;
            operation = _currentWorkers[workerOpIndex].operation;
            operation.callback( e );

            // The worker becomes available again.
            _currentWorkers.splice(workerOpIndex, 1);
            _availableWorkers.push(worker);
        }

        _nextOperation();
    }

    return {
        addOperation: _addOperation,
        terminateOperation: _terminateOperation,
    };
})('worker.js');

In your worker.js module you have to pass back the operation id.

onmessage = function(e) {
    var id = e.data.id;
    var inputValue = e.data.message.inputValue;

    // Do the work
    var outputValue = inputValue+1;

    // Return the result with the id
    postMessage({id: id, result: outputValue});
}


Example Usage

From the main logic, you would break up your work into small operations and then repeatedly call addOperation. For example:

var opID1 = operationQueue.addOperation( {startValue:1, endValue:10}, workFinished);
var opID2 = operationQueue.addOperation( {startValue:11, endValue:20}, workFinished);
var opID3 = operationQueue.addOperation( {startValue:21, endValue:30}, workFinished);

function workFinished(e) {
    console.log('The result is: ' + e.result);
}

You can terminate an in-process operation by calling terminateOperation with the operation ID returned from addOperation().

operationQueue.terminateOperation(opID1);

It's a very simple system to incorporate into already existing main logic.

Conclusion

Web Workers are an integral part of creating high performance JavaScript. If you are not used to writing programs with concurrency in mind, it can be difficult to start doing so. I hope I have shown how easy it can be with a system like an operation queue. It should be noted that this isn't only useful for things like fractal calculation, but also anything like a network request that could block on the main thread and freeze the UI.

I hope you have found this useful. Feel free to contact me if you have any comments or questions.

Question or Comment?