Async calls which remember their return values in node.js

Pretty much everyone and anyone who has coded node.js has used or encountered the async module at some point. Async.js provides a range of methods to simplify asynchronous code at the same time as making the intention of that code more explicit. Collection methods like “each” and “map”, along with control flow methods like “series”, “parallel” and “waterfall” make it clear whether the code can be run in parallel or sequentially, making it easy to take advantage of node’s non-threading model. It also makes for code that is much less deeply nested and easy to read.

There was one case which I frequently encountered though, with async wasn’t able to deal with. This was the case where I had multiple methods which each returned something I wanted, and they could all be run in parallel, but I needed to know when they were all complete.

The simplest example of this case is rendering a webpage where I needed to retrieve data from multiple sources before rendering out the final page. Retrieving data meant calls to Mongo or to an SQL database, so these were definitely tasks which should be taking advantage of node’s multitasking model… So running them with “series” was a bad idea. But if I ran it with “parallel” or “reduce”, I wouldn’t be able to get back the results. What I wanted was to be able to hand over a handful of functions and get back a hash of function-to-results.

This is how to do just that.

    gather: function (hash, done){
        var count = Object.keys(hash).length;
        var returned = 0;
        var errs = {};
        var returns = {};
 
        var localDone = function(err, data, key){
            errs[key] = err;
            returns[key] = data;
            returned++;
 
            if (returned == count){
                done(errs, returns);
            }
        };
 
        Object.keys(hash).forEach(function(key) {
            if(hash.hasOwnProperty(key)){
                process.nextTick(function(){
                    hash[key](function(err, data){
                       localDone(err, data, key);
                    });
                });
            }
        });
    }

This can then be called like this:

   util.gather({
        user: function(done){
            users.findByUsername(req.params.user,done);
        },
        data: function(done){
            data.findByUsername(req.params.user,done);
        }

Which then returns a hash with the keys ‘user’ and ‘data’, each containing the data from the relevant function, all in the shortest possible time.

 

 

 

One thought on “Async calls which remember their return values in node.js

  1. The value returned by a callback is bubbled up the chain of promises. The same is true of rejections. This allows for rejections to be handled at a higher level than the function that called the promise-returning function that is rejected. This also allows promise-based APIs to compose responses for rejections and successes by changing the value or error as it is bubbled.

Leave a Reply

Your email address will not be published. Required fields are marked *

*