nodejs parallel callback design pattern


I'm trying to find a good pattern to execute a bunch of parallel tasks.

Let me define some task to exemplify. Tasks a, b, c, d, e, f, g execute as a(function(er, ra){//task a returned, ra is result}), so do b to g

There are also some tasks that should be execute after some task is done, let's call them ab, bc, abc, bd, bcd, af, fg, means when a and b has returned ab(ra, rb) should be executed at once, and when b and c returned, bc(rb, rc) should be executed at once, and if a, b, c all returned, abc(ra, rb, rc) should be executed.

For the simplest case, if there is only a and b, I can do something like this:

    var count = 2, _ra, _rb;
    function update(){if(--count == 0) cb(null, _ra, _rb)}
    a(function(er, ra){_ra = ra; update()});
    b(function(er, ra){_rb = rb; update()});
})(function(er, ra, rb){
    ab(ra, rb);

As you can see, a and b execute in parallel, and when both are done, ab(ra, rb) execute.

But how can I do more things for a lot of parallel tasks?

10/25/2013 11:05:45 PM

What you actually want is a deferred pattern though like futures.

function defer(f) {
    // create a promise.
    var promise = Futures.promise();
    f(function(err, data) {
        if (err) {
            // break it
        } else {
            // fulfill it
    return promise;
var da = defer(a), db = defer(b), dc = defer(c), dd = defer(d), de = defer(e), df = defer(f), dg = defer(g);

// when a and b are fulfilled then call ab
// ab takes one parameter [ra, rb]
Futures.join(da, db).when(ab);
Futures.join(db, dc).when(bc);
// abc takes one parameter [ra, rb, rc]
Futures.join(da, db, dc).when(abc);
Futures.join(db, dd).when(bd);
Futures.join(db, dc, dd).when(bcd);
Futures.join(da, df).when(af);
// where's e ?
Futures.join(da,db,dc,dd,de,df,dg).fail(function() {
5/13/2011 7:54:38 PM

Licensed under: CC-BY-SA with attribution
Not affiliated with: Stack Overflow