Async and recursion in nodejs

Question

Beginning with express and mongoose i often need to do some batch operations on collections. However it usually involves callbacks which is a pain given how concurrency is coded in nodejs. so basically

//given a collection C 
var i = 0;
var doRecursive = function(i){
    if(i<C.length){
      C[i].callAsync(err,result){
        i=+1;
        return doRecursive(i);
       }
    }else{
      return done();
    }
}
doRecursive(i);

Now i dont remember what is the max stack before i get a stackover flow with node , but i guess with 10 000 elements , it wont do. I wonder if there are other ways to handle this, if yes , what are they? thanks

Well, asynchronous functions will help with stack overflows as the callback will be in a different call stack. But, by breaking out into a different stack, it renders return rather useless. — Jonathan Lonowski
– Jonathan Lonowski, Commented Jul 12, 2013 at 19:09

Jonathan Lonowski · Accepted Answer · 2013-07-12 19:19:38Z

5

If the goal is to iterate an collection asynchronously, there are numerous control flow libraries available.

A good example is async and its reduce function:

async.reduce(C, 0, function (memo, item, callback) {
    item.callAsync(function (err, result) {
        if (err) {
            callback(err);
        } else {
            callback(null, memo + result);
        }
    });
}, function (err, result) {
    // ...
});

Note: It's not entirely clear what value you wanted to get from doRecursion, so this just uses addition for an example.

answered Jul 12, 2013 at 19:19

Jonathan Lonowski

124k35 gold badges203 silver badges202 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

mpm Over a year ago

I know async. does async has a way to generalize recursion so recursion can be written in a "prettier" fashion ? People are telling me there should be no stackoverflow issue with recursion and async calls, i need to test it to be sure. EDIT : i see you are using Async.reduce , interesting ,did not know about that one.

dandavis · Accepted Answer · 2013-07-12 19:42:12Z

1

i think you can simply self-iterate instead of true recursion, since you're not drilling into a deep object:

function doRecursive (C, i){
    i=i||0;
    if(i<C.length){
       C[i].callAsync(err, function(result){
          doRecursive(C, ++i);
       });
    }else{
       done();
    }
};

doRecursive(C);

this does not create a tall stack if the code functions as labeled. i localized C so that it executes faster and is potentially re-usable on other collections. the pattern also makes it easy to defer it for long-running operations, just by changing

doRecursive(C, ++i);

to

setTimeout( doRecursive.bind(this, C, ++i), 50 );

answered Jul 12, 2013 at 19:42

dandavis

16.8k5 gold badges43 silver badges38 bronze badges

Collectives™ on Stack Overflow

Async and recursion in nodejs

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related