What's the proper way of chaining async functions in Node.js?

Question

I have an interesting case where I need to do a few queries in MongoDB using Mongoose, but the response is returning before I can complete all of them.

I have two document types, list and item. In one particular call, I need to get all of the lists for a particular user, then iterate over each of them and fetch all of the items and append them to the appropriate list before returning.

List.find({'user_id': req.params.user_id}, function(err, docs){
  if (!err) {
    if (docs) {

      var results = [];

      _und.each(docs, function(value, key) {

        var list = value.toObject();
        list.items = [];

        Item.find({'list_id': value._id}, function(err, docs) {
          if (!err) {
            _und.each(docs, function(value, key) { list.items.push(value.toObject()); });
            results.push(list);
          }
          else {
            console.log(err);
          }
        });
      });

      res.send(results);

(_und is how I've imported underscore.js)

Obviously the issue are the callbacks, and since there's multiple loops I can't return within a callback.

Perhaps this is a case where I would need to get the count in advance and check it on every iteration to decide when to return the results. This doesn't seem elegant though.

Raynos · Accepted Answer · 2012-03-01 11:02:56Z

Code solution

First of all the issue is with the code. Your sending the results before the Item.find queries finish. You can fix this quite easily

var count = docs.length + 1;
next()
_und.each(docs, function(value, key) {

    var list = value.toObject();
    list.items = [];

    Item.find({
        'list_id': value._id
    }, function(err, docs) {
        if (!err) {
            _und.each(docs, function(value, key) {
                list.items.push(value.toObject());
            });
            // push asynchronous
            results.push(list);
            next()
        }
        else {
            console.log(err);
        }
    });
});

function next() {
    --count === 0 && finish()
}

function finish() {
    res.send(results)
}

The easiest way is reference counting, you default count to the number of documents. Then every time your finished getting an item you call next and decrement the count by one.

Once your finished getting all items your count should be zero. Note that we do .length + 1 and call next immediately. This gaurds against the the case where there are no documents, which would otherwise do nothing.

Database solution

The best solution is to use mongo correctly. You should not be doing what is effectively a join in your code, it's slow and inefficient as hell. You should have a nested document and denormalize your list.

so list.items = [Item, Item, ...]

As a further aside, avoid mongoose, it's inefficient, use the native mongo driver.

4dan · Accepted Answer · 2012-03-01 05:34:18Z

1

I use with this module: https://github.com/caolan/async

answered Mar 1, 2012 at 5:34

4dan

1,07310 silver badges14 bronze badges

Collectives™ on Stack Overflow

What's the proper way of chaining async functions in Node.js?

2 Answers 2

Code solution

Database solution

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Code solution

Database solution

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related