nested callbacks in jquery begone!!

A week or two ago I noticed a bunch of people having trouble because of their nested callbacks, or with flow control in general, in jQuery. I said something about this on twitter, saying people should use jQuery’s Deferred object, and several people commented that Deferred is a really difficult concept to explain. I kind of disagree. Or rather, I think the difficulty comes from how we’re trying to explain it. So I wanted to share my experience, which begins not with “Oh jQuery has this new cool Deferred thing I guess I should go figure out WTF that is,” but with me doing lots and lots of nested callbacks.

I had this application that needed to send a bunch of updates to the server, and then redraw. The structure of the data being updated was sort of like object.rows[n].images[], and all those pieces – the parent object, each of its rows, and the images for each row if present – had to be updated via their own API calls. So the code looked sort of like this:

app.saveObj = function(obj) {
	obj.rowsToSave = obj.rows.length;
	$.post(someURL,obj,function(data) {
		$.each(obj.rows,function() {
			var row = this;
			row.parentId = data.id;
			obj.imgsToSave += row.imgs.length;
			$.post(otherURL,row,function(data2) {
				$.each(row.imgs,function() {
					this.rowId = data2.id;
					$.post(uploadURL,this,function() {
						obj.imgsToSave--;
						if (obj.imgsToSave == 0) {
							obj.rowsToSave--;
							if (obj.rowsToSave == 0) {
								app.renderObj(obj);
							}
						}
					});
				});
				if (row.imgs.length == 0) {
					obj.rowsToSave--;
					if (obj.rowsToSave == 0) {
						app.renderObj(obj);
					}
				}
			});
		});
	});
};

Or something like that. That’s essentially pseudo-code, and I know there were lots of functions that abstracted certain parts of that out, but it was more of less like that. I know there was some goofy shit with passing references to parents and parents-of-parents around in the callbacks, and those counters were definitely in there (unavoidably, I think). That’s nuts. At the time I’d never heard of deferreds or promises, and I thought flow control was a feature of shower heads. But even totally ignorant of the real solutions, I knew enough to feel immensely dirty about my own.

I think there are a lot of people in that same boat right now.. There are a couple different solutions that I learned about along the way, but if you’re using jQuery, the Deferred object is kind of the most obvious. I formed that opinion thanks to these resources, which you should totally check out if you haven’t:

(Ordered by when I saw them)
Rebecca Murphey’s post Deferreds coming to jQuery?
Eric Hynds’ post Using Deferreds in jQuery 1.5
Dan Heberden’s jqcon presentation Deferreds – Putting Laziness to Work

Looking at the mess of code above and knowing what I know now, here’s how it could be rewritten to be somewhat less shameful:

app.saveRows = function(obj) {
	var d = new $.Deferred(),
		count = obj.rows.length;
	$.each(obj.rows,function(i, row) {
		row.parentId = obj.id;
		$.post(otherURL,row,function(data2) {
			count--;
			$.when(app.saveImgs(row))
			.then(function() {
				if (count == 0) {
					d.resolve();
				}
			});
		});
	});
	return d.promise();
};

app.saveImgs = function(row) {
	var d = new $.Deferred(),
		count = row.imgs.length;
	$.each(row.imgs,function() {
		$.post(uploadURL,this,function() {
			count--;
			if (count == 0) {
				d.resolve();
			}
		});
	}
	return d.promise();
};

app.saveObj = function(obj) {
	$.when($.post(someURL,obj,function(data) { obj.id = data.id; }))
	.then(app.saveRows(obj))
	.then(app.renderObj(obj));
};

I don’t want to present this as though it’s super elegant or a magical solution. It’s actually more lines of code and, as far as I know, the stupid counters can’t be gotten around. Yeah, $.Deferred.resolve() is basically substituted for calling the callback. On the other hand, it seems much cleaner to me than the fake example, and you’re just going to have to trust me when I say that it’s worlds cleaner than the code I actually wrote. Mostly, it’s sustainable. The callbacks wait neatly chained for the event that indicates they should be called instead of being carried around through nested function call after nested function call (though the first example is condensed, that’s essentially what’s happening there – keeping the anonymous callback functions inline just makes it appear more connected).

I just fixed the front page of this site after a year, and added in deferreds to help manage some nasty bullshit with animations (not my strong suit) getting messed up if the animation function was called again before it had finished. I had two concurrent animations running each time I wanted to “flip a page”, and then they both had a couple of cleanup things they’d do upon completion. Now that list of final tasks happens only once both have run, and it’s out of the callbacks and nicely chained. It’s readable and consistent. As before, it’s not really solving a problem for me – the bug with the animations was related to a variable changing by the time the callbacks ran – but it makes for better code.

For me, the difficulty in understanding deferreds came in realizing that they aren’t magic. JavaScript only works this one way, so you still have to be able to think of your code in terms of, “I need this to happen and then I need something to trigger a different thing.” You have to map out the dependencies in your head or on paper and figure out the places where you need to wait. To me, nesting and passing callbacks has always felt conceptually like a lot of overlapping loops, loops that get paused part of the way through and then your app has several levels of functions waiting to move forward. I like deferreds because they make those feel more separate. Do this thing, and only once it’s done, begin this other thing. Under the hood I assume it’s still just passing around references to functions, but deferreds are a much easier way to think about this stuff.

So anyway. If you’ve also been waiting to grasp the nature of the problem deferreds magically solve to look into them them, you should stop. You should also stop nesting and passing around callbacks. It’s difficult to read, it’s difficult to manage, and it scales just like shit. I’m not really the best person to be talking about this, but I can say fairly confidently that deferreds offer an improvement. And, you know, maybe there is a magical solution somewhere in there that I just don’t know about yet. If you find one, you should tell me.

Tags:

4 Responses to “nested callbacks in jquery begone!!”

  1. Rebecca Murphey Says:

    So, yes, you’ve just substituted .resolve() for the callback, but what’s really exciting is that you’ve broken things up into stuff that you can actually *reuse* — you can’t say that about a bunch of nested anonymous functions. If you craft each of the steps well, and each of the steps returns a promise, then the mixing and matching opportunities are endless. Many times now I’ve found myself using a method that returns a promise in a way that I didn’t plan for when I wrote the method, and now pretty much any async method I write returns a promise.

    It gets even more fun when you have methods that may or may not be async — with the power of .when, you can have a ton of flexibility inside the maybe-async method while code that consumes the output of the maybe-async method doesn’t have to give a crap about what’s going on behind the scenes.

  2. garann Says:

    @Rebecca – Sorry, I didn’t make that point very well.. The example code up there has nested anonymous functions, but what I was thinking about when I said resolve() was just a substitution for a callback was having a separate named function that would be reusable. But you hit on the big thing I missed, which is that with deferreds, those functions never have to know about the rest of the flow. They just do their thing and the flow control is abstracted out. Not that, again, you can’t separate some of that functionality from the flow control without deferreds, but at some point the flow control code itself still gets nested, even if the thing you want controlled doesn’t.

    Which is to say: thank you!

  3. Edgar Says:

    hmm i’m having a bit of a tough time here. Yes callbacks work. although they were ugly, yes this seems cleaner and more reusable, at least when you try and read the code in some time from now you wont have to follow all these functions calling other function calling other.

    I’m having a problem now. Is this suposed to be used only once? i guess if it’s resolved it’s closed. Should one use some other thing when you want to run it more times or am i doing it wrong?

    Should i be using jquery Callbacks Object instead?

  4. garann Says:

    @Edgar – You’d use it once each time you performed the async operation. Like, in the above code, the then() functions will get called every time that request finishes, which is to say, every time app.saveObj() is called.

    I have to admit I don’t know much about the Callbacks object myself.. My impression is that you’d use it if you wanted more control over which callbacks were getting executed when. Sounds like the normal implementation of Deferred would work ok for your scenario, though?