using javascript templates

You know the feeling when you’re a bit obsessed with a topic and you discover someone else is just learning about it? It would be like if you used Vim and I said I only know like five Vim commands. You’d be like, “TIME TRAVELER! I should have known when you tried to bite through your change at that coffee shop!” It’s not that I recently traveled through centuries, robbed an H&M, and learned to write ‘th’ as two separate letters, though – it’s just that I’ve never really used Vim (and I don’t want to, and I don’t want you to try and talk me into it). So it is with me and JavaScript templates. I was surprised, therefore, to search these internets and not be able to find a comprehensive blog post on how to use them. Due to my perhaps abnormal interest in the topic, I thought I would try to provide one.

To get started with JS templates, there are two things you need to agree to, as a default position (down the road, optimization might take you to a different conclusion). First:

It is bad to write HTML in your JavaScript.

That’s easy. It’s so easy I’m not even going to explain it. To do otherwise is difficult to maintain and fraught with ridiculous problems like extremely long strings and incorrectly escaped apostrophes. Second:

It is bad to send data from your server as HTML instead of JSON.

If you don’t want your pages to be dynamic, that’s perfectly fine. So fine, in fact, that you’re excused from reading the rest of this blog post and can go back to whittling yourself a neckbeard trimmer. If you do, though, sending HTML in response to an XHR is silly. What are you going to do with that? Nothing, Einstein. Nothing is what you’re going to do with it. You can insert it into the page once and after that it’s just more junk. If you’re not going to send data back from the server you might as well have static pages.

Still reading? Ok, great. So we have established that we need templates, because the alternative we’re left with is lots of DOM manipulation and nobody wants that. Let’s talk how these things get used.

replacing sections of a page

This is the most basic component. You get some data. You have a template. You hydrate said template. You attach the result to the DOM. And you need to think no more deeply about it than that.. if all you’re making is a proof of concept. For performance, we should probably add a requirement that these templates be precompiled, and that the resulting functions be cached somewhere. For flexibility, we probably want these templates to be composable, so that we don’t have to re-render more of the page than is affected by whatever data we’ve received. So add an additional requirement for partials. If we want don’t want to present all the data exactly as it’s received, we need to be able to control the scope the template function executes in, so that we have access to utilities that transform raw numbers and strings into things like currency and proper names. How we do that depends on the template engine we choose, but we need to be able to do it somehow.

packaging templates

How templates get loaded is an elephant that reveals itself very quickly in whatever room you happen to be designing your application in. Most template engines will accept any string as input, meaning the templates themselves can exist anywhere. Some people think that means they should be chunks of HTML in your normal, server-rendered page. I think that’s foolhardy and inflexible. Templates should go in their own files. In some engines, partials can be defined inline, and in certain scenarios that makes sense – for example, if you have a module that needs to be re-rendered frequently, and some subsection of it will be re-rendered alone only if some error occurs. This assumes, though, that the error state partial isn’t useful anywhere else in the app. Otherwise it should be its own file.

A lot of us start experimenting with templates naïvely lazy-loading them with XHRs. If you begin to break your templates up intelligently, though, that quickly turns into a lot of HTTP requests. Templates need to be packaged up and delivered the same way any resources do, minimizing requests while maximizing the number of manageable chunks the client has the opportunity to cache. You don’t want to concatenate all your templates into one long JS file as variables, though – then you just have the same old line break and character escaping headaches again. You want to concatenate the compiled functions they produce. Those functions can then end up in the same nice page-specific concatenated JS file you were going to send down to the client anyway.

templatizing CSS

While we’re talking about resource loading, it’s a good time to mention that a lot of template engines are output agnostic and will build any kind of code you want, including CSS. So if you’re making changes that might also require loading a lot of CSS you might not need, or that will break your page under the wrong circumstances, you can put that in a template, too, if you like. This might be nice if you want to A/B test some changes based on a factor you’ll only know in a certain client-side state, for instance. Your CSS can be a template function you feed your parameters into to output the correct version without needing to change classes in the DOM, you can attach the resultant CSS to the DOM, and you’re done.

templates outside of SPAs

Mostly we talk about templates for single-page apps, because a good deal of rendering needs to occur there, but that’s not the only place they’re useful. There are one-off areas on a lot of otherwise static pages that might have cause to re-render outside of any formal concept of an app. And then there are the static pages themselves. As soon as a page needs to be rendered once on the server and then another n times on the client, we have a case for shared templates. Before you say it, no, this doesn’t automatically mean you have to use Node, and it doesn’t mean you automatically have to use Mustache. Use whatever the hell you were going to use anyway. If there’s no parser for that template engine with that server-side language, write a parser. Or, you know, use Node or Mustache. But remember that this is computer programming and there’s always a way to do it, it’s just a matter of writing more code.

Of course, once you start using JavaScript-y templates to render some pages, you have to ask whether you want to use JavaScript-y templates to render all your pages. Maintaining two templating systems server-side might get to be kind of a pain in the ass eventually. You also have to ask how easily the JS-y templates can be used in your other templates as partials, or whether that’s even an option. (I might be completely trolling you here.) The goal of sharing templates is not having to write two templates for the same thing. To that end, it’s probably best, if a page contains a JS-y partial, to just convert the whole page to your JS-y template syntax. If that turns out to be a lot of pages, it may indeed be worthwhile to convert the remaining static pages to the same syntax, just for consistency, and in case you want to someday Ajax-ify your Terms of Service. If it’s just your one or two “fancy” pages, it’s probably fine to just keep them in separate directories and live with the ambivalence.

controlling rendering

If you’re doing this server-side, presumably whatever parser you’re using already hooks into your routing and controllers. So that’s awesome. If you’re doing it client-side and using an application framework, the pattern seems to be that either rendering is completely up to you and it’s a piece of cake, or your framework is so tightly tied to your markup that we’re not even having this conversation. If you’re not using a client-side framework you will probably need to write something. The nice thing is that you concatenated all your templates as compiled functions and passed them all in when the page loaded, or required them for the section of code where they’re needed, so all you have to do is pass them data and put the result in the right place. This is where it becomes handy to have all your templates and partials on the same object. Some engines do this already, some will need to be managed by you. This is so you can write a nice abstraction that accepts the name of the template to render, its data, and its container, and you don’t have to worry about which partials it’ll be using. Some engines also need any helper functions used within them passed in each time they’re rendered, and that’s another thing you probably want to keep separate from your main code.

widgets and plugins

There’s an inherent annoyance in creating utilities for distribution (even if it’s just among various pieces of your own app) that need to manage their own HTML and CSS. You want the person using the widget to be able to implement it with no more than an initialization call and some small set of options. You don’t want them to have to import the JS, HTML, and CSS on every page where it’s used, particularly if those pages might be built up from other server-side templates and the place the widget is used might be a partial within that system. At minimum, it’s an improvement to only have to import the JS, and have it take care of the rest (even better would be to use AMD or something to load the widget).

Putting your HTML and CSS in templates makes distribution easier, cause you have something with (presumably) a different file extension that can live in the same directory as the script that will load it and not look like it’s been misorganized. Better still, the minified version of the widget can contain the compiled templates as functions, so its users only need a single file. If you offer an option to let users supply their own path to a template, you can make your templates overridable defaults, though you probably want to set some ground rules about what template engine is allowed and how templates will be loaded and cached (e.g., are you expecting to use require.js).

mixing template engines

I’m going to be straight with you people: we have too many template engines and that’s a problem because of everything above. Templates themselves help us modularize our HTML and whatever else in the same way we can our JS, but using a bunch of different types of templates makes things messier because we then have modules we can’t share. If we want to combine the modules for use in some implementation, we have to load all the template engines needed. And that’s no big deal, the modules can take care of doing that, provided it’s not via some naïve mechanism that will add a script tag to the page potentially loading the same engine again and again. Except that to make that provision we have to assume they’re using a sophisticated loader, which is also a dependency.

The best solution is simply to not mix template engines. The likelihood that two different parts of your app require templates to be marked-up or parsed in a different way is kind of small. Choosing a template engine is like choosing a library in certain ways (which is why it’s nice when libraries like Underscore or Dojo provide their own): it allows your app to be consistent in how it handles things, and pretty much needs to be decided before you can get any large-scale work done affecting disparate parts of the app. If template strings (or quasi-literals) become available in everyday JavaScript, they won’t address all the things templates do, but they will hopefully create more consistency in how various template engines work, and ideally, narrow the field a little bit. In the short term, however, it’s best to choose a template engine you like and just use that one. If there’s a third party utility you need that uses something different, or none at all, it’s probably trivial to create a version that cooperates with your template engine by changing the delimiters in the template and the functions called to compile and render it. If you really need multiple template engines, you could create a wrapper and call that instead of the template engine directly. That would also make it easier to change your mind in the future.


That’s everything I could think of. If I missed something, let me know!

Traducción español, gracias a Webhostinghub.com.

Tags:

12 Responses to “using javascript templates”

  1. Angus Croll Says:

    Nice article Garann.

    Just one thing: I disagree with “It is bad to send data from your server as HTML”. Sure, you would usually want to send the response in JSON form if for no other reason than to have an easily addressable data packet; but I see no reason that one or more of those JSON properties couldn’t be a chunk of HTML rendered on the server.

    This strategy doesn’t mean no templates – it’s just sometimes it’s more practical (and performant) to have you templates live on the server and run over Node, Java, Ruby or whatever. Server-side templating obviates the need to duplicate state on the client and can vastly reduce the size of the initial JavaScript load.

    There’s no right or wrong here – either way its MVC – but I think it comes down to whether the product is sufficiently interactive and “app-like” to warrant client-side MVC.

  2. Kyle Simpson Says:

    Great article. Agreed with pretty much every observation you made.

    One thing I wish you could go into more depth on, at least your thoughts, is how (or even if!?) to manage and use code-behind with your template partials. It always seemed bizarre to me to use templates for separation of concerns (no templates in code), and then violate SoC by putting code into your templates. There has to be better patterns for this, or the conclusion is code-behind as a pattern is more harm than good.

    On the other side of the spectrum, how do you feel about the logic that is required to transform (sometimes heavily) your data so that logic-less templates (mustache, etc) can actually do their job? Where does all that data transformation logic live? How do you keep it separate from the templates, but keep those tightly integrated transformations in sync with your templates, when either or both of them change?

    To me, this kind of discussion is what’s really missing from the template engine universe right now. I have strong opinions on it that I’ve baked into my templating engine “grips”, but I really would like to know how other smart people think about this stuff for real world scenarios?

  3. Mike McNally Says:

    Hi Garann. I’ve been using doT.js on an overhaul of our web application. This post has given me significant relief in that it suggests I haven’t been doing horribly sinful things. I’ve really enjoyed the flexibility of moving from JSP (ugh) to doT (actually I still use JSP to build JSON responses, a purpose for which it works OK).

    One thing that I do that’s not really emphasized in your post is to exploit the template system for common application widgets. For example, my site has a convention of marking forms etc. with little [?] buttons (we call them, “what’s this? bubbles”). You click/tap on a question mark button and up pops some explanatory text, often with live data mixed in (“Here’s why your account balance is $123″ etc). Well to do that there’s some boilerplate HTML that’s repeated to set things up for the JavaScript that responds to the clicks – class names, positioning elements, etc. In the JSP world, I used a custom tag, but in doT I just have a template that does the opening and closing markup. That’s good for several reasons: it avoids repeating that markup in template after template, saving some space. It also means that if I need to update the “What’s this?” mechanism, I can do it in just one place.

    I realize that “calling a template from inside a template” is heretical to some people, as one could think of it as “logic”. Personally, I don’t think of it like that at all, and even if I did I think that sticking to DRY principles is far more important. (I also realize that not all template systems allow this; in my case it’s one of the main reasons I love doT so much.)

    Thanks for the awesome post!

  4. Luís Cardoso Says:

    First, nice article. It speaks to some of the issues that I have been encountering regarding client side templates.

    In the end of the article you talk about having a nice abstraction that accepts the name of the template. Currently I am thinking about taking that a bit further by organizing my client side code in a way that for every view (as in MV* views) there is a folder which contains the view itself, corresponding template and CSS/LESS/whatever. In this approach the template and CSS would be automatically associated with the view. Do you think this is a good approach? I think it will help a lot with maintainability and reduce the time that I spend making boilerplate code but I am a bit afraid that I will run into limitation as my views become more complex.

    Thanks!

  5. Mike McNally Says:

    @Kyle – because I was converting a JSP (server-side template) application to client-side, I decided to ease my pain by concocting some build-time tools to allow me to keep my client-side template code in the same files with the server-side JSP that builds JSON responses. That way, as I realize needs for things like boolean flags that require server-side data/logic, I can update the JSON-builder code in the same file as the template that consumes it. It’s turned out to be really handy, if a little weird.

    All the tools do is split up the JSP source into the pure JSP part (which is handled pretty ordinarily by the rest of the server-side code), and then one or more template files. Those are then translated by doT and collected into a single blob for the client. (My app isn’t big enough for me to worry about separate template bundles and require.js or anything like that.)

  6. garann Says:

    @Angus – Ah, but I didn’t exactly say that. ;) I said “… instead of JSON”. As part of some JSON is a different story, cause you still have access to the data without having to scrape it out of some markup. I do see your point, though. And I agree that client-side rendering is not the right choice for every application. I think there’s a stronger argument for it in very application-y applications, where there’s a high level and probability of interactivity for everything on the page, and referring back to the server for the application’s state might become onerous. Even there, though, optimizing is something that can only be done on a case-by-case basis. I guess my stance is that if you’re sending data to the client, you should expect as a default to use it as data, and thus you should expect as a default to do client-side rendering. If you ultimately find that it makes more sense to do it server-side, of course that’s the right answer.

  7. garann Says:

    @Kyle – I kind of stayed away from that topic, because I didn’t want to get to specific to any one approach. I’m in favor of violating SoC up to a point, but I’m also very aware that then you have to be able to parse your template as JS on the backend, if you’re sharing templates, and that can be a big problem. But the alternative is having duplicate presentation logic libraries for the client and the server, and I’m not sure if that’s objectively better. Well, I guess there’s also the third path, transforming the data when it’s requested, which is what I think you mean by “code-behind”? (My .NET experience might be confusing my interpretation here.) I see that as hurting the flexibility of the data returned, though, because you might have to parse a string to get the data you actually want.

    I don’t think there’s a single ideal approach. One of my favorite things about using JS on both the front- and back-end is that you can then use the same data transformation layer, in the same file, for each case. If you don’t have that luxury, I think you have to accept that you’re going to violate SoC, DRY, or both. That might be a reason to question whether you need to do rendering on both sides after all.

  8. garann Says:

    @Mike – Thanks! Self-contained widgets are one of my favorite use cases for templates, actually, and that’s totally an oversight on my part. I’m going to add something. Also, I agree that doT is awesome. :D

  9. garann Says:

    @Luis – Yeah, that sounds like a sane way of organizing it to me. Would your template and CSS also contain the various states the View could be in? I think this ties into what @Mike is saying about widgets, so I’ll take it into account when I add a bit about those. Thanks!

  10. Andreas Müller Says:

    Nice Article! Have you tried client side templating tools, e.g. google closure templates, mustache, jquery-templates?

  11. Pete Peterson Says:

    I would like to add a point of frustration with JS templating and the whole front-end framework movement in general.

    Firstly, mostly the frameworks are built, or intended for single page apps as you pointed out. The problems in having maintainable code, templates and resources across a larger site comes down to having strict design guidelines, so the bulk of the JS logic and the templates can be reused. This is minor for a new build, horrible for retrofitting an older larger system that has very little reusable “modules”.

    Secondly, as @Kyle pointed out, and in your response to him. There is some inherent logic that needs to be handled in displaying some information. That logic, at least I’ve found, has to be replicated on both the back end and the front end in some cases. This drives me crazy, but I don’t know of way around the issue unless all the rendering gets done either on the front end or the back end. There are pros and cons to either so….whats a guy to do other than cringe and replicate logic? I find that there are more cases where violating SoC, DRY than I feel comfortable with.

    I’ve found the roadblocks and issues with client-side rendering to be a huge PITA. I would love a much simpler solution, but have yet to find one that works. Perhaps this approach is truly best suited for a strict design pattern on new sites. IDK.

    I am glad to know that what I have done is generally matching your approach. I’m interested to see how this space develops. I would love to see more REAL world examples how others are solving some of these issues. The de facto TODO app examples are worthless in this regard.

  12. Enrique Amodeo Says:

    Good post, specially the part about self contained widgets.

    I’ve been reading the comments and I think the topic about transforming your data so that it can be used to hydrate a logic less template is very important.

    Some people want logic full template because they want to embed that kind of transformation logic into the template, but that violates the SoC. Templates should be used to render HTML (or CSS…) but not to transform data too, they are completely unrelated concerns. That’s why I prefer logic less templates.

    It is usually the case you need two different models. A model to use for business logic and another one for template rendering. The transformation between both can be done either in a method of the former, or in an adapter/wrapper for the template. This transformation can take care of formatting issues (currency, dates) and even structural changes. The formatting logic usually have internationalization issues, so it’s not so simple as may appear at first look. Of course if both data structures are exactly the same, and no special formatting is needed, this intermediate transformation layer is not needed. This way we avoid violating SoC and have a more testable system.