The strange world of Node.js design patterns

A world apart

Of course, the design patterns from the Gang of Four are still the reference for writing sound architectures, but we all know that in Javascript it seems all the rules we learnt with other languages are broken. Design patterns make no exception, be prepared to rethink them in Javascript, or forge new ones. Traditional design patterns in Javascript may be implemented in different variations, while simple coding tricks might be considered patterns because broadly used, understood and effective. Also don’t be surprised if some commonly recognized anti-patterns are actually widely used in Javascript/Node.js (for example, proper encapsulation is most of the time overlooked, because of the complexity in obtaining it, leading often to an Object orgy).

The list

The one that follows is a short list of common design patterns used in Node.js applications, it does not want to show you again how a Singleton or an Observer pattern are implemented in Javascript, but instead wants to go through some distinctive practices used in Node.js that can be generalized as design patterns.

This list is just a result of what I have learnt by writing Node.js apps and by looking at other people’s code, so it does not want to be complete nor exact. Any suggestion is greatly welcome.

And I’m pretty sure you may have already seen or used some of these patterns without even knowing it…

Directory require (pseudo plugins)

This is definitely one of the most popular patterns. It consists in simply requiring all the modules from a directory, that’s it. Although its simplicity it’s one of the most popular (and handy) techniques. Npm has plenty of modules implementing this pattern: require-all, require-many, require-tree, require-namespace, require-dir, require-directory, require-fu just to name a few.

Depending on the way it is used, requiring a directory can be considered a simple utility function or a sort of plugin system where the dependencies are not hardcoded into the requiring module, but are injected from the directory contents.

Example: simple usage

var requireDir = require('require-all');
var routes = requireDir('./routes');

app.get('/', routes.home);
app.get('/register', routes.auth.register);
app.get('/login', routes.auth.login);
app.get('/logout', routes.auth.logout);

Example: advanced usage (improve decoupling, extensibility)

var requireFu = require('require-fu');

requireFu(__dirname + '/routes')(app);

Where each module in /routes, is a function which defines its own url route:

module.exports = function(app) {
  app.get("/about", function(req, res) {
    // do things
  });
}

In this second example, you can add a new route by simply creating a new file, with no need to modify the requiring module. This is definitely more powerful, and offers better decoupling between the requiring and the required modules.

The App object (home brewed dependency injection)

This pattern is also very common in other platforms/languages, but the dynamic nature of Javascript makes this very effective (therefore popular) in Node.js. It consist of creating a single object that represent the backbone of an app. Usually this object is instantiated in the application entry point, and is used as glue for the different application services. I would say this is very similar to a Facade, but in Node.js it is also widely used to implement a very rudimentary dependency injection container.

A typical example of this pattern is when an application has an object called App (or with the same name of the application) where all the services are initialized and then attached to this big object.

Example

var app = new MyApp();

app.db = require('./db');
app.log = new require('./logger')();
app.express = require('express')();
app.i18n = require('./i18n').initialize();

app.models = require('./models')(app);

require('./routes')(app);

The App object can be then passed around to be used by other modules, in the form of function argument or required.

When most of the dependencies of an application are attached to this backbone object, modules using it will actually have those dependencies injected from the outside .

Now, be aware that if you use the App object pattern without providing some kind of abstraction over the loaded dependencies, you can easily end up in having an all-knowing object, hard to maintain, and that actually corresponds to the God object anti-pattern.

But fortunately there are some libraries that can help you with that, as for example Broadway, an architectural framework which implements a quite neat version of this pattern, offering a good abstraction and more control over service lifecycle.

Example

var app = new broadway.App();
app.use(require("./plugins/helloworld"));
app.init(...);
app.hello("world");

// ./plugins/helloworld

exports.attach = function (options) {
  // "this" is our App object!
  this.hello = function (world) {
    console.log("Hello "+ world + ".");
  };
};

Function hooks (monkey patching meets AOP)

Function hooks is another design pattern typical of a dynamic language as Javascript, and as you may guess also very popular in Node.js. It consists in augmenting the behaviour of a function (or method) by intercepting its execution. Often by applying this technique the developer can intercept the call before its execution (pre hook) or after (post hook). The peculiarity is that in Node.js it’s often used in combination with monkey-patching and that’s what makes it very powerful but also dangerous.

Example

var hooks = require('hooks'),
    Document = require('./path/to/some/document/constructor');

// Add hooks' methods: `hook`, `pre`, and `post`
for (var k in hooks) {
  Document[k] = hooks[k];
}

Document.prototype.save = function () {
  // ...
};

// Define a middleware function to be invoked after 'save'
Document.post('save', function createJob (next) {
  this.sendToBackgroundQueue();
  next();
});

If you ever used Mongoose then you must have seen this pattern in action for sure, otherwise npm offers you a plethora of modules to choose from. But it doesn’t stop here, in the Node.js community Aspect Oriented Programming (AOP) is often synonym of function hooks , take a look again at npm to see what I mean. Can we really call this AOP? My answers is NO. AOP requires cross-cutting concerns applied to a pointcut, not some specific behavior manually attached to a single (or even a set) of functions. On the other hand an hypothetical Node.js AOP solution may easily exploit hooks to apply advice to a set of functions, based for example on a pointcut defined with a regular expression and by scanning all the modules/methods in search for a match.

Pipelines (a.k.a. middleware)

This is the essence of Node.js, Pipelines are everywhere, under various forms, uses and purposes. Generically, a Pipeline is a series of processing units connected together, where the output of one unit is the input for the next one. In Node.js, this often means a series of functions in the form:

function(/* input/output */, next) {
    next(/* err and/or output */)
}

You might know it as middleware and when you think middleware you say Connect or Express, but the usage of this pattern goes well beyond that. Hooks for example, a popular hooks implementation (we talked about it previously) connects all the pre/post functions in a pipeline (middleware) to “give you maximum flexibility”.

Most of the time you will implement this pattern in some way by using async.waterfall or async.auto, or using a chain of promises and maybe not just as flow control, but to make some part of you application extensible.

Example: Async

async.waterfall([
    function(callback){
        callback(null, 'one', 'two');
    },
    function(arg1, arg2, callback){
        callback(null, 'three');
    }
]};

Another Node.js popular component has the traits of a Pipeline. Of course I’m talking about the hero called streams, and what is a stream if it cannot be piped? While middleware and in general function chains are a generic solution to control flow and extensibility, streams are better suited to process flowing data in the form of bytes or objects.

Example: streams

fs.createReadStream("data.gz")
    .pipe(zlib.createGunzip())
    .pipe(through(function write(data) {
        //... make the data awesome ...
        this.queue(data);
    })
    // Write to File
    .pipe(fs.createWriteStream("out.txt"));

Conclusions

We have seen how the nature of Node.js pushes the developers towards the use of some recurring patterns and techniques and we went through some of them, showing how they can effectively solve common problems, if used properly. We have seen also how different a pattern can look depending on the way it is implemented. Feel free to leave a feedback and to let me know of any other pattern I missed in this list.