Creating a Template Engine Part 2: The Code

Last time we ripped off a bunch of libraries to create a string interpolation script and explained the composite pattern with a view to ultimately being able to parse and populate this template:

<ul>
    ${#if responses.length}
        ${#each responses as response}
        <li>${response.name}</li>
        ${#end each}
    ${#end if}
    ${#if !responses.length}
    <li><em>No responses</em></li>
    ${#end if}
</ul>

If you don’t know how we’re handling string interpolation or you don’t know what the composite pattern is, check last week’s post for the explanation. The overall tree that our composite will build will look like this:

Tree─┬─TextBranch (1)
     ├─IfBranch (2)───EachBranch (3)───TextBranch (4)
     ├─IfBranch (5)───TextBranch (6)
     └─TextBranch (7)

… and the contents of these leaves and branches look like this:

  1. <ul>
  2. ${#if responses.length}
  3. ${#each responses as response}
  4. <li>${response.name}</li>
  5. ${#if !responses.length}
  6. <li><em>No responses</em></li>
  7. </ul>

There will be a few other text branches containing new lines, but these are the key ones.

There will also be a “Tree” to keep all the branches together, decipher the template string and create the composite and we’ll put a facade in-front of it to make sure that the branches can’t be manipulated once they’ve been created. So, without further adieu, let’s start looking at the code to make a template engine.

The “TextBranch”

The “TextBranch” is very simple, it’s mainly a wrapper for the util.String.supplant function from last week. We’re using the “Crockford Classless” style here, so as well as a “TextBranch”, we’ll need a utility function that can extend one object with the properties of another. There is a native function to do this, but (at time of writing) not every browser has it so we’ll create a fallback.

util.Object.assign = Object.assign || function (source, ...objects) {

    objects.forEach(function (object) {

        Object.keys(object).forEach(function (key) {
            source[key] = object[key];
        });

    });

    return source;

};

function makeTextBranch(text) {

    var textBranch = {

        type: "text",

        render: function (data) {
            return util.String.supplant(text, data);
        }

    };

    return Object.freeze(textBranch);

}

Each of the branches will have a type property so we can work out what it is, mainly for validation. The render method is the one that will actually get the data to display and we pass the contents of the branch as text to the constructor function.

The “BaseBranch”

The “IfBranch” and “EachBranch” will have some specific functionality but they will also share some. We’ll call the shared functionality a “BaseBranch”; the “BaseBranch” will allow us to add child branches, render them all and have a parent (we’ll need that to keep track of the nesting). I also want to create a utility function for executing a method on all array items, passing them the same data. While this doesn’t need to be its own function, I frequently find that creating utility functions like this not only makes them re-usable, but simplifies the code where they’re used. (We can also make it generic so it would work on a NodeList or another collection – we might need that later on.)

util.Array = {};

var arrayMap = Array.map || function (array, handler, context) {
    return Array.prototype.map.call(array, handler, context);
};

function arrayInvoke(array, method, ...args) {

    return arrayMap(array, function (entry) {
        return entry[method](...args);
    });

}

util.Object.assign(util.Array, {
    map: arrayMap,
    invoke: arrayInvoke
});
function makeBaseBranch() {

    var branches = [];
    var parent;
    var baseBranch = {

        type: "base",

        addBranch: function (branch) {
            branches.push(branch);
        },

        getBranches: function () {
            return branches.concat();
        },

        render: function (data) {
            return util.Array.invoke(branches, "render", data).join("");
        },

        setParent: function (branch) {
            parent = branch;
        },

        getParent: function () {
            return parent;
        }

    };

    return Object.freeze(baseBranch);

}

When we eventually make the “Tree” to tie all of these branches together, we’ll be able to close an “IfBranch” or “EachBranch” by simply getting it’s parent and using that to continue our work.

The “IfBranch”

An “IfBranch” will require us to work out what the template is checking for. Allowing the “IfBranch” to send its contents to something like eval() or new Function() would open a huge security hole so instead we parse the branch and check it against some pre-determined checks. So, before we can write the “IfBranch”, we’ll need those checks:

var tests = {

    "<": function (data, value) {
        return data < value;
    },

    ">": function (data, value) {
        return data > value;
    },

    "<=": function (data, value) {
        return data <= value;
    },

    ">=": function (data, value) {
        return data >= value;
    },

    "!==": function (data, value) {
        return data !== value;
    },

    "===": function (data, value) {
        return data === value;
    },

    // The regular expression won't match this, but we'll need it for a case
    // like: `${#if responses.length}`
    truthy: function (data) {
        return !!data;
    }

};

// We should create some aliases to make life easier on developers.
tests["!="] = tests["!=="];
tests["=="] = tests["==="];

Since we’re checking with the strict equality operator (=== instead of ==) the value will need to be decoded – we’ll need another couple of utility functions to make that work.

util.Number = {};
util.Function = {};

// Checks to see if a value is numeric
// (either a number or a numeric string).
util.Number.isNumeric = function (number) {
    return !isNaN(parseFloat(number)) && isFinite(number);
};

// Creates a function with pre-populated arguments but allows
// those arguments to leave spaces.
util.Function.curry = function (func, ...args) {

    return function (...innerArgs) {

        var allArgs = [];

        args.forEach(function (arg) {

            allArgs.push(
                arg === undefined
                    ? innerArgs.shift()
                    : arg
            );

        });

        return func(...allArgs.concat(innerArgs));

    };

};

function decode(value) {

    var parts;

    switch (value) {

    case "null":
        value = null;
        break;

    case "undefined":
        value = undefined;
        break;

    case "false":
    case "true":
        value = value === "true";
        break;

    default:

        parts = value.match(/^(["'`])?([\s\S]+)?\1$/);

        if (typeof parts[1] === "string") {
            value = parts[2];
        } else if (util.Number.isNumeric(value)) {
            value = +value;
        } else {

            value = util.Function.curry(
                util.Object.access,
                undefined,
                value
            );

        }

    }

    return value;

}

The decode function allows a developer to check against different things much like they would with a native if statement meaning that any of these would be valid conditions:

<!-- checking against a number -->
${#if variable === 10}
<!-- checking against a string -->
${#if variable === "10"}
<!-- also checking against a string -->
${#if variable === "ten"}
<!-- checking against a data property -->
${#if variable === ten}
<!-- checking against a nested data property -->
${#if variable === ten.info}
<!-- checking against truthiness -->
${#if variable}
<!-- checking against falsiness -->
${#if !variable}

We now have all the groundwork necessary for the “IfBranch” itself. We just parse the branch to work out what the developer is asking for. If the condition is truthy, we render the branch (which is done by simply passing the data to the `render` method of the parent “BaseBranch”) and if falsy, we just return an empty string, ignoring all the branch’s children.

function makeIfBranch(condition) {

    // Parsing.
    var regexp = /\${#if\s+(!)?([\S]+)(\s*([<>!=]{1,3})\s*([\S]+))?\}/;
    var parts = condition.match(regexp);

    // Checking the parts and falling back where necessary.
    var negation = parts[1];
    var propertyPath = parts[2];
    var test = parts[4] || "truthy";
    var value = parts[5] || "";

    // Creating the branch.
    var baseBranch = makeBaseBranch();
    var ifBranch = util.Object.assign({}, baseBranch, {

        type: "if",

        render: function (data) {

            var property = util.Object.access(data, propertyPath);
            var decoded = decode(value);
            var shouldRender = true;

            if (test && typeof tests[test] === "function") {

                shouldRender = tests[test](
                    negation === "!"
                        ? !property
                        : property,
                    typeof decoded === "function"
                        ? decoded(data)
                        : decoded
                );

            }

            return shouldRender
                ? baseBranch.render(data)
                : "";

        }

    });

    return Object.freeze(ifBranch);

}

The “EachBranch”

Compared to the “IfBranch”, the “EachBranch” is much simpler. It still takes some parsing with a regular expression to work out what’s required, but the actual functionality is mainly looping over the children and rendering each of them in turn.

Despite the simplicity, the “EachBranch” will change the data subtly. Consider data and each statements like this:

<!-- Data:
{
    responses: [
        {name: "Alpha"},
        {name: "Bravo"},
        {name: "Charlie"}
    ]
}
-->

${#each responses as response}
${#each responses as index to response}

In that example, response and index aren’t part of the data, but the child branches will need to be able to access them (and, of course, other information from the data as well). But, just for good measure, index isn’t always required (and shouldn’t be included if the developer didn’t ask for it – it may accidentally override something). We can extend the data without affecting the original by using util.Object.assign but we’re also going to need a function that can convert an object into an array of key/value pairs. For an array, we can just use map and for an array-like structure we can use util.Array.map … but that means we also need a way to identify an array-like structure.

util.Object.pair = function (object) {

    var pairs = [];

    Object.keys(object).forEach(function (key) {

        pairs.push({
            key: key,
            value: object[key]
        });

    });

    return pairs;

};

util.Array.isArrayLike = function (array) {

    var arrayMaxLength = Math.pow(2, 32) - 1;

    return (
        array !== null
        && array !== undefined
        && (
            Array.isArray(array)
            || (
                util.Number.isNumeric(array.length)
                && array.length >= 0
                && array.length < arrayMaxLength
            )
            || (
                window.Symbol
                    ? typeof array[Symbol.iterator] === "function"
                    : false
            )
        )
    );

};

Now we’ve got those, we can build the “EachBranch” itself. The only thing that I found tricky about this is that you have to loop over the data *before* looping over the branches, otherwise the branches get rendered in the wrong order. Feel free to swap those two lines to see what I mean.

function pair(object) {

    return util.Array.isArrayLike(data)
        ? util.Array.map(data, function (value, key) {

            return {
                key: key,
                value: value
            };

        })
        : util.Object.pair(data);

}

function makeEachBranch(condition) {

    // Parsing.
    var regexp = /\$\{#each\s+([\w]+)\s+as(\s([\w]+)\s+to)?\s+([\w]+)\}/;
    var parts = condition.match(regexp);

    // Aliases (for readability).
    var dataKey = parts[1];
    var iterationKey = parts[3];
    var iterationValue = parts[4];

    // Creating the branch.
    var eachBranch = util.Object.assign({}, makeBaseBranch(), {

        type: "each",

        render: function (data) {

            var rendered = "";
            var datum = pair(data[dataKey]);
            var branchData = util.Object.assign({}, data);

            datum.forEach(function (pair) {

                eachBranch.getBranches().forEach(function (branch) {

                    branchData[iterationValue] = pair.value;

                    if (iterationKey) {
                        branchData[iterationKey] = pair.key
                    }

                    rendered += branch.render(branchData);

                });

            });

            return rendered;

        }

    });

    return Object.freeze(eachBranch);

}

The branchData is only a shallow clone. If you want a better cloning function, util.Object.clone from encapsulation part 3 can be used.

The Tree

Now that we’ve got the logic of our branches, we need a tree to combine them all. The tree will handle the hierarchy and starts off with a few basic ideas: get a branch, add branches to that branch and render the branch. We also use the tree to validate the template – the render method will only work if the current branch is a “BaseBranch” (since a “TextBranch” would just be a child of a “BaseBranch”) because a dangling “IfBranch” or “EachBranch” means that the template wasn’t complete when we tried to render it.

function makeTree() {

    var currentBranch;
    var tree = {

        init: function () {
            tree.setBranch(makeBaseBranch());
        },

        setBranch: function (branch) {
            currentBranch = branch;
        },

        getBranch: function () {
            return currentBranch;
        },

        addBranch: function (branch) {
            currentBranch.addBranch(branch);
        },

        render: function (data) {

            if (currentBranch.type !== "base") {

                throw new SyntaxError(
                    "Unclosed " + currentBranch.type + " branch"
                );

            }

            return currentBranch.render(data);

        }

    }

    return Object.freeze(tree);

}

So far, so good. There are just 2 more methods that make the tree complete and they manage the hierarchy itself. This is done by opening a new branch (we set currentBranch to our new branch meaning addBranch affects the new branch) and closing a branch (currentBranch gets set to the new branch’s parent). We also continue the validation by ensuring that we only open a branch we recognise and only close the branch we’re expecting.

function makeTree() {

    var types = {
        each: makeEachBranch,
        text: makeTextBranch,
        "if": makeIfBranch
    };

    // ...

    var tree = {

        // ...

        openBranch: function (type, content) {

            var newBranch;

            if (!types[type]) {
                throw new TypeError("Unknown branch type " + type);
            }

            newBranch = types[type](content);
            newBranch.setParent(currentBranch);
            tree.addBranch(newBranch);
            tree.setBranch(newBranch);

        },

        closeBranch: function (type) {

            var branchType = currentBranch.type;

            if (branchType !== type) {

                throw new SyntaxError(
                    "Expecting type " + branchType + " but got " + type
                );

            }

            tree.setBranch(branch.getParent());

        }

    };

    // ...

}

And that’s the tree … yeah – it’s that simple. The tree will handle the nesting for us. All we need to do now is parse the template and pass it to the tree piece by piece. We also need a ways to protect the tree itself from the branches being manipulated against our wishes and the best way to do that is with:

The Facade

As its name suggests, the facade sits in front of the tree and exposes only the render method, keeping the branches protected. We also take the opportunity to set up the tree using the same object.

You may have noticed that we haven’t been allowing our branches to be escaped. We correct that in the facade, allowing the branches themselves to simply concentrate on their purpose (we just won’t create an “IfBranch” from an escaped “IfBranch” string). We can do this with the util.String.tokenise function that we wrote in the previous post.

var BRANCH_PART = /(^|\\|\r|\n)?\$\{#[^\}]+\}/;
var PROCESS_BRANCH = /(^|\\|\r|\n)?\$\{#(\w+)\s+([^\}]+)\}$/;

function setup(tree) {

    tree.init();
    util.String.tokenise(string, BRANCH_PART).forEach(function (part) {

        var match = part.match(PROCESS_BRANCH);

        if (match && match[1] !== "\\") {

            if (match[2] === "end") {
                tree.closeBranch(match[3]);
            } else {
                tree.openBranch(match[2], match[0]);
            }

        } else {
            tree.addBranch(makeTextBranch(part));
        }

    });

}

function makeTemplate(string) {

    var tree = makeTree();
    var template = {
        render: tree.render
    };

    setup(tree);

    return Object.freeze(template);

}

Now that we have everything we need, here’s now to use it:

// Note: Don't use the \ to escape a new line - trust me!
var string = "<ul>\
    ${#if responses.length}\
        ${#each responses as response}\
        <li>${response.name}</li>\
        ${#end each}\
    ${#end if}\
    ${#if !responses.length}\
    <li><em>No responses</em></li>\
    ${#end if}\
</ul>"
var template = makeTemplate(string);
template.render({
    responses: [
        "Alpha",
        "Bravo",
        "Charlie"
    ]
});
// ->
//  <ul>
//      <li>Alpha</li>
//      <li>Bravo</li>
//      <li>Charlie</li>
//  </ul>

template.render({
    responses: []
});
// ->
//  <ul>
//      <li><em>No responses</em></li>
//  </ul>

Oh yeah – templates can be re-used – did I forget to mention that?

You can see this template working in this fiddle:

Taking it Further

This is only a basic template engine, but there are a few ways that it can be improved, if that’s something you’d be interested in:

  • The “IfBranch” could be expanded to allow an “else” statement (I would do that by having 2 “BaseBranches” inside the “IfBranch” with the “else” statement switching the focus from the positive “BaseBranch” to the negative “BaseBranch” – the render method would render one of these branches).
  • The “Tree” can check how much work it has to do before parsing the string. A regular expression (or even indexOf) can be run to see if the string given to makeTemplate has process branches, just requires interpolation or even requires no processing and should just be returned. This technique is based on the process of “Space partitioning” in games (and you apply the same technique to util.String.supplant).
  • Custom branch types can be defined allowing for more functionality to be added. As long as those branches have a render method that accepts a data object and returns a string, the branch will work with the others.
  • String interpolation could use the native template strings.
  • The “IfBranch” can be expanded to allow “and” or “or” conditions and even nested brackets.

A lot of the popular engines allow some of these improvements – they’re popular for a reason. Under the hood, most of them essentially do what I’ve described over these two posts.

Final Words

I hope you have enjoyed this explanation of JavaScript template engines. All the code that is in both parts can be found in my GitHub repo.

Leave a Reply

Your email address will not be published. Required fields are marked *