An ECMAScript 6, CommonJS and RequireJS Project

I’ve been writing a lot of CommonJS code recently – the sort that you would include in Node projects on the server side. I’ve recently had a thought that I would like to do a browser-side project. However, how do you produce a browser library that can be consumed by everyone?

The different styles of modules

Let’s say I have a class Client(). If I were operating in Node or Browserify, I’d do something like this:

var Client = require('my-client-package');

var myclient = new Client();

This is called CommonJS format. I like it – it’s nice and clean. However, that’s not the only way to potentially consume the library. You can also bring it in with RequireJS:

define(['Client'], function(Client) {
    var myclient = new Client();

});

Finally, you could also register the variable as a global and bring it in with a script HTML tag:

<script src="node_modules/my-client-package/index.js"></script>
<script>
    var client = new Client();
</script>

You can find a really good writeup of the differences between CommonJS and AMD in an article by Addy Osmani.

Three different techniques. If we were being honest, they are all valid and have their place, although you might have your favorite technique. As a library developer, I want to support the widest range of JavaScript developers which means supporting three different styles of code. This brings me to UMD format. I named it “Ugly Module Definition”, and you can see why when you look at the code:

(function (root, factory) {
    if (typeof define === 'function' && define.amd) {
        // AMD. Register as an anonymous module.
        define(['b'], function (b) {
            return (root.returnExportsGlobal = factory(b));
        });
    } else if (typeof module === 'object' && module.exports) {
        // Node. Does not work with strict CommonJS, but
        // only CommonJS-like enviroments that support module.exports,
        // like Node.
        module.exports = factory(require('b'));
    } else {
        // Browser globals
        root.returnExportsGlobal = factory(root.b);
    }
}(this, function (b) {
    // Use b in some fashion

    return {// Your exported interface };
}));

Seriously, could this code be any uglier? I like writing my code in ECMAScript 2015, also known as ES6. So, can I write a class in ES6 and then transpile it to the right format? Further, can I set up a project that has everything I need to test the library? It turns out I can. Here is how I did it.

Project Setup

These days, I tend to create a directory for my project, put some stuff in it and then push it up to a newly created GitHub repository. I’m going to assume you have already created a GitHub user and then created a GitHub repository called ‘my-project’. Let’s get started:

mkdir my-project
cd my-project
git init
git remote add origin https://github.com/myuser/my-project
npm init --yes
git add package.json
git commit -m "First Commit"
git push -u origin master

Perhaps unshockingly, I have a PowerShell script for this functionality since I do it so often. All I have to do is remember to check in things along the way now and push the repository to GitHub at the end of my work.

My Code

I keep my code in the src directory, The tests are in the test directory. The distribution file is in the dist directory. Let’s start with looking at my src/Client.js code:

export default class Client {
    constructor(options = {}) {
    }
}

Pretty simple, right? The point of this is not to concentrate on code – it’s about the build process. I’ve also got a test in the test/Client.js file:

/* global describe, it */

// Testing Library Functions
import { expect } from 'chai';

// Objects under test
import Client from '../src/Client';

describe('Client.js', () => {
    describe('constructor', () => {
        it('should return a Client object', () => {
            let client = new Client();
            expect(client).to.be.instanceof(Client);
        });
    });
});

I like to use Mocha and Chai for my tests, so this is written with that combination in mind. Note the global comment on the first line – that prevents Visual Studio Code from putting green squiggles underneath the mocha globals.

Build Modules

I decided some time along the way that I won’t use gulp or grunt unless I have to. In this case, I don’t have to. My toolset includes:

Let’s take a look at my package.json:

{
    "name": "my-project",
    "version": "0.1.0",
    "description": "A client library written in ES6",
    "main": "dist/Client.js",
    "scripts": {
        "pretest": "eslint src test",
        "test": "mocha",
        "build": "babel src --out-file dist/Client.js --source-maps"
    },
    "keywords": [
    ],
    "author": "Adrian Hall <adrian@shellmonger.com>",
    "license": "MIT",
    "devDependencies": {
        "babel-cli": "^6.3.17",
        "babel-plugin-transform-es2015-modules-umd": "^6.3.13",
        "babel-preset-es2015": "^6.3.13",
        "babel-register": "^6.3.13",
        "chai": "^3.4.1",
        "eslint": "^1.10.3",
        "mocha": "^2.3.4"
    },
    "babel": {
        "presets": [
            "es2015"
        ],
        "plugins": [
            "transform-es2015-modules-umd"
        ]
    }
}

A couple of regions need to be discussed here. Firstly, I’ve got two basic npm commands I can run:

  • npm test will run the tests
  • npm run build will build the client library

I’ve got a bunch of devDependencies to implement this build system. Also note the “babel” section – this is what would normally go in the .babelrc – you can also place it in your package.json file.

The real secret sauce here is the build script. This uses a module transform to create a UMD format library from your ES6 code. You don’t even have to worry about reading that ES5 code – it’s ugly, but it works.

Editor Files

I use Visual Studio Code, so I need a jsconfig.json file in the root of my project:

{
    "compilerOptions": {
        "target": "ES6"
    }
}

This tells Visual Studio Code to use ES6 syntax. I’m hopeful the necessity of this will go away soon. I’m hoping that I’m not the only one who is contributing to this repository. Collaboration is great, but you want to set things up so that people coming newly in to the project can get started with your coding style straight away. I include a .editorconfig file as well:

root = true

[*]
charset = utf-8
indent_style = space
indent_size = 4
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true

[*.json]
insert_final_newline = false

You can read about editorconfig files on their site. This file is used by a wide variety of editors – if your editor is on the list, you should also install the plugin.

ESLint Configuration

I have a .eslintrc.js file at the root of the project. I’ve got that in a gist since it is so big and I just cut and paste it into the root directory.

Test Configuration

My test directory is different – it expects to operate within mocha, so I need an override to tell eslint that this is all about mocha. Here is the test/.eslintrc file:

module.exports = exports = {
    "env": {
        "es6": true,
        "mocha": true
    }
};

I also need a mocha.opts file to tell mocha that the tests are written in ES6 format:

--compilers js:babel-register

Wrapping up

You will need a dist directory. I place a README.md file in there that describes the three use cases for the library – CommonJS, AMD and globals. That README.md file is really only there to ensure the dist directory exists when you clone the repository.

I also need to add a README.md at the root of the project. It’s required if I intend to publish the project to the NPM repository. Basic instructions on how to install and use the library is de rigeur, but you can put whatever you want in there in reality.

I have not addressed jsdoc yet – you should be doing it in your source files, and it should be a postbuild step in your package.json file.

You can now run the tests and build through the npm commands and get a library that can be used across the board.

Web Dev Tools 101: Modules (Part 1)

If you have anything but a simple application you are going to want to modularize your code. At that point, dependency hell ensues as different modules depend on different other modules that depend on different libraries that depend on different other libraries and so on. If you don’t use a module loader then you have to load those libraries and modules in the right order, which can be hell. If you want a more in-depth look at module formats, check out the article by Addy Osmani – it’s pretty definitive, if a little more technical.

TL;DR

Module Loaders are a necessity. Write your modules in ECMAScript 6 format and use the babel-core transpiler to compile them into something you can use.

Why do I care?

I started by thinking “well, in my html file, I just include jQuery and Bootstrap and move on, right?” I did that for a while. Then I found a new JavaScript library for Sparklines. It depended on jQuery, so load it after jQuery. Then I had my component that was written as a jQuery plug-in (so also after jQuery), that depended on the jquery.sparkline library (so after that as well). As I wrote and added more components to my application, it quickly got out of hand. I was spending many minutes in each cycle ensuring that the files were added in the right order. Then I got a conflict – two libraries that depended on two different versions of another library. It was time for modules.

Modules are a code organization methodology. You should be writing all your code as modules. It allows you to expose only a public interface and then hide the implementation away. It’s considered bad practice to put objects in the global namespace these days. Modules allow you to achieve that.

Module Loaders take the hard work of dependency management from you. When writing a module, a module loader allows you to define what your module depends on and then the module pattern allows you to expose just the bits of your code that you want (the public interface). When another module wants your code, it requires your code, which is then loaded in a new namespace within the other module.

Unfortunately, each module loader requires a different pattern for writing modules. Before deciding on a module loader you have to decide on a module format that you want to write.

My Options

There are four options right now:

  1. The AMD module – most commonly used in client-side code.
  2. The CommonJS module – most commonly used in server-side code.
  3. The ECMAScript 6 module – the new one.
  4. The UMD module – an attempt at a middle ground.

Let’s take a look at a simple module and how to call it in each case. Our module will print 10 numbers onto the console. We’re going to be obnoxious and use jQuery for this functionality. The jQuery code is this:

$(document).ready(function() {
    function logger(arr) {
        $.each(arr, function(idx, val) {
            console.log(idx + ": " + val);
        });
    }

    logger([1,2,3,4,5,6,7,8,9,10]);
}

The idea is that we put the function logger into a module so that we can require it in our main program.

The AMD Format

AMD stands for Asynchronous Module Definition. It’s major design goal is lazy loading. If you have a Single Page Application, then you might have dozens of pages. Let’s say you have a charting library – they tend to be big. You only want to load that when it’s needed. If you load it all at the beginning then it can really affect the startup time of your application. AMD is great for this case. I’ve already shown off this type of module when I was developing the Bubble Search Box. The module pattern is fairly basic:

define([
    // List of requires
]), function(/* List of variables */) {
    // Your code
    return {
        // Your public interface
    }
});

At the top level, I use a require() function to require in the bits I need. My main page would look like this:

require([
    'jquery',
    './mymodule'
], function($, mod) {
    $(document).ready() {
        mod.logger([1,2,3,4,5,6,7,8,9,10]);
    }
});

My module looks like this:

define([
    'jquery'
], function($) {
    function internalLogger(arr) {
        $.each(arr, function(idx, val) {
            console.log(idx + ": " + val);
        });
    }

    return {
        logger: internalLogger
    }
});

Here I am defining some internal code in internalLogger() – it’s an example of a private interface. Then I’m returning an object with my public interface – in this case, a function logger() that points to my internal representation. This is a nice format. Some other nice things about this format is that I can include other things, like CSS, using plug-ins. This means I can package a visual component up with it’s CSS as well – the CSS only gets loaded if the module gets loaded, allowing for modularization of your CSS code as well.

The downside is that each load is a roundtrip. Got 100 components? That’s 100 round-trips to the server. That can severely affect the performance of your application. One of the major performance boosts you can get is to concatenate all your files together, minify them with a tool like uglify, then load that single file. The resulting file may be 100K, but that’s better than 100 x 1K transfers.

The CommonJS Format

One of the problems with AMD is that you have to define all your requirements up front, and co-ordinate the variables within the function parameters with the order of the requirements. That’s easy when there are one or two, but how about 20? It’s much more problematic. The CommonJS module format gets around that. Here is my CommonJS formatted module:

var $ = require('jquery');

function internalLogger(arr) {
    $.each(arr, function(idx, val) {
        console.log(idx + ": " + val);
    });
}

exports.logger = internalLogger;

My CommonJS formatted main file is similar:

var $ = require('jquery'),
    mod = require('./mymodule');
    
$(document).ready() {
    mod.logger([1,2,3,4,5,6,7,8,9,10]);
}

You might think “wait a minute – all that code is in the global namespace”. Yes it is – right now. When the entire app is constructed, each module is wrapped in an anonymous function. The CommonJS format improves the two bad things about AMD. The problem of correlating the requires with the parameters is fixed by requiring each module into the variable separately. The problem of the multiple round-trips is fixed by making a composite file of all your modules and including that composite file in your html page. The down side of this is that you have to include everything at the beginning – even that large charting library that you may or may not use.

I personally like the readability of the CommonJS format a little better than the AMD format. With the AMD format I have to keep on going back to the top of the file to figure out what is required and what isn’t. With the CommonJS format, I can just do a definition search for the variable – it’s explicit.

The ECMAScript 6 Format

Both the AMD format and the CommonJS format will continue to work given the same loaders in ECMAScript 6, but ECMAScript 6 has a built-in module loader. In the interim period – before browsers support the built-in module loader – the babel-core engine compiles the ES6 module code to a format corresponding to whatever you specify – CommonJS or AMD. This allows you to write and include ECMAScript 6 modules now, then pick lazy loading or up-front loading at build time. Let’s look at my ECMAScript 6 module:

import * as $ from 'jquery';

function internalLogger(arr) {
    $.each(arr, function(idx, val) {
        console.log(idx + ": " + val);
    });
}

module.exports = {
    logger: internalLogger
};

In the CommonJS and AMD formats, the require() and define() functions were just that – functions. Here we have explicit keywords to manage the module definition. Our public interface is similarly exported here. I like the explicit nature of the definition. This is also as readable as the CommonJS format and the import syntax should make it easier for IDEs to handle auto-complete or intellisense. My main program is similarly easy to read:

import * as $ from 'jquery';
import { logger } from './mymodule';

$(document).ready() {
    logger([1,2,3,4,5,6,7,8,9,10]);
}

For the main program I’ve added an alternate syntax for importing – importing a single function from a module. Again – it’s very explicit. I love the new syntax.

The UMD Format

The final format I’m going to look at is the UMD format. This is a format that is designed to be a hybrid between AMD and CommonJS, for those that can’t make up their mind. In the context of modules, UMD stands for Universal Module Definition. Our simple module looks like this in UMD format:

(function (root, factory) {
    if (typeof define === 'function' &amp;&amp; define.amd) {
        // AMD. Register as an anonymous module.
        define(['jquery'], factory);
    } else if (typeof exports === 'object') {
        // Node. Does not work with strict CommonJS, but
        // only CommonJS-like environments that support module.exports,
        // like Node.
        module.exports = factory(require('jquery'));
    } else {
        // Browser globals (root is window)
        root.returnExports = factory(root.jQuery);
    }
}(this, function ($) {
    function internalLogger(arr) {
        $.each(arr, function(idx, val) {
            console.log(idx + ": " + val);
        });
    }

    return {
        logger: internalLogger
    };
}));

This module can be used in AMD environments and CommonJS environments because of the scaffolding at the top of the module. However it is making assumptions on what you are using as your module loader. This is probably a really good pattern if you are writing a library for distribution and you want to support AMD and CommonJS, but the scaffolding seems overkill for your own application code.

The Verdict

If you followed the advice of my last article then the format is obvious – go for the ECMAScript 6 format. However this does come with some baggage. The good news is that you can use Babel-Core to compile the modules to either AMD or CommonJS.

If you aren’t using ECMAScript 6, then the choice is more driven by “in the browser” vs. “on the server”. If you are writing an application that is designed to be run in the browser then use the AMD format. If your application is designed to be run on the server (in a Node or IO container, for example), then use CommonJS format.

In my next article, I’ll discuss the Module Loaders for each format in more detail.