An ECMAScript 6, CommonJS and RequireJS Project

I’ve been writing a lot of CommonJS code recently – the sort that you would include in Node projects on the server side. I’ve recently had a thought that I would like to do a browser-side project. However, how do you produce a browser library that can be consumed by everyone?

The different styles of modules

Let’s say I have a class Client(). If I were operating in Node or Browserify, I’d do something like this:

var Client = require('my-client-package');

var myclient = new Client();

This is called CommonJS format. I like it – it’s nice and clean. However, that’s not the only way to potentially consume the library. You can also bring it in with RequireJS:

define(['Client'], function(Client) {
    var myclient = new Client();

});

Finally, you could also register the variable as a global and bring it in with a script HTML tag:

<script src="node_modules/my-client-package/index.js"></script>
<script>
    var client = new Client();
</script>

You can find a really good writeup of the differences between CommonJS and AMD in an article by Addy Osmani.

Three different techniques. If we were being honest, they are all valid and have their place, although you might have your favorite technique. As a library developer, I want to support the widest range of JavaScript developers which means supporting three different styles of code. This brings me to UMD format. I named it “Ugly Module Definition”, and you can see why when you look at the code:

(function (root, factory) {
    if (typeof define === 'function' && define.amd) {
        // AMD. Register as an anonymous module.
        define(['b'], function (b) {
            return (root.returnExportsGlobal = factory(b));
        });
    } else if (typeof module === 'object' && module.exports) {
        // Node. Does not work with strict CommonJS, but
        // only CommonJS-like enviroments that support module.exports,
        // like Node.
        module.exports = factory(require('b'));
    } else {
        // Browser globals
        root.returnExportsGlobal = factory(root.b);
    }
}(this, function (b) {
    // Use b in some fashion

    return {// Your exported interface };
}));

Seriously, could this code be any uglier? I like writing my code in ECMAScript 2015, also known as ES6. So, can I write a class in ES6 and then transpile it to the right format? Further, can I set up a project that has everything I need to test the library? It turns out I can. Here is how I did it.

Project Setup

These days, I tend to create a directory for my project, put some stuff in it and then push it up to a newly created GitHub repository. I’m going to assume you have already created a GitHub user and then created a GitHub repository called ‘my-project’. Let’s get started:

mkdir my-project
cd my-project
git init
git remote add origin https://github.com/myuser/my-project
npm init --yes
git add package.json
git commit -m "First Commit"
git push -u origin master

Perhaps unshockingly, I have a PowerShell script for this functionality since I do it so often. All I have to do is remember to check in things along the way now and push the repository to GitHub at the end of my work.

My Code

I keep my code in the src directory, The tests are in the test directory. The distribution file is in the dist directory. Let’s start with looking at my src/Client.js code:

export default class Client {
    constructor(options = {}) {
    }
}

Pretty simple, right? The point of this is not to concentrate on code – it’s about the build process. I’ve also got a test in the test/Client.js file:

/* global describe, it */

// Testing Library Functions
import { expect } from 'chai';

// Objects under test
import Client from '../src/Client';

describe('Client.js', () => {
    describe('constructor', () => {
        it('should return a Client object', () => {
            let client = new Client();
            expect(client).to.be.instanceof(Client);
        });
    });
});

I like to use Mocha and Chai for my tests, so this is written with that combination in mind. Note the global comment on the first line – that prevents Visual Studio Code from putting green squiggles underneath the mocha globals.

Build Modules

I decided some time along the way that I won’t use gulp or grunt unless I have to. In this case, I don’t have to. My toolset includes:

Let’s take a look at my package.json:

{
    "name": "my-project",
    "version": "0.1.0",
    "description": "A client library written in ES6",
    "main": "dist/Client.js",
    "scripts": {
        "pretest": "eslint src test",
        "test": "mocha",
        "build": "babel src --out-file dist/Client.js --source-maps"
    },
    "keywords": [
    ],
    "author": "Adrian Hall <adrian@shellmonger.com>",
    "license": "MIT",
    "devDependencies": {
        "babel-cli": "^6.3.17",
        "babel-plugin-transform-es2015-modules-umd": "^6.3.13",
        "babel-preset-es2015": "^6.3.13",
        "babel-register": "^6.3.13",
        "chai": "^3.4.1",
        "eslint": "^1.10.3",
        "mocha": "^2.3.4"
    },
    "babel": {
        "presets": [
            "es2015"
        ],
        "plugins": [
            "transform-es2015-modules-umd"
        ]
    }
}

A couple of regions need to be discussed here. Firstly, I’ve got two basic npm commands I can run:

  • npm test will run the tests
  • npm run build will build the client library

I’ve got a bunch of devDependencies to implement this build system. Also note the “babel” section – this is what would normally go in the .babelrc – you can also place it in your package.json file.

The real secret sauce here is the build script. This uses a module transform to create a UMD format library from your ES6 code. You don’t even have to worry about reading that ES5 code – it’s ugly, but it works.

Editor Files

I use Visual Studio Code, so I need a jsconfig.json file in the root of my project:

{
    "compilerOptions": {
        "target": "ES6"
    }
}

This tells Visual Studio Code to use ES6 syntax. I’m hopeful the necessity of this will go away soon. I’m hoping that I’m not the only one who is contributing to this repository. Collaboration is great, but you want to set things up so that people coming newly in to the project can get started with your coding style straight away. I include a .editorconfig file as well:

root = true

[*]
charset = utf-8
indent_style = space
indent_size = 4
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true

[*.json]
insert_final_newline = false

You can read about editorconfig files on their site. This file is used by a wide variety of editors – if your editor is on the list, you should also install the plugin.

ESLint Configuration

I have a .eslintrc.js file at the root of the project. I’ve got that in a gist since it is so big and I just cut and paste it into the root directory.

Test Configuration

My test directory is different – it expects to operate within mocha, so I need an override to tell eslint that this is all about mocha. Here is the test/.eslintrc file:

module.exports = exports = {
    "env": {
        "es6": true,
        "mocha": true
    }
};

I also need a mocha.opts file to tell mocha that the tests are written in ES6 format:

--compilers js:babel-register

Wrapping up

You will need a dist directory. I place a README.md file in there that describes the three use cases for the library – CommonJS, AMD and globals. That README.md file is really only there to ensure the dist directory exists when you clone the repository.

I also need to add a README.md at the root of the project. It’s required if I intend to publish the project to the NPM repository. Basic instructions on how to install and use the library is de rigeur, but you can put whatever you want in there in reality.

I have not addressed jsdoc yet – you should be doing it in your source files, and it should be a postbuild step in your package.json file.

You can now run the tests and build through the npm commands and get a library that can be used across the board.

Web Dev Tools 101: Modules (Part 2)

Hopefully by now, you’ve settled on a module format. If you haven’t, then please read my prior article for some information on that topic. You cannot select a module loader unless you know what sort of modules you are writing. If you want the short version:

  • ECMAScript 6 – use the babel-core engine to compile to the AMD or CommonJS format
  • AMD – good for browser applications
  • CommonJS – good for server applications

What we will look at today is the various major module loaders and how to use them in the browser. If you are using NodeJS, then the CommonJS loader is built-in, so you don’t need to worry about it.

TL;DR

If you are using ECMAScript 6 then use the babel-core compiler to compile your modules into an equivalent AMD or CommonJS format, depending on requirements.

What to look for

There are only 4 things to look for in a module loader when you are starting out.

  1. Support for Module Format: Your chosen module loader needs to support the modules you are going to write plus the modules in libraries that you need.
  2. Asynchronous or not?: Module loaders are only really used in the browser (node has its own loader support for CommonJS). If you are a multi-page application, then you get major benefits from caching by using asynchronous loading. If you are a single-page application, it’s likely less important but caching is still good. I recommend an Asynchronous loader.
  3. Support for Gulp and Grunt?: If your module loader requires pre-processing of the modules, then it should support grunt or gulp – your chosen task runner.
  4. Support for loading plugins: Sometimes you want to lazy-load your CSS and HTML as well, mostly so you can componentize your CSS and templates. If your module loader doesn’t support it, you can’t have it.

The Candidates

I’ve listed the major candidates I found out on the Internet. Note that ECMAScript 6 support is not in there. Babel-core will compile your ES6 modules to one of the formats (AMD or CommonJS), then you use a module loader as normal. Just look for the requirements.

RequireJS

RequireJS was the original standlone module loader and is still probably the most used module loader out there today. It supports AMD natively. You can wrap your CommonJS modules in a simple wrapper. It’s got lazy (asynchronous) loading and it doesn’t need any pre-processing of the modules (beyond the ES6 to ES5 conversion that babel-core does). It also has a third-party plugin for CSS. This is a really good choice if you are just starting out and don’t know what else you need out of a module loader.

Downsides are few. However, there is a major one. If you are using certain frameworks (Angular is a culprit here) in a single-page application, then using RequireJS is more complex than it needs to be. In this case, you may want to re-visit the module loader topic based on the recommendations of that framework.

Browserify

Browserify is a bundling application, not a module loader. Why then do I include it here? Well, the workflow is “create a whole tree of stuff incorporating all your modules (written in CommonJS or AMD via a plugin) and produces a single Javascript file. You can then include that Javascript file in your HTML page and away you go.

Of course, this gets rid of all the benefits of lazy loading – most of which you don’t want anyway. The one you do want is when you have that large charting library. For that purpose, Browserify has a require functionality that is in CommonJS. The problem will be, of course, that now you have to mix require() (the old syntax) and import (the ES6 syntax) in your code, so you lose the benefit of writing code in ES6.

Browserify does get around the complexity described around AngularJS. If you are or are intending on using a framework like Angular, you may want to check out Browserify in that context.

Webpack

Webpack is another bundling application, much like Browserify. It has native support for AMD formatted modules (which isn’t to say it lazy loads them – just that it supports them). You can use loader plugins to pre-process files, so you can distribute ES6, Coffeescript or the like if you include their preprocessors as well. You can load external libraries, just like Browserify, although implementing lazy-loading is much easier. It has support for grunt and gulp, just like Browserify. Basically, it’s Browserify plus native support for AMD formatted modules and lazy loading.

The others

Yes, there are a whole load of other dynamic or asynchronous module loaders out there. None of them have a following like the ones above. Here is a list for investigation purposes.

Geez – come on guys. We don’t need more module loaders!

The Verdict

I mentioned in a prior article that I like ECMAScript 6, so I’m going to compile my code to another format anyway. That format is AMD as more module loaders support asynchronous loading using AMD than the others. It’s hardly a ringing endorsement, I know, but this area is likely to be in flux while the browsers start supporting ECMAScript 6 anyways.

My AMD loader of choice is RequireJS, but I’ll be spending more time with Webpack over the next few weeks to decide if that is worth the investment.

Use RequireJS to auto-load the Search Bubble

Over the last two posts (here and here) I’ve been creating my search bubbble with an intent to componentize my work. It’s a really simple example – just a search icon on a header bar and a pop-up bubble. The intent is to learn the techniques that I can apply in bigger projects. In this article on the subject I’m going to integrate RequireJS to auto-load my components and libraries.

I will note here that it isn’t worth the trouble in this size project. I’ve got a pretty good reuseable component for the search bubble from the last article. I’m loading jQuery and Bootstrap from the CDN. What could possibly need more infrastructure? Well, I’m glad you asked. One component doesn’t seem worth it. Let’s say that navigation bar was a component and it included icons that could be loaded to the left and right, and each of them relied on other components. Now, those components maybe relied on Javascript libraries that you downloaded from the Internet. And those libraries relied on other libraries as well. Now you have a whole nested tree of dependencies several layers thick.

What order do you load the various Javascript files in? How slow is your page load going to be now?

In reality, you want to load just the libraries for the components you are going to be using, you want those libraries to be defined as dependencies and you want them loaded only at the point in time you need them. Enter RequireJS – an Asynchronous Module Definition (AMD) API for Javascript Modules. If you follow some fairly basic rules, this makes it easy for you to write modules that are aware of their dependencies and handles all the messy loading stuff for you.

Let’s take a look at the new index.html file I created right at the beginning:

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8">
    <title>My Test Page</title>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.2/css/bootstrap.min.css">
    <link href="css/site.css" rel="stylesheet">
</head>
<body>
    <header>
        <ul>
            <li id="nav-search"><span class="glyphicon glyphicon-search"></span></li>
        </ul>
    </header>
    <script src="js/lib/require.js" data-main="js/init"></script>
</body>
</html>

Not much has changed. I’ve organized things a little – the CSS is being loaded from a specific CSS directory and the JavaScript is being loaded from a JS directory. Let’s take a look at that JS directory:

js-directory-structure

In my js directory I have two other directories. js/components is where I’ll put my code. The only thing in js/lib is require.js – you can download this from the RequireJS web site. Notice the data-main element in my code? This is “bootstrapping” – use require.js, but then load this other piece of code that tells RequireJS how to get your application started. There are two parts to the js/init.js file. The first is configuration for RequireJS:

require.config({
  baseUrl: 'js',
  paths: {
    'jquery':   'https://code.jquery.com/jquery-1.11.2.min',
    'bootstrap':'https://maxcdn.bootstrapcdn.com/bootstrap/3.3.2/js/bootstrap.min'
  },
  shim: {
    'bootstrap': { deps : [ 'jquery' ]}
  }
});

The baseUrl element tells RequireJS where to find our Javascript files. If I didn’t have the baseUrl then we would have to tell every single component the explicit path. Everything is relative to my js directory. Next, I have to tell RequireJS where my libraries are. In this case, I’m loading the libraries from the CDN. jQuery is aware of RequireJS but only if we refer to it as jquery. Similarly, I don’t want to be referring to Bootstrap with it’s full URL – let’s just call it something simple. My final shim section tells RequireJS that bootstrap is dependent on jquery. In general, you only need to do this for libraries downloaded from the Internet.

My original Javascript had some initialization code to create the search box object. That application initialization code goes next:

requirejs(['jquery', 'components/searchbox', 'bootstrap'], function($, searchbox) {
  $(document).ready(function() {
    var s = new searchbox('#nav-search');
  });
});

Let’s take a look at this a little bit. The function call has two components – an array of things this code depends on, then the function to call when all the requirements have been satisfied.
The elements of the array become variables to my function, allowing me to encapsulate the entire functionality into a module.

Remember the Namespace I created to hold the component class? That is no longer needed because of this encapsulation.

Once all those three libraries are loaded the function will be called. This function registers an event handler for when the document is ready and then instantiates the search box code.

I’vealready done most of the work in the js/components/searchbox.js, but let’s take a look at the basic form of a “class-style” AMD module:

define(['jquery', 'components/utils'], function($, utils) {
    return function Searchbox(id) {
        // My class goes here
        return {
            // My public interface goes here
        };
    };
});

I’m bringing in a non-class-style module called utils – I’ll put the createDIV() and generateGUID() functions there. For now, all I need to do is include the searchbox class from my prior post:

define(['jquery', 'components/utils'], function($, utils) {
  return function Searchbox(id) {
    var clickId     = id,
        searchBoxId = utils.generateGUID();
      
    var clickHandler = function(evt) {
      var elem = $(evt.currentTarget),
        srch = $('#' + evt.data.id);
      
      // Find out how many search boxes there are
      //   -> More than 1, then destroy them
      if (srch.length > 0) {
        srch.remove();
        return;
      }
      
      // Compute the size and position relative to what was clicked
      var left   = elem.position().left - 24,
        top    = elem.position().top + elem.height() + 16,
        width  = 300,
        height = 50;
        
      // If there are no search boxes, then fall through
      // to here and create one.
      var srch = utils.createDIV(searchBoxId, 'bubble', left, top, width, height);  
      srch.html("<div class='search'><input type='text' name='search' placeholder='Search'></div>");
    };
    
    $(clickId).css({ 'cursor': 'pointer'})
      .click({ id: searchBoxId }, clickHandler);
    
    // Public Interface
    return {
      getID: function() { 
        return clickId; 
      },
      getSearchBox: function() {
        var s = $('#'+searchBoxId);
        return (s.length > 0) ? s : null;
      }
    };
  };
});

So that’s the class-style module. What about a non-class-style module? One that’s just a collection of functions I want to use? In the searchbox.js file, I’m depending on components/utils and I use utils.createDIV() and utils.generateGUID(). The only thing I need to do is encapsulate these functions in an object. In the class-style module, I return a function – the constructor of the class. In the collection-of-functions, I return an object that has all the functions listed in it. In the searchbox.js file I’m bringing in components/utils, so place this code in js/components/utils.js:

define(['jquery'], function($) {
  return {
    createDIV: function(id, cssClass, left, top, width) {
      return $("<div id='" + id + "' class='" + cssClass + "'></div>")
        .appendTo('body')
        .css({
          'display': 'block',
          'position': 'absolute',
          'left': left + 'px',
          'top': top + 'px',
          'width': width + 'px'
        });
    },
    
    generateGUID: function() {
      var d = new Date().getTime(),
        guid = 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function (c) {
          var r = (d + Math.random() * 16) % 16 | 0;
          d = Math.floor(d / 16);
          return (c == 'x' ? r : (r & 0x3 | 0x8)).toString(16);
        });

      return guid;
    }
  };
});

Can I go further with this? Absolutely. I can, for example, move the header HTML code into a component and add that as a component. That header component can include other components and so on. Now I can modularize the code in such a way that it is just a blank body element and a require.js call. Everything is done on the client and in Javascript. Of course, that’s an extreme case, but you can see the possibilities there.