Web Dev Tools 101: Testing

You were going to test your application, weren’t you? Until recently, I was the kind of person who ran my application in Google Chrome and Internet Explorer on each little piece of functionality I developed. If that functionality ran, then I called the whole application good. This has all sorts of problems for the serious developer. If you are a serious developer then you need to be thinking about testing.


There are several pieces to testing. The below is what I’m going to use:

Why do I care?

In the bad old days, Javascript programmers got a really bad rap as bad programmers. The code wasn’t efficient and modular, there was lots of bad practices, and practically zero automated testing. Then the environment grew up a bit and now we have all this infrastructure available to us. This includes testing environments.

Let’s be honest, you don’t want to test your code. You NEED to test your code. There are two ways of doing that. The “ad-hoc” method where you run your application, click around, and see if it fails, and the “rigorous” method where you write unit and UI tests to test your code, you write the unit tests before you write your code and you run your unit tests before checking in your code.

Fortunately, there are a lot of choices when it comes to testing libraries these days. You need a test runner (the thing you integrate into your build process for testing), a test framework (a format that you write all your tests in), sometimes an assertion library, and something to emulate a browser so you don’t have to.

Test Runners

There are only two test runners and only one real contender in this category – so good news there. The two contenders are Karma (you should use this) and Chutzpah (erm – no). Here is what I was looking for when looking for a test runner.

  1. Support on grunt and gulp: You want to integrate testing into your process.
  2. Support for device testing: You want to test on real devices and real browsers.
  3. Support for headless testing: You don’t want to test on real devices all the time.
  4. Test Coverage Reporting: You want to know how much is tested and get better over time.
  5. Visual Studio Support: I like Visual Studio – this is important to me!


Produced by a collaboration between Google and Angular, Karma (previously Testacular – not a good name) is pretty much the de-facto standard in test runners. It has plugins for grunt and gulp, so it can be easily integrated into your workflow. It can work with Jasmine, Mocha and QUnit – the three major test frameworks (more on these below). It has support for PhantomJS and slimerJS (the headless browsers – more on these below) and can support multiple real browsers and devices. It can also integrate with Istanbul – a code coverage tool. There is even a Visual Studio Test Adapter.

Indeed, there is really very little down side to Karma.


In the interest of being balanced, here is another option. Chutzpah seems to have been written with Visual Studio in mind. It comes with a Visual Studio Test Adapter from the author (rather than a third party), integrates with Jasmine, Mocha and QUnit and has support for PhantomJS.

And there it stops. No gulp/grunt integration. No SlimerJS support. No remote device support. Those a really really important things to me. I just don’t think I can support a test runner without those.

Test Frameworks

Test Frameworks are what take your test suite and actually execute tests. There are three really popular ones – Jasmine, Mocha and QUnit – and a host of others. All of them come with an assertion library, but you can use your own as well if you want (something like should.js, expectJS, or chai). These assertion libraries just improve the test framework – they are not required.


How about this for a Jasmine test description:

describe("The 'toBe' matcher compares with ===", function() {
  it("and has a positive case", function() {

  it("and can have a negative case", function() {

I can actually read this! It’s Javascript and it’s semantic. Those of you who have been following me for a while know I love elegant Javascript. When you can read it, you’ve won. As one would expect, there are a whole host of assertions out of the box but you can integrate with an assertion library as well (I like Chai, for reference).

So what’s the downfall of this library? Jasmine is targeted as Behavior Driven Development. I can see it being used for Test Driven Development as well, though. QUnit is designed for unit testing, so it’s more in the TDD area. Yes, I’m splitting hairs.


Mocha is sort of a very flexible version of Jasmine and QUnit. There is less done for you but it’s more flexible as a result of all the plug-ins available. There is no assertion library (use Chai), or Spy Framework (use sinon.js). It can be configured for BDD, TDD or both though. Basically, if Jasmine or QUnit can’t do it for you and you would find yourself going towards the other tool, it’s time to move to Mocha.

The tests for Mocha are very similar to Jasmine:

var assert = require("assert");
describe('Array', function(){
  describe('#indexOf()', function(){
    it('should return -1 when the value is not present', function(){
      assert.equal(-1, [1,2,3].indexOf(5));
      assert.equal(-1, [1,2,3].indexOf(0));

Note how we have to bring in the assert library first. One isn’t built in, so that’s more scaffolding. Otherwise, it should look very similar to the Jasmine test case.


QUnit is done by the same fine people who brought you jQuery. It’s got plenty of tutorial and documentation materials and fine examples. it’s got support for gulp and grunt and has a whole host of plugins to handle BDD (which isn’t handled out of the box) and PhantomJS. A typical QUnit test looks like this:

QUnit.test("prettydate basics", function( assert ) {
    var now = "2008/01/28 22:25:00";
    assert.equal(prettyDate(now, "2008/01/28 22:24:30"), "just now");
    assert.equal(prettyDate(now, "2008/01/28 22:23:30"), "1 minute ago");
    assert.equal(prettyDate(now, "2008/01/28 21:23:30"), "1 hour ago");
    assert.equal(prettyDate(now, "2008/01/27 22:23:30"), "Yesterday");
    assert.equal(prettyDate(now, "2008/01/26 22:23:30"), "2 days ago");
    assert.equal(prettyDate(now, "2007/01/26 22:23:30"), undefined);

My major problem with this is that it just isn’t as readable as the Jasmine version. I like the level of documentation and help you can get with QUnit though.

Headless Browsers

At some point you are going to want to test on a real browser. There are basically three flavors of browser – Gecko (most notably Firefox), WebKit (most notably Chrome) and Internet Explorer. I can’t help you with Internet Explorer – you’ll have to use a real device for that. However, I can help you with Gecko and WebKit.


PhantomJS includes the WebKit javascript engine. As a result of this, it can emulate Chrome and Safari precisely. This makes it an ideal test companion as you can run the test suite against a “real browser” without having the browser open.


SlimerJS is the same thing, but for the Gecko javascript engine. Unlike PhantomJS, it isn’t truly headless – you still need a graphical environment and you will see windows. This is really only a problem on Windows as you can use xvfb (a virtual frame buffer) on Mac and Linux to simulate a screen that you don’t see.

The Verdict

After doing all this research, the choice for me was rather obvious. I’m going to start out with the Karma test runner, run my tests with Jasmine (using the Chai assertion library when the default library isn’t enough) and when I come to do my UI tests I’ll be able to use real devices or the PhantomJS headless browser.