Determined to tackle this last unknown, we began to experiment with different testing tools. We left no stone unturned, and tried out both familiar and unfamiliar libraries. By the end, we had chosen a collection of tools that gave us confidence in both our tests and our codebase overall.
Our Testing Goals
Early on, we sketched out a set of goals that we wanted our testing setup to accomplish:
- Our entire stack should be shared across the server and client
- Our entire stack should feel familiar to our current front-end developers
We also knew that other Boxers would be our main audience, and wanted their transition onto our new stack to be painless. With them in mind, we had a preference for tools and patterns that were already being used in our current codebase. Our goal was to keep the tools that were working well, and to update the tools that weren't.
Defining these two goals early on was crucial for focusing our search. Instead of arguing over who liked which tool more (or which library had a more clever name) we were able to cut through the noise and save valuable time and energy.
Getting Started: A Testing Framework
The first thing we needed was a testing framework. At the time, we had been using QUnit for all of our front-end tests. QUnit was created long ago for jQuery developers, and provides a bare-bones interface for writing front-end tests. While we were happy with it at the time, we were eager to see what else was out there.
After just a few searches, we were completely blown away. While QUnit had a bare-bones interface, every framework we looked at went above and beyond with extra features and powerful nesting patterns. Vows.js, for example, had an impressive mechanism for testing asynchronous callbacks. Frameworks like Intern came with advanced features like built-in support for popular task-runners.
Eventually, we settled on Mocha. We first looked at Mocha for its completely flat “QUnit interface.” While this felt familiar to us, it wasn’t a perfect copy. Besides, we had seen the promised lands of nested test suites and layered setups, and we wanted more. Luckily, Mocha provided some other, more contemporary styles that still felt familiar. The BDD-style interface had all the features that we wanted while remaining explicit and strait-forward. And because of Mocha's well-tested client-side support, we could use it on the front-end as well. Mission accomplished.
With Mocha in hand, we were ready to start writing tests. At first, we were happy with Node's core assert module. It did a good job of checking values quickly and easily, but when we began to write front-end tests we realized we could no longer use it. For the front-end, we decided to pick up the popular assertion library Chai, while keeping Node’s assertion module on the server. While both worked well on their own, small cracks began to form between the two. Chai, for instance, uses a `===` strict equals on deepEquals, while Node's assert module doesn't. Chai also supports a much larger collection of assertion helpers than the core assert module, so developers were forced to be mindful of which environment allowed each assertion helper.
Frustrated, we moved to unify. While a basic port of the Node.js module exists for the front-end, we felt that if we were going to share an assertion library, we should use the one that was made to do so. Chai is now our preferred assertion library, and we’re in the process of converting all of our tests to use it.
Stubs and Mocks
Unfortunately, assertions alone only get you so far. When testing more complex functions, you need a way to influence behavior and test code under explicit conditions. While it's important to always stay true to your code's original behavior, sometimes you need to be certain that some external dependency will return "true", or that an API call will yield with an expected value. For this, we use Sinon.
Sinon is a stubbing and mocking framework. Stubs are simple functions with explicit behavior, and they can be used to eliminate external dependencies from our tests. Mocks go one step further, and let you set expectations for your code. If the wrong arguments are passed to a function or if that function is accidentally never called, your mock can throw an error. Together, these two features give you more complete control over your module and testing environment.
We were already using Sinon for front-end testing at Box, and have been incredibly pleased with it so far. We quickly added it to our Node.js testing stack and have been happy ever since.
Our final challenge was Node's `require()` function. By loading external files privately within each file, we had no access to those modules or any of their behaviors. We couldn't stub them, and we definitely couldn't spy on them. To really control our tests, we needed the ability to bring these modules under our command.
There are a few different ways to solve this problem, depending on how much control you wish to have over your modules. For simply managing what is returned by require, you could use Mockery or Proxyrequire. Both tools wrap the require function in a special loader that will first check for and return any custom overrides that you define. This let you set stubs and mocks to return for some modules, while allowing others to be loaded normally. Other tools go one level deeper by modifying the actual module itself. Rewire, for instance, injects functions into your code for getting and setting private variables within your module scope. This lets you not only replace loaded modules, but also test and modify private functions within them.
After experimenting with each of these tools, we eventually settled on Mockery. Of the group, Mockery makes the least modifications to Node’s core behavior, and we liked that it inserted itself in front of require without monkey patching our original modules. We also liked that Mockery warns us when a module is loaded without permission. While it can be a pain to white-list each allowed module, it encourages defensive testing and gives us confidence that nothing is accidentally required.
Everyone has different goals when writing code, and the tools we use should always help us accomplish them. We set out to unify our front-end and back-end testing environments in a way that our developers would already be familiar with, and the testing stack we have today reflects that:
- Mocha is a testing framework with a strait-forward interface that feels familiar to our front-end and anyone with QUnit experience
- Chai is an assertion library that behaves the same on both Node.js and the front-end
- Sinon is a stubbing and mocking framework that eliminates external dependencies and can make assertions on certain behaviors
- Mockery proxies the require function and provides control over what is loaded in each test
Choosing the right libraries and frameworks takes time, but the investment is well worth it. Together these tools enable our developers to write tests with confidence, resulting in a more stable, high-quality product. If you'd like to do the same, think about your own testing goals. Then find the tools that help you accomplish them.
Interested in getting started with Node.js testing? Check out this blog post for a more in-depth look at some of the tools discussed above.