Experience: Introducing JMockit To The Team
Today, almost every test we have has a mocked class, and an expectation set on some dependency. Many tests verify how many times a particular internal/private method or a method on a dependency was invoked, some have VOs mocked, or have assertions on return values inside verifications block. A few have gone to the lengths of mocking constructors of certain objects because there was no way to inject them into tested class. We are not even considering the private and static method mocks here.
When I look at the tests that we have today, and look back on the last few months, I wonder if I made mistake while introducing the mocking tool. In the trainings we had, we discussed the purpose of a mocking tool, issues due to overuse, indicators of overuse; heck there was even slide dedicated to this in the developer induction we have here. Architects were involved in many code reviews and tried avoiding these pitfalls, but clearly it was not enough.
Tests are supposed to improve the design of the code. Since the highly tangled, coupled classes, classes breaking SRP (Single Responsibility Principle) are difficult to test, we tend to fix them. As the size of the class increases, the functionality it has grows making it difficult to test, so we split it. As the dependencies being created makes it difficult to test, we change the class to allow injecting them. We end up splitting large methods, redesigning methods with side effects, removing unneeded code, decoupling from the libraries all for making the code testable and effectively get cleaner, maintainable, verifiable code. All of this, only if we wrote tests correctly.
If we started modelling our tests to match our code, we not only lose all the benefits, but the tests start becoming coupled to our code. That brings down the speed of refactoring or new development because every time we change code, we need to change the tests to match the implementation. That brings down our overall productivity. And finally blaming it on tests, we would stop writing them altogether. Back to square one! Mocking tools have a purpose, but if we mock everything, we get our tests tightly coupled to the implementation, adding to the problem. Mocking simple data-structures and VOs is not meaningful, we never test their methods separately, they are not supposed to be tested. (And yes, VOs are data structures, let us not get into that here.) Mocking external libraries is risky, because we then verify their actual behaviour only at runtime, which is exactly what we are trying to avoid by writing tests.
JMockit is a powerful tool, a little too powerful, and harmful if we are not careful. Despite the misuse that we have done, it could do all those things is itself marvelous. I am not convinced to blame it on JMockit, it is as Uncle Ben said to Peter Parker: ‘With great power comes great responsibility’. What we did, is our fault, we should have been more responsible with it. I wish I could go back and change the way we used it, or overused it, but our life is not a Git repository.
Luckily though, we have identified these issues and their severity before they start heavily weighing us down. What we need to do first is avoid more such tightly coupled tests from getting in. The course of action now seems like along with training the team on how to be more responsible with tests (which by the way needs to be a continuous process), we need to identify reviewers and train them on spotting such instances. Revising the review process to have reviews through identified reviewers and not just peer reviews.