Author: Haroon Khalil
-
Tests should be fast
Tests are a developer’s safety net. Whenever we perform maintenance or evolution in source code, we use the feedback from the test suite to understand whether the system is working as expected. The faster we get feedback from the test code, the better. Slower test suites force us to run the tests less often, making…
-
Principles of maintainable test code
What does good test code look like? There is a great deal of literature about test code quality, which I rely on in this section. Much of what I say here can be found in the works of Langr, Hunt, and Thomas (2015); Meszaros (2007); and Beck (2019)—as always, with my own twist.
-
Introduction
You have probably noticed that once test infected, the number of JUnit tests a software development team writes and maintains can become significant. In practice, test code bases grow quickly. Moreover, we have observed that Lehman’s law of evolution, “Code tends to rot, unless one actively works against it” (1980), also applies to test code. A…
-
Testing other types of web systems
The higher we go in levels of testing, such as web testing, the more we start to think about the frameworks and environment our application runs in. Our web application is responsive; how do we test for that? If we use Angular or React, how do we test it? Or, if we use a non-relational…
-
DSLs and tools for stakeholders to write tests
We wrote the system tests ourselves with lots of Java code. At this level, it is also common to see more automation. Some tools, such as the Robot framework and Cucumber even allow you to write tests in language that is almost completely natural. These tools make a lot of sense if you want others…
-
Proper code infrastructure is key
Integration and system tests both require a decent infrastructure behind the scenes. Without it, we may spend too much time setting up the environment or asserting that behavior was as expected. My key advice here is to invest in test infrastructure. Your infrastructure should help developers set up the environment, clean up the environment, retrieve…
-
Be careful with methods that are covered but not tested
Larger tests exercise more classes, methods, and behaviors together. In addition to all the trade-offs discussed with larger tests, the chances of covering a method but not testing it are much higher. Vera-Pérez and colleagues (2019) coined the term pseudo-tested methods. These methods are tested, but if we replace their entire implementation with a simple return null, tests still…
-
Perform cost/benefit analysis
One of the testing mantras is that a good test is cheap to write but can capture important bugs. Unit tests are cheap to write, so we do not have to think much about cost. Larger tests may not be cheap to write, run, or maintain. I have seen integration test suites that take hours…
-
How do all the testing techniques fit?
Our goal was to explore techniques that would help you engineer test cases systematically. We discuss a more orthogonal topic: how large should our tests be? I have shown you examples of larger component tests, integration tests, and system tests. But regardless of the test level, engineering good test cases should still be the focus.…
-
Final notes on larger tests
I close with some points I have not yet mentioned regarding larger tests.