Category: Structural Testing And Code Coverage

How much of the production code is exercised by a test. What they all miss is whether the assertions that these tests make are good and strong enough to capture bugs. If we introduce a bug in the code, even in a line covered by a test, will the test break? As mentioned earlier, coverage… Continue reading Mutation testing

We have talked a lot about what to test and cover. Let’s quickly discuss what not to cover. Achieving 100% coverage may be impossible or not even desirable. For example, the code snippet in listing 3.12 returns the full path of a specific directory. The code may throw a URISyntaxException, which we catch and wrap around a RuntimeException. (For… Continue reading What should not be covered?

We have used the program’s control flow as a way to derive different tests. Another way of approaching structural testing is to look at the data flow: examining how the data flows to different parts of the program. For example, imagine that a variable is defined, then modified one, two, or three times in other parts of… Continue reading Other coverage criteria

MC/DC is increasingly valuable as expressions become more complicated. Listing 3.11 shows an example of a complex expression that I extracted from Chilenski’s 2001 paper. It is an anonymized version of a condition found in a level A flight simulation program and contains an impressive 76 conditions. Achieving path coverage in such a complex expression… Continue reading MC/DC when expressions are too complex and cannot be simplified

This is a popular question among practitioners and researchers. If we settle for a less-rigorous criterion, such as line coverage instead of branch coverage, we might miss something. Plus this question brings the focus back to the metric, which we do not want. Which criterion to use depends on the context: what you are testing… Continue reading What coverage criterion to use

I have purposefully skipped talking much about achieving 100% line coverage or branch coverage or other coverage. I do not believe that achieving a number should be the goal. Nevertheless, given how prevalent those numbers are in practice, it is important to understand them. First, let’s talk about the metrics themselves. NOTE Formulas vary among the… Continue reading What does it mean to achieve 100% coverage?

I find it interesting that some people rage against code coverage. A prevalent opinion is, “If I write a test case with no assertions, I achieve 100% coverage, but I am not testing anything!” This is true. If your tests have no assertions, they do not test anything, but the production code is exercised. However,… Continue reading Why do some people hate code coverage?

Now that you have a clear picture of structural testing, the coverage criteria you can use for guidance, and how to use structural testing in combination with specification-based testing, let me discuss a few interesting points.

If code is the source of all truth, why can’t we just do structural testing? This is a very interesting question. Test suites derived only with structural testing can be reasonably effective, but they may not be strong enough. Let’s look at an example (see the “counting clumps” problem, inspired by a CodingBat assignment. The… Continue reading Structural testing alone often is not enough

The most challenging part of specification-based testing is identifying boundaries. They are tricky to find, given the way we write specifications. Luckily, they are much easier to find in source code, given how precise code has to be. The idea of identifying and testing on and off points fits nicely in structural testing. For example,… Continue reading Boundary testing and structural testing