Going back to the tax calculation example, we need to reflect on pre-conditions that the method needs to function properly, as well as its post-conditions: what the method guarantees as outcomes. We already mentioned a pre-condition: the method does not accept negative numbers. A possible post-condition of this method is that it also does not return negative numbers. Once the… Continue reading Pre-conditions and post-conditions
Category: Uncategorized
Introduction
Imagine a piece of software that handles a very complex financial process. For that big routine to happen, the software system chains calls to several subroutines (or classes) in a complex flow of information: that is, the results of one class are passed to the next class, whose results are again passed to the next… Continue reading Introduction
Mutation testing
How much of the production code is exercised by a test. What they all miss is whether the assertions that these tests make are good and strong enough to capture bugs. If we introduce a bug in the code, even in a line covered by a test, will the test break? As mentioned earlier, coverage… Continue reading Mutation testing
What should not be covered?
We have talked a lot about what to test and cover. Let’s quickly discuss what not to cover. Achieving 100% coverage may be impossible or not even desirable. For example, the code snippet in listing 3.12 returns the full path of a specific directory. The code may throw a URISyntaxException, which we catch and wrap around a RuntimeException. (For… Continue reading What should not be covered?
Other coverage criteria
We have used the program’s control flow as a way to derive different tests. Another way of approaching structural testing is to look at the data flow: examining how the data flows to different parts of the program. For example, imagine that a variable is defined, then modified one, two, or three times in other parts of… Continue reading Other coverage criteria
MC/DC when expressions are too complex and cannot be simplified
MC/DC is increasingly valuable as expressions become more complicated. Listing 3.11 shows an example of a complex expression that I extracted from Chilenski’s 2001 paper. It is an anonymized version of a condition found in a level A flight simulation program and contains an impressive 76 conditions. Achieving path coverage in such a complex expression… Continue reading MC/DC when expressions are too complex and cannot be simplified
What coverage criterion to use
This is a popular question among practitioners and researchers. If we settle for a less-rigorous criterion, such as line coverage instead of branch coverage, we might miss something. Plus this question brings the focus back to the metric, which we do not want. Which criterion to use depends on the context: what you are testing… Continue reading What coverage criterion to use
What does it mean to achieve 100% coverage?
I have purposefully skipped talking much about achieving 100% line coverage or branch coverage or other coverage. I do not believe that achieving a number should be the goal. Nevertheless, given how prevalent those numbers are in practice, it is important to understand them. First, let’s talk about the metrics themselves. NOTE Formulas vary among the… Continue reading What does it mean to achieve 100% coverage?
Why do some people hate code coverage?
I find it interesting that some people rage against code coverage. A prevalent opinion is, “If I write a test case with no assertions, I achieve 100% coverage, but I am not testing anything!” This is true. If your tests have no assertions, they do not test anything, but the production code is exercised. However,… Continue reading Why do some people hate code coverage?
Structural testing in the real world
Now that you have a clear picture of structural testing, the coverage criteria you can use for guidance, and how to use structural testing in combination with specification-based testing, let me discuss a few interesting points.