Category: Uncategorized

If code is the source of all truth, why can’t we just do structural testing? This is a very interesting question. Test suites derived only with structural testing can be reasonably effective, but they may not be strong enough. Let’s look at an example (see the “counting clumps” problem, inspired by a CodingBat assignment. The… Continue reading Structural testing alone often is not enough

The most challenging part of specification-based testing is identifying boundaries. They are tricky to find, given the way we write specifications. Luckily, they are much easier to find in source code, given how precise code has to be. The idea of identifying and testing on and off points fits nicely in structural testing. For example,… Continue reading Boundary testing and structural testing

Let’s try specification-based testing and structural testing together on a real-world example: the leftPad() function from Apache Commons Lang. Left-pad a string with a specified string. Pad to a size of size. The method returns a left-padded string, the original string if no padding is necessary, or null if a null string is input. For example, if we give “abc” as the string… Continue reading Specification-based and structural testing: A running example

You may have noticed that some of the criteria we have discussed are more rigorous than others. For example, a single test is enough to achieve 100% line coverage, but two tests are needed for 100% branch coverage. Some strategies subsume other strategies. Formally, a strategy X subsumes strategy Y if all elements that Y exercises are… Continue reading Criteria subsumption, and choosing a criterion

You may wonder what to do in the case of loops, such as for and while. The code block inside the loop may be executed different numbers of times, making testing more complicated. Think of a while(true) loop, which can be non-terminating. To be rigorous, we would have to test the program with the loop block executed one time, two times,… Continue reading Handling loops and similar constructs

The question is how to (mechanically) select such test cases. Let’s continue using the same if statement from the CountWords program (from listing 3.4). The statement takes three booleans as input: (1) whether the current character is a letter and whether this letter is (2) “s” or (3) “r”. Generically, this is the same as the A && (B || C) example we just discussed.… Continue reading Creating a test suite that achieves MC/DC

Let’s take a simple abstract example: if(A && (B || C)), where A, B, and C all evaluate to booleans. MC/DC dictates the following: If conditions have only binary outcomes (that is, true or false), the number of tests required to achieve 100% MC/DC coverage is N + 1, where N is the number of conditions in the decision (as shown by Chilenski [2001]). Note that N… Continue reading An abstract example

Devising test suites that maximize the number of bugs they can identify while minimizing the effort/cost of building the test suite is part of any tester’s job. The question is, what can we do about complex, lengthy if statements? Modified condition/decision coverage (MC/DC) is a good answer. The MC/DC criterion looks at combinations of conditions, as path… Continue reading Complex conditions and the MC/DC coverage criterion

A developer aiming for path coverage covers all the possible paths of execution of the program. While ideally this is the strongest criterion, it is often impossible or too expensive to achieve. In a single program with three conditions, where each condition could be independently evaluated to true or false, we would have 23 = 8 paths to cover. In… Continue reading Path coverage

Condition + branch coverage considers not only possible branches but also each condition of each branch statement. For example, the first if statement in the CountWords program contains three conditions: !Character.isLetter(str.charAt(i)), last == ‘s’, and last == ‘r’. Therefore, a developer aiming for condition + branch coverage should create a test suite that exercises each of those individual conditions being evaluated to true and false at least once and the entire branch… Continue reading Condition + branch coverage