arXiv++ Combinatorics

Modern software systems often consist of many different components, each with a number of options. Although unit tests may reveal faulty options for individual components, functionally correct components may interact in unforeseen ways to cause a fault. Covering arrays are used to test for interactions among components systematically. A two-stage framework, providing a number of concrete algorithms, is developed for the efficient construction of covering arrays. %Our framework divides the construction in two stages. In the first stage, a time and memory efficient randomized algorithm covers most of the interactions. In the second stage, a more sophisticated search covers the remainder in relatively few tests. In this way, the storage limitations of the sophisticated search algorithms are avoided; hence the range of the number of components for which the algorithm can be applied is extended, without increasing the number of tests. Many of the framework instantiations can be tuned to optimize a memory-quality trade-off, so that fewer tests can be achieved using more memory. The algorithms developed outperform the currently best known methods when the number of components ranges from 20 to 60, the number of options for each ranges from 3 to 6, and $t$-way interactions are covered for $t\in \{5,6\}$. In some cases a reduction in the number of tests by more than $50\%$ is achieved.

A covering array $\mathsf{CA}(N;t,k,v)$ is an $N\times k$ array with entries in $\{1, 2, \ldots , v\}$, for which every $N\times t$ subarray contains each $t$-tuple of $\{1, 2, \ldots , v\}^t$ among its rows. Covering arrays find application in interaction testing, including software and hardware testing, advanced materials development, and biological systems. A central question is to determine or bound $\mathsf{CAN}(t,k,v)$, the minimum number $N$ of rows of a $\mathsf{CA}(N;t,k,v)$. The well known bound $\mathsf{CAN}(t,k,v)=O((t-1)v^t\log k)$ is not too far from being asymptotically optimal. Sensible relaxations of the covering requirement arise when (1) the set $\{1, 2, \ldots , v\}^t$ need only be contained among the rows of at least $(1-ε)\binom{k}{t}$ of the $N\times t$ subarrays and (2) the rows of every $N\times t$ subarray need only contain a (large) subset of $\{1, 2, \ldots , v\}^t$. In this paper, using probabilistic methods, significant improvements on the covering array upper bound are established for both relaxations, and for the conjunction of the two. In each case, a randomized algorithm constructs such arrays in expected polynomial time.

Covering arrays find important application in software and hardware interaction testing. For practical applications it is useful to determine or bound the minimum number of rows, CAN$(t,k,v)$, in a covering array for given values of the parameters $t,k$ and $v$. Asymptotic upper bounds for CAN$(t,k,v)$ have earlier been established using the Stein-Lovász-Johnson strategy and the Lovász local lemma. A series of improvements on these bounds is developed in this paper. First an estimate for the discrete Stein-Lovász-Johnson bound is derived. Then using alteration, the Stein-Lovász-Johnson bound is improved upon, leading to a two-stage construction algorithm. Bounds from the Lovász local lemma are improved upon in a different manner, by examining group actions on the set of symbols. Two asymptotic upper bounds on CAN$(t,k,v)$ are established that are tighter than the known bounds. A two-stage bound is derived that employs the Lovász local lemma and the conditional Lovász local lemma distribution.