Accelerating Black Box Testing with Light-Weight Learning

Black box testing can employ randomness for generating test sequences. Often, even a large number of test sequences may sample a minuscule portion of the overall behaviors, thus missing failures of the system under test. The challenge is to reconcile the tradeoff between good coverage and high complexity. Combining black box testing with learning (a sequence of increasingly more accurate) models for the tested system was suggested for improving the coverage of black box testing. The learned models can be used to perform more comprehensive exploration, e.g., using model checking. We present a light-weight approach that employs machine learning ideas in order to improve the coverage and accelerate the testing process. Rather than focus on constructing a complete model for the tested system, we construct a kernel, whose nodes are consistent with prefixes of test sequences that were examined so far; as part of the testing process, we keep refining and expanding the kernel. We detect whether the kernel itself contains faulty executions. Otherwise, we exploit the kernel to generate further test sequences that use only a reduced set of representative prefixes.