EMPIRICAL EXPLORATION OF SOFTWARE TESTING

Abstract

Despite several advances in software engineering research and development, the quality of software products remains a considerable challenge. For all its theoretical limitations, software testing remains the main method used in practice to control, enhance, and certify software quality. This doctoral work comprises several empirical studies aimed at analyzing and assessing common software testing approaches, methods, and assumptions. In particular, the concept of mutant subsumption is generalized by taking into account the possibility for a base program and its mutants to diverge for some inputs, demonstrating the impact of this generalization on how subsumption is defined. The problem of mutant set minimization is revisited and recast as an optimization problem by specifying under what condition the objective function is optimized. Empirical evidence shows that the mutation coverage of a test suite depends broadly on the mutant generation operators used with the same tool and varies even more broadly across tools. The effectiveness of a test suite is defined by its ability to reveal program failures, and the extent to which traditional syntactic coverage metrics correlate with this measure of effectiveness is considered.