Evaluating the FITTEST automated testing tools: An industrial case study

Cu D. Nguyen, Bilha Mendelson, Daniel Citron, Onn Shehory, Tanja E.J. Vos, Nelly Condori-Fernandez

Research output: Contribution to journalConference articlepeer-review

5 Scopus citations


This paper aims at evaluating a set of automated tools of the FITTEST EU project within an industrial case study. The case study was conducted at the IBM Research lab in Haifa, by a team responsible for building the testing environment for future development versions of an IBM system management product. The main function of that product is resource management in a networked environment. This case study has investigated whether current IBM Research testing practices could be improved or complemented by using some of the automated testing tools that were developed within the FITTEST EU project. Although the existing Test Suite from IBM Research (TSibm) that was selected for comparison is substantially smaller than the Test Suite generated by FITTEST (TSfittest), the effectiveness of TSfittest, measured by the injected faults coverage is significantly higher (50% vs 70%). With respect to efficiency, by normalizing the execution times, we found the TSfittest runs faster (9.18 vs. 6.99). This is due to the fact that the TSfittest includes shorter tests. Within IBM Research and for the testing of the target product in the simulated environment: the FITTEST tools can increase the effectiveness of the current practice and the test cases automatically generated by the FITTEST tools can help in more efficient identification of the source of the identified faults. Moreover, the FITTEST tools have shown the ability to automate testing within a real industry case.

Original languageEnglish
Article number6681376
Pages (from-to)332-339
Number of pages8
JournalInternational Symposium on Empirical Software Engineering and Measurement
StatePublished - 2013
Externally publishedYes
Event2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2013 - Baltimore, MD, United States
Duration: 10 Oct 201311 Oct 2013


Dive into the research topics of 'Evaluating the FITTEST automated testing tools: An industrial case study'. Together they form a unique fingerprint.

Cite this