Integrated test case generation and prioritization based on multiple coverage criteria

Purpose: Software testing is crucial for ensuring software quality, but it often consumes significant time and resources. Test case generation tools can generate test cases which cover the elements of tested program. However, some types of test case generation algorithms, such as those based on symbolic execution, can also gather coverage information during the test case generation process, which is undoubtedly valuable for other testing tasks. Test case prioritization (TCP) helps detect software faults earlier, achieving better testing outcomes under limited resources. Our goal is to enhance TCP without increasing costs, making it applicable across all white-box testing while improving its effect. Methods: We developed MCTCG, a symbolic execution-based test case generation tool that produces test case sets meeting various coverage criteria. During generation, MCTCG analyzes program coverage for elements like statements and branches to guide the next step. We also proposed TCP _all−first , a prioritization method that uses the coverage information of MCTCG to prioritize test cases based on the principle of 'maximizing coverage step by step.' It selects test cases that cover the most uncovered elements until full coverage is achieved. Since TCP _all−first directly uses the coverage data of MCTCG, there is no need to re-execute the test cases to determine priority, allowing test case generation and prioritization to occur in the same process, saving significant testing resources. Results and conclusion: We evaluated our approach using multiple open-source programs. The results show that MCTCG successfully generates high-coverage test case sets according to various coverage criterias, with higher bug detection rates. TCP _all-first uses the coverage information obtained during the generation process to determine test case prioritization. Compared with the best performing among other TCP methods, TCP _all−first improved the APFD value by 4.11% and reduced the first-fail metric by 8.7%. Moreover, TCP _all−first required the fewest test cases to detect the same number of bugs compared to all other methods. Therefore, we believe that MCTCG effectively enhances test case set coverage while gathering coverage information, and TCP _all−first extends the applicability of TCP and improves coverage-based TCP techniques.