Test Case Generation with Hecate: To Infinity and Beyond!

Developing dependable cyber-physical systems often requires engineers to uncover software defects. Search-based software testing (SBST) is a wellestablished approach to support this activity, yet broader industrial uptake calls for solid empirical evidence across multiple benchmarks and application domains. In this replication study, we report our experience evaluating SBST for generating failure-revealing test cases on two representative controllers: the software controller of an electric-bike (e-Bike) motor and the software controller of an autonomous drone executing line-following tasks. For the e-Bike controller, we replicate our prior ssessment of Hecate, an SBST framework for Simulink® models, and analyze its effectiveness and efficiency. For the drone controller, we extend the evaluation by comparing Hecate with S-TaLiRo using the same set of requirements and testing budget. The results indicate that SBST can expose failures within practical time limits in both domains, while the comparison also sheds light on differences in failure-discovery capability and search efficiency between the two tools. We present our lessons learned, discuss implications for industrial adoption, and outline how these findings can help advance current practice.