Validating Model Transformations using Search-based Software Testing

Abstract

Model Transformations (MTs) are the cornerstone of Model-Driven Engineering (MDE). MTs systematically transform input models to output models. Validating MTs establishes credibility of MDE which finds its applicability in avionics and automotive industries, however; it is a non-trivial task as models are complex structures consisting of attributes and associations of various cardinalities. These models conform to their corresponding Metamodel (MM) which defines the model structure. A MM further imposes additional constraints on models that they must satisfy. These constraints manifest as Boolean expressions adding to the complexity of models, making it all the more challenging to validate MTs for which test models need to be generated. Previous studies showed that formal techniques for Test Model Generation (TMG) involved overhead of intermediate formalism, were time consuming and suffered from combinatorial explosion. In contrast to formal techniques, Search-based Software Testing (SBST) demonstrated effective and efficient TMG. SBST relies on search algorithms guided by heuristics and a Fitness Function (FF) defined using different coverage criteria targeting model constraints. Previously, FF based on weaker criteria such as Decision Coverage (DC) has been widely studied. Few studies cater stronger condition-based coverage criteria only by reusing DC‟s FF. This results in inadequate coverage for stronger criteria that are often mandated as standards in MDE industries. To better cater condition-based criteria, we propose a five-step approach employing SBST. A novel condition-based FF is also proposed. Modified Condition/Decision Coverage (MC/DC) is selected as the coverage criterion. Alternating Variable Method (AVM) is selected as the search algorithm. An existing tool named EsOCL is extended to realize our approach. Two case studies of varying complexity are used to evaluate our approach in terms of coverage and success rate. Our condition-based FF is compared with the widely studied DC‟s FF. Results are verified by means of an extensive analysis. Our results demonstrate a significant improvement of ~36.2% in terms of coverage and ~0.3% in terms of success rate. Our proposed approach advocates for the efficacy of our condition-based FF which delivers promising results, ranging from weaker to stronger coverage criteria, in comparison to existing DC‟s FF