Post on 05-Aug-2015
Dipartimento di Ingegneria e Scienze
Università degli Studi dell’Aquila
dell’Informazione e Matematica
Mining Correlations of ATL Transformation and Metamodel Metrics
Juri Di RoccoDavide Di RuscioLudovico IovinoAlfonso Pierantonio
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
3IntroductionOver the last decades many MDE technologies have been conceived to support a wide range of modeling and model management activities
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
4IntroductionAvailability of powerful languages and tools for developing, testing, and chaining model transformations
Limited support for analysing and understanding common characteristics of model transformations
• what are the main constructs typically used when developing transformations ?
• to what extent is the development of model transformations affected by the complexity of the corresponding metamodels ?
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
5IntroductionAvailability of powerful languages and tools for developing, testing, and chaining model transformations
Limited support for analysing and understanding common characteristics of model transformations
• what are the main constructs typically used when developing transformations ?
• to what extent is the development of model transformations affected by the complexity of the corresponding metamodels ?
MMs MMtT
?
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
6IntroductionSeveral metrics are available to measure ATL transformations
Quality attributes have been also defined and they have been aligned to a set of metrics
None of the existing approaches deal with transformation metrics correlation
Correlating transformation and metamodels metrics is also unexplored
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
7Contribution
Correlation of several metrics
• 28 metamodel metrics
• 35 transformation metrics
It has been applied on a corpus of
• 91 ATL transformations
• 72 corresponding metamodels
Preparatory study to estimate the required effort to develop model transformations depending on the structural characteristics of the input and target metamodels
Analysis process for understanding model transformations characteristics
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
8Proposed analysis process
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
9
Metrics calculation
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
10Metrics calculationConsists of the application of metrics on a data set of metamodels and transformations
Sample metamodel metrics
Sample transformation metrics
Davide Di Ruscio
11Metrics calculationThe metrics calculation has been implemented by exploiting a model-driven toolchain
Artifact 1
Artifact 2
Artifact n
MetricsCalculator
MetricsCSV generator
Davide Di Ruscio
12Metrics calculationThe metrics calculation has been implemented by exploiting a model-driven toolchain
Artifact 1
Artifact 2
Artifact n
MetricsCalculator
MetricsCSV generator
The Metrics Calculator is able to calculate for each artifact all
the considered metrics
Davide Di Ruscio
13Metrics calculationThe metrics calculation has been implemented by exploiting a model-driven toolchain
Artifact 1
Artifact 2
Artifact n
MetricsCalculator
MetricsCSV generator
The Metrics Calculator is an ATL transformation whose
target models conform to the Metrics metamodel
Davide Di Ruscio
14Metrics calculationThe metrics calculation has been implemented by exploiting a model-driven toolchain
Artifact 1
Artifact 2
Artifact n
MetricsCalculator
MetricsCSV generator
Generating CVS files enables the adoption of statistical
tools like IBM SPSS, Microsoft Excel, and Libreoffice Calc for
subsequent analysis of the generated data
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
15
Calculation, selection, and statistical
significance of metrics correlation
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
16Calculation of metrics correlations
Correlation is used to detect cross-links and assess relationships among observed data
Pearson’s and Spearman’s coefficients to measure the correlations among calculated metamodel and transformation metrics
Pearson’s correlations Spearman’s correlations
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
17Calculation of metrics correlations
Both Pearson’s and Spearman’s correlation indexes assume values in the range of -1.00 (perfect negative correlation) and +1.00 (perfect positive correlation)
A correlation with value 0 indicates that between two variables there is no correlation
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
18Selection of metrics correlations
Pearson’s correlation indexes for all the values of the ATL transformation metrics
Spearman’s correlation indexes for all the values of the ATL transformation and metamodel metrics
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
19Selection of metrics correlations
ATL transformation metrics correlations
All the values greater than 0.8 have been highlighted to select the metrics that are most related according to the Pearson’s index.
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
20Selection of metrics correlations
ATL transformation metrics correlations
All the values greater than 0.8 have been highlighted to select the metrics that are most related according to the Pearson’s index.
Number of output patterns (OP) <->
number of bindings (B)
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
21Selection of metrics correlations
ATL transformation metrics correlations
All the values greater than 0.8 have been highlighted to select the metrics that are most related according to the Pearson’s index.
Number of Transformation Rules (TR) <->
Number of Rules with a Filter Condition on the Input Pattern (RWF)
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
22Selection of metrics correlations
ATL transformation metrics correlations
All the values greater than 0.8 have been highlighted to select the metrics that are most related according to the Pearson’s index.
Number of Helper (H) <->
Number of Helper with Context (HWC)
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
23Selection of metrics correlations
ATL transformation and metamodel metrics correlations
All the values greater than 0.6 have been highlighted to select the metrics that are most related according to the Spearman’s index.
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
24Selection of metrics correlations
ATL transformation and metamodel metrics correlations
All the values greater than 0.6 have been highlighted to select the metrics that are most related according to the Spearman’s index.Number of structural features in the output metamodel (SF - OUTPUT)
<-> Number of binding (B)
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
25Statistical significance of metric correlations
Just because two variables are related, it does not necessarily mean that one directly causes the other.
It is necessary to assess that the performed analysis is statistically
significant or not
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
26Statistical significance of metric correlations
Significance level of a statistical hypothesis refers to the probability that the random sample that has been chosen is not representative
• the lower the significance level, the more confident one can be in replicating the performed results
T-test has been used to establish if the identified correlation coefficients were statistically significant
• the threshold 0.05 has been considered
• correlations which induce a T-test value above the threshold have been rejected
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
27Data analysisThe aim is to discuss and interpret the most relevant correlations between structural characteristics which have been found in the previous stages
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
28Data analysisHow transformation rules are influenced by target metamodels
Number of output metaclasses (OUT MC)
<-> Number of Transformation Rules
(TR)
Transformation development is typically output driven (developer tries to cover all the metaclasses
in the target metamodel)
Spearman’s index: 0.746Significance value: 0.0002
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
29Data analysisHow the total number of transformation input patterns are influenced by the source metamodels
Number input metaclasses (IN MC)
<-> Number of Input Patterns
(IP)
Even though not evident like in the previous case, IN MC and IP seem
to increase together and this might be related to the “coverage”
characteristic of the transformations in the considered
corpus
Spearman’s index: 0.692Significance value: 0.0001
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
30Data analysisHow the structural features in the target metamodel influence the number of bindings
Number of output structural features
(OUT SF) <->
Number of Bindings (B)
Both OUT SF and B seem to increase together and this might
be related to the “coverage” characteristic of the
transformations in the considered corpus
Spearman’s index: 0.808Significance value: 0.07
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
31Data analysisHow general purpose and domain specific metamodels affect the complexity of model transformations
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
32Data analysisHow general purpose and domain specific metamodels affect the complexity of model transformations
Number of Rules with a Filter Condition on the Input Pattern
In GPL2GPL and GPL2DSL transformations only parts of metamodels and hierarchies are considered
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
33Data analysisHow general purpose and domain specific metamodels affect the complexity of model transformations
Number of Rules with a Do Section
In GPL2GPL and DSL2GPL the use of the imperative “do” block is higher than the other cases
Typically imperative constructs are used when the input and output metamodels are completely different
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
34Data analysisHow general purpose and domain specific metamodels affect the complexity of model transformations
Number of Rules with a Using clause
The use of the using clause is very limited and it seems to be one of the less used constructs of ATL
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
35Data analysisHow general purpose and domain specific metamodels affect the complexity of model transformations
Number of calls to resolveTemp
It seems to be never used in GPL2DSL transformations and equally distributed in the other cases
It results to be one of the most complex construct of the ATL language
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
36Data analysisHow developers use the ATL language
Transformations in the considered corpus are mainly developed in a declarative way: • Most of the transformations are developed by means of matched
rules• Helper with contexts are more than those without contexts, which
are usually used as variables in transformations described in an imperative way
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
37ConclusionsAn approach to analyze model transformations by considering also the corresponding metamodels has been discussed
The main goal is to better understand the characteristics of model transformations and how their complexity is related to the complexity of metamodels
A correlation analysis has been performed to identify the most cross-linked metrics
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
38Future workExtend the corpus of artifacts in order to:
• validate the identified correlations
• better investigate the significance value for those correlations that currently are below the threshold
Include in the analysis further kinds of artifacts typically involved in any MDE approach
Rely on the results achieved so far in order to define an approach supporting the early cost estimation for developing model transformations
7th International Workshop on Modeling in Software Engineering – ICSE 2015 Davide Di Ruscio
39Future workExtend the corpus of artifacts in order to:
• validate the identified correlations
• better investigate the significance value for those correlations that currently are below the threshold
Include in the analysis further kinds of artifacts typically involved in any MDE approach
Rely on the results achieved so far in order to define an approach supporting the early cost estimation for developing model transformations
Item for the panel discussion ?
?Thank you