Mining Metrics for Understanding Metamodel Characteristics
-
Upload
alfonso-pierantonio -
Category
Presentations & Public Speaking
-
view
177 -
download
1
description
Transcript of Mining Metrics for Understanding Metamodel Characteristics
Dipartimento di Ingegneria e Scienze
Università degli Studi dell’Aquiladell’Informazione e Matematica
Mining Metrics for Understanding Metamodel Characteristics
Alfonso Pierantonio Juri Di RoccoDavide Di RuscioLudovico Iovino
Alfonso Pierantonio
2OutlineIntroduction MotivationMeasuring metamodelsCalculation of metrics correlationSelection of metrics correlationData AnalisysConclusionsFuture works
Alfonso Pierantonio
3IntroductionMetamodels are a key concept in Model-Driven Engineering: they represent the «trait-d'union» among all constituent components Metamodels formally define the modeling primitives used in modeling activities
Alfonso Pierantonio
4MotivationIt is of crucial relevance to investigate common characteristics of metamodels, in order to have a better understanding on– how they evolve over time– what is the impact of metamodel changes throughout the
modeling ecosystem
Metamodel
Alfonso Pierantonio
5MotivationIt is of crucial relevance to investigate common characteristics of metamodels, in order to have a better understanding on– how they evolve over time– what is the impact of metamodel changes
throughout the modeling ecosystem
Analyse metamodels in order to evalutate their structural characteristics and the impact they might have during the whole metamodel life-
cycle especially in case of metamodel evolutions
Analysing metamodel characteristics by investigating
the correlations of different metrics applied on a corpus of more than
450 metamodels
Alfonso Pierantonio
7Measuring MetamodelsThe applied process is able to identify linked structural characteristics Understand how they might change depending on the nature of metamodels
Alfonso Pierantonio
8Metrics calculationConsists of the application of metrics on a data set of metamodelsThe applied metrics are borrowed from [1] and we added new ones by leading to a set of 28 metricsAn excerpt of the considered metrics have been considered in rest of the presentation
[1] W. James, Z. Athansios, M. Nicholas, R. Louis, K. Dimitios, P. Richard, and P. Fiona. What do metamodels really look like? Frontiers of Computer Science, 2013.
Alfonso Pierantonio
9Metrics calculationThe corpus of the analyzed metamodels has been obtained by retrieving metamodels from different repositories, i.e., EMFText Zoo, ATLZoo, Github, GoogleCodeFor a total number of 466 metamodels belonging to different technical spaces and domains
Alfonso Pierantonio
10Metrics CalculationThe metrics calculation has been implemented by exploiting a model-driven toolchain
Alfonso Pierantonio
11Metrics CalculationThe metrics calculation has been implemented by exploiting a model-driven toolchainThe Metamodel Metrics
Calculator is able to calculate for each metamodel all the
considered metrics
Alfonso Pierantonio
12Metrics CalculationThe metrics calculation has been implemented by exploiting a model-driven toolchain
The Metamodel Metrics Calculator is an ATL
transformation whose target models conform to the Metrics
metamodel
Alfonso Pierantonio
13Metrics CalculationThe metrics calculation has been implemented by exploiting a model-driven toolchain
Generating CVS files enables the adoption of statistical
tools like IBM SPSS, Microsoft Excel, and Libreoffice Calc for
subsequent analysis of the generated data
Alfonso Pierantonio
14Calculation of metrics correlationsCorrelation is used to detect cross-links and assess relationships among observed dataWe have considered Pearson’s and Spearman’s coefficients to measure the correlations among calculated metamodel metricsSpearman only for highlighting curvilinear correlationsBoth Pearson’s and Spearman’s correlation indexes assume values in the range of -1.00 (perfect negative correlation) and +1.00 (perfect positive correlation)A correlation with value 0 indicates that between two variables there is no correlation
Alfonso Pierantonio
15Selection of metrics correlationsWe have calculated the Pearson’s and Spearman’s correlation indexes for all the values of the considered metricsFor each couple we have selected the coefficient index (between Pearson and Spearman) having higher correlation values
Alfonso Pierantonio
16Selection of metrics correlationsThe bar chart shows the degree of metric correlation: the higher the bar, the more the metric is correlated with other metrics.
Alfonso Pierantonio
17Pearson’s correlation matrixAll the values greater than 0.7 or lesser than -0.73 have been highlighted to select the metrics that are most related according to the Pearson’s index.
Alfonso Pierantonio
18Pearson’s correlation matrixAll the values greater than 0.7 or lesser than -0.73 have been highlighted to select the metrics that are most related according to the Pearson’s index.
Alfonso Pierantonio
19Data analysisThe aim is to discuss and interpret the most relevant correlations between structural characteristics which have been found in the previous stages
Alfonso Pierantonio
20Data analysis: abstractionHow the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes
Alfonso Pierantonio
21Data analysis: abstractionHow the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes
Alfonso Pierantonio
22Data analysis: abstractionHow the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes
MGHLMaximum hierarchical depth
Alfonso Pierantonio
23Data analysis: abstractionHow the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes
MHSMaximum Hierarchy Siblings
Alfonso Pierantonio
24Data analysis: abstractionHow the number of metaclasses is related to the adoption of abstraction constructs, ie. abstract metaclasses and supertypes
MTCWSNumber of metaclasses having at least one super type
Alfonso Pierantonio
25Metrics
Alfonso Pierantonio
26Data analysis: abstractionThe #metaclasses and the #classes with supertypes are strongly correlated (with Pearson index 0.99) When the metamodel grows in size, the number of inheritance hierarchies also increases, however the hierarchy depth is somehow independent from the metamodel size.
Interestingly, metamodel designers prefer to add siblings in hierarchies instead of adding new hierarchy levels
Alfonso Pierantonio
27Data analysis: abstractionThis is testified by the graph that shows the values of the MHS (Max Hierarchy Sibling) and MGHL (Max generalization hierarchical level) metricsConfirmed by the Pearson correlation indexes between MC and MHS (0.70) and that between MC and MGHL (0.66).
Alfonso Pierantonio
28Data analysis: abstractionIn metamodels with at most 50 metaclasses:– the number of supertypes in
hierarchy is in between 0 and 20
– the number of siblings in a hierarchy is in between 0 and 10
– the maximum height of a hierarchy is in between 0 and 5
Alfonso Pierantonio
29Data analysis: structural featuresHow structural features are used with hierarchiesWe can consider the average number of features (ASF) and the total number of metaclasses with supertypes (MCWS) metricsThe Spearman approach permits to identify a greater correlation index
Alfonso Pierantonio
30Data analysis: structural featuresIncreasing the #metaclasses with supertypes, the average # of structural features in a metaclass decreases
Alfonso Pierantonio
31Data analysis: structural featuresBy considering metamodels having in between 1 and 50 metaclasses, the average number of features (excluding the inherited ones) of a metaclass ranges between 1 and 5
Alfonso Pierantonio
32Data analysis: structural featuresHow the number of featureless metaclasses is related to hierarchies heightThis can indicate how specializations of metaclasses can introduce or reduce structural features in metamodelsMCWS and IFLMC are strongly correlated as supported by the Pearson’s index having value 0.890
Alfonso Pierantonio
33Data analysis: structural featuresHow the number of featureless metaclasses is related to hierarchies height
Alfonso Pierantonio
34Data analysis: structural featuresBy increasing the number of metaclasses with super types, the number of metaclasses without attributes or references increases tooThis means that when hierarchies are introduced, usually existing features are subject to refactoring operations mainly to move them to super classes and to create leaves in the hierarchies inheriting features from the super types.
Alfonso Pierantonio
35Data analysis: isolated metaclassesHow isolated metaclasses are distributed In this case we considered another statistical instrument named Pareto analysis, which is commonly referred to as the 80-20 principle (20% of the causes account for 80% of the defects)About 80% of the analyzed metamodels have a percentage of isolated metaclasses in the range 0-19%, by testifying the fact that isolated metaclasses are not commonly used
Alfonso Pierantonio
36Data analysis: isolated metaclassesWe can claim that they are used only for testing or educational purposes
Alfonso Pierantonio
37ConclusionsWe proposed a number of metrics which can be used to acquire objective, transparent, and reproducible measurements of metamodels.The major goal is to better understand the main characteristic of metamodels, how they are coupled, and how they change depending on the metamodel structure A correlation analysis has been performed to identify the most cross-linked metrics, which have, in turn, been computed over 450 metamodels
Alfonso Pierantonio
38ConclusionsThese figures have been discussed in detail highlighting the followings- the adoption of inheritance is proportional to
the size of metamodels- the number of metaclasses with supertypes are
inversely proportional to the average number of structural features
- the number of metaclasses with supertypes is proportional to the number of metaclasses without attributes or references
- isolated metaclasses are not commonly used apart from testing or educational purposes.
Alfonso Pierantonio
39ConclusionsThese figures have been discussed in detail highlighting the following
Thank you