DEVELOPMENT OF MBSE/UML MATURITY
MODEL
ÖZLEM DEMIRCI
MASTER THESIS 2010
INFORMATICS
DEVELOPMENT OF MBSE/UML MATURITY MODEL
ii
SUBJECT
INFORMATION TECHNOLOGIES AND MANAGEMENT IN
INFORMATICS
DEVELOPMENT OF MBSE/UML
MATURITY MODEL
ÖZLEM DEMIRCI
This exam work has been carried out at the School of Engineering in Jönköping in the
subject area IT and Management in Informatics. The work is the two-year university
diploma program of the Master of Science program.
The authors take full responsibility for opinions, conclusions and findings presented.
Examiner: Vladamir Tarasov
Supervisor: Christer Thörn
Abstract
iii
Abstract
Model Based Software Engineering (MBSE) is becoming popular in the industry today. Many
organizations, also including small and medium scales organizations are adopting modeling
approaches. Models are becoming the main artifact of the software development process and
they provide organization to test their product and detect problems before the implementation.
UML (Unified Modeling Language) is the most common modeling language that is used by
organizations during modeling process so that focus is on UML in the organizations.
Therefore organizations that are developing software products in the sense of MBSE and
using UML as a main modeling language are the interests of this paper. These organizations
that are performing modeling approaches are not aware of how advanced or un-advanced they
are performing the related modeling approaches during model based software development
process. They are unaware of measuring and evaluating their modeling practices. This is
because of lack of standards and guidelines regarding model based software development. By
considering this unawareness of organizations, various affects that can effect on model based
development process are examined like modeling process, quality assurance techniques,
personnel competence on modeling practices, training of personnel, design and structure of
the UML Models, modeling tools, transformation issues, and syntax and semantic regulations
of UML. These are considered as most essential aspects regarding the occurrence in literature.
Regarding these aspects, a MBSE/UML Maturity Model is provided where 6 maturity levels
regarding the aim of modeling within the organizations and aspects required to be filled out.
Therefore this Maturity Model different than traditional software development maturity
models consider various aspects at different levels of organization and consider models as the
main development artifacts. The practices and tasks required to be performed are based on
models and modeling approaches. This MBSE/UML Maturity Model provides both
practitioners and researchers to evaluate and measure the proficiency of modeling approaches
performed in the organizations.
Acknowledgments
iv
Acknowledgments
I would like to thank my supervisor Christer Thörn at Jönköping School of Engineering for
helping me to shape the idea behind this thesis work and supporting me every step of the
research process. Thank You.
Key Words
v
Key Words
Model Based Software Engineering (MBSE), Model Driven Development (MDD), UML
(Unified Modeling Process), Maturity Model
Table of Contents
vi
Table of Contents
1. INTRODUCTION ............................................................................................................. 1
1.1 Background .................................................................................................................. 1
1.2 Problem Description .................................................................................................... 2
1.3 Purpose ........................................................................................................................ 3
1.4 Research Questions ...................................................................................................... 4
1.5 Topic Vocabulary ........................................................................................................ 5
1.6 Disposition ................................................................................................................... 5
2. THEORETICAL BACKGROUND ................................................................................... 7
2.1 Previous Research ........................................................................................................ 7
2.2 Models and Model Based Software Engineering (MBSE) .......................................... 8
2.3 Modeling Languages and UML ................................................................................. 10
2.3.1 Types of UML Diagrams ................................................................................... 12
2.3.2 UML Profiles ...................................................................................................... 15
2.3.3 OCL (Object Constraint Language) ................................................................... 17
2.3.4 Meta-Model ........................................................................................................ 18
2.4 DSL (Domain Specific Language) ............................................................................ 18
2.5 Maturity Model .......................................................................................................... 19
3. METHODOLOGY ........................................................................................................... 20
3.1 Data Sources .............................................................................................................. 20
Table of Contents
vii
3.2 Data Collection .......................................................................................................... 21
3.3 Data Analysis ............................................................................................................. 21
3.4 Maturity Model Development Method ...................................................................... 22
3.5 Quality of Science ..................................................................................................... 25
4. EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF ASPECTS 26
4.1 Quality of UML Diagrams in Modeling .................................................................... 26
4.2 Evaluation Criteria for MBSE/UML Maturity Model ............................................... 28
4.2.1 Essential Aspects for MBSE/UML Maturity Models ........................................ 28
4.2.2 Indicators for Each Aspect in MBSE/UML Maturity Model ............................. 32
4.2.3 Different Levels of Organization for MBSE/UML Maturity Model ................. 42
4.2.4 The General Matrix for MBSE/UML Maturity Model ...................................... 43
5. MBSE/UML Maturity Model ........................................................................................... 47
5.1 MBSE/UML Maturity Model Structure and Levels .................................................. 47
5.2 MBSE/UML Maturity Level Dimension ................................................................... 48
5.2.1 Maturity Level 0: No Modeling ......................................................................... 49
5.2.2 Maturity Level 1: Basic Modeling ..................................................................... 49
5.2.3 Maturity Level 2: Initial MBSE/UML ............................................................... 50
5.2.4 Maturity Level 3: Defined MBSE/UML ............................................................ 51
5.2.5 Maturity Level 4: Integrated MBSE/UML ......................................................... 52
5.2.6 Maturity Level 5: Optimized MBSE/UML ........................................................ 53
Table of Contents
viii
6. Assessment of Maturity Levels of MBSE/UML Maturity Model ................................... 54
7. VALIDTY OF MBSE/UML MATURITY MODEL ....................................................... 63
8. CONCLUSION ................................................................................................................ 64
REFERENCES ......................................................................................................................... 66
APPENDIX .............................................................................................................................. 71
Appendix 1: MBSE/UML Modeling Process Evaluation Questionnaire ............................. 71
Appendix 2: MBSE/UML Quality Assurance Techniques Evaluation Questionnaire ........ 78
Appendix 3: MBSE/UML Personnel Competence Evaluation Questionnaire vol. 1........... 85
Appendix 4: MBSE/UML Personnel Competence Evaluation Questionnaire vol. 2........... 92
Appendix 5: MBSE/UML Tools Evaluation Questionnaire ................................................ 94
Appendix 6: MBSE/UML Design Evaluation Questionnaire ............................................ 100
Appendix 7: MBSE/UML Syntax and Semantic Evaluation Questionnaire ...................... 108
Appendix 8: MBSE/UML Transformation Evaluation Questionnaire .............................. 110
Table of Figures
ix
Table of Figures
Figure 1 A sample framework for MBSE/UML Maturity Model ............................................... 4
Figure 2 Disposition ................................................................................................................... 6
Figure 3 MDD Adopting Spectrum, adapted from [1] ............................................................. 10
Figure 4 Krutcher 4+1 view model .......................................................................................... 12
Figure 5 Sample Creating a New Blog Account Use Case Diagram [11] .............................. 14
Figure 6 Sample Activity Diagram [11] .................................................................................. 15
Figure 7 Sample Class Diagram for Blog Account .................................................................. 15
Figure 8 Example of UML Profile ........................................................................................... 17
Figure 9 Sample OCL example for UML Profile ―colors‖ [39] ............................................. 18
Figure 10 Followed Data Analysis Steps ................................................................................. 22
Figure 11 Maturity Model Development Steps ........................................................................ 23
Figure 12 A Sample Declaration of Internal and External Stakeholder .................................. 33
Figure 13 A Sample ATM UML Use Case Diagram ................................................................ 34
Figure 14 Effects of Crossing and Bends [22] ......................................................................... 37
Figure 15 The Structure of MBSE/UML Maturity Model (adapted from [47] and [48]) ........ 47
Figure 16 MBSE/UML Maturity Model Levels ........................................................................ 49
INTRODUCTION
1
1.INTRODUCTION
1.1 Background
“My method is different. I do not rush into actual work. When I get a new idea, I start at once
building it up in my imagination, and make improvements and operate the device in my mind.
When I have gone so far as to embody everything in my invention, every possible
improvement I can think of, and when I see no fault anywhere, I put into concrete form the
final product of my brain” said Nikola Tesla. And in software engineering similar method has
recently been widely used.
Nikola Tesla did not talk about the use of modeling as a process in software development, but
the idea can be considered more or less as same. Even though models are been used in the
software industry for a long time, the activity of using modeling as a process in software
development has recently become widespread since the need to abstract the system and its
behaviors have been considered by both researchers and practitioners. Software engineers did
not start development by implementation of code phase in software life cycle, including steps
like requirement specifications, analysis, design, implementation and maintenance. The aim
of these processes is to improve the productivity and to solve possible problems at early
stages of development. By applying the principles of Model Based Software Engineering
(MBSE) where the models are developed as solutions for particular problems during software
development process, these aims can be achieved much easier. For instance, if the modeling is
performed in an advanced way it is possible to detect errors early in the development life
cycle or to test the software models before the implementation is executed.
Modeling can be performed for different applications in software development. Besides the
large scale enterprises, both medium and small sized enterprises are also interested in
modeling. Due to the wide application of modeling, numerous informal and formal
approaches to modeling have been developed, such as Entity Relationship Diagrams (ERD)
for modeling data, Specification and Description Language (SDL) for modeling
telecommunication systems, formal modeling languages such as Z and B, and the Unified
Modeling Language (UML) which is the modeling language most widely used in industry
today [1].
Modeling a system is not simple. There are various affects and aspects need to be considered.
For instance, modeling a large scale system is complex since it isn‟t possible to model some
of the aspects and elements of such a system because of lack of modeling approaches and a
single modeling language. Moreover, based on literature review, it can be stated that there
exists no single modeling language that can cover all the needs to model a system. However,
UML (Unified Modeling Language) that is proposed by OMG is the most common modeling
language used in industry and also an interest of researchers. Modeling language is the core of
INTRODUCTION
2
using modeling as a process. Therefore the syntax need be well defined and regulated. What
we mean by syntax here is the rules, regulations, and notations to describe a modeling
language and semantic is the meaning of expressions stated in the language. In addition to
these, modeling requires a good quality of design since there exists both technical and un-
technical people, so the design of the models should be clear and comprehensible. The
modeling process need to be well documented and the participants in this process need to be
well trained and skilled. All these issues are considered by the industry but sometimes they do
not give that much importance to one or more of these. Since this is a new field, a proper,
qualified, and sufficient approach to adopt this new process do not exist.
Therefore, in this thesis a maturity model for modeling approaches in organization is
presented by indicating the aspects that need to be considered at different levels of
abstraction. Moreover, the activities and tasks that needs to be performed in order to use
modeling as a process in an advanced way is provided. A focus on UML is provided in this
thesis since it is the most common modeling language in the industry. The maturity model
provided here includes some levels for the proficiency of the organization using modeling
approaches and UML together. The goal of this maturity model is that an organization and
also researchers can assess their proficiency level regarding their modeling approaches and
can get knowledge what else they need to do in order to perform modeling approaches in an
advanced way.
1.2 Problem Description
Regarding the popularity of adopting these new approaches and advantages of modeling,
organizations are started to implement these modeling practices and activities. They face with
lack of standardization on MBSE approaches and lack of standardization on modeling
languages that required to be used while adopting modeling approaches, and several other
lacks, insufficient issues, and problems. Organizations that adopt these MBSE approaches are
aware of how advanced they are performing and what they need to do in order to improve
their modeling practices. In other words, they adopt modeling approaches, but do not know
how to evaluate and measure themselves with respect to their practices regarding their
organizational and modeling goals. This leads to problems and discussions on how they can
improve their modeling practices.
There exists several studies on Model Driven Development and models such as quality goals
that need to be achieved, the lacks of modeling languages utilized in adopting modeling
approaches, different and various issues having an impact on modeling process, and other
researches related to on this field. All this researches are essential for people and
organizations that are involved in adopting modeling approaches. The existing maturity
models and/or frameworks are developed for traditional software development processes, so
organizations which are dealing with MBSE are aware of how mature they are applying these
modeling approaches within their organizations. For instance, CMMI (Capability Maturity
INTRODUCTION
3
Model Integration) - a maturity model proposed by SEI (Software Engineering Institute)
describes process improvement activities required to be performed within the organization.
SPICE (Software Process Improvement and Capability Determination) is also supporting
continuous process improvement activities required to be performed within the organization.
Both are developed for traditional software development process so they are not providing
and/or supporting activities and tasks required to be performed in order to improve the
proficiency of model based approaches. There is a research going on developing a Model
Driven Development (MDD) Maturity Model within the MODELWARE1 project which is
taking CMMI as basis and reference so that only process improvement activities are
considered. Moreover CMMI is too complex for many small and medium size organizations.
Therefore this maturity model can be perceived as too complex for small and medium size
organizations as well.
When these factors and problems are taken into consideration, it seems useful to be able to
evaluate the proficiency level of organizations‟ modeling approaches utilization with regard to
the best practices of the organization and the insufficient practices of the organization. An
improvement on model based approaches can be provided.
1.3 Purpose
The goal of this master thesis is to develop a maturity model for measuring and evaluating the
level of MBSE/UML proficiency in an organization. As discussed in previous section (1.2)
MDD is becoming popular in the industry. However, any standards for proficiency in using
modeling approaches do not exist.
As mentioned previously, there exists several maturity models and each measures and
evaluates different aspects at different levels. However, the aspects that they evaluate are
specific and mostly only one aspect. Moreover these existing maturity models are developed
for traditional software development process so that the tasks and activities that need to be
performed differ from MDD where the models became the key artifact of the process.
Therefore, this thesis aims to develop a maturity model that evaluates and measures the
different aspects of MBSE/UML at different levels of organization like individual, team, etc.
Considering that both practitioners and researchers are interested in to evaluate the
proficiency level of adopting modeling approaches within the organization, these both parties
will be able to benefit from the outcomes of this thesis. The practitioners will be able to
examine their modeling practices and the results for their practices so that they will know
what they are doing at advanced level, what are the best practices of them, and what they need
to do in order to improve. Like CMMI, a different way to improve the development practices
and tasks will be shown in this thesis. The researchers will also have benefit from this thesis.
1 MODELWARE is a project co-funded by the European Commission under the “Information
Society Technologies” Sixth Framework Programme (2002-2006).
INTRODUCTION
4
They will be aware of what practices and tasks an organization should consider while
adopting modeling approaches. Researchers will be able to examine the best practices,
evaluation methods of proficiency of modeling approaches, and essential aspects that need to
be considered during MDD process.
The outcomes and results are;
A maturity model for MBSE/UML approaches with a description of aspects and
levels that are targets for evaluations like shown in Figure 1.
Figure 1 A sample framework for MBSE/UML Maturity Model
Sets of instruments for how to perform the evaluations (ex: questionnaires,
experiments, etc.)
A review of existing approaches to maturity in models, evaluations, etc.
1.4 Research Questions
Even though the aim of this thesis is to develop a maturity model for evaluating the
proficiency usage of modeling approaches, the work was guided by research questions. With
these research questions, it is possible to assign and decide the important aspects, practices
and tasks that require to be performed during modeling.
RQ1: What model quality goals are important in MBSE approaches that use UML as a core
modeling language?
RQ2: How to evaluate the quality and proficiency in usage of UML and MBSE approaches
during a software development process?
INTRODUCTION
5
The results that emerge from these research questions enable us to develop the criteria for
different aspects at different abstraction levels of modeling process in order to evaluate the
proficiency of modeling approaches applied in the organizations.
1.5 Topic Vocabulary
Many abbreviations are used in this paper so here the most used abbreviations are presented
below.
MDD= Model Driven Development
MBSE= Model Based Software Engineering
MDA= Model Driven Architecture
MBD= Model Based Development
UML= Unified Modeling Language
OCL= Object Constraint Language
1.6 Disposition
This thesis is divided into 8 chapters which is shown and described briefly in Figure 2 below.
A general knowledge is provided about the thesis and related topics like MBSE, UML, and
maturity models so that the reader can become familiar with the rest of the paper.
The methodology chapter provides the research process of the thesis, how data is collected
and what kind of data is collected so that the reader will be able to understand how the
research is performed.
The rest of the chapters are about the findings from the data sources and analyzing them
according to the purpose of the thesis. In chapter 4, the results for the research questions are
provided. Moreover how these results are analyzed can be also read in chapter 4. These results
constitute the key areas of the topic.
A MBSE/UML Maturity Model is provided in chapter 5. The development of that is based on
the findings from chapter 4, previous studies, and previous research performed on this field.
Then in chapter 6, it is defined how the maturity of organization will be assessed so that the
instruments designed for this aim is presented there.
Chapter 7 indicates how the MBSE/UML Maturity Model will be validated.
Chapter 8 concludes all tasks performed for the thesis.
INTRODUCTION
6
Figure 2 Disposition
THEORETICAL BACKGROUND
7
2.THEORETICAL BACKGROUND
2.1 Previous Research
Previous research on this field in the industry is mostly addressing the model quality goals
that are required to be considered in order to develop good quality models since models are
the main artifacts of the MDD process. For instance, “Definitions and approaches to model
quality in model-based software development”, Parastoo Mohagheghi et al. [1] examines
various articles published on quality of models and define 6C model quality goals
(correctness, completeness, consistency, etc.) for model quality that is commonly used in the
industry. In [22] the authors define model goals regarding UML structure and MBSE
principles. They define different model characteristics, different modeling purposes, and
relationships among them. Then they provide a UML model quality framework for different
phases of software development process. Moreover, in article [9] it is indicated that model
quality goals require to be defined properly while adopting MBSE principles. [23], [8], [7],
[24], and [5] are other researches that deals with model quality goals. The common thing of
these researches is all state that model goals are essential and require to be defined. Moreover
they all indicate that UML‟s semantic structure is lacking when a MDD process is considered.
Defining the model quality goals is not adequate. It has to be supported by providing some
metrics and measurement techniques so that they can be measure and evaluate if the
developed models meet the model quality goals or not. [25], [26], [27], [5], [50], [29], [30],
and [31] provide some metrics that can be applicable to measure and evaluate whether the
model quality goals are met or not. For instance, Baroni et al. [30] emphasize on defining
metrics by using OCL (Object Constraint Language) for UML model quality. Lange [5]
defines metrics and rules. He also defines model quality characteristic and modeling purposes.
Then he relates them to each model quality characteristics regarding the modeling purposes.
His consideration is related to UML model quality which is the same aim as this thesis.
Since MBSE is not only developing models for communication, analysis, and prediction but
also for transformation and code generation. Therefore, the requirements for developing
models as solution and implementation source require additional activities and tasks to be
performed. Moreover many researches [1], [5], [7], and [8] indicate that UML‟s lack of
semantic is an issue required to be considered while adopting MBSE principles so the
concepts like Meta-Modeling, UML Profiles, OCL, and DSL (Domain Specific Languages)
come up. For instance [32], [33], and [34] emphasizes on meta-modeling and [32] provides a
simple meta-model developed for modeling the blueprints of a software system.
With respect to the research on model quality, some experiment and case studies show that
while adopting modeling approaches, defining the model quality goals and measuring them
are not the only issues require to be performed. For instance, [35] is a case study that
THEORETICAL BACKGROUND
8
examines the Motorola‟s MDD process and encounters some issues like lack of common
tools, lack of abstraction, lack of well-defined semantics, lack of integrated and migration
tools, lack of team experience on modeling, ill-defined processes, and so on. Moreover it
states that Motorola is developing a Modeling Challenge Levels to provide guideline for
MDD process adoption. In addition to these, Parastoo Mohagheghi et al. [36] examines two
organizations that are adopting MDD and states that because of lack of standards, guidelines,
and ill-process definitions, organizations are faced with several problems. The author of [37]
also states the issues require to be performed properly like process definition, training of
personal, maturity of modeling technology and maturity of model related methods. The
common of these case studies and experiment reports is that organizations adopting MDD are
unaware of how mature they are performing these activities and tasks. For this issue, there is
only little research [2] performed as mentioned previously.
2.2 Models and Model Based Software Engineering (MBSE)
It is quite better to know and understand what is model at first. Models are representations of
a (software) system at an abstract level [10]. A model tries to capture and present a specific
aspect of a system so that only the information related to this aspect is represented. A model
can consist of several diagrams in order to represent the part of system clearly. For instance,
UML 2.0 proposes 13 different diagrams and each is described for various usages and
representation styles (will be discussed in 2.2 Modeling Languages and UML). These various
types of diagrams provide modelers and/or designers to develop the models at different
abstraction levels by describing different features and representations of the system so that
everyone involved in the development process can understand what is required and described,
and essentially the roles assigned.
The aim of using models differs and there exist many ways of using models so the
requirements to describe and develop the models also differ. Models can be used to test the
parts of the system before implement it so that the productivity can be improved. Moreover
models can be used to provide a clear and coherent communication between stakeholders,
clients, programmers, and so on; or the other field that models used is to present the
integration and interoperability of distributed systems. In MBSE, they are aimed to perform
transformation and achieve code from the models. The Figure 3 below shows the adoption of
modeling within the organization by indicating the different aims of it. For instance, if the
organization aims to implement the code from the models, the requirements for the code,
transformation, and transformation tool that need to be considered and presented in the
models. Let us assume that an organization will develop software programmed by C++, so the
object-oriented requirements that need to be captured and the models should be developed
based on this. Therefore the concepts like inheritance, classes come up and a specific part of
THEORETICAL BACKGROUND
9
the system requires to be developed regarding this structure. Consequently, there exists
different ways, aims, and different domains that models are presented.
All these fields that models used indicate us that models have been used in software
engineering world for a long time. They mostly have been used at the analysis and design
steps of software development life cycle in order to describe the problem and requirements
but solutions. Now by the proposition of MDD, the models became main artifact of software
development process and they are intended to be developed as solutions.
After understanding what is model and for what aims they are mostly used in industry, now it
will be clearer to understand what MBSE and MDD are. There exist several names for those
concepts. Some states that Model Driven Engineering (MDE) rather than MBSE; and some
utilize Model Based Software Development (MBSD) or Model Based Development (MBD)
rather than Model Driven Development (MDD). In this paper, we will use MBSE and MDD
in order to decrease the complexity of existing terms.
In article [11], model based software engineering is defined as performing software
engineering on the level of abstraction and the authors simply define MBSE as the
functionality of the system is presented by the models where the implementation details are
hidden.
In article [13], the models are described as a main artifact in the software development and
they define MBSE as “Model-based software engineering covers software development
methods, where models are the main artifacts and some or all other artifacts are derived from
the models [13]”.
The main difference between traditional software development and model based development
is that models are the artifacts in the model driven development. The authors in [1] indicates
that MDE (Model Driven Engineering) covers approaches where development is carried out
using models; often at different abstraction levels and from multiple views; and they also
focuses on that the models are the core sources in order to perform transformations. That
basically means that models can be used to develop other models or either way to develop the
source code.
THEORETICAL BACKGROUND
10
Figure 3 MDD Adopting Spectrum, adapted from [1]
Model Driven Development (MDD) approaches are used in the paradigm of MBSE. In MDD,
the complexities of the large software systems are handled by raising the level of abstraction.
[14] MDD is defined as “a software engineering approach consisting of the application of
models and model technologies to raise the level of abstraction at which developers create and
evolve software, with the goal of both simplifying (making easier) and formalizing
(standardizing, so that automation is possible) the various activities and tasks that comprise
the software life cycle”. In article [15], the authors are also defining MDD as model based
software engineering approaches where the parts of the software development process are
executed automatically using model transformations.
In this thesis, the common way of using MBSE and MDD approaches is to evaluate the usage,
adopting proficiency of these concepts in the industry so that the proficiency is dependent on
the different usage principles of these approaches and models regarding different domains and
contexts. The criteria, the evaluation methods, and best practices that need to be applied will
be discussed later in this thesis.
2.3 Modeling Languages and UML
Since modeling is being used in different areas; there exist many modeling languages and
standards. Majority of them emphasizes on graphical notation because modeling is considered
as a graphic. However, a modeling language can also be textual. In addition to these some of
the modeling languages are executable so that a source code can be generated from the
models. Therefore, regarding the type and aim of modeling language, the set of rules and
structure of them differ. For instance, a graphical modeling language consists of set of
diagrams and a set of rules exists to indicate the relationship between diagrams.
The modeling approaches require a modeling language as it mentioned previously and
modeling language provides standards, a set of rules, and conventions in order to improve the
THEORETICAL BACKGROUND
11
quality and develop models. At that point, UML which is the de facto Standard for modeling
software systems in industry is considered since when adopting modeling standards and
guidelines within an organization the first goal is to settle a common notation as Scott W.
Ambler stated in his book “The Elements of UML style [38]”. He states that at that point
UML is a good start since it defines a common notation and semantics. Moreover OMG
(Object Management Group)2 states that modeling is the designing of software applications
before coding and using a model, those responsible for a software development project's
success can assure themselves that business functionality is complete and correct, end-user
needs are met, and program design supports requirements for scalability, robustness, security,
extendibility, and other characteristics, before implementation in code renders changes
difficult and expensive to make. OMG„s UML helps to specify, visualize, and document
models of software systems, including their structure and design, in a way that meets all of
these requirements.
The utilization area of UML in the industry is wide. UML models are used at several stages
during software engineering, such as the early analysis of the problem domain, documenting
the architecture or specifying the detailed design of a system. The abstraction level of the used
models varies for the different stages. A UML model serves one or more specific purposes
depending on the development stage and the stakeholders who are using the model [5].
Moreover, in the book “Learning UML 2.0” [11] authors mention the quotes of Fowler who
identifies the usage of models (UML models) in a precise way and states that UML models
used in industry are assigned to three different roles which are;
UML as Sketch: The models are designed to represent the communication rather than
describing the system specifications so that if the organization is using the UML models as a
sketch than the models that need to be understood and agreed by all users. The models need to
be comprehensible.
UML as Blueprint: UML models are designed detailed and provide detailed information for
the programmers to develop the code. Therefore the UML models need to be correct and
complete.
UML as Programming Language: UML models are designed and developed as executable.
The focus and the aim is developing models first and then implementing them into the code
by using transformation tools.
2 http://www.omg.org
THEORETICAL BACKGROUND
12
2.3.1 Types of UML Diagrams
The latest version of UML which is UML 2.0 consists of 13 different diagrams and each has
different purposes. However, some are related to each other. For instance, the class diagrams
are used to show the static design view of a system while sequence diagrams are used to show
the interaction between the objects. Russell Miles and Kim Hamilton in their book “Learning
UML 2.0 [11]” mentions the views of UML diagrams on overall and he focuses on the views
defined by Philippe Kruchten‟s 4+1 view model where a model is broken into several views
and each view captures specific aspect of the system that is modeled. As shown in Figure 4,
the views are logical, process, development, physical, and use case views.
Figure 4 Krutcher 4+1 view model
The Table 1 below provides basic description about Krutchen‟s 4+1 view model and example
UML diagrams.
Table 1 Krutchen’s View Model Description
View Type Description UML Diagram
Logical View System‟s parts and how they
interact each other are described.
Class, Object, State
Machine, and Interaction
Diagrams
Process View The steps and/or tasks of the
processes are described.
Activity Diagrams
Development The system‟s parts organization
into modules and components is
Package and Component
THEORETICAL BACKGROUND
13
View described. Diagrams
Physical View How the final deployed system is
performed regarding to the 3 other
views of the model is described.
Deployment Diagram
Use Case View What the systems should do from
the outside perspective is described.
Use Case, Description, and
Overview Diagrams
OMG also identifies different perspective of UML diagrams. They provide 3 different
perspectives which are shown and described in Table 2.
Table 2 UML’s Types of Diagrams by OMG
Types of Diagrams Example UML Diagrams
Structural Diagrams Class Diagrams
Object Diagrams
Behavioral
Diagrams
Use Case Diagrams
Sequence diagram
Collaboration diagram
State-chart diagram
Activity diagram
Implementation
Diagrams
Component diagram
Deployment diagram
THEORETICAL BACKGROUND
14
As mentioned previously, even though each diagram is developed for different aims, while
modeling a system these diagrams indicate different issues of the system regarding the other
developed UML diagrams. For instance, let us assume that a modeler is developing a model
which is representing a blog page which allows users to have a blog account in order to post
some entries. The Figure 5 simply illustrates the use case diagram of such a system including
the actors and what they can perform. This use case diagram can be thought as a tool to
provide communication with the client so that the client will be able to comprehend what the
software product will do.
In MBSE, UML diagrams are also used to provide specific features of the system and/or the
software product that will be produced. These specifications will provide developers to
comprehend the problem and then how it is required to be solved. When the example of blog
page system is considered, the UML activity shown in Figure 6 simply illustrates how the
tasks and/or activities of development process will be done. It shows how a decision will be
performed when activities are depending on a condition. For instance, in this example it can
be considered as when a blog account user intends to add a new entry, there are some
limitations to publish his/her entry. It is assumed that if the entry is longer than 1000 words
then it exceeds the word size limit or if the entry is empty then it cannot be published either.
Figure 5 Sample Creating a New Blog Account Use Case Diagram [11]
THEORETICAL BACKGROUND
15
Figure 6 Sample Activity Diagram [11]
Russell Miles and Kim Hamilton states that in their book “Learning UML 2.0 [11]” “Use
cases describe the behavior of your system as a set of concerns. Classes describe the different
types of objects that are needed within your system to meet those concerns. [11]” Regarding
this statement, a sample class diagram of blog account example is shown in Figure 7.
Figure 7 Sample Class Diagram for Blog Account
2.3.2 UML Profiles
It is not necessary to define a new language in order to add some constraints or add new UML
elements by the proposition of UML profiles by OMG. This feature allows customizing the
UML elements such as adding new elements, defining restrictions or constraints. In [39]
authors define UML profiles as set of extensions supporting customizations by using UML‟s
THEORETICAL BACKGROUND
16
extension mechanisms. By this feature developers are enabled to develop UML models in
particular domains.
OMG also defines UML Profiles in “Catalog of UML Profile Specifications3”as “A UML
profile is a specification that does one or more of the following Table 3”:
Table 3 UML Profile Specification by OMG adapted from ―Catalog of UML Profile
Specifications‖
Identifies a subset of the UML meta-model.
Specifies “well-formedness rules” beyond those specified by the identified
subset of the UML meta-model.
“Well-formedness rule” is a term used in the normative UML meta-model
specification to describe a set of constraints written in UML‟s Object
Constraint Language (OCL) that contributes to the definition of a meta-model
element.
Specifies “standard elements” beyond those specified by the identified subset
of the UML meta-model.
“Standard element” is a term used in the UML meta-model specification to
describe a standard instance of a UML stereotype, tagged value or constraint.
Specifies semantics, expressed in natural language, beyond those specified by
the identified subset of the UML meta-model.
Specifies common model elements, expressed in terms of the profile.”
There exist already defined UML Profiles such as UML Profile for CORBA for CCM
(CORBA Component Model) or UML Profile Scheduling. For instance, defining UML
Profile for CORBA is intended to be developed in order to be able to model CORBA
interfaces which support expressing the semantics of CORBA in UML tools also being able to
perform transformations from the models. However, organizations by themselves are also
allowed to define UML Profiles regarding their need. For instance, [39] provides a UML
Profile for MultiTEL (Multimedia Telecommunication Services) in which UML Profile
provides a standard way for designing and documenting systems and application frameworks.
3 Catalog of UML Profile Specifications by OMG
http://www.omg.org/technology/documents/profile_catalog.htm
THEORETICAL BACKGROUND
17
Another UML Profile example is provided by [39] which is a simple example that aims to add
two new elements named weights and colors as shown in Figure 8.
Figure 8 Example of UML Profile
2.3.3 OCL (Object Constraint Language)
OCL is provided by OMG as a modeling specification and OCL is a formal, declarative
language that is used to provide expressions to UML models. In another way it can be defined
as specification of formal constraints in context of a UML model. OCL provides a set of rules
that applicable to UML models. The specifications of constraints are essential since pre- and
post-conditions of operations, invariants, derivation rules for attributes and association etc. are
expressed by constraints. Moreover the restrictions are defined by constraints. Any kind of
UML element can be related with constraints including the UML Profiles that are defined for
new elements. The aforementioned example for adding two new elements which are weights
and colors, the authors of [39] indicates that only classes and associations can be colored. As
a result for this constraint, the following OCL shown in Figure 9 is derived by [39];
THEORETICAL BACKGROUND
18
Figure 9 Sample OCL example for UML Profile ―colors‖ [39]
2.3.4 Meta-Model
A meta-model is a model describing the model. Meta-models capture the different aspects of
the system and make different models to follow the same guideline so that all models
describing the system achieve consistency.
The authors of “Meta-Modeling Semantics of UML[4]” states that “The UML semantics is
described using a meta-model that is presented in terms of three views: the abstract syntax,
well-formedness rules, and modeling element semantics.”
Moreover OMG presents MOF (Meta Object Facility)4 which is a standard used to define the
UML meta-model. It is required to extend the UML meta-model because of its syntax and
semantic lacks. Therefore, MOF is used for this issue.
Therefore meta-modeling is a technique used to provide descriptive modeling language rules
by aligning formal semantics to UML.
2.4 DSL (Domain Specific Language)
DSL is a form of computer languages that are dedicated to a particular domain. Martin
Fowler5 describes it as “The basic idea of a domain specific language (DSL) is a computer
4 http://www.omg.org/mof/
context UML::InfrastructureLibrary::Core::
Constructs::Association
inv : self.isStereotyped(“Coloured”) implies
self.connection->forAll
(isStereotyped(“Coloured”)
implies color=self.color)
THEORETICAL BACKGROUND
19
language that's targeted to a particular kind of problem, rather than a general purpose
language that's aimed at any kind of software problem.” If the wide application areas of
modeling approaches are considered, it can be achieved that DSLs provide better
understanding for different domain problems since each domain has different requirements
and features need to be considered.
2.5 Maturity Model
A maturity model provides a systematic framework to assess how mature the organizations
are developing software products where maturity is defined as “developed fully” by Oxford
Concise Dictionary. Therefore the purpose of a maturity model can be described as
identifying the capabilities and approaches of organization that will allow them to manage
their different processes like personnel competence, test, software development processes
more reliable and successful. There exists several maturity models proposed for different
issues related to software development process in software industry. For instance, there are
maturity models developed by concerning the software process quality like CMM (Capability
Maturity Model), CMMI (Capability Maturity Model Integration)6, SPICE
7 (Software Process
Improvement and Capability Determination), and so on. In software industry, there also exists
maturity models developed for assessing the maturity level of project management, test
processes, architecture, personnel competence, and so on. All these maturity frameworks have
common issues as described by Information Technology and Telecommunications SIG8 as
following;
They describe a number of discrete stages, or levels where each level represents a
progression of overall maturity of the organization. The CMM, and many other models,
are comprised of five levels, from an ad hoc, non-repeatable process (level 1) to an
exceedingly mature, robust and tightly structured capability (level 5).
They are comprised of a number of capability areas.
They explore a set of practices that together collectively define how the overall discipline
They tend to be organizationally focused.
While not processes or process frameworks, they are prescriptive of the processes that an
organization should adopt.
In summary, they are developed to assess the maturity level of organizations on specific
tasks.
5 http://martinfowler.com/
6 http://www.sei.cmu.edu/cmmi/start/faq/related-faq.cfm
7 http://www.isospice.com/categories/ISO{47}IEC-15504-Standard/
8 http://pmi-ittelecom.org/pmtopics/maturity-models-a-framework-for-organizational-improvement/
METHODOLOGY
20
3.METHODOLOGY
A systematic and planned research support finding and getting reliable answers. By keeping
the academic work structured and systematic, the readers will be able to understand the whole
work easier. Moreover, the reliability of the results will be achieved.
3.1 Data Sources
Data sources are the carriers of data (information) [16]. There exist two types of data sources
which are secondary data and primary data. Secondary data are information collected by
others for purposes that can be different from ours. Primary data are original data collected by
us for the research problem at hand [16].
In this thesis, secondary data collection is performed since the sources of primary data like
communication, experiments, and observations are not performed because of time limitations.
However, secondary data sources provided and supported essential and sufficient data to get
the reliable results.
MBSE is a new concept in software engineering. Therefore knowledge about its features,
characteristics, and how it is performed in the industry were required in order to understand
the whole concept. Moreover, since this thesis‟s focus is on UML diagrams usage in MBSE
and UML is known as its complexity because of the multi diagrammatic architecture, UML
by its variety of characteristics and usage fields that need to be examined properly. Therefore
these two knowledge requirement leaded to perform a research by the process of reviewing
literature from previous researches and books published. In order to get these publications,
reliable secondary data sources such as databases exist in Jönköping University Library are
scanned by some keywords. Therefore many publications like e-books and books, academic
articles, course notes, case studies, and experiment reports are scanned. Moreover since UML
is proposed by OMG, the OMG web-site was also another secondary data source that
examined frequently.
Any experiments or interviews are not handled during thesis but existing experiments and
case studies that are performed by others were interest of the this thesis in order to get
answers for the research questions defined in section 1.4.
All these findings are analyzed regarding on mostly research questions so that efficient and
sufficient results are achieved. For instance, the model quality goals and evaluation of them
supported to determine the best practices at different levels of abstraction.
METHODOLOGY
21
3.2 Data Collection
Data sources that are performed in this thesis were secondary data. Therefore qualitative
methods are performed. In qualitative research, the findings are not arrived at by statistical
methods or other procedures of quantification [16]. Even though many researchers believes
that quantitative research is more suitable or scientific; there are some says that methods
and/or techniques can be more scientific and better, it just depends on what the researchers are
looking for and how they will be able to get results for their research questions. Therefore as
long as everything is performed in a systematic way, the results will be sufficient and reliable.
3.3 Data Analysis
In order to analyze the collected data, content analysis method is applied during this process.
Klaus Krippendorff [40] defines content analysis as “an empirically grounded method,
exploratory in process, predictive or inferential in intent.” Also Robert P. Weber [41] states
that “content analysis is a research method that uses a set of procedures to make valid
inferences from text”. The most essential step of qualitative content analysis approach is to
make categorization. In [42] the author briefly indicates that categories are in the center of
analysis and “the aspects of text interpretation, following the research questions, are putted
into categories, which were carefully founded and revised within the process of analysis
[42]”.
In this thesis, the aim is to be able to define the most essential aspects regarding the
MBSE/UML principles within the organization. Therefore, measuring the occurrence of these
factors in the literature (conferences, workshops, experience reports, empirical studies, and
case studies) will provide to make determination of these aspects to be evaluated within the
proposed MBSE/UML Maturity Model.
In order to achieve this, the process shown is Figure 10 is followed.
METHODOLOGY
22
Figure 10 Followed Data Analysis Steps
3.4 Maturity Model Development Method
While developing a maturity model, seven guidelines of [51] is considered as basis in order to
be able develop a reasonable and usable maturity model. Design science addresses research
through the building and evaluation of artifacts designed to meet the identified business need
[51]. Design science aims at the improvement of problem-solving capabilities by creating
innovative artifacts, such as constructs, models, methods and instantiations [52]. Therefore,
developed maturity models require to be considered as innovative artifacts while assessing the
proficiency of organizations‟ model based development process. While adopting these
guidelines in this thesis, some changes are done since this maturity model development
process is supported by some research questions and literature review. The Table 4 below
indicates the description of these seven guidelines that are adapted from [51] and descriptions
of them in this thesis work.
METHODOLOGY
23
Figure 11 Maturity Model Development Steps
Results from the research questions and literature review conducted some of the steps of this
design-science research. As mentioned in previous section, after the determination of aspects,
organization levels, and indicators for model based development process. The maturity levels
are conducted with respect to the organizations‟ aim on model based development. This
process is shown in Figure 11.
Table 4 Design-Science Research Guideline (quoted from [51])
Guideline Description in General Description in This
Thesis
Guideline 1: Design as an
Artifact
Design-science research must
produce a viable artifact in the
form of a construct, a model, a
method, or an instantiation
A maturity model is
developed as a viable
artifact that provides an
assessment of
proficiency level of an
METHODOLOGY
24
[51]. organization on model
based development.
Guideline 2: Problem Relevance The solutions should be
provided for essential and
relevant business problems.
Essential aspects
regarding the model
based development are
retrieved from literature
review by underlying the
occurrence of the aspects
in literature.
Guideline 3: Design Evaluation The utility, quality, and
efficacy of a design artifact
must be rigorously
demonstrated via well-
executed evaluation methods
[51].
The results are required
to be carried out with
scientific methods.
Guideline 4: Research
Contributions
Relevant contributions require
to be performed in the design
areas.
Existing maturity models
relevant with model
based development
require to be examined.
Guideline 5: Research Rigor The selected methods require
to be adjusted rigorously.
The selected and used
methods require being
well-defined.
Guideline 6: Design as a Search
Process
Solutions that are proposed
should be developed,
evaluated, and improved
iteratively.
Maturity Model requires
to be developed step by
step.
Guideline 7: Communication of
Research
Design-science research must
be presented effectively both to
technology-oriented as well as
A usability analysis after
the development of
maturity model requires
METHODOLOGY
25
management-oriented
audiences [51].
to be performed.
3.5 Quality of Science
The method used in this thesis research is qualitative. Therefore, the reliability and validity of
data collected and analyzed are more difficult than the quantitative research. However,
performing a systematic review covers this problem and provides reliable and valid data. As
described in previous section, identifying what exactly are we looking for from the answers of
research questions makes the research question more structured. Therefore, the results
gathered are certain. Then the measuring the occurrence of these data in the literatures ensures
the reliability of the data collected since by applying content analysis, sort of quantitative data
are collected from the qualitative research and at that point, the influence of case studies and
experience reports support that data collected are based on some experiences and real-time
issues.
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
26
4.EVALUATION OF MBSE/UML
APPROACHES AND DEFINITION OF
ASPECTS
4.1 Quality of UML Diagrams in Modeling
Today‟s software systems are complex so modeling is considered to be performed more than
sketches and blueprints and at that point UML‟s multi diagram architecture is also causing a
complexity problems since some of the diagrams and semantic are not described properly.
Therefore it is both complexes to model a large scale system and to use UML in this modeling
process.
In this thesis, we are looking at both UML and modeling concurrently so that the lacks that
will be stated here will be based on using UML as a modeling language in modeling process.
This is also interest of many researchers. However, the variety use of modeling approaches
such as UML as blueprints, sketches, and programming language leads researchers to
examine the UML quality in different modeling approaches.
As it is also discussed and mentioned in articles [1], [17], [18], [5], [6], [4], [5] UML has lack
of semantics and syntax regarding the transformation of software development artifacts from
models. Therefore possible problems and defects arise in the models. For instance, the lack of
formal syntax causes not to be able automated verification of design specifications. There
exist several industrial experiences and researchers on semi-formal semantic and syntax
structure of UML and some provides solutions to decrease the problems that occur because of
these lacks.
It is not only semi-formal structure of UML leads models to have problems, because of multi
diagrammatic architecture of UML many researchers and practitioners state that this
architecture structure leads inconsistency between diagrams in model. Moreover the
interaction between diagrams also cannot be well described.
In article [1], Parastoo Mohagheghi et al. define 6 classes of model quality goals shown in
Table 5.
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
27
Table 5 6C Model Quality Goals
Model Quality Goal Description
Correctness Including right elements and correct relations between them, and
including correct statements about the domain [1]
Not violating rules and conventions; for example adhering to
language syntax, style rules, naming guidelines or other rules or
conventions [1].
Completeness Completeness is defined as having all the necessary information
that is relevant and being detailed enough according to the
purpose of modeling [1].
Consistency Consistency is defined as no contradictions in the model [1].
Comprehensibility Comprehensibility is defined as being understandable by the
intended users; either human users or tools [1].
Confinement Confinement is defined as being in agreement with the purpose of
modeling and the type of system; such as including relevant
diagrams and being at the right abstraction level. A model is a
description from which detail has been removed intentionally [1].
Changeability Changeability is defined as supporting changes or improvements
so that models can be changed or evolved rapidly and
continuously [1].
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
28
In addition to these, the quality of models also depends on aesthetics, the layout of diagrams,
the design, the tools used and so on.
In this thesis, our aim is to provide a maturity model for evaluating the usage of modeling
approaches so that our focus will be on these model quality goals too. These model quality
goals will provide aspects that need to be considered during the modeling process and then
how to achieve these goals or in which proficiency level an organization is performing these
will be examined.
4.2 Evaluation Criteria for MBSE/UML Maturity Model
In this section, the results for research question 2 “How to evaluate the quality and
proficiency in usage of UML and MBSE principles during software development process?” is
examined by literature review regarding UML new features, standards; and MBSE
approaches. The results gathered provide decision making while determining the important
aspects that require to be evaluated at different levels of organization.
Different aspects are considered at different levels of organization since various aspects not
only one have an impact on whole model based development process. Moreover while
evaluating the proficiency of organization on modeling, these aspects require to be considered
in order to support organizations to make improvements on their modeling practices.
In addition to these, since one of the aims of this thesis is to evaluate various aspects at
different levels of organization, in this section, brief information for levels of organization and
aspects required to be considered for each level provided. Moreover, after determining the
aspects and levels of organization; the indicators that will be evaluated and measured
separately are considered and provided in this section.
4.2.1 Essential Aspects for MBSE/UML Maturity Models
This evaluation criterion is important since MBSE approaches and UML are being used for
different purposes in the industry. Some organizations are only applying these to be able
generate code from the models whereas some are just using it to develop sketches. Therefore,
the aim of performing MBSE approaches in organization will differ and requirements for this
adoption will be different. Just to make this part clearer, a simple example can be provided.
For instance, an organization adopts modeling in order to generate code from the models
developed and the domain of the system is a defense system which requiring high security and
safety. In order to manage this, the models that require to provide some implementation
details and the tools that are used should be certified tools. On the other hand, let assume that
another organization is only aiming to use MBSE approaches as sketches so they should not
provide implementation details in their models; otherwise, their models will not be the ones to
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
29
cover the organization‟s purposes so too much unnecessary work and time consuming will
occur.
During the writing of this thesis, the literature examined provided the necessary and essential
aspects require to be estimated while adopting modeling approaches. According to [19], lack
of process development definitions, lack of quality assurance activities, and the design of
models cause a decrease on organization‟s improvements on models. In addition to these,
based on the case studies performed, they found that the characteristics of the designer
(modeler) like skills, motivation, and training has an impact on the quality of the models.
Moreover, time pressure is also examined as another effect on quality of models.
[43], [44], [45], [37], [4-3], and [19] also mention about importance of well-defined modeling
process. For instance, the authors of [44] states that “Processes must be generic enough to be
shared and capitalized at company or department level, but must be customizable enough to
be relevant at project level. Processes describe roles, activities and work products for each
involved stakeholder.” Moreover in [43] the authors also state that modeling process affects
the quality of MBSE so that while evaluating the proficiency of organization on MBSE, the
modeling process should be defined properly. The importance of utilizing a defined process in
software engineering has been known for several years. However, most “tried and tested”
processes are not tailored for MDE, which does not make any assumptions on the software
development process or the design methodology [20].
In [43], [35], and [19] the effects of modeling language, the knowledge and skills of modelers,
and the applied quality assurance techniques are also mentioned. For instance, based on [43]
the used modeling language should be applicable with domain and should be able to represent
the complexity of the system if it is complex. In article [35], team inexperience on modeling,
modeling language used, and modeling tools used stated as a challenge while adopting MBSE
within the organization.
In addition to these, the article [46] focuses on the importance of transformation and examines
the Motorola‟s experience on MDD. It indicates that the automatic generation of code from
developed models requires UML profiles or DSL (Domain Specific Language).
As mentioned several times in this thesis, UML‟s lack of semantic and syntax structure is
another issue to be estimated. For instance, as indicated in [19], [3], and [14] in order to
present and state the UML diagrams precisely a well-defined syntax and semantic is required.
By defining these, the consistency and comprehensibility will be achieved. However, it is
require to enforce the personnel who develops the models to follow these set of rules.
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
30
Table 6 Number of Occurrences of Aspects in Literature
Essential Aspects Occurrence in Literature
Articles, Journals
(n=31)
Case Studies,
Empirical Studies,
Experience Reports
(n= 21)
Modeling Process 13 11
Quality Assurance
Techniques
11 10
Personnel Competence 7 12
Tools 11 13
Design 13 9
Syntax and Semantic 16 12
Transformation 10 10
In total 52 articles are examined while determining the evaluation aspects for MBSE/UML
Maturity Model, the literature consists of two different parts. One is the articles that are
research papers developed systematically; the others are the empirical studies like surveys,
experience reports, etc. The Table 6 above shows the most occurred aspects regarding the
literature review where UML is considered as a main modeling language in MDD process.
The aspects stated in the Table 6 are same as the aspects that are determined as evaluation
criteria for MBSE/UML Maturity Model. However, it is important to state that during the
literature review if they are mentioned about the UML‟s lack of syntax and semantic, it is
related to aspect named syntax and semantic. Or when Meta-Modeling, DSL, and UML
Profiles are also categorized into syntax and semantic; and some are related to transformation
aspect. This categorization depends on how these concepts are mentioned and discussed. Then
these specific aspects like Meta-Models, DSL, and UML Profiles are utilized as indicators for
each aspect.
Therefore in this thesis, the focus will be on aspects which are modeling process, quality
assurance, personnel competence, tools, design, syntax and semantic, and transformation
shown in Table 7.
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
31
Table 7 Definition of Aspects
Aspects Definition
Modeling Process Modeling Process can be defined as a set of steps that need to be
followed during the development process considering the models
are the main artifacts. These steps can be like planning, requirement
specification, and model design like the ones in traditional software
development process but regarding the models.
Quality Assurance
Techniques
These are the techniques or methods that allow organization to
measure its development practices and evaluate if the organization
goals and model goals are gained or not.
Personnel Competence Here personnel competence is considered as motivation of
participants, the knowledge and experience of them, skills of them.
Moreover if it is necessary, training activities need to be included.
Tools Tools are used in modeling approaches and depending on the
modeling aim of organization different types of tools are used. For
instance, tools for graphical view, tools for automatic code
generation.
Design
In this thesis, the design aspect represents the layout, aesthetics of
UML diagrams and models. Moreover it also indicates the different
models require to be developed such as requirement model,
business model, domain model, and design model.
Syntax and Semantic In this thesis, syntax is considered as regulations, notations, and
rules used in order to describe a modeling language and semantic is
considered as meaning of the expressions stated within the
modeling language.
Transformation In this thesis, transformation means being able to perform
transformations from model to model, model to code, or model to
text.
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
32
4.2.2 Indicators for Each Aspect in MBSE/UML Maturity Model
After defining the aspects, it will be clearer to examine how to evaluate these aspects. Based
on the literature review performed, many researchers are discussing why these aspects are
important to be considered and they only focus on these aspects separately. However, in order
to measure and evaluate the proficiency of organization on modeling approaches all these
aspects need to be evaluated so that organizations will be able to examine how advanced or
un-advanced they are performing modeling approaches.
In this thesis, for each aspect some indicators are defined to be evaluated. These indicators are
the general titles and they consist of evaluation methods, some checklists, questionnaires, or
interviews and so on. These evaluation techniques will be discussed later. Now the focus is on
the main indicators that will be evaluated and will be evaluation criteria which are shown in
Table 10 below.
Modeling Process
Because of lack of standardized model based software development process definition,
organizations need to give importance to define a modeling process before they start adopting
modeling approaches. The traditional software development processes can be considered and
based on them; a new process for model based development can be defined regarding the
main artifact as models. The defined modeling process should be applicable to the field that
modeling will be performed. The standards, regulations need to be defined in order to avoid
from inconsistency and controversy that can occur later in the development process.
Moreover, the defined model based development process need to consider the modeling goals
so that it is required to define the modeling goals and document them.
In addition to these, it is important for organization to support and motivate staff to document
everything regarding the standards so that the changes and/or regulations can be tracked and
ordered properly.
Personnel in the model based development process regarding their knowledge, skills, and
experiments need to be evaluated since they are the ones who will develop the models and
final product. Moreover, personnel should agree on the defined modeling process and feel
confident with that. Therefore, it would provide an advantage if the personnel is also involved
in defining the model development process.
The modeling goals, the modeling process, and skills of personnel are considered but it is also
necessary to define quality assurance techniques to be able evaluate the results. This will be
examined with the quality assurance technique aspect. However, while defining the modeling
process and modeling goals, quality assurance techniques also need to be considered.
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
33
Model based development processes include many actors like stakeholders and personnel
from different departments of organization and with different roles. Stakeholders can be both
external and internal as shown in Figure 12.
Figure 12 A Sample Declaration of Internal and External Stakeholder
Each actor within the development process and organization is responsible of sharing
knowledge with others and each has to consider the organization standards, regulations, and
goals. For instance, each modeler can be responsible for different models but which are
supplementary for others too. Therefore, each modeler should know each other‟s work and
should provide consistency interaction and communication with each other. Let assume that a
modeler is responsible for providing a use case diagram to show how the general project will
work and this use case is based on the requirements of the external stakeholder who is client
in this scenario. Let also assume that this sample scenario simply indicates how an ATM
machine works as shown in Figure 13.
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
34
Figure 13 A Sample ATM UML Use Case Diagram
Regarding all these issues that need to be evaluated the indicators that will be used while
evaluating the proficiency level of modeling process aspect in adopting modeling approaches
can be listed like;
Defined Modeling Process,
Organization Workflow Schema,
Documentation,
Traceability,
Communication and Interaction Facilities,
Confidence of Personnel, and
Applicability to Domain.
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
35
Quality Assurance Techniques
If organizations intend to adopt modeling approaches in a proficient level, performing the
quality assurance techniques becomes a must since they are the ones which enable
organizations to evaluate themselves and their works.
The main issue is to evaluate if the organization goals and model goals are fulfilled or not so
that by referring the defined model based software development process and defined goals, a
proper documentation of them and a defined quality assurance plan are necessary. For
instance, a quality assurance plan should consist of goals definition, test plans, training
evaluations, verification and validation, and so on. For instance, quality assurance techniques
should evaluate and validate the 6C model goals [1] that mentioned previously in section 4.1.
In addition to these, quality standards need to be applied so it is necessary to enforce
participants involved in model development process to obey quality standards.
Therefore regarding these the indicators for this aspect are;
Defined Quality Assurance Plan,
Audits and Reviews,
Defined Quality Management Plan,
Defined Test Plan for Model Quality Goals, and
Domain Testability.
Personnel Competence
Personnel competence is another aspect that needs to be considered and evaluated since the
motivation and knowledge of participants have a reasonable effect on the whole development
process. The participants are the ones who develop the models and the final product so that
their affect cannot be underestimated. Therefore it has to be examined that how the staff have
a knowledge about modeling, the tools that will be used, and/or domain. For instance, it is
necessary to know if a modeler has experienced with the modeling tool that now will be used
in model development process or what kind of related certificates and/or participations to
related seminars and/or courses a modeler attended and had can provide knowledge about
training of him. Moreover the knowledge requirement does not only have to be about
modeling approaches, depending on the requirements for the model based development it is
possible to require knowledge on domain.
Regarding the aspects like quality assurance techniques and modeling process, the motivation,
documentation facilities, and communication facilities of personnel need to be considered.
The indicators for Quality Assurance Techniques Aspect are;
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
36
Defined Quality Assurance Plan
Defined Quality Management Plan
Audits and Reviews
Defined Test Plan for Model Quality Goals
Confidence and Compatibility of Personnel in Defined Plans
Domain Testability
Tools
There exist several tools for modeling activities which can be model editors, model
simulators, model executors, transformers, and so on. Regarding the modeling aim, the tool
selection is a critical decision making activity in MBSE. Therefore it is necessary for
organization to evaluate the existing tools including the ones that they already use within the
organization. For instance, if organization intends to transform code from the models then
tools like transformers, transformation editors are needed to be used. Moreover the policy of
organization is also another affect while using and making a decision on using some specific
tools. Therefore the indicators which are;
Policy for Use,
Integration Facility,
Simulation Ability of Use,
Applicability of Domain (certified or secure tools),
Tools Evaluation Facility, and
Repository and Tool Reuse need to be considered.
Design
Even though, most of the requirements for layout and aesthetic rules are required to be
declared by syntax and semantic there are still some other issues that affect the design of
UML diagrams. For instance, as mentioned in [22] and [19] the crossings and bends, size, and
colors are the ones to be estimated during the development of UML diagrams in order to
provide better and comprehensible view of diagrams. As shown in Figure 14, the crossings
and bends have a reasonable effect on comprehensibility of developed UML diagrams. The
most preferable one in this figure is (b) since the view of diagram is not complicated to
understand.
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
37
Figure 14 Effects of Crossing and Bends [22]
Moreover in order to improve the visualization of UML diagrams some perceptual concepts
are required to be performed which are defined in Table 8. These layout and aesthetic rules
are related to semantic structure of UML so that the results from the other aspect “Syntax and
Semantic” that will be presented later will be also indicators for design aspect.
Table 8 Definition of Visualization Principles
The Visualization Principles Definition
Law of Good Figure In this thesis, law of good figure refers to simplicity of UML
diagrams where only necessary information is represented.
Law of Similarity Similar elements (e.g., common shape or color) appear to be
grouped together [22]. In this thesis it refers to same
meaning too.
Law of Continuation As stated in [22] the belonging elements of UML diagrams
should be grouped together.
Law of Connectedness In this thesis, law of connectedness refers to that lines that
connect the belonging UML elements are required to be
connected to each other properly to increase the visualization
of them and support the group of elements.
Law of Familiarity In this thesis it also refers as stated in [22] where law of
familiarity is defined as elements are more likely to be
grouped together if the groups seem familiar or meaningful.
In addition to these, in this thesis design is not considered as layout and aesthetics of UML
diagrams but also as representing the different goals of models where the modeling goals
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
38
should be defined in development of modeling process and quality assurance techniques. In
this thesis, 4 different models will be taken into account which is requirement models, domain
models, design models, and business models where the definitions of them provided in Table
9.
Table 9 Definitions of Different Models in MDD
Different Models Definition
Requirement
Models
The requirement models are the models that are developed in order
to represent the requirements of clients. Therefore the main UML
diagrams that will be developed are use cases.
Business Models Business models represent the business requirements and
information about system platform and technology are not be
shown.
Design Models Design Models are developed in order to provide detailed
information about the system and product that will be developed.
Both functional and non-functional requirements are shown. For
instance, if organization aims to execute code from the models then
in design models these execution requirements and specifications
should be provided. Example UML diagrams are class diagrams,
sequence diagrams, activity diagrams.
Domain Models Domain Model is the model that defines how a business works
without reference to software systems, similarly to OMG‟s
Computation Independent Model [2].
The indicators that will be taken into consideration are;
Law of good figure [22]
Law of similarity [22]
Law of continuation [22]
Law of connectedness [22]
Law of familiarity [22]
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
39
Development of Semantic
Development of Requirement Model
Development of Business Model
Development of Domain Model
Development of Design Model
Skills and Experience
Moreover by indicating these different models, the aim is that each model is responsible to
show another goal and if design model is not related to requirement model then the model
quality cannot be achieved. Moreover if automatic code generation is required then design
model should satisfy all necessary requirements, system and technological information related
to that. That is why, these are considered as indicators.
Syntax and Semantic
Because of lack of definition and regulations on syntax and semantic structure of UML, this
aspect has a great impact on adopting modeling languages. Therefore organizations should
declare syntax and semantic regulations and rules clearly and properly. That will provide
consistency between each model that developed by different modelers. Therefore this
declaration should be understood by personnel and personnel need to be enforced to obey
these regulations. Moreover while organization is defining the syntax and semantic structure,
it is essential to consider the support of modeling language and modeling tool since it is
possible that a modeling tool cannot allow any other notation declared. Therefore the defined
syntax and semantic regulations should be applicable by modeling tools.
Like the other aspects, syntax and semantic evaluation also depend on some results from the
other aspects. For instance, the UML knowledge of personnel which take roles on
development process needed to be examined.
The indicators for this aspect are;
Consistency with Modeling Goals,
Syntax and Semantic Definition,
Modeling Language Support,
Personnel Knowledge and Skills, and
Applicability to Tools
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
40
Transformation
Regarding the definition of transformation, organizations can be willing to perform some
transformation based on their modeling and organization goals. With respect to the aim of
organization and/or modeling approaches, the developed models need to fulfill these
requirements. For instance, if the aim is to be able to generate code automatically in a specific
language then the specifications for this programming language should be indicated in the
models. Moreover, the knowledge and skills of personnel also have an impact on this aspect
too. For instance, in order to generate code automatically from the models UML profiles can
be required to be developed. Therefore the developed models need to fulfill the requirements
for the transformation activities.
The indicators for transformation aspect are;
Automatic Code Generation Requirements [20],
Executable Model [20],
Model Reusability, Changeability,
Generation of Domain Specific Artifacts[20],
Model Representation of Detailed System Specifications, and
Knowledge and Experience of Personnel on Transformation and Transformation
Tools,
The Table 10 below briefly demonstrates the indicators for each aspect.
Table 10 Indicators for Each Aspect
Aspects Indicators
Modeling
Process
Defined Modeling Process [43], [44], [45], [37], [20],
and [19],
Organization Workflow Schema,
Communication and Interaction Facilities [45],
Confidence of Personnel [44], and
Applicability to Domain [43].
Quality
Assurance
Defined Quality Assurance Plan [19], [35]
Audits and Reviews,
Defined Quality Management Plan[35],
Defined Test Plan for Model Quality Goals, and
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
41
Techniques Domain Testability.
Confidence and Compatibility of Personnel in Defined
Plans
Personnel
Competence
Training Certificates and/or Knowledge Level,
Project Skills Requirement Definition
Motivation,
Skills (Tools, Modeling, Domain) [43], [35]
Defined Training Plan,
Domain Knowledge and Skills[43],,
Domain Skills Requirement Definition.
Tools
Policy for Use,
Integration Facility [44], [35], [2]
Simulation Ability of Use [2],
Applicability of Domain (certified or secure tools),
Tools Evaluation Facility,
Repository and Tool Reuse.
Knowledge, experience of Personnel on Tools
Design
Law of good figure [22]
Law of similarity [22]
Law of continuation [22]
Law of proximity [22]
Law of connectedness [22]
Law of familiarity [22]
Development of Semantic
Development of Requirement Model
Development of Business Model
Development of Domain Model
Development of Design Model
Skills and Experience
Syntax and
Semantic
Modeling Language Support,
Formal Language to define syntax and semantic (ex:
meta model [3-24])
Syntax and Semantic Definition[19], [3], and [14],
Personnel Knowledge and Skills,
Applicability to Tools [43],
Consistency with Modeling Goals.
Transformation
Automatic Code Generation Requirements [20]
Executable Model [20]
Model Reusability, Changeability,
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
42
Generation of Domain Specific Artifacts[20],
Model Representation of Detailed System
Specifications.
Knowledge and Experience of Personnel on
Transformation and Transformation Tools,
4.2.3 Different Levels of Organization for MBSE/UML Maturity Model
As mentioned several time in this thesis, depending on the different levels of organization, the
requirements, the activities and practices, and the responsibilities require to be considered and
implemented change. For instance, organizations are responsible for managing and supporting
the modeling approaches as whole. The focus for them is fulfilling the organizational goals,
enforcing the obeying the organizational standards, and some other issues like these.
Regarding the knowledge on the existing maturity models and framework, CMM is developed
for at the organizational level where PCMM developed for at personnel level considering both
individual and team activities and practices. Based on them, the different levels of
organization are determined for this thesis and it is considered essential to examine the whole
activities and practices at different levels of organization which are;
Organizational Level,
Project Level,
Personnel Level, and
Domain Level.
The Table 11 below briefly indicates these levels of organization with a brief description.
Table 11 Levels of Organization
Levels of Organization Definition
Organizational
Management
Support
Organizational Management and Support activities and
practices performed at this level in order to provide and support
organizational change and improvement.
Project At the level of project, a specific project is considered. Based
on that project, the relative activities and practices need to be
performed for advance proficiency in modeling approaches are
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
43
considered.
Personnel
Engineering
Management
Modeler
Stakeholder
Team/ Group
Modeling activities where personnel are involved and perform
are basics for this level of organization. Their relationships,
training, skills, tasks, and so on are the issues that require to be
considered.
Domain
Business and
Financial
Health Care
Communication
Defense
IT
At the level of domain, different domains are considered since
regarding the each project that will be developed different
domains are point at issue. Moreover requirements, structure,
and specifications for each domain differ so that it affects the
whole model development process.
4.2.4 The General Matrix for MBSE/UML Maturity Model
There exist 7 different aspects need to be considered at 4 different levels of organization as
discussed previously and the Table 12 below demonstrates all the aspects that require to be
considered at different levels of organization regarding the indicators that require to be
evaluated.
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
44
Table 12 General Matrix for MBSE/UML Proficiency Framework
Aspects
Modeling Process Quality Assurance
Techniques
Personnel
Competence
Tools Design
Syntax and Semantic Transformation
Levels
Organizational
Management
Support
Defined Modeling
Process,
Organization
Workflow Schema,
Defined Quality
Assurance Plan,
Audits and
Reviews,
Defined Quality
Management Plan,
Defined Training
Plan
Policy for Use,
Tools
Evaluation
Facility,
Repository and
Tool Reuse
Development of
Business Model
Modeling Language
Support,
Syntax and Semantic
Definition,
Applicability to Tools
Transformation Goal
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
45
Project Defined Project/Model
Goals
Defined Test Plan
for Model Quality
Goals
Project Skills
Requirement
Definition
Integration
Facility,
Simulation
Ability of Use
Development of
Requirement
Model
Development of
Design Model
Law of good
figure [22]
Law of
similarity [22]
Law of
continuation
[22]
Law of
connectedness
[22]
Consistency with
Modeling Goals
Automatic Code
Generation
Requirements, [20]
Model
Representation of
Detailed System
Specifications,
Model Reusability,
Changeability
Executable Model
[20]
EVALUATION OF MBSE/UML APPROACHES AND DEFINITION OF
ASPECTS
46
Law of
familiarity [22]
Development of
Semantic
Personnel
Engineering
Management
Modeler
Stakeholder
Team/ Group
Communication and
Interaction Facilities,
Confidence of
Personnel
Confidence and
Compatibility of
Personnel in
Defined Plans
Training
Certificates and/or
Knowledge Level
Motivation,
Skills,
Domain
Knowledge and
Skills
Knowledge,
experience of
Personnel on
Tools
Skills and
Experience
Personnel Knowledge
and Skills
Knowledge and
Experience of
Personnel on
Transformation and
Transformation
Tools,
Domain
Business and
Financial
Health Care
Communication
Defense
IT
Applicability to
Domain
Domain Testability Domain Skills
Requirement
Definition
Applicability of
Domain
(certified or
secure tools)
Development of
Domain Model
Generation of
Domain Specific
Artifacts [20]
MBSE/UML Maturity Model
47
5.MBSE/UML Maturity Model
5.1 MBSE/UML Maturity Model Structure and Levels
The structure of the proposed MBSE/UML Maturity Model (shown in Figure 15) where [47]
and [48] are adapted while developing the maturity model, is built upon regarding different
indications listed as following;
The assessment of essential aspects during MDD process,
The assessment of indicators for each aspect at different levels of organization.
Figure 15 The Structure of MBSE/UML Maturity Model (adapted from [47] and [48])
As mentioned previously, the determination of different aspects provide a general evaluation
criteria to measure how mature organizations are adapting modeling approaches. Therefore
each aspect is considered while determining the maturity levels of MBSE/UML Maturity
Model. Regarding this, the proposed maturity levels will contain these aspects as general
requirements to be performed. However, they will not be the only issue to be considered. The
MBSE/UML Maturity Model
48
indicators for each aspect at different levels of organization will be the ones that the results
for maturity dimension are measured and evaluated. Basically, in addition to the aspects, the
indicators will assess the determination to which maturity level the organizations belong.
Briefly, the MBSE/UML Maturity Level of organization will be assessed regarding these
following issues;
If the aspects assigned for different maturity levels exist or not,
If the results from the indicators evaluations comply the appropriate outcomes
corresponding with the assigned maturity level.
5.2 MBSE/UML Maturity Level Dimension
While determining the maturity levels of MBSE/UML Maturity Model [2], [48], and the
results from literature review which mostly the issues discussed by MDD experience reports,
surveys, and case studies are adapted. It is quite necessary to underline that only while
building up the maturity level of MBSE/UML Maturity Model, existing maturity models for
traditional software development process and MDD process are considered in order to provide
a familiar model for organizations. As mentioned in purpose section in this thesis, different
than the existing maturity models, here the discussion is on MDD process not traditional
software development process; moreover, several aspects at different levels of organization
are the main consideration of this MBSE/UML Maturity Model where the most existing
models are only considering one or few aspects at one organizational level.
After clearing the difference of this proposed maturity model structure, now the maturity level
dimension can be discussed properly.
In this thesis, six MBSE/UML Maturity Model Levels are defined as shown in Figure 16.
MBSE/UML Maturity Model
49
Figure 16 MBSE/UML Maturity Model Levels
5.2.1 Maturity Level 0: No Modeling
No modeling is applied within the organization. Organization is still performing traditional
software development. None of the indicators are applied within the organization with respect
to MBSE/UML approaches.
5.2.2 Maturity Level 1: Basic Modeling
MBSE/UML approaches are not applied within the organization. However, individuals are
developing models for their own help or in order to facilitate communication and interaction
between different stakeholders. In this maturity level, models are developed as sketches so
that MDD‟s main principle which is developing models as main artifacts in software
MBSE/UML Maturity Model
50
development process is not achieved. Therefore organization is still performing traditional
software development process.
5.2.3 Maturity Level 2: Initial MBSE/UML
MDD principles are applied within the organization. However, this is performed at project
level not at organizational level. A modeling process is defined including modeling goals
definition, some quality assurance techniques description but application of them is still not
mature enough. Organization considers their personnel knowledge and skills on MBSE/UML.
However, there are no defined training activities to support personnel‟s level of knowledge on
MBSE/UML. Moreover by development of design models, organization is capable of
implementing the code from the models. However, the code implementation is performed
manually.
The indicators for different aspects at different level of organization in MBSE/UML Maturity
Model Level 2 are shown in Table 13.
Table 13 Aspects and Indicators Assessed in MBSE/UML Maturity Level 2
Aspects Indicators
Modeling Process
Defined Modeling Process
Defined Project/Model Goals
Communication and Interaction Facilities
Quality Assurance Techniques
Defined Quality Assurance Plan
Defined Quality Management Plan
Personnel Competence
Project Skills Requirement Definition
Motivation
Skills
Tools
Tools Evaluation Facility
Repository and Tool Reuse
Design
Development of Requirements Model
Development of Design Models
Law of good figure
Law of similarity
Law of continuation
Law of connectedness
Law of familiarity
MBSE/UML Maturity Model
51
Syntax and Semantic
Consistency with Modeling Goals
Transformation
Model Representation of Detailed System
Specifications
5.2.4 Maturity Level 3: Defined MBSE/UML
Organizations are more familiar with MBSE/UML principles and adapting modeling
approaches at organizational level as well as at project level. In this level, organizations
develop business models where business requirements are represented (see 4.2.2 for
description of business models). After the development of business and design models,
organization is capable of generating code from the models automatically. However, not all
code generation can be achieved automatically. In order to provide a common understanding
and interpreting of models, meta-modeling is performed. Moreover UML Profiles are
developed or organizations use the proposed UML Profiles by OMG in order to enable the
auto-code generation. In addition to these, organizations are considering the personnel
competence in MBSE/UML also.
The indicators that assess the maturity of organization on MBSE/UML Maturity Model Level
3 are shown in Table 14 below. However, the indicators stated previously in maturity level 2
are also required to be assessed.
Table 14 Aspects and Indicators Assessed in MBSE/UML Maturity Level 3
Aspects Indicators
Modeling Process
(All the indicators stated in Maturity Level
2)
Quality Assurance Techniques
Audits and Reviews
Defined Test Plan for Model Quality Goals
Confidence and Compatibility of
Personnel in Defined Plans
Personnel Competence
Defined Training Plan
Training Certificates and/or Knowledge
Level
MBSE/UML Maturity Model
52
Tools
Knowledge and Experience of Personnel
on Tools
Policy for Use
Design
Development of Business Models
Skills and Experience
Syntax and Semantic
Syntax and Semantic Definition
Modeling Language Support
Personnel Knowledge and Skills
Applicability to Tools
Transformation
Transformation Goal
Automatic Code Generation Requirements
Knowledge and Experience of Personnel
on Transformation and Transformation
Tools
5.2.5 Maturity Level 4: Integrated MBSE/UML
Organizations have well-defined meta-models in order to perform generation of models, code,
and/or text from developed models automatically. Domain models are also developed to
represent the specific requirements and concepts of particular domain. The consistency
between different models is achieved. Transformation of development artifacts with respect to
MBSE/UML principles is performed. DSL are utilized and defined.
The indicators that assess the maturity of organization on MBSE/UML Maturity Model Level
4 are shown in Table 15 below. However, the indicators stated previously in maturity level 3
are also required to be assessed.
Table 15 Aspects and Indicators Assessed in MBSE/UML Maturity Level 4
Aspects Indicators
Modeling Process
Organization Workflow Schema
Applicability to Domain
Quality Assurance Techniques
Domain Testability
MBSE/UML Maturity Model
53
Personnel Competence
Domain Knowledge and Skills
Domain Skills Requirement Definition
Tools
Applicability to Domain (certified or secure
tools)
Integration Facility
Simulation Ability of Use
Design
Development of Domain Models
Syntax and Semantic
(All the indicators stated in Maturity Level
3)
Transformation
Generation of Domain Specific Artifacts
5.2.6 Maturity Level 5: Optimized MBSE/UML
Briefly, in addition to the tasks, activities performed on MBSE/UML Maturity Model Level 4,
the developed models are the models that can be executed and reused. Therefore there is a
strict differentiation between business, domain, and design models. To provide this
distinction, some of these models are developed regarding the particular domain and project
whereas some are developed to represent the general structure of the system so that they can
be reused later. In this maturity level, the MBSE/UML aim of organization is completely to be
able to execute the models for final code. Therefore, the main artifact in the software
development process is developing the models as solutions.
In this maturity level, in addition to the indicators stated previously, for transformation aspect
organizations are capable of performing models as reusable and executable as listed in Table
16 below.
Table 16 Aspects and Indicators Assessed in MBSE/UML Maturity Level 5
Aspects Indicators
Transformation
Model Reusability, Changeability
Executable Models
Assessment of Maturity Levels of MBSE/UML Maturity Model
54
6.Assessment of Maturity Levels of
MBSE/UML Maturity Model
In previous chapter, the maturity levels of MBSE/UML Maturity Model are described briefly.
As mentioned there, the indicators are the assessment keys while stating the maturity level of
organization. After organizations start MBSE/UML development, the indicators stated for
each level requires to be achieved in different scales which mean an indicator for one level
does not have to be performed in advanced way. However for maturity levels 4 and 5 all
indicators are required to be resulted advanced.
The instruments are conducted in order to evaluate the indicators that require to be performed
within the organization regarding MDD process. Different questionnaires are designed as
instruments.
The relations of instruments with MBSE/UML Maturity Model‟s aspects and participants who
are intended to answer the questionnaires are provided in Table 17 and the questionnaires are
stated in Appendix part of this thesis.
Table 17 Instruments’ Focus of Aspect and Participants
Aspects Instruments Intended Participants
Modeling Process MBSE/UML Modeling Process
Questionnaire (see Appendix 1)
Project Manager and
Quality Manager
Quality Assurance Techniques MBSE/UML Quality Assurance
Technique Questionnaire (see
Appendix 2)
Quality Manager
Personnel Competence MBSE/UML Personnel
Competence Questionnaire 1
(see Appendix 3)
Developers (designer,
software engineer,
modeler, etc.)
Assessment of Maturity Levels of MBSE/UML Maturity Model
55
MBSE/UML Personnel
Competence Questionnaire 2
(see Appendix 4)
Project Manager and
Quality Manager
Tools MBSE/UML Tool
Questionnaire (see Appendix 5)
Developers, Project
Manager, and Quality
Manager
Design MBSE/UML Design
Questionnaire (see Appendix 6)
Developers, Quality
Manager
Syntax and Semantic MBSE/UML Syntax and
Semantic Questionnaire (see
Appendix 7)
Developer, Quality
Manager
Transformation MBSE/UML Transformation
Questionnaire (see Appendix 8)
Developer
As mentioned previously, 6 maturity levels are proposed and each maturity level requires
being performed different aspects with different indicators at different levels of organization.
Therefore, for each aspect, different questionnaires are conducted to assess if the indicators
are performed or not.
In total 85 questions are designed to be asked. The Table 18 below indicates the total of
questions designed for different indicators. Moreover, it should be noted that some of the
questions that are asked in different aspects are also related to the other aspects. For instance,
while evaluating if the personnel skills are qualified for syntax and semantic aspect, this also
results if the organization is aware of their personnel knowledge and skills which is supposed
to be performed in personnel competence aspect. Therefore for some indicators and aspects,
the results can be also gathered from different questions.
Some of the questions that are designed to be asked are;
Assessment of Maturity Levels of MBSE/UML Maturity Model
56
Do you assure that the granularity of your developed models is sufficient enough to
represent the details of a particular system?
Do you use DSL (Domain Specific Languages) within your MDD process?
Do you separate the domain specific concepts in your domain model from your
business and design model?
Table 18 Number of Questions Designed to be asked for Each Indicators within Different
Aspects
Aspects Indicators Total
Number of
Questions
Asked
Modeling Process
Defined Modeling Process,
5
Organization Workflow Schema,
1
Defined Project/Model Goals,
7
Communication and Interaction
Facilities,
Confidence of Personnel,
8
Applicability to Domain,
2
Documentation,
6
Traceability
2
Quality Assurance
Defined Quality Assurance Plan,
11
Assessment of Maturity Levels of MBSE/UML Maturity Model
57
Techniques Audits and Reviews,
4
Defined Quality Management
Plan 6
Defined Test Plan
5
Confidence and Compatibility of
Personnel in Defined Plans 1
Domain Testability
2
Personnel
Competence
Defined Training Plan,
10
Project Skills Requirement
Definition 1
Training Certificates and/or
Knowledge Level , 1
Motivation,
4
Skills,
6
Domain Knowledge and Skills
3
Domain Knowledge
Requirements Definition 2
Tools
Policy for Use,
5
Assessment of Maturity Levels of MBSE/UML Maturity Model
58
Tools Evaluation Facility,
3
Repository and Tool Reuse,
2
Integration Facility,
2
Simulation Ability of Use,
1
Knowledge and Experience of
Personnel on Tools, 5
Applicability to Domain (Ex.
certified or secure tools) 3
Design
Development of Business Model,
4
Development of Requirement
Model, 5
Development of Design Model,
5
Law of Good Figure,
2
Law of Similarity,
2
Law of Continuation,
2
Law of Connectedness,
2
Assessment of Maturity Levels of MBSE/UML Maturity Model
59
Law of Familiarity,
2
Development Model for
Semantic, 1
Development of Domain Model
5
Meta-Model
5
OCL
1
Syntax and
Semantic
Modeling Language Support,
4
Syntax and Semantic Definition,
5
Applicability to Tools,
2
Consistency with Modeling
Goals, 2
Personnel Knowledge and Skills
3
Transformation
Transformation Goal,
7
Automatic Code Generation
Requirements, 3
Model Representation of Detailed
System Specifications, 2
Assessment of Maturity Levels of MBSE/UML Maturity Model
60
Model Reusability, Changeability,
7
Executable Models,
5
Generation of Domain Specific
Artifacts 2
Most of the answers for questions are assigned with numerical values from 0 to 4. A sample is
shown in Table 19 below.
Table 19 Sample of Assigned Numerical Values for Answer Types
Answer Type 1 Answer Type 2 Assigned
Value
Definitely Yes Complete Support 4
Usually Partly Support 3
Planned but not applied Moderate Support 2
Not Sure Poorly Support 1
Definitely No Not at all 0
Assessment of Maturity Levels of MBSE/UML Maturity Model
61
Moreover, some multiple choice questions are designed in order to examine which of the
practices the organization is performing; then regarding this questions with answers shown
above are asked to measure how mature they are performing those practices. In addition to
these, the answers for some particular questions consist of scales and these scales are also
assigned to numerical values from 0 to 4 to support easy to calculate as shown in Table 20
below.
Table 20 Sample of Assigned Values for Questions
Answer Type Assigned
Value
Between 0% and 20% 4
Between 20% and 40% 3
Between 40% and 60% 2
Between 60% and 80% 1
Between 80% and 100% 0
The evaluation of these answers will be performed regarding the indicators that require to be
performed with respect to MBSE/UML Maturity Levels which are defined in previous
section. For instance, assuming that the evaluation is being performed to assess whether the
organization is in Maturity Level 4 or not where organization is required to separate the
domain concepts in domain model than business and design models. Therefore the answer for
“Do you separate the domain specific concepts in your domain model from your business and
design model?” should have numerical value of 4.
In addition to these, it is important to underline that organizations do not have to be on
Maturity Level 4 or 5 all the time because if their modeling aim is not being able to execute
code from the models or generate model, code, and/or text from the models automatically,
being Level 3 will be adequate for them. However, their advance in modeling still depends on
Assessment of Maturity Levels of MBSE/UML Maturity Model
62
the requirements of Level 3 so that they are required to be performed the modeling practices
with respect to that Level.
In addition to measurable questionnaire questions, structured questions are developed in order
to identify some terminologies within a particular field. For instance, to gather information
about what exactly the organization‟s aim on modeling is. By having information about this,
the questionnaire results can be assessed regarding this issue. An example for a structured
question is “Which of the artifacts like code, model, and/or text do you generate from your
developed models?” If the answer is “they do not generate any of them” then the
questionnaire for transformation aspect is not necessary to be evaluated since the organization
do not purpose for this and this leads that they are performing modeling not in MBSE/UML
Maturity Model 4 and 5. Therefore these types of questions will support the focus of
evaluation which means that for which level the focus should be. The unstructured questions
are most likely to collect some qualitative data on examining how they are performing
modeling within the organization. For instance, a sample question for this is “What do you do
if your personnel do not document their work?” The answer for this is open-ended. Therefore
it will just provide to elicit as much information as possible on documentation maintenance.
When the results from all types of questions within the questionnaire are combined with each
other; more reliable data will be gathered for assessing the organizations‟ Maturity Level on
MBSE/UML.
VALIDTY OF MBSE/UML MATURITY MODEL
63
7.VALIDTY OF MBSE/UML
MATURITY MODEL
A usability analysis is considered for validating the maturity model provided in the thesis. The
goal of a usability analysis is to pinpoint areas of confusion and ambiguity for the user which,
when improved will increase the efficiency and quality of a users‟ experience [49]. Since the
aim of MBSE/UML Maturity Model provide assessment criteria for organizations who are
adopting MBSE/UML principles within their organizations. Therefore the proposed Maturity
Model requires being easy to apply and being satisfied.
To validate these features, as defined in previous chapter, different questionnaires are
conducted for different aspects and these aspects‟ importance is validated by literature review.
For instance, when the occurrence of the aspects are conducted, it is gained that modeling
process is considered more important by researchers than the practitioners but still both state
that modeling process require to be well-defined. Therefore assessing these aspects in
MBSE/UML Maturity Model is providing reliability and validity. The questionnaires
designed will be required to be filled out by the intended personnel within a software
company who adopts MBSE/UML principles within the organization. The results will be
analyzed and evaluated regarding the criteria defined in previous chapters so that the
MBSE/UML Maturity Level of organization will be achieved. During this process, it can be
expected that the feedback can come up regarding the different aims or policies of
organization. However, the results are predicted as sufficient enough to capture many
different issues that come up.
CONCLUSION
64
8.CONCLUSION
The increasing adoption of MBSE/UML principles in software industry require some
standards and proficiency evaluations since many organizations are applying these new
principles and related approaches within the organization but with a lack of knowledge how
advanced or complete they are performing. The existing standards and definitions consider the
traditional software development process. Therefore applying them for MDD process is not
sufficient and effective since in MDD process models are developed as solutions and code is
aimed to be executed from the developed models automatically.
In this thesis, a MBSE/UML Maturity Model is proposed in order to provide organizations to
evaluate themselves with respect to their MBSE/UML approaches. Rather than many maturity
models presented in the software industry, this Maturity Model provide organizations to
assess themselves in different aspects regarding modeling like modeling process, personnel
competence, modeling tools, UML diagrams and models design, modeling quality assurance
techniques, syntax and semantic of UML, and transformation which are determined with
respect to the results from literature review. Therefore organizations will be able to assess
themselves in many essential aspects of MDD adaption process.
While developing this Maturity Model, a systematic literature review is performed and
essential aspects discussed by the researchers and practitioners are examined. In order to
retrieve the essential features of the models, answers for the research question 1 is analyzed so
that it is achieved that quality assurance activities related to models are one of the key roles
while executing the code from the models. The results for this research question is contributed
the indicators for design and syntax and semantic aspects of the MBSE/UML Maturity Model.
Second research question is also provided identification of indicators while evaluating the
proficiency of utilization of UML and modeling approaches. For instance, if organization is
defining their own meta-models this lead us that they are using UML in an advanced way.
Moreover during the writing process of this thesis, MDD Maturity Model [2] is found so that
later in the thesis, this proposed maturity model inspired the process of determining the
dimension of the maturity levels of MBSE/UML Maturity Model.
The most important factor that require to be considered and performed while adapting MDD
is defining a sufficient MDD process which is required to be performed by the organization.
Then the aspects like quality assurance techniques, training, motivation, and skills of
personnel, design of UML diagrams and models, syntax and semantic structure of UML,
modeling tools, and transformation tasks all related to modeling as process that require to be
considered. Some organizations are aware of these aspects. However, it is not known how
competence they are performing related tasks and activities. At that point, this proposed
MBSE/UML Maturity Model will provide organizations to assess themselves in different
aspects with respect to modeling as a process so that it will be easier for the organizations to
CONCLUSION
65
improve themselves in modeling process, modeling quality assurance techniques, modeling
tools, personnel competence based on modeling, transformation facilities of organization,
design issues, and syntax and semantic.
REFERENCES
66
REFERENCES
[1] Parastoo Mohagheghi , Vegard Dehlen, and Tor Neple: Definitions and approaches to
model quality in model-based software development– A review of literature, Elsevier B.V.,
April 2009
[2] Erkuden Rios, Teodora Bozheva, Aitor Bediaga, and Nathalie Guilloreau, MDD Maturity
Model: A Roadmap for Introducing Model-Driven Development, A. Rensink and J. Warmer,
2006
[3] Andy Evans, Robert France, Kevin Lano, and Bernhard Rumpe, The UML as a Formal
Modeling Notation , Computer Standards and Interfaces, Volume 19, Number 7, 1997
[4] Andy Evans¸ Robert France, Kevin Lano, and Bernhard Rumpe, Meta Modeling
Semantics of UML, Proceedings of UML'99, 1999
[5] Christian F.J. Lange, Improving the Quality of UML Models, in Practice International
Conference on Software Engineering Proceedings of the 28th international conference on
Software engineering, available on ACM, 2006
[6] David Harel and Bernhard Rumpe, Modeling Languages Syntax, Semantics and All That
Stuff, Technical report, Jerusalem, Israel, 2000
[7] Christian F.J. Lange and Michel R.V. Chaudron, Effects of Defects in UML Models –An
Experimental Investigation, ICSE‟06 available on ACM, 2006
[8] Helen C. Purchase, Jo-Anne Allder, and David Carrington, Graph Layout Aesthetics in
UML Diagrams: User Preferences, Journal of Graph Algorithms and Applications, 2001
[9] Parastoo Mohagheghi and Vegard Dehlen, Developing a Quality Framework for Model-
Driven Engineering, Springer-Verlag Berlin Heidelberg, 2008
[10] J. Rumbaugh, I. Jacobson, and G. Booch, The Unified Modeling Language Reference
Manual, Addison-Wesley, 1999.
[11] Russell Miles and Kim Hamilton, Learning UML 2.0., O‟Reilly Media, 2006
[12] Christof Lutteroth, AP1: A Platform for Model-Based Software Engineering, Springer-
Verlag Berlin Heidelberg, 2007
REFERENCES
67
[13] Alar Raabe, Feature Matching In Model-Based Software Engineering, Springer, 2006
[14] B. Hailpern and P. Tarr, Model-driven development: The good, the bad, and the ugly,
IBM Systems Journal, 2006
[15] Stefan Winkler and Jens von Pilgrim, A survey of traceability in requirements
engineering and model-driven development, Springer-Verlag, 2009
[16] Pervez Ghauri and Kjell, Gronhaug, Research Methods in Business Studies 3rd
Edition,
Prentice Hall, 2005
[17] Alain April, Jane Huffman Hayes, Alain Abran, and Reiner Dumke, Software
Maintenance Maturity Model (SMmm): The software maintenance process model, John Wiley
& Sons, Ltd, 2004
[18] Christian F.J. Lange, Bart DuBois, Michel R.V. Chaudron, and Serge Demeyer, An
Experimental Investigation of UML Modeling Conventions, Springer-Verlag Berlin
Heidelberg, 2006
[19] Christian F.J. Lange and Michel R.V. Chaudron, Defects in Industrial UML Models – A
Multiple Case Study –, the ACM/IEEE 10th International Conference on Model Driven
Engineering Languages and Systems, 2007
[20] Parastoo Mohagheghi and Vegard Dehlen, Where Is the Proof? - A Review of
Experiences from Applying MDE in Industry, Springer-Verlag Berlin Heidelberg, 2008
[21] Xabier Larrucea, Ana Belen García Díez, and Jason Xabier Mansell, Practical Model
Driven Development process, European Workshop on Model Driven Engineering, European
Software Institute, 2004
[22] Christian F.J. Lange and Michel R.V. Chaudron, Managing Model Quality in UML-
based Software Development, Pre-Proceedings IEEE International Workshop on Software
Technology and Engineering Practice, STEP 2005, 2005
[23] Helen C. Purchase, Matthew McGilI, Linda Colpoys, and David Carrington, Graph
drawing aesthetics and the comprehension of UML class diagrams: an empirical study,
Australian Computer Society, Inc., Proceedings of the 2001 Asia-Pacific symposium on
Information visualization - Volume 9, 2001
[24] John Krogstie, Evaluating UML Using a Generic Quality Framework, Idea Group Inc.,
2003
REFERENCES
68
[25] Norman E. Fenton and Shari Lawrence Pfleeger. Software Metrics, A Rigorous and
Practical Approach, Thomson Computer Press, second edition, 1996
[26] Stephen H. Kan, Metrics and Models in Software Quality Engineering, Addison Wesley
Professional, 2nd edition, September 2002
[27] S. R. Chidamber and C. F. Kemerer. A metrics suite for objectoriented design. IEEE
Transactions on Software Engineering, 20(6):476–493, 1994
[28] Lange, C.: Improving the Quality of UML Models in Practice. In: Proceedings of 28th
International Conference on Software Engineering (ICSE 2006), pp. 993–996, ACM, New
York, 2006
[29] Kim, H. and Boldyreff, C.: Developing Software Metrics Applicable to UMLModels, In
Proceedings of the 6th ECOOP Workshop on Quantitative Approaches in Object- Oriented
Engineering, 2002
[30] Baroni, A., Braz, S., Abreu, and F.B.: Using OCL to Formalize Object-Oriented Design
Metrics Definitions, In Proceedings of the 6th ECOOP Workshop on Quantitative Approaches
in Object-Oriented Software Engineering, 2002
[31] McQuillan, J. and Power, J.: A Metamodel for the Measurement of Object-Oriented
Systems: An Analysis using Alloy, In Proceedings of the 2008 international Conference on
Software Testing, Verification (ICST), pp. 228–297. IEEE Computer Society Press,
Washington, 2008
[32] Tony Bloomfield, MDA, Meta-Modelling and Model Transformation: Introducing New
Technology into the Defence Industry, in: ECMDA-FA 2005, pp. 9 18, Springer, 2005
[33] Evans A., France R., Lano K., and Rumpe B., The UML as a Formal Modeling Notation,
UML'98 Proceedings, Springer-Verlag, 1998
[34] Evans A., France R., Lano K., and Rumpe B., Meta-modelling Semantics of UML,
Behavioural Specifications for Businesses and Systems, Chapter 4, 1999
[35] Baker P., Loh S., and Weil F., Model-Driven Engineering in a Large Industrial Context
— Motorola Case Study, MoDELS 2005. LNCS, vol. 3713, pp. 476–491, Springer-Verlag,
2005
[36] Mohagheghi P., Fernandez M.A., Martell J.A., Fritzsche M., and Gilani W.,MDE
Adoption in Industry: Challenges and Success Criteria, M.R.V. Chaudron (Ed.): MODELS
2008 Workshops, LNCS 5421, pp. 54–59, Springer-Verlag, 2009
REFERENCES
69
[37] Miroslaw Staron, Adopting Model Driven Software Development in Industry – A Case
Study at Two Companies, O. Nierstrasz et al. (Eds.): MoDELS 2006, LNCS 4199, pp. 57 –
72, Springer-Verlag Berlin Heidelberg, 2006
[38] Scott W. Ambler, The Elements of UML Style, Cambridge University Press, 2003
[39] Lidia Fuentes-Fernández and Antonio Vallecillo-Moreno, An introduction to UML
Profiles, The European Journal for the Informatics Professional, Vol. 5, No. 2. , April 2004
[40] Klaus Krippendorff, Content Analysis- An Introduction to its Methodology, Sage
Publications Inc., 2004
[41] Robert Philip Weber, Basic Content Analysis 2nd
Edition, Sage Publications Inc., 1990
[42] Philipp Mayring, Qualitative Content Analysis, Qualitative Social Research [On-line
Journal], avaliable online CiteSeer, 2000
[43] Parastoo Mohagheghi and Jan Aagedal, Evaluating Quality in Model-Driven
Engineering, International Workshop on Modeling in Software Engineering (MISE'07), 2007
[44] Eric Jouenne and Véronique Normand, Tailoring IEEE 1471 for MDE Support, N.
Jardim Nunes et al. (Eds.), UML 2004 Satellite Activities, LNCS 3297, pp. 163-174, Springer
Verlag Berlin Heidelberg, 2005
[45] Mohagheghi P., Fernandez M. A., Martell J.A., Fritzsche M., and Gilani W., MDE
Adoption in Industry: Challenges and Success Criteria, M.R.V. Chaudron (Ed.): MODELS
2008 Workshops, LNCS 5421, pp. 54–59, Springer-Verlag Berlin Heidelberg, 2009
[46] William E. McUmber and Betty H.C. Cheng, A General Framework for Formalizing
UML with Formal Languages, in Proceedings of the 23rd International Conference on
Software Engineering, ICSE 2001, 2001
[47] Beecham, S. and Hall, T., Expert panel questionnaire: validating a requirements process
improvement model, avalaible online
http://homepages.feis.herts.ac.uk/~pppgroup/requirements_cmm.htm, last visit May 2010,
2003
[48] SEI, Capability Maturity Model Integration (CMMISM), Version 1.1. SEI, CMU/SEI-
2002-TR-029, 2002
[49] International Standards Organization, ISO DIS 9241-11: Guidance on Usability, 1994
REFERENCES
70
[50] OMG, Introduction To OMG's Unified Modeling Language (UML), available:
www.omg.org/gettingstarted/what_is_uml.htm, last visit May 2010, 2005
[51] Hevner AR, March ST, Park J, and Ram S, Design science in information systems
research, MIS Quarterly 28(1):75–105, 2004
[52] March ST and Smith G, Design and natural science research on information technology,
Decision Support Systems 15(4):251–266, 1995
APPENDIX
71
APPENDIX
Appendix 1: MBSE/UML Modeling Process Evaluation Questionnaire
This survey is conducted in order to measure the indicators that are essential in Modeling
Process of organization. These indicators are defined in MBSE/UML Maturity Model.
1. Does your organization have a defined MDD Process? If no please leave this survey, if
yes please continue with question 2 in this survey.
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
2. Is MDD Process of your organization documented properly?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
APPENDIX
72
3. Does MDD Process document include description of organization workflow for MDD
projects?
Definitely Yes
Partly
Poorly
Not sure
Definitely No
4. Which of the following does MDD Process document include?
Configuration Management Plan
Training Plan
Project Plan
Project Management Plan
Risk Management
Test Plan
Quality Management Plan
Requirements Specification
Software Description Document
Modeling Goals Document
APPENDIX
73
Modeling Standards Document
Other:
5. Does MDD Process Document provide information about documentation maintenance
regarding facilities like model updates and changes?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
6. Are the tasks, roles, and activities assigned personal defined in MDD Process?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
7. Does your organization consider and state modeling and tools standards?
Definitely Yes
Usually
APPENDIX
74
Planned but not applied
Not sure
Definitely No
8. Are methods and/or techniques regarding measurement of modeling quality the tasks
defined?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
9. Do you estimate the size, effort, and cost for software projects with respect to
adapting modeling as a development process?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
10. Do you enforce your personal to document their work properly?
APPENDIX
75
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
11. Are the commitments and project risks with respect to modeling being traced
according to MDD project plan?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
12. Are you able to trace whether your defined modeling requirements are achieved by
models or not?
Definitely Yes
Usually
Planned but not applied
Not sure
APPENDIX
76
Definitely No
13. Do you define the methods and/or techniques to perform measurement and analysis
of your modeling tasks?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
14. Do you manage the communication and/or interaction facilities between all
participants with respect to the project by means of modeling?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
16. Do you consider the issues and concepts about modeling domain while defining the
MDD Process?
Definitely Yes
Usually
APPENDIX
77
Planned but not applied
Not sure
Definitely No
17. Do you think that your MDD Process is applicable with the project domain?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
18. Which of the following do you define with respect to MDD process??
Definitely
Yes Usually
Planned but
no applied Not sure
Definitely
No
Modeling Goals
regarding particular
project
Modeling Goals
regarding particular
Domain
APPENDIX
78
Definitely
Yes Usually
Planned but
no applied Not sure
Definitely
No
Modeling Goals
regarding
organization goals
Appendix 2: MBSE/UML Quality Assurance Techniques Evaluation
Questionnaire
1. Does your organization apply any quality assurance techniques for modeling process?
If the answer is no, please leave the survey, otherwise please continue to answer survey
questions by question 2.
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
2. Does Quality Assurance Techniques that your organization applies include tasks for
MDD process? If the answer is no, please leave the survey, otherwise please continue to
answer survey questions by question 3.
Definitely Yes
Usually
Planned but not applied
Not sure
APPENDIX
79
Definitely No
3. Does your organization document quality assurance techniques defined for MDD? If
your answer is definitely no, please leave the survey; otherwise please continue to answer the
survey questions by question 4.
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
4. Which of the following does your organization define in Quality Assurance Plan
Document?
Definitely
Yes Partly Moderate Poorly
Definitely
No
Model Quality
Standards
Identification
Modeling Purpose
Identification
Evaluation of
Software (Modeling)
Tools
APPENDIX
80
Definitely
Yes Partly Moderate Poorly
Definitely
No
Evaluation of
Modeling
Requirements
Analysis Process
Evaluation of Model
Design Process
Evaluation of Test
Models
Evaluation of Model
Acceptance Testing
Process
Evaluation of Model
Configuration
Management Process
Perform Model
Configuration Audits
Evaluation of
APPENDIX
81
Definitely
Yes Partly Moderate Poorly
Definitely
No
Training Facilities
Evaluation of
Domain Testability
5. Which of the following documents are expected to be produced during MDD process?
Definitely
Yes Usually
Planned but
not applied
mostly
Not sure Definitely
No
Modeling Needs
Statement
Project Plan
Model Configuration
Management Plan
Quality Assurance
Plan
APPENDIX
82
Definitely
Yes Usually
Planned but
not applied
mostly
Not sure Definitely
No
Feasibility Study
Risk Analysis
System and
Modeling Tools
Support and
Acquisition Plan
System Security and
Privacy Plan
Functional and Non-
Functional
Requirements
Document
Granularity of
Models Document •
(Ex. In design phase,
what is the Level of
abstraction in design
phase)
APPENDIX
83
Definitely
Yes Usually
Planned but
not applied
mostly
Not sure Definitely
No
Domain
Requirements
Document
Semantic Description
Internal Audit Plan
System/Subsystem
and Modeling Tools
Specifications
Validation,
Verification, and
Testing Plan for
Models
Training Plan
Model
Transformation Plan
APPENDIX
84
Definitely
Yes Usually
Planned but
not applied
mostly
Not sure Definitely
No
Modeling Tools,
Transformation
Tools, Modeling
Language
Requirements,
Programing
Language
Specification,
Simulation and
Integration
Identification
Model Checking
Maintenance Manual
6. Which of the following reviews does your organization apply?
Definitely
Yes Usually
Planned but
not applied
mostly
Not sure Definitely
No
Model Requirements
Review
APPENDIX
85
Definitely
Yes Usually
Planned but
not applied
mostly
Not sure Definitely
No
Design Review
Modeling Review
Preliminary Model
Design Review
Model Testing
Review
Model
Transformation
Readiness Review
Model Acceptance
Test Review
Appendix 3: MBSE/UML Personnel Competence Evaluation Questionnaire
vol. 1
This questionnaire is conducted to assess the maturity of the personnel within the organization
regarding the MBSE/UML. Moreover if any activities are performed for improving the
personnel competence like any training programs, this questionnaire also can measure the
APPENDIX
86
competence of these activities held by organization. This is required to be filled out by
developers, modelers, designers, and software engineers.
1. How would you rank your previous knowledge on MBSE approaches? *
1 2 3 4 5
nothing
advanced
2. How would you rank your previous knowledge on UML? *
1 2 3 4 5
nothing
advanced
3. Which of the following do you have? *
Any MBSE related certificates
Academic courses on MBSE
Attendance to conferences, workshops about MBSE
None
Other:
4. How would you rank your current knowledge on MBSE approaches? *
APPENDIX
87
1 2 3 4 5
learned nothing
learned in depth
5. How would you rank your current knowledge on UML? *
1 2 3 4 5
learned nothing
learned in depth
6. How useful were the materials provided to you during this project? *
1 2 3 4 5
not useful
very useful
7. How useful were the materials for your career? *
1 2 3 4 5
not useful
very useful
8. Regarding your experience on MBSE/UML approaches within your organization,
how would you rank the learning curve? *
APPENDIX
88
1 2 3 4 5
insufficient
very sufficient
9. For a particular MBSE/UML project, how would you rank your knowledge on
relevant domain? *
1 2 3 4 5
nothing
very well
10. During a particular MBSE/UML project or MBSE/UML adapting to your
organization, how can you rank your knowledge on the tools that are used? *
1 2 3 4 5
never used
experienced
11. During a particular MBSE/UML project or MBSE/UML adapting to your
organization, were you confident with the tools used? *
1 2 3
not confident
very confident
APPENDIX
89
12. During a particular MBSE/UML project, were the materials are sufficient for a
particular domain? *
1 2 3
not sufficient
very sufficient
13. Which of the following are required to be considered regarding the future training
on MBSE/UML approaches?*
Meta Modeling
UML Profile
OCL
DSL
Round-Trip Engineering Facilities
Other:
14. How useful would it be to learn more about MBSE/UML approaches? *
1 2 3 4 5
not useful
very useful
15. How would you rank the courses given to you by the organization regarding the
MBSE/UML approaches? *
APPENDIX
90
1 2 3 4 5
not given
very sufficient
16. Which of the following UML modes have you worked previously? *
UML as sketches
UML as blueprint
UML as implementation language
None
Other:
17. Which of the following UML modes, did you become familiar currently? *
UML as sketches
UML as blueprint UML as implementation language
None
Other:
18. How did the courses and/or materials provided by your organization improve your
knowledge and/or skills on UML? *
1 2 3
APPENDIX
91
insufficient
sufficient
19. How would you rank your UML knowledge and/or skills currently? *
1 2 3 4 5
nothing
advanced
20. What specific tasks are required to be considered during learning curve regarding
UML? *
Meta Modeling
UML Profiles
OCL
Other:
21. Which of the following MBSE tasks you have knowledge and/or skills? *
Model to Model Transformation
Model to Text Transformation
Model to Code Transformation
Round-Trip Engineering Facilities
Other:
APPENDIX
92
22. How would you rank your familiarity with the modeling process of your
organization *
1 2 3 4 5
not known
well known
23. How would you rank your knowledge on modeling and organization goals of your
organization? *
1 2 3 4 5
do not know
well known
Appendix 4: MBSE/UML Personnel Competence Evaluation Questionnaire
vol. 2
This questionnaire is required to be filled out by the ones who are responsible for personnel
competence activities in addition to the project and quality managers. The aim of this survey
is to evaluate how the personnel competence activities are performed within the organization
regarding MBSE/UML.
1. Do you define a training plan regarding MBSE/UML knowledge requirements of your
personnel?
Definitely Yes
Usually
Planned but not applied
Not sure
APPENDIX
93
Definitely No
2. Are you aware of your personnel's knowledge, skills, and experiences on
MBSE/UML?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
3. Do you analyze the specific MBSE/UML requirements for particular projects?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
4. Do you analyze the requirements for particular domain?
Definitely Yes
Usually
Planned but not applied
Not sure
APPENDIX
94
Definitely No
5. Regarding the requirements of domain and/or project, do you define any training
plans to support knowledge of your personnel?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
Appendix 5: MBSE/UML Tools Evaluation Questionnaire
1. Are your personnel aware of your organization's policy of tool use?
Definitely Yes
Usually
Not sure
Definitely No
2. How does the modeling tools (e.g. model editors, model executors, transformation
editors, and so on) support integration with other tools that support MDD process like
Microsoft Visual Studio, NetBeans, and so on?
1 2 3 4 5
Not Supported
Totally Supported
APPENDIX
95
3. Which of the following modeling tool types is your organization using?
Model Editors
Model Executors
Model Simulators
Model Repositories
Transformers
Transformation Repositories
Other:
4. Are you able to simulate the models developed?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
Other:
5. Are you able to execute code from your developed models by using transformation
tools?
Definitely Yes
Usually
APPENDIX
96
Planned but not applied
Not sure
Definitely No
Other:
6. How would you rank the following issues of your UML modeling tool used for MDD
process?
Not
Supported
Slightly
Supported
Moderately
Supported
Very
Supported
Repository Support
Round-Trip Engineering
HTML Document
Full UML 2.0 Support
(or other versions that is
used within
organization)
Data Model Integration
APPENDIX
97
Not
Supported
Slightly
Supported
Moderately
Supported
Very
Supported
Versioning
Model Navigation
Printing Support
Diagram Views
Exporting Diagrams
Scripting
Robustness
Platform Applicability
Domain Applicability
APPENDIX
98
Not
Supported
Slightly
Supported
Moderately
Supported
Very
Supported
Auto-Generation
XMI Support
7. Which of the following issues are essential for UML modeling tool that will be used
during MDD process?
Not
Essential
Seldomly
Essential
Essential
once in a
while
Essential
most of the
time
Very
Essential
Repository Support
Round-Trip
Engineering
HTML Document
Full UML 2.0 (or
other versions)
Support
APPENDIX
99
Not
Essential
Seldomly
Essential
Essential
once in a
while
Essential
most of the
time
Very
Essential
Data Model
Integration
Versioning
Model Navigation
Printing Support
Diagram Views
Exporting Diagrams
Scripting
Robustness
APPENDIX
100
Not
Essential
Seldomly
Essential
Essential
once in a
while
Essential
most of the
time
Very
Essential
Platform
Applicability
Domain
Applicability
Auto-Generation
XMI Support
Appendix 6: MBSE/UML Design Evaluation Questionnaire
This survey is conducted by regarding the well-formedness rules and modeling conventions
proposed by OMG.
Model Development
This section includes questions to analyze what kind of models are developed by
organization.
1. Do you develop a model or models to represent the following requirements?
Definitely
Yes Usually
Planned but
not applied Not sure
Definitely
No
APPENDIX
101
Definitely
Yes Usually
Planned but
not applied Not sure
Definitely
No
User Requirements
System
Requirements
Business
Requirements
Domain
Requirements
Syntax and Semantic
Requirements
2. Which of the following model are you developing? The requirement models are the
models that are developed in order to represent the requirements of clients. Business models
represent the business requirements and information about system platform and technology
are not be shown. Design Models are developed in order to provide detailed information about
the system and product that will be developed. Both functional and non-functional
requirements are shown. Domain Model is the model that defines how a business works
without reference to software systems, similarly to OMG‟s Computation Independent Model.
Definitely
Yes Usually
Planned but
not applied Not sure
Definitely
No
APPENDIX
102
Definitely
Yes Usually
Planned but
not applied Not sure
Definitely
No
Requirements Model
Design Model
Business Model
Domain Model
3. Do you separate the concepts of business, design, and domain models? If you are not
developing at least two of these models, you can skip this question and continue with
following questions.
Definitely
Yes Usually
Planned but
not applied Not sure
Definitely
No
Requirements Model
Design Model
Business Model
APPENDIX
103
Definitely
Yes Usually
Planned but
not applied Not sure
Definitely
No
Domain Model
UML Diagrams Design Issues
4. Do you apply any metrics to your UML Diagrams to measure the quality of them? ex.
metrics for measuring the complexity of UML Diagrams.
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
This part is conducted for those who apply metrics to their UML Models.
5. Which of the following issues of UML Diagrams' layout do you measure?
Definitely
Yes Usually
Planned but
not applied Not sure
Definitely
No
Complexity
APPENDIX
104
Definitely
Yes Usually
Planned but
not applied Not sure
Definitely
No
Completeness
Modifiability
Familiarity
Similarity
Continuity of points
and lines
Connectedness of
UML Elements
6. Do you think metrics that are defined measure the actual attributes that are intended
in metric definition?
Definitely Yes
Usually
Planned but not applied
APPENDIX
105
Not sure
Definitely No
7. Do you calculate the metrics automatically by means of tool?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
8. With your defined metrics, are you able to measure the model quality goals defined?
(ex. consistency, correctness, comprehensibility, etc. of models)
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
Meta Model Design Issues
This part of survey is conducted for those who perform meta-modeling within their
organization. The questions are derived from the OMG propose for UML well-formedness
rules and modeling conventions.
APPENDIX
106
9. OMG proposes that initial embedded capitals required to be used while naming the
Meta Classes. Do you apply this to your Meta Classes? Please indicate in others, if you do
not have meta models.
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
Other:
10. Are names of meta associations written in the same manner as meta classes (e.g.,
‘ElementReference’).
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
11. Do you define a specific notation for the followings?
Definitely
Yes Usually
Planned but
not applied Not sure
Definitely
No
APPENDIX
107
Definitely
Yes Usually
Planned but
not applied Not sure
Definitely
No
For names that
consist of appended
nouns/adjectives
For Boolean meta
attribute names
For Enumeration
types
12. While referring to meta classes, meta associations, meta attributes, etc. in the text,
the exact names as they appear in the model are always used. Proposed by OMG
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
13. If a mandatory section does not apply for a meta class, the text ‘No additional XXX’
is used, where ‘XXX’ is the name of the heading. If an optional section is not applicable,
it is not included. Proposed by OMG
Definitely Yes
APPENDIX
108
Usually
Planned but not applied
Not sure
Definitely No
OCL Specification
14. Do you define any metrics to measure the structural complexity of OCL expressions?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
We do not have OCL Expressions
Appendix 7: MBSE/UML Syntax and Semantic Evaluation Questionnaire
1. Which of the following is your aim on applying modeling approaches?
No Modeling
Modeling as Sketches
Modeling as Blueprints
Modeling as Transformation
Modeling as Execution
2. How would you rank the support of defined syntax and semantic expressions by your
modeling language (UML) regarding your modeling aim?
APPENDIX
109
Complete Support
Partly Support
Moderate Support
Poorly Support
Not at all
3. Do you require defining additional syntax and semantic declaration for your modeling
process?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
4. How do you consider that UML's support on allowing developers to define additional
syntax and semantic declarations?
Complete Support
Partly Support
Moderate Support
Poorly Support
Not at all
5. Do you think that your defined syntax and semantic regulations are complying to
your modeling goals? If your answer is "Definitely No" for question 3, then you can skip this
question and continue to survey by following questions.
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
6. Are you confident that the modeling tools that you are using are applicable with UML
and UML's additional features like UML Profile, Meta Models?
APPENDIX
110
Definitely Yes
Usually
Moderately
Not sure
Definitely No
7. Is UML's syntax and semantic, profiles, meta-models are adequate to represent the
domain specific regulations?
Definitely Yes
Usually
Moderately
Not sure
Definitely No
8. Which of the following issues do you apply and/or useregarding your MDD process?
Meta-Modeling
UML Profiles
Syntax and Semantic Definition by using formal languages like Z
Modeling Conventions
Well-Formedness Rules
None
Other:
Appendix 8: MBSE/UML Transformation Evaluation Questionnaire
General Statement
General Requirement Analysis for Model Transformation
1. Which of the following artifacts do you generate from your developed models?If your
answer is none, please leave this survey.
Code
Model
APPENDIX
111
Text
None
Other:
2. Do you generate models, text, and/or code from models manually? If your answer is
'Definitely No' and 'Not Sure', please skip the section 'Automatic Generation from Models'
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
Automatic Generation from Models
3. Are you able generating artifacts like code, model, and text from models
automatically?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
Other:
4. Do you assure that the granularity of your developed models is sufficient enough to
represent the details of a particular system?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
5. Which of the following are you using while generating artifacts from your models
automatically?
APPENDIX
112
UML Profile
Meta Models
UML Profile with OCL description
Domain Specific Languages (DSL)
Developing own code generators
Other:
6. What is the percentage of code that you generate from your developed models
automatically?
7. Do you use DSL within your MDD process?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
Other:
Domain Concepts
8. How would you rank the effect of domain for a particular project?
1 2 3 4 5
Not Related
Very Related
9. Do you develop domain models for particular project?
Definitely Yes
Usually
APPENDIX
113
Planned but not applied
Not sure
Definitely No
10. Do you separate the domain specific concepts in domain model from your business
and design models?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
Model Reusability, Changeability, and Executable Model
11. Are you able to do changes on your meta-model and update the models developed
based on your meta-model representations?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
12. Are you able to update all related models when an update is required in one specific
model?
Definitely Yes
Usually
Planned but not applied
Not sure
Definitely No
13. What is the percentage range of your developed models that you can reuse for other
projects?
APPENDIX
114
14. How would you rank your models' reusability, changeability, and executable?
Not at All Slightly
Sufficient
Moderately
Sufficient
Very
Sufficient
Reusability
Changeability
Executable
Top Related