Sept. 9, 2009
Seminar at Beijing University
1
Deriving and Analysing the Quality of Software
Architectural Designs
Hong ZhuDepartment of Computing, Oxford Brookes University,
Oxford OX33 1HX, UKEmail: [email protected]
Sep
t. 9,
200
9
2Seminar at Beijing University
OutlineOverview of existing work and open
problemsOur approach
Graphic model of software qualityDerivation of software quality model from designAnalysis of quality models
ConclusionComparison with related worksFuture work
Sep
t. 9,
200
9
3Seminar at Beijing University
The Notion of Quality Quality, in particular software quality, is an elusive
concept and varying from people to people. Garvin’s definitions of quality
Transcendent view: Quality is universally recognisable. It is related to a comparison of features and characteristics of products.
Product-based view: Quality is a precise and measurable variable. Differences in quality reflect the differences in quantities of some product attributes.
User-based view: Quality is the fitness of intended uses.Manufacturing-based view: Quality is conformation to
the specifications.Value-based view: A quality product is one that
provides performance at an acceptable price or conformance at an acceptable cost.
Sep
t. 9,
200
9
4Seminar at Beijing University
Software Quality Models Models about software quality in terms of
The factors that affect software quality• Attributes directly indicate the quality of he system,
e.g. reliability, correctness, user friendliness, etc. • Attributes indirectly related to quality, e.g. internal
complexity, etc. The interrelations between the factors
• Causality models: Causal relationship, in terms of stereo-type relations
• Quantitative models: Quantitative relations expressed as numerical functions, Numerical values of the basic attributes are given, e.g.
through using metrics/measurementsThe overall quality in an attribute on a more high level of
abstraction is calculated numerically
Sep
t. 9,
200
9
5Seminar at Beijing University
Hierarchical Models of SQA set of quality related properties organised
into a hierarchical structureE.g. decompose quality into a number of quality
attributes and attributes into a number of factors, and then a number of measures, etc.
Positive causal relationship is representedExamples:
McCall model (1977)Boehm model (1978)ISO model (1992)Bansiya-Davis model of OO designs (2002), etc.
Sep
t. 9,
200
9
6Seminar at Beijing University
McCall’s Model
Product operations
Product transition
Product
revision
Maintainability: Can I fix it? Flexibility:Can I change it?Testability:Can I test it?
Portability: Will I be able to use it on another machine?
Reusability:Will I be able to reuse some of the software?
Interoperatability:Will I be able to interface it with another machine / software?
Correctness: Does it do what I want? Reliability: Does it do it accurately all the time?Efficiency: Will it run on my machine as well as it can?Integrity: Is it secure?Usability: Can I run it?
Sep
t. 9,
200
9
7Seminar at Beijing University
ISO 9126 Model
Software quality
ReliabilityMaturity
Fault toleranceRecoverability
Portability
AdaptabilityInstallability ConformanceReplaceability
Maintainability
AnalysabilityChangeability
StabilityTestability
EfficiencyTime behaviour
Resource behaviour
UsabilityUnderstandability
Learnability Operability
Functionality
SuitabilityAccuracy
InteroperabilityCompliance
Security
Sep
t. 9,
200
9
8Seminar at Beijing University
Relational Models of SQ A quality model consists of:
A number of quality attributes and factors, etc. A set of stereo types of relationships between them,
normally include • Positive relation:
High on one aspect of quality implies inevitably high on the other
• Negative relation: High on one aspect of quality implies inevitably low on the other
• Neutral relation: High or low on one aspect of quality does not imply on the other aspect the quality is high or low.
Examples:Perry model (1991)Gillies model (1992, 1997)
Sep
t. 9,
200
9
9Seminar at Beijing University
Perry’s ModelCorrectness
Reliability
Efficiency
Integrity
Usability
Maintainability
Testability
Flexibility
Portability
Reusability
Interoperability
CR
E
I
U
M
T
F
P
R
I
Direct
Inverse
Neutral
Sep
t. 9,
200
9
10Seminar at Beijing University
Roles in Software DevelopmentUnderstanding software quality
To measure an existing system’s quality, To predict a system’s quality during
developmentTo analyse quality problems during development
and/or in operation of a systems
Guidelines to development activitiesTo elicit users’ requirements on software quality To perform quality assurance activities in
software development and maintenance
Sep
t. 9,
200
9
11Seminar at Beijing University
Key Features of Existing SQ Models
Feature Pros Cons
Universal Applicable to all software systems
Not address system specific issues
Simplicity Stereo types of relationships between quality attributes
Incapable of dealing with complicated relationships between quality attributes
Black-box Treat software systems as black-boxes
Provide little help to the designers to consider system’s structure
Empirical Developed-based on empirical knowledge from experiences
Cannot be validated or verified of its correctness
Sep
t. 9,
200
9
12Seminar at Beijing University
Our Approach Representing software quality models in a diagrammatic
notation (Zhang, Zhu, et al. 2002) to improve expressiveness of quality models to represent
complicated relationships between factors to associate quality features with the structure of the system
Deriving software quality models from architectural designs (Zhang, et al. 2002, Zhu, 2005) to systematically derive quality models from software architectural
designs by employing extended hazard analysis concepts and techniques
Automating the analysis of quality models to help software designers (Zhang, et al, 2006a, 2006b) to perform various analysis tasks based on graphic quality models
by developing and using automated software tools
Sep
t. 9,
200
9
13Seminar at Beijing University
Graphic Notation for Modelling SQ
Subject[+|-] PropertyPhenomenon
Node: representing quality carrying properties and quality attributes and observable phenomena
Link without annotation: representing logic relations between the nodes.
Link with Annotation: providing additional information of the rationale of the relation between nodes.
[+ | - ]
Annotation of Reasons
[+ | - ]
Interface is not displayed properly
Client side- Compatibility
Font size too small to be illegible.
System- Usability
Difficult to operate
Node A Node BLink
Sep
t. 9,
200
9
14Seminar at Beijing University
Example: Web-based ApplicationsHTML files
+ Well StructuredLarge size
HTML files+ Navigability
Small number of hyperlinks
System+ UsabilityEasy to find required info
Web Server- Responsiveness
Long response time
HTML files- Correctness
Contains broken links
Online Help- AvailabilityNot available
Client side- Compatibility
Font size too small to be illegible when displayed
user’s platform
System- Usability
Cannot find required info
Server side- Performance
Execution speed is slow
Server side+ Load
Highly demanded
Files are considered as
unavailable when time-out
Simpler hyperlink network usually easier
to navigate
Less nodes means less links
Need long time to transmit the files
Web page cannot be displayed
properly
Unable to obtain files through hyperlinks
Unable to get help when experiencing difficulty
Sep
t. 9,
200
9
15Seminar at Beijing University
Deriving Quality Models: The HASARD MethodsHASARD: Hazard Analysis of Software ARchitectural DesignInput: A software architectural model Output: A quality model in graphic notationProcess:
Hazard identificationA hazard identification method is applied to identify all quality sensitive observable phenomena of the components, connectors, the system, etc.
Cause-consequence analysisThe causal relationships between the identified hazards are recognized
Model assemblingThe information obtained in the previous steps are assembled together and represented in the graphical notation
Quality concern recognitionThe quality carrying properties/quality attributes that a phenomenon demonstrates are recognized according to the nature of the phenomenon
Sep
t. 9,
200
9
16Seminar at Beijing University
Hazard IdentificationExtended Hazard and Operability Studies
(HAZOP)The method relies on determining answers to
questions of what-if nature. By applying a set of guide words systematically
to develop a collection of what-if questions. They are applied to the attributes of various
components and connectors of the system being studied.
If a deviation from the normal working of the component is credible, the behaviour of the component is considered as a possible hazard.
Sep
t. 9,
200
9
17Seminar at Beijing University
Guide Words for Software Hazard Identification
Guide word
Applicable attribute
Interpretations
No Date/control signals
No data / control signal exchanged; No data / control signals produced/received.
Component property/ function
The component / connector does not have the designed property / function.
Component / connector
The system does not contain the component / connector.
More Quantitative parameters
The value of the parameter is too large.
Less Quantitative parameters
The value of the parameter is too small.
Sep
t. 9,
200
9
18Seminar at Beijing University
As well as
Event or activity Redundant data sent, or event/activity occurred in addition to the intended ones.
Component / connector
Other components / connectors are added in addition to the intended ones.
Part of Structured data Only a part of the data produced, stored or received.
Structure events Only a part of the events happened.
Reverse Direction of flow The information flow in the opposite direction.
Event The opposite event happened.
Other than
Data/control signals, quantitative/qualitative parameters
Incorrect data / control signals produced;The parameter has a value different from the design.
Component/system’s function or property
The component has a functionality / property different from the designed.
Component/ connector Other kind of component / connector is contained.
Sep
t. 9,
200
9
19Seminar at Beijing University
Early Periodical events The event happened earlier than expected.
Late Periodical events The event happened later than expected.
Before Temporal orders The event happened in the order earlier than designed.
After Temporal orders The event happened in the order later than designed.
Sep
t. 9,
200
9
20Seminar at Beijing University
Example: Internet ConnectionGuide word
Hazard/Failure mode Causes Consequences
No The internet connection passes no messages between the client and server.
Physically disconnected; Traffic jam; Software failure; Network server down.
Client cannot communicate with the server.
More More messages are delivered than the clients and the server send out, e.g. duplicated messages.
Hacker’s attack; Heavy traffic on the Internet caused resending packages.
System clash; Overload on the server or the client.
Less Fewer messages are delivered than the server and the clients send out, i.e. lost messages.
Discontinued Internet connection; Heavy traffic on the Internet; Software failure.
Incomplete transactions; System clash; Damage the integrity of the data and program on the server and/or client.
Sep
t. 9,
200
9
21Seminar at Beijing University
As well as
Messages are delivered to other destinations in addition to the designated receiver.
Hacker’s attack; Software failure.
Leak of sensitive information.
Part of Only a part of the packets of a message is delivered to the destination client or server.
Discontinued Internet connection; Heavy traffic on the Internet; Software failure.
Software failure; Production of incorrect computation results if incompleteness is not detected.
Other than
A message not from the client (or the server) is passed to the server (or client).
Hacker’s attack; Other software system’s failure
System failure; Damage the integrity of the data and the program.
Other than
The message is in a different format.
The client or the server is modified; Fault in the software.
System failure.
Sep
t. 9,
200
9
22Seminar at Beijing University
Cause-Consequence Analysis It aims at understanding the causal relationships
between hazards. It can be performed backward or forward, or a
combination of both. Forward analysis
• Search for the potential effects, i.e. consequences, of a hazard until the consequence is terminal.
• A hazard or failure mode is terminal: if it does not affect any other component of the system or If It does not cause any other hazards/failures. if we are not interested in its further consequences.
Backward analysis • Search for the causes of a hazard until the hazard is
primitive. • A hazard or failure mode is primitive:
if its causes cannot be further identified without additional knowledge about the system
if we are not interested in its causes
Sep
t. 9,
200
9
23Seminar at Beijing University
Results of Cause-Consequence Analysis
Sep
t. 9,
200
9
24Seminar at Beijing University
Constructing Graphic Quality Model Input:
The records of the cause-consequence analysis Output:
A partially completed graphical representation of quality model
The transformation:Each record of the hazard or failure mode becomes a
node with the component and phenomenon as specified in the record.
Each row in the record becomes a link from the node that represents the cause to the node that represents the consequence.
The explanation column of the row forms the reason of the link.
Sep
t. 9,
200
9
25Seminar at Beijing University
Example:
Sep
t. 9,
200
9
26Seminar at Beijing University
Identification of Quality Concerns
For each node in the diagram Identify the quality attribute or quality carrying property
that the phenomenon demonstrates• Comparing the observable phenomenon with the
definitions of a set of quality attributes and quality-carrying properties
• Recognising a new attribute or property, if no existing one matches.
Fill the identified quality attribute into the slot of the node Examples,
‘a hyperlink is broken’ demonstrates the quality attribute correctness of the HTML file.
‘Server is down’ is related to the availability of the server.
Sep
t. 9,
200
9
27Seminar at Beijing University
Analysis of Graphic Quality ModelsContribution factors
What are the factors that affect a specific quality issue that the user (designer) is interested in?
HTML filesWell Structured
Large size
Web Server- Responsiveness
Long response time
Server side- Performance
Execution speed is slow
Server side- Load balance
Some servers have high demand, but some not
Need long time to transmit the
files
Example: Factors of server’s responsiveness
Sep
t. 9,
200
9
28Seminar at Beijing University
Impacts of design decisions What are the consequences
of a design decision?
Simpler hyperlink network usually
easier to navigate
HTML files+ Navigability
Small number of hyperlinks
Web Server- Responsiveness
Long response time
System- Usability
Cannot find required information
System+ Usability
Easy to find required information
Less nodes means less links
Need long time to transmit the files
Files are considered as unavailable when
time-out
HTML files+ Well Structured
Large size
Example: The impacts of HTML files are of large sizes
Sep
t. 9,
200
9
29Seminar at Beijing University
Quality risks When a negative quality property is present, where does the risk come
from? And what are the implications of the problem?
Example: (partial model)The risk of long response time in a web-based application: its causes and implications
HTML files+ Well Structured
Large size
Web Server- Responsiveness
Long response time
System- Usability
Cannot find required info
Server side- Performance
Execution speed is slow
Server side- Load balance
Some servers have high demand, but some not
Files are considered as
unavailable when time-out
Need long time to transmit the files
Sep
t. 9,
200
9
30Seminar at Beijing University
Configure is easy
Overhead due to using middleware
System
Scalability
Adding more resources is easy
System
Performance
Response time is short
System
Flexibility
Use J2EE technology
-
More processing
power
Relationships between quality issues How two quality issues are interrelated?
Example:Relationships between flexibility and performance
Sep
t. 9,
200
9
31Seminar at Beijing University
Trade-off points:When a design decision has positive impact on one quality attribute but negative impact on another quality attribute, a trade-off must be made to balance between the two. Where are the points in a complicated design that need to make trade-offs? Or simply, where are the trade-off points?
It is hard to reuse
Reduce the interaction between EJB components
system- Economy
More development time
EJB componetstructureness
Large size
System performance
Shorter response time
Example:
EJB components’ size is a trade-off point in systems implemented using EJB.
Sep
t. 9,
200
9
32Seminar at Beijing University
Automation of Quality Analysis SQUARE: Software QUality and ARchitecture modeling Environment.
Tools to support software architectural modelling Tools to support the derivation of software quality models from architectural
design Graph algorithms are designed and implemented to automate the analysis
of software quality;
Hazard Analysis
Tools
Quality Analysis
Tools
Quality Model Editor
Architecture Model Editor
SA Model
Quality Model
Quality Analysis Results
Model Repository
Project Manager
Overall structure of SQUARE toolkits
Sep
t. 9,
200
9
33Seminar at Beijing University
Screen Snapshot of SQUARE toolkits
Sep
t. 9,
200
9
34Seminar at Beijing University
Case Study The subject:
A real e-commerce system of online trading of medicine. The system is operated by the local medicine trading
regulation authority who supplies medicines to all state-owned hospitals in the province.
Its main functions:• customer relationship management, • product catalogue management, • online trade management, • online auction of medicine supplies, • online information advertisement, • a search engine for medicine information, and so on.
The system is implemented in J2EEThe system has been in operation for more than one
year at the time of case study
Sep
t. 9,
200
9
35Seminar at Beijing University
Process of The Case StudyStep 1: Construct the architectural modelIt is a reverse engineering process:
Review of the design documents• The system’s design document consists of four main parts:
(a) a simple and schematic J2EE architecture showing that the system uses J2EE as implementation technology; (b) interface design, which consists of a lot of HTML files; (c) database design, which consists of about 63 database tables; (d) a simple UML class diagram that contains a dozen of classes and shows the logical view of the system.
Review of parts of the source code. • When design information was not well documented, parts of the
source code was reviewed Construction of architectural model Confirmation of architectural model by some of the chief developers
of the system• The draft architectural model was reviewed and corrected • The final version of the model was approved of its accuracy
Sep
t. 9,
200
9
36Seminar at Beijing University
Architecture of CRM Subsystem
Sep
t. 9,
200
9
37Seminar at Beijing University
Step 2: Construction of Quality Model
Employed the HASARD method and the SQUARE toolkits
The quality model was reviews and corrected in several iterations with the collaboration with the developers
The final version was confirmed for its accuracy and correctness by the developers
The final result: 48 nodes 60 links between the nodes.
Sep
t. 9,
200
9
38Seminar at Beijing University
Step 3: Analysis of the Quality ModelThe quality model was analysed using the SQUARE
analysis tools Identification of quality risks and quality trade-off points Derivation of the impacts of design decision and the contribution
factors on certain quality attributes The results were feed back to the developers
A workshop was run to validate the outcomes • All our findings were consistent with what has been observed in
the development and operation of the system. • Some of the phenomena observed in the operation of the
system were first time satisfactorily explained A number of specific suggestions on the improvement of the
system’s architecture were made• Some were taken by the development team in the development
of the new release of the system. • Some would result in major changes of system’s architecture
and regrettably cannot be implemented within the budget of the new releases.
Sep
t. 9,
200
9
39Seminar at Beijing University
Example 1: Causes of usability problemThe quality problem concerned:
‘Users often cannot find desired information on the website’Analysis goal:
Find the factors that may cause the problem Method:
Select the node that contains ‘user’ as the entity and “Cannot find info” as its phenomenon
Use the tool to find the contribution factors to this nodeResult:
The tool generated a sub-diagram that contains 25 nodes out of the 48 nodes in the quality model.
Interpretation of the result: Most components of the system affect usability of the system Usability is a very sensitive quality issue in the design
The outcome: Details are provided by the tool about what affect usability Useful direction to enhance usability
Sep
t. 9,
200
9
40Seminar at Beijing University
Example 2: Contribution Factors to Server’s AvailabilityProblem to be analysed:
What are the factors that affect the server’s availability?Method:
Select the node that contains ‘server’ as entity and ‘availability’ as property.
Use the tool to find the contribution factors.Result:
A sub-diagram is generated, that shows that the factors include hardware reliability, software reliability, power supply, system security, and maintenance.
Outcome: Suggested measures to improve availability To prevent hackers from attacking the server To ensure a reliable power supply To use reliable hardware to run the server To use fault tolerance to avoid software crashes on invalid input To provide maintenance tools to enable online maintenance To reduce the time that the system has to be shut down for
maintenance
Sep
t. 9,
200
9
41Seminar at Beijing University
Quality factors that affect server’s availability
Sep
t. 9,
200
9
42Seminar at Beijing University
Example 3: Identify the quality trade-off points Problem to be analysed:
What are the trade-off points in the design? Method:
Select a node and to find the consequences of the node using the tool Identify whether is leads to both positive result and negative result
It is hard to reuse
Reduce the interaction between EJB beans
system- Economy
More development time
EJB BeansGranularity
Large size
System performance
Shorter response time
Examples: • The size of
HTML files is a trade-off point. (See diagram on slide 25)
• The granularity of the session beans.
Sep
t. 9,
200
9
43Seminar at Beijing University
Comparison with related works Scenario-based analysis of software architectures
SAAM, ATAM, etc.: • Builds no quality model, • not automated
Our approach: • More systematic to derive quality model from design, • Can be automated for analysis once a model is constructed, • Models can be relatively easily validated
Hazard analysis of software system HAZOP, FMEA, etc.:
• Focused on safety, • Systematic, • But not interrelationships between quality attributes
Our approach: • More general rather than just for safety, • Focuses on the relationships between attributes
Sep
t. 9,
200
9
44Seminar at Beijing University
Future work
Combining with scenario-based methods to construct graphic quality models to represent results of scenario analysis
Analysis other quality features, e.g.Process-oriented quality features: how does a design
affect the software development process? Such as reusability, modifiability, portability, etc.
More case studiesExisting case study is on e-Commerce web-based
applicationsWork in progress: real-time and embedded systems
Sep
t. 9,
200
9
45Seminar at Beijing University
References
H. Zhu, Y. Zhang, Q. Huo and S. Greenwood, Application of Hazard Analysis to Software Quality Modelling, in Proc. of COMPSAC’02, pp. 139-144, IEEE CS, 2002.
H. Zhu, Software Design Methodology: From Principles to Architectural Styles, Elsevier, 2005.
Q. Zhang, J. Wu, and H. Zhu, Tool Support to Model-based Quality Analysis of Software Architectures, Proc. Of COMPSAC’06, IEEE CS, 2006.
Q. Zhang and H. Zhu, Automated Analysis of Software Designs with Graphic Quality Models, WSEAS Transactions on Computers, Vol. 5, Issue 10, Oct. 2006, ISSN 1109-2750, pp2232-2237.
Sep
t. 9,
200
9
46Seminar at Beijing University
Acknowledgement
The work reported in this talk is based on the research collaborated with Dr. Yanlong Zhang of Manchester Metropolitan University, UK, Dr. Sue Greenwood of Oxford Brookes University, UK, Ms. Qian Zhang and Mr. Jian Wu of National University of Defence Technology, China.
Top Related