2011 05-27-icse
-
Upload
heiko-koziolek -
Category
Technology
-
view
431 -
download
0
description
Transcript of 2011 05-27-icse
© ABB Group April 9, 2023 | Slide 1
Q-ImPrESSAn Industrial Case Study on Quality Impact Prediction for Evolving Service-oriented Software
Heiko Koziolek, ABB Corporate Research, GermanyBastian Schlich, Carlos Bilich, Roland Weiss, Steffen Becker, Klaus Krogmann, Mircea Trifu, Raffaela Mirandola, Anne Koziolek
Industrial automation: Process Control Systems
© ABB Group April 9, 2023 | Slide 2
MotivationRelease History for an ABB Process Control System
© ABB Group April 9, 2023 | Slide 3
2004 2005 2006 2007 2008 2009 2010
Version A First version release with complete system
concept Single environment from independent solutions Outstanding Operations Offering Function based Engineering Redundant Controllers and I/O capabilities Connectivity for Harmony and Melody FF, Redundant Profibus, HART, ABB Drives
Version B Increased system size SIL 2 Integrated Safety Connectivity for DCI and MOD 300 Alarm and Event Improvements Remote Clients via MS Terminal
Services
Version C Online Upgrade Capability Multi-User / Distributed Engineering Large screen / Multi-screen
enhancements Digital Security Improvements
Version C1 Increased system
size Multi-system
Integration SPI Integration
(PETI) MODBUS TCP
Version C2 Virtualization support MS WPF Graphics SIL3 Safety IEC 61850 (Intel Elect Devices) New PM866 controller (2x PM864) New S800 I/O (non-red HART) New Power Supplies with smaller
footprint Evolution Libraries MOD300 and Infi90
Version C3 Windows 7 support Alarm Analysis and Alarm
Shelving WirelessHART Integration Profinet, Ethernet IP, DeviceNet New Controller PM891 (2x
PM866) Engineering efficiency
improvements Detailed difference reporting Foundation Fieldbus
improvements
MotivationProblems of software evolution at ABB
Continuous evolution of ABB software systems
New requirements, technologies, failure reports
Software maintenance and evolutionare a large cost factor for ABB software
development
Current practice
Experience to rationalize design decisions
Prototyping for new technologies, performance impacts
Unknown change impacts on performance/reliability
Apply model-based prediction methodsfor systematic decision supportto save costs and achieve higher quality?
© ABB Group April 9, 2023 | Slide 4
Q-ImPrESS MethodOverview
Q-ImPrESS Workbench
Case Study
© ABB Group April 9, 2023 | Slide 7
Manual ModelingSteps executed to create a Q-ImPrESS model
Modelling static structure• Analyzed architectural documenation• Identified four key use cases• Abstraction level: process = component
Modelling dynamic structure
• Created testbed, installed system• Recorded component transitions• Derived transition probabilities
Validating the model• Created Q-ImPrESS model in workbench• Applied Q-ImPrESS consistency checker• Discussed the model with architects
© ABB Group April 9, 2023 | Slide 8
~2.5 person months
Manual ModelingQ-ImPrESS model of the ABB process control system
© ABB Group April 9, 2023 | Slide 9
Performance PredictionSteps executed to determined resource demands
Preparing measurements• Abstracted memory, virtualization
overheads• Designed experiment runs per use case
Measuring performance• Used Windows Performance Monitor• Determined >20 CPU and HD demands
Predicting performance• Annotated Q-ImPrESS model• Executed SimCom & LQN solver• Modelled alternatives and ran predictions
© ABB Group April 9, 2023 | Slide 10
~1 person month
Performance PredictionSample predictions for different design alternatives
© ABB Group April 9, 2023 | Slide 11
• Achieved prediction error below 30 percent• Easy to analyze different evolution scenarios
Pro
• Data collection consumed more time than expected• Many bottlenecks below the architectural level
Con
Performance PredictionResults: Measurements vs. Simulation Results
© ABB Group April 9, 2023 | Slide 12
Workload PerfMonMeasured
SimuComPrediction
Error (%) LQNSPrediction
Error (%)
30 17.146 12.467 27.288 12.464 27.305
60 26.681 22.366 16.174 22.343 16.260
90 31.902 32.347 1.395 32.322 1.317
120 39.016 42.432 8.754 42.329 8.490
150 51.929 51.943 0.027 51.760 0.326
Reliability PredictionSteps executed to determine failure probabilities
Preparing measurements• Conducted literature search• Selected software reliability growth model
from IEEE 1633-2008 (Littlewood/Verrall)
Measuring reliability• Acquired access to bug tracking system• Conducted curve fitting per subsystem• Determined subsystem failure probabilities
Predicting reliability• Annotated Q-ImPrESS model• Executed PRISM model checker• Conducted sensitivity analysis
© ABB Group April 9, 2023 | Slide 13
~1 person month
Each line shows how the system reliability changes if we change one subsystem reliability (8 subsystems in total)
Reliability PredictionSample sensitivity analysis
© ABB Group April 9, 2023 | Slide 14
Reliability PredictionResults
© ABB Group April 9, 2023 | Slide 15
• Identification of sensitive subsystems• Better planning of test budget distribution• Ability to analyze different usage profiles
Pro
• Abstraction level too high: (e.g., no hardware reliability)• Data collection for lower abstraction levels difficult• Validation difficult: would take long to get data
Con
More research and tool development needed
ConclusionsSummary
Q-ImPrESS
provides a structured method and useful tool support
is best used for evolutionary changes, not full redesigns
still needs to demonstrate costs/benefits
© ABB Group April 9, 2023 | Slide 16
ConclusionsFuture Work
Future work desired by ABB:
More robust reverse engineering tools
Model transformations from UML to Q-ImPrESS
Tools and best practices for data collection
© ABB Group April 9, 2023 | Slide 17
© ABB Group April 9, 2023 | Slide 18