Post on 27-Sep-2020
Predicting Failures at SAP
A talk by Kim Herzig
ReproductionMining Metrics to Predict Component Failures
Zeller et al.
Structure of this Talk
1. Building failure predictor
2. Working at SAP and initial problems
3. Current status
4. Future work
Hypotheses
Hypothesis
Increase in complexity metrics of an entity E correlates with the number of post-release defects of E
There is a common subset of metrics for which H1 applies in all projects
There is a combination of metrics which significantly predicts the post-release defects of new entities within a
project
Predictors obtained using H3 from one project also predict failure-prone entities in other projectssimilar
H1
H2
H3
H4
RoadmapStep 1: Map bug report to code changes
get reported bugs(post release)
get version datafor reported bugs
get source code
source of graphics: Mining Metrics to Predict Component Failures, Zeller et al.
BugDatabase
VersionDatabase
SourceCode
RoadmapStep 2: Map failures to defects in entities
number of failures found
source of graphics: Mining Metrics to Predict Component Failures, Zeller et al.
BugDatabase
VersionDatabase
SourceCode
Entity Entity Entity
RoadmapStep 3: Predict failure for new entities
How to build this?
source of graphics: Mining Metrics to Predict Component Failures, Zeller et al.
Entity Entity Entity
Entity Predictor FailureProbability
BugDatabase
VersionDatabase
SourceCode
Building Failure Predictor
compute complexity metrics for each entity
e: m(e)m(e)
metrics data
Entity
EntityEntity
Building Failure Predictor
find metrics m(e) correlating to
quality(e): failure predictor
metrics data
Failure Predictor
Entity
EntityEntity
To Be Reproduced !For more details read paper
SAP vs. Microsoft• Other programming language (ABAP)‣ no metric tools!‣ ABAP highly database oriented
• Customers involved in processes‣ not all processes are in house (communication)‣ bug database more like ticket system‣ many different releases with different features‣ Service packs contain new feature
• New terrain for SAP‣ pure future work‣ many current projects but lack of experience
Goals for SAP
• Global Quality Governance:
‣ Help to improve SAP quality strategy
‣ Make development processes more efficient
‣ Integrate into product standard framework
• Development Teams:
‣ Which module to test most?
‣ Identifying critical components!
How does SAP work?online!
R/3
• Application Server
‣ programs written in ABAP/4
‣ programs executed on server
• Development Environment
‣ completely online (no data local available)
‣ integrated version repository
Initial Problems
Limited Access
• Working on Maintenance Servers
‣ Code frequently shipped
‣ Limited data access
‣ Limited development permission
Bug Data
• Ticket System
• separate system (no integration)
• used by clients and internally
• difficult data access
• Version Repository
• integrated in development system
• no direct connection to ticket system
ABAP/4
• Source code stored online (database)‣ how to get code? see later!
• Mixture between functional and object-oriented programming‣ What metrics to be used?
• Integration of SQL‣ Additional metrics?
• Gimmicks like ‘implicit function calls’‣ Hard to detect!
*
* Allgemeiner Berichtsaufbereitungsprozessor
What can be reused?
Basic Framework
• Maps package name to code objects
• Gets source code (incomplete)
• Computes McCabe complexity
• Gets code deltas from repository (untested)
Structure of this Talk
1. Building failure predictor
2. Working at SAP and initial problems
3. Current status
4. Future work
Initial Summary
we have we need
bug data source code
version data metrics data
Requirements
• SAP wants to reuse code‣ programming ABAP/4
• Keep adoption effort as low as possible‣ programming JAVA
• Use as many code information as possible
• Compute metrics efficiently
Let’s get started!
identify code objects
Getting Source
≥ 20 TABLES
initial package
compute dependencies
dependent packages
Source Code
Meta Data
Connectto R/3
The Hybrid
Start GUI
RFC call collect sourcecode
compute basicmetrics
CodeObjects,Metrics/Meta- Data
persistentDB
store results
compute metametrics
SAP Reuse
collect sourcecode
compute basicmetrics
ABAP/4 GUI
ABAP/4 tables
compute meta metrics(to be implemented)
Compute Basic Metrics
Definition: can be computed by simply scanning code
• While assembling code objects
• Granularity: Function/Method - Level
• Collect meta data for later processing(parameter types, x calls y, etc.)
Definition: can not be computed by simply scanning code
• Uses meta data collected by basic metrics
• Computed on database (MSSQL)
• Never computed at SAP
Compute Meta-Metrics
The Metrics
The Metrics- Module related metrics- Counting number of classes, functions, etc.- Detect database connectivity- Look at include statements
- Function related metrics- McCabe, FanIn, FanOut etc- Usage of global variables
- Class related metrics- Inheritance, Coupling
The Metrics
~50% of metrics implemented
metrics data available byend of February
Work to Come
• Measure entity quality‣ Assumption: every delta is bug fix‣ filtering bug data (reopened bugs, delta size)
• Build the failure predictor‣ which metrics correlate to quality
• Statistic analysis‣ try to merge with JAVA approach (Tilman)
• Reproduction on other systems‣ Let’s get excited!
Thank you!Questions?
Questions?Thank you!