PROMISE 2011: What Prediction Model Should Be?
-
Upload
cs-ncstate -
Category
Technology
-
view
2.565 -
download
1
description
Transcript of PROMISE 2011: What Prediction Model Should Be?
WHAT PREDICTION MODEL SHOULD BE ?
Institute of Software Chinese Academy of SciencesQing Wang
2
Why we use prediction model ? Understand the past behavious of
process Control the present behavious of process Predict the future behavious of process
3
How to understand the past behavious ?
Based on the historical data which has been collected from relative processes
Apply appropriate measure and analysis techniques to analyze the data quantitatively.
Try to find some nature characteristics Is stable ? Nature bounds ? Does some quantitative relationship among
process attributes ? Can be stratified /segmented or composed
further ?
4
How to control the present behavious ?
Based on the nature bounds (baselines) to establish the desired objectives of project
Compare the present data to say if it is compliant with the nature bounds.
Take proper action to correct the deviation
Monitor and evaluate the effect of correct action
5
How to predict the future behavious ?
Based on the quantitative relationship which established depend on the understanding of past behavious.
Predict the interim outcome and final outcome of project to know if the project is going well toward the project objectives.
6
What is the essential ? Differ from estimation model
Deterministic, uncertainty, …… Use past and present data to predict future Some dependent X factors, predict
independent Y outcomes The measurement for Xs and Y make sense Data collection timely and frequently Dynamical Statistical, probability, simulation-based
7
Must be careful when establish prediction model ?
Use real world and relevant data Align the necessary project objectives SMART principle
Specific Measurable Attainable Relevant Time-bound
Cost-benefit balance Feedback and refinement
Practical Case – A CMMI ML4 organization
Issue: the organization found there are more than 90% projects delay is due to test and fix bugs need much more time than estimated
Question: How to predict the effort and schedule of test process?
Method: Identify the factors which impact the effort and
schedule of test process Establish a prediction model use these factors Measure and control these factors Predict the effort and schedule of test based on
the factors when they were got
The process
Each user case has been implemented by requirement analyze, design and coding
reviews for each user case for each development activity were conducted respectively and defect data was recorded
Each iteration has a unique testing and data was recorded The subsystem of each iteration was delivered to customer as a
intermediate result, they must be on schedule and have good quality
System testing was conducted when all iterations were completed (each release)
Process improvement objectives Goal
G1: manage the schedule of each iteration and reduce the defect as soon as possible
G2: manage the system test to detect and fix defects as many as possible
G1: in each iteration
All defect data related requirement analysis, design and coding were collected before testing
Can we find some quantitative relationship between these data and the effort of testing process?
There are two primary activities in testing process
Detect defects Fix defects
Get data from 16 history projects
13
Defect injection analysis - 1
Defect injected in design
Defect injected in requirement analysis
Defect injection analysis - 2
Defect injected in codingThe factor can be control , it can be used to establish prediction model
Y is the effort of fix defects.
If the predicted effort beyond estimate, they should assign more engineers to assure on schedule deliver of each iteration.
G2: for system testing
16
Use controllable factor to predict
Similar prediction model for system testing can be established.
The possible impact factors could be: The defects injected in requirement, design
and coding The defects removed respectively The defects removed in integrated testing
of each iteration ……
17
Summary
All data can be get The factors which used to establish
prediction model are controllable and independent
Statistical techniques are used to do analysis
Prediction object is relevant to the organization process improvement objectives
The prediction model can be refined along with the using and more data was got
18 Thanks !