Simplifying effort estimation based on use case points

Summerized By Shaheen 1

SIMPLIFYING EFFORT ESTIMATION BASED ON USE CASE POINTS

Prepared by: AbdulRahman Shaheen


Agenda

Introduction Experiment Setup Experiment Framework Experiment Theses and Results Threats to Validity Conclusion


Introduction Software Estimation is key aspect of

software project management. Software Project Outsourcing estimation

problems:Underestimation: can lead to delays or poor

quality software.Overestimation: might result in bid rejection.

Possible Solution:Simplify the process of Effort Estimation while

maintaining or improving the quality of estimation.


History

Function Point Analysis (FPA) introduced 1979 by Albrecht.

Mark II FPA in 1991 by Symon .

UCP in 1993 by Karner.


Use Cases


Use Case Point Methods

Assess Complexity of Actors aWeight(c): {1, 2, 3}

1. -Simple: API Communication

2. -Averge: Protocol communication

3. -Complex: GUI communication aCardinality: number of actors belonging to

the class Unadjusted Weight Factor


Use Case Point Methods - cont

Assess Complexity of Use Casescomplexity depends on number of transactionsGiven use case u, number of transactions

#trans(u)

uWeight(c) = 5 for simple, 10 for average, and 15 for complex;

uCardinality(c) is the number of use-cases assigned to class c (depends on a described system).


Technical and Environmental Factors



Add Adjustment Factor(Technical + Environmental)

TF_weighti is the weight of the ith technical complexity factor (see Table 1);

valuei is the predicted degree of influence of the ith technical complexity factor on the project (value between 0 and 5).



• EF_weighti is the weight of the ith environmental factor (see Table 1);

• valuei is the predicted degree of influence of the ith environ mental factor on the project (value between 0 and 5).


Use Case Point Methods -cont

UUCP = UAW + UUCW UCP = UUCP * TCF * EF Effort Estimations

Effort = UCP * PF○ [Karner]

PF = 20 hours / UCP

○ [Schneider and Winters]PF=20h if EF<3PF=28h if 3<EF<4PF=36h if EF>4


Use Case Point Methods -cont Calibrating UCP with historical data:


Projects Setup 14 projects 277 to 3593 man hours Projects labeled according to their

development environmentA – G: industrial projectsH – N: Pazon Univ. (PUT)

OrigionU:budgetedS2B: student to businessI: software development company


Projects Information


Projects UCP


Subsets of Datasets used in the study


Framework

UCP calculations:1. Reviewing use cases and reject:

○ Business use case.○ Include + extend use cases, if they describe

the same logic with lower abstraction○ Unimplemented + duplicated.

2. Counting transactions and steps○ External expert was used for validation of

correctness of step counting.

3. Obtaining TCF and EF

4. Calculating UCP


Framework

Evaluation of prediction accuracy1. Ordinary least squares regression (OLS)

○ pi is the project for which effort is estimated;○ ᵝSize is the slope for Size;○ ᵝ0 is the constant or intercept (set to 0);○ Size (pi) is the value of the size metric

calculated for the project pi.


Framework Evaluation of prediction accuracy

2. Multiple regression

○ pi is the project for which effort is estimated;○ ᵝSize is the slope for Size;○ ᵝFactor is the slope for Factor;○ ᵝ0 is the constant or intercept (set to 0);○ Size (pi) is the value of the size metric calculated for

the project pi.○ Factor (pi) is the value of the additional factor included

in the regression model (chosen among the EFs, TCFs, and team size) for the project pi


Framework


Framework

The evaluation criteria○ magnitude of relative error (MRE) [inversely]

Agregated using Mean of MRE (MMRE)

○ mean relative error (Mean RE)bias of the estimates

○ Pred (e)=k/n [proportional]prediction quality is calculated on a set of n projects,

where k is the number of projects for which estimation error (MRE) is less than or equal to e

To test Statistical Significance: two-tailed Wilcoxon signed-rank test with the significance level a set to 0.05 was used.


Experiment Actors complexity in Use Case Points

UAW had only minor impact on the accuracy of the effort estimation based on UCP

Adjustment factors in Use Case PointsProblems:

○ Lack of standardized (agreed) scale○ Not verified weights of factors

Investigation○ Overlapping○ Minor influence on estimation

Result:○ 13 to 4 technical complexity factors ○ 8 to 2 environmental factors


Experiment Use Case Points – steps vs. transactions

Transactions: either executed fully or not at all.Counting steps will simplify calculation of UCP

method.Thesis: The value of UCP calculated based on

steps is the same as if calculated based on transactions.

Results are not the same.UCP-T = 0.584 * UCP-S

○ Can not be generalized.○ Depends on SRS writing styles.


Experiment Thesis

The accuracy of UCP calculated based on steps is not worse than if calculated based on transactions.

Results:Almost the same, however, on average UCP-S more

accurate.Minor tendency to overestimate.Mean RE close to Zero. (almost no bias)

Conclusion: if a company use standard style of writing they can use UCP-S instead of UCP-T.


Experiment Can the estimation be further simplified to

count just the total number of steps in all use cases?

Thesis: The Steps metric, which is the total number of steps in all use cases describing the system, can be used to estimate effort with similar accuracy to UCP.

Result:Can be used to simplify the effort.Suffers from sensitivity to writing styles.


Use-case transactions methods

The Transactions metric is defined as the total number of transactions that can be identified in all use cases describing the system under development.

The TTPoints metric is also based on the number of transactions, but it includes additional information about the semantics of transactions (twelve semantic transaction types have been identified so far ).


Experiment Thesis: Use-case transactions can be used to

provide effort estimation with better accuracy than UCP calculated based on steps.

Investigation: compare two projects with:different writing styles.Same actual effort.

Result: Based on post-productivity factor UCP-T is better if there historical data suffer either from writing style or abstraction level.

Lower Variability on prediction


Experiment

TTPoints outperform all step based metrics.However, calculation is complex.


Threats to validity – internal validity Projects developed by different

organizations.Maturity of development process is different.Some records does not show activities.

Objectiveness of Transaction [style of writing]Solved by allowing only one person to

extract the transactions and cross validate it by external expert.


Threats to validity – Conclusion validity Large number of direct comparison

Can be solved using Bonferroni correction. Power of 10

sensitivity analysis was performed: and conclusion was “whenever we were not able to reject the null hypothesis, it could mean that either the null hypothesis was really true or the effect size was too small to be detected in this study”.


Threats to validity – External validity Size of Data Sets:

Largest data sets used to study UCP heterogeneity of the analyzed set of

projectsSame cross-validation procedure for all

combinations.○ MMRE shows the difference in data set does not

influence the visibility of findings.

Data-intensive systemsThis study can be generalized to this kind of

systems.


Conclusion

Potential ways to simplify effort estimation:Reject UAW [minor impact on accuracy]Reduce Technical and Environmental

Factors.Use steps to calculate UCP. Further more,

you can use steps metric.Transaction Metrics can replace UCP-T.

Measures based on Steps are sensitive to writing styles.


Future Work

Developing standards of writing Use Cases. This might include enforcement to use specific phrases to describe use cases.

Study if a combination or all suggested simplification can be used together to reduce effort while maintaining accuracy.

Simplifying effort estimation based on use case points

Technology

Transcript of Simplifying effort estimation based on use case points