THE INFLUENCE OF ORGANIZATIONAL STRUCTURE ON SOFTWARE QUALITY: AN EMPIRICAL … · 2009. 3. 12. ·...
18
THE INFLUENCE OF ORGANIZATIONAL STRUCTURE ON SOFTWARE QUALITY: AN EMPIRICAL CASE STUDY Nachiappan Nagappan Microsoft Research Brendan Murphy Microsoft Research Victor R. Basili University of Maryland Presentation: Gregory Dymarek 1 Sunday, March 1, 2009
Transcript of THE INFLUENCE OF ORGANIZATIONAL STRUCTURE ON SOFTWARE QUALITY: AN EMPIRICAL … · 2009. 3. 12. ·...
THE INFLUENCE OF ORGANIZATIONAL STRUCTURE ON SOFTWARE QUALITY: AN EMPIRICAL
CASE STUDYNachiappan Nagappan
Microsoft ResearchBrendan Murphy
Microsoft ResearchVictor R. Basili
University of Maryland
Presentation: Gregory Dymarek
1Sunday, March 1, 2009
TO FOLLOW
‣Context
‣Experiment itself (details)
‣Results
‣Related work
‣Conclusions and Future Work
‣Questions
2Sunday, March 1, 2009
CONTEXT
‣Software development context
‣Complex projects within organizations where many teams working together for a project
‣Product quality is strongly affected by organization structure (Brooks)
‣New metric schema (organizational)
‣Subject: Microsoft Windows Vista development
3Sunday, March 1, 2009
MOTIVATION
How does organizational complexity influence
quality? Can we identify measures of the
organizational structure? How well do they do
at predicting quality, e.g., do they do a better job
of identifying problems components than earlier
used metrics [churn, coverage, bugs, etc.]?
4Sunday, March 1, 2009
EXPERIMENT
‣A tree map of the organization structure
‣Measures that quantify organizational complexity
‣Version control system access- 3404 binaries (50 million lines of code)- each binary with metrics and post-release failures
‣Correlation with the collected organizational metrics to provide empirical evidence
5Sunday, March 1, 2009
EXPERIMENT - ORG STRUCTURE
ORG
TEAM
MANAGERS
Level 0: organizations (AB, AC, AD)Level 1: managers with their teams (ABA, ABB, ABC, etc.)Level 2: lower managers (ABCA)Level 3: developers (E1, E2, etc.)
6Sunday, March 1, 2009
EXPERIMENT - METRICS (1)
‣Number of Engineers (NOE)how many did touch a particular binary
‣Number of Ex-Engineers (NOEE)how many of them left the company (subset of NOE)
‣Edit Frequency (EF)total number of times the binary code was edited
‣Depth of Master Ownership (DMO)depth in hierarchy of the owner (75%) of the binary
7Sunday, March 1, 2009
EXPERIMENT - METRICS (2)‣Percentage of organization contributing to development (PO)number of people reporting to DMO level to the size of the whole org (not whole company)
‣Level of Org Code Owenership (OCO)how much of the contributions belong to owner organization
‣Overall Organization Ownership (OOW)ratio of contributing people at DMO to all contributing people (PO considering edits)
‣Organization Intersection Factor (OIF)number of organizations touching the binary
8Sunday, March 1, 2009
RESULTS - COLLECTING
‣Eight measures for each binary that quantify organizational structure
‣Constructing prediction model using step-wise regression and principal component analysis
‣Dependant variable: post-release failures
9Sunday, March 1, 2009
RESULTS - ASSESSING (1)
‣Using precision and recall measures to measure the accuracy of the model; 50 random splits
Precision ≈ 87%(50 random splits)
10Sunday, March 1, 2009
RESULTS - ASSESSING (2)
Recall ≈ 84%(50 random splits)
11Sunday, March 1, 2009
RESULTS - ASSESSING (3)
Sensitivity(Spearmen‘s rank correlation)
(50 random splits)12Sunday, March 1, 2009
RESULTS - COMPARISON
13Sunday, March 1, 2009
RELATED WORK
‣Two scopes:- organizational research from the software perspective- predicting faults/failures
‣Geographically distributed organizations(Herbsleb & Grinter)
‣SE decisions from the viewpoint of coordination(Herbsleb & Mockus)
14Sunday, March 1, 2009
CONCLUSIONS‣Introduced organizational measures
‣Investigate the relation organization <-> quality
‣Comparison against traditional measures
‣Outcome: significant precision, recall and sensitivity with better prediction than traditional measures
‣All the data is from one software system
‣Cannot be generalized beyond the environment as of potentially large number of relevant context variables
15Sunday, March 1, 2009
FUTURE WORK
‣Conduct the research within other companies
‣Virtual organizations (open source projects)
‣Social and cognitive aspects of the work
16Sunday, March 1, 2009
REFERENCES‣ F. P. Brooks, Jr. The mythical man-month (anniversary ed.). Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1995.
‣ T. L. Graves, A. F. Karr, J. S. Marron, and H. Siy. Predicting fault incidence using software change history. IEEE Trans. Softw. Eng., 26(7):653–661, 2000.
‣ J. D. Herbsleb and A. Mockus. Formulation and preliminary test of an empirical theory of coordination in software engineering. In ESEC/FSE-11: Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering, pages 138–137, New York, NY, USA, 2003. ACM.
‣ N. Nagappan, B. Murphy, and V. Basili. The influence of organizational structure on software quality: an empirical case study. In ICSE ’08: Proceedings of the 30th international conference on Software engineering, pages 521–530, New York, NY, USA, 2008. ACM.
17Sunday, March 1, 2009
QUESTIONS
?18Sunday, March 1, 2009
