An Empirical Study of Build System Migrations in Practice (ICSM 2012)
57
An Empirical Study of Build System Migrations in Practice Case Studies on KDE and the Linux Kernel 1 R. Suvorov , B. Adams, M. Nagappan, A. E. Hassan,Y. Zou
-
Upload
bram-adams -
Category
Technology
-
view
349 -
download
1
description
As the build system, i.e. the infrastructure that constructs executable deliverables out of source code and other resources, tries to catch up with the ever-evolving source code base, its size and already significant complexity keep on growing. Recently, this has forced some major software projects to migrate their build systems towards more powerful build system technologies. Since at all times software developers, testers and QA personnel rely on a functional build system to do their job, a build system migration is a risky and possibly costly undertaking, yet no methodology, nor best practices have been devised for it. In order to understand the build system migration process, we empirically studied two failed and two successful attempts of build system migration in two major open source projects, i.e. Linux and KDE, by mining source code repositories and tens of thousands of developer mailing list messages. The major contributions of this paper are: (a) isolating the phases of a common methodology for build system migrations, which is similar to the spiral model for source code development (multiple iterations of a waterfall process); (b) identifying four of the major challenges associated with this methodology: requirements gathering, communication issues, performance vs. complexity of build system code, and effective evaluation of build system prototypes; (c) detailed analysis of the first challenge, i.e., requirements gathering for the new build system, which revealed that the failed migrations did not gather requirements rigorously. Based on our findings, practitioners will be able to make more informed decisions about migrating their build system, potentially saving them time and money. http://mcis.polymtl.ca/publications/2012/ICSM2012Roman.pdf
Transcript of An Empirical Study of Build System Migrations in Practice (ICSM 2012)
An Empirical Study of Build System
Migrations in PracticeCase Studies on KDE and the Linux Kernel
1
R. Suvorov, B. Adams, M. Nagappan, A. E. Hassan, Y. Zou
Analogy: Car Assembly
2
+4x +assembly
+
Build System Transforms Source Code & Resources into Executables
3
+bdSys
+
bdSys is at the Heart of the Development Process!
4
bdSys+
Developers
QA Personnel Testers
bdSys Technology is “different”
5
bdSys Technology is “different”
5
Our record so far is a project we inherited with an Ant script weighing in at 10,000 lines of XML. Needless to say, this project required an entire team devoted to keeping the build working - a complete waste of resources.[Jez Humble & David Farley]
KDE 4 is leaving the aging "autotool" build chain behind. Some developers,
not only in KDE, like to nickname
the autotools as "auto-hell"
because of its difficult to comprehend architecture.[Alexander Neundorf]
I've been examining the existing kernel configuration system, and concluded that the best favor we could do everybody involved with it is to take it out behind the barn and shoot it through the head. [Eric Raymond]
... yet a bdSys Must Keep on Evolving
6
MLO
C
..., in fact a bdSys Changes Relatively More than Source Code
71
(c) McIntosh et al.: “An Empirical Study of
Build Maintenance Effort”
..., in fact a bdSys Changes Relatively More than Source Code
71
ArgoUML Hibernate Eclipse GCC Git Linux Mozilla PLPlot PostgreSQL0
0.51
1.52
2.53
3.54
4.55
# changes / source file# changes / build file
(c) McIntosh et al.: “An Empirical Study of
Build Maintenance Effort”
..., in fact a bdSys Changes Relatively More than Source Code
71
ArgoUML Hibernate Eclipse GCC Git Linux Mozilla PLPlot PostgreSQL0
0.51
1.52
2.53
3.54
4.55
# changes / source file# changes / build file
(c) McIntosh et al.: “An Empirical Study of
Build Maintenance Effort”
..., in fact a bdSys Changes Relatively More than Source Code
71
ArgoUML Hibernate Eclipse GCC Git Linux Mozilla PLPlot PostgreSQL0
0.51
1.52
2.53
3.54
4.55
# changes / source file# changes / build file
(c) McIntosh et al.: “An Empirical Study of
Build Maintenance Effort”
In Other Words ...
8
In Other Words ...
8
Software Projects need HELP Maintaining their Build Systems
The “easy” way out: Build System Migrations
9
The “easy” way out: Build System Migrations
9
10
11
12
12
Build system migrations are not that easy! But why?
Major Contributions
13
Common Migration
Methodology
Major Migration Challenges
Major Challenge: Requirements
Gathering
Study Setup
• Linux & KDE:
• Two established projects: 21 & 16 years old
• Large: 25 & 4.3 MLOC
• 1 failed & 1 successful migration each
• Data sources:
• git repos: 300 K & 100 K commits
• 4 mailing lists, over 1.5 million messages
14
Quantitative Study of Developer Commits
• git repositories
• Native git tools + bash scripts
• Statistics on source & bdSys code:
• Evolution
• Churn
• Distinguish between “normal” periods & migrations
15
Qualitative Analysis of Developer Communication
• Personal emails to migration participants
• Mailing lists
• Mass-downloaded and analyzed
• Statistics on number of messages:
• By groups (roles)
• By period of time
• Distinguish between “normal” periods & migrations
16
Major Contributions
17
Common Migration
Methodology
Major Migration Challenges
Major Challenge: Requirements
Gathering
18
Common Migration
Methodology
The Spiral Model of bdSys Migration
19
1. Planning 2. Risk Analysis
3. Development4. Evaluation
Why? Migration is Incremental!
• Linux kernel
• 33K+ source code files
• 2,300+ subdirectories
• 5 major subsystems
• KDE
• 11K+ source code files
• 22 packages
20
21
Common Migration
Methodology
21
Common Migration
Methodology
spiral model!
21
Common Migration
Methodology
spiral model!
migration happens
incrementally
22
Common Migration
Methodology
Major Migration Challenges
spiral model!
migration happens
incrementally
Communication AmongKey Participants
23
Communication AmongKey Participants
23
bdSys Managermajor decisions
Communication AmongKey Participants
23
bdSys Managermajor decisions
bdSys Championleading bdSys dev’t
Communication AmongKey Participants
23
bdSys Managermajor decisions
bdSys Championleading bdSys dev’t
bdSys ExpertbdSys contributor
Communication AmongKey Participants
23
bdSys Managermajor decisions
bdSys Championleading bdSys dev’t
bdSys ExpertbdSys contributor
Core Developersource contributor
Major Challenges
• Communication issues
• Build experts form a sub-community often reluctant to communicate effectively
• Performance vs. complexity
• Improving performance often means introducing hacks
• Effective evaluation
• When is migration "successfully finished"? How to measure?
• Requirements gathering (see remainder)
24
25
Common Migration
Methodology
Major Migration Challenges
spiral model!
migration happens
incrementally
25
Common Migration
Methodology
Major Migration Challenges
requirementsspiral model!
migration happens
incrementally
25
Common Migration
Methodology
Major Migration Challenges
requirements
communicationspiral model!
migration happens
incrementally
25
Common Migration
Methodology
Major Migration Challenges
requirements
communication
performance
spiral model!
migration happens
incrementally
25
Common Migration
Methodology
Major Migration Challenges
requirements
communication
performance
evaluation
spiral model!
migration happens
incrementally
26
Common Migration
Methodology
Major Migration Challenges
Major Challenge: Requirements
Gathering
requirements
communication
performance
evaluation
spiral model!
migration happens
incrementally
Example bdSys Requirements
• Platform independence
• Two new major platforms: Mac OS X & Windows
• High-level configuration
• Single-pass methodology
• Away with multiple-steps build!
• Simple and platform-independent syntax
27
Requirements Gathering Process1. Stakeholder identification - who?
• Who needs to be involved?
2. Elicitation
• What requirements are there?
3. Analysis
• Get all requirement data. Resolve conflicts, refine and organize.
4. Specification & Maintenance
• Document & order resulting requirements.
28
How SCons failed• Stakeholder identification
• Only 150 of 800 developers attended aKademy
• Elicitation
• Too little, too late: “trawled” for requirements
• Analysis
• Not enough attention paid to this phase!
• Specification & Maintenance
• Not done at all!
29
30
“the main SCons tree has intolerable problems, and there are so many layers [...] that it barely
resembles what a normal SCons build system looks like. Don't get me wrong; I use SCons at work and I
really like it, but if it isn't capable of handling KDE then why are we trying to make it do so? Is it really
worth keeping a KDE specific build system (which is what we are moving towards)?
[...] Perhaps it’s time to cut our losses and run.” [Jaison Lee]
As a Result:
Rise and Fall of SCons
31
% m
ax (
LOC
)
How succeeded
• Stakeholder identification
• Learned from SCons. Involved experts early.
• Elicitation
• Reuse some of SCons’ data.Active participation by experts!
• Analysis
• Identified conflicting requirements early.
• Specification & Maintenance
• An initiative by the build champion.32
33
Common Migration
Methodology
Major Migration Challenges
Major Challenge: Requirements
Gathering
spiral model!requirements
communication
performance
evaluation
migration happens
incrementally
33
Common Migration
Methodology
Major Migration Challenges
Major Challenge: Requirements
Gathering
spiral model!requirements
communication
performance
evaluation
migration happens
incrementally
involve all stakeholders
33
Common Migration
Methodology
Major Migration Challenges
Major Challenge: Requirements
Gathering
spiral model!requirements
communication
performance
evaluation
migration happens
incrementally
involve all stakeholders
identify &
resolve conflicts
33
Common Migration
Methodology
Major Migration Challenges
Major Challenge: Requirements
Gathering
spiral model!requirements
communication
performance
evaluation
migration happens
incrementally
involve all stakeholders
identify &
resolve conflicts
efficient elicitation
34
34
Common Migration
Methodology
spiral model!
migration happens
incrementally
34
Major Migration Challenges
requirements
communication
performance
evaluation
Common Migration
Methodology
spiral model!
migration happens
incrementally
34
Major Migration Challenges
requirements
communication
performance
evaluation
Common Migration
Methodology
spiral model!
migration happens
incrementally
Major Challenge: Requirements
Gathering
involve all stakeholders
identify &
resolve conflicts
efficient elicitation
