Post on 24-May-2020
Continuous integration in a social-coding world
Bogdan Vasilescu @b_vasilescu
/ Department of Mathematics and Computer Science
Bogdan Vasilescu, Stef van Schuylenburg, Jules Wulms, !Alexander Serebrenik, Mark G. J. van den Brand!
Alexander Serebrenik @aserebrenik
How are GitHub developers using CI?
30th International Conference on Software Maintenance and Evolution, Victoria, BC, Canada, September 28 – October 3, 2014
#Developers #Commits #Pull Requests
Characteristics of the GitHub projects in our sample
1
10
100
1000
10000
100000
Sample of large and active GH repos!* not forks * ≥ 1 year * both commits
and pull requests
* Java, Python, or Ruby
* ≥ 10 changes last month
* ≥ 10 contributorsrails
elasticsearch ruby gradle
223 projects
2. Commits vs. Pull Requests: !Which builds succeed more often?!
Distribution of p−values from the Chi^2 tests
0.01
0.05 0.1
0.5 1
(45 / 84) statistically significant at 95%
Pull request builds are more successful!
0.2 0.5 1.0 2.0 5.0 10.0 20.0
Odds Ratios
(39 / 45) > 1
3. Not all projects are the same…!
Initiate CI process
Source Control
Build
TestReport
Commit
Tests
Developers
1. Usage of the Travis-CI service!
0
20
40
60
80
Java Python RubyProgramming.language
count
configured and using (113 / 223)
configured but not using !(93 / 223)
not configured (17 / 223)
• Python and Ruby
• Older than two years
• Not too many contributors
Pull request builds are more successful for:
• ≥5 successful pull requests
• ≥5 failed pull requests • ≥5 successful commits • ≥5 failed requests
(84 / 113)²
Prog. language Age (years) Contributors
Java Python Ruby < 2 2–4 > 4 < 17 17—33 > 33!
# projects 10 34 40 24 42 18 29 27 28
… s.t. χ2 p < 0.05 3 19 23 9 25 11 18 15 12
reject Stouffer H0? ✗ ✓ ✓ ✗ ✓ ✓ ✓ ✓ ✗
% odds ratio > 1 n/a 89 87 n/a 92 82 89 80 n/a