Managing Community Contributions: Lessons Learned from a Case

Study on Android and Linux

Nicolas BettenburgBram Adams

Ahmed E. HassanQueen’s University

Daniel M. GermanUniversity of Victoria

QUEEN

’SUNIVERSITY

ANNUAL

REPORT

2008

ANNUAL RE

PORT2008

Queen’s University

Kingston, Ontario

Canada k7l 3n6

Que

en’s

Uni

vers

ity08

-026

9

!

!

!

!

!

"!

!"#$%&'#()*')'(%+'*'(,(!'*!-.,(%*

!

!

#$%&$'($)!*+"+!!

!

(,-./&0/#-1002!,!345678!9:!;<;=75;!>787!4?@8AB7B!A3B!CA?ACD=<!D3C87A;7B!D3!?87?A8A=D93!:98!

948! =<?DCAE! 6ACF/=9/;CG99E! AC=DHD=<! H9E457I! ! J895! =G7! G7E?! B7;F! =9! (A3378K!

59;=!;<;=75;!>98F7B!>7EEI!!&GD;!;7C=D93!9:!87?98=!;455A8DL7;!=G7!87;4E=;I!

&G7!87;DB73C7!-95?4=78!17E?!M7;F!;A=7EED=7!>A;!9?73!:895!#7?=75678!N 8B!

=9! "+ =G! =9! ;78H7! 37>! A3B! 87=483D3@! ;=4B73=;I! ! &GD;! E9CA=D93! ;78HDC7B! OPO!

;=4B73=;! >D=G! D;;47;! EDF7! 4;78! QM! ?A;;>98B! 87;7=;K! 37=>98F! ACC7;;K! A3B!

>D87E7;;!D3=7837=!C93:D@48A=D93;I!

R?@8AB7;! =9!948!>D87E7;;! CA?A6DED=<! 73A6E7B!4;! =9! ;4??98=! 9H78!PKO++! ;D54E=A3794;!4;78;! =GD;! #7?=75678I! ! &GD;!

H9E457!D;!=G7!GD@G7;=!>7!GAH7!7H78!7S?78D73C7B!93!CA5?4;I!!,;!CA3!6773!;773!:895!=G7!@8A?G!67E9>K!=G7!8A5?/4?!

:895!;45578!GA;!6773!B8A5A=DCI!!'A3<!;=4B73=;!39>!A88DH7!93!CA5?4;!>D=G!54E=D?E7!>D87E7;;!B7HDC7;!T=<?DCAEE<!A!

EA?=9?!A3B!A3!D%9BU!A3B!7S?7C=!46DV4D=94;!A3B!87EDA6E7!C9H78A@7I!!

!

2

What is a community contribution?

2

What is a community contribution?

A lot of things:

2

What is a community contribution?

A lot of things:

Documentation

Graphics/Art

Source Code

Testing

Error-Report

Music

Stuffed Animals

Installation Party

Bug Fix

Feedback

FeatureAdvertisement

3

What is a community contribution?

For our study:

Documentation

Graphics/Art

Source Code

Testing

Error-Report

Music

Stuffed Animals

Installation Party

Bug Fix

Feedback

FeatureAdvertisement

4

Why would you want community contributions?

Community Member• proposes and implements features• suggests bug fixes• translates code/documentation• tests new features• provides fast feedback• brings new users to project• public relations/advertisement

4

Why would you want community contributions?

Community Member• proposes and implements features• suggests bug fixes• translates code/documentation• tests new features• provides fast feedback• brings new users to project• public relations/advertisement

FOR FREE!

5

Competitive Advantage Market Strategy

LiabilityQuality

AssurancePatents

Product Identity

User Integration

Issues with community contributions

6

What is an effective process to enable and manage community contributions?

Are contributors actively engaged in the development?

Are community contributions reviewed in a timely fashion?

What kinds of contributions can be expected – and which parts of the software product do they target?

Research Questions:

7

8

Submission

ProjectRepository

Feedback

Feedback

OK OK

Conception

Contributor

Review Verification Integration

Discussion

Idea

Contribution

Feedback

(1)

(2) (3) (4) (5)

Figure 1: Conceptual Model of the CollaborationManagement Process.

this conceptual model is implemented in two popular large

software systems, and (3) we identify techniques and pro-

cesses that are valuable for industry.

The rest of this paper is organized as follows. We present

a conceptual model for collaboration management that we

derived from seven popular for-profit and non-for-profit open

source systems in section 2. In section 3 we discuss the

design of our detailed case study and present our results in

section 4. We identify and present related work in section 5,

followed by our concluding remarks presented in section 6.

2. CONCEPTUAL MODEL OF COLLABO-

RATION MANAGEMENT

In order to establish a common ground in terms of termi-

nology and concepts, we first derive a conceptual model of

contribution management styles. This model was derived

through a study of the processes and practices of seven

popular for-profit and non-for-profit open source software

systems, based on available documentation, press releases,

white papers and previous research. These seven software

systems are summarized in Figure 1.

The APACHE web server and LINUX operating system

are both non-for-profit open source projects. Even though

OPENSOLARIS and FEDORA are non-for-profit open source

systems, they build the basis for commercial products, which

contain features that are cherry-picked from the open source

products. The MySQL relational database and the ECLIPSEdevelopment environment are both available for free to the

community, but also target the business market. Whereas

MySQL aims at customer support, hardware certification

and increased rate of updates for enterprises, ECLIPSE is

surrounded by a rich market of commercial products that

build on top of the open source software. The ANDROIDplatform and operating system for mobile devices is unique

in that the source code is freely available, yet drives the

commercial agendas of industry partners.

Overall, the conceptual model defines five phases that a

contribution must undergo before it can be integrated into

the next software release (Figure 1). The following subsec-

tions discuss each phase, illustrated by concrete examples

from the seven analyzed systems.

2.1 Conception

Similar to classical software development, prospective con-

tributors with an idea for a new feature or bug fix need early

feedback and support to work out their ideas into a concrete

design. Such discussions often take place in project mail-

ing lists (LINUX, ECLIPSE, OPENSOLARIS, APACHE), issue

tracking systems (ANDROID, ECLIPSE), wikis (FEDORA),

or discussion forums (MySQL). Within one project, combi-

nations of these venues can be used depending on the sensi-

tivity of the feature or bug fix. The outcome of the concep-

tion step is either a concrete design (usually after multiple

feedback rounds), or a rejection of the proposed feature or

bug fix if the idea does not align with the project’s goals.

The conception phase is not mandatory – in most projects

contributors can skip this phase altogether and start their

collaboration with a concrete source code submission.

2.2 Submission

Once the design for a contribution has been fleshed out in

source code, the contributor sends this contribution to the

project. Since open contribution projects usually receive

contributions from many people, intellectual property in-

fringements are a substantial concern [11]. All seven projects

that we studied acknowledge this risk and have specific poli-

cies for their submission process that guarantee traceability.

In LINUX, community contributions must contain a “signed-

off” record that acts as a substitute for an electronic signa-

ture. ECLIPSE, FEDORA, MySQL and APACHE completely

disallow contributions through mailing lists. Instead, they

require a submission to be carried out formally by opening

a new issue in their bug tracking systems. The ANDROIDproject goes one step further and uses a separate system on

top of the distributed version control system, called GER-RIT. This system automatically collects all submissions to

the project repositories and notifies project members of the

contribution. GERRIT was built as a successor to the highly

popular Mondrian tool, internally used by Google, and was

built to handle the complete contribution management pro-

cess (steps two to five in the conceptual model) through a

web-based interface.

2.3 Review

After a submission has been formally submitted for con-

sideration, senior members of the project are notified. All

seven projects require a formal peer review to be carried

out for every community contribution. The goal of this peer

review is to determine the fit of the contribution for the

project and to ensure that contributions meet the quality

standards of the project. Reviewers are either appointed

automatically (e.g., ANDROID), or step up voluntarily (e.g.,

LINUX). Reviewers then do an inspection of the contributed

code and judge the overall quality and fit of the submission.

If there are concerns with the contribution, reviewers can

give feedback to the contributor, ranging from high-level

comments down to code line-by-line comments. To enable

the latter, the ANDROID project built its own dedicated re-

view system as a web application. Most other projects use

issue tracking systems or more low-tech technologies such as

email to support the review process. There are three po-

tential outcomes of the review: the contribution is accepted

as-is, the contribution needs to be reworked, or the contri-

bution is rejected. If necessary the contributor should then

address any concerns raised and resubmit an updated revi-

sion of the contribution (but sometimes it is abandoned and

never reworked).

2

8

Submission

ProjectRepository

Feedback

Feedback

OK OK

Conception

Contributor

Review Verification Integration

Discussion

Idea

Contribution

Feedback

(1)

(2) (3) (4) (5)

Figure 1: Conceptual Model of the CollaborationManagement Process.

this conceptual model is implemented in two popular large

software systems, and (3) we identify techniques and pro-

cesses that are valuable for industry.

The rest of this paper is organized as follows. We present

a conceptual model for collaboration management that we

derived from seven popular for-profit and non-for-profit open

source systems in section 2. In section 3 we discuss the

design of our detailed case study and present our results in

section 4. We identify and present related work in section 5,

followed by our concluding remarks presented in section 6.

2. CONCEPTUAL MODEL OF COLLABO-

RATION MANAGEMENT

In order to establish a common ground in terms of termi-

nology and concepts, we first derive a conceptual model of

contribution management styles. This model was derived

through a study of the processes and practices of seven

popular for-profit and non-for-profit open source software

systems, based on available documentation, press releases,

white papers and previous research. These seven software

systems are summarized in Figure 1.

The APACHE web server and LINUX operating system

are both non-for-profit open source projects. Even though

OPENSOLARIS and FEDORA are non-for-profit open source

systems, they build the basis for commercial products, which

contain features that are cherry-picked from the open source

products. The MySQL relational database and the ECLIPSEdevelopment environment are both available for free to the

community, but also target the business market. Whereas

MySQL aims at customer support, hardware certification

and increased rate of updates for enterprises, ECLIPSE is

surrounded by a rich market of commercial products that

build on top of the open source software. The ANDROIDplatform and operating system for mobile devices is unique

in that the source code is freely available, yet drives the

commercial agendas of industry partners.

Overall, the conceptual model defines five phases that a

contribution must undergo before it can be integrated into

the next software release (Figure 1). The following subsec-

tions discuss each phase, illustrated by concrete examples

from the seven analyzed systems.

2.1 Conception

Similar to classical software development, prospective con-

tributors with an idea for a new feature or bug fix need early

feedback and support to work out their ideas into a concrete

design. Such discussions often take place in project mail-

ing lists (LINUX, ECLIPSE, OPENSOLARIS, APACHE), issue

tracking systems (ANDROID, ECLIPSE), wikis (FEDORA),

or discussion forums (MySQL). Within one project, combi-

nations of these venues can be used depending on the sensi-

tivity of the feature or bug fix. The outcome of the concep-

tion step is either a concrete design (usually after multiple

feedback rounds), or a rejection of the proposed feature or

bug fix if the idea does not align with the project’s goals.

The conception phase is not mandatory – in most projects

contributors can skip this phase altogether and start their

collaboration with a concrete source code submission.

2.2 Submission

Once the design for a contribution has been fleshed out in

source code, the contributor sends this contribution to the

project. Since open contribution projects usually receive

contributions from many people, intellectual property in-

fringements are a substantial concern [11]. All seven projects

that we studied acknowledge this risk and have specific poli-

cies for their submission process that guarantee traceability.

In LINUX, community contributions must contain a “signed-

off” record that acts as a substitute for an electronic signa-

ture. ECLIPSE, FEDORA, MySQL and APACHE completely

disallow contributions through mailing lists. Instead, they

require a submission to be carried out formally by opening

a new issue in their bug tracking systems. The ANDROIDproject goes one step further and uses a separate system on

top of the distributed version control system, called GER-RIT. This system automatically collects all submissions to

the project repositories and notifies project members of the

contribution. GERRIT was built as a successor to the highly

popular Mondrian tool, internally used by Google, and was

built to handle the complete contribution management pro-

cess (steps two to five in the conceptual model) through a

web-based interface.

2.3 Review

After a submission has been formally submitted for con-

sideration, senior members of the project are notified. All

seven projects require a formal peer review to be carried

out for every community contribution. The goal of this peer

review is to determine the fit of the contribution for the

project and to ensure that contributions meet the quality

standards of the project. Reviewers are either appointed

automatically (e.g., ANDROID), or step up voluntarily (e.g.,

LINUX). Reviewers then do an inspection of the contributed

code and judge the overall quality and fit of the submission.

If there are concerns with the contribution, reviewers can

give feedback to the contributor, ranging from high-level

comments down to code line-by-line comments. To enable

the latter, the ANDROID project built its own dedicated re-

view system as a web application. Most other projects use

issue tracking systems or more low-tech technologies such as

email to support the review process. There are three po-

tential outcomes of the review: the contribution is accepted

as-is, the contribution needs to be reworked, or the contri-

bution is rejected. If necessary the contributor should then

address any concerns raised and resubmit an updated revi-

sion of the contribution (but sometimes it is abandoned and

never reworked).

2

8

Submission

ProjectRepository

Feedback

Feedback

OK OK

Conception

Contributor

Review Verification Integration

Discussion

Idea

Contribution

Feedback

(1)

(2) (3) (4) (5)

Figure 1: Conceptual Model of the CollaborationManagement Process.

this conceptual model is implemented in two popular large

software systems, and (3) we identify techniques and pro-

cesses that are valuable for industry.

The rest of this paper is organized as follows. We present

a conceptual model for collaboration management that we

derived from seven popular for-profit and non-for-profit open

source systems in section 2. In section 3 we discuss the

design of our detailed case study and present our results in

section 4. We identify and present related work in section 5,

followed by our concluding remarks presented in section 6.

2. CONCEPTUAL MODEL OF COLLABO-

RATION MANAGEMENT

In order to establish a common ground in terms of termi-

nology and concepts, we first derive a conceptual model of

contribution management styles. This model was derived

through a study of the processes and practices of seven

popular for-profit and non-for-profit open source software

systems, based on available documentation, press releases,

white papers and previous research. These seven software

systems are summarized in Figure 1.

The APACHE web server and LINUX operating system

are both non-for-profit open source projects. Even though

OPENSOLARIS and FEDORA are non-for-profit open source

systems, they build the basis for commercial products, which

contain features that are cherry-picked from the open source

products. The MySQL relational database and the ECLIPSEdevelopment environment are both available for free to the

community, but also target the business market. Whereas

MySQL aims at customer support, hardware certification

and increased rate of updates for enterprises, ECLIPSE is

surrounded by a rich market of commercial products that

build on top of the open source software. The ANDROIDplatform and operating system for mobile devices is unique

in that the source code is freely available, yet drives the

commercial agendas of industry partners.

Overall, the conceptual model defines five phases that a

contribution must undergo before it can be integrated into

the next software release (Figure 1). The following subsec-

tions discuss each phase, illustrated by concrete examples

from the seven analyzed systems.

2.1 Conception

Similar to classical software development, prospective con-

tributors with an idea for a new feature or bug fix need early

feedback and support to work out their ideas into a concrete

design. Such discussions often take place in project mail-

ing lists (LINUX, ECLIPSE, OPENSOLARIS, APACHE), issue

tracking systems (ANDROID, ECLIPSE), wikis (FEDORA),

or discussion forums (MySQL). Within one project, combi-

nations of these venues can be used depending on the sensi-

tivity of the feature or bug fix. The outcome of the concep-

tion step is either a concrete design (usually after multiple

feedback rounds), or a rejection of the proposed feature or

bug fix if the idea does not align with the project’s goals.

The conception phase is not mandatory – in most projects

contributors can skip this phase altogether and start their

collaboration with a concrete source code submission.

2.2 Submission

Once the design for a contribution has been fleshed out in

source code, the contributor sends this contribution to the

project. Since open contribution projects usually receive

contributions from many people, intellectual property in-

fringements are a substantial concern [11]. All seven projects

that we studied acknowledge this risk and have specific poli-

cies for their submission process that guarantee traceability.

In LINUX, community contributions must contain a “signed-

off” record that acts as a substitute for an electronic signa-

ture. ECLIPSE, FEDORA, MySQL and APACHE completely

disallow contributions through mailing lists. Instead, they

require a submission to be carried out formally by opening

a new issue in their bug tracking systems. The ANDROIDproject goes one step further and uses a separate system on

top of the distributed version control system, called GER-RIT. This system automatically collects all submissions to

the project repositories and notifies project members of the

contribution. GERRIT was built as a successor to the highly

popular Mondrian tool, internally used by Google, and was

built to handle the complete contribution management pro-

cess (steps two to five in the conceptual model) through a

web-based interface.

2.3 Review

After a submission has been formally submitted for con-

sideration, senior members of the project are notified. All

seven projects require a formal peer review to be carried

out for every community contribution. The goal of this peer

review is to determine the fit of the contribution for the

project and to ensure that contributions meet the quality

standards of the project. Reviewers are either appointed

automatically (e.g., ANDROID), or step up voluntarily (e.g.,

LINUX). Reviewers then do an inspection of the contributed

code and judge the overall quality and fit of the submission.

If there are concerns with the contribution, reviewers can

give feedback to the contributor, ranging from high-level

comments down to code line-by-line comments. To enable

the latter, the ANDROID project built its own dedicated re-

view system as a web application. Most other projects use

issue tracking systems or more low-tech technologies such as

email to support the review process. There are three po-

tential outcomes of the review: the contribution is accepted

as-is, the contribution needs to be reworked, or the contri-

bution is rejected. If necessary the contributor should then

address any concerns raised and resubmit an updated revi-

sion of the contribution (but sometimes it is abandoned and

never reworked).

2

8

Submission

ProjectRepository

Feedback

Feedback

OK OK

Conception

Contributor

Review Verification Integration

Discussion

Idea

Contribution

Feedback

(1)

(2) (3) (4) (5)

Figure 1: Conceptual Model of the CollaborationManagement Process.

this conceptual model is implemented in two popular large

software systems, and (3) we identify techniques and pro-

cesses that are valuable for industry.

The rest of this paper is organized as follows. We present

a conceptual model for collaboration management that we

derived from seven popular for-profit and non-for-profit open

source systems in section 2. In section 3 we discuss the

design of our detailed case study and present our results in

section 4. We identify and present related work in section 5,

followed by our concluding remarks presented in section 6.

2. CONCEPTUAL MODEL OF COLLABO-

RATION MANAGEMENT

In order to establish a common ground in terms of termi-

nology and concepts, we first derive a conceptual model of

contribution management styles. This model was derived

through a study of the processes and practices of seven

popular for-profit and non-for-profit open source software

systems, based on available documentation, press releases,

white papers and previous research. These seven software

systems are summarized in Figure 1.

The APACHE web server and LINUX operating system

are both non-for-profit open source projects. Even though

OPENSOLARIS and FEDORA are non-for-profit open source

systems, they build the basis for commercial products, which

contain features that are cherry-picked from the open source

products. The MySQL relational database and the ECLIPSEdevelopment environment are both available for free to the

community, but also target the business market. Whereas

MySQL aims at customer support, hardware certification

and increased rate of updates for enterprises, ECLIPSE is

surrounded by a rich market of commercial products that

build on top of the open source software. The ANDROIDplatform and operating system for mobile devices is unique

in that the source code is freely available, yet drives the

commercial agendas of industry partners.

Overall, the conceptual model defines five phases that a

contribution must undergo before it can be integrated into

the next software release (Figure 1). The following subsec-

tions discuss each phase, illustrated by concrete examples

from the seven analyzed systems.

2.1 Conception

Similar to classical software development, prospective con-

tributors with an idea for a new feature or bug fix need early

feedback and support to work out their ideas into a concrete

design. Such discussions often take place in project mail-

ing lists (LINUX, ECLIPSE, OPENSOLARIS, APACHE), issue

tracking systems (ANDROID, ECLIPSE), wikis (FEDORA),

or discussion forums (MySQL). Within one project, combi-

nations of these venues can be used depending on the sensi-

tivity of the feature or bug fix. The outcome of the concep-

tion step is either a concrete design (usually after multiple

feedback rounds), or a rejection of the proposed feature or

bug fix if the idea does not align with the project’s goals.

The conception phase is not mandatory – in most projects

contributors can skip this phase altogether and start their

collaboration with a concrete source code submission.

2.2 Submission

Once the design for a contribution has been fleshed out in

source code, the contributor sends this contribution to the

project. Since open contribution projects usually receive

contributions from many people, intellectual property in-

fringements are a substantial concern [11]. All seven projects

that we studied acknowledge this risk and have specific poli-

cies for their submission process that guarantee traceability.

In LINUX, community contributions must contain a “signed-

off” record that acts as a substitute for an electronic signa-

ture. ECLIPSE, FEDORA, MySQL and APACHE completely

disallow contributions through mailing lists. Instead, they

require a submission to be carried out formally by opening

a new issue in their bug tracking systems. The ANDROIDproject goes one step further and uses a separate system on

top of the distributed version control system, called GER-RIT. This system automatically collects all submissions to

the project repositories and notifies project members of the

contribution. GERRIT was built as a successor to the highly

popular Mondrian tool, internally used by Google, and was

built to handle the complete contribution management pro-

cess (steps two to five in the conceptual model) through a

web-based interface.

2.3 Review

After a submission has been formally submitted for con-

sideration, senior members of the project are notified. All

seven projects require a formal peer review to be carried

out for every community contribution. The goal of this peer

review is to determine the fit of the contribution for the

project and to ensure that contributions meet the quality

standards of the project. Reviewers are either appointed

automatically (e.g., ANDROID), or step up voluntarily (e.g.,

LINUX). Reviewers then do an inspection of the contributed

code and judge the overall quality and fit of the submission.

If there are concerns with the contribution, reviewers can

give feedback to the contributor, ranging from high-level

comments down to code line-by-line comments. To enable

the latter, the ANDROID project built its own dedicated re-

view system as a web application. Most other projects use

issue tracking systems or more low-tech technologies such as

email to support the review process. There are three po-

tential outcomes of the review: the contribution is accepted

as-is, the contribution needs to be reworked, or the contri-

bution is rejected. If necessary the contributor should then

address any concerns raised and resubmit an updated revi-

sion of the contribution (but sometimes it is abandoned and

never reworked).

2

8

Submission

ProjectRepository

Feedback

Feedback

OK OK

Conception

Contributor

Review Verification Integration

Discussion

Idea

Contribution

Feedback

(1)

(2) (3) (4) (5)

Figure 1: Conceptual Model of the CollaborationManagement Process.

this conceptual model is implemented in two popular large

software systems, and (3) we identify techniques and pro-

cesses that are valuable for industry.

The rest of this paper is organized as follows. We present

a conceptual model for collaboration management that we

derived from seven popular for-profit and non-for-profit open

source systems in section 2. In section 3 we discuss the

design of our detailed case study and present our results in

section 4. We identify and present related work in section 5,

followed by our concluding remarks presented in section 6.

2. CONCEPTUAL MODEL OF COLLABO-

RATION MANAGEMENT

In order to establish a common ground in terms of termi-

nology and concepts, we first derive a conceptual model of

contribution management styles. This model was derived

through a study of the processes and practices of seven

popular for-profit and non-for-profit open source software

systems, based on available documentation, press releases,

white papers and previous research. These seven software

systems are summarized in Figure 1.

The APACHE web server and LINUX operating system

are both non-for-profit open source projects. Even though

OPENSOLARIS and FEDORA are non-for-profit open source

systems, they build the basis for commercial products, which

contain features that are cherry-picked from the open source

products. The MySQL relational database and the ECLIPSEdevelopment environment are both available for free to the

community, but also target the business market. Whereas

MySQL aims at customer support, hardware certification

and increased rate of updates for enterprises, ECLIPSE is

surrounded by a rich market of commercial products that

build on top of the open source software. The ANDROIDplatform and operating system for mobile devices is unique

in that the source code is freely available, yet drives the

commercial agendas of industry partners.

Overall, the conceptual model defines five phases that a

contribution must undergo before it can be integrated into

the next software release (Figure 1). The following subsec-

tions discuss each phase, illustrated by concrete examples

from the seven analyzed systems.

2.1 Conception

Similar to classical software development, prospective con-

tributors with an idea for a new feature or bug fix need early

feedback and support to work out their ideas into a concrete

design. Such discussions often take place in project mail-

ing lists (LINUX, ECLIPSE, OPENSOLARIS, APACHE), issue

tracking systems (ANDROID, ECLIPSE), wikis (FEDORA),

or discussion forums (MySQL). Within one project, combi-

nations of these venues can be used depending on the sensi-

tivity of the feature or bug fix. The outcome of the concep-

tion step is either a concrete design (usually after multiple

feedback rounds), or a rejection of the proposed feature or

bug fix if the idea does not align with the project’s goals.

The conception phase is not mandatory – in most projects

contributors can skip this phase altogether and start their

collaboration with a concrete source code submission.

2.2 Submission

Once the design for a contribution has been fleshed out in

source code, the contributor sends this contribution to the

project. Since open contribution projects usually receive

contributions from many people, intellectual property in-

fringements are a substantial concern [11]. All seven projects

that we studied acknowledge this risk and have specific poli-

cies for their submission process that guarantee traceability.

In LINUX, community contributions must contain a “signed-

off” record that acts as a substitute for an electronic signa-

ture. ECLIPSE, FEDORA, MySQL and APACHE completely

disallow contributions through mailing lists. Instead, they

require a submission to be carried out formally by opening

a new issue in their bug tracking systems. The ANDROIDproject goes one step further and uses a separate system on

top of the distributed version control system, called GER-RIT. This system automatically collects all submissions to

the project repositories and notifies project members of the

contribution. GERRIT was built as a successor to the highly

popular Mondrian tool, internally used by Google, and was

built to handle the complete contribution management pro-

cess (steps two to five in the conceptual model) through a

web-based interface.

2.3 Review

After a submission has been formally submitted for con-

sideration, senior members of the project are notified. All

seven projects require a formal peer review to be carried

out for every community contribution. The goal of this peer

review is to determine the fit of the contribution for the

project and to ensure that contributions meet the quality

standards of the project. Reviewers are either appointed

automatically (e.g., ANDROID), or step up voluntarily (e.g.,

LINUX). Reviewers then do an inspection of the contributed

code and judge the overall quality and fit of the submission.

If there are concerns with the contribution, reviewers can

give feedback to the contributor, ranging from high-level

comments down to code line-by-line comments. To enable

the latter, the ANDROID project built its own dedicated re-

view system as a web application. Most other projects use

issue tracking systems or more low-tech technologies such as

email to support the review process. There are three po-

tential outcomes of the review: the contribution is accepted

as-is, the contribution needs to be reworked, or the contri-

bution is rejected. If necessary the contributor should then

address any concerns raised and resubmit an updated revi-

sion of the contribution (but sometimes it is abandoned and

never reworked).

2

8

Submission

ProjectRepository

Feedback

Feedback

OK OK

Conception

Contributor

Review Verification Integration

Discussion

Idea

Contribution

Feedback

(1)

(2) (3) (4) (5)

Figure 1: Conceptual Model of the CollaborationManagement Process.

this conceptual model is implemented in two popular large

software systems, and (3) we identify techniques and pro-

cesses that are valuable for industry.

The rest of this paper is organized as follows. We present

a conceptual model for collaboration management that we

derived from seven popular for-profit and non-for-profit open

source systems in section 2. In section 3 we discuss the

design of our detailed case study and present our results in

section 4. We identify and present related work in section 5,

followed by our concluding remarks presented in section 6.

2. CONCEPTUAL MODEL OF COLLABO-

RATION MANAGEMENT

In order to establish a common ground in terms of termi-

nology and concepts, we first derive a conceptual model of

contribution management styles. This model was derived

through a study of the processes and practices of seven

popular for-profit and non-for-profit open source software

systems, based on available documentation, press releases,

white papers and previous research. These seven software

systems are summarized in Figure 1.

The APACHE web server and LINUX operating system

are both non-for-profit open source projects. Even though

OPENSOLARIS and FEDORA are non-for-profit open source

systems, they build the basis for commercial products, which

contain features that are cherry-picked from the open source

products. The MySQL relational database and the ECLIPSEdevelopment environment are both available for free to the

community, but also target the business market. Whereas

MySQL aims at customer support, hardware certification

and increased rate of updates for enterprises, ECLIPSE is

surrounded by a rich market of commercial products that

build on top of the open source software. The ANDROIDplatform and operating system for mobile devices is unique

in that the source code is freely available, yet drives the

commercial agendas of industry partners.

Overall, the conceptual model defines five phases that a

contribution must undergo before it can be integrated into

the next software release (Figure 1). The following subsec-

tions discuss each phase, illustrated by concrete examples

from the seven analyzed systems.

2.1 Conception

Similar to classical software development, prospective con-

tributors with an idea for a new feature or bug fix need early

feedback and support to work out their ideas into a concrete

design. Such discussions often take place in project mail-

ing lists (LINUX, ECLIPSE, OPENSOLARIS, APACHE), issue

tracking systems (ANDROID, ECLIPSE), wikis (FEDORA),

or discussion forums (MySQL). Within one project, combi-

nations of these venues can be used depending on the sensi-

tivity of the feature or bug fix. The outcome of the concep-

tion step is either a concrete design (usually after multiple

feedback rounds), or a rejection of the proposed feature or

bug fix if the idea does not align with the project’s goals.

The conception phase is not mandatory – in most projects

contributors can skip this phase altogether and start their

collaboration with a concrete source code submission.

2.2 Submission

Once the design for a contribution has been fleshed out in

source code, the contributor sends this contribution to the

project. Since open contribution projects usually receive

contributions from many people, intellectual property in-

fringements are a substantial concern [11]. All seven projects

that we studied acknowledge this risk and have specific poli-

cies for their submission process that guarantee traceability.

In LINUX, community contributions must contain a “signed-

off” record that acts as a substitute for an electronic signa-

ture. ECLIPSE, FEDORA, MySQL and APACHE completely

disallow contributions through mailing lists. Instead, they

require a submission to be carried out formally by opening

a new issue in their bug tracking systems. The ANDROIDproject goes one step further and uses a separate system on

top of the distributed version control system, called GER-RIT. This system automatically collects all submissions to

the project repositories and notifies project members of the

contribution. GERRIT was built as a successor to the highly

popular Mondrian tool, internally used by Google, and was

built to handle the complete contribution management pro-

cess (steps two to five in the conceptual model) through a

web-based interface.

2.3 Review

After a submission has been formally submitted for con-

sideration, senior members of the project are notified. All

seven projects require a formal peer review to be carried

out for every community contribution. The goal of this peer

review is to determine the fit of the contribution for the

project and to ensure that contributions meet the quality

standards of the project. Reviewers are either appointed

automatically (e.g., ANDROID), or step up voluntarily (e.g.,

LINUX). Reviewers then do an inspection of the contributed

code and judge the overall quality and fit of the submission.

If there are concerns with the contribution, reviewers can

give feedback to the contributor, ranging from high-level

comments down to code line-by-line comments. To enable

the latter, the ANDROID project built its own dedicated re-

view system as a web application. Most other projects use

issue tracking systems or more low-tech technologies such as

email to support the review process. There are three po-

tential outcomes of the review: the contribution is accepted

as-is, the contribution needs to be reworked, or the contri-

bution is rejected. If necessary the contributor should then

address any concerns raised and resubmit an updated revi-

sion of the contribution (but sometimes it is abandoned and

never reworked).

2

9

What is an effective process to enable and manage community contributions?

Are contributors actively engaged in the development?

Are community contributions reviewed in a timely fashion?

What kinds of contributions can be expected – and which parts of the software product do they target?

Research Questions:

10

Reviewer59%

Contributor41%

Reviewer44%

Contributor56%

LINUX ANDROID

10

Reviewer59%

Contributor41%

Reviewer44%

Contributor56%

LINUX ANDROID

~ 2 contributions

per person

~2-4 contributions

per person

10

Reviewer59%

Contributor41%

Reviewer44%

Contributor56%

LINUX ANDROID

~ 2 contributions

per person

~2-4 contributions

per person

avg. activity ~2 months

avg. activity ~2 months

11

What is an effective process to enable and manage community contributions?

Are contributors actively engaged in the development?

Are community contributions reviewed in a timely fashion?

What kinds of contributions can be expected – and which parts of the software product do they target?

Research Questions:

12

Time until a first response to a contribution is received

LINUX (2009)

Date contribution was submitted

Tim

e un

til fi

rst r

eply

(in

hour

s)

J Ma J No

13

13

400% growth of community

13

400% growth of community

2x submissions per contributor

13

Mailing-Liststo manage

contributions

400% growth of community

2x submissions per contributor

14

14

Googleactively decidedagainst E-Mail!

14

Googleactively decidedagainst E-Mail!

Instead:web-application

(Gerrit)to manage

contributions

15

“Weʼre now responding to [Android] platform contributions faster, with most changes currently getting looked at within a few business days of being uploaded, and few changes staying inactive for more than a few weeks at a time.

Weʼre trying to review early and review often. [...]

I hope that the speedy process will lead to more interactivity during the code reviews.”

Jean-Baptiste QueruOpen-Source Management Android

16

Time until a first response to a contribution is received

ANDROID

Date contribution was submitted

Tim

e un

til fi

rst r

eply

(in

hour

s)

Ma A No F Ma

17

LINUX (2009) ANDROID

Time until a final conclusion to a contribution is obtained

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0Overall time taken for review (log hours)

Dens

ity E

stim

ate

REJECT

ACCEPT

0.00

0.05

0.10

0.15

0 5Overall time taken for review (log hours)

Dens

ity E

stim

ate

REJECT

ACCEPT

18

What is an effective process to enable and manage community contributions?

Are contributors actively engaged in the development?

Are community contributions reviewed in a timely fashion?

What kinds of contributions can be expected – and which parts of the software product do they target?

Research Questions:

19

(1) Pareto Distribution:20% of subsystems received 80% of contributions in both projects.

20

(2) Major Subsystems have high acceptance rates:

Between 50% and 91% of contributions are accepted in both projects.

21

(3) Specific Subsystems have very low acceptance rates:

Certain subsystems in Android are very sensitive and rather kept private.

22

TO SUMMARIZE:

• Business Model based on Product Halo • Contribution management model• Top priority: keep users happy• Management of contributions important• Give fast feedback• Let them know of acceptance right away• Users contribute to ‘favourite’ subsystems• For commercial OSS some subsystems ‘off

limits’

23

24

2 Principles:

“The worth of a company is proportional to the number of connected users”

Metacalfe’s Law.

“As more people get involved in a community, participation begets more participation”

Bass’ diffusion model.

Why having user communities is desirable