27 au 29 mars 2013

Leading a Community-Driven open source project

Vincent MassolXWiki CommitterCTO XWiki SAS

Vincent Massol, June 2017

(When a company is behind the project)

Vincent Massol

• Speaker Bio

• Committer XWiki & CTO XWiki SAS

• Projects

• XWiki (community-driven open source project) - 11y

• Past: Maven, Apache Cargo, Apache Cactus, Pattern Testing - 6y

• Other Credentials:

• LesCastCodeurs podcast about Java news

• Creator of OSSGTP open source group in Paris

• 3 books: JUnit in Action, Maven: A Developer’s Notebook, BBWM

Agenda

• The XWiki project

• Communication

• Build

• Continuous Integration

• Tests

• XWiki Days

• Governance & Marketing

• Roadmap Process

• Release Process

• Committers Diversity

• Increasing Contributions

• Measuring Progress

• Future

What is XWiki? (1/2)• A structured open source enterprise wiki

What is XWiki? (2/2)

• A platform for developing content-based web applications

Project Stats

Source: http://dev.xwiki.org/xwiki/bin/view/Community/ProjectHealth

Mailing lists activity

Communication

• Dev mailing lists (devs, notifs)

• ~8/day

• Forum (Discourse) - New

• ~10/day

• IRC/Matrix for discussions

• But all important messages must go through the list

• Commit emails + JIRAs + daily wiki activity on the notifications list

• Code reviews (not systematic)Source: http://dev.xwiki.org/xwiki/bin/view/Community/MailingLists & http://dev.xwiki.org/xwiki/bin/view/Community/Chat

Build

• Maven-based with several custom plugins

• Active Quality vs Passive Quality. Examples:

• Checkstyle with additional custom rules

• Verify that Script Services are not located in the internal package

• Verify that @since javadoc tags have the correct format

• Verify header licenses

• Backward compatibility checks with Revapi

• Enforcer checks

• Verify we don't use Commons Logging or Log4j (since we use SLF4J)

• Verify that Test Percentage Coverage don't go downSource: http://dev.xwiki.org/xwiki/bin/view/Community/Building

Continuous Integration• Some jobs with -Pquality profile to perform

Quality checks

• Catching false positives before sending mail. Examples:

• JVM crash

• GitHub connection issue

• X Display not ready for UI tests

• Display screenshot of failing test in job report

• Started using Jenkinsfile for xwiki-contrib

• Pipeline Job to generate full TPC

Source: http://ci.xwiki.org/

Tests (1/3)

• Unit tests: JUnit + Mockito

• Integration tests with automatic component testing and injection. Custom JUnit @Rule

• Functional UI tests with JUnit + Selenium/WebDriver

• Page Object strategy

• WCAG tests

• HTML5 validity tests

• Manual tests (performance + QA @ XWiki SAS

• Total TPC = 73.2%Source: http://dev.xwiki.org/xwiki/bin/view/Community/Testing

Tests (2/3) - Manual Tests

Source: http://test.xwiki.org

Tests (3/3) - Manual Tests

XWiki Days

• Every Thursday

• Examples

• Bug Fixing Day (BFD)

• Test Improvement Day

• Deprecation Fixing Day

• Improvement Fixing Day

• Doc Improvement Day

• Pull Request Day

• Worked really well for BFD

Source: http://dev.xwiki.org/xwiki/bin/view/Community/XWikiDays

Governance - General

• Complete separation from XWiki SAS since 2005-2006

• Employees are not committers

• xwiki.org has its own governance rules (similar to the ASF)

• Committership

• Voting: +1, 0, -1 (72 hours)

• Only 1 non-XWiki SAS committer needed to guarantee honesty!

• Lazy consensus

• All decisions done by committers and community (non-binding)

Governance - Marketing (1/2)

• Governance for website

• Question: How to reconcile XWiki SAS need for marketing and keep it fair?

• Sponsoring Companies

• At least 1 active committer

• Active = at least 1 commit / year

• Top Sponsoring Company (TSC) = company with the most active committers (= XWiki SAS)

Source: http://dev.xwiki.org/xwiki/bin/view/Community/Governance

Governance - Marketing (2/2)

• Marketing allowed on:

• Download page

• Support page (for professional support)

• TSC allowed to gather emails and redistribute to other SC (if >= 3 active committers)

Source: http://dev.xwiki.org/xwiki/bin/view/Community/Governance

• SC can have their Extensions repo by default in XWiki is >= 3 active committers

• TSC can advertise on extensions.xwiki.org

Roadmap

• Time-boxed releases every month (3w RC and 1w final)Source: http://www.xwiki.org/xwiki/bin/view/Roadmaps/

Release Process• Ongoing Release Notes and

reference documentation

• Marked in JIRA with 2 custom fields

• Rolling Release Managers

Source: http://dev.xwiki.org/xwiki/bin/view/ReleasePlans/

• Create Release Plan for the release

Release Process - Plan (1/2)

Source: http://dev.xwiki.org/xwiki/bin/view/ReleasePlans/

Release Process - Plans (2/2)• Release in JIRA

• Check that all issues are documented

• Check Release Notes

• Import translations

• Build the Release

• Create mail announcement

• Push to Maven Central

• Update Docker official image

• etcSource: http://dev.xwiki.org/xwiki/bin/view/ReleasePlans/ReleasePlan845

Committers Diversity• XWiki SAS: 38

• Core Committers: 17 (11 actives)

• Contrib Committers: 69 (17 from XWiki SAS, 22 active)

• 64% of Core Committers are from XWiki SAS

• But 90%+ of code committed by XWiki SAS devs

• Problem: XWiki SAS hires contributors…

XWiki SAS Contrib Committers

CoreCommitters

21 6 11 6 46

Why Community-Driven for XWiki?• In 2005: Copied from ASF, Community over Code

• But community-driven model under attack

• Not enough contributions, 90%+ of code from XWiki SAS

• Creates separation in company (those working in OSS vs others)

• Why continue?

• Love of open community by founders

• Ethics: “real open source”

• Senior developer retention & HR <— over 8 years in dev team

• Still getting some contributions (translations, support help, extensions, etc)

• Increased usage? Having a deciding community means that even if company goes away the project can live on?

• We want other companies and individuals to make a living on XWiki!

Increasing Contributions (1/2)

• Idea 1: Slim down the core into modules and extensions

• Concept of Extensions in XWiki, that can be installed at runtime

• Moved from ‘xwiki’ GitHub org to ‘xwiki-contrib’ and still going on

• Anyone given commit permissions on ‘xwiki-contrib’

• Idea 2: XWiki SAS hires more devs

• Idea 3: Good dev documentation

• Idea 4: Feeling of welcome and good support on IRC + Forums

Increasing Contributions (2/2)

• Idea 5: Make it easy to perform some type of contributions, e.g. http://l10n.xwiki.org for translations

• Idea 6: Why would people contribute when they just need to ask and wait a bit to see it done?

• Idea 7: If some devs work too fast then occasional contributors cannot follow up

Not easy. Maybe it’s just not possible when there’s a company paying devs to work on project?

Measuring Progress - Active Installs

Source: http://dev.xwiki.org/xwiki/bin/view/Community/ProjectHealth

• Opt out mechanism

• Send anonymous data (no IP recorded)

• Send XWiki version, Java version, Servlet Container, Installed Extensions & versions, Distributions

Future• XWiki Foundation to officially separate open source project from XWiki SAS

company

• CLA Discussion (ex Amazon)

• CI functional tests executing on various environments inside Docker containers

• Work on reducing flaky tests number

• Automate performance tests

• Unbreakable build with automatic merge on Release branch when CI jobs passes (using Pipeline)

• Maybe more Paying Apps (XWiki SAS). Too many users don’t play the open source game… Idea:

• Open source

• Free for anyone who shows a contribution to OSS

Q&A

Me