CommunityMetricsby Stackalytics

Ilya ShakhatHerman Narkaytis

H

● Processing of commits and reviews, easy expandable to other types of contribution

● Contribution statistics must be sliced by release, project, company, engineer

● Dataset must be refreshed several times a day ● Every API query must be within a second● Every deployment show the same results

Design Goals

I

● Contributions stats on commits, LOCs, reviews, patch sets, emails, filed and resolved bugs, drafted and completed blueprints

● Filtering by release, project type, module, company or engineer

● Reports ● JSON API● Crawlable by Google

What is Stackalytics now

I

Data Source

Launchpad Gerrit Mail Archives

blueprints

bugs

change requests

patch sets

reviews

Git

commits emails

Stackalytics

record (type, user_id, module, release, date, company)

I

SELECT SUM(x) FROM table GROUP BY y

1M records on 1 CPU VM from indexed table

● MySQL ~ 30 sec● Mongo > 1 min● Memcached ~ 10 sec

Backend Technologies

H

Architecture

Persistence Layer

git gerrit launchpad mail archives

Back-end

memcached

uwsgi + flask

Front-end

uwsgi + flask

Client-side

js libs

I

● The processor: ○ polls changes from git, gerrit, launchpad and mail

archives○ stores new records or updates existing in memcached○ stores list of record ids that changed

● The dashboard (each uwsgi process):○ reads list of changed record ids○ reads records from memcached○ updates internal caches

Data Processing

I

Email

Affiliation

Gerrit id

User Profile

Launchpad id

I

● Different types of contribution give different pieces:○ commits - emails, name○ reviews - email, gerrit id, name○ blueprints and bugs - launchpad id, name○ mail archives - email, name

● Match pieces:○ retrieve launchpad id by email○ check if there is launchpad id matching gerrit id

● Affiliation is got by domain

Autobuilding a User Profile

I

Needed when:

● User has more than one affiliation● Not all emails are listed in Launchpad profile● Company domain is not in listed in domain list

Add profile into default_data.json!

Custom User Profile

I

Project Types:

● OpenStack - based on the official programs.yaml from governance project - split by integrated, incubated and other

● Stackforge - all under ‘stackforge’ organization

Module Groups:

● Official programs● Custom configured in default_data.json

Project Classification

I

Stackalytics in Numbers

Contribution Summary

OpenStack:

● 100k commits ● 90k change requests● 300k patch sets● 120k “patch looks good”● 90k “patch needs more work”● 40k filed bugs and 25k resolved● 7.7k drafted blueprints and 3.9k completed

+ Stackforge = more than 1.5mln records!

H

Reports

● Shows individual contribution into the project for last month or 3 months

● Used as a hint to filter candidates into core team

Metrics:

● Reviews - shows how active contributor is● Disagreement ratio - low when core’s vote is the same● Commits and mailing activity

Contribution Summary Report

I

● Shows all contributions done by person or company● Shows punch card with usual hours of work● Usually on first page in Google by query <Name>

Activity Report

I

Person-day Effort

● A metric that shows a number of days a person does anything in OpenStack

● Used to see how many people involved on daily basis

In Juno:

● Total: 3.5k● Only 1k work more than 2 weeks per cycle● Only 200 engineers work full-time

+50% since Havana!

H

● A central list of all drivers for OpenStack● Vendors can add info about their drivers, including

description, reference to docs and CI gerrit id ● DriverLog data is used for marketplace● CI votes metric in Stackalytics

Ironic team joined recently!

DriverLogβ

I

DriverLogβ

I

● List of all members registered at openstack.org● Filtered by time of joining and companies● List new companies that join community recently

Summit registration period 3 months ago:

● 3x registration ● +200 companies

All times:

● 17k members from 3.8k companies

Member Directory

I

Member Directory

I

● Goal: introduce KPI for engineers and company● Need easy, very easy way to configure ● What about writing some text in Google Docs?

Define goal statements as:

● number of commits / reviews / bugs / etc.● to achieve position # in the top rating

KPI Report

H

Release Kilo:

Group ceilometer-group:

User dbelova:

- make 100 reviews during 30 days

- make 5 commits during 30 days

- become core in ceilometer

Company Mirantis:

- top 2 by commits

KPI Script Example

H

KPI Report Workflow

1) Create Google Doc, share it

3) Copy&Paste the address

2) Write the script

4) Bookmark Stackalytics addressH

Links

● www.stackalytics.com● Wiki: wiki.openstack.org/wiki/Stackalytics ● API Docs: stackalytics.readthedocs.org

● Stackalytics’ fork for OpenDayLight community: http://spectrometer.opendaylight.org

Links