Making your first contribution to OpenStack

Julie Pichon, EuroPython, July 22nd 2014

Etherpad: http://tinyurl.com/ep-openstack-etherpad

Slides: http://tinyurl.com/ep-openstack

The Cloud

http://en.wikipedia.org/wiki/OpenStack#Components

Compute (Nova) Object Storage (Swift)

Image Service (Glance) Identity Service (Keystone)

Dashboard (Horizon) Block Storage (Cinder)

Networking (Neutron) Orchestration (Heat)

Telemetry (Ceilometer) Database (Trove)

http://en.wikipedia.org/wiki/OpenStack#Components

Compute (Nova) Object Storage (Swift)

Image Service (Glance) Identity Service (Keystone)

Dashboard (Horizon) Block Storage (Cinder)

Networking (Neutron) Orchestration (Heat)

Telemetry (Ceilometer) Database (Trove)

http://en.wikipedia.org/wiki/OpenStack#Components

Compute (Nova) Object Storage (Swift)

Image Service (Glance) Identity Service (Keystone)

Dashboard (Horizon) Block Storage (Cinder)

Networking (Neutron) Orchestration (Heat)

Telemetry (Ceilometer) Database (Trove)

http://en.wikipedia.org/wiki/OpenStack#Components

Compute (Nova) Object Storage (Swift)

Image Service (Glance) Identity Service (Keystone)

Dashboard (Horizon) Block Storage (Cinder)

Networking (Neutron) Orchestration (Heat)

Telemetry (Ceilometer) Database (Trove)

http://en.wikipedia.org/wiki/OpenStack#Components

Compute (Nova) Object Storage (Swift)

Image Service (Glance) Identity Service (Keystone)

Dashboard (Horizon) Block Storage (Cinder)

Networking (Neutron)(as Quantum)

Orchestration (Heat)

Telemetry (Ceilometer) Database (Trove)

http://en.wikipedia.org/wiki/OpenStack#Components

Compute (Nova) Object Storage (Swift)

Image Service (Glance) Identity Service (Keystone)

Dashboard (Horizon) Block Storage (Cinder)

Networking (Neutron) Orchestration (Heat)

Telemetry (Ceilometer) Database (Trove)

http://en.wikipedia.org/wiki/OpenStack#Components

Compute (Nova) Object Storage (Swift)

Image Service (Glance) Identity Service (Keystone)

Dashboard (Horizon) Block Storage (Cinder)

Networking (Neutron) Orchestration (Heat)

Telemetry (Ceilometer) Database (Trove)

http://en.wikipedia.org/wiki/OpenStack#Components

Compute (Nova) Object Storage (Swift)

Image Service (Glance) Identity Service (Keystone)

Dashboard (Horizon) Block Storage (Cinder)

Networking (Neutron) Orchestration (Heat)

Telemetry (Ceilometer) Database (Trove)

Meta:

● Documentation● Quality Assurance (Tempest)● Infrastructure (not today)

Also:

● Clients

Nova (Compute)

$ nova list

Helpful trick: $ nova --debug list

python-novaclient

REST

APIs

$ curl -i 'http://my_ip:5000/v2.0/tokens' -X POST -H "Accept: application/json" -H "Content-Type: application/json" -d '{"auth": {"tenantName": "demo", "passwordCredentials": {"username": "demo", "password": "123456"}}}'$ curl -i 'http://my_ip:8774/v2/project_id/servers/detail' -X GET -H "Accept: application/json"-H "X-Auth-Project-Id: demo" -H "X-Auth-Token: my_token"

Keystone (Identity)

REST

APIs

Session objectives● Have all the necessary accounts and tools set

up for contributing● Know how to submit a contribution to an

OpenStack component● Understand the OpenStack contribution

process● Have a first contribution completed or

underway, and know what are the next steps

https://wiki.openstack.org/wiki/How_To_Contribute

Also: http://docs.openstack.org/infra/manual/developers.html

Setting up (one time only)

● Launchpad (bug tracking, single sign-on)● The OpenStack Foundation (voting)● Gerrit and the Contributor Licence Agreement

(reviews, submitting patches)● Keypair (so you can upload your patches to

Gerrit)● Basic check: git review -s

Protip: Use the same email for all these accounts

contributor

Launchpad

sign in

Gerrit

sign up

upload keypair OpenStackFoundation

sign up

compare email addresses to confirm membership

Extra reading: https://wiki.openstack.org/wiki/GerritWorkflow

● Launchpad– https://launchpad.net/ -> Register -> Create account

● The OpenStack Foundation– https://www.openstack.org/join/ (Use the same email as LP)

– “Affiliation:” None (unless you're here for your company!)

– “Statement of interest:” e.g. “I want to contribute code”, “I want to improve the documentation”, ...

● Gerrit and the CLA– https://review.openstack.org (Sign in with LP account)

– Sign the Individual CLA (ICLA)

● Keypair– Upload at https://review.openstack.org/#/settings/ssh-keys

Bonus levelSimple check (gerrit account + keypair) with git review:

● Clone a repo from https://github.com/openstack or cd to /opt/stack (DevStack)● sudo pip install git-review● git review -s

DevStack tips and tricks● What is DevStack?

– Best tested on Ubuntu 12.04 and Fedora 20

– Extra reading: http://devstack.org/overview.html

● Always in a VM● Good for testing and actually seeing● Not a requirement for contributing though

– /opt/stack/

– https://github.com/openstack/

● Need help troubleshooting? Ask for help on IRC.– #openstack-101, #openstack-dev (Freenode)

– https://wiki.openstack.org/wiki/IRC

Tip #1: Use the local.conf file

$ cat devstack/local.conf

[[local|localrc]]

OS_PASSWORD=123456

ADMIN_PASSWORD=123456

MYSQL_PASSWORD=123456

RABBIT_PASSWORD=123456

SERVICE_PASSWORD=$ADMIN_PASSWORD

SERVICE_TOKEN=123456

# If you need the latest version

RECLONE=yes

# Can be handy to help debug stack.sh failures

VERBOSE=True

# Neutron (naming based on Quantum)

disable_service n­net

enable_service q­svc q­agt q­dhcp q­l3 q­meta quantum q­lbaas

(See http://devstack.org/configuration.html for even more options)

Tip #2: On using screen

● http://www.gnu.org/software/screen/manual/screen.html#Default-Key-Bindings

● screen -r, screen -ls● Detaching the screen session: C-a, C-d● Navigating: C-a “● (C-a ?)

Applied screen: restarting a service

Example: Imagining you made a change in /opt/stack/keystone

screen ­r

C­a “ # Choose “key” for keystone

C­c # To kill the service

Press up­arrow + Enter, to restart the last command

C­a C­d # To leave

Tip #3: openrc files and communicating with OpenStack

$ nova list

$ source ~/devstack/openrc # credentials

$ nova list

$ nova help

$ keystone user­list

$ source ~/devstack/openrc admin

$ openstack ­­help # New common client

A simple openrc example

$ cat ./example_openrc

export OS_USERNAME=username

export OS_PASSWORD=password

export OS_TENANT_NAME=projectName

export OS_AUTH_URL=https://identityHost:portNumber/v2.0

See http://docs.openstack.org/user-guide/content/cli_openrc.html#openrc-create

Getting started

Soft skills

“Assume good faith.”

Getting started

Culture

● Friendly community● Reviewer vs. Committer team

Getting started

Process● +1, -1

Everyone is welcome and encouraged to participate in reviews

● +2, -2Core reviewers team

● Continuous Integration (CI) gets a vote too● Unit tests, integration tests, style

● To get your patch merged:● +1 from CI● +2 from two cores (+A)

the contributor guidelines

Read

Usethe bug tracker

the contributor guidelines

Buildthe software

Findthe bug

Fixit!

Submityour patch

The Open-Source contribution process

Sources

● Code:– /opt/stack (DevStack)

– github.com/openstack

● Docs:– github.com/openstack-manuals– https://wiki.openstack.org/wiki/Documentation/HowTo/FirstTimers

● QA/Integration tests:– github.com/openstack/tempest (DevStack required)

– http://docs.openstack.org/developer/tempest/

How to choose a bug?

● Low hanging fruits– e.g. https://bugs.launchpad.net/horizon/+bugs?field.tag=low-hanging-fruit

● Browsing the bug tracker– e.g. https://bugs.launchpad.net/neutron

● Sometimes a project has additional docs– e.g. https://wiki.openstack.org/wiki/NeutronStarterBugs

● Launchpad tips and tricks:

– Low priority bugs can sometimes be smaller (not always true)

– Advanced search: by tag (low-hanging-fruits)

– Advanced search: Assignee -> “Nobody”

– Advanced search: Status New/Confirmed/Triaged

– Find something that piques your interest or touches an area of knowledge

Launchpad

● run_tests.sh often has a coverage flag– usually ./run_tests.sh ­­coverage

– otherwise: ./run_tests.sh ­­help

● Reports should appear in a new ./reports/ directory

● Unit tests and additional non-Python requirements: have a look at the README file– e.g. for Nova: python-dev libssl-dev python-pip git-core libxml2-

dev libxslt-dev pkg-config libffi-dev libpq-dev libmysqlclient-dev

How to choose a bug?Adding a unit test

● Find and fix a typo!

● Main docs: guides, API references, ...– http://docs.openstack.org/

– Sources: https://github.com/openstack/openstack-manuals

– More info on how to build the docs: https://wiki.openstack.org/wiki/Documentation/HowTo

● Developer docs: within the code repositories– e.g. http://docs.openstack.org/developer/heat/ is generated from

/doc/ at https://github.com/openstack/heat/

– Find a typo, fix Sphinx syntax errors... (http://sphinx-doc.org/)

– Usually can build the docs with e.g. “tox -e docs” (pip install tox first)

How to choose a bug?Documentation patch

● Often out of date or copy-paste errors● Projects: python-<codename>client● Where? Usually:

– python-<codename>client/<codename>client/<version>/

– Pick a resource type and read the file!

● Correct the entire file

How to choose a bug?Client doc strings

(https://review.openstack.org/#/c/69376/)

commit ea8c9554c947b7fd30adfb1249a9317f258901ab

Author: Julie Pichon <jpichon@redhat.com>

Date:   Mon Jan 27 14:28:28 2014 +0000

    Fix typos in the volumes and snapshots docstrings

    

    Change­Id: I0e4f3813b8e263ea59e182db153c5b4c6fd2a177

diff ­­git a/cinderclient/v2/volumes.py b/cinderclient/v2/volumes.py

index 524456d..7f77bb9 100644

­­­ a/cinderclient/v2/volumes.py

+++ b/cinderclient/v2/volumes.py

@@ ­187,7 +187,7 @@ class VolumeManager(base.ManagerWithFind):

     def get(self, volume_id):

         """Get a volume.

 

­        :param volume_id: The ID of the volume to delete.

+        :param volume_id: The ID of the volume to get.

         :rtype: :class:`Volume`

         """

         return self._get("/volumes/%s" % volume_id, "volume")

@@ ­225,7 +225,7 @@ class VolumeManager(base.ManagerWithFind):

     def update(self, volume, **kwargs):

         """Update the name or description for a volume.

 

­        :param volume: The :class:`Volume` to delete.

+        :param volume: The :class:`Volume` to update.

         """

         if not kwargs:

             return

● https://wiki.openstack.org/wiki/How_To_Contribute ● Help with translations, break things and file bugs (DevStack),

do reviews, ...● Bug triage – This can help you find a task!

– See the excellent blog post, “A path towards contributing (via commits) in OpenStack” http://www.mattfischer.com/blog/?p=602

– Bugs in “NEW” state: confirm if can reproduce (DevStack)● Yes? Set status to CONFIRMED● No? Ask questions, ask for clearer steps to reproduce, optionally mark

“INCOMPLETE”

– http://wiki.openstack.org/BugTriage

How to choose a bug?Other tasks

Now

● This should cover most non-Python dependencies:– sudo apt-get install python-dev libssl-dev libxml2-dev libxslt-dev

pkg-config libffi-dev libpq-dev libmysqlclient-dev– sudo yum install python-devel openssl-devel gcc libxslt-devel

mysql-devel postgresql-devel libffi-devel

● Choose a project● Go into the repository and start ./run_tests.sh

This will create the virtual environment in .venv and download the required dependencies● Short Break

Find a task and work on it

● Browse the Launchpad bug tracker (low hanging fruit or other) – maybe better for sprints

● Increase the test coverage by adding a unit test● Improve the docs● Fix a client's doc strings● Confirm bugs with DevStack● https://wiki.openstack.org/wiki/How_To_Contribute

● (http://tinyurl.com/ep-openstack )

Formatting your patch to community standards

● See “Summary of GIT commit message structure” in wiki

https://wiki.openstack.org/wiki/GitCommitMessages#Summary_of_GIT_commit_message_structure

● Style guidelines: http://docs.openstack.org/developer/hacking/

● Check no pep8/hacking errors:

– ./run_tests.sh -p● Check the tests pass:

– ./run_tests.sh

Switch libvirt get_cpu_info method over to use config APIs

The get_cpu_info method in the libvirt driver currently uses XPath queries to extract information from the capabilities XML document. Switch this over to use the new config class LibvirtConfigCaps. Also provide a test case to validate the data being returned.

DocImpact

Closes­Bug: #1003373

Implements: blueprint libvirt­xml­cpu­model

Change­Id: I4946a16d27f712ae2adf8441ce78e6c0bb0bb657

Submitting a patch

● Preparing your patch

– Git commit message structure, pep8, unit tests● sudo pip install git-review● In the repository:

– git review -s● To add a ChangeId to your existing commit:

– git commit --amend– (exit editor, no need to change anything)

git-review

● For the cautious:– git log -n1 # check your commit has a ChangeId– git review -n # dry run, doesn't push anything out yet

● git review # sends your patch to Gerrit

● “The Gate”– Unit tests, Integration tests, style checks

$ git review ­n

Please use the following command to send your commits to review:

    git push gerrit HEAD:refs/publish/master/bug/1324982

$  git review

remote: Resolving deltas: 100% (2/2)

remote: Processing changes: new: 1, refs: 1, done    

remote: 

remote: New Changes:

remote:   https://review.openstack.org/96815

remote: 

To ssh://jpichon@review.openstack.org:29418/openstack/horizon.git

 * [new branch]      HEAD ­> refs/publish/master/bug/1324982

Gate jobs

git review bonus commands

● git review can help you do reviews, too– git review -l

– git review -d <review_id>

The Next Steps

● Your patch will get reviewed, be patient● You can help by doing reviews of other patches yourself

– Learn about the project's culture and structure

● Where to ask for help?– IRC: #openstack, #openstack-dev, #openstack-101

● https://wiki.openstack.org/wiki/IRC

– Mailing list: openstack-dev@lists.openstack.org (filters!)● https://wiki.openstack.org/wiki/MailingLists

– http://ask.openstack.org/– Don't stay stuck! Ask for help on IRC.

What to do next?

● EuroPython sprints● Re-read https://wiki.openstack.org/wiki/How_To_Contribute● Find a slightly harder bug. After resolving one, try to leave the remaining low

hanging fruits for the next newcomer.● Review, triage, translate... Lots of ways to help!● Respond to review feedback, if you receive any● Join the mailing list (filters, filters), idle on IRC, become part of the

community● Summit! Conference + Design Summit. Participate in the conversation.

– Next one in November in Paris

– Free ticket for “ATC”

– http://www.openstack.org/summit/

What did we learn?

● What accounts are required and why● The OpenStack development process● Glimpse into the OpenStack developer toolbox

– DevStack, Gerrit, Launchpad

● Where/How you can help● The Life of a Contribution● Where to ask for help

● “Next action” - plan it now– Take a couple of minutes to figure out what you want to do next

and write it down

Questions?

Feedback form: http://tinyurl.com/ep-openstack-survey