project-team-guide/doc/source/project-setup/python.rst

title

Python Project Guide

Python Project Guide

The goal of this document is to explain OpenStack wide standard practices around the use of Python.

It describes the use of a Python virtual environment, the install of both system and project-specific dependencies and, finally, running the tests.

Note

Make sure you have installed python3-dev or python-dev before running these tox checks.

Virtual Environment

It is recommended that you use virtualenv to create an isolated Python environment, with no reliance on system packages other than truly global things like Python itself.

virtualenv bundles three other important Python tools – pip, wheel and setuptools. All combined, this will get you a fully up to date development environment going without root access – and without confusing your package manager.

Installing

1. Pick a place to make the virtual enviroment, e.g ~/workspace;
2. Get a copy of virtualenv to run locally from source, according to the virtualenv installation guide. Note that you need to invoke the virtualenv.py script against ~/workspace;
3. Activate your virtualenv.

Installing System Dependencies

Prior to installing the Python packages and running the tests, there are system packages that will need installing in order to allow various Python packages to compile.

bindep project exists to address this need. It is a tool for checking the presence of binary packages needed by an application or library:

$ pip install bindep

If the project you are working on has a bindep.txt, bindep will read system requirements from there. Just call it along with your package manager:

$ sudo [apt-get | yum] install $(bindep -b)

If there is no such file, in order to learn what system dependencies need to be installed, you should look at the documentation of the specific project you are interested in.

Running Python Unit Tests

Before submitting your change, you should test it. Repositories generally have several categories of tests:

• Style Checks -- Check source code for style issues
• Unit Tests -- Self contained in each repository
• Integration Tests -- Require a running OpenStack environment

The tests available and the tools used to implement these tests vary from project to project. This section assumes you have all system dependencies needed by the project installed. It covers how to run the style check and unit tests. Both are run through tox, so you need to install it:

$ sudo pip install tox

Run The Tests

Navigate to the repository's root directory and execute:

$ tox

Note

Completing this command may take a long time, depending on system resources. You might not see any output until tox is complete.

Run One Set of Tests

tox will run your entire test suite in the environments specified in the repository tox.ini:

[tox]

envlist = <list of available environments>

To run just one test suite in envlist execute:

$ tox -e <env>

so for example, run the test suite in py27:

$ tox -e py27

Running the style checks

Just run:

$ tox -e pep8

Run One Test

To run individual tests with tox:

If testr is in tox.ini, for example:

[testenv]

... "python setup.py testr --slowest --testr-args='{posargs}'"

Run individual tests with the following syntax:

$ tox -e <env> -- path.to.module.Class.test

So for example, run the test_memory_unlimited test in openstack/nova:

$ tox -e py27 -- nova.tests.unit.compute.test_claims.ClaimTestCase.test_memory_unlimited

If nose is in tox.ini, for example:

[testenv]

... "nosetests {posargs}"

Run individual tests with the following syntax:

$ tox -e <env> -- --tests path.to.module:Class.test

So for example, run the list test in openstack/swift:

$ tox -e py27 -- --tests test.unit.container.test_backend:TestContainerBroker.test_empty

Debugging Python Unit Tests

You can debug tests with pdb. To begin, insert set_trace() where you wish to break:

import pdb; pdb.set_trace()

If testr is in tox.ini, the testtools.run command should be used to run tests. However, due to a bug, it is not possible to simply pass a regex to this tool. Instead, first generate a list of tests to run and then pipe this list through testtools.run:

$ source .tox/py27/bin/activate
$ testr list-tests test_name_regex > my-list
$ python -m testtools.run discover --load-list my-list

Alternatively, some projects provide a debug in their tox envlist, which is based on oslo_debug_helper. Run individual tests with pdb enabled with the following syntax:

$ tox -e debug -- path.to.module.Class.test