openstack
/
requirements
Code Issues Proposed changes

updated requirements repo readme 

The purpose of the requirements repository, it's history, and
consistent review guidelines are something that's currently lore
passed down through the ages.

We should write this stuff down, and come up with consistent
guidelines for submitters and reviewers to get us all onto a single
page wrt reviewing and acceptance criteria.

Change-Id: I72f0f5771fee637d0b651d76217ce3959a4b92b7
This commit is contained in:
Sean Dague 2014-08-25 15:34:54 -04:00
parent 3d28e020d0
commit 12d999e49c
1 changed files with 206 additions and 19 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

225
README.rst
View File

@ -1,25 +1,212 @@
Global dependencies for OpenStack Projects
============================================
Global Requirements for OpenStack Projects
============================================
All official OpenStack projects should be added to
projects.txt. Once they are added here the gate/check
jobs for the projects will use the OpenStack
internal pypi mirror to ensure stability. The
continuous integration infrastructure will also
sync up the requirements across all the official
projects and will create reviews in the participating
projects for any mis-matches.
Why Global Requirements?
========================
This process above will ensure that users of OpenStack
will have one single set of python package requirements/
dependencies to install and run the individual OpenStack
components.
During the Havana release cycle we kept running into coherency issues
with trying to install all the OpenStack components into a single
environment. The issue is that syncing of ``requirements.txt`` between
projects was an eventually consistent problem. Some projects would
update quickly, others would not. We'd never have the same versions
specified as requirements between packages.
To use this, run:
Because of the way that python package installation with pip works,
this means that if you get lucky you'll end up with a working
system. If you don't you can easily break all of OpenStack on a
requirements update.
An example of how bad this had gotten is that python-keystoneclient
would typically be installed / uninstalled 6 times during the course
of a devstack gate run during Havana. If the last version of python
keystoneclient happened to be incompatible with some piece of
OpenStack a very hard to diagnose break occurs.
We also had an issue with projects adding dependencies of python
libraries without thinking through the long term implications of those
libraries. Is the library actively maintained? Is the library of a
compatible license? Does the library duplicate the function of existing
libraries that we already have in requirements? Is the library python
3 compatible? Is the library something that already exists in Linux
Distros that we target (Ubuntu / Fedora). The answer to many of these
questions was no.
Global requirements gives us a single place where we can evaluate
these things so that we can make a global decision for OpenStack on
the suitability of the library.
Solution
========
The mechanics of the solution are relatively simple. We maintain a
central list of all the requirements (``global-requirements.txt``)
that are allowed in OpenStack projects. This is enforced for both
``requirements.txt`` and ``test-requirements.txt``.
Enforcement for Test Runs
-------------------------
When installing with devstack, we overwrite the ``requirements.txt``
and ``test-requirements.txt`` for **all** installed projects with the
versions from ``global-requirements.txt``. This ensures that we will
get a deterministic set of requirements installed in the test system,
and it won't be a guessing game based on the last piece of software to
be installed.
Enforcement in Projects
-----------------------
All projects that have accepted the requirements contract (as listed
in ``projects.txt``) are expected to run a requirements compatibility
job that ensures that they can not change any lines in global
requirements to versions not in ``global-requirements.txt``. It also
ensures that those projects can't add a requirement that's not already
in ``global-requirements.txt``.
Automatic Sync of Accepted Requirements
---------------------------------------
If an updated requirement is proposed to OpenStack and accepted to
``global-requirements.txt``, the system then also automatically pushes
a review request for the new requirements definition to the projects
that include it.
For instance: if a review is accepted to ``global-requirements.txt``
that increases the minimum version of python-keystoneclient, the
system will submit patches to all the OpenStack projects that list
python-keystoneclient as a requirement or test requirement to match
this new version definition.
This is intended as a time saving device for projects, as they can
fast approve requirements syncs and not have to manually worry about
whether or not they are up to date with the global definition.
Running
=======
To run the requirements update manually, run::
python update.py path/to/project
Entries in requirements.txt and test-requirements.txt
will have their versions updated to match the entires
listed here. Any entries in the target project which
do not first exist here will be removed. No entries
will be added.
Entries in requirements.txt and test-requirements.txt will have their
versions updated to match the entries listed here. Any entries in the
target project which do not first exist here will be removed. No
entries will be added.
Review Guidelines
=================
There are a set of questions that every reviewer should ask on any
proposed requirements change (and ones that proposers should pre
answer to make things go smoother).
General Review Criteria
-----------------------
- No specifications for library versions should contain version caps
As a community we value early feedback of broken upstream
requirements, so version caps should be avoided except when dealing
with exceptionally unstable libaries.
If a library is exceptionally unstable, we should also be
considering whether we want to replace it over time with one that
*is* stable, or to contribute to the upstream community to help
stabilize it.
- Libraries should contain a sensible known working minimum version
Bare library names are bad. If it's unknown what a working minimum
is, look at the output of pip freeze at the end of a successful
devstack/tempest run and use that version. At least that's known to
be working now.
- Commit message should refer to consuming projects(s)
Preferably, the comments should also identify which feature or
blueprint requires the new specification. Ideally, changes should
already be proposed, so that its use can be seen.
For new Requirements
--------------------
- Is the library actively maintained?
We *really* want some indication that the library is something we
can get support on if we or our users find a bug, and that we
don't have to take over and fork the library.
Pointers to recent activity upstream and a consistent release model
are appreciated.
- Is the library good code?
It's expected, before just telling everyone to download arbitrary 3rd
party code from the internet, that the submitter has taken a deep dive
into the code to get a feel on whether this code seems solid enough
to depend on. That includes ensuring the upstream code has some
reasonable testing baked in.
- Is the library python 3 compatible?
OpenStack will eventually need to support python 3. At this point
adding non python 3 compatible libraries should only be done under
*extreme* need. It should be considered a very big exception.
- Is the library license compatible?
Preferably Apache2, BSD, MIT licensed. LGPL is ok. GPL or AGPL is
verboten. Any other oddball license should be rejected.
- Is the library already packaged in the distros we target (Ubuntu
latest / Fedora latest)?
By adding something to OpenStack ``global-requirements.txt`` we are
basically demanding that Linux Distros package this for the next
release of OpenStack. If they already have, great. If not, we should
be cautious of adding it. :ref:`finding-distro-status`
- Is the function of this library already covered by other libraries
in ``global-requirements.txt``?
Everyone has their own pet libraries that they like to use, but we
do not need 3 different request mocking libraries in OpenStack.
If this new requirement is about replacing an existing library with
one that's better suited for our needs, then we also need the
transition plan to drop the old library in a reasonable amount of
time.
For Upgrading Requirements Versions
-----------------------------------
- Why is it impossible to use the current version definition?
Everyone likes everyone else to use the latest version of their
code. However, deployers really don't like to be constantly updating
things. Unless it's actually **impossible** to use the minimum
version specified in ``global-requirements.txt``, it should not be
changed.
Leave that decision to deployers and distros.
.. _finding-distro-status:
Finding Distro Status
---------------------
From the OpenStack distro support policy:
OpenStack will target its development efforts to latest Ubuntu/Fedora,
but will not introduce any changes that would make it impossible to
run on the latest Ubuntu LTS or latest RHEL.
As such we really need to know what the current state of packaging is
on these platforms (and ideally Debian and SuSE as well).
For people unfamiliar with Linux Distro packaging you can use the
following tools to search for packages:
- Ubuntu - http://packages.ubuntu.com/
- Fedora - https://apps.fedoraproject.org/packages/