From be1017fa8bdfee786c6923516d51d95087f5b7f4 Mon Sep 17 00:00:00 2001 From: Tobias Henkel Date: Sun, 9 Dec 2018 11:29:35 +0100 Subject: [PATCH] Add spec for multi ansible version support Currently zuul only supports one specific ansible version at once. This complicates upgrading ansible because ansible often breaks backwards compatibility and so we need to synchronize the upgrade on the complete deployment which is often not possible. Instead we want to support multiple ansible versions at once so we can handle the lifecycle of ansible versions by adding new versions and deprecating old ones. Change-Id: If00c971cec3b90406ec1bf9b37e9fecf38c0c03f Story: 2004572 Task: 28345 --- doc/source/developer/specs/index.rst | 1 + .../specs/multiple-ansible-versions.rst | 157 ++++++++++++++++++ 2 files changed, 158 insertions(+) create mode 100644 doc/source/developer/specs/multiple-ansible-versions.rst diff --git a/doc/source/developer/specs/index.rst b/doc/source/developer/specs/index.rst index 675ce13d2c..846749fd19 100644 --- a/doc/source/developer/specs/index.rst +++ b/doc/source/developer/specs/index.rst @@ -14,3 +14,4 @@ for those changes and help us work on them collaboratively. :maxdepth: 1 container-build-resources + multiple-ansible-versions diff --git a/doc/source/developer/specs/multiple-ansible-versions.rst b/doc/source/developer/specs/multiple-ansible-versions.rst new file mode 100644 index 0000000000..8ff8e2204f --- /dev/null +++ b/doc/source/developer/specs/multiple-ansible-versions.rst @@ -0,0 +1,157 @@ +Multiple Ansible versions +========================= + +.. warning:: This is not authoritative documentation. These features + are not currently available in Zuul. They may change significantly + before final implementation, or may never be fully completed. + +Currently zuul only supports one specific ansible version at once. This +complicates upgrading ansible because ansible often breaks backwards +compatibility and so we need to synchronize the upgrade on the complete +deployment which is often not possible. + +Instead we want to support multiple ansible versions at once so we can handle +the lifecycle of ansible versions by adding new versions and deprecating old +ones. + + +Requirements +------------ + +We want jobs to be able to pick a specific ansible version to run. However as we +have lots of stuff that overrides things in ansible we will let the job only +select a minor version (e.g. 2.6) in a list of supported versions. This is also +necessary from a security point of view so the user cannot pick a specific +bugfix version that is known to contain certain security flaws. Zuul needs to +support a list of specific minor versions from where it will pick the latest +bugfix version or a pinned version if we need to. + +We need virtual environments for ansible which are not relocatable. Because of +this we must install ansible after the installation. Further we need to support +sdist based installations as well as wheel based installations. However wheel +based installations don't have any possibility for post install hooks. Thus +to stay consistent we will require to manually run a post installation script +to install the ansible versions. This script will work standalone as well as +within the executor process context. This will make it possible to let the +executor optionally install ansible at startup time if there is interest. +This can be useful for the quick start. However the recommended way will be the +manual run. + +We also need a configuration that specifies additional packages that need to +be installed along with ansible. This is required because different deployers +use different ansible connections or modules that have additional optional +dependencies (e.g. winrm). + + +Job configuration +----------------- + +There will be a new job attribute ``ansible-version`` that will instruct zuul +to take the specified version. This attribute will be validated against a list +of supported ansible versions that zuul currently can handle. Zuul will throw +a configuration error if a job selects an unsupported or unparsable version. +If no ``ansible-version`` is defined zuul will pick up the ansible version +marked as default. We will also need a mechanism to deprecate ansible versions +to prepare removing old versions. We could add labels to the supported versions. +To express that. The supported versions we will start with will be: + +* 2.5 (deprecated) +* 2.6 +* 2.7 (default) + +The default ansible version will be configurable on global and tenant level. +This way updating the default to newer versions can be made much less +disruptive. Being able to specify the default version on tenant level makes +it easy to canary release ansible updates in a larger multi tenant environment. + +We will also need to be able to pin a version to a specific bugfix version in +case the latest one is known to be broken. This will also be handled by the +installation mechanisms described below. + + +Installing ansible +------------------ + +We currently pull in ansible via the ``requirements.txt``. This will no longer +be sufficient. Instead zuul itself needs to take care of installing the +supported versions into a pre-defined directory structure using the venv module. +The executor will have two new config options: + +* ``ansible-root``: The root path where ansible installations will be found. The + default will be ``/lib/zuul/executor-ansible``. All supported + ansible installations will live inside a venv in the path + ``ansible-root/``. + +* ``manage-ansible``: A boolean flag that tells the executor manage the + installed ansible versions itself. The default will be ``true``. + + If set to ``true`` the executor will install and upgrade all supported + ansible versions on startup. + + If set to ``false`` the executor will validate the presence of all supported + ansible versions on startup and throw an error on missing installations. + +The management of the ansible installations will be performed by a script +installed along the zuul binaries. This will install and update every supported +version of ansible into a specified ``ansible-root`` directory. + +This script +can then be used by the executor or externally depending on the configuration. + + +Dockerized deployment +--------------------- + +In a dockerized deployment it is preferable to pre-install ansible during the +image build. This can be done by calling the installation script during the +image build. The official images will contain all supported versions. + + +Ansible module overrides +------------------------ + +We currently have many ansible module overrides. These may or may not be +targeted to a specific ansible version. Currently they are organized into the +folder ``zuul/ansible``. In order to support multiple ansible versions without +needing to fork everything there this will be reorganized into: + +* ``zuul/ansible/generic``: Overrides and modules valid for all supported + ansible versions. +* ``zuul/ansible/``: Overrides and modules valid for a specific + version. + +If there are overrides that are valid for a range of ansible versions we can +define it in the lowest version and make use of symlinking to the other versions +in order to minimize additional maintenance overhead by not forking an override +where possible. Generally we should strive for having as much as possible in the +generic part to minimize the maintenance effort of these. + + +Deprecation policy +------------------ + +We should handle deprecating and removing supported ansible versions similar to +the deprecation policy described in zuul-jobs: +https://zuul-ci.org/docs/zuul-jobs/policy.html + +Further we should make sure that whenever we release a new version of zuul we +should make sure that the list of supported ansible versions is a subset of +what is supported by ansible at that time. The list of supported ansible +versions can be found here: +https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html#release-status + +We also should notify the users when they use deprecated ansible versions. This +can be done in two ways. First the executor will emit a warning to the logs when +it encounters a job that uses a deprecated ansible version. The executor already +can return warnings together with the build result. These will be added directly +to the reporting to the code review system. This can be used to warn about +deprecated ansible versions at a prominent location instead of burying it +somewhere in megabytes of logs. + + +Testing +------- + +We also have a set of tests that validate the security overrides. We need to +test them for all supported ansible versions. Where needed we also need to fork +or add additional version specific tests.