With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.
With that we also update metdata to reflect current state.
Change-Id: Ib390f06177fbac77a4285e3be86e6461fd769597
Currently, rally_upper_constraints_url always points to master branch.
It is not a valid behavior because u-c from master may not work for
stable branches.
This change fixes rally_upper_constraints_url.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/887592
Change-Id: I016e457c0e4b7819d6d65af3bc35e06061f92d1c
At the moment we don't have control what version of rally-openstack
is being installed. So we are installing latest version available in
PyPi. However rally package version is defined in
rally-openstack constraints.
With this patch we add variables to be able to adjust source of
rally-openstack as well as pick version for installation.
We also temporary disable upgrade jobs to resolve circular dependency
Needed-By: https://review.opendev.org/c/openstack/openstack-ansible/+/847272
Change-Id: I8100dfd3544c0530815fb877b44b960d404f635b
Rally ships it's own version of upper-constraints [1].
As we move openstack-ansible to using the new pip resolver this
causes several issues. The rally specific version of u-c contains
constraints for pip/setuptools/wheel which will inevitably be in
conflict with those in global-requirement-pins. This results in
several problems.
* Rally is branchless and so applying branch-specific versions
of openstack u-c is incorrect
* Rally should be installed using the rally u-c rather than
openstack u-c
* Rally should not use the repo server to build wheels as the repo
server wheel build venv will have different versions of pip/
setuptools/wheel to those specificed in rally u-c.
* Rally venv build should not be subject to openstack-ansible
global-requirement-pins as these are in conflict with rally u-c.
This patch introduces the changes necessary to build rally in a self
contained manner on the utility host with no use of the repo server.
It also includes a revert of I73887784cbc3ce6089a42c67871dbc6c3d54acf6.
[1] https://opendev.org/openstack/rally-openstack/raw/branch/master/upper-constraints.txt
Change-Id: Idbd71ade1ab82d2d218e560b2f10c38116ff47e3
Move it to the service setup host (defaults to utility[0]) instead
of the galera[0] host, and use galera_address (defaults to internal VIP)
as the endpoint instead of a local connection on the db host.
Change-Id: I3d7688ea29f1eb4d65dd50ae82d157f988e404af
We had all of the upper-constraints hardwired inside the role
which was not ideal. This patch grabs the latest upper-constraints
as we are installing the latest package in this case.
Change-Id: Icfac74a031d9059d50772d8423c5ba3f6aa00407
This comes out of a latest pip freeze, finding what's not
in upper constraints, after install rally-openstack.
Change-Id: Ib62520f0302efe51fe03e97722c5e8d452a3f7c8
The variables rally_developer_mode and rally_venv_download
no longer carry any meaning. This review changes rally to
do the equivalent of what developer_mode was all the time,
meaning that it always builds the venv and never requires
the repo server, but it will use a repo server when available.
As part of this, we move the source build out of its own file
because it's now a single task to include the venv build role.
This is just to make it easier to follow the code.
Rally doesn't uses upper-constraints. We rely on stable releases,
for that reason, we don't need vars like rally_git_contraints,
rally_git_repo and rally_git_install_branch.
Change-Id: Iad66086bde1680e4a101f75a17700ab83c459124
In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.
We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.
This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:
1. Replaces 'developer mode' with an equivalent mechanism
that uses the common role and is simpler to understand.
We will also simplify the provisioning of pip install
arguments when doing this.
2. Uses Rally's own in-repo upper constraints, instead of
the OpenStack requirements repo constraints.
Depends-On: https://review.openstack.org/598957
Change-Id: I3f70f7879d399c36fb8d145e69ce4a609157ec30
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Given that the openstack_hosts role installs pip and virtualenv,
we do not need this extra task/var.
Change-Id: Id3c2329f0f0b86d88fcd748c7e4504400c942ef2
This is motivated by the recent change of rally
project [1] which states that In-tree OpenStack
plugins will be removed from the Rally main package soon.
Also, we do not have constraint for it, we need to pin it
to keep consistent build.
[1] https://github.com/openstack/rally/blob/master/CHANGELOG.rst#100---2018-06-20
Change-Id: I9121867301fc69ba6623110fa6e54c2da0b5ae2c
Closes-Bug: 1782372
There is no record for why we implement the database creation outside
of the role in the playbook, when we could do it inside the role.
Implementing it inside the role allows us to reduce the quantity of
group_vars duplicated from the role, and allows us to better document
the required variables in the role. The delegation can still be done
as it is done in the playbook too.
In this patch we implement a new variable called 'rally_db_setup_host'
which is used in the role to allow delegation of the database setup
task to any host, but defaults to the first member of the galera_all
host group. We also document the variable 'rally_galera_address' which
has been used for a long time, but never documented. A bunch of unused
variables have also been removed.
Change-Id: Ifb89e35e3acaef33ce4ce24b3f5affc6c4e8e494
Rally does not adhere to the global requirements process, so its
requirements extend beyond those managed by upper-constraints, and
sometimes are incompatible with upper constraints.
We have no need to track their development from git commit to git
commit, and can rather rely on stable releases which we verify are
working with the upper constraints whenever we do a pinning bump.
This keeps things simpler for us, and also allows the repo build
process in the integrated build to do a constrained venv build
instead of doing it unconstrained and breaking stable branch builds
down the line.
Depends-On: https://review.openstack.org/570906
Change-Id: If3d0120cf6f6f78463cf7805bc046382c3b6d44e
The cmd2-0.9.0 is python3-only and the rally venv is built
unconstrained, so it's trying to bring in the latest version
which is incompatible with python2. We pin the library to
ensure that the venv build completes correctly.
We also re-arrange the package list to be alphabetically
sorted.
Depends-On: https://review.openstack.org/570879
Change-Id: I1e259ce98fc8517c0cd00a04d6fe7b3f0703f5b4
virtualenv-tools has a bug which gets triggered in gates: it can't
change the shebang of a virtualenv python bin/ files if they
were generated with a virtualenv script whose shebang ends with
python2 instead of python.
Because we can't modify virtualenv-tools, we use shell scripts
instead.
Change-Id: I438ae40b42bcf61a1be56a5f52581444a9b181a3
Partial-Bug: #1741634
Rally installs ansible, in an unconstrained way.
To pass linters, we skip the lint job on the test
shell command.
Change-Id: Ic830cf81b3cc691aafb7a97819e0ce68e48dd809
Fixes the ability to deploy a venv in cases where:
1) developer_mode is not enabled
2) A cached venv is not downloaded from the repo server
Additional cleanup to the developer_mode venv deployment
logic is implemented by adding a *_venv_download var
which is used to decouple developer_mode from the
cached venv extraction process so that a deployer
can force venv builds in-place (disable cached
venv usage) without enabling developer mode
constraints.
Change-Id: I6939e47455898c07003f1b480e6d65b5d06e4c68
The current constraints generation for the
installation involves multiple tasks and multiple
variables.
Using multiple tasks extends the installation time
unnecessarily and the additional variables are
unnecessary.
This patch aims to simplify the mechanism and
hopes to speed it up a little.
Change-Id: I4633c71c515ab03a4e72ef08e1bb682dedc8c079
It was duplicating the meaning/purpose of the role
variable `rally_role_project_group`
Also ensure that additional tasks are limited to only
a single host in the host group.
Change-Id: I3d5cc822cc0d3c2b0b3ba7b05a9fe1b6b9e3a839
The default value of "rally_all" remains, but this
allows deployers to target the role to multiple
groups now.
Change-Id: Id1bbda227032f66270fe5656250be7794a1dd4af
This patch adds a var used by the repo_build process to determine
which host group to check for members of before building the pip
packages required by this role. This provides the ability to
optimise the repo build process to only building the packages
that are required for an environment.
Depends-On: Idda16b4f382eee57c7469af898859d6d81d4eb30
Change-Id: Iedb4c0c91daa291f881a9fc00c5b8c40bb312ce3
The current method of installing the distribution packages required is
set in the tasks and cannot be changed by a deployer.
Currently the apt task always installs the latest package. This results
in unexpected binary changes when a deployer may simply be trying to
execute a configuration change.
This patch adds the ability for a deployer to change the desired state
so that the results are predictable.
Change-Id: I1708ca8285ffa2bbd1a989b187ef3c6d9dd005c2
Remove all tasks and variables related to toggling between installation
of rally inside or outside of a Python virtual environment.
Installing within a venv is now the only supported deployment.
Additionally, a few changes have been made to make the creation of the
venv more resistant to interruptions during a run of the role.
* unarchiving a pre-built venv will now also occur when the venv
directory is created, not only after being downloaded
* virtualenv-tools is run against both pre-built and non pre-built venvs
to account for interruptions during or prior to unarchiving
Change-Id: I815dff623fce1440a5d60f64bf69f4f563371998
Implements: blueprint only-install-venvs
Debian-specific vars and logic have been moved to tasks
that will execute only on those distributions.
Change-Id: I6a9d04382b1f1badebf12483ce7d1d02b43f8903
Implements: blueprint multi-platform-host
Dmitriy Rabotyagov