We delegate openstack client operations to the utility host nowadays,
so no reason to install them into ansible venv.
Change-Id: I4a183598150186f54ef52bef9e10ee915e9b85de
The git clones of required roles contained in ansible-role-requirements
are currently single-threaded and can take a significant time when
running scripts/bootstrap-ansible.sh. A new module, git_requirements has
been implemented in order to multithreaded this, which takes
input as provided in clone_roles in get-ansible-requirements.yml.
The result of this implementation is a sizeable speed increase (~factor
of five across the full bootstrap_ansible script) for the git clone
repos task when tested on a two core VM.
The new module requires all repo information ahead of time in order to
distribute the clones between threads. This is unlike the previous
implementation, which performed the clones in sequence and generated
the appropriate data (destination URL, default refspec etc) per loop
iteration. This information is now generated within the ansible module,
with defaults supplied at the task level.
The retry and delay parameters used for the loop with the built in ansible
git module have been re-implemented in the new module to allow force
cloning, alongside retries and time delays for failed clones.
Change-Id: I3cdb248204c5e03ef98803d366e3a19ed7f8bdd8
As CI images have pre-installed latest virtualenv package wich breaks
deployments. This will ensure that this package will be dropped and
appropriate one will be installed later.
This also makes ubuntu use appropriate python3 venv tooling and
drops py2 packages installation
Additionally virtualenv is dropped from requirements and ubuntu
upgrade jobs is marked as non-voting until backporting change
to previous release
Change-Id: I50931e0f68176bbaebfb68eec9f84241816c7fe3
With this, the requirement for the os_* modules
changes from shade to openstacksdk. The 'ansible-next'
job is also updated to test stable-2.7.
Depends-On: https://review.openstack.org/#/c/611765
Change-Id: If32f88592caa849c3c7c9701d4803530d9e549ed
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Adding this python library enables us to use the
json_query filter, which is amazing, and deserves
to be president.
Change-Id: I196a2a65268a00a6077b39de6309b5111e98ce97
In order for the octavia role to upload the amphora image using
tags, we need to have access to the openstack CLI.
This will be necessary until the Ansible os_image module can
replace this shell task.
Change-Id: Ic75483dbab8a5dd43672b3234dbd19bad3d6e735
In order to pass the requirements test, every lib in
the *requirements.txt files must have a lower bound.
Change-Id: Ib20057b9cc2860167c6eeed064c303519c4c9b5c
In order to reduce the packages required to pip install on to the hosts,
we install the extra library into the ansible-runtime venv so that tasks
interacting with keystone using our keystone module can do so by using
localhost delegation. Using this strategy, the library does not need to
be installed on the target host.
Change-Id: I7213252ac7fe59550379db3fe755996c1349a959
We stopped using pycrypto for secrets generation a while back
but the requirement remained in requirements.txt. This patch
removes it.
Change-Id: I7cfce70bbf630d1cf15fcc358bdba3371c03ceee
This patch removes urrlib3 and requests from the requirements.txt
file so that the CentOS Ceph gate will pass.
At the moment, yum tries to install python-urllib3 and throws errors
when it finds that urrlib3 files already exist (since they were
installed via pip).
This also breaks our periodic builds which, in turn, makes infra
quite displeased.
Closes-Bug: 1715888
Change-Id: I2a8b0ebcca9839d033552130688356c055f71553
The current ansible bootstrap process tried to
pin the versions of pip, setuptools and wheel
on the host and also uses inherited python
packages from the host. This causes problems
when the host has a version of setuptools which
is cannot be changed (perhaps due to some bug)
or when the host has otherwise undesirable
python packages.
The ansible bootstrap process only needs to
be concerned with whether pip is installed
and understands how to use constraints.
From there we can bootstrap the venv using
get-pip and completely avoid package conflicts
with the host. Once Ansible is bootstrapped,
the pip_install role will ensure that pip,
setuptools and wheel are correctly bootstrapped
and pinned across all hosts.
The pip_install role now uses constraints for
the initial get-pip [1], so we can also remove
the list of pip pins from group_vars.
We remove pip, setuptools and wheel from the
requirements.txt to allow the packages on the
host to be installed in the bootstrap without
forcing a change of pip/setuptools/wheel at the
same time (which causes failures in some
circumstances).
[1] https://review.openstack.org/483905
Change-Id: Ida84fb6bb726e1332f0e29ade51b67a5721f0785
If not forcing to install urllib3, ansible on the deploy node
will reuse the urrlib3 from site-packages, which could be older
than the one in global-requirements, because ansible doesn't
list it in its dependencies, so we don't explicit force a
certain version to be installed.
On my machine, I had an urllib3 installed with a version of
1.7.1, and all the lookups with https had SSL issues. Moving to
urllib3 fixed the thing.
This only cares about the ansible side, to ensure ansible venv
has the proper version. Hosts targetted by ansible (for example
when using get_url), also need a fix. This will be done in
different patches.
Change-Id: Ia07969ac9aee90724eecc6657affd1dbe16045a7
The ansible.cfg file has been removed to allow a deployer to specify their
own configurations without having to modify the ansible.cfg file within the
playbooks directory.
Change-Id: I94a8647edd04c21c34f7892f3179a8aa66548ed3
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch adds the packages needed to resolve SNI issues that may
be seen when executing tasks on the deployment host.
Change-Id: I2d7c58ffe39faf204485dffb39d982f3ff26c34a
To ensure that the Ansible venv has the right packages installed,
requirements.txt is updated with the appropriate packages.
To ensure that the OS for the deployment host is appropriately
prepared for the Ansible venv creation, to install pip, and
python packages into the venv, OS packages are installed.
Fedora install support is removed from the script as it is an
unsupported platform and unmaintained section.
Change-Id: Ib4e219294f5fd93c6a50814955ab97424a79dafd
This implements the following changes:
- The sources-branch-updater script will only update SHA's/tags
for OpenStack roles. This is primarily for usage in stable
branches where external roles must not be changed after release.
- The script will no longer update the *requirements.txt files.
These files are managed by the OpenStack Global Requirements
management process.
- The pins for pip, setuptools and wheels are now set in the
global-requirement-pins.txt file in order to ensure that it
takes precedence in the repo build process above all other
requirements.
- The order of the requirements.txt file has been changed to ensure
that pip, setuptools and wheels is installed first whenever the
requirements.txt file is used.
- The pin comment has been removed from the requirements.txt file
as it no longer applies..
Change-Id: Ib98e63153ace7f02b9bbce878aa67fbdddd784b6
This patch updates all the roles to the latest available SHA's,
updates all the OpenStack Service SHA's and also updates the
appropriate python requirements pins.
Change-Id: Ifc77c02d456500651e8adcaf9338f81601e2c148
This change allows Ansible to be installed and executed from within
a virtual environment.
Closes-Bug: 1431324
Implements: blueprint ansible-2-1-support
Change-Id: I2fb19dd8218753f75bd7550ed7ba9b79f777083d
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch updates all the roles to the latest available stable
SHA's, updates all the OpenStack Service SHA's and also updates
the appropriate python requirements pins.
Note that 'ceilometer_api' has been removed from the standard
tempest testing in the integrated build as the tests have been
removed from the tempest repository with
I0775bcc15dc9cbae6e075fe92f44b5f6c9b9d5d2
The ceilometer testing will have to be re-introduced using the
tempest plugin framework.
Change-Id: Iea7e4213893fb331c410c2d09e3258493f4643f3
Upstream has removed the paramiko restriction of <2 and now has a
dependency on paramiko >2. This is the commit where the requirement
changed: [0]
[0] - e379813e9c
Related-Change-Id: I2369638282b4fefccd8484a5039fcfa9795069a7
Change-Id: I06ba8a9da100a3b02e8eb7f15042f9573b71ed59
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
* Corrected the repo server log directory. This change updates the repos
server log directory which was being incorrectly indexed. The old container
build process would create a service log directory based on "properties".
The "properties" log directory is unused or is re-defined by the various
service roles. This change simply corrects the log paths which the
`rsyslog_client` role uses to ensure all logs are shipped to the right
places.
* Add a log directory creation task to the utility container. This play
incorrectly assumed that the log directory based on "properties" would
be automatically created. This update simply makes that more explicit.
* paramiko has been pinned to match global requirements and use a version
<2. This will ressolve runtime issues in Newton with an unbound
requirements.
Closes-Bug: #1576755
Change-Id: I15d841106ec9a13555b9737c9388f40557f5bec5
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>