At the moment it's not possible to apply different versions of
services to the different groups due to playbook vars having
prescedence over group_vars. However, it can be quite important
to such use cases, for example for phased rollouts of newer versions.
This will also reduce amount of unnecessary variables that are included
for each host, since only required git details will be loaded.
Closes-Bug: #2007296
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_rally/+/881954
Change-Id: Icaa3a958926d9f9aa6cb649bd9f3da9449dd7490
We use sshd role only for aio bootstrap. In fact, what we need is
to simply install and launch daemon.
Using a role for that is overkill considering they have a tendency to
fail against newer versions.
Change-Id: I650a6e611eb53d95fdf94c86d3086bedf5f506c9
All references to Gentoo, SUSE, Debian stretch and Centos-7 are removed.
Conditional tasks, ternary operators and variables are simplified where possible
OS specific variables files are generalised where possible
Change-Id: I42e40adc20a122ea5bba292b1f6fea9e79b3c365
Correct lint errors that arise from these version updates.
This also fixes ceph deployment after backporting breaking changes
to stable-5.0 branch.
Change-Id: Iaa6bd137ebc4bf6cc2071712eb3f53a08c85d75e
Also use ansible interpreter discovery rather than hardcoding the
path to the python executable which may vary depending on distro.
Change-Id: Id6d1ec65106b5b8eabb152b0238a6f9c0a2d97da
On some systems, the aio bootstrap script will fail due to an issue with fact
gathering. This patch runs the setup module prior to the bootstrap process.
Reference: Eavesdrop IRC Logs - openstack-ansible.2018-11-23.log.html
URL: https://pasted.tech/l/418bb8177f794d3f
Change-Id: I52a7ebb946cc068013a0f7dbbc43989abf1cbd02
Closes-Bug: #1851248
This patch removes all references and stops using repo_build as
all the integrated roles have their wheels and virtualenvs
built inside the role.
This patch removes all references and usage of pip_install as we
no longer need to do this.
Change-Id: Ie48b738c9281578e48b1811481d2a8da3832dc1b
In this patch we change the 'scenario' for the AIO bootstrap
to allow it to be overloaded with any additional service names
which correspond to an appropriate conf.d AIO configuration file.
This allows us to implement jobs using a scenario called something
like 'aio_lxc_octavia' and it will implement the base AIO and add
the octavia service.
In order to make use of this for the experimental jobs, we change
the job names so that the appropriate services are deployed.
Related-bug: #1759784
Change-Id: If6b7e401021ea57a908a616ec8b86e7aa6798558
To ensure there are no clashes with the inventory and other uses
beyond the AIO bootstrap, we namespace the container_tech variable
and the install_method variable. This also allows us to use those
variables as the common base in all conditionals, rather than repeat
the use of the env var lookup here and there.
We also tidy up a bit of content in the user_variables.yml template and
the bootstrap-aio.yml playbook.
Finally, we remove the unnecessary specification of the
'bootstrap_host_scenario', 'install_method' and 'openstack_confd_entries'
variables in the playbook, given that they're now in the role defaults.
Change-Id: I5c276a2efed5e1b300fc72f2767abb008199571c
Nspawn containers can attach directly to the physical host interface
using macvlan which allows us to greatly simplify connectivity in test
instances.
Changes to the user variable files have been made to allow the services
to function on a simplified stack.
Depends-On: I083042a791d9213b9b1872a239dc18dc6c7ae46e
Change-Id: Iaa7cfb051d26bcd080df15ef8949d3cf16208ea9
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Interfaces such as eth12 are required to be working for functional
tests such as healthcheck/tempest, and we waste a lot of CI time
when these interfaces do not come up properly during host preparation.
This change makes the build fail if either:
* The network bridges do not come up after creation
* The new interfaces eth12/13/14 are not up after host bootstrap
Change-Id: Idf83b48beab29e9044fbc0e6d4eacc89f68bf18f
None of the hosts need pip installed any more. Everything
installed on the host is now a distribution package, and
all pip packages are inside a virtualenv. As such, we make
the final changes to eliminate the installation of pip on
the host.
1. We change the pip_install role settings to only put
pip.conf down, and not bother installing pip. The
pip.conf remains necessary to provide any pip installs
done in venvs with the details of the repo server.
2. We update the utility container playbook so that it
installs everything into a venv, then symlinks the
appropriate openstack client binaries to /usr/local/bin
for convenient access. This is only done for source
based installs.
3. We update the ceph radosgw keystone setup to make it
use the appropriate service host, and to make use of
the correct runtime venv. It also now makes use of
native ansible modules instead of our own.
Depends-On: https://review.openstack.org/587840
Depends-On: https://review.openstack.org/587849
Depends-On: https://review.openstack.org/589643
Depends-On: https://review.openstack.org/590011
Depends-On: https://review.openstack.org/590178
Change-Id: Iac018386e98d1531b605c66bccf3bcce10226e19
Add new 'aio_distro_basekit' jobs to test the minimal basekit deployment
using distribution packages for the OpenStack services.
We can skip all repo-* related playbooks and roles since we are not
building pip packages for OpenStack services anymore. Finally, we can
populate the utility container using the distribution packages for the
OpenStack client instead of using the wheel packages.
Change-Id: Ia8c394123b5588fff8c4acbe1532ed5a6dc7e8ec
Depends-On: https://review.openstack.org/#/c/583161/
Depends-On: https://review.openstack.org/#/c/567530/
Depends-On: https://review.openstack.org/#/c/580455/
Implements: blueprint openstack-distribution-packages
Currently the pip_install role does not configure any
constraints when exercising the AIO bootstrap, creating
the possibility that the initial bootstrap may not have
the right software versions.
This patch ensures that the same constraints are used
for the AIO bootstrap as was done for the Ansible
bootstrap.
Change-Id: I6aa467d4f1f6db33c824f4fcf8e46adb0a26cfa0
It looks like the `aio` scenario was renamed to `aio_lxc`, however
there are still a few references to `aio`. This commit updates those
references to `aio_lxc`.
Change-Id: Ia080bc4f979cc9b29ea645725d25442edbb73668
This change adds in a dynamic table to our documentation so that we're
able to generate a table of all available scenarios and mark where
services are tested within. To make the dynamic scenario generation
easier the AIO scenario conf setup has been moved into it's own variable
file.
The file doc/source/contributor/scenario-table-gen.html has been touched
allowing sphinx to import and generate the required table as needed.
Change-Id: Ief5d180491d3a330890d6bfa06d43f565dd26103
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The bootrastap-aio.sh script uses the bootstrap-aio.yml file to
provision an All-in-One setup but this playbook lacks of proxy
support resulting in timeout inside of corporate networks. This
change add support to networks that need to reach a proxy server.
Change-Id: I3b28b8a39d050ced0ebca5ffdfbe1a7b78b9a7f5
Convert the existing sftp variable into a dictionary so it can contain
the correct sftp path for SUSE hosts.
Change-Id: Ie5ade74f9fc5a6fa8eef4af35b269d87125674d6
This patch provides the necessary files and changes in existing files to
deploy tacker component. Tacker is an orchestrator and VNF manager, which
is widely used as MANO component in NFV type of deployments
Change-Id: I339c9cc032f871766a89e24c2ada38063fc7ac39
Configure glance to only use file to avoid swift, etc.
Depends-On: I7b8a2314a23281a4369d431aba280e0c0dc66f22
Change-Id: I6054f754d2671beacd83d3a0fe950d49fa221342
The current ansible bootstrap process tried to
pin the versions of pip, setuptools and wheel
on the host and also uses inherited python
packages from the host. This causes problems
when the host has a version of setuptools which
is cannot be changed (perhaps due to some bug)
or when the host has otherwise undesirable
python packages.
The ansible bootstrap process only needs to
be concerned with whether pip is installed
and understands how to use constraints.
From there we can bootstrap the venv using
get-pip and completely avoid package conflicts
with the host. Once Ansible is bootstrapped,
the pip_install role will ensure that pip,
setuptools and wheel are correctly bootstrapped
and pinned across all hosts.
The pip_install role now uses constraints for
the initial get-pip [1], so we can also remove
the list of pip pins from group_vars.
We remove pip, setuptools and wheel from the
requirements.txt to allow the packages on the
host to be installed in the bootstrap without
forcing a change of pip/setuptools/wheel at the
same time (which causes failures in some
circumstances).
[1] https://review.openstack.org/483905
Change-Id: Ida84fb6bb726e1332f0e29ade51b67a5721f0785
We want to set-up a test-site for translations as a separate scenario of
an AIO. This will assist the translations team to test translations
throughout the cycle.
In order for Trove to build successfully we need some Network setup
which is added in this patch.
Change-Id: I0a8bef400e5c99d9258022fac7e740632ed4b796
Additionally, this adds Designate back into integrated build since
Designate is now working and requirements have caught up.
Change-Id: Ib8a6e2fe1aeb3dc475f8c0a4bdf212392ff8cdfe
Additionally, disable designate from the integrated build until their
gate issues are unblocked and patch
https://review.openstack.org/#/c/445069/ can merge.
Change-Id: Ia5b3e4a51c907f5a27710d201c0bef07d397fd53
In order to facilitate periodic deployment and upgrade testing,
this patch implements an 'action' as a second CLI parameter for
the gate-check-commit script.
This seperates the 'action' from the 'scenario', allowing us to
execute a gate-check-commit for multiple scenarios instead of
being stuck with only one. That is why the 'upgrade' scenario
is removed.
This patch is partnered with this project-config change:
https://review.openstack.org/419517
The patch also tidies a few things up in order to make the
script a little more readable.
Change-Id: Ie62efc188d5eafd5bb64eb53f14c191dd50bef33
This change removes ceilometer from the integrated gate. Sadly the
ceilomteter roles and playbooks have been a little neglected which has
resulted in a slow and sometimes unstable system. To improve gate times
ceilometer has been pulled out of the integrated gate. Once we're able
to get the ceilometer roles and playbooks updated to the latest
release and we're able to resolve all of the deprecations we should
put these rolse back in rotation for the integrated gate.
Change-Id: I116928a1db96e242ed87177578581fe7bf16a001
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This is so variables can be overridden in playbooks
that include bootstrap-aio.yml. Otherwise, the variables
specified at time of the role invocation will always take
precedence.
Change-Id: Ie2df5f4b18795edbc4abec1c348057106537f857
* Add a new variable UPGRADE_BASEBRANCH and use that to check out a
stable branch, then run the bootstrap process. This will require a
change to project-config so that the job names will add in the branch
name, so we can pick it up instead of providing a value when none is
present.
* Add a section to confd_overrides for the upgrade scenario to fix the
following error
TASK [bootstrap-host : Deploy user conf.d configuration] ***********************
fatal: [localhost]: FAILED! => {"failed":
true, "msg": "{{ confd_overrides[scenario] }}: 'dict object' has no
attribute u'upgrade'"}
Change-Id: Ia5fd197e81da0a1fd55327d155f181c8792bc199
Adds ceph-ansible integration to OSA using the upstream
roles ceph-common, ceph-mon, and ceph-osd to support deployment
of a Ceph cluster for use with OSA.
The integration should be considered experimental at this point,
however an integrated functional testing scenario named "ceph"
is added which fully exercises the ceph-ansible deployment and
configures OSA to use the deployed Ceph cluster. Once this is added
to the OSA gate tests for the integrated build, that should help
quickly stabilize any issues that arise with this integration.
Change-Id: I275708096a71f66d73be1400c8a592c598382c1d
The pip_install role is using variables requiring facts to be
present. The best way to fix this issue is to gather_facts
in the bootstrap-aio playbook.
Change-Id: Idfbcf54faa90013d229b415463ce664dbda7f334
Closes-Bug: 1647481
Add a designate container to those installed in an AIO
and run the os-designate-install playbook to install and
start the designate services in the container.
Change-Id: I869d91161dd00076fbdae67046537ffe3e1c4706
This change is being done to ensure that we have a consistent and
performant SSH configuration in place throughout all of our gates.
Change-Id: I4b2da075400dd7abb9826e55bf14bf10b126b5df
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
In order to prepare for the ability to implement multiple
integrated gate scenarios, this patch does the following:
- Ensures that all inventory groups are complete and can
be entered into the inventory with their own host groups
instead of being forced to use the shared-os-infra group.
- Splits all OpenStack service environment configurations
into individual conf.d files so that scenarios can be
selective about which services are deployed.
- Implements the ability to set the desired scenario via
an environment variable. This has already been
implemented in OpenStack-CI.
- The current AIO is set as the default scenario, and it
is set even if there is no environment variable present.
Depends-On: I5441f8f9a9c02cc0a501a9b9f7c6cbe2604f4b65
Change-Id: I99541d36aed6d4b9f83746c1cd1a5521b310f1f6
This patch converts the AIO bootstrap process to use Ansible
instead of bash scripting. The patch also minimises the options
available to focus the role concerned to just handle an AIO
bootstrap, but gives it just enough flexibility to allow the
use of an external MongoDB database for Ceilometer/Aodh and
for a deployer to specify a secondary disk for the AIO to
consume.
A major change is that the AIO bootstrap process no longer
assumes that it can destroy a secondary boot device. It
requires a device name to be provided. This prevents horrible
surprises.
TODO (in subsequent patches):
- update the developer AIO docs
- convert run-playbooks.sh into an Ansible playbook
Implements: blueprint convert-aio-bootstrap-to-ansible
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Change-Id: I6028952e7260388873f57db47cc3e08126ecc530
Dmitriy Rabotyagov