Commit Graph

127 Commits

Author SHA1 Message Date
Michal Nasiadka b04486df07 Bump ansible-core versions to 2.15 and 2.16
Change-Id: Iab40eb92c7e4a9092471bef9d4477a4fa34f1c85
2024-03-14 06:13:38 +00:00
Michal Nasiadka bc203433f1 Add support for ansible-core
Collections version taken from official Ansible 7 package [1]
and limited to the same major version.

[1]: https://github.com/ansible-community/ansible-build-data/blob/main/7/ansible-7.7.0.yaml

Change-Id: I0f1ea9717397adc2d2c3215d74ec3759f351a57a
2023-11-08 09:52:20 +00:00
Michal Nasiadka 00dfa4233c Remove remnants of Monasca and deps
Following Monasca initial removal in [1]

[1]: I6fc7842bcda18e417a3fd21c11e28979a470f1cf

Change-Id: I94d6f102e8da3882f37f3007639b917c49f907a9
2023-10-02 13:10:03 +02:00
Michal Nasiadka 71dd6eb983 ansible: bump min version to 2.14 and max to 2.15
Change-Id: Ia982df80d688522dbaa5fc3e462c1a009d512cb6
2023-08-30 12:24:06 +00:00
Matt Crees 60282285bd Add command to force reset the state of RabbitMQ
This command can be invoked with ``kolla-ansible rabbitmq-reset-state``.
This is primarily designed to be used when enabling HA queues[1].

As such, this also updates the RabbitMQ documentation to use this
command.

[1] https://docs.openstack.org/kolla-ansible/latest/reference/message-queues/rabbitmq.html#high-availability

Change-Id: I6ad95a3618fc1a34af56657ef99ef14dc979f17a
2023-08-25 10:09:58 +00:00
Zuul a34f1cd7e6 Merge "Add support for checking Octavia cert expiration" 2023-08-17 07:21:28 +00:00
Michal Nasiadka 10fc1b748a ansible: bump min to 2.13 and max to 2.14.2
We limit to 2.14.2 due to a regression in ansible-core [1] that breaks
conditional include_task loops in handlers. This is used for controlled
restarts of MariaDB and RabbitMQ.

[1]: 65366f663d

Change-Id: I57425680a4cdbf0daeb9b2cc35920f1b933aa4a8
Co-Authored-By: Michal Nasiadka <michal@stackhpc.com>
2023-05-18 20:04:48 +01:00
Matt Crees d9451f49f3 Add support for checking Octavia cert expiration
Adds a flag ``kolla-ansible octavia-certificates --check-expiry <days>``
to the ``octavia-certificates`` command to check if the certificates
will expire within a given number of days.

Change-Id: I869b8afd85fe282d823ecf3593aa22f94a61b2a0
2023-04-27 15:22:12 +01:00
Zuul 861e9467ad Merge "Revert "ansible: bump min to 2.13 and max to 2.14"" 2023-04-20 13:20:52 +00:00
Dr. Jens Harbott b98a71e5d4 Revert "ansible: bump min to 2.13 and max to 2.14"
This reverts commit 9867060b6b.

Reason for revert: seems this broke some jobs

Change-Id: I1ca81214ece403351c0a522ea05bf07802e4c4c0
2023-04-20 09:26:11 +00:00
Zuul 6873931f1d Merge "cli: fix find globals.d" 2023-04-19 18:29:39 +00:00
Michal Nasiadka 9867060b6b ansible: bump min to 2.13 and max to 2.14
Change-Id: Ibc9cc91f64b0450de3cae6e2830b4ff2c52c0395
2023-04-13 14:10:28 +00:00
Michal Nasiadka 35b92825b1 cli: fix find globals.d
Currently when the /etc/kolla/globals.d directory does not exist
it writes an error on stderr.

Change-Id: I2d4b3dd9dde3d383af213dc6fd376bc14c650a7d
2023-03-31 14:03:14 +02:00
Matt Crees 6c2aace8d6 Integrate oslo-config-validator
Regularly, we experience issues in Kolla Ansible deployments because we
use wrong options in OpenStack configuration files. This is because
OpenStack services ignore unknown options. We also need to keep on top
of deprecated options that may be removed in the future. Integrating
oslo-config-validator into Kolla Ansible will greatly help.

Adds a shared role to run oslo-config-validator on each service. Takes
into account that services have multiple containers, and these may also
use multiple config files. Service roles are extended to use this shared
role. Executed with the new command ``kolla-ansible validate-config``.

Change-Id: Ic10b410fc115646d96d2ce39d9618e7c46cb3fbc
2022-12-21 17:19:09 +00:00
Michal Nasiadka fb447eb70a Improve kolla-ansible error on missing Ansible
Also improve the error message to point out currently supported
Ansible version.

Change-Id: Iabd11f03ff789f7fab8bec068bc9b4fb4dd1a2be
2022-11-22 06:23:04 +00:00
Michal Nasiadka 98a691a7c9 Bump Ansible min version to 2.12/5 and max to 2.13/6
Change-Id: Ie09bf108250a71d539002dd5ccfa63dd71bcfe90
2022-10-04 11:45:28 +00:00
Tim Beermann 591f366ed7 Remove sanity checks
"Smoke tests" for barbican, cinder, glance and keystone have been removed as discussed in PTG April 2022.

Signed-off-by: Tim Beermann <beermann@osism.tech>
Change-Id: I613287a31e0ea6aede070e7e9c519ab2f5f182bd
2022-05-13 17:41:34 +00:00
Mark Goddard 80b311bef7 libvirt: add nova-libvirt-cleanup command
Change Ia1239069ccee39416b20959cbabad962c56693cf added support for
running a libvirt daemon on the host, rather than using the nova_libvirt
container. It did not cover migration of existing hosts from using a
container to using a host daemon.

This change adds a kolla-ansible nova-libvirt-cleanup command which may
be used to clean up the nova_libvirt container, volumes and related
items on hosts, once it has been disabled.

The playbook assumes that compute hosts have been emptied of VMs before
it runs. A future extension could support migration of existing VMs, but
this is currently out of scope.

Change-Id: I46854ed7eaf1d5b5e3ccd8531c963427848bdc99
2022-03-21 11:54:54 +00:00
Mark Goddard f63f1f3082 Install openstack.kolla collection
This change adds an Ansible Galaxy requirements file including the
openstack.kolla collection. A new 'kolla-ansible install-deps' command
is provided to install the requirements.

With the new collection in place, this change also switches to using the
baremetal role from the openstack.kolla collection, and removes the
baremetal role from this repository.

Depends-On: https://review.opendev.org/c/openstack/ansible-collection-kolla/+/820168

Change-Id: I9708f57b4bb9d64eb4903c253684fe0d9147bd4a
2022-02-21 14:26:48 +00:00
Zuul d5dcb66479 Merge "Add Ansible 5 aka core 2.12 support" 2022-01-20 20:53:03 +00:00
Radosław Piliszek e63bbed18d Clean up chrony cleanup
In the Yoga cycle we no longer need kolla chrony container removal
procedures.

Change-Id: I4dc246cf0fd68838470bf9e9bf749fa9be4d6670
2022-01-18 15:37:51 +00:00
Radosław Piliszek 7372911a19 Add Ansible 5 aka core 2.12 support
Also bumps the minimum to 4.

Change-Id: Ia373f9cc3bb69eba0288bbb3e497e8cadb7cc4d3
2022-01-07 18:08:55 +00:00
Adrian Andreias d4a2c14039 Add kolla-ansible --version option
Closes-Bug: #1878280
Change-Id: I0d39a015639adf3d75780edc765bebcb85c2b42e
Signed-off-by: Adrian Andreias <adrian@fleio.com>
2021-11-15 13:44:50 +02:00
Radosław Piliszek cd13bde880 Fix missing Ansible version in the error message
Change-Id: Iefa8f78142c502ac9e8ebdbb95c94b2b05f99e5b
Closes-Bug: #1948979
2021-10-27 19:17:05 +00:00
Zuul d85af34ccd Merge "Bump up Ansible max supported ver to 4.x" 2021-09-24 16:43:01 +00:00
Zuul 59e6688a0c Merge "Add check and diff options to kolla-ansible" 2021-09-24 16:30:46 +00:00
Michał Nasiadka 1b650534c0 Bump up Ansible max supported ver to 4.x
This change bumps up max supported Ansible version
to 4.x (ansible-core 2.11.x) and minimum to 2.10.

Change-Id: I8b9212934dfab3831986e8db55671baee32f4bbd
2021-09-23 10:45:31 +00:00
Michal Arbet 0e720b382b Add check and diff options to kolla-ansible
This patch is adding --check and --diff options
to kolla-ansible, which cause that kolla-ansible
run will be more verbose and able to run in
semi dry-run mode.

The --diff option for kolla-ansible can be used alone or
with --check. When you run in diff mode, any module that
supports diff mode reports the changes made or, if used
with --check, the changes that would have been made.
Diff mode is most common in modules that manipulate files
(for example, the template module) but other modules might
also show ‘before and after’ information
(for example, the user module).

For more information check [1].

[1] https://docs.ansible.com/ansible/latest/user_guide/playbooks_checkmode.html#using-diff-mode

Change-Id: Ifb82ea99e5af82540e938eab9e2a442b2820d7df
2021-09-21 17:08:39 +02:00
Zuul e06e531089 Merge "Add kolla-ansible gather-facts command" 2021-09-20 18:54:29 +00:00
Mark Goddard d9a3758952 Add kolla-ansible gather-facts command
In some situations it may be helpful to populate the fact cache on
demand. The 'kolla-ansible gather-facts' command may be used to do this.

One specific case where this may be helpful is when running kolla-ansible
with a --limit argument, since in that case hosts that match the limit
will gather facts for hosts that fall outside the limit. In the extreme
case of a limit that matches only one host, it will serially gather
facts for all other hosts. To avoid this issue, run 'kolla-ansible
gather-facts' without a limit to populate the fact cache in parallel
before running the required command with a limit.

Change-Id: I79db9bca23aa1bd45bafa7e7500a90de5a684593
2021-08-25 16:45:39 +01:00
Marc 'risson' Schmitt 839ec629bf
tools: use /usr/bin/env bash instead of /bin/bash
Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>
Change-Id: I374f7427a4318d00ca474367818117e11789ec13
2021-08-17 14:29:33 +02:00
Will Szumski 6c72fa8117 Support multiple inventories
Multiple inventories can now be passed to `kolla-ansible`.  This can be
useful to construct a common inventory that is shared between multiple
environments.

Change-Id: I2ac5d7851b310bea2ba362b353f18c592a0a6a2e
2021-07-29 15:25:07 +00:00
Zuul 8f05a309d1 Merge "Fix exit code with bogus command name" 2021-07-26 07:46:36 +00:00
Zuul 7e5db84e56 Merge "Support editable installation in all cases" 2021-06-17 09:13:43 +00:00
Mark Goddard 84ac7b3096 chrony: allow to remove the container
The chrony container is deprecated in Wallaby, and disabled by default.
This change allows to remove the container if chrony is disabled.

Change-Id: I1c4436072c2d47a95625e64b731edb473384b395
2021-06-02 17:28:35 +00:00
Mark Goddard 86ddc94ec2 Fix exit code with bogus command name
Running this:

$ kolla-ansible bogus-command

Should show usage & give a non-zero exit code. Previously it gave a zero
exit code. This change fixes the issue.

Closes-Bug: #1929397

Change-Id: I580c208d61d5efe115f936dfb8f3f6508acd91b2
2021-05-25 08:52:19 +00:00
Michal Arbet 22a6765f5e Support editable installation in all cases
An editable installation allows changes to be made to the source code
directly, and have those changes applied immediately without having to
reinstall.

    pip install -e /path/to/kolla-ansible

Above is currently working only in virtualenv, but there is no reason to
not allow in all cases. This is usefull for example when user is
building his own docker container with editable kolla-ansible installed
from git without virtualenv.

Change-Id: I185f7c09c3f026fd6926a26001393f066ff1860d
2021-04-08 12:31:00 +00:00
Zuul 6c18e5814e Merge "Remove Monasca Log Transformer" 2021-03-24 18:21:04 +00:00
Michał Nasiadka 1ccccbcf0a Bump up supported Ansible version to 2.10
Min version stays as 2.9

Change-Id: I7ec8c5eb36757248c9aa016dc7d4e495ec5bb635
2021-03-08 11:40:31 +00:00
Doug Szumski 0743a9bf4b Remove Monasca Log Transformer
Historically Monasca Log Transformer has been for log
standardisation and processing. For example, logs from different
sources may use slightly different error levels such as WARN, 5,
or WARNING. Monasca Log Transformer is a place where these could
be 'squashed' into a single error level to simplify log searches
based on labels such as these.

However, in Kolla Ansible, we do this processing in Fluentd so
that the simpler Fluentd -> Elastic -> Kibana pipeline also
benefits. This helps to avoid spreading out log parsing
configuration over many services, with the Fluentd Monasca output
plugin being yet another potential place for processing (which
should be avoided). It therefore makes sense to remove this
service entirely, and squash any existing configuration which
can't be moved to Fluentd into the Log Perister service. I.e.
by removing this pipeline, we don't loose any functionality,
we encourage log processing to take place in Fluentd, or at least
outside of Monasca, and we make significant gains in efficiency
by removing a topic from Kafka which contains a copy of all logs
in transit.

Finally, users forwarding logs from outside the control plane,
eg. from tenant instances, should be encouraged to process the
logs at the point of sending using whichever framework they are
forwarding them with. This makes sense, because all Logstash
configuration in Monasca is only accessible by control plane
admins. A user can't typically do any processing inside Monasca,
with or without this change.

Change-Id: I65c76d0d1cd488725e4233b7e75a11d03866095c
2021-03-03 17:20:18 +00:00
Mark Goddard 519ca1c083 Fix installation with pip install --user
If kolla-ansible is installed via pip install --user, currently the
kolla-ansible script is unable to locate the installed playbooks.
This leads to a failure when running commands.

This change fixes the issue by checking for the user's .local directory
as a possible installation path.

This fixes some of the scenario tests which were failing after switching
to a user installation in Ifaf1948ed5d42eebaa62d7bad375bbfc12b134d5.
Most tests did not fail since the kolla-ansible script in the source
checkout was used.

Closes-Bug: #1915527

Change-Id: I5b47a146627d06bb3fe4a747c5f20290c726b0f9
2021-02-12 17:58:04 +00:00
zengchen228 aaab1d1b68 Fix kolla-ansible to work with pyenv-virtualenv
One of the pyenv-virtualenv-set-up aliases depends on a symlink.
It seems pyenv runs the bash script from such a path and it fails
because of a failing comparison (VIRTUAL_ENV not detected).

The VIRTUAL_ENV is ensured to be fully resolved as well for safety.

This requires readlink from GNU coreutils but all supported platforms
have it by default.

Extra comments included, as well as simplification of directory
detection - readlink handles this (not that `bin` itself was
ever a symlink...).

Closes-Bug: #1903887
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Change-Id: I2fe6eb13ce7be68d346b1b3b7036859f34c896c4
2020-11-18 20:41:01 +00:00
Mark Goddard 894f4912ac octavia: generate certificates automatically
implemented as a separate command (kolla-ansible octavia-certificates)

Implements: blueprint implement-automatic-deploy-of-octavia

Co-Authored-By: wu.chunyang <wuchunyang@yovole.com>
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Change-Id: I2c5b26ce9e363f35c523865904a582f7960aa682
2020-10-08 16:50:30 +02:00
Pierre Riteau c5c6d995d3 Bump minimum Ansible version to 2.9
Change-Id: I5befc72a4894d625ca352b27df9d3aa84a2f5b2c
2020-09-23 17:48:01 +02:00
Zuul 9a8341c2a7 Merge "Performance: Run common role in a separate play" 2020-07-17 15:43:22 +00:00
Zuul 59b2f18de4 Merge "Support editable virtualenv installation for development" 2020-07-15 11:50:53 +00:00
Michal Nasiadka bf985930d0 Evaluate PASSWORDS_FILE later
Currently seting --configdir on kolla-ansible CLI doesn't set properly the path
for the passwords file.

Change-Id: I38d215b721ec256be6cfdd6313b5ffb90c2a3f4c
Closes-Bug: #1887180
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2020-07-10 17:32:35 +02:00
Mark Goddard 56ae2db7ac Performance: Run common role in a separate play
The common role was previously added as a dependency to all other roles.
It would set a fact after running on a host to avoid running twice. This
had the nice effect that deploying any service would automatically pull
in the common services for that host. When using tags, any services with
matching tags would also run the common role. This could be both
surprising and sometimes useful.

When using Ansible at large scale, there is a penalty associated with
executing a task against a large number of hosts, even if it is skipped.
The common role introduces some overhead, just in determining that it
has already run.

This change extracts the common role into a separate play, and removes
the dependency on it from all other roles. New groups have been added
for cron, fluentd, and kolla-toolbox, similar to other services. This
changes the behaviour in the following ways:

* The common role is now run for all hosts at the beginning, rather than
  prior to their first enabled service
* Hosts must be in the necessary group for each of the common services
  in order to have that service deployed. This is mostly to avoid
  deploying on localhost or the deployment host
* If tags are specified for another service e.g. nova, the common role
  will *not* automatically run for matching hosts. The common tag must
  be specified explicitly

The last of these is probably the largest behaviour change. While it
would be possible to determine which hosts should automatically run the
common role, it would be quite complex, and would introduce some
overhead that would probably negate the benefit of splitting out the
common role.

Partially-Implements: blueprint performance-improvements

Change-Id: I6a4676bf6efeebc61383ec7a406db07c7a868b2a
2020-07-07 15:00:47 +00:00
Mark Goddard 5fb37ae584 Support editable virtualenv installation for development
An editable installation allows changes to be made to the source code
directly, and have those changes applied immediately without having to
reinstall.

    pip install -e /path/to/kolla-ansible

Change-Id: I023d96d25edd9d2fafd4415743e298af72a861a1
2020-07-02 18:15:40 +01:00
Mark Goddard 4d8d678240 Fix unexpected message without globals.d
Recently a feature was merged to support pulling in multiple
configuration files from a globals.d directory. However, if this
directory does not exist, we get the following error when executing
kolla-ansible:

    find: '/etc/kolla/globals.d': No such file or directory

This change addresses this by redirecting find command stderr to
/dev/null.

TrivialFix

Change-Id: Ie5aa511a5ebf3355817a7c3bb65b09ac5dcf2b67
2020-06-29 14:11:05 +01:00