Added tar option to allow the current user to unpack local
or downloaded data (from swift) into ceph-ansible/fetch_dir,
preserving the existing metadata.
Added also a "/*" as trailing character on the *path field* of
the *archive* ansible task to create a tar.gz containing all
the fetch_dir content whithout including the parent directory
in the resulting path: in this way we should be able to avoid
nested fetch_dir.
1. Added a step to fix broken environment excluding the nested
fetch_dir from the archive sections;
2. Added a new task to delete from filesystem the nested
section to make everything clean: this task is executed
according to the output of the previous one (the archive task).
Change-Id: I45c394bd6b5c09730399202b476ed913e8f334c0
Related-Bug: 1816094
We should avoid use of become: true where unnecessary because
additional files might get created with the wrong permissions.
Change-Id: I4dc71fd23134a9e4a5b2b6e93d2cf45735e71711
Related-Bug: 1813832
The ceph-ansible tasks are now invoked with the tripleo-admin user,
which doesn't by default have write access to /var/lib/mistral, but it
does have sudo access.
This change makes /var/lib/mistral/overcloud/ceph-ansible be owned by
the tripleo-admin user so that subsequent tasks can write to that
directory.
Related-Bug: #1813832
Change-Id: I98bb38078be84cbda3e9a9e338af0d054dc53420
dmidecode can return some additional data if SMBIOS is updated; this
ensures output matches the UUID format.
Change-Id: I5c61c0049bfd16114894cf4db3b79f94b6d9291b
Related-Bug: 1762460
Forces lowercase matching of user provided node data with dmidecode
output.
Change-Id: I844a2f7660dff5feda8397706671ac0efd37949f
Co-Authored-By: Francesco Pantano <fpantano@redhat.com>
Closes-Bug: 1814070
The NodeDataLookup parameter should be valid JSON and we should
help the user by adding type checking for this early in the
deployment so that the user doesn't experience the related bug.
Change-Id: Id9d2208f1cbaba9234d7657420cd7efcad3507a0
Related-Bug: #1784967
Related-Bug: #1814070
PythonInterpreter defaults to /usr/bin/python. If a user overrides
this default, e.g. to something like python3, then we should use it.
Modify ceph-base.yml to use the PythonInterpreter parameter. The
variable will already be set to ansible_python_interpreter by the
calling ansible execution.
Change-Id: If599855c00d0ab8861ea7f873d410f9a880d35be
Closes-Bug: #1811974
In 459b2664d9 (Handle missing or bad
dmidecode) we accept return values of 0, 1 or 2 from command modules
that call 'dmidecode'. However we then unconditionally look at the
stdout key in the result object. When the dmidecode binary is missing
the result dictionary from the command module doesn't contain a stdout
key (resulting in something like):
---
fatal: [overcloud-novacomputeppc64le-1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'stdout'\n\nThe error appears to have been in '/var/lib/mistral/overcloud/ceph-ansible/nodes_uuid_playbook.yml': line 14, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n failed_when: machine_uuid.rc not in [0, 1, 2]\n - name: generate host vars from nodes data\n ^ here\n"}
---
This change just adds a default('') filter to the lookup so a missing
key will fallback to an empty dictionary
Closes-Bug: 1790447
Change-Id: I7db180674c3696508a7f449e2e825e7083a00f6e
Prior to routed networks we only had one subnet per network.
With routed networks each network can have multiple subnets.
The NetCidrMapValue should contain a list storing the cidr
of each subnet for each network.
Ceph:
list_join is used to make a comma separated list of
cidrs for public_network, monitor_address_block,
cluster_network and radosgw_address_block.
Partial: blueprint tripleo-routed-networks-templates
Depends-On: Ia8e219b30d4f8b199b882e95fe2834252a92c15a
Depends-On: I1ace0a02e6aa2610559fee0d8576e6f1bc98d699
Change-Id: I68e064d23ec5d43f59146d974cae604d2c5fdb52
When running dry-run, --check option is used with Ansible CLI which
means the shell command don't actually run so the curl isn't producting
any change. This patch fixes the tasks that relied on the curl outputs
to only run these tasks when the curl changed and produced something.
Change-Id: I5b722ac111db0a1b6272a95496251107e23dd764
Closes-Bug: #1803986
Originally the playbook failed to make user aware that they need
to create the CephAnsibleFetchDirectoryBackup. However, if they
specified the path, then they should know about the directory.
Autocreate it so that the user only needs to specify a path.
Change-Id: I06eca1cf8987c9366492973cbe861e1a6b47f744
The jinja2 condition to look for "ansible_connection: local"
did not contain enough whitespace and resulted in an invalid
Ansible inventory. This patch adds the missing whitespace by
indenting the condition.
Change-Id: Ia114edafe6f1b37d6e6d47af146ab35adeb79efd
Closes-Bug: #1800148
Ensures the hosts group is not created in the ceph-ansible inventory
if all nodes of a given group are blacklisted.
Closes-Bug: 1798044
Change-Id: Ibf25e28d02cee6308f3e9a2660e001ba73d756c0
The standalone deployer adds "ansible_connection: local" to facilitate
all-in-one deployments. This patch passes on this setting when generating
the inventory used by ceph-ansible.
Change-Id: I694c4b3c7fb98e11d7a52eed4072a37471c0e405
So far the tasks for external update/upgrade were not using the step
mechanism as other tasks, we had a single step. As external
deploy/update/upgrade tasks are being used for more things nowadays,
it's likely that we'll need to go towards a similar model like we have
for deploy/update/upgrade tasks -- proper usage of steps.
For now we have just 2:
* Step 0 for setting global facts, and performing validations.
* Step 1 for actual update/upgrade tasks. (There's an upcoming change
to run online data migrations in step 1).
Change-Id: I1933bd0eedab71caab56c0e5d93ba7927fb7c20f
Partial-Bug: #1793332
The CLI commands running Ansible can crash if we send too much
single-line log output their way. This was happening on upgrades, when
we run Ansible with verbosity level 1.
The fix is twofold:
* If ceph-ansible finishes successfully, we don't print the
ceph-ansible output into the main log.
* If ceph-ansible fails, we do print the output, but we print it
line-by-line, which should give us much better readability than
before, and we shouldn't break the limits of the Mistral-Zaqar-CLI
message passing.
Change-Id: I6e0fc36749e74fce25f414c2547e49e2a20437ab
Closes-Bug: #1795689
Attempt to not create GnocchiRbdPool pool when it is set to empty
string [1] resulted in related bug which was not resolved by
originally proposed fix [2].
1. https://review.openstack.org/#/c/575571
2. https://review.openstack.org/#/c/570043
Change-Id: Ie7a42822be89cced480302d40180b9972d191004
Closes-Bug: 1776987
dmidecode isn't functional on all architectures. Don't treat a missing
binary or a missing DMI table as fatal from an install POV.
Change-Id: I33c50ee00ac0b478839b2536f0b965e444e66e53
Related-Bug: 1790447
When scaling ceph monitors, ceph-ansible uses context from the
fetch_directory to prevent new monitors from behaving like they
are the only monitors.
Save the fetch_directory after each ceph-ansible playbook run;
and if there is a previously saved fetch directory, restore it
before each playbook run.
Fetch directory can be saved on the undercloud in Swift or if
the new LocalCephAnsibleFetchDirectoryBackup parameter is passed
then it will be saved in a directory local to the undercloud
instead.
Note that https://review.openstack.org/#/c/567782 only resolves
1769769 for Queens/Pike where Mistral runs ceph-ansible. This
change resolves 1769769 when using config-download.
Change-Id: I0591be8419828cc32f976afce8be1b787b783c23
Depends-On: Icce658f803a608ee4b7df34b0b8297ecabcdb0ee
Related-Bug: #1769769
Running `openstack overcloud external-update run` will update all
external services. This commit adds possibility of running `openstack
overcloud external-update run --tags ceph` to specifically update just
Ceph. It works analogically for upgrades.
Change-Id: Ic1786b6dbfa54516bfb836b450fc35452dca8cb5
Partial-Bug: #1783949
Correct playbook selection is now done automatically using
external_update_tasks and external_upgrade_tasks. The
CephAnsiblePlaybook is still usable and highest priority if set to
something else than the default. It can also be set to [] to
completely skip ceph-ansible playbook execution.
Closes-Bug: #1783949
Change-Id: If7955b31eda432dd61331281ad3d8f25a335fdfe
Depends-On: Ifdf5f1175849f79b98c76b4bcdef2a70f8f1fedb
We obsoleted the CephFS native parameter but missed to update it
in the Ceph profile.
Change-Id: I12a91da7df6079d391fa9583c4f2c926d123044f
Closes-Bug: #1783341
Previously we tried setting from within the Ceph service some Ansible
settings before launching the nodes-uuid playbook; we should instead
just point to the global ansible.cfg file.
Also includes setting environment variables not in the ansible.cfg.
Co-Authored-By: John Fulton <fulton@redhat.com>
Closes-Bug: #1782423
Change-Id: Id220ebba5d9881eae943c7459b2328d3384b7370
The openstack_keys map can have permissions for an empty pool
which results in an invalid kerying.
Co-Authored-By: Giulio Fidente <gfidente@redhat.com>
Change-Id: Ic5ae53d9ab52ea5e7c3f75a240a7a7f4bb5632ba
Closes-Bug: 1776987
Restores compatibility with NodeDataLookup parameter for the
deployment of Ceph via config-download.
Change-Id: I06764271adb73ba73e15e3f7d8d553532db727ef
Closes-Bug: 1778617
Within the double-quoted capability string replace back-slashed
double quotes with single quotes.
Change-Id: Iff6e8f52c28a2de826e1582775727df85384b696
Closes-Bug: #1777249
Updates the format of the CephX keys caps to a new one which
does not need backward compatibility in ceph-ansible
Change-Id: Icd36ac32ec0ed708e66fe638bcbf54cee2d1ae69
By setting the value of rule_name explicitly, we prevent backward
incompatibility issues because the default which ceph-ansible uses
might fit a particular version of Ceph, not all.
Change-Id: I275c1ca53ea79eea607cbbb58aa21cae6d6be80b
Closes-Bug: 1776252
Luminous can assign application types to pools and ceph-ansible
supports this, provided it is passed an appropriate list of maps.
The list of maps is defined in openstack_pools and this change
updates this variable to contain the correct default application
type.
Includes update to environments/disable-telemetry.yaml so that
the GnocchiRbdPool is not created if telemetry is disabled.
Change-Id: I3966a8a1b170202e020684a36ecece78599296b5
Closes-Bug: #1772743
The new master branch should point now to rocky.
So, HOT templates should specify that they might contain features
for rocky release [1]
Also, this submission updates the yaml validation to use only latest
heat_version alias. There are cases in which we will need to set
the version for specific templates i.e. mixed versions, so there
is added a variable to assign specific templates to specific heat_version
aliases, avoiding the introductions of error by bulk replacing the
the old version in new releases.
[1]: https://docs.openstack.org/heat/latest/template_guide/hot_spec.html#rocky
Change-Id: Ib17526d9cc453516d99d4659ee5fa51a5aa7fb4b
Recent changes [1] [2] in ceph-ansible break old way of escaping
quotes in the Manila keyring caps.
1. 82ccbdafbc
2. 424815501a
Change-Id: I7ff5df0e602aff000c9a4213231b7abc68871fc4
Closes-Bug: 1769436
Modify the ceph-ansible execution so that it uses the ansible.cfg
that is shipped with ceph-ansible. ceph-ansible's own testing uses
this file and the project's maintainers recommend it be used.
This configuration file also uses ControlMaster=auto which is not
used by config download at this time. Adding this setting to the
ceph-ansible execution should make it take less time.
Change-Id: I774132eb8b7a43c921c0696635d56b4136301346
Closes-Bug: 1767252
Hardcoding the Ceph version to use in the templates causes issues in
some generic tasks during upgrade, when the cluster has nodes in
mixed state.
Closes-Bug: 1766502
Change-Id: I20bfd46c1db41e2d2ebd3f810591cabf27f2e664
With the migration to external_deploy_tasks, we can remove support
for workflow_tasks in the Ceph services.
Change-Id: Ib7851e02424d4adb1490e8ed36327dc4a573c44f
Uses external_deploy_tasks instead of workflow_tasks for the
deployment of Ceph, via ceph-ansible.
Initially, external_deploy_tasks are added alongside workflow_tasks so
that CI will pass. Once CI is updated to use config-download for these
jobs, we will remove the workflow_tasks.
Co-Authored-By: James Slagle <jslagle@redhat.com>
Co-Authored-By: John Fulton <fulton@redhat.com>
Implements: blueprint ceph-ansible-external-deploy-tasks
Change-Id: I4b88e97c38ff394023a92bec5631b3cd0f6e293e
Dan Prince