Make sure scripts are created with the executable bit set.
Change-Id: I731902411e987b4ea7c2aa84fef869fe5e1c25ae
Add a oooq comptatibility layer, documentation and example.
Change-Id: I30fe6359f1c0098ff9bcdd5939724491d94ef199
Add support for applying w/a before and after upgrade
This change adds the ability to apply workarounds before and after
the overcloud upgrade process has finished. This allows the user
to workaround particular issues that show up after the upgrade
process has finished.
Change-Id: I21a7e885bcc466af6bf80410ba2cc8d03865cb33
Fix missing quotes
This changes adds missing quotes to the node_upgrade_script.yml
templates. Currently the task is failing because of the missing quotes.
Change-Id: Ied9217df374d09bf90e6878a7c79e22042f44d99
Add support for applying workarounds post undercloud upgrade
This change adds support for applying workarounds after the undercloud
upgrade process has finished as part of the undercloud upgrade.
Change-Id: If85900969c0d591cf6024408d958a82fa8c8a534
Adjust overcloud_converge_upgrade_script script
This change adjusts the overcloud_converge_upgrade_script to allow
running the upgrade converge stage. In addition it adjust the ssh
config file to skip host key check so the non-controller script does
not get stuck waiting for user input.
Change-Id: Ic38f325c61e90165a5322ef754f7e5514ed8e687
Append working_dir to logs generated
Change-Id: I6bc9f0c58ad8684ed03dee042e9cfb2bdc6835f6
Install ceph-ansible during undercloud upgrade
ceph-ansible is required to be installed manually for deployments
with ceph nodes. This change installs the ceph-ansible package
before the undercloud upgrade.
Change-Id: If8918a38250a10681d965d0715ebc17078166336
Use openstack overcloud container image prepare command
This change adds the use of openstack overcloud container image prepare
command for generating the environment file containing the container
image names and local registry address.
Change-Id: I174f7e3aae415d51224cf73da83a859e90eed095
Do not rely on ansible inventory for upgrading non controller nodes
Currently we are relying on the ansible inventory to provide groups
containing compute nodes and their facts when creating the upgrade
scripts. In order to remove this requirement and provide easier
integration this change discovers the compute and swift storage nodes
from the undercloud. In addition it adds a wait loop for instance live
migration to complete before and after upgrading compute nodes and adds
support for swift storage nodes upgrade.
Change-Id: Ia4b2e81845c3fec9036c4695f0dd1746d4c5c6b8
Add silent and nobuffer options to curl command
Change-Id: I0f4bd71b67d4827717d9f7d3fc075fc396eb6363
Adjust tht environment files and custom roles data during upgrade
This change switches the environment files used in the overcloud deploy
command to their variants used for deploying containers. In addition, if
using a custom roles data file for composable roles deployments, this
change adjusts the local roles data copy with the changes introduced in
Pike.
Change-Id: Icd3c3b67342c0bd10fc3d28ed89a94fc9f714db4
Fix overcloud_deploy_script var location in oooq_test playbook
Change-Id: I01f18a5413ff223cbca900521037e739ebc43d5d
Disconnect ssh session before uploading images to local registry
This change disconnects the running ssh session in order to allow
the stack user to connect to the docker daemon. This acts as a
workaround for https://bugs.launchpad.net/tripleo/+bug/1699477
Change-Id: Ia06fe8581ba17525cbcb2d955d7947b7c546811d
Add undercloud_reboot var and reboot only on ovs or kernel updates
This change adds the undercloud_reboot var to manipulate if undercloud
reboots should be made or not and does the reboot only when kernel or
ovs updates occur.
Change-Id: Ic89291c5791bbd098204b2c85793335e0faa8d94
Add InfraRed docs
This change adds the steps required to run the tripleo-upgrade role
as an InfraRed plugin.
Change-Id: I08fde6c8954ec4eeedc47971607bec592fc801a1
Add docs for running the role manually from the undercloud
Change-Id: I6d5bb6479b6e1fb45e34aa776d3c315cfae98ae4
Do not stop services before undercloud upgrade
According to the docs stopping services should be done by the undercloud
upgrade process so we do not need to manually do it.
Change-Id: I7f14257b4e90d3a0610c0f90e856096643325861
Specifically become_user: root when running ovs command
Currently become_user is set to the undercloud user, usually
stack, when invoking the undercloud_validate_upgrade.yaml playbook.
This change overrides become_user and sets it to root when running
ovs-vsctl command as it requires privilege escalation.
Change-Id: Ia93245f4a2d09d73849c03401d38ffb25e7be802
Use ip address instead of name when rebooting
OOOQ doesn't set a name for the undercloud in hosts so in case the
reboot is triggered the virthost cannot reach the undercloud. This
change switches the wait condition and calls the ip address instead
of the name.
Change-Id: I2121e2dabe9794d31fe70bbfa0ff8e53da6b3b1b
Add reload ssh tag to the Kill SSH task
Add a tag to the Kill SSH task so it can be avoided when
running the role manually from the undercloud.
Change-Id: Iec088cc174d7e8270fbea0698ee76227b45842f7
Add option to create script with --setup-heat-output option
According to BZ#1477962 in order to be able to run non controller
upgrade scripts after major upgrade composable steps we need to run
the deploy command with the --setup-heat-outputs option.
Change-Id: I8137ff18047a130c5ea8dca2ce11378eabc30329
Add deprecated params to custom roles data file
In Pike there were additional flags assigned to the default roles
data file. This change adds these flags during the upgrade process
to the roles defined in the custom roles data file.
Change-Id: I58c2f30ff74d2302027d7488dc03c5146c371649
Use a default value for the HOME env var
Change-Id: I40e2f46f1311dc4b797977c8136ec4037505ef05
Update python-openstackclient before undercloud upgrade
Updating python-openstackclient before undercloud upgrade is a
requirement to get undercloud upgrade passing.
Change-Id: I4ab67f9af68036a29e11bb2457d70535bd94b7b0
Create environment file for injecting undercloud certificate
When undercloud is SSL enabled the overcloud nodes need to be aware
of the undercloud CA cert. This change creates this file during upgrade
and adds it to the overcloud deploy commands when the overcloud nodes
are not able to reach the undercloud ssl enabled public endpoints.
Change-Id: I79a03299bc28d0ca2dbd83c28087a4c56f6b2271
Convert puppet ceph parameters to ceph ansible during upgrade
This is a workaround for BZ#1488855.
Change-Id: I58ac44b2166abddf56a327a4ee09457139c831da
Remove setup heat outputs workaround
https://review.openstack.org/#/c/502470/ allows us to obtain the
RoleConfig output so there's no need to run the setup heat outputs
step anylonger. This change remove this step.
Change-Id: I7235b8625eea4f77a055d0ab1862d0318f3a776f
Echo bug number in workarounds script
This change replaces the bug number comments with an echo statement.
This provides easier way to debug what workarounds failed to apply
when the workarounds script is run.
Change-Id: I02c5534427ee5cf7b7af618f36577c1008d50992
Fix uc_keystone_conn condition
This change adds an addition condition to inject the undercloud
certificate in the CAMap only when undercloud ssl is enabled.
Change-Id: Iae023922bf1ed0bb22e86710b122c65a8f1568a3
Gather facts only from undercloud node
Facts are only required for the undercloud node. This change adjusts
the existing playbooks to gather facts only from undercloud node in
order to save some time and not rely on nodes which are not required
to be reachable.
Change-Id: Idceec3d10d84ef112da558109d9904a1d8c6ed93
Do not include docker.yaml and docker-ha.yaml environments
As described in BZ#1466744 the docker.yaml and docker-ha.yaml
environments are currently included by default so we do not need
to specifically include them.
Change-Id: I44a72ddd65cf816003ceca21ef33470a3ab125a7
Reboot compute nodes post upgrade
As a post upgrade requirement we need to reboot the nodes in case
of an OVS upgrade. This change runs a post upgrade check for non
controller nodes and reboots the nodes if an ovs upgrade has been
detected. It also adds additional validation for compute nodes to
make sure that the nova-compute service is enabled after reboot.
Change-Id: I583e589118aabae84f8e1dc9ec2c4b43ca17a250
Add L3 agent connectivity check during upgrade
This change adds a check which validates that ICMP
connectivity with a floating IP is not interrupted
during the major upgrade composable step.
Change-Id: Iee55af85b9a2c3ece86731e043130d191ff6a821
Run pre docker composable upgrade workarounds at correct position
This change moves the pre docker composable upgrade workarounds to
be run right before the docker composable upgrade step.
Change-Id: I604ea2eb6202d48b0f771ea80e5e731df687600e
Use bool with ansible booleans
use bool filter when using ansible booleans.
Change-Id: Ibeb59772e935cc28a661ccddcaa4773388ce296d
Add option for creating workloads before upgrade
This change adds the option to launch an instance before
starting upgrade. This operation is useful when doing
tests such as instance live migration during upgrade or
floating ip connectivity testing during upgrade. The
script requires a network defined in the external_network_name
var which provides external connectivity to exist beforehand.
Change-Id: Ib39e41b36fac7794ea515c8a9d56141866dcfeed
Fix pre compute upgrade check
This change adds the MIGRATING state to be checked before the
compute nodes upgrade.
Change-Id: I0073a7e69a71a044882d4760dbb49cd4f455dd89
Fix workload_launch position
Change-Id: I6193ac6a60165bb20ece6277067c05696ed6d3b1
Run non controller node pre upgrade script
This change runs the non controller node pre upgrade script.
In addition it exposes the option to run instances migration
between compute nodes during upgrade.
Change-Id: Ief55eecdc85bb620f637c4ed4d9b5bc3243b37d1
Update roles_data adjustments to latest changes
This change updates the roles_data file adjustments to the latest
changes.
Change-Id: Ic787b135cdf96b33829e05140e069b398df7196f
Use docker and docker-ha environment files for upstream deployments
Change-Id: I70bb9767e97f616729adff983fff065858a6dcdc
Convert services environments to services-docker only for upstream
Per BZ#116463 in downstream the environments used for extra service
enablement now point to docker resources.
Change-Id: I379622ec2749ac8b485aec79a7500308ef74214e
Echo debug message to differentiate live migration from block
Change-Id: I69e7f381543d84fcd308acbd3a90f5d0ac23ae1b
Accept <= 5% ICMP packet loss during upgrade connectivity check
Change-Id: I34d1de225c0e391035e22e18f63356e04afbbfd5
Reorganize playbooks to separate upgrade/update
This change adds separate directories for upgrade/update which
provides a better separation between updates and upgrades.
Change-Id: Icf1a09514fb0e6236535ae32265bbd3805918478
Run block migrate multiple times
Block migrate doesn't work seem to work if triggered once but it
does if the command is run for a second time. This change runs the
block migrate command multiple times to make sure that the instance
gets migrated.
Change-Id: I8b9a9ecae21f7ce49a03945afec66b9e671622b7
Ensure files/ are part of the setup.cfg files to copy
When installing tripleo-upgrade into a .quickstart
environment, the files/ folder wasn't getting copied, which is necessary
at least for "adjust ssh config to skip host key check" in
create-upgrade-scripts.yaml.
Change-Id: I7d862ec5c13ba719923c90cc40790b842b582999
Add tasks for undercloud minor update
Start adding the minor undercloud update tasks
Change-Id: I33705b270e2d5e6a28f1cad8179e1f4b3e4ea975
Remove timeouts from upgrade scripts
Depending on the number of deployed nodes upgrade could take longer
and we want to rely on the heat stack timeout. This change clears
any manual timeouts set in the upgrade scripts.
Change-Id: I5d141e2cc13621d3be5fb0c27b0ac3c3fc30d424
Minor updates of RHOS 12.
Manage minor update workflow from within tripleo-upgrade repo.
Change-Id: I8c6771af4825ce166e8470413ca4687be0a58cb9
Reboot controller nodes post upgrade
This change adds the option to reboot controller nodes post upgrade
and performs basic verifications that the clustered services are
reported as up.
Change-Id: I370d421e5968ae50bd1ff140cdfcf98a4db03a5f
Don't force ssh_config on everybody.
This add an option to be able to not overwrite the ssh_config file.
As a side note the ssh_config is missing from the repo, so by default
this task is broken.
Change-Id: Idfb78e2b7226a7e6295acd3f250bbfb48d0a103d
Fix filter used in the node upgrade scripts
This change is fixes the current filter used for node upgrade
scripts so that deployment with $domain.tld format are supported.
Change-Id: I18f43c440bb93e0fcefb664c7d716ff9368673a4
Run live migration multiple times
Change-Id: I7c028defd3cb9080efa7bdbe9daa6ed201df8640
Manually inject undercloud SSL cert to overcloud nodes
Per BZ#1501779 the compute nodes do not get their trusted store
updated when using a CAMap and upgrade fails. This change updates
the overcloud nodes trusted store manually so the overcloud nodes
are prepared for update. This should translate in a documentation
step that before upgrade starts the user needs to make sure the
overcloud nodes are able to reach the undercloud SSL public
endpoint.
Change-Id: Ib95a29c608803504a866ae71cbc0082faf3c194f
Replace puppet external ceph environment with ceph-ansible one
This change replaces during the upgrade the external ceph puppet
environment file with its ceph-ansible equivalent.
Change-Id: I9020e8f7c43f91259b551caa2e20f03be1424106
Append deprecated params only to predefined roles
We should append the deprecated params only to predefined roles
in order to avoid failures such as reported in BZ#1501237.
Change-Id: I3a7c332b35da9639fb6f8e5b38234dc0c55d8499
Split the post controller scripts into per services scripts
This change splits the post upgrade controller scripts into
per service checks and adds them to a common directory so
they can be shared between update and upgrade.
Change-Id: I8f2fb6162a5acb8a92057400a7b04e6e2388abaa
Add the ability to specify a remote docker registry
This change adds the ability to specify a remote docker registry
to be used for downloading the Docker images on the undercloud or
be used directly by the overcloud nodes during upgrade.
Change-Id: I132a8b94f9a101d1c9c624d202bb01527dc2b844
Fix BZ#1499677 workaround condition
In addition to empty gvwstate.dat file there might be situations
where the gvwstate.dat file is missing after reboot. This patch
addresses this condition for BZ#1499677 workaround.
Change-Id: I295b133248f48ab41b1748225cbe9359662b280d
Cleanup galera resource instead of rebooting node for BZ#1499677
Instead of rebooting the node while implementing the workaround for
BZ#1499677 we should simply cleanup the Galera resource. This way
we can save some time and potential issues caused by an additional
reboot.
Change-Id: I391daeae41321baec1cbd8c458132a3161cd96d5
[UPDATES] Introduce option for minor updates workarounds.
To speed up testing of minor updates it might be required
to apply some patches before they are landed.
Hence we need a flag to differentiate if workarounds are required
or not
Change-Id: I642e4ade204f5fd30ec9433f1d90a2d539287c5e
Do not pipe curl output in container images environment script
Curl can sometimes exit with exit code 23 when its output is piped
into another command. To avoid this errors we save the curl output
to a file.
Change-Id: I4123b6c66ae2873c11631f229cb8e3eec5a5a66b
Use service environment files when generating the images environment
In the last build openstack overcloud container image prepare only
generates the parameters for the services included by default. In
order to make it work when extra non-default services are enabled
we need to pass the environment files to the prepare command. This
change addresses this issue.
Change-Id: I86ab6faaffcd4c7cc1a07e9a6ed1e890cb5cf980
Place the oooq deploy command into overcloud_deploy_script var
This change places the openstack overcloud deploy command with its
arguments in the location defined by the overcloud_deploy_script
var. This way we don't require oooq users to specifically set the
overcloud_deploy_script to a hardcoded location and make it less
confusing.
Change-Id: Id2b14fcffbd169c342df4b5b9105dff81e18e3a0
Replace ceph radosgw environment during upgrade
Change-Id: I489211f39941bba5b1ca2ddf1b635c3bdb0151fe
Avoid losing undercloud connection in TripleO CI.
When running the role in the TripleO upstream CI
the connectivity to the undercloud gets lost when
rebooting the undercloud after upgrade or killing
the ssh service, this makes the playbook fail.
As a solution, a flag tripleo_ci has been added.
This flag will default to false, and when set to
true no undercloud reboot, nor ssh killing will
be executed.
Change-Id: If4a303fff49bbe55cdfb7142d8dd69264ab47ab4
Align deployment-files option with IR.
deployment-files option is not a list of choices in IR,
adjust it accordingly.
Change-Id: I6889e9b75f842cc466278fed5dbf85a80cb58ee0
Append docker-ha only when needed.
Before appending the docker-ha.yaml env
file, we need to check if the overcloud
was deployed with pacemaker. If so, then
we'll add the env file to the upgrade
script.
Change-Id: I9867d86b6d23385c576d2f8c5a25ab3333f7113d
Specify tht directory used in upgrade script.
When deploying with oooq, the generated script
overcloud-deploy.sh is reused in order to append
the cooresponding env files for the upgrade.
However, if the location of the tripleo-heat-templates
directory is different from the used when deploying
then the upgrade does not succeed.
This change modifies the tripleo-heat-templates
location used to upgrade when the directory found
in overcloud-deploy.sh is different. If it is the
same no change is done.
Change-Id: I55ada3e75b7463b1c14c8734410d2591cf162e67
Don't append DockerInsecureRegistryAddress
This is no longer required as the prepare command detects whether the
registry is secure and DockerInsecureRegistryAddress as necessary.
Depends on upstream https://review.openstack.org/#/c/514473/
Related-Bug: #1722632
Change-Id: Ia9d91f6280600c59d0079c5d1f26a00f04040426
Fix the controller regex during roles_data conversion
The Ocata roles_data controller role might include a comment for
the controller role name. This change adjusts the regexp used during
the roles data conversion to take into consideration that comment.
Change-Id: I7b43b1fcb9e477de8e1265ef6aa6ba5149e82d47
Use prepare --set for ceph image parameter
This is more maintainable, and consistent with other uses of prepare.
Change-Id: Ieec88e271973a248192c2b247cd2c5e0cccbfb7c
Add storage environment files to be used for containers prepare
This change adds the storage environments files to be matched when
creating the environment file containing the Docker images names
which gets created via container image prepare.
Change-Id: I34c3cdaab1b63ce3f43d748372d35143bc12b8b4
Add environment file containing required DPDK changes during upgrade
Change-Id: Iabaf16e18ee7546bd6275f8f84226892423a6c95
Create failed_upgrade log files.
Most of the stack failures in the TripleO
CI are registered inside two log files
failed_upgrade_list.log and failed_upgrade.log.
This patch adds the option of inject the
stack failures list command into these two
log files, as well as priting out the detailed
stack failure list (--long).
Change-Id: I4fad989818f67ad0a73e45b47f835750f18c3bb6
Replace storage-environemnt.yaml for upstream only
Per BZ#1502862 the Ceph environment files switch during upgrade is
only needed for upstream deployments. This change does the changes
to accommodate this.
Change-Id: Ia3adb120c9b524c66d069593b0779b3399295fd4
Do not update python-openstackclient before upgrade
BZ#1488471 was fixed so there's no need to update the python-
openstackclient package before upgrade.
Change-Id: I2aba7515ec43926ec4f8de5c701467dea31dba1b
Be more aggressive on accepted packet loss during upgrade
Tests have shown that the packet loss shouldn't eceed 1% during
the upgrade steps. This change adjusts the accepted level to this.
Change-Id: I9e47ea56a78e4e9eab40fca609cbedaecfcf1e14
Add more tags for the upgrade process.
This enable one to either do only a small part of the whole process.
This can be useful for debugging or development.
Change-Id: Ic6cc9a1e6aa2793fde65636d2ad92bc174173252
Use new method of discovering tag and adjust local registry upload
This change uses the new mechanism of retrieving the tag from the
latest image provided in the registry.
Change-Id: I7e063f13c7d4812e9986452774881235b620bd0e
Swap baremetal environment file for containerized.
In oooq, when upgrading from baremetal
to containerized overcloud two different
environment files are used. These env
files are located in [0].
When upgrading using tripleo-upgrade, we
need to convert that environment file name
to its corresponding containers version.
[0] https://github.com/openstack/tripleo-heat-templates/tree/master/ci/environments
Change-Id: I6c9fad2f402a162cf663c5089e79c2e10f3d0928
Add condition to create local docker registry.
The only way to not execute the docker
registry environment file creation task
is via tags, which is not easy to handle
in TripleO CI.
As tripleo-quickstart already prepares
the local docker registry file, there
is no need to execute it. So a new
parameter 'create_docker_registry' is
been added.
Also, the 'force' option is being added
to avoid overwritting a provided script
with the role template.
Change-Id: I800d6696b8dbb83f05f3d9381c6e5689558f4b77
Prepare workloads before update/upgrade.
Prepare scripts for managing workload on oc before running
update/upgrade operation.
Allow to run ping test during minor update.
Change-Id: I1d5754f36f53588c97c646aa4e1380e9ca5938bc
Remove tag parsing from the image prepare command
Parsing the tag is not needed anymore and the one returned by the
container image prepare command can be used.
Change-Id: If782fc655da7b22e4d4a803509e9cc8c49774368
[UPDATES] Run minor update per role.
With recent changes it's advised to perform minor update in batches:
role-by-role.
This change limits the scope of update with '--nodes <Role>' option.
Change-Id: I0bc03873b749dc9c15b13cacbfff78cead4360d8
4572055ffb