x
/
fuel-plugin-contrail
Code Issues Proposed changes

Contrail packages and configuration upgrade using custom task graphs 

Change-Id: Iadd64b7b0a63eba6adfa25a1ed37910d3235c059
Signed-off-by: Illia Polliul <ipolliul@mirantis.com>
This commit is contained in:
Oleksandr Martsyniuk 2016-09-02 15:47:25 +03:00 committed by Illia Polliul
parent 77623586f1
commit d119971b94
6 changed files with 267 additions and 306 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
doc/source/limitations.rst
View File

@ -19,3 +19,6 @@ Limitations
* Changing the default OpenStack project name is not supported. Default project name should be ``admin``.
* The password of OpenStack ``admin`` user should not contain following characters: ``$``, `````, ``\\`` and ``!``
* Upgrade procedure based on custom deployment graphs does not support upgrading nodes with roles dpdk, tsn and vmware.
Contrail packages upgrades should be done manually on these nodes.

59
doc/source/upgrade.rst
View File

@ -12,8 +12,10 @@ to be upgraded without touching other OpenStack components.
The packages are updated using the plugin-based repository, and configuration files
are updated using the templates included in the latest plugin version.
Controllers and compute nodes are upgraded separately, using puppet manifests
provided with plugin. Other contrail-specific roles such as DPDK-compute, SR-IOV-compute,
provided with plugin. Other contrail-specific roles such as DPDK-compute, VMWare-compute,
and TSN are not supported yet.
More information on running custom deployment graphs can be found in
`Fuel User Guide <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/maintain-environment/custom-graph.html>`_
Prequisites
-----------
@ -29,6 +31,9 @@ Package versions supported:
Update the packages on Fuel Master node
---------------------------------------
In case Fuel Contrail plugin package should be upgraded, please execute steps 1-3, otherwise
proceed to step 4.
#. Obtain the latest package of Fuel Contrail plugin that supports your Fuel version.
#. Copy the rpm package downloaded at previous step to the Fuel Master node
@ -49,7 +54,7 @@ Update the packages on Fuel Master node
.. code-block:: console
scp contrail-install-packages_3.0.2.1-4~liberty_all.deb \
scp contrail-install-packages_3.1.0.0-25~Ubuntu-14.04.4-mitaka_all.deb \
<Fuel Master node ip>:/var/www/nailgun/plugins/contrail-5.0/
ssh <Fuel Master node ip> /var/www/nailgun/plugins/contrail-5.0/install.sh
@ -75,11 +80,31 @@ starts. The Neutron service will be restarted in case if contrail core plugin wi
ssh <the Fuel Master node ip>
cd /var/www/nailgun/plugins/contrail-5.0/
#. Start the upgrade of control plane:
# Check ID of contrail plugin:
.. code-block:: console
fuel plugins
# Check ID of your env:
.. code-block:: console
fuel2 env list
# Upload upgrade graph:
.. code-block:: console
fuel2 graph upload --plugin <plugin-ID> --type contrail_upgrade_control --file upgrade_control.yaml
#Verify the graph has been uploaded:
.. code-block:: console
./upgrade.sh controllers
fuel2 graph list --env <env-ID>
# Execute the custom graph to upgrade control plane:
.. code-block:: console
fuel2 graph execute --env <env-ID> --type contrail_upgrade_control
#. Run the contrail service verification steps from :doc:`/verification` to ensure that all
Contrail services are up and running.
@ -96,18 +121,38 @@ The task upgrades compute hosts one by one, in ascending order by node ID.
The instances running on particular compute node will lose network connectivity
during the vRouter upgrade, this can take up to 5 min.
#. Log in to the Fuel Master node, change the working directory to the plugin's folder:
#. Log in to Fuel Master node, change the working directory to plugin folder:
.. code-block:: console
ssh <the Fuel Master node ip>
cd /var/www/nailgun/plugins/contrail-5.0/
#. Start the upgrade of control plane:
# Check ID of contrail plugin:
.. code-block:: console
fuel plugins
# Check ID of your env:
.. code-block:: console
fuel2 env list
# Upload upgrade graph:
.. code-block:: console
fuel2 graph upload --plugin <plugin-ID> --type contrail_upgrade_compute --file upgrade_compute.yaml
#Verify the graph has been uploaded:
.. code-block:: console
./upgrade.sh computes
fuel2 graph list --env <env-ID>
# Execute the custom graph to upgrade compute hosts:
.. code-block:: console
fuel2 graph execute --env <env-ID> --type contrail_upgrade_compute
#. Log in to compute nodes and verify output of the ``contrail-status`` command.
You can verify the version of the vRouter package by running ``contrail-version`` command.

181
upgrade.sh
View File

@ -1,181 +0,0 @@
#!/bin/bash
#
# Copyright 2016 Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
# This script provides the functions to upgrade the contrail packages and config files
# for existing Mirantis OpenStack environment deployed wth Fuel Contrail Plugin.
# It makes use of puppet manifests located in deployment_scripts/upgrade folder
# and contrail puppet module shipped within plugin.
#
# Upgrade steps
# 1. Upgrade contrail plugin to version 3.0.1+
# 2. Copy Contrail distribution packages to plugin folder on master node.
# 3. Run ./install.sh to populate the repositories with new packages.
# 4. Start the upgrade of control plane with ./upgrade.sh controllers.
# 5. After control plane has been upgraded, run ./upgrade.sh computes.
#
# Please note, that neutron operations will be unavailable during the process
# of contrail nodes upgrade (15-20 min window).
# The instances on compute nodes will lose network connectivity during the vrouter
# upgrade on particular compute node (upto 5 min).
# Functions
# Returns a list of nodes with role defined in argument, which are in 'ready' state,
# or a list of all ready nodes if no argument supplied
get_nodes_list () {
local _role
_role=$1
if [ -z "$_role" ]; then
_nodes=$(fuel node 2>/dev/null | grep 'ready' | cut -d "|" -f 1,7 | awk '{printf $1 ","}' | sed -e 's|,$||')
elif [ "$_role" == "compute" ]; then
_nodes=$(fuel node 2>/dev/null | grep 'ready' | cut -d "|" -f 1,7 | grep "$_role" | grep -v compute-vmware | awk '{printf $1 ","}' | sed -e 's|,$||')
elif [ "$_role" == "analytics" ]; then
_nodes=$(fuel node 2>/dev/null | grep 'ready' | cut -d "|" -f 1,7 | grep "$_role" | grep -v contrail-analytics-db | awk '{printf $1 ","}' | sed -e 's|,$||')
else
_nodes=$(fuel node 2>/dev/null | grep 'ready' | cut -d "|" -f 1,7 | grep "$_role" | awk '{printf $1 ","}' | sed -e 's|,$||')
fi
echo "$_nodes"
}
# Run the task on particular node
start_task_on_node () {
local _nodes=$1
local _tasks=$2
fuel node --node-id "$_nodes" --task "$_tasks" 2>/dev/null
}
# Wait for task to complete or fail
wait_for_tasks () {
local _timeout=60
local _status
while true ; do
_status=$(fuel task 2>/dev/null | sort -n | tail -1 | cut -d "|" -f 2 | tr -d " ")
case $_status in
ready)
break
;;
error)
exit 1
;;
*)
sleep $_timeout
;;
esac
done
}
# Steps
# Check the args, exit if none
[ $# -eq 0 ] && { echo "Usage: $0 <controllers|computes>"; exit 1; }
# Add the upgrade tasks to plugins deployment_tasks as skipped
grep -q "# Contrail upgrade tasks" deployment_tasks.yaml || cat upgrade_tasks.yaml >> deployment_tasks.yaml
# Sync the deployment tasks
fuel plugins --sync 2>/dev/null
# Rsync the plugins manifests
nodes=$(get_nodes_list)
start_task_on_node "$nodes" rsync_core_puppet
wait_for_tasks
# Run the upgrade tasks
case $1 in
controllers*)
# Start the pre-upgrade tasks on contrail nodes
nodes=$(get_nodes_list contrail)
start_task_on_node "$nodes" upgrade-contrail-pre
wait_for_tasks
# Stop the contrail config services
nodes=$(get_nodes_list contrail-config)
start_task_on_node "$nodes" upgrade-contrail-stop-config
wait_for_tasks
# Stop the contrail collector services
nodes=$(get_nodes_list contrail-analytics)
if [ -n "$nodes" ]; then
start_task_on_node "$nodes" upgrade-contrail-stop-analytics
else
nodes_cfg=$(get_nodes_list contrail-config)
start_task_on_node "$nodes_cfg" upgrade-contrail-stop-analytics
fi
wait_for_tasks
# Start the upgrade tasks on database nodes
nodes=$(get_nodes_list contrail-db)
start_task_on_node "$nodes" upgrade-contrail-db
wait_for_tasks
# Start the upgrade tasks on config nodes
nodes=$(get_nodes_list contrail-config)
start_task_on_node "$nodes" upgrade-contrail-config
wait_for_tasks
# Start the upgrade tasks on analytics database nodes
nodes=$(get_nodes_list contrail-analytics-db)
start_task_on_node "$nodes" upgrade-contrail-db
wait_for_tasks
# Start the upgrade tasks on collector nodes
nodes=$(get_nodes_list contrail-analytics)
if [ -n "$nodes" ]; then
start_task_on_node "$nodes" upgrade-contrail-analytics
else
nodes_cfg=$(get_nodes_list contrail-config)
start_task_on_node "$nodes_cfg" upgrade-contrail-analytics
fi
wait_for_tasks
# Start the upgrade tasks on control nodes, one by one
nodes=$(get_nodes_list contrail-control)
for node in ${nodes//,/ }
do
start_task_on_node "$node" upgrade-contrail-control
wait_for_tasks
done
# Start the post-upgrade tasks on contrail nodes
nodes=$(get_nodes_list contrail)
start_task_on_node "$nodes" upgrade-contrail-post
wait_for_tasks
# Start the upgrade tasks on config nodes
nodes=$(get_nodes_list controller)
start_task_on_node "$nodes" upgrade-contrail-controller
wait_for_tasks
;;
computes*)
# Start the repo tasks on compute nodes
nodes=$(get_nodes_list compute)
start_task_on_node "$nodes" contrail-repository
wait_for_tasks
# Start the upgrade tasks on compute nodes, one by one
nodes=$(get_nodes_list compute)
for node in ${nodes//,/ }
do
start_task_on_node "$node" upgrade-contrail-compute
wait_for_tasks
done
;;
*)
echo "Incorrect args specified"; exit 1
;;
esac

50
upgrade_compute.yaml Normal file
View File

@ -0,0 +1,50 @@
# Contrail upgrade tasks
# How to use:
# 1. Check ID of contrail plugin:
# fuel plugins
# 2. Check ID of your env:
# fuel2 env list
# 3. Upload upgrade graph:
# fuel2 graph upload --plugin <plugin-ID> --type contrail_upgrade_compute --file upgrade_tasks.yaml
# 4. Verify the graph has been uploaded:
# fuel2 graph list --env <env-ID>
# 5. Execute the custom graph to upgrade contrail:
# fuel2 graph execute --env <env-ID> --type contrail_upgrade_compute
- id: plugins_rsync
type: puppet
version: 2.1.0
groups: ['/.*/']
required_for: [pre_deployment_start]
condition:
yaql_exp: &have_plugins "len($.plugins) > 0"
parameters:
puppet_manifest: /etc/puppet/modules/osnailyfacter/modular/plugins/plugins_rsync.pp
puppet_modules: /etc/puppet/modules
timeout: 600
- id: plugins_setup_repositories
type: puppet
version: 2.1.0
groups: ['/.*/']
requires: [plugins_rsync]
required_for: [pre_deployment_start]
condition:
yaql_exp: *have_plugins
parameters:
puppet_manifest: /etc/puppet/modules/osnailyfacter/modular/plugins/plugins_setup_repositories.pp
puppet_modules: /etc/puppet/modules
timeout: 600
- id: upgrade-contrail-compute
type: puppet
version: 2.0.0
required_for: [deploy_end]
requires: [deploy_start]
groups: [compute]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-compute.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
strategy:
type: one_by_one

162
upgrade_control.yaml Normal file
View File

@ -0,0 +1,162 @@
# Contrail upgrade tasks
# How to use:
# 1. Check ID of contrail plugin:
# fuel plugins
# 2. Check ID of your env:
# fuel2 env list
# 3. Upload upgrade graph:
# fuel2 graph upload --plugin <plugin-ID> --type contrail_upgrade_control --file upgrade_tasks.yaml
# 4. Verify the graph has been uploaded:
# fuel2 graph list --env <env-ID>
# 5. Execute the custom graph to upgrade contrail:
# fuel2 graph execute --env <env-ID> --type contrail_upgrade_control
- id: plugins_rsync
type: puppet
version: 2.1.0
groups: ['/.*/']
required_for: [pre_deployment_start]
condition:
yaql_exp: &have_plugins "len($.plugins) > 0"
parameters:
puppet_manifest: /etc/puppet/modules/osnailyfacter/modular/plugins/plugins_rsync.pp
puppet_modules: /etc/puppet/modules
timeout: 600
- id: plugins_setup_repositories
type: puppet
version: 2.1.0
groups: ['/.*/']
requires: [plugins_rsync]
required_for: [pre_deployment_start]
condition:
yaql_exp: *have_plugins
parameters:
puppet_manifest: /etc/puppet/modules/osnailyfacter/modular/plugins/plugins_setup_repositories.pp
puppet_modules: /etc/puppet/modules
timeout: 600
- id: upgrade-contrail-pre
type: puppet
version: 2.0.0
required_for: [deploy_end]
requires: [deploy_start]
groups: [primary-contrail-controller,contrail-controller,
primary-contrail-analytics,contrail-analytics,
primary-contrail-analytics-db, contrail-analytics-db]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-pre.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-stop-config
type: puppet
version: 2.0.0
required_for: [deploy_end]
requires: [upgrade-contrail-pre]
groups: [primary-contrail-controller,contrail-controller]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-stop-config.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-stop-analytics
type: puppet
version: 2.0.0
required_for: [deploy_end]
requires: [upgrade-contrail-pre]
groups: [primary-contrail-analytics,contrail-analytics]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-stop-analytics.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-db
type: puppet
version: 2.0.0
required_for: [deploy_end]
requires: [upgrade-contrail-pre]
groups: [primary-contrail-controller,contrail-controller,primary-contrail-analytics-db, contrail-analytics-db]
cross-depends:
- name: upgrade-contrail-stop-config
- name: upgrade-contrail-stop-analytics
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-db.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-config
type: puppet
version: 2.0.0
required_for: [deploy_end]
requires: [upgrade-contrail-pre]
groups: [primary-contrail-controller,contrail-controller]
cross-depends:
- name: upgrade-contrail-stop-config
- name: upgrade-contrail-db
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-config.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-analytics
type: puppet
version: 2.0.0
required_for: [deploy_end]
requires: [upgrade-contrail-stop-analytics]
groups: [primary-contrail-controller,contrail-controller,primary-contrail-analytics,contrail-analytics]
cross-depends:
- name: upgrade-contrail-config
- name: upgrade-contrail-db
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-analytics.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-control
type: puppet
version: 2.0.0
required_for: [deploy_end]
requires: [upgrade-contrail-pre]
groups: [primary-contrail-controller,contrail-controller]
cross-depends:
- name: upgrade-contrail-config
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-control.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
strategy:
type: one_by_one
- id: upgrade-contrail-post
type: puppet
version: 2.0.0
required_for: [deploy_end]
requires: [deploy_start,upgrade-contrail-pre]
groups: [primary-contrail-controller,contrail-controller,
primary-contrail-analytics,contrail-analytics,
primary-contrail-analytics-db, contrail-analytics-db]
cross-depends:
- name: upgrade-contrail-db
- name: upgrade-contrail-config
- name: upgrade-contrail-analytics
- name: upgrade-contrail-control
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-post.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-controller
type: puppet
version: 2.0.0
required_for: [deploy_end]
requires: [deploy_start]
groups: [primary-controller,controller]
cross-depends:
- name: upgrade-contrail-config
- name: upgrade-contrail-analytics
- name: upgrade-contrail-control
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-controller.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720

118
upgrade_tasks.yaml
View File

@ -1,118 +0,0 @@
# Contrail upgrade tasks
# DO NOT MODIFY THE FOLLOWING LINES
- id: upgrade-contrail-pre
skipped: true
type: puppet
required_for: [post_deployment_end]
requires: [post_deployment_start]
role: [primary-contrail-db, contrail-db,
primary-contrail-config,contrail-config,
primary-contrail-analytics,contrail-analytics,
primary-contrail-control,contrail-control]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-pre.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-stop-config
skipped: true
type: puppet
required_for: [post_deployment_end]
requires: [post_deployment_start]
role: [primary-contrail-config,contrail-config]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-stop-config.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-stop-analytics
skipped: true
type: puppet
required_for: [post_deployment_end]
requires: [post_deployment_start]
role: [primary-contrail-config,contrail-config,primary-contrail-analytics,contrail-analytics]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-stop-analytics.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-db
skipped: true
type: puppet
required_for: [post_deployment_end]
requires: [post_deployment_start]
role: [primary-contrail-db, contrail-db]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-db.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-config
skipped: true
type: puppet
required_for: [post_deployment_end]
requires: [post_deployment_start]
role: [primary-contrail-config,contrail-config]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-config.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-analytics
skipped: true
type: puppet
required_for: [post_deployment_end]
requires: [post_deployment_start]
role: [primary-contrail-config,contrail-config,primary-contrail-analytics,contrail-analytics]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-analytics.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-control
skipped: true
type: puppet
required_for: [post_deployment_end]
requires: [post_deployment_start]
role: [primary-contrail-control,contrail-control]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-control.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-post
skipped: true
type: puppet
required_for: [post_deployment_end]
requires: [post_deployment_start]
role: [primary-contrail-db, contrail-db,
primary-contrail-config,contrail-config,
primary-contrail-analytics,contrail-analytics,
primary-contrail-control,contrail-control]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-post.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-controller
skipped: true
type: puppet
required_for: [post_deployment_end]
requires: [post_deployment_start]
role: [primary-controller,controller]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-controller.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720
- id: upgrade-contrail-compute
skipped: true
type: puppet
required_for: [post_deployment_end]
requires: [post_deployment_start]
role: [compute]
parameters:
puppet_manifest: puppet/upgrade/upgrade-contrail-compute.pp
puppet_modules: puppet/modules:/etc/puppet/modules
timeout: 720