openstack
/
ha-guide
Code Issues Proposed changes

Delete XML HA Guide 

Only publish the RST HA Guide.
Publish it under docs.openstack.org/ha-guide, so not as draft anymore.

Change-Id: I4afef0100d21aaa7215ac7f1ee49fdd4c3e7a4e3
This commit is contained in:
Andreas Jaeger 2015-10-12 09:37:09 +02:00
parent d344c74f4e
commit ad5004e927
71 changed files with 18 additions and 9832 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

37
README.rst
View File

@ -6,44 +6,13 @@ This repository contains the OpenStack High Availability Guide.
For more details, see the `OpenStack Documentation wiki page
<http://wiki.openstack.org/Documentation>`_.
Prerequisites
=============
`Apache Maven <http://maven.apache.org/>`_ must be installed to build the
documentation.
To install Maven 3 for Ubuntu 12.04 and later, and Debian wheezy and later::
apt-get install maven
On Fedora 20 and later::
yum install maven
On openSUSE 13.1 and later::
zypper ar http://download.opensuse.org/repositories/devel:/tools:/building/openSUSE_13.1/devel:tools:building.repo
zypper install maven
Building
========
The root directory of the *OpenStack High Availability Guide*
is ``doc/high-availability-guide``.
is ``doc/ha-guide``.
To build the guide, move into the directory ``doc/high-availability-guide``,
then run the ``mvn`` command in that directory::
cd doc/high-availability-guide/
mvn clean generate-sources
The generated PDF documentation file is::
doc/high-availability-guide/target/docbkx/webhelp/high-availability-guide/high-availability-guide.pdf
The root of the generated HTML documentation is::
doc/high-availability-guide/target/docbkx/webhelp/high-availability-guide/index.html
To build the guide, run ``tox -e docs``.
Testing of changes and building of the manual
=============================================
@ -55,8 +24,6 @@ gating jobs.
If you like to run individual tests, run:
* ``tox -e checkniceness`` - to run the niceness tests
* ``tox -e checksyntax`` - to run syntax checks
* ``tox -e checkdeletions`` - to check that no deleted files are referenced
* ``tox -e checkbuild`` - to actually build the manual
tox will use the openstack-doc-tools package for execution of these

12
doc-tools-check-languages.conf
View File

@ -2,14 +2,18 @@
# directories to be set up
declare -A DIRECTORIES=(
["ja"]="high-availability-guide glossary"
["zh_CN"]="high-availability-guide glossary"
)
# books to be built
declare -A BOOKS=(
["ja"]="high-availability-guide"
["zh_CN"]="high-availability-guide"
# No translations currently
# Example how to enable:
#["ja"]="ha-guide"
)
# draft books
declare -A DRAFTS=(
["ja"]="ha-guide"
)
# Where does the top-level pom live?

13
doc/high-availability-guide/api/section_api_pacemaker.xml
View File

@ -1,13 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="s-api-pacemaker">
<title>Configure Pacemaker group</title>
<para>Finally, we need to create a service <literal>group</literal>
to ensure that the virtual IP is linked to the API services
resources:</para>
<screen>group g_services_api p_api-ip p_keystone p_glance-api p_cinder-api \
p_neutron-server p_glance-registry p_ceilometer-agent-central</screen>
</section>

16
doc/high-availability-guide/api/section_api_vip.xml
View File

@ -1,16 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="s-api-vip">
<title>Configure the VIP</title>
<para>First, you must select and assign a virtual IP address (VIP)
that can freely float between cluster nodes.</para>
<para>This configuration creates <literal>p_ip_api</literal>, a
virtual IP address for use by the API node
(<literal>192.168.42.103</literal>):</para>
<screen>primitive p_api-ip ocf:heartbeat:IPaddr2 \
params ip="192.168.42.103" cidr_netmask="24" \
op monitor interval="30s"</screen>
</section>

72
doc/high-availability-guide/api/section_ceilometer_agent_central.xml
View File

@ -1,72 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="s-ceilometer-agent-central">
<title>Highly available Telemetry central agent</title>
<para>Telemetry (ceilometer) is the metering and monitoring service in
OpenStack. The Central agent polls for resource utilization
statistics for resources not tied to instances or compute nodes.</para>
<note>
<para>Due to limitations of a polling model, a single instance of this agent
can be polling a given list of meters, unless workload partitioning has
been configured for multiple central agents. In this setup, we install
this service on the API nodes also in the active / passive mode.</para>
</note>
<para>Making the Telemetry central agent service highly available in
active / passive mode involves managing its daemon with the Pacemaker
cluster manager.</para>
<note>
<para>You will find at <link xlink:href="http://docs.openstack.org/developer/ceilometer/install/manual.html#installing-the-central-agent">this page</link>
the process to install the Telemetry central agent.</para>
</note>
<section xml:id="_add_the_telemetry_central_agent_resource_to_pacemaker">
<title>Add the Telemetry central agent resource to Pacemaker</title>
<para>First of all, you need to download the resource agent to your system:</para>
<screen><prompt>#</prompt> <userinput>cd /usr/lib/ocf/resource.d/openstack</userinput>
<prompt>#</prompt> <userinput>wget https://raw.github.com/madkiss/openstack-resource-agents/master/ocf/ceilometer-agent-central</userinput>
<prompt>#</prompt> <userinput>chmod a+rx *</userinput></screen>
<para>You may then proceed with adding the Pacemaker configuration for
the Telemetry central agent resource. Connect to the Pacemaker cluster
with <literal>crm configure</literal>, and add the following cluster
resources:</para>
<programlisting>primitive p_ceilometer-agent-central \
ocf:openstack:ceilometer-agent-central \
params config="/etc/ceilometer/ceilometer.conf" \
op monitor interval="30s" timeout="30s"</programlisting>
<para>This configuration creates</para>
<itemizedlist>
<listitem>
<para><literal>p_ceilometer-agent-central</literal>, a resource for managing the
Ceilometer Central Agent service</para>
</listitem>
</itemizedlist>
<para><literal>crm configure</literal> supports batch input, so you may copy
and paste the above into your live pacemaker configuration, and then make
changes as required.</para>
<para>Once completed, commit your configuration changes by entering
<literal>commit</literal> from the <literal>crm configure</literal> menu.
Pacemaker will then start the Ceilometer Central Agent service, and its
dependent resources, on one of your nodes.</para>
</section>
<section xml:id="_configure_telemetry_central_agent_service">
<title>Configure Telemetry central agent service</title>
<para>Edit <filename>/etc/ceilometer/ceilometer.conf</filename>:</para>
<programlisting language="ini"># We use API VIP for Identity Service connection:
os_auth_url=http://192.168.42.103:5000/v2.0
# We send notifications to High Available RabbitMQ:
notifier_strategy = rabbit
rabbit_host = 192.168.42.102
[database]
# We have to use MySQL connection to store data:
sql_connection=mysql://ceilometer:password@192.168.42.101/ceilometer
# Alternatively, you can switch to pymysql,
# a new Python 3 compatible library and use
# sql_connection=mysql+pymysql://ceilometer:password@192.168.42.101/ceilometer
# and be ready when everything moves to Python 3.
# Ref: https://wiki.openstack.org/wiki/PyMySQL_evaluation</programlisting>
</section>
</section>

93
doc/high-availability-guide/api/section_cinder_api.xml
View File

@ -1,93 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="s-cinder-api">
<title>Highly available Block Storage API</title>
<para>Making the Block Storage (cinder) API service highly available in
active / passive mode involves:</para>
<itemizedlist>
<listitem>
<para>Configuring Block Storage to listen on the VIP address</para>
</listitem>
<listitem>
<para>Managing Block Storage API daemon with the Pacemaker cluster
manager</para>
</listitem>
<listitem>
<para>Configuring OpenStack services to use this IP address</para>
</listitem>
</itemizedlist>
<note>
<para>Here is the
<link xlink:href="http://docs.openstack.org/kilo/install-guide/install/apt/content/ch_cinder.html">documentation</link>
for installing Block Storage service.</para>
</note>
<section xml:id="_add_block_storage_api_resource_to_pacemaker">
<title>Add Block Storage API resource to Pacemaker</title>
<para>First of all, you need to download the resource agent to your
system:</para>
<screen><prompt>#</prompt> <userinput>cd /usr/lib/ocf/resource.d/openstack</userinput>
<prompt>#</prompt> <userinput>wget https://raw.github.com/madkiss/openstack-resource-agents/master/ocf/cinder-api</userinput>
<prompt>#</prompt> <userinput>chmod a+rx *</userinput></screen>
<para>You can now add the Pacemaker configuration for Block Storage API
resource. Connect to the Pacemaker cluster with <literal>crm configure</literal>,
and add the following cluster resources:</para>
<programlisting>primitive p_cinder-api ocf:openstack:cinder-api \
params config="/etc/cinder/cinder.conf" os_password="secretsecret" os_username="admin" \
os_tenant_name="admin" keystone_get_token_url="http://192.168.42.103:5000/v2.0/tokens" \
op monitor interval="30s" timeout="30s"</programlisting>
<para>This configuration creates</para>
<itemizedlist>
<listitem>
<para><literal>p_cinder-api</literal>, a resource for manage Block
Storage API service</para>
</listitem>
</itemizedlist>
<para><literal>crm configure</literal> supports batch input, so you may
copy and paste the above into your live pacemaker configuration, and
then make changes as required. For example, you may enter
<literal>edit p_ip_cinder-api</literal> from the <literal>crm configure</literal>
menu and edit the resource to match your preferred virtual IP address.</para>
<para>Once completed, commit your configuration changes by entering
<literal>commit</literal> from the <literal>crm configure</literal> menu.
Pacemaker will then start the Block Storage API service, and its dependent
resources, on one of your nodes.</para>
</section>
<section xml:id="_configure_block_storage_api_service">
<title>Configure Block Storage API service</title>
<para>Edit <filename>/etc/cinder/cinder.conf</filename>:</para>
<programlisting language="ini"># We have to use MySQL connection to store data:
sql_connection=mysql://cinder:password@192.168.42.101/cinder
# Alternatively, you can switch to pymysql,
# a new Python 3 compatible library and use
# sql_connection=mysql+pymysql://cinder:password@192.168.42.101/cinder
# and be ready when everything moves to Python 3.
# Ref: https://wiki.openstack.org/wiki/PyMySQL_evaluation
# We bind Block Storage API to the VIP:
osapi_volume_listen = 192.168.42.103
# We send notifications to High Available RabbitMQ:
notifier_strategy = rabbit
rabbit_host = 192.168.42.102</programlisting>
</section>
<section xml:id="_configure_openstack_services_to_use_highly_available_block_storage_api">
<title>Configure OpenStack services to use highly available Block
Storage API</title>
<para>Your OpenStack services must now point their Block Storage API
configuration to the highly available, virtual cluster IP address — rather
than a Block Storage API server’s physical IP address as you normally
would.</para>
<para>You must create the Block Storage API endpoint with this IP.</para>
<note>
<para>If you are using both private and public IP, you should create
two Virtual IPs and define your endpoint like this:</para>
</note>
<screen><prompt>$</prompt> <userinput>keystone endpoint-create --region $KEYSTONE_REGION \
--service-id $service-id --publicurl 'http://PUBLIC_VIP:8776/v1/%(tenant_id)s' \
--adminurl 'http://192.168.42.103:8776/v1/%(tenant_id)s' \
--internalurl 'http://192.168.42.103:8776/v1/%(tenant_id)s'</userinput></screen>
</section>
</section>

110
doc/high-availability-guide/api/section_glance_api.xml
View File

@ -1,110 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="s-glance-api">
<title>Highly available OpenStack Image API</title>
<para>The OpenStack Image service offers a service for discovering, registering,
and retrieving virtual machine images. To make the OpenStack Image API
service highly available in active / passive mode, you must:</para>
<itemizedlist>
<listitem>
<para>Configure the OpenStack Image service to listen on the VIP address.</para>
</listitem>
<listitem>
<para>Manage the OpenStack Image API daemon with the Pacemaker cluster
manager.</para>
</listitem>
<listitem>
<para>Configure OpenStack services to use this IP address.</para>
</listitem>
</itemizedlist>
<note>
<para>Here is the <link xlink:href="http://docs.openstack.org/kilo/install-guide/install/apt/content/ch_glance.html">documentation</link>
for installing the OpenStack Image API service.</para>
</note>
<section xml:id="_add_openstack_image_api_resource_to_pacemaker">
<title>Add OpenStack Image API resource to Pacemaker</title>
<para>First of all, you need to download the resource agent to your
system:</para>
<screen><prompt>#</prompt> <userinput>cd /usr/lib/ocf/resource.d/openstack</userinput>
<prompt>#</prompt> <userinput>wget https://raw.github.com/madkiss/openstack-resource-agents/master/ocf/glance-api</userinput>
<prompt>#</prompt> <userinput>chmod a+rx *</userinput></screen>
<para>You can now add the Pacemaker configuration for the OpenStack Image API
resource. Connect to the Pacemaker cluster with <literal>crm
configure</literal>, and add the following cluster resources:</para>
<programlisting>primitive p_glance-api ocf:openstack:glance-api \
params config="/etc/glance/glance-api.conf" os_password="secretsecret" \
os_username="admin" os_tenant_name="admin" os_auth_url="http://192.168.42.103:5000/v2.0/" \
op monitor interval="30s" timeout="30s"</programlisting>
<para>This configuration creates</para>
<itemizedlist>
<listitem>
<para><literal>p_glance-api</literal>, a resource for managing
OpenStack Image API service</para>
</listitem>
</itemizedlist>
<para><literal>crm configure</literal> supports batch input, so you may
copy and paste the above into your live Pacemaker configuration, and
then make changes as required. For example, you may enter <literal>edit
p_ip_glance-api</literal> from the <literal>crm configure</literal>
menu and edit the resource to match your preferred virtual IP address.</para>
<para>Once completed, commit your configuration changes by entering
<literal>commit</literal> from the <literal>crm configure</literal> menu.
Pacemaker will then start the OpenStack Image API service, and its
dependent resources, on one of your nodes.</para>
</section>
<section xml:id="_configure_openstack_image_service_api">
<title>Configure OpenStack Image service API</title>
<para>Edit <filename>/etc/glance/glance-api.conf</filename>:</para>
<programlisting language="ini"># We have to use MySQL connection to store data:
sql_connection=mysql://glance:password@192.168.42.101/glance
# Alternatively, you can switch to pymysql,
# a new Python 3 compatible library and use
# sql_connection=mysql+pymysql://glance:password@192.168.42.101/glance
# and be ready when everything moves to Python 3.
# Ref: https://wiki.openstack.org/wiki/PyMySQL_evaluation
# We bind OpenStack Image API to the VIP:
bind_host = 192.168.42.103
# Connect to OpenStack Image registry service:
registry_host = 192.168.42.103
# We send notifications to High Available RabbitMQ:
notifier_strategy = rabbit
rabbit_host = 192.168.42.102</programlisting>
</section>
<section xml:id="_configure_openstack_services_to_use_high_available_openstack_image_api">
<title>Configure OpenStack services to use high available OpenStack
Image API</title>
<para>Your OpenStack services must now point their OpenStack Image API
configuration to the highly available, virtual cluster IP address — rather
than an OpenStack Image API server’s physical IP address as you normally
would.</para>
<para>For OpenStack Compute, for example, if your OpenStack Image API
service IP address is <literal>192.168.42.103</literal> as in the
configuration explained here, you would use the following
configuration in your <filename>nova.conf</filename> file:</para>
<programlisting language="ini">[glance]
...
api_servers = 192.168.42.103
...</programlisting>
<note>
<para>In versions prior to Juno, this option was called
<literal>glance_api_servers</literal> in the
<literal>[DEFAULT]</literal> section.</para>
</note>
<para>You must also create the OpenStack Image API endpoint with this
IP.</para>
<note>
<para>If you are using both private and public IP addresses, you should
create two Virtual IP addresses and define your endpoint like this:</para>
</note>
<screen><prompt>$</prompt> <userinput>keystone endpoint-create --region $KEYSTONE_REGION \
--service-id $service-id --publicurl 'http://PUBLIC_VIP:9292' \
--adminurl 'http://192.168.42.103:9292' \
--internalurl 'http://192.168.42.103:9292'</userinput></screen>
</section>
</section>

102
doc/high-availability-guide/api/section_keystone.xml
View File

@ -1,102 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="s-keystone">
<title>Highly available OpenStack Identity</title>
<para>OpenStack Identity is the Identity Service in OpenStack and used by
many services. Making the OpenStack Identity service highly available in
active / passive mode involves</para>
<itemizedlist>
<listitem>
<para>Configure OpenStack Identity to listen on the VIP address,</para>
</listitem>
<listitem>
<para>Managing OpenStack Identity daemon with the Pacemaker cluster
manager,</para>
</listitem>
<listitem>
<para>Configure OpenStack services to use this IP address.</para>
</listitem>
</itemizedlist>
<note>
<para>Here is the <link xlink:href="http://docs.openstack.org/kilo/install-guide/install/apt/content/ch_keystone.html">documentation</link>
for installing OpenStack Identity service.</para>
</note>
<section xml:id="_add_openstack_identity_resource_to_pacemaker">
<title>Add OpenStack Identity resource to Pacemaker</title>
<para>First of all, you need to download the resource agent to your
system:</para>
<screen><prompt>#</prompt> <userinput>cd /usr/lib/ocf/resource.d</userinput>
<prompt>#</prompt> <userinput>mkdir openstack</userinput>
<prompt>#</prompt> <userinput>cd openstack</userinput>
<prompt>#</prompt> <userinput>wget https://raw.github.com/madkiss/openstack-resource-agents/master/ocf/keystone</userinput>
<prompt>#</prompt> <userinput>chmod a+rx *</userinput></screen>
<para>You can now add the Pacemaker configuration for OpenStack Identity
resource. Connect to the Pacemaker cluster with <literal>crm configure</literal>,
and add the following cluster resources:</para>
<programlisting>primitive p_keystone ocf:openstack:keystone \
params config="/etc/keystone/keystone.conf" os_password="secretsecret" \
os_username="admin" os_tenant_name="admin" os_auth_url="http://192.168.42.103:5000/v2.0/" \
op monitor interval="30s" timeout="30s"</programlisting>
<para>This configuration creates <literal>p_keystone</literal>, a resource
for managing the OpenStack Identity service.</para>
<para><literal>crm configure</literal> supports batch input, so you may
copy and paste the above into your live pacemaker configuration, and
then make changes as required. For example, you may enter
<literal>edit p_ip_keystone</literal> from the <literal>crm configure</literal>
menu and edit the resource to match your preferred virtual IP address.</para>
<para>Once completed, commit your configuration changes by entering
<literal>commit</literal> from the <literal>crm configure</literal> menu.
Pacemaker will then start the OpenStack Identity service, and its dependent
resources, on one of your nodes.</para>
</section>
<section xml:id="_configure_openstack_identity_service">
<title>Configure OpenStack Identity service</title>
<para>You need to edit your OpenStack Identity configuration file
(<filename>keystone.conf</filename>) and change the bind
parameters:</para>
<para>On Havana:</para>
<programlisting language="ini">bind_host = 192.168.42.103</programlisting>
<para>The <literal>admin_bind_host</literal> option lets you use a private
network for the admin access.</para>
<programlisting language="ini">public_bind_host = 192.168.42.103
admin_bind_host = 192.168.42.103</programlisting>
<para>To be sure all data will be highly available, you should be sure
that you store everything in the MySQL database (which is also highly
available):</para>
<programlisting language="ini">[catalog]
driver = keystone.catalog.backends.sql.Catalog
...
[identity]
driver = keystone.identity.backends.sql.Identity
...</programlisting>
</section>
<section xml:id="_configure_openstack_services_to_use_the_highly_available_openstack_identity">
<title>Configure OpenStack services to use the highly available OpenStack
Identity</title>
<para>Your OpenStack services must now point their OpenStack Identity
configuration to the highly available, virtual cluster IP address — rather
than a OpenStack Identity server’s physical IP address as you normally
would.</para>
<para>For example with OpenStack Compute, if your OpenStack Identity
service IP address is <literal>192.168.42.103</literal> as in the
configuration explained here, you would use the following line in your
API configuration file (<literal>api-paste.ini</literal>):</para>
<programlisting language="ini">auth_host = 192.168.42.103</programlisting>
<para>You also need to create the OpenStack Identity Endpoint with this
IP.</para>
<note>
<para>If you are using both private and public IP addresses, you should
create two Virtual IP addresses and define your endpoint like this:</para>
<screen><prompt>$</prompt> <userinput>keystone endpoint-create --region $KEYSTONE_REGION \
--service-id $service-id --publicurl 'http://PUBLIC_VIP:5000/v2.0' \
--adminurl 'http://192.168.42.103:35357/v2.0' \
--internalurl 'http://192.168.42.103:5000/v2.0'</userinput></screen>
</note>
<para>If you are using the horizon dashboard, you should edit the
<literal>local_settings.py</literal> file:</para>
<programlisting>OPENSTACK_HOST = 192.168.42.103</programlisting>
</section>
</section>

101
doc/high-availability-guide/api/section_neutron_server.xml
View File

@ -1,101 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="s-neutron-server">
<title>Highly available OpenStack Networking server</title>
<para>OpenStack Networking is the network connectivity service in OpenStack.
Making the OpenStack Networking Server service highly available in
active / passive mode involves the following tasks:</para>
<itemizedlist>
<listitem>
<para>Configure OpenStack Networking to listen on the virtual
IP address,</para>
</listitem>
<listitem>
<para>Manage the OpenStack Networking API Server daemon with the
Pacemaker cluster manager,</para>
</listitem>
<listitem>
<para>Configure OpenStack services to use the virtual IP address.</para>
</listitem>
</itemizedlist>
<note>
<para>Here is the <link xlink:href="http://docs.openstack.org/kilo/install-guide/install/apt/content/ch_networking.html">documentation</link>
for installing OpenStack Networking service.</para>
</note>
<section xml:id="_add_openstack_networking_server_resource_to_pacemaker">
<title>Add OpenStack Networking Server resource to Pacemaker</title>
<para>First of all, you need to download the resource agent to your
system:</para>
<screen><prompt>#</prompt> <userinput>cd /usr/lib/ocf/resource.d/openstack</userinput>
<prompt>#</prompt> <userinput>wget https://raw.github.com/madkiss/openstack-resource-agents/master/ocf/neutron-server</userinput>
<prompt>#</prompt> <userinput>chmod a+rx *</userinput></screen>
<para>You can now add the Pacemaker configuration for
OpenStack Networking Server resource. Connect to the Pacemaker cluster
with <literal>crm configure</literal>, and add the following cluster
resources:</para>
<programlisting>primitive p_neutron-server ocf:openstack:neutron-server \
params os_password="secretsecret" os_username="admin" os_tenant_name="admin" \
keystone_get_token_url="http://192.168.42.103:5000/v2.0/tokens" \
op monitor interval="30s" timeout="30s"</programlisting>
<para>This configuration creates <literal>p_neutron-server</literal>,
a resource for manage OpenStack Networking Server service</para>
<para><literal>crm configure</literal> supports batch input, so you may
copy and paste the above into your live pacemaker configuration, and
then make changes as required. For example, you may enter
<literal>edit p_neutron-server</literal> from the
<literal>crm configure</literal> menu and edit the resource to match
your preferred virtual IP address.</para>
<para>Once completed, commit your configuration changes by entering
<literal>commit</literal> from the <literal>crm configure</literal>
menu. Pacemaker will then start the OpenStack Networking API
service, and its dependent resources, on one of your nodes.</para>
</section>
<section xml:id="_configure_openstack_networking_server">
<title>Configure OpenStack Networking server</title>
<para>Edit <filename>/etc/neutron/neutron.conf</filename>:</para>
<programlisting language="ini"># We bind the service to the VIP:
bind_host = 192.168.42.103
# We bind OpenStack Networking Server to the VIP:
bind_host = 192.168.42.103
# We send notifications to Highly available RabbitMQ:
notifier_strategy = rabbit
rabbit_host = 192.168.42.102
[database]
# We have to use MySQL connection to store data:
connection = mysql://neutron:password@192.168.42.101/neutron
# Alternatively, you can switch to pymysql,
# a new Python 3 compatible library and use
# connection=mysql+pymysql://neutron:password@192.168.42.101/neutron
# and be ready when everything moves to Python 3.
# Ref: https://wiki.openstack.org/wiki/PyMySQL_evaluation</programlisting>
</section>
<section xml:id="_configure_openstack_services_to_use_highly_available_openstack_networking_server">
<title>Configure OpenStack services to use highly available OpenStack
Networking server</title>
<para>Your OpenStack services must now point their OpenStack Networking
Server configuration to the highly available, virtual cluster IP
address — rather than an OpenStack Networking server’s physical IP
address as you normally would.</para>
<para>For example, you should configure OpenStack Compute for using highly
available OpenStack Networking server in editing
<literal>nova.conf</literal> file:</para>
<programlisting language="ini">neutron_url = http://192.168.42.103:9696</programlisting>
<para>You need to create the OpenStack Networking server endpoint with
this IP.</para>
<note>
<para>If you are using both private and public IP addresses, you should
create two Virtual IP addresses and define your endpoint like
this:</para>
</note>
<screen><prompt>$</prompt> <userinput>keystone endpoint-create --region $KEYSTONE_REGION --service-id $service-id \
--publicurl 'http://PUBLIC_VIP:9696/' \
--adminurl 'http://192.168.42.103:9696/' \
--internalurl 'http://192.168.42.103:9696/'</userinput></screen>
</section>
</section>

111
doc/high-availability-guide/bk-ha-guide.xml
View File

@ -1,111 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<book xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="bk-ha-guide">
<title>OpenStack High Availability Guide</title>
<info>
<author>
<personname>
<firstname/>
<surname/>
</personname>
<affiliation>
<orgname>OpenStack Contributors</orgname>
</affiliation>
</author>
<copyright>
<year>2012</year>
<year>2013</year>
<year>2014</year>
<holder>OpenStack Contributors</holder>
</copyright>
<releaseinfo>current</releaseinfo>
<productname>OpenStack</productname>
<pubdate/>
<legalnotice role="apache2">
<annotation>
<remark>Copyright details are filled in by the template.</remark>
</annotation>
</legalnotice>
<abstract>
<para>This guide describes how to install, configure, and manage
OpenStack for high availability.</para>
</abstract>
<revhistory>
<revision>
<date>2015-04-30</date>
<revdescription>
<itemizedlist spacing="compact">
<listitem>
<para>This guide has various updates for the Kilo
release, such as adding MariaDB, updates to the
MySQL information, corosync and networking
updates.</para>
</listitem>
</itemizedlist>
</revdescription>
</revision>
<revision>
<date>2014-10-17</date>
<revdescription>
<itemizedlist spacing="compact">
<listitem>
<para>This guide has gone through editorial changes to
follow the OpenStack documentation
conventions. Various smaller issues have been
fixed.</para>
</listitem>
</itemizedlist>
</revdescription>
</revision>
<revision>
<date>2014-05-16</date>
<revdescription>
<itemizedlist spacing="compact">
<listitem>
<para>Conversion to DocBook.</para>
</listitem>
</itemizedlist>
</revdescription>
</revision>
<revision>
<date>2014-04-17</date>
<revdescription>
<itemizedlist spacing="compact">
<listitem>
<para>Minor cleanup of typos, otherwise no major revisions for
Icehouse release.</para>
</listitem>
</itemizedlist>
</revdescription>
</revision>
<revision>
<date>2012-01-16</date>
<revdescription>
<itemizedlist spacing="compact">
<listitem>
<para>Organizes guide based on cloud controller and compute nodes.</para>
</listitem>
</itemizedlist>
</revdescription>
</revision>
<revision>
<date>2012-05-24</date>
<revdescription>
<itemizedlist spacing="compact">
<listitem>
<para>Begin trunk designation.</para>
</listitem>
</itemizedlist>
</revdescription>
</revision>
</revhistory>
</info>
<xi:include href="http://git.openstack.org/cgit/openstack/openstack-manuals/plain/doc/common/ch_preface.xml"/>
<xi:include href="ch_intro.xml"/>
<xi:include href="part_active_passive.xml"/>
<xi:include href="part_active_active.xml"/>
<xi:include href="http://git.openstack.org/cgit/openstack/openstack-manuals/plain/doc/common/app_support.xml"/>
</book>

21
doc/high-availability-guide/ch_api.xml
View File

@ -1,21 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ch-api">
<title>API node cluster stack</title>
<para>The API node exposes OpenStack API endpoints onto external network (Internet).
It must talk to the cloud controller on the management network.</para>
<xi:include href="api/section_api_vip.xml"/>
<xi:include href="api/section_keystone.xml"/>
<xi:include href="api/section_glance_api.xml"/>
<xi:include href="api/section_cinder_api.xml"/>
<xi:include href="api/section_neutron_server.xml"/>
<xi:include href="api/section_ceilometer_agent_central.xml"/>
<xi:include href="api/section_api_pacemaker.xml"/>
</chapter>

15
doc/high-availability-guide/ch_controller.xml
View File

@ -1,15 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ch-controller">
<title>Cloud controller cluster stack</title>
<para>The cloud controller runs on the management network and must talk to all other services.</para>
<xi:include href="controller/section_mysql.xml"/>
<xi:include href="controller/section_rabbitmq.xml"/>
</chapter>

22
doc/high-availability-guide/ch_ha_aa_controllers.xml
View File

@ -1,22 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ha-aa-controllers">
<title>OpenStack controller nodes</title>
<para>OpenStack controller nodes contain:</para>
<itemizedlist>
<listitem>
<para>All OpenStack API services</para>
</listitem>
<listitem>
<para>All OpenStack schedulers</para>
</listitem>
<listitem>
<para><application>Memcached</application> service</para>
</listitem>
</itemizedlist>
<xi:include href="ha_aa_controllers/section_run_openstack_api_and_schedulers.xml"/>
<xi:include href="ha_aa_controllers/section_memcached.xml"/>
</chapter>

33
doc/high-availability-guide/ch_ha_aa_db.xml
View File

@ -1,33 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ha-aa-db">
<title>Database</title>
<para>The first step is installing the database that sits at the heart of
the cluster. When we talk about high availability (HA), we talk about several
databases (for redundancy) and a means to keep them synchronized. In
this case, we choose the <application>MySQL</application> database,
along with <application>Galera</application> plug-in for synchronous multi-master
replication.</para>
<para>The Galera Cluster plug-in is a multi-master Cluster based on
synchronous replication. It is a high availability solution, which provides
high system uptime, no data loss, and scalability
for growth.</para>
<note>
<para>The choice of database is not a foregone conclusion; you are not
required to use <application>MySQL</application>. It is, however, a
fairly common choice in OpenStack installations, so we will cover it
here.</para>
</note>
<note>
<para>MySQL with Galera is by no means the only way to achieve database HA.
MariaDB Galera Cluster (<link xlink:href="https://mariadb.org/">https://mariadb.org/</link>)
and Percona XtraDB Cluster (<link xlink:href="http://www.percona.com/">http://www.percona.com/</link>)
also work with Galera. You also have the option to use PostgreSQL, which
has its own replication, or another database HA option.</para>
</note>
<xi:include href="ha_aa_db/section_mysql_galera.xml"/>
<xi:include href="ha_aa_db/section_mariadb_galera_rh.xml"/>
</chapter>

185
doc/high-availability-guide/ch_ha_aa_haproxy.xml
View File

@ -1,185 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ha-aa-haproxy">
<title>HAProxy nodes</title>
<para>HAProxy is a very fast and reliable solution offering high availability, load balancing, and proxying
for TCP and HTTP-based applications. It is particularly suited for web sites crawling under very high loads
while needing persistence or Layer 7 processing. Supporting tens of thousands of connections is clearly
realistic with today’s hardware.</para>
<para>
For installing HAProxy on your nodes, you should consider its
<link xmlns:xlink="http://www.w3.org/1999/xlink"
xlink:href="http://haproxy.1wt.eu/#docs">official documentation</link>.
And also, you have to consider that HAProxy should not be a single point
of failure so you need to ensure its availability by other means,
such as Pacemaker or Keepalived. It is advisable to have multiple HAProxy
instances running, where the number of these instances is a small odd
number like 3 or 5. Also it is a common practice to collocate HAProxy
instances with existing OpenStack controllers.
</para>
<para>Here is an example for the HAProxy configuration file:</para>
<programlisting>global
chroot /var/lib/haproxy
daemon
group haproxy
maxconn 4000
pidfile /var/run/haproxy.pid
user haproxy
defaults
log global
maxconn 4000
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout check 10s
listen dashboard_cluster
bind &lt;Virtual IP&gt;:443
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:443 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:443 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:443 check inter 2000 rise 2 fall 5
listen galera_cluster
bind &lt;Virtual IP&gt;:3306
balance source
option httpchk
server controller1 10.0.0.4:3306 check port 9200 inter 2000 rise 2 fall 5
server controller2 10.0.0.5:3306 backup check port 9200 inter 2000 rise 2 fall 5
server controller3 10.0.0.6:3306 backup check port 9200 inter 2000 rise 2 fall 5
listen glance_api_cluster
bind &lt;Virtual IP&gt;:9292
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:9292 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:9292 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:9292 check inter 2000 rise 2 fall 5
listen glance_registry_cluster
bind &lt;Virtual IP&gt;:9191
balance source
option tcpka
option tcplog
server controller1 10.0.0.1:9191 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:9191 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:9191 check inter 2000 rise 2 fall 5
listen keystone_admin_cluster
bind &lt;Virtual IP&gt;:35357
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:35357 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:35357 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:35357 check inter 2000 rise 2 fall 5
listen keystone_public_internal_cluster
bind &lt;Virtual IP&gt;:5000
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:5000 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:5000 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:5000 check inter 2000 rise 2 fall 5
listen nova_ec2_api_cluster
bind &lt;Virtual IP&gt;:8773
balance source
option tcpka
option tcplog
server controller1 10.0.0.1:8773 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:8773 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:8773 check inter 2000 rise 2 fall 5
listen nova_compute_api_cluster
bind &lt;Virtual IP&gt;:8774
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:8774 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:8774 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:8774 check inter 2000 rise 2 fall 5
listen nova_metadata_api_cluster
bind &lt;Virtual IP&gt;:8775
balance source
option tcpka
option tcplog
server controller1 10.0.0.1:8775 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:8775 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:8775 check inter 2000 rise 2 fall 5
listen cinder_api_cluster
bind &lt;Virtual IP&gt;:8776
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:8776 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:8776 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:8776 check inter 2000 rise 2 fall 5
listen ceilometer_api_cluster
bind &lt;Virtual IP&gt;:8777
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:8777 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:8777 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:8777 check inter 2000 rise 2 fall 5
listen spice_cluster
bind &lt;Virtual IP&gt;:6080
balance source
option tcpka
option tcplog
server controller1 10.0.0.1:6080 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:6080 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:6080 check inter 2000 rise 2 fall 5
listen neutron_api_cluster
bind &lt;Virtual IP&gt;:9696
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.1:9696 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:9696 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:9696 check inter 2000 rise 2 fall 5
listen swift_proxy_cluster
bind &lt;Virtual IP&gt;:8080
balance source
option tcplog
option tcpka
server controller1 10.0.0.1:8080 check inter 2000 rise 2 fall 5
server controller2 10.0.0.2:8080 check inter 2000 rise 2 fall 5
server controller3 10.0.0.3:8080 check inter 2000 rise 2 fall 5</programlisting>
<para>After each change of this file, you should restart HAProxy.</para>
<note>
<para>Note that the Galera cluster configuration commands indicate
two of the three controllers are backup. That should be done to ensure
only one node serves write requests because OpenStack support for
multi-node writes is not production-ready yet.</para>
</note>
</chapter>

37
doc/high-availability-guide/ch_ha_aa_network.xml
View File

@ -1,37 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ha-aa-network">
<title>OpenStack network nodes</title>
<para>OpenStack network nodes contain:</para>
<itemizedlist>
<listitem>
<para>Neutron DHCP agent</para>
</listitem>
<listitem>
<para>Neutron L2 agent</para>
</listitem>
<listitem>
<para>Neutron L3 agent</para>
</listitem>
<listitem>
<para>Neutron metadata agent</para>
</listitem>
<listitem>
<para>Neutron LBaaS agent</para>
</listitem>
</itemizedlist>
<note>
<para>The neutron L2 agent does not need to be highly available. It has to
be installed on each data forwarding node and controls the virtual
networking drivers as Open vSwitch or Linux Bridge. One L2 agent runs per
node and controls its virtual interfaces. That's why it cannot be
distributed and highly available.</para>
</note>
<xi:include href="ha_aa_network/section_run_neutron_dhcp_agent.xml"/>
<xi:include href="ha_aa_network/section_run_neutron_l3_agent.xml"/>
<xi:include href="ha_aa_network/section_run_neutron_metadata_agent.xml"/>
<xi:include href="ha_aa_network/section_run_neutron_lbaas_agent.xml"/>
</chapter>

25
doc/high-availability-guide/ch_ha_aa_rabbitmq.xml
View File

@ -1,25 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ha-aa-rabbitmq">
<title>RabbitMQ</title>
<para>RabbitMQ is the default AMQP server used by many OpenStack services.
Making the RabbitMQ service highly available involves the following
steps:</para>
<itemizedlist>
<listitem>
<para>Install RabbitMQ</para>
</listitem>
<listitem>
<para>Configure RabbitMQ for HA queues</para>
</listitem>
<listitem>
<para>Configure OpenStack services to use Rabbit HA queues</para>
</listitem>
</itemizedlist>
<xi:include href="ha_aa_rabbitmq/section_install_rabbitmq.xml"/>
<xi:include href="ha_aa_rabbitmq/section_configure_rabbitmq.xml"/>
<xi:include href="ha_aa_rabbitmq/section_configure_openstack_services_to_use_rabbitmq.xml"/>
</chapter>

127
doc/high-availability-guide/ch_intro.xml
View File

@ -1,127 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ch-intro">
<title>Introduction to OpenStack High Availability</title>
<para>High Availability systems seek to minimize two things:</para>
<variablelist>
<varlistentry>
<term>System downtime</term>
<listitem><para>Occurs when a user-facing service is unavailable
beyond a specified maximum amount of time.</para></listitem>
</varlistentry>
<varlistentry>
<term>Data loss</term>
<listitem><para>Accidental deletion or destruction of
data.</para></listitem>
</varlistentry>
</variablelist>
<para>Most high availability systems guarantee protection against system
downtime and data loss only in the event of a single failure. However,
they are also expected to protect against cascading failures, where a
single failure deteriorates into a series of consequential failures.</para>
<para>A crucial aspect of high availability is the elimination of single
points of failure (SPOFs). A SPOF is an individual piece of equipment or
software which will cause system downtime or data loss if it fails.
In order to eliminate SPOFs, check that mechanisms exist for redundancy
of:</para>
<itemizedlist>
<listitem>
<para>Network components, such as switches and routers</para>
</listitem>
<listitem>
<para>Applications and automatic service migration</para>
</listitem>
<listitem>
<para>Storage components</para>
</listitem>
<listitem>
<para>Facility services such as power, air conditioning, and fire
protection</para>
</listitem>
</itemizedlist>
<para>In the event that a component fails and a back-up system must take on
its load, most high availability systems will replace the failed
component as quickly as possible to maintain necessary redundancy. This
way time spent in a degraded protection state is minimized.</para>
<para>Most high availability systems will fail in the event of multiple
independent (non-consequential) failures. In this case, most systems will
protect data over maintaining availability.</para>
<para>High-availability systems typically achieve an uptime percentage of
99.99% or more, which roughly equates to less than an hour of cumulative
downtime per year. In order to achieve this, high availability systems
should keep recovery times after a failure to about one to two minutes,
sometimes significantly less.</para>
<para>OpenStack currently meets such availability requirements for its own
infrastructure services, meaning that an uptime of 99.99% is feasible for
the OpenStack infrastructure proper. However, OpenStack does not guarantee
99.99% availability for individual guest instances.</para>
<para>Preventing single points of failure can depend on whether or not a
service is stateless.</para>
<section xml:id="stateless-vs-stateful">
<title>Stateless vs. Stateful services</title>
<para>A stateless service is one that provides a response after your
request, and then requires no further attention. To make a stateless
service highly available, you need to provide redundant instances and
load balance them. OpenStack services that are stateless include
<systemitem class="service">nova-api</systemitem>,
<systemitem class="service">nova-conductor</systemitem>,
<systemitem class="service">glance-api</systemitem>,
<systemitem class="service">keystone-api</systemitem>,
<systemitem class="service">neutron-api</systemitem> and
<systemitem class="service">nova-scheduler</systemitem>.</para>
<para>A stateful service is one where subsequent requests to the service
depend on the results of the first request. Stateful services are more
difficult to manage because a single action typically involves more than
one request, so simply providing additional instances and load balancing
will not solve the problem. For example, if the Horizon user interface
reset itself every time you went to a new page, it wouldn't be very
useful. OpenStack services that are stateful include the OpenStack
database and message queue.</para>
<para>Making stateful services highly available can depend on whether you
choose an active/passive or active/active configuration.</para>
</section>
<section xml:id="ap-intro">
<title>Active/Passive</title>
<para>In an active/passive configuration, systems are set up to bring
additional resources online to replace those that have failed. For
example, OpenStack would write to the main database while maintaining a
disaster recovery database that can be brought online in the event that
the main database fails.</para>
<para>Typically, an active/passive installation for a stateless service
would maintain a redundant instance that can be brought online when
required. Requests may be handled using a virtual IP address to
facilitate return to service with minimal reconfiguration
required.</para>
<para>A typical active/passive installation for a stateful service
maintains a replacement resource that can be brought online when
required. A separate application (such as Pacemaker or Corosync) monitors
these services, bringing the backup online as necessary.</para>
</section>
<section xml:id="aa-intro">
<title>Active/Active</title>
<para>In an active/active configuration, systems also use a backup but will
manage both the main and redundant systems concurrently. This way, if
there is a failure the user is unlikely to notice. The backup system is
already online, and takes on increased load while the main system is
fixed and brought back online.</para>
<para>Typically, an active/active installation for a stateless service
would maintain a redundant instance, and requests are load balanced using
a virtual IP address and a load balancer such as HAProxy.</para>
<para>A typical active/active installation for a stateful service would
include redundant services with all instances having an identical state.
For example, updates to one instance of a database would also update all
other instances. This way a request to one instance is the same as a
request to any other. A load balancer manages the traffic to these
systems, ensuring that operational systems always handle the
request.</para>
<para>These are some of the more common ways to implement these high
availability architectures, but they are by no means the only ways to do
it. The important thing is to make sure that your services are redundant,
and available; how you achieve that is up to you. This document will
cover some of the more common options for highly available
systems.</para>
</section>
</chapter>

23
doc/high-availability-guide/ch_network.xml
View File

@ -1,23 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ch-network">
<title>Network controller cluster stack</title>
<para>The network controller sits on the management and data network, and needs to be connected to the Internet if an instance will need access to the Internet.</para>
<note>
<para>Pacemaker requires that both nodes have different hostnames. Because of that, RA scripts could require some adjustments since the Networking
scheduler will be aware of one node, for example a virtual router attached to a single L3 node. For example, both nodes could set different hostnames
in the configuration files, and when the l3-agent started by Pacemaker, the node's hostname will be changed to network-controller automatically.
Whichever node starts the l3-agent will have the same hostname.</para>
</note>
<xi:include href="network/section_highly_available_neutron_l3_agent.xml"/>
<xi:include href="network/section_highly_available_neutron_dhcp_agent.xml"/>
<xi:include href="network/section_highly_available_neutron_metadata_agent.xml"/>
<xi:include href="network/section_manage_network_resources.xml"/>
</chapter>

31
doc/high-availability-guide/ch_pacemaker.xml
View File

@ -1,31 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ch-pacemaker">
<title>The Pacemaker cluster stack</title>
<para>OpenStack infrastructure high availability relies on the
<link xlink:href="http://www.clusterlabs.org">Pacemaker</link> cluster
stack, the state-of-the-art high availability and load balancing stack
for the Linux platform. Pacemaker is storage and application-agnostic,
and is in no way specific to OpenStack.</para>
<para>Pacemaker relies on the
<link xlink:href="http://www.corosync.org">Corosync</link> messaging
layer for reliable cluster communications. Corosync implements the
Totem single-ring ordering and membership protocol. It also provides
UDP and InfiniBand based messaging, quorum, and cluster membership to
Pacemaker.</para>
<para>Pacemaker interacts with applications through resource agents
(RAs), of which it supports over 70 natively. Pacemaker can also
easily use third-party RAs. An OpenStack high-availability
configuration uses existing native Pacemaker RAs (such as those
managing MySQL databases or virtual IP addresses), existing third-party
RAs (such as for RabbitMQ), and native OpenStack RAs (such as those
managing the OpenStack Identity and Image services).</para>
<xi:include href="pacemaker/section_install_packages.xml"/>
<xi:include href="pacemaker/section_set_up_corosync.xml"/>
<xi:include href="pacemaker/section_starting_corosync.xml"/>
<xi:include href="pacemaker/section_start_pacemaker.xml"/>
<xi:include href="pacemaker/section_set_basic_cluster_properties.xml"/>
</chapter>

229
doc/high-availability-guide/controller/section_mysql.xml
View File

@ -1,229 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE section [
<!ENTITY % openstack SYSTEM "../openstack.ent">
%openstack;
]>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="s-mysql">
<title>Highly available MySQL</title>
<para>MySQL is the default database server used by many OpenStack
services. Making the MySQL service highly available involves:</para>
<itemizedlist>
<listitem>
<para>Configuring a DRBD device for use by MySQL</para>
</listitem>
<listitem>
<para>Configuring MySQL to use a data directory residing on that DRBD
device</para>
</listitem>
<listitem>
<para>Selecting and assigning a virtual IP address (VIP) that can freely
float between cluster nodes</para>
</listitem>
<listitem>
<para>Configuring MySQL to listen on that IP address</para>
</listitem>
<listitem>
<para>Managing all resources, including the MySQL daemon itself, with
the Pacemaker cluster manager
</para>
</listitem>
</itemizedlist>
<note>
<para><link xlink:href="http://galeracluster.com/">MySQL/Galera</link> is
an alternative method of configuring MySQL for high availability. It is
likely to become the preferred method of achieving MySQL high
availability once it has sufficiently matured. At the time of writing,
however, the Pacemaker/DRBD based approach remains the recommended one
for OpenStack environments.</para>
</note>
<section xml:id="_configure_drbd">
<title>Configure DRBD</title>
<para>The Pacemaker based MySQL server requires a DRBD resource from
which it mounts the <literal>/var/lib/mysql</literal> directory. In this
example, the DRBD resource is simply named <literal>mysql</literal>:</para>
<formalpara>
<title><literal>mysql</literal> DRBD resource configuration
(<filename>/etc/drbd.d/mysql.res</filename>)</title>
<para>
<programlisting>resource mysql {
device minor 0;
disk "/dev/data/mysql";
meta-disk internal;
on node1 {
address ipv4 10.0.42.100:7700;
}
on node2 {
address ipv4 10.0.42.254:7700;
}
}</programlisting>
</para>
</formalpara>
<para>This resource uses an underlying local disk (in DRBD terminology, a
backing device) named <filename>/dev/data/mysql</filename>
on both cluster nodes, <literal>node1</literal> and <literal>node2</literal>.
Normally, this would be an LVM Logical Volume specifically set aside for
this purpose. The DRBD meta-disk is internal, meaning DRBD-specific metadata
is being stored at the end of the disk device itself. The
device is configured to communicate between IPv4 addresses
<literal>10.0.42.100</literal> and <literal>10.0.42.254</literal>, using
TCP port 7700. Once enabled, it will map to a local DRBD block device
with the device minor number 0, that is, <filename>/dev/drbd0</filename>.</para>
<para>Enabling a DRBD resource is explained in detail in
<link xlink:href="http://www.drbd.org/users-guide-8.3/s-first-time-up.html">
the DRBD User's Guide</link>. In brief, the proper sequence of commands
is this:</para>
<screen><prompt>#</prompt> <userinput>drbdadm create-md mysql</userinput><co xml:id="CO3-1"/>
<prompt>#</prompt> <userinput>drbdadm up mysql</userinput><co xml:id="CO3-2"/>
<prompt>#</prompt> <userinput>drbdadm -- --force primary mysql</userinput><co xml:id="CO3-3"/></screen>
<calloutlist>
<callout arearefs="CO3-1">
<para>Initializes DRBD metadata and writes the initial set of metadata
to <literal>/dev/data/mysql</literal>. Must be completed on both
nodes.</para>
</callout>
<callout arearefs="CO3-2">
<para>Creates the <literal>/dev/drbd0</literal> device node,
attaches the DRBD device to its backing store, and
connects the DRBD node to its peer. Must be
completed on both nodes.</para>
</callout>
<callout arearefs="CO3-3">
<para>Kicks off the initial device synchronization, and puts the device
into the <literal>primary</literal> (readable and writable) role. See
<link xlink:href="http://www.drbd.org/users-guide-8.3/ch-admin.html#s-roles">
Resource roles</link> (from the DRBD User's Guide) for a more
detailed description of the primary and secondary roles in DRBD. Must
be completed on one node only, namely the one
where you are about to continue with creating your filesystem.</para>
</callout>
</calloutlist>
</section>
<section xml:id="_creating_a_file_system">
<title>Creating a file system</title>
<para>Once the DRBD resource is running and in the primary role (and
potentially still in the process of running the initial device
synchronization), you may proceed with creating the filesystem for
MySQL data. XFS is generally the recommended filesystem due to its
journaling, efficient allocation, and performance:</para>
<screen><prompt>#</prompt> <userinput>mkfs -t xfs /dev/drbd0</userinput></screen>
<para>You may also use the alternate device path for the DRBD device, which
may be easier to remember as it includes the self-explanatory resource
name:</para>
<screen><prompt>#</prompt> <userinput>mkfs -t xfs /dev/drbd/by-res/mysql</userinput></screen>
<para>Once completed, you may safely return the device to the secondary
role. Any ongoing device synchronization will continue in the
background:</para>
<screen><prompt>#</prompt> <userinput>drbdadm secondary mysql</userinput></screen>
</section>
<section xml:id="_prepare_mysql_for_pacemaker_high_availability">
<title>Prepare MySQL for Pacemaker high availability</title>
<para>In order for Pacemaker monitoring to function properly, you must
ensure that MySQL's database files reside on the DRBD device. If you
already have an existing MySQL database, the simplest approach is to
just move the contents of the existing <filename>/var/lib/mysql</filename>
directory into the newly created filesystem on the DRBD device.</para>
<warning>
<para>You must complete the next step while the MySQL database server is
shut down.</para>
</warning>
<screen><prompt>#</prompt> <userinput>mount /dev/drbd/by-res/mysql /mnt</userinput>
<prompt>#</prompt> <userinput>mv /var/lib/mysql/* /mnt</userinput>
<prompt>#</prompt> <userinput>umount /mnt</userinput></screen>
<para>For a new MySQL installation with no existing data, you may also run
the <command>mysql_install_db</command> command:</para>
<screen><prompt>#</prompt> <userinput>mount /dev/drbd/by-res/mysql /mnt</userinput>
<prompt>#</prompt> <userinput>mysql_install_db --datadir=/mnt</userinput>
<prompt>#</prompt> <userinput>umount /mnt</userinput></screen>
<para>Regardless of the approach, the steps outlined here must be completed
on only one cluster node.</para>
</section>
<section xml:id="_add_mysql_resources_to_pacemaker">
<title>Add MySQL resources to Pacemaker</title>
<para>You can now add the Pacemaker configuration for
MySQL resources. Connect to the Pacemaker cluster with <command>crm
configure</command>, and add the following cluster resources:</para>
<programlisting>primitive p_ip_mysql ocf:heartbeat:IPaddr2 \
params ip="192.168.42.101" cidr_netmask="24" \
op monitor interval="30s"
primitive p_drbd_mysql ocf:linbit:drbd \
params drbd_resource="mysql" \
op start timeout="90s" \
op stop timeout="180s" \
op promote timeout="180s" \
op demote timeout="180s" \
op monitor interval="30s" role="Slave" \
op monitor interval="29s" role="Master"
primitive p_fs_mysql ocf:heartbeat:Filesystem \
params device="/dev/drbd/by-res/mysql" \
directory="/var/lib/mysql" \
fstype="xfs" \
options="relatime" \
op start timeout="60s" \
op stop timeout="180s" \
op monitor interval="60s" timeout="60s"
primitive p_mysql ocf:heartbeat:mysql \
params additional_parameters="--bind-address=192.168.42.101" \
config="/etc/mysql/my.cnf" \
pid="/var/run/mysqld/mysqld.pid" \
socket="/var/run/mysqld/mysqld.sock" \
log="/var/log/mysql/mysqld.log" \
op monitor interval="20s" timeout="10s" \
op start timeout="120s" \
op stop timeout="120s"
group g_mysql p_ip_mysql p_fs_mysql p_mysql
ms ms_drbd_mysql p_drbd_mysql \
meta notify="true" clone-max="2"
colocation c_mysql_on_drbd inf: g_mysql ms_drbd_mysql:Master
order o_drbd_before_mysql inf: ms_drbd_mysql:promote g_mysql:start</programlisting>
<para>This configuration creates</para>
<itemizedlist>
<listitem>
<para><literal>p_ip_mysql</literal>, a virtual IP address for use by MySQL
(<literal>192.168.42.101</literal>),</para>
</listitem>
<listitem>
<para><literal>p_fs_mysql</literal>, a Pacemaker managed filesystem
mounted to <filename>/var/lib/mysql</filename> on whatever node
currently runs the MySQL service,</para>
</listitem>
<listitem>
<para><literal>ms_drbd_mysql</literal>, the master/slave set
managing the <literal>mysql</literal> DRBD resource,</para>
</listitem>
<listitem>
<para>a service <literal>group</literal> and <literal>order</literal>
and <literal>colocation</literal> constraints to ensure resources are
started on the correct nodes, and in the correct sequence.</para>
</listitem>
</itemizedlist>
<para><command>crm configure</command> supports batch input, so you may
copy and paste the above into your live pacemaker configuration, and then
make changes as required. For example, you may enter <literal>edit p_ip_mysql</literal>
from the <command>crm configure</command> menu and edit the resource to
match your preferred virtual IP address.</para>
<para>Once completed, commit your configuration changes by entering
<literal>commit</literal> from the <command>crm configure</command> menu.
Pacemaker will then start the MySQL service, and its dependent resources,
on one of your nodes.</para>
</section>
<section xml:id="_configure_openstack_services_for_highly_available_mysql">
<title>Configure OpenStack services for highly available MySQL</title>
<para>Your OpenStack services must now point their MySQL configuration to
the highly available, virtual cluster IP address&mdash;rather than a
MySQL server's physical IP address as you normally would.</para>
<para>For OpenStack Image, for example, if your MySQL service IP address is
<literal>192.168.42.101</literal> as in the configuration explained here,
you would use the following line in your OpenStack Image registry
configuration file (<filename>glance-registry.conf</filename>):</para>
<programlisting language="ini">sql_connection = mysql://glancedbadmin:&lt;password&gt;@192.168.42.101/glance</programlisting>
<para>No other changes are necessary to your OpenStack configuration. If the
node currently hosting your database experiences a problem
necessitating service failover, your OpenStack services may experience
a brief MySQL interruption, as they would in the event of a network
hiccup, and then continue to run normally.</para>
</section>
</section>

215
doc/high-availability-guide/controller/section_rabbitmq.xml
View File

@ -1,215 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE section [
<!ENTITY % openstack SYSTEM "../openstack.ent">
%openstack;
]>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="s-rabbitmq">
<title>Highly available RabbitMQ</title>
<para>RabbitMQ is the default AMQP server used by many OpenStack
services. Making the RabbitMQ service highly available involves:</para>
<itemizedlist>
<listitem>
<para>configuring a DRBD device for use by RabbitMQ,</para>
</listitem>
<listitem>
<para>configuring RabbitMQ to use a data directory residing on
that DRBD device,</para>
</listitem>
<listitem>
<para>selecting and assigning a virtual IP address (VIP) that can freely
float between cluster nodes,</para>
</listitem>
<listitem>
<para>configuring RabbitMQ to listen on that IP address,</para>
</listitem>
<listitem>
<para>managing all resources, including the RabbitMQ daemon itself, with
the Pacemaker cluster manager.</para>
</listitem>
</itemizedlist>
<note>
<para><link xlink:href="http://www.rabbitmq.com/ha.html">Active-active mirrored queues</link>
is another method for configuring RabbitMQ versions 3.3.0 and later
for high availability. You can also manage a RabbitMQ cluster with
active-active mirrored queues using the Pacemaker cluster manager.</para>
</note>
<section xml:id="_configure_drbd_2">
<title>Configure DRBD</title>
<para>The Pacemaker based RabbitMQ server requires a DRBD resource from
which it mounts the <filename>/var/lib/rabbitmq</filename> directory.
In this example, the DRBD resource is simply named
<literal>rabbitmq</literal>:</para>
<formalpara>
<title><literal>rabbitmq</literal> DRBD resource configuration
(<filename>/etc/drbd.d/rabbitmq.res</filename>)</title>
<para>
<programlisting>resource rabbitmq {
device minor 1;
disk "/dev/data/rabbitmq";
meta-disk internal;
on node1 {
address ipv4 10.0.42.100:7701;
}
on node2 {
address ipv4 10.0.42.254:7701;
}
}</programlisting>
</para>
</formalpara>
<para>This resource uses an underlying local disk (in DRBD terminology, a
backing device) named <filename>/dev/data/rabbitmq</filename> on both
cluster nodes, <literal>node1</literal> and <literal>node2</literal>.
Normally, this would be an LVM Logical Volume specifically set aside for
this purpose. The DRBD meta-disk is internal, meaning DRBD-specific
metadata is being stored at the end of the disk device itself. The device
is configured to communicate between IPv4 addresses
<literal>10.0.42.100</literal> and <literal>10.0.42.254</literal>,
using TCP port 7701. Once enabled, it will map to a local DRBD block
device with the device minor number 1, that is,
<filename>/dev/drbd1</filename>.</para>
<para>Enabling a DRBD resource is explained in detail in
<link xlink:href="http://www.drbd.org/users-guide-8.3/s-first-time-up.html">the DRBD
User's Guide</link>. In brief, the proper sequence of commands is this:</para>
<screen><prompt>#</prompt> <userinput>drbdadm create-md rabbitmq</userinput><co xml:id="CO4-1"/>
<prompt>#</prompt> <userinput>drbdadm up rabbitmq</userinput><co xml:id="CO4-2"/>
<prompt>#</prompt> <userinput>drbdadm -- --force primary rabbitmq</userinput><co xml:id="CO4-3"/></screen>
<calloutlist>
<callout arearefs="CO4-1">
<para>Initializes DRBD metadata and writes the initial set of
metadata to <filename>/dev/data/rabbitmq</filename>. Must be
completed on both nodes.</para>
</callout>
<callout arearefs="CO4-2">
<para>Creates the <filename>/dev/drbd1</filename> device node,
attaches the DRBD device to its backing store, and connects
the DRBD node to its peer. Must be completed on both nodes.</para>
</callout>
<callout arearefs="CO4-3">
<para>Kicks off the initial device synchronization, and puts the
device into the <literal>primary</literal> (readable and writable)
role. See <link xlink:href="http://www.drbd.org/users-guide-8.3/ch-admin.html#s-roles">
Resource roles</link> (from the DRBD User's Guide) for a more
detailed description of the primary and secondary roles in DRBD.
Must be completed on one node only, namely the one where you
are about to continue with creating your filesystem.</para>
</callout>
</calloutlist>
</section>
<section xml:id="_create_a_file_system">
<title>Create a file system</title>
<para>Once the DRBD resource is running and in the primary role (and
potentially still in the process of running the initial device
synchronization), you may proceed with creating the filesystem for
RabbitMQ data. XFS is generally the recommended filesystem:</para>
<screen><prompt>#</prompt> <userinput>mkfs -t xfs /dev/drbd1</userinput></screen>
<para>You may also use the alternate device path for the DRBD device,
which may be easier to remember as it includes the self-explanatory
resource name:</para>
<screen><prompt>#</prompt> <userinput>mkfs -t xfs /dev/drbd/by-res/rabbitmq</userinput></screen>
<para>Once completed, you may safely return the device to the secondary
role. Any ongoing device synchronization will continue in the
background:</para>
<screen><prompt>#</prompt> <userinput>drbdadm secondary rabbitmq</userinput></screen>
</section>
<section xml:id="_prepare_rabbitmq_for_pacemaker_high_availability">
<title>Prepare RabbitMQ for Pacemaker high availability</title>
<para>In order for Pacemaker monitoring to function properly, you must
ensure that RabbitMQ's <filename>.erlang.cookie</filename> files
are identical on all nodes, regardless of whether DRBD is mounted
there or not. The simplest way of doing so is to take an existing
<filename>.erlang.cookie</filename> from one of your nodes, copying
it to the RabbitMQ data directory on the other node, and also
copying it to the DRBD-backed filesystem.</para>
<screen><prompt>#</prompt> <userinput>scp -p /var/lib/rabbitmq/.erlang.cookie node2:/var/lib/rabbitmq/</userinput>
<prompt>#</prompt> <userinput>mount /dev/drbd/by-res/rabbitmq /mnt</userinput>
<prompt>#</prompt> <userinput>cp -a /var/lib/rabbitmq/.erlang.cookie /mnt</userinput>
<prompt>#</prompt> <userinput>umount /mnt</userinput></screen>
</section>
<section xml:id="_add_rabbitmq_resources_to_pacemaker">
<title>Add RabbitMQ resources to Pacemaker</title>
<para>You may now proceed with adding the Pacemaker configuration for
RabbitMQ resources. Connect to the Pacemaker cluster with
<command>crm configure</command>, and add the following cluster
resources:</para>
<programlisting>primitive p_ip_rabbitmq ocf:heartbeat:IPaddr2 \
params ip="192.168.42.100" cidr_netmask="24" \
op monitor interval="10s"
primitive p_drbd_rabbitmq ocf:linbit:drbd \
params drbd_resource="rabbitmq" \
op start timeout="90s" \
op stop timeout="180s" \
op promote timeout="180s" \
op demote timeout="180s" \
op monitor interval="30s" role="Slave" \
op monitor interval="29s" role="Master"
primitive p_fs_rabbitmq ocf:heartbeat:Filesystem \
params device="/dev/drbd/by-res/rabbitmq" \
directory="/var/lib/rabbitmq" \
fstype="xfs" options="relatime" \
op start timeout="60s" \
op stop timeout="180s" \
op monitor interval="60s" timeout="60s"
primitive p_rabbitmq ocf:rabbitmq:rabbitmq-server \
params nodename="rabbit@localhost" \
mnesia_base="/var/lib/rabbitmq" \
op monitor interval="20s" timeout="10s"
group g_rabbitmq p_ip_rabbitmq p_fs_rabbitmq p_rabbitmq
ms ms_drbd_rabbitmq p_drbd_rabbitmq \
meta notify="true" master-max="1" clone-max="2"
colocation c_rabbitmq_on_drbd inf: g_rabbitmq ms_drbd_rabbitmq:Master
order o_drbd_before_rabbitmq inf: ms_drbd_rabbitmq:promote g_rabbitmq:start</programlisting>
<para>This configuration creates</para>
<itemizedlist>
<listitem>
<para><literal>p_ip_rabbitmq</literal>, a virtual IP address for
use by RabbitMQ (<literal>192.168.42.100</literal>),</para>
</listitem>
<listitem>
<para><literal>p_fs_rabbitmq</literal>, a Pacemaker managed
filesystem mounted to <filename>/var/lib/rabbitmq</filename>
on whatever node currently runs the RabbitMQ service,</para>
</listitem>
<listitem>
<para><literal>ms_drbd_rabbitmq</literal>, the master/slave set
managing the <literal>rabbitmq</literal> DRBD resource,</para>
</listitem>
<listitem>
<para>a service group and order and colocation constraints to
ensure resources are started on the correct nodes, and in the
correct sequence.</para>
</listitem>
</itemizedlist>
<para><command>crm configure</command> supports batch input, so you
may copy and paste the above into your live pacemaker configuration,
and then make changes as required. For example, you may enter
<literal>edit p_ip_rabbitmq</literal> from the
<command>crm configure</command> menu and edit the resource to
match your preferred virtual IP address.</para>
<para>Once completed, commit your configuration changes by entering
<literal>commit</literal> from the <command>crm configure</command>
menu. Pacemaker will then start the RabbitMQ service, and its
dependent resources, on one of your nodes.</para>
</section>
<section xml:id="_configure_openstack_services_for_highly_available_rabbitmq">
<title>Configure OpenStack services for highly available RabbitMQ</title>
<para>Your OpenStack services must now point their RabbitMQ
configuration to the highly available, virtual cluster IP
address&mdash;rather than a RabbitMQ server's physical IP address
as you normally would.</para>
<para>For OpenStack Image, for example, if your RabbitMQ service
IP address is <literal>192.168.42.100</literal> as in the
configuration explained here, you would use the following line
in your OpenStack Image API configuration file
(<filename>glance-api.conf</filename>):</para>
<programlisting language="ini">rabbit_host = 192.168.42.100</programlisting>
<para>No other changes are necessary to your OpenStack configuration.
If the node currently hosting your RabbitMQ experiences a problem
necessitating service failover, your OpenStack services may
experience a brief RabbitMQ interruption, as they would in the
event of a network hiccup, and then continue to run normally.</para>
</section>
</section>

60
doc/high-availability-guide/figures/Check_mark_23x20_02.svg
View File

@ -1,60 +0,0 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://web.resource.org/cc/"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
width="19.21315"
height="18.294994"
id="svg2"
sodipodi:version="0.32"
inkscape:version="0.45"
sodipodi:modified="true"
version="1.0">
<defs
id="defs4" />
<sodipodi:namedview
id="base"
pagecolor="#ffffff"
bordercolor="#666666"
borderopacity="1.0"
gridtolerance="10000"
guidetolerance="10"
objecttolerance="10"
inkscape:pageopacity="0.0"
inkscape:pageshadow="2"
inkscape:zoom="7.9195959"
inkscape:cx="17.757032"
inkscape:cy="7.298821"
inkscape:document-units="px"
inkscape:current-layer="layer1"
inkscape:window-width="984"
inkscape:window-height="852"
inkscape:window-x="148"
inkscape:window-y="66" />
<metadata
id="metadata7">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
</cc:Work>
</rdf:RDF>
</metadata>
<g
inkscape:label="Layer 1"
inkscape:groupmode="layer"
id="layer1"
transform="translate(-192.905,-516.02064)">
<path
style="fill:#000000"
d="M 197.67968,534.31563 C 197.40468,534.31208 196.21788,532.53719 195.04234,530.37143 L 192.905,526.43368 L 193.45901,525.87968 C 193.76371,525.57497 194.58269,525.32567 195.27896,525.32567 L 196.5449,525.32567 L 197.18129,527.33076 L 197.81768,529.33584 L 202.88215,523.79451 C 205.66761,520.74678 208.88522,517.75085 210.03239,517.13691 L 212.11815,516.02064 L 207.90871,520.80282 C 205.59351,523.43302 202.45735,527.55085 200.93947,529.95355 C 199.42159,532.35625 197.95468,534.31919 197.67968,534.31563 z "
id="path2223" />
</g>
</svg>
Side by Side

Before

Width:  |  Height:  |  Size: 2.1 KiB

1
doc/high-availability-guide/figures/README
View File

@ -1 +0,0 @@
Place any figures and illustrations in this directory.

28
doc/high-availability-guide/ha_aa_controllers/section_memcached.xml
View File

@ -1,28 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_memcached">
<title>Memcached</title>
<para>Most OpenStack services can use <application>Memcached</application> to
store ephemeral data, such as tokens. Although <application>Memcached</application>
does not support typical forms of redundancy, such as clustering, OpenStack
services can use almost any number of instances by configuring multiple
hostnames or IP addresses. The <application>Memcached</application>
client implements hashing to balance objects among the instances. Failure of
an instance only impacts a percentage of the objects and the client automatically
removes it from the list of instances.</para>
<para>To install and configure it, read the
<link xlink:href="http://code.google.com/p/memcached/wiki/NewStart">
official documentation</link>.</para>
<para>Memory caching is managed by oslo-incubator, so the way to use
multiple memcached servers is the same for all projects.</para>
<para>Example configuration with two hosts:</para>
<programlisting>memcached_servers = controller1:11211,controller2:11211</programlisting>
<para>By default, <literal>controller1</literal> handles the caching service
but if the host goes down, <literal>controller2</literal> does the job. For
more information about <application>Memcached</application> installation,
see the <link xlink:href="http://docs.openstack.org/admin-guide-cloud/">
OpenStack Cloud Administrator Guide</link>.</para>
</section>

74
doc/high-availability-guide/ha_aa_controllers/section_run_openstack_api_and_schedulers.xml
View File

@ -1,74 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_run_openstack_api_and_schedulers">
<title>Run OpenStack API and schedulers</title>
<section xml:id="_api_services">
<title>API services</title>
<para>All OpenStack projects have an API service for controlling all the
resources in the Cloud. In active/active mode, the most common setup is
to scale out these services on at least two nodes and to use load balancing
and a virtual IP address (with <application>HAProxy</application> and
<application>Keepalived</application> in this setup).</para>
<para>To use highly available and scalable API services, we need to
ensure that:</para>
<itemizedlist>
<listitem>
<para>You use virtual IP addresses when configuring OpenStack Identity
endpoints.
</para>
</listitem>
<listitem>
<para>All OpenStack configuration files should refer to virtual
IP addresses.</para>
</listitem>
</itemizedlist>
<note>
<para>The monitor check is quite simple since it just establishes a TCP
connection to the API port. Comparing to the active/passive mode using
<application>Corosync</application> and resource agents,
we do not check if the service is actually running. That is why all
OpenStack API services should be monitored by another tool, for example
<application>Nagios</application>.</para>
</note>
</section>
<section xml:id="_schedulers">
<title>Schedulers</title>
<para>OpenStack schedulers are used to determine how to dispatch compute,
network, and volume requests. The most common setup is to use RabbitMQ as a
messaging system. Those services are connected to the messaging back end
and can scale out:</para>
<itemizedlist>
<listitem>
<para><systemitem class="service">nova-scheduler</systemitem></para>
</listitem>
<listitem>
<para><systemitem class="service">nova-conductor</systemitem></para>
</listitem>
<listitem>
<para><systemitem class="service">cinder-scheduler</systemitem></para>
</listitem>
<listitem>
<para><systemitem class="service">neutron-server</systemitem></para>
</listitem>
<listitem>
<para><systemitem class="service">ceilometer-collector</systemitem></para>
</listitem>
<listitem>
<para><systemitem class="service">heat-engine</systemitem></para>
</listitem>
</itemizedlist>
<para>Please refer to the <link linkend="s-rabbitmq">RabbitMQ section</link>
for configuring these services with multiple messaging servers.</para>
</section>
<section xml:id="_ceilometer_central">
<title>Telemetry Central agent</title>
<para>The Telemetry Central agent can be configured to partition its
polling workload between multiple agents, enabling high availability.
Please refer to <link xlink:href="http://docs.openstack.org/admin-guide-cloud/telemetry-data-collection.html#support-for-ha-deployment-of-the-central-and-compute-agent-services">
this section</link> of the <citetitle>OpenStack Cloud Administrator Guide</citetitle>
for the requirements and implementation details of this configuration.</para>
</section>
</section>

152
doc/high-availability-guide/ha_aa_db/section_mariadb_galera_rh.xml
View File

@ -1,152 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ha-aa-db-mariadb-galera-rh">
<title>MariaDB with Galera (Red Hat-based platforms)</title>
<para>MariaDB with Galera provides synchronous database replication in an
active-active, multi-master environment. High availability for the data
itself is managed internally by Galera, while access availability will be
managed by HAProxy.</para>
<para>This guide assumes that three nodes are used to form the MariaDB
Galera cluster. Unless otherwise specified, all commands need to be
executed on all cluster nodes.</para>
<procedure>
<title>To install MariaDB with Galera</title>
<step>
<para>
Red Hat-based distributions include Galera packages in their repositories. To
install the most current version of the packages, run the following command:
</para>
<screen><prompt>#</prompt> <userinput>yum install -y mariadb-galera-server xinetd rsync</userinput></screen>
</step>
<step>
<para>(Optional) Configure the <application>clustercheck</application> utility.</para>
<para>If HAProxy is used to load-balance client access to MariaDB,
as described in <link linkend="ha-aa-haproxy">the HAProxy section</link>
of this document, you can use the <application>clustercheck</application> utility
to improve health checks.
</para>
<substeps>
<step>
<para>Create file <filename>etc/sysconfig/clustercheck</filename>
with the following contents:
</para>
<programlisting language="ini">MYSQL_USERNAME="clustercheck"
MYSQL_PASSWORD=<replaceable>PASSWORD</replaceable>
MYSQL_HOST="localhost"
MYSQL_PORT="3306"</programlisting>
<warning>
<para>Make sure a sensible password is used.</para>
</warning>
</step>
<step>
<para>Configure monitor service (used by HAProxy):</para>
<para>Create file <filename>/etc/xinetd.d/galera-monitor</filename>
with the following contents:</para>
<programlisting language="ini">service galera-monitor
{
port = 9200
disable = no
socket_type = stream
protocol = tcp
wait = no
user = root
group = root
groups = yes
server = /usr/bin/clustercheck
type = UNLISTED
per_source = UNLIMITED
log_on_success =
log_on_failure = HOST
flags = REUSE
}</programlisting>
</step>
<step>
<para>Create the database user required by <application>clustercheck</application>:</para>
<screen><prompt>#</prompt> <userinput>systemctl start mysqld</userinput>
<prompt>#</prompt> <userinput>mysql -e "CREATE USER 'clustercheck'@'localhost' IDENTIFIED BY '<replaceable>PASSWORD</replaceable>';"</userinput>
<prompt>#</prompt> <userinput>systemctl stop mysqld</userinput></screen>
</step>
<step>
<para>Start xinetd (required by <application>clustercheck</application>):</para>
<screen><prompt>#</prompt> <userinput>systemctl daemon-reload</userinput>
<prompt>#</prompt> <userinput>systemctl enable xinetd</userinput>
<prompt>#</prompt> <userinput>systemctl start xinetd</userinput></screen>
</step>
</substeps>
</step>
<step>
<para>Configure MariaDB with Galera.</para>
<substeps>
<step>
<para>
Create the Galera configuration file <filename>/etc/my.cnf.d/galera.cnf</filename>
with the following contents:
</para>
<programlisting language="ini">[mysqld]
skip-name-resolve=1
binlog_format=ROW
default-storage-engine=innodb
innodb_autoinc_lock_mode=2
innodb_locks_unsafe_for_binlog=1
max_connections=2048
query_cache_size=0
query_cache_type=0
bind_address=<replaceable>NODE_IP</replaceable>
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_name="galera_cluster"
wsrep_cluster_address="gcomm://<replaceable>PRIMARY_NODE_IP</replaceable>, <replaceable>SECONDARY_NODE_IP</replaceable>, <replaceable>TERTIARY_NODE_IP</replaceable>"
wsrep_slave_threads=1
wsrep_certify_nonPK=1
wsrep_max_ws_rows=131072
wsrep_max_ws_size=1073741824
wsrep_debug=0
wsrep_convert_LOCK_to_trx=0
wsrep_retry_autocommit=1
wsrep_auto_increment_control=1
wsrep_drupal_282555_workaround=0
wsrep_causal_reads=0
wsrep_notify_cmd=
wsrep_sst_method=rsync</programlisting>
</step>
<step>
<para>Open firewall ports used for MariaDB and Galera communications:</para>
<screen><prompt>#</prompt> <userinput>firewall-cmd --add-service=mysql</userinput>
<prompt>#</prompt> <userinput>firewall-cmd --add-port=4444/tcp</userinput>
<prompt>#</prompt> <userinput>firewall-cmd --add-port=4567/tcp</userinput>
<prompt>#</prompt> <userinput>firewall-cmd --add-port=4568/tcp</userinput>
<prompt>#</prompt> <userinput>firewall-cmd --add-port=9200/tcp</userinput>
<prompt>#</prompt> <userinput>firewall-cmd --add-port=9300/tcp</userinput>
<prompt>#</prompt> <userinput>firewall-cmd --add-service=mysql --permanent</userinput>
<prompt>#</prompt> <userinput>firewall-cmd --add-port=4444/tcp --permanent</userinput>
<prompt>#</prompt> <userinput>firewall-cmd --add-port=4567/tcp --permanent</userinput>
<prompt>#</prompt> <userinput>firewall-cmd --add-port=4568/tcp --permanent</userinput>
<prompt>#</prompt> <userinput>firewall-cmd --add-port=9200/tcp --permanent</userinput>
<prompt>#</prompt> <userinput>firewall-cmd --add-port=9300/tcp --permanent</userinput></screen>
</step>
<step>
<para>Start MariaDB cluster:</para>
<substeps>
<step>
<para>On node 1:</para>
<screen><prompt>#</prompt> <userinput>sudo -u mysql /usr/libexec/mysqld --wsrep-cluster-address='gcomm://' &amp;</userinput></screen>
</step>
<step>
<para>On nodes 2 and 3:</para>
<screen><prompt>#</prompt> <userinput>systemctl start mariadb</userinput></screen>
</step>
<step>
<para>Once the output from <application>clustercheck</application>
is <option>200</option> on all nodes, restart MariaDB on node 1:</para>
<screen><prompt>#</prompt> <userinput>kill &lt;<replaceable>mysql PIDs</replaceable>&gt;</userinput>
<prompt>#</prompt> <userinput>systemctl start mariadb</userinput></screen>
</step>
</substeps>
</step>
</substeps>
</step>
</procedure>
</section>

195
doc/high-availability-guide/ha_aa_db/section_mysql_galera.xml
View File

@ -1,195 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ha-aa-db-mysql-galera">
<title>MySQL with Galera</title>
<para>Rather than starting with a vanilla version of MySQL, and then adding
Galera, you will want to install a version of MySQL patched for wsrep
(Write Set REPlication) from <link xlink:href="https://launchpad.net/codership-mysql">https://launchpad.net/codership-mysql</link>.
The wsrep API is suitable for configuring MySQL High Availability in
OpenStack because it supports synchronous replication.</para>
<para>Note that the installation requirements call for careful attention. Read
the guide <link xlink:href="https://launchpadlibrarian.net/66669857/README-wsrep">https://launchpadlibrarian.net/66669857/README-wsrep</link>
to ensure you follow all the required steps.</para>
<para>And for any additional information about Galera, you can access this guide:
<link xlink:href="http://galeracluster.com/documentation-webpages/gettingstarted.html">http://galeracluster.com/documentation-webpages/gettingstarted.html</link></para>
<orderedlist numeration="arabic" inheritnum="ignore" continuation="restarts">
<title>Installing Galera through a MySQL version patched for wsrep:</title>
<listitem>
<para>
Setup the repository for Ubuntu 14.04 "trusty" (most recent).
Install the software properties, the key, and the repository:
</para>
<screen><prompt>#</prompt> <userinput>apt-get install software-properties-common</userinput>
<prompt>#</prompt> <userinput>apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xcbcb082a1bb943db</userinput>
<prompt>#</prompt> <userinput>add-apt-repository 'deb http://ams2.mirrors.digitalocean.com/mariadb/repo/5.5/ubuntu trusty main'</userinput></screen>
<note>
<para>
You can change the mirror to one near you on:
<link xlink:href="https://downloads.mariadb.org/mariadb/repositories/">downloads.mariadb.org</link>
</para>
</note>
</listitem>
<listitem>
<para>
Update your system and install the required packages:
<screen><prompt>#</prompt> <userinput>apt-get update</userinput>
<prompt>#</prompt> <userinput>apt-get install mariadb-galera-server galera</userinput></screen>
</para>
<warning>
<para>
If you have mariaDB already installed you need to re-apply all the permissions from the installation guide. It will purge all privileges!
</para>
</warning>
</listitem>
<listitem>
<para>Adjust the configuration:</para>
<para>In the <filename>/etc/mysql/my.conf</filename> file, make the following changes:</para>
<programlisting language="ini">query_cache_size=0
binlog_format=ROW
default_storage_engine=innodb
innodb_autoinc_lock_mode=2
innodb_doublewrite=1</programlisting>
</listitem>
<listitem>
<para>
Create the <filename>/etc/mysql/conf.d/wsrep.cnf</filename> file.
</para>
<para>Paste the following lines in this file:</para>
<programlisting language="ini">[mysqld]
wsrep_provider=/usr/lib/galera/libgalera_smm.so
wsrep_cluster_name="Openstack"
wsrep_sst_auth=wsrep_sst:wspass
wsrep_cluster_address="gcomm://<replaceable>PRIMARY_NODE_IP</replaceable>,<replaceable>SECONDARY_NODE_IP</replaceable>"
wsrep_sst_method=rsync
wsrep_node_address="<replaceable>PRIMARY_NODE_IP</replaceable>"
wsrep_node_name="<replaceable>NODE_NAME</replaceable>"
</programlisting>
<para>
Replace <replaceable>PRIMARY_NODE_IP</replaceable> and
<replaceable>SECONDARY_NODE_IP</replaceable> with the IP addresses
of your primary and secondary servers.
</para>
<para>
Replace <replaceable>PRIMARY_NODE_IP</replaceable> with the hostname of the server. This is set for logging.
</para>
<para>Copy this file to all other databases servers and change the value of <literal>wsrep_cluster_address</literal> and <literal>wsrep_node_name</literal> accordingly.</para>
</listitem>
<listitem>
<para>Start mysql as root and execute the following queries:</para>
<screen><prompt>mysql&gt;</prompt> <userinput>SET wsrep_on=OFF; GRANT ALL ON *.* TO wsrep_sst@'%' IDENTIFIED BY 'wspass';</userinput></screen>
<para>Remove user accounts with empty user names because they cause problems:</para>
<screen><prompt>mysql&gt;</prompt> <userinput>SET wsrep_on=OFF; DELETE FROM mysql.user WHERE user='';</userinput></screen>
</listitem>
<listitem>
<para>Check that the nodes can access each other through the firewall.
Depending on your environment, this might mean adjusting iptables, as
in:</para>
<screen><prompt>#</prompt> <userinput>iptables --insert RH-Firewall-1-INPUT 1 --proto tcp \
--source &lt;my IP&gt;/24 --destination &lt;my IP&gt;/32 --dport 3306 \
-j ACCEPT</userinput>
<prompt>#</prompt> <userinput>iptables --insert RH-Firewall-1-INPUT 1 --proto tcp \
--source &lt;my IP&gt;/24 --destination &lt;my IP&gt;/32 --dport 4567 \
-j ACCEPT</userinput></screen>
<para>This might also mean configuring any NAT firewall between nodes to
allow direct connections. You might need to disable SELinux, or
configure it to allow <systemitem class="service">mysqld</systemitem>
to listen to sockets at unprivileged ports.</para>
</listitem>
</orderedlist>
<para>
For the next step create a back-up file of the <filename>debian.cnf</filename> file in <filename>/etc/mysql</filename> on all database servers.
Should something go wrong just copy the back-up file back.
</para>
<screen><prompt>#</prompt> <userinput>cp debian.cnf debian.cnf.old</userinput></screen>
<para>
Make sure you have SSH root access on the other servers. From the primary database server, copy the <filename>debian.cnf</filename> file to all other servers by running the following command:
</para>
<screen><prompt>#</prompt> <userinput>scp /etc/mysql/debian.cnf root@IP-address:/etc/mysql</userinput></screen>
<para>After the copy make sure that all files are the same, you can do this by using the following command:</para>
<screen><prompt>#</prompt> <userinput>md5sum debian.cnf</userinput></screen>
<para>From the <filename>debian.cnf</filename> get the database password:</para>
<screen><prompt>#</prompt> <userinput>cat /etc/mysql/debian.cnf</userinput></screen>
<para>The result will look like this:</para>
<screen><computeroutput>[client]
host = localhost
user = debian-sys-maint
password = FiKiOY1Lw8Sq46If
socket = /var/run/mysqld/mysqld.sock
[mysql_upgrade]
host = localhost
user = debian-sys-maint
password = FiKiOY1Lw8Sq46If
socket = /var/run/mysqld/mysqld.sock
basedir = /usr
</computeroutput></screen>
<para>
The below query should be run on every server except the primary node. This will make sure that you can restart the database again.
Do not forget to add the password from the <filename>debian.cnf</filename>. To do this, run:
</para>
<screen><prompt>mysql&gt;</prompt> <userinput>GRANT SHUTDOWN ON *.* TO ‘debian-sys-maint’@’localhost' IDENTIFIED BY '&lt;debian.cnf password&gt;';</userinput>
<prompt>mysql&gt;</prompt> <userinput>GRANT SELECT ON `mysql`.`user` TO ‘debian-sys-maint’@’localhost' IDENTIFIED BY '&lt;debian.cnf password&gt;';</userinput></screen>
<para>Stop all the mysql servers and start the first server with the following command:</para>
<screen><prompt>#</prompt> <userinput>service mysql start --wsrep-new-cluster</userinput></screen>
<para>All other nodes can now be started using:</para>
<screen><prompt>#</prompt> <userinput>service mysql start</userinput></screen>
<para>Verify the wsrep replication by logging in as root under mysql and running the following command:</para>
<screen><prompt>mysql&gt;</prompt> <userinput>SHOW STATUS LIKE ‘wsrep%’;</userinput>
<computeroutput>+------------------------------+--------------------------------------+
| Variable_name | Value |
+------------------------------+--------------------------------------+
| wsrep_local_state_uuid | d6a51a3a-b378-11e4-924b-23b6ec126a13 |
| wsrep_protocol_version | 5 |
| wsrep_last_committed | 202 |
| wsrep_replicated | 201 |
| wsrep_replicated_bytes | 89579 |
| wsrep_repl_keys | 865 |
| wsrep_repl_keys_bytes | 11543 |
| wsrep_repl_data_bytes | 65172 |
| wsrep_repl_other_bytes | 0 |
| wsrep_received | 8 |
| wsrep_received_bytes | 853 |
| wsrep_local_commits | 201 |
| wsrep_local_cert_failures | 0 |
| wsrep_local_replays | 0 |
| wsrep_local_send_queue | 0 |
| wsrep_local_send_queue_avg | 0.000000 |
| wsrep_local_recv_queue | 0 |
| wsrep_local_recv_queue_avg | 0.000000 |
| wsrep_local_cached_downto | 1 |
| wsrep_flow_control_paused_ns | 0 |
| wsrep_flow_control_paused | 0.000000 |
| wsrep_flow_control_sent | 0 |
| wsrep_flow_control_recv | 0 |
| wsrep_cert_deps_distance | 1.029703 |
| wsrep_apply_oooe | 0.024752 |
| wsrep_apply_oool | 0.000000 |
| wsrep_apply_window | 1.024752 |
| wsrep_commit_oooe | 0.000000 |
| wsrep_commit_oool | 0.000000 |
| wsrep_commit_window | 1.000000 |
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced |
| wsrep_cert_index_size | 18 |
| wsrep_causal_reads | 0 |
| wsrep_cert_interval | 0.024752 |
| wsrep_incoming_addresses | &lt;first IP&gt;:3306,&lt;second IP&gt;:3306 |
| wsrep_cluster_conf_id | 2 |
| wsrep_cluster_size | 2 |
| wsrep_cluster_state_uuid | d6a51a3a-b378-11e4-924b-23b6ec126a13 |
| wsrep_cluster_status | Primary |
| wsrep_connected | ON |
| wsrep_local_bf_aborts | 0 |
| wsrep_local_index | 1 |
| wsrep_provider_name | Galera |
| wsrep_provider_vendor | Codership Oy &lt;info@codership.com&gt; |
| wsrep_provider_version | 25.3.5-wheezy(rXXXX) |
| wsrep_ready | ON |
| wsrep_thread_count | 2 |
+------------------------------+--------------------------------------+</computeroutput></screen>
</section>

16
doc/high-availability-guide/ha_aa_network/section_run_neutron_dhcp_agent.xml
View File

@ -1,16 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_run_neutron_dhcp_agent">
<title>Run neutron DHCP agent</title>
<para>The OpenStack Networking service has a scheduler that lets you
run multiple agents across nodes. Also, the DHCP agent can be
natively highly available.
You can configure the number of DHCP agents per network using the
parameter <option>dhcp_agents_per_network</option> in
<filename>neutron.conf</filename>.
By default this is equal to 1. To achieve high availability
assign more than one DHCP agent per network.</para>
</section>

82
doc/high-availability-guide/ha_aa_network/section_run_neutron_l3_agent.xml
View File

@ -1,82 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_run_neutron_l3_agent">
<title>Run neutron L3 agent</title>
<para>The neutron L3 agent is scalable, due to the scheduler
that allows distribution of virtual routers across multiple nodes.
The following options are available to make a router highly
available:</para>
<itemizedlist>
<listitem>
<para>Automatic L3 agent failover for routers via the
<option>allow_automatic_l3agent_failover = True</option> configuration
option in <filename>/etc/neutron/neutron.conf</filename>.</para>
</listitem>
<listitem>
<para>Use Layer 3 High Availability with VRRP. The following
configuration options need to be set in <filename>/etc/neutron/neutron.conf</filename>
to enable it:</para>
<informaltable rules="all" width="80%">
<col width="40%"/>
<col width="10%"/>
<col width="50%"/>
<thead>
<tr>
<th>Option</th>
<th>Value to set</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<para>l3_ha</para>
</td>
<td>
<para><option>True</option></para>
</td>
<td>
<para>All routers will be highly available by default.</para>
</td>
</tr>
<tr>
<td>
<para>max_l3_agents_per_router</para>
</td>
<td>
<para>2</para>
</td>
<td>
<para>Maximum number of network nodes to be used for the HA
router. The value can be larger than 2 but needs to be at least
2.</para>
</td>
</tr>
<tr>
<td>
<para>min_l3_agents_per_router</para>
</td>
<td>
<para>2</para>
</td>
<td>
<para>Minimum number of network nodes to be used for the HA
router. A new router creation will fail unless there are
at least <option>min_l3_agents_per_router</option> network
nodes available. The value should not be less than 2.</para>
</td>
</tr>
</tbody>
</informaltable>
</listitem>
<listitem>
<para>Using the active/passive solution to run the Neutron L3 agent in
failover mode with <application>Pacemaker</application>. See the
<link linkend="ha-using-active-passive">active/passive section</link>
of this guide.</para>
</listitem>
</itemizedlist>
</section>

13
doc/high-availability-guide/ha_aa_network/section_run_neutron_lbaas_agent.xml
View File

@ -1,13 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_run_neutron_lbaas_agent">
<title>Run neutron LBaaS agent</title>
<para>Currently, there's no native feature to make the LBaaS agent highly
available using the default plug-in HAProxy. A common way to make
HAProxy highly available is to use the VRRP (Virtual Router Redundancy
Protocol). Unfortunately, this is not yet implemented in the LBaaS
HAProxy plug-in.</para>
</section>

13
doc/high-availability-guide/ha_aa_network/section_run_neutron_metadata_agent.xml
View File

@ -1,13 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_run_neutron_metadata_agent">
<title>Run neutron metadata agent</title>
<para>There is no native feature to make this service highly available.
At this time, the Active / Passive solution exists to run the neutron
metadata agent in failover mode with <application>Pacemaker</application>.
See the <link linkend="ha-using-active-passive">active/passive
section</link> of this guide.</para>
</section>

64
doc/high-availability-guide/ha_aa_rabbitmq/section_configure_openstack_services_to_use_rabbitmq.xml
View File

@ -1,64 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_configure_openstack_services_to_use_rabbitmq">
<title>Configure OpenStack services to use RabbitMQ</title>
<para>We have to configure the OpenStack components to use at least two
RabbitMQ nodes.</para>
<para>Do this configuration on all services using RabbitMQ:</para>
<itemizedlist>
<listitem>
<para>RabbitMQ HA cluster host:port pairs:</para>
<programlisting language="ini">rabbit_hosts=rabbit1:5672,rabbit2:5672</programlisting>
</listitem>
<listitem>
<para>How frequently to retry connecting with RabbitMQ:</para>
<programlisting language="ini">rabbit_retry_interval=1</programlisting>
</listitem>
<listitem>
<para>How long to back-off for between retries when connecting to
RabbitMQ:</para>
<programlisting language="ini">rabbit_retry_backoff=2</programlisting>
</listitem>
<listitem>
<para>Maximum retries with trying to connect to RabbitMQ (infinite by
default):</para>
<programlisting language="ini">rabbit_max_retries=0</programlisting>
</listitem>
<listitem>
<para>Use durable queues in RabbitMQ:</para>
<programlisting language="ini">rabbit_durable_queues=true</programlisting>
</listitem>
<listitem>
<para>Use HA queues in RabbitMQ (x-ha-policy: all):</para>
<programlisting language="ini">rabbit_ha_queues=true</programlisting>
</listitem>
</itemizedlist>
<note>
<para>If you change the configuration from an old setup which did not use
HA queues, you should interrupt the service:</para>
<screen><prompt>#</prompt> <userinput>rabbitmqctl stop_app</userinput>
<prompt>#</prompt> <userinput>rabbitmqctl reset</userinput>
<prompt>#</prompt> <userinput>rabbitmqctl start_app</userinput></screen>
</note>
<note>
<para>Services currently working with HA queues:</para>
<itemizedlist>
<listitem>
<para>OpenStack Compute</para>
</listitem>
<listitem>
<para>OpenStack Block Storage</para>
</listitem>
<listitem>
<para>OpenStack Networking</para>
</listitem>
<listitem>
<para>Telemetry</para>
</listitem>
</itemizedlist>
</note>
</section>

61
doc/high-availability-guide/ha_aa_rabbitmq/section_configure_rabbitmq.xml
View File

@ -1,61 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_configure_rabbitmq">
<title>Configure RabbitMQ</title>
<para>We are building a cluster of RabbitMQ nodes to construct a
RabbitMQ broker, a logical grouping of several Erlang nodes.</para>
<para>We have to consider that while exchanges and bindings will survive
the loss of individual nodes, queues and their messages will not because a
queue and its contents is located on one node. If we lose this node, we
also lose the queue.</para>
<para>Mirrored queues in RabbitMQ improve the availability
of service since it will be resilient to failures.</para>
<para>We consider that we run (at least) two RabbitMQ servers and we call the
nodes <literal>rabbit1</literal> and <literal>rabbit2</literal>. To build a
broker, we need to ensure that all nodes have the same Erlang cookie
file.</para>
<para>To do so, stop RabbitMQ everywhere and copy the cookie from the first
node to the other node(s):</para>
<screen><prompt>#</prompt> <userinput>scp /var/lib/rabbitmq/.erlang.cookie \
root@<replaceable>NODE</replaceable>:/var/lib/rabbitmq/.erlang.cookie</userinput></screen>
<para>On the target nodes ensure the correct owner, group, and permissions of the
<filename>.erlang.cookie</filename> file:</para>
<screen><prompt>#</prompt> <userinput>chown rabbitmq:rabbitmq /var/lib/rabbitmq/.erlang.cookie</userinput>
<prompt>#</prompt> <userinput>chmod 400 /var/lib/rabbitmq/.erlang.cookie</userinput></screen>
<para>Start RabbitMQ on all nodes and verify the nodes are running:</para>
<screen><prompt>#</prompt> <userinput>rabbitmqctl cluster_status</userinput>
<computeroutput>Cluster status of node rabbit@<replaceable>NODE</replaceable>...
[{nodes,[{disc,[rabbit@<replaceable>NODE</replaceable>]}]},
{running_nodes,[rabbit@<replaceable>NODE</replaceable>]},
{partitions,[]}]
...done.</computeroutput></screen>
<para>Run the following commands on all nodes except the first one:</para>
<screen><prompt>#</prompt> <userinput>rabbitmqctl stop_app</userinput>
<computeroutput>Stopping node rabbit@<replaceable>NODE</replaceable>...
...done.</computeroutput>
<prompt>#</prompt> <userinput>rabbitmqctl join_cluster rabbit@rabbit1</userinput>
<prompt>#</prompt> <userinput>rabbitmqctl start_app</userinput>
<computeroutput>Starting node rabbit@<replaceable>NODE</replaceable> ...
...done.</computeroutput></screen>
<para>To verify the cluster status:</para>
<screen><prompt>#</prompt> <userinput>rabbitmqctl cluster_status</userinput>
<computeroutput>Cluster status of node rabbit@<replaceable>NODE</replaceable>...
[{nodes,[{disc,[rabbit@rabbit1]},{ram,[rabbit@<replaceable>NODE</replaceable>]}]},{running_nodes,[rabbit@<replaceable>NODE</replaceable>,rabbit@rabbit1]}]</computeroutput></screen>
<para>If the cluster is working, you can now proceed to creating users and
passwords for queues.</para>
<para>To ensure that all queues, except those with auto-generated names, are
mirrored across all running nodes it is necessary to set the policy key
<literal>ha-mode</literal> to <literal>all</literal>. Run the following
command on one of the nodes:</para>
<screen><prompt>#</prompt> <userinput>rabbitmqctl set_policy ha-all '^(?!amq\.).*' '{"ha-mode": "all"}'</userinput></screen>
<note>
<para>More information about <link xlink:href="http://www.rabbitmq.com/ha.html">
highly available queues</link> and
<link xlink:href="https://www.rabbitmq.com/clustering.html">
clustering</link> can be found in the official RabbitMQ
documentation.</para>
</note>
</section>

68
doc/high-availability-guide/ha_aa_rabbitmq/section_install_rabbitmq.xml
View File

@ -1,68 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_install_rabbitmq">
<title>Install RabbitMQ</title>
<section xml:id="_on_ubuntu_debian">
<title>On Ubuntu and Debian</title>
<para>RabbitMQ is packaged on both distros:</para>
<screen><prompt>#</prompt> <userinput>apt-get install rabbitmq-server</userinput></screen>
<note>
<para>
<link xlink:href="http://www.rabbitmq.com/install-debian.html">
Official manual for installing RabbitMQ on Ubuntu and Debian</link>
</para>
</note>
</section>
<section xml:id="_on_fedora_rhel">
<title>On Fedora and RHEL</title>
<para>RabbitMQ is packaged on both distros:</para>
<screen><prompt>#</prompt> <userinput>yum install rabbitmq-server</userinput></screen>
<note>
<para>
<link xlink:href="http://www.rabbitmq.com/install-rpm.html">
Official manual for installing RabbitMQ on Fedora and RHEL</link>
</para>
</note>
</section>
<section xml:id="_on_opensuse_sles">
<title>On openSUSE and SLES</title>
<procedure>
<title>On openSUSE:</title>
<step>
<screen><prompt>#</prompt> <userinput>zypper install rabbitmq-server</userinput></screen>
</step>
</procedure>
<note>
<para>
<link xlink:href="http://www.rabbitmq.com/install-rpm.html">
Official manual for installing RabbitMQ on openSUSE</link>
</para>
</note>
<para>
<link xlink:href="http://www.rabbitmq.com/install-rpm.html">
Official manual for installing RabbitMQ on openSUSE</link>
</para>
<procedure>
<title>On SLES 12:</title>
<step>
<screen><prompt>#</prompt> <userinput>zypper addrepo -f obs://Cloud:OpenStack:Kilo/SLE_12 Kilo</userinput></screen>
<note>
<para>The packages are signed by GPG key 893A90DAD85F9316. You should
verify the fingerprint of the imported GPG key before using
it.</para>
<programlisting>Key ID: 893A90DAD85F9316
Key Name: Cloud:OpenStack OBS Project &lt;Cloud:OpenStack@build.opensuse.org&gt;
Key Fingerprint: 35B34E18ABC1076D66D5A86B893A90DAD85F9316
Key Created: Tue Oct 8 13:34:21 2013
Key Expires: Thu Dec 17 13:34:21 2015</programlisting>
</note>
</step>
<step>
<screen><prompt>#</prompt> <userinput>zypper install rabbitmq-server</userinput></screen>
</step>
</procedure>
</section>
</section>

67
doc/high-availability-guide/includes/corosync.conf
View File

@ -1,67 +0,0 @@
totem {
version: 2
# Time (in ms) to wait for a token <1>
token: 10000
# How many token retransmits before forming a new
# configuration
token_retransmits_before_loss_const: 10
# Turn off the virtual synchrony filter
vsftype: none
# Enable encryption <2>
secauth: on
# How many threads to use for encryption/decryption
threads: 0
# This specifies the redundant ring protocol, which may be
# none, active, or passive. <3>
rrp_mode: active
# The following is a two-ring multicast configuration. <4>
interface {
ringnumber: 0
bindnetaddr: 192.168.42.0
mcastaddr: 239.255.42.1
mcastport: 5405
}
interface {
ringnumber: 1
bindnetaddr: 10.0.42.0
mcastaddr: 239.255.42.2
mcastport: 5405
}
}
amf {
mode: disabled
}
service {
# Load the Pacemaker Cluster Resource Manager <5>
ver: 1
name: pacemaker
}
aisexec {
user: root
group: root
}
logging {
fileline: off
to_stderr: yes
to_logfile: no
to_syslog: yes
syslog_facility: daemon
debug: off
timestamp: on
logger_subsys {
subsys: AMF
debug: off
tags: enter|leave|trace1|trace2|trace3|trace4|trace6
}
}

11
doc/high-availability-guide/includes/mysql.res
View File

@ -1,11 +0,0 @@
resource mysql {
device minor 0;
disk "/dev/data/mysql";
meta-disk internal;
on node1 {
address ipv4 10.0.42.100:7700;
}
on node2 {
address ipv4 10.0.42.254:7700;
}
}

2
doc/high-availability-guide/includes/pacemaker-api.crm
View File

@ -1,2 +0,0 @@
group g_services_api p_api-ip p_keystone p_glance-api p_cinder-api \
p_neutron-server p_glance-registry p_ceilometer-agent-central

3
doc/high-availability-guide/includes/pacemaker-api_vip.crm
View File

@ -1,3 +0,0 @@
primitive p_api-ip ocf:heartbeat:IPaddr2 \
params ip="192.168.42.103" cidr_netmask="24" \
op monitor interval="30s"

3
doc/high-availability-guide/includes/pacemaker-ceilometer_agent_central.crm
View File

@ -1,3 +0,0 @@
primitive p_ceilometer-agent-central ocf:openstack:ceilometer-agent-central \
params config="/etc/ceilometer/ceilometer.conf" \
op monitor interval="30s" timeout="30s"

4
doc/high-availability-guide/includes/pacemaker-cinder_api.crm
View File

@ -1,4 +0,0 @@
primitive p_cinder-api ocf:openstack:cinder-api \
params config="/etc/cinder/cinder.conf" os_password="secretsecret" os_username="admin" \
os_tenant_name="admin" keystone_get_token_url="http://192.168.42.103:5000/v2.0/tokens" \
op monitor interval="30s" timeout="30s"

3
doc/high-availability-guide/includes/pacemaker-glance_api.crm
View File

@ -1,3 +0,0 @@
primitive p_glance-api ocf:openstack:glance-api \
params config="/etc/glance/glance-api.conf" os_password="secretsecret" os_username="admin" os_tenant_name="admin" os_auth_url="http://192.168.42.103:5000/v2.0/" \
op monitor interval="30s" timeout="30s"

3
doc/high-availability-guide/includes/pacemaker-keystone.crm
View File

@ -1,3 +0,0 @@
primitive p_keystone ocf:openstack:keystone \
params config="/etc/keystone/keystone.conf" os_password="secretsecret" os_username="admin" os_tenant_name="admin" os_auth_url="http://192.168.42.103:5000/v2.0/" \
op monitor interval="30s" timeout="30s"

33
doc/high-availability-guide/includes/pacemaker-mysql.crm
View File

@ -1,33 +0,0 @@
primitive p_ip_mysql ocf:heartbeat:IPaddr2 \
params ip="192.168.42.101" cidr_netmask="24" \
op monitor interval="30s"
primitive p_drbd_mysql ocf:linbit:drbd \
params drbd_resource="mysql" \
op start timeout="90s" \
op stop timeout="180s" \
op promote timeout="180s" \
op demote timeout="180s" \
op monitor interval="30s" role="Slave" \
op monitor interval="29s" role="Master"
primitive p_fs_mysql ocf:heartbeat:Filesystem \
params device="/dev/drbd/by-res/mysql" \
directory="/var/lib/mysql" \
fstype="xfs" \
options="relatime" \
op start timeout="60s" \
op stop timeout="180s" \
op monitor interval="60s" timeout="60s"
primitive p_mysql ocf:heartbeat:mysql \
params additional_parameters="--bind-address=50.56.179.138" \
config="/etc/mysql/my.cnf" \
pid="/var/run/mysqld/mysqld.pid" \
socket="/var/run/mysqld/mysqld.sock" \
log="/var/log/mysql/mysqld.log" \
op monitor interval="20s" timeout="10s" \
op start timeout="120s" \
op stop timeout="120s"
group g_mysql p_ip_mysql p_fs_mysql p_mysql
ms ms_drbd_mysql p_drbd_mysql \
meta notify="true" clone-max="2"
colocation c_mysql_on_drbd inf: g_mysql ms_drbd_mysql:Master
order o_drbd_before_mysql inf: ms_drbd_mysql:promote g_mysql:start

4
doc/high-availability-guide/includes/pacemaker-network-dhcp.crm
View File

@ -1,4 +0,0 @@
primitive p_neutron-dhcp-agent ocf:openstack:neutron-dhcp-agent \
params config="/etc/neutron/neutron.conf" \
plugin_config="/etc/neutron/dhcp_agent.ini" \
op monitor interval="30s" timeout="30s"

4
doc/high-availability-guide/includes/pacemaker-network-l3.crm
View File

@ -1,4 +0,0 @@
primitive p_neutron-l3-agent ocf:openstack:neutron-agent-l3 \
params config="/etc/neutron/neutron.conf" \
plugin_config="/etc/neutron/l3_agent.ini" \
op monitor interval="30s" timeout="30s"

4
doc/high-availability-guide/includes/pacemaker-network-metadata.crm
View File

@ -1,4 +0,0 @@
primitive p_neutron-metadata-agent ocf:openstack:neutron-metadata-agent \
params config="/etc/neutron/neutron.conf" \
agent_config="/etc/neutron/metadata_agent.ini" \
op monitor interval="30s" timeout="30s"

2
doc/high-availability-guide/includes/pacemaker-network.crm
View File

@ -1,2 +0,0 @@
group g_services_network p_neutron-l3-agent p_neutron-dhcp-agent \
p_neutron-metadata_agent

4
doc/high-availability-guide/includes/pacemaker-neutron_server.crm
View File

@ -1,4 +0,0 @@
primitive p_neutron-server ocf:openstack:neutron-server \
params os_password="secretsecret" os_username="admin" os_tenant_name="admin" \
keystone_get_token_url="http://192.168.42.103:5000/v2.0/tokens" \
op monitor interval="30s" timeout="30s"

5
doc/high-availability-guide/includes/pacemaker-properties.crm
View File

@ -1,5 +0,0 @@
property no-quorum-policy="ignore" \ # <1>
pe-warn-series-max="1000" \ # <2>
pe-input-series-max="1000" \
pe-error-series-max="1000" \
cluster-recheck-interval="5min" # <3>

27
doc/high-availability-guide/includes/pacemaker-rabbitmq.crm
View File

@ -1,27 +0,0 @@
primitive p_ip_rabbitmq ocf:heartbeat:IPaddr2 \
params ip="192.168.42.100" cidr_netmask="24" \
op monitor interval="10s"
primitive p_drbd_rabbitmq ocf:linbit:drbd \
params drbd_resource="rabbitmq" \
op start timeout="90s" \
op stop timeout="180s" \
op promote timeout="180s" \
op demote timeout="180s" \
op monitor interval="30s" role="Slave" \
op monitor interval="29s" role="Master"
primitive p_fs_rabbitmq ocf:heartbeat:Filesystem \
params device="/dev/drbd/by-res/rabbitmq" \
directory="/var/lib/rabbitmq" \
fstype="xfs" options="relatime" \
op start timeout="60s" \
op stop timeout="180s" \
op monitor interval="60s" timeout="60s"
primitive p_rabbitmq ocf:rabbitmq:rabbitmq-server \
params nodename="rabbit@localhost" \
mnesia_base="/var/lib/rabbitmq" \
op monitor interval="20s" timeout="10s"
group g_rabbitmq p_ip_rabbitmq p_fs_rabbitmq p_rabbitmq
ms ms_drbd_rabbitmq p_drbd_rabbitmq \
meta notify="true" master-max="1" clone-max="2"
colocation c_rabbitmq_on_drbd inf: g_rabbitmq ms_drbd_rabbitmq:Master
order o_drbd_before_rabbitmq inf: ms_drbd_rabbitmq:promote g_rabbitmq:start

11
doc/high-availability-guide/includes/rabbitmq.res
View File

@ -1,11 +0,0 @@
resource rabbitmq {
device minor 1;
disk "/dev/data/rabbitmq";
meta-disk internal;
on node1 {
address ipv4 10.0.42.100:7701;
}
on node2 {
address ipv4 10.0.42.254:7701;
}
}

2008
doc/high-availability-guide/locale/high-availability-guide.pot
View File

File diff suppressed because it is too large Load Diff

1602
doc/high-availability-guide/locale/ja.po
View File

File diff suppressed because it is too large Load Diff

2647
doc/high-availability-guide/locale/zh_CN.po
View File

File diff suppressed because it is too large Load Diff

45
doc/high-availability-guide/network/section_highly_available_neutron_dhcp_agent.xml
View File

@ -1,45 +0,0 @@
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_highly_available_neutron_dhcp_agent">
<title>Highly available neutron DHCP agent</title>
<para>The neutron DHCP agent distributes IP addresses to the VMs with dnsmasq (by
default). High availability for the DHCP agent is achieved by adopting
Pacemaker.</para>
<note>
<para>Here is the <link
xlink:href="http://docs.openstack.org/admin-guide-cloud/networking_multi-dhcp-agents.html">documentation</link> for installing neutron DHCP agent.</para>
</note>
<section xml:id="_add_neutron_dhcp_agent_resource_to_pacemaker">
<title>Add neutron DHCP agent resource to Pacemaker</title>
<para>First of all, you need to download the resource agent to your system:</para>
<screen><prompt>#</prompt> <userinput>cd /usr/lib/ocf/resource.d/openstack</userinput>
<prompt>#</prompt> <userinput>wget https://raw.github.com/madkiss/openstack-resource-agents/master/ocf/neutron-agent-dhcp</userinput>
<prompt>#</prompt> <userinput>chmod a+rx neutron-agent-dhcp</userinput></screen>
<para>You may now proceed with adding the Pacemaker configuration for
neutron DHCP agent resource. Connect to the Pacemaker cluster with <literal>crm
configure</literal>, and add the following cluster resources:</para>
<programlisting>primitive p_neutron-dhcp-agent ocf:openstack:neutron-agent-dhcp \
params config="/etc/neutron/neutron.conf" \
plugin_config="/etc/neutron/dhcp_agent.ini" \
op monitor interval="30s" timeout="30s"</programlisting>
<para>This configuration creates:</para>
<itemizedlist>
<listitem>
<para><literal>p_neutron-agent-dhcp</literal>, a
resource for managing the neutron DHCP Agent service.</para>
</listitem>
</itemizedlist>
<para><literal>crm configure</literal> supports batch input, so you may copy and paste the
above into your live pacemaker configuration, and then make changes as
required.</para>
<para>Once completed, commit your configuration changes by entering <literal>commit</literal>
from the <literal>crm configure</literal> menu. Pacemaker will then start the neutron DHCP
agent service, and its dependent resources, on one of your nodes.</para>
</section>
</section>

49
doc/high-availability-guide/network/section_highly_available_neutron_l3_agent.xml
View File

@ -1,49 +0,0 @@
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_highly_available_neutron_l3_agent">
<title>Highly available neutron L3 agent</title>
<para>The neutron L3 agent provides L3/NAT forwarding to ensure external network access
for VMs on tenant networks. High availability for the L3 agent is achieved by
adopting Pacemaker.</para>
<note>
<para>Here is the <link xlink:href="http://docs.openstack.org/admin-guide-cloud/networking_config-agents.html#configure-l3-agent">documentation</link>
for installing neutron L3 agent.</para>
</note>
<section xml:id="_add_neutron_l3_agent_resource_to_pacemaker">
<title>Add neutron L3 agent resource to Pacemaker</title>
<para>First of all, you need to download the resource agent to your system:</para>
<screen><prompt>#</prompt> <userinput>cd /usr/lib/ocf/resource.d/openstack</userinput>
<prompt>#</prompt> <userinput>wget https://raw.github.com/madkiss/openstack-resource-agents/master/ocf/neutron-agent-l3</userinput>
<prompt>#</prompt> <userinput>chmod a+rx neutron-l3-agent</userinput></screen>
<para>You may now proceed with adding the Pacemaker configuration for
neutron L3 agent resource. Connect to the Pacemaker cluster with <literal>crm
configure</literal>, and add the following cluster resources:</para>
<programlisting>primitive p_neutron-l3-agent ocf:openstack:neutron-agent-l3 \
params config="/etc/neutron/neutron.conf" \
plugin_config="/etc/neutron/l3_agent.ini" \
op monitor interval="30s" timeout="30s"</programlisting>
<para>This configuration creates</para>
<itemizedlist>
<listitem>
<para><literal>p_neutron-l3-agent</literal>, a resource for manage Neutron L3 Agent service
</para>
</listitem>
</itemizedlist>
<para><literal>crm configure</literal> supports batch input, so you may copy and paste the
above into your live pacemaker configuration, and then make changes as
required.</para>
<para>Once completed, commit your configuration changes by entering <literal>commit</literal>
from the <literal>crm configure</literal> menu. Pacemaker will then start the neutron L3 agent
service, and its dependent resources, on one of your nodes.</para>
<note>
<para>This method does not ensure a zero downtime since it has to recreate all
the namespaces and virtual routers on the node.</para>
</note>
</section>
</section>

41
doc/high-availability-guide/network/section_highly_available_neutron_metadata_agent.xml
View File

@ -1,41 +0,0 @@
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_highly_available_neutron_metadata_agent">
<title>Highly available neutron metadata agent</title>
<para>Neutron metadata agent allows Compute API metadata to be reachable by VMs on tenant
networks. High availability for the metadata agent is achieved by adopting
Pacemaker.</para>
<note>
<para>Here is the <link xlink:href="http://docs.openstack.org/kilo/config-reference/content/networking-options-metadata.html">documentation</link> for installing Neutron Metadata Agent.</para>
</note>
<section xml:id="_add_neutron_metadata_agent_resource_to_pacemaker">
<title>Add neutron metadata agent resource to Pacemaker</title>
<para>First of all, you need to download the resource agent to your system:</para>
<screen><prompt>#</prompt> <userinput>cd /usr/lib/ocf/resource.d/openstack</userinput>
<prompt>#</prompt> <userinput>wget https://raw.github.com/madkiss/openstack-resource-agents/master/ocf/neutron-metadata-agent</userinput>
<prompt>#</prompt> <userinput>chmod a+rx neutron-metadata-agent</userinput></screen>
<para>You may now proceed with adding the Pacemaker configuration for
neutron metadata agent resource. Connect to the Pacemaker cluster with <literal>crm
configure</literal>, and add the following cluster resources:</para>
<programlisting>primitive p_neutron-metadata-agent ocf:openstack:neutron-metadata-agent \
params config="/etc/neutron/neutron.conf" \
agent_config="/etc/neutron/metadata_agent.ini" \
op monitor interval="30s" timeout="30s"</programlisting>
<para>This configuration creates</para>
<itemizedlist>
<listitem>
<para><literal>p_neutron-metadata-agent</literal>, a resource for manage Neutron Metadata Agent
service
</para>
</listitem>
</itemizedlist>
<para><literal>crm configure</literal> supports batch input, so you may copy and paste the
above into your live Pacemaker configuration, and then make changes as
required.</para>
<para>Once completed, commit your configuration changes by entering <literal>commit</literal>
from the <literal>crm configure</literal> menu. Pacemaker will then start the neutron metadata
agent service, and its dependent resources, on one of your nodes.</para>
</section>
</section>

15
doc/high-availability-guide/network/section_manage_network_resources.xml
View File

@ -1,15 +0,0 @@
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_manage_network_resources">
<title>Manage network resources</title>
<para>You can now add the Pacemaker configuration for
managing all network resources together with a group.
Connect to the Pacemaker cluster with <literal>crm configure</literal>, and add the following
cluster resources:</para>
<programlisting>group g_services_network p_neutron-l3-agent p_neutron-dhcp-agent \
p_neutron-metadata_agent</programlisting>
</section>

25
doc/high-availability-guide/openstack.ent
View File

@ -1,25 +0,0 @@
<!-- The master of this file is in openstack-manuals repository, file
doc/common/entities/openstack.ent.
Any changes to the master file will override changes in other
repositories. -->
<!-- Some useful entities borrowed from HTML -->
<!ENTITY ndash "&#x2013;">
<!ENTITY mdash "&#x2014;">
<!ENTITY nbsp "&#160;">
<!ENTITY times "&#215;">
<!ENTITY hellip "&#133;">
<!-- Useful for describing APIs in the User Guide -->
<!ENTITY COPY '<command xmlns="http://docbook.org/ns/docbook">COPY</command>'>
<!ENTITY GET '<command xmlns="http://docbook.org/ns/docbook">GET</command>'>
<!ENTITY HEAD '<command xmlns="http://docbook.org/ns/docbook">HEAD</command>'>
<!ENTITY PUT '<command xmlns="http://docbook.org/ns/docbook">PUT</command>'>
<!ENTITY POST '<command xmlns="http://docbook.org/ns/docbook">POST</command>'>
<!ENTITY DELETE '<command xmlns="http://docbook.org/ns/docbook">DELETE</command>'>
<!ENTITY CHECK '<inlinemediaobject xmlns="http://docbook.org/ns/docbook">
<imageobject>
<imagedata fileref="figures/Check_mark_23x20_02.svg"
format="SVG" scale="60"/>
</imageobject>
</inlinemediaobject>'>

44
doc/high-availability-guide/pacemaker/section_install_packages.xml
View File

@ -1,44 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_install_packages">
<title>Install packages</title>
<para>On any host that is meant to be part of a Pacemaker cluster, you must
first establish cluster communications through the Corosync messaging
layer. This involves installing the following packages (and their
dependencies, which your package manager will normally install
automatically):</para>
<itemizedlist>
<listitem>
<para><package>pacemaker</package> (Note that the crm shell should be
downloaded separately.)</para>
</listitem>
<listitem>
<para>
<package>crmsh</package>
</para>
</listitem>
<listitem>
<para>
<package>corosync</package>
</para>
</listitem>
<listitem>
<para>
<package>cluster-glue</package>
</para>
</listitem>
<listitem>
<para><package>fence-agents</package> (Fedora only; all other
distributions use fencing agents from
<package>cluster-glue</package>)</para>
</listitem>
<listitem>
<para>
<package>resource-agents</package>
</para>
</listitem>
</itemizedlist>
</section>

54
doc/high-availability-guide/pacemaker/section_set_basic_cluster_properties.xml
View File

@ -1,54 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_set_basic_cluster_properties">
<title>Set basic cluster properties</title>
<para>Once your Pacemaker cluster is set up, it is recommended to set a few
basic cluster properties. To do so, start the <command>crm</command> shell
and change into the configuration menu by entering
<literal>configure</literal>. Alternatively, you may jump straight into
the Pacemaker configuration menu by typing <command>crm configure</command>
directly from a shell prompt.</para>
<para>Then, set the following properties:</para>
<programlisting>property no-quorum-policy="ignore" \ # <co xml:id="CO2-1"/>
pe-warn-series-max="1000" \ # <co xml:id="CO2-2"/>
pe-input-series-max="1000" \
pe-error-series-max="1000" \
cluster-recheck-interval="5min" # <co xml:id="CO2-3"/></programlisting>
<calloutlist>
<callout arearefs="CO2-1">
<para>Setting <option>no-quorum-policy="ignore"</option> is required
in 2-node Pacemaker clusters for the following reason: if quorum
enforcement is enabled, and one of the two nodes fails, then the
remaining node can not establish a majority of quorum votes necessary
to run services, and thus it is unable to take over any resources. In
this case, the appropriate workaround is to ignore loss of quorum in
the cluster. This should only only be done in 2-node clusters: do not
set this property in Pacemaker clusters with more than two nodes. Note
that a two-node cluster with this setting exposes a risk of
split-brain because either half of the cluster, or both, are able to
become active in the event that both nodes remain online but lose
communication with one another. The preferred configuration is 3 or
more nodes per cluster.</para>
</callout>
<callout arearefs="CO2-2">
<para>Setting <option>pe-warn-series-max</option>,
<option>pe-input-series-max</option> and
<option>pe-error-series-max</option> to 1000 instructs Pacemaker to
keep a longer history of the inputs processed, and errors and warnings
generated, by its Policy Engine. This history is typically useful in
case cluster troubleshooting becomes necessary.</para>
</callout>
<callout arearefs="CO2-3">
<para>Pacemaker uses an event-driven approach to cluster state
processing. However, certain Pacemaker actions occur at a configurable
interval, <option>cluster-recheck-interval</option>, which defaults to
15 minutes. It is usually prudent to reduce this to a shorter interval,
such as 5 or 3 minutes.</para>
</callout>
</calloutlist>
<para>Once you have made these changes, you may <literal>commit</literal>
the updated configuration.</para>
</section>

397
doc/high-availability-guide/pacemaker/section_set_up_corosync.xml
View File

@ -1,397 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_set_up_corosync">
<title>Set up Corosync</title>
<para>Besides installing the <package>Corosync</package> package, you must
also create a configuration file, stored in
<filename>/etc/corosync/corosync.conf</filename>. Corosync can be configured
to work with either multicast or unicast IP addresses.
</para>
<section xml:id="_set_up_corosync_multicast">
<title>Set up Corosync with multicast</title>
<para>Most distributions ship an example configuration file (<filename>corosync.conf.example</filename>)
as part of the documentation bundled with the <package>Corosync</package>
package. An example Corosync configuration file is shown below:</para>
<formalpara>
<title>Corosync configuration file (<filename>corosync.conf</filename>)</title>
<para>
<programlisting language="ini">totem {
version: 2
# Time (in ms) to wait for a token <co xml:id="CO1-1"/>
token: 10000
# How many token retransmits before forming a new
# configuration
token_retransmits_before_loss_const: 10
# Turn off the virtual synchrony filter
vsftype: none
# Enable encryption <co xml:id="CO1-2"/>
secauth: on
# How many threads to use for encryption/decryption
threads: 0
# This specifies the redundant ring protocol, which may be
# none, active, or passive. <co xml:id="CO1-3"/>
rrp_mode: active
# The following is a two-ring multicast configuration. <co xml:id="CO1-4"/>
interface {
ringnumber: 0
bindnetaddr: 192.168.42.0
mcastaddr: 239.255.42.1
mcastport: 5405
}
interface {
ringnumber: 1
bindnetaddr: 10.0.42.0
mcastaddr: 239.255.42.2
mcastport: 5405
}
}
amf {
mode: disabled
}
service {
# Load the Pacemaker Cluster Resource Manager <co xml:id="CO1-5"/>
ver: 1
name: pacemaker
}
aisexec {
user: root
group: root
}
logging {
fileline: off
to_stderr: yes
to_logfile: no
to_syslog: yes
syslog_facility: daemon
debug: off
timestamp: on
logger_subsys {
subsys: AMF
debug: off
tags: enter|leave|trace1|trace2|trace3|trace4|trace6
}}</programlisting>
</para>
</formalpara>
<calloutlist>
<callout arearefs="CO1-1">
<para>The <option>token</option> value specifies the time, in
milliseconds, during which the Corosync token is expected to be
transmitted around the ring. When this timeout expires, the token is
declared lost, and after <option>token_retransmits_before_loss_const</option>
lost tokens the non-responding processor (cluster node) is declared
dead. In other words,
<option>token</option> × <option>token_retransmits_before_loss_const</option>
is the maximum time a node is allowed to not respond to cluster
messages before being considered dead. The default for
<option>token</option> is 1000 (1 second), with 4 allowed
retransmits. These defaults are intended to minimize failover times,
but can cause frequent "false alarms" and unintended failovers in case
of short network interruptions. The values used here are safer, albeit
with slightly extended failover times.</para>
</callout>
<callout arearefs="CO1-2">
<para>With <option>secauth</option> enabled, Corosync nodes mutually
authenticate using a 128-byte shared secret stored in
<filename>/etc/corosync/authkey</filename>, which may be generated with
the <command>corosync-keygen</command> utility. When using
<option>secauth</option>, cluster communications are also
encrypted.</para>
</callout>
<callout arearefs="CO1-3">
<para>In Corosync configurations using redundant networking (with more
than one <option>interface</option>), you must select a Redundant
Ring Protocol (RRP) mode other than <literal>none</literal>.
<literal>active</literal> is the recommended RRP mode.</para>
</callout>
<callout arearefs="CO1-4">
<para>There are several things to note about the recommended interface
configuration:</para>
<itemizedlist>
<listitem>
<para>The <option>ringnumber</option> must differ between all
configured interfaces, starting with 0.</para>
</listitem>
<listitem>
<para>The <option>bindnetaddr</option> is the network address of
the interfaces to bind to. The example uses two network addresses
of <literal>/24</literal> IPv4 subnets.</para>
</listitem>
<listitem>
<para>Multicast groups (<option>mcastaddr</option>) must not be
reused across cluster boundaries. In other words, no two distinct
clusters should ever use the same multicast group. Be sure to
select multicast addresses compliant with
<link xlink:href="http://www.ietf.org/rfc/rfc2365.txt">RFC 2365,
"Administratively Scoped IP Multicast"</link>.</para>
</listitem>
<listitem>
<para>For firewall configurations, note that Corosync communicates
over UDP only, and uses <literal>mcastport</literal> (for receives)
and <literal>mcastport - 1</literal> (for sends).</para>
</listitem>
</itemizedlist>
</callout>
<callout arearefs="CO1-5">
<para>The <literal>service</literal> declaration for the
<literal>pacemaker</literal> service may be placed in the
<filename>corosync.conf</filename> file directly, or in its own
separate file,
<filename>/etc/corosync/service.d/pacemaker</filename>.</para>
<note>
<para>If you are using Corosync version 2 on Ubuntu 14.04, remove
or comment out lines under the service stanza, which enables
Pacemaker to start up.</para>
</note>
</callout>
</calloutlist>
<para>Once created, the <filename>corosync.conf</filename> file (and the
<filename>authkey</filename> file if the <option>secauth</option> option
is enabled) must be synchronized across all cluster nodes.</para>
</section>
<section xml:id="_set_up_corosync_unicast">
<title>Set up Corosync with unicast</title>
<para>Some environments may not support multicast. For such cases,
Corosync should be configured for unicast.
An example fragment of the Corosync configuration file is shown below:</para>
<formalpara>
<title>Corosync configuration file fragment (<filename>corosync.conf</filename>)</title>
<para>
<programlisting language="ini">totem {
#...
interface {
ringnumber: 0
bindnetaddr: 192.168.42.0
broadcast: yes <co xml:id="CO1-6"/>
mcastport: 5405
}
interface {
ringnumber: 1
bindnetaddr: 10.0.42.0
broadcast: yes
mcastport: 5405
}
transport: udpu <co xml:id="CO1-7"/>
}
nodelist { <co xml:id="CO1-8"/>
node {
ring0_addr: 192.168.42.1
ring1_addr: 10.0.42.1
nodeid: 1
}
node {
ring0_addr: 192.168.42.2
ring1_addr: 10.0.42.2
nodeid: 2
}
}
#...</programlisting>
</para>
</formalpara>
<calloutlist>
<callout arearefs="CO1-6">
<para>If the <option>broadcast</option> is set to yes, the broadcast address
is used for communication. If this option is set, <option>mcastaddr</option>
should not be set.</para>
</callout>
<callout arearefs="CO1-7">
<para>The <option>transport</option> directive controls the transport mechanism used.
To avoid the use of multicast entirely, a unicast transport parameter <option>udpu</option> should
be specified. This requires specifying the list of members in <option>nodelist</option>
directive; this could potentially make up the membership before deployment.
The default is <option>udp</option>. The transport type can also be set to <option>udpu</option> or <option>iba</option>.</para>
</callout>
<callout arearefs="CO1-8">
<para>Within the <option>nodelist</option> directive, it is possible to specify specific
information about nodes in cluster. Directive can contain only the <option>node</option>
sub-directive, which specifies every node that should be a member of the membership,
and where non-default options are needed. Every node must have at least the <option>ring0_addr</option>
field filled.</para>
<note>
<para>For UDPU, every node that should be a member of the membership must
be specified.</para>
</note>
<para>Possible options are:</para>
<para>The <option>ringX_addr</option> specifies the IP address of one of the nodes.
X is the ring number.</para>
<para>The <option>nodeid</option> configuration option is optional when
using IPv4 and required when using IPv6. This is a 32-bit value
specifying the node identifier delivered to the cluster membership
service. When using IPv4, this defaults to the 32-bit IP address to
which the system is bound with the ring identifier of 0. The node
identifier value of zero is reserved and should not be used.</para>
</callout>
</calloutlist>
</section>
<section xml:id="_set_up_corosync_votequorum_full_size">
<title>Set up Corosync with votequorum library for a full-size cluster</title>
<para>This section describes a full-size cluster configuration with three or
more members which is appropriate for production deployments.
For a two-node configuration that can be used for demostrations and testing,
please go to the next section.
</para>
<para>Votequorum library is a part of the Corosync project. It provides
an interface to the vote-based quorum service and it must be explicitly
enabled in the Corosync configuration file. The main role of the votequorum
library is to avoid split-brain situations, but it also provides a mechanism
to:
<itemizedlist>
<listitem>
<para>Query the quorum status</para>
</listitem>
<listitem>
<para>Get a list of nodes known to the quorum service</para>
</listitem>
<listitem>
<para>Receive notifications of quorum state changes</para>
</listitem>
<listitem>
<para>Change the number of votes assigned to a node</para>
</listitem>
<listitem>
<para>Change the number of expected votes for a cluster to be
quorate</para>
</listitem>
<listitem>
<para>Connect an additional quorum device to allow small clusters
remain quorate during node outages</para>
</listitem>
</itemizedlist>
Votequorum library has been created to replace and eliminate from advanced
cluster configurations qdisk, disk-based quorum daemon for CMAN.
</para>
<formalpara>
<title>Votequorum service configuration within Corosync</title>
<para>
<programlisting language="ini">quorum {
provider: corosync_votequorum <co xml:id="CR0-1"/>
expected_votes: 7 <co xml:id="CR0-2"/>
wait_for_all: 1 <co xml:id="CR0-3"/>
last_man_standing: 1 <co xml:id="CR0-4"/>
last_man_standing_window: 10000 <co xml:id="CR0-5"/>
}</programlisting>
</para>
</formalpara>
<calloutlist>
<callout arearefs="CR0-1"><para>Provider <option>corosync_votequorum</option>
enables votequorum library, this is the only required option.</para>
</callout>
<callout arearefs="CR0-2"><para>The cluster is fully operational with
<option>expected_votes</option> 7 nodes (each node has 1 vote),
quorum: 4. If list of nodes is specified as <option>nodelist</option>
the <option>expected_votes</option> is ignored.</para>
</callout>
<callout arearefs="CR0-3"><para><option>wait_for_all</option>
When starting up a cluster (all nodes down) it will hold the cluster quorum
until all of the nodes become online and joint the cluster first time
(new in Corosync 2.0).</para>
</callout>
<callout arearefs="CR0-4"><para><option>last_man_standing</option> enable
Last Man Standing (LMS) feature (disabled by default: 0). If a cluster is on
the quorum edge (expected_votes: 7; online nodes: 4) for time longer than
configured in <option>last_man_standing_window</option>, the cluster can
recalculate quorum and continue operating even if the next node will be
lost. This logic is repeated until the number of online nodes in the cluster
reach 2. In order to allow cluster step down from 2 members to only 1,
what is not recommended <option>auto_tie_breaker</option> option needs
to be set.</para>
</callout>
<callout arearefs="CR0-5"><para><option>last_man_standing_window</option>
is time required to recalculate the quorum after one or most hosts have been
lost from the cluster. To do the new quorum recalculation, the cluster needs
to have quorum at least for <option>last_man_standing_window</option>,
time in [ms] (default: 10000ms).</para>
</callout>
</calloutlist>
</section>
<section xml:id="_set_up_corosync_votequorum_two_hosts_cluster">
<title>Set up Corosync with votequorum library for two-host clusters</title>
<para>The two-node cluster configuration is a special case that Pacemaker
supports for demonstration and testing; it is a special feature of the
votequorum library and is not recommended for production environments.
</para>
<formalpara>
<title>Multicast votequorum service configuration for two-host
Corosync cluster</title>
<para>
<programlisting language="ini">quorum {
provider: corosync_votequorum <co xml:id="CR1-0"/>
expected_votes: 2
two_node: 1 <co xml:id="CR1-1"/>
}</programlisting>
</para>
</formalpara>
<calloutlist>
<callout arearefs="CR1-0"><para><option>corosync_votequorum</option>
enables votequorum provider library.</para>
</callout>
<callout arearefs="CR1-1"><para>Put the cluser into two-node operational
mode (default: 0).
<note><para>Setting <option>two_node</option> to 1 enables
<option>wait_for_all</option> by default. It is still possible to
override <option>wait_for_all</option> by explicitly setting it
to 0. If more than 2 nodes join the cluster, the
<option>two_node</option> option is automatically disabled.</para>
</note>
<important>
<para>Disabling <option>wait_for_all</option> in a two-node cluster
may be dangerous as it can lead to a situation (stonith deathmatch)
where each node comes up, assumes the other is down, and fences
peer in order to safely start clustered services. The peer
eventually comes up and repeats the process until the underlying
fault is rectified.</para>
</important>
</para>
</callout>
</calloutlist>
<formalpara>
<title>Unicast (UDP) votequorum service configuration for two-host
Corosync cluster</title>
<para>
<programlisting language="ini">quorum {
provider: corosync_votequorum <co xml:id="CR1-4"/>
two_node: 1 <co xml:id="CR1-5"/>
}
nodelist { <co xml:id="CR1-6"/>
node {
ring0_addr: 192.168.1.1
}
node {
ring0_addr: 192.168.1.2
}
}</programlisting>
</para>
</formalpara>
<calloutlist>
<callout arearefs="CR1-4"><para><option>corosync_votequorum</option>
enables votequorum provider library.</para>
</callout>
<callout arearefs="CR1-5"><para>For the cluster to work with two members only
<option>two_node</option> artificially sets quorum below mathematical
majority.</para>
</callout>
<callout arearefs="CR1-6"><para>Unicast Corosync configuratrion requires
<option>nodelist</option> option to explicitly provide a list of cluster
members.</para>
</callout>
</calloutlist>
</section>
</section>

48
doc/high-availability-guide/pacemaker/section_start_pacemaker.xml
View File

@ -1,48 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_start_pacemaker">
<title>Start Pacemaker</title>
<para>Once the Corosync services have been started and you have established
that the cluster is communicating properly, it is safe to start
<systemitem class="service">pacemakerd</systemitem>, the Pacemaker
master control process:</para>
<itemizedlist>
<listitem>
<para>
<command>/etc/init.d/pacemaker start</command> (LSB)
</para>
</listitem>
<listitem>
<para>
<command>service pacemaker start</command> (LSB, alternate)
</para>
</listitem>
<listitem>
<para>
<command>start pacemaker</command> (upstart)
</para>
</listitem>
<listitem>
<para>
<command>systemctl start pacemaker</command> (systemd)
</para>
</listitem>
</itemizedlist>
<para>Once the Pacemaker services have started, Pacemaker will create a default
empty cluster configuration with no resources. You may observe
Pacemaker's status with the <command>crm_mon</command> utility:</para>
<screen><computeroutput>============
Last updated: Sun Oct 7 21:07:52 2012
Last change: Sun Oct 7 20:46:00 2012 via cibadmin on node2
Stack: openais
Current DC: node2 - partition with quorum
Version: 1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c
2 Nodes configured, 2 expected votes
0 Resources configured.
============
Online: [ node2 node1 ]</computeroutput></screen>
</section>

62
doc/high-availability-guide/pacemaker/section_starting_corosync.xml
View File

@ -1,62 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<section xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="_starting_corosync">
<title>Starting Corosync</title>
<para>Corosync is started as a regular system service. Depending on your
distribution, it may ship with an LSB init script, an
upstart job, or a systemd unit file. Either way, the service is
usually named <systemitem class="service">corosync</systemitem>:</para>
<itemizedlist>
<listitem>
<para>
<command>/etc/init.d/corosync start</command> (LSB)
</para>
</listitem>
<listitem>
<para>
<command>service corosync start</command> (LSB, alternate)
</para>
</listitem>
<listitem>
<para>
<command>start corosync</command> (upstart)
</para>
</listitem>
<listitem>
<para>
<command>systemctl start corosync</command> (systemd)
</para>
</listitem>
</itemizedlist>
<para>You can now check the Corosync connectivity with two tools.</para>
<para>The <command>corosync-cfgtool</command> utility, when invoked with
the <option>-s</option> option, gives a summary of the health of the
communication rings:</para>
<screen><prompt>#</prompt> <userinput>corosync-cfgtool -s</userinput>
<computeroutput>Printing ring status.
Local node ID 435324542
RING ID 0
id = 192.168.42.82
status = ring 0 active with no faults
RING ID 1
id = 10.0.42.100
status = ring 1 active with no faults</computeroutput></screen>
<para>The <command>corosync-objctl</command> utility can be used to dump the
Corosync cluster member list:</para>
<screen><prompt>#</prompt> <userinput>corosync-objctl runtime.totem.pg.mrp.srp.members</userinput>
<computeroutput>runtime.totem.pg.mrp.srp.435324542.ip=r(0) ip(192.168.42.82) r(1) ip(10.0.42.100)
runtime.totem.pg.mrp.srp.435324542.join_count=1
runtime.totem.pg.mrp.srp.435324542.status=joined
runtime.totem.pg.mrp.srp.983895584.ip=r(0) ip(192.168.42.87) r(1) ip(10.0.42.254)
runtime.totem.pg.mrp.srp.983895584.join_count=1
runtime.totem.pg.mrp.srp.983895584.status=joined</computeroutput></screen>
<para>You should see a <literal>status=joined</literal> entry for each of
your constituent cluster nodes.</para>
<note>
<para>If you are using Corosync version 2, use the <command>corosync-cmapctl</command> utility
as it is a direct replacement for <command>corosync-objctl</command>.</para>
</note>
</section>

13
doc/high-availability-guide/part_active_active.xml
View File

@ -1,13 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<part xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ha-using-active-active">
<title>HA using active/active</title>
<xi:include href="ch_ha_aa_db.xml"/>
<xi:include href="ch_ha_aa_rabbitmq.xml"/>
<xi:include href="ch_ha_aa_haproxy.xml"/>
<xi:include href="ch_ha_aa_controllers.xml"/>
<xi:include href="ch_ha_aa_network.xml"/>
</part>

12
doc/high-availability-guide/part_active_passive.xml
View File

@ -1,12 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<part xmlns="http://docbook.org/ns/docbook"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="ha-using-active-passive">
<title>HA using active/passive</title>
<xi:include href="ch_pacemaker.xml"/>
<xi:include href="ch_controller.xml"/>
<xi:include href="ch_api.xml"/>
<xi:include href="ch_network.xml"/>
</part>

78
doc/high-availability-guide/pom.xml
View File

@ -1,78 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<parent>
<groupId>org.openstack.docs</groupId>
<artifactId>parent-pom</artifactId>
<version>1.0.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>openstack-high-availability-guide</artifactId>
<packaging>jar</packaging>
<name>OpenStack High Availability Guide</name>
<properties>
<!-- This is set by Jenkins according to the branch. -->
<release.path.name></release.path.name>
<comments.enabled>0</comments.enabled>
</properties>
<!-- ################################################ -->
<!-- USE "mvn clean generate-sources" to run this POM -->
<!-- ################################################ -->
<build>
<plugins>
<plugin>
<groupId>com.rackspace.cloud.api</groupId>
<artifactId>clouddocs-maven-plugin</artifactId>
<!-- version set in ../pom.xml -->
<executions>
<execution>
<id>generate-webhelp</id>
<goals>
<goal>generate-webhelp</goal>
</goals>
<phase>generate-sources</phase>
<configuration>
<!-- These parameters only apply to webhelp -->
<enableDisqus>${comments.enabled}</enableDisqus>
<disqusShortname>openstack-ha-guide</disqusShortname>
<enableGoogleAnalytics>1</enableGoogleAnalytics>
<googleAnalyticsId>UA-17511903-1</googleAnalyticsId>
<generateToc>
appendix toc,title
article/appendix nop
article toc,title
book toc,title,figure,table,example,equation
chapter toc,title
section toc
part toc,title
qandadiv toc
qandaset toc
reference toc,title
set toc,title
</generateToc>
<!-- The following elements sets the autonumbering of sections in output for chapter numbers but no numbered sections-->
<sectionAutolabel>0</sectionAutolabel>
<tocSectionDepth>1</tocSectionDepth>
<sectionLabelIncludesComponentLabel>0</sectionLabelIncludesComponentLabel>
<webhelpDirname>high-availability-guide</webhelpDirname>
<pdfFilenameBase>high-availability-guide</pdfFilenameBase>
</configuration>
</execution>
</executions>
<configuration>
<!-- These parameters apply to pdf and webhelp -->
<xincludeSupported>true</xincludeSupported>
<sourceDirectory>.</sourceDirectory>
<includes>
bk-ha-guide.xml
</includes>
<canonicalUrlBase>http://docs.openstack.org/high-availability-guide/content</canonicalUrlBase>
<glossaryCollection>${basedir}/../glossary/glossary-terms.xml</glossaryCollection>
<branding>openstack</branding>
</configuration>
</plugin>
</plugins>
</build>
</project>

46
doc/pom.xml
View File

@ -1,46 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.openstack.docs</groupId>
<artifactId>parent-pom</artifactId>
<version>1.0.0-SNAPSHOT</version>
<packaging>pom</packaging>
<modules>
<module>high-availability-guide</module>
</modules>
<profiles>
<profile>
<id>Rackspace Research Repositories</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<repositories>
<repository>
<id>rackspace-research</id>
<name>Rackspace Research Repository</name>
<url>http://maven.research.rackspacecloud.com/content/groups/public/</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>rackspace-research</id>
<name>Rackspace Research Repository</name>
<url>http://maven.research.rackspacecloud.com/content/groups/public/</url>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<build>
<plugins>
<plugin>
<groupId>com.rackspace.cloud.api</groupId>
<artifactId>clouddocs-maven-plugin</artifactId>
<version>2.1.4</version>
</plugin>
</plugins>
</build>
</project>

2
tools/build-all-rst.sh
View File

@ -3,4 +3,4 @@
mkdir -p publish-docs
tools/build-rst.sh doc/ha-guide --build build \
--target draft/ha-guide
--target ha-guide

15
tox.ini
View File

@ -20,27 +20,25 @@ whitelist_externals =
commands = {posargs}
[testenv:checklinks]
commands = openstack-doc-test --check-links {posargs}
commands = true
[testenv:checkniceness]
commands =
openstack-doc-test --check-niceness {posargs}
doc8 doc
[testenv:checksyntax]
commands =
openstack-doc-test --check-syntax {posargs}
# Check that .po and .pot files are valid:
bash -c "find doc -type f -regex '.*\.pot?' -print0|xargs -0 -n 1 msgfmt --check-format -o /dev/null"
[testenv:checkdeletions]
commands = openstack-doc-test --check-deletions {posargs}
commands = true
[testenv:checkbuild]
commands =
# Build and copy RST Guides
{toxinidir}/tools/build-all-rst.sh
# Build DocBook Guide
# This only generates the index page
openstack-doc-test --check-build {posargs}
[testenv:publishdocs]
@ -50,8 +48,6 @@ commands =
commands =
# Build and copy RST Guides
{toxinidir}/tools/build-all-rst.sh
# Build DocBook Guide
openstack-doc-test --check-build --publish
[testenv:checklang]
# Generatedocbook needs xml2po which cannot be installed
@ -59,7 +55,10 @@ commands =
# use sitepackages.
sitepackages=True
whitelist_externals = doc-tools-check-languages
commands = doc-tools-check-languages doc-tools-check-languages.conf test all
commands =
doc-tools-check-languages doc-tools-check-languages.conf test all
# Check that .po and .pot files are valid:
bash -c "find doc -type f -regex '.*\.pot?' -print0|xargs -0 -n 1 msgfmt --check-format -o /dev/null"
[testenv:buildlang]
# Run as "tox -e buildlang -- $LANG"