From 827a5176d99479ec9762a7d8dd3571057505f7ec Mon Sep 17 00:00:00 2001 From: Andrew Smith Date: Tue, 26 Jul 2016 10:29:13 -0400 Subject: [PATCH] [AMQP 1.0] AMQP 1.0 Driver User Guide Document update This patch includes updates to the user guide document for the amqp-dispatch-router feature branch. Includes information for deploying driver to operate with router or broker intermediaries. Change-Id: Icb32743a82bf0f1e5969a215d880dc736a22f408 Closes-Bug: #1596649 --- doc/source/AMQP1.0.rst | 507 +++++++++++++++++++++++++++++++++++------ 1 file changed, 438 insertions(+), 69 deletions(-) diff --git a/doc/source/AMQP1.0.rst b/doc/source/AMQP1.0.rst index 31c5cc351..f38f0c78d 100644 --- a/doc/source/AMQP1.0.rst +++ b/doc/source/AMQP1.0.rst @@ -1,6 +1,6 @@ -------------------------- -AMQP 1.0 Protocol Support -------------------------- +----------------------------------------- +AMQP 1.0 Protocol Driver Deployment Guide +----------------------------------------- .. currentmodule:: oslo_messaging @@ -8,24 +8,98 @@ AMQP 1.0 Protocol Support Introduction ============ -This release of oslo.messaging includes an experimental driver that -provides support for version 1.0 of the Advanced Message Queuing -Protocol (AMQP 1.0, ISO/IEC 19464). +The AMQP 1.0 Protocol Driver is a messaging transport backend +supported in oslo.messaging. The driver maps the base oslo.messaging +capabilities for RPC and Notification message exchange onto version +1.0 of the Advanced Message Queuing Protocol (AMQP 1.0, ISO/IEC +19464). The driver is intended to support any messaging intermediary +(e.g. broker or router) that implements version 1.0 of the AMQP +protocol. -The current implementation of this driver is considered -*experimental*. It is not recommended that this driver be used in -production systems. Rather, this driver is being provided as a -*technical preview*, in hopes that it will encourage further testing -by the AMQP 1.0 community. +More detail regarding the AMQP 1.0 Protocol is available from the +`AMQP specification`_. -More detail regarding the driver's implementation is available from the `specification`_. +.. _AMQP specification: http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-overview-v1.0-os.html -.. _specification: https://git.openstack.org/cgit/openstack/oslo-specs/tree/specs/juno/amqp10-driver-implementation.rst +More detail regarding the driver's implementation is available from +the `oslo specification`_. + +.. _oslo specification: https://git.openstack.org/cgit/openstack/oslo-specs/tree/specs/juno/amqp10-driver-implementation.rst + +======== +Abstract +======== + +The AMQP 1.0 driver is one of a family of oslo.messaging backend +drivers. It currently supports two types of message intermediaries. +The first type is an AMQP 1.0 messaging broker and the second type is +an AMQP 1.0 message router. The driver should support additional +intermediary types in the future but may require additions to driver +configuration parameters in order to do so. + ++--------------+-----------+------------+------------+-----------+ +| Intermediary | RPC | Notify | Message | Topology | +| Type | Pattern | Pattern | Treatment | | ++--------------+-----------+------------+------------+-----------+ +| Message | Yes | `Limited`_ | Direct | Single or | +| Router | | | Messaging | Mesh | ++--------------+-----------+------------+------------+-----------+ +| Message | Yes | Yes | Store and | Single or | +| Broker | | | Forward | Cluster | ++--------------+-----------+------------+------------+-----------+ + +Direct Messaging +---------------- + +The direct messaging capabilities provided by the message router +intermediary type are recommended for the oslo.messaging RPC +messaging pattern. + +.. _Limited: + +Additionally, the message router intermediary may be used for the +Notification messaging pattern with the consideration that +notification messages will be dropped when there is no active +consumer. The message router does not provide durability or +store-and-forward capabilities for notification messages. + +The driver can readily scale operation from working with a single +instances of a message router to working with a large scale routed +mesh interconnect topology. + +Store and Forward +----------------- + +The store and forward capabilities provide by the message broker +intermediary type are recommended for the oslo.message Notification +messaging pattern. This driver is able to work with a single instance +of a message broker or a clustered broker deployment. + +Oslo.messaging provides a mechanism to configure separate backends for +RPC and Notification communications. This document provides deployment and +configuration information for use of this driver in oslo.messaging. + +Addressing +---------- + +A new address syntax was added to the driver to support efficient +direct message routing. This new syntax will also work with a broker +intermediary backend but is not compatible with the address syntax +previously used by the driver. In order to allow backward compatibility, +the driver will attempt to identify the intermediary type for the +backend in use and will automatically select the 'legacy' syntax for +broker-based backends or the new 'routable' syntax for router-based +backends. An `address mode`_ configuration option is provided to +override this dynamic behavior and force the use of either the legacy +or routable address syntax. ============= Prerequisites ============= +Protocol Engine +--------------- + This driver uses the Apache QPID `Proton`_ AMQP 1.0 protocol engine. This engine consists of a platform specific library and a python binding. The driver does not directly interface with the engine API, @@ -37,100 +111,372 @@ is layered on top of the protocol engine. .. _Pyngus: https://github.com/kgiusti/pyngus In order to run the driver the Proton Python bindings, Proton -library, Proton header files, and Pyngus must be installed. +library, Proton header files, and Pyngus must be installed. Pre-built +packages for both Pyngus and the Proton protocol engine are available +for various Linux distributions (see `packages`_ below). It is +recommended to use the pre-built packages if they are available for +your platform. -Pyngus is available via `Pypi`__. +The Proton package includes a C extension that links to the Proton +library. If this library is not installed, then the Proton install +script will attempt to download the necessary Proton C source files from +the Apache repository and build the library locally. -.. __: https://pypi.python.org/pypi/pyngus +In order to build the Proton C source locally, there are a number of +tools and libraries that need to be present: -While the Proton Python bindings are available via `Pypi`__, it -includes a C extension that requires the Proton library and header -files be pre-installed in order for the binding to install properly. -If the target platform's distribution provides a pre-packaged version -of the Proton Python binding (see packages_ below), it is recommended -to use these pre-built packages instead of pulling them from Pypi. +* The tools and library necessary for Python development +* The `SWIG`_ wrapper generator +* The `OpenSSL`_ development libraries and headers +* The `Cyrus SASL`_ development libraries and headers -.. __: https://pypi.python.org/pypi/python-qpid-proton +.. _SWIG: http://www.swig.org/index.php +.. _OpenSSL: https://www.openssl.org +.. _Cyrus SASL: https://cyrusimap.org -The driver also requires a *broker* that supports version 1.0 of the -AMQP protocol. +**Note well**: Currently the Proton Pypi package only supports building +the C extension on Linux systems. -The driver has only been tested using `qpidd`_ in a `patched -devstack`_ environment. The version of qpidd **must** be at least -0.26. qpidd also uses the Proton engine for its AMQP 1.0 support, so +Router Intermediary +------------------- + +This driver supports a *router* intermediary that supports version 1.0 +of the AMQP protocol. The direct messaging capabilities provided by +this intermediary type are recommended for oslo.messaging RPC. + +The driver has been tested with `qpid-dispatch-router`_ router in a +`devstack`_ environment. The version of qpid-dispatch-router +**must** be at least 0.6.0. The qpid-dispatch-router also uses the +Proton engine for its AMQP 1.0 support, so the Proton library must be +installed on the system hosting the qpid-dispatch-router daemon. + +.. _qpid-dispatch-router: http://qpid.apache.org/components/dispatch-router/ + +Pre-built packages for the router are available. See `packages`_ below. + +Broker Intermediary +------------------- + +This driver supports a *broker* intermediary that supports version 1.0 +of the AMQP protocol. The store and forward capabilities provided by +this intermediary type are recommended for oslo.messaging Notifications. + +The driver has been tested with the `qpidd`_ broker in a `devstack`_ +environment. The version of qpidd **must** be at least +0.34. qpidd also uses the Proton engine for its AMQP 1.0 support, so the Proton library must be installed on the system hosting the qpidd daemon. .. _qpidd: http://qpid.apache.org/components/cpp-broker/index.html -.. _patched devstack: https://review.openstack.org/#/c/109118/ -At present, RabbitMQ does not work with this driver. This driver -makes use of the *dynamic* flag on the link Source to automatically -provision a node at the peer. RabbitMQ's AMQP 1.0 implementation has -yet to implement this feature. +Pre-built packages for the broker are available. See `packages`_ below. -See the `specification`_ for additional information regarding testing +See the `oslo specification`_ for additional information regarding testing done on the driver. ============= Configuration ============= -driver ------- +Transport URL Enable +-------------------- -It is recommended to start with the default configuration options -supported by the driver. The remaining configuration steps described -below assume that none of the driver's options have been manually -overridden. +In oslo.messaging, the transport_url parameters define the OpenStack service +backends for RPC and Notify. The url is of the form: - **Note Well:** The driver currently does **not** support the generic - *amqp* options used by the existing drivers, such as - *amqp_durable_queues* or *amqp_auto_delete*. Support for these are - TBD. + transport://user:pass@host1:port[,hostN:portN]/virtual_host + +Where the transport value specifies the rpc or notification backend as +one of **amqp**, rabbit, zmq, etc. + +To specify and enable the AMQP 1.0 driver for RPC, in the section +[DEFAULT] of the service configuration file, specify the +'transport_url' parameter: + +:: + + [DEFAULT] + transport_url = amqp://username:password@routerhostname:5672 + +To specify and enable the AMQP 1.0 driver for Notify, in the section +[NOTIFICATIONS] of the service configuration file, specify the +'transport_url' parameter: + +:: + + [NOTIFICATIONS] + transport_url = amqp://username:password@brokerhostname:5672 + +Note, that if a 'transport_url' parameter is not specified in the +[NOTIFICATIONS] section, the [DEFAULT] transport_url will be used +for both RPC and Notify backends. + +Driver Options +-------------- + +It is recommended that the default configuration options provided by +the AMQP 1.0 driver be used. The configuration options can be modified +in the oslo_messaging_amqp section of the service configuration file. + +Connection Options +^^^^^^^^^^^^^^^^^^ + +In section [oslo_messaging_amqp]: + +#. idle_timeout: Timeout in seconds for inactive connections. + Default is disabled. + +#. connection_retry_interval: Seconds to pause before attempting to + re-connect. + +#. connection_retry_backoff: Connection retry interval increment after + unsuccessful failover attempt. + +#. connection_retry_interval_max: The maximum duration for a + connection retry interval. + +Message Send Options +^^^^^^^^^^^^^^^^^^^^ + +In section [oslo_messaging_amqp]: + +#. link_retry_delay: Time to pause between re-connecting to an AMQP 1.0 link. + +#. default_reply_timeout: The deadline for an rpc reply message delivery. + +#. default_send_timeout: The deadline for an rpc cast or call message delivery. + +#. default_notify_timeout: The deadline for a sent notification + message delivery. + +.. _address mode: + +Addressing Options +^^^^^^^^^^^^^^^^^^ + +In section [oslo_messaging_amqp]: + +#. addressing_mode: Indicates addressing mode used by the driver. + +#. server_request_prefix: Legacy address prefix used when sending to a + specific server. + +#. broadcast_prefix: Legacy broadcast prefix used when broadcasting to + all servers. + +#. group_request_prefix: Legacy address prefix when sending to any + server in a group. + +#. rpc_address_prefix: Routable address prefix for all generated RPC addresses. + +#. notify_address_prefix: Routable address prefix for all generated + Notification addresses. + +#. multicast_address: Appended to address prefix when sending a fanout address. + +#. unicast_address: Appended to address prefix when sending to a + particular RPC/Notification server. + +#. anycast_address: Appended to address prefix when sending to a group + of consumers. + +#. default_notification_exchange: Exchange name used in notification + addresses if not supplied by the application. + +#. default_rpc_exchange: Exchange name used in RPC addresses if not + supplied by the application. + +SSL Options +^^^^^^^^^^^ + +In section [oslo_messaging_amqp]: + +#. ssl_ca_file: A file containing the trusted Certificate Authority's + digital certificate (in PEM format). This certificate is used to + authenticate the messaging backend. + +#. ssl_cert_file: A file containing a digital certificate (in PEM + format) that is used to identify the driver with the messaging bus + (i.e. client authentication). + +#. ssl_key_file:A file containing the private key used to sign the + ssl_cert_file certificate (PEM format, optional if private key is + stored in the certificate itself). + +#. ssl_key_password: The password used to decrypt the private key + (not required if private key is not encrypted). + +SASL Options +^^^^^^^^^^^^ + +In section [oslo_messaging_amqp]: + +#. sasl_config_dir: Path to the *directory* that contains the SASL + configuration. + +#. sasl_config_name: The name of SASL configuration file (without + .conf suffix) in sasl_config_dir + +#. username: SASL user identifier for authentication with the message + bus. Can be overridden by URL. + +#. password: Password for username + +AMQP Generic Options (**Note Well**) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The AMQP 1.0 driver currently does **not** support the generic *amqp* +options used by pre-1.0 drivers such as *amqp_durable_queues* or +*amqp_auto_delete*. + +qpid-dispatch-router +-------------------- + +First, verify that the Proton library has been installed and is +imported by the qpid-dispatch-router intermediary. This can be checked +by running: + +:: + + $ qdrouterd --help + +and looking for references to qpid-dispatch include and config path +options in the help text. If no qpid-dispatch information is listed, +verify that the Proton libraries are installed and that the version of +the qdrouterd is greater than or equal to 0.6.0. + +Second, configure the address patterns used by the driver. This is +done by adding the following to /etc/qpid-dispatch/qdrouterd.conf. + +If the legacy syntax for the addressing mode is required, include the +following: + +:: + + address { + prefix: unicast + distribution: closest + } + + address { + prefix: exclusive + distribution: closest + } + + address { + prefix: broadcast + distribution: multicast + } + +For the routable syntax addressing mode, include the following: + +:: + + address { + prefix: openstack.org/om/rpc/multicast + distribution: multicast + } + + address { + prefix: openstack.org/om/rpc/unicast + distribution: closest + } + + address { + prefix: openstack.org/om/rpc/anycast + distribution: balanced + } + + address { + prefix: openstack.org/om/notify/multicast + distribution: multicast + } + + address { + prefix: openstack.org/om/notify/unicast + distribution: closest + } + + address { + prefix: openstack.org/om/notify/anycast + distribution: balanced + } + + +**Note well**: For any customization of the `address mode`_ and syntax used, +it is required that the address entity configurations in the +/etc/qpid-dispatch/qdrouterd.conf be updated. qpidd ----- First, verify that the Proton library has been installed and is -imported by the qpidd broker. This can checked by running:: +imported by the qpidd broker. This can checked by running: + +:: $ qpidd --help and looking for the AMQP 1.0 options in the help text. If no AMQP 1.0 options are listed, verify that the Proton libraries are installed and -that the version of qpidd is greater than or equal to 0.26. +that the version of qpidd is greater than or equal to 0.34. -Second, configure the address patterns used by the driver. This is -done by adding the following to /etc/qpid/qpidd.conf:: +Second, configure the default address patterns used by the +driver for a broker-based backend. This is done by adding the +following to /etc/qpid/qpidd.conf: + +:: queue-patterns=exclusive queue-patterns=unicast topic-patterns=broadcast These patterns, *exclusive*, *unicast*, and *broadcast* are the -default values used by the driver. These can be overridden via the -driver configuration options if desired. If manually overridden, +legacy addressing values used by the driver. These can be overridden via the +driver configuration options if desired (see above). If manually overridden, update the qpidd.conf values to match. -services --------- +.. _devstack: -The new driver is selected by specifying **amqp** as the transport -name. For example:: +================ +DevStack Support +================ - from oslo import messaging - from oslo.config import cfg +The plugin for the AMQP 1.0 olso.messaging driver is supported by +DevStack. The plugin supports the use of either the broker or router +intermediary. - amqp_transport = messaging.get_transport(cfg.CONF, - "amqp://me:passwd@host:5672") +In local.conf [localrc] section, the `devstack-plugin-amqp1`_ plugin +repository must be enabled. For example: +:: -The new driver can be loaded and used by existing applications by -specifying *amqp* as the RPC backend in the service's configuration -file. For example, in nova.conf:: + [[local|localrc]] + enable_plugin amqp1 https://git.openstack.org/openstack/devstack-plugin-amqp1 - rpc_backend = amqp +Set the username and password variables if needed for the +configuration: + +:: + + AMQP1_USERNAME=queueuser + AMQP1_PASSWORD=queuepassword + +To configure DevStack for operation of the router intermediary, set +the AMQP1 service variable to "qdr": + +:: + + AMQP1_SERVICE=qdr + +Alternatively, to configure DevStack for operation of the broker +intermediary, set the AMQP1 service variable to "qpid": + +:: + + AMQP1_SERVICE=qpid + +.. _devstack-plugin-amqp1: https://github.com/openstack/devstack-plugin-amqp1.git .. _packages: @@ -138,20 +484,37 @@ file. For example, in nova.conf:: Platforms and Packages ====================== -Pyngus is available via Pypi. +PyPi +---- + +Packages for `Pyngus pypi`_ and the `Proton pypi`_ engine are available on Pypi. + +.. _Pyngus pypi: https://pypi.python.org/pypi/pyngus +.. _Proton pypi: https://pypi.python.org/pypi/python-qpid-proton -Pre-built packages for the Proton library and qpidd are available for -some popular distributions: RHEL and Fedora --------------- -Packages exist in EPEL for RHEL/Centos 7, and Fedora 19+. +Packages exist in EPEL for RHEL/Centos 7, and Fedora 24+. Unfortunately, RHEL/Centos 6 base packages include a very old version of qpidd that does not support AMQP 1.0. EPEL's policy does not allow a newer version of qpidd for RHEL/Centos 6. The following packages must be installed on the system running the +intermediary daemon: + ++--------------+--------------------------+ +| Intermediary | Package | ++--------------+--------------------------+ +| qdrouterd | qpid-dispatch-router | +| | python-qpid-proton | ++--------------+--------------------------+ +| qpidd | qpid-cpp-server | +| | qpid-proton-c | ++--------------+--------------------------+ + + qpidd daemon: - qpid-cpp-server (version 0.26+) @@ -173,17 +536,23 @@ yet available in the Ubuntu repository. The version of qpidd on both platforms is too old and does not support AMQP 1.0. Until the proper package version arrive the latest packages can be -pulled from the `Apache Qpid PPA`_ on Launchpad:: +pulled from the `Apache Qpid PPA`_ on Launchpad: + +:: sudo add-apt-repository ppa:qpid/released .. _Apache Qpid PPA: https://launchpad.net/~qpid/+archive/ubuntu/released +The following packages must be installed on the system running the +qdrouterd daemon: + +- qdrouterd (version 0.6.0+) + The following packages must be installed on the system running the qpidd daemon: -- qpidd (version 0.26+) -- libqpid-proton2 +- qpidd (version 0.34+) The following packages must be installed on the systems running the services that use the new driver: