Update to use oslo.messaging service for RPC and Notify

This introduces oslo.messaging variables that define the RPC and
Notify transports for the OpenStack services. These parameters replace
the rabbitmq values and are used to generate the messaging
transport_url for the service. The association of the messaging
backend server to the oslo.messaging services will then be transparent
to the trove service.

This patch:
* Add oslo.messaging variables for RPC and Notify to defaults
* Update transport_url generation (add for notification)
* Add oslo.messaging to tests inventory
* Update tests
* Update docs
* Update extras
* Add release note

Change-Id: Ia01317343ae6fbc790d64b5ba282c8c069750d45
This commit is contained in:
Andrew Smith 2018-06-12 12:00:26 -04:00 committed by Andy Smith
parent 8ebd2d73bb
commit 3c4f4127b4
10 changed files with 102 additions and 39 deletions

View File

@ -119,14 +119,26 @@ trove_galera_use_ssl: "{{ galera_use_ssl | default(False) }}"
trove_galera_ssl_ca_cert: "{{ galera_ssl_ca_cert | default('/etc/ssl/certs/galera-ca.pem') }}"
trove_galera_connection_string: "mysql+pymysql://{{ trove_galera_user}}:{{ trove_galera_password }}@{{ trove_galera_address }}/{{ trove_galera_database_name }}?charset=utf8{% if trove_galera_use_ssl | bool %}&ssl_ca={{ trove_galera_ssl_ca_cert }}{% endif %}"
## Oslo Messaging vars
# RPC
trove_oslomsg_rpc_transport: rabbit
trove_oslomsg_rpc_servers: "{{ rabbitmq_servers }}"
trove_oslomsg_rpc_port: 5672
trove_oslomsg_rpc_use_ssl: False
trove_oslomsg_rpc_userid: trove
trove_oslomsg_rpc_vhost: /trove
# Notify
trove_oslomsg_notify_transport: rabbit
trove_oslomsg_notify_servers: "{{ rabbitmq_servers }}"
trove_oslomsg_notify_port: 5672
trove_oslomsg_notify_use_ssl: False
trove_oslomsg_notify_userid: trove
trove_oslomsg_notify_vhost: /trove
# Rabbit vars
trove_control_exchange: trove
trove_rabbit_notification_topic: notification
trove_rabbitmq_userid: trove
trove_rabbitmq_vhost: /trove
trove_rabbitmq_use_ssl: False
trove_rabbitmq_port: 5672
trove_rabbitmq_servers: "{{ rabbitmq_servers }}"
# The trove guest agent in the deployed DB VMs need access to OpenStack services (keystone, swift, etc)
# and also to rabbitmq. The way it gets access (networking) these services may differ.
@ -141,8 +153,10 @@ trove_provider_network: "{{ provider_networks|map(attribute='network')|selectatt
trove_provider_ip_from_q: "{{ trove_provider_network['ip_from_q'] }}"
# The name of the network address pool
trove_container_net_name: "{{ trove_provider_ip_from_q }}_address"
trove_guest_rabbitmq_servers: "{% for host in groups[rabbitmq_host_group] %}{{ hostvars[host]['container_networks'][trove_container_net_name]['address'] }}{% if not loop.last %},{% endif %}{% endfor %}"
trove_guest_rabbitmq_use_ssl: "{{ trove_rabbitmq_use_ssl }}"
trove_guest_oslomsg_rpc_servers: "{% for host in groups[oslomsg_rpc_host_group] %}{{ hostvars[host]['container_networks'][trove_container_net_name]['address'] }}{% if not loop.last %},{% endif %}{% endfor %}"
trove_guest_oslomsg_rpc_use_ssl: "{{ trove_oslomsg_rpc_use_ssl }}"
trove_guest_oslomsg_notify_servers: "{% for host in groups[oslomsg_notify_host_group] %}{{ hostvars[host]['container_networks'][trove_container_net_name]['address'] }}{% if not loop.last %},{% endif %}{% endfor %}"
trove_guest_oslomsg_notify_use_ssl: "{{ trove_oslomsg_notify_use_ssl }}"
# For OpenStack services that have public, admin, and internal access, use the public ones for the guest VMs.
trove_guest_auth_url: "{{ keystone_service_publicurl }}"
trove_guest_swift_url: "{{ trove_service_publicuri_proto }}://{{ external_lb_vip_address }}:{{ swift_proxy_port }}/v1/AUTH_"
@ -168,7 +182,8 @@ trove_service_net_endpoint_type: internal
trove_enable_secure_rpc_messaging: "True"
trove_required_secrets:
- trove_galera_password
- trove_rabbitmq_password
- trove_oslomsg_rpc_password
- trove_oslomsg_notify_password
- trove_service_password
- trove_admin_user_password
- trove_taskmanager_rpc_encr_key

View File

@ -10,15 +10,15 @@ Configuring Trove
Trove provides DBaaS to an OpenStack deployment. It deploys guest VMs that
provide the desired DB for use by the end consumer. The trove guest VMs need
connectivity back to the trove services via RPC (rabbitmq) and the OpenStack
services. The way these guest VM get access to those services could be via
internal networking (in the case of rabbitmq) or via public interfaces (in the
case of OpenStack services). For the example configuration, we'll designate
a provider network as the network for trove to provision on each guest VM. The
guest can then connect to rabbitmq via this network and to the OpenStack
services externally. Optionally, the guest VMs could use the internal network
to access OpenStack services, but that would require more containers being
bound to this network.
connectivity back to the trove services via RPC (oslo.messaging) and the
OpenStack services. The way these guest VM get access to those services could be
via internal networking (in the case of oslo.messaging) or via public interfaces
(in the case of OpenStack services). For the example configuration, we'll
designate a provider network as the network for trove to provision on each guest
VM. The guest can then connect to oslo.messaging via this network and to the
OpenStack services externally. Optionally, the guest VMs could use the internal
network to access OpenStack services, but that would require more containers
being bound to this network.
The deployment configuration outlined below may not be appropriate for
production environments. Review this very carefully with your own security
@ -45,14 +45,14 @@ An example entry into ``openstack_user_config.yml`` is shown below:
net_name: "dbaas-mgmt"
group_binds:
- neutron_linuxbridge_agent
- rabbitmq
- oslomsg_rpc
Make sure to modify the other entries in this file as well.
The ``net_name`` will be the physical network that is specified when creating
the neutron network. The default value of ``dbaas-mgmt`` is also used to
lookup the addresses of the rabbitmq container. If the default is not used then
some variables in ``defaults\main.yml`` will need to be overwritten.
lookup the addresses of the rpc messaging container. If the default is not used
then some variables in ``defaults\main.yml`` will need to be overwritten.
By default this role will not create the neutron network automaticaly. However,
the default values can be changed to create the neutron network. See the

View File

@ -0,0 +1,20 @@
---
features:
- Support separate oslo.messaging services for RPC and Notifications
to enable operation of separate and different messaging backend servers.
deprecations:
- |
The rabbitmq server parameters have been replaced by corresponding
oslo.messaging RPC and Notify parameters in order to abstract the
messaging service from the actual backend server deployment.
- trove_oslomsg_rpc_servers replaces trove_rabbitmq_servers
- trove_oslomsg_rpc_port replaces trove_rabbitmq_port
- trove_oslomsg_rpc_use_ssl replaces trove_rabbitmq_use_ssl
- trove_oslomsg_rpc_userid replaces trove_rabbitmq_userid
- trove_oslomsg_rpc_vhost replaces trove_rabbitmq_vhost
- added trove_oslomsg_notify_servers
- added trove_oslomsg_notify_port
- added trove_oslomsg_notify_use_ssl
- added trove_oslomsg_notify_userid
- added trove_oslomsg_notify_vhost
- added trove_oslomsg_notify_password

View File

@ -5,7 +5,7 @@ debug = {{ debug }}
trove_auth_url = {{ trove_auth_url }}
conductor_manager = trove.conductor.manager.Manager
trove_conductor_workers={{ trove_conductor_workers }}
transport_url = rabbit://{% for host in trove_rabbitmq_servers.split(',') %}{{ trove_rabbitmq_userid }}:{{ trove_rabbitmq_password }}@{{ host }}:{{ trove_rabbitmq_port }}{% if not loop.last %},{% else %}/{{ trove_rabbitmq_vhost }}{% endif %}{% endfor %}
transport_url = {{ trove_oslomsg_rpc_transport }}://{% for host in trove_oslomsg_rpc_servers.split(',') %}{{ trove_oslomsg_rpc_userid }}:{{ trove_oslomsg_rpc_password }}@{{ host }}:{{ trove_oslomsg_rpc_port }}{% if not loop.last %},{% else %}/{{ trove_oslomsg_rpc_vhost }}{% endif %}{% endfor %}
{# There must be a blank line above or the following line will be appended to the previous. #}
control_exchange = {{ trove_control_exchange }}
@ -21,4 +21,7 @@ enabled = {{ trove_profiler_enabled }}
connection = "{{ trove_galera_connection_string }}"
[oslo_messaging_rabbit]
ssl={{ trove_rabbitmq_use_ssl }}
ssl={{ trove_oslomsg_rpc_use_ssl }}
[oslo_messaging_notifications]
transport_url = {{ trove_oslomsg_notify_transport }}://{% for host in trove_oslomsg_notify_servers.split(',') %}{{ trove_oslomsg_notify_userid }}:{{ trove_oslomsg_notify_password }}@{{ host }}:{{ trove_oslomsg_notify_port }}{% if not loop.last %},{% else %}/{{ trove_oslomsg_notify_vhost }}{% endif %}{% endfor %}

View File

@ -6,7 +6,7 @@ nova_proxy_admin_user = {{ trove_service_user_name }}
nova_proxy_admin_pass = {{ trove_service_password }}
nova_proxy_admin_tenant_name = {{ trove_service_project_name }}
transport_url = rabbit://{% for host in trove_rabbitmq_servers.split(',') %}{{ trove_rabbitmq_userid }}:{{ trove_rabbitmq_password }}@{{ host }}:{{ trove_rabbitmq_port }}{% if not loop.last %},{% else %}/{{ trove_rabbitmq_vhost }}{% endif %}{% endfor %}
transport_url = {{ trove_oslomsg_rpc_transport }}://{% for host in trove_oslomsg_rpc_servers.split(',') %}{{ trove_oslomsg_rpc_userid }}:{{ trove_oslomsg_rpc_password }}@{{ host }}:{{ trove_oslomsg_rpc_port }}{% if not loop.last %},{% else %}/{{ trove_oslomsg_rpc_vhost }}{% endif %}{% endfor %}
{% if trove_swift_enabled is defined %}
swift_url = {{ trove_guest_swift_url }}
@ -56,7 +56,10 @@ log_file = trove-guestagent.log
[oslo_messaging_rabbit]
# Connect over SSL for RabbitMQ. (boolean value)
# Deprecated group/name - [DEFAULT]/rabbit_use_ssl
ssl = {{ trove_guest_rabbitmq_use_ssl }}
ssl = {{ trove_guest_oslomsg_rpc_use_ssl }}
[oslo_messaging_notifications]
transport_url = {{ trove_oslomsg_notify_transport }}://{% for host in trove_oslomsg_notify_servers.split(',') %}{{ trove_oslomsg_notify_userid }}:{{ trove_oslomsg_notify_password }}@{{ host }}:{{ trove_oslomsg_notify_port }}{% if not loop.last %},{% else %}/{{ trove_oslomsg_notify_vhost }}{% endif %}{% endfor %}
# ========== Datastore Specific Configuration Options ==========

View File

@ -3,7 +3,7 @@
[DEFAULT]
debug = {{ debug }}
update_status_on_fail = True
transport_url = rabbit://{% for host in trove_rabbitmq_servers.split(',') %}{{ trove_rabbitmq_userid }}:{{ trove_rabbitmq_password }}@{{ host }}:{{ trove_rabbitmq_port }}{% if not loop.last %},{% else %}/{{ trove_rabbitmq_vhost }}{% endif %}{% endfor %}
transport_url = {{ trove_oslomsg_rpc_transport }}://{% for host in trove_oslomsg_rpc_servers.split(',') %}{{ trove_oslomsg_rpc_userid }}:{{ trove_oslomsg_rpc_password }}@{{ host }}:{{ trove_oslomsg_rpc_port }}{% if not loop.last %},{% else %}/{{ trove_oslomsg_rpc_vhost }}{% endif %}{% endfor %}
{# There must be a blank line above or the following line will be appended to the previous. #}
control_exchange = {{ trove_control_exchange }}
@ -100,6 +100,9 @@ enabled = {{ trove_profiler_enabled }}
#trace_sqlalchemy = True
[oslo_messaging_rabbit]
ssl={{ trove_rabbitmq_use_ssl }}
ssl={{ trove_oslomsg_rpc_use_ssl }}
[oslo_messaging_notifications]
transport_url = {{ trove_oslomsg_notify_transport }}://{% for host in trove_oslomsg_notify_servers.split(',') %}{{ trove_oslomsg_notify_userid }}:{{ trove_oslomsg_notify_password }}@{{ host }}:{{ trove_oslomsg_notify_port }}{% if not loop.last %},{% else %}/{{ trove_oslomsg_notify_vhost }}{% endif %}{% endfor %}
{% include 'include_db.j2' %}

View File

@ -4,7 +4,7 @@ debug = {{ debug }}
bind_host = {{ trove_service_host }}
bind_port = {{ trove_service_port }}
trove_api_workers={{ trove_api_workers }}
transport_url = rabbit://{% for host in trove_rabbitmq_servers.split(',') %}{{ trove_rabbitmq_userid }}:{{ trove_rabbitmq_password }}@{{ host }}:{{ trove_rabbitmq_port }}{% if not loop.last %},{% else %}/{{ trove_rabbitmq_vhost }}{% endif %}{% endfor %}
transport_url = {{ trove_oslomsg_rpc_transport }}://{% for host in trove_oslomsg_rpc_servers.split(',') %}{{ trove_oslomsg_rpc_userid }}:{{ trove_oslomsg_rpc_password }}@{{ host }}:{{ trove_oslomsg_rpc_port }}{% if not loop.last %},{% else %}/{{ trove_oslomsg_rpc_vhost }}{% endif %}{% endfor %}
{# There must be a blank line above or the following line will be appended to the previous. #}
control_exchange = {{ trove_control_exchange }}
@ -116,6 +116,9 @@ enabled = {{ trove_profiler_enabled }}
#ca_file = /path/to/ca_file
[oslo_messaging_rabbit]
ssl = {{ trove_rabbitmq_use_ssl }}
ssl = {{ trove_oslomsg_rpc_use_ssl }}
[oslo_messaging_notifications]
transport_url = {{ trove_oslomsg_notify_transport }}://{% for host in trove_oslomsg_notify_servers.split(',') %}{{ trove_oslomsg_notify_userid }}:{{ trove_oslomsg_notify_password }}@{{ host }}:{{ trove_oslomsg_notify_port }}{% if not loop.last %},{% else %}/{{ trove_oslomsg_notify_vhost }}{% endif %}{% endfor %}
{% include 'include_db.j2' %}

View File

@ -7,6 +7,12 @@ openstack1
infra1
openstack1
[oslomsg_rpc_all]
infra1
[oslomsg_notify_all]
infra1
[rabbitmq_all]
infra1

View File

@ -18,13 +18,20 @@
trove_developer_mode: True
trove_galera_password: "secrete"
trove_rabbitmq_port: "{{ rabbitmq_port }}"
trove_rabbitmq_servers: "{{ rabbitmq_servers }}"
trove_rabbitmq_use_ssl: "{{ rabbitmq_use_ssl }}"
trove_rabbitmq_password: "secrete"
trove_rabbitmq_userid: trove
trove_rabbitmq_vhost: /trove
trove_guest_rabbitmq_servers: "{{ trove_rabbitmq_servers }}"
trove_oslomsg_rpc_port: "{{ oslomsg_rpc_port }}"
trove_oslomsg_rpc_servers: "{ oslomsg_rpc_servers }}"
trove_oslomsg_rpc_use_ssl: "{{ oslomsg_rpc_use_ssl }}"
trove_oslomsg_rpc_password: "secrete"
trove_oslomsg_rpc_userid: trove
trove_oslomsg_rpc_vhost: /trove
trove_guest_oslomsg_rpc_servers: "{{ trove_oslomsg_rpc_servers }}"
trove_oslomsg_notify_port: "{{ oslomsg_notify_port }}"
trove_oslomsg_notify_servers: "{ oslomsg_notify_servers }}"
trove_oslomsg_notify_use_ssl: "{{ oslomsg_notify_use_ssl }}"
trove_oslomsg_notify_password: "secrete"
trove_oslomsg_notify_userid: trove
trove_oslomsg_notify_vhost: /trove
trove_guest_oslomsg_notify_servers: "{{ trove_oslomsg_notify_servers }}"
trove_guest_auth_url: "{{ trove_auth_url }}"
trove_requirements_git_install_branch: master
trove_service_password: "secrete"

View File

@ -21,10 +21,13 @@
user: root
gather_facts: true
pre_tasks:
- include: common/ensure-rabbitmq.yml
vhost_name: "{{ trove_rabbitmq_vhost }}"
user_name: "{{ trove_rabbitmq_userid }}"
user_password: "{{ trove_rabbitmq_password }}"
- include: common/ensure-oslomsg.yml
rpc_vhost: "{{ trove_oslomsg_rpc_vhost }}"
rpc_user: "{{ trove_oslomsg_rpc_userid }}"
rpc_password: "{{ trove_oslomsg_rpc_password }}"
notify_vhost: "{{ trove_oslomsg_notify_vhost }}"
notify_user: "{{ trove_oslomsg_notify_userid }}"
notify_password: "{{ trove_oslomsg_notify_password }}"
roles:
- role: "os_trove"
vars_files: