Remove notification agent support from the plugin
Change-Id: I3f3332efb59d27f08368c5d0571cdbd9750b96b5
This commit is contained in:
parent
4fec0439df
commit
f7a306221a
|
@ -140,7 +140,6 @@ class redis::main (
|
|||
'coordination/backend_url' : value => redis_backend_url($redis_hosts, $redis_sentinel_port, $timeout, $master_name);
|
||||
'coordination/heartbeat' : value => '1.0';
|
||||
'coordination/check_watchers' : value => $timeout;
|
||||
'notification/workload_partitioning': value => true
|
||||
}
|
||||
|
||||
service { 'ceilometer-agent-central':
|
||||
|
|
|
@ -4,14 +4,13 @@ Ceilometer Redis plugin
|
|||
Ceilometer Redis Plugin aims to install Redis to MOS environment and provide a coordination mechanism for
|
||||
`Ceilometer agents <https://ceilometer.readthedocs.org/en/latest/architecture.html>`_ and Alarm Evaluator
|
||||
through the `tooz library <http://docs.openstack.org/developer/tooz/>`_ with a `Redis backend <http://redis.io>`_
|
||||
The plugin supports coordination for the following Ceilometer services: central agent, notification agent
|
||||
and alarm-evaluator. Each of these services are running on every controller after the plugin
|
||||
is installed. All of them are joined into the corresponding coordination group (one coordination group
|
||||
per each service). It differs from the default configuration when there should be only one central agent
|
||||
and alarm-evaluator per cloud. The plugin also configures redis-server under pacemaker to monitor its process.
|
||||
The plugin configures `redis-sentinel <http://redis.io/topics/sentinel>`_ to monitor the state of the redis
|
||||
cluster, to elect new master during failovers, to forward ceilometer services to new elected redis master,
|
||||
to organize sync between redis nodes.
|
||||
The plugin supports coordination for the following Ceilometer services: central agent and alarm-evaluator.
|
||||
Each of these services are running on every controller after the plugin is installed. All of them are joined
|
||||
into the corresponding coordination group (one coordination group per each service). It differs from the default
|
||||
configuration when there should be only one central agent and alarm-evaluator per cloud. The plugin also configures
|
||||
redis-server under pacemaker to monitor its process. The plugin configures `redis-sentinel <http://redis.io/topics/sentinel>`_
|
||||
to monitor the state of the redis cluster, to elect new master during failovers, to forward ceilometer services to new
|
||||
elected redis master, to organize sync between redis nodes.
|
||||
|
||||
|
||||
Central agent
|
||||
|
@ -32,23 +31,6 @@ If there are several alarm evaluators and no coordination enabled, all of them w
|
|||
every configurable time interval. The alarm sets for evaluators should be disjoint. So, coordination is responsible
|
||||
for providing the set of alarms to evaluate to each alarm-evaluator in the cloud.
|
||||
|
||||
Notification agent
|
||||
------------------
|
||||
Before Liberty, there was no need to coordinate Ceilometer notification agents. But starting from Liberty, samples
|
||||
transformations started to be handled not by compute/central agents as it was before, but by a notification agent.
|
||||
Some of Ceilometer transformers have a local cache where it is stored the data from previously processed samples.
|
||||
For example, "cpu_util" metric are obtained from two consecutive Samples with "cpu" metric: one is subtracted from
|
||||
another and divided to an amount of cpu (this information is stored in Sample's metadata).
|
||||
Thus, it should be guaranteed that all the Samples which should be transformed by one transformer, will go to the
|
||||
same notification agent. If some of the samples go to another, the cache cannot be shared and some data will be lost.
|
||||
|
||||
To handle this process properly, IPC queues was introduced - inter process communication queues in message bus
|
||||
(RabbitMQ). With coordination enabled, each notification agent has two set of listeners: for main queues and for IPC
|
||||
queues. All notification agents listen to _all_ the main queues (where we have all messages from OpenStack services
|
||||
and polling-based messages from central/compute agents) and re-publish messages to _all_ IPC queues. Coordination
|
||||
starts to work at this point: every notification agent in the cloud has it's own set of IPC queues to listen to. Thus,
|
||||
we can be sure that local cache on each notification agent contains all the previous data required for transformation.
|
||||
|
||||
|
||||
Requirements
|
||||
------------
|
||||
|
@ -69,8 +51,31 @@ Limitations
|
|||
This requirement is mandatory because Redis needs an odd number of nodes to
|
||||
choose the master successfully.
|
||||
|
||||
* In MOS 8.0, there are no transformers configured by default. The plugin doesn't add any of them into
|
||||
ceilometer's pipeline.yaml. Thus, you need to configure it manually if you want to use transformers.
|
||||
If you don't need this feature, it is recommended to disable coordination for the notification agents.
|
||||
* Before Liberty, there was no need to coordinate Ceilometer notification agents. Starting from Liberty, samples
|
||||
transformations started to be handled not by compute/central agents as it was before, but by a notification agent.
|
||||
Some of Ceilometer transformers have a local cache where they store the data from the previously processed samples.
|
||||
For example, "cpu_util" metric are obtained from two consecutive Samples with "cpu" metric: one is subtracted from
|
||||
another and divided by an amount of cpu (this information is stored in Sample's metadata).
|
||||
Thus, it should be guaranteed that all the Samples which should be transformed by one transformer, will go to the
|
||||
same notification agent. If some of the samples go to another, the cache cannot be shared and some data will be lost.
|
||||
|
||||
To handle this process properly, IPC queues was introduced - inter process communication queues in message bus
|
||||
(RabbitMQ). With coordination enabled, each notification agent has two set of listeners: for main queues and for IPC
|
||||
queues. All notification agents listen to _all_ the main queues (where we have all messages from OpenStack services
|
||||
and polling-based messages from central/compute agents) and re-publish messages to _all_ IPC queues. Coordination
|
||||
starts to work at this point: every notification agent in the cloud has it's own set of IPC queues to listen to. Thus,
|
||||
we can be sure that local cache on each notification agent contains all the previous data required for transformation.
|
||||
|
||||
After some investigations, some performance issues were found with IPC approach. That's why in In MOS 8.0 all basic
|
||||
transformations (cpu_util, disk.read.requests.rate, disk.write.requests.rate, disk.read.bytes.rate, disk.write.bytes.rate,
|
||||
disk.device.read.requests.rate, disk.device.read.bytes.rate, disk.device.write.bytes.rate, network.incoming.bytes.rate,
|
||||
network.outgoing.bytes.rate, network.incoming.packets.rate, network.outgoing.packets.rate) were moved back to compute
|
||||
nodes, i.e. for the basic set of transformations there is no need to run notification agent in coordination mode.
|
||||
That's the reason why the plugin does't support coordination for notification agents, although it is possible to configure
|
||||
notification agents to run in coordination mode manually. Anyway, it is not recommended.
|
||||
|
||||
If you have any custom transformers, you need to be sure that they are cache-less, i.e. are based only on
|
||||
``unit_conversion`` transformer or the ``arithmetic`` transformer. If it's not the case, you may consider the following
|
||||
options: run only one notification agent in the cloud or install this plugin and do all configuration manually.
|
||||
|
||||
|
||||
|
|
|
@ -2,7 +2,7 @@ User Guide
|
|||
==========
|
||||
|
||||
Once the Ceilometer Redis plugin plugin has been installed (following :ref:`Installation Guide`), you can
|
||||
create *OpenStack* environments with Ceilometer whose Central agents, Notification agent and Alarm evaluator
|
||||
create *OpenStack* environments with Ceilometer whose Central agents and Alarm evaluator
|
||||
work in workload_partitioned mode.
|
||||
|
||||
Ceilometer installation
|
||||
|
@ -110,15 +110,6 @@ How to check that plugin works
|
|||
.... 2015-11-05T10:26:26 |
|
||||
.... 2015-11-05T10:26:17 |
|
||||
|
||||
#. For the notification agent: Check that IPC queues are created and have consumers:
|
||||
ubuntu@ubuntu:/opt/stack/ceilometer$ sudo rabbitmqctl list_queues name messages consumers | grep ceilo
|
||||
ceilometer-pipe-meter_source:meter_sink-0.sample 0 1
|
||||
ceilometer-pipe-meter_source:meter_sink-1.sample 0 1
|
||||
ceilometer-pipe-meter_source:meter_sink-2.sample 0 1
|
||||
ceilometer-pipe-meter_source:meter_sink-3.sample 0 1
|
||||
ceilometer-pipe-meter_source:meter_sink-4.sample 0 1
|
||||
|
||||
By default, you should see 10 queues in this list. Every queue should have one and only one consumer.
|
||||
|
||||
#. For the alarm evaluator, it is possible to see that everything works as expected only from the logs. Grep the
|
||||
line "extract_my_subset". There should be different "My subset: [" results on each alarm evaluator instance.
|
||||
|
|
Loading…
Reference in New Issue