summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorAshwin Agate <ashwin.agate@suse.com>2018-07-18 17:54:40 -0700
committerJoseph Davis <joseph.davis@suse.com>2018-12-14 08:15:49 -0800
commitf01ecbb9ad6e9897864c2be84e40849efd63f239 (patch)
tree9f5f174ed5f65989cec0e2620260a152a8eac258
parent819eeb8fa53d9d1c8da7d04b76bba5036892562b (diff)
Monasca Events Publishing Spec
Monasca Events API was developed to store events data in Elasticsearch. There is still a need to collect and publish Openstack Notifications as Monasca Events. This commit includes a specification for creating a new Monasca Events Listener service that would be independent of Ceilometer. This is being proposed for the Stein cycle. Monasca Ceilometer project currently publishes ceilometer samples to Monasca API. An alternate specification proposes to extend Monasca Ceilometer project and add a new events publisher which would publish Openstack notifications (or events) to Monasca Events API. This solution was proposed for Rocky but will not be implemented as the Ceilometer Events feature has been deprecated on master. Co-Authored-By: Joseph Davis <joseph.davis@suse.com> Co-Authored-By: Ashwin Agate <ashwin.agate@suse.com> Story: 2003023 Task: 23047 Change-Id: I8ed7a19c0565a0cd613866934eb2540ae46c847d
Notes
Notes (review): Code-Review+2: Witold Bedyk <witold.bedyk@est.fujitsu.com> Code-Review+1: Joseph Davis <joseph.davis@suse.com> Code-Review+2: Doug Szumski <doug@stackhpc.com> Workflow+1: Witold Bedyk <witold.bedyk@est.fujitsu.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Wed, 19 Dec 2018 15:46:32 +0000 Reviewed-on: https://review.openstack.org/583803 Project: openstack/monasca-specs Branch: refs/heads/master
-rw-r--r--README.rst9
-rw-r--r--doc/source/specs/rocky/index.rst8
l---------doc/source/specs/rocky/not-implemented1
-rw-r--r--specs/rocky/not-implemented/monasca-events-publishing.rst705
-rw-r--r--specs/stein/approved/monasca-events-listener.rst586
5 files changed, 1306 insertions, 3 deletions
diff --git a/README.rst b/README.rst
index ee3f8d2..11bfa73 100644
--- a/README.rst
+++ b/README.rst
@@ -25,11 +25,14 @@ The layout of this repository is::
25 priorities/<release>/ 25 priorities/<release>/
26 specs/<release>/ 26 specs/<release>/
27 27
28Where there are two sub-directories in ``specs``: 28Where there are three sub-directories in ``specs``:
29 29
30specs/<release>/approved 30specs/<release>/approved
31 specifications approved, but not yet implemented 31 Specifications approved, but not yet implemented
32 32
33specs/<release>/implemented 33specs/<release>/implemented
34 implemented specifications 34 Implemented specifications
35 35
36specs/<release>/not-implemented
37 Specifications that were approved but are not expected to be implemented.
38 These are kept for historical reference.
diff --git a/doc/source/specs/rocky/index.rst b/doc/source/specs/rocky/index.rst
index ccb3149..3c27fee 100644
--- a/doc/source/specs/rocky/index.rst
+++ b/doc/source/specs/rocky/index.rst
@@ -24,3 +24,11 @@ Rocky approved (but not implemented) specs:
24 :maxdepth: 1 24 :maxdepth: 1
25 25
26 approved/* 26 approved/*
27
28Rocky deprecated specs:
29
30.. toctree::
31 :glob:
32 :maxdepth: 1
33
34 not-implemented/*
diff --git a/doc/source/specs/rocky/not-implemented b/doc/source/specs/rocky/not-implemented
new file mode 120000
index 0000000..93d2bbc
--- /dev/null
+++ b/doc/source/specs/rocky/not-implemented
@@ -0,0 +1 @@
../../../../specs/rocky/not-implemented \ No newline at end of file
diff --git a/specs/rocky/not-implemented/monasca-events-publishing.rst b/specs/rocky/not-implemented/monasca-events-publishing.rst
new file mode 100644
index 0000000..db67291
--- /dev/null
+++ b/specs/rocky/not-implemented/monasca-events-publishing.rst
@@ -0,0 +1,705 @@
1..
2 This work is licensed under a Creative Commons Attribution 3.0 Unported
3 License.
4
5 http://creativecommons.org/licenses/by/3.0/legalcode
6
7================================================
8Monasca Events Publishing - from Ceilometer
9================================================
10
11https://storyboard.openstack.org/#!/story/2003023
12
13Monasca Events API [1] was developed to store Openstack Notification data in Elasticsearch. There is
14still a need to collect and publish Openstack Notifications to Monasca Events API. Monasca
15Ceilometer project[2] currently publishes ceilometer samples[3] to Monasca API. We are proposing to
16extend Monasca Ceilometer project and add a new events publisher which will publish Openstack
17notifications (or events)[3] to Monasca Events API.
18
19UPDATE: This spec is being superceded by the ../../stein/approved/monasca-events-listener.rst spec,
20but is kept here for reference.
21
22
23Problem description
24===================
25
26All Openstack services generate a lot of notifications or events which contain large amounts of
27operational and state information about the service and its resources. This notification data is not
28currently available in Monasca.
29
30Ceilometer data processing pipeline[3] provides an extensible mechanism of publishing samples and
31events using a custom publisher. Ceilometer samples represent a quantity that can be measured (for
32e.g. the size of a volume) and events represent an occurrence of an event and do not have any
33associated quantity (e.g. volume was created).
34
35Monasca Ceilometer project currently provides a samples publisher. Monasca Ceilometer samples
36publisher converts Ceilometer samples to Monasca Metrics format which are then published to Monasca
37API. There is no corresponding events publisher to Monasca yet.
38
39By adding an event publisher to Monasca Ceilometer project we could take advantage of Ceilometer's
40event publishing mechanism to publish events to Monasca Events API.
41
42Ceilometer consists of different data collection components - namely Polling Agent, Notification
43Agent and Compute Agent. (Please see [7] for System Architecture diagram) Ceilometer also has a data
44storage and retrieval component, which would be Monasca in our case.
45
46Samples publisher and new proposed events publisher run within Ceilometer's Notification Agent
47component and are part of Notifcation Agent's data processing pipeline. Monasca
48Ceilometer presumes the need to install and deploy Ceilometer Notification Agent component (doesn't
49need Polling Agent or Compute Agent deployed) on all the control nodes. Ceilometer Notification
50Agent is Highly Available (HA) and can run on multiple nodes. We will have to evaluate its
51performance in terms of scaling for events, but we haven't run into performance/scale problems
52with current samples publisher.
53
54
55Use Cases
56---------
57
58#. Openstack notification data would be stored in Elasticsearch
59 via the Monasca Events API
60
61 Example sequence from Nova notification to Monasca API
62 #. Nova completes the creation of a VM
63 #. Nova generates a Notification message to oslo.messaging
64 #. Ceilometer Notification agent receives the Notification message
65 #. Ceilometer translates the Notification to a Monasca API format according to the configuration
66 #. Ceilometer Event Publisher publishes formatted Notification to Monasca Events API
67 #. Monasca Events API receives and validates formatted Notification
68 #. Monasca Events stores event Notification in configured Elasticsearch instance
69
70
71Proposed change
72===============
73
74#. Openstack Notifications consist of envelope and payload fields
75
76 Example Openstack Notification data format:
77
78 .. code-block:: javascript
79
80 {
81 "_context_auth_token": "42630b3ea13242fcad20e0a92d0207f1",
82 "_context_domain": null,
83 "_context_instance_lock_checked": false,
84 "_context_is_admin": true,
85 "_context_project_domain": null,
86 "_context_project_id": "a4f77",
87 "_context_project_name": "admin",
88 "_context_quota_class": null,
89 "_context_read_deleted": "no",
90 "_context_read_only": false,
91 "_context_remote_address": "192.168.245.4",
92 "_context_request_id": "req-5948338c-f223-4fd8-9249-8769f7a3e460",
93 "_context_resource_uuid": null,
94 "_context_roles": [
95 "monasca-user",
96 "admin",
97 "KeystoneAdmin"
98 ],
99 "_context_service_catalog": [
100 {
101 "endpoints": [
102 {
103 "adminURL": "http://192.168.245.8:8776/v2/a4f77",
104 "internalURL": "http://192.168.245.8:8776/v2/a4f77",
105 "publicURL": "http://192.168.245.9:8776/v2/a4f77",
106 "region": "region1"
107 }
108 ],
109 "name": "cinderv2",
110 "type": "volumev2"
111 },
112 {
113 "endpoints": [
114 {
115 "adminURL": "http://192.168.245.8:8776/v1/a4f77",
116 "internalURL": "http://192.168.245.8:8776/v1/a4f77",
117 "publicURL": "http://192.168.245.9:8776/v1/a4f77",
118 "region": "region1"
119 }
120 ],
121 "name": "cinder",
122 "type": "volume"
123 }
124 ],
125 "_context_show_deleted": false,
126 "_context_tenant": "a4f77",
127 "_context_timestamp": "2015-09-18T20:54:23.468522",
128 "_context_user": "be396488c7034811a200a3cb1d103a28",
129 "_context_user_domain": null,
130 "_context_user_id": "be396488c7034811a200a3cb1d103a28",
131 "_context_user_identity": "be396488c7034811a200a3cb1d103a28 a4f77 - - -",
132 "_context_user_name": "admin",
133 "_unique_id": "ff9699d587bf4283a3c367ab88be1541",
134 "event_type": "compute.instance.create.start",
135 "message_id": "c6149ba1-34b3-4367-b8c2-b1d6f073742d",
136 "payload": {
137 "access_ip_v4": null,
138 "access_ip_v6": null,
139 "architecture": null,
140 "availability_zone": null,
141 "cell_name": "",
142 "created_at": "2015-09-18 20:55:25+00:00",
143 "deleted_at": "",
144 "disk_gb": 1,
145 "display_name": "testeee",
146 "ephemeral_gb": 0,
147 "host": null,
148 "hostname": "testeee",
149 "image_meta": {
150 "base_image_ref": "df0c8",
151 "container_format": "bare",
152 "disk_format": "qcow2",
153 "min_disk": "1",
154 "min_ram": "0"
155 },
156 "image_name": "glanceaaa3",
157 "image_ref_url": "http://192.168.245.5:9292/images/df0c8",
158 "instance_flavor_id": "1",
159 "instance_id": "abd2ef5c-0381-434a-8efc-d7b39b28a2b6",
160 "instance_type": "m1.tiny",
161 "instance_type_id": 4,
162 "kernel_id": "",
163 "launched_at": "",
164 "memory_mb": 512,
165 "metadata": {},
166 "node": null,
167 "os_type": null,
168 "progress": "",
169 "ramdisk_id": "",
170 "reservation_id": "r-1ghilddw",
171 "root_gb": 1,
172 "state": "building",
173 "state_description": "",
174 "tenant_id": "a4f77",
175 "terminated_at": "",
176 "user_id": "be396488c7034811a200a3cb1d103a28",
177 "vcpus": 1
178 },
179 "priority": "INFO",
180 "publisher_id": "compute.ccp-compute0001-mgmt",
181 "timestamp": "2015-09-18 20:55:37.639023"
182 }
183
184#. All the fields with the prefix of '_context" are the envelope fields, the
185 other interesting fields are
186
187 #. 'message_id' - notification identifier
188 #. 'payload' - contains most of the relevant and useful information in JSON format
189 #. 'priority' - notification priority
190 #. 'publisher_id' - notification publisher
191 #. 'timestamp' - notification timestamp
192
193#. Ceilometer event publishing framework converts the Openstack notifications to events format[4].
194 Event publishing framework also has the ability to extract only some of the 'payload' data into
195 a flat set of key-value pairs called 'traits' and publish the normalized 'event' with 'traits'
196 extracted from the payload using a custom publisher.
197
198 Extraction of certain fields into traits from the payload is
199 driven by configuration file, but by default "publisher_id",
200 'request_id', 'tenant_id', 'user_id' and 'project_id'
201 fields are always extracted and added as 'traits'.
202
203 The event can also have an optional field called 'raw' which has original
204 notification, provided 'store_raw' option is set in ceilometer.conf
205
206 Questions/TODO:
207
208 * Q1: Does the store_raw field only apply to events, or to all notifications processed by
209 Ceilometer?
210 * We will have to find it out if it has any adverse impact on sample publisher. Though in the
211 case of samples, monasca sample publisher definitely does not submit raw payload, so it must
212 be getting dropped.
213
214 Example Ceilometer Event data format:
215
216 .. code-block:: javascript
217
218 {
219 "event_type": "compute.instance.create.start",
220 "message_id": "c6149ba1-34b3-4367-b8c2-b1d6f073742d",
221 "generated": "2015-09-18 20:55:37.639023",
222 "traits": {
223 "publisher_id": "compute.ccp-compute0001-mgmt",
224 "request_id": "req-5948338c-f223-4fd8-9249-8769f7a3e460",
225 "tenant_id": "a4f77",
226 "project_id": "a4f77",
227 "user_id": "be396488c7034811a200a3cb1d103a28"
228 },
229 "raw": { "_context_auth_token": "42630b3ea13242fcad20e0a92d0207f1",
230 "_context_domain": null,
231 ...
232 ...
233 "event_type": "compute.instance.create.start",
234 "message_id": "c6149ba1-34b3-4367-b8c2-b1d6f073742d",
235 "payload": {
236 "access_ip_v4": null,
237 "access_ip_v6": null,
238 "architecture": null,
239 "availability_zone": null,
240 "cell_name": "",
241 "created_at": "2015-09-18 20:55:25+00:00",
242 "deleted_at": "",
243 "disk_gb": 1,
244 "display_name": "testeee",
245 "ephemeral_gb": 0,
246 "host": null,
247 "hostname": "testeee",
248 "image_meta": {
249 "base_image_ref": "df0c8",
250 "container_format": "bare",
251 "disk_format": "qcow2",
252 "min_disk": "1",
253 "min_ram": "0"
254 },
255 "image_name": "glanceaaa3",
256 "image_ref_url": "http://192.168.245.5:9292/images/df0c8",
257 "instance_flavor_id": "1",
258 "instance_id": "abd2ef5c-0381-434a-8efc-d7b39b28a2b6",
259 "instance_type": "m1.tiny",
260 "instance_type_id": 4,
261 "kernel_id": "",
262 "launched_at": "",
263 "memory_mb": 512,
264 "metadata": {},
265 "node": null,
266 "os_type": null,
267 "progress": "",
268 "ramdisk_id": "",
269 "reservation_id": "r-1ghilddw",
270 "root_gb": 1,
271 "state": "building",
272 "state_description": "",
273 "tenant_id": "a4f77",
274 "terminated_at": "",
275 "user_id": "be396488c7034811a200a3cb1d103a28",
276 "vcpus": 1
277 }
278 }
279 }
280
281
282#. Key-Value pairs that can be extracted from 'payload' in form of traits
283 can be defined in events definitions file.
284
285 For example the following events definitions yaml specifies that for
286 all events which have a prefix of "compute.instance.*" then
287 add "user_id", "instance_id", and "instance_type_id" as traits,
288 after extracting values from "payload.user_id", "payload.instance_id",
289 and "payload.instance_type_id" respectively.
290
291 .. code-block:: yaml
292
293 ---
294 - event_type: compute.instance.*
295
296 traits: &instance_traits
297 user_id:
298 fields: payload.user_id
299 instance_id:
300 fields: payload.instance_id
301 instance_type_id:
302 type: int
303 fields: payload.instance_type_id
304
305 We are for now proposing not to use this feature, of defining traits for each event
306 extracting since we have the ability to store entire payload, via
307 Monasca Events API.
308
309 We can certainly look at enabling this feature in the future if we run into trouble storing
310 entire JSON "payload" in Elasticsearch. This is certainly a nifty way to trim the amount
311 of data that will be stored.
312
313#. The proposed new Custom Monasca Ceilometer event publisher will run within Ceilometer's
314 Notification Agent component. It will leverage Ceilometer's data processing pipeline[3] which
315 converts notifications to Ceilometer's event format. At the end of its processing, Monasca
316 Ceilometer event publisher will convert Ceilometer Event data into Monasca Event format[6] and
317 publish the monasca event to Monasca Events API.
318
319#. Monasca Events API allows a field called 'payload' which can be in an arbitrary
320 nested JSON format. Monasca-Ceilometer event publisher will extract JSON field called
321 'payload' from 'raw' (JSON path notation: 'raw.payload'), publish the payload from the
322 original notification to Monasca Events API.
323
324 Example Monasca Event Format:
325
326 .. code-block:: javascript
327
328 events: [
329 {
330 dimensions": {
331 "service": "compute.ccp-compute0001-mgmt",
332 "topic": "notification.sample",
333 "hostname": "nova-compute:compute
334 },
335 event: {
336
337 "event_type": "compute.instance.create.start",
338
339 "payload": {
340 "access_ip_v4": null,
341 "access_ip_v6": null,
342 "architecture": null,
343 "availability_zone": null,
344 "cell_name": "",
345 "created_at": "2015-09-18 20:55:25+00:00",
346 "deleted_at": "",
347 "disk_gb": 1,
348 "display_name": "testeee",
349 "ephemeral_gb": 0,
350 "host": null,
351 "hostname": "testeee",
352 "image_meta": {
353 "base_image_ref": "df0c8",
354 "container_format": "bare",
355 "disk_format": "qcow2",
356 "min_disk": "1",
357 "min_ram": "0"
358 },
359 "image_name": "glanceaaa3",
360 "image_ref_url": "http://192.168.245.5:9292/images/df0c8",
361 "instance_flavor_id": "1",
362 "instance_id": "abd2ef5c-0381-434a-8efc-d7b39b28a2b6",
363 "instance_type": "m1.tiny",
364 "instance_type_id": 4,
365 "kernel_id": "",
366 "launched_at": "",
367 "memory_mb": 512,
368 "metadata": {},
369 "node": null,
370 "os_type": null,
371 "progress": "",
372 "ramdisk_id": "",
373 "reservation_id": "r-1ghilddw",
374 "root_gb": 1,
375 "state": "building",
376 "state_description": "",
377 "tenant_id": "a4f77",
378 "terminated_at": "",
379 "user_id": "be396488c7034811a200a3cb1d103a28",
380 "vcpus": 1
381 }
382 },
383 publisher_id: "compute.ccp-compute0001-mgmt",
384 priority: "INFO"
385 }
386 ]
387
388#. If no traits are specified in events pipeline yaml configuration file for an event
389 Ceilometer's data processing pipeline will add the following default traits:
390
391 * service: (All notifications should have this) notification’s publisher
392 * tenant_id
393 * request_id
394 * project_id
395 * user_id
396
397 Note: "service" is not the service that produced the event as in say "compute", "glance",
398 "cinder" but rather notification RabbitMQ publisher that produced the event
399 e.g. "compute.ccp-compute0001-mgmt" so is not very useful.
400
401#. Ceilometer event data is converted to Monasca event data format before being published to Monasca
402 Event API. Following fields in Monasca Event data are not available in current Ceilometer Event
403 data format:
404
405 * "service"
406 * "dimensions.topic"
407 * "event.priority"
408
409 We are proposing removing these fields from Monasca Event format (will be done as a separate
410 spec/implementation process) for the following reasons:
411
412 "service": Currently Openstack notifications do not specify a service, that
413 generated the notification in a consistent way. It might be possible to create an external
414 mapping file which maps event name to a service but its hard to maintain such mapping over a
415 period of time.
416
417 "dimensions.topic": This field is not available in the source Openstack notification
418
419 "event.priority": This field is not currently available in Ceilometer Event format. It is
420 available in the source Openstack notification. Note: If we think this field can be useful we can
421 propose adding it to the Ceilometer Event format.
422
423#. Following new fields will be added to Monasca Event data as dimensions:
424
425 * "dimensions.publisher_id": Identifier for the publisher that generated the event. Maps to
426 "traits.publisher_id" in Ceilometer event data.
427 * "dimensions.user_id": Identifier for user that generated the event. Maps to "traits.user_id" in
428 Ceilometer event data.
429 * "dimensions.project_id": Identifier of the project that generated the event. Maps to
430 "traits.project_id" or "traits.tenant_id" in Ceilometer event data.
431
432#. hostname is available in the event payload, but its location might differ from event to event. We
433 can use Ceilometer's event definitions config to always add a trait called "hostname" to all
434 events. e.g. for compute.instance.* will have a trait called "hostname", which grabs data from
435 "payload.hostname"
436
437 .. code-block:: yaml
438
439 ---
440 - event_type: compute.instance.*
441
442 traits: &instance_traits
443 user_id:
444 fields: payload.hostname
445
446#. The proposed new Monasca Ceilometer event publisher will have the ability to submit event
447 data in a batch and at a configurable frequency (similar to current samples publisher). The
448 event data will be published if the items in the current batch reach their maximum size
449 (config setting) or if certain time interval has elapsed since the last publish
450 (config setting). This will make sure that the batch does not get huge at the same time
451 there is no significant delay in publishing of the events to Monasca Events API.
452
453#. Monasca Ceilometer event publisher will use service credentials from ceilometer configuration
454 file (in "[monasca]" section) to get keystone token.
455
456 Example "[monasca]" section in ceilometer config file
457 .. code-block:: text
458
459 [monasca]
460 service_auth_url = https://localhost:5000/v3
461 service_password = secretpassword
462 service_username = ceilometer
463 service_interface = internalURL
464 service_auth_type = password
465 # service_project_id may also be used
466 service_project_name = admin
467 service_domain_name = Default
468 service_region_name = RegionOne
469
470 The publisher will then make a POST request to Monasca Events /v1.0/events REST api[8] to publish
471 events to Monasca Events API. The URL for the instance of Monasca Events API will be configured
472 in the Ceilometer 'events-pipeline.yaml' file. This has the added advantage of allowing
473 different events to be published differently (see Ceilometer pipeline documentation [10]).
474
475#. "tenant_id" and "user_id" that the notification relates to are available in "payload" section
476 of the notification, and these notifications are generated by each service itself.
477
478 There is no additional "Openstack-operator-agent" like component or functionality required to
479 fetch that data from the service and publish to monasca event api on behalf of the original
480 tenant.
481 Ceilometer publishing pipeline simply extracts these "tenant_id" and "user_id" fields from the
482 "payload" and makes those fields available as "tenant_id" and "user_id" traits, which would then
483 be mapped to "dimensions.project_id" and "dimensions.user_id" fields in monasca events format.
484
485 In other words, original "tenant_id" and "user_id" values are available in
486 the payload of the notification, and will make its way to "dimensions.tenant_id"
487 and "dimensions.user_id" in Monasca Event.
488
489 Questions/TODO:
490 * Q: Do we need to do anything special to handle multi-tenancy in monasca-events api like being
491 done for metrics[9] ? Would original user_id and tenant_id in "dimensions.user_id" and
492 "dimensions.tenant_id" fields in dimensions serve this purpose?
493 * Q: In Ceilometer V2 API (which has been deprecated and removed), when querying data the role
494 "admin" could access data for all tenants, whereas a user with "ceilometer-admin" role could
495 access only data for a particular tenant. Can we implement something like this for
496 monasca-events api when querying for data?
497
498#. Monasca Ceilometer event publisher will also retry submitting a batch, in case Monasca
499 Events API is temporarily unavailable or down. The retry frequency, the number of retries
500 and the number of items that can be in the retry batch will also be set via configuration.
501
502
503Alternative Solutions
504---------------------
505
506#. Standalone monasca event agent which reads Openstack notifications published to RabbitMQ
507 (on "notification" topic) and publishes them to Monasca Events API.
508 Pro:
509 * No dependency on Telemetry project.
510 * May be simple to develop if leverage the oslo.messaging functionality.
511 * Ceilometer has *deprecated* the events functionality in the Stein release. [13]
512 Con:
513 * Another agent to convince users to install on their systems.
514 * Reinventing work already done in the Ceilometer agent. The OpenStack community already uses Ceilometer and contributes updates when something fails.
515 This alternate solution will be detailed in a separate spec, as it is likely
516 the long term solution Monasca will need.
517
518#. Openstack Panko [5] is a event storage and REST API for Ceilometer.
519 Pro:
520 * An 'official' subproject within Telemetry, so there is some community recognition.
521 Con:
522 * Its primary storage is in a relational database which has problems with scale.
523 * It is not maintained actively and not ready for production. [11]
524 * It will be deprecated eventually. [12]
525
526Data model impact
527-----------------
528
529None
530
531REST API impact
532---------------
533
534#. We are proposing to tweak the Monasca Event data format by removing and adding following
535 fields as mentioned in "Proposed change" section above.
536
537 Remove fields (JSON path notation): "service", "dimensions.topic",
538 "dimensions.hostname" and "event.priority"
539
540 Add fields (JSON path notation): "dimensions.publisher_id", "dimensions.user_id" and
541 "dimensions.project_id"
542
543 This change will have an impact on Monasca Events API.
544
545Security impact
546---------------
547
548The proposed Monasca Ceilometer events publisher will collect and publish
549Openstack event (notification) data to Monasca API. Openstack notification
550data does not have any sensitive data like 'tokens'.
551Notifications do contain 'user_id' and 'project_id' fields but do not
552contain any Personally Identifiable Information (PII) for the user or
553the project.
554
555
556Other end user impact
557---------------------
558
559None.
560
561Performance Impact
562------------------
563
564#. The number of notifications(events) generated by different services will depend on the capacity
565 of the cloud along with the number of resources being created by the users.
566
567 For example, if there was a large number of compute VM's being created or destroyed it could
568 lead to a surge in number of notifications (events) that would have to be published. Optimum
569 configuration options related to say event batch size and event batch interval would have to be
570 documented, to reduce any adverse affect on performance.
571
572#. Monasca Ceilometer publisher runs within Ceilometer Notification Agent component and invoked as a
573 last step in its data processing pipeline. It is an additional component that will have to to be
574 deployed on all the controller nodes. We will have to evaluate the performance impact of
575 Ceilometer Notification Agent when publishing events to Monasca Events API.
576
577
578Other deployer impact
579---------------------
580
581#. The proposed new Monasca-Ceilometer events publisher will introduce
582 few new configuration options like
583 * events api endpoint
584 * events batch interval
585 * events batch size
586 * events retry interval
587
588#. Monasca Ceilometer Events publisher will have to to be added to Ceilometer's
589 "[ceilometer.event.publisher]" section entry_points.txt
590
591 For example:
592
593 [ceilometer.event.publisher]
594 monasca = ceilometer.publisher.monclient:MonascaEventsPublisher
595
596#. As part of developing new Monasca Ceilometer Events publisher devstack plugin would be updated to
597 add the above configuration changes.
598
599
600Developer impact
601----------------
602
603#. The proposed change to Monasca Event Format will have an impact on existing Monasca Event API,
604 since Monasca Event Format will have to be tweaked. (See REST API Impact section above)
605
606
607Implementation
608==============
609
610Assignee(s)
611-----------
612
613Primary assignee:
614 joadavis, aagate
615
616Other contributors:
617 <launchpad-id or None>
618
619
620Work Items
621----------
622
623#. Implement new Monasca Ceilometer Events publisher.
624
625#. Implement monasca-ceilometer devstack plugin changes to deploy
626 new events publisher.
627
628#. Implement unit tests for Events publisher.
629
630#. Implement change to Monasca Event format in Monasca Events API.
631
632
633Dependencies
634============
635
636#. Monasca Events API 1.0: https://storyboard.openstack.org/#!/story/2001654
637
638#. Monasca Ceilometer project: https://github.com/openstack/monasca-ceilometer
639
640#. Ceilometer Data processing and pipelines:
641https://docs.openstack.org/ceilometer/pike/admin/telemetry-data-pipelines.html
642
643Testing
644=======
645
646#. New Monasca Ceilometer Event publisher unit tests will be added, which can test publishing with
647 various config options events batch size, events batch interval, handling retry when Monasca
648 Event API is not available.
649
650#. Adding tempest tests for Monasca Ceilometer events publisher could be looked at as part of
651 separate effort.
652
653 Please note that current Monasca Ceilometer samples publisher does not have tempest tests either
654 so having tempest tests for both events and samples publisher could be considered in the future.
655
656Documentation Impact
657====================
658
659#. New Monasca Events Publisher config options will be documented
660
661 * events api endpoint
662 * events batch interval
663 * events batch size
664 * events retry interval
665
666#. Recommended values for each of the config options will also be documented based on the size of
667 the cloud and resources for Cloud Operators.
668
669References
670==========
671
672[1] Monasca Events API 1.0: https://storyboard.openstack.org/#!/story/2001654
673
674[2] Monasca Ceilometer project: https://github.com/openstack/monasca-ceilometer
675
676[3] Ceilometer Data processing and pipelines:
677https://docs.openstack.org/ceilometer/pike/admin/telemetry-data-pipelines.html
678
679[4] Ceilometer Events: https://docs.openstack.org/ceilometer/latest/admin/telemetry-events.html
680
681[5] Openstack Panko: https://github.com/openstack/panko
682
683[6] Monasca Event Format:
684https://github.com/openstack/monasca-events-api/blob/master/doc/api-samples/v1/req_simple_event.json
685
686[7] Ceilometer System Architecture Diagram:
687https://docs.openstack.org/ceilometer/ocata/architecture.html
688
689[8] Monasca Events POST v1.0 API:
690https://github.com/openstack/monasca-events-api/blob/master/api-ref/source/events.inc
691
692[9] Cross-Tenant Metric Submission:
693https://github.com/openstack/monasca-agent/blob/master/docs/MonascaMetrics.md#cross-tenant-metric-submission
694
695[10] Ceilometer pipeline yaml documentation:
696https://docs.openstack.org/ceilometer/latest/admin/telemetry-data-pipelines.html
697
698[11] No future for Panko or Aodh:
699https://julien.danjou.info/lessons-from-openstack-telemetry-deflation/
700
701[12] Ceilometer Events deprecated means Panko also deprecated:
702http://eavesdrop.openstack.org/irclogs/%23openstack-telemetry/%23openstack-telemetry.2018-10-10.log.html
703
704[13] Ceilometer Events marked as deprecated in Stein:
705https://review.openstack.org/#/c/603336/ \ No newline at end of file
diff --git a/specs/stein/approved/monasca-events-listener.rst b/specs/stein/approved/monasca-events-listener.rst
new file mode 100644
index 0000000..c01d5b1
--- /dev/null
+++ b/specs/stein/approved/monasca-events-listener.rst
@@ -0,0 +1,586 @@
1..
2 This work is licensed under a Creative Commons Attribution 3.0 Unported
3 License.
4
5 http://creativecommons.org/licenses/by/3.0/legalcode
6
7=======================
8Monasca Events Listener
9=======================
10
11https://storyboard.openstack.org/#!/story/2003023
12
13Monasca Events API [1]_ was developed to store event data in Elasticsearch.
14A new application could use the Monasca Events API to post an event directly for processing
15and storage.
16However, a general collection service is needed to capture the existing OpenStack Notifications
17already generated by OpenStack Services and pass them to Monasca for storage and processing.
18This specification proposes creating a new Monasca Events Listener to capture events and pass
19them to Monasca services.
20
21
22Problem description
23===================
24
25All Openstack services generate a lot of notifications or events which contain large amounts of
26operational and state information about the service and its resources. Services such as Nova [14]_
27already publish to a RabbitMQ topic.
28This notification data is not currently available in Monasca.
29
30Ceilometer data processing pipeline [3]_ provides an extensible mechanism of publishing samples and
31events using a custom publisher. Ceilometer samples represent a quantity that can be measured
32(e.g. the size of a volume) and events represent an occurrence of an event and do not have any
33associated quantity (e.g. volume was created). The Telemetry project also has the Panko [5]_
34service for indexing and storing these events.
35
36A previous `spec <../../rocky/not-implemented/monasca-events-publishing.html>`_ was created to
37specify an enhancement to Ceilometer to allow collected events to be published to the Monasca
38Events API.
39However, with the recent deprecation of Ceilometer's Event functionality [13]_, this is no longer
40a long-term option.
41
42This spec is for creating a new Monasca service which would listen for OpenStack Event messages
43(often called notifications) and process them through Monasca by consuming the RabbitMQ message
44and producing a post to the Monasca Events API with that event.
45
46This service could use Ceilometer, Vitrage, Watcher, or another service as an example of how to
47listen to notifications from OpenStack services such as Nova [14]_.
48
49It is being proposed to make the Monasca Events Listener service a part of the Monasca Agent
50code base and reuse code, including the existing monasca_setup script and config.
51
52
53Use Cases
54---------
55
56#. Openstack notification data would be stored in Elasticsearch via the Monasca services
57
58 Example sequence:
59
60 #. Nova completes the creation of a VM
61 #. Nova generates a Notification message to oslo.messaging
62 #. oslo.messaging posts the message to RabbitMQ
63 #. Monasca Event Listener receives the Notification message from RabbitMQ
64 #. Monasca Event Listener validates and translates the Notification to a Monasca Events
65 API format according to the configuration
66 #. Monasca Event Listener publishes formatted Notification to Monasca Events API
67 #. Monasca Events API receives and validates formatted Notification
68 #. Monasca Events API stores event Notification in configured Elasticsearch instance
69
70
71Proposed change
72===============
73
74#. Monasca Event Listener will be a new service which can be run in a Highly
75 Available configuration by running an instance of Monasca Event Listener
76 on each Controller node in a cloud. Each node will listen to OpenStack
77 notifications in RabbitMQ and convert notifications to a post to the Monasca
78 Events API. The Monasca Events API then passes the notification on as Kafka
79 messages on the 'monevents' topic, where the Monasca Persister will receive
80 and store them. By the nature of RabbitMQ clients, the load will be
81 distributed between the Monasca Events Listener instances (only one listener
82 will process each RabbitMQ message).
83
84#. Monasca Event Listener will filter messages from OpenStack services based on
85 specification of event_types to be collected. This will reduce 'noise' and
86 focus event collection on those events that are deemed valuable.
87 This filtering specification can be from a configuration file, or optionally
88 could be controlled through a new API implemented as part of the Monasca
89 Events API service.
90
91#. OpenStack Notifications consist of envelope and payload fields. See [15]_ and [16]_ for
92 examples.
93
94 Example OpenStack Notification data format:
95
96 .. code-block:: javascript
97
98 {
99 "_context_auth_token": "42630b3ea13242fcad20e0a92d0207f1",
100 "_context_domain": null,
101 "_context_instance_lock_checked": false,
102 "_context_is_admin": true,
103 "_context_project_domain": null,
104 "_context_project_id": "a4f77",
105 "_context_project_name": "admin",
106 "_context_quota_class": null,
107 "_context_read_deleted": "no",
108 "_context_read_only": false,
109 "_context_remote_address": "192.168.245.4",
110 "_context_request_id": "req-5948338c-f223-4fd8-9249-8769f7a3e460",
111 "_context_resource_uuid": null,
112 "_context_roles": [
113 "monasca-user",
114 "admin",
115 "KeystoneAdmin"
116 ],
117 "_context_service_catalog": [
118 {
119 "endpoints": [
120 {
121 "adminURL": "http://192.168.245.8:8776/v2/a4f77",
122 "internalURL": "http://192.168.245.8:8776/v2/a4f77",
123 "publicURL": "http://192.168.245.9:8776/v2/a4f77",
124 "region": "region1"
125 }
126 ],
127 "name": "cinderv2",
128 "type": "volumev2"
129 },
130 {
131 "endpoints": [
132 {
133 "adminURL": "http://192.168.245.8:8776/v1/a4f77",
134 "internalURL": "http://192.168.245.8:8776/v1/a4f77",
135 "publicURL": "http://192.168.245.9:8776/v1/a4f77",
136 "region": "region1"
137 }
138 ],
139 "name": "cinder",
140 "type": "volume"
141 }
142 ],
143 "_context_show_deleted": false,
144 "_context_tenant": "a4f77",
145 "_context_timestamp": "2015-09-18T20:54:23.468522",
146 "_context_user": "be396488c7034811a200a3cb1d103a28",
147 "_context_user_domain": null,
148 "_context_user_id": "be396488c7034811a200a3cb1d103a28",
149 "_context_user_identity": "be396488c7034811a200a3cb1d103a28 a4f77 - - -",
150 "_context_user_name": "admin",
151 "_unique_id": "ff9699d587bf4283a3c367ab88be1541",
152 "event_type": "compute.instance.create.start",
153 "message_id": "c6149ba1-34b3-4367-b8c2-b1d6f073742d",
154 "payload": {
155 "access_ip_v4": null,
156 "access_ip_v6": null,
157 "architecture": null,
158 "availability_zone": null,
159 "cell_name": "",
160 "created_at": "2015-09-18 20:55:25+00:00",
161 "deleted_at": "",
162 "disk_gb": 1,
163 "display_name": "testeee",
164 "ephemeral_gb": 0,
165 "host": null,
166 "hostname": "testeee",
167 "image_meta": {
168 "base_image_ref": "df0c8",
169 "container_format": "bare",
170 "disk_format": "qcow2",
171 "min_disk": "1",
172 "min_ram": "0"
173 },
174 "image_name": "glanceaaa3",
175 "image_ref_url": "http://192.168.245.5:9292/images/df0c8",
176 "instance_flavor_id": "1",
177 "instance_id": "abd2ef5c-0381-434a-8efc-d7b39b28a2b6",
178 "instance_type": "m1.tiny",
179 "instance_type_id": 4,
180 "kernel_id": "",
181 "launched_at": "",
182 "memory_mb": 512,
183 "metadata": {},
184 "node": null,
185 "os_type": null,
186 "progress": "",
187 "ramdisk_id": "",
188 "reservation_id": "r-1ghilddw",
189 "root_gb": 1,
190 "state": "building",
191 "state_description": "",
192 "tenant_id": "a4f77",
193 "terminated_at": "",
194 "user_id": "be396488c7034811a200a3cb1d103a28",
195 "vcpus": 1
196 },
197 "priority": "INFO",
198 "publisher_id": "compute.ccp-compute0001-mgmt",
199 "timestamp": "2015-09-18 20:55:37.639023"
200 }
201
202#. All the fields with the prefix of '_context' are the envelope fields, the
203 other interesting fields are
204
205 #. 'message_id' - notification identifier
206 #. 'payload' - contains most of the relevant and useful information in JSON format
207 #. 'priority' - notification priority
208 #. 'publisher_id' - notification publisher
209 #. 'timestamp' - notification timestamp
210
211#. Monasca Event Listener converts the OpenStack notifications to Monasca events format.
212 This format will be suitable for Kafka messaging and will match the expected data fields
213 of the Monasca Persister. This conversion and validation should be common between the
214 Monasca Event Listener and Monasca Event API.
215
216#. The Kafka client connection will handle communication issues such as reconnections and
217 resending as needed.
218
219#. Monasca Events API allows a field called 'payload' which can be in an arbitrary
220 nested JSON format.
221 TODO: mappings
222
223 Example Monasca Event Format:
224
225 .. code-block:: javascript
226
227 events: [
228 {
229 dimensions": {
230 "service": "compute.ccp-compute0001-mgmt",
231 "topic": "notification.sample",
232 "hostname": "nova-compute:compute
233 },
234 event: {
235
236 "event_type": "compute.instance.create.start",
237
238 "payload": {
239 "access_ip_v4": null,
240 "access_ip_v6": null,
241 "architecture": null,
242 "availability_zone": null,
243 "cell_name": "",
244 "created_at": "2015-09-18 20:55:25+00:00",
245 "deleted_at": "",
246 "disk_gb": 1,
247 "display_name": "testeee",
248 "ephemeral_gb": 0,
249 "host": null,
250 "hostname": "testeee",
251 "image_meta": {
252 "base_image_ref": "df0c8",
253 "container_format": "bare",
254 "disk_format": "qcow2",
255 "min_disk": "1",
256 "min_ram": "0"
257 },
258 "image_name": "glanceaaa3",
259 "image_ref_url": "http://192.168.245.5:9292/images/df0c8",
260 "instance_flavor_id": "1",
261 "instance_id": "abd2ef5c-0381-434a-8efc-d7b39b28a2b6",
262 "instance_type": "m1.tiny",
263 "instance_type_id": 4,
264 "kernel_id": "",
265 "launched_at": "",
266 "memory_mb": 512,
267 "metadata": {},
268 "node": null,
269 "os_type": null,
270 "progress": "",
271 "ramdisk_id": "",
272 "reservation_id": "r-1ghilddw",
273 "root_gb": 1,
274 "state": "building",
275 "state_description": "",
276 "tenant_id": "a4f77",
277 "terminated_at": "",
278 "user_id": "be396488c7034811a200a3cb1d103a28",
279 "vcpus": 1
280 }
281 },
282 publisher_id: "compute.ccp-compute0001-mgmt",
283 priority: "INFO"
284 }
285 ]
286
287#. Following fields in Monasca Event data may not be available in the OpenStack notification
288 data format:
289
290 * "service"
291 * "dimensions.topic"
292 * "event.priority"
293
294 We are proposing removing these fields from Monasca Event format (will be done as a separate
295 spec/implementation process) for the following reasons:
296
297 "service": Currently OpenStack notifications do not specify a service, that
298 generated the notification in a consistent way. It might be possible to create an external
299 mapping file which maps event name to a service but its hard to maintain such mapping over a
300 period of time.
301
302 "dimensions.topic": This field is not available in the source OpenStack notification.
303 However, the Monasca Event Listener may be able to save the RabbitMQ topic that the notification
304 was collected from. In that case, this field should be used.
305
306 "event.priority": This field is not currently available in Ceilometer Event format. It is
307 available in the source OpenStack notification. Note: If we think this field can be useful we can
308 propose adding it to the Monasca Event Listener format.
309
310#. Following new fields will be added to Monasca Event data as dimensions:
311
312 * "dimensions.publisher_id": Identifier for the publisher that generated the event.
313 * "dimensions.user_id": Identifier for user that generated the event.
314 * "dimensions.project_id": Identifier of the project that generated the event.
315
316#. 'hostname' is available in the event payload, but its location might differ from event to event.
317
318#. The proposed new Monasca Event Listener will have the ability to submit event
319 data in a batch and at a configurable frequency (similar to current samples publisher). The
320 event data will be published if the items in the current batch reach their maximum size
321 (config setting) or if certain time interval has elapsed since the last publish
322 (config setting). This will make sure that the batch does not get huge at the same time
323 there is no significant delay in publishing of the events to Monasca Events API.
324
325#. Monasca Event Listener will have a configuration file to configure connection information for
326 both RabbitMQ and Monasca Events API.
327
328#. The "tenant_id" and "user_id" that the notification relates to are available in "payload"
329 section of the notification, and these notifications are generated by each service itself.
330
331 There is no additional "OpenStack-operator-agent" like component or functionality required to
332 fetch that data from the service and publish to monasca event api on behalf of the original
333 tenant.
334 (Ceilometer publishing pipeline simply extracts these "tenant_id" and "user_id" fields from the
335 "payload" and makes those fields available as "tenant_id" and "user_id" traits, which would then
336 be mapped to "dimensions.project_id" and "dimensions.user_id" fields in monasca events format.)
337
338 In other words, original "tenant_id" and "user_id" values are available in
339 the payload of the notification, and will make its way to "dimensions.tenant_id"
340 and "dimensions.user_id" in Monasca Event.
341
342 Questions/TODO:
343
344 * Q: Do we need to do anything special to handle multi-tenancy in monasca-events api like being
345 done for metrics [9]_ ? Would original user_id and tenant_id in "dimensions.user_id" and
346 "dimensions.tenant_id" fields in dimensions serve this purpose?
347
348 * A: Monasca Events Listener can start with sending all events to a single "admin" tenant
349 and if required in the future some other process could copy select metrics back to tenant
350 projects.
351
352 * Q: In Ceilometer V2 API (which has been deprecated and removed), when querying data the role
353 "admin" could access data for all tenants, whereas a user with "ceilometer-admin" role could
354 access only data for a particular tenant. Can we implement something like this for
355 monasca-events api when querying for data?
356
357 * A: In Monasca API every request is scoped to the project, so there is no equivalent of
358 Ceilometer's "admin" role to query data for all projects. So placing all events in to
359 an "admin" project may be the best approach.
360
361 * Q: How should services which generate notifications but do not include a tenant_id be
362 handled? For example Keystone [16]_.
363 How does Ceilometer handle such events?
364
365 * A: If all events are in an "admin" project then admin metrics like shared ceph cluster
366 load or provider network load can be copied back to tenants so they may understand how
367 infrastructure is affecting their workload.
368
369 * Note: Configuration of Elasticsearch cluster is out of scope for this spec. If needed
370 could assign a separate Elasticsearch cluster to the events API to avoid overloads.
371
372
373Alternative Solutions
374---------------------
375
376#. Reuse the Ceilometer functionality to collect and publish events to the
377 Monasca Events API. While this may be less work initially, Ceilometer has
378 deprecated the Events functionality as of Stein. [13]_
379
380#. An alternate Events Listener was proposed that would listen for RabbitMQ events
381 then publish them directly to the 'monevents' topic in Kafka. Discussion on this
382 can be seen in the git history for this document and in IRC logs [18]_.
383
384 Pro:
385
386 * A much simpler approach, more efficient that HTTP hop through another service.
387 * No need for batching in service, as RabbitMQ and Kafka clients would handle fast
388 throughput and short network interruptions.
389
390 Con:
391
392 * Nova Cells v2 each have their own RabbitMQ instances. While most deployments
393 would likely have a centralized RabbitMQ, it is not required in the documented
394 architecture.
395 * Regions may also cause separation of RabbitMQ instances that need to be monitored.
396 * While it might be possible to have a service/agent in each Cell publish back to
397 a centralized to Kafka directly, our authentication and networking for Kafka was
398 not designed to support that.
399
400#. OpenStack Panko [5]_ is a event storage and REST API for Ceilometer and could
401 be used instead of a Monasca solution.
402
403 Pro:
404
405 * An 'official' subproject within Telemetry, so there is some community recognition.
406
407 Con:
408
409 * Its primary storage is in a relational database which has problems with scale.
410 * It is not maintained actively and not ready for production. [11]_
411 * It will be deprecated eventually. [12]_
412
413
414Data model impact
415-----------------
416
417None
418
419REST API impact
420---------------
421
422#. We are proposing to tweak the Monasca Event data format by removing and adding following
423 fields as mentioned in "Proposed change" section above.
424
425 Remove fields or make them optional (JSON path notation): "service", "dimensions.topic",
426 "dimensions.hostname" and "event.priority"
427
428 Add fields (JSON path notation): "dimensions.publisher_id", "dimensions.user_id" and
429 "dimensions.project_id"
430
431 This change will have an impact on Monasca Events API.
432
433Security impact
434---------------
435
436The proposed Monasca Event Listener will collect and publish OpenStack event
437(notification) data to Monasca Events API. OpenStack notification data does not
438have any sensitive data like 'tokens'.
439Notifications do contain 'user_id' and 'project_id' fields but do not
440contain any Personally Identifiable Information (PII) for the user or
441the project.
442
443
444Other end user impact
445---------------------
446
447None.
448
449Performance Impact
450------------------
451
452#. The number of notifications(events) generated by different services will depend on the capacity
453 of the cloud along with the number of resources being created by the users.
454
455 For example, if there was a large number of compute VM's being created or destroyed it could
456 lead to a surge in number of notifications (events) that would have to be published. Optimum
457 configuration options related to say event batch size and event batch interval would have to be
458 documented, to reduce any adverse affect on performance.
459
460#. The proposed Monasca Event Listener is a new service, so performance is unknown. However, the
461 Monasca API has been shown to have a high performance throughput.
462
463
464Other deployer impact
465---------------------
466
467#. The proposed Monasca Event Listener will introduce a
468 few new configuration options like
469
470 * RabbitMQ connection information
471 * Monasca Events API endpoint URL
472 * events batch interval
473 * events batch size
474 * events retry interval
475 * Keystone credentials for obtaining a token
476 * Conversion options for OpenStack notifications to the Monasca Event format. This may
477 be stored in separate pipeline configuration files, similar to how transform specs
478 are configured in Monasca Transform.
479
480#. As part of developing new Monasca Event Listener devstack plugin would be
481 updated to add the above configuration changes.
482
483
484Developer impact
485----------------
486
487#. The proposed change to Monasca Event Format will have an impact on existing Monasca Event API,
488 since Monasca Event Format will have to be tweaked. (See REST API Impact section above)
489
490Implementation
491==============
492
493Assignee(s)
494-----------
495
496Primary assignee:
497 joadavis, aagate
498
499Other contributors:
500 <launchpad-id or None>
501
502
503Work Items
504----------
505
506#. Implement new Monasca Event Listener.
507
508 * Connection to RabbitMQ for OpenStack Notifications
509 * Add filtering of notifications for configured event_types
510
511 * Specification in configuration file
512 * (Optional) Creation of a new API to configure event_type subscriptions
513
514 * Validation of OpenStack Notification data and format
515 * Conversion of data format to meet Monasca Events requirements
516 * Publishing to Monasca Events API
517 * Configuration of conversion specifications per-event type
518
519#. Implement monasca devstack plugin changes to deploy
520 new Events Listener service.
521
522#. Implement unit tests for Monasca Event Listener.
523
524
525Dependencies
526============
527
528None
529
530Testing
531=======
532
533#. New Monasca Event Listener unit tests will be added, which can test publishing with
534 various config options events batch size, events batch interval, handling retry when Monasca
535 Event API is not available.
536
537#. Adding tempest tests for Monasca Event Listener could be looked at as part of
538 separate effort.
539
540
541Documentation Impact
542====================
543
544#. New Monasca Event Listener config options will be documented
545
546#. Recommended values for each of the config options will also be documented based on the size of
547 the cloud and resources for Cloud Operators.
548
549References
550==========
551
552.. [1] Monasca Events API 1.0: https://git.openstack.org/cgit/openstack/monasca-events-api/
553
554[2] Monasca Ceilometer project: https://github.com/openstack/monasca-ceilometer
555
556.. [3] Ceilometer Data processing and pipelines: https://docs.openstack.org/ceilometer/pike/admin/telemetry-data-pipelines.html
557
558[4] Ceilometer Events: https://docs.openstack.org/ceilometer/latest/admin/telemetry-events.html
559
560.. [5] Openstack Panko: https://github.com/openstack/panko
561
562[6] Monasca Event Format: https://github.com/openstack/monasca-events-api/blob/master/doc/api-samples/v1/req_simple_event.json
563
564[7] Ceilometer System Architecture Diagram: https://docs.openstack.org/ceilometer/ocata/architecture.html
565
566[8] Monasca Events POST v1.0 API: https://github.com/openstack/monasca-events-api/blob/master/api-ref/source/events.inc
567
568.. [9] Cross-Tenant Metric Submission: https://github.com/openstack/monasca-agent/blob/master/docs/MonascaMetrics.md#cross-tenant-metric-submission
569
570[10] Ceilometer pipeline yaml documentation: https://docs.openstack.org/ceilometer/latest/admin/telemetry-data-pipelines.html
571
572.. [11] No future for Panko or Aodh: https://julien.danjou.info/lessons-from-openstack-telemetry-deflation/
573
574.. [12] Ceilometer Events deprecated means Panko also deprecated: http://eavesdrop.openstack.org/irclogs/%23openstack-telemetry/%23openstack-telemetry.2018-10-10.log.html
575
576.. [13] Ceilometer Events marked as deprecated in Stein: https://review.openstack.org/#/c/603336/
577
578.. [14] Nova notification version update lists services effected (see "Deprecating legacy notifications"): https://etherpad.openstack.org/p/nova-ptg-stein
579
580.. [15] Nova notification reference: https://docs.openstack.org/nova/latest/reference/notifications.html#existing-versioned-notifications
581
582.. [16] Keystone notification reference: https://docs.openstack.org/keystone/latest/advanced-topics/event_notifications.html#example-notification-project-create
583
584[17] Monasca Events API publisher to Kafka: https://github.com/openstack/monasca-events-api/blob/master/monasca_events_api/app/common/events_publisher.py
585
586.. [18] Monasca IRC meeting Dec 15, 2018: http://eavesdrop.openstack.org/meetings/monasca/2018/monasca.2018-12-12-15.00.log.html \ No newline at end of file