Commit Graph

4868 Commits

Author SHA1 Message Date
Zuul b91b9d914d Merge "Remove unused cache_key_mangler" 2024-03-28 06:24:10 +00:00
Zuul 110d136715 Merge "Bump hacking" 2024-03-18 11:10:43 +00:00
Takashi Kajinami 6ad28c20bb Fix wrong stacklevel in deprecation warnings
We should point the class which owns the methods.

Change-Id: Ib3e86c858f00fa0d384ae63493db64d20ed4b5e5
2024-02-21 23:56:06 +09:00
Takashi Kajinami 09c897f61c Remove unused cache_key_mangler
... so that we can remove usage of md5, which is not FIPs compliant.

Change-Id: I1e10e028ff716ba5ba72997b72750d3c7876d66d
2024-01-30 19:52:03 +09:00
Takashi Kajinami 7e7465140f Bump hacking
hacking 3.0.x is too old.

Change-Id: I970dcf9673ef75c719714f565bd4df2eb47c0989
2024-01-27 19:06:09 +09:00
Zuul b8716094c9 Merge "Add 'user_id' label in prometheus publisher" 2024-01-04 09:54:41 +00:00
Yadnesh Kulkarni 8f54f95134 Create new meter to poll power usage
IPMI sensor 'Current' captures current & power consumption metrics.

With the help of new pollster "hardware.ipmi.power", ceilometer
ipmi agent can differentiate between current and power metrics as
both are generated from the same sensor(Current).

Power metrics are captured using a slightly different command than
other sensors which is "ipmitool get sensor 'Pwr Consumption'".

Closes-Bug: #2038425
Change-Id: I0a8af40626cd44dca9743fba63c8dbda8729d054
2024-01-03 13:17:03 +00:00
Zuul f1e6594e52 Merge "Add opentelemetry publisher base on http" 2024-01-02 09:00:52 +00:00
Zuul f7184475b0 Merge "Deprecate OpenContrail support" 2023-12-20 11:13:06 +00:00
Zuul eb23c39717 Merge "Deprecate OpenDaylight support" 2023-12-20 11:11:28 +00:00
minruigao c972cecb37 Add opentelemetry publisher base on http
Opentelemetry is a standard data protocol of observability.
This patch add a new publisher about 'opentelemetry', it use
http to send data in json format to opentelemetry collector.
In the future, we can try to send data to opentelemetry by gRPC.

Change-Id: Ifa20d29d27e35d98999bf0e7ae519ebcedf811aa
2023-12-18 16:47:29 +08:00
Takashi Kajinami 36e47d7432 Deprecate OpenContrail support
The current OpenContrail neutron plugin, which is available in
GitHub[1] is not compatible with the recent OpenStack releases, mainly
because of openstacksdk pinned to <2.

Because the plugin can't be used with recent releases of OpenStack, we
do not expect any users using it as their network backend in their
OpenStack deployment.

[1] https://github.com/tungstenfabric/tf-neutron-plugin

Change-Id: I66cec47a9238dab68445838e5599bfe1c45d2e4e
2023-12-15 04:38:27 +00:00
Takashi Kajinami a92d77b182 Deprecate OpenDaylight support
The networking-odl project[1] has been retired because of inactivity,
and we no longer expect that any user may use OpenDaylight as network
backend of their OpenStack cluster.

[1] https://opendev.org/openstack/networking-odl

Change-Id: Ie70ffbc20d936a3f4cc2eb37bf7e3dce865a3191
2023-12-15 11:11:25 +09:00
minruigao c51dcb56db Add 'user_id' label in prometheus publisher
Sometimes we need user_id in prometheus metrics so that we can filter
cpu or memory data by user_id.

Change-Id: I92f2120d707c4288e8a02016e50bb4ce4be32c89
2023-12-13 19:14:00 +08:00
Zuul 7d471b92d5 Merge "Fallback to oslo_cache.dict caching backend" 2023-12-04 07:38:30 +00:00
Rafael Weingärtner 3159880013 Add logs for the `tenant_name_discovery=True` flow
While troubleshooting, we felt the needs to understand the execution
flow when `tenant_name_discovery=True`. Therefore, we added logs to
see its execution.

Change-Id: I711fe20915f4cf88f61b465cda3db91bd41817d3
2023-11-29 10:23:35 -03:00
Zuul ad2c9900bd Merge "Add logs in the SampleEndpoint class" 2023-11-24 13:11:15 +00:00
Rafael Weingärtner 12518e5e29 Add logs in the SampleEndpoint class
While troubleshooting the issue reported on [1], we needed
to see the logs for the push workflow. However, there were
some processes that had no logs to help us identify the issue.

This patch is proposing a log addition, and a raise to an exception,
to facilitate future troubleshootings.

[1] https://review.opendev.org/c/openstack/ceilometer/+/901607

Change-Id: I36a195965712469e33e73de809f290edd42c8b51
2023-11-22 16:42:29 -03:00
Rafael Weingärtner de91333fed Fix `TelemetryIpc` when using `tenant_name_discovery=False`
Patches [1-2] introduced the variables `project_name` and
`user_name` for the Ceilometer samples that are collected.
The gathering of these attributes is controlled with the use
of the parameter `tenant_name_discovery` , which defaults to
False.

However, when it (the parameter) is set as False, an error will
happen at [2], as the key will not exist in the dictionary.

This patch fixes the described issue.

Without this patch, the error that happens is the follwoing:
```
ERROR ceilometer.pipeline.sample [-] Fail to process notification: KeyError: 'user_name'
ERROR ceilometer.pipeline.sample Traceback (most recent call last):
ERROR ceilometer.pipeline.sample   File "/usr/lib/python3/dist-packages/ceilometer/pipeline/sample.py", line 44, in process_notifications
ERROR ceilometer.pipeline.sample     p(list(self.build_sample(message)))
ERROR ceilometer.pipeline.sample   File "/usr/lib/python3/dist-packages/ceilometer/telemetry/notifications.py", line 34, in build_sample
ERROR ceilometer.pipeline.sample     user_name=sample_dict['user_name'],
ERROR ceilometer.pipeline.sample KeyError: 'user_name'
ERROR ceilometer.pipeline.sample

```

[1] https://review.opendev.org/c/openstack/ceilometer/+/877647
[2] 2a5f63da95
Change-Id: I54a0fd43ac2da1b9a8bcdeae7ad7d160bc07f35b
2023-11-21 14:09:23 -03:00
Jaromir Wysoglad d2e247cf38 Remove monasca
This removes the monasca integration. Monasca was recently
declared as inactive project. In addition the monasca related
unit tests were very verbose and they were outputing a lot
of errors. Lately one of the unittests even started failing
randomly.

Change-Id: I468079b2a790245bd682c80d0116fb0da60d0e7c
2023-11-08 04:30:36 -05:00
Zuul 46461a5ef4 Merge "Fix ConnectionRefused in TCP publisher" 2023-11-06 12:46:24 +00:00
Yadnesh Kulkarni 7925cf9fcc Fallback to oslo_cache.dict caching backend
If ceilometer has invalid or missing caching configurations,
'oslo_cache.dict' will be set as the default caching backend.

Project and user names will be discovered only when caching
and "tenant_name_discovery" parameters are enabled.

Also "tenant_name_discovery" parameter must be evaluated
from "polling" section and not "DEFAULT".

Change-Id: I149e408f5a65f214101c0516ed8046d297127145
2023-10-17 12:30:11 +00:00
Jaromir Wysoglad 9c96730f15 Fix ConnectionRefused in TCP publisher
If ceilometer is started before the receiving end of the TCP
publisher, the TCP publisher's constructor ends with
an exception "ConnectionRefusedError". After that
ceilometer doesn't try to reconnect again.

I tested the change without and then with sg-core
running as a receiver. When sg-core wasn't running,
ceilometer would keep logging errors as expected. When I
started sg-core, metrics started to flow from ceilometer to
sg-core (without the need to touch the notification agent)

This change adds another "except" to catch the exception.
I also noticed, that the way the LOG.Error(...) is done
in the publisher doesn't work and it ends with a syntax
error. (_() accepts only one parameter, not two). So
I fixed the LOG.Errors.

Change-Id: Ib231dd0e6762159eeb09c02aed84a80561fbe188
2023-10-16 09:32:05 -04:00
Takashi Kajinami 641bad556b notification: Dump loaded options during start up
... so that operators can review effective values (except for secret
information such as password) from the log.

The same was already implemented in polling agents but has never been
added to agent-notification yet.

Change-Id: I69be0229d7b692ee06fc406e3f15bac33f46ad54
2023-09-19 04:42:34 +00:00
Zuul cf43abbfbf Merge "Refactor TCP provider" 2023-09-15 13:58:52 +00:00
Zuul 304c48a398 Merge "Deprecate support for VMWare vSphere" 2023-09-14 15:26:21 +00:00
Zuul cba79595a5 Merge "Add file publisher entrypoint to publish events" 2023-09-14 15:13:53 +00:00
Takashi Kajinami 297089a622 Deprecate support for VMWare vSphere
The vmwareapi virt driver in nova was marked experimenta[1] and may be
removed in a future release. Once this driver is removed, we may no
longer have any way to integrate VMWare vSphere hypervisor in OpenStack
deployments.

[1] 713ef7839670bd5d73af52c211a14932adc933ba

Change-Id: Ifb17fed132fb4acb4a23049ed2f4e9eeabd0055d
2023-09-11 22:38:08 +09:00
Takashi Kajinami 7660339b4a Deprecate Windows OS support
Unfortunately the Winstackers project has been retired because of no
maintainers[1][2], and OpenStack on Windows host is no longer
maintained.

[1] https://lists.openstack.org/pipermail/openstack-discuss/2022-November/031044.html
[2] https://review.opendev.org/c/openstack/governance/+/886880

Change-Id: I1b810d41ae07d2f6ddce4119a89b06a594f10b84
2023-09-11 11:01:22 +00:00
Yadnesh Kulkarni 3f874056c3 Add file publisher entrypoint to publish events
Implementation to publish event using file publisher already exists.

Due to the missing entrypoint for file publisher, events couldn't
be written to a file.

Closes-Bug: 2033052

Change-Id: Iee69978121874263d6de67dbddd420e44b6bff4c
2023-09-11 05:18:24 +00:00
Erno Kuvaja e62d6d9ed5 Refactor TCP provider
This change refactors the TCP provider to use socket.create_connection
avoiding the need to guess and do lots of messy handling between IPv6
and IPv4.

Also makes the error handling more specific expecting the errors socket
is emitting rather than catch all. removes logging stack trace on
connection issues.

We introduce break out of the message loop in case reconnect fails to
avoid spamming.

Change-Id: Id40fa0e023ad9ca9c05a06b9053bd97b48b4989c
2023-09-08 14:42:05 +01:00
Zuul 7d0ac89c3f Merge "jsonpath-rw-ext uses operator.add for "+" and operator.sub for "-". operator.add can do string concatenation, but operator.sub cannot do string subtraction. so the original hack does not work, perhaps we should use sub to get an empty string." 2023-08-07 12:41:09 +00:00
Yadnesh Kulkarni 9aad30748e Include 'hardware.ipmi.fan' to resource type ipmi_sensor
With this change, fan metrics collected by ipmi agent will appear in  Gnocchi with "RPM" unit.

Change-Id: I9b328e23e013a510a525746a41e491888c66e37a
2023-06-22 08:42:43 +00:00
Yadnesh Kulkarni 838a32681c Make multiple attempts to obtain gnocchiclient
As of now a single attempt is made to obtain gnocchiclient, if
this request fails due any intermittent issue, ceilometer fails to
load gnocchi publisher.

This change uses tenacity to retry fetching gnocchiclient if it
fails with keystone exceptions "ServiceUnavailable",
"DiscoveryFailure" or "ConnectTimeout" before assuming gnocchi service  is not available.

Upon restarting ceilometer services, "event_pipeline.yaml" definitions
file is logged during info mode which is unnecessary . This config
file must be logged only when debug is enabled.

Resolves: rhbz#2167428
Change-Id: Ia223b6518948765a74bb920829b6f76c354a3919
2023-06-09 15:28:21 +05:30
Dou Yuan a0c56384b5 jsonpath-rw-ext uses operator.add for "+" and operator.sub for "-". operator.add can do string concatenation, but operator.sub cannot do string subtraction. so the original hack does not work, perhaps we should use sub to get an empty string.
Change-Id: I9183a4a9f9cc345d8b4508bd7de1afa0eccd89d1
2023-05-22 20:49:02 +08:00
Rafael Weingärtner b9c151bb2d Add logs to help identify when a polling task finishes
During a troubleshooting session, we needed to discover the time
that it was taking for a polling task to finish. However, there
were no clear logs showing that. We could see when the task would
start, but not when it finished without errors. Therefore, we are
suggesting in this patch this small addition to the code base, which
can help operators to identify when a polling task has finished.

We added it in INFO log level because the message regarding the start
of the polling is also in INFO. However, we can change it to DEBUG if
the community thinks that DEBUG log level is better in this context.

Change-Id: I7358add1e45abe4a1288e51efa2f91d07c6acacd
2023-05-09 20:31:48 -03:00
OpenStack Proposal Bot 9eaa0d5887 Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I41671793e5c3249c8e5fd222bf37f83392c7c92a
2023-05-09 04:25:39 +00:00
Zuul da679054bd Merge "Make TCP publisher log warning instead of failing." 2023-04-24 15:52:29 +00:00
Jaromír Wysoglad 8699146e1a Make TCP publisher log warning instead of failing.
Change-Id: I56ca759c2d43830f4e342130f27b75bdba65c469
2023-04-24 07:27:52 -04:00
Yadnesh Kulkarni 6e339d3e74 Add vanity names to notification samples
This change adds "project_name" and "user_name" fields to the
polling samples created from notifications of "event_type".

Also move caching helper functions into "ceilometer/cache_utils.py"
to make them accessible throughout the project.

Change-Id: I68bd4ee096b28a2fd952e749d56a6b3eed9bfb94
2023-04-04 19:02:46 +05:30
Zuul 7dd58d6f75 Merge "Add vanity names to telemetry polling notifications" 2023-03-24 15:31:15 +00:00
Takashi Kajinami 9b47161121 [coordination] backend_url should be secret
The backend_url option can sometimes contain secrets.

For example when redis coordination backend is used and authentication
is enabled in redis, the plain redis password is put as an URL element.

[coordination]
backend_url=redis://:password@127.0.0.1:6379

Closes-Bug: #2012246
Change-Id: I9b61dd459445e0d32a305835fb05a072c32848a4
2023-03-20 18:36:10 +09:00
Yadnesh Kulkarni 2a5f63da95 Add vanity names to telemetry polling notifications
This change adds "project_name" and "user_name" fields
to polling samples which is related to the identification
of vanity names change 79454d6b22

Change-Id: I5fbe97439e7fadbdd8fd2641c49f1c88fbc416fc
2023-03-02 12:55:03 +05:30
Zuul 57c626015a Merge "Post individual JSONs instead of the whole list" 2023-02-21 09:50:15 +00:00
OpenStack Proposal Bot e83a6802ca Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I811d00ac0995980b4f6ab17b580f164d956d0bfe
2023-01-27 03:52:43 +00:00
Zuul 80909b4b24 Merge "Add TCP publisher" 2023-01-20 09:54:10 +00:00
jwysogla ed55b5f4a4 Add TCP publisher
This commit adds a TCP publisher. The publisher works similarly
to the UDP publisher.

Change-Id: Iac662018039e74ad59ac9c7fa4db994da540ef2f
2023-01-13 18:09:05 -05:00
Zuul 39f3721209 Merge "NoUniqueMatch: ClientException on Gnocchi publisher" 2022-12-21 12:16:31 +00:00
Zuul 2686aeb695 Merge "Change oslo_cache implementation" 2022-12-21 07:39:12 +00:00
Zuul 4815899038 Merge "Fix barbican credentials retrieval" 2022-12-20 13:37:36 +00:00