This removes the monasca integration. Monasca was recently
declared as inactive project. In addition the monasca related
unit tests were very verbose and they were outputing a lot
of errors. Lately one of the unittests even started failing
randomly.
Change-Id: I468079b2a790245bd682c80d0116fb0da60d0e7c
The openstack Ussuri and Victoria versions no longer support the
RHEL7/CentOS7 and python2 environment packages. Correct the
problems in the latest installation document for RDO.
Change-Id: Ie03815f25d3a9ec99506a9dc14810a6800d3c42c
Problem description
===================
The hardcoded pollsters are defined by namespaces, so they
are instantied based on the namespaces provided to the
'AgentManager'.
The dynamic pollsters, on the other hand, are always instantied,
independent of the provided namespaces.
Proposal
========
To allow operators to define in which namespaces the dynamic
pollster will be deployed, we propose to add a new configuration
'namespaces' in the dynamic pollsters yaml configuration.
This configuration will support a single entry or a list of the
namespaces that the pollster must be instantiated.
Change-Id: I39ba0c3dd312a0601e02f8cfcab7a44e585a8a7f
Problem description
===================
Some OpenStack APIs do not always return exactly the
same metadata for all resources. As an example, we
have the server API, which might not return the
'OS-EXT-SRV-ATTR:host' attribute; it depends on the
virtual machine status.
Therefore, if the operator configures to retrieve
the value of the 'OS-EXT-SRV-ATTR:host' attribute in
the response, when the VM is in the 'RUNNING' state,
it will work properly; however, if it is in 'ERROR'
or other state, it will throw a 'key not found'
error when we are collecting the metadata.
Proposal
========
To allow operators to skip the extra_metadata_fields
gathering based on the collected sample attributes. We
propose to add a new 'extra_metadata_fields_skip' in
the dynamic pollster YAML definition where operators
can define some rules to skip gathering extra_metadata
for some samples based on their attributes.
Change-Id: I40176328e1863283890870098418c0944a75bad9
Depends-On: https://review.opendev.org/c/openstack/ceilometer/+/852021
Problem description
===================
Today we have some hardcoded pollsters that are gathering
data from running virtual machines through libvirt or
different programs running in the compute nodes. However,
the Dynamic pollster definition does not support this kind of
operations to gather data, it only supports HTTP Rest
requests to collect data. Therefore, it is not possible to
use the dynamic pollster definition to create a YML based
pollster that runs and collects data from Libvirt in the
compute nodes.
Proposal
========
To allow host commands/scripts in the Dynamic pollsters,
we propose to add a new pollster definition using the
`os.subprocess` lib to run host commands to collect
Host/VMs data and store them in the configured backend.
This will provide more flexibility and make the
Dynamic pollsters able to be used in Ceilometer compute
instances as well.
Change-Id: I50b8dc341ce457780416b41d138e35f5a0d083b6
Depends-On: https://review.opendev.org/c/openstack/ceilometer/+/850253
Problem description
===================
The dynamic pollsters only support APIs that produce
JSON responses. Therefore the dynamic pollsters do not
support APIs where the response is an XML or not Restful
compliant APIs with HTTP 200 within a plain text message
on errors.
Proposal
========
To allow the dynamic pollsters to support other APIs
response formats, we propose to add a response handling
that supports multiple response types. It must be
configurable in the dynamic pollsters YAML. The default
continues to be JSON.
Change-Id: I4886cefe06eccac2dc24adbc2fad2166bcbfdd2c
A unicode prefix is meaningless in Python 3. Because now ceilometer
supports only Python 3, we can remove the prefix.
Change-Id: I7bc91be21df646d8bbc7793eec28a93179a3eefa
... and discovery/inspector plugins depending on the pollster. These
were implemented to gather metrics via SNMP daemon in TripleO-managed
deployment but these are no longer valid since Telemetry services and
Nova were removed from undercloud.
Change-Id: If9a6b695ba799c766314a88328ea8a779407acc0
... because it is of no use since neutron-lbaas was retired and was
replaced by Octavia. This feature was officially deprecated during
Yoga cycle[1] so we are ready to remove it.
[1] 8917c73964
Change-Id: Ic145c23cc0b0372ef78f4a45ffb084bec24936c3
Sometimes we want/need to add/gather extra metadata for the samples
being handled by Ceilometer Dynamic pollsters, such as the project
name, domain id, domain name, and other metadata that are not always
accessible via the OpenStack component where the sample is gathered.
For instance, when gathering the status of virtual machines (VMs) from
Nova, we only have the tenant_id, which must be used as the project_id.
However, for billing and later invoicing one might need/want the
project name, domain id, and other metadata that are available in
Keystone (and maybe some others that are scattered over other
components). To achieve that, one can use the OpenStack metadata
enrichment option. This feature is only available to OpenStack
pollsters, and can only gather extra metadata from OpenStack APIs.
This patch introduces a new option in the OpenStack Dynamic pollsters,
that enable operators to enrich sample's metadata with information that
can be captured in other OpenStack APIs such as Keystone, Nova, Neutron,
and many others.
Change-Id: I079bf26cf26fcbcf678dba10469516a1dcb52c0d
Gnocchi API coundn't be use if we don't install and configure uwsgi
or other HTTP daemon. And gnocchi install guide didn't describe
those.
Add the describe of installing uWSGI and the brief relationship
between uWSGI and Gnocchi API. And tell the reader that they can
also use Apache httpd and mod_wsgi, or any other HTTP daemon to
run Gnocchi API.
Closes-Bug: #1750933
Change-Id: Iba8c1fe7a64569de4ca366251c495731a8d7a779
This patch introduces a new option for the dynamic pollsters'
sub-system to define the request timeout for requests executed
by the dynamic pollsters. The default timeout value will be 30
seconds. If it (the timeout option) is set to `None`, this means
that the request never times out on the client-side. Therefore,
one might have problems if the server never closes the connection.
The pollsters are executed serially, one after the other; if the
request hangs, all pollsters (including the non-dynamic ones) will
stop executing.
This option is being introduced after a failure that we had when
using the dynamic pollsters; one of the APIs did not respond, and the
server did not close the connection of the request, which made
Ceilometer dynamic pollsters stop gathering data. To avoid such a
situation, we recommend the community review and merge this patch.
Change-Id: I4735f20d75d329ccafc797fe285d9fef579f593e
This issue is to allow operators to create expressions that can reference the
sample itself. Currently, the expression is allowed only for fields in the
sample, and not the sample itself.
We discovered this requirement when developing pollsters for Cinder;
the attribute `volume_image_metadata` is not returned in the API response when
it does not exist, therefore, we need a method to check if it exist before
trying to retrieve it.
Change-Id: I5d3eb04534afaeb556ddab4ef603a205745883b8
glance-registry service was deprecated in Queens release and has been
removed in Victoria [1].
[1] https://review.opendev.org/#/c/738671/
Change-Id: Ib33a3f49647d86fae826f416825a85433730b391
Do not include ceilometer.conf.sample since PDF building fails with the
large file. This fix PDF building of openstack-tox-docs job.
Change-Id: Idaf811f71bab0b49de5b68a6a69a7872cb481ffb
html_static_path doesn't exist and the following warning is present in
logs:
```
WARNING: html_static_path entry '_static' does not exist
```
These changes drop this config.
Change-Id: I5ca35d1658c8734dd74d1c4dd16a8516f16c39b6
This pull request enables OpenStack pollsters to configure
Ids(project, user, and resource) as the non-openstack pollster.
This is useful as some of the OpenStack APIs use other keys to represent
values such as project_id and user_id.
Change-Id: Id5ae1720ad714675722bc857a142d7bd667fe7ed
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems
Update Sphinx version as well.
Set openstackdocs_pdf_link to link to PDF file. Note that
the link to the published document only works on docs.openstack.org
where the PDF file is placed in the top-level html directory. The
site-preview places the PDF in a pdf directory.
Disable openstackdocs_auto_name to use 'project' variable as name.
Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.
openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.
See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html
Change-Id: Ieaaff8278e93f85bd711a340f9621ae18841e720
Make a few cleanups:
- Remove python 2.7 stanza from setup.py
- Add requires on python >= 3.6 to setup.cfg so that pypi and pip
know about the requirement
- Remove obsolete sections from setup.cfg:
* Wheel is not needed for python 3 only repo
* Some other sections are obsolete
- Update classifiers
- Update requirements, no need for python_version anymore
Change-Id: I93294c42a967ca19b9b04ead9cd636b52f1cc038
Some APIs split their responses into pages and each response has
a link to the next one.
To allow operators to deal with this kind of APIs, we propose to
extend the Dynamic pollsters to navigate through the API
responses and join all responses into a single one.
To enable it, the operator will need to configure the parameter
`next_sample_url_attribute` with a mapper to the response's next
page attribute.
Change-Id: Ida0a73d2964f192e6c63a6b7e8003ef2b52bd710
The initial idea for this feature comes from the `categories` fields that we
can find in the `summary` object of the RadosGW API. Each user has a
`categories` attribute in the response; in the `categories` list, we can find
the object that presents ain a granular fashion the consumption of different
RadosGW API operations such as GET, PUT, POST, and may others.
In that context, and having in mind that we have APIs with similar data
structures, we developed an extension for the dynamic pollster that enables
multi-metric processing for a single pollster. It works as follows.
The pollster name will contain a placeholder for the variable that
identifies the "submetric". E.g. `dynamic.radosgw.api.request.{category}`.
The placeholder `{category}` indicates the object attribute that is in the list
of objects that we use to load the sub metric name. Then, we must use a special
notation in the `value_attribute` configuration to indicate that we are dealing
with a list of objects. This is achieved via `[]` (brackets); for instance, in
the `dynamic.radosgw.api.request.{category}`, we can use `[categories].ops`
as the `value_attribute`. This indicates that the value we retrieve is a list
of objects, and when the dynamic pollster processes it, we want it (the
pollster) to load the `ops` value for the sub metrics being generated
Depends-On: https://review.opendev.org/#/c/694519/
Change-Id: I6ed4632f209ac51a07687476ca316212659d72bb
Signed-off-by: Rafael Weingärtner <rafael@apache.org>
This PR enables the use of python operation to transform the attributes that
are extracted in the JSON response that the Dynamic pollster handle. By
enabling operators to define transformations on the fly, we can provide even
more flexibility for the dynamic pollsters' system.
One example of use case is the RadosGW that uses <project_id$project_id> as the
username. With this implementation, one can create configurations in the
dynamic pollster to clean that variable. It is as simple as defining
`resource_id_attribute: "user | value.split('$')[0].strip()"`
The operations are separated by `|` symbol. The first element of the expression
is the key to be retrieved from the JSON object. The other elements are
operations that can be applied to the `value` variable. The value variable is
the variable we use to hold the data.
Depends-On: https://review.opendev.org/#/c/693088/
Change-Id: I9ee209410491b3f04259e1a5c62ac20461070ae1
Signed-off-by: Rafael Weingärtner <rafael@apache.org>
Takashi Kajinami