This patch allows CloudKitty to use InfluxDB v2 with Flux queries. This
type of query uses less CPU and RAM to be processed in the InfluxDB
backend.
Change-Id: I8ee3c92776aa69afbede353981a5fcd65dd7d099
Depends-On: https://review.opendev.org/c/openstack/requirements/+/895629
Story: 2010863
Task: 48539
After the commit
8778c64776
The module PBR in openstack started to validating the parameters when
creating an embedded WSGI, now if invalid arguments are given
to the cloudkitty-api it raises an error as we are facing in the
devstack when using `CLOUDKITTY_USE_MOD_WSGI=False`:
cloudkitty-api[86126]: usage: cloudkitty-api [-h] [--port PORT] [--host IP] -- [passed options]
cloudkitty-api[86126]: cloudkitty-api: error: unrecognized arguments: --config-file=/etc/cloudkitty/cloudkitty.conf
This PR also extracts the upgrade database workflow to a function
to be used in grenade tests
Change-Id: Ifc1a8563a9efcae2abaa6f8eb036405a93ff296d
To facilitate the switch from Elasticsearch to OpenSearch, the ES
backend has been duplicated and renamed where appropriate to OpenSearch.
The OpenSearch implementation was modified in places for compatibility
with OpenSearch 2.x, for example:
- remove mapping name from bulk API URL
- replace put_mapping by post_mapping
This will allow for the future removal of the Elasticsearch backend.
Change-Id: I88b0a30f66af13dad1bd75cde412d2880b4ead30
Co-Authored-By: Pierre Riteau <pierre@stackhpc.com>
DevStack is moving to adopt VENV to manage its python versions.
However, CloudKittty DevStack integration was not using the
DevStack VENV variable. This, in turn, causes issues with tempest
tests, as they are based on a DevStack deployment.
We need to merge this patch to fix the tempest tests.
Change-Id: I17de617557fb86c002814941325d71e3c08e0e72
Change DevStack plugin to deploy OpenSearch 1.x instead of Elasticsearch
when using the elasticsearch storage driver. This is in preparation for
supporting OpenSearch instead of Elasticsearch. Note that CloudKitty
does not work with OpenSearch 2.x yet, likely due to the removal of
mapping types [1].
[1] https://opensearch.org/docs/2.0/breaking-changes/#remove-mapping-types-parameter
Change-Id: I572cba384a9a16ee2bbfe07b5c316ed5abbb5681
This adds support for the elasticsearch v2 storage driver to the devstack
plugin. Two non-voting check jobs running a the tempest test suite
against a devstack deployment using the elasticsearch driver have also been
added.
Story: 2006332
Task: 36078
Task: 36080
Change-Id: Ib531f83e2c8e5c2494b36af250d7630e32f49f40
This is done in order to get quicker results when creating a devstack.
Production deployments should keep the default value.
Change-Id: I899311ba1d87f23133b650567a63863005caf56c
This updates various parts of the devstack plugin:
* The fetcher is now configurable via the "CLOUDKITTY_FETCHER" variable and
defaults to gnocchi (this allows the user to have immediate results).
* The "CLOUDKITTY_SERVICES" variable was removed as it is not used anymore.
* The storage backend does now default to "influxdb". If this storage backend is
selected, influxdb is installed. This can be done on fedora and ubuntu. The
storage backend to use in devstack can be configured through the
``CLOUDKITTY_STORAGE_BACKEND`` and ``CLOUDKITTY_STORAGE_VERSION`` variables.
* Some details about available variables have been added to the devstack
documentation. Since the "admin/quick_deployment" section did only contain
the devstack documentation, it has been removed for now.
* Given that the "ceilometer-low" archive-policy (default in devstack) only
provides the "mean" aggregation method, it is now the aggregation method
used in the default metrics.yml file.
Change-Id: I37452772de163b5fafc502917af870c86a3d38b2
In order to respect the convention [backendtype_backendname]
for all collectors and storage backends in Cloudkitty configuration
some sections names has changed.
* For the Gnocchi collector, [collector_gnocchi] has been added
in addition to [gnocchi_collector].
This last one will be removed in the future.
* For the hybrid storage, [storage_hybrid] has been added
in addition to [hybrid_storage].
This last one will be removed in the future.
* For the keystone fetcher, [fetcher_keystone] has been added
in addition to [keystone_fetcher].
This last one will be removed in the future.
* For the hybrid storage, [fetcher_source] has been added
in addition to [source_fetcher].
This last one will be removed in the future.
* Fix Prometheus section name.
Change-Id: I3435a151de4b384f2f2540225504abe6c99b928e
Story: 2001053
Task: 4654
This adds a v2 storage interface as well as a gnocchi backend for it to
cloudkitty. This new interface will allow us to implement the v2 API, which will
be more metric-oriented.
This new storage uses the new dataframe format ('desc' is now split up into
'groupby' and 'metadata', following the collect logic); and supports grouping,
filters and pagination.
Modifications:
* The tenant/scope state management isn't handled by the storage backend
anymore. A StateManager object has been added.
* All internal interactions with the storage backend use the v2 interface.
An adapter class has been added to ensure retrocompatibility with the
v1 interface.
* A base class for functional tests testing the v2 storage interface has been
added. It contains generic tests, which should allow easier testing for new
v2 backends.
* Some developer documentation explaining how to implement a v2 storage
backend has been added. User documentation will be updated once the v2
storage backend is considered stable.
The v1 is still the default storage version. Note that this new version is
considered unstable and should only be used for upstream development.
Change-Id: I5e9b95c79292582fab3017289d35ee310e35ffea
Story: 2001372
Task: 6585
Task: 14361
Task: 24535
This adds the CLOUDKITTY_STORAGE_BACKEND option to CloudKitty's devstack
plugin, allowing users to choose the storage backend they want to use.
Change-Id: I442131cfbbb6f6229a5e8e2a9efe4a4b33338ac9
CloudKitty's ceilometer collector and Gnocchi's devstack plugin do not exist
anymore, so the documentation has been updated accordingly.
Change-Id: Ief49cffae2a7c5a5a292eae5850ef10e21f275bb
As announced during the Queens cycle, all ceilometer-collector
related code has been removed.
Change-Id: I03f9a89ae4bc970fbc421dd6964be95cdacfe758
Task: 6293
Story: 2001503
TEMPEST_SERVICES global variable is not supported
by devstack since long back.
- I380dd20e5ed716a0bdf92aa02c3730359b8136e4
- I9c24705e494689f09a885eb0a640efd50db33fcf
Service availability of tempest known services will be
set by devstack with local check.
- I02be777bf93143d946ccbb8e9eff637bfd1928d4
cloudkitty does not have tempest plugin so it does
not need to have service register in tempest.
This commit removes the TEMPEST_SERVICES setting.
Change-Id: Ica3d142ad1c95379771fc1f045dd7e80ca0452a5
Related-Bug: #1743688
This patch introduces the implementation for registering
default policy rules in code. Default rules are defined under
cloudkitty.common.policies. Each API's policies are defined in a
sub-folder under that path and __init__.py contains all the
default policies in code which are registered in the ``init``
enforcer function in cloudkitty/common/policy.py.
This commit does the following:
- Creates the ``policies`` module that contains all the default
policies in code.
- Adds the base policy rules into code (context_is_admin,
admin_or_owner and default rules).
- Add policies in code for current APIs
- Add a tox env to generate default policy sample file
- Delete policy.json from repo as policies in code will be used.
Change-Id: I257e8cefc2b699fc979c717531cd9ba77233d94b
Implements: blueprint policy-in-code
To avoid source code updates with metrology conf update,
the metrology configuration is separated from Cloudkitty
configuration file and placed in a yaml one.
Task: 5724
Story: 2001215
Change-Id: Icc098c40bc52c2589e89d705d9d711d0ce2fb557
When using mod-wsgi with devstack, apache is configured and restarted just
after cloudkitty-processor is started, causing a ConnexionRefused error when
ck-proc tries to authenticate against keystone and making the devstack fail.
This adds a sleep to avoid this issue.
Change-Id: I080a364be2f783610252a40dc70f2eb5babd1467
* Horizon is enabled by default, so there is no need to enable it.
* The Gnocchi plugin should be enabled
* The CloudKitty services need to be enabled
Change-Id: I2b0bac8c8c62479a4701f66efc2703e12ecbaaa3
To reach Cloudkitty API endpoints, an authentification method must be set.
No authentification or Keystone authentification are allowed.
This should be set in the configuration file, within the DEFAULT block,
using the auth_strategy field.
Task: 4902
Story: 2001168
Change-Id: I3f9d5700e4c2d1a69b5514fd9932a62238ee659c
Recommands to setup cloudkitty through an other WSGI services
like Apache 'mod_wsgi'. And the community has set a community wide goal
in Pike cycle: "Control Plane API endpoints deployment via WSGI"
https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html
Work Item: Add WSGI support
1. Provide WSGI application script file.
2. Removing the cloudkitty-api command line.
3. Adding cloudkitty-api wsgi_scripts, by 'cloudkitty-api -p 8889' to run.
Work Item: Make the devstack setup ck-api with wsgi
1. Switch devstack jobs to deploy control-plane API services in WSGI with Apache.
2. Default to deploy with Apache by global ENABLE_HTTPD_MOD_WSGI_SERVICES, in
local.conf expose CLOUDKITTY_USE_MOD_WSGI=False to run without Apache.
Work Item: Update the docs about installation
1. Installing the cloudkitty-api behind mod_wsgi.
2. Updating the installation about the cloudkitty-api.
Implements: blueprint wsgi-support
Change-Id: I207587c5360bb80c0e856cd0239e4073578951aa
Since [1], systemd is the default process init. Therefore,
CK devstack plugin still uses screen_it function instead of
generic run_process function. This leads to strange behavior
of the plugin.
This patch fixes this problem.
[1] https://review.openstack.org/#/c/461716/
Story: 2001047
Task: 4634
Change-Id: I57c71c85f2a31cd90afeaf7f32754f1f1a503753
A recent patch in devstack [1] forces the use of path access instead
of port for Keystone authentication, leading to connection error in CK
when launching the devstack.
Now CK devstack plugin uses $KEYSTONE_SERVICE_URI to support now and
future keystone devstack modification.
[1] https://review.openstack.org/#/c/456344/
Change-Id: I7ef964185b029036e7ea7abfb555162e9f2d6ec6
Actually use CLOUDKITTY_COLLECTOR to setup the ceilometer/gnocchi
collectors.
Update the documentation accordingly.
Change-Id: I7308c0597c0ae9dac435cf1c0cc2e3b4c07ba063
The plugin.sh file contained a reference to the localrc configuration
file, but devstack is now configured by a local.conf file.
Change-Id: I37a73b9ef1dc1f8f1a6f640f65240997e132e9fd
ck-api log complains:
Option "verbose" from group "DEFAULT" is deprecated for removal.
Its value may be silently ignored in the future.
If this option is not set explicitly, there is no such warning.
Furthermore, the default value of verbose is true, so there is
no need to set this value in devstack plugin.
Change-Id: I0dbf2ec729482883c87a89f2a13a406ff4998754
Static files needs to be compressed in Horizon. Since CloudKitty is
loaded as a devstack plugin its files are copied after Horizon
initialization. This patch add calls to django-admin to process new
files.
Change-Id: I51b703f682b74a35a1fb6cad64856093011ee2e2
Users using the cloudkitty devstack plugin want to use cloudkitty.
Because of that it makes no sense to not enable the cloudkitty services
provided by this devstack plugin by default.
Change-Id: I7188397233bcc43df39a66d407d92f6a844f374c
Pedro Henrique