It appears that the default metrics.yml needs force_granularity: 300
otherwise cloudkitty-processor will fail to process metrics from
Gnocchi. As this is what at least I (in Debian) ship in packages, it
would be nice to have a working default.
Change-Id: I28b43d6131182252c0576d9323bbcdbfb1dbbd0a
Story: 2008598
Task: 41771
This is useful for operators to setup haproxy and know that one of
the API is alive. I've done such a patch on many other projects,
and this is unifying all.
Change-Id: I14214679849c1cd378c7487254761f1d4d73e10e
This adds a v2 API root to CloudKitty. An example endpoint along with some
developer documentation has been added. The API root is now served with Flask
instead of pecan.
Change-Id: I4ede52dae531631f7fe708400def01bc08a64dec
Story: 2004208
Task: 27717
Task: 27718
Task: 27719
Work items:
- Added "user_id" to default groupby attributes
- Replaced "flavor" metadata by "flavor_name" for instances (it is the
default attribute name in gnocchi)
- Replaced network.incoming/outgoing.bytes by
network.incoming/outgoing.bytes.rate. The previous metrics were cumulative,
whereas the new ones are rates.
Change-Id: I55271bfbdd5a3457bea249dbefefd93a78ee057e
This updates various parts of the devstack plugin:
* The fetcher is now configurable via the "CLOUDKITTY_FETCHER" variable and
defaults to gnocchi (this allows the user to have immediate results).
* The "CLOUDKITTY_SERVICES" variable was removed as it is not used anymore.
* The storage backend does now default to "influxdb". If this storage backend is
selected, influxdb is installed. This can be done on fedora and ubuntu. The
storage backend to use in devstack can be configured through the
``CLOUDKITTY_STORAGE_BACKEND`` and ``CLOUDKITTY_STORAGE_VERSION`` variables.
* Some details about available variables have been added to the devstack
documentation. Since the "admin/quick_deployment" section did only contain
the devstack documentation, it has been removed for now.
* Given that the "ceilometer-low" archive-policy (default in devstack) only
provides the "mean" aggregation method, it is now the aggregation method
used in the default metrics.yml file.
Change-Id: I37452772de163b5fafc502917af870c86a3d38b2
This patch provides a refactoring of the metric
configuration model (and file description) to
improve genericity, maintainability and usage for
non-openstack deployment.
The new metric yaml format is defined in the
attached story task and is validated on load with
voluptuous.
Now, a processor is dedicated to one collector and
one storage backend. Thus, collector and storage
configuration go back to the cloudkitty oslo conf.
Collectors have been refactored to have a code as similar as possible,
in order to ease comprehension for new contributors.
Story: 2001883
Task: 14354
Task: 14355
Task: 14431
Change-Id: I948dd9cd5c113bdaa4e49c532354938ffb45f0e7
* Add a SourceFetcher, which allows to retrieve new type of data sources.
It allows new collectors to be added.
(examples: Prometheus, Influxdb, Kubernetes...)
* Base Cloudkitty on metrics instead of services and resources
for collecting metrics. This new architecture allows Cloudkitty
to be more agnostic and so to rate containers metrics as the same way
as virtual machines metrics.
* Centralize metrology information in metrics.yml under metrics names.
Task: 6291
Story: 2001501
Change-Id: I00ca080cf05dfc03a3363720f85b79e003eda9be
Allow users and operators to control the conversions units,
by filling source and destinations units for the metrics.
Differents units can now be affected to metrics from a same resource.
Task: 6077
Story: 2001396
Change-Id: I79713063d530737df20d4d910afc7c7e489aa2c5
This patch add the service "radosgw.usage" to collectors
in order to allow rating on ceph object storage usage per project.
The service unit is set to GB.
The following work has been done:
- Add radosgw.usage transformer for ceilometer
- Add radosgw.usage collect function to ceilometer and gnocchi (new
and deprecated format)
- update doc
Change-Id: Ia49c76a285950475ed0a9f15d35b2f23b11e5ef3
This patch introduces the implementation for registering
default policy rules in code. Default rules are defined under
cloudkitty.common.policies. Each API's policies are defined in a
sub-folder under that path and __init__.py contains all the
default policies in code which are registered in the ``init``
enforcer function in cloudkitty/common/policy.py.
This commit does the following:
- Creates the ``policies`` module that contains all the default
policies in code.
- Adds the base policy rules into code (context_is_admin,
admin_or_owner and default rules).
- Add policies in code for current APIs
- Add a tox env to generate default policy sample file
- Delete policy.json from repo as policies in code will be used.
Change-Id: I257e8cefc2b699fc979c717531cd9ba77233d94b
Implements: blueprint policy-in-code
To avoid source code updates with metrology conf update,
the metrology configuration is separated from Cloudkitty
configuration file and placed in a yaml one.
Task: 5724
Story: 2001215
Change-Id: Icc098c40bc52c2589e89d705d9d711d0ce2fb557
To reach Cloudkitty API endpoints, an authentification method must be set.
No authentification or Keystone authentification are allowed.
This should be set in the configuration file, within the DEFAULT block,
using the auth_strategy field.
Task: 4902
Story: 2001168
Change-Id: I3f9d5700e4c2d1a69b5514fd9932a62238ee659c
Recommands to setup cloudkitty through an other WSGI services
like Apache 'mod_wsgi'. And the community has set a community wide goal
in Pike cycle: "Control Plane API endpoints deployment via WSGI"
https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html
Work Item: Add WSGI support
1. Provide WSGI application script file.
2. Removing the cloudkitty-api command line.
3. Adding cloudkitty-api wsgi_scripts, by 'cloudkitty-api -p 8889' to run.
Work Item: Make the devstack setup ck-api with wsgi
1. Switch devstack jobs to deploy control-plane API services in WSGI with Apache.
2. Default to deploy with Apache by global ENABLE_HTTPD_MOD_WSGI_SERVICES, in
local.conf expose CLOUDKITTY_USE_MOD_WSGI=False to run without Apache.
Work Item: Update the docs about installation
1. Installing the cloudkitty-api behind mod_wsgi.
2. Updating the installation about the cloudkitty-api.
Implements: blueprint wsgi-support
Change-Id: I207587c5360bb80c0e856cd0239e4073578951aa
Sed s/manager_mappings/manager_mapping/g in policy file
to keep consistent with API
Closes-bug: #1661818
Change-Id: Ie23c3bc55f6c78028c86c6655d71de3330442347
Update policy of get_total/summary, admin user can
get rate info of all tenants and non-admin user can
get rate info of itself.
For example, get total api change as follows:
cloudkitty total-get
-- get total of current tenant(get tenant from context)
cloudkitty total-get -t tenant_id
-- get total of specified tenant if allowed(admin or owner)
cloudkitty total-get --all-tenants
-- get total of all tenants if allowed(admin)
Change-Id: I47dd5f310f18b7e5211165069692827047cdccd6
closes-bug:#1644399
Some of the option names in cloudkitty.conf.sample are not
correctly configured as well as some options are missed.
This patch also delete cloudkitty.conf.sample file from git so that we will
no longer maintain that file [1].
[1] https://review.rdoproject.org/r/#/c/3646/
Closes-bug: #1592219
Closes-bug: #1608970
Change-Id: I9e916bbd5a13652fb7e96f443ece91f03bf16126
Add `get summary api` to support that get total
price of each tenant and resource, also can order
by price
Change-Id: Iafab81ac539e70c958ed03460c165d950a2b022e
Implements: blueprint price-groupby-fields
This patch allow the user to query the API in order to:
* Get current collector configuration (period, services, etc)
* Get collected metadata list for a given service (using currently
configured collector)
For this, the following work have been made:
* Each transformer can now export available metadata for a given
resource (using a FakeData object and the strip_resource_data method)
* Each collector can now export information for a given resource
about metadata (using associated transformer) and unit
* A new REST controller 'info' provides API entrypoints to those
information:
* configuration with GET /v1/info/config
* all active services information with GET /v1/info/services
* given active service information with GET /v1/info/services/SERVICE
Change-Id: I02b1bc5709371785748661b63c5e6f0705ce891b
Implements: blueprint user-experience-improvement (PARTIAL)
This sets up the HTTPProxyToWSGI middleware in front of Cloudkitty.
The purpose of this middleware is to set up the request URL correctly in
case there is a proxy in front of cloudkitty.
Change-Id: I8bff0002be65720a268d39c25fb909b3d1187560
This adds the CORS support middleware to CloudKitty, allowing a deployer
to optionally configure rules under which a javascript client may
break the single-origin policy and access the API directly.
For CloudKitty, we used the same gabbi fixtures and harnesses created for
ceilometer. A custom gabbi-paste.ini was created so that the API tests run
against the full wsgi application, and appropriate fixtures and tests were
created to correctly simulate the configuration state.
Hooks to ensure that cloudkitty's required HTTP headers are represented
both in the runtime defaults, and in the automatically generated config
file, have also been added.
OpenStack CrossProject Spec:
http://specs.openstack.org/openstack/openstack-specs/specs/cors-support.html
Oslo_Middleware Docs:
http://docs.openstack.org/developer/oslo.middleware/cors.html
OpenStack Cloud Admin Guide:
http://docs.openstack.org/admin-guide-cloud/cross_project_cors.html
DocImpact: Add link to CORS configuration in admin cloud guide.
Change-Id: I3ef96ca6c78c5e369fb09425871d0a57bf15ad8a
Transitioned from deprecated oslo namespace to newer ones.
Updated requirements.
Updated configuration.
Change-Id: Ic395878a21564ceed6febe85179523efd9956eb9
Global improvement of the API.
Added support for policy engine.
Added new policy rights to manage the collector mappings.
Updated documentation.
Added contrib script to do simple API tests.
Change-Id: Ibb9dc1e0c7ead75922d777a669e0270b632c4631
Add a new auth_strategy option allowing to disable keystone auth to make
CK fully standalone if needed/wanted.
Update the sample config file with the new option.
Change-Id: I8926f1518382640395e34b4e5321d99f2bc2d7eb
Provide a default policy which requires to be admin for every API call except
rating/quote, report/total and storage/list_data_frames (which will require a
bit more control to return only the current tenant data for non-admin users).
Also provide a custom Context class to extract roles information from the API
request.
hashmap controllers now use a new base class which provides a custom
_route method. This will avoid to define policies for core and external
configuration APIs.
Change-Id: Ie3feb4e926270b95ab813a5d24854d1df1758a5e
Update the sample file.
Provide a simple test to make sure that we don't list removed options in
the sample file.
Change-Id: I9efa59d862f25f73d92ff852d4e9e7b29761e3e4
Right now every authenticated user can perform administration actions on
CloudKitty's API. This patch uses oslo.context and oslo.policy to enable
a policy engine.
Enforcement of the policy will be added in another patchset.
Change-Id: Ia88a37b259f12aee00f65876686d12411297b8fb
As CloudKitty is a rating and not a billing component, we needed to
rename all components and change paths on the API.
We've created a compatibility layer to help people with the migration.
The old API objects are now deprecated and shouldn't be used, they will
be removed in a near future.
Change-Id: I9e264f4ed01f4c94366eb51f612d634312b6a2f7
Move all the packaging related files to contrib/.
Rework the .spec file to make automated builds easier.
Change-Id: I96482fdb838f46f9dcb57c725f3029a1ccf428f9
Removed _connect abstract method as we can't assume the collector will
connect somewhere and should be handled by the collector himself.
Removed now useless state manager in orchestrator(handled by the storage
backend).
Kept the keystone instance in the orchestrator for future token
validation purpose.
Change-Id: I471ed1ad94345c83a637247d814a30b38743b100
Easy addition to see how extra collections can be added.
Did rewrite get_active_instances to active_resources.
Makes it more useful, plus gets rid of the unsafe prefix
if someone puts 'active_instances' in the conf.
Change-Id: Id78f3a786bc91a34aa9751c7bb3a4fcc276d1778
Updated gitignore to skip additional files.
Updated configuration.
Pinned WSME version to 0.6.1 as 0.6.2 is causing troubles with python
3.3 and later.
Updated tox configuration to integrate coverage and configuration
updating entries.
Change-Id: Ibf474cf978758e8d5e299e523fbc26687a6e4aa7
Added SQLAlchemy storage backend.
Modified Orchestrator to handle new storage backends.
write_orchestrator code should be considered legacy, it'll be removed
next versions. A new process cloudkitty-writer will replace this
feature.
Change-Id: I723371309e12754b2ebd58afab5bbf7a47d69704
This patch provides a .spec file (RHEL7 support only), systemd service
files and a logrotate configuration.
Change-Id: I103e14cab2e63a55aa6aed7441ac35986f546c48
API and processors now talk via oslo.messaging.
Updated openstack-common to add lockutils support.
Change-Id: I0b06a8c8b6612eda6a2ff8785a4f0d516347f09a
Sam Morrison