Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: I8ac5e339700316b07b7c548f24ffc8dd83110df3
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.
Fix problems found.
Change-Id: Ica99c9a4affd2b693ef74f1e3ef7c8ea2c6a46d6
There have been a few changes in ceilometer repository, specifically
https://review.openstack.org/#/c/596378/, which warrant an update
post-Rocky.
Change-Id: I9e127e0976be4c44bd56a977a755564a88fc1b44
As of Queens the Ceilometer V2 API has been removed. Storage
driver related code in monasca-ceilometer can thus be removed.
Backport some refactoring that came up in creating
https://review.openstack.org/#/c/562400/
Realized some class names could be clearer and fit better with
Ceilometer.
Update test-requirements.txt with Rocky release of
python-monascaclient. Also include more license info.
Specify py35 instead of py34 in tox.ini (gates already using py35).
Include updates to devstack configuration to keep up with master
ceilometer. These include adding values for the [monasca] section
in ceilometer.conf and refreshing setup.cfg as well as updating
how the connection url to Monasca API is configured.
Also include some testing notes in devstack/README.md
Change-Id: I9291dfc06d67f18109a0ff66184bd418c6361977
I encountered a bug whith the following configuration in
monasca_field_definitions.yaml:
metadata:
cpu:
- flavor_id: flavor["id"]
- flavor_name: flavor["name"]
The problem was in _get_value_metadata_for_key(): sometimes keys
from ceilometer samples can be unicode.
Change-Id: I8914c2238f3a450393f07e6309f57451c8c45cf2
Following changes are necessary to keep monasca-ceilometer
in sync with Ceilometer upstream
* Remove api/health.py (since ceilometer v2 api was removed)
* Remove storage/impl_monasca.py (since ceilometer storage drivers were removed)
* Updated opts.py and service.py (to keep in sync with ceilometer master)
* removed api, storage driver tests
* updated /etc/ceilometer/ceilometer.conf with new [monasca] options
Change-Id: I044f3e512db52a128bac8d5c5dcac76549a3b1c2
Ceilometer in Ocata made several changes to
move away from oslo config's global conf.
Instead ceilometer/service.py now creates a
oslo config object that gets passed to the
publisher.
To keep up with the changes had to make
the following changes in monasca-ceilometer
1.) Monasca Publisher now extends
ConfigPublisherBase, with conf as argument
2.) monasca_client now takes conf as an
argument during intialization.
3.) Added monasca_ceilometer_opts to
intialize all ConfigOpts. This will have
to be regiested in ceilometer/opts.py.
(will need a additional line in opts.py)
4.) Introduced three new properties
service_username, service_password and
service_auth_url since username, password
and auth_url were removed from
[service_credentials] section.
5.) Added api/health.py, service.py
and opts.py from ceilometer code base
which were modified to read
monasca ceilometer options via
monasca_ceilometer_opts.py
6.) Added ostestr to test requirements
and updated tox.ini to run tests in
serial (to avoid problems with
singleton mapping classes, stepping
on each other test values when
run in parallel)
List of Ceilometer changes to remove
global conf
[1] https://review.openstack.org/#/c/384834
[2] https://review.openstack.org/#/c/384624
[3] https://review.openstack.org/#/c/386025
This change also modifies test-requirements.txt
to bring this master branch up to the stable/pike
version of ceilometer, and python-monascaclient
up to 1.7.1 (pike).
Change-Id: Ieb7962a50fdb5d12ea6238c96ed116e8b2c83c48
To enhance flexibility when mapping a ceilometer sample to the
monasca fields, add support for jsonpath parsing of metadata.
For example, the cinder samples have metadata stored in lists,
which the current parsing (including . notation) cannot handle.
Similar requests have been made in the past, and this should
give enough power to meet most requests.
For a jsonpath that does not resolve to a simple leaf (string
value), a CeiloscaMappingDefinitionException is raised.
See https://storyboard.openstack.org/#!/story/2000954
Change-Id: I8d7f5e1b87a1de6078b4d397e96dab624fde42bb
If enable_api_pagination is turned on
ceilometer statistics api call with group by
did not return any results. This review fixes
problem with indentation. Also fixing similar
problems with metric_list and measurement list
calls in monasca client.
Also added unit tests to test metrics list,
measurement list and statistics list when
pagination is enabled and also when disabled.
Change-Id: I2394da8c0c32d8516ac8136fcaaef87e546966b0
Adding capabilities to ceilosca to be able to query metrics that
are already being collected by monasca using mappings files
there are two types of mappings:
1. Static Mapping: Currently it is used to map any static info
about ceilometer meters like the type of meter or unit
2. Extensive monasca mapping: currently it is used to map any
monasca metric to a ceilometer meter example:
meter_metric_map:
- name: "disk.device.write.requests"
monasca_metric_name: "vm.io.write_ops"
resource_id: $.dimensions.resource_id
project_id: $.dimensions.tenant_id
user_id: $.dimensions.user_id
region: "NA"
type: "cumulative"
unit: "request"
source: "NA"
resource_metadata: $.measurement[0][2]
As you can see for this mapping fields on left side of ":"
are used to map ceilometer fields and fields on right side
for referring to monasca fields
Both of these mapping files are configurable and can be set in
ceilometer configuration file
cloud_name, cluster_name, and control plane fields
are now added as dimensions by monasca publisher when
publishing metrics to monasca api which is necessary
in multi-region deployment of notification agent.
monasca publisher does not use oslo.service LoopingCall
but instead uses futurist periodics library to enable
batching.
monasca client now pages through monasca api results
if enable_api_pagination is enabled in configuration.
This flag is disabled by default but should be enabled
if monasca api supports paging using "offsets"
parameter.
Tox testing is now targeting stable/newton branch of
ceilometer.
DocImpact
Change-Id: I83b96325cb79d82858cf529935e5d699a509f6c3
In some part in the code we import objects. In the Openstack style
guidelines they recommend to import only modules.
http://docs.openstack.org/developer/hacking/#imports
Change-Id: Ibdb1fc5b52b07ab9098dd9a72aecd938d1515132
- Leverages monsaca devstack and uses new devstack plugin model
to setup ceilosca.
- Also some refactoring in directory structure
Change-Id: I097c86f92e1ec09ceeb3d0d4831ff8c51c3aa1a3
When available metric list should include the timestamp for metric
list query as well to have better performance in avg case.
Change-Id: I5200e90454c419d24f554c69bcfa3c8c97c0353d
Complex query computes the single complex query to multiple simple
queries which only use AND operator, each simple query will invoke
get_samples() method. Complex query collects all the results till
limit constraint is satisfied.
Change-Id: I0da398d4d3627fcbfe2c686acf5b0fa9a68492b7
Implements get_resources and get_samples using
iterative approach with limit, monasca measurements
api accepts a limit, but cannot be used here since
user can query for measurements from different metrics.
Change-Id: I87d070c60af3435d20917ffc5d07adf8ed749ee2
Queries for metrics to obtain all dimensions
before quering for statistics with merge_metrics
as False, which returns response with dimensions.
The statistics response with dimensions is used to
perform in-memory computations based on the groupby
dimension. Only a single groupby is supported.
The current implementation is memory-intensive and
can be improved further to constrain memory usage.
This patch is an initial step in that direction.
Change-Id: I98ad72337ed50b2f6703afc2e7c4e4689590b2c7
When statistics API is used with custom
aggregates as input, there is an attribute
error in client as it expects an aggregate
dict in response.
This adds the aggregate dict in response
when input has an aggregate function.
Change-Id: Ide13f5d16b6e4ef11cdcb9c98d85200efaf0cb91
When timestamp/timestamp range is specified,
statistics API fails because timestamps are not
being converted to ISO 8601 formatted times, also
when parsing the response, the timezone offsets are
not being dropped.
Monasca API expects timestamps in ISO 8601 format
and ceilometer statistics API expects timestamp in
response in a timezone-naive format.
This patch fixes that so that statistics API works
when timestamps are specified in input filter.
Change-Id: I67055a661478fc15a06c3714f431c484f303658c
List of changes:
* Changed oslo-incubator imports to oslo-service
* Refactored tests to functional and unit packages
* Upgraded test requirement versions
* Added py34 test capability
* Upgraded hacking(pep8) version
* Added devstack plugin and settings to deploy Ceilosca
* Modified setup.cfg to use liberty entry points
* Refactored storage driver to compatible with v2 api liberty changes
Change-Id: Id6711e6661cde14df3decba6f56e52d391981f42
Monasca has added a check for null data. When the publisher does not
find the meter value or metadata it defaults it to Null.
This causes Monasca API to reject the sample. This is particularly
evident in the Central and Compute Agents since they collect data
that may be null.
Change-Id: I51ca07ec10c6b414b6a26c057edc09487053f982
Instead of sequentially querying for
measurements of a metric with different
dimensions, this patch queries them
at same time using mult-tasking.
Change-Id: Ief0d588ed1cc6d6ed0d1a198e888d545c32c188c
The batching is disabled by default, but we really want to show
how the feature improve performance and I think we should have
it on by default. The batch size has been changed to 1000 also to
be more performant.
The batch timeout has been reduced to 15s.
Change-Id: I276f0e3fa20927a9ccbbedc36238c88c753dab57
These scripts allow installtion of ceilosca on a local machine.
- Sets up devstack
- Sets up ceilosca, modifies config and copies files
- Sets up monasca on the same local box
Necessary information is in deployer/README.md
Change-Id: I2da63ee7075cee5401594408d44dd8f5881d9fe6
Currently, monasca does not return metadata for get meters and statistics,
so metaquery for meters and statistics cannot be supported in ceilosca yet.
Change-Id: I2772e936a6b4aa928048c4fb916a036b9f3df61c
When publishing metrics to Monasca-api fails, the
failed metrics need to be handled in some manner,
otherwise risk data loss.
Currently, Ceilometer notification agent does not
support requeuing of samples on publisher error.
This patch adds the ability to enable metrics
archival (via conf) using FilePublisher in Ceilometer.
This ability is switched off by default.
Change-Id: I482d4466ae106e5292d12a768e8acd4a240f4049
API unit tests are enabled by copying the
ceilometer config files in venv and using
test instance of Stevedore which allows
control over building extensions using custom
entry points.
Change-Id: I601c301d7e0353392498b374a8d6e4741487dac7
- Force pep8 to use correct python interpreter
- Add py27 checks
- Use smaller requirements file for pep8 checks
Change-Id: I4a2a642f60c17986739355187c67f403febb79fc
Martin Chacon Piza