When rewriting data for compression, it's possible a file
<metric>_<splitkey>_<aggregation_method>_<granularity> is missing,
because of some data corruption. In that case the following traceback happens:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/gnocchi/storage/_carbonara.py", line 557, in process_new_measures
ignore_too_old_timestamps=True)
File "/usr/lib/python2.7/site-packages/gnocchi/carbonara.py", line 217, in set_values
before_truncate_callback(self)
File "/usr/lib/python2.7/site-packages/gnocchi/storage/_carbonara.py", line 551, in _map_add_measures
for aggregation in agg_methods))
File "/usr/lib/python2.7/site-packages/gnocchi/storage/_carbonara.py", line 675, in _map_no_thread
return list(itertools.starmap(method, list_of_args))
File "/usr/lib/python2.7/site-packages/gnocchi/storage/_carbonara.py", line 316, in _add_measures
oldest_mutable_timestamp)
File "/usr/lib/python2.7/site-packages/gnocchi/storage/_carbonara.py", line 249, in _store_timeserie_split
offset, data = split.serialize(key, compressed=write_full)
AttributeError: 'NoneType' object has no attribute 'serialize'
This patch makes the driver log a warning and return, so it just ignores
the failure and continues anyway.
Change-Id: I4f367b2418c8be0067746c88bcce74ca756acf4e
The Content-Type is not automatically set by WebOb anymore. That also
broke the CORS test as keystonemiddleware is broken.
This patch moves the test to gabbi. While not strictly identical – it
does not test Keystone anymore – it ought to be enough.
Change-Id: I0f23481d5f75694da23d05c9ef88005a0f2c27d7
Adding options from middleware not enabled by default is not a good
idea.
Change-Id: I3ed922c84e6d37f2424262d108e6bdc5a03e2858
(cherry picked from commit 14a7cbaa64)
oslo.db 4.15.0 breaks our gate because it assumes we use the
oslo.db provision module.
This have been fixed by Ie8c454528ce3aa816c04fbb4beb69f4b5ec57e9c
Also oslo.db now cleans the database resources. We was mocking self.db
to avoid that before because of the provision module dependencies.
But since this module is no more required, we can use the oslo.db
facility.
This change does that but keep compatibility with oslo.db < 4.15.0
Change-Id: I69f8ff9f702064e8fc5bf4018ebc6f3b2a8ea1a8
(cherry picked from commit f5794af695)
Because the api binary script is generated by pbr now, the api
startup parameters need to be adapted accordingly in devstack
script or the api will be unable to start if the devstack deploy
mode is 'simple'.
Change-Id: I72abc3a83a4a2cf993198b0b1ac98c79ee26ec56
Closes-Bug: #1626979
(cherry picked from commit b8c0c23dd5)
This adds a fallback method when unserializing measures so Gnocchi 3 is
able to read measures stored in Gnocchi 2 format.
Change-Id: Ic863ace3ac8df430e5de93f17a45f73398ed143f
(cherry picked from commit 0c66859f82)
If the coordinator fails to start (e.g. ToozConnectionError), the metricd
subprocess started by Cotyledon fails to start and raises an error. That means
Cotyledon will retry to spawn the process with no delay at all, spamming the
CPU with forking request.
This patches fatorize the coordination retrieval and connection code into one
function in gnocchi.utils, which makes sure both metricd and the Carbonara
based drivers leverage the same code to retry to connection to the coordinator
with some delay.
Change-Id: I83157c5fdb0a3e488a9b788d48d974de80219dbb
(cherry picked from commit 04917de585)
This reverts commit 84443ca4d4.
Upgrade with some storage drivers such as Swift consumes a lot of I/O with high
latency, so running in parallel improve speed.
Change-Id: I65106a5fbb8cd19235274ea55fb19fe2ecff9093
(cherry picked from commit c95b98f765)
new oslo.config enforces type. it should give warning but we uuid
type doesn't give ValueError or TypeError, which are caught. rather
it gives AttributeError
Change-Id: Ib8fe3f0e6f37f2ca5c0785e73b2c770b71fac9e6
Current implementation based on pandas.Timestamp can't handle keys that
go further than 2^32 seconds after epoch, which makes e.g. archive
policies with very high granularity failing.
Change-Id: Idb81345544cc25e36447473e5115d9d856766c83
(cherry picked from commit 9a95873f40)
This sets up the HTTPProxyToWSGI middleware in front of Gnocchi. The
purpose of thise middleware is to set up the request URL correctly in
case there is a proxy (For instance, a loadbalancer such as HAProxy)
in front of Gnocchi.
So, for instance, when TLS connections are being terminated in the
proxy, and one tries to get the versions from the / resource of
Gnocchi, one will notice that the protocol is incorrect; It will show
'http' instead of 'https'. So this middleware handles such cases.
Thus helping Keystone discovery work correctly.
The HTTPProxyToWSGI is off by default and needs to be enabled via a
configuration value.
Change-Id: Ic5526cf37e70335fa2cc70946a271253f227f129
Closes-Bug: #1590608
(cherry picked from commit 2b1ed8d452)
The query argument used was mistyped, and the results were actually 'mean' and
not 'min' in the end. Same in doc.
Change-Id: Iac5683eecc923a4150e93ecb23cff343faad988b
Related-Bug: #1630306
Last gabbi version checks that data have the right content-type.
In theses tests we say that data in plain/text and we put json that's not
correct, old version was silencly ignored and now this is correctly
checked.
Same when we put data but no content-type.
Since our tests are wrong, this change fixes them.
Change-Id: I9a60db296467b08ef32f956471ef5e4f81f462af
(cherry picked from commit 35c4a08dd4)
Gnocchi actually uses oslo_log.log.get_default_log_levels which is provided by
oslo.log starting at 2.3.0.
Change-Id: Idd01d577a0b85a6603f5c69f783c951c7a0bd438
(cherry picked from commit b36b66eedf)
gnocchi-metricd does not shutdown gracefully because we block on
metric queue.get(). This change adds a timeout to allow the process
to terminate every 10 seconds.
Change-Id: Id7917464009b19a0e27ff84694803f97f43cfac7
If you run more than one tooz watcher at the same time, we might end up
updating block_size and block_index in parallel, causing incorrect
computing of the block to grab.
It's easier and simpler to just run the watchers serially.
Change-Id: I1d7f4f725a1c62c7511af6189d8deaebf836f31d
The variable 'self.previously_scheduled_metrics'
initially is Set type, afterwards it is assigned
a List type data ('metrics'), this causes the
error:
2016-09-20 17:14:27.919 28772 ERROR gnocchi.cli TypeError: unsupported
operand type(s) for -: 'set' and 'list'
Change-Id: Iec5ab651d82fdd8d058f8a3232a6cd93df1e3cf5
Closes-Bug: #1625529
- is not allowed in extra name, and are silently ignored.
We also rename them to make them the extra purpose clearer.
Change-Id: I4470050e1a196577e2c667ce7c2949d82655f77c
On PostgreSQL, migration scripts are run within transaction, so things
can be already committed or not. Adding value on an Enum already commit
in PostgreSQL won't work within a new transaction, so we have to delete
the old enum and create a new one.
Closes-Bug: #1624504
Change-Id: I8227021bc0e3dc63d1a28512fe7c610da4c6dc63
If computation is really fast elapsed can be zero.
Tests something fail on that with:
File "gnocchi/storage/_carbonara.py", line 530, in process_new_measures
* computed_points['number']) / elapsed)
ZeroDivisionError: float division by zero
This change fixes that.
Change-Id: I350db83fb90566f985f8555fcb6f2d077bb806d4
OpenDev Sysadmins