Importing some modules lead to registering config options that may
collide with config options from a project that calls the import. This
patch wraps the side effect that registers config options into a
function that needs to be called in case the caller wants to register
the options.
This solution is also not perfect as it guards the common options to be
registered only once even if the function is called multiple times. This
is to solve problems in unittests, ideally we should always call the
function just once even in our testing suites.
Resolves-Bug: #1968606
Change-Id: Ic1532eb8de887ff1b1085206df11f53e22f7f524
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
If a metering label is removed during a data collection and processing, and
error might happen. This patch address such situations, and creates an execution
flow that avoids such problems.
Change-Id: Iba68173c304dccf97521dcec9a87fce67e21cec2
Extend neutron metering agent to generate Granular metering data.
The rationale here is to have data (bytes and packets) not just in
a label basis, but also in tenant, router, and router-label, and tenant-label
basis. This allows operators to develop more complex network monitoring
solutions.
Moreover, I added documentation to explain what is the neutron metering agent,
its configs, and different message formats.
Change-Id: I7b6172f88efd4df89d7bed9a0af52f80c61acbe0
Implements: https://blueprints.launchpad.net/neutron/+spec/granular-metering-data
Closes-Bug: #1886949
Now that we use setproctitle for neutron-server workers (and
neutron-keepalived-state-change), this has the side effect of changing
the process name for agents, impacting some monitoring systems. More
details in launchpad bug.
This patch fixes it by setting the name with setproctitle to:
agent name (original process name).
Also use the newly introduced name constants to replace existing
hardcoded uses.
Change-Id: I74c3a4d3e9f833752571a75f196560cd45529385
Closes-Bug: #1881297
This change makes the metering agent retry creating the iptables
managers for each router and applying the metering rules.
This is needed in case the metering agent starts before some or all of
the namespaces are created.
Change-Id: Ifc565feb98c7f02df5c2831a3607c3e526a2e703
Closes-Bug: #1807153
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
NeutronLibImpact
Depends-On: https://review.openstack.org/#/c/647836/
Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
The neutron.common.rpc module has been in neutron-lib for awhile now and
neutron is shimmed to use neutron-lib already.
This patch removes neutron.common.rpc and switches the code over to use
neutron-lib's implementation where needed.
NeutronLibImpact
Change-Id: I733f07a8c4a2af071b3467bd710290eee11a4f4c
If the Metering agent fails to send its report_state
to the server, it logs an exception:
Failed reporting state!: MessagingTimeout: Timed out...
If it then tries a second time and succeeds it just
goes on happily. It would be nice if it logged that
it had success on the subsequent attempt so someone
looking at the logs know it recovered.
Change-Id: If5522e5a975e05f195f1760d7fbd60264ac91fc3
New releases of oslo.config support a 'mutable' parameter to Opts.
This is only respected when the new method mutate_config_files is
called instead of reload_config_files. Neutron delegates making this
call to oslo.service. This was provided in patchset
Icec3e664f3fe72614e373b2938e8dee53cf8bc5e
Further patches will be needed to make select config options be
marked as mutable. This change enables support for oslo provided
config options to be updated via SIGHUP such as log level.
Task: 6389
Story: 2001545
Change-Id: I9442965607f3248706464643c6d87a04edcae24e
The neutron.common.topics module was rehomed into neutron-lib with
commit Ie88b84949cbd55a4e7ad06341aab77b286cdc485
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.
NeutronLibImpact
Change-Id: Ia4a4604c259ce862597de80c6deeb3d408bf0e95
This problem will cause self.host to be overridden by cfg.CONF.host
instead of the passed host value when calling MeteringAgent.__init__(host).
Change-Id: I50c5136e1cf1eb0db28edd02129f963948a54741
neutron-lib contains the synchronized lockutils decorator as well as
the SYNCHRONIZED_PREFIX global. This patch consumes them from
neutron-lib and removes them from neutron.
NeutronLibImpact
Change-Id: I729da348e340509f2d09f8a6436716e2398f1583
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
Currently the metering agent is using the old import method,
use stevedore instead.
DocImpact
Two places in the networking guide should change to
'driver = iptables' from current format.
Partial-Bug: #1504536
Change-Id: I1e6d196a3ada8fbfc2b70d6a983984d8db09bbd0
Refactoring Neutron configuration options for agent common config to be
in neutron/conf/agent/common. This will allow centralization of all
configuration options and provide an easy way to import.
Partial-Bug: #1563069
Change-Id: Iebac0cdd3bcfd0135349128921b7ad7a1a939ab8
Needed-By: Ib676003bbe909b5a9013a3178b12dbe291d936af
When syncing data from neutron server, metering-agent may receive
information about routers that are not hosted by the l3-agent on the
same host, because the server didn't filter them out. This could lead to
the following problems:
* metering-agent tries to setup iptables rules for a router that is not
on the host
* metering-agent tries to get get traffic counters for a router that
was once on the host but is already removed
* metering-agent not sets up iptables rules for a router that is
removed then added back to the host, because nothing about the
router is changed from metering-agent's perspective
This commit fixes the aforementioned problems by making metering-agent
only receive information about routers that are on the same host, and
update metering-agent's knowledge about which routers it should care.
However, there could still be problem if one removes then adds a router
back to the same l3-agent, or just sets the router's admin_state_up
property to False then True in a short time(shorter than the interval
between two syncs). Because the metering-agent sees nothing changed
while during the same time the router's namespace is removed and added
back on the host. Thus metering-agent will fail to get such router's
traffic counters. This commit also make iptables-driver to forget such
routers and leave the metering-agent to reconfigure them later.
Closes-Bug: #1580548
Change-Id: Ia6ae82c676582b06710d6f96b9938c215258182d
Signed-off-by: Hunt Xu <mhuntxu@gmail.com>
Refactoring neutron services config opts to be in neutron/conf/services
so that all the configuration options for services reside in a
centralized location. This simplifies the process of looking up the
extension config opts and provides an easy way to import.
Change-Id: Iad255b020910ee54bc6cce25f4f786376dfe3705
Partial-Bug: #1563069
Currently the interval between two reports is wrong. For example, with
below setting in file metering_agent.ini
measure_interval = 30
report_interval = 600
The Ceilometer will get the reports from meter-agent per 630 seconds. The
reason is the test condition is wrong in the code, we should use >=
instead of >
add a unit test
neutron.tests.unit.services.metering.agents.test_metering_agent.
TestMeteringOperations.test_notification_report_interval
to test the report interval
Change-Id: Id5820f845d20bbe5fb5a9b8b9695499aa36c5d9e
Closes-Bug: #1569875
Python 3 deprecated the logger.warn method, see:
https://docs.python.org/3/library/logging.html#logging.warning
so we prefer to use warning to avoid DeprecationWarning.
Closes-Bugs: #1529913
Change-Id: Icc01ce5fbd10880440cf75a2e0833394783464a0
Co-Authored-By: Gary Kotton <gkotton@vmware.com>
- This does NOT break other projects that rely on neutron.i18n,
as this change includes a debtcollector shim to maintain those
older entry points, until they can migrate.
- Also updates _i18n.py to the latest pattern defined by oslo_i18n
- Guidance and template are from the reference:
http://docs.openstack.org/developer/oslo.i18n/usage.html
Partially-Closes-Bug: #1519493
Change-Id: I1aa3a5fd837d9156da4643a367013c869ed8bf9d
This optimization is needed for big clusters with hundreds
of agents where the spike of activity may trigger a burst
of RPC requests that would prevent neutron-server from processing
agent heart beats in time, triggering resource rescheduling.
This will be further optimized by running dedicated RPC workers
for state reports processing.
Related-Bug: #1496410
Change-Id: Id86a1f962aaa4f64011d57ae55d240f890cca4f7
Fix a situation for the _purge_metering_info function
when the items will never be deleted from the metering_info.
Delete the metering_info dict and use the metering_infos instead.
Fix the problem with changing a dictionary during iteration.
Add the unit tests for the _purge_metering_info and
_add_metering_info functions.
Co-Authored-By: Yaroslav Isakov <yisakov@mirantis.com>
Change-Id: I9031a5f27ae6438ffd5c5a48b0cf5cdc6867eff3
Closes-Bug: #1490581
oslo.service has graduated, so neutron should consume it.
Closes-Bug: #1466851
Depends-On: Ie0fd63f969f954029c3c3cf31337fbe38f59331a
Depends-On: I2093b37d411df9a26958fa50ff523c258bbe06ec
Depends-On: I4823d344878fc97e66ddd8fdae25c13a34dede40
Change-Id: I0155b3d8b72f6d031bf6f855488f80acebfc25d4
Now we send all labels and rules per rule create/delete
and rebuild whole iptables chains.
In this patch we send only affected rule and create/
delete only this rule from iptables.
Change-Id: I58ebd8d810c62980c09a340ee1680be17c12b74a
Closes-Bug: #1400280
It's mostly a matter of changing imports to a new location.
Non-obvious changes needed:
* pass overwrite= argument to oslo_context since oslo.log reads context
from its thread local store and not local.store from incubator
* don't store context at local.store now that there is no code that
would consume it
* LOG.deprecated() -> versionutils.report_deprecated_feature()
* dropped LOG.audit check from hacking rule since now the method does
not exist
* WritableLogger is now located in oslo_log.loggers
Dropped log module from the tree. Also dropped local module that is now
of no use (and obsolete, as per oslo team).
Added versionutils back to openstack-common.conf since now we use the
module directly from neutron code and not just as a dependency of some
other oslo-incubator module.
Note: tempest tests are expected to be broken now, so instead of fixing
all the oslo.log related issues for the subtree in this patch, I only
added TODOs with directions for later fix.
Closes-Bug: #1425013
Change-Id: I310e059a815377579de6bb2aa204de168e72571e
This should remove the last bits of passing root_helper around
for execute(). Some tests store and pass the root_helper argument
outside of this, and have been left alone.
Depends-On: I1e3b64e5a1d6cff2aebc638710487bbdbdba61d4
Depends-On: I5d51fa5edc7f427d84d7ae49d3a4c972dca8ac37
Depends-On: I21c6e7795c8b2e8f87bbd2b00a8eb7cb215ece74
Depends-On: I6566e17a1ce07a3bebf2a1a3653ad7e7f397f0b9
Change-Id: I38c901c65fb0c8598589ec56d5afd50d2f882f4f
Partially-Implements: blueprint rootwrap-daemon-mode
Oslo project decided to move away from using oslo.* namespace for all their
libraries [1], so we should migrate to new import path.
This patch applies new paths for:
- oslo.config
- oslo.db
- oslo.i18n
- oslo.messaging
- oslo.middleware
- oslo.rootwrap
- oslo.serialization
- oslo.utils
Added hacking check to enforce new import paths for all oslo libraries.
Updated setup.cfg entry points.
We'll cleanup old imports from oslo-incubator modules on demand or
if/when oslo officially deprecates old namespace in one of the next
cycles.
[1]: https://blueprints.launchpad.net/oslo-incubator/+spec/drop-namespace-packages
Depends-On: https://review.openstack.org/#/c/147248/
Depends-On: https://review.openstack.org/#/c/152292/
Depends-On: https://review.openstack.org/#/c/147240/
Closes-Bug: #1409733
Change-Id: If0dce29a0980206ace9866112be529436194d47e
Mostly trivial import changes.
- oslo.i18n no longer provide install() method to inject _() into
globals(), so removed all calls to it;
- removed Babel from dependencies (it will now be grabbed by oslo.i18n);
- updated tox.ini to ignore import violations for oslo.i18n.
Change-Id: I6623d551f512fb7fe9bf35ee734ed6d4c6cbc287
This change enforces log hints use and removes debug level log
translation, modifications are validated through a hacking rule.
Validate that hacking rules apply to directories:
- neutron/services/metering
Change-Id: Ic763e81e8b06973013115af45c7c09a3e69c673b
Partial-bug: #1320867
Drop one more use of the RpcProxy compatibility class in favor of
direct use of oslo.messaging APIs.
Part of blueprint drop-rpc-compat.
Change-Id: I3c68aa246d88447f793aa9890f10f5043a41338d
We have git to track authorship, so let's not pad source files
with it as well.
A hacking check has been added for this. The value is N322.
Change-Id: Iab0b64d417e0bb41a6b455e2ac377deee64ec3ee
The cfg.CFG argument is ignored and misleading. This change removes it
and updates all callers.
Closes-Bug: #1355565
Change-Id: I2fcece85d1abe848e5c01125cfb62c02f2dcbc86
Most of this code will probably stay with us for quite some time, so
let's make things easier and consider them as our way of doing RPC.
blueprint oslo-messaging
Change-Id: Iaf353b23f9c54b82d1e02a6bd5a5960cec827c88
Jakub Libosvar