Trivial fix
MeteringPlugin has the meter_rpc initialization, and the
MeteringDbMixin never used that.
Change-Id: Ieabb10b25a1c455937d61ee2426ebfdb2ce603d9
Extend neutron metering agent to generate Granular metering data.
The rationale here is to have data (bytes and packets) not just in
a label basis, but also in tenant, router, and router-label, and tenant-label
basis. This allows operators to develop more complex network monitoring
solutions.
Moreover, I added documentation to explain what is the neutron metering agent,
its configs, and different message formats.
Change-Id: I7b6172f88efd4df89d7bed9a0af52f80c61acbe0
Implements: https://blueprints.launchpad.net/neutron/+spec/granular-metering-data
Closes-Bug: #1886949
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues. Didn't think it was going to be
close to 100 files when I started.
Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
The functionality within neutron.db.common_db_mixin is available via
neutron-lib APIs. This patch removes common_db_mixin and updates any
uses of it to use neutron-lib instead.
Depends-On: https://review.openstack.org/#/c/636159/
NeutronLibImpact
Change-Id: I2388f90b37abb09408809dda8c21da551bcd94bb
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
NeutronLibImpact
Depends-On: https://review.openstack.org/#/c/647836/
Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
NeutronLibImpact
Change-Id: I97120faeec73690592ed21a5ec3c6202f61e1429
The APIs our consumers are using from neutron.db_utils were rehomed into
neutron-lib with https://review.openstack.org/#/c/540161/
This patch consumes them by removing the rehomed APIs and using lib's
implementation where applicable.
NeutronLibImpact
Change-Id: I7ee53bce917feae8e37bf278eb3121a5af47131c
The is_extension_supported function now lives in neutron-lib. This patch
removes the function from neutron and uses lib's version instead.
NeutronLibImpact
Change-Id: Iccb72e00f85043b3dff0299df7eb1279655e313e
This patch includes Meter Label OVO in neutron/db/metering/metering_db.py.
Change-Id: I39e811676b17d89b7bc60873adeb082181fd5786
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Router OVO is created in patch [1].
This patch uses Router object in metering_db
[1]https://review.openstack.org/#/c/516961/
Change-Id: If551787d923231a1efb876fa000d4dda304c6636
The metering extension's API definition was rehomed into neutron-lib
with commit I1b3285287be2429f7cf9913a7e6204db26371ced. This patch
consumes the API definition removing the rehomed code from neutron
and updating references to use lib's code.
NeutronLibImpact
Change-Id: Iacd69a3b332ee494946befc2f35638410e27cfb9
Even though when using a host filter only one agent will
be returned, have get_sync_data_metering() RPC code
iterate through the list returned instead of using
agent[0].id. Noticed while looking at callers of
get_l3_agents() for an OVO change that modifies the order
the agents are returned. The code is essentially still
the same.
Also added a test using the host filter.
Trivialfix
Change-Id: I33df669258f77015a47809dd7e7df402de31e708
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
The well known service type constants are in
neutron_lib.plugins.constants, but for legacy reasons a few still exist
and are referenced from neutron_lib.constants that we'd like to remove.
This patch switches references over to neutron_lib's plugin constants.
Change-Id: I1861448cec303725b30cef8f42029f467f9e03a3
This patch introduces and implements OVO for metering
Change-Id: I168e113de6651b3a91183ed148338a26bef663cb
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Co-Authored-By: Eli Qiao <liyong.qiao@intel.com>
Now that CommonDbMixin is just a shim we can stop using it.
Instead, use the model_query and resource_extend functions directly.
Related-Blueprint: neutron-lib
Change-Id: If1252c42c49cd59dba7ec7c02c9b887fdc169f51
Those were moved under neutron.db.models start of Ocata. We should be
able to clean them up now.
NeutronLibImpact
Change-Id: Iadbf44d52ee8e30712807384152a29ce1a8b8f72
IPTables driver for metering was not handling the DVR router
namespaces properly for configuring the metering forward rules.
This patch addresses the issue by configuring the iptables
manager based on the availability of the namespace and selecting
the respective namespaces such as router namespace and snat
namespace with the right external device.
Change-Id: I6790d6ff42d9f8fa220e1a231ae94cbf8b60506c
Closes-Bug: #1506567
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
When syncing data from neutron server, metering-agent may receive
information about routers that are not hosted by the l3-agent on the
same host, because the server didn't filter them out. This could lead to
the following problems:
* metering-agent tries to setup iptables rules for a router that is not
on the host
* metering-agent tries to get get traffic counters for a router that
was once on the host but is already removed
* metering-agent not sets up iptables rules for a router that is
removed then added back to the host, because nothing about the
router is changed from metering-agent's perspective
This commit fixes the aforementioned problems by making metering-agent
only receive information about routers that are on the same host, and
update metering-agent's knowledge about which routers it should care.
However, there could still be problem if one removes then adds a router
back to the same l3-agent, or just sets the router's admin_state_up
property to False then True in a short time(shorter than the interval
between two syncs). Because the metering-agent sees nothing changed
while during the same time the router's namespace is removed and added
back on the host. Thus metering-agent will fail to get such router's
traffic counters. This commit also make iptables-driver to forget such
routers and leave the metering-agent to reconfigure them later.
Closes-Bug: #1580548
Change-Id: Ia6ae82c676582b06710d6f96b9938c215258182d
Signed-off-by: Hunt Xu <mhuntxu@gmail.com>
Use reader and writer for db operations with metering.
Partially-Implements blueprint: enginefacade-switch
Change-Id: If76d8d7c2cf92c6d9a41bece96b9209e1133541d
Extract all the common utils from common_db_mixin.py in preparation
for moving them to neutron-lib.
This is a preliminary step in preparation for refactoring the
CommonDbMixin class and moving it to neutron-lib also.
Partial Blueprint: neutron-lib
Change-Id: I3cba375a8162cb68e8f988f22f5c8b1ce7915180
Currently, creating meter-label-rule with non-existent meter-label
is returned with 500 error, because there is no error handling for
foreign key constraint error from DB.
This patch adds error handling for the error from DB
and raises MeteringLabelNotFound error.
Change-Id: I05d6c70cfe1c54618c55cba435bc8e3301948738
Closes-Bug: #1608378
This patch will separate metering db models from mixins
for OVO implementation work.
Change-Id: If1bdb64c36e1993d62269d390d3d18144b6c492d
Partial-Bug: #1597913
All occurences of ``tenant_id`` across the database are renamed
to ``project_id``. Both options are equally valid, but ``project_id``
is preferred.
To inform external users about the change, HasTenant class was
deprecated.
UpgradeImpact
Partially-Implements: blueprint keystone-v3
Change-Id: I87a8ef342ccea004731ba0192b23a8e79bc382dc
I saw a patch set [1] that was running in to circular imports and
removing these. I thought it'd be better to remove them all in one
shot.
[1] https://review.openstack.org/#/c/261222/1/neutron/db/agents_db.py
Change-Id: If1a9ba3091ac99f01f73b98a7459358c3efd4dd5
The check of the tenant done in the method _get_tenant_id_for_create()
is already did by the Neutron Controller in prepare_request_body(),
with a call to attributes.populate_tenant_id().
Moreover, when the Controller processes a "create" requests, it
will add the 'tenant_id' to the resource dict.
Thus, _get_tenant_id_for_create() can be deleted.
Calls to this method are replaced by the res['tenant_id'].
Changes have to be done in UT to explicitly add the tenant_id while
creating resources, since the UT framework is bypassing the controller code
that automatically adds the tenant_id to the resource.
Co-Authored-By: Hong Hui Xiao <xiaohhui@cn.ibm.com>
Closes-Bug: #1513825
Change-Id: Icea06dc81344e1120bdf986a97a6b1094bbb765e
Depends-On: I31022e9230fc5404c6a94edabbb08d2b079c3a09
Depends-On: Iea3f014ef17a1e1b755cd2efe99afd1a36ebbc6a
Depends-On: I604602d023e0cbf7f6591149f914d73217d7a574
- This does NOT break other projects that rely on neutron.i18n,
as this change includes a debtcollector shim to maintain those
older entry points, until they can migrate.
- Also updates _i18n.py to the latest pattern defined by oslo_i18n
- Guidance and template are from the reference:
http://docs.openstack.org/developer/oslo.i18n/usage.html
Partially-Closes-Bug: #1519493
Change-Id: I1aa3a5fd837d9156da4643a367013c869ed8bf9d
Now we send all labels and rules per rule create/delete
and rebuild whole iptables chains.
In this patch we send only affected rule and create/
delete only this rule from iptables.
Change-Id: I58ebd8d810c62980c09a340ee1680be17c12b74a
Closes-Bug: #1400280
It's mostly a matter of changing imports to a new location.
Non-obvious changes needed:
* pass overwrite= argument to oslo_context since oslo.log reads context
from its thread local store and not local.store from incubator
* don't store context at local.store now that there is no code that
would consume it
* LOG.deprecated() -> versionutils.report_deprecated_feature()
* dropped LOG.audit check from hacking rule since now the method does
not exist
* WritableLogger is now located in oslo_log.loggers
Dropped log module from the tree. Also dropped local module that is now
of no use (and obsolete, as per oslo team).
Added versionutils back to openstack-common.conf since now we use the
module directly from neutron code and not just as a dependency of some
other oslo-incubator module.
Note: tempest tests are expected to be broken now, so instead of fixing
all the oslo.log related issues for the subtree in this patch, I only
added TODOs with directions for later fix.
Closes-Bug: #1425013
Change-Id: I310e059a815377579de6bb2aa204de168e72571e
Oslo project decided to move away from using oslo.* namespace for all their
libraries [1], so we should migrate to new import path.
This patch applies new paths for:
- oslo.config
- oslo.db
- oslo.i18n
- oslo.messaging
- oslo.middleware
- oslo.rootwrap
- oslo.serialization
- oslo.utils
Added hacking check to enforce new import paths for all oslo libraries.
Updated setup.cfg entry points.
We'll cleanup old imports from oslo-incubator modules on demand or
if/when oslo officially deprecates old namespace in one of the next
cycles.
[1]: https://blueprints.launchpad.net/oslo-incubator/+spec/drop-namespace-packages
Depends-On: https://review.openstack.org/#/c/147248/
Depends-On: https://review.openstack.org/#/c/152292/
Depends-On: https://review.openstack.org/#/c/147240/
Closes-Bug: #1409733
Change-Id: If0dce29a0980206ace9866112be529436194d47e
Convert several places in the code that were using the RpcCallback
compatibility class. This class eased the transition to
oslo.messaging. This patch drops the usage of this class in favor of
direct usage of the appropriate oslo.messaging API.
Part of blueprint drop-rpc-compat.
Change-Id: I955958e836635d43dabc7b39d050c1418b18cf8f
Mostly trivial import changes.
- oslo.i18n no longer provide install() method to inject _() into
globals(), so removed all calls to it;
- removed Babel from dependencies (it will now be grabbed by oslo.i18n);
- updated tox.ini to ignore import violations for oslo.i18n.
Change-Id: I6623d551f512fb7fe9bf35ee734ed6d4c6cbc287
Rodolfo Alonso Hernandez