This patch makes necessary changes to ML2 type drivers and plugin
manager for network segment range extension support when it is loaded.
When the network segment range extension is not loaded, no impact to the
current flow.
When the extension is loaded,
- populating a range that is managed from the configuration file [1]_,
such as "VLAN IDs", "VXLAN VNI IDs", "GRE tunnel IDs",
"Geneve VNI IDs" to the network segment range DB table as a "default"
and "shared" entry to maintain backward compatibility;
- reloading the "default" segment ranges when Neutron server
starts/restarts;
- creating a set of "default" network segment ranges out of the
ML2-config-file-defined ranges [1]_ and the segment allocation
operations are always retrieving the information from the DB to have
the network segment ranges fully administered via API;
- when a tenant allocates a segment, it will first allocate from an
available segment range assigned to the tenant, and then a shared
range if no tenant specific allocation is possible.
[1] /etc/neutron/plugins/ml2/ml2_conf.ini
Co-authored-by: Allain Legacy <Allain.legacy@windriver.com>
Partially-implements: blueprint network-segment-range-management
Change-Id: I522940fc4d054f5eec1110eb2c424e32e8ae6bad
neutron-lib contains a number of the plugin related constants from
neutron.plugins.common.constants. This patch consumes those constants
from neutron-lib and removes them from neutron. In addition the notion
of the dummy plugin service type is moved strictly into the test
package of neutron since it's not a real service plugin.
NeutronLibImpact
Change-Id: I767c626f3fe6159ab3abd6a7ae3cb9893b79bf66
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
This patch integrates the Oslo-Versioned Objects created for
VlanAllocation, VxlanAllocation, VxlanEndpoints, GreAllocation,
GreEndpoints, GeneveAllocation and GeneveEndpoints into the
code base.
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Change-Id: I0d592bae452876b24c28ca4cc4bf6392b5ab6493
Co-Authored-By: Victor Morales <victor.morales@intel.com>
Those were moved under neutron.db.models start of Ocata. We should be
able to clean them up now.
NeutronLibImpact
Change-Id: Iadbf44d52ee8e30712807384152a29ce1a8b8f72
Refactoring neutron ml2 drivers configuration options to be in
neutron/conf/plugins/ml2/drivers. This would allow centralization of
all configuration options and provides an easy way to import.
Change-Id: I825bfed769bff92de010733b16f4a4b5171a052c
Partial-Bug: #1563069
This patch will relocate the VxlanAllocation and VxlanEndpoints db
model from plugins/ml2/drivers to db/models/plugins/ml2 and separate
it from mixins for OVO implementation.
Change-Id: I14702ddcc5c32ba34b5f905a15cd816a69ef2b49
Partial-Bug: #1597913
- This does NOT break other projects that rely on neutron.i18n,
as this change includes a debtcollector shim to maintain those
older entry points, until they can migrate.
- Also updates _i18n.py to the latest pattern defined by oslo_i18n
- Guidance and template are from the reference:
http://docs.openstack.org/developer/oslo.i18n/usage.html
Partially-Closes-Bug: #1519493
Change-Id: I1aa3a5fd837d9156da4643a367013c869ed8bf9d
This adds a new tox environment, genconfig, which generates sample
neutron core configuration file using oslo-config-generator.
Updates to some configuration option help messages to reflect useful
details that were missing in the code but were present in config files.
It also adds details to devref on how to update config files.
Partially-Implements: blueprint autogen-neutron-conf-file
DocImpact
Change-Id: I1c6dc4e7d479f1b7c755597caded24a0f018c712
Closes-bug: #1199963
Co-Authored-By: Louis Taylor <louis@kragniz.eu>
Currently gre/vxlan type drivers have specific sync_allocations
implementations to perform the same feature. This change abstracts
(vxlan) implementation and moves it to TunnelTypeDriver[1] in order to
share implementation between gre/vxlan; this change also adds some
unittests.
Current gre/vxlan and new implementations of sync_allocations are not
Galera-compliant with multi-writers, a follow-up will update it to
fully support Galera.
[1] neutron.plugins.ml2.drivers.type_tunnel
Change-Id: I188a7cf718d811084475f6783d844588de5d60ea
Currently, gre/vxlan sync_allocations and _parse_tunnel_ranges both
check tunnel id values. This change removes the check in gre/vxlan
sync_allocations as they duplicate _parse_tunnel_ranges check and is
less fine.
Change-Id: I5827468aeaec5d6c79d469132b129aeb7da171e2
gre and vxlan type drivers have similar implementations for multiple
methods:
* get_endpoint_by_host
* get_endpoint_by_ip
* delete_endpoint
* get_endpoints
* add_endpoint
This change abstracts these methods and moves the abstractions to the
new class EndpointTunnelTypeDriver.
Change-Id: Iab97f8283b6bf5586334958de950664f6e74202a
The function `xrange` was renamed to `range` in Python 3.
* Remove `xrange` occurences so that Python 3 tests can pass. Use
`six.moves.range` instead to get the right function in both cases.
* Generalize the use of the efficient `range` (ex-`xrange`) in
critical sections (when iterating over large lists).
* Simplify code.
* Add a hacking check to prevent future usage of `xrange`.
Change-Id: I080acaaa1d4753619fbbb76dddba6d946d84e73f
Partially implements: blueprint neutron-python3
It is quite confusing to have values for network type in common.constants.py
instead of having in plugins.common.constants.py.
Currently, the plugins/common/constants.py consists network_type constants
like VLAN, VXLAN, GRE etc. but values for network type like ranges
are defined in common.constants.py which is not good, it is better to have
both things at the same place.
This patch set addresses the same.
Moved out few methods which are predominantly used in plugins
from common.utils.py to plugins.common.utils.py.
Removed constants which were used in neutron-fwaas from
plugins.common.constants.py: https://review.openstack.org/#/c/168709/
Closes-Bug: #1441043
Change-Id: Iecfb15c541ed5d3cce95ba48f072af7fa60ac6f1
ML2 will check the config parameters for the MTU settings.
It will check the segment, path and physnet mtu settings.
Change-Id: I58b57e01ec9bcafd7cdcfbf03149e98c3a1291ed
Partially-Implements: blueprint mtu-selection-and-advertisement
It's mostly a matter of changing imports to a new location.
Non-obvious changes needed:
* pass overwrite= argument to oslo_context since oslo.log reads context
from its thread local store and not local.store from incubator
* don't store context at local.store now that there is no code that
would consume it
* LOG.deprecated() -> versionutils.report_deprecated_feature()
* dropped LOG.audit check from hacking rule since now the method does
not exist
* WritableLogger is now located in oslo_log.loggers
Dropped log module from the tree. Also dropped local module that is now
of no use (and obsolete, as per oslo team).
Added versionutils back to openstack-common.conf since now we use the
module directly from neutron code and not just as a dependency of some
other oslo-incubator module.
Note: tempest tests are expected to be broken now, so instead of fixing
all the oslo.log related issues for the subtree in this patch, I only
added TODOs with directions for later fix.
Closes-Bug: #1425013
Change-Id: I310e059a815377579de6bb2aa204de168e72571e
All the Neutron code was scanned for places where in_ is being used
and added checks to ensure that the input is not an empty sequence.
Change-Id: I1e27f94ea350ce1dfabdd7eb14e4397ca29e8eb7
Closes-Bug:1264579
Oslo project decided to move away from using oslo.* namespace for all their
libraries [1], so we should migrate to new import path.
This patch applies new paths for:
- oslo.config
- oslo.db
- oslo.i18n
- oslo.messaging
- oslo.middleware
- oslo.rootwrap
- oslo.serialization
- oslo.utils
Added hacking check to enforce new import paths for all oslo libraries.
Updated setup.cfg entry points.
We'll cleanup old imports from oslo-incubator modules on demand or
if/when oslo officially deprecates old namespace in one of the next
cycles.
[1]: https://blueprints.launchpad.net/oslo-incubator/+spec/drop-namespace-packages
Depends-On: https://review.openstack.org/#/c/147248/
Depends-On: https://review.openstack.org/#/c/152292/
Depends-On: https://review.openstack.org/#/c/147240/
Closes-Bug: #1409733
Change-Id: If0dce29a0980206ace9866112be529436194d47e
neutron.db.model_base.BASEV2 defines __table_args__ attribute. Its
subclasses overloading __table_args__ should inherit __table_args__ from
parent class.
A migration script is not needed as __table_args__ are not overloaded in
neutron.db.migration.models.frozen module.
Closes-Bug: #1413985
Change-Id: Ied625252714e2e952f9831d0d03c4573306fe29d
ml2_vxlan_allocations, ml2_gre_allocations, ml2_vlan_allocations tables
have the 'allocated' field.
There are a lot of similar queries to these tables which look
like the following:
SELECT ml2_vxlan_allocations.vxlan_vni
AS ml2_vxlan_allocations_vxlan_vni,
ml2_vxlan_allocations.allocated
AS ml2_vxlan_allocations_allocated
FROM ml2_vxlan_allocations
WHERE ml2_vxlan_allocations.allocated = 0 LIMIT 1;
Performing such selects can take quite a lot of time and if a transaction
which performs allocation is executed in parallel, it can lead to
allocation failure and retry.
Adding an index on "allocated" column significantly improves
the performance. For ml2_vlan_allocations table created
an index on (physical_network, allocation) together.
Example for MySQL for execution of query
select * from ml2_vxlan_allocations where allocated = 0;
when on the table with ~3 mln entries, ~500K of which
have allocated = 0:
+-----------------------+---------------------+
|No index on "allocated"| Index on "allocated"|
+---------------------------------------------+
| 2.02 sec | 0.43 sec |
+-----------------------+---------------------+
Closes-Bug: #1412348
Change-Id: Ie90ba611dcae6bd0cb7686a0c7b29b9484eae693
This patch set basically addressing the review comment
given in the following patch-set:
https://review.openstack.org/#/c/121000/
Co-Authored-By: Romil Gupta <romilg@hp.com>
Change-Id: I12cbb4737e4938b8da808cb19f5e9d1e290f2559
Description:
Stale GRE and VXLAN tunnel endpoints persists in neutron db this should be
deleted from the database. Also, if local_ip of L2 agent changes the
stale tunnel ports and flows persists on br-tun on other Compute Nodes and
Network Nodes for that remote ip this should also be removed.
Implementation
Plugin changes:
Added host column in 'ml2_vxlan_endpoints' and 'ml2_gre_endpoints' table.
Added delete_endpoint method for deleting the stale endpoints from db.
Modified tunnel_sync() method to accommodate these changes.
Modified testcases in test_type_vxlan.py
Modified testcases in test_type_gre.py
Agent changes:
Added tunnel_delete rpc for removing stale ports and flows from br-tun.
tunnel_sync rpc signature upgrade to obtain 'host'.
Added testcases for TunnelRpcCallbackMixin().
This patch-set only deals with plugin side changes.
Partial-Bug: #1179223
Change-Id: I75c6581fcc9f47a68bde29cbefcaa1a2a082344e
Mostly trivial import changes.
- oslo.i18n no longer provide install() method to inject _() into
globals(), so removed all calls to it;
- removed Babel from dependencies (it will now be grabbed by oslo.i18n);
- updated tox.ini to ignore import violations for oslo.i18n.
Change-Id: I6623d551f512fb7fe9bf35ee734ed6d4c6cbc287
All the existing LOG.info, LOG.warning, LOG.error and LOG.critical
messages should have _LI, _LW, _LE and _LC respectively.
Also, debug level log shouldn't be translated.
This patch set will cover the ml2 directory under neutron/plugins.
Partial-Bug: #1320867
Change-Id: I9d78d23bbc14e7c536c6ddf2dc4f52c67faeb667
Currently, there is no check which validates the values of
tunnel range for VXLAN/GRE networks. The VXLAN VNI is 24 bit
which have range between 1 to 2^24 - 1. Similarly, GRE key field
is 32 bit which have range between 1 to 2^32 - 1.
Closes-Bug: #1306488
Co-Authored-By: romilg romilg@hp.com
Change-Id: Idb3d0f41166df589a1e90394d9319901b5f9b439
We have git to track authorship, so let's not pad source files
with it as well.
A hacking check has been added for this. The value is N322.
Change-Id: Iab0b64d417e0bb41a6b455e2ac377deee64ec3ee
gre and vxlan type drivers have similar implementation for multiple
methods. This change abstracts these methods and move them to
TunnelTypeDriver.
sync_allocations method is not shared between gre and vxlan type drivers
because they use different strategies and the targeted one is unclear.
Change-Id: I717b70eb718f7ae82f1bc0c81543b0c438e8bddd
SELECT FOR UPDATE expression, which is triggered with the use of the
SQLAlchemy Query object's with_lockmode('update') method, is
detrimental to performance and scalability of the database
performance code in Neutron due to the lock contention it produces.
SELECT FOR UPDATE can be entirely avoided in add_endpoint methods
with the use of single-shot SELECT and INSERT expressions and the
correction of VxlanEndpoint primary key: indeed previously it was not
possible to create multiple endpoints with the same ip, now the model
primary key constraint ensures it.
Change-Id: Id69fbc15c8f51b4b275cd742312e6ff6802d8c0f
Partial-Bug: #1330562
ML2 provider networks partial specs let admins choose some provider
network attributes and let neutron choose remaining attributes. This
change provides the implementation for GRE/VXLAN provider networks.
In practice, for GRE/VXLAN provider networks provider:segmentation_id
choice can be delegated to neutron, in such case neutron will try to
find a network in tenant network pools which respects provided
provider attributes.
DocImpact
Related to blueprint provider-network-partial-specs
Partial-Bug: #1330562
Change-Id: I720d7d04f6e3453145e888d9e4d9b5e381d0f7d4
ML2 provider networks partial specs let admins choose some provider
network attributes and let neutron choose remaining attributes. This
change provides the implementation for VLAN provider networks.
In practice, for VLAN provider networks provider:physical_network
and provider:segmentation_id choices can be delegated to neutron,
in such case neutron will try to find a network in tenant network
pools which respects provided provider attributes.
DocImpact
Related to blueprint provider-network-partial-specs
Partial-Bug: #1330562
Change-Id: I2c52c71167edaa153b2e04681273e2f1be8d03aa
If a model mentions 'default' it is better to set 'server_default'
as well because this 'defaut' can be reached only via code. This
change adds 'server_default' parameter to models and in database,
so if someone will use SQL query it will also use default value.
For model Router added parameter server_default. In migration it was
added by this change https://review.openstack.org/105212.
For model MeteringLabelRule added parameter server_default.
In migration it will be added by heal script due to problems
with Metering plugin and grenade.
Partial-Bug: #1295539
Change-Id: Iab6fb4eca73e4918c16dac2945fd8552ae2a0c73
SELECT FOR UPDATE expression, which is triggered with the use of the
SQLAlchemy Query object's with_lockmode('update') method, is
detrimental to performance and scalability of the database
performance code in Neutron due to the lock contention it produces.
SELECT FOR UPDATE can be entirely avoided in release_segment methods
with the use of single-shot UPDATE and DELETE expressions, and this
patch clears a number of uses of SELECT FOR UPDATE by consolidating
multiple SQL expressions into one.
Partial-Bug: #1330562
Change-Id: I29ffcafc8d4d73ac1cb50c94df5da85514d47a3f
Vxlan type driver may take long time to initialize
vxlan allocation table. Optimize db performance by issuing
raw sql inserts coalesced into bulk statements.
Also optimize deleting logic.
Proposed patch gives ~2x performance gain in comparison with
original code on Mysql and Postgesql backends
Change-Id: I801d967e8e3c0260593f289097d17270ef0b391e
Partial-Bug: #1324875
At neutron startup VxlanTypeDriver syncs tunnel range from conf
to DB. In case multiple servers deployment restarting several
servers at the same time could lead to DB exceptions being thrown.
Need to synchronize between neutron servers by locking VxlanAllocation
table.
Change-Id: Idf9908f039070b9194612484603f592f1a8d74b8
Closes-Bug: #1307295
When multiple region of segment_id is specified, segment_id can leak
(will be never reused) on deletion of network.
Change-Id: Ia27b7e2c0ff4ab58be304e54123a679a5e35ab00
Closes-Bug: #1260262
This patch removes new definitions of common network type constants (TYPE_FLAT,
TYPE_LOCAL, etc.) and modifies uses of aforementioned constants to a common
place where constants are defined (neutron.plugins.common.constants). This
patch does not change values that are equal in value but different in name:
NETWORK_TYPE_FLAT vs TYPE_FLAT. A second changeset will be made to handle that
case.
Unit tests were modified as well when they referred to the constant.
Finally, the ovs agent code refers to the OVS plugin constants directly and
these had to be changed as well. A TODO flag was put in that file due to use
of another plugin specific constant.
Network types that were only defined in a single plugin, such as mellanox's
infiniband (IB) network type was not carried over to the common constants file.
Fixes-bug: #1196170
Change-Id: Ib6bfc8275496a24bf247946d177c734b62ae44bb
The migration for the ML2 VXLAN TypeDriver was incorrectly setting
primary key attributes on separate lines for 'ip_address' and
'udp_port'. Also, a primary key cannot have autoincrement set, so
add this to 'udp_port' as well.
Fixes bug 1213134
Change-Id: I0b2e2685bf524dd9c0333ac4511287ad6a5eb87e
there was a copy/paste error in GRE/VXlan type drivers implementation.
the segment wasn't returned while validating the provider network.
Change-Id: I7df6d2e714d09618644f935a9ed41354b62de9d0
Fixes: bug #1202244
Add an ML2 Type Driver for VXLAN networks to allow the creation
of VXLAN networks with the OVS agent.
Change-Id: I7fcc384cc44d5adc752510847d8ba5f46bbb79fb
Implements: blueprint ml2-vxlan
Kailun Qin