in [1] a lock was introduced with the goal of preventing subnets from
being deleted while ports are being created in them in parallel.
This was acheived by aquiring an exclusive lock on the row of the
subnet in the Subnet table when adding/modifying a port or deleting
the subnet.
However as this was a exclusive lock it also prevented concurrent port
modifications on the same subnet from happening. This can cause
performance issues on environment with large shared subnets (e.g. a
large external subnet).
To reduce the lock contention for this case we split the lock in two
parts:
* For normal port operations we will aquire a shared lock on the
row of the subnet. This allows multiple such operations to happen in
parallel.
* For deleting a subnet we will aquire an exclusive lock on the row of
the subnet. This lock can not be aquired when there is any shared
lock currently on the row.
With this we maintain the same locking level as before, but reduce the
amount of lock contention happening and thereby improve throughput.
The performance improvement can be measured using rally test [2].
(improving from 21 to 18 seconds).
Alternatively it can be tested using 250 parallel curl calls to create a
port in the same network. This improves from 113s to 42s.
[1]: https://review.opendev.org/c/openstack/neutron/+/713045
[2]: https://github.com/openstack/rally-openstack/blob/master/samples/tasks/scenarios/neutron/create-and-delete-ports.json
Closes-Bug: #2009055
Change-Id: I31b1a9c2f986f59fee0da265acebbd88d2f8e4f8
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/db.
Trivialfix
Change-Id: I9311cfe5efc51552008072d84aa238e5d0c9de60
With [1] gateway is no longer set for subnet created
with prefix delegation, but when adding the subnet
to the router it fails as it expects gateway to be
set.
This patch ensures gateway is set temporary to the first IP
of the subnet as it used to be just like the temporary CIDR.
Also need to ensure dhcp configuration is skipped to avoid the
original issue[2].
[1] https://review.opendev.org/c/openstack/neutron/+/699465
[2] https://bugs.launchpad.net/neutron/+bug/1856675
Closes-Bug: #1962306
Related-Bug: #1856675
Change-Id: I512f7d98ac99bb0ef06fd2acba09482e3436d18d
It is available in Neutron lib since version 1.16 so pretty long time
now.
Also use segment api definition from neutron-lib, it's available
since version 1.19.0. The api definition from neutron-lib also
avoids circulary dependency b/w standard-attr-segment and segment
extension[1].
[1] https://review.opendev.org/c/openstack/neutron-lib/+/577866
Change-Id: I13699f8c494a15d8bb9e13f767f2725f7cab9f4f
Related-Bug: #1765008
When needing to create a point to point connection via a subnet,
generally and /31 is the recommended cidr. Neutron supports /31
disabling dhcp and gateway on a subnet. /32 is also supported in
openstack.
Closes-Bug: #1580927
Change-Id: I3bfa3efb9fb8076656b16c89d2f35d74efde12b7
This change is needed for enabling floating IP's on routed networks.
To be able to create a subnet that spans all segments of a routed network,
a special subnet service type of 'network:routed' is used to denote a
network that can span all segments of a routed network.
To create floating IP's on a routed network, the subnet must be created
with a service_type of 'network:routed'. After the subnet has been
created, floating IP's can be allocated and associated as before.
See the design spec https://review.opendev.org/#/c/486450/ for
reference.
One caveat for this approach is that it requires the underlying
infrastructure to be aware of and able to route /32 host routes
for the floating IP. This implies that in practice, use of the
'network:routed' service type should be done in conjunction with
one or both of the following:
1. Third-party SDN backend that handles this service type in its
own way
2. neutron-dynamic-routing and the BGP service plugin for announcing
the appropriate next-hops for floating IP's. This is compatible
with DVR and non-DVR environments.
Depends-On: Ibde33bdacba6bd1e9c41cc69d0054bf55e1e6454
Change-Id: I9ae9d193b885364d5a4d90538880d8e9fbc8df74
Co-Author: Thomas Goirand <zigo@debian.org>
Partial-Bug: #1667329
OVN creates localport [1] for each network that has metadata
and allocate IP address from subnet within this network that has
DHCP enabled. The traffic from this port will never go outside
the chassis.
While using multiple segments with subnet linked to each segment
OVN needs to create an allocation of IP address for each of those
subnets [2] in order to generate data for OVN NBDB IPv4 DHCP Options.
The change [3] started to validate that condition, while multiple
IP addresses from different segments are tried to be allocated on
one port. We can skip this for OVN Metadata port, because there
is no reason to prevent those kind of allocation for OVN.
[1] http://www.openvswitch.org/support/dist-docs/ovn-architecture.7.html
[2] 5f42488a9a/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py (L2279)
[3] https://review.opendev.org/#/c/709444/
Change-Id: Ib51cde89ed873f48db4daebc27a0980da9cc0f19
Closes-Bug: 1871608
If IP overlapping is not allowed, when a new subnet is created, the
assigned CIDR is compared to any other subnet CIDR, searching for
overlapped IP addresses.
To retrieve those subnet CIDRs, there is no need to retrieve the
entire subnet OVO (including all synthetic and lazy fields), but
only the "id" and the "cidr" fields from the "subnet" table in the
database.
In a development machine, with 500 subnets created, those are the
average times executing both methods:
- Full subnet OVO retrieval ("get_subnets"): 0.9 secs
- "get_subnet_cidrs": 0.025 secs
Change-Id: I804cf08cbaeec544fbd9645af64d713499052257
The field "in_use" is added to "subnet" DB definition. This DB
register column is a flag used to mark a register as in use
by other transaction. When a write DB transaction writes any
value on this field, the register is locked for any other
concurrent transaction. If two DB transactions try to set this
column at the same time, one of them will fail.
This DB lock is implemented in "subnet" and is used during the
subnet deletion and the port IP assignation, where all the port
network subnets are retrieved to provide an IP address on the subnet
CIDR.
As reported in the related bug, it was possible to assign an IP
to a port and, before the port creation command finished, delete the
subnet where the IP belonged. This patch introduces this subnet lock
during the IP assignation and at the beginning of the subnet deletion
process. At the end of both transactions, the DB engine checks if the
lock operation (write "in_use" column) is possible or the subnet
register was already requested by other DB transaction.
Change-Id: I45a724917389814e83400f5854ada175dfce2b7b
Closes-Bug: #1865891
For v6_stateless IP addresses for all stateless
subnets within a network are implicitly included.
When using segments implicitly allocating addresses
across subnets on different segments is incorrect.
IPs from subnets on differnt segments was allocated
when no host binding information was available
but a subnet_id in fixed_ips request was present.
This change adds filtering based on segment_id when
fixed_ips are used. If fixed_ips are not all on the
same segment exception FixedIpsSubnetsNotOnSameSegment
is raised.
Related: rhbz#1803989
Related-Bug: #1864333
Related-Bug: #1865138
Closes-Bug: #1864225
Change-Id: I336ae76283f29dd226344fb454aaa0e4aac030ea
When a subnet is created and allocated, the gateway IP is created
based on the subnet CIDR. In case of IPv6 prefix delegation, this
CIDR is a temporary one. In this case the gateway IP cannot be
assigned yet and the value stored in the DB should be "None".
IpamBackendMixin._gateway_ip_str must read properly the IP version
stored in the "subnet" variable, under the key "ip_version"
instead of "version".
Closes-Bug: #1856726
Closes-Bug: #1856675
Change-Id: I9313c880cc458f08dc3a1b0ff13187b764ba7042
After this patch, during delete subnets with fixed ip, there are high
chances a subnet can't be found(because it just deleted) while
deleting another subnet and Raise BadRequest instead of Notfound.
The behavior change led to heat gate failure, since Heat expect to
see NotFound exception instead of BadRequest.
Also the error message is really confusing while delete subnets
(you can check out more detail in
https://storyboard.openstack.org/#!/story/2006962)
hence, I think to revert this patch will make sense to maintain the
behavior before a proper solution comes out.
Story: 2006962
Task: 37652
This reverts commit cae66a4d8d.
Change-Id: I9fc200de376668b662cd8c2bfba01483dc9bb677
This change relaxes the constraint that all subnets of the same
address family on a network must be allocated from the same subnet
pool. It allows subnets that share the same address scope to
co-exist on a network. If there is no address scope involved the
subnet pool / network affinity constraints continue to enforced
as done previously.
Change-Id: I33bd17c723b3e8d409415bda008440f8ed9cfa68
Closes: 1830240
Implements: subnets-different-pools-same-net
This patch switches over to neutron-lib's version of the the
_AuthenticBase, AuthenticEUI and AuthenticIPNetwork classes by
deleting them from neutron and using lib's version instead.
Depends-On: https://review.opendev.org/#/c/659881/
NeutronLibImpact
Change-Id: Ia3d3db401d6abcb9c9965b945bcd4c199f8e812b
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Today the neutron common exceptions already live in neutron-lib and are
shimmed from neutron. This patch removes the neutron.common.exceptions
module and changes neutron's imports over to use their respective
neutron-lib exception module instead.
NeutronLibImpact
Change-Id: I9704f20eb21da85d2cf024d83338b3d94593671e
The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
NeutronLibImpact
Change-Id: I97120faeec73690592ed21a5ec3c6202f61e1429
If the segment_id in the update request equals the segment_id
already associated with the subnet being updated, there is no
need for further update related segment validations.
Closes-Bug: #1793391
Change-Id: I65cd5b92b219c14dd07ae72457af243756c4ed02
Migrate to using Segments OVO objects
in the IPAM backend mixin code.
Change-Id: Ib98ed0687f98f6c82df8895a887cab8b8c8d38a6
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
The APIs our consumers are using from neutron.db_utils were rehomed into
neutron-lib with https://review.openstack.org/#/c/540161/
This patch consumes them by removing the rehomed APIs and using lib's
implementation where applicable.
NeutronLibImpact
Change-Id: I7ee53bce917feae8e37bf278eb3121a5af47131c
Migrate to using Service Type OVO objects
in the IPAM backend mixin code
Change-Id: Ifcbaf471f4b5f6e6ddf277c35f01e7cba71fd087
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
To enable the possibility to migrate a non-routed network to a
routed network allow updating the segment_id of a subnet.
Only allow the operation if:
- The network only has one segment
- The network only has one subnet
- The current segment_id == None
APIImpact: The segment_id attribute of subnets now allows put operation.
Closes-Bug: #1692490
Depends-On: Iffda823a149a1143f46ee9a05e9640b34bf42c51
Change-Id: I1aee29dfb59e9769ec0f1cb1f5d2933bc5dc0dc5
Fix W503 (line break before binary operator) pep8 warnings
and no longer ignore new failures.
Trivialfix
Change-Id: I7539f3b7187f2ad40681781f74b6e05a01bac474
The segment extension's API definition was rehomed into neutron-lib with
commit I8b598c9aff2f0b28e5b11d3ab3570aeae2df1a4b
This patch consumes it by using lib's version of the module, removing
the rehomed code and updating the segment extension's test to mock
out standard_attr to ensure the description of segments isn't
overridden by the standardattrdescription extension.
NeutronLibImpact
Change-Id: Icea5981a5b920a30e159a0271052b9cc43236bc5
There was one code path where the existing device_owner
value, which is used for subnet selection when service_type's
are in use was not passed to _imap_get_subnets(). This could
trigger one of two exceptions - HostNotCompatibleWithFixedIps()
or IpAddressGenerationFailureNoMatchingSubnet() depending
on the environment. Pass it along if known.
Also update the IpAddressGenerationFailureNoMatchingSubnet
exception to print the network_id and service_type values since
it could aid in debugging the problem quicker.
Change-Id: Ic13901b92cac05e8ddf1092b66aa5bcc5623fa8a
Closes-bug: #1637366
This patch switch to use Subnet OVO in
ipam_backend_mixin.IpamBackendMixin._save_subnet
method.
To use Subnet OVO there, additional changes in _make_subnet_args()
method are also required. Now this method returns:
* 'project_id' instead of 'tenant_id',
* 'cidr' as instance of netaddr.IPNetwork,
* 'gateway_ip' as instance of netaddr.IPAddress
Change-Id: I9fb284e98394b1a41ec8af919b65dc512663ff6a
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
This patch switches to use _get_subnet_object() instead of
_get_subnet() method in ipam_backend_mixin module to use Subnet
OVO instead of db model class directly.
Change-Id: Ibdc91ba20b560dc42319565b5f4aeadba0dfcb29
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
This reverts commit 1470baf02b.
As Neutron uses own objects registry this commit should be safe
to merge again and it shoudn't break projects like midonet.
Change-Id: I51833eebb4b24207f395e1d4095b1809f191a480
Commit I9642ed9b513a43c5558f9611f43227299707284a rehomed the
PROVISIONAL_IPV6_PD_PREFIX constant into neutron-lib. This patch
consumes it removing the constant in neutron and using lib's version
of it instead.
NeutronLibImpact
Change-Id: I107cb5e0ff2f3e2c5bb9dc501f420d0be08735a0
The IP allocation extension's API definition was rehomed into
neutron-lib with commit Ifebe1d5e623a16ae46598c19b7d904fe913ce2d3
This patch consumes the API definition by removing the rehomed code and
using the APIExtensionDescriptor for the extension class.
NeutronLibImpact
Change-Id: Ic44c55f6aa2c6ea8f67f9d17e904ad695ff655df
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
Fix a usage regression, use case #2 in Nova Neutron Routed Networks spec
https://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/neutron-routed-networks.html
Currently ip allocation is always deferred if binding:host_id is not
specified when routed networks are used. This causes initial fixed-ips
data provided by the user to be _lost_ unless the user also specify the
host.
Since the user specified the IP or Subnet to allocate in, there is no
reason to defer the allocation.
a) It is a common pattern, especially in Heat templates to:
1. Create a port with fixed-ips specifying a subnet.
2. Create a server and associate the existing port.
b) It is also common to use Neutron IPAM as a source to get VIP
addresses for clusters on provider networks.
This change enables these use cases with routed networks.
DocImpact: "The Networking service defers assignment of IP addresses to
the port until the particular compute node becomes apparent." This is no
longer true if fixed-ips is used in the port create request.
Change-Id: I86d4aafa1f8cd425cb1eeecfeaf95226d6d248b4
Closes-Bug: #1695740
Felix Huettner
yatinkarel