When port with IP address from the auto allocation subnet (SLAAC or
dhcp_stateless) is created or updated and fixed IP address is specified,
Neutron will fail as in case of such subnets IP address is assigned
automatically based on the subnet's prefix and port's MAC address.
But in case when given IP address is actually correct EUI64 generated IP
address (the same as Neutron would generate for that port) there is no
need to raise an exception and fail request.
Additionally this patch fixes imports sections in the
ipam_pluggable_backend module.
Closes-bug: #1991398
Change-Id: Iaee5608c9581228a83f7ad75dbf2cc31dafaa9ea
In the ML2 plugin in create_port_bulk method, we are iterating over
list of the ports to be created and do everything for all ports in
single DB transaction (which makes totally sense as this is bulk
request).
But one of the things which was done during that huge transaction was
allocation of the IP addresses for all ports. That action is prone for
race conditions and can fail often, especially when there is no many IP
addresses available in the subnet(s) for the ports.
In case of the error while allocating IP address even for one port from
the whole bulk request, whole create_port_bulk method was retried so
allocations (and everything else) for all ports was reverted and started
from scratch. That takes a lot of time so some requests may be processed
very long time, like e.g. 2-3 minutes in my tests.
To reproduce that issue I did simple script which created network with
/24 subnet and then sent 24 requests to create 10 ports in bulk in each
request. That was in totall 240 ports created in that subnet.
I measured time of the creation of all those ports in the current master
branch (without this patch) and with the patch. Results are like below:
+-----+---------------+------------+---------------------------+
| Run | Master branch | This patch | Simulate bulk by creation |
| | [mm:ss] | [mm:ss] | of 10 ports one by one |
+-----+---------------+------------+---------------------------+
| 1 | 01:37 | 01:02 | 00:57 |
| 2 | 02:06 | 00:40 | 01:03 |
| 3 | 02:08 | 00:41 | 00:59 |
| 4 | 02:14 | 00:45 | 00:55 |
| 5 | 01:58 | 00:45 | 00:57 |
| 6 | 02:37 | 00:53 | 01:05 |
| 7 | 01:59 | 00:42 | 00:58 |
| 8 | 02:01 | 00:41 | 00:57 |
| 9 | 02:39 | 00:42 | 00:55 |
| 10 | 01:59 | 00:41 | 00:56 |
+-----+---------------+------------+---------------------------+
| AVG | 00:02:07 | 00:00:45 | 00:58 |
+-----+---------------+------------+---------------------------+
Closes-Bug: #1954763
Change-Id: I8877c658446fed155130add6f1c69f2772113c27
Allow IP allocation with different segments also for IPv6
auto allocation for OVN service ports.
See change: Ib51cde89ed873f48db4daebc27a0980da9cc0f19
Change-Id: Ic6db2f66be596756b1fed9ffeffc7b049f6e957f
Closes-Bug: #1944948
Pass existing port dict to IPAM delete and l3 callback handler
to save 2 port DB requests.
This gives ~10-15% improvement.
Change-Id: Iab5ed8a57aaf68f626e5b386f72d941ef7b22227
OVN creates localport [1] for each network that has metadata
and allocate IP address from subnet within this network that has
DHCP enabled. The traffic from this port will never go outside
the chassis.
While using multiple segments with subnet linked to each segment
OVN needs to create an allocation of IP address for each of those
subnets [2] in order to generate data for OVN NBDB IPv4 DHCP Options.
The change [3] started to validate that condition, while multiple
IP addresses from different segments are tried to be allocated on
one port. We can skip this for OVN Metadata port, because there
is no reason to prevent those kind of allocation for OVN.
[1] http://www.openvswitch.org/support/dist-docs/ovn-architecture.7.html
[2] 5f42488a9a/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py (L2279)
[3] https://review.opendev.org/#/c/709444/
Change-Id: Ib51cde89ed873f48db4daebc27a0980da9cc0f19
Closes-Bug: 1871608
When creating additional subnets with ipv6 auto-addressing
ip allocation was added to existing ports without filtering
on current allocation's segment.
This adds fitering to only add auto-address allocation when
the new subnet is on the same segment as the ports current
ipam allocations.
Related: rhbz#1803989
Related-Bug: #1864225
Related-Bug: #1865138
Closes-Bug: #1864333
Change-Id: I75ae14c64db076434ca9897ba9a6d97702e233ad
For v6_stateless IP addresses for all stateless
subnets within a network are implicitly included.
When using segments implicitly allocating addresses
across subnets on different segments is incorrect.
IPs from subnets on differnt segments was allocated
when no host binding information was available
but a subnet_id in fixed_ips request was present.
This change adds filtering based on segment_id when
fixed_ips are used. If fixed_ips are not all on the
same segment exception FixedIpsSubnetsNotOnSameSegment
is raised.
Related: rhbz#1803989
Related-Bug: #1864333
Related-Bug: #1865138
Closes-Bug: #1864225
Change-Id: I336ae76283f29dd226344fb454aaa0e4aac030ea
Add a new DB table "network_subnet_lock". The primary key will be the
network_id. When a subnet is created, inside the write context during
the "subnet" object creation, a register in the mentioned table is
created or updated. This will enforce the serialization of the "subnet"
registers belonging to the same network, due to the write lock in the
DB.
This will solve the problem of attending several "subnet" creation
requests, described in the related bug. If several subnets with the
same CIDR are processed in parallel, the implemented logic won't reject
them because any of them will not contain the information of each other.
This DB lock will also work in case of distributed servers because the
lock is not enforced in the server logic but in the DB backend.
Change-Id: Iecbb096e0b7e080a3e0299ea340f8b03e87ddfd2
Closes-Bug: #1852777
After this patch, during delete subnets with fixed ip, there are high
chances a subnet can't be found(because it just deleted) while
deleting another subnet and Raise BadRequest instead of Notfound.
The behavior change led to heat gate failure, since Heat expect to
see NotFound exception instead of BadRequest.
Also the error message is really confusing while delete subnets
(you can check out more detail in
https://storyboard.openstack.org/#!/story/2006962)
hence, I think to revert this patch will make sense to maintain the
behavior before a proper solution comes out.
Story: 2006962
Task: 37652
This reverts commit cae66a4d8d.
Change-Id: I9fc200de376668b662cd8c2bfba01483dc9bb677
When updating a port with the fixed_ips request the
fixed_configured argument should be set to true when
calling _ipam_get_subnets() so that all subnets are
returned if host is not set.
Otherwise the ip allocation will be deffered and an
empty list of possible subnets for the port is
returned. Which in turn led to raising an error that
the network requires subnets to allocate an IP
address.
Closes-Bug: #1844124
Change-Id: I2e690ea0cf5fa0614e39be2b0e83afad3daa7f48
For auto-address IPv6 subnets postcommit has update port action
if the net already has ports. This results in
"cannot be called within a transaction" error for bulk IPv6 subnet
create.
Closes-Bug: #1822582
Change-Id: Ia32ec4c11c0793e7df07dcce19c122b3c7f865e1
Rather than iterating through all ports and leveraging the heavyweight
ml2 method, use a set of optimized functions that attempts to make bulk
port operations as speedy as possible.
The test test_bulk_ports_before_and_after_events_outside_of_txn is
deleted because with the change to using port OVO, the session will
not be closed, and it is not correct to check for it.
Change-Id: Ieea0e6074cd31a2d09ae92f1f3c8d375c6d8ecc2
Implements: blueprint speed-up-neutron-bulk-creation
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
IP allocation was initially deffered due to lack of binding
information. On port update the with both `mac_address` and
`binding_host_id`` in the request 'fixed_ips: []' was
appended to the new_port data. This caused the check for
fixed_ips_requested to return True, which in turn cause
deferred_ip_allocation to evaluates False.
Only set the new_port default fixed_ips to original_ips if
the original port had fixed_ips.
Closes-Bug: #1811905
Change-Id: If98a82f8432b09a29f9d0cc6627e9649b43bc4a1
Some L3 ports can directly modify the IP address now,
but there are some types of device_owner, for instance
network:router_centralized_snat, should not allow to
change the IP address, otherwise it will increase L3
agent code complexity.
Since router public gateway port is related to the
external network, and this port can be used for some
service, like VPN. So after this patch we will only
allow the gateway port to update the IP address
directly, aka device_owner network:router_gateway.
The following l3 router ports will not be allowed to
directly update the IP address:
network:ha_router_replicated_interface
network:router_centralized_snat
network:router_ha_interface
network:router_interface
network:router_interface_distributed
Remove one unit test case since it will not occur in
neutron.
Change-Id: I2890a71abc82cb781b601afb879bdcca10e39257
Closes-Bug: #1796824
Implement a new function called create_port_obj_bulk that optimizes
bulk port creation operations by streamlining ensuring network existence
and operates on an array of port data.
Change-Id: Ie819c215944514d0bb43c2ce87394825bda41e94
Partially-Implements: blueprint speed-up-neutron-bulk-creation
The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
NeutronLibImpact
Change-Id: I97120faeec73690592ed21a5ec3c6202f61e1429
If the segment_id in the update request equals the segment_id
already associated with the subnet being updated, there is no
need for further update related segment validations.
Closes-Bug: #1793391
Change-Id: I65cd5b92b219c14dd07ae72457af243756c4ed02
To enable the possibility to migrate a non-routed network to a
routed network allow updating the segment_id of a subnet.
Only allow the operation if:
- The network only has one segment
- The network only has one subnet
- The current segment_id == None
APIImpact: The segment_id attribute of subnets now allows put operation.
Closes-Bug: #1692490
Depends-On: Iffda823a149a1143f46ee9a05e9640b34bf42c51
Change-Id: I1aee29dfb59e9769ec0f1cb1f5d2933bc5dc0dc5
Don't mix old and new session management styles. Mixing them is not
guaranteed to work. The method is not switched to new enginefacade yet,
so stick to the old style. Since OVO create() method already opens a
nested transaction, do nothing in IPAM layer.
This change fixes an issue where sometimes, whenever
add_auto_addrs_on_network_ports raises a retriable exception like
StaleDataError, corresponding IPAllocation is still persisted in the
database without being rolled back and without the corresponding
IpamAllocation model persisted. Later, this situation may break port and
network deletion for resources affected by the issue, failing with the
following error:
Unable to find IP address 2003::f816:3eff:fed2:5006 on subnet
eed623f4-cbad-488c-b230-7f4a8b1514eb
It happens because whenever writer.using context manager exits, and it's
top level manager, it commits the changes to database. Since
writer.using doesn't know that it's being executed in scope of the old
facade .begin() call, it doesn't postpone commitment as needed.
The bug was introduced in Pike as part of the switch to the new
enginefacade (that was never fully completed), specifically by:
I50be115ea69f805b48b02aebe4259ec2c839830e
For existing setups that already have IPAllocation objects without
IpamAllocation counterparts, the solution is to remove those
dangling IPAllocations manually from the database. (That's of course
assuming they use Neutron DB IPAM plugin in the first place.)
Note: If we ever decide to remove nested transactions from OVO base
layer, we may need to revisit this code to keep ignore-on-failure
semantics.
Change-Id: Ic9e146f51d9a120011892828d3a67b0630f4c5ce
Closes-Bug: #1706750
Commit I9642ed9b513a43c5558f9611f43227299707284a rehomed the
PROVISIONAL_IPV6_PD_PREFIX constant into neutron-lib. This patch
consumes it removing the constant in neutron and using lib's version
of it instead.
NeutronLibImpact
Change-Id: I107cb5e0ff2f3e2c5bb9dc501f420d0be08735a0
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
Change If1eb4046865f43b15ba97c52e2d0b9343dc72c19 fixed bugs 1666493 and 1655567
that reported exception IpAddressAlreadyAllocated being raised during the
creation of IPv6 auto-address subnets. This patchset removes code that was
added by change I22b8f1f537f905f4b82ce9e50d6fcc5bf2210f9f to root-cause these
bugs.
By removing log statements, this patchset contributes to reduce the
'Neutron log obesity epidemic'
Change-Id: I28c58dc4a957df833d277f0d08ce831c7ee07c68
Partial-Bug: #1707307
Automatically allocated fixed IP addresses were not reflected on cached
port records, which triggered FlushError if the allocating method failed
and triggered a retry.
Change-Id: Ia7725094827b554aa6d928c7daa026959a237991
Closes-Bug: #1706750
This patch integrates the Oslo-Versioned Object created for
IPAllocation model class.
Change-Id: Ibb731c4635d89c848081fac73f216d2ecf10b599
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
When creating an IPv6 auto-address subnet for a network, ports can be
created or updated concurrently in the same network, before the subnet
creation concludes. In such a situation, an IpAddressAlreadyAllocated
exception may be raised, because the subnet create request attempts to
allocate auto addresses to the concurrently created or updated ports,
which have been already allocated by the port requests.
This patchset adds code to the IPAM to skip the attempt to assign the
auto address to ports if they already have received them.
Change-Id: If1eb4046865f43b15ba97c52e2d0b9343dc72c19
Closes-Bug: #1655567
Fix a usage regression, use case #2 in Nova Neutron Routed Networks spec
https://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/neutron-routed-networks.html
Currently ip allocation is always deferred if binding:host_id is not
specified when routed networks are used. This causes initial fixed-ips
data provided by the user to be _lost_ unless the user also specify the
host.
Since the user specified the IP or Subnet to allocate in, there is no
reason to defer the allocation.
a) It is a common pattern, especially in Heat templates to:
1. Create a port with fixed-ips specifying a subnet.
2. Create a server and associate the existing port.
b) It is also common to use Neutron IPAM as a source to get VIP
addresses for clusters on provider networks.
This change enables these use cases with routed networks.
DocImpact: "The Networking service defers assignment of IP addresses to
the port until the particular compute node becomes apparent." This is no
longer true if fixed-ips is used in the port create request.
Change-Id: I86d4aafa1f8cd425cb1eeecfeaf95226d6d248b4
Closes-Bug: #1695740
This patch integrates the Oslo-Versioned Object created for
IPAllocationPool model class.
Change-Id: Ifca5be29d322f1e85c45ca77371df37c9bf5b27f
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
On MAC update add an entry for auto-addressed IPs so
that they get removed and re-allocated.
Also add a check to ensure the length of fixed_ips in the result
is what was expected i.e. ensure we replaced the IP's(as is the current
behaviour) and not just added to the list.
Change-Id: Id1eadce7a18ba82d6c1a42f1afb0267f076f9308
Closes-Bug: #1671548
On MAC update add an entry for auto-addressed IPs so
that they get removed and re-allocated.
Also Fix the unit test that was testing for this, so that
it checks the IP for the new updated port (it was checking
the port before the update).
Change-Id: I0a83efa7b8dfa1f6979566846195fb349547018a
Closes-Bug: #1671548
Change I50be115ea69f805b48b02aebe4259ec2c839830e moved the
dict construction inside of the transaction to deal with the
expired fixed_ips relationship. However, this no longer matches
the pattern we have everywhere else of waiting until after commit
to build the response so this patch moves it back out and loads
the fixed_ips relationships after altering them so they can be
referenced outside of the new engine facade transactions.
Partially-Implements blueprint: enginefacade-switch
Change-Id: I9149d8ab3bf977f0247e542e0c47731d900dd119
Use reader and writer for db operations.
Partially-Implements blueprint: enginefacade-switch
Depends-On: Iba3520ac6cfb6b82b2013df9b8e1aee64b10a11c
Change-Id: I50be115ea69f805b48b02aebe4259ec2c839830e
The patch handles the case when a port is updated
with an IP from a new IPv6 auto-address subnet
See bug for details on how the issue can affect DHCP
functionality.
Closes-Bug: #1678104
Change-Id: If2473f2db3ca16b5f46d3280e79a49756d1c098a
Neutron-lib 1.1.0 is now out and contains the portbindings
API definition (as per commit [1]). This patch moves neutron
references over to the neutron-lib version.
NeutronLibImpact
- Consumers using the public constants within neutron's
portbindings API extension must now use the values
from neutron-lib.
[1] 87e42f993c07ae320159d5123662ee9f3bd4d903
Change-Id: I669af9b4c712877772d91a03857ab108714001d4
Slawek Kaplonski