The correct usage when using keyword and varargs is
def func(arg1, *args, arg2=None, **kwargs):
Otherwise you can end up in having multiple values passed
for the parameters in case the method is called with
keyword arguments.
Fixed two calls to _get_ports_query() that were not
passing the 'filters' argument correctly.
Start enforcing this as well.
TrivialFix
Change-Id: Id9d6d841133241bbc87a589117468c4e699c310a
When a router interface is created, the corresponding subnet gateway IP
is tested first [1]. If the subnet has no gateway IP, the router
interface cannot be created. This IP will be assigned to this port.
The Neutron API also prevents from modifying the subnet gateway IP
if assigned to a router interface [2]. However the API is not
preventing the subnet gateway IP deletion. This patch is adding
this check.
This patch is being tested in the neutron-tempest-plugin [3].
[1]de58c1b995/neutron/db/l3_db.py (L902-L904)
[2]de58c1b995/neutron/db/db_base_plugin_v2.py (L715)
[3]https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/904710
Closes-Bug: #2036423
Change-Id: I4c7b399a3a052749abdb88fb50be628ee91b63a0
Do not allow the subnet cidr of :: to be used when
creating a subnet, except in the case IPv6 prefix
delegation has been specified in the request.
Closes-bug: #2028159
Change-Id: I480e9a117513996f3c070acd4ba39c2b9fe9c0f1
When creating a subnet using a subnetpool, we were
failing to validate all the passed API arguments in
the dictionary, leading to a case where you could
specify an invalid DNS nameserver. For example,
using an IPv4 nameserver on an IPv6 subnet. This
could cause daemons the l3-agent starts, like radvd,
to fail to start correctly, leading to a loss of
connectivity.
Specifying a subnet by cidr without a subnetpool
did already correctly fail with an IP version
mismatch error, this is just an edge case that
was never tested.
Since _validate_subnet() was called in so many places
it was moved to a common location and is only not
called for IPv6 prefix-delegation subnets.
Closes-bug: #2036877
Change-Id: I6302e9a373cf93e706cec10f87c3beaf632a0391
This functionality will be kept in the code as experimental as long
as no bugs are reported againts this feature.
This patch also marks the config option "ipv6_pd_enabled" as
experimental. In order to enable this flag, it is needed to configure
the "experimental.ipv6_pd_enabled" flag too.
Related-Bug: #1916428
Change-Id: I27aeed74f308d5bdf0210e76d9557f95b66c71bf
in [1] a lock was introduced with the goal of preventing subnets from
being deleted while ports are being created in them in parallel.
This was acheived by aquiring an exclusive lock on the row of the
subnet in the Subnet table when adding/modifying a port or deleting
the subnet.
However as this was a exclusive lock it also prevented concurrent port
modifications on the same subnet from happening. This can cause
performance issues on environment with large shared subnets (e.g. a
large external subnet).
To reduce the lock contention for this case we split the lock in two
parts:
* For normal port operations we will aquire a shared lock on the
row of the subnet. This allows multiple such operations to happen in
parallel.
* For deleting a subnet we will aquire an exclusive lock on the row of
the subnet. This lock can not be aquired when there is any shared
lock currently on the row.
With this we maintain the same locking level as before, but reduce the
amount of lock contention happening and thereby improve throughput.
The performance improvement can be measured using rally test [2].
(improving from 21 to 18 seconds).
Alternatively it can be tested using 250 parallel curl calls to create a
port in the same network. This improves from 113s to 42s.
[1]: https://review.opendev.org/c/openstack/neutron/+/713045
[2]: https://github.com/openstack/rally-openstack/blob/master/samples/tasks/scenarios/neutron/create-and-delete-ports.json
Closes-Bug: #2009055
Change-Id: I31b1a9c2f986f59fee0da265acebbd88d2f8e4f8
Some files are using strings access_as_shared or access_as_external
instead of using defined constants ACCESS_SHARED and ACCESS_EXTERNAL.
This commit is doing the cleaning it does not bring any functional
change.
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: Ib75326c762776c5259740cb2f0abc1163842f95d
A network's MTU is now only valid if it is the minimum value
allowed based on the IP version of the associated subnets,
68 for IPv4 and 1280 for IPv6.
This minimum is now enforced in the following ways:
1) When a subnet is associated with a network, validate
the MTU is large enough for the IP version. Not only
would the subnet be unusable if it was allowed, but the
Linux kernel can fail adding addresses and configuring
network settings like the MTU.
2) When a network MTU is changed, validate the MTU is large
enough for any currently associated subnets. Allowing a
smaller MTU would render any existing subnets unusable.
Closes-bug: #1988069
Change-Id: Ia4017a8737f9a7c63945df546c8a7243b2673ceb
This improves the performance of the database when fetching a list of ports
for a project user. This change creates an inner join with the networks
belonging to the ports.
Previous SQL query:
SELECT ports ...
FROM network, ports ...
WHERE ports.project_id = <project>
OR ports.network_id = networks.id
AND networks.project_id = <project>
Current SQL query:
SELECT ports ...
FROM ports
INNER JOIN networks ON networks.id = ports.network_id
WHERE ports.project_id = <project>
OR networks.project_id = <project>
Closes-Bug: #2016704
Change-Id: I9c49a307956ecfbf8bd2e866cefb21a212c38bd6
When networks are created using REST APIs, if the MTU isn't specified
in the request, then a default value of 0 is used. Some use cases, such
as the auto-allocated-topology workflow, call the plugin directly to
create networks, bypassing the layer that inserts this default value.
Commit 68625686a4 introduced a different
default value at the DB layer, defined by a constant in neutron-lib.
If the maximum MTU size has been configured lower than this constant,
then the user receives an exception, even though they didn't provide
a value for MTU.
This patch changes the default value used in the DB layer, so that
it's consistent with the workflow seen via REST APIs.
Change-Id: Ica21e891cd2559942abb0ab2b12132e7f6cdd835
Closes-Bug: #1896933
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/db.
Trivialfix
Change-Id: I9311cfe5efc51552008072d84aa238e5d0c9de60
The goal of this patch is to make the Neutron code compliant
with SQLAlchemy 2.0.
All SQL transactions must be executed inside an explicit
writer/reader context. SQLAlchemy no longer will create an
implicit transaction if the session has no active transaction.
A warning message, only available in debug mode, is added. When
an ORM session calls "do_orm_execute", if there is no active
transaction, a warning message with a traceback will be logged
to help to debug the regression introduced.
Related-Bug: #1964575
Change-Id: I3da37fee205b8d67d10673075b9130147d9eab5f
This patch removes the ``IpAddressAllocationNotFound`` exception. This
exception was raised when a IPAM register was called to be deleted
but not found.
As reported in the LP bug, this IPAM register deletion can be called
several times if a port fails during the creation. The IPAM register
deletion calls the DB deletion but doesn't raise any exception if the
register does not exist. The code ensures the IPAM register is
deleted and there is no need to fail if it is not present anymore.
This patch also removes the exception catch and try in "update_port",
that was added in [0] as a fix for [1]. That was added because the
subnet deletion code involved a port update call [2] during the
IP allocation deletion, if any port was still present in the subnet.
Since [3], this code is not needed because the subnet deletion does
not call a port update anymore.
[0]https://review.opendev.org/c/openstack/neutron/+/373536
[1]https://bugs.launchpad.net/neutron/+bug/1622616
[2]https://github.com/openstack/neutron/blob/pike-em/neutron/db/db_base_plugin_v2.py#L1017-L1018
[3]https://review.opendev.org/c/openstack/neutron/+/713045
Closes-Bug: #1965807
Related-Bug: #1954763
Related-Bug: #1622616
Change-Id: I5b96b3a91aacffe118ddbb91a75c4892818ba97a
When new defaults and scope enforcement are used in Neutron, elevated
context needs to be used to update router's external gateway when new
subnet is added to the external network. Otherwise, router can belongs
to the other project than external network and it will not be updated at
all.
Closes-Bug: #1959331
Change-Id: I888ddf66a15cd20039ff26baccd170da128e1eb7
This new parameter "bound_drivers" is a dictionary with the binding
levels and the driver name. E.g.:
port['vif_details']['bound_drivers'] = {'0': 'openvswitch'}
port['vif_details']['bound_drivers'] = {'0': 'ovn'}
If the port is not bound, this key won't be present in "vif_details".
This information is important for Nova, along with the VIF type, to
adequate the port plugin strategy or to know what kind of plugin
events are expected; currently, depending on the driver and the
connection type, Neutron sends a different set of vif-plugged events.
This is specially critical during live migration process, where the
network communication should be halted as little as possible.
Related-Bug: #1821058
Change-Id: I1c42fa4f44cc2311e874b2b9bf2bd40ffd142e91
In subnet update API call Neutron checks if gateway_ip was send to be
updated and if so, it checkes if old gateway_ip isn't already allocated
to some router port. If it's already used, Neutron returns 409 response.
This is valid behaviour but sometimes, some automation tools may do
subnet update request and pass the same gateway ip as already used by
the subnet. In such case, as gateway_ip is actually not changed Neutron
should not raise exception in that validation.
Closes-Bug: #1955121
Change-Id: Iba90b44331fdc63273fd3d19c583a24b5295c0ac
When there is attempt to delete network with ports,
a general error message is displayed that one or more
ports are in use on the network. This patch proposes
to also return the ports which are in use as part of
the message.
Also modify test_delete_network_if_port_exists unit
test to check for port id and network id in Error
message.
Also bump required version of neutron-lib to 2.18.0
as that's needed for custom message in NetworkInUse
Exception.
Depends-On: https://review.opendev.org/c/openstack/neutron-lib/+/821806
Closes-Bug: #1953716
Change-Id: Ib0b40402746c6a487a226b238907142384608d3c
Reduce the object retrieval to one single field to improve the
collection count.
Bumped neutron-lib to 2.16.0. This version contains [1], needed for
this patch.
[1]https://review.opendev.org/c/openstack/neutron-lib/+/807686
Related-Bug: #1942863
Change-Id: I160e8084e97b23a2bacb49ceb40efbac2d0715be
Instead of executing a subquery inside a select, use a proper filter
condition instead.
Closes-Bug: #1936675
Change-Id: I97e9ca244c0716510fcd4ec81d54046be9c5f8f8
This patch sanitizes the MAC address coming from a user input:
- The "base_mac" address configuration parameter.
- The "port.mac_address" stored in the database, if the script
provided is not executed.
This patch relays on [1], that will sanitize any input coming from
the server API.
This patch adds a new script to sanitize all "port.mac_address"
registers stored in the dabatabase.
[1]https://review.opendev.org/c/openstack/neutron-lib/+/788300
Related-Bug: #1926273
Change-Id: I8572906cc435feda1f82263fd94dda47fc1526e1
Don't eagerly load several fields which are not used in
port list and show.
Port list and show speed-up is ~10-25%.
Depends on neutron-lib change:
https://review.opendev.org/c/openstack/neutron-lib/+/788729
Change-Id: I4bfcac16a54b766e0dc97f58ba4048eb709fa88f
This reverts commit 062336e59b.
Now, we have proper fix for the system_scope='all' in elevated context
in the neutron-lib so we can revert temporary fix made at the end of the
Wallaby cycle.
Related-Bug: #1920001
Conflicts:
neutron/api/rpc/agentnotifiers/dhcp_rpc_agent_api.py
neutron/common/utils.py
neutron/db/address_group_db.py
neutron/services/segments/db.py
Change-Id: Ife9b647b403bdd76a8a99984ea8858bf95c96bc3
- pass network dict from ml2 plugin to _create_subnet_postcommit
- skip ipam subnet fetch for non ipv6 auto-address subnets
- don't count subnets (DB request) if subnet has no segment
Change-Id: Iaecfda2700c5316cb25a93496d24ece366e40a4a
- don't re-fetch subnet object from DB
- network is not used in SubnetContext when deleting subnet
Above gives ~35% improvement
Change-Id: I34f850782092f771482a297ae1e68a63ffb027c1
This patch switches over to the payload style of callbacks for
NETWORK based events. As part of this change a few shims are needed
to handle cases where some callbacks don't yet use payloads and others
do. Once we move over to payloads for all callbacks the shims can be
removed.
NeutronLibImpact
Change-Id: I889364b5d184d47a79fe6ed604ce13a4b334acfa
- pass existing network dict to PRECOMMIT_DELETE ml2 handler;
- skip MTU and segment update handling in case network is
about to be deleted.
Above gives up to 50% net delete speed-up
Change-Id: I07c70db027f2ae03ffb5a95072e019e8a5fdc411
On port and network update ML2 plugin gets a db object
and can pass it to db_base_plugin_v2 when doing super call.
Change-Id: I1213f59fe643807f303e3ad7f24925fa333a5512
Pass existing port dict to IPAM delete and l3 callback handler
to save 2 port DB requests.
This gives ~10-15% improvement.
Change-Id: Iab5ed8a57aaf68f626e5b386f72d941ef7b22227
In case when enforce_new_defaults is set to True and new policy rules
are used, context.is_admin flag isn't really working as it was with old
rules.
But in case when elevated context is needed, it means that we need
context which has full rights to the system. So we should also set
"system_scope" parameter to "all" to be sure that system scope queries
can be done with such elevated context always.
It is needed e.g. when elevated context is used to get some data from
db. In such case we need to have db query which will not be scoped to
the single project_id and with new defaults to achieve that system_scope
has to be set to "all".
Proper fix for that should be done in neutron-lib and it is proposed
in [1] already but as we are have frozen neutron-lib version for
stable/wallaby already this patch for neutron is temporary fix for that
issue.
We can revert that patch as soon as we will be in Xena development cycle
and [1] will be merged and released.
[1] https://review.opendev.org/c/openstack/neutron-lib/+/781625
Related-Bug: #1920001
Change-Id: I0068c1de09f5c6fae5bb5cd0d6f26f451e701939
Brian Haley
yatinkarel