Currently routers with ha=true are NOT rescheduled form a dead L3 agent, even
when `allow_automatic_l3agent_failover` is enabled.
This is contrary to what the user expects and the feature description states:
"Automatically reschedule routers from offline
L3 agents to online L3 agents."
There is no distinction made between HA and non-HA routers. Also HA and
automatic-failover can work together:
* HA allows for a fast failover to a standby router
* Automatic failover then restores back full redundancy in case the failed L3
agent, which HA failed away from, does not come back within a certain time.
Closes-Bug: #2050236
Change-Id: I1e5ee5048f61eef7fa4d9de25e69bf0e0a5ea442
neutron.object.base._use_db_facade() is unused according
to codesearch, except for the test cases. Remove it.
TrivialFix
Change-Id: I8f183584bf66bac157694c51769409dba2c79da2
This method populates the SG rules in a dictionary. Each SG rule
inherits the "stateful" value of the SG. Prior to this patch, each
SG rule was isuing a database call to retrieve the SG register.
In this patch, the SG "stateful" retrieval is done in one database
query for all SG. That improves the performance of this method
reducing the database access to only one single call.
This improvement, as commented in the LP bug, affects to
ML2/LinuxBridge. ML2/OVS agent uses a cached RPC implementation
that not requires to perform any RPC call/database query.
Closes-Bug: #2045950
Change-Id: Iafd0419a1d1eeb25d5589edc2570ebf287450957
In ML2/OVN there is a static common configuration parameter to define
if the routers (more in particular the floating IPs) can be distributed
or centralized:
[ovn] enable_distributed_floating_ip
This patch writes this value on each new router created. It also
implements a maintenance method to populate this flag when the
Neutron API is restarted and the value changed.
Closes-Bug: #2022058
Change-Id: Ib109b09fde4db8738c1d0b3e394c201492d210c6
The following resources have been updated with new policies for
tags:
* Port
* Subnet
* Network
* Router
* FloatingIP
* NetworkSegmentRange
* NetworkSegment
* SecurityGroup
* Trunk
* Subnetpool
The admin can now enforce specific policies for the resource tags
for the creation, update and deletion actions.
NOTE: a follow-up patch, with a new Launchpad bug reference, will
be created to move the ``Tagging`` class from
``ExtensionDescriptor`` to ``APIExtensionDescriptor``, and
refactor the ``TaggingController`` to be a standard
``neutron.api.v2.base.Controller``. Any API resource using
the second controller will use the path used by the wsgi
hooks, in particular the policy hook. That will make unnecessary
to manually call the ``policy.enforce`` method from the
extension class methods.
Closes-Bug: #2037002
Change-Id: I9f3e032739824f268db74c5a1b4f04d353742dbd
This new extension adds a new synthetic field, "belongs_to_default_sg",
to the security group rule OVO. This read only boolean field determines
if the security group rule belongs to a default security group or not.
This new field will be used in a new set of policy rules. By default,
these new rules will allow to create and delete security group rules
into the default security group of a project only to the admin user
NOTE: the follow-up patch will introduce the policy rules check,
during the creation/deletion operations, of the
"belongs_to_default_sg" field and the user executing this action.
Partial-Bug: #2019960
Change-Id: I0b3ded52e1ff8160c5804c59635c0fd34ce9995b
This table has a 1:1 relationship with the "port" table, providing
the "hardware_offload_type" field (string).
The "neutron-lib" library minimum version is 3.8.0, that contains
[1].
NOTE: once the OSC patch is merged [2], the documentation will be
updated to reflect how to create a hardware offloaded port without
manually defining the port binding profile,
[1]https://review.opendev.org/c/openstack/neutron-lib/+/882726
[2]https://review.opendev.org/c/openstack/python-openstackclient/+/892792
Partial-Bug: #2013228
Change-Id: I04f232d6c43e39f254c4559caf041dcf05acec21
Currently, agents in DVR mode requesting a router update fetch all the
FIPs on a network from the DB rather than just the FIPs that are
relevant to the specific host requesting the update.
While not noticable in smaller networks with a limited number of
floating IPs, this can add significant overhead in larger networks
with many FIPs and hosts.
That overhead comes from Python mapping the responses from the DB into
objects, making extra DB calls per FIP returned and adding additional
iterations to the loop in _get_dvr_sync_data. These objects are mostly
discarded later on and not updated nor included in the RPC response.
This change ensures that we only fetch FIPs from the DB that are bound
to the host requesting the update or those which are in a pre-live
migration state (as they may be migrated to the host in question).
Closes-Bug: #2028185
Change-Id: I199b0b1456aa15dadcc24cafc89db1072d224efd
After updating pylint, it started emitting additional "R"
warnings in some cases, fix some of them.
use-a-generator,
unnecessary-lambda-assignment,
consider-using-max-builtin,
consider-using-generator,
consider-using-in,
use-list-literal,
consider-using-from-import
Trivialfix
Change-Id: Ife6565cefcc30b4e8a0df9121c9454cf744225df
The SQL clause "GROUP BY" in PostgreSQL requires the presence of all
fields provided in the "SELECT" clause; this is not happening in
MariaDB not MySQL.
Since [1] (and the related patch [2] that are part of the same bug
fix), when a resource with RBAC registers is selected, the "GROUP BY"
clause is added to reduce the number of returned registers. The
filed used is "id", present in all the RBAC controlled resources
('network', 'qospolicy', 'securitygroup', 'addressscope',
'subnetpool', 'addressgroup'). That is in opposition to what was
stated in the first paragraph "requires the presence of all
fields provided in the "SELECT" clause". However it is possible
to group only by a primary key [3].
In [4] a prior change was introduced that modifies the "Subnet" entity
to include a child table "SubnetServiceType". That introduces two new
fields ("SubnetServiceType" fields) that are not present in the
"GROUP BY" clause, causing the reported error in the LP bug.
This patch is adding these two fields (that form the composite primary
key of "SubnetServiceType" table) to the "GROUP BY" clause, when
needed; if the query is executed by an administrator, the RBAC query
won't be performed (an administrator has always permissions) and the
"GROUP BY" clause won't be issued.
[1]https://review.opendev.org/c/openstack/neutron-lib/+/884878
[2]https://review.opendev.org/c/openstack/neutron/+/884877
[3]https://learnsql.com/blog/must-appear-in-group-by-clause/
[4]https://review.opendev.org/c/openstack/neutron/+/744512
Closes-Bug: #2028003
Change-Id: I18e65d79e56fe5995076eb9166da23fc14c92fc5
This patch adds DB model, OVO class and DB migration script for
SG rules template used for every new SG created.
It also implements Create/Get/Delete actions for that new resource and
adds API policies for those APIs
Related-Bug: #1983053
Change-Id: Ib3cde1710edd400b972f493b13666d0679a7753c
After updating pylint, it started emitting additional "C"
warnings in some cases, fix some of them.
use-implicit-booleaness-not-comparison,
use-maxsplit-arg,
unnecessary-dunder-call
Trivialfix
Change-Id: I53558e6bc52aaea590f6c147aacc062c08099464
PostgreSQL requires to add the table keys selected in the SELECT clause
in the later GROUP BY clause. This patch is adding
"SubnetPool.address_scope_id" to this GROUP BY clause. Because the
subnet pool (for IPv4) is unique for the FIP subnet, the returned
elements in this query will be the same.
Closes-Bug: #2019186
Change-Id: Ia446e17a44b1a260971ae237841451edb97ce39f
A method is added as opposed to having a synthetic field on a router for
performance reasons: gateways will only be queried when needed to use
the external gateways feature API calls.
Partial-Bug: #2002687
Change-Id: Iddde9d986b024109bdb7c2aa777a1b017b6a35ab
The context decorator needs to have the "context" parameter in the
first position or the first argument must be "self" or "cls",
followed by the "context" object.
Closes-Bug: #2017784
Change-Id: Ib80f7c72e78854226b227e354792320c78fed5d0
We saw auto_schedule_routers took over 40 seconds
for a DVR enabled environment with option
auto_schedule_routers enabled.
Adding new arguments to get_router_agents_count and
dealing with routers separately depending on whether
it is a regular router or HA. The benefits are
we do not need to loop over every router available in
environment. Another reason for doing this is that
get_router_agents_count is used solely to heal
routers with less than required agents so number of
routers with less agents is small for most of the times.
Related-Bug: #1973656
Change-Id: Ic29275815a8c32cee7a6470509687a18fa594514
The method ``get_l3_agents_by_router_ids`` is retrieving the
``RouterL3AgentBinding`` registers along with the ``Agent`` information.
This left outer join should be explictly defined using the database
model.
Expected query:
SELECT routerl3agentbindings.router_id AS ...
FROM routerl3agentbindings
LEFT OUTER JOIN agents ON agents.id = routerl3agentbindings.l3_agent_id
WHERE routerl3agentbindings.router_id IN (XXX)
Closes-Bug: #2012662
Change-Id: Ife9ba5d774934d2cd1a86b3281b78e854d88de3c
Previously we would expect duplicates rows from MySQL and then filter
them with _unique_floatingip_iterator. This means that we were
converting rows to ORM-mapped objects unnecessarily. This conversion is
very CPU intensive.
Instead, we can remove the duplicates as a part of the query which means
that only unique rows are returned and the number of conversions from
rows to ORM-mapped objects is reduced significantly. It also means that
the _unique_floatingip_iterator function is no longer needed.
Change-Id: I05136302f7b8abc0a985b91c993008595234ad6b
Signed-off-by: Adam Oswick <adam@adamoswick.co.uk>
This patch is covering an edge case that could happen when the number
of DHCP agents ("dhcp_agents_per_network") or L3 agents
("max_l3_agents_per_router") has been reduced and there are more agents
assigned than the current number. If the user removes any agent
assignation from a L3 router or a DHCP agent, it is possible to remove
first the lower binding assigned registers.
Now the method ``get_vacant_binding_index`` calculates the number of
agents bound and the number required. If a new one is needed, the
method returns first the lower binding indexes not used.
Closes-Bug: #2006496
Change-Id: I25145c088ffdca47acfcb7add02b1a4a615e4612
The from_self() method in SQLAlchemy is currently
being considered for removal from the library,
with a deprecation phase throughout 1.4 and then
removal by SQLAlchemy 2.0.
The from_self() method takes an ORM query object,
turns it into a subquery, then returns a new query
object that will SELECT from that subquery, while transparently
altering subsequent criteria added to the query to
be stated in terms of the subquery. The current
design direction of SQLAlchemy hopes to
de-emphasize the "transparently altering criteria"
part of the above use case, and to move users towards a
more explicit and model of usage where a subquery should
be created and used explicitly using the aliased()
construct, which is now very mature and can be used in ways
that were not available when from_self() was first introduced.
On the SQLAlchemy side, from_self() has proven to be one
of the most difficult features to maintain and test
as it can easily lead to extremely complicated scenarios, and
while I am also experimenting with some alternatives that
may still retain some of the "automatic translation" features,
those features are still proving to add similar internal
complexity which is having me lean towards the original
plan of removing open-ended "entity translation" features
like that of from_self() at least through the start
of the 2.0 series.
A code search for all of Openstack shows that the
two files modified here are the only usages of the
from_self() method throughout all of searchable Openstack
code. This speaks to the general obscurity of this method,
although neutron's Subnet code is actually using this
method as intended. The new approach necessarily changes
some of the method signatures here so that the explicit
"subquery" entity can be passed; code searches again
show that these methods are not being called anywhere
outside, so the query_filter_service_subnets method
becomes the private _query_entity_service_subnets method.
References: https://github.com/sqlalchemy/sqlalchemy/issues/5368
Closes-Bug: #2004263
Change-Id: Icec998873221ac8e6a1566a157b2044c1f6cd7f3
In order to decide if a port is a hardware offloaded port by just
reading the OVN Logical_Switch_Port register, it is needed the
VNIC type and the binding profile cababilities.
The maintenance task will only update those LSP ports that belong
to direct Neutron ports to avoid an unnecessary database load.
Related-Bug: #1998608
Change-Id: I5febe9d3eeef6c5b5f6d972b9e8ebfef541458ac
The ``neutron-remove-duplicated-port-bindings`` script removes the
duplicated port binding registers ("ml2_port_bindings" table) that
have status=INACTIVE.
This patch also removes the corresponding port binding levels
("ml2_port_binding_levels" table) associated to those inactive port
bindings.
Closes-Bug: #2000078
Change-Id: I12fa0764cd0ff509f1859b61060d64cc5a54a7b9
This updates the exception with a log message informing that
multi-segments is supported by OVS only at that point.
This also add fullstack tests that validates multisegs deployment on a
physnet.
Closes-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: I3811a4ca28906dd29100c602de7fa4a3595393ab
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
some remaining ones in miscellaneous directories.
Also cleanup any remaining code that I missed in this
series, or has changed since I started.
Trivialfix
Change-Id: I17b4779020a7bfb369c3e721ab6638cd4a6ab50c
The table "router_extra_attributes" is a child of "router" table.
Each register contains extra information that completes the router
description. When using ML2/OVS mechanism driver, the methods that
create and populate the "router_extra_attributes" register are always
called from the L3 DVR, L3 HA and availability zones extensions.
When using ML2/OVN, those extensions are not loaded and therefore the
"router_extra_attributes" register is not created.
Despite this register is currently not used in ML2/OVN (it will be in
future features), there are some project expecting the
"router_extra_attributes" register to be always created (for example,
neutron-dynamic-routing [1]).
This patch enforces the child register creating always when a router is
created. This register is populated with the default values. This new
register does not affect any current operation related to ML2/OVN nor
ML2/OVS.
There is a 1:1 relationship between "routers" and
"router_extra_attributes". The child register is deleted by the database
engine when the "routers" register is deleted (ondelete="CASCADE").
[1]https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/863713
Closes-Bug: #1995974
Change-Id: Ic546e40513402fa101c9687acce382cd6b84356c
Use "value > constant" syntax and not vice-versa. Also
removed disable of misplaced-comparison-constant in
.pylintrc so future ones are caught.
Trivialfix
Change-Id: I733864e7437213bfb6edde24f207b2c9861998c6
A new script to remove the duplicated port bindings was added. This
script will list all ``ml2_port_bindings`` records in the database,
finding those ones with the same port ID. Then the script removes
those ones with status=INACTIVE. This script is useful to remove
those leftovers that remain in the database after a failed live
migration.
"dry_run" mode is possible if selected in "[cli_script] dry_run"
boolean config option. The duplicated port bindings are printed in
the shell but not deleted.
Related-Bug: #1979072
Change-Id: I0de5fbb70eb852f82bd311616557985d1ce89bbf
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/objects.
Trivialfix
Change-Id: I5e885e2b2a09d48197de30d8c6de5d74c10f5987
In ``check_baremetal_ports_dhcp_options``, the ports need to be
retrieved based on the VNIC type. The VNIC type is stored in
"ml2_port_bindings" table, not "ports" table. Because the ML2Plugin
``get_ports`` method can filter only by "ports" database register
fields, it is needed to retrieve those ports using a custom database
query and then use the plugin method to retrieve the ports, converted
to dictionaries and the extensions processed.
Closes-Bug: #1979643
Change-Id: Iab227d9de75e4b1b927043ce27bdc346066478e8
Port create/update are most time-consuming operations
on subnet creation. As example, in those cases where
several subnets are created over the same network the
response time for those port operations is linearly
increased as the total subnets increases.
This patch improves the number of queries required on port
operations in order to reduce the response time.
Closes-Bug: #1977831
Change-Id: I0fccf36a2035e8f6c2fa8dab0307358da600c8f7
The method ``_get_filter_query`` returns an active query and must
be called always inside a DB context.
Closes-Bug: #1976357
Change-Id: Idea915bd71bfb778f5009be9cf47eace83e11437
Christian Rohmann