Remove B303 (md5, sha1 for python<3.9) and
remove B311 (Standard pseudo-random generators are
not suitable for security/cryptographic purpose) from
the skip list of bandit execution.
Change-Id: I6e9e61e7f94dc9ca339942529af8997adef45e38
When a HA router is created and the HA is not yet, before creating
the router, the Neutron server creates the HA network and the
corresponding subnet.
The HA network cannot be duplicated (see previous patches related to
this bug). But the subnet, that is created in another database
transaction, cannot be present when the router creation call tries
to create the HA port.
This patch adds a HA subnet check before creating the router and the
HA port. Even if the subnet check fails and the worker tries to
create this subnet, if the process fails with ``InvalidInput``, that
means other worker created the subnet before and the current one
fails because tries to create the same subnet with the same CIDR.
In this case, we dismiss the exception and continue with the router
creation.
Closes-Bug: #2016198
Change-Id: I82225fcc6248bb0fd68959ceb1daabff423d81ff
This patch implements the new network HA boolean field API extension.
This field is an input only parameter for POST operations (creation).
By default is "False". When enabled, the Neutron server will create
a ``ha_router_networks`` register in the same transaction of the
network creation.
If by any circumstance (a race condition, for example), another
``ha_router_networks`` exists in the same project, a
``DBDuplicateEntry`` exception will be raised and the transaction
will be rolled back.
Partial-Bug: #2016198
Change-Id: Ie42c13ecbe4abcad9229b71f6942e393fd0f2e4e
After updating pylint, it started emitting additional "E"
warnings in some cases, fix them.
unsubscriptable-object,
unsupported-delete-operation
These were associated with the OVN AgentCache code. Instead
of using a subscript, create get/delete methods to do the
same thing.
used-before-assignment
Re-factor some code so it's clear to pylint variables are
being assigned properly.
Trivialfix
Change-Id: I4a5ccb7f33465705e59b5274c41db3c371862b1e
If someone removes the port with device owner router_ha_interface,
then we can get unexpected router behavior like doubling
arp response packets. This patch prohibits removing such a port.
Closes-Bug: #2008270
Change-Id: Ief031801c1a3e3dd64e6cbf65e27f04f2bef9cba
The table "router_extra_attributes" is a child of "router" table.
Each register contains extra information that completes the router
description. When using ML2/OVS mechanism driver, the methods that
create and populate the "router_extra_attributes" register are always
called from the L3 DVR, L3 HA and availability zones extensions.
When using ML2/OVN, those extensions are not loaded and therefore the
"router_extra_attributes" register is not created.
Despite this register is currently not used in ML2/OVN (it will be in
future features), there are some project expecting the
"router_extra_attributes" register to be always created (for example,
neutron-dynamic-routing [1]).
This patch enforces the child register creating always when a router is
created. This register is populated with the default values. This new
register does not affect any current operation related to ML2/OVN nor
ML2/OVS.
There is a 1:1 relationship between "routers" and
"router_extra_attributes". The child register is deleted by the database
engine when the "routers" register is deleted (ondelete="CASCADE").
[1]https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/863713
Closes-Bug: #1995974
Change-Id: Ic546e40513402fa101c9687acce382cd6b84356c
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/db.
Trivialfix
Change-Id: I9311cfe5efc51552008072d84aa238e5d0c9de60
The goal of this patch is to make the Neutron code compliant
with SQLAlchemy 2.0.
All SQL transactions must be executed inside an explicit
writer/reader context. SQLAlchemy no longer will create an
implicit transaction if the session has no active transaction.
A warning message, only available in debug mode, is added. When
an ORM session calls "do_orm_execute", if there is no active
transaction, a warning message with a traceback will be logged
to help to debug the regression introduced.
Related-Bug: #1964575
Change-Id: I3da37fee205b8d67d10673075b9130147d9eab5f
This patch switches over to callback payloads for ROUTER
AFTER_CREATE, AFTER_UPDATE and AFTER_DELETE events.
Change-Id: Ie818ffbb1a291faa80501157b46ff6671d5c26ba
This patch switches over to callback payloads for ROUTER
BEFORE_CREATE, PRECOMMIT_CREATE, BEFORE_UPDATE and
PRECOMMIT_DELETE events.
Change-Id: I4a52c773d3f753c918df0986f1d261083156651c
This reverts commit 062336e59b.
Now, we have proper fix for the system_scope='all' in elevated context
in the neutron-lib so we can revert temporary fix made at the end of the
Wallaby cycle.
Related-Bug: #1920001
Conflicts:
neutron/api/rpc/agentnotifiers/dhcp_rpc_agent_api.py
neutron/common/utils.py
neutron/db/address_group_db.py
neutron/services/segments/db.py
Change-Id: Ife9b647b403bdd76a8a99984ea8858bf95c96bc3
This method, when called from outside an active session, must create
a new one, given the context.
This patch also changes the L3HARouterAgentPortBinding.port and
L3HARouterAgentPortBinding.agent relationships to be "joined". That
will retrieve the port and the agent DB registers in the main query
and the returned object won't need to make subqueries to retrieve
them. The SQL query looks like
http://paste.openstack.org/show/806275/.
Change-Id: I7870b869a755054ef1989f86ebdd3470ec5cf435
Closes-Bug: #1930397
Method _ensure_vr_id_and_network is not used in l3_hamode_db.py,
hence method and related tests can be removed.
Closes-Bug: #1715371
Change-Id: I9a64e683f060573195a88b3c4ee34abbd941fa44
In case when enforce_new_defaults is set to True and new policy rules
are used, context.is_admin flag isn't really working as it was with old
rules.
But in case when elevated context is needed, it means that we need
context which has full rights to the system. So we should also set
"system_scope" parameter to "all" to be sure that system scope queries
can be done with such elevated context always.
It is needed e.g. when elevated context is used to get some data from
db. In such case we need to have db query which will not be scoped to
the single project_id and with new defaults to achieve that system_scope
has to be set to "all".
Proper fix for that should be done in neutron-lib and it is proposed
in [1] already but as we are have frozen neutron-lib version for
stable/wallaby already this patch for neutron is temporary fix for that
issue.
We can revert that patch as soon as we will be in Xena development cycle
and [1] will be merged and released.
[1] https://review.opendev.org/c/openstack/neutron-lib/+/781625
Related-Bug: #1920001
Change-Id: I0068c1de09f5c6fae5bb5cd0d6f26f451e701939
This patch implements the last code bits pending to
conclude the new DB engine facade migration.
Due to the resultant interactions in the modified code, is
not possible to submit smaller patches; this code must be
migrated at once.
Partially-Implements blueprint: enginefacade-switch
Signed-off-by: Slawek Kaplonski <skaplons@redhat.com>
Co-Authored-By: Rodolfo Alonso Hernandez <ralonsoh@redhat.com>
Change-Id: Id3f09b78c8d0a8daa7ec4fa6f5bf79f7d5ab8f8b
There is no real reason we should be using some of the
terms we do, they're outdated, and we're behind other
open-source projects in this respect. Let's switch to
using more inclusive terms in all possible places.
Change-Id: I99913107e803384b34cbd5ca588451b1cf64d594
With python 3.x, six.text_type and six.string_type
are just str.
Also removed a six.integer_type since it was the only
one left in a file.
Another step in removing all of six usage from neutron.
Change-Id: I5208dc41bff1983ecd323286f427296b722da62a
There are some extreme conditions which will result the unbound
router gateway port. Then all the centralized floating IPs will
not be reachable since the gateway port was set to 4095 tag.
This patch adds the HA status to the router related port
processing code path. If it is HA router, the gateway port
will go to the right HA router processing code branch.
Closes-Bug: #1827754
Change-Id: Ida1c9f3a38171ea82adc2f11cb17945d6e2434be
HA routers are using keepalived and needs to have virtual_router_id
configured. As routers which belongs to same tenant are using same
ha network, those values have to be different for each router.
Before this patch this value was always taken as first available value
from available_vr_ids range.
In some (rare) cases, when more than one router is created in parallel
for same tenant it may happen that those routers would have same vr_id
choosen so keepalived would treat them as single application and only
one router would be ACTIVE on one of L3 agents.
This patch changes this behaviour that now random value from available
vr_ids will be chosen instead of taking first value always.
That should mittigate this rare race condition that it will be (almost)
not noticable for users.
However, proper fix should be probably done as some additional
constraint in database layer. But such solution wouldn't be possible to
backport to stable branches so I decided to propose this easy patch
first.
Change-Id: Idb0ed744e54976dca23593fb2d7317bf77442e65
Related-Bug: #1823314
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
NeutronLibImpact
Depends-On: https://review.openstack.org/#/c/647836/
Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
During L3 agent restart, the dvr ha router gateway port
binding host may change because the multiple ha router
scheduled hosts.
After this patch, we return the 'master' ha binding host
directly during the gateway port create. And do not let
the original 'master' (current is backup) host override
the gateway port binding host.
Closes-Bug: #1793529
Change-Id: Icb2112c7f0bd42c4f4b1cf32d6b83b6d97f85ef7
The retry_if_session_inactive decorator was rehomed into neutron-lib
[1]. This patch consumes it by removing the function from neutron and
using neutron-libs version where appropriate.
NeutronLibImpact
[1] https://review.openstack.org/#/c/557040/
Change-Id: I3e3289f33e62d45933d0fbf165bb4b25078f22d5
The method _create_ha_port_binding returns only single binding
but add_ha_port using it use plural bindings while safe_execution
of create_ha_port_binding. I noticed this while converting l3ha_model
to ovo in [1], which required access to db_obj of bindings.
[1]. https://review.openstack.org/#/c/602497
Change-Id: Ifabed41c59b2a946e968fdb0748c17e654bd4fcc
Based on the current import style guide,
https://docs.openstack.org/charm-guide/latest/coding-guidelines.html#import-style
it is recommended to not import a function directly, but instead
import the module and use module.function().
Do this for the is_distributed_router function in l3_hamode_db.
Trivialfix
Change-Id: Ie52f3a2480f337d90535baed90a4b6d824d3a51f
The remainder of the neutron.plugins.common.utils were rehomed into
neutron-lib with [1][2]. This patch consumes them by using the functions
from neutron-lib, and removing the neutron.plugins.common.utils module
all together as it's fully rehomed now.
NeutronLibImpact
[1] https://review.openstack.org/#/c/560950/
[2] https://review.openstack.org/#/c/554546/
Change-Id: Ic0f7b37861f078ce8c5ee92d97e977b8d2b468ad
After the addition of a new resource and related events with [1],
this patch adds the necessary notifications for l3 flavor,
resource(ROUTER_CONTROLLER) and events(PRECOMMIT_ADD_ASSOCIATION and
PRECOMMIT_DELETE_ASSOCIATIONS) so that l3 flavor driver can subscribe to
them when flavor is changed.
Apply callback priority to ensure that the ordering of callback the
following.
- l3_*_db callbacks to extend l3 extended attributes
This callbacks need to be called first so that rest callbacks can
see those extended attributes.
- l3 driver controller callbacks
- l3 flavor driver callbacks
extra routes/l3_gwmode/l3_hamode need care because they are
updated via update_router but within different db transaction.
[1] I1e72ee843851004d26410a90da4030ab3b024741
Closes-Bug: #1745633
Co-Authored-By: Manjeet Singh Bhatia<manjeet.s.bhatia@intel.com>
Change-Id: If20b11f0587f1ed30db72d97c15b20d4c6e87543
Depends-On: https://review.openstack.org/#/c/541766/
Fix W503 (line break before binary operator) pep8 warnings
and no longer ignore new failures.
Trivialfix
Change-Id: I7539f3b7187f2ad40681781f74b6e05a01bac474
A HA router should always be active unless all of the agents hosting
that router go down. In that event, the router should switch to
standby. This behavior changed with review:
https://review.openstack.org/#/c/411784
That review seemed to be accounting for a flakey message bus. This
change should account for that, but also revert to the original behavior
of the router state only changing when its backing agent hosts are down.
Change-Id: I89c3b2546382624f175f8de4de621c3e53adf527
Closes-Bug: 1682145
This patch switches callbacks over to the payload object style events
[1] for PRECOMMIT_UPDATE based notifications. To do so a DBEventPayload
object is used with the publish() method to pass along the related data.
In addition a few UTs are updated to work with the changes. Finally
a few shims are put into place to allow PRECOMMIT_UPDATE based events to
use payloads while still supporting the existing kwarg style events.
NeutronLibImpact
[1] https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html#event-payloads
Change-Id: Ie6d27df01cd7b87894efc80946d41eb1ebe25bef
The is_extension_supported function now lives in neutron-lib. This patch
removes the function from neutron and uses lib's version instead.
NeutronLibImpact
Change-Id: Iccb72e00f85043b3dff0299df7eb1279655e313e
The L3_AGENT_MODE_DVR_NO_EXTERNAL and DVR_SNAT_BOUND constants were
rehomed into neutron-lib with Ieb9374f5483a0ab2306592ab901686ca374db1c8
This patch consumes them by removing them from neutron and using the
constants from neutron-lib instead.
NeutronLibImpact
Change-Id: Ib63a523721a2fa3d1a978a729de28e6a2e560ef6
There is a race condition in update_routers_states that may result in
some fixed ips incorrectly deallocated from router ports. This may
happen if update_routers_states fetches ports' state before another
thread updates the list; then update_routers_states passes port payloads
with old fixed ips into update_port, which results in ip address
deallocation. Among other things, l3 agent will detect the change and
remove the affected subnet prefix from radvd configuration file, since
it doesn't configure extra_subnets for RA.
There is no need to pass full port payload into update_port just to set
host. This patch replaces the payload with a dict of one key - host.
This allows core plugin to handle just this host field change, leaving
existing allocations (and other port attributes) intact.
Change-Id: Ib2c661d6e2cb8e34676fd83e19b6cf65c232545d
Closes-Bug: #1743658
Commit I81748aa0e48b1275df3e1ea41b1d36a117d0097d added the l3 extension
API definition to neutron-lib and commit
I2324a3a02789c798248cab41c278a2d9981d24be rehomed the l3 exceptions,
while Ifd79eb1a92853e49bd4ef028e7a7bd89811c6957 shims the l3
exceptions.
This patch consumes the l3 api def by:
- Removing the code from neutron that's now in lib.
- Using lib's version of the code where applicable.
- Tidying up the related unit tests as now that the l3 api def from lib
is used the necessary fixture is already setup in the parent chain when
setting up the unit test class.
NeutronLibImpact
Change-Id: If2e66e06b83e15ee2851ea2bc3b64ad366e675dd
Before this change, DVR_SNAT agents would get no routers when
asking for updates due to provisioning of DHCP ports on the
node they are running on. This means that there's no connectivity
between the DHCP port and the network gateway (that may be
hosted on a different node), and therefore things like DNS may
break when a VM attempts resolution when talking to the affected
DHCP port.
This change relaxed a conditional that prevents the right list of
routers to be compiled and returned from the server to the agent.
The agent on the other hand needs to make sure to allocate the
right type of router based on what is being returned from the server.
Closes-bug: #1733987
Change-Id: I6124738c3324e0cc3f7998e3a541ff7547f2a8a7
The l3 ext ha mode extension's API definition was rehomed into
neutron-lib with commit Ie407d56cdac6996133fcd855754185c74707e992
This patch consumes the API definition by removing/using the rehomed
code and using the APIExtensionDescriptor for the extension class.
NeutronLibImpact
Change-Id: I8f728c8707172ed7340fb90cce43b885c61938c2
elajkat