Remove B303 (md5, sha1 for python<3.9) and
remove B311 (Standard pseudo-random generators are
not suitable for security/cryptographic purpose) from
the skip list of bandit execution.
Change-Id: I6e9e61e7f94dc9ca339942529af8997adef45e38
We saw auto_schedule_routers took over 40 seconds
for a DVR enabled environment with option
auto_schedule_routers enabled.
Adding new arguments to get_router_agents_count and
dealing with routers separately depending on whether
it is a regular router or HA. The benefits are
we do not need to loop over every router available in
environment. Another reason for doing this is that
get_router_agents_count is used solely to heal
routers with less than required agents so number of
routers with less agents is small for most of the times.
Related-Bug: #1973656
Change-Id: Ic29275815a8c32cee7a6470509687a18fa594514
This patch is covering an edge case that could happen when the number
of DHCP agents ("dhcp_agents_per_network") or L3 agents
("max_l3_agents_per_router") has been reduced and there are more agents
assigned than the current number. If the user removes any agent
assignation from a L3 router or a DHCP agent, it is possible to remove
first the lower binding assigned registers.
Now the method ``get_vacant_binding_index`` calculates the number of
agents bound and the number required. If a new one is needed, the
method returns first the lower binding indexes not used.
Closes-Bug: #2006496
Change-Id: I25145c088ffdca47acfcb7add02b1a4a615e4612
If not set availability_zone_hits when create router, should use
configuration parameter default_availability_zones.
At present, only the creation parameters are validate, and the default
availability zones not validate.
Creating a network is the same as creating a route.
Closes-bug: #1938261
Change-Id: I1c7f50b69a31d725b762e3061f09a0bd5b077a58
This reverts commit 062336e59b.
Now, we have proper fix for the system_scope='all' in elevated context
in the neutron-lib so we can revert temporary fix made at the end of the
Wallaby cycle.
Related-Bug: #1920001
Conflicts:
neutron/api/rpc/agentnotifiers/dhcp_rpc_agent_api.py
neutron/common/utils.py
neutron/db/address_group_db.py
neutron/services/segments/db.py
Change-Id: Ife9b647b403bdd76a8a99984ea8858bf95c96bc3
In case when enforce_new_defaults is set to True and new policy rules
are used, context.is_admin flag isn't really working as it was with old
rules.
But in case when elevated context is needed, it means that we need
context which has full rights to the system. So we should also set
"system_scope" parameter to "all" to be sure that system scope queries
can be done with such elevated context always.
It is needed e.g. when elevated context is used to get some data from
db. In such case we need to have db query which will not be scoped to
the single project_id and with new defaults to achieve that system_scope
has to be set to "all".
Proper fix for that should be done in neutron-lib and it is proposed
in [1] already but as we are have frozen neutron-lib version for
stable/wallaby already this patch for neutron is temporary fix for that
issue.
We can revert that patch as soon as we will be in Xena development cycle
and [1] will be merged and released.
[1] https://review.opendev.org/c/openstack/neutron-lib/+/781625
Related-Bug: #1920001
Change-Id: I0068c1de09f5c6fae5bb5cd0d6f26f451e701939
This patch implements the last code bits pending to
conclude the new DB engine facade migration.
Due to the resultant interactions in the modified code, is
not possible to submit smaller patches; this code must be
migrated at once.
Partially-Implements blueprint: enginefacade-switch
Signed-off-by: Slawek Kaplonski <skaplons@redhat.com>
Co-Authored-By: Rodolfo Alonso Hernandez <ralonsoh@redhat.com>
Change-Id: Id3f09b78c8d0a8daa7ec4fa6f5bf79f7d5ab8f8b
With python 3.x, classes can use the metaclass= logic
to not require usage of the six library.
One step in removing all of six usage from neutron.
Change-Id: I2f815e412d9a96eb5faf2b3bb3a1e393a9db9309
Michael Bayer while analysing neutron process function call trace,
suggested to run queries against specific columns rather than full
ORM entities as it can help reduce load both at the DB level and
in the Python level since they are much faster to fetch as
non-ORM entities. In this patch we are trying that on simpler
queries to improve neutron performance.
Co-Authored-By: Joe Talerico <jtaleric@redhat.com>
Change-Id: I6a41e9487a4427f876442bbeeae61974e892225e
A publically consumed db api is available in neutron-lib. This patch
consumes the autonested_transaction by removing it's def from neutron
and replacing all such usages to now use neutron-lib's version of
autonested_transaction.
NeutronLibImpact
Change-Id: I8d7cbb9b90151d252959e9fce8937f3e4e456811
The externally consumed APIs from neutron.db.api were rehomed into
neutron-lib with https://review.openstack.org/#/c/557040/
This patch consumes the retry_db_errors function from lib by:
- Removing retry_db_errors from neutron.db.api
- Updating the imports for retry_db_errors to use it from lib
- Using the DB API retry fixture from lib in the UTs where applicable
- Removing the UTs for neutron.db.api as they are now covered in lib
NeutronLibImpact
Change-Id: I1feb842d3e0e92c945efb01ece29856335a398fe
Commit I81748aa0e48b1275df3e1ea41b1d36a117d0097d added the l3 extension
API definition to neutron-lib and commit
I2324a3a02789c798248cab41c278a2d9981d24be rehomed the l3 exceptions,
while Ifd79eb1a92853e49bd4ef028e7a7bd89811c6957 shims the l3
exceptions.
This patch consumes the l3 api def by:
- Removing the code from neutron that's now in lib.
- Using lib's version of the code where applicable.
- Tidying up the related unit tests as now that the l3 api def from lib
is used the necessary fixture is already setup in the parent chain when
setting up the unit test class.
NeutronLibImpact
Change-Id: If2e66e06b83e15ee2851ea2bc3b64ad366e675dd
The router_ids argument to auto_schedule_routers() is
unused, and was marked for deprecation in Queens.
Change-Id: Ie97b1ad05e294b5fe763ae8d7319800eb16ea3dc
Commit I1d4ded9959c05c65b04b118b1c31b8e6db652e67 rehomed the
availability zone extension's API definition into neutron-lib. This
patch consumes it, removing the rehomed logic in neutron and switching
over to lib's version of it.
NeutronLibImpact
Change-Id: I761381de0d6e26a0380386700e7921b824991669
Refactoring neutron db config opts to be in neutron/conf/db/ and
neutron/conf/agent/database/ so that all the configuration options
reside in a centralized location. This simplifies the process of
looking up the config opts and provides an easy way to import.
Change-Id: I4da9bb48d49b99e8c2b34a5c1b83e7eb95b70b82
Partial-Bug: #1563069
This was removed in I227ca60422545e40d3bbb8baf2b41a8ce14f4294 and
it is no longer being exercised.
TrivialFix
Change-Id: Idf9611675be871fac31d4cf9b4335c7ace911bb1
This patch introduces and integrates OVO for L3 HA Router.
Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com>
Co-Authored-By: Vu Cong Tuan <tuanvc@vn.fujitsu.com>
Change-Id: I3463921dec415dd073503ab9470588193d08ce87
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
When a router is migrated between DVR+HA and HA(i.e DVR+HA->HA and
HA->DVR+HA), redundant "network:router_ha_interface" ports are created.
For binding a HA router to a agent, existing code can create multiple
ports, but is using only one port and other ports remain still in DB
until router is deleted.
In this patch, when duplicate entry is detected while binding,
we remove the port which is created at that time for binding.
Closes-Bug: 1715370
Change-Id: I0c6c5f9dfce1bf2d99672b5cf9e072bfb0228a17
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
The L3 HA RPC code that creates HA interfaces can race
with an HA router deletion on the server side. The L3 HA
code ends up creating a port on the HA network while the
server side is deleting the router and the HA network.
This stops the L3 HA network from being deleted because
it has a new port without a bound segment, which leaves the
HA network in a segmentless condition and no ports after
the L3 RPC code cleans up its port.
This adjusts the L3 RPC logic to attempt an HA network cleanup
whenever it encounters the concurrent router deletion case
to ensure that the HA network gets cleaned up.
To make this more robust in the future, we may need the L3
HA code to recognize when an HA network has no segments and
automatically create a new one.
Change-Id: Idd301f6df92e9bc37187e8ed8ec00004e67da928
Closes-Bug: #1696537
This patch integrates the OVO created for RouterL3AgentBinding into
the code base.
Change-Id: I0af665a97087ad72431d58f04089a804088ef005
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
This patch proposes a (rather major) refactor to the L3 scheduler.
Basically, the auto_schedule_routers() code-path was split to 2
different code-paths, each dealing with a different case (unscheduled
routers vs underscheduled routers), in addition to the API-initiated
schedule() logic. This patch removes the 2 code-paths in favor of moving
most of the logic into schedule(). While the result is a slightly
longer schedule(), the benefit is that a lot of the previous
unmaintainable code-paths of auto_schedule_routers() are now removed.
Yay! :D
Related-Bug: #1609738
Change-Id: I227ca60422545e40d3bbb8baf2b41a8ce14f4294
The option was deprecated [1] for removal in Newton
and is being removed in Ocata.
[1] Deprecated in patch with Gerrit Change-Id of:
I8a5fc74a96c784d474aefe2d9b27eeb66521ca82
DocImpact remove all references to the option.
Change-Id: I3a9195ff6fd18fad9f85cec03a632e7e52d954e7
Closes-Bug: #1555042
This reverts commit 9c3c19f07c.
Following the merge of Ie98d5e3760cdb17450aea546f4b61f5ba14baf1c, the
creation of new router uses RouterL3AgentBinding and its' new
binding_index attribute to ensure correctness of the resources. As such,
the ALLOCATING state (which was used to do just that) is no longer
needed and can be removed.
Closes-Bug: #1609738
Change-Id: Ib04e08df13ef4e6b94bd588854a5795163e2a617
In some cases, the creation of the ha_vr_id could have occurred twice.
For example, if a router was created with a given external_gateway then
a vr_id would be allocated twice: once from super().create_router()
(which triggers the scheduler on its own), and once from the
create_router() logic.
This patch modifies some code paths to make this allocation more robust.
For example the code that allocates a new vr_id will now also assign it
to the router in the same transaction, to make sure atomicity.
Closes-Bug: #1654032
Change-Id: I82c33aee5cfcc086f60fc74ed4d7bd7d443a3370
Change I3447ea5bcb7c57365c6f50efe12a1671e86588b3 introduced a new
running-index for RouterL3AgentBinding, binding_index, which helps to
keep count of how many bindings a router has for each agent (and how
many bindings in total). Since we were able use this DB column to make
sure concurrency doesn't break on creating a new HA router, we also
postponed the creation of L3HARouterAgentPortBinding to after the first
binding was successfully created.
This patch proposes a change to the way routers are scheduled to an
agent: when creating a new HA router, no L3HARouterAgentPortBinding
entities will be created until after the corresponding
RouterL3AgentBinding was successfully created.
In other words, instead of pre-creating the L3HARouterAgentPortBinding
objects without assigning it to an agent, we'll create them only after
the RouterL3AgentBinding were successfully created.
Related-Bug: #1609738
Change-Id: Ie98d5e3760cdb17450aea546f4b61f5ba14baf1c
Solve a problem with an unexpected NoneType returned by
_get_routers_can_schedule called from within
_schedule_ha_routers_to_additional_agent when using:
router_scheduler_driver =
neutron.scheduler.l3_agent_scheduler.AZLeastRoutersScheduler
This was leading to problems with starting neutron-l3-agent on network
nodes, causing HA routers to fail to start.
Closes-Bug: #1641879
Change-Id: I33c5a6214670f0ada9c2293b0eb2ff243f6f7b1b
Change I3447ea5bcb7c57365c6f50efe12a1671e86588b3 added a binding_index
to the RouterL3AgentBinding table. In certain (concurrent) cases, a row
with the same binding_index might be used twice, which will raise
DBDuplicateEntry. However, that change didn't retry on this case at all
code-paths, so this patch rectifies this issue.
Closes-Bug: #1633042
Change-Id: I228b0084a8e7c48e78bc2ea6a3fccf7437210e73
Change Idef2fe3e16b245da849e2d29c5578e5f5d081dc4 relocated the
RouterL3AgentBinding model, but the last rebase didn't change all the
references of this model - there was one left.
Change-Id: Id48d48413ecc2f55fc889b8d19a6760c16434d96
Partial-Bug: #1597913
In the l3_agent_scheduler.py file, some functions accept both the
'plugin' and 'context' argument. However, some functions expect
'context, plugin' (context first) and some functions expect
'plugin, context' (context last). I'm a real nit-picker and this
bothered me for a while, so here's a fix :)
Since the base scheduler class expects 'plugin, context', some functions
couldn't be changed to accept the other variation. Instead, context will
always be last. Also, modified unit tests to make sure they test.
This also fixes an odd-ordering in one of the dhcp scheduler's private
functions.
Change-Id: I825e108170a29d5ecaa0f0883bb0a171b5fdb895
As there would be issue of cyclic imports while
implementation of objects for l3agentbinding which has
db models definition and mixins in same file, this patch will
relocate l3agentbinding models.
Change-Id: Idef2fe3e16b245da849e2d29c5578e5f5d081dc4
Partial-Bug: #1597913
This adjusts the L3 scheduler framework to ask the L3 plugin if
a given router should be scheduled before scheduling it. To maintain
backwards compatibility, this new method is implemented to return True
in the base class.
The L3 plugin with flavor support overrides this to lookup the driver
associated with a router and check if the driver requires the L3
scheduling framework. This will allows the coexistence of flavors that
needs scheduling and flavors that don't.
Change-Id: I17a64c59eaf5d8605ff8ec2a29e491673be960e7
Implements: blueprint multi-l3-backends
The patch proposes adding a new binding_index to the
RouterL3AgentBinding table, with an additional Unique Constraint that
enforces a single <router_id, binding_id> per router. This goes a long
way into fixing 2 issues:
1. When scheduling a non-HA router, we only use binding_index=1. This
means that only a single row containing that router_id can be
committed into the database. This in fact prevents over-scheduling of
non-HA routers. Note that for the HA router case, the binding_index
is simply copied from the L3HARouterAgentPortBinding (since they are
always created together they should always match).
2. This sets the ground-work for a refactor of the l3 scheduler - by
using this binding and db-based limitation, we can schedule a router
to agents using the RouterL3AgentBinding, while postponing the
creation of L3HARouterAgentPortBinding objects for the agents until
they ask for it (using sync_routers). This will be a major
improvement over todays "everything can create
L3HARouterAgentPortBinding" way of things).
Closes-Bug: #1535557
Change-Id: I3447ea5bcb7c57365c6f50efe12a1671e86588b3
When l3 agents do fullsync, the server configured router_auto_schedule
will schedule ha routers whose l3 agent count don't reach
max_l3_agents_per_router to the agent. Need to check compatibility
before binding.
Change-Id: Ie9869eae2bd01c19da5820b4e7ecb7cba7b5692a
Closes-Bug: 1598422
This adjusts the create_object_with_dependency helper function
to attempt to cleanup any dependency it was responsible for creating
if it encounters a failure in trying to attach a child to the
dependency.
Change-Id: I363f3a299c55e5063b4239028728bb5593132010
Related-Bug: #1548285
elajkat