Remove B303 (md5, sha1 for python<3.9) and
remove B311 (Standard pseudo-random generators are
not suitable for security/cryptographic purpose) from
the skip list of bandit execution.
Change-Id: I6e9e61e7f94dc9ca339942529af8997adef45e38
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/db.
Trivialfix
Change-Id: I9311cfe5efc51552008072d84aa238e5d0c9de60
In case when 2 neutron servers are trying to remove network from the
same DHCP agent it may happend that one of them will not be able
to find binding object anymore thus deletion of that binding object
will raise ObjectNotFound exception.
This patch adds proper handling of such case to not raise ugly
stacktrace in neutron logs.
Closes-Bug: #1970759
Change-Id: I67d516c4583aa0c20416114b92a6d69ece5b970c
It has been reported by bug #1959750 that for a given network the
agent is scheduling all segments reported by agent.
This commit is fixing the issue by ensuring scheduling segments per
network that they belong to only.
A test has been updated to demonstract the fact that for a given
network we only schedule the related segments.
Closes-bug: #1959750
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: Icd57570004055903c6817a75d814ed65db3fa53c
In case when subnet in segment X was created before hosts from
that segment was mapped in the Neutron DB, network wasn't scheduled
to the DHCP agents in that new segment.
Now DHCP scheduler is triggered in such case.
Closes-bug: #1917811
Change-Id: Ic9e64aa4ecdc3d56f00c26204ad931b810db7599
As spotted in Focal testing patch [0], pep8 test fails with many
C0321 false-positives, reported in pylint as current version does not
support python 3.8 [1]
Use a newer version of pylint and astroid, fixing or disabling some of
the new checks: no-else-*, unnecessary-comprehension, import-outside-toplevel
[0] https://review.opendev.org/#/c/738163/
[1] https://github.com/PyCQA/pylint/issues/2737
Change-Id: Ie646b7093aa8634fd950c136a0eba9adcf56591c
Currently, if dhcp mapping from a network is removed, it is reassigned
to the network. This is because of the Network Scheduler's schedule
function, which considers balancing the networks with the agents, whether
enable_dhcp is set on its subnets or not. It does not take into account
the network_auto_schedule config option. This is particularly disturbing
when considering backends which have their provide their own DHCP.
With this patch, if network_auto_schedule is set to False, networks wont
be automatically scheduled to DHCP Agents. If DHCP is to be mapped to a
network, it can be mapped using the CLI itself.
While it may seem that this change is breaking what is already working,
but as mentioned earlier, if there are network backends which provide DHCP
support themselves, they wont need the automatic mapping, which the term
"network_auto_schedule" actually stands for.
Closes-Bug: #1647421
Change-Id: If1a6a2a174d0f737415efa2abce518722316a77b
The patch proposes adding a new binding_index to the
NetworkDhcpAgentBinding table, with an additional Unique
Constraint that enforces a single <network_id, binding_index>
per network.
1. When a network is triggered to be auto-scheduled to DHCP
agents, the number of DHCP agents is constrained by
dhcp_agents_per_network in neutron.conf. This prevents
too many DHCP agents from being scheduled in the first place.
2. If users manually schedule a network to specific DHCP
agents, the binding_index increments to show the number of
DHCP agents hosting this network.
Co-Authored-By: Oleg Bondarev <obondarev@mirantis.com>
Change-Id: I1bc3f8b69c337f7c1cf7375509a0da61def9baf1
Closes-Bug: #1535554
Neutron-LBaaS has now been retired and there will be no Train
release[1]. This patch removes neutron-lbaas references from
neutron.
[1] https://review.opendev.org/658494
Closes-Bug: #1833125
Change-Id: I0fe3fbaf4adf7fb104632fd94cd093e701e12289
Added log information about the hostable DHCP agents per network.
Added log information when an agent is declared "down" because the
heartbeat is too old.
Those log messages are included according to [1].
[1] https://bugs.launchpad.net/neutron/+bug/1799555/comments/8
Change-Id: I7b30499a86e5ae5de49813dfca01788fd5fce139
Related-Bug: #1799555
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
When a network is removed from a dhcp agent, in some scenarios if the
agent releases its port concurrently, there is chance that the
unscheduling will fail due to that the target port is not found.
Catch the PortNotFound exception as an expected error under this type of
concurrent circumstance and logs it to move forward.
Closes-Bug: #1775496
Change-Id: Ib51b364f6ced0de7685c8ee07c1d292308d919f5
Signed-off-by: Kailun Qin <kailun.qin@intel.com>
The dhcpagentscheduler extension's API definition was rehomed into
neutron-lib with https://review.openstack.org/#/c/520751/
This patch consumes it by using the API definition and it's constants
where applicable.
NeutronLibImpact
Change-Id: Ib0c97268f01885f6daacb3d1cdbbd94bb6020d60
The agent extension's API definition was moved into neutron-lib with
commit Iae4746aa2e3f68a7120696a12a6ae717115dbc31
This patch consumes it by removing the rehomed code and using the
APIExtensionDescriptor for the parent class of the extension.
NeutronLibImpact
Change-Id: I18aebec1e2b5b361d9b96192d4a59e5424c6a80b
Agent object has been merged [1].
This patch uses Agent object in agents_db and test_agents_db.
We also introduce a new function (get_agents_object) and keep
the old function (get_agents_db) for backward compatibility.
[1] https://review.openstack.org/#/c/297887/
Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com>
Change-Id: I4c4283cb1aa05d52dca00cc249e094ea7d55b1d3
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
The neutron-lib commit I360545b6ee4291547e0c5c8e668ad03d3efa4725 moved
the externally consumed globals from neutron.common.constants into lib.
With the exception of PROVISIONAL_IPV6_PD_PREFIX all other constants
in neutron.common.constants should only be used in neutron, and will
hopefully remain that way. External consumers needing access to other
common constants should move them into lib first.
NeutronLibImpact
Change-Id: Ie4bcffccf626a6e1de84af01f3487feb825f8b65
Refactoring neutron db config opts to be in neutron/conf/db/ and
neutron/conf/agent/database/ so that all the configuration options
reside in a centralized location. This simplifies the process of
looking up the config opts and provides an easy way to import.
Change-Id: I4da9bb48d49b99e8c2b34a5c1b83e7eb95b70b82
Partial-Bug: #1563069
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
Renames the AgentStatusCheckWorker class to PeriodicWorker
and moves it into the worker module since there isn't anything
agent-specific about it and it can be used for other periodic
jobs server side.
TrivialFix
Change-Id: Ic7a55ef534f64e6bfc60ae38bb0e139a0078510b
This worker would fail to start again if stop() or reset()
was called on it because of some bad conditional logic. This
doesn't appear to impact the current in-tree use case but
it should behave correctly.
Closes-Bug: #1641788
Change-Id: Id6334c1ef6c99bd112ada31e8fe3746d7e035356
Change I7be4ce2513e49e6da46a7bdffb8538613f0be7c7 relocated the Agent
model (and a few other functions), but some references to old
functions/model still remain. These cause a considerable amount of
warnings when running unit tests and the code itself.
Change-Id: Id026cae75bfa56b1023f8a1c4e5db750cf0bff5f
Partial-Bug: #1597913
This patch set is for breaking the circular dependency between
Agent/AgentVersionedObject.
See:https://review.openstack.org/#/c/297887/ for details.
Change-Id: I7be4ce2513e49e6da46a7bdffb8538613f0be7c7
Partial-Bug: #1597913
Co-Authored-By: Victor Morales <victor.morales@intel.com>
Co-Authored-By: Sindhu Devale <sindhu.devale@intel.com>
Since a network can now be broken up by segments, each segment will
need to have its own DHCP Agent. This maintains backwards
compatibility when a network does not have a segment. However,
once a segment is created on a network, a dhcp agent should be
scheduled per segment with a dhcp enabled subnet.
The scheduling happens by filtering the candidate dhcp agents by
the hosts that are bound to that segment.
Partially-Implements: blueprint routed-networks
Change-Id: If73211978e14b7533a1213cfb8c2c155a408f19e
There is a bug in pep8, when 'select' used, it omits all default checks
and runs only those specified by 'select'. We got hit by this issue
since I2d26534230ffe5d01aa0aab6ec902f81cfba774d was merged which lead to
almost no static checks in pep8 job.
Also note that off_by_default decorator has no effect for now because
factory in hacking is triggered after ignored checks are collected.
There will be a follow-up patch for that in order to make pep8 doing
its job quickly.
[1] https://github.com/PyCQA/pycodestyle/issues/390
Related-Bug: 1594756
Change-Id: I8e27f40908e1bb4307cc7c893169a9d99f3433c4
There were several patches attempting to move threading after forking
workers. By the time of development it was assumed to land in Mitaka but
it eventually landed in Newton [1], deprecating some methods but leaving
wrong name of cycle where methods should be removed.
[1] I3a32a95489831f0d862930384309eefdc881d8f6
Change-Id: Ic30014c0f94da4b6dd0323663d98be3b1cdd77da
This patch set is for breaking the circular dependency between
Agent/NetworkDhcpAgentBinding. The goal of this is to implement
Agent OVO
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Change-Id: I3f2a8bcc6f8644e94e377dc916fba6743cb230bc
This reverts commit b1cdba1696
Original patch was reverted because it broke neutron plugin's
backward compatibility and needed more work.
This patch fixes that problems:
1) original behaviour of add_agent_status_check,
start_periodic_l3_agent_status_check and
start_periodic_dhcp_agent_status_check methods is deprecated but kept
for using in third part plugins for backward compatibility
2) new add_agent_status_check_worker, add_periodic_l3_agent_status_check
and add_periodic_dhcp_agent_status_check method are implemented
instead and are used for implementing plugins in neutron codebase
Closes-Bug: #1569404
Change-Id: I3a32a95489831f0d862930384309eefdc881d8f6
The method to get dhcp agents for networks takes as a an argument
the ability to send a list of network_ids. However, this method
is really only being used to send in a list of one network_id, but
there don't seem to be any usages where it is being used for many
network_ids. Since its not being used, this hasn't been caught yet.
Change-Id: I6f07ed50b29448d279a4fd5f9d392af3a8490340
Closes-Bug: #1587973
The logic of reschedule_routers_from_down_agents() can be useful for
additional types of resource rescheduling such as loadbalancers, as described at
Id8d3218bf1e52722cc10ddcd34e3e734eef90658
Related-Bug: #1565511
Change-Id: I7871d68246d82f343e99730c09f81bcc7800bcce
I think this needs a little more thought. This broke OVN at least and
I don't think that's good. We need to figure out how to be compatible
with existing plugins, even out of tree ones.
This reverts commit 1cafff0871.
Change-Id: Ie087fb11213cc85911483c2d32c463fa9c973e54
Forking a process when multiple threads are running is an unsafe
operation and could cause a lot of problems because only current
thread will continue working in child thread. Any locked by other
thread resource will remain locked forever.
We faced with this problem during oslo.messaging development and
added workaround to hide this problem:
https://review.openstack.org/#/c/274255/
I tried to fix this problem in oslo.service:
https://review.openstack.org/#/c/270832/
but oslo folks said that this fix is ugly and it is wrong way to add
workarounds to common libraries because projects use them incorrectly.
I think that is fair.
So this patch fixes incorrect usage of oslo libraries. In this patch
I extended functionality of NeutronWorker and add there
`worker_process_count` parameter which determines how many processes
should be spawned for this worker. If `worker_process_count` = 0 - don't
create process and spawn thread in scope of current process for worker
Then I moved all background tasks to workers and return them by
`get_workers` method. start_plugin_workers collects plugin's workers
using `get_workers` method and starts in ProcessLauncher first workers
with `worker_process_count` > 0 and only after this starts threaded
workers by simple Launcher
Closes-bug: #1569404
Change-Id: I0544f1d47ae53d572adda872847a56fa0b202d2e
Updating ports is incompatible with the transaction
semantics that happen inside of ML2. See partial-bug for
more details.
This transaction served no purpose wrapped around the port
update because there should only be one port per agent per
network and if that port fails to update, the code to delete
the binding won't be hit anyway.
Change-Id: Ica2c5a19e10ae1773e49537b5ff4ae0eacaeb388
Partial-Bug: #1540844
In case of intermittent DB failures router and network auto-rescheduling
tasks may fail due to error on fetching down bindings from db.
Need to put this queries under try/except to prevent unexpected exit.
Closes-Bug: #1546110
Change-Id: Id48e899a5b3d906c6d1da4d03923bdda2681cd92
Python 3 deprecated the logger.warn method, see:
https://docs.python.org/3/library/logging.html#logging.warning
so we prefer to use warning to avoid DeprecationWarning.
Closes-Bugs: #1529913
Change-Id: Icc01ce5fbd10880440cf75a2e0833394783464a0
Co-Authored-By: Gary Kotton <gkotton@vmware.com>
The previous code was doing a DB lookup for each network's
availability zone which was significantly impacting the
performance of network listings.
This patch adjusts the network model to be automatically joined
to the DHCP agents table that the AZ code uses to populate the
AZs for the network.
Change-Id: I908ceb1a68e0eed7c304e3ff82279ad6fa406167
Closes-Bug: #1525740
elajkat