MechanismDriverContext has an attribute _plugin_context, which carries
the current context with it. This is used by many ml2 drivers, as it is
the only way for them to get the current context. We now make this a
public API by adding a property to MechanismDriverContext that returns
_plugin_context as a read-only attribute.
Change-Id: If9b05655286f42081cf26c90c563429ca2e63244
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/plugins.
Trivialfix
Change-Id: Id9138652f5f07ef12fa682e182fe210019e8f975
When processing ports that are incompatible with l2_population,
certain methods do unnecessary db calls to retrieve agent port
info from the db. This info is used only if the processed
port is of the correct network type. The check is done after
the db call, but before the info is used.
This commit moves the check earlier in the method. The db will
be called only when the port is supposed to be set up with the
l2pop.
Closes-Bug: #1983558
Change-Id: I5a83bfce60a933af781b1fd96037c7de6b2b1f38
Currently the ovs agent calls update_device_list with the
agent_restarted flag set only on the first loop iteration. Then the
server knows to send the l2pop flooding entries for the network to
the agent. But when a compute node with many instances on many
networks reboots, it takes time to readd all the active devices and
some may be readded after the first loop iteration. Then the server
can fail to send the flooding entries which means there will be no
flood_to_tuns flow and broadcasts like dhcp will fail.
This patch fixes that by renaming the agent_restarted flag to
refresh_tunnels and setting it if the agent has not received the
flooding entries for the network.
Change-Id: I607aa8fa399e72b037fd068ad4f02b6210e57e91
Closes-Bug: #1853613
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues. Didn't think it was going to be
close to 100 files when I started.
Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
When the vlan and vxlan both exist in env, and l2population
and arp_responder are enabled, if we update a port's ip address
from vlan network, there will be arp responder related flows
added into br-tun, this will cause too many arp reply for
one arp request, and vm connections will be unnormal.
Closes-Bug: #1824504
Change-Id: I1b6154b9433a9442d3e0118dedfa01c4a9b4740b
Ovs-agent can be very time-consuming in handling a large number
of ports. At this point, the ovs-agent status report may have
exceeded the set timeout value. Some flows updating operations
will not be triggerred. This results in flows loss during agent
restart, especially for hosts to hosts of vxlan tunnel flow.
This fix will let the ovs-agent explicitly, in the first rpc loop,
indicate that the status is restarted. Then l2pop will be required
to update fdb entries.
Closes-Bug: #1813703
Closes-Bug: #1813714
Closes-Bug: #1813715
Closes-Bug: #1794991
Closes-Bug: #1799178
Change-Id: I8edc2deb509216add1fb21e1893f1c17dda80961
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
With high concurrency more than 1 port may be activated on an
OVS agent at the same time (like VM port + a DVR port),
so the patch mitigates the condition by checking for 1 or 2
first active ports.
Given that the condition also contains "or self.agent_restarted(context)"
which makes it True first 180 sec (by default) after agent restart,
I believe the downside of changing 1 to 2 should be negligible.
Please see bug for more details on the issue.
Closes-Bug: #1789846
Change-Id: Ieab0186cbe05185d47bbf5a31141563cf923f66f
When HA router's interface on host is going DOWN but router
is still available on this host, L2 population
mechanism driver will now send to other hosts info to remove
fdb unicast entries to this port on host.
It will not send FLOODING_ENTRY because this port is still on
host but in standby mode and might be transformed to master
in future.
This solves issue with migration router from Legacy to HA.
In such case, port which was originally attached to legacy
router is transformed to be HA backup port before changing
its status to DOWN.
Now in such case unicast entries to this port and backup
node will be removed properly so packets to HA router will
be really send to host which is master node for router.
Closes-Bug: #1785582
Change-Id: Icc14e5f5d40fc6fbb49e0f7b18cc3b15ebec8508
neutron-openvswitch-agent will refresh flows when it's restarted.
But the port's binding status is not changed, update_port_postcommit
will be skipped at function '_update_individual_port_db_status' in
'neutron/plugins/ml2/plugin.py', l2pop don't handle DVR ports, the
fdb entries about DVR port will not be added.
So, we can't skip DVR port at notify_l2pop_port_wiring when agent
is restared.
Closes-Bug: #1773286
Change-Id: I54e3db4822830a0c83daf7b5150575f8d6e2497b
Fix W503 (line break before binary operator) pep8 warnings
and no longer ignore new failures.
Trivialfix
Change-Id: I7539f3b7187f2ad40681781f74b6e05a01bac474
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
The error sneaked in with Ib6e59ab3405857d3ed4d82df1a80800089c3f06e
where is_ha_router_port expects a NeutronContext object but we still
pass PortContext instead.
Change-Id: I593af5d050de00ddea7d758007d9856c4b97695f
Closes-Bug: #1703938
The well known service type constants are in
neutron_lib.plugins.constants, but for legacy reasons a few still exist
and are referenced from neutron_lib.constants that we'd like to remove.
This patch switches references over to neutron_lib's plugin constants.
Change-Id: I1861448cec303725b30cef8f42029f467f9e03a3
The ml2 MechanismDriver is now in neutron-lib along with its associated
constants. This patch switches over to the lib versions of those, but
leaves a shim of the MechanismDriver that just ref's the driver from
lib. This shim allows our broad consumer base of the driver to switch
over at their leisure.
NeutronLibImpact
Change-Id: I99e3de6d933a1bb341394f85415fb07306a82a01
This patch integrates Router Extra Attributes OVO and uses proper
context in calling methods for object operations.
The other integration parts of this OVO in l3_agentschedulers_db.py
and l3_attrs_db.py are being done in patch [1] and [2] respectively.
[1] - I0af665a97087ad72431d58f04089a804088ef005
[2] - Id5ed0a541d09cd1105f9cb067401e2afa8cd9b83
Change-Id: Ib6e59ab3405857d3ed4d82df1a80800089c3f06e
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
get_agent_by_host can return None in the l2pop
driver so we need to check for that case before
we blindly try to decode configuration values on
the result.
There are a couple of cases that can lead to this.
* The deployment can be misconfigured and is missing
either a tunneling_ip option for the agent on a
host or is missing an L2 agent with that host_id
entirely.
* Multiple mech drivers are in use and a port is being
deleted from an agentless host.
Related-Bug: #1533013
Closes-Bug: #1672564
Change-Id: I1e79f600172edad1e31e8231a0a6a2c55f46804c
In delete_port_postcommit, a DVR port (port['device_owner'] =
DEVICE_OWNER_ROUTER_SNAT) can match on l2pop_db.HA_ROUTER_PORTS[1],
but can not get any fdb entries by _get_ha_port_agents_fdb. Then,
the fdb_entries[network_id]['ports'] is been overwritten by {}. So
the associated flow entries will not be deleted.
Closes-Bug: #1668277
Change-Id: I7b621157fe85945acd99e4f08b6370d2f9c3d44d
Introduce get_reader_session() and get_writer_session()
and replace get_session() with them.
Mark get_session as depricated.
Stop using get_engine from legacy facade. Use writer engine for
places where it is required.
Partially-Implements blueprint: enginefacade-switch
Change-Id: I28b741bfa27bf04cbe273586e6e3e00e14fbe683
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
Refactoring neutron ml2 plugin l2pop driver config opts to be in
neutron/conf/plugins/ml2/drivers/l2pop so that all the
configuration options for l2pop drivers reside in a centralized
location. This simplifies the process of looking up the l2pop driver
config opts and provides an easy way to import.
Change-Id: Ia06c0c31ef40c691d67cfc5d514fdff907f5dca4
Partial-Bug: #1563069
This patch introduces context in methods where it is needed for
object operations done during the integration of Router Extra
Attributes OVO.
Used in integration patch: https://review.openstack.org/#/c/381209/
Change-Id: I4dea67207220a19abf5d3cc754f02150b3621550
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
_get_agent_fdb may return None so we need to check for
that before we try to iterate over a key inside of it
in delete_port_postcommit.
Closes-Bug: #1622996
Change-Id: I2256df0e08380e550f32248fb9589ee43b0923ff
This patch makes L3 HA failover not depended on neutron components
(during failover).
All HA agents(active and backup) call update_device_up/down after wiring
the ports. But l2pop driver is called for only active agent as port
binding in DB reflects active agent. Then l2pop creates unicast and
multicast flows for active agent.
On failover, flows to new active agent is created. For this to happen -
all of database, messaging server, neutron-server and destination L3
agent should be active during failover. This creates two issues -
1) When any of the above resources(i.e neutron-server, .. ) are dead,
flows between new master and other agents won't be created and
L3 Ha failover is not working. In same scenario, L3 Ha failover will
work if l2pop is disabled.
2) Packet loss during failover is higher as above neutron resources
interact multiple times, so will take time to create l2 flows.
In this change, we allow plugin to notify l2pop when update_device_up/down
is called by backup agents also. Then l2pop will create flood flows to
all HA agents(both active and slave). L2pop won't create unicast flow for
this port, instead unicast flow is created by learning action of table 10
when keepalived sends GARP after assigning ip address to master router's
qr-xx port. As flood flows are already created and unicast flow is
dynamically added, L3 HA failover is not depended on l2pop.
This solves two isses
1) with L3 HA + l2pop, failover will work even if any of above agents
or processes dead.
2) Reduce failover time as we are not depending on neutron to create
flows during failover.
We use L3HARouterAgentPortBinding table for getting all HA agents of a
router port. HA router port on slave agent is also considered for l2pop
distributed_active_network_ports and agent_network_active_port_count
Closes-bug: #1522980
Closes-bug: #1602614
Change-Id: Ie1f5289390b3ff3f7f3ed7ffc8f6a8258ee8662e
As the L2population mechanism driver does not impact VLAN transparency
support, this patch overrides the check_vlan_transparency method from
the MechanismDriver base class and returns 'True', In this way,
inclusion of the L2population mechanism driver in the list of configured
mechanism_drivers does not prevent the creation of VLAN transparent
networks.
Change-Id: I8b26e899180544e2ad49ef470b9624015fd4d5de
Closes-Bug: #1611920
Now the ML2 core plugin maps driver errors to MechanismDriverError
and hides the error details from the caller.
This patch change MechanismDriverError from an instance of
NeutronException to an instance of MultipleExceptions. Add add
exceptions from mechanism driver as inner_exceptions of
MultipleExceptions. As a result, the api layer will unwrap the
MechanismDriverError and return the real error to client.
Change-Id: I3a46932848d59f7f027640bfb598650f064b0a12
Closes-bug: #1273730
The issue might happen when VMs are intensively created/deleted.
With the patch deleted ports will be just skipped.
Closes-Bug: #1610303
Change-Id: I32b0de9c452cf973d687c72e8381584012c9f3b4
As part of making DVR portbinding implementation generic, we rename
dvr portbinding functions as distributed portbinding functions.
In next patch we make dvr logic for port binding generic,
to be useful for all distributed router ports(for example, HA).
Partial-Bug: #1595043
Partial-Bug: #1522980
Change-Id: I402df76c64299156d4ed48ac92ede1e8e9f28f23
* Detect neutron.i18n import (neutron._i18n is recommended)
* Check builtins _ usage
* 'builtins = _' in tox.ini is no longer required.
* Introduce hacking rule doctest framework.
Newly added check_builtins_gettext() hacking check takes
token as argument. It is not a good idea to pass a tokenized
line manually. Instead it is reasonable to use docstring based
tests used in hacking repo.
Change-Id: Ib7464658fc4c8a6f1b03af6ab46f0bd3ee0bfb18
Currently neutron DCHP scheduler assumes that that every server running
a dhcp-agent can reach every network. Typically the scheduler can
wrongly schedule a vlan network on a dhcp-agent that has no reachability
to the network it's supposed to serve (ex: network's physical_network
not supported).
Typically such usecase can append if:
* physical_networks are dedicated to a specific service and we don't
want to mix dnsmasqs related to different services (for
isolation/configuration purpose),
* physical_networks are dedicated to a specific rack (see example
diagram http://i.imgur.com/NTBxRxk.png), the rack interconnection can
be handled outside of neutron or inside when routed-networks will be
supported.
This change makes the DHCP scheduler network reachability aware by
querying plugin's filter_hosts_with_network_access method.
This change provides an implementation for ML2 plugin delegating host
filtering to its mechanism drivers: it aggregates the filtering done by
each mechanism or disables filtering if any mechanism doesn't overload
default mechanism implementation[1] (for backward compatibility with
out-of-tree mechanisms). Every in-tree mechanism overloads the default
implementation: OVS/LB/SRIOV mechanisms use their agent mapping to filter
hosts, l2pop/test/logger ones return empty set (they provide to "L2
capability").
This change provides a default implementation[2] for other plugins
filtering nothing (for backward compatibility), they can overload it to
provide their own implementation.
Such host filtering has some limitations if a dhcp-agent is on a host
handled by multiple l2 mechanisms with one mechanism claiming network
reachability but not the one handling dhcp-agent ports. Indeed the
host is able to reach the network but not dhcp-agent ports! Such
limitation will be handled in a follow-up change using host+vif_type
filtering.
[1] neutron.plugin.ml2.driver_api.MechanismDriver.\
filter_hosts_with_network_access
[2] neutron.db.agents_db.AgentDbMixin.filter_hosts_with_network_access
Closes-Bug: #1478100
Co-Authored-By: Cedric Brandily <zzelle@gmail.com>
Change-Id: I0501d47404c8adbec4bccb84ac5980e045da68b3
This fixes the problem that when two or more ports in a network
are migrated to a host that did not previously have any ports in
the same network, the new host is sometimes not told about the
IP/MAC addresses of all the other ports in the network. In other
words, initial L2population does not work, for the new host.
This is because the l2pop mechanism driver only sends catch-up
information to the host when it thinks it is dealing with the first
active port on that host; and currently, when multiple ports are
migrated to a new host, there is always more than one active port so
the condition above is never triggered.
The fix is for the ML2 plugin to set a port's status to DOWN when
its binding info changes.
This patch also fixes the bug when nova thinks it should not wait
for any events from neutron because all ports are already active.
Closes-bug: #1483601
Closes-bug: #1443421
Closes-Bug: #1522824
Related-Bug: #1450604
Change-Id: I342ad910360b21085316c25df2154854fd1001b2
None of the L2populationDbMixin methods actually use 'self' for
anything. As the class is basically just used as a namespace and
modules already provide that, this patch gets rid of the mixin. This
makes the code simpler and easier to debug as inheritance doesn't buy
us anything in this case.
Change-Id: Ibf4dfe49a2ebc32d3909d3d7b579d2bb2ea3f61d
When l2pop is enabled and an unbound port is deleted l2pop mech
driver raises an exception as a result of patch:
https://review.openstack.org/#/c/263471/
As a result of the same patch, when an unbound port's IP
address is changed l2pop sends a fanout RPC message needlessly.
Change-Id: Ia81c03dcdf7aef9528c9c2b9527399251fa6aad7
Closes-Bug: #1533013
It's a method called 'get_port_infos' that returned
4 values, most of which were not a function of the port.
What the method did or its naming was very unclear to me
so I broke it up according to its usage.
Change-Id: Ib1fd589e9123fc5009f0887dd400c550efa687a3
- This does NOT break other projects that rely on neutron.i18n,
as this change includes a debtcollector shim to maintain those
older entry points, until they can migrate.
- Also updates _i18n.py to the latest pattern defined by oslo_i18n
- Guidance and template are from the reference:
http://docs.openstack.org/developer/oslo.i18n/usage.html
Partially-Closes-Bug: #1519493
Change-Id: I1aa3a5fd837d9156da4643a367013c869ed8bf9d
This patch is a regression issue for patch[1].
Only those agents which expose tunneling-ip will be considered in
determining data-path tunnels in deployments with l2pop ON.
[1] https://review.openstack.org/#/c/236970/
Change-Id: I7c3b911d5e7448b4e8dee15bb50df33a6e9d5cfe
Closes-Bug: #1407959
Per the discussion in bug 1362242, the log statement doesn't
appear to be associated with anything bad happening. Therefore,
this patch lowers the log level from WARNING to DEBUG to avoid
polluting operator's logs.
Change-Id: I55b6d1d30ed2ed55659c9a6809531cfcbc57f17c
Signed-off-by: Ryan Moats <rmoats@us.ibm.com>
Closes-Bug: 1362242
Check that a port mac address hasn't changed during the precommit
phase of the port update rather than the post commit so the resulting
exception actually stops it from happening.
Change-Id: I62f120b3c954fa4251a7d676cf2c623e6da5a98b
Closes-Bug: #1486379
Sebastian Lohff
