As part of the Secure RBAC community goal, we should switch options
"enforce_new_defaults" and "enforce_scope" to be True by default.
It will be still possible to fallback to old policy rules by configuring
those config options to False in Neutron config.
Change-Id: I09c0026ccf87e6c0bb1fa59165c03dc508fba6fa
oslo.db 12.1.0 uncovered some issues in functional
tests, Fix:-
- test__get_dvr_subnet_ids_on_host_query by adding a db reader
context as without it _get_dvr_subnet_ids_on_host_query method was
returning a query object with an active db context and other
db transactions can't be executed with an active context.
- test_get_total_reservations_map by adding a db reader context
for get_total_reservations_map call
- _TestIsSessionActive by calling neutron.objects.network as it looks
for active session and since now autocommit is False for active
session we need to have _connections[1] in transaction object.
[1] https://review.opendev.org/c/openstack/neutron/+/843256
Co-authored-by: Yatin Karel <ykarel@redhat.com>
Closes-bug: #1989057
Change-Id: Ic0b91f03b4eea817fa130e28236d7e007b481e30
When I writing 'ndp_proxy' service plugin, I found I couldn't get enough
informations about router from the callback system (Such as: the origin
request body of user send). So, for write service plugin that related
router plugin more concisely I commit this patch.
This patch proposal two changes about router callback publish events:
1. Add 'request_body' parameter to some event's payload
2. add 'BEFORE_UPDATE' event for router gateway
Related-bug: #1877301
Change-Id: I5f6a4e6f0b7c5feb794ddb7efbd07d01bad91af8
This patch switches the code over to the payload style of callbacks [1]
for ROUTER_INTERFACE events for those that are not using them yet.
The unit tests are also updated where needed to account for the
payload style callbacks and publish() method. In addition, a few
callback methods that use the retry_if_session_inactive() decorator are
separated out from the callback so that the context can still be
passed and detected by retry_if_session_inactive logic.
NeutronLibImpact
[1]
https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html
Change-Id: I8d9f8296952dfb10fcccd6afd72e90a5d4f379eb
In case when related dvr router is configured by L3 agent, it is first
added to the tasks queue and then processed as any other router hosted
on the L3 agent.
But if L3 agent will ask neutron server about details of such router,
it wasn't returned back as this router wasn't really scheduled to the
compute node which was asking for it. It was "only" related to some
other router scheduled to this compute node. Because of that router's
info wasn't found in reply from the neutron-server and L3 agent was
removing it from the compute node.
Now _get_router_ids_for_agent method from the l3_dvrscheduler_db module
will check router serviceable ports for each dvr router hosted on the
compute node and will then find all routers related to it. Thanks to
that it will return routers which are on the compute node only because
of other related routers scheduled to this host and such router will not
be deleted anymore.
Change-Id: I689d5135b7194475c846731d846ccf6b25b80b4a
Closes-Bug: #1884527
DEVICE_OWNER_NETWORK_PREFIX should be used for ports
belonging to the network, DEVICE_OWNER_NEUTRON_PREFIX
was only intended to be used for LBAAS ports.
Change-Id: I091f4449979e6e6d5263f58e1d14bc053abe3b3b
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This completes removal of all references to "import mock",
changing to "from unittest import mock" in fullstack and
functional tests.
Added a hacking check to enforce it in future patches.
Change-Id: Ifcaf1c21bea0ec3c35278e49cecc90a101a82113
This patch will remove all the control plane
arp updates for DVR routers. Based on the recent
patch that merged[1], DVR routers will no longer
require the control plane ARP update but will
depend on the ARP Responder for ARP updates.
[1] https://review.opendev.org/#/c/651905/
Change-Id: I538aa6d68fbb5ff8431f82ba76601ee34c1bb181
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues. Didn't think it was going to be
close to 100 files when I started.
Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
When new DVR serviceable port appears on new node we need
to update node's l3 agent with all routers which have the
port's subnets, including connected routers.
We don't need to update all nodes hosting these routers.
It costs us much as all l3 agents then go back to neutron server
and request routers info for no good reason.
This was one of the main issues with DVR at scale fixed in Mitaka.
Change-Id: I99d01d7bf29f236eff0f80d1ae8659f64ac55d39
Related-Bug: #1830456
All of the externally consumed variables from neutron.common.constants
now live in neutron-lib. This patch removes neutron.common.constants
and switches all uses over to lib.
NeutronLibImpact
Depends-On: https://review.openstack.org/#/c/647836/
Change-Id: I3c2f28ecd18996a1cee1ae3af399166defe9da87
Functional test case for [1] to prevent agressive on L3 provider service.
Related DB error: ProviderResourceAssociation.
[1] https://review.openstack.org/#/c/638380/
Change-Id: I95792ad6cf63d13dc790a1ec636860dde82a23c6
Related-Bug: #1816771
In patch [1] I changed most of functional tests classes to
inherit from neutron.tests.functional.base.BaseLoggingTestCase
class to enable logging of results for such tests.
I missed two classes then, so this patch fixes it and adds
inherit from same base class to tests which still didn't have
logging enabled.
[1] https://review.openstack.org/#/c/619266/
Change-Id: I9afbe241ee26c8cdc807d2f4fa1c285ff1f07d60
Patch [1] added handling of AgentNotFoundByTypeHost exception in
create_fip_agent_gw_port_if_not_exists method in
neutron.db.l3_dvr_db.DVRResourceOperationHandler class.
Unfortunatelly there was mistake there and such exception wasn't
catched properly.
That caused issue e.g. in functional-python35 tests but wasn't catched
properly in functional tests using python27.
This patch changes this to handle such exception properly.
[1] https://review.openstack.org/#/c/574917/
Change-Id: If43f2c944ff46d8b05fbcf68231fd04bed147ba0
In case when 2 dvr routers are connected to each other with
tenant network, those routers needs to be always deployed
on same compute nodes.
So this patch changes dvr routers scheduler that it will create
dvr router on each host on which there are vms or other dvr routers
connected to same subnets.
Co-Authored-By: Swaminathan Vasudevan <SVasudevan@suse.com>
Closes-Bug: #1786272
Change-Id: I579c2522f8aed2b4388afacba34d9ffdc26708e3
FloatingIP create throws an error if the L3 agent is not running
on the given host for DVR Routers.
This happens when we try to create a DVR Router in the Legacy
Router deployed cloud.
FloatingIP create checks for the agent type based on the host,
and when agents are not available on the given host, it raises
an exception.
This patch will fixes by handling the exception raised, when the
agent is not available.
Change-Id: I80522d12087495861e832cf1c7a3fe9a7830d386
Closes-Bug: #1776566
This reverts commit fbe308bdc1.
This does not help the ARP update for the unbound Allowed-address-pair
IP, since the temporary ARP update (NUD: reachable) goes to incomplete
state when the router tries to re-ARP for the IP, before it responds to
a VM, since DVR routers does not allow the ARP requests to flow through
the br-tun.
Closes-bug: #1773999
Change-Id: I9977c8cbbbc1e68565249e7f80c59319fe967300
Allowed_address_pair IP when associated with a network port will
inherit the services MAC.
Right now the ARP entry is updated with the last MAC that it is
associated with. But when allowed_address_pair IPs are used in
the context of VRRP the MAC keeps switching between the MASTER
and SLAVE. VRRP instance sends out GARP, but the ARP entry in the
router namespace is not getting updated based on the GARP.
This might cause the VRRP IP and the service using the IP to fail.
Since we having been adding the ARP entry with NUD state as
PERMANENT, the ARP entries are set for ever and does not adopt the
GARP sent out by the VRRP instance.
This will cause instances associated with DVR routers to have a
service interruption.
So the proposed patch will add the ARP entry for the Allowed address
pair with NUD for 'REACHABLE'.
This allows the Allowed_address_pair IP MAC to be updated on the
fly.
Change-Id: I43c3471f5d259e8c2ee1685398a06a4680c0bfcd
Closes-Bug: #1608400
The neutron.common.topics module was rehomed into neutron-lib with
commit Ie88b84949cbd55a4e7ad06341aab77b286cdc485
This patch consumes it by removing the rehomed module from neutron
and using the module from neutron-lib instead.
NeutronLibImpact
Change-Id: Ia4a4604c259ce862597de80c6deeb3d408bf0e95
The L3_AGENT_MODE_DVR_NO_EXTERNAL and DVR_SNAT_BOUND constants were
rehomed into neutron-lib with Ieb9374f5483a0ab2306592ab901686ca374db1c8
This patch consumes them by removing them from neutron and using the
constants from neutron-lib instead.
NeutronLibImpact
Change-Id: Ib63a523721a2fa3d1a978a729de28e6a2e560ef6
Centralized floating IP return to Error state when
the 'dvr_no_external' agent restarts.
The sync data received from the server was not handling
the agent properly and so was not update the 'dvr_snat_bound'
flag.
This would initiate an floating IP Error state.
This patch will fix the issue mentioned above.
Closes-Bug: #1741411
Change-Id: Id1cf26ffba8262ba7b3e5f41faa4cb28ba9dcb7d
Commit 088e317cd2 breaks floating IP
creation.
This changed the API and mandated that 'subnet_id' and
'floating_ip_address' be passed in the API.
This also cleans up tests with invalid inputs.
Change-Id: I7bc758eed56610c269485681f67a5379730d8ef5
Closes-Bug: #1738612
Commit I81748aa0e48b1275df3e1ea41b1d36a117d0097d added the l3 extension
API definition to neutron-lib and commit
I2324a3a02789c798248cab41c278a2d9981d24be rehomed the l3 exceptions,
while Ifd79eb1a92853e49bd4ef028e7a7bd89811c6957 shims the l3
exceptions.
This patch consumes the l3 api def by:
- Removing the code from neutron that's now in lib.
- Using lib's version of the code where applicable.
- Tidying up the related unit tests as now that the l3 api def from lib
is used the necessary fixture is already setup in the parent chain when
setting up the unit test class.
NeutronLibImpact
Change-Id: If2e66e06b83e15ee2851ea2bc3b64ad366e675dd
In the current code, if user specifies floating-ip-address
and subnet, we only process the subnet when creating
the fip port.
This patch adds floating_ip_address and subnet_id to
fip port's fixed_ips, if floating_ip_address is not in the subnet,
InvalidIpForSubnet exception will be raised.
This patch also fixes a default value error in tests.
Change-Id: I436353690839281ca7e13eaf792249306b71dd4b
Closes-Bug: #1732890
The external network extension's API definition was rehomed into
neutron-lib with I9933b91d1e82db3891b3b72f06e94316e56a4f15. This patch
consumes it, switch over to neutron-lib's modules and removing the
rehomed code in neutron.
NeutronLibImpact
Change-Id: I696b52265b9528082cd2524f05febe2338376488
With the current change in allowing the unbound fip
to be associated with the snat node, we are seeing
that all floating IPs that are associated with an
unbound port are created at the snat node.
This is also applicable for floating IPs that are
created just before associating the port to a VM.
We have seen such scenarios in the test cases.
This is the right behavior as per design. But when
the port is bound to a host, the floating IP should
be migrated to the respective host.
This patch fixes the issue by sending notification to
the respective node, when the port is bound and also
clear the fip from the snat node.
Closes-Bug: #1718788
Change-Id: I6b1f3ffc3c3336035632f6a82d3a87b3be57b403
With a recent change to the neutron server code, the server was
processing floating IPs that were not bound to the respective
agent during fullsync operation.
Change to always initialize floating IP host info so callers
can determine if info should be sent to an agent or not.
Also changed the logic that decides when the server should
send a floating IP to an agent to be easier to understand.
Closes-bug: #1713927
Change-Id: Ic916225e0a11c3fb8cd94437ca063e0d3295a569
When router interfaces are added to DVR router, if the router has
gateway configured, then the internal csnat ports are created for
the corresponding router interfaces.
We have seen recently after the csnat port is created if the
RouterPort table update fails, there is a DB retry that is happening
and that retry operation is creating an additional csnat port.
This additional port is not getting removed automatically when the
router interfaces are deleted.
This issue is seen when testing with a simple heat template as
per the bug report.
This patch fixes the issue by calling the RouterPort create with
delete_port_on_error context.
Change-Id: I916011f2200f02556ebb30bce30e349a8023602c
Closes-Bug: #1709774
DVR supports both East/West and North/South routing. While the
SNAT is centralized the DNAT is mostly distributed. There are
certain circumstances where the DNAT might be centralized when
the ports are unbound.
In order to have a well defined behavior and when there are
no external network connectivity available in the compute host,
the DNAT functionality is centralized.
In order to achieve this we are introducing a new agent type
option 'dvr_no_external' to centralize the DNAT.
This new L3 agent type ('dvr_no_external') would only allow the East/West
routing to occur in the compute host and the DNAT or Floating IP will be
configured in the centralized network node.
Change-Id: Ia5d7336e478e0fa5ba62b7ae5ed0c56656116d94
Partial-Bug: #1667877
Unbound ports that are associated with a Floating IP and connected to
DVR Routers will not be serviced by the DVR Routers, unless we bind it
to a valid host.
This server side patch allows the neutron server to schedule the
unbound port Floating IP on the network node or the node with dvr_snat
agent where the SNAT functionality resides.
The DNAT rules for the unbound ports will be configured in the SNAT
namespace on the network node.
Related-Bug: #1583694
Change-Id: I05d0bfb3fa275b1e4e479928000cf8494da858f6
Router scheduling was happening before the schedule call
so sometimes the router was being scheduled to a different
host than the test was expecting.
This fixes it by explicitly setting the router to not be HA
and calls schedule before adding interfaces that would trigger
scheduling.
Closes-Bug: #1698058
Change-Id: I6c6319f1da72546a9d21af198daf9a2a67e6dec8
DVR snat ports and gateway ports are not bound to any host
and so we don't see the snat namespace getting created.
The issue is the _build_routers_list in l3_dvr_db.py is not called due
to the inheritance order.
Change-Id: I56f9de31524aeef262cf2a78be3abf8487c21a12
Closes-Bug: #1695101
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
In order to route traffic between the internal subnets and the
external subnet that belong to the same address_scopes we need
to create the gateway port and the fip namespace irrespective of
the configured floatingips for the internal subnet.
This will consume an additional IP from the external subnet on
all nodes, but with the introduction of service_type networks,
this will not be an issue any more.
This patch is the first in series that creates the agent gateway
port and the fip namespace on every node when the gateway is set
for the router. For every router created it will connect the
router namespace to the fip namespace.
Partial-Bug: #1577488
DocImpact: Document the change in behavior for fip-agent-gw create
Change-Id: I30c4f7fc250e486fe9a71b68540e783e90a6cf15
Neutron-lib 1.1.0 is now out and contains the portbindings
API definition (as per commit [1]). This patch moves neutron
references over to the neutron-lib version.
NeutronLibImpact
- Consumers using the public constants within neutron's
portbindings API extension must now use the values
from neutron-lib.
[1] 87e42f993c07ae320159d5123662ee9f3bd4d903
Change-Id: I669af9b4c712877772d91a03857ab108714001d4
The ML2 plugin was missing a port AFTER_UPDATE event in its
binding commit operations. This was resulting in port binding
results not being serialized for push notifications.
This patch just adds in the AFTER_UPDATE event to the port
binding commit operation.
Partially-Implements: blueprint push-notifications
Change-Id: I67276a3efe97b60499601b1aceeee27ad8961062
This patch proposes the allowance of changing a router's 'distributed'
or 'ha' attributes without any restriction as to the previous state of
these attributes, as long as the router's admin_state_up is False.
Closes-Bug: #1654991
Change-Id: Iad4dd77f74ba01876e0091112a1e24a3463993d1
This gets rid of the inheritance of remove_router_interface
in the DVR mixin and converts it to use the callback framework.
Change-Id: Iaa66549b2c49ad5fe7a2877b6fca9782630ff731
DVR had a complete copy and paste of the l3_db
add_router_interface. This patch gets rid of that
by adding in a some args to existing callbacks. \o/
This also has a variable rename from 'new_port' to
'new_router_intf' to clearly indicate that it's referring
to the creation of a new router interface regardless of
the creation of a core plugin port.
Change-Id: I6192c41419a992be9d0ded338f7a87ebcefda6af
This shouldn't be happening in a transaction. It creates
port resources in ML2 that need to have correct commit
semantics.
This refactors it to leverage the callback framework and
unifies the codepath for creating these DVR ports after
a router creation and after a router update with a migration.
Related-Bug: #1540844
Partially-Implements: blueprint push-notifications
Partially-Implements: blueprint enginefacade-switch
Change-Id: I15d8d32b54087e4a2ca1bfa74b46fa95adc4b95d
The L3DvrTestCaseMigration class was just inheriting
and subsequently running all of the tests in the
L3DvrTestCase class. It doesn't have a separate setup
or mixin from the parent class so it was just duplicating
all of the tests for no reason.
Create a "base" test class so the migration class
can just run a single test.
Change-Id: Idbe347fa26e64c600cf3056d16a99c4ad46a2586
With this change the L3 DVR code for add_router_interface
will always generate ROUTER_INTERFACE AFTER_CREATE notifications,
even in the absence of a gateway on the router.
Change-Id: I2eae49b06cad0c42950ac76f1db7ea6e37e308b9
Closes-Bug: 1629861
Slawek Kaplonski