Extend the get_hosts_mapped_with_segments method to add
optional filters to include/exclude based on agent type.
Uses a joined query, when both include and exclude
filtering is used togheter the exclude filter is most
significant.
Partial-Bug: #2040172
Change-Id: I2cfd52a2657fad989e24e974fda470ecd960262b
Signed-off-by: Harald Jensås <hjensas@redhat.com>
Trying to run the TestSegmentHostMappingNoStore class
unit tests fails if run standalone:
{0} neutron.tests.unit.extensions.test_segment.
TestSegmentHostMappingNoStore.
test_no_segmenthostmapping_when_disable_segment [0.001858s] ... FAILED
Error is about a missing option:
oslo_config.cfg.NoSuchOptError: no such option service_plugins
in group [DEFAULT]
It needs to register the common config options.
Trivialfix
Change-Id: I3e91d0019535b7d8f96d21fa4e255222c1db18d1
As part of the Secure RBAC community goal, we should switch options
"enforce_new_defaults" and "enforce_scope" to be True by default.
It will be still possible to fallback to old policy rules by configuring
those config options to False in Neutron config.
Change-Id: I09c0026ccf87e6c0bb1fa59165c03dc508fba6fa
This module needs to ensure that common config options (service_plugins
option especially) is registered. Otherwise this test module can't be
successfully run without other tests as it was then failing due to
unregistered config option.
Trivial-fix
Change-Id: I0031bb4dd16976b699bbd4b6b211acf8255a6ef1
This updates the exception with a log message informing that
multi-segments is supported by OVS only at that point.
This also add fullstack tests that validates multisegs deployment on a
physnet.
Closes-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: I3811a4ca28906dd29100c602de7fa4a3595393ab
During the Zed PTG it was decided to handle unsupported features in
Neutron as experimental. See section titled "When we say something is
not supported?", day 2 in [1]. The agreement was:
"We keep existing jobs for linuxbridge driver for example, but when the
tests start to fail we skip them and finally we stop the job also.
To make it clear for operators we add warning logs highlighting that the
given feature/driver is experimental, and introduce cfg option to enable
such features explicitly."
This commit implements this agreement, initially with Linuxbridge
Depends-On: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/845646
[1] https://lists.openstack.org/pipermail/openstack-discuss/2022-April/028164.html
Change-Id: Ib18efa3f472736b58c8967847b1061da0e3897d7
It is available in Neutron lib since version 1.16 so pretty long time
now.
Also use segment api definition from neutron-lib, it's available
since version 1.19.0. The api definition from neutron-lib also
avoids circulary dependency b/w standard-attr-segment and segment
extension[1].
[1] https://review.opendev.org/c/openstack/neutron-lib/+/577866
Change-Id: I13699f8c494a15d8bb9e13f767f2725f7cab9f4f
Related-Bug: #1765008
Not Neutron nor other active projects use "PortBindingMixin" class or
the table "portbindingports" anymore.
Closes-Bug: #1956980
Change-Id: I34424a271f6c66cd99852c6109a96a4dcf374913
When:
* the segments service plugin is enabled and
* we have multiple rpc worker processes (as in the sum of rpc_workers
and rpc_state_report_workers, since both kind processes agent
state_reports) and
* many ovs-agents report physnets,
then rabbitmq dispatches the state_report messages between the workers
in a round robin fashion, therefore eventually the state_reports of the
same agent will hit all rpc workers.
Unfortunately all worker processes have a 'reported_hosts' set to
remember from which host it has seen agent reports already. But right
after a server start when that set is still empty, each worker will
unconditionally write the received physnet-segment information into
the db. This means we multiply the load on the db and rpc workers by
a factor of the rpc worker count.
This patch tries to reduce the load on the db by adding another early
return before the unconditional db write.
Change-Id: I935186b6ee95f0cae8dc05869d9742c8fb3353c3
Closes-Bug: #1952730
When two segments are concurrently created this could have resulted in
both threads creating a segment, thus resulting in two segments with
different segmentation ids. To prevent this we now introduce a new
unique constraint onto the networksegments table, which requires
(network_id, network_type, physical_network) to be unique, which allows
only a single segment with a single segmentation id to exist per
combination of these three values.
With the constraint in place a DB error will be thrown, which will cause
allocate_dynamic_segment() to be executed again and this time it will
find the already existing segment. To make sure that no additional DB
objects are created when segment creation failed we need to put all of
the allocation code into a DB transaction.
Change-Id: I407ae88d69ed971bf8d9a9b79120366f33bb56fd
Closes-Bug: #1791233
Some functional and unit tests uses range and len built-in functions to iterate
over items. Those can be replaced by enumerate which makes the code more
readable.
Change-Id: Icb507681578dbb50128aaf017bd598d98c26a6d4
This patch switches over to the payload style of callbacks for
NETWORK based events. As part of this change a few shims are needed
to handle cases where some callbacks don't yet use payloads and others
do. Once we move over to payloads for all callbacks the shims can be
removed.
NeutronLibImpact
Change-Id: I889364b5d184d47a79fe6ed604ce13a4b334acfa
This patch switches over to payload style of callbacks for the remaining
SEGMENT based callback events. Unit tests are also updated as needed
to handle the switch.
NeutronLibImpact
Change-Id: I3624228238babc64fe02bf0cc56eb0c1f6d35e73
- pass existing network dict to PRECOMMIT_DELETE ml2 handler;
- skip MTU and segment update handling in case network is
about to be deleted.
Above gives up to 50% net delete speed-up
Change-Id: I07c70db027f2ae03ffb5a95072e019e8a5fdc411
This change is needed for enabling floating IP's on routed networks.
To be able to create a subnet that spans all segments of a routed network,
a special subnet service type of 'network:routed' is used to denote a
network that can span all segments of a routed network.
To create floating IP's on a routed network, the subnet must be created
with a service_type of 'network:routed'. After the subnet has been
created, floating IP's can be allocated and associated as before.
See the design spec https://review.opendev.org/#/c/486450/ for
reference.
One caveat for this approach is that it requires the underlying
infrastructure to be aware of and able to route /32 host routes
for the floating IP. This implies that in practice, use of the
'network:routed' service type should be done in conjunction with
one or both of the following:
1. Third-party SDN backend that handles this service type in its
own way
2. neutron-dynamic-routing and the BGP service plugin for announcing
the appropriate next-hops for floating IP's. This is compatible
with DVR and non-DVR environments.
Depends-On: Ibde33bdacba6bd1e9c41cc69d0054bf55e1e6454
Change-Id: I9ae9d193b885364d5a4d90538880d8e9fbc8df74
Co-Author: Thomas Goirand <zigo@debian.org>
Partial-Bug: #1667329
When a network is deleted, only one Placement call per segment
is done to remove the associated (if existing) resource provider.
Before this patch, each time a subnet was deleted, the segment
resource provider was updated. When no subnets were present in the
related segment, the associated resource provider was deleted.
This optimization improves the network deletion time (see Launchpad
bug). E.g.: a network with two segments and ten subnets, the Neutron
server processing time dropped from 8.2 seconds to 4.4 seconds (note
that the poor performance was due to the modest testing environment).
Along with the segment RP optimization during the network deletion,
this patch also skips the router subnet update. Because all subnets
in the network are going to be deleted, there is no need to update
them during the network deletion process.
Change-Id: Ifd50027911a9ca3508e80e0de9a6cc45b67006cf
Closes-Bug: #1878916
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line
inconsistency.
Fixed an inconsistency in the OVSBridge.deferred() definition
as it needs to also have an *args argument.
Fixed an issue where an l3-agent test was mocking
functools.partial, causing a python3.8 failure.
Unit tests only, removing from tests/base.py affects
functional tests which need additional work.
Change-Id: I40e8a8410840c3774c72ae1a8054574445d66ece
On subnet delete, the supplied subnet in the post hook
could contain a subnet without certain items, leading to
a KeyError in the segment plugin. Fix a number of these
occurences so this cannot happen.
Also fixed similar code in the segment tests.
Change-Id: I645610febde446b78ed6edd868e699673648a4de
Closes-bug: #1868724
When a port has only one IP allocation on auto-allocation
subnet which is associated with a segment, do not allow
the delete of the subnet. Raise SubnetInUse exception instead.
Related: rhbz#1803989
Related-Bug: #1864225
Related-Bug: #1864333
Closes-Bug: #1865138
Change-Id: I9fb0f05ede42afa1a349635b1936028edf540a1f
When creating additional subnets with ipv6 auto-addressing
ip allocation was added to existing ports without filtering
on current allocation's segment.
This adds fitering to only add auto-address allocation when
the new subnet is on the same segment as the ports current
ipam allocations.
Related: rhbz#1803989
Related-Bug: #1864225
Related-Bug: #1865138
Closes-Bug: #1864333
Change-Id: I75ae14c64db076434ca9897ba9a6d97702e233ad
For v6_stateless IP addresses for all stateless
subnets within a network are implicitly included.
When using segments implicitly allocating addresses
across subnets on different segments is incorrect.
IPs from subnets on differnt segments was allocated
when no host binding information was available
but a subnet_id in fixed_ips request was present.
This change adds filtering based on segment_id when
fixed_ips are used. If fixed_ips are not all on the
same segment exception FixedIpsSubnetsNotOnSameSegment
is raised.
Related: rhbz#1803989
Related-Bug: #1864333
Related-Bug: #1865138
Closes-Bug: #1864225
Change-Id: I336ae76283f29dd226344fb454aaa0e4aac030ea
The patch in the Depends-On line changed the class of an exception
raised in neutron-lib. This change adapts neutron code where we expected
the exception changed.
Change-Id: I34f7d2aab0af0b985d9165465fe9e7c656041745
Depends-On: https://review.opendev.org/695205
When updating a port with the fixed_ips request the
fixed_configured argument should be set to true when
calling _ipam_get_subnets() so that all subnets are
returned if host is not set.
Otherwise the ip allocation will be deffered and an
empty list of possible subnets for the port is
returned. Which in turn led to raising an error that
the network requires subnets to allocate an IP
address.
Closes-Bug: #1844124
Change-Id: I2e690ea0cf5fa0614e39be2b0e83afad3daa7f48
Today a number of classes define their supported_extension_aliases
using static strings rather than API definition ALIASes. This patch
switches them to use the ALIAS where applicable.
Change-Id: I716270c68a9fcd850c3c26de31bc13ea16def23d
IP allocation was initially deffered due to lack of binding
information. On port update the with both `mac_address` and
`binding_host_id`` in the request 'fixed_ips: []' was
appended to the new_port data. This caused the check for
fixed_ips_requested to return True, which in turn cause
deferred_ip_allocation to evaluates False.
Only set the new_port default fixed_ips to original_ips if
the original port had fixed_ips.
Closes-Bug: #1811905
Change-Id: If98a82f8432b09a29f9d0cc6627e9649b43bc4a1
This patch adds a check to determine if the 'segments' service plugin is
enabled. The segment-host mapping db table should only be saved and
updated to the db table if the users configure the 'segments' service
plugin in the config file. The data should be available only in a routed
network resource situation.
Change-Id: I65a42aa2129bef696906a18d82575461dc02ba21
Closes-Bug: #1799328
Originally segments extension used it's own placement client.
Now in neutron-lib there is a working actively designed and tested
client for placement, thus we can change segments to use that, and
remove the old placement client from neutron.
Change-Id: I2e7b53fdedb1837ddb0d685718d9a811ddac5733
Depends-On: https://review.openstack.org/580665
If the segment_id in the update request equals the segment_id
already associated with the subnet being updated, there is no
need for further update related segment validations.
Closes-Bug: #1793391
Change-Id: I65cd5b92b219c14dd07ae72457af243756c4ed02
Ensure that host routes are maintained for each subnet within
a network. Subnets associated with different segments on the
same network get host_routes entries added/removed as subnets
are created, deleted or updated.
This change handle the host_routes for the peer subnets on the
same network when a subnet is created or deleted.
Also adds a shim api extension.
APIImpact: Host routes are now calculated for routed networks.
Closes-Bug: #1766380
Change-Id: Iafbabe6352283e7f1a535a7b147bd81fb32f0ed1
Ensure that host routes are maintained for each subnet within
a network. Subnets associated with different segments on the
same network get's host_routes enties added/removed as
subnets are created, deleted or updated.
This change handle the host_routes for the subnet that is
created or updated.
Partial-Bug: #1766380
Change-Id: If6792d121e7b8e1ab4c7a548982a42e69023da2b
The unittests do some mocks of the keystoneauth Session contructor to
make sure Sessions are being constructed properly. In keystoneauth 3.7.0
we added two new options that can be used from oslo.config files, and
those options in turn now get passed to the Session constructor - which
breaks the mocks.
The thing is, neutron doesn't need to test that
load_session_from_conf_options works, keystoneauth tests that. What we
want to test here is that setting various config options has a specific
result. So instead of testing the entire args list (which could change)
test for the specific results we're concerned about.
Change-Id: I3048cdb8f8b07f6b243a0ab7d08484278c60995f
Needed-By: https://review.openstack.org/569447
With the work done in [1], [2] and [3], we should be able to normalize
the description for segments by relying on that from the
standardattrdescription description.
This patch removes the 'description' from the segment extension's
attribute map as well as adds the standard attribute description and
shim extension in [1] as required extensions to ensure the description
is in fact added. Also the UTs are updated to inject the std attr
description into the segment API attributes for testing and no longer
test that description can be None (not supported with std attr
description).
Additional work is needed to close up the partial bug herein:
- Updating the segment API def/ref in neutron-lib.
- Consuming the former in neutron so that the segment extension uses
the API from neutron-lib.
[1] https://review.openstack.org/#/c/562320/
[2] https://review.openstack.org/#/c/558318/
[3] https://review.openstack.org/#/c/562345/
Change-Id: I693222b7041f4f6f7f6a1096a22727a66364636a
Partial-Bug: 1757513
To enable the possibility to migrate a non-routed network to a
routed network allow updating the segment_id of a subnet.
Only allow the operation if:
- The network only has one segment
- The network only has one subnet
- The current segment_id == None
APIImpact: The segment_id attribute of subnets now allows put operation.
Closes-Bug: #1692490
Depends-On: Iffda823a149a1143f46ee9a05e9640b34bf42c51
Change-Id: I1aee29dfb59e9769ec0f1cb1f5d2933bc5dc0dc5
It looks the sorting and pagination are natively supported in
segment db layer but the segment controller doesn't mark these
features as supported. This patch fixes it.
Change-Id: I325396bb0256196ee6c6fc42cc812a0c75ae3827
Closes-Bug: #1763161
This patch removes a handful of unused exceptions from
neutron.common.exceptions as follows:
The following exceptions already live in neutron-lib and are thus
duplicates in neutron:
PlacementEndpointNotFound
PlacementResourceProviderNotFound
PlacementInventoryNotFound
PlacementAggregateNotFound
PlacementInventoryUpdateConflict
NetworkVlanRangeError
PhysicalNetworkNameError
MacAddressGenerationFailure
The following exception is not used anywhere today:
TenantNetworksDisabled
NeutronLibImpact
Change-Id: Ie8227954681292ead1da0dedc950c54028ce84f1
The tests are failing when executed separately with -m testtools.run
because they rely on another test class to register ml2 and ml2_vlan
option sections.
TrivialFix
Change-Id: I7a9a5c7c7f62783aceccd9e6ea62aacd9753d0d4
Currently, errors related to lack of microversion support are completely
masked because of the way eventlet.spawn_n() works. These changes
improve error handling so that deployers are explicitly notified when
Nova API microversions are not supported. This is useful because without
microversion support, the entire flow will break and hosts will not be
associated with segments properly. Deployers will have no idea why
because errors are not printed to log files setup by the standard Python
logging framework.
Related-Bug: #1759004
Change-Id: I9c6151ffa90394612e7f81ededd4a309dd8ab7fb
Zuul