Extend the get_hosts_mapped_with_segments method to add
optional filters to include/exclude based on agent type.
Uses a joined query, when both include and exclude
filtering is used togheter the exclude filter is most
significant.
Partial-Bug: #2040172
Change-Id: I2cfd52a2657fad989e24e974fda470ecd960262b
Signed-off-by: Harald Jensås <hjensas@redhat.com>
This updates the exception with a log message informing that
multi-segments is supported by OVS only at that point.
This also add fullstack tests that validates multisegs deployment on a
physnet.
Closes-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: I3811a4ca28906dd29100c602de7fa4a3595393ab
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/services.
Trivialfix
Change-Id: Ie9779b257981bc80e69639cdaa4d7dfd0ffa5809
Method ``_add_segment_host_mapping_for_segment`` is called by the event
(resources.SEGMENT, events.PRECOMMIT_CREATE), from
``SegmentDbMixin._create_segment_db``, and is called inside a database
writer context. That means it is irrelevant to check if the session is
active (must be always).
Closes-Bug: #1975542
Change-Id: Ib19dacf886486876237ed1157fb95ae157ed430e
This patch adds debug messages when a segment (or a set of
segments) is mapped to a host (or a set of hosts). The
deletion of an entry is logged too.
Trivial-Fix
Change-Id: I614dc6b8da9596b748db285709f9fbb146560b6f
It is available in Neutron lib since version 1.16 so pretty long time
now.
Also use segment api definition from neutron-lib, it's available
since version 1.19.0. The api definition from neutron-lib also
avoids circulary dependency b/w standard-attr-segment and segment
extension[1].
[1] https://review.opendev.org/c/openstack/neutron-lib/+/577866
Change-Id: I13699f8c494a15d8bb9e13f767f2725f7cab9f4f
Related-Bug: #1765008
When:
* the segments service plugin is enabled and
* we have multiple rpc worker processes (as in the sum of rpc_workers
and rpc_state_report_workers, since both kind processes agent
state_reports) and
* many ovs-agents report physnets,
then rabbitmq dispatches the state_report messages between the workers
in a round robin fashion, therefore eventually the state_reports of the
same agent will hit all rpc workers.
Unfortunately all worker processes have a 'reported_hosts' set to
remember from which host it has seen agent reports already. But right
after a server start when that set is still empty, each worker will
unconditionally write the received physnet-segment information into
the db. This means we multiply the load on the db and rpc workers by
a factor of the rpc worker count.
This patch tries to reduce the load on the db by adding another early
return before the unconditional db write.
Change-Id: I935186b6ee95f0cae8dc05869d9742c8fb3353c3
Closes-Bug: #1952730
This patch switches over to callback payloads for PORT
AFTER_DELETE events.
Some shims were removed.
Change-Id: If69e37b84fe1b027777b1d673b3d08a6651a979e
This patch switches the code over to the payload style of callbacks [1]
for PORT AFTER_CREATE events. In addition it adds a branch/shim to the
dhcp_rpc_agent_api to support both payload and kwarg style callbacks.
NeutronLibImpact
[1]
https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html
Change-Id: I25d43d4f8f2390b07e0d11c631f894d88669bbe0
- pass network dict from ml2 plugin to _create_subnet_postcommit
- skip ipam subnet fetch for non ipv6 auto-address subnets
- don't count subnets (DB request) if subnet has no segment
Change-Id: Iaecfda2700c5316cb25a93496d24ece366e40a4a
This patch switches over to the payload style of callbacks for
NETWORK based events. As part of this change a few shims are needed
to handle cases where some callbacks don't yet use payloads and others
do. Once we move over to payloads for all callbacks the shims can be
removed.
NeutronLibImpact
Change-Id: I889364b5d184d47a79fe6ed604ce13a4b334acfa
This patch switches over to payload style of callbacks for the remaining
SEGMENT based callback events. Unit tests are also updated as needed
to handle the switch.
NeutronLibImpact
Change-Id: I3624228238babc64fe02bf0cc56eb0c1f6d35e73
- pass existing network dict to PRECOMMIT_DELETE ml2 handler;
- skip MTU and segment update handling in case network is
about to be deleted.
Above gives up to 50% net delete speed-up
Change-Id: I07c70db027f2ae03ffb5a95072e019e8a5fdc411
In case when enforce_new_defaults is set to True and new policy rules
are used, context.is_admin flag isn't really working as it was with old
rules.
But in case when elevated context is needed, it means that we need
context which has full rights to the system. So we should also set
"system_scope" parameter to "all" to be sure that system scope queries
can be done with such elevated context always.
It is needed e.g. when elevated context is used to get some data from
db. In such case we need to have db query which will not be scoped to
the single project_id and with new defaults to achieve that system_scope
has to be set to "all".
Proper fix for that should be done in neutron-lib and it is proposed
in [1] already but as we are have frozen neutron-lib version for
stable/wallaby already this patch for neutron is temporary fix for that
issue.
We can revert that patch as soon as we will be in Xena development cycle
and [1] will be merged and released.
[1] https://review.opendev.org/c/openstack/neutron-lib/+/781625
Related-Bug: #1920001
Change-Id: I0068c1de09f5c6fae5bb5cd0d6f26f451e701939
When a network is deleted, only one Placement call per segment
is done to remove the associated (if existing) resource provider.
Before this patch, each time a subnet was deleted, the segment
resource provider was updated. When no subnets were present in the
related segment, the associated resource provider was deleted.
This optimization improves the network deletion time (see Launchpad
bug). E.g.: a network with two segments and ten subnets, the Neutron
server processing time dropped from 8.2 seconds to 4.4 seconds (note
that the poor performance was due to the modest testing environment).
Along with the segment RP optimization during the network deletion,
this patch also skips the router subnet update. Because all subnets
in the network are going to be deleted, there is no need to update
them during the network deletion process.
Change-Id: Ifd50027911a9ca3508e80e0de9a6cc45b67006cf
Closes-Bug: #1878916
Improve port retrieval in method
"_validate_auto_address_subnet_delete". Instead of requesting each
port individually, a single DB query is executed to retrieve all
the ports with IP allocation in a in a subnet.
Change-Id: I7875142ebecd17663e17847fb14997200d7ae5c8
Related-Bug: #1865138
On subnet delete, the supplied subnet in the post hook
could contain a subnet without certain items, leading to
a KeyError in the segment plugin. Fix a number of these
occurences so this cannot happen.
Also fixed similar code in the segment tests.
Change-Id: I645610febde446b78ed6edd868e699673648a4de
Closes-bug: #1868724
When a port has only one IP allocation on auto-allocation
subnet which is associated with a segment, do not allow
the delete of the subnet. Raise SubnetInUse exception instead.
Related: rhbz#1803989
Related-Bug: #1864225
Related-Bug: #1864333
Closes-Bug: #1865138
Change-Id: I9fb0f05ede42afa1a349635b1936028edf540a1f
For v6_stateless IP addresses for all stateless
subnets within a network are implicitly included.
When using segments implicitly allocating addresses
across subnets on different segments is incorrect.
IPs from subnets on differnt segments was allocated
when no host binding information was available
but a subnet_id in fixed_ips request was present.
This change adds filtering based on segment_id when
fixed_ips are used. If fixed_ips are not all on the
same segment exception FixedIpsSubnetsNotOnSameSegment
is raised.
Related: rhbz#1803989
Related-Bug: #1864333
Related-Bug: #1865138
Closes-Bug: #1864225
Change-Id: I336ae76283f29dd226344fb454aaa0e4aac030ea
The patch in the Depends-On line changed the class of an exception
raised in neutron-lib. This change adapts neutron code where we expected
the exception changed.
Change-Id: I34f7d2aab0af0b985d9165465fe9e7c656041745
Depends-On: https://review.opendev.org/695205
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues. Didn't think it was going to be
close to 100 files when I started.
Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
By documentation segments plugin was designed to use placement
microversion 1.1, force to use that.
Change-Id: Ibb8d6bcce7f0fe1070b9eeb2ad632dfb58a3a015
Depends-On: https://review.opendev.org/663978
Related-Bug: #1828543
This patch switches over to payload style callbacks for all
SEGMENT_HOST_MAPPING events.
NeutronLibImpact
Change-Id: I71888b2b72b486c32991d651cdc608adb2149500
The functionality within neutron.db.common_db_mixin is available via
neutron-lib APIs. This patch removes common_db_mixin and updates any
uses of it to use neutron-lib instead.
Depends-On: https://review.openstack.org/#/c/636159/
NeutronLibImpact
Change-Id: I2388f90b37abb09408809dda8c21da551bcd94bb
Today a number of classes define their supported_extension_aliases
using static strings rather than API definition ALIASes. This patch
switches them to use the ALIAS where applicable.
Change-Id: I716270c68a9fcd850c3c26de31bc13ea16def23d
getting the full collection (which includes iterating over it
and populating attributes) just to gather the count of it
is wasteful. we can just use the count api.
Change-Id: I1b216cb2c8c5b612f12554454d5721a14975f138
Closes-Bug: #1821708
Reduces E128 warnings by ~260 to just ~900,
no way we're getting rid of all of them at once (or ever).
Files under neutron/tests still have a ton of E128 warnings.
Change-Id: I9137150ccf129bf443e33428267cd4bc9c323b54
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
The _resource_extend module is already rehomed into neutron-lib and is
shimmed in neutron. This patch removes the module as no active
consumers are using it.
NeutronLibImpact
Change-Id: I1550075fa5fa2aa2f1a88ee7189d311a1fe78391
This patch switches callbacks over to the payload object style events
for AGENT AFTER_CREATE and AFTER_UPDATE based notifications. To do
so a DBEventPayload object is used with the publish() method to
pass along the API related data.
Change-Id: Ibefa495be41c91957c2e8d797130e569bccc3765
This patch adds a check to determine if the 'segments' service plugin is
enabled. The segment-host mapping db table should only be saved and
updated to the db table if the users configure the 'segments' service
plugin in the config file. The data should be available only in a routed
network resource situation.
Change-Id: I65a42aa2129bef696906a18d82575461dc02ba21
Closes-Bug: #1799328
Originally segments extension used it's own placement client.
Now in neutron-lib there is a working actively designed and tested
client for placement, thus we can change segments to use that, and
remove the old placement client from neutron.
Change-Id: I2e7b53fdedb1837ddb0d685718d9a811ddac5733
Depends-On: https://review.openstack.org/580665
This patch switches over to the payload style of callbacks for SEGMENT
BEFORE_DELETE events. It does so by using the publish callback function
with DBEventPayload payloads. Corresponding listener callbacks are
updated where necessary.
NeutronLibImpact
Change-Id: I64031129ad458d4468a316a9385288ced091d333
Make the same status information available to notification consumers
(resource==agent, event==after_create/after_update) as it was already
available where the notification is sent in class AgentDbMixin.
Change-Id: Ie74091da934c7e49fd29ae4c6f930a7eb47e14b2
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
NeutronLibImpact
Change-Id: I97120faeec73690592ed21a5ec3c6202f61e1429
The retry_if_session_inactive decorator was rehomed into neutron-lib
[1]. This patch consumes it by removing the function from neutron and
using neutron-libs version where appropriate.
NeutronLibImpact
[1] https://review.openstack.org/#/c/557040/
Change-Id: I3e3289f33e62d45933d0fbf165bb4b25078f22d5
Enforce validation on filter parameters on list requests.
If an API request contains an unknown or unsupported parameter,
the server will return a 400 response instead of silently ignoring
the invalid input.
In resource attributes map, all filter parameters are annotated by
the ``is_filter`` keyword. Attributes with is_filter set to True
are candidates for validation.
Enabling filter validation requires support from core plugin and
all service plugins so each plugin need to indicate if it supports
the validation by setting ``__filter_validation_support`` to True.
If this field is not set, the default is False and validation is
turned off. Right now, the ML2 plugin and all the in-tree service
plugin support filter validation. Out-of-tree plugins will have
filter validation disabled by default.
An API extension is introduced to allow API users to discover this
new API behavior. This feature can be disabled by cloud operators
if they choose to do that. If it is disabled, the extension won't
be presented.
Depends-On: Ic3ab5b3ffdc378d570678b9c967cb42b0c7a8a9b
Depends-On: I4397df1c35463a8b532afdc9c5d28b37224a37b4
Depends-On: I3f2e6e861adaeef81a1a5819a57b28f5c6281d80
Depends-On: I1189bc9a50308df5c7e18c329f3a1262c90b9e12
Depends-On: I057cd917628c77dd20c0ff7747936c3fec7b4844
Depends-On: I0b24a304cc3466a2c05426cdbb6f9d99f1797edd
Change-Id: I21bf8a752813802822fd9966dda6ab3b6c4abfdc
Partial-Bug: #1749820
Harald Jensås