Update hacking to a more recent version, along with
flake8-import-order.
Remove N347 (import mock library) check as that is the
default with later hacking versions.
Update the builtins override of '_' to be the neutron.i18n
version due to the code triggering a false positive. This
is done in a couple of other projects as well.
Fix a number of new warnings it found.
Added some ignore directives for new whitespace issues
found in the test tree, can fix later.
TrivialFix
Change-Id: I5923255af86cf1fa11ab8e3b03bb9efac7dd7b58
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
some remaining ones in miscellaneous directories.
Also cleanup any remaining code that I missed in this
series, or has changed since I started.
Trivialfix
Change-Id: I17b4779020a7bfb369c3e721ab6638cd4a6ab50c
Sometimes Neutron is failing to send notification to Nova
due to timeout, refused connection or another HTTP error.
Retry send in those cases.
Closes-Bug: #1987780
Change-Id: Iaaccec770484234b704f70f3c144efac4d8ffba0
Nova notifier logs for external events only provide the event
details without showing its connection to the associated request_id.
That complicates debugging of corner cases for external events as we
cannot be sure to which exact request a logged event is related.
Make the matching request_ids logged alongside external event
details, whenever it is available.
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
Change-Id: I047a28744a0bf52ace9f9a205993c6037a21083c
The Nova Notifier can be called simultaneously by several RPC
callbacks from the agents (DHCP, L2), trying to update the provisioning
status of a port. In order to handle each context notifier enable flag,
a thread local variable is used. This will isolate the flag update
if two entities inform at the same time and one RPC callback is attended
during the processing of the other one.
This patch also removes the debug messages added to debug this issue.
Closes-Bug: #1958363
Change-Id: Ie670fba4b3afe427747732d2c3948d92311e960e
I should helps with debugging why the nova notifier is at some point
disabled for one neutron worker thus notifications aren't send to
nova from that worker.
Related-Bug: #1958363
Change-Id: Ifd88d5bc7515b5f29240bd12be1d3257b308fd67
This patch switches over to callback payloads for PORT
AFTER_DELETE events.
Some shims were removed.
Change-Id: If69e37b84fe1b027777b1d673b3d08a6651a979e
The goal of this patch is to avoid the connection disruption during
the live-migration using OVS. Since [1], when a port is migrated,
both the source and the destination hosts are added to the profile
binding information. Initially, the source host binding is activated
and the destination is deactivated.
When the port is created in the destination host (created by Nova),
the port was not configured because the binding was not activated.
The binding (that means, all the OpenFlow rules) was done when Nova
sent the port activation. That happend when the VM was already
running in the destination host. If the OVS agent was loaded, the
port was bound seconds later to the port activation.
Instead, this patch enables the OpenFlow rule creation in the
destination host when the port is created.
Another problem are the "neutron-vif-plugged" events sent by Neutron
to Nova to inform about the port binding. Nova is expecting one single
event informing about the destination port binding. At this moment,
Nova considers the port is bound and ready to transmit data.
Several triggers were firing expectedly this event:
- When the port binding was updated, the port is set to down and then
up again, forcing this event.
- When the port binding was updated, first the binding is deleted and
then updated with the new information. That triggers in the source
host to set the port down and the up again, sending the event.
This patch removes those events, sending the "neutron-vif-plugged"
event only when the port is bound to the destination host (and as
commented before, this is happening now regardless of the binding
activation status).
This feature depends on [2]. If this Nova patch is not in place, Nova
will never plug the port in the destination host and Neutron won't be
able to send the vif-plugged event to Nova to finish the
live-migration process.
Because from Neutron cannot query Nova to know if this patch is in
place, a new temporary configuration option has been created to enable
this feature. The default value will be "False"; that means Neutron
will behave as before.
[1]https://bugs.launchpad.net/neutron/+bug/1580880
[2]https://review.opendev.org/c/openstack/nova/+/767368
Closes-Bug: #1901707
Change-Id: Iee323943ac66e566e5a5e92de1861832e86fc7fc
This option allows to configure Number of times nova or ironic client
should retry on any failed http call.
Default value for this new option is "3".
Change-Id: I795ee7ca729646be0411a1232bf218015c65010f
Closes-Bug: #1883712
This patch removes the dependency from ironicclient for the ironic
event notifiers in favor of openstacksdk.
Also, increasing minimum required versions for mock and
openstacksdk.
Change-Id: Ib76e19d29f0ae3db6d181578b638da699181f60d
This patch is the first one of a series of patches improving how the L3
agents update the router HA state to the Neutron server.
This patch partially reverts the previous patch [1]. When the batch
notifier sends events, it calls the callback method passed during the
initialization, in this case AgentMixin.notify_server. The batch
notifier spawns a new thread in charge of sending the notifications and
then wait the specified "batch_interval" time. If the callback method is
not synchronous with the notify thread execution (what [1] implemented),
the thread can finish while the RPC client is still sending the
HA router states. If another HA state update is received, then both
updates can be executed at the same time. It is possible then that a new
router state can be overwritten with an old one still not sent or
processed.
The batch notifier is refactored, to improve what initally was
implemented [2] and then updated [3]. Currently, each new event thread
can update the "pending_events" list. Then, a new thread is spawned to
process this event list. This thread decouples the current execution
from the calling thread, making the event processing a non-blocking
process.
But with the current implementation, each new process will spawn a new
thread, synchronized with the previous and new ones (using a
synchronized decorator). That means, during the batch interval time, the
system can have as many threads waiting as new events received. Those
threads will end secuentially when the previous threads end the batch
interval sleep time.
Instead of this, this patch receives and enqueue each new event and
allows only one thread to be alive while processing the event list. If
at the end of the processing loop new events are stored, the thread will
process then.
[1] I3f555a0c78fbc02d8214f12b62c37d140bc71da1
[2] I2f8cf261f48bdb632ac0bd643a337290b5297fce
[3] I82f403441564955345f47877151e0c457712dd2f
Partial-Bug: #1837635
Change-Id: I20cfa1cf5281198079f5e0dbf195755abc919581
While working on improving the osprofiler report in neutron I'm finding
various places where the profiling can be extended by propagating the
profiler info further then before.
This change starts to profile BatchNotifier threads (e.g. various
callbacks to nova) when the code starting the thread was already
profiled (it's not at the moment, but that will be another change).
In this change we use osprofiler.profiler.clean(). Since that has became
part of the public interface of osprofiler in version 2.3.0, we bump
osprofiler version to 2.3.0 both in lower-constraints and requirements.
Change-Id: Ibd08e097b6f8457c50f8ba9e4a63b96e7e3182bc
Partial-Bug: #1833674
This patch adds an ironic notifier that sends notifications
to ironic endpoint /v1/events. The events are triggered by
port updates and deletions. Only ports with vnic_type
baremetal are honored.
Story: 1304673
Task: 22263
Closes-Bug: #1828367
Implements: blueprint event-notifier-ironic
Authored-By: Vasyl Saienko <vsaienko@mirantis.com>
Co-Authored-By: Harald Jensås <hjensas@redhat.com>
Co-Authored-By: Julia Kreger <juliaashleykreger@gmail.com>
Change-Id: I0bb3187a88a7f20adb8c60e24945db159afb83f1
Neutron makes call to nova for sending notifications but here no
context object user/admin available, so generated a request_id
and passed it as global_request_id during novaclient initialization
so that nova will log both global_request_id and it's own generated
request_id in the context formatter for traceability. As Notifier
class is used as singleton mode, need to create novaclient every time
events are sent from neutron to nova.
Modified unit tests wherever applicable.
Oslo spec I65de8261746b25d45e105394f4eeb95b9cb3bd42
Change-Id: I94257bd6ec9ec6b9a1f509c27c439e6305e43e63
Fix W503 (line break before binary operator) pep8 warnings
and no longer ignore new failures.
Trivialfix
Change-Id: I7539f3b7187f2ad40681781f74b6e05a01bac474
This patch switches callbacks over to the payload object style events
[1] for BEFORE_RESPONSE and AFTER_REQUEST based notifications. To do
so an APIEventPayload object is used with the publish() method to
pass along the API related data. In addition a few UTs are updated to
work with the changes.
NeutronLibImpact
[1] https://docs.openstack.org/neutron-lib/latest/contributor/callbacks.html#event-payloads
Change-Id: Ibd8559e0db9dcc995abf8937a0cb764b21a18531
neutron-lib contains the synchronized lockutils decorator as well as
the SYNCHRONIZED_PREFIX global. This patch consumes them from
neutron-lib and removes them from neutron.
NeutronLibImpact
Change-Id: I729da348e340509f2d09f8a6436716e2398f1583
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
While running unit tests plugins/ml2/drivers/test_type_vlan.py
the following message is observed:
""" UserWarning: Discovering extensions only by contrib path is
no longer supported since all contrib extensions have either
been made required or removed. The only_contrib argument is
deprecated and will be removed in a future release.
"""
From the source code you can see the TODO message to remove
the argument in Queens.
264c22a9cc/novaclient/client.py (L181)
This commit removes the argument and the stderr captured by unit
tests.
Change-Id: Ia4270c69c1519342636206fa439640240a1e13bb
When Nova gets a network-changed event, it rebuilds the
entire network info cache for the instance if it does not
have a specific port ID. This can be costly and redundant
when performing something like a live migration with multiple
ports attached to the same instance.
This change simply adds the port ID to the network-changed event
since we have it in scope. Nova can use it or not, but at least
the information is provided for context.
Change-Id: Ifdaef05208d09ddd9587fed6214cf388e5265ba4
Closes-Bug: #1691602
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
Nova V2.1 API has been available since Kilo, and the API is CURRENT.
Neutron is using Nova API via novaclient and the used Nova API is
V2.1 on the gate as the default Nova endpoint. So this patch doesn't
change the used Nova API actually, but current novaclient skips checking
expected API version in a response from Nova API.
This patch will be helpful to avoid an issue when enabling this check
on novaclient side.
NOTE: Cinder is using Nova v2.1 API with the similar patch as
Iadd3363265be6c5a8ed46704a712da6c15e2b046
Change-Id: Ie3a10f7078f4213da77be522e3866f82b5e8683c
This adjusts the batching logic in the Nova notifier to immediately
send and then sleep to allow batching of subsequent calls in the batch
interval.
So rather than always wait for 2 seconds to elapse while batching,
batching will only occur in the 2 second period after a call is made.
This turns the batch notifier into a standard queuing rate limiter.
The upside to this is a single port create results in an immediate
notification to Nova without a delay.
The downside is now that a sudden burst of 6 port creations to a
previously idle server will result in 2 notification calls to Nova
(1 for the first call and another for the other 5).
Closes-Bug: #1564648
Change-Id: I82f403441564955345f47877151e0c457712dd2f
This is a revert of change I3d53bff8278dabafd929ecbea0b4b3b441c9e1cf
The nova notifier was updated to notify nova on ports with the
baremetal: device_owner, these ports are owned by Ironic not Nova, so
nova is getting notifications that it doesn't understand.
Change-Id: I8318a682163f6a5b739be68ce56973c43d0e32f2
Closes-Bug: #1656010
Depends-On: I43c3af9f424a65211ef5a39f13e4810072997339
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
By adding a singleton getter method and using it in the places
that use nova notifiers, we ensure that only one object exists
so all callers share the same batching queues and the sqlalchemy
events will only be fired once.
Change-Id: I89752d9c69feb578f0294339aae1a5cf51ec124b
Closes-Bug: #1622503
Remove deprecation warnings for various constants
and exceptions that have moved to neutron_lib.
Fix miscellaneous other deprecations.
Uses constants instead of l3_constants when importing
neutron-lib constants.
Co-Authored By: Henry Gessau <gessau@gmail.com>
Co-Authored By: Gary Kotton <gkotton@vmware.com>
Change-Id: Ib0e8ff5c3e23677c1009241a1818cbc8a3430c38
Use the only_contrib option that was added by
I030f4c55c2795c7f7973f5f12e54b9819c4a5578 to speed up nova_client calls
and skip the search for nova_client extensions to reduce
/var/log/audit noise.
Change-Id: Ic97b342a3633ffdf05b02ddd81baad88e1605a75
Closes-Bug: #1509500
Interface should be correctly removed from ironic when port-delete.
Nova should receive notify when baremetal port deleted.
Change-Id: I3d53bff8278dabafd929ecbea0b4b3b441c9e1cf
Partial-Bug: #1606229
If we try to delete a VM and to delete the floating IP
associated with the VM at the same time, depending
on the order according to which these requests are processed
Neutron might fail in the deletion of the floating IP,
raising a PortNotFound error. This happens because Neutron
notifies Nova of the network change event and it tries to get
the port to which the FIP is associated. If the port is not there,
Neutron shouldn't raise, it shouldn't send any notification.
Change-Id: Ic72313ad1f787b3cb528e806c843f1fd01eb12f2
Closes-bug: #1586931
- during live migration on pre migration step nova plugs instance
vif device on the destination compute node;
- L2 agent on destination host detects new device and requests device
info from server;
- server does not change port status since port is bound to another
host (source host);
- L2 agent processes device and sends update_device_up to server;
- again server does not update status as port is bound to another host;
Nova notifications are sent only in case port status change so in this case
no notifications are sent.
The fix is to explicitly notify nova if agent reports device up from a host
other than port's current host.
This is the fix on neutron side, the actual fix of the bug is on nova side:
change-id Ib1cb9c2f6eb2f5ce6280c685ae44a691665b4e98
Closes-Bug: #1414559
Change-Id: Ifa919a9076a3cc2696688af3feadf8d7fa9e6fc2
Once the decoupling started with 60c05a6d57, this cleanup
patch continues the effort by improving the decoupling
between the API layer and Nova.
Change-Id: I1d7d4b80ee77deefce18df22f76cab81750c0397
This patch adds a the Nova notifier calls to the pecan
notification hook and adds some simple functional tests
for them as well.
This patch also alters the operations of the policy enforcement
hook, which now stores in the request context the original
value of the object rather than the value of the object as it
would have appeared to the plugin - that is to say a merge
between the original objects and the parameters passed on to
the request body. Such 'frankenobject' is indeed of no use for
the notifiers.
Partial-Bug: #1552979
Change-Id: I4a37197eb38afe15d2b368f4e355226824478792
The NotFound response from Nova happens under normal circumstances, so
it should be a debug message, not a warning. In particular, if nova
itself deletes the port, when we then notify about it, it's going to say
not found.
Change-Id: I782a0fb9f39c2533bcf62b4e7ceb655f067c0282
Signed-off-by: Russell Bryant <rbryant@redhat.com>
When the neutron notification to nova was updated to use novaclient the
nova_url parameter was disabled. This prevents administrators from
using anything but the publicURL as the proper endpoint to notify nova.
This patch adds an option to pass on to novaclient for the
endpoint_type so that the administrator can set the notification url to
public, internal or admin.
Change-Id: I405f761944449cab6b8c8895f98419f79cd74cad
Closes-Bug: #1478471
DocImpact: Need to add a new option to the neutron configuration
reference.
keystoneauth was split out last cycle as a library specifically to deal
with doing auth functions so that people who do not need to do keystone
CRUD operations can just consume only the auth session parts. As part
of modernizing keystone interactions, use keystoneauth instead of
keystoneclient.
Depends-On: I1f754a9a949ef92f4e427a91bbd1b1e73e86c8c4
Change-Id: Ia6fb7d8bb07d3101cf1b1afc318f3dcac0a37afc
- This does NOT break other projects that rely on neutron.i18n,
as this change includes a debtcollector shim to maintain those
older entry points, until they can migrate.
- Also updates _i18n.py to the latest pattern defined by oslo_i18n
- Guidance and template are from the reference:
http://docs.openstack.org/developer/oslo.i18n/usage.html
Partially-Closes-Bug: #1519493
Change-Id: I1aa3a5fd837d9156da4643a367013c869ed8bf9d
Now that we have the constant defined, we should reuse it from other
code to avoid potential typos.
Change-Id: Iebb270be46b116df3441370dc1a6784571311aa9
Neutron nova_* options have been deprecated in Kilo in favour of options
in [nova] section.
This change removes these options in order to simplify code, clarify
configuration and avoid a warning in unittests.
DocImpact
UpgradeImpact
Closes-Bug: #1509092
Related-Bug: #1403686
Change-Id: I950f72ad7d7d41cfa006f8c7105934301cd56faa
Brian Haley