assertItemsEqual was removed from Python's unittest.TestCase in
Python 3.3 [1][2]. We have been able to use them since then, because
testtools required unittest2, which still included it. With testtools
removing Python 2.7 support [3][4], we will lose support for
assertItemsEqual, so we should switch to use assertCountEqual.
NOTE(dmllr): added hacking check
[1] - https://bugs.python.org/issue17866
[2] - https://hg.python.org/cpython/rev/d9921cb6e3cd
[3] - testing-cabal/testtools#286
[4] - testing-cabal/testtools#277
Change-Id: I7c20fec08e5dc9f67b34100c925ea6724bbd25f0
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line
inconsistency.
Fixed an inconsistency in the OVSBridge.deferred() definition
as it needs to also have an *args argument.
Fixed an issue where an l3-agent test was mocking
functools.partial, causing a python3.8 failure.
Unit tests only, removing from tests/base.py affects
functional tests which need additional work.
Change-Id: I40e8a8410840c3774c72ae1a8054574445d66ece
When there are many calls to enable security groups on ports there
can be sometimes race condition between refresh recource_cache
with data get by "pull" call to neutron server and data received
with "push" rpc message from neutron server.
In such case when "push" message comes with information about
updated port (with enabled port_security), in local cache this port
is already updated so local AFTER_UPDATE call is not called for
such port and its rules in firewall are not updated.
It happend quite often in fullstack security groups test because
there are 4 ports created in this test and all 4 are updated to
apply SG to it one by one.
And here's what happen then in details:
1. port 1 was updated in neutron-server so it sends push notification
to L2 agent to update security groups,
2. port 1 info was saved in resource cache on L2 agent's side and agent
started to configure security groups for this port,
3. as one of steps L2 agent called
SecurityGroupServerAPIShim._select_ips_for_remote_group() method;
In that method RemoteResourceCache.get_resources() is called and this
method asks neutron-server for details about ports from given
security_group,
4. in the meantime neutron-server got port update call for second port
(with same security group) so it sends to L2 agent informations about 2
ports (as a reply to request sent from L2 agent in step 3),
5. resource cache updates informations about two ports in local cache,
returns its data to
SecurityGroupServerAPIShim._select_ips_for_remote_group() and all
looks fine,
6. but now L2 agent receives push notification with info that port 2 is
updated (changed security groups), so it checks info about this port
in local cache,
7. in local cache info about port 2 is already WITH updated security
group so RemoteResourceCache doesn't trigger local notification about
port AFTER UPDATE and L2 agent doesn't know that security groups for this
port should be changed
This patch fixes it by changing way how items are updated in
the resource_cache.
For now it is done with record_resource_update() method instead of
writing new values directly to resource_cache._type_cache dict.
Due to that if resource will be updated during "pull" call to neutron
server, local AFTER_UPDATE will still be triggered for such resource.
Change-Id: I5a62cc5731c5ba571506a3aa26303a1b0290d37b
Closes-Bug: #1742401
Neutron lib contains the latest callbacks and thus this patch removes
the callbacks package from neutron entirely.
NeutronLibImpact
Change-Id: I14e45fd5d2d3c816bb39f8ace56f7be460bac0d6
We need to check if the results from a bulk_pull are stale because
the resource might have been updated and concurrently pushed from
the server while the bulk_pull query was being fulfilled.
Change-Id: I755a1cb2e0037ec2316161a09ad462bc4b09f397
Closes-Bug: #1707699
This adjusts the record_resource_delete method to ignore duplicate
calls to the same resource ID so we don't generate multiple
AFTER_DELETE callbacks for the same thing.
This can happen in security groups if a security group is deleted
and the local handler starts deleting SG rules and then we receive
an explicit rule deletion from the server triggered by the user.
Change-Id: I8ff58e178641328fe8fed526399e9aa9bef82a6f
Partially-Implements: blueprint push-notifications
This adjusts get_resources filters to take a tuple of values
instead of a single value. This gets translated into an OR query
on the server side.
This is used in the dependent patch to query for all ports in a
set of security groups.
Change-Id: I10ce263d3eb89cfec3bae4f66d3ef59365a18e15
Partially-Implements: blueprint push-notifications
This gets rid of the bulk_flood call and adjusts
the cache to query the server on demand as it's asked
for things it hasn't been asked for before.
Change-Id: I58f3d4dd9bcf545fd9dca8cd42673d705db06c10
Partially-Implements: blueprint push-notifications
Logic for subscribing to RPC callbacks, flooding the initial cache,
storing updates, and generating local callback events.
To see how this will be used, see Change
Ib2234ec1f5d328649c6bb1c3fe07799d3e351f48.
Partially-Implements: blueprint push-notifications
Change-Id: I71f4f5b5ed4e19b8a70a65d22b5378be44d2bd73
Nurmatov Mamatisa