Default SG rules created as template in the Neutron DB are now used to
create security group rules for each new default and non-default SG
created in Neutron.
Closes-bug: #1983053
Change-Id: Iaf27deb955c3844409fcd36239511478e9607a82
As part of the Secure RBAC community goal, we should switch options
"enforce_new_defaults" and "enforce_scope" to be True by default.
It will be still possible to fallback to old policy rules by configuring
those config options to False in Neutron config.
Change-Id: I09c0026ccf87e6c0bb1fa59165c03dc508fba6fa
This patch switches over to callback payloads for SECURITY_GROUP
events. To do so a few shims are put into place the handle both
payload and kwarg style callbacks; these shims will be removed once
all events use payloads. In addition a few UT updates are included to
get the tests working properly with payloads.
Change-Id: I6161a8b387812808c4d679f882a3193c93235647
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line
inconsistency.
Fixed an inconsistency in the OVSBridge.deferred() definition
as it needs to also have an *args argument.
Fixed an issue where an l3-agent test was mocking
functools.partial, causing a python3.8 failure.
Unit tests only, removing from tests/base.py affects
functional tests which need additional work.
Change-Id: I40e8a8410840c3774c72ae1a8054574445d66ece
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues. Didn't think it was going to be
close to 100 files when I started.
Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
The core resource API definitions are in neutron-lib and are already
setup in a RESOURCES map, similar to neutron's global
RESOURCE_ATTRIBUTE_MAP. However, a number of consumers directly use
RESOURCE_ATTRIBUTE_MAP and moreover can perform some interesting
operations on it while mockin" for tests. For that reason this
patch proposes we phase in the use of neutron's RESOURCES map rather
than a rip and replace of RESOURCE_ATTRIBUTE_MAP.
This patch removes the API definitions for the core neutron resources
by updating neutron's global RESOURCE_ATTRIBUTE_MAP to reference lib's
RESOURCE map. In addition the AttributeMapMemento class is removed; it
no longer servers a purpose and neutron-lib's fixture should be used
in its place.
This patch should be safe to merge without worry of impacting consumers.
No one is using AttributeMapMemento [1] and the global map only changed
its reference value.
[1] http://codesearch.openstack.org/?q=AttributeMapMemento
Change-Id: Ib04fddcbd4465074452b71b16befa4e33b27259e
The callback modules have been available in neutron-lib since commit [1]
and are ready for consumption.
As the callback registry is implemented with a singleton manager
instance, sync complications can arise ensuring all consumers switch to
lib's implementation at the same time. Therefore this consumption has
been broken down:
1) Shim neutron's callbacks using lib's callback system and remove
existing neutron internals related to callbacks (devref, UTs, etc.).
2) Switch all neutron's callback imports over to neutron-lib's.
3) Have all sub-projects using callbacks move their imports over to use
neutron-lib's callbacks implementation.
4) Remove the callback shims in neutron-lib once sub-projects are moved
over to lib's callbacks.
5) Follow-on patches moving our existing uses of callbacks to the new
event payload model provided by neutron-lib.callback.events
This patch implements #2 from above, moving all neutron's callback
imports to use neutron-lib's callbacks.
There are also a few places in the UT code that still patch callbacks,
we can address those in step #4 which may need [2].
NeutronLibImpact
[1] fea8bb64ba7ff52632c2bd3e3298eaedf623ee4f
[2] I9966c90e3f90552b41ed84a68b19f3e540426432
Change-Id: I8dae56f0f5c009bdf3e8ebfa1b360756216ab886
We should use addCleanup instead of tearDown because:
- tearDown is executed only if the test succeeds
- cleanups (defined with addCleanup) are executed even if the test fails
This change removes useless tearDown and transforms remaining ones into
addCleanup.
Change-Id: I44bd26dcb5c8456126a35cb807f0bafc772c0ab0
Creating default security groups should not be done inside
another transaction because the creation code emits AFTER events.
Handlers for the push notification code expect to be able to read
from the DB during an after event using a new transaction, which will
not work if they are inside of exising transactions.
This patch adjusts the default security group creation logic to avoid
using layered transactions for the default security group creation and
adjusts ML2 to stop calling ensure_default_security_group inside of a
port/network transaction.
Partially-Implements: blueprint push-notifications
Change-Id: Iaa83c8664d5bfde31fdcdd694f6f18d9ef9bf14a
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
Stevedore documentation suggest that full import paths are not supposed
to be user visible. Since unit tests emulate users when configuring
oslo.config, we better off relying on well known plugin aliases than
internal details.
For in-tree that may be not a big deal, but with it we set a bad example
for third parties that may later find their tests broken eg. when we
decide to move code around.
TrivialFix
Change-Id: I7bd036ac3df7e7f4c678356d0a793e7d38599dda
Went through all of the docstrings in Neutron and did
some cleanup. I'm sure there are bunch more that I have missed.
Change-Id: Ib29d2de1c580880c89ed4fd069e1515d0977a3e7
map() returns an iterator in python 3. In a case that a list is expected,
we wrap map() in a list call.
Change-Id: I623d854c410176c8ec43b732dc8f4e087dadefd9
Blueprint: neutron-python3
`contextlib.nested` is deprecated since Python 2.7 and incompatible with
Python 3. This patch removes all its occurences by using the helper
script at [1].
This is a necessary step to allow us running all unit tests with
Python 3 (not just a small subset as it is done now).
[1]: https://github.com/adrienverge/context_unnester
Change-Id: I8d1de09ff38ed0af9fb56f423a2c43476408e0fb
Blueprint: neutron-python3
Closes-Bug: #1428424
This patch changes the get_port_from_device_mac() and
get_sg_ids_grouped_by_port() methods in ML2 db.py module so that
they do not create a new database session (via get_session()), but
instead reuse the session associated with the caller's context.
In order to make the session that is associated with the caller's
context available to these ML2 DB methods, the
get_ports_from_devices plugin API in securitygroups_rps_base.py
needs to be modified so that the context can be passed down to the
ML2 plugin. (A similar change is made to the get_port_from_device
plugin API for consistency.)
Change-Id: I3f990895887e156de929bd7ac3732df114dd4a4b
Closes-Bug: 1441205
This patch adds a AttributeMapMemento class that can be used for
restoring the RESOURCE_ATTRIBUTE_MAP on test tear down. Tests containing
their own cleanup code have been modified to use it instead.
Change-Id: I7ce5182bdfb8f541741a327feada63a29ddac2ae
In the test_full_uuids_skip_port_id_lookup test in test_security_group.py,
there are a couple of problems with how a mock return value is being set
for a database query.
The first problem is that in this line:
fmock = sess_mock.query.return_value.outerjoin.return_value.filter
there is a missing '.return_value' missing between 'sess_mock' and 'query'.
The second problem is that in this line:
fmock.return_value.all.return_value = []
the 'all.return_value' should not be used.
For reference, the query for which this mock return value is being set
is in the get_sg_ids_grouped_by_port() method in ML2's db.py:
query = session.query(models_v2.Port,
sg_db.SecurityGroupPortBinding.security_group_id)
query = query.outerjoin(sg_db.SecurityGroupPortBinding,
models_v2.Port.id == sg_binding_port)
query = query.filter(or_(*or_criteria))
This patch fixes the problems mentioned above so that the query above
returns an empty list for the test_full_uuids_skip_port_id_lookup test.
Change-Id: I2cec2c27fcdc82557c91205d202a6ac79987e92a
Closes-Bug: 1444009
This change ensures that the structure of the unit test tree matches
that of the code tree to make it obvious where to find tests for a
given module. A check is added to the pep8 job to protect against
regressions.
The plugin test paths are relocated to neutron/tests/unit/plugins
but are otherwise ignored for now.
Change-Id: If307593259139171be21a71c58e3a34bf148cc7f
Partial-Bug: #1440834
Slawek Kaplonski