Update hacking to a more recent version, along with
flake8-import-order.
Remove N347 (import mock library) check as that is the
default with later hacking versions.
Update the builtins override of '_' to be the neutron.i18n
version due to the code triggering a false positive. This
is done in a couple of other projects as well.
Fix a number of new warnings it found.
Added some ignore directives for new whitespace issues
found in the test tree, can fix later.
TrivialFix
Change-Id: I5923255af86cf1fa11ab8e3b03bb9efac7dd7b58
Default SG rules created as template in the Neutron DB are now used to
create security group rules for each new default and non-default SG
created in Neutron.
Closes-bug: #1983053
Change-Id: Iaf27deb955c3844409fcd36239511478e9607a82
As part of the Secure RBAC community goal, we should switch options
"enforce_new_defaults" and "enforce_scope" to be True by default.
It will be still possible to fallback to old policy rules by configuring
those config options to False in Neutron config.
Change-Id: I09c0026ccf87e6c0bb1fa59165c03dc508fba6fa
Remove security_groups_shared_filtering_lib extension and
use security-groups-shared-filtering from neutron-lib as
it is available since version 2.17.0 [0].
[0] https://review.opendev.org/c/openstack/neutron-lib/+/812617
Change-Id: Ife9b1ae47f5b447898bce0d8b44500f91f6dfbfb
Related-Bug: #1942615
security_groups_db._check_security_group is supposed to check the
security_group_id of the _create_security_group_rule payload.
When using an integer e.g. 0, as security_group_id, the check
succededs because mysql accepts following query:
SELECT * FROM securitygroups WHERE id in (0)
Forcing validation of security_group_id as uuid fixes the problem
Closes-Bug: #1968343
Change-Id: I7c36b09309c1ef66608afacfb281b6f4b06ea5b8
Add the shared field to security group API responses and support
using shared as a query filter.
A follow-up patch will remove the temporary api def once it is merged
and released in neutron-lib.
Related-Bug: #1942615
Depends-On: https://review.opendev.org/c/openstack/neutron-lib/+/812617
Change-Id: Ic04be8f0b7097c8aed19365f06089aa7af333eb9
The quota driver ``ConfDriver`` was deprecated in Liberty release.
``NullQuotaDriver`` is created for testing although it could be used
in production if no quota enforcement is needed. However, because
the Quota engine is not plugable (is an extension always loaded), it
could be interesting to make it plugable as any other plugin.
This patch also creates a Quota engine driver API class that should be
used in any Quota engine driver. Currently it is used in the three
in-tree drivers implemented: ``NullQuotaDriver``, ``DbQuotaDriver``
and ``DbQuotaNoLockDriver``.
Change-Id: Ib4af80e18fac52b9f68f26c84a215415e63c2822
Closes-Bug: #1928211
New API extension was added in [1] to extend security group rules with
"normalized_cidr" read only attribute.
This patch implements this API extension in Neutron ML2 plugin and
extends security group rules with "normalized_cidr" value.
[1] https://review.opendev.org/#/c/743630/
Related-Bug: #1869129
Change-Id: I65584817a22f952da8da979ab68cd6cfaa2143be
- Add api extension and db model changes to support remote_address_group_id
in SG rules.
- RPC and firewall agent changes will be in the follow-up patches.
Change-Id: I99681736d05eefd82bdba72b3866eab9468ef5dd
Implements: blueprint address-groups-in-sg-rules
This patch adds the verification of whether admin context when
verifying the valid security groups of port.
Change-Id: I2674bdc448d9a091b9fe8c68f0866fd19141c6be
Closes-Bug: #1890539
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line
inconsistency.
Fixed an inconsistency in the OVSBridge.deferred() definition
as it needs to also have an *args argument.
Fixed an issue where an l3-agent test was mocking
functools.partial, causing a python3.8 failure.
Unit tests only, removing from tests/base.py affects
functional tests which need additional work.
Change-Id: I40e8a8410840c3774c72ae1a8054574445d66ece
A security group rule where port_range_min:port_range_max
is 1:65535 is specifying all ports, but it is not optimal
for backends to try and implement this potentially large
rule.
Since it is essentially the entire port range, change
min:max to be None, making the rule specify the entire
protocol instead.
Change-Id: Iff22e2fc84d679e20a5a04b8516750c6ea949078
Closes-bug: #1848213
It was noticed in another change that argument ordering in an
assertEqual() were backwards in test_securitygroup.py, this
fixes them all at once.
Trivialfix
Change-Id: Idcb9b371914ec7fb64920fad55bd90994d28fe9f
The security group code was changed recently to always
normalize IPv6 ICMP protocol names to 'ipv6-icmp', but it
did not cover when a number is used instead. Normalize
protocol number 1 to 58 for IPv6 ICMP as well.
Change-Id: Ife8263196f3d678d8455f07834c9f6c1330acc00
Closes-bug: #1582500
Currently, 'icmp', 'ipv6-icmp' and 'icmpv6' can be
specified as an IPv6 ICMP protocol value. This can
lead to duplicate entries in the DB for doing exactly
the same thing.
Change to always be 'ipv6-icmp' so this doesn't happen.
Existing rules using one of the old values will now be
returned with 'ipv6-icmp' as the protocol value.
Depends-on: https://review.opendev.org/660206
Depends-on: https://review.opendev.org/660387
Change-Id: I7cd146691dce1a690e1d2c309dfd54b4a0032f76
Partial-Bug: #1582500
'res' is never used after the call to self._delete(),
just like the line below it, so don't save it.
Trivialfix
Change-Id: Iaaba249cfd658afe3cf0ea8543a9f001cc7b8908
After taking a closer look at bug 1818385, I found a couple
of follow-on things to fix in the security group code.
First, there are very few protocols that accept ports,
especially via iptables. For this reason I think it's
acceptable that the API rejects them as invalid.
Second, UDPlite has some interesting support in iptables. It
does not support using --dport directly, but does using
'-m multiport --dports 123', and also supports port ranges using
'-m multiport --dports 123:124'. Added code for this special
case.
Change-Id: Ifb2e6bb6c7a2e2987ba95040ef5a98ed50aa36d4
Closes-Bug: #1818385
Used to be, we would return an empty list. Now, as of change
https://review.openstack.org/#/c/630401/, we don't return the
field at all. That's an API regression.
Go back to returning an empty list.
Change-Id: I295076155eea518152e2479f93f3cf1ea811a207
Today the neutron common exceptions already live in neutron-lib and are
shimmed from neutron. This patch removes the neutron.common.exceptions
module and changes neutron's imports over to use their respective
neutron-lib exception module instead.
NeutronLibImpact
Change-Id: I9704f20eb21da85d2cf024d83338b3d94593671e
We switched from swapping the tenant_id in the context to explicitly
checking the db column. Switch back, and a test that checks for
not breaking this rather odd behavior. At least, until we decide
to fix it as a bug.
Change-Id: I6af4d414b1972e14692a8356ef95db7323e3a09a
The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
NeutronLibImpact
Change-Id: I97120faeec73690592ed21a5ec3c6202f61e1429
This patch removes the explicit API definition of the 'description'
attribute from the security group API extension. This
attributes is redundant as its added by default by the
standardattrdescription extension and moreover are clobbered by the
latter.
In addition this patch updates the UTs to account for the extension
manager not adding the standardattrdescription to the security group
definition during test runtime.
Also note that the segment API extension has the same issue, but will be
handled with the consumption of https://review.openstack.org/#/c/562331/
Change-Id: I1fcc2adf13792705fc4bb6faf757ee59faee1349
Partial-Bug: 1757513
Fixed all pep8 E265 errors and changed tox.ini to no longer
ignore them. Also removed an N536 comment missed from a
previous change.
Change-Id: Ie6db8406c3b884c95b2a54a7598ea83476b8dba1
Fix W503 (line break before binary operator) pep8 warnings
and no longer ignore new failures.
Trivialfix
Change-Id: I7539f3b7187f2ad40681781f74b6e05a01bac474
Iptables only supports port-ranges for certain protocols,
others will generate failures, possibly leaving the agent
looping trying to apply rules. Change to not allow port
ranges outside of the list of known good protocols.
Change-Id: I5867f77fc5aedc169b42f50def0424ff209c164c
Closes-bug: #1749667
Today our unit test code uses various ways to "patch" the global
RESOURCE_ATTRIBUTE_MAP as well as extension specific maps in some cases.
This patch consolidates such patching whereby tests should use neutron's
AttributeMapMemento in their setup() chain (only once) if they update
the global map and they should individually handle backup/restore of per
extension map updates. This change will simplify the code and make it
easier to phase-in API definition usage with neutron-lib where we have
some as API definitions and others not. Longer term the
AttributeMapMemento will be replace with neutron-lib's fixture as we
move all extension maps to API definitions in neutron-lib.
Change-Id: I2586f0b11b107d7f57214a0d65bcf7c38a5f0ebb
according to https://wiki.openstack.org/wiki/Python3,
now we should avoid using six.iteritems and replace
it with dict.items.
Change-Id: I8753e80b34c0f86cf70aebc3bcbd3392ee933f62
Partial-Bug: #1680761
This patch disallows cases that following name is specified.
1. name whose type is not String.
2. name whose characters is more than 255.
Change-Id: Ib72a4e480b62a22da2171ed24449321f2b27258b
Closes-Bug: #1610764
Use reader and writer for db operations.
Partially-Implements blueprint: enginefacade-switch
Depends-On: Iba3520ac6cfb6b82b2013df9b8e1aee64b10a11c
Change-Id: I50be115ea69f805b48b02aebe4259ec2c839830e
Currently, Bulk Creation of security groups
is enabled, but due to a mismatch in the function
signature, it is not possible to create security
groups in bulk.
This patch allows the same.
Change-Id: Id1c7a520aa1bb58984b199c86aca2267b98e6b3e
Closes-Bug: #1604662
Currently, we check whether specified value in list
matches with UUID format or not.
However, we don't check whether data type of specified value of
security_groups key is list in the first place.
This patch adds check above.
Change-Id: If027df3ff34b219e4a043934b9461ac0fcb40156
Closes-Bug: #1477921
Neutron Manager is loaded at the very startup of the neutron
server process and with it plugins are loaded and stored for
lookup purposes as their references are widely used across the
entire neutron codebase.
Rather than holding these references directly in NeutronManager
this patch refactors the code so that these references are held
by a plugin directory.
This allows subprojects and other parts of the Neutron codebase
to use the directory in lieu of the manager. The result is a
leaner, cleaner, and more decoupled code.
Usage pattern [1,2] can be translated to [3,4] respectively.
[1] manager.NeutronManager.get_service_plugins()[FOO]
[2] manager.NeutronManager.get_plugin()
[3] directory.get_plugin(FOO)
[4] directory.get_plugin()
The more entangled part is in the neutron unit tests, where the
use of the manager can be simplified as mocking is typically
replaced by a call to the directory add_plugin() method. This is
safe as each test case gets its own copy of the plugin directory.
That said, unit tests that look more like API tests and that rely on
the entire plugin machinery, need some tweaking to avoid stumbling
into plugin loading failures.
Due to the massive use of the manager, deprecation warnings are
considered impractical as they cause logs to bloat out of proportion.
Follow-up patches that show how to adopt the directory in neutron
subprojects are tagged with topic:plugin-directory.
NeutronLibImpact
Partially-implements: blueprint neutron-lib
Change-Id: I7331e914234c5f0b7abe836604fdd7e4067551cf
This adds a way for standard attribute models to declare
the API resources they show up in. It then adds a utility
function to the standard_attr module to grab a map of all
API resources and their corresponding models.
This can be used by any processing code that wants to add
fields to standard attribute resources.
This also adjusts the existing extensions to leverage this
new functionality.
Partially-Implements: blueprint add-neutron-extension-resource-timestamp
Change-Id: Idc8923d0e983fcb0690f8cb5b55a5aff8690154f
Among non-obvious changes, description field in standardattributes object
extension is now nullable, as in the database. This changes object hashes,
but we should be safe to touch it without providing a new object version
conversion rules because affected objects are not yet exposed to RPC
anywhere.
To avoid object API complexity, base class added support for
get_objects() keyword arguments that are not of list type (in which
case, we pass a single entry list down into _get_collection).
Also, some unit tests were changed to pass correct UUIDs for tenants, to
avoid validation warnings triggered by strict typing for subnetpool
project_id field.
Depends-On: I1c1bf544af4463b0302b62c8cce492932504bb01
Co-Authored-By: Ihar Hrachyshka <ihrachys@redhat.com>
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Change-Id: I1415c7a29af86d377ed31cce40888631a34d4811
Since neutron_lib 0.2.0 contains PROTO_NAME_IPV6_ICMP_LEGACY, remove
it from neutron.common.constants
Change-Id: Idd150ce1cbe660fc9529e5d46678f37142490a28
Closes-bug: #1585047
Now the security group rules can be added with same content but different
description. This should be prevented to stop creating duplicated sg rules.
Change-Id: Ibafe39f9652ecd24ad9536e6abc7c4f4384b3a22
Closes-bug: #1583601
The quotas tests of security group rule have a magic number.
This is the number to avoid the resources created by default.
Fixed to assign the appropriate number by the resources lists.
Change-Id: I9e1639d640ef297fd3e0f5a7721c1aa7cd6eada9
Closes-bug: #1567743
Brian Haley