The policy rule ``shared_security_group`` allows to create new policy
rules checking if a security group rule belongs or not to the project
default security group.
By default the behaviour has not changed. If an administrator wants
to prevent a non-privileged user from creating or deleting rules in the
default security group, the ``create_security_group_rule`` and
``delete_security_group_rule`` can be overriden. An example is provided
in the unit tests.
Closes-Bug: #2019960
Change-Id: I6c90b61df0e726ef07f177801069baf30c49de67
The method ``delete_security_group_rule`` is publishing the
BEFORE_DELETE event before starting the security group rule deletion.
This event is published using a wrap method called
``SecurityGroupDbMixin._registry_publish``. This method is capturing
any ``CallbackFailure`` exception and raising a
``SecurityGroupRuleInUse`` one. That makes no sense because:
* We are hidding the real cause of the callback failure.
* The BEFORE_DELETE is not checking that the security group rule is
being used (NOTE 1).
* If any new implementation makes this check, the corresponding callback
should return explicitly this exception.
The method ``_create_security_group_rule`` is publishing the
BEFORE_CREATE event before starting the security group rule creation.
The same argument applies here: the callback manager should return the
exception raise by the callback method (NOTE 2).
In a follow-up patch, this events will be captured to check the
permissions related to the user creating or deleting the security group
rule. In case of error, it will be needed to raise a ``NotAuthorized``
derived exception, instead of a ``InUse`` one.
NOTE 1: this is the current use of BEFORE_DELETE event in the
OpenStack repository:
* [2] Omni project had no activity for the last 4 years.
* [3] networking-arista: the method ``run_cmds_on_all_switches``, that
calls ``run_openstack_sg_cmds``, returns its own exceptions.
* [4] networking-opencontrail: same justification.
* [5] The ML2/OVN mechanism driver, that will raise an exception if the
OVN ACL deletion doesn't succeed.
NOTE 2: this is the current use of BEFORE_DELETE event in the
OpenStack repository:
* [2] Omni project had no activity for the last 4 years.
[1]https://codesearch.openstack.org/?q=%5C.SECURITY_GROUP_RULE&i=nope&literal=nope&files=&excludeFiles=&repos=
[2]https://opendev.org/x/omni/src/branch/master/neutron/neutron/plugins/ml2/drivers/aws/callbacks.py
[3]https://opendev.org/x/networking-arista/src/branch/master/networking_arista/ml2/security_groups/arista_security_groups.py
[4]https://opendev.org/x/networking-opencontrail/src/branch/master/networking_opencontrail/ml2/opencontrail_sg_callback.py
[5]https://opendev.org/openstack/neutron/src/branch/master/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py
Partial-Bug: #2019960
Change-Id: I8d5f5392fb7a6ab9b20e9222c143f4e67c925cae
This new extension adds a new synthetic field, "belongs_to_default_sg",
to the security group rule OVO. This read only boolean field determines
if the security group rule belongs to a default security group or not.
This new field will be used in a new set of policy rules. By default,
these new rules will allow to create and delete security group rules
into the default security group of a project only to the admin user
NOTE: the follow-up patch will introduce the policy rules check,
during the creation/deletion operations, of the
"belongs_to_default_sg" field and the user executing this action.
Partial-Bug: #2019960
Change-Id: I0b3ded52e1ff8160c5804c59635c0fd34ce9995b
Default SG rules created as template in the Neutron DB are now used to
create security group rules for each new default and non-default SG
created in Neutron.
Closes-bug: #1983053
Change-Id: Iaf27deb955c3844409fcd36239511478e9607a82
This patch adds DB model, OVO class and DB migration script for
SG rules template used for every new SG created.
It also implements Create/Get/Delete actions for that new resource and
adds API policies for those APIs
Related-Bug: #1983053
Change-Id: Ib3cde1710edd400b972f493b13666d0679a7753c
This change allows to pass to ``_make_security_group_rule_dict`` method
the Neutron OVO. That could include synthetic fields added in the OVO
(SQL view) that are not present in the database register.
This change will be needed in next patches to increase the information
returned by this method, using new synthetic fields added to the
security group rule OVO.
Partial-Bug: #2019960
Change-Id: Ic0e697bb212c7795a40c0b9be01345db26c2874e
This reverts commit 6358495720.
Reason for revert: This is generating a lot of
"SecurityGroupNotFound" errors in neutron-server.log in
the tempest-integrated-networking job.
Closes-Bug: #2019449
Related-Bug: #2008712
Change-Id: I077fe87435f61bd29d5c1efc979c2adebca26181
Based on bug #2008712 if we have a security-group which
is the remote group of a 2nd security-group, the backend
never deletes the rule of the 2nd group which
remote_group_id is the original security-group.
By AFTER_DELETE event for each rule that has the
security_group_id as remote_group_id, we can make the
mech drivers do their work and delete these rules in the
backend.
Change-Id: I207ecf7954b06507e03cb16b502ceb6e2807e0e7
Closes-Bug: #2008712
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/db.
Trivialfix
Change-Id: I9311cfe5efc51552008072d84aa238e5d0c9de60
The attempt to list security groups for a project, or any
random string, can create a default SG for it. Only allow if
privileges support it.
Closes-bug: #1988026
Change-Id: Ieef7011f48cd2188d4254ff16d90a6465bbabfe3
Do not retrieve the security group port bindings in the database
transaction that deletes a security group. In the previous context,
if a security group port binding is present on the database,
the method raises a ``SecurityGroupInUse``. It is unneeded to
retrieve them again.
This patch also changes the
``SecurityGroupPortBinding.security_group_id`` foreign key. Now if
the security group is deleted, any security group port binding related
will be too, using the database engine. That will ensure no leftover
remains in the database. Although the check done in
"delete_security_group" before the security group is deleted, there is
a minimal possibility of race condition between the first database
transaction (SG port binding check) and the second one (SG deletion).
Trivial-Fix
Change-Id: I1c9c2dd95b98a7cc77509b0d537d7c7766765275
Removed unneeded database context in
``_create_port_security_group_binding``. This method is called always
from inside an active database transaction, when the port is udpated
or created.
The retry context only makes sense when a method is called outside
an active database transaction. It doesn't retry the command if the
context passed has an active transaction.
Trivial-Fix
Change-Id: I8f84c1bce0d1ce2538789e3511fd8f7b64fcd41a
When listing the security group rules, the DB object can be used
instead the OVO. That will save the conversion time. In a
development environment with 1000 SG rules, the DB query took around
100 ms and the same time to load the OVOs from the DB objects.
The ``NeutronDbObject.get_objects`` now can return the DB objects
without converting them to OVO if "return_db_obj=True".
Trivial-Fix
Change-Id: I245032cf99b1b042226703f5ddb4625e176924c5
Add the shared field to security group API responses and support
using shared as a query filter.
A follow-up patch will remove the temporary api def once it is merged
and released in neutron-lib.
Related-Bug: #1942615
Depends-On: https://review.opendev.org/c/openstack/neutron-lib/+/812617
Change-Id: Ic04be8f0b7097c8aed19365f06089aa7af333eb9
The goal of [1] is to, in case of failing when removing the quota
reservation, continue the operation. Any expired reservation will
be removed automatically in any driver.
If the DB transaction fails, it should affect only to the reservation
trying to be deleted. This is why this patch isolates the
"remove_reservation" method and guarantees it is called outside an
active DB session. That guarantees, in case of failure, no other DB
operation will be affected.
This patch also partially reverts [2] but still checks the security
group rule quota when a new security group is created. Instead of
creating and releasing a quota reservation for the security group
rules created, now only the available quota limit is checked before
creating them. That won't prevent another operation to create security
group rules in parallel, exceeding the available quota. However, this
is not even guaranteed with the current quota driver.
[1]https://review.opendev.org/c/openstack/neutron/+/805031
[2]https://review.opendev.org/c/openstack/neutron/+/701565
Closes-Bug: #1943714
Change-Id: Id73368576a948f78a043d7cf0be16661a65626a9
SECURITY_GROUP and SECURITY_GROUP_RULE resources now
is using payload style callback. Shim can be removed
Change-Id: Icd8e61100b5e2db24a737e852441548eca8f8424
With new scopes, when e.g. project admin context is used to ensure
default SG for different tenant, elevated context needs to be used
to make db query. Otherwise default SG will not be found and attempt to
create it in DB may fail with DuplicateDbEntry error.
Closes-Bug: #1938910
Change-Id: Ib884be6aa12bd0d3faf83f3e753f8e7aad503b68
The quota driver ``ConfDriver`` was deprecated in Liberty release.
``NullQuotaDriver`` is created for testing although it could be used
in production if no quota enforcement is needed. However, because
the Quota engine is not plugable (is an extension always loaded), it
could be interesting to make it plugable as any other plugin.
This patch also creates a Quota engine driver API class that should be
used in any Quota engine driver. Currently it is used in the three
in-tree drivers implemented: ``NullQuotaDriver``, ``DbQuotaDriver``
and ``DbQuotaNoLockDriver``.
Change-Id: Ib4af80e18fac52b9f68f26c84a215415e63c2822
Closes-Bug: #1928211
With new system/project scopes it may happen that project admin
will try to list security groups for different project and during
that call Neutron will ensure that default security group is created.
In such case elevated context needs to be used to get SG object from
the database otherwise, SG will not be found and error 500 will be
returned through the API.
Use of elevated context is fine here as if user don't have access to
the SG, it will be filtered out by policy mechanism and it will not
be returned through API.
Closes-Bug: #1934115
Change-Id: I0ca07d1a1aaf05c1992aea9e29575580d7933324
This patch switches over to callback payloads for SECURITY_GROUP
events. To do so a few shims are put into place the handle both
payload and kwarg style callbacks; these shims will be removed once
all events use payloads. In addition a few UT updates are included to
get the tests working properly with payloads.
Change-Id: I6161a8b387812808c4d679f882a3193c93235647
This patch switches the code over to the payload style of callbacks
for PORT BEFORE_CREATE and PRECOMMIT_CREATE events
Change-Id: Ia6903be1af4de575ab9d82c9cb0c88290d07abb3
This patch switches over to the payload style of callbacks for
NETWORK based events. As part of this change a few shims are needed
to handle cases where some callbacks don't yet use payloads and others
do. Once we move over to payloads for all callbacks the shims can be
removed.
NeutronLibImpact
Change-Id: I889364b5d184d47a79fe6ed604ce13a4b334acfa
Getting default SG for a project is quite a frequent operation
that happens for example on each port create/update events.
Since default SG id does not change often for a project/tenant
(if at all) it makes sense to cache it in memory to avoid DB access
each time.
Change-Id: I2493eb65ddd548435cf846e1ae7ac65ce8bc14a4
If security group API is disabled, there is no point to create default
security group for tenant when e.g. network is created.
Closes-Bug: #1913297
Change-Id: Ib73babdd563e3e8c21ce6f63456cc87af414c5aa
New API extension was added in [1] to extend security group rules with
"normalized_cidr" read only attribute.
This patch implements this API extension in Neutron ML2 plugin and
extends security group rules with "normalized_cidr" value.
[1] https://review.opendev.org/#/c/743630/
Related-Bug: #1869129
Change-Id: I65584817a22f952da8da979ab68cd6cfaa2143be
- Add api extension and db model changes to support remote_address_group_id
in SG rules.
- RPC and firewall agent changes will be in the follow-up patches.
Change-Id: I99681736d05eefd82bdba72b3866eab9468ef5dd
Implements: blueprint address-groups-in-sg-rules
This patch implements the last code bits pending to
conclude the new DB engine facade migration.
Due to the resultant interactions in the modified code, is
not possible to submit smaller patches; this code must be
migrated at once.
Partially-Implements blueprint: enginefacade-switch
Signed-off-by: Slawek Kaplonski <skaplons@redhat.com>
Co-Authored-By: Rodolfo Alonso Hernandez <ralonsoh@redhat.com>
Change-Id: Id3f09b78c8d0a8daa7ec4fa6f5bf79f7d5ab8f8b
Included standard attributes ID in some OVO dictionaries to improve
the OVN revision numbers operation. Having this ID, there is no need
to retrieve it from the database.
The following OVOs have been updated:
- port
- network
- subnet
- security group
- security group rule
- router
Closes-Bug: #1904188
Change-Id: Ia16335a2fc8f9324b9489692c76a73e4ef5bef96
This patch adds the verification of whether admin context when
verifying the valid security groups of port.
Change-Id: I2674bdc448d9a091b9fe8c68f0866fd19141c6be
Closes-Bug: #1890539
As spotted in Focal testing patch [0], pep8 test fails with many
C0321 false-positives, reported in pylint as current version does not
support python 3.8 [1]
Use a newer version of pylint and astroid, fixing or disabling some of
the new checks: no-else-*, unnecessary-comprehension, import-outside-toplevel
[0] https://review.opendev.org/#/c/738163/
[1] https://github.com/PyCQA/pylint/issues/2737
Change-Id: Ie646b7093aa8634fd950c136a0eba9adcf56591c
Method _ensure_default_security_group wasn't atomic as it first tries to get
default SG and if that not exists in DB, it tries to create it.
It may happend, like e.g. in Calico plugin that between
get_default_sg_id method and create_security_group method, this default
SG will be created by other neutron worker. And in such case there will
be Duplicate entry exception raised.
So this patch is adding handling of such exception.
Change-Id: I515c310f221e7d9ae3be59a26260538d1bc591c2
Closes-Bug: #1883730
With python 3.x, six.text_type and six.string_type
are just str.
Also removed a six.integer_type since it was the only
one left in a file.
Another step in removing all of six usage from neutron.
Change-Id: I5208dc41bff1983ecd323286f427296b722da62a
Allow the subscriber to know the deleted security group name. It can
help some downstream callback utilization to use the deleted sg name for
additional workflow.
Change-Id: Ia321ff96cf445d20f082779d3f6a96fac07b0943
now filter security groups by tenant_id, will auto ensure default security
group, but project_id not. tenant_id is deprecated, we are used to filtering
by project_id.
Closes-bug: #1875849
Change-Id: I3f1a2698768e2d66811bb290a892a1b089dff4ed
During processing of security group rule list API call Neutron will
now ensure that default security group for project given in the filters
or in the context exists.
It is similar to what is done for list of security groups or creation of
new network/port in the project.
Change-Id: Id6fee5a752968b356b884d939b708a420016c9bc
Closes-Bug: #1864171
Rodolfo Alonso Hernandez