Remove B303 (md5, sha1 for python<3.9) and
remove B311 (Standard pseudo-random generators are
not suitable for security/cryptographic purpose) from
the skip list of bandit execution.
Change-Id: I6e9e61e7f94dc9ca339942529af8997adef45e38
There is a chance on real environment that a port group doesn't have any
correspondent security group (and there are maintenance tasks that will
remove them). This patch avoids a DriverError from Neutron in case we
are in an environment with a port group that was mistakenly left over
due to any reason. Instead, a Warning log will be raised.
Related-bug: #2032929
Change-Id: I42208557c8522d6fbc29df8a3c7d0367cace31e4
Right now, as per kernel limitation, the burst limit is not correctly
enforcing the rate and burst when using the ovn "log-related" option and
stateless security groups. We log exactly double the burst. Creating a
new meter that limits the rate and burst to half of the expected ones is
a workaround that solves the issue.
Closes-bug: #2032929
Signed-off-by: Elvira García <egarciar@redhat.com>
Change-Id: Ib0047d38c58bcebb23c8887e7934987ff8c8a432
After updating pylint, it started emitting additional "R"
warnings in some cases, fix some of them.
use-a-generator,
unnecessary-lambda-assignment,
consider-using-max-builtin,
consider-using-generator,
consider-using-in,
use-list-literal,
consider-using-from-import
Trivialfix
Change-Id: Ife6565cefcc30b4e8a0df9121c9454cf744225df
There is the possibility that db_remove raises an error if an ACL was
deleted on parallel for other reasons while deleting a log object. On a
normal situation, this command would remove the no-longer-needed
'log-related' property, but since the ACL is no longer there, it will
raise an error. For this case, it is not problematic to have an ACL
deleted mid-operation, so it should not raise any error.
Related-Bug: #2019887
Signed-off-by: Elvira García <egarciar@redhat.com>
Change-Id: I154393529134b5861e0ef0283257ef964fe057fd
Up until now, we needed to remove all logging objects to see the
meter-band properties being changed after a server restart. Now we check
for inconsistencies between the neutron configuration and the OVN
meter-band object after a restart. The function create_ovn_fair_meter is
now located in the ovn_driver instead of the log_driver so as to be able
to call it from the maintenance task.
Closes-bug: #2017145
Signed-off-by: Elvira García <egarciar@redhat.com>
Change-Id: I24cef85ed68c893a740445707f88296d763c8de8
Before this patch, we would only get logged the client to server side of
the communication. The OVN allow-related ACL option was implemented [0]
so as to be able to log also the packets that are going from server to
client. This patch implements the addition of that feature in Neutron
and needs OVN version 22.03 or updated 21.12.
[0] https://patchwork.ozlabs.org/project/ovn/patch/20220201141118.1846390-1-mmichels@redhat.com/
Closes-Bug: #2003706
Change-Id: I72d061c333f53e07f6feedec032e2c0b06a61248
Signed-off-by: Elvira García <egarciar@redhat.com>
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
some remaining ones in miscellaneous directories.
Also cleanup any remaining code that I missed in this
series, or has changed since I started.
Trivialfix
Change-Id: I17b4779020a7bfb369c3e721ab6638cd4a6ab50c
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/services.
Trivialfix
Change-Id: Ie9779b257981bc80e69639cdaa4d7dfd0ffa5809
Previously, only the first log object created that associated to a
certain ACL would be able to make changes to the True/False property of
that ACL. This patch makes the driver to take in consideration each log
object created to enable or disable an ACL logging status. A functional
test is added so as to ensure correct behaviour of this feature.
Closes-Bug: #1996780
Change-Id: Ib9663495f30562f79babef163729a0c43812089d
Signed-off-by: Elvira García <egarciar@redhat.com>
During log delete we fetch all the acls for
all the pgs if there is only one log object
and do clear log of all these acls, but if
one or more acls of any of the pgs is removed
concurrently, db_set fails as acl is not found.
This patch proposes to only do log clear of those
acls which are available and add log message for the
acls which were deleted concurrently.
Also add a unit test for this case where one of
the acl get's missing.
Closes-Bug: #1971569
Change-Id: I58487024c8d0352776307f0185f0812bb00036ae
If a security group object modified an Access Control List in OVN, it
could not be modified by another object. This change prevents the driver
from associating more ACLs than needed if the network log object created
is to log the dropped traffic. Functional tests consider the existence
of events now.
Closes-Bug: #1956763
Change-Id: Ib392b46308aaf76386a747870c5600790a268d51
Added "update_network" method implementation to all "L3AgentExtension"
child classes. This new method will be added as a new abstract method
in future neutron-lib releases.
Needed-By: https://review.opendev.org/c/openstack/neutron-lib/+/818536
Related-Bug: #1950454
Change-Id: Ib50d37d57c3d6a69a68876ae04f468325516d4eb
NOTE: this patch is an ammend of [1].
When a SG or a port is deleted, the related log entry should be
too. A log entry has the following fields:
- log.resource_id = SG ID
- log.target_id = port ID
[1] was deleting all log entries, related or not with the SG ID
deleted. This is because "get_logs_bound_sg" returned all log entries,
including those ones without any "resource_id" or "target_id".
Now this method can return only the log entries related to a port or
a SG, excluding those ones without those two parameters populated.
Closes-Bug: #1939558
[1]https://review.opendev.org/c/openstack/neutron/+/804237
Change-Id: Icb92327a06486e168ce064532d819347e6031cc1
The OVS Firewall has a singleton class that manages the conjuction IDs
to used in the OF rules. Those conjuntions are used to group rules
related to remote security group IDs.
Now each time the OVS agent is started, the OVS Firewall initial
conjunction ID is calculated based on the present OF rules. This value
and the next one used won't clash with any present rule in the
integration bridge during the initial transient period.
Related-Bug: #1934917
Change-Id: Ie2e4441f766947a2164dec2d1555c7049428903f
When security group is deleted, log entries configured for that SG
weren't removed and stayed in the DB as orphaned entries.
That patch fixes that by subscribe logapi service plugin to the
AFTER_DELETE security group events.
Closes-bug: #1939558
Change-Id: Ibffbbd3b529c31372715d9a064f7cb6bada8c424
Map OpenStack SG stateful=False to OVN ACL allow-stateless action verb.
The verb is added in the latest OVN release, 21.06. Inspect db schema to
determine if the new action is supported by OVN before trying to create
it. Fall back to allow-related when it's not supported yet.
Also-Needs: I7343fb609fab91c20490842378747f7265241e82
This will require ovsdbapp version bump with the patch mentioned above
to make it work.
Change-Id: Ic1c36fb71a9d03e8697583a1ea9453d4c0052f74
This is patchset 2 of 2 for OVN driver handling of security-group-logging.
It includes the core changes and tests for this feature.
This feature requires OVN 20.12 [0] or newer. Functional test will be
skipped for non-supported versions.
Related-Bug: 1468366
Closes-Bug: 1914757
[0]: 880dca99ea
Change-Id: Ic86fa70eb34c9b178267b80de1f8883a3ef03e98
Signed-off-by: Flavio Fernandes <flaviof@redhat.com>
Support security group rules with remote_address_group_id in openvswitch
firewall. This change reuses most of the firewall functions handling remote
security groups to also process remote address groups. The conjunctive flows
for a rule with remote_adress_group_id are similar to others with
remote_group_id but have different conj_ids.
Change-Id: I8c69e62ba56b0d3204e9c12df3133126071b92f7
Implements: blueprint address-groups-in-sg-rules
Move the duplicated code to a base class BaseNeutronAgentOSKenApp.
Authored-By: Swaminathan Vasudevan <SVasudevan@suse.com>
Co-Authored-By: Slawek Kaplonski <skaplons@redhat.com>
Related-Bug: #1774459
Change-Id: I10c64c318cf04692c30eff6cb718450ea174caf0
Bump astroid test requirement to 2.4.0
Older versions trigger an error on wrapt dependency:
https://github.com/PyCQA/astroid/issues/755
Bump pylint accordingly to new astroid
Fix some new PEP8 warnings appearing with new versions, and filter out
the larget I202 "Additional newline in a group of imports" one for now
Drop psutil from functional requirements, it indicated an old version
and we have it in common requirements now
Bump a series of lower-constraints and requirements to work with new pip
resolver, testing with steps outlined at:
http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019285.html
This includes eventlet 0.22.1, previous versions triggered a hard to
track error on enum34
Cap cryptography in lower-constraints to prevent discovery failure in
relevant job (other jobs have it capped via upper-constraints)
Change-Id: Ie74ea517a403e6e2a7a4e0a245dd20e5281339e8
Closes-Bug: #1907242
As spotted in Focal testing patch [0], pep8 test fails with many
C0321 false-positives, reported in pylint as current version does not
support python 3.8 [1]
Use a newer version of pylint and astroid, fixing or disabling some of
the new checks: no-else-*, unnecessary-comprehension, import-outside-toplevel
[0] https://review.opendev.org/#/c/738163/
[1] https://github.com/PyCQA/pylint/issues/2737
Change-Id: Ie646b7093aa8634fd950c136a0eba9adcf56591c
There were a couple of versions of oslo.log that were
not backwards-compatible. Now that a fixed version was
released, bump lower-constraints to require it and
remove the temporary workaround.
Change-Id: If38105ceaa48a0520ae8243982b736d0bc99ec3a
Related-bug: #1871840
Versions of oslo.log >= 4.1.2 require a second argument when
initializing the OSJournalHandler class. While I've started
a review to fix that regression [0], bump oslo.log to a later
version that requires this new flag and pass it to fix the
gate.
Also had to blacklist the OVN tempest IPv6 hotplug test
since it is failing too much to pass the check jobs.
[0] https://review.opendev.org/732457
Change-Id: Ic9bbb43aa832ad6cc45d57328b40afe4468ddfca
Related-bug: #1871840
Related-bug: #1881558
With python 3.x, classes can use the metaclass= logic
to not require usage of the six library.
One step in removing all of six usage from neutron.
Change-Id: I2f815e412d9a96eb5faf2b3bb3a1e393a9db9309
Consume the logapi constants from neutron-lib by removing them from
neutron and using them from neutron-lib instead.
NeutronLibImpact
Depends-On: https://review.opendev.org/#/c/660826/
Change-Id: I55edf26598d38de34038f005b05ba3d2e96972aa
Today a number of classes define their supported_extension_aliases
using static strings rather than API definition ALIASes. This patch
switches them to use the ALIAS where applicable.
Change-Id: I716270c68a9fcd850c3c26de31bc13ea16def23d
The neutron.common.rpc module has been in neutron-lib for awhile now and
neutron is shimmed to use neutron-lib already.
This patch removes neutron.common.rpc and switches the code over to use
neutron-lib's implementation where needed.
NeutronLibImpact
Change-Id: I733f07a8c4a2af071b3467bd710290eee11a4f4c
Today the neutron common exceptions already live in neutron-lib and are
shimmed from neutron. This patch removes the neutron.common.exceptions
module and changes neutron's imports over to use their respective
neutron-lib exception module instead.
NeutronLibImpact
Change-Id: I9704f20eb21da85d2cf024d83338b3d94593671e
The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
NeutronLibImpact
Change-Id: I97120faeec73690592ed21a5ec3c6202f61e1429
A log request for SNAT resource should specify resource_type='snat'
and resource_id=router_id. So this patch adds a validation method for
the incoming log request.
Co-Authored-By: Kim Bao Long <longkb@vn.fujitsu.com>
Change-Id: I4fff5a9c8877d192aab780eed9be90452711fb3d
Partial-Bug: #1752290
This change is a follow-up to Ib6ced838a7ec6d5c459a8475318556001c31bdf,
reintroducing a single place for applying the NORMAL action to
egress traffic, which is necessary to fix a regression introduced
by Ib6ced838a7ec6d5c459a8475318556001c31bdf.
Change-Id: I60d299275effd9ef35c8007773d3c9fcabfa50fa
Partial-Bug: 1789878
This patch aims to fix a co-existence problem between security_group
and firewall_group log resources due to incorrect log querying from
database.
Change-Id: Ic60ad436e0fbb23cdae0e63eaeb73130ebf02089
Closes-Bug: #1787119
Packet sent to table 91 are considered accepted by the egress pipeline
and NORMAL action is used by default in this table. However, if we
create a security group logging resource, then ovs flows log will be
added into this table with higher priority. Therefore packet matches
with ovs flows log will be sent to CONTROLLER and never forward.
So this patch append action=NORMAL into ovs flows log to forward
the packet and send it to CONTROLLER for logging.
Closes-Bug: #1787106
Change-Id: I6e95e2e646ec8a5507c7f140ab2c4a56be8404c3
In production environment, it's quite difficult to require operator
to enable debug log level. This patch changes LOG.debug to LOG.info
of packet_in_handler function. Therefore operator can get packet log
data while debug=False or True in config file. This patch also adds
timestamp to log_file if local_output_log_base is specified.
Partial-Bug #1781372
Change-Id: I729e57caecd502d018084c3c766278006bcbf0c9
Currently, we cannot get ACCEPT packet log because there are some
changed related to ovs firewall code since ovs firewall logging has
been merged.
Regarding to performance perspective, we only log first accepted packet.
So we only need to forward first accepted packet of each connection
session to table 91 and table 92.
So this patch fixes these issues.
Closes-Bug: #1782576
Change-Id: Ib6ced838a7ec6d5c459a8475318556001c31bdf0
This patch allows logging plugin to handle resource callback
events AFTER_UPDATE/AFTER_CREATE/AFTER_DELETE of security
group, firewall group.
Co-Authored-By: Kim Bao Long <longkb@vn.fujitsu.com>
Partial-Bug: #1720727
Change-Id: I1cb170de1dbb7ac5380d386d850241f3c4a2f225
elajkat
yatinkarel