Commit Graph

860 Commits

Author SHA1 Message Date
Rodolfo Alonso Hernandez 70ddf4eef5 Add "socket" NUMA affinity policy
This new extension adds a new parameter to the NUMA affinity policy
list: "socket". The "socket" NUMA affinity policy has been supported
in Nova since [1].

[1]https://review.opendev.org/c/openstack/nova/+/773792

Closes-Bug: #2052786
Change-Id: Iad2d4c461a2aceef6ed2d5e622cce38362d79687
2024-03-21 16:04:24 +00:00
Rodolfo Alonso Hernandez 8830fed191 Open the 2024.2 (Dalmatian) DB branch
According to the TC resolution [1] and the release identification
document [2], the release number will be used as primary identifier in
the development cycle. Release name will be used in marketing.

[1]https://governance.openstack.org/tc/resolutions/20220524-release-identification-process.html
[2]https://governance.openstack.org/tc/reference/release-naming.html

Change-Id: I6fc7a154fa77a8792f93eb40c9b4497c0e67ae2f
2024-03-20 10:00:19 +00:00
Rodolfo Alonso Hernandez 80f547ad1d Add a "port" child table "porthardwareoffloadtype"
This table has a 1:1 relationship with the "port" table, providing
the "hardware_offload_type" field (string).

The "neutron-lib" library minimum version is 3.8.0, that contains
[1].

NOTE: once the OSC patch is merged [2], the documentation will be
updated to reflect how to create a hardware offloaded port without
manually defining the port binding profile,

[1]https://review.opendev.org/c/openstack/neutron-lib/+/882726
[2]https://review.opendev.org/c/openstack/python-openstackclient/+/892792

Partial-Bug: #2013228
Change-Id: I04f232d6c43e39f254c4559caf041dcf05acec21
2023-08-19 06:08:51 +00:00
Rodolfo Alonso Hernandez 8aad094985 Drop unused tables from the Neutron database
This is the list of tables removed from the Neutron database:
* neutron.db.migration.alembic_migrations.external.REPO_CISCO_TABLES
* neutron.db.migration.alembic_migrations.external.REPO_VMWARE_TABLES
* neutron.db.migration.alembic_migrations.external.REPO_BROCADE_TABLES
* neutron.db.migration.alembic_migrations.external.REPO_NUAGE_TABLES
* other tables: "cisco_csr_identifier_map"

Closes-Bug: #2038555
Change-Id: Ia39f759dd7567970b00856e334786e4df74966cf
2023-10-10 16:17:14 +00:00
Brian Haley c282077058 Fix pointless-string-statement pylint warning
For some reason, kilo_initial.py is triggering this warning,
so fix it by using comments instead of a docstring.

Also updated the template to use comments as well, so any
migrations created in the future use them, even though
none seem to be causing a similar warning right now.

Closes-bug: #2036763
Change-Id: I63b7ac83fdd3ebc6251979f1376daf2105489fdc
2023-09-21 10:43:46 -04:00
Rodolfo Alonso Hernandez 7e0ba8bcda Open the 2024.1 (Caracal) DB branch
According to the TC resolution [1] and the release identification
document [2], the release number will be used as primary identifier in
the development cycle. Release name will be used in marketing.

[1]https://governance.openstack.org/tc/resolutions/20220524-release-identification-process.html
[2]https://governance.openstack.org/tc/reference/release-naming.html

Change-Id: I1ffd5878489176f33146688de03367101ede2305
2023-09-09 22:13:40 +00:00
Zuul 7b0cf39008 Merge "[sqlalchemy-20] TableClause.insert constructs Insert object" 2023-08-22 15:25:04 +00:00
Miro Tomaska 43b4c9ebc5 [sqlalchemy-20] TableClause.insert constructs Insert object
Replace with the sqalchemy insert [1] function which is
compatible across versions

[1] https://docs.sqlalchemy.org/en/20/core/dml.html#sqlalchemy.sql.expression.insert

Closes-Bug: #2030903
Change-Id: I0abc9c0dfc65e3e031246f514ff6e1e3593a00f4
2023-08-10 20:21:56 +00:00
Dmitrii Shcherbakov 89702218db Add extra router attributes for ECMP and BFD
* enable_default_route_ecmp
* enable_default_route_bfd

Partial-Bug: #2002687
Change-Id: I3fcd0458d20f20ce40378f90f073f37c41400865
2023-07-25 17:03:34 +02:00
Slawek Kaplonski e41fae522b Default SG api rules template - DB and OVO models
This patch adds DB model, OVO class and DB migration script for
SG rules template used for every new SG created.
It also implements Create/Get/Delete actions for that new resource and
adds API policies for those APIs

Related-Bug: #1983053
Change-Id: Ib3cde1710edd400b972f493b13666d0679a7753c
2023-07-07 10:43:34 +02:00
Christian Rohmann b4eb5d71ab Drop redundant index on ports table
There already exists a unique constraint on the same columns, making
an additional index redundant.

Closes-Bug: #1988421
Change-Id: I8236ec1f685a3ae7c503d3ff8148138a875d702a
2023-07-03 07:51:32 +00:00
Rodolfo Alonso Hernandez 08fe84f443 [sqlalchemy-20] Remove redundant indexes from some tables
The following tables and columns were defined as primary key and key
(index). A column defined as primary key creates an index in the table.
The is no need to create a second one.

Tables and columns affected:
* portdataplanestatuses, port_id
* portdnses, port_id
* portuplinkstatuspropagation, port_id
* qos_policies_default, project_id
* quotausages, resource
* quotausages, project_id
* subnet_dns_publish_fixed_ips, subnet_id
* segmenthostmappings, segment_id
* segmenthostmappings, host
* networkdnsdomains, network_id
* floatingipdnses, floatingip_id

Closes-Bug: #2024044
Change-Id: I271c109a597eb0aa088a7a9c785e8631bfaa01d7
2023-06-23 11:24:22 +00:00
Zuul ac0b0c50a2 Merge "Make "project_id" in "L3HARouterNetwork" unique constraint" 2023-06-09 20:37:03 +00:00
Rodolfo Alonso Hernandez e8cd39b3d7 Make DB migration creating indexes in RBACs conditional
This patch makes conditional the existing DB migration that adds
the new indexes "target_tenant" and "action" in the "*rbacs" tables.
The rationale of this patch is to be able to manually improve older
systems by just manually creating the indexes in the database.
Once these indexes are added, those operations including RBACs
checks (all these called from non-admin user to RBAC administrated
resourced) will be improved.

This patch is avoiding the migration issue a system could find if
these indexes have been manually added and then the system is
upgraded. The new check added will first retrieve the table indexes;
if the index is already present, the index addition is skipped.

Closes-Bug: #2020802
Change-Id: I1962fbc844bb67180e9071bcee01f8e95853bdda
2023-05-29 15:26:11 +00:00
yatin 452973c0db Revert "Use a writer context for the online alembic migrations"
This reverts commit 82029c2c51.

Reason for revert: fullstack tests broken with this.

Change-Id: Ibd75c3526a3a8c4138c5f266448299786ccbf045
Related-Bug: #2019802
Related-Bug: #2015847
2023-05-22 13:29:55 +00:00
Rodolfo Alonso Hernandez 43c756d728 [alembic] Alembic operations require keywords only arguments
Since [1] (release 1.11.0), the Alembic operations "now enforce
keyword-only arguments as passed as keyword and not positionally"
(from the commit message).

This change is compatible with the previous versions (as confirmed
in the CI).

[1]df75e85489

Closes-Bug: #2019948
Change-Id: Ic5e7790377668d83be2cebbba8f6bfa09f7cee5e
2023-05-17 17:21:03 +02:00
Rodolfo Alonso Hernandez 6a2ccfac32 Make "project_id" in "L3HARouterNetwork" unique constraint
There could be just only one HA network per project. This database
enforcement guarantees this limitation.

Partial-Bug: #2016198
Change-Id: Ieb8aac6244d384b0af522f9ba145e9367de2c8ef
2023-05-10 09:15:05 +02:00
Bence Romsics 0390ada97c port-hints: api extension
api extension
db model
db migration
ovo (including changes affecting push rpc)
extension driver
policies

To enable this:

* neutron-db-manage upgrade 6f1145bff34c
* ml2_conf.ini:
  [ml2]
  extension_drivers += port_hints

This patch also bumps neutron-lib requirement to 3.5.0.

Change-Id: I80816618285d742775bc0534510c0f874f84ed2e
Partial-Bug: #1990842
Related-Change (spec): https://review.opendev.org/c/openstack/neutron-specs/+/862133
Related-Change (n-lib api-def): https://review.opendev.org/c/openstack/neutron-lib/+/870080
2023-05-09 11:49:17 +02:00
Zuul d893368356 Merge "Reduce lock contention on subnets" 2023-05-05 20:12:21 +00:00
Felix Huettner c0af5b3b5e Reduce lock contention on subnets
in [1] a lock was introduced with the goal of preventing subnets from
being deleted while ports are being created in them in parallel.
This was acheived by aquiring an exclusive lock on the row of the
subnet in the Subnet table when adding/modifying a port or deleting
the subnet.

However as this was a exclusive lock it also prevented concurrent port
modifications on the same subnet from happening. This can cause
performance issues on environment with large shared subnets (e.g. a
large external subnet).

To reduce the lock contention for this case we split the lock in two
parts:

* For normal port operations we will aquire a shared lock on the
  row of the subnet. This allows multiple such operations to happen in
  parallel.
* For deleting a subnet we will aquire an exclusive lock on the row of
  the subnet. This lock can not be aquired when there is any shared
  lock currently on the row.

With this we maintain the same locking level as before, but reduce the
amount of lock contention happening and thereby improve throughput.

The performance improvement can be measured using rally test [2].
(improving from 21 to 18 seconds).
Alternatively it can be tested using 250 parallel curl calls to create a
port in the same network. This improves from 113s to 42s.

[1]: https://review.opendev.org/c/openstack/neutron/+/713045
[2]: https://github.com/openstack/rally-openstack/blob/master/samples/tasks/scenarios/neutron/create-and-delete-ports.json

Closes-Bug: #2009055
Change-Id: I31b1a9c2f986f59fee0da265acebbd88d2f8e4f8
2023-05-05 17:01:43 +02:00
Rodolfo Alonso Hernandez 82029c2c51 Use a writer context for the online alembic migrations
``DBConnection`` class does not retrieve the URL string
correctly and cannot connect to the database. This patch
replaces it with default writer context that any database
operation transaction uses in Neutron.

Closes-Bug: #2015847

Change-Id: I60523690bc170ec4ba81312ada0f2d83542cae67
2023-04-25 08:25:45 +00:00
Rodolfo Alonso Hernandez 97aa84b69a Open the 2023.2 (Bobcat) DB branch
According to the TC resolution [1] and the release identification
document [2], the release number will be used as primary identifier in
the development cycle. Release name will be used in marketing.

[1]https://governance.openstack.org/tc/resolutions/20220524-release-identification-process.html
[2]https://governance.openstack.org/tc/reference/release-naming.html

Change-Id: I0890b344096d8f8350fedd9c123e27844b671882
2023-03-21 10:14:04 +01:00
Zuul bf568cb24e Merge "Change the release tag to use the release identification" 2023-03-11 20:24:38 +00:00
Rodolfo Alonso Hernandez d4a85833a7 [sqlalchemy-20] The Session.begin.subtransactions flag is deprecated
Closes-Bug: #2008276
Change-Id: I5472db98c6ae24b8b1e6add2c0d73aeb1ed016ca
2023-03-03 08:04:14 +00:00
Rodolfo Alonso Hernandez 39b65575cd Change the release tag to use the release identification
According to the TC resolution [1] and the release identification
document [2], the release number will be used as primary identifier in
the development cycle. Release name will be used in marketing.

[1]https://governance.openstack.org/tc/resolutions/20220524-release-identification-process.html
[2]https://governance.openstack.org/tc/reference/release-naming.html

Change-Id: Ia3da9bf6d6113edacb0e21de9a51bd60d5e2465b
2023-03-02 00:41:19 +01:00
Rodolfo Alonso Hernandez 41846a6e8b [sqlalchemy-20] The Connection.connect() method is considered legacy
Closes-Bug: #2008227
Change-Id: I3dfa2688f7288dda43883b3cfba45e5f15f182fb
2023-03-01 14:36:51 +00:00
Sahid Orentino Ferdjaoui eeff5b3c81 db: add segment_index to the unique constraint
For multi segments support we have update the unique contraint so
`segment_index` will be part of it.

Related-Bug: #1791233
Partial-Bug: #1956435
Partial-Bug: #1764738
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
Change-Id: Ic564131dcd7525fc5f24c3282688e3584cd2e2e0
2022-11-23 08:43:21 +01:00
Brian Haley 55b16d7b7c Fix some pylint indentation warnings
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/db.

Trivialfix

Change-Id: I9311cfe5efc51552008072d84aa238e5d0c9de60
2022-11-03 19:50:54 -04:00
Rodolfo Alonso Hernandez 1fe4edab80 Open Antelope DB branch
Change-Id: I45da9a285536ddb04d6cb1def2e5fbcb866b4baf
2022-09-23 01:20:41 +02:00
Rodolfo Alonso Hernandez 7857a3194b Don't retrieve SG port bindings when deleting a SG
Do not retrieve the security group port bindings in the database
transaction that deletes a security group. In the previous context,
if a security group port binding is present on the database,
the method raises a ``SecurityGroupInUse``. It is unneeded to
retrieve them again.

This patch also changes the
``SecurityGroupPortBinding.security_group_id`` foreign key. Now if
the security group is deleted, any security group port binding related
will be too, using the database engine. That will ensure no leftover
remains in the database. Although the check done in
"delete_security_group" before the security group is deleted, there is
a minimal possibility of race condition between the first database
transaction (SG port binding check)  and the second one (SG deletion).

Trivial-Fix

Change-Id: I1c9c2dd95b98a7cc77509b0d537d7c7766765275
2022-08-10 20:51:29 +02:00
Zuul 3ffbf831cf Merge "Forbid create ndp proxy on same router with same ip address" 2022-07-20 22:47:19 +00:00
Rodolfo Alonso Hernandez d7ba5948ff Replace "Inspector.from_engine()" with "sqlalchemy.inspect()"
Since SQLAlchemy 1.4, the method "Inspector.from_engine()" is
deprecated.

This patch is related to [1].

[1]https://review.opendev.org/c/openstack/neutron/+/808103

Related-Bug: #1943155
Change-Id: Ic26da2e20c0878204cf9565e173b4825cb36f34c
2022-06-29 18:42:18 +00:00
yangjianfeng 9dd06e9c44 Forbid create ndp proxy on same router with same ip address
Create multiple ndp proxies with same ip address within one router
is invalid. The related database constraint was missed in previous
patchsets. The patch add some codes fixed this error.

Additionally, Fixed two typo errors.

Related-Bug: #1877301
Change-Id: Iab24ad78a3d4d9b0ee584cf0986328c9ae2bd16a
2022-06-04 10:27:13 +08:00
yatinkarel 5957e90575 Create an index for subnetpools.address_scope_id
The method "get_network_address_scope" filters
"Subnetpool" with "address_scope_id" using an
exact match. Making the column indexed will improve
the query performance.

Closes-Bug: #1973726
Change-Id: Ib3f8e18ba28b277d5fa02dd386ca80a5a113c247
2022-06-01 19:41:53 +05:30
Rodolfo Alonso Hernandez db2ae854cf Create an index for "ports.network_id"
The method ``_port_filter_hook``, that is added in any "Port" SELECT
command, filters the database "Port" registers by "network_id", using
an exact match. This query speed will improve if this column is
indexed in the database engine.

Closes-Bug: #1973349
Change-Id: Ia20f96dc78ea04bb0ab4665e6d47a6365789d2c9
2022-05-13 13:49:36 +00:00
Jakub Libosvar 4d3a274765 Don't register config options on imports
Importing some modules lead to registering config options that may
collide with config options from a project that calls the import. This
patch wraps the side effect that registers config options into a
function that needs to be called in case the caller wants to register
the options.

This solution is also not perfect as it guards the common options to be
registered only once even if the function is called multiple times. This
is to solve problems in unittests, ideally we should always call the
function just once even in our testing suites.

Resolves-Bug: #1968606
Change-Id: Ic1532eb8de887ff1b1085206df11f53e22f7f524
Signed-off-by: Jakub Libosvar <libosvar@redhat.com>
2022-04-13 05:49:15 +00:00
Pedro Martins b271c82d10 Extend database to support portforwardings with port range
This patch is the second of a series of patches
to implement floating ip port forwarding with
port ranges.

The specification is defined in:
https://github.com/openstack/neutron-specs/blob/master/specs/wallaby/port-forwarding-port-ranges.rst

Implements: blueprint floatingips-portforwarding-ranges
Related-Bug: #1885921
Change-Id: I43e0b669096df865f37c74ddbd050b3b177fd5e5
2022-03-15 09:10:23 -03:00
Rodolfo Alonso Hernandez 5a60701f1f Open Zed DB branch
Change-Id: I03a2b5ee881864226bbe5ef78e38707213cbbbfe
2022-03-05 19:27:21 +00:00
Yang JianFeng a0a25cb15c [Server Side] L3 router support ndp proxy
Change-Id: I9b92702af8a235443a2fa1aea3997f3d40a03fc3
Partial-Bug: #1877301
2022-02-03 10:07:46 +08:00
Rodolfo Alonso Hernandez ebe9e046c6 Create an index for "agents.host" column
This index improves any query filtering by agent host name.

Closes-Bug: #1955639
Change-Id: Ice4fd6319d4c7d403bdc74a25e3d7cac31fb4c9f
2022-01-13 05:20:58 +00:00
Rodolfo Alonso Hernandez 0e1ec52118 Remove "PortBindingMixin" class and related DB table
Not Neutron nor other active projects use "PortBindingMixin" class or
the table "portbindingports" anymore.

Closes-Bug: #1956980
Change-Id: I34424a271f6c66cd99852c6109a96a4dcf374913
2022-01-08 14:53:21 +00:00
Rodolfo Alonso Hernandez 8813b0ed2d Replace "target_tenant" with "target_project" in RBAC OVOs and models
This is part of the remaining technical debt of the specs
https://specs.openstack.org/openstack/neutron-specs/specs/newton/moving-to-keystone-v3.html

Blueprint: https://blueprints.launchpad.net/neutron/+spec/keystone-v3

Change-Id: I2d2fd4d1802c9dfe0778ac8fdddc7b9a8afe7d25
2021-12-03 10:48:57 +00:00
Oleg Bondarev cd1d96863e Add Local IP Extension and DB
This adds Local IP API extension, DB and OVO models, DB mixin,
migration and service plugin.

Partial-Bug: #1930200
Change-Id: I0ab7c5e9bc918f7fad282673ac6e32e1b01985c5
2021-11-11 10:08:23 +03:00
Sebastian Lohff b993ebb407 Fix dynamic segment allocation race condition
When two segments are concurrently created this could have resulted in
both threads creating a segment, thus resulting in two segments with
different segmentation ids. To prevent this we now introduce a new
unique constraint onto the networksegments table, which requires
(network_id, network_type, physical_network) to be unique, which allows
only a single segment with a single segmentation id to exist per
combination of these three values.

With the constraint in place a DB error will be thrown, which will cause
allocate_dynamic_segment() to be executed again and this time it will
find the already existing segment. To make sure that no additional DB
objects are created when segment creation failed we need to put all of
the allocation code into a DB transaction.

Change-Id: I407ae88d69ed971bf8d9a9b79120366f33bb56fd
Closes-Bug: #1791233
2021-10-07 13:20:04 +02:00
Rodolfo Alonso Hernandez f8c879ddbf Add new indexes to RBAC DB models
Added two new indexes to all RBAC DB models: "target_tenant" and
"action".

The DB models affected are "networkrbacs", "qospolicyrbacs",
"securitygrouprbacs", "addressscoperbacs", "subnetpoolrbacs" and
"addressgrouprbacs".

The goal of this patch is to speed up the model query if RBAC apply to
this object. If the object query scope is a project, [1] will be added
to the DB query. If "action" and "target_tenant" are indexed, the exact
match filtering will be faster.

[1]890d62a3df/neutron_lib/db/model_query.py (L123-L131)

Change-Id: I0a70a1a500fad52ca55006d6e2ebc1044aef0fc8
Closes-Bug: #1918145
2021-09-30 12:55:02 +00:00
Zuul 79c2b5f05d Merge "Add API extension for QoS minimum pps rule" 2021-09-30 11:17:24 +00:00
Zuul 106fa3e6d3 Merge "Replace "Inspector.from_engine()" with "sqlalchemy.inspect()"" 2021-09-29 14:14:29 +00:00
Przemyslaw Szczerbik 56044db26d Add API extension for QoS minimum pps rule
This patch implements support for CRUD operations for QoS minimum
packet rate, for example:

DELETE /qos/policies/$POLICY_ID/minimum_packet_rate_rules/$RULE_ID

Placement or dataplane enforcement is not implemented yet.

Partial-Bug: #1922237
See-Also: https://review.opendev.org/785236
Change-Id: Ie994bdab62bab33737f25287e568519c782dea9a
2021-09-29 12:27:30 +02:00
Rodolfo Alonso Hernandez 42fda206e9 Replace "Inspector.from_engine()" with "sqlalchemy.inspect()"
Since SQLAlchemy 1.4, the method "Inspector.from_engine()" is
deprecated.

Error message:
"The from_engine() method on Inspector is deprecated and will be removed
in a future release.  Please use the sqlalchemy.inspect() function on an
Engine or Connection in order to acquire an Inspector. (deprecated
since: 1.4)"

Minimum version of SQLAlchemy required is bumped to 1.4.23.

Change-Id: I6cf5944ccb3a0532cbf123ddc0d7df6b6de80af1
Closes-Bug: #1943155
2021-09-23 13:41:50 +00:00
Rodolfo Alonso Hernandez 3d7929591c Open Yoga DB branch
Change-Id: If0fa70e8ebab3c704c2350e28953ebfcc3a79fa9
2021-09-20 15:08:07 +00:00