Port create/update are most time-consuming operations
on subnet creation. As example, in those cases where
several subnets are created over the same network the
response time for those port operations is linearly
increased as the total subnets increases.
This patch improves the number of queries required on port
operations in order to reduce the response time.
Closes-Bug: #1977831
Change-Id: I0fccf36a2035e8f6c2fa8dab0307358da600c8f7
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line
inconsistency.
Fixed an inconsistency in the OVSBridge.deferred() definition
as it needs to also have an *args argument.
Fixed an issue where an l3-agent test was mocking
functools.partial, causing a python3.8 failure.
Unit tests only, removing from tests/base.py affects
functional tests which need additional work.
Change-Id: I40e8a8410840c3774c72ae1a8054574445d66ece
Add a new DB table "network_subnet_lock". The primary key will be the
network_id. When a subnet is created, inside the write context during
the "subnet" object creation, a register in the mentioned table is
created or updated. This will enforce the serialization of the "subnet"
registers belonging to the same network, due to the write lock in the
DB.
This will solve the problem of attending several "subnet" creation
requests, described in the related bug. If several subnets with the
same CIDR are processed in parallel, the implemented logic won't reject
them because any of them will not contain the information of each other.
This DB lock will also work in case of distributed servers because the
lock is not enforced in the server logic but in the DB backend.
Change-Id: Iecbb096e0b7e080a3e0299ea340f8b03e87ddfd2
Closes-Bug: #1852777
Removed E125 (continuation line does not distinguish itself
from next logical line) from the ignore list and fixed all
the indentation issues. Didn't think it was going to be
close to 100 files when I started.
Change-Id: I0a6f5efec4b7d8d3632dd9dbb43e0ab58af9dff3
Except test_subnet, there are no more object tests that would require
the plugin configured, so let's not go through the hassle of setting it
up. (For the record, this setup was not needed since
If1252c42c49cd59dba7ec7c02c9b887fdc169f51).
Change-Id: Iafb886c17dbf9a91a3514b0caa1bcd14093a796c
New facade is enabled by setting new_facade = True for the object of
interest. With new_facade on, all OVO actions will use the new reader /
writer decorator to activate sessions.
There are two new facade decorators added to OVO: db_context_reader and
db_context_write that should be used instead of explicit
autonested_transaction / reader.using / writer.using in OVO context.
All neutron.objects.db.api helpers now receive OVO classes / objects
instead of model classes, since they need to know which type of engine
facade to use for which object. While it means we change signatures for
those helper functions, they are not used anywhere outside neutron tree
except vmware-nsx unit tests, and the latter pass anyway because the
tests completely mock out them disregarding their signatures.
This patch also adds several new OVO objects to be able to continue
using neutron.objects.db.api helpers to persist models that previously
didn't have corresponding OVO classes.
Finally, the patch adds registration for missing options in
neutron/tests/unit/extensions/test_qos_fip.py to be able to debug
failures in those unit tests. Strictly speaking, this change doesn't
belong to the patch, but I include it nevertheless to speed up merge in
time close to release.
There are several non-obvious changes included, specifically:
- in neutron.objects.base, decorator() that refreshes / expunges models
from the active session now opens a subtransaction for the whole span of
call / refresh / expunge, so that we can safely refresh model regardless
of whether caller opened another parent subtransaction (it was not the
case for create_subnetpool in base db plugin code).
- in neutron.db.l3_fip_qos, removed code that updates obj.db_model
relationship directly after corresponding insertions for child policy
binding model. This code is not needed because the only caller to the
_process_extra_fip_qos_update method refetches latest state of floating
ip OVO object anyway, and this code triggers several unit test failures.
- unit tests checking that a single commit happens for get_object and
get_objects are no longer valid for new facade objects that use reader
decorator that doesn't commit but close. This change is as intended, so
unit tests were tweaked to check close for new facade objects.
Change-Id: I15ec238c18a464f977f7d1079605b82965052311
Related-Bug: #1746996
This method allows to update matching resources with requested values
without doing full fetch/set/update cycle. This is also handy to "lock"
records in database with UPDATE WHERE.
Change-Id: I2347fedbecef823babe3d8038f5a74b21fc0a602
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Most of the _create_test methods create a property and only uses
its id during their execution. This change pretends to simplify
the returned data and reduce the number of lines used for the
creation of test data used during the execution of Unit Tests for
Oslo-Versioned Objects.
Change-Id: Icfc251a7d460246e91e260a4ac503d10c4031170
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
This method can be used by subclasses to inject specific values into
randomly generated values for objects, for example, when we need to link
those automatic test objects to other objects through foreign keys.
Change-Id: Ic32ebf5ef055b25e4c765a9bf71ca4657d3b3d70
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Adding SubnetServiceType to Subnet OVO as synthetic field.
It is not required to change the Subnet version, since it is not used.
Change-Id: I0ccff39dcb5e41351b0a4aeba7c98cfba24a1689
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
DNSNameServerDbObjectTestCase was overriding self.db_objs and
self.obj_fields if the attributes did not have unique order/address
fields generated by get_random_fields. But since
Id1ca4ce7b134d9729e68661cedb2f5556e58d6ff landed, we should have also
updated self.objs, otherwise test_filtering_by_fields will fail later
when it will try to find an object with attributes that were not used
when creating the object targeted by the filtering attempt.
Instead of adding the update for self.objs in the
DNSNameServerDbObjectTestCase test class, I went with an alternative
approach, getting rid of overriding logic completely. The rationale for
the path is that there is nothing wrong in duplicate address and order
field values (at least as per underlying model definition), and hence
our tests should be resilient against that kind of scenario.
So instead of comparing all fields for an object, just make sure that
the order monotonically goes up/down in the sorted result, and ignore
other fields to be strictly ordered.
Change-Id: Ic956072de5dab336f83b04bddfa9da967b2865b2
Closes-Bug: #1623425
This should reflect the code under test better, and is needed for one of
patches in the review queue (I130609194f15b89df89e5606fb8193849edd14d8)
to pass some of those test cases.
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Change-Id: Id1ca4ce7b134d9729e68661cedb2f5556e58d6ff
Limit the SQL query number when loading the synthetic fields for Neutron
DB objects. If synthetic fields is defined in SQL model as orm.relationship,
the objects are loaded with main object. No need to query for those resources
once more.
It has impact on performance in scale, to have constant number of SQL queries
when number of resources stored in DB is increasing.
Change-Id: Ie54d9f51642edec1f4533062eb8306b78a1b6f20
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Fields types and nullable parameters were adjusted according to usage model
and tests expectations.
Added segment_id as it was added in Subnet DB model in Newton.
Change-Id: I0d4a9934af9253321593a6bd64edfd6f660c26f1
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Problem found when querying for shared subnet. Current implementation for
filtering is applying the filter_by(id=xxx) to NetworkRBAC id, not the
subnets.id field.
Change-Id: I9343ccbbb9723582db4365d671505d0bb6eba72d
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Subnet 'shared' attribute is dependent from network 'shared' state.
Added helper methods in rbac_db to calculate the share state based on
rbac_entries.
Refactor existing RbacNeutronDbObjectMixin is_shared_with_tenant method
to be able to use it within already opened session.
Change-Id: I235cf3d37fefa27c005cff992a08fd324e910e75
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Added sorting by 'order' field in DNSNameServer.get_objects()
Loading the synthetic fields for Subnet object, dns_nameservers needs
to be sorted by order.
Change-Id: Icd833eae062b23555d4e53cf2cbd5bcd17cf2108
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
This patch is introducing the following OVOs:
- DNSNameServer
- Route
- IPAllocation
- Subnet
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Change-Id: I3fca197098f24f68ea2b2915851c68dc170e781c
Fernando Royo