Now, after [1] was merged all tests where neutron_lib.context.Context is
used needs to have configured config because every time
instance of the neutron_lib.context.Context class is created it tries to
initialize policies and check if it's service_role or not.
Until now this was done only in the BaseTestCase class so all tests
which inherits from the DietTestCase or SqlTestCaseLight classes did not
have it and those tests were failing.
Additionally this patch removes adding CONF.reset() to the cleanUp in
some tests as it's not needed anymore because it is done in the setUp of
the DietTestCase class.
Closes-Bug: #2025753
[1] https://review.opendev.org/c/openstack/neutron-lib/+/887191
Change-Id: I52597ab066c3d7a2d835278431e00b63c8f55c46
This new quota driver, ``DbQuotaNoLockDriver``, does not create a lock
per (resource, project_id) but retrieves the instant (resource,
project_id) usage and the current (resource, project_id) reservations.
If the requested number of resources fit the available quota, a new
``Reservation`` register is created with the amount of units requested.
All those operations are done inside a DB transaction context. That
means the amount of resources and reservations is guaranteed inside
this transaction (depending on the DB backend isolation level defined)
and the new reservation created will not clash with other DB transation.
That will guarantee the number of resources and instant reservations
never exceed the quota limits defined for this (resource, project_id).
NOTES:
- This change tries to be as unobtrusive as possible. The new driver
uses the same ``DbQuotaDriver`` dabatase tables (except for
``QuotaUsage``) and the same Quota engine API, located in
``neutron.quota``. However, the Quota engine resources implements some
particular API actions like "dirty", that are not used in the new
driver.
- The Pecan Quota enforcement hooks,
``neutron.pecan_wgsi.hooks.quota_enforcement``, execute actions like
"resync", "mark_resources_dirty" or "set_resources_dirty", that has
no meaning in the new driver.
- The isolation between the Quota engine and the Pecan hook, and the
driver itself is not clearly defined. A refactor of the Quota engine,
Quota service, Quota drivers and a common API between the driver and
the engine is needed.
- If ``DbQuotaDriver`` is deprecated, ``CountableResource`` and
``TrackedResource`` will be joined in a single class. This resource
class will have a count method (countable) or a hard dependency on a
database table (tracked resource). The only difference will be the
"count" method implementation.
Closes-Bug: #1926787
Change-Id: I4f98c6fcd781459fd7150aff426d19c7fdfa98c1
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.
This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line
inconsistency.
Fixed an inconsistency in the OVSBridge.deferred() definition
as it needs to also have an *args argument.
Fixed an issue where an l3-agent test was mocking
functools.partial, causing a python3.8 failure.
Unit tests only, removing from tests/base.py affects
functional tests which need additional work.
Change-Id: I40e8a8410840c3774c72ae1a8054574445d66ece
If a resource was registered in the quota engine before
being set as a tracked resource, it would have been registered
as a CountableResource instead of a TrackedResource.
This would cause a failure later when the resources were
being unregistered because the quota engine thought it
should have been a TrackedResource and tried to call
unregister_events on it.
This patch just prevents resources from being registered
as both types. It also callse unregister_all_resources as
part of the standard test case cleanup since the vast majority
of tests are using a plugin that may register resources.
Closes-Bug: #1612222
Change-Id: I8a40f38d7c0e5aeca257ba62115fa9b02ad5aa93
As previously implemented, the TestTrackedResource class is designed
to inject random failures into the gate. It generates random numbers
within the range of 0..10000, and will fail if it generates duplicate
random numbers during its run.
This patch creates UUIDs instead of random numbers, and makes the
chance of an collision vanishingly small.
Change-Id: I0cf535d1c5a3995a50b506aafce10e983872dcb7
Closes-bug: #1494021
This patch deals with the lock wait timeout and the deadlock errors
observed under high concurrency (api_workers >= 4) with the pymysql
driver. It includes the following changes:
- Stop setting dirty status for resource usage when creating
reservation, as usage of reserved resources is not tracked anymore;
- Add a variable, increasing delay when retrying make_reservation
upon a DBDeadlock error in order to reduce the chances of further
collisions;
- Enable transaction retry upon DBDeadlock errors for set_quota_usage;
- Do not resync quota usage while making reservation. This puts a lot
of stress on the database and is also wasteful since resource usage
is very likely to change again once the transaction is committed;
- Use autonested_transaction to simplify logic around when the
nested flag should be used.
Change-Id: I7a335f9ebea3c0d6fee6e6b757554e045a66075c
Closes-Bug: #1486134
Related-Blueprint: better-quotas
This patch introduces application logic support for tracking
Neutron resource usage data, thus introducing a different
way of enforcing quota limits, which now relies on records
tracking resource usage for each tenant.
When these records go "out-of-sync" with actual resource usage,
the quota usage table entry is marked "dirty".
And it will be resynchronized the next time a resource count is
requested. Changes in resource utilization are detected using
SQLAlchemy events.
This patch however does not automatically enable resource usage
tracking for any plugin. Plugins must explicitly declare for which
resources they wish to enable tracking. To this aim, this patch
provides the @tracked_resources decorator.
As some operators might wish to not use resource usage tracking, this
patch adds the track_quota_usage configuration option. If set to
False, DB-level usage tracking will not be performed, and
resources will be counted as before, ie: invoking a plugin method.
Usage tracking is performed using a new resource type,
TrackedResource, which can work side by side with CountableResource,
the one Neutron used so far. To this aim a ResourceRegistry class
has been introduced for registering and managing resources. Most
of the resource management code previously embedded in the
QuotaEngine class is being moved to ResourceRegistry as a part of
this patch.
Partially implements blueprint better-quota
Change-Id: If461399900973a38d7b82e0b3ac54fc052f09529
Slawek Kaplonski