Switch to using oslo_db.sqlalchemy.enginefacade instead, as this is
required for SQLAlchemy 2.x support.
Change-Id: I055fed2ef2f4953cdd7b215ccafaa27767e915ae
Instance reservations can be created with the affinity parameter set to
True (affinity), False (anti-affinity) or null (no affinity requested).
Change Icf26d1b7dffe64f82d7084dcebb5df1c7c9f106d updated the database
schema with Alembic but missed updating the SQLAlchemy model.
Change-Id: Ic754692722a784b1cb47ffeebce58136a528ad14
The current implementation of enforcement exceptions
during lease creation and update transitions the lease into an error
state, which is undesirable.This update introduces non-fatal handling
for the following enforcement exceptions:
- MaxLeaseDurationException
- ExternalServiceFilterException
Now, these exceptions will not cause the lease status to be
updated to an error state, allowing smoother lease management.
Test Plan:
PASS: Verify behavior for lease creation with non-fatal
enforcement exceptions
PASS: Verify behavior for lease update with non-fatal
enforcement exceptions
PASS: Confirm consistent lease status for non-fatal exceptions
Negative Testing:
PASS: Ensure that lease status remains unaffected for
non-fatal exceptions
PASS: Validate expected outcomes for various enforcement scenarios
Change-Id: I5d807d33e07d80f69f9734735c11e8d801bf5cde
Adds the new config option ``cafile``, which is passed into the
Session invocations for SSL verification.
Partial-Bug: #2045281
Change-Id: I2ec5bc7ac929534175d380d2e3e535a5e7abd962
In the event that a given physical node has reliability issues, it can
be difficult for users to "get around" the problem because we currently
deterministically pick the first node from the list of nodes available.
Adding randomness to this process helps spread the probability that a
given lease will nab a node that has issues (but has not yet been
identified as such.)
Co-Authored-By: Matt Crees <mattc@stackhpc.com>
Change-Id: I0f0ddb14ed5d21f0ccc5c9659c821d58cea5cbde
oslo.db 12.1.0 has changed the default value for the 'autocommit'
parameter of 'LegacyEngineFacade' from 'True' to 'False'. This is a
necessary step to ensure compatibility with SQLAlchemy 2.0. However, we
are currently relying on the autocommit behavior and need changes to
explicitly manage sessions. Until that happens, we need to override the
default.
Co-Authored-By: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Ic7c5703c1d1009f6c402766aeb20fa9869495470
Co-Authored-By: Jacob Colleran <jakecoll@uchicago.edu>
Co-Authored-By: Jason Anderson <jasonanderson@uchicago.edu>
Co-Authored-By: Pierre Riteau <pierre@stackhpc.com>
Change-Id: I0728f556829ba84e222c27bd8c407738b4be2f76
According to Openstack summit session [1],
stestr is maintained project to which all Openstack projects should migrate.
Let's switch to stestr as other projects have already moved to it.
[1] https://etherpad.openstack.org/p/YVR-python-pti
Change-Id: I9d22be39bec67fc17841367a0ef0ac439143af6c
When using eventlet and greenthreads, it can happen that the context
switches during the processing of an exception, which can clear the
exception context. The save_and_reraise_exception utility is designed
for this use-case.
Co-Authored-By: Pierre Riteau <pierre@stackhpc.com>
Change-Id: I7911eeef7fcaaee6c4dde5869d458da41684c37d
The url_for function was defaulting to the public endpoint if no
specific endpoint type was passed as argument.
Also switch from admin to internal keystone endpoint by default and
replace use of a hardcoded endpoint type by the corresponding
configuration option.
As explained in [1], a recent devstack change switched off the creation
of an admin endpoint for keystone. Now that we support configuring
endpoint types, switch to using the public interface for keystone when
deploying blazar with devstack.
[1] https://review.opendev.org/c/openstack/blazar/+/816627
Change-Id: If20c20f6cfb6aa23cb6e19020301bf59044aa79c
This allows users to query for resource properties of hosts,
allowing them to be used in creating leases more effectively. Two new
API endpoints are added for hosts, ``/properties`` and
``/properties/<property_name>``, which allow for listing available
properties, and updating a property respectively. Properties can be
listed with detail, showing possible values and visibility. Admins can
list public and private properties.
A new database table is added ``resource_properties``, which stores
property names and resource types. Resource specific property tables
(e.g. ``computehost_extra_capabilities``) entries store a foreign key to
``resource_properties``, rather than the capability name.
Implements blueprint resource-properties-discovery-api
Change-Id: Ib9f1140c44c5e4fbef6e019c48a842869368cb21
This commit makes endpoint types configurable and uses the internal
endpoint as default for Blazar, except admin for reaching keystone to
keep legacy behaviour for now.
Update devstack configuration to use public endpoints, since internal
endpoints are not created by devstack anymore (except for keystone).
Change-Id: I6414be91a1b629df680641050cfd415252b0a3c5
Closes-Bug: #1960335
If a lease update raises an exception that is recoverable (such as
specifying an invalid date range) the lease would be set to ERROR
status. Other cases with wrong input have similar effects.
This change allows for non fatal exceptions to be specified in
the lease status code. If one is raised, the lease is restored
to the original status.
Closes-Bug: #1786031
Change-Id: I73df4587d223dc4582cb15e0f4cb8de1a02ffee7
This change modifies the DB utility to fetch reservation allocations so
that a list of reservations is returned with host_ids as an attribute to
each reservation.
This allows the host plugin to include more information when listing
allocations, namely start and end dates. This can be used to see when
resources are available, and for the creation of a reservation calendar
(blueprint calendar-view).
Change-Id: I464898bdeda6b55a028c8a14a54eb137c92a83c1
The get_reservation_allocations_by_host_ids function builds an
allocation dictionary that looks like this:
allocations = {
'r1': ['host1'],
'r2': ['host1', 'host2']
}
The list returned by this function was only using the first host of each
reservation:
allocations[r['id']][0]
Loop over hosts to build the list of (reservation, lease, host) tuples.
Change-Id: I55b3cb4e736f6f747c8b3254c62fa40bac3f288f
Closes-Bug: #1958307
If two leases have compute hosts in common, and the second lease starts
exactly when the first lease ends, there is the possibility of a race.
The Blazar manager can first run the start_lease event of the second
lease. This event would fail since the end_lease event of the first
lease would still be UNDONE, and the compute hosts in common would still
be in the aggregate associated with the first lease, instead of being in
the freepool.
This patch changes event execution code so that events are executed
concurrently if possible, with the following constraints:
- events are executed strictly in order, i.e. events are started only
after all previous events have completed
- when events are at the same time, we first execute before_end_lease
events (unless there is a start_lease at the same time), then
end_lease events, followed by start_lease events, ensuring the bug
described above does not happen. Finally, we run any before_end_lease
which had a corresponding start_lease event at the same time.
It also has the side effect of providing better stack traces for event
execution failures, since we call wait() on all GreenThread objects.
Co-Authored-By: Jason Anderson <jasonanderson@uchicago.edu>
Change-Id: Ie2339db18e8baee379fbea082f1238ec44fca6b1
Closes-Bug: #1785841
The oslo context library has built-in mechanisms to deserialize a
context object from a set of headers; Blazar's built in extension of the
context class was ignoring several possibly-important pieces of
information, notably the Keystone domain name.
To fix, this removes much of the custom logic in the BlazarContext and
keeps only the two important bits:
1. A stack of contexts is maintained to allow for nested operations w/
different sets of credentials
2. The service_catalog is preserved. It's unclear if this is really
needed long-term, but some code still relies on it. Also unclear why
the oslo context doesn't include this when parsing headers.
Support for multiple domains is included as part of this changeset.
Before, it was assumed that all users (admins and project users) were
part of the default domain.
Closes-Bug: #1881162
Change-Id: I75fcd97cf7a53d17c909620fcf41a8b5a3699dfa
If an error occurs during lease create and an aggregate isn't created,
an exception will be raised in cleanup_resources when trying to delete
the missing aggregate. Handle this gracefully and continue.
Closes-Bug: #1938241
Change-Id: I866bc7c369fb390f8c9204e36c06fadea17f6dd5
The six library has been removed once in change
Id572aec8a6fda7676eb03d17c66fd33f9801f25d. However it was added back in
Ic106bee4fc3f5c401c4f8d8ecb1c4e735560bcc2.
This change removes six again.
Change-Id: Ib7300cfb4fd8d8c81cff780fbdb44d537e29680a
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
Implements filter-based architecture for usage enforcement.
One filter is initially included: MaxLeaseDurationFilter allows
operators to define the maximum duration of a lease in seconds. All
filters must be enabled and configured in the [enforcement] group of
blazar.conf.
When ending a lease, the allocations currently associated with the
reservations of the lease are sent to the enforcement layer for
processing. This works fine when deleting a lease, but when a lease
naturally ends, that is, when the current time is greater than the
lease's recorded end time, the allocation lookup returns 0 results
because we have a filter on the time of the lease/reservation by
default, which starts from datetime.now. This is really not what we want
to do when explicitly filtering on a lease/reservation, so this updates
the generic query function to not apply this filtering in this case.
It turns out, it was possible for usage enforcement's on_end hook to run
multiple times if the first reservation teardown failed. In this
circumstance, the end_lease event would be IN_PROGRESS but the
reservations would be in an error state. The code only checks if any
reservation is DELETED before trying to run on_end again.
To fix this, only try to run on_end if we detected that there was an
UNDONE end_lease event; that's the only time in which we'll actually be
responsible for running this logic.
This also updates the delete logic to be a bit more consistent with the
internal way we track states. It is not enough to check dates, because
it could be that Blazar never got around to actually starting the
leases; we should be checking events. This also makes the logic a bit
easier to understand.
Change-Id: Ic106bee4fc3f5c401c4f8d8ecb1c4e735560bcc2
Co-Authored-By: Jason Anderson <jasonanderson@uchicago.edu>
Co-Authored-By: Pierre Riteau <pierre@stackhpc.com>
Implements: blueprint flexible-reservation-usage-enforcement
As per the community goal of migrating the policy file format from JSON
to YAML [1], we need to do two things:
1. Change the default value of '[oslo_policy] policy_file' config option
from 'policy.json' to 'policy.yaml' with upgrade checks.
2. Deprecate the JSON formatted policy file on the project side via
warning in documentation and release notes.
[1] https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Change-Id: I9ca5459a6968d93dee1bf24b3e0012a4cc7020b5
Blazar is currently not re-using the rpc transport, this
will cause connections to leak. This patch introduces
a commonly shared rpc pattern for openstack services
that ensures that the transport is always re-used.
Change-Id: Ib49d7820b2ce4bc6cc9bbbd91c075631b7fb56bb
The `name` keyword argument for Operations.alter_column() has been
deprecated since version 0.8. It was finally removed in Alembic 1.5.0,
causing the following error during installation of Blazar:
TypeError: alter_column() got multiple values for keyword argument 'name'
Change-Id: I05541da4d87575f65d8e78f0fc54bd415328b9f2
Closes-Bug: #1912502
There are several DB functions that are not referenced anywhere but in
tests. It's possible that there was some intent to surface them as some
sort of reporting API, but that hasn't ever happened. Removing them as
dead code.
Change-Id: I2766db82ff18af9cd7eeefe237ca8ea329e11ccd
The auth_url parameter is constructed from several configuration values,
including os_auth_prefix which defaults to an empty string. This can
result in a Keystone auth_url containing a double slash, which appears
to cause issues in some deployments.
Change-Id: I9e78e5ef1c29c88d3dc16447a78a6e53c5a71567
Closes-Bug: #1856658
Zuul