Merge "Spec: Placement API support for instance reservation"

This commit is contained in:
Zuul 2018-07-18 07:38:56 +00:00 committed by Gerrit Code Review
commit 4f1090e73b
1 changed files with 408 additions and 0 deletions

View File

@ -0,0 +1,408 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
==============================================
Placement API support for instance reservation
==============================================
https://blueprints.launchpad.net/blazar/+spec/placement-api
Placement API [#placement_api]_ was introduced to Nova at the 14.0.0 Newton
release to separate API and data model for tracking resource provider
inventories and usages. It can be used for improving instance reservation.
Problem description
===================
Current instance reservation has the following constraints:
* A user has to create instance reservation with the anti-affinity policy.
Therefore, the amount of instances in one reservation cannot be larger than
the amount of hosts.
* A user has to specify the server group when the user launches instances on
reserved resources. If it is not specified, more instances than the reserved
amount are possibly launched.
Use Cases
---------
* A user wants to reserve instance resources with arbitrary affinity policy.
* A user wants to reserve more instances than the number of hosts.
Proposed change
===============
Use the `custom resource class`_ to represent reservation resources and use the
`nested resource provider`_ to manage capacity and usage of reservation
resources.
The following sections describe how Blazar interacts with Nova and Placement
for supporting instance reservation.
When creating a host:
---------------------
1. Get hypervisor information and store it into the computehosts table.
2. Create a `nested resource provider`_ as a child of the compute node
resource provider by calling the `Create resource provider API`_. The UUID
of the compute node resource provider can be retrieved by calling the `List
resource providers API`_ with the **name** option query,
e.g. ``GET /placement/resource_proiders?name=compute-1``.
The child resource provider is referred to as *'reservation provider'* in
the following sections.
Create reservation provider request body example:
``POST /placement/resource_providers``
.. sourcecode:: json
{
"name": "blazar_compute-1",
"parent_provider_uuid": "542df8ed-9be2-49b9-b4db-6d3183ff8ec8"
}
.. note::
"542df8ed-9be2-49b9-b4db-6d3183ff8ec8" is the UUID of the "compute-1"
compute node.
3. Add the host into the freepool.
When creating a lease:
----------------------
1. Look for available resources with arbitrary affinity policy.
2. Update the computehost_allocations table.
3. Create a custom resource class **CUSTOM_RESERVATION_{reservation UUID}** by
calling the `Create resource class API`_.
Create resource class request body example:
``POST /placement/resource_classes``
.. sourcecode:: json
{
"name": "CUSTOM_RESERVATION_4D17D41A_830D_47B2_91C7_4F9FC0AE611E"
}
.. note::
Use upper case and under score for the custom resource class name because
lower case and hyphen cannot be used.
4. Create a private flavor which has
``resources:CUSTOM_RESERVATION_{reservation UUID}=1`` in its `extra_spec`_.
.. note::
* A host aggregate is not created for each instance reservation anymore
because reserved hosts can be distinguished by the reservation provider
inventory.
* A server group is not created anymore because the proposed approach does
not depend on the ServerGroup(Anti)AffinityFilter.
When starting a lease:
----------------------
1. Add the custom resource class **CUSTOM_RESERVATION_{reservation UUID}** into
the reservation provider's inventory by calling the `Update resource
provider inventories API`_ with the **total** parameter which equals to the
amount of instances reserved for the host.
Update resource provider inventories request body example:
``PUT /placement/resource_providers/{reservation_provider_uuid}/inventories``
.. sourcecode:: json
{
"inventories": {
"CUSTOM_RESERVATION_4D17D41A_830D_47B2_91C7_4F9FC0AE611E": {
"total": 3,
"allocation_ratio": 1.0,
"min_unit": 1,
"max_unit": 1,
"step_size": 1
},
"snip"
},
"resource_provider_generation": 5
}
.. note::
Existing hosts which were created before this spec is implemented do not
have the reservation provider. So, check if the reservation provider exists
and create it if it does not exist before this step.
2. Add the lease owner's project to the private flavor access rights list.
.. note::
The previous implementation of starting lease should be kept until the
previous instance reservation is deprecated and completely removed. The
previous instance reservations can be distinguished by checking the
aggregate_id or server_group_id column in the instance_reservations table.
When launching instances (from user point of view):
---------------------------------------------------
1. A lease owner uses the private flavor and the instance is launched on the
reserved host which has the **CUSTOM_RESERVATION_{reservation UUID}** in
it's child resource provider inventory, i.e. reservation provider inventory.
Consumption of **CUSTOM_RESERVATION_{reservation UUID}** resources in the
reservation provider inventory is claimed by the Nova scheduler. It means
that usage of reserved resources is automatically tracked by the Placement.
.. note::
It still depends on the *BlazarFilter* though the *BlazarFilter* will be
ideally removed in the future. The *BlazarFilter* is changed to check if
``resources:CUSTOM_RESERVATION_*`` is in flavor extra specs to distinguish
the request from normal, i.e. non-reserved, instance creation requests.
`Traits`_ or other features would be able to be used for solving
*BlazarFilter* dependency. It would be addressed by another blueprint.
On the other hand, dependency on the following filters are solved. These
filters are not needed any more.
* AggregateInstanceExtraSpecsFilter
* AggregateMultiTenancyIsolationFilter
* ServerGroupAntiAffinityFilter
Note that above filters and existing logic in the BlazarFilter should be
kept to keep backward compatibility until the previous instance reservation
is deprecated and completely removed.
When terminating a lease:
-------------------------
1. Delete related instances and the private flavor.
2. Remove the **CUSTOM_RESERVATION_{reservation UUID}** class from the
reservation provider's inventory by calling the `Delete resource provider
inventory API`_.
3. Delete the **CUSTOM_RESERVATION_{reservation_UUID}** resource class by
calling the `Delete resource class API`_.
.. note::
The previous implementation of terminating lease should be kept until the
previous instance reservation is deprecated and completely removed. The
previous instance reservations can be distinguished by checking the
aggregate_id or server_group_id column in the instance_reservations table.
When deleting a host:
---------------------
1. Delete the reservation provider which is associated with the host by
calling the `Delete resource provider API`_.
2. Remove the host from the freepool.
3. Update the computehosts table.
.. _custom resource class: https://specs.openstack.org/openstack/nova-specs/specs/ocata/implemented/custom-resource-classes.html
.. _nested resource provider: https://specs.openstack.org/openstack/nova-specs/specs/ocata/approved/nested-resource-providers.html
.. _Create resource provider API: https://developer.openstack.org/api-ref/placement/#create-resource-provider
.. _List resource providers API: https://developer.openstack.org/api-ref/placement/#list-resource-providers
.. _Create resource class API: https://developer.openstack.org/api-ref/placement/#create-resource-class
.. _extra_spec: https://specs.openstack.org/openstack/nova-specs/specs/pike/implemented/custom-resource-classes-in-flavors.html
.. _Update resource provider inventories API: https://developer.openstack.org/api-ref/placement/#update-resource-provider-inventories
.. _Delete resource provider inventory API: https://developer.openstack.org/api-ref/placement/#delete-resource-provider-inventory
.. _Delete resource class API: https://developer.openstack.org/api-ref/placement/#delete-resource-class
.. _Traits: https://specs.openstack.org/openstack/nova-specs/specs/pike/implemented/resource-provider-traits.html
.. _Delete resource provider API: https://developer.openstack.org/api-ref/placement/#delete-resource-provider
Alternatives
------------
Dummy resources approach
^^^^^^^^^^^^^^^^^^^^^^^^
Update inventories of the general resources, e.g. VCPU, of compute nodes in the
freepool to be **zero** or **reserved**. And add dummy resources like
**CUSTOM_VCPU_{reservation UUID}** into the inventory. This approach
complicates resource usage tracking because real usage of each general resource
cannot be seen through the top level compute node inventory.
Traits approach
^^^^^^^^^^^^^^^
Use `Traits`_ to express reserved resources. The problem is that traits are
just traits and they cannot be used for managing capacity and usage of reserved
resources.
Data model impact
-----------------
The **affinity** column of the instance_reservations table is changed to allow
``NULL``. ``NULL`` means ``no affinity policy is applied`` while ``True`` means
``affinity is applied`` and ``False`` means ``anti-affinity is applied``.
.. _instance_reservations table:
The instance_reservations table:
.. sourcecode:: none
ALTER TABLE instance_reservations
ALTER COLUMN affinity NULL;
After the previous instance reservation is deprecated and completely removed,
drop the following columns in the instance_reservations table:
.. sourcecode:: none
ALTER TABLE instance_reservations
DROP COLUMN aggregate_id, server_group_id;
REST API impact
---------------
The **affinity** parameter of the `Create lease API`_ is changed to be an
optional parameter. If the **affinity** parameter is not given, no affinity
policy is applied.
.. _Create lease API: https://developer.openstack.org/api-ref/reservation/v1/index.html#create-lease
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
None
Other deployer impact
---------------------
* The Placement API has to be newer than or equal to Ver. 1.29.
* To upgrade from the previous version, run the DB upgrade script and the
instance_reservations table schema will be updated.
Developer impact
----------------
None
Implementation
==============
Assignee(s)
-----------
Primary assignee:
<tetsuro>
Other contributors:
<hiro-kobayashi>
Work Items
----------
Base:
* Update DB schema: update the instance_reservations table.
* Add placement library in blazar/utils/openstack module.
To support the host creation:
* Update the create_computehost() of the host plugin to call Placement APIs and
update related tables.
To support the host deletion:
* Update the delete_computehost() of the host plugin to delete Placement
related resources.
To support the lease creation:
* Update the query_available_hosts() to return how many instance can be
launched on each available host.
* Update the pickup_hosts() to support arbitrary affinity policy.
* Update the reserve_resource() and update_reservation() to support multiple
allocations which have the same pair of reservation_id and computehost_id.
* Update the _create_resources() of the instance plugin to create the
**CUSTOM_RESERVATION_{reservation UUID}** class and add it into the private
flavor extra specs.
To support starting the lease:
* Update the on_start() of the instance plugin to add the
**CUSTOM_RESERVATION_{reservation UUID}** into the reservation provider
inventory. The **total** parameter equals to the number of entries of the
computehost_allocations table which have the same reservation id and
computehost id.
To support launching reserved instances:
* Update the *BlazarFilter*.
To support termination of the lease:
* Update the on_end() of the instance plugin to remove the custom resource from
the reservation provider inventory and delete the class itself.
Others:
* Update the api module and the python-blazarclient to support arbitrary
affinity policies.
* Update the blazar-dashboard to support arbitrary affinity policies.
* Update documentation.
Dependencies
============
WIP: Check Placement API development status.
Testing
=======
* Add unit tests for new features of each method described in the work items
section.
* Add test scenarios of instance reservation with the affinity policy and no
affinity policy.
Documentation Impact
====================
* Parameter description of the Create Lease API reference will be updated.
* Instance reservation part of the Command-Line Interface Reference will be
updated.
* Release notes will be added.
References
==========
.. [#placement_api] https://docs.openstack.org/nova/latest/user/placement.html
History
=======
.. list-table:: Revisions
:header-rows: 1
* - Release Name
- Description
* - Rocky
- Introduced