openstack
/
blazar
Code Issues Proposed changes

Add resource properties discovery API 

This allows users to query for resource properties of hosts,
allowing them to be used in creating leases more effectively. Two new
API endpoints are added for hosts, ``/properties`` and
``/properties/<property_name>``, which allow for listing available
properties, and updating a property respectively. Properties can be
listed with detail, showing possible values and visibility. Admins can
list public and private properties.

A new database table is added ``resource_properties``, which stores
property names and resource types. Resource specific property tables
(e.g. ``computehost_extra_capabilities``) entries store a foreign key to
``resource_properties``, rather than the capability name.

Implements blueprint resource-properties-discovery-api

Change-Id: Ib9f1140c44c5e4fbef6e019c48a842869368cb21
This commit is contained in:
Mark Powers 2021-09-21 17:31:43 +00:00
parent 4a2bb67fbf
commit f5e6d24826
24 changed files with 894 additions and 85 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

88
api-ref/source/v1/hosts.inc
View File

@ -128,6 +128,7 @@ Request
.. rest_parameters:: parameters.yaml
- host_id: host_id_path
- private: property_private
Response
--------
@ -328,3 +329,90 @@ Response
.. literalinclude:: ../../../doc/api_samples/hosts/allocation-get-resp.json
:language: javascript
List Resource Properties
========================
.. rest_method:: GET v1/os-hosts/properties
Get all resource properties from host
**Response codes**
Normal response code: 200
Error response codes: Bad Request(400), Unauthorized(401), Forbidden(403),
Internal Server Error(500)
Request
-------
.. rest_parameters:: parameters.yaml
- detail: resource_property_detail
- all: resource_property_all
Response
--------
.. rest_parameters:: parameters.yaml
- resource_properties: resource_properties
- property: resource_properties_property
- private: resource_properties_private
- values: resource_properties_values
**Example of List Resource Properties Response**
.. literalinclude:: ../../../doc/api_samples/hosts/host-property-list.json
:language: javascript
**Example of List Resource Properties With Detail Response**
.. literalinclude:: ../../../doc/api_samples/hosts/host-property-list-detail.json
:language: javascript
Update Resource Properties
==========================
.. rest_method:: PATCH v1/os-hosts/properties/{property_name}
Update a host resource properties
**Response codes**
Normal response code: 200
Error response codes: Bad Request(400), Unauthorized(401), Forbidden(403),
Internal Server Error(500)
Request
-------
.. rest_parameters:: parameters.yaml
- property_name: property_name
- private: property_private
**Example of Update Resource Properties**
.. literalinclude:: ../../../doc/api_samples/hosts/host-property-update.json
:language: javascript
Response
--------
.. rest_parameters:: parameters.yaml
- created_at: created_at
- updated_at: updated_at
- id: resource_property_id
- resource_type: resource_property_resource_type
- property_name: resource_properties_property
- private: resource_property_private
**Example of List Resource Properties Response**
.. literalinclude:: ../../../doc/api_samples/hosts/host-property-update-res.json
:language: javascript

89
api-ref/source/v1/parameters.yaml
View File

@ -42,6 +42,12 @@ lease_id_path:
in: path
required: true
type: string
property_name:
description: |
The name of the property.
in: path
required: true
type: string
# variables in query
@ -57,6 +63,19 @@ allocation_reservation_id_query:
in: query
required: false
type: string
resource_property_all:
description: |
Whether to include all resource properties, public and private.
in: query
required: false
type: string
resource_property_detail:
description: |
Whether to include values along for each property and if the property
is private.
in: query
required: false
type: string
# variables in body
@ -406,6 +425,13 @@ leases:
in: body
required: true
type: array
property_private:
description: |
Whether the property is private.
in: body
required: true
type: boolean
reservation:
description: |
A ``reservation`` object.
@ -627,6 +653,69 @@ reservations_optional:
in: body
required: false
type: array
resource_properties:
description: |
A list of ``resource_property`` objects.
in: body
required: true
type: array
resource_properties_private:
description: |
Whether the property is private.
in: body
required: false
type: boolean
resource_properties_property:
description: |
The name of the property.
in: body
required: true
type: any
resource_properties_values:
description: |
A list of values for the property.
in: body
required: false
type: array
resource_property:
description: |
The updated ``resource_property`` object.
in: body
required: true
type: any
resource_property_id:
description: |
The updated ``resource_property`` UUID.
in: body
required: true
type: string
resource_property_private:
description: |
Whether the updated ``resource_property`` is private.
in: body
required: true
type: boolean
resource_property_property_name:
description: |
The updated ``resource_property`` property_name.
in: body
required: true
type: string
resource_property_resource_type:
description: |
The updated ``resource_property`` resource type.
in: body
required: true
type: string
updated_at:
description: |
The date and time when the object was updated.

11
blazar/api/v1/oshosts/service.py
View File

@ -86,3 +86,14 @@ class API(object):
:type query: dict
"""
return self.manager_rpcapi.get_allocations(host_id, query)
@policy.authorize('oshosts', 'get_resource_properties')
def list_resource_properties(self, query):
"""List resource properties for hosts."""
return self.manager_rpcapi.list_resource_properties(query)
@policy.authorize('oshosts', 'update_resource_properties')
def update_resource_property(self, property_name, data):
"""Update a host resource property."""
return self.manager_rpcapi.update_resource_property(
property_name, data)

14
blazar/api/v1/oshosts/v1_0.py
View File

@ -79,3 +79,17 @@ def allocations_list(req, query):
def allocations_get(req, host_id, query):
"""List all allocations on a specific host."""
return api_utils.render(allocation=_api.get_allocations(host_id, query))
@rest.get('/properties', query=True)
def resource_properties_list(req, query=None):
"""List computehost resource properties."""
return api_utils.render(
resource_properties=_api.list_resource_properties(query))
@rest.patch('/properties/<property_name>')
def resource_property_update(req, property_name, data):
"""Update a computehost resource property."""
return api_utils.render(
resource_property=_api.update_resource_property(property_name, data))

5
blazar/api/v1/utils.py
View File

@ -53,6 +53,9 @@ class Rest(flask.Blueprint):
def put(self, rule, status_code=200):
return self._mroute('PUT', rule, status_code)
def patch(self, rule, status_code=200):
return self._mroute('PATCH', rule, status_code)
def delete(self, rule, status_code=204):
return self._mroute('DELETE', rule, status_code)
@ -79,7 +82,7 @@ class Rest(flask.Blueprint):
if status:
flask.request.status_code = status
if flask.request.method in ['POST', 'PUT']:
if flask.request.method in ['POST', 'PUT', 'PATCH']:
kwargs['data'] = request_data()
if flask.request.endpoint in self.routes_with_query_support:

17
blazar/db/api.py
View File

@ -385,13 +385,11 @@ def host_extra_capability_create(values):
return IMPL.host_extra_capability_create(values)
@to_dict
def host_extra_capability_get(host_extra_capability_id):
"""Return a specific Host Extracapability."""
return IMPL.host_extra_capability_get(host_extra_capability_id)
@to_dict
def host_extra_capability_get_all_per_host(host_id):
"""Return all extra_capabilities belonging to a specific Compute host."""
return IMPL.host_extra_capability_get_all_per_host(host_id)
@ -410,7 +408,6 @@ def host_extra_capability_update(host_extra_capability_id, values):
def host_extra_capability_get_all_per_name(host_id,
extra_capability_name):
return IMPL.host_extra_capability_get_all_per_name(host_id,
extra_capability_name)
@ -525,3 +522,17 @@ def reservable_fip_get_all_by_queries(queries):
def floatingip_destroy(floatingip_id):
"""Delete specific floating ip."""
IMPL.floatingip_destroy(floatingip_id)
# Resource Properties
def resource_properties_list(resource_type):
return IMPL.resource_properties_list(resource_type)
def resource_property_update(resource_type, property_name, values):
return IMPL.resource_property_update(resource_type, property_name, values)
def resource_property_create(values):
return IMPL.resource_property_create(values)

9
blazar/db/exceptions.py
View File

@ -40,3 +40,12 @@ class BlazarDBInvalidFilter(BlazarDBException):
class BlazarDBInvalidFilterOperator(BlazarDBException):
msg_fmt = _('%(filter_operator)s is invalid')
class BlazarDBResourcePropertiesNotEnabled(BlazarDBException):
msq_fmt = _('%(resource_type)s does not have resource properties enabled.')
class BlazarDBInvalidResourceProperty(BlazarDBException):
msg_fmt = _('%(property_name)s does not exist for resource type '
'%(resource_type)s.')

101
blazar/db/migration/alembic_migrations/versions/02e2f2186d98_resource_property.py Normal file
View File

@ -0,0 +1,101 @@
# Copyright 2022 OpenStack Foundation.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""resource property
Revision ID: 02e2f2186d98
Revises: f4084140f608
Create Date: 2020-04-17 15:51:40.542459
"""
# revision identifiers, used by Alembic.
revision = '02e2f2186d98'
down_revision = 'f4084140f608'
import uuid
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import mysql
def upgrade():
op.create_table('resource_properties',
sa.Column('id', sa.String(length=36), nullable=False),
sa.Column('resource_type', sa.String(255), nullable=False),
sa.Column('property_name', sa.String(255),
nullable=False),
sa.Column('private', sa.Boolean, nullable=False,
server_default=sa.false()),
sa.Column('created_at', sa.DateTime(), nullable=True),
sa.Column('updated_at', sa.DateTime(), nullable=True),
sa.PrimaryKeyConstraint('id'),
sa.UniqueConstraint('resource_type', 'property_name'))
if op.get_bind().engine.name != 'sqlite':
connection = op.get_bind()
host_query = connection.execute("""
SELECT DISTINCT "physical:host", capability_name
FROM computehost_extra_capabilities;""")
capability_values = [
(str(uuid.uuid4()), resource_type, capability_name)
for resource_type, capability_name
in host_query.fetchall()]
if capability_values:
insert = """
INSERT INTO resource_properties
(id, resource_type, property_name)
VALUES {};"""
connection.execute(
insert.format(', '.join(map(str, capability_values))))
op.add_column('computehost_extra_capabilities',
sa.Column('property_id', sa.String(length=255),
nullable=False))
connection.execute("""
UPDATE computehost_extra_capabilities c
LEFT JOIN resource_properties e
ON e.property_name = c.capability_name
SET c.property_id = e.id;""")
op.create_foreign_key('computehost_resource_property_id_fk',
'computehost_extra_capabilities',
'resource_properties', ['property_id'], ['id'])
op.drop_column('computehost_extra_capabilities', 'capability_name')
def downgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.add_column('computehost_extra_capabilities',
sa.Column('capability_name', mysql.VARCHAR(length=64),
nullable=False))
if op.get_bind().engine.name != 'sqlite':
connection = op.get_bind()
connection.execute("""
UPDATE computehost_extra_capabilities c
LEFT JOIN resource_properties e
ON e.id=c.property_id
SET c.capability_name = e.property_name;""")
op.drop_constraint('computehost_resource_property_id_fk',
'computehost_extra_capabilities',
type_='foreignkey')
op.drop_column('computehost_extra_capabilities', 'property_id')
op.drop_table('resource_properties')

160
blazar/db/sqlalchemy/api.py
View File

@ -29,6 +29,9 @@ from blazar.db import exceptions as db_exc
from blazar.db.sqlalchemy import facade_wrapper
from blazar.db.sqlalchemy import models
RESOURCE_PROPERTY_MODELS = {
'physical:host': models.ComputeHostExtraCapability,
}
LOG = logging.getLogger(__name__)
@ -663,26 +666,28 @@ def host_get_all_by_queries(queries):
hosts_query = hosts_query.filter(filt)
else:
# looking for extra capabilities matches
extra_filter = model_query(
models.ComputeHostExtraCapability, get_session()
).filter(models.ComputeHostExtraCapability.capability_name == key
).all()
# looking for resource properties matches
extra_filter = (
_host_resource_property_query(get_session())
.filter(models.ResourceProperty.property_name == key)
).all()
if not extra_filter:
raise db_exc.BlazarDBNotFound(
id=key, model='ComputeHostExtraCapability')
for host in extra_filter:
for host, property_name in extra_filter:
print(dir(host))
if op in oper and oper[op][1](host.capability_value, value):
hosts.append(host.computehost_id)
elif op not in oper:
msg = 'Operator %s for extra capabilities not implemented'
msg = 'Operator %s for resource properties not implemented'
raise NotImplementedError(msg % op)
# We must also avoid selecting any host which doesn't have the
# extra capability present.
all_hosts = [h.id for h in hosts_query.all()]
extra_filter_hosts = [h.computehost_id for h in extra_filter]
extra_filter_hosts = [h.computehost_id for h, _ in extra_filter]
hosts += [h for h in all_hosts if h not in extra_filter_hosts]
return hosts_query.filter(~models.ComputeHost.id.in_(hosts)).all()
@ -755,9 +760,19 @@ def host_destroy(host_id):
# ComputeHostExtraCapability
def _host_resource_property_query(session):
return (
model_query(models.ComputeHostExtraCapability, session)
.join(models.ResourceProperty)
.add_column(models.ResourceProperty.property_name))
def _host_extra_capability_get(session, host_extra_capability_id):
query = model_query(models.ComputeHostExtraCapability, session)
return query.filter_by(id=host_extra_capability_id).first()
query = _host_resource_property_query(session).filter(
models.ComputeHostExtraCapability.id == host_extra_capability_id)
return query.first()
def host_extra_capability_get(host_extra_capability_id):
@ -766,8 +781,10 @@ def host_extra_capability_get(host_extra_capability_id):
def _host_extra_capability_get_all_per_host(session, host_id):
query = model_query(models.ComputeHostExtraCapability, session)
return query.filter_by(computehost_id=host_id)
query = _host_resource_property_query(session).filter(
models.ComputeHostExtraCapability.computehost_id == host_id)
return query
def host_extra_capability_get_all_per_host(host_id):
@ -777,6 +794,13 @@ def host_extra_capability_get_all_per_host(host_id):
def host_extra_capability_create(values):
values = values.copy()
resource_property = resource_property_get_or_create(
'physical:host', values.get('property_name'))
del values['property_name']
values['property_id'] = resource_property.id
host_extra_capability = models.ComputeHostExtraCapability()
host_extra_capability.update(values)
@ -797,7 +821,7 @@ def host_extra_capability_update(host_extra_capability_id, values):
session = get_session()
with session.begin():
host_extra_capability = (
host_extra_capability, _ = (
_host_extra_capability_get(session,
host_extra_capability_id))
host_extra_capability.update(values)
@ -809,9 +833,8 @@ def host_extra_capability_update(host_extra_capability_id, values):
def host_extra_capability_destroy(host_extra_capability_id):
session = get_session()
with session.begin():
host_extra_capability = (
_host_extra_capability_get(session,
host_extra_capability_id))
host_extra_capability = _host_extra_capability_get(
session, host_extra_capability_id)
if not host_extra_capability:
# raise not found error
@ -819,15 +842,16 @@ def host_extra_capability_destroy(host_extra_capability_id):
id=host_extra_capability_id,
model='ComputeHostExtraCapability')
session.delete(host_extra_capability)
session.delete(host_extra_capability[0])
def host_extra_capability_get_all_per_name(host_id, capability_name):
def host_extra_capability_get_all_per_name(host_id, property_name):
session = get_session()
with session.begin():
query = _host_extra_capability_get_all_per_host(session, host_id)
return query.filter_by(capability_name=capability_name).all()
return query.filter(
models.ResourceProperty.property_name == property_name).all()
# FloatingIP reservation
@ -1115,3 +1139,101 @@ def floatingip_destroy(floatingip_id):
raise db_exc.BlazarDBNotFound(id=floatingip_id, model='FloatingIP')
session.delete(floatingip)
# Resource Properties
def _resource_property_get(session, resource_type, property_name):
query = (
model_query(models.ResourceProperty, session)
.filter_by(resource_type=resource_type)
.filter_by(property_name=property_name))
return query.first()
def resource_property_get(resource_type, property_name):
return _resource_property_get(get_session(), resource_type, property_name)
def resource_properties_list(resource_type):
if resource_type not in RESOURCE_PROPERTY_MODELS:
raise db_exc.BlazarDBResourcePropertiesNotEnabled(
resource_type=resource_type)
session = get_session()
with session.begin():
resource_model = RESOURCE_PROPERTY_MODELS[resource_type]
query = session.query(
models.ResourceProperty.property_name,
models.ResourceProperty.private,
resource_model.capability_value).join(resource_model).distinct()
return query.all()
def _resource_property_create(session, values):
values = values.copy()
resource_property = models.ResourceProperty()
resource_property.update(values)
with session.begin():
try:
resource_property.save(session=session)
except common_db_exc.DBDuplicateEntry as e:
# raise exception about duplicated columns (e.columns)
raise db_exc.BlazarDBDuplicateEntry(
model=resource_property.__class__.__name__,
columns=e.columns)
return resource_property_get(values.get('resource_type'),
values.get('property_name'))
def resource_property_create(values):
return _resource_property_create(get_session(), values)
def resource_property_update(resource_type, property_name, values):
if resource_type not in RESOURCE_PROPERTY_MODELS:
raise db_exc.BlazarDBResourcePropertiesNotEnabled(
resource_type=resource_type)
values = values.copy()
session = get_session()
with session.begin():
resource_property = _resource_property_get(
session, resource_type, property_name)
if not resource_property:
raise db_exc.BlazarDBInvalidResourceProperty(
property_name=property_name,
resource_type=resource_type)
resource_property.update(values)
resource_property.save(session=session)
return resource_property_get(resource_type, property_name)
def _resource_property_get_or_create(session, resource_type, property_name):
resource_property = _resource_property_get(
session, resource_type, property_name)
if resource_property:
return resource_property
else:
rp_values = {
'resource_type': resource_type,
'property_name': property_name}
return resource_property_create(rp_values)
def resource_property_get_or_create(resource_type, property_name):
return _resource_property_get_or_create(
get_session(), resource_type, property_name)

21
blazar/db/sqlalchemy/models.py
View File

@ -155,6 +155,23 @@ class Event(mb.BlazarBase):
return super(Event, self).to_dict()
class ResourceProperty(mb.BlazarBase):
"""Defines an resource property by resource type."""
__tablename__ = 'resource_properties'
id = _id_column()
resource_type = sa.Column(sa.String(255), nullable=False)
property_name = sa.Column(sa.String(255), nullable=False)
private = sa.Column(sa.Boolean, nullable=False,
server_default=sa.false())
__table_args__ = (sa.UniqueConstraint('resource_type', 'property_name'),)
def to_dict(self):
return super(ResourceProperty, self).to_dict()
class ComputeHostReservation(mb.BlazarBase):
"""Description
@ -252,7 +269,9 @@ class ComputeHostExtraCapability(mb.BlazarBase):
id = _id_column()
computehost_id = sa.Column(sa.String(36), sa.ForeignKey('computehosts.id'))
capability_name = sa.Column(sa.String(64), nullable=False)
property_id = sa.Column(sa.String(36),
sa.ForeignKey('resource_properties.id'),
nullable=False)
capability_value = sa.Column(MediumText(), nullable=False)
def to_dict(self):

9
blazar/manager/oshosts/rpcapi.py
View File

@ -64,3 +64,12 @@ class ManagerRPCAPI(service.RPCClient):
"""List all allocations on a specified computehost."""
return self.call('physical:host:get_allocations',
host_id=host_id, query=query)
def list_resource_properties(self, query):
"""List resource properties and possible values for computehosts."""
return self.call('physical:host:list_resource_properties', query=query)
def update_resource_property(self, property_name, values):
"""Update resource property for computehost."""
return self.call('physical:host:update_resource_property',
property_name=property_name, values=values)

29
blazar/plugins/base.py
View File

@ -14,7 +14,11 @@
# limitations under the License.
import abc
import collections
from blazar import context
from blazar.db import api as db_api
from blazar import policy
from oslo_config import cfg
from oslo_log import log as logging
@ -98,6 +102,31 @@ class BasePlugin(object, metaclass=abc.ABCMeta):
"""Wake up resource."""
pass
def list_resource_properties(self, query):
detail = False if not query else query.get('detail', False)
all_properties = False if not query else query.get('all', False)
resource_properties = collections.defaultdict(list)
include_private = all_properties and policy.enforce(
context.current(), 'admin', {}, do_raise=False)
for name, private, value in db_api.resource_properties_list(
self.resource_type):
if include_private or not private:
resource_properties[name].append(value)
if detail:
return [
dict(property=k, private=False, values=v)
for k, v in resource_properties.items()]
else:
return [dict(property=k) for k, v in resource_properties.items()]
def update_resource_property(self, property_name, values):
return db_api.resource_property_update(
self.resource_type, property_name, values)
def before_end(self, resource_id):
"""Take actions before the end of a lease"""
pass

6
blazar/plugins/instances/instance_plugin.py
View File

@ -809,9 +809,9 @@ class VirtualInstancePlugin(base.BasePlugin, nova.NovaClientWrapper):
extra_capabilities = {}
raw_extra_capabilities = (
db_api.host_extra_capability_get_all_per_host(host_id))
for capability in raw_extra_capabilities:
key = capability['capability_name']
extra_capabilities[key] = capability['capability_value']
for capability, capability_name in raw_extra_capabilities:
key = capability_name
extra_capabilities[key] = capability.capability_value
return extra_capabilities
def get(self, host_id):

36
blazar/plugins/oshosts/host_plugin.py
View File

@ -302,9 +302,9 @@ class PhysicalHostPlugin(base.BasePlugin, nova.NovaClientWrapper):
extra_capabilities = {}
raw_extra_capabilities = (
db_api.host_extra_capability_get_all_per_host(host_id))
for capability in raw_extra_capabilities:
key = capability['capability_name']
extra_capabilities[key] = capability['capability_value']
for capability, property_name in raw_extra_capabilities:
key = property_name
extra_capabilities[key] = capability.capability_value
return extra_capabilities
def get(self, host_id):
@ -383,7 +383,7 @@ class PhysicalHostPlugin(base.BasePlugin, nova.NovaClientWrapper):
raise e
for key in extra_capabilities:
values = {'computehost_id': host['id'],
'capability_name': key,
'property_name': key,
'capability_value': extra_capabilities[key],
}
try:
@ -396,7 +396,7 @@ class PhysicalHostPlugin(base.BasePlugin, nova.NovaClientWrapper):
host=host['id'])
return self.get_computehost(host['id'])
def is_updatable_extra_capability(self, capability):
def is_updatable_extra_capability(self, capability, property_name):
reservations = db_utils.get_reservations_by_host_id(
capability['computehost_id'], datetime.datetime.utcnow(),
datetime.date.max)
@ -413,7 +413,7 @@ class PhysicalHostPlugin(base.BasePlugin, nova.NovaClientWrapper):
# the extra_capability.
for requirement in requirements_queries:
# A requirement is of the form "key op value" as string
if requirement.split(" ")[0] == capability['capability_name']:
if requirement.split(" ")[0] == property_name:
return False
return True
@ -428,37 +428,33 @@ class PhysicalHostPlugin(base.BasePlugin, nova.NovaClientWrapper):
new_keys = set(values.keys()) - set(previous_capabilities.keys())
for key in updated_keys:
raw_capability = next(iter(
raw_capability, property_name = next(iter(
db_api.host_extra_capability_get_all_per_name(host_id, key)))
capability = {
'capability_name': key,
'capability_value': values[key],
}
if self.is_updatable_extra_capability(raw_capability):
capability = {'capability_value': values[key]}
if self.is_updatable_extra_capability(
raw_capability, property_name):
try:
db_api.host_extra_capability_update(
raw_capability['id'], capability)
except (db_ex.BlazarDBException, RuntimeError):
cant_update_extra_capability.append(
raw_capability['capability_name'])
cant_update_extra_capability.append(property_name)
else:
LOG.info("Capability %s can't be updated because "
"existing reservations require it.",
raw_capability['capability_name'])
cant_update_extra_capability.append(
raw_capability['capability_name'])
property_name)
cant_update_extra_capability.append(property_name)
for key in new_keys:
new_capability = {
'computehost_id': host_id,
'capability_name': key,
'property_name': key,
'capability_value': values[key],
}
try:
db_api.host_extra_capability_create(new_capability)
except (db_ex.BlazarDBException, RuntimeError):
cant_update_extra_capability.append(
new_capability['capability_name'])
cant_update_extra_capability.append(key)
if cant_update_extra_capability:
raise manager_ex.CantAddExtraCapability(

25
blazar/policies/oshosts.py
View File

@ -79,7 +79,30 @@ oshosts_policies = [
'method': 'GET'
}
]
)
),
policy.DocumentedRuleDefault(
name=POLICY_ROOT % 'get_resource_properties',
check_str=base.RULE_ADMIN,
description='Policy rule for Resource Properties API.',
operations=[
{
'path': '/{api_version}/os-hosts/resource_properties',
'method': 'GET'
}
]
),
policy.DocumentedRuleDefault(
name=POLICY_ROOT % 'update_resource_properties',
check_str=base.RULE_ADMIN,
description='Policy rule for Resource Properties API.',
operations=[
{
'path': ('/{api_version}/os-hosts/resource_properties/'
'{property_name}'),
'method': 'PATCH'
}
]
),
]

21
blazar/tests/api/v1/oshosts/test_v1_0.py
View File

@ -100,6 +100,10 @@ class OsHostAPITestCase(tests.TestCase):
self.list_allocations = self.patch(service_api.API,
'list_allocations')
self.get_allocations = self.patch(service_api.API, 'get_allocations')
self.list_resource_properties = self.patch(service_api.API,
'list_resource_properties')
self.update_resource_property = self.patch(service_api.API,
'update_resource_property')
def _assert_response(self, actual_resp, expected_status_code,
expected_resp_body, key='host',
@ -237,3 +241,20 @@ class OsHostAPITestCase(tests.TestCase):
res = c.get('/v1/{0}/allocation?{1}'.format(
self.host_id, query_params), headers=self.headers)
self._assert_response(res, 200, {}, key='allocation')
def test_resource_properties_list(self):
with self.app.test_client() as c:
self.list_resource_properties.return_value = []
res = c.get('/v1/properties', headers=self.headers)
self._assert_response(res, 200, [], key='resource_properties')
def test_resource_property_update(self):
resource_property = 'fake_property'
resource_property_body = {'private': True}
with self.app.test_client() as c:
res = c.patch('/v1/properties/{0}'.format(resource_property),
json=resource_property_body,
headers=self.headers)
self._assert_response(res, 200, {}, 'resource_property')

40
blazar/tests/db/sqlalchemy/test_sqlalchemy_api.py
View File

@ -193,7 +193,7 @@ def _get_fake_host_extra_capabilities(id=None,
computehost_id = _get_fake_random_uuid()
return {'id': id,
'computehost_id': computehost_id,
'capability_name': name,
'property_name': name,
'capability_value': value}
@ -507,6 +507,12 @@ class SQLAlchemyDBApiTestCase(tests.DBTestCase):
"""Create one host and test extra capability queries."""
# We create a first host, with extra capabilities
db_api.host_create(_get_fake_host_values(id=1))
db_api.resource_property_create(dict(
id='a', resource_type='physical:host', private=False,
property_name='vgpu'))
db_api.resource_property_create(dict(
id='b', resource_type='physical:host', private=False,
property_name='nic_model'))
db_api.host_extra_capability_create(
_get_fake_host_extra_capabilities(computehost_id=1))
db_api.host_extra_capability_create(_get_fake_host_extra_capabilities(
@ -533,6 +539,20 @@ class SQLAlchemyDBApiTestCase(tests.DBTestCase):
db_api.host_get_all_by_queries(['nic_model == ACME Model A'])
))
def test_resource_properties_list(self):
"""Create one host and test extra capability queries."""
# We create a first host, with extra capabilities
db_api.host_create(_get_fake_host_values(id=1))
db_api.resource_property_create(dict(
id='a', resource_type='physical:host', private=False,
property_name='vgpu'))
db_api.host_extra_capability_create(
_get_fake_host_extra_capabilities(computehost_id=1))
result = db_api.resource_properties_list('physical:host')
self.assertListEqual(result, [('vgpu', False, '2')])
def test_search_for_hosts_by_composed_queries(self):
"""Create one host and test composed queries."""
@ -580,9 +600,13 @@ class SQLAlchemyDBApiTestCase(tests.DBTestCase):
db_api.host_destroy, 2)
def test_create_host_extra_capability(self):
result = db_api.host_extra_capability_create(
_get_fake_host_extra_capabilities(id=1))
self.assertEqual(result['id'], _get_fake_host_values(id='1')['id'])
db_api.resource_property_create(dict(
id='id', resource_type='physical:host', private=False,
property_name='vgpu'))
result, _ = db_api.host_extra_capability_create(
_get_fake_host_extra_capabilities(id=1, name='vgpu'))
self.assertEqual(result.id, _get_fake_host_values(id='1')['id'])
def test_create_duplicated_host_extra_capability(self):
db_api.host_extra_capability_create(
@ -594,8 +618,8 @@ class SQLAlchemyDBApiTestCase(tests.DBTestCase):
def test_get_host_extra_capability_per_id(self):
db_api.host_extra_capability_create(
_get_fake_host_extra_capabilities(id='1'))
result = db_api.host_extra_capability_get('1')
self.assertEqual('1', result['id'])
result, _ = db_api.host_extra_capability_get('1')
self.assertEqual('1', result.id)
def test_host_extra_capability_get_all_per_host(self):
db_api.host_extra_capability_create(
@ -609,8 +633,8 @@ class SQLAlchemyDBApiTestCase(tests.DBTestCase):
db_api.host_extra_capability_create(
_get_fake_host_extra_capabilities(id='1'))
db_api.host_extra_capability_update('1', {'capability_value': '2'})
res = db_api.host_extra_capability_get('1')
self.assertEqual('2', res['capability_value'])
res, _ = db_api.host_extra_capability_get('1')
self.assertEqual('2', res.capability_value)
def test_delete_host_extra_capability(self):
db_api.host_extra_capability_create(

119
blazar/tests/plugins/oshosts/test_physical_host_plugin.py
View File

@ -13,6 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import collections
import datetime
from unittest import mock
@ -81,18 +82,13 @@ class PhysicalHostPluginSetupOnlyTestCase(tests.TestCase):
self.fake_phys_plugin.project_domain_name)
def test__get_extra_capabilities_with_values(self):
ComputeHostExtraCapability = collections.namedtuple(
'ComputeHostExtraCapability',
['id', 'property_id', 'capability_value', 'computehost_id'])
self.db_host_extra_capability_get_all_per_host.return_value = [
{'id': 1,
'capability_name': 'foo',
'capability_value': 'bar',
'other': 'value',
'computehost_id': 1
},
{'id': 2,
'capability_name': 'buzz',
'capability_value': 'word',
'computehost_id': 1
}]
(ComputeHostExtraCapability(1, 'foo', 'bar', 1), 'foo'),
(ComputeHostExtraCapability(2, 'buzz', 'word', 1), 'buzz')]
res = self.fake_phys_plugin._get_extra_capabilities(1)
self.assertEqual({'foo': 'bar', 'buzz': 'word'}, res)
@ -229,7 +225,7 @@ class PhysicalHostPluginTestCase(tests.TestCase):
# NOTE(sbauza): 'id' will be pop'd, we need to keep track of it
fake_request = fake_host.copy()
fake_capa = {'computehost_id': '1',
'capability_name': 'foo',
'property_name': 'foo',
'capability_value': 'bar',
}
self.get_extra_capabilities.return_value = {'foo': 'bar'}
@ -296,11 +292,10 @@ class PhysicalHostPluginTestCase(tests.TestCase):
host_values = {'foo': 'baz'}
self.db_host_extra_capability_get_all_per_name.return_value = [
{'id': 'extra_id1',
'computehost_id': self.fake_host_id,
'capability_name': 'foo',
'capability_value': 'bar'
},
({'id': 'extra_id1',
'computehost_id': self.fake_host_id,
'capability_value': 'bar'},
'foo'),
]
self.get_reservations_by_host = self.patch(
@ -310,7 +305,7 @@ class PhysicalHostPluginTestCase(tests.TestCase):
self.fake_phys_plugin.update_computehost(self.fake_host_id,
host_values)
self.db_host_extra_capability_update.assert_called_once_with(
'extra_id1', {'capability_name': 'foo', 'capability_value': 'baz'})
'extra_id1', {'capability_value': 'baz'})
def test_update_host_having_issue_when_storing_extra_capability(self):
def fake_db_host_extra_capability_update(*args, **kwargs):
@ -320,11 +315,10 @@ class PhysicalHostPluginTestCase(tests.TestCase):
self.db_utils, 'get_reservations_by_host_id')
self.get_reservations_by_host.return_value = []
self.db_host_extra_capability_get_all_per_name.return_value = [
{'id': 'extra_id1',
'computehost_id': self.fake_host_id,
'capability_name': 'foo',
'capability_value': 'bar'
},
({'id': 'extra_id1',
'computehost_id': self.fake_host_id,
'capability_value': 'bar'},
'foo'),
]
fake = self.db_host_extra_capability_update
fake.side_effect = fake_db_host_extra_capability_update
@ -340,7 +334,7 @@ class PhysicalHostPluginTestCase(tests.TestCase):
host_values)
self.db_host_extra_capability_create.assert_called_once_with({
'computehost_id': '1',
'capability_name': 'qux',
'property_name': 'qux',
'capability_value': 'word'
})
@ -348,11 +342,10 @@ class PhysicalHostPluginTestCase(tests.TestCase):
host_values = {'foo': 'buzz'}
self.db_host_extra_capability_get_all_per_name.return_value = [
{'id': 'extra_id1',
'computehost_id': self.fake_host_id,
'capability_name': 'foo',
'capability_value': 'bar'
},
({'id': 'extra_id1',
'computehost_id': self.fake_host_id,
'capability_value': 'bar'},
'foo'),
]
fake_phys_reservation = {
'resource_type': plugin.RESOURCE_TYPE,
@ -2388,6 +2381,74 @@ class PhysicalHostPluginTestCase(tests.TestCase):
self.fake_phys_plugin._check_params(values)
self.assertEqual(values['before_end'], 'default')
def test_list_resource_properties(self):
self.db_list_resource_properties = self.patch(
self.db_api, 'resource_properties_list')
# Expecting a list of (Reservation, Allocation)
self.db_list_resource_properties.return_value = [
('prop1', False, 'aaa'),
('prop1', False, 'bbb'),
('prop2', False, 'aaa'),
('prop2', False, 'aaa'),
('prop3', True, 'aaa')
]
expected = [
{'property': 'prop1'},
{'property': 'prop2'}
]
ret = self.fake_phys_plugin.list_resource_properties(
query={'detail': False})
# Sort returned value to use assertListEqual
ret.sort(key=lambda x: x['property'])
self.assertListEqual(expected, ret)
self.db_list_resource_properties.assert_called_once_with(
'physical:host')
def test_list_resource_properties_with_detail(self):
self.db_list_resource_properties = self.patch(
self.db_api, 'resource_properties_list')
# Expecting a list of (Reservation, Allocation)
self.db_list_resource_properties.return_value = [
('prop1', False, 'aaa'),
('prop1', False, 'bbb'),
('prop2', False, 'ccc'),
('prop3', True, 'aaa')
]
expected = [
{'property': 'prop1', 'private': False, 'values': ['aaa', 'bbb']},
{'property': 'prop2', 'private': False, 'values': ['ccc']}
]
ret = self.fake_phys_plugin.list_resource_properties(
query={'detail': True})
# Sort returned value to use assertListEqual
ret.sort(key=lambda x: x['property'])
self.assertListEqual(expected, ret)
self.db_list_resource_properties.assert_called_once_with(
'physical:host')
def test_update_resource_property(self):
resource_property_values = {
'resource_type': 'physical:host',
'private': False}
db_resource_property_update = self.patch(
self.db_api, 'resource_property_update')
self.fake_phys_plugin.update_resource_property(
'foo', resource_property_values)
db_resource_property_update.assert_called_once_with(
'physical:host', 'foo', resource_property_values)
class PhysicalHostMonitorPluginTestCase(tests.TestCase):

12
doc/api_samples/hosts/host-property-list-detail.json Normal file
View File

@ -0,0 +1,12 @@
{
"resource_properties": [
{
"property": "gpu",
"private": false,
"values": [
"True",
"False"
]
}
]
}

7
doc/api_samples/hosts/host-property-list.json Normal file
View File

@ -0,0 +1,7 @@
{
"resource_properties": [
{
"property": "gpu"
}
]
}

10
doc/api_samples/hosts/host-property-update-res.json Normal file
View File

@ -0,0 +1,10 @@
{
"resource_property": {
"created_at": "2021-12-15T19:38:16.000000",
"updated_at": "2021-12-21T21:37:19.000000",
"id": "19e48cd0-042d-4044-a69a-d44d672849b5",
"resource_type": "physical:host",
"property_name": "gpu",
"private": true
}
}

3
doc/api_samples/hosts/host-property-update.json Normal file
View File

@ -0,0 +1,3 @@
{
"private": true
}

140
doc/source/cli/host-reservation.rst
View File

@ -51,6 +51,49 @@ Result:
..
3. (Optional) Add extra capabilities to host to add other properties. These can
be used to filter hosts when creating a reservation.
.. sourcecode:: console
# Using the blazar CLI
blazar host-update --extra gpu=True compute-1
# Using the openstack CLI
openstack reservation host set --extra gpu=True compute-1
..
Result:
.. sourcecode:: console
Updated host: compute-1
..
Multiple ``--extra`` parameters can be included. They can also be specified in
``host-create``. Properties can be made private or public. By default, they
are public.
.. sourcecode:: console
# Using the blazar CLI
blazar host-capability-update gpu --private
# Using the openstack CLI
openstack reservation host capability update gpu --private
..
Result:
.. sourcecode:: console
Updated host extra capability: gpu
..
2. Create a lease
-----------------
@ -128,6 +171,103 @@ Result:
..
3. Alternatively, create leases with resource properties.
First list properties.
.. sourcecode:: console
# Using the blazar CLI
blazar host-capability-list
# Using the openstack CLI
openstack reservation host capability list
..
Result:
.. sourcecode:: console
+----------+
| property |
+----------+
| gpu |
+----------+
..
List possible values for a property
.. sourcecode:: console
# Using the blazar CLI
blazar host-capability-show gpu
# Using the openstack CLI
openstack reservation host capability show gpu
..
Result:
.. sourcecode:: console
+-------------------+-------+
| Field | Value |
+-------------------+-------+
| capability_values | True |
| | False |
| private | False |
| property | gpu |
+-------------------+-------+
..
Create a lease.
.. sourcecode:: console
# Using the blazar CLI
blazar lease-create --physical-reservation min=1,max=1,resource_properties='["=", "$gpu", "True"]' --start-date "2020-06-08 12:00" --end-date "2020-06-09 12:00" lease-1
# Using the openstack CLI
openstack reservation lease create --reservation resource_type=physical:host,min=1,max=1,resource_properties='[">=", "$vcpus", "2"]' --start-date "2020-06-08 12:00" --end-date "2020-06-09 12:00" lease-1
..
Result:
.. sourcecode:: console
+---------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+---------------+---------------------------------------------------------------------------------------------------------------------------------------------+
| action | |
| created_at | 2020-06-08 02:43:40 |
| end_date | 2020-06-09T12:00:00.000000 |
| events | {"status": "UNDONE", "lease_id": "6638c31e-f6c8-4982-9b98-d2ca0a8cb646", "event_type": "before_end_lease", "created_at": "2020-06-08 |
| | 02:43:40", "updated_at": null, "time": "2020-06-08T12:00:00.000000", "id": "420caf25-dba5-4ac3-b377-50503ea5c886"} |
| | {"status": "UNDONE", "lease_id": "6638c31e-f6c8-4982-9b98-d2ca0a8cb646", "event_type": "start_lease", "created_at": "2020-06-08 02:43:40", |
| | "updated_at": null, "time": "2020-06-08T12:00:00.000000", "id": "b9696139-55a1-472d-baff-5fade2c15243"} |
| | {"status": "UNDONE", "lease_id": "6638c31e-f6c8-4982-9b98-d2ca0a8cb646", "event_type": "end_lease", "created_at": "2020-06-08 02:43:40", |
| | "updated_at": null, "time": "2020-06-09T12:00:00.000000", "id": "ff9e6f52-db50-475a-81f1-e6897fdc769d"} |
| id | 6638c31e-f6c8-4982-9b98-d2ca0a8cb646 |
| name | lease-1 |
| project_id | 4527fa2138564bd4933887526d01bc95 |
| reservations | {"status": "pending", "lease_id": "6638c31e-f6c8-4982-9b98-d2ca0a8cb646", "resource_id": "8", "max": 1, "created_at": "2020-06-08 |
| | 02:43:40", "min": 1, "updated_at": null, "hypervisor_properties": "", "resource_properties": "[\"=\", \"$gpu\", \"True\"]", "id": |
| | "4d3dd68f-0e3f-4f6b-bef7-617525c74ccb", "resource_type": "physical:host"} |
| start_date | 2020-06-08T12:00:00.000000 |
| status | |
| status_reason | |
| trust_id | ba4c321878d84d839488216de0a9e945 |
| updated_at | |
| user_id | |
+---------------+---------------------------------------------------------------------------------------------------------------------------------------------+
..
3. Use the leased resources
---------------------------

7
releasenotes/notes/resource-property-discovery-42df197a1a49bd76.yaml Normal file
View File

@ -0,0 +1,7 @@
---
features:
- |
Adds a host resource property discovery API, which allows users to
enumerate what properties are available, and current property values.
Properties can be made private by operators, which filters them from the
public list.