Commit Graph

86 Commits

Author SHA1 Message Date
Zuul aa14e9ccde Merge "Support microversion 1.39" 2022-06-30 14:37:56 +00:00
Zuul 5e46c52c5c Merge "Replace deprecated assertRaisesRegexp" 2022-06-28 16:27:35 +00:00
Balazs Gibizer 9545623054 Support microversion 1.39
The CLI now support the following syntax both for openstack resource
providers list and allocation candidate list commands:

  --required T1,T2 --required T3

and it means (T1 or T2) and T3.

In the allocation candidate list command the above can be used both
outside and in a --group context.

Story: 2005345
Story: 2005346

Depends-On: https://review.opendev.org/c/openstack/placement/+/826719

Change-Id: I38ff55bdd072f3a9c1ed03e28192d045cb4096cf
2022-06-01 14:09:29 +02:00
l Luis db84950f55 Replace deprecated assertRaisesRegexp
The assertRaisesRegexp method has been deprecated since it was renamed
to assertRaisesRegex in Python 3.2.

https://docs.python.org/3/library/unittest.html#deprecated-aliases

Change-Id: I4d19f3e0314f912e83d931bf061d0eacb13987f4
2021-11-10 17:08:44 +08:00
songwenping 1f2fdddd9c Remove usage of six
Remove six-library Replace the following items with Python 3 style code.
- six.raise_from
- six.StringIO
- six.moves
- requirement.txt

Change-Id: If3d810a1dbe7d506aa55cc07717babec8393b8c5
2021-10-25 07:26:36 +00:00
songwenping 99d5dcc170 Remove usage of six
Remove six-library Replace the following items with Python 3 style code.
- six.string_types
- six.text_type

Change-Id: I8d06743b7a21cfa4db88f9503e2ab9247f6685b8
2021-10-03 07:24:34 +00:00
Balazs Gibizer 0ea8f1b05d Fix allocation show / unset with empty allocation
Add a guards for empty allocations in the allocation parsing code of
allocation show and allocation unset.

Closes-Bug: #1942740
Change-Id: Ic0e6e981d6602a76935f6bc6d9ffb0a707a5b1a9
2021-09-06 15:11:47 +02:00
Balazs Gibizer 087dd64dd2 Repro allocation show bug with empty allocation
This patch adds a functional test case to reproduce the bug when an
allocation show command on an empty (non existent) allocation causes
KeyError as the code expects project_id in the response but the
response is {"allocations":{}}. Also similarly for allocation unset
where microversion 1.38 fails on parsing out consumer_type from the
empty allocation too.

Related-Bug: #1942740

Change-Id: Iea27c9376480e3e94d12eec56b466331e5c6f171
2021-09-06 15:10:37 +02:00
melanie witt 5e38a59317 Add support for microversion 1.38 consumer types
Story: 2005473
Task: 42878

Depends-On: https://review.opendev.org/c/openstack/placement/+/679486

Change-Id: I1b7d7095e62dc80191ff3e199959fb4d7df6f68e
2021-08-12 20:37:38 +00:00
Zuul b0a74d094d Merge "Mark microversion 1.37 supported" 2021-07-31 01:20:41 +00:00
Sean Mooney 281ace20df default to max version when no session
This change fixes a regresssion intoduced
by the supprot for microverion auto-negotiation
which resulted in the inablity to generate
bash autocompltion.

When the client_manager does not have a api session established
such as when being invoked for auto completion this
change modifies get_version to return version.MAX_VERSION_NO_GAP

Story: #2008958
Task: #42592
Change-Id: I90e1e3caeaa22ff51df515226b3ac5c7b64c88b6
2021-06-09 14:38:57 +00:00
Zuul 3a3710c6ee Merge "Note env OS_PLACEMENT_API_VERSION support" 2021-05-20 12:57:44 +00:00
Zuul 869769e3bf Merge "Add "--resource-class" to allocation unset" 2021-05-14 15:22:20 +00:00
Tetsuro Nakamura 39befda0d7 Verify result for inventory set --dry-run
There were tests to verify that 'resource provider inventory set'
command with --dry-run and --aggregate option changes nothing on
the server side but there was no test to verify the result of that
command. This patch adds the test.

Change-Id: I8d2c25a654928d8d46d5e1d0ee203600c9a72600
2021-05-13 06:29:04 +00:00
Tetsuro Nakamura a4ac717a48 Add "--resource-class" to allocation unset
The ``openstack resource provider allocation unset`` command now
supports ``--resource-class`` option, which accepts string of a
resource class. This will remove allocations for the given resource
class from all the providers. If ``--provider`` option is also
specified, allocations to remove will be limited to the given resource
class of the given resource provider.

example1::

  # remove VGPU allocation from provider P for this consumer.
  allocation unset <consumer_uuid> --provider P --resource-class VGPU

example2::

  # remove VGPU allocations from all providers for this consumer.
  allocation unset <consumer_uuid> --resource-class VGPU

Change-Id: I91baf9de5205ec15023706a1646556850302d3d9
Story: #2006779
Task: #37304
2021-05-13 06:23:54 +00:00
Tetsuro Nakamura 9778535263 Note env OS_PLACEMENT_API_VERSION support
The placement client (osc-placement) supports the
``OS_PLACEMENT_API_VERSION`` environment variable,
but it is not described in the ``openstack -h`` command.

So this patch adds the description as well as other
environment variables.

Change-Id: I41a3718bfa27ecf370c7375b4f761cda47866b1a
2021-05-12 02:55:15 +00:00
Tetsuro Nakamura 05ca1ee2e1 Switch default to use latest microversion
The default value of ``--os-placement-api-version`` is now changed to ``1``
from ``1.0``, which by default enables negotiation with server to choose
max microversion supported both by server and client.

Change-Id: I0ad02ee5c3e57aa126bfba739738280bbf3a97d0
Story: #2005448
Task: #30497
2021-05-12 01:53:13 +00:00
Tetsuro Nakamura 8ac8c8627c Support auto-negotiated microversion
To improve the usability for those who are not familiar with
microversion, this patch adds microversion negotiation to pick up max
microversion supported both by client and by server.

This negotiation is enabled by specifying only major api version,
``--os-placement-api-version 1``. Note that the default microversion
remains to be ``1.0`` in this patch and will be switched to the
negotiated version in the following patch.

Change-Id: I2998ff0f3941bf226a942969adf75564a8d5a065
Story: #2005448
Task: #30497
2021-05-12 01:52:56 +00:00
Balazs Gibizer 8b3ed3c663 Mark microversion 1.37 supported
Microversion 1.37 adds support for re-parenting and un-parenting RPs.
The client supports these operations with the new microversion out of
the box. So this patch just marks the microversion as supported.

Story: 2008764
Task: 42132
Depends-On: https://review.opendev.org/c/openstack/placement/+/784020
Change-Id: I62f5a620e6639c29d4a8ed99f7e42b72f7af1a74
2021-04-15 07:49:04 +00:00
Zuul 499b929e2f Merge "Include usage in 'inventory list', 'inventory show'" 2021-03-22 17:34:07 +00:00
Tetsuro Nakamura eb5a6258fe Support granular allocation candidate list
The ``openstack allocation candidate list`` command now supports
``--group`` and ``--group-policy`` option. The ``--group`` option is
accepts integer to group granular requests. If specified, following
given options of resources, required/forbidden traits, and aggregates
are associated to that group and will be satisfied by the same resource
provider in the response. Can be repeated to get candidates from
multiple resource providers in the same resource provider tree.

If multiple groups are supplied, the separate groups may or may not
be satisfied by the same provider. If you want the groups to be
satisfied by different providers, set ``--group_policy`` to
``isolate``.

For example:
    ``openstack allocation candidate list \
        --group 1 --resource VCPU=3 --required HW_CPU_X86_SSE \
        --group 2 --resource VCPU=4 \
        --group_policy isolate``

The granular request is supported in 1.25, but this is tested via
microversion 1.29 since the API is unaware of nested provider
before that version. Note that 1.26-1.29 has no impact to client:
  1.25 - Adds support for granular resource requests via numbered
         querystring groups in GET /allocation_candidates.
  1.26 - Add ability to specify inventory with reserved value equal
         to total.
  1.27 - Include all resource class inventories in `provider_summaries`
         field in response of `GET /allocation_candidates` API even if
         the resource class is not in the requested resources.
  1.28 - Add support for consumer generation
  1.29 - Support nested providers in `GET /allocation_candidates` API.

Change-Id: Ib32181abd12db0c2983065ea9846e911033d53f1
2020-09-11 06:23:09 +00:00
Pawel Baclawski db98792e46 Support multiple member_of query parameter
The ``openstack allocation candidate list`` and the ``openstack
allocation candidate list`` command now supports ``--member-of``
option, which accepts a list of comma-separated UUIDs of the
resource provider aggregates. If this is specified, the returned
resource providers must be associated with at least one of the
aggregates identified by uuid. This option can be repeated to
add(restrict) the condition with ``--os-placement-api-version 1.24``
or greater.

This deprecates the ``--aggregate-uuid`` option because it is
difficult to keep using this option to support multiple ``member_of``
feature added in microversion 1.24.

Change-Id: I0f3068c411c504380918f27f994734528fe55c77
Co-Autored-By: Tetsuro Nakamura <tetsuro.nakamura.bc@hco.ntt.co.jp>
Story: #2006940
Task: #37613
2020-09-11 06:23:09 +00:00
Zuul 8d852bc956 Merge "trivial: Fix formatting of command help texts" 2020-09-11 01:13:08 +00:00
Zuul 874032c9c9 Merge "Remove six.PY3" 2020-09-11 01:13:07 +00:00
Zuul 3986993cd4 Merge "Update command help information" 2020-09-11 01:07:31 +00:00
likui cfdebe3c78 Remove six.PY3
The Python 2.7 Support has been dropped since Ussuri.
So remove hacking rules for compatibility between python 2 and 3.

Change-Id: I5ec70dc8153dc0ab0bf0323ffb814822566da6cc
2020-09-04 23:20:05 +08:00
Stephen Finucane dee5bffb28 Include usage in 'inventory list', 'inventory show'
The 'resource provider inventory list' and 'resource provider inventory
show' commands describe everything about some given inventory *except*
their usage, which is a pretty important metric in quickly assessing
availability of a given resource. Instead, usage is hidden behind the
'resource provider usage show' command. There doesn't appear to be any
reason for this to the case, aside from wanting to directly map to the
placement API which shouldn't be a goal. Make these commands more
friendly by including usage information. We may want to consider
deprecating the 'resource provider usage show' command in the future,
but that's left for another day.

Change-Id: Iafedac3ff7a5ae60e6027741c44904babf64d3f1
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2020-08-07 15:12:02 +01:00
Stephen Finucane 644feaa50e trivial: Fix formatting of command help texts
Use the reST, Luke.

Change-Id: Ie4a75c14c67a615e81ba97b60e6a5d61d2e0f764
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2020-08-07 10:25:56 +01:00
Zuul 35cc92e603 Merge "Provide a useful message in case of 5xx error" 2020-08-06 18:05:28 +00:00
likui 7aed6dcd06 Update command help information
update openstack allocation candidate list help.

Change-Id: Iad530edaec39545a5573196f4c33ca1b29808fd7
2020-08-05 17:25:55 +08:00
Ghanshyam Mann df0527de30 Use unittest.mock instead of third party mock
mock has been removed from requirements files of its deps
which used to install mock for this repo gate.

Fixing gate by moving it to unittest.mock.

Change-Id: I5e0ef2a74d03333ebd9d4be1ef7fcb040e54ed30
2020-08-03 17:46:51 -05:00
Andreas Jaeger 2abc37868b Cleanup py27 support
This repo is now testing only with Python 3, so let's make
a few cleanups:
- Remove python 2.7 stanza from setup.py
- Add requires on python >= 3.6 to setup.cfg so that pypi and pip
  know about the requirement
- Remove obsolete sections from setup.cfg
- Update classifiers
- Update requirements, no need for python_version anymore
- Switch to using sphinx-build
- Use newer openstackdocstheme version
- Remove install_command from tox.ini, the default is fine
- Update to hacking 3.0, fix warnings

Change-Id: If66a87c519ce82ee7890ac3e133afef13fb35bd2
2020-04-13 11:28:10 +02:00
Tetsuro Nakamura d80651a389 Improve tests for warning messages
The functional test for allocation create did test that it displays
the warning message if the microversion is not high enough, but didn't
validate the returned json result.

This patch changes it to validate the actual result as well as the
warning message.

Change-Id: I8280075a020fb4a10b82b710d5e0ca997a3028fd
2020-01-10 07:12:15 +00:00
Matt Riedemann 37f521e53c Follow up to I627bfd1ff699d075028da6afafbe7fb9b2f13058
This addresses comments from that change:

* https://review.opendev.org/#/c/691422/7/osc_placement/resources/allocation.py@207
* https://review.opendev.org/#/c/691422/7/osc_placement/resources/allocation.py@219
* https://review.opendev.org/#/c/691422/7/osc_placement/resources/allocation.py@224
* https://review.opendev.org/#/c/691422/7/releasenotes/notes/allocation-unset-beb0d904c8bc4228.yaml@6

Change-Id: I444fa2ec69d3fabb588c5baf58e61c4c80517c13
2019-11-25 12:30:41 -05:00
Andrey Volkov 75b3aa094f Provide a useful message in case of 5xx error
If server response does not have an appropriate json format,
the message was useless. For example, if placement-api is down,
we had something like this:

$ openstack resource provider delete
Expecting value: line 1 column 1 (char 0)

This was because the code assumed that the response had details in
it. That is correct for 4xx errors, but that's not the case for 5xx
errors. This patch changes it not to retrieve the details from
response on 5xx error, and just raise keystone exceptions.

Now we have something like this:

$ openstack resource provider delete
Service Unavailable (HTTP 503)

Story: #2005437
Story: #2005438
Task: #30478
Task: #30479

Change-Id: I07fa0b626b7529699179c8cd8e254fd66455c930
Co-Autored-By: Tetsuro Nakamura <tetsuro.nakamura.bc@hco.ntt.co.jp>
2019-11-25 10:25:39 +00:00
Matt Riedemann 0acbe401c5 Add support for microversion 1.28 in allocation set
This adds support to the resource provider allocation set command
for microversion 1.28 which uses consumer generations to ensure
atomic updates of consumer allocations.

Change-Id: I8f85ee4ff4c39fba69a78bbe642f79a706504df8
Story: #2006912
Task: #37557
2019-11-19 12:41:38 -05:00
Matt Riedemann 9b86d94559 Add resource provider allocation unset command
Adds the "openstack resource provider allocation unset" command which
allows removing resource provider(s) from the set of allocations for
the given consumer. This can be useful when a consumer has allocations
against more than one resource provider and
"openstack resource provider allocation delete" is undesirable as it
removes all allocations for the consumer. The new unset command requires
"--os-placement-api-version 1.12" or greater.

Note that the --provider option can be specified multiple times.

If the resulting allocations are all gone and 1.28 or greater is not
being used, a DELETE is called rather than PUT since
PUT /allocations/{consumer_id} before 1.28 does not allow an empty
"allocations" dict in the request body. In this case nothing is output
similar to the "openstack resource provider allocation delete" command.

Change-Id: I627bfd1ff699d075028da6afafbe7fb9b2f13058
Story: #2006779
Task: #37304
2019-11-19 12:38:42 -05:00
Chris Dent 306a91e9e8 Be explicit about auth type in functional tests
When using the old style of '--os-url' we didn't need to be
explicit, it deduced the right type. And python-openstackclient
>= 4.0.0 it does for --os-endpoint. But when using '--os-endpoint'
with <4.0.0 we need to state 'admin_token' explicitly.

This form works in both 3.19.x and 4.0.x.

Change-Id: Ie9c76a23658b46997dcb17ba3fb7f1e700eca506
2019-09-24 16:00:29 +01:00
Chris Dent f49b9fcc81 Use os-endpoint instead of os-url for functional tests
The functional tests use an explicit endpoint and token for auth
to the placement fixture. Before openstack client 4.0.0 os-url
was a way to specify the endpoint. Now os-endpoint is the way.

Update accordingly.

Note that this doesn't impact the behavior of the plugin itself,
just the tests.

Change-Id: Id4f8f0224f61c1741e6a1a8369ec053bb1824e8a
2019-09-23 13:13:15 +01:00
melanie witt 8bbca01dbd Follow up for Ib0cbb58d0adbbcfe83ee48d2ff6c9af1a516a7ae
Change-Id: I594c4ee874dbca5caa52f1c6e60e924d39a641d4
2019-09-04 21:30:21 +00:00
melanie witt 5cf7d77ab4 Add --dry-run option to 'resource provider inventory set'
This adds a new '--dry-run' option which gives the ability to preview
changes before effecting them.

Story: 2006318
Task: 36190

Change-Id: I24bbe9657c0b83a93f0d4c762992c16e91b4ec47
2019-09-04 21:25:33 +00:00
melanie witt 0ce562fbb0 Add --amend option to 'resource provider inventory set'
This adds a new '--amend' option which can update resource provider
inventory without requiring the user to pass a full replacement for
inventory. It does a GET for existing inventory, applies user provided
updates, and does a PUT of the updated inventory.

Story: 2006318
Task: 36049

Change-Id: I4ed1016852144377485959e65077803f5f2c672c
2019-09-04 03:36:46 +00:00
melanie witt f08cb39813 Add --aggregate option to 'resource provider inventory set'
This adds a new '--aggregate' option which can set resource provider
inventory for all resource providers that are members of the specified
aggregate.

The main motivation for this option is to give operators a tool to
make setting overcommit allocation ratios on a per-aggregate basis
easier, after the functionality from the old
Aggregate[Core|Ram|Disk]Filter scheduler filters was removed back in
Ocata [1].

[1]
http://lists.openstack.org/pipermail/openstack-dev/2018-January/126283.html

Story: 2006318
Task: 36189

Change-Id: Ib0cbb58d0adbbcfe83ee48d2ff6c9af1a516a7ae
2019-08-28 21:03:39 +00:00
Pawel Baclawski e52fa969f9 Add support for 1.22 microversion
The 1.22 microversion of placement adds support for
excluding resource providers and allocation candidates
with specified traits. A forbidden trait may be specified
with --forbidden option.

Change-Id: I4760fe6119e29d80f63ec7bd6c4a5f2395a9291e
2019-05-31 09:47:13 +02:00
Tetsuro Nakamura 250a8d18cd Expose version error message generically
The change in https://review.openstack.org/#/c/652100/ has improved
the error message for the required placement api version::

 Operation or argument is not supported with version 1.0

had become::

 Operation or argument is not supported with version 1.0; \
  requires at least version 1.1

However, this was only applied for aggregate use cases. This patch
follows it up to apply the improved error message generically to other
commands than aggregate and other version compare function than
"greater than or equal to".

Change-Id: I28226a1dbc003a8b6366e528469152c141b40593
Story: #2005448
Task: #30498
2019-05-20 11:44:13 +00:00
Chris Dent da8cd4d68b Use PlacementFixture in functional tests
Change the functional tests to use the PlacementFixture instead
of devstack as the source of a placement API. This speeds up
the tests considerably and lowers the number of dependencies.

There are four primary changes:

* For each test a PlacementFixture is instantiated, using the
  usual in-RAM db and in-process placement.

* Because of some exceedingly confusing optimizations in
  osc_lib and python-openstackclient, done to improve start
  up time, a session to placement was caching the service
  url. This meant that after a first test succeeded, every
  subsequent one would not because it was trying to talk
  to a fake hostname that was no longer being intercepted.

  The workaround for this was to monkeypatch the method
  in the ClientCache class which provides access to a client (per
  service-type). The replacement method makes a new client
  every time.

* The previous tests would subprocess out to a real call of
  the openstack command and then interpret the results.

  Now, a run() method on OpenStackShell is called instead.
  This accepts arguments in the same way, but we need to
  a) capture stderr and stdout, b) make a try/except for
  SystemExit to get some error responses (mostly from
  the argparse lib which has a tendency to exit for you
  instead of politely telling you it wants to), c) deal
  with errors from commands ourself rather than using
  exceptions from the subprocess module.

  Switching to this form means that logging becomes in-process
  and more visible. To accomodate this the Capture fixture
  from placement is used. This was chosen because we are already
  pulling in the PlacementFixture. If this seems icky, I can fix
  it with a local one. This was the shorter path.

* The legacy dsvm jobs have been removed in favor of "standard"
  functional jobs for 2.7 and 3.6 that require openstack/placement.
  The playbooks associated with the legacy jobs are removed.

  tox.ini is adjusted to reflect this new setup. Because tox-siblings
  functional is being used, we don't want to share tox envs with the unit
  tests. The 3.5 functional job is removed because we no longer target
  that.

After running these for a while it became clear that there were
intermittent failures being caused by subunit attachments being too
large. This was eventually traced back to logging from all packages
being set to DEBUG even when something else was requested. That was
traced back to a poor interaction between the way that osc does
logging and the way oslo_logging does logging (used by placement and
thus the placement fixture). The workaround, embodied in the
RESET_LOGGING list in osc_placement/tests/functional/base.py, is to
get and reset the log level for a subset of the packages that are
used.

Change-Id: I7deda200b372ff6a7ba67b0c4fa0e53c4fa16ffc
Story: 2005411
Task: 30428
2019-05-03 15:04:56 -06:00
Matt Riedemann cf0d72191f Improve aggregate version check error messages with min_version
The aggregate set/list commands require at least version 1.1 but
if you don't specify any version we default to 1.0 and you get this
unhelpful message:

  Operation or argument is not supported with version 1.0

This change adds a simple min_version kwarg to the version compare
method such that if we fail the version check and the kwarg is passed
we use it to improve the message so the user doesn't have to read
the help of the command to figure out what they did wrong, e.g.:

  Operation or argument is not supported with version 1.0; \
  requires at least version 1.1

This only handles the aggregate set/list commands and could be used
elsewhere but this is just a start.

Change-Id: I28a859918ab78c8aa47e54928cb14339c7c75498
Story: #2005448
Task: #30498
2019-04-17 05:09:12 +00:00
Zuul f2d41ee1df Merge "Microversion 1.21 support" 2019-04-12 16:41:00 +00:00
Zuul 5417b38509 Merge "Add support for 1.19 microversion" 2019-04-12 15:46:59 +00:00
Xiaopengli dc3ede32df Fix the metavar on "resource provider aggregate set"
The "name" metavar is wrong since it is for the resource
provider UUID argument. This fixes the metavar to match
the argument.

Story: #2005446
Task: #30490

Change-Id: I1e9076a36c6cd27212bd76ee04307296fc937168
2019-04-12 09:06:39 -04:00