Reorganize guidelines into guidelines directory and create
current_guideline that is softlink to the latest approved guideline.
The same with add-ons guidelines - they are moved to guidelines
directory within add-ons one and soft links are created pointing
to the latest guidelines for each add-on.
Also cleaned up some tooling that hardwired where guidelines lived.
Change-Id: I5ad4b91b1afb44a0a6987b339f7efba14f395302
Will do the same for next guideline.
Removed notion of core projects and replaced as required for
OpenStack Powered Logo.
Change-Id: Iee17aaa06b7c0792fade0f9b012909e11d0d9f29
We stopped talking about the cinder v2 API with change Ic0d18d89b9
a year ago, so update the volumes capabilities descriptions so that
they no longer mention being "added as advisory to highlight the
transition from v2 to v3."
Change-Id: I29aa97c459d340bb10dea68d907d4214fb76b5c4
The Cinder v2 APIs are deprecated since Pike
- https://docs.openstack.org/releasenotes/cinder/pike.html#deprecation-notes
and its capabilities were also marked as Deprecated in
- Ie104503e1655e1dad4737b86bc047c6c363a6569.
Same for volume reserve capabilities which were marked
as Deprecated in
- I1599c1c3721e72a415be65af8b474206b174f241
Pike release was last supported in 2018.11 guidelines and
2019.11 guidelines onwards, Cinder v2 and reserve capabilities
were not included in guidelines.
This is time to move those Deprecated capabilities in the Removed list
and later in the next cycle we can completely cleanup those as well
as their test information.
Change-Id: Ic0d18d89b917d0fd428ecbde40c9a2dd7682eeee
This patch adds a new capability for Nova: the "create multiple
servers" API [1], which allows a user to create multiple servers with
a single API call. Although this API is not widely used by tools such
as gophercloud, libcloud, or Terraform, it meets most other criteria
and is not specific to any particular backend.
[1]
https://developer.openstack.org/api-ref/compute/#create-multiple-servers
Change-Id: I5bdbc1504fb088392e55b76c2ce70f19d6aa69c6
The Cinder v2 API moved from Supported to Deprecated status in Pike.
Although the 2018.11 guideline will still cover releases as far back
as Ocata, most clouds supported v3 even in Ocata and v3 tests have
been required since the 2018.02 guideline. Deprecating the v2
capabilities in the 2018.11 guideline should not negatively impact
existing cloud certifications, is very unlikely to affect users, and
will pave the way for newer clouds to not have to carry a deprecated
API.
This patch moves all volumes-v2 capabilities from required to
deprecated status.
Change-Id: Ie104503e1655e1dad4737b86bc047c6c363a6569
In 2018.02 we set the "required_since" field to "2018.02" on a couple
of capabilities to mark them as moving from advisory to required.
However, they're still in the "advisory" list in the components section.
This patch cleans them up by removing them from the advsiory list and
placing them in the required list.
We also assigned volumes-v3-upload a high enough score to qualify as
advisory and added it to the JSON document, but did not add it to the
advisory section. This patch adds it.
Further, this patch removes two capabilities from the "removed" list
as is customary after a cycle.
Change-Id: Ib7c1216f3b3045e709e7fefa8b7ccce9922959b5
There are a number of inconsistencies between the latest active
guidelines (next.json, 2018.02.json, and 2017.09.json) and
Tempest. This patch corrects the following errors:
* There are several duplicte tests between the new ServersTestJSON
and the old ServersTestManualDisk classes. This patch removes
all of the tests from ServersTestManualDisk as duplicates. While
technically the tests from ServersTestManualDisk should be
flagged, doing do would remove active tests from the guideline
because of name collisions in RefStack test result processing.
No functional change is made to the required capabilitied by
removing the duplicate entries, and adding a aliases to the
ServersTestJSON entry preserves old valid test results.
* A syntax error for an idempotent id was fixed.
* Some network router tests were updated to require admin capabilities
and have been flagged.
* Some volume tests were removed from Tempest or refactored, and
have been flagged.
* Some volume and object store tests were renamed, and new aliases
have been added to reflect the name changes.
Change-Id: Ib26ed051f42b8a528e6dd349141ec49f330243f3
Idempotent id expects the form of id-<UUID>. One of the entries
is of the form <UUID>. This patch corrects the typo.
Change-Id: Ie5ad764487097ef841750e0cd5a0046ab7448ac0
In general, when attach/detach volumes, there are several
internal APIs which only used to update volume status and
to indicate the intermediate state. For example, os-attach
and os-detach, they have already been discussed in the
maillist [2] and marked as deprecated in patch [1].
Since the complete attach/detach process has already been
covered by "compute-volume-attach" capability.
This patch proposes to deprecate volume "reserve&unreserve"
capability as well. These tests could break defcore testing
for OpenStack clouds, especially for public clouds.
1. Remove V2 and V3 reserve&unreserve from 2017.09
2. Remove V2 and V3 reserve&unreserve from 2018.02
3. Remove V2 and V3 reserve&unreserve from next
[1]: d234631f4f
[2]: http://lists.openstack.org/pipermail/interop-wg/2017-April/000116.html
Change-Id: I1599c1c3721e72a415be65af8b474206b174f241
The 2018.02 Guideline was the first to be published in the new
schema 2.0 format. [1] The change to 2.0 was done with the agreement
that it would probably be easiest to write 2018.02 in the new format
and then create a new next.json from it (rather than the other way
around) due to the timing. This patch transforms next.json to
schema 2.0 by using the existing 2018.02 document as the base. Once
this lands, we'll be able to begin work on the next round of scoring
using the new 2.0 format, as well as do the usual round of housekeeping
like removing old capabilities and moving advisory ones to required.
For simplicity, we'll do those in future patches.
This patch also removes one test from the listing as it has been
combined with another test under the other test's idempotent ID.
See discussion in [2] for more background. Leaving the test here
but also swapping the name and alias as we normally do in next.json
would cause a duplicate key issue with the docs jobs.
[1] https://git.openstack.org/cgit/openstack/interop/schema/2.0.json
[2] https://review.openstack.org/#/c/548990/3/2018.02.json@2924
Change-Id: I12457d66e804f6f6673c604542848d41ac1396a2
This commit creates the 2018.02 guideline that will be presented to
the Board of Directors at the February 2018 meeting for approval.
This commit moves all capabilities that were advisory in 2017.09
to required status.
This patch also refreshes the next.json document to prepare for the
fall 2018 guideline drafting process. As the meeting date for the
BoD meeting at the fall PTG has not yet been determined, it uses an
arbitrary date in August for the time being (we can change this
later once we have a firm date). It also moves items that were
advisory in 2018.02 to required status.
This patch can be merged now as it sets the 2018.02 document to
advisory status--we will change it to approved if and when the BoD
votes to approve it next month.
Change-Id: I634e2dc1273e7b67103c86436b4ea5489cae3543
Just a quick first pass at this, looking forward to getting a bit
of input. Only a few existing capabilities needed updates, but
there are two new capabilities, objectstore-slo-support and
objectstore-dlo-support. Obviously, these may require some
discussion, and I will need to write new tests for the dlo
capability.
Change-Id: Ie658f499a21e4f0bb92d7aeba1387f0497752989
The Additional Properties Wavier on Nova responses was intended
to be cancelled in the 2017.01 guideline. This required official
removal from the board approved guideline. This patch removes the
Additional Properties Waiver from next.json, to be approved for
the 2018.01 guideline and officially cancelling the waiver.
Change-Id: Ia65eb241d5e82ac4dd91c3e3b41f5b69dbdb8a43
This proposes scoring for the 2018.1 guidelines to include
the compute capability to create keypairs with a specified
type.
This microversion is important for Windows guests since it
allows the creation or import of x509 keypairs.
Notes:
- This feature is based on the compute API 2.2 microversion
and was introduced in the Kilo release:
https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id2https://developer.openstack.org/api-ref/compute/#create-or-import-keypair
- This feature is considered widely deployed since it is
not configurable or backend-specific.
- It is considered not used by tools given it depends on
those tools supporting compute API microversions, and it
is assumed that most tools are still lacking in this area.
- This capability is discoverable since client code can
determine if the microversion is available on the server
via version discovery, i.e. compute-list-api-versions.
- While this feature was in the last release, it is not
considered "required" since microversions are opt-in per
request by the client.
NOTE: Since this is the first proposed capability that
depends on a microversion, it is assumed that the definition
of what "required" means will need some adjusting to account
for weighing microversion-based capabilities differently.
- There is an existing Tempest test for this microversion
with id 89d59d43-f735-441a-abcf-0601727f47b6.
Change-Id: I62554618e38f3f7ca1b508387a1eef190cdcef63
Test [1] was promoted from the advisory group in 2017.01 to become
a required test in 2017.09. Since it requires a second set of user
credential, it should be flagged until the issue is fixed. Or better
yet, we just move it back to the advisory section.
[1] tempest.api.identity.v3.test_projects.IdentityV3ProjectsTest.test_list_projects_returns_only_authorized_projects
Change-Id: I83916504b6f87ff748f6f10b6c0e29b5c3fe6ad8
The Board of Directors deferred it's August 2017 meeting until
September, so the Guideline we intended to ask approval for in August
will now not be voted on until September. Guidelines are named by
the year and month of their approval, so this patch updates references
to "2017.08" to "2017.09".
Change-Id: If7e19ea4524b7c944d503cbda73052ef0b0f55d0
Test [1] is a typo. The correct name is [2]. The test_volume_bootable test belongs to
the test_volumes_actions.py not test_volumes_get.py file. Thus, no Tempest version will
be able to run this test because its name won't be able to match any name from the test
list produced by Tempest. Following the practice in [3], fix 2017.01.json by adding the
correct name as alias since this is an approved guideline. For 2017.08 and next.json,
replace the incorrect name with the correct one.
[1] tempest.api.volume.test_volumes_get.VolumesActionsTest.test_volume_bootable
[2] tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_volume_bootable
[3] https://review.openstack.org/#/c/450346/
Change-Id: I9d17b6f9366740878f1b2918d38da775f4b7e620
Add keystone get catalog capability in next.json. Non-admin test case was
added in tempest
Change-Id: Ie28ca460cf4a11a9431dfa01e216396f5787acc3
Depends-On: Ia745dff050f35cc5a3fc3744d58c6fe1e875e56f
A new volume api test [1] will be added to the function
test_update_volume_metadata_item, the test name will be
changed on Tempest side. This patch adds aliases for
following this Tempest change.
[1] https://review.openstack.org/#/c/479152
Change-Id: Ica8b57f5f5392e2bf47e368e9b10ef9fa5695c30
Depends-On: Idb9d198383ce93453707f1204806e4ae82c1e46a
A new snapshot api test [1] will be added to the function
test_update_snapshot_metadata_item, the test name will be
changed on Tempest side. This patch adds aliases for
following this Tempest change.
[1] https://review.openstack.org/#/c/479124
Change-Id: I672bbcb9983e7a3ace7769722d6ff97f7bc0ccc8
Depends-On: I3697406e5541bbdb191e4d6e63831be31b471e78
This is the second patch for I700b7009bfb3a93ade832198f4d58f487d641af8
It provide all the aliases for volume-v3-* which are
advisory capabilities.
The change is due to tempest renaming of its VolumeV2 classes.
Tempest change is to reuse the same test case for both V2 and V3.
Avoiding confusion about the version being tested by removing "V2"
from the name. Notice that Cinder version should be setup at
tempest.conf file and defaults to V2.
Change-Id: I5d709698297d043b5cb4a40c263f30cc00908238
Create the 2017.08 Guideline candidate, to be moved from review to
approved status if and when approved by the OpenStack Foundation
Board of Directors. A vote is expected at the August 22, 2017
Board meeting.
Note: this patch also adds back test information for two Capabilities
that were deprecated (and keeps them marked as not required). The
tests associated with these were inadvertently removed in
Iabb1d0820300442e407d8b92dfb575ac71471d06 but should instead have
simply had their "required-since" status nullified, as removing
them entirely breaks the jsonToRst.py script and leaves users
unable to easily find the tests that are no longer required due to
the deprecation.
Change-Id: Ifce3fe1e3401e7f887ddeae2a41078dca7e16625
Add keystone validate token capability into next.json/ 2017.08.
non-admin test case is now available in tempest. Further
details on commit Ice1a241445d532ee2c4b1ad8d2c4c896d755798d
TC call GET on /v3/auth/tokens API.
Depends-On: Ice1a241445d532ee2c4b1ad8d2c4c896d755798d
Change-Id: I062e6148e90ae84d34f2df4577eb581ce76d021b
The recent version of Tempest (at least from v15) has renamed
all the previous VolumeV2* test case to Volume*. However the
current defcore guideline has not added such alias, which has
resulted in error when user wants to use new version of tempest
to test.
This patch is the part 1 of the fix, due to the large amount of
work. This patch has provided necessary aliases for all the
volume-v2-* capabilities (that means mission complete for required
capabilities) and some of the volume-v3-* ones.
We should expect another patch to provide the aliases for the
remaining volume-v3-* which are at the moment advisory capabilities
Co-Authored-By: Luz Cazares <luz.cazares@intel.com>
Signed-off-by: zhipengh <huangzhipeng@huawei.com>
Change-Id: I700b7009bfb3a93ade832198f4d58f487d641af8
In this flagging request two test cases are flagged:
"test_update_port_with_security_group_and_extra_attributes"
and "test_update_port_with_two_security_groups_and_extra
_attributes"
Reasoning: the concept of a network in OpenStack usually
refers to a l2 network, instead of concept like vpc in
aws. The flagged test cases require the cloud to support
subnets within a l2 network, which is a rather rare use
case for typical IPv4 public cloud implementation.
Specifically, in the first flagged test case update_port
action involves creating a new port in a new subnet and
update the previous port to the new one. The second test
case deals with a similar problem in a similar manner.
Proposal: flag these two test cases in defcore guideline
2016.08.json, 2017.01.json and next.json
Change-Id: I247d9247ba751f2d566ec2eb08905a264e784d7e
Signed-off-by: zhipengh <huangzhipeng@huawei.com>
test_list_servers_by_limits will be removed from Tempest
by https://review.openstack.org/463257, this is to remove
it from json file
Change-Id: I052e4305a08e2c63277903357ffe2696f2e85c95
ArkadyKanevsky