Reorganize guidelines into guidelines directory and create
current_guideline that is softlink to the latest approved guideline.
The same with add-ons guidelines - they are moved to guidelines
directory within add-ons one and soft links are created pointing
to the latest guidelines for each add-on.
Also cleaned up some tooling that hardwired where guidelines lived.
Change-Id: I5ad4b91b1afb44a0a6987b339f7efba14f395302
Reorganize guidelines into previous_guideline directory
and current_guideline that is softlink to the latest approved guideline.
Cleaned up some tooling that hardwired where guidelines lived.
Change-Id: Ia6be9ca6326718488ee5668df3806da5f76dc456
Clouds are hard. English, doubly so. We have a longstanding
spelling error in the schema: "superceded" should be spelled
"superseded". This patch corrects it and updates all the documents
the typo was propagated to over time.
Change-Id: I72f24bf026919f4ccce0ce0c42e1c33c4042aca4
This commit changes the 2015.07 Guideline from "approved"
to "superseded" and changes the 2016.08 Guideline from "draft"
to "approved" status. This patch should not be merged until
the Board of Directors has approved the 2016.08 Guideline. Voting
is planned for the August 23 Board meeting.
Note: this change also regenerates documentation for all other
superceded Guidelines.
Change-Id: I630978bdff9c166a4a3340f8c26bcdcbf7013cbf
NOTE: This patch starts the process of creating a new
draft process for Board approval in Section C6.
As discussed in the board meeting, this change specifically
identifies that only the latest two guidelines are available
for vendor validation. Older guidelines are still approved
and must remain available because vendors have validated
against them (thus superseded not deprecated).
Change-Id: Id2c2a23c9a8e204480dcebde6fe436292120ba8a
The .rst Guideline files currently say that they're generated from
"the master JSON verison" but don't actually contain a link to their
JSON counterparts. This patch makes that reference a hyperlink so
folks can more easily locate the JSON files that are the authoritative
documents. This is especially useful when viewing rendered versions
of the RST docs.
The patch also adjusts the script we use to generate the RST docs
from their JSON sources so future Guidelines include similar
linkage.
While we're at it, we've also removed a chunk of code in the
RST generator script so that it doesn't rely on the "name" field
since we never really added those fields universally, which could
cause the script to fail. Instead we just use the "project" field.
Further, a few capabilities that are marked as removed are lacking
the aforementioned "project" field, so those have been added where
they were missing.
Following the changes to the RST generating script, it has been
re-run so that the existing 2015 Guideline RST documents have
the same formatting. While the formatting has changed (added links
and modified capitalization), the content has not (no capabilities
or tests added/removed).
Co-Authored-By: Mark T. Voelker <mvoelker@vmware.com>
Co-Authored-By: Rob Hirschfeld <rob@zehicle.com>
Change-Id: Idbdd7d6825aa8af2404b9e2791fa67240cdef239
In a previous pull request, it was agreeded to remove the tests
that needed the "stop" function available to end users.
https://review.openstack.org/#/c/180365/
While performing due dilligence, I realized there was one stop
test that I had missed during the previous commit, so I would
like to flag this test under the same reasoning used in
the previous request.
Change-Id: I2b405052f58be5d7ac48955ced5fcde7ea982e45
Concerns have been raised (by both community members and John Dickinson,
the Swift PTL) about a subset of the Swift tests that are currently
included that test ACL policy and authentication to objects in Swift.
John's reccomendation was to flag these tests as they are not testing
the Swift API and to remove them in the future.
Change-Id: I0f543ef8863abc0299545c20c4db7ef7aa930c0b
The Images v2 API ListImages test creates images to appear in the image
list, using the Glance image-create call to do this. There is, however,
an alternative route to image creation in the Images v2 API, namely, to
use an import task. A cloud provider may wish to restrict the standard
image-create to admin use only, for example, and restrict regular end
users to use of the import task (which allows an opportunity to perform
validation or security scanning on the image data before creating an
image in Glance). It is also reasonable for a provider not to allow
any upload of external images into a cloud and instead only allow image
creation via the Nova image-create server action. The key point is
that given that the ListImages test is supposed to test that the image
list functionality is present and working correctly, the test should be
agnostic as to how the images got into Glance in the first place. The
test should be flagged until it can be modified to take into account the
various legitimate restrictions that providers may wish to place on how
images are introduced into their OpenStack clouds.
This is also reported as tempest bug:
https://bugs.launchpad.net/tempest/+bug/1453265
Change-Id: I197d90f4644c4c558c50aa143f2ec57921a3985b
While executing the test_get_object_after_expiry_time test from the
ObjectExpiryTest, I experienced intermittent test failures. On
further investigation, I discovered that this test has a hard
coded "sleep for safety" step. While this value is sufficient
for test environments, there is higher latency and processing
time in a cloud under load. I've filed a Tempest issue
(https://bugs.launchpad.net/tempest/+bug/1452915) that
offers more reliable solutions to this problem, which I would
be more than glad to implement.
However, I can very rarely get this test to pass as is, as I
suspect others may be having as well. I'd ask to flag this test
while this Tempest issue is being worked through.
Change-Id: I2f1080ece519f1857d30f2aefa265a980757536d
In Nova, the various server actions can be limited by the
policy.json file deployed with the API. This policy
file provides the vendor with granular control over the actions
that can be performed by users in different roles. This means
that the “admin-ness” of a feature can be determined by
configuration. For example, in Rackspace’s OpenStack
public cloud the stop/start and lock/unlock behaviors are
not available to standard users and reserved for
administrators only. Because of this, it would be impossible
for these tests to run successfully in the scope of a
public user.
These two pairs of operations are only an example. It is
likely that other vendors would have similar constraints,
so for that reason I’m proposing that these tests be flagged.
Change-Id: I26e012b138e4c78d25f1dd57a4f845e82dc1fdb8
In the compute-servers sections of tests, there are four resize tests.
When using Nova's libvirt driver, and no shared-storage, resize relies
on transferring storage via ssh.
This requires password-less ssh to be setup between all compute nodes
in the cloud. This kind of setup may raise security concerns.
For example, resize is disabled by default in the HP Helion OpenStack
distribution because of this security concerns, as well as in the HP
Helion OpenStack public cloud.
This would cause any cloud/product sharing the same security concern to
fail or skip these tests, so I'm proposing they should be temporarily
flagged.
The blueprint [0], when implemented, would address the security concerns,
so it would be the first step to remove the flag from resize tests.
[0] https://blueprints.launchpad.net/nova/+spec/migrate-libvirt-volumes
Change-Id: I9273b7edac000007225daa7e0b7f03ef0b6e23ed
In the Compute tests, there are tests that need to generate
images as a prerequisite before the main tests can begin.
Since these are Compute tests, I would expect that these
prerequisites would be via the Compute API, but they use
the Glance v1 API instead. Since the capability being tested
should be Compute in isolation, I would say this test should
be flagged until the depdency on Glance v1 is removed.
Change-Id: I122034315463589ab81d5c549da6142e71c91a0e
This test currently has a bug that occurs when the Tempest
is configured to not use its isolated tenant feature.
This feature creates new accounts on the fly for individual
tests. This feature can be disabled if the test environment
does not allow for generation of accounts on the fly. This
bug emerges when the Tempest configuration is set to not
allow tenant isolation, but the test overrides that
configuration. From the comments in the Tempest code,
the expected behavior in this situation is that the
test should be skipped
(https://github.com/openstack/tempest/blob/master/tempest/common/credentials.py#L31).
Instead, an exception is raised, which causes the test to
erroneously fail.
I have filled issue with Tempest concerning this issue
(https://bugs.launchpad.net/tempest/+bug/1452118).
I propose that this test be flagged until this bug is resolved.
Change-Id: Ic0f043b8248620eade9af1f9a9033a027a0cbaec
In the compute-servers sections of tests, the
ServerActionsTestJSON.test_get_console_output test fails on some Nova
backends because it's not a universally supported feature. For example,
it isn't supported on VMware:
http://git.io/vJIAj
This would cause any cloud/product using such a backend to fail DefCore
tests, so the test should be flagged.
Change-Id: Ie3778cf8d1100fdb0a9065fabfb8254e0a7747b1
When the new 2015.04.json capabilities list was created,
the flagged broken tests were not moved forward to the new
file.
This patch reconciles the flagged test lists between the
two standards.
Change-Id: I0513bd41c623060cfed15d57e86994722d581597
Not sure how we missed this one while we were doing
the more complex 2015.next -> 2015.04 changes.
This guideline WAS approved by the board and the
status needs to be updated to reflect that.
Change-Id: I829d898876a444e4381e907d36e83ba612528cd8
As per the process, we want to use generated RST
instead of hand created files.
In generating the files, I found a comma error in 2015.03
and removed spaces from 2015.04 so there are minor
changes to those files.
Because 2015.next is still in process, it does not make
sense to generate the RST file at this time. We do
NOT want to have to maintain multiple files for each
change. I've removed the 2015.next.rst file.
Also added check to make sure we're using the
correct JSON source using the hardcoded DefCore
repo path as an extra check.
Change-Id: I443f4c1cb84eb2c42ad9e6914cee0ce6d498a9bb
As per DefCore discussion, I've moved the DS JSON over
to the guidelines.
Unlike the original version, I've added the concept
of required, advisory, depricated, removed and
informational so that it matches the process flows.
To be complete, I've done it to all three Guidelines
since they were all the same.
Change-Id: Ia5a906df5228bf34bb24c96e730e2452743a17c8
Also, correct JSON denormalization of the missed
guidelines addition for 2015.04 for 2015.04 and 2015.next
Change-Id: Ia9473b50d45d436c136ca58147229bd1fd130bbf
PS: we may want to drop the denormalization (patch?)
The board approved adding Juno to the guideline at the
2015 Apr 2 meeting by unanimous consent.
With history on both files!
Change-Id: Ic5833667f7a76176036b5dbbf3ee9884d937ed05
ArkadyKanevsky