From c384bc4ae0064673c1507ef668ea1e91fc34f6d4 Mon Sep 17 00:00:00 2001 From: Felipe Monteiro Date: Tue, 10 Jul 2018 20:14:04 -0400 Subject: [PATCH] trivial: Fix typos in the test_removal documentation page This patchset fixes typos (capitalization mistakes, grammar mistakes) found in the test_removal.rst documentation page. Change-Id: Ia9a6b9d7a2a151a460a620be9fa6875f2efda00f --- doc/source/test_removal.rst | 66 ++++++++++++++++++------------------- 1 file changed, 33 insertions(+), 33 deletions(-) diff --git a/doc/source/test_removal.rst b/doc/source/test_removal.rst index ddae6e2ab7..b22503b7ad 100644 --- a/doc/source/test_removal.rst +++ b/doc/source/test_removal.rst @@ -1,21 +1,21 @@ Tempest Test Removal Procedure ============================== -Historically tempest was the only way of doing functional testing and -integration testing in OpenStack. This was mostly only an artifact of tempest +Historically, Tempest was the only way of doing functional testing and +integration testing in OpenStack. This was mostly only an artifact of Tempest being the only proven pattern for doing this, not an artifact of a design -decision. However, moving forward as functional testing is being spun up in -each individual project we really only want tempest to be the integration test -suite it was intended to be; testing the high level interactions between -projects through REST API requests. In this model there are probably existing -tests that aren't the best fit living in tempest. However, since tempest is +decision. However, moving forward, as functional testing is being spun up in +each individual project, we really only want Tempest to be the integration test +suite it was intended to be: testing the high-level interactions between +projects through REST API requests. In this model, there are probably existing +tests that aren't the best fit living in Tempest. However, since Tempest is largely still the only gating test suite in this space we can't carelessly rip out everything from the tree. This document outlines the procedure which was developed to ensure we minimize the risk for removing something of value from -the tempest tree. +the Tempest tree. -This procedure might seem overly conservative and slow paced, but this is by -design to try and ensure we don't remove something that is actually providing +This procedure might seem overly conservative and slow-paced, but this is by +design to try to ensure we don't remove something that is actually providing value. Having potential duplication between testing is not a big deal especially compared to the alternative of removing something which is actually providing value and is actively catching bugs, or blocking incorrect patches @@ -27,24 +27,24 @@ Proposing a test removal 3 prong rule for removal ^^^^^^^^^^^^^^^^^^^^^^^^ -In the proposal etherpad we'll be looking for answers to 3 questions +In the proposal etherpad we'll be looking for answers to 3 questions: #. The tests proposed for removal must have equiv. coverage in a different project's test suite (whether this is another gating test project, or an in tree functional test suite). For API tests preferably the other project will - have a similar source of friction in place to prevent breaking api changes - so that we don't regress and let breaking api changes slip through the + have a similar source of friction in place to prevent breaking API changes + so that we don't regress and let breaking API changes slip through the gate. #. The test proposed for removal has a failure rate < 0.50% in the gate over the past release (the value and interval will likely be adjusted in the future) .. _`prong #3`: -#. There must not be an external user/consumer of tempest +#. There must not be an external user/consumer of Tempest that depends on the test proposed for removal The answers to 1 and 2 are easy to verify. For 1 just provide a link to the new -test location. If you are linking to the tempest removal patch please also put +test location. If you are linking to the Tempest removal patch please also put a Depends-On in the commit message for the commit which moved the test into another repo. @@ -91,32 +91,32 @@ you would run the following: #. paste the output table with numbers and the mysql command you ran to generate it into the etherpad. -Eventually a cli interface will be created to make that a bit more friendly. +Eventually a CLI interface will be created to make that a bit more friendly. Also a dashboard is in the works so we don't need to manually run the command. The intent of the 2nd prong is to verify that moving the test into a project -specific testing is preventing bugs (assuming the tempest tests were catching -issues) from bubbling up a layer into tempest jobs. If we're seeing failure +specific testing is preventing bugs (assuming the Tempest tests were catching +issues) from bubbling up a layer into Tempest jobs. If we're seeing failure rates above a certain threshold in the gate checks that means the functional testing isn't really being effective in catching that bug (and therefore -blocking it from landing) and having the testing run in tempest still has +blocking it from landing) and having the testing run in Tempest still has value. However for the 3rd prong verification is a bit more subjective. The original intent of this prong was mostly for refstack/defcore and also for things that running on the stable branches. We don't want to remove any tests if that -would break our api consistency checking between releases, or something that -defcore/refstack is depending on being in tempest. It's worth pointing out +would break our API consistency checking between releases, or something that +defcore/refstack is depending on being in Tempest. It's worth pointing out that if a test is used in defcore as part of interop testing then it will -probably have continuing value being in tempest as part of the +probably have continuing value being in Tempest as part of the integration/integrated tests in general. This is one area where some overlap -is expected between testing in projects and tempest, which is not a bad thing. +is expected between testing in projects and Tempest, which is not a bad thing. Discussing the 3rd prong """""""""""""""""""""""" There are 2 approaches to addressing the 3rd prong. Either it can be raised -during a qa meeting during the tempest discussion. Please put it on the agenda +during a qa meeting during the Tempest discussion. Please put it on the agenda well ahead of the scheduled meeting. Since the meeting time will be well known ahead of time anyone who depends on the tests will have ample time beforehand to outline any concerns on the before the meeting. To give ample time for @@ -133,17 +133,17 @@ nature of ML. Exceptions to this procedure ---------------------------- -For the most part all tempest test removals have to go through this procedure +For the most part all Tempest test removals have to go through this procedure there are a couple of exceptions though: -#. The class of testing has been decided to be outside the scope of tempest. +#. The class of testing has been decided to be outside the scope of Tempest. #. A revert for a patch which added a broken test, or testing which didn't actually run in the gate (basically any revert for something which shouldn't have been added) #. Tests that would become out of scope as a consequence of an API change, as described in `API Compatibility`_. Such tests cannot live in Tempest because of the branchless nature of - Tempest. Such test must still honor `prong #3`_. + Tempest. Such tests must still honor `prong #3`_. For the first exception type the only types of testing in tree which have been declared out of scope at this point are: @@ -160,8 +160,8 @@ there is equivalent testing elsewhere. Tempest Scope ^^^^^^^^^^^^^ -Starting in the liberty cycle tempest has defined a set of projects which -are defined as in scope for direct testing in tempest. As of today that list +Starting in the liberty cycle Tempest, has defined a set of projects which +are defined as in scope for direct testing in Tempest. As of today that list is: * Keystone @@ -171,18 +171,18 @@ is: * Neutron * Swift -anything that lives in tempest which doesn't test one of these projects can be +Anything that lives in Tempest which doesn't test one of these projects can be removed assuming there is equivalent testing elsewhere. Preferably using the `tempest plugin mechanism`_ -to maintain continuity after migrating the tests out of tempest. +to maintain continuity after migrating the tests out of Tempest. .. _tempest plugin mechanism: https://docs.openstack.org/tempest/latest/plugin.html API Compatibility """"""""""""""""" -If an API introduces a non-discoverable, backward incompatible change, and -such change is not backported to all versions supported by Tempest, tests for +If an API introduces a non-discoverable, backward-incompatible change, and +such a change is not backported to all versions supported by Tempest, tests for that API cannot live in Tempest anymore. This is because tests would not be able to know or control which API response to expect, and thus would not be able to enforce a specific behavior.