Merge "Clarify testing for interop programs"

resolutions/20180307-trademark-program-test-location.rst

@@ -0,0 +1,73 @@
+.. _201180307_interop_test_location:
+
+=======================================================================
+ 2018-03-07 Location of tempest tests for OpenStack Trademark Programs
+=======================================================================
+
+Introduction
+============
+
+When :ref:`20160504_defcore_test_location` was proposed, the trademark program
+was simpler, involving only a small subset of the projects available in
+OpenStack.
+
+With the proposed "add on" and "vertical" trademark programs, more projects are
+involved. This growth introduces some scaling problems into the management of
+tests for the trademark programs. These issues were discussed at the Rocky PTG.
+The outcome of those discussions are reported here.
+
+This resolution supersedes :ref:`20160504_defcore_test_location`.
+
+Updated Resolution
+==================
+
+Whereas the Trademark Program has made it clear that they have the tooling and
+willingness to support trademark tests located in any of
+
+* The `Tempest repository`_
+* Project specific tempest plugins (e.g., `image-trademark-tempest-plugin`)
+* Vertical specific tempest plugins (e.g.,
+  `nfv-vertical-trademark-tempest-plugin`)
+* A global Trademark tempest plugin (e.g.,
+  `openstack-trademark-tempest-plugin`)
+
+as long as they are tests that can be run by tempest.
+
+And the QA Team, Trademark Program and individual projects have shared
+responsibility for the management of trademark tests, with varying degrees of
+bandwidth for review attention.
+
+And trademark tests are expected to be changed rarely.
+
+It is resolved that trademark tests (and candidates thereof) for individual,
+projects should live in one, and only one, of four possible locations, each
+with differing strategies for review and repository ownership (status in
+:ref:`projects`):
+
+* The `Tempest repository`_, with approval reviews performed by the QA Team,
+  repository ownership via the QA Team (already true).
+* A vertical specific tempest plugin, with approval reviews performed by the QA
+  Team and the Trademark Program, repository ownership via the Trademark
+  Program.
+* A global Trademark tempest plugin, with approval reviews performed by the QA
+  Team and the Trademark Program, repository ownership via the Trademark
+  Program.
+* A project specific tempest plugin, specifically limited to trademark-oriented
+  tests, with approval reviews performed by the project, the QA Team, and the
+  Trademark Program, repository ownership via the Trademark Program if they
+  accept, otherwise the project team.
+
+The choice is up to the individual projects, in consultation with the QA Team
+and the Trademark Program.
+
+.. note:: No overarching direction is provided here on what tooling must be
+          used in the tests but it is generally expected that the tooling
+          should be aggreed by the people who are reviewing the code. If a
+          project wants to use tooling other than the core tempest libraries
+          (e.g., gabbi_) they should prefer a project specific plugin and be
+          aware that their choice of tooling may limit the ability for
+          non-project parties to review tests. Any tooling chosen must be able
+          to produce tests that are run by tempest.
+
+.. _gabbi: https://gabbi.readthedocs.io/
+.. _Tempest repository: http://git.openstack.org/cgit/openstack/tempest

resolutions/superseded/20160504-defcore-test-location.rst

@@ -1,7 +1,12 @@
+.. _20160504_defcore_test_location:
+
 ============================================================
  2016-05-04 Recommendation on Location of Tests for DefCore
 ============================================================
 
+.. note:: This resolution has been superseded by 
+          ref:`201180305_interop_test_location`
+
 At the Newton summit discussion about `QA, DefCore, and
 interoperability testing`_ the DefCore committee asked for direction
 about where they should look for tests to consider for verifying