From d8fb682a2bd17c111f5dd4f8341fb1b300ac2d4d Mon Sep 17 00:00:00 2001 From: "Mark T. Voelker" Date: Wed, 8 Jun 2016 09:07:44 -0400 Subject: [PATCH] Change doc references to DefCore At the Board of Directors meeting on April 24, 2016 the Board of Directors indicated that as DefCore has evolved it's focus on interoperability and it's working structure, it's name may be a source of some confusion to those outside of the community or who are new to the community. The Board made an informal request that the DefCore Committee consider changing it's name to more clearly reflect it's focus and structure. This patch is the first step in that process. It updates references in various documents to change the name "DefCore Committee" to "Interop Working Group" as agreed at the summer 2016 DefCore Committee Sprint [1]. It should be noted that this patch should be considered a work in progress to generate discussion until the new name is approved by the DefCore Committee and the Board of Directors. Should we elect to go forward with the new name, some other actions will also need to be taken, including but not limited to: 1. We will need to consider updating references on the OpenStack wiki. 2. We will need to consider updating the name of our IRC channel, mailing list, Launchpad project, and git repository. Most of these changes will need to be carefully coordinated with the OpenStack Infrastructure team. 3. We will need to take into account external resources that point to DefCore artifacts, such as Foundation-maintained websites (such as: http://www.openstack.org/interop ). 4. We will need to coordinate with RefStack to minimize impact. 5. We will need to clearly communicate the name change to the rest of the community. Note also that I've intentionally left many historical documents that have been superceded (such as the 2015A process docs, Guidelines that are no longer used, etc) in tact. There seemed little value in spending time on them and cluttering the patch with them since they're now obsolete. [1] https://etherpad.openstack.org/p/DefCoreSummer2016Sprint Change-Id: I79d337c193e75c54d49f1d847468f6347e2ef2b3 --- HACKING.rst | 48 +-- README.rst | 27 +- doc/source/conf.py | 2 +- doc/source/guidelines/next.rst | 10 +- doc/source/process/2017A.rst | 314 ++++++++++++++++++ doc/source/process/CoreCriteria.rst | 23 +- doc/source/process/CoreDefinition.rst | 4 +- doc/source/process/DesignatedSections.rst | 10 +- doc/source/process/GovernanceProcess.rst | 8 +- doc/source/process/Lexicon.rst | 55 +-- doc/source/process/PlatformCap.rst | 4 +- doc/source/process/ProcessCycles.rst | 8 +- doc/source/schema/1.6.rst | 4 +- next.json | 4 +- setup.cfg | 6 +- tools/jsonToRst.py | 4 +- .../additional_properties_waiver.rst | 34 +- ...{DefCoreSpec.rst => interop_test_spec.rst} | 7 +- working_materials/scoring.txt | 66 +++- working_materials/tabulate_scores.py | 2 +- 20 files changed, 503 insertions(+), 137 deletions(-) create mode 100644 doc/source/process/2017A.rst rename working_materials/{DefCoreSpec.rst => interop_test_spec.rst} (93%) diff --git a/HACKING.rst b/HACKING.rst index d57592ac..61792d11 100644 --- a/HACKING.rst +++ b/HACKING.rst @@ -1,22 +1,23 @@ -DefCore Style Commandments -========================== +Interop Working Group Commandments +=================================== - Step 1: Read the OpenStack Style Commandments http://docs.openstack.org/developer/hacking/ -- Step 2: Read the following DefCore process documents in the following recommended order: +- Step 2: Read the following Interop Working Group process + documents in the following recommended order: - `Core Definition `_ - - `OpenStack DefCore Process 2015A `_ + - `OpenStack Interop WG Process 2016A `_ - Step 3: Read on -DefCore Specific Commandments ------------------------------ +Interop Working Group Specific Commandments +-------------------------------------------- - [D300] When adding tests to "flagged" lists, generally only the most current Board-approved .json file and the .next.json file should be - modified. There is no need to modify older guidelines unless the most - current Board-approved guideline doesn't cover the OpenStack release + modified. There is no need to modify older Guidelines unless the most + current Board-approved Guideline doesn't cover the OpenStack release you are concerned with. - [D301] The "tests" lists in the .json capabilities lists are immutable once approved by the Board. Therefore if you desire to flag a test, @@ -42,9 +43,9 @@ DefCore Specific Commandments field within newly required capabilities. The patch should include the matching generated RST version of the JSON file. This patch should be marked as -1 workflow until after approval. -- [D305] DefCore guidelines generally cover the most recent three - releases of OpenStack, though the DefCore Committee has the power to - determine otherwise. The "releases" section of the .next.json file +- [D305] Interop Guidelines generally cover the most recent three + releases of OpenStack, though the Interop Working Group has the + power to determine otherwise. The "releases" section of the .next.json file should generally be updated shortly after the Board approves a release so that contributors can see what releases the proposed Guideline targets. @@ -62,7 +63,7 @@ DefCore Specific Commandments - [D309] The "reason" field of the "flagged" section must begin with the flag type. For example: - ``"reason" : "[D400] The Foo test doesn't meet DefCore criteria because ..."`` + ``"reason" : "[D400] The Foo test doesn't meet Core Criteria because ..."`` - [D310] If you believe a test needs to be flagged but the reason for doing so doesn't appear in the list below, you must do the following: @@ -76,22 +77,23 @@ DefCore Specific Commandments file adding your proposed new flag to the list below. #. If at all possible, please include a link to code and/or test runs which demonstrate the reason a new flag type is needed. - #. The DefCore committee will discuss and consider the flagging proposal as - well as the proposed new reason. They may accept or decline either proposal. + #. The Interop Working Group will discuss and consider the flagging + proposal as well as the proposed new reason. They may accept or decline + either proposal. - [D311] Once a test has been flagged, it will remain flagged for that Guideline. -- [D312] When a new guideline is proposed for Board approval, no flagged tests - will be included in the guideline. Flags will be added in subsequent patches. +- [D312] When a new Guideline is proposed for Board approval, no flagged tests + will be included in the Guideline. Flags will be added in subsequent patches. -DefCore Test Flagging Guidelines --------------------------------- +Interop Working Group Test Flagging Guidelines +----------------------------------------------- -The DefCore Committee may "flag" tests to mark them as not required for a -given Guideline. There are different flag types; each flag type indicates a -fairly narrow category of reasons for flagging a given test. +The Interoper Working Group may "flag" tests to mark them as not +required for a given Guideline. There are different flag types; each flag +type indicates a fairly narrow category of reasons for flagging a given test. Valid reasons for flagging a test are limited to the following: -- [D400] The test is for a Capability that fails to meet DefCore Criteria +- [D400] The test is for a Capability that fails to meet the Criteria as set out in the `Core Criteria document `_. - [D401] The test fails or is skipped due to a bug in the test and the bug is @@ -99,7 +101,7 @@ Valid reasons for flagging a test are limited to the following: - [D402] The test fails or is skipped due to a bug in the code that provides the Capability and the bug is accepted by the OpenStack project which maintains the Capability. -- [D403] The test fails because other non-DefCore Capabilities are also +- [D403] The test fails because other non-required Capabilities are also tested. - [D404] Flag Not Found - Use this flag if none of the others fit. - [D405] The test reflects an implementation choice that is not widely diff --git a/README.rst b/README.rst index 2dc57777..e9b0876e 100644 --- a/README.rst +++ b/README.rst @@ -1,32 +1,35 @@ ================================================= -Understanding the DefCore Guidelines +Understanding the Interoperability Guidelines ================================================= -This repository contains DefCore committee managed files that provide guidance -for the OpenStack community. +This repository contains files managed by the Interop Working +Group that provide guidance for the OpenStack community. -NOTE: Changes to file requires approval of the DefCore committee chair(s). +NOTE: Changes to file requires approval of the Interop Working +Group chair(s). -DefCore Process Documentation -============================= +Interop Working Group Process Documentation +============================================ -Want to certify as a vendor? Consult `OpenStack Interop +Are you a vendor who wants to get a license to use the OpenStack trademark +and logo? Consult `OpenStack Interop `_. -The /doc/source/process directory contains details about the DefCore process. +The /doc/source/process directory contains details about the +Interop Working Group process. The /doc/source/schema directory contains details about the JSON schema versions used to express Guidelines. -The /doc/source/guidelines directory contains RST versions of the DefCore -Guidelines approved by the OpenStack Board of Directors. +The /doc/source/guidelines directory contains RST versions of the +Interop Guidelines approved by the OpenStack Board of Directors. :Core Definition: doc/source/process/CoreDefinition.rst :Process Goverance: doc/source/process/2015A.rst (please check for latest) :Designated Sections: doc/source/process/DesignatedSections.rst :Core Criteria: doc/source/process/CoreCriteria.rst -:DefCore Governance: doc/source/process/GovernanceProcess.rst +:Interop WG Governance: doc/source/process/GovernanceProcess.rst :Platform and Components: doc/source/process/PlatformCap.rst -:DefCore Cycles: doc/source/process/ProcessCycles.rst +:Interop WG Cycles: doc/source/process/ProcessCycles.rst :Terminology: doc/source/process/Lexicon.rst diff --git a/doc/source/conf.py b/doc/source/conf.py index b1a0d864..6519a783 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -184,7 +184,7 @@ latex_elements = { # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ - ('index', 'defcore.tex', u'OpenStack DefCore Documentation', + ('index', 'defcore.tex', u'OpenStack Interop WG Documentation', u'OpenStack Foundation', 'manual'), ] diff --git a/doc/source/guidelines/next.rst b/doc/source/guidelines/next.rst index 33799d54..c643e717 100644 --- a/doc/source/guidelines/next.rst +++ b/doc/source/guidelines/next.rst @@ -1,9 +1,9 @@ -====================== -OpenStack DefCore next -====================== +========================================= +OpenStack Interoperability Guideline next +========================================= :Status: draft -:Replaces: 2016.01 +:Replaces: 2016.08 :JSON Master: http://git.openstack.org/cgit/openstack/defcore/tree/next.json This document outlines the mandatory capabilities and designated @@ -14,7 +14,7 @@ This document was generated from the `master JSON version `_. Releases Covered ============================== -Applies to Kilo, Liberty, Mitaka, Newton +Applies to Liberty, Mitaka, Newton, Ocata Platform Components ============================== diff --git a/doc/source/process/2017A.rst b/doc/source/process/2017A.rst new file mode 100644 index 00000000..a89217a5 --- /dev/null +++ b/doc/source/process/2017A.rst @@ -0,0 +1,314 @@ +OpenStack Interop Working Group Process 2017A +============================================== + +:Status: Draft +:Replaces: 2016A + +This document describes the Interop Working Group's working +process required by the OpenStack bylaws and approved by the OpenStack +Technical Committee and Board. + +Expected Time line: +--------------------------------------- + ++------------+-----------+--------------------------------------+-----------+ +| Time Frame | Milestone | Activities | Lead By | ++============+===========+======================================+===========+ +| -3 months | S-3 | Draft status | Interop WG| ++------------+-----------+--------------------------------------+-----------+ +| -2 months | S-2 | ID new Capabilities | Community | ++------------+-----------+--------------------------------------+-----------+ +| -1 month | S-1 | Score Capabilities | Interop WG| ++------------+-----------+--------------------------------------+-----------+ +| Summit | S | Review status | Community | ++ + +--------------------------------------+-----------+ +| | | Advisory/Deprecated items selected | Interop WG| ++------------+-----------+--------------------------------------+-----------+ +| +1 month | S+1 | Self-testing | Vendors | ++------------+-----------+--------------------------------------+-----------+ +| +2 months | S+2 | Test Flagging | Interop WG| ++------------+-----------+--------------------------------------+-----------+ +| +3 months | S+3 | Approve Guidance | Board | ++------------+-----------+--------------------------------------+-----------+ + +Note: The Interop Working Group may accelerate the process to correct errors +and omissions. + +Process Definition +-------------------------------------- + +The Guideline process has two primary phases: Draft and Review. +During the Draft phase (A), the Interop Working Group is working +with community leaders to update and score the components of the Guideline. +During the Review phase (B), the general community and vendors have an +opportunity to provide input and check the Guidelines (C) against actual +implementations. The Review phase ends with Board approval of the draft +Guideline (D). + +This section provides specific rules and structure for each phase. + +NOTE: To ensure continuity of discussion, process components defined below +must _not_ reuse numbers in future revisions. The numbering pattern +follows draft, section and sub-item numbering, e.g.: 2015A.B2.2. This +requirement may create numbering gaps in future iterations that will help +indicate changes. + +Guidelines Draft Phase (A) +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Starting: S-3 + +A1. New Guidelines Start From Previous Guidelines + + 1. New Guidelines start from the previous Board approved document. + 2. New Guidelines are given the preliminary name of "next.json". + +A2. Community Groups Tests into Capabilities + + 1. The Interop Working Group coordinates community + activities with the Technical Leadership to revise the capabilities + based on current technical needs and functionality. + 2. Capabilities must correspond to projects which are part of the + "TC-approved release" as designated by the TC (see bylaws of the + Foundation, section 4.1(b)(iii)). + 3. Groupings may change between iterations. + 4. Tests must have unique identifiers that are durable across releases + and changes in grouping. + 5. Tests must be under OpenStack Technical Committee governance. + 6. The Interop Working Group will provide the test groupings + in JSON format for scoring. + 7. The Interop Working Group will provide a human-readable summary + of the Guideline generated from the JSON version. + +A3. Interop Working Group Collects Recommendations for + Designated Sections + + 1. Designated Sections will not be removed without being deprecated in the + previous Guideline. + 2. Designated Sections will not be added without being advisory in the + previous Guideline. + 3. Designated Sections will not be added or be made advisory unless the + corresponding code base is designated as part of the "TC-approved release" + by the Technical Committee (see bylaws of the Foundation, section + 4.1(b)(iii)). + 4. Technical leadership may, but is not required to, assist the + Interop Working Group with defining advisory Designated + Sections for projects that have advisory or required Capabilities. + 5. Designated Sections may be sufficiently defined for Guidelines using + general descriptions. + 6. The Interop Working Group will present A3.4 descriptions to + the Board for approval. + 7. Technical leadership may, but is not required to, provide more specific + details describing the Designated Sections for a project. + 8. Designated Sections will be included in the JSON Guideline. + +A4. Interop Working Group identifies required capabilities + + 1. The Interop Working Group uses Board approved scoring + criteria to evaluate Capabilities. + 2. The Interop Working Group needs Board approval to change + scoring criteria. + 3. Scoring criteria factors or weights cannot change after Draft is + published. + 4. The Interop Working Group identifies a cut-off score for + determining that a Capability is required. + 5. Capabilities will not be removed without being deprecated in the + previous Guideline. + 6. Capabilities will not be added without being advisory in the previous + Guideline. + 7. For the "widely deployed" adoption criteria, the size of the + pool being considered will match the scope of the community being + considered. Capabilities will be evaluated based on their use in their + Component. Components will be evaluated based on their use in the + Platform. + +A5. Foundation Staff recommends OpenStack Components and OpenStack Platform + Scope + + 1. Foundation Staff recommends Capabilities to include in each OpenStack + Component. + 2. Foundation Staff recommends which Components are required for + the OpenStack Powered Platform. + 3. To support the Foundation recommendation, the Interop + Working Group will apply the approved scoring criteria to + evaluate if a component should be included in the + Platform (see A4). + +A6. Additional Capabilities and Tests + + 1. The Interop Working Group will work with the community to + define new Capabilities. + 2. Test grouping for new Capabilities will be included in the + Interop Working Group documents. + 3. The Interop Working Group will publish a list of missing + Capabilities and Capabilities with inadequate test coverage. + +A7. The Interop Working Group creates recommendation for Draft. + + 1. The Interoper Working Group coordinates activities to create + the Guideline draft. + 2. The Interop Working Group may choose to ignore recommendations + with documented justification. + +Guidelines Review Phase (B) +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Starting: Summit + +B1. All Reference Artifacts are reviewed via Gerrit + + 1. Draft Guideline + 2. Designated sections + 3. Test-Capability groupings + 4. Flagged Test List + 5. Capability Scoring criteria and weights + 6. May not be in Gerrit: Working materials (spreadsheets, etc) + +B2. Presentation of Draft Guidelines for Review + + 1. The Interop Working Group will present Draft Guidelines to + the Board for review. + 2. The Interop Working Group will distribute Draft Guidelines + to the community for review. + 3. Foundation Staff will provide Draft Guidelines to vendors for review. + 4. A link to the Gerrit document must be provided with the review materials. + +B3. Changes to Guideline made by Gerrit Review Process + + 1. Community discussion including vendors must go through Gerrit. + 2. All changes to draft must go through Gerrit process. + 3. The Interop Working Group will proxy for users who do not + participate in the Gerrit process with attribution. + +B4. For Gerrit reviews, Interop Working Group Co-Chairs act as + Joint PTLs + + 1. Interop Working Group Co-Chairs serve as "core" reviewers (+2). + 2. Requests for changes must be submitted as patches by the requesting + party. + 3. Interop Working Group members may proxy change requests as + long as the requesting party is explicitly acknowledged. + 4. One Interop Working Group Co-Chair must be Board member. + 5. One Interop Working Group Co-Chair will be elected by the + Interop Working Group. Election quorum is composed of + attendees present during the election meeting. + 6. Additional core reviewers (+2) can be appointed by Co-Chairs. + 7. Election meetings must be posted at least one meeting prior. + +Community Review & Vendor Self-Test (C) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Starting: S and continues past S+3 + +C1. Vendor Self-Tests + + 1. Vendors are responsible for executing tests identified by the + Interop Working Group. + 2. The Foundation may, but is not required to, provide tooling for + running tests. + 3. The Foundation may, but is not required to, define a required + reporting format. + 4. Self-test results may be published by Vendors in advance of Foundation + review, but must be clearly labeled as "Unofficial Results - Not Yet + Accepted By The OpenStack Foundation". + 5. Vendors who publish self-tests MUST provide them in the same format that + would be submitted to the OpenStack Foundation but MAY provide additional + formats if they choose to do so. + 6. Self-test results cannot be used as proof of compliance. + +C2. Vendor submits results to Foundation for review + + 1. The Foundation determines the acceptable format for submissions. + 2. The Foundation has final authority to determine if Vendor meets + criteria. + 3. The Foundation will provide a review of the results within 30 days. + +C3. Vendor Grievance Process + + 1. Vendors may raise concerns with specific tests to the Interop + Working Group. + 2. The Interop Working Group may choose to remove tests from + a Guideline (known as flagging). + 3. The Interop Working Group will acknowledge vendor requests + to flag tests within 30 days. + +C4. Results of Vendor Self-Tests will be open + + 1. The Foundation will make the final results of approved vendors + available to the community. + 2. The Foundation will not publish incomplete or unapproved results. + 3. Only "pass" results will be reported. Skipped and failed results will + be omitted from the reports. + 4. Reports will include individual test results, not just Capability + scoring. + 5. Vendors are required to submit a description of the system and + configuration used to achieve the results. + 6. The Foundation may require vendors to submit specific details of the + configuration and may also require use of a specific format for + reporting. + +C5. API Usage Data Report + + 1. The Foundation will provide Interop Working Group with an + open report about API usage based on self-tests. + 2. To the extent the data is available, Capabilities beyond the + Interoperability Guideline list will be included in the report. + +C6. Only Two Approved Guidelines at a time: + + 1. Vendors seeking Foundation validation are limited to using the two + latest approved Guidelines. + + 2. Since past validations are respected, older Guidelines will be + maintained as superseded for historical reference. + + 3. Guideline status progresses as follows: + + :draft: initial work, pre-summit (S-3) discussion material + :review: as presented at summit (S) for community review + :approved: board approved, one of the two official guidelines + :superseded: board approved, now superseded by two latest guidelines + +Guideline Approval (D) +^^^^^^^^^^^^^^^^^^^^^^ + +Starting: S+3 + +D1. Board will review and approve Interoperability Guidelines from draft + + 1. Guidelines are set at the Platform, Component and Capability level + only. + 2. The Interop Working Group will submit the human-readable + summary of Capabilities (see section A2[6]) to the Board for approval. + 3. By voting to approve the summary, the Board delegates responsibility + for maintaining test groupings to the Interop Working Group + subject to the limitations described in section D2. + 4. Guidelines only apply to the identified releases (a.k.a. release + tags). + +D2. Interop Working Group has authority on test categorization + + 1. The Interop Working Group can add flagged tests before and + after Guideline approval. + 2. The Interop Working Group cannot add additional Tests to + Capability mappings after approval. + 3. The Interop Working Group maintains the test to Capability + mappings in the JSON representation. + +D3. Designated Sections only enforced for projects with required Capabilities + + 1. Designated Sections may be defined for any project. + 2. Designated Sections apply to the releases (a.k.a. release tags) + identified in the Guideline. + 3. Designated Sections will be included in the JSON Capabilities file + to ensure a single source of identification. + +D4. Guidelines are named based on the date of Board approval + + 1. Naming pattern will be: 4-digit year, dot (period), and 2-digit month. + + +Functional Information +---------------------- +:Format: RestructuredText +:Layout: 1.0 diff --git a/doc/source/process/CoreCriteria.rst b/doc/source/process/CoreCriteria.rst index 4919d42b..6b35cd76 100644 --- a/doc/source/process/CoreCriteria.rst +++ b/doc/source/process/CoreCriteria.rst @@ -19,8 +19,8 @@ categories: :System: the capability integrates with other parts of OpenStack These categories summarize critical values that we want in OpenStack and so -make sense to be the primary factors used when we select DefCore capabilities. -While we strive to make the DefCore process objective and quantitive, we +make sense to be the primary factors used when we select required Capabilities. +While we strive to make the process objective and quantitive, we must recognize that these choices drive community behavior. .. figure:: ../images/Defcore_capabilities_criteria.png @@ -60,11 +60,11 @@ ALIGNS WITH TECHNICAL DIRECTION capabilities. * **"Stable"** Capabilities are required to be stable for >2 releases because - we do not want DefCore capabilities that do not have dependable APIs. + we do not want required Capabilities that do not have dependable APIs. * **"Complete"** Where the code being tested has a designated area of alternate - implementation (extension framework) as per the DefCore Principles, there - should be parity in capability tested across extension implementations. + implementation (extension framework) as per the Interoperability Principles, + there should be parity in capability tested across extension implementations. This also implies that the capability test is not configuration specific or locked to non-open technology. @@ -78,9 +78,9 @@ PLAYS WELL WITH OTHERS This can be a very subjective measure and we expect to refine this definition over time. -* **"DefCore in Last Release"** A required capability should stay a required - capability. This makes DefCore capabilities sticky per release. Leaving - DefCore is disruptive to the ecosystem. +* **"Required in Last Release"** A required capability should stay a required + capability. This makes Capabilities sticky per release. Leaving + the Interoperability Guidelines is disruptive to the ecosystem. TAKES A SYSTEM VIEW ------------------- @@ -92,11 +92,12 @@ TAKES A SYSTEM VIEW required capabilities. * **"Proximity"** (sometimes called a Capability Cluster) selects for - capabilities that are related to DefCore capabilities. This helps ensure - that related capabilities are managed together. + capabilities that are related to other required capabilities. This helps + ensure that related capabilities are managed together. NON-ADMIN --------- The original 13th "non-admin" criteria has been removed because Admin -APIs cannot be used for interoperability and are not considered DefCore. +APIs cannot be used for interoperability and are not considered for end +users. diff --git a/doc/source/process/CoreDefinition.rst b/doc/source/process/CoreDefinition.rst index a4cf4231..41fbf90d 100644 --- a/doc/source/process/CoreDefinition.rst +++ b/doc/source/process/CoreDefinition.rst @@ -19,8 +19,8 @@ suggest changes to the by-laws to clarify the definition of core. Implementation ============== -* The `Governance/DefCoreCommittee - `_ is +* The `Governance/InteropWG + `_ is working to manage this. * Meetings and agendas will be posted to that page, hosted on Google Hangouts and (generally) open to the community. diff --git a/doc/source/process/DesignatedSections.rst b/doc/source/process/DesignatedSections.rst index d56918bf..deb73f2f 100644 --- a/doc/source/process/DesignatedSections.rst +++ b/doc/source/process/DesignatedSections.rst @@ -16,9 +16,9 @@ Designated Sections Selection Guidance _Approved 2014 Dec 2_ -There DefCore committee identified 10 selection criteria. The first seven are -technical from the TC and last three allow the Board to resolve issues without -needed a technical judgement. +The Interop Working Group identified 10 selection criteria. The first +seven are technical from the TC and last three allow the Board to resolve +issues without needing a technical judgement. 1. Designated if the code provides the project external REST API @@ -64,13 +64,13 @@ needed a technical judgement. 2. The goal is guidance on where we want upstream contributions not a code inspection police state. Guidance will be revised per release - as part of the DefCore process. + as part of the Interop Working Group process. Designated Sections =================== Effective April 2015, approved Designated Sections are maintained -in the Board approved DefCore Guidelines. The 2015.03 Guideline +in the Board approved Interoperability Guidelines. The 2015.03 Guideline was set to match the Board action of 2014 December 2. Please see the current Guidelines to determine which Designated diff --git a/doc/source/process/GovernanceProcess.rst b/doc/source/process/GovernanceProcess.rst index 508c2018..db7d0031 100644 --- a/doc/source/process/GovernanceProcess.rst +++ b/doc/source/process/GovernanceProcess.rst @@ -4,13 +4,13 @@ Governance Process * Meetings will be open to all community members. Information about how and when to join meetings will be posted on the defcore-committee@lists.openstack.org mailing list and/or the - `DefCore Committee wiki page - `_. + `Interop WG wiki page + `_. * Members are expected to to their homework. We will not be rehashing due to time limits. Minutes from prior meetings will be posted to the - `DefCore Committee wiki page - `_. + `Interop WG wiki page + `_. * Process documents and other "rules of the road" will be maintained and voted on in `Gerrit diff --git a/doc/source/process/Lexicon.rst b/doc/source/process/Lexicon.rst index a3ee6eb6..a040395f 100644 --- a/doc/source/process/Lexicon.rst +++ b/doc/source/process/Lexicon.rst @@ -1,9 +1,9 @@ -OpenStack DefCore Lexicon -========================================= +OpenStack Interop Working Group Lexicon +======================================== Licensing the OpenStack commercial-use marks requires passing tests of -DefCore required capabilities, and including designated sections of code. +required capabilities, and including designated sections of code. There are multiple marks available for vendors depending on which capability groupings are passed. @@ -11,17 +11,16 @@ TERMS: ---------------------------------------- Advisory - Capabilities that have been suggested for the next guideline. + Capabilities that have been suggested for the next Guideline. Capability The functionality ensured by a set of tests collected into - a group as defined by the DefCore committee Required - capabilities that - are required to meet the guideline. + a group as defined by the Interop Working Group. Certify or Accredit - DefCore does not do any of these things for OpenStack clouds. These - actions would fall under the governance of the Foundation trademark - policy. + The Interop Working Group does not do any of these things + for OpenStack clouds. These actions would fall under the + governance of the Foundation trademark policy. Community The universe of people and companies that are involved in the OpenStack @@ -41,25 +40,23 @@ Core uses. DefCore - The OpenStack board committee that manages commercial definition - of OpenStack for trademark purposes. - -DefCore Process - The process used by DefCore to score capabilities and - select criteria. + The original name for the OpenStack board committee that managed + commercial definition of OpenStack for trademark purposes. As the + group evolved, it's name was eventually changed to the Interop + Working Group. Deprecated - Capabilities that will be removed in the next guideline. + Capabilities that will be removed in the next Guideline. Designated Sections Portions of the OpenStack codebase that must be used to provide required capabilities in a product wishing to use the OpenStack - trademark. Designated sections fulfill one or more of the following + trademark. Designated sections fulfill one or more of the following criteria: they provide the project-external REST API, or are shared and provide common functionality for all options, or implement logic - that is critical for cross-platform operation. Designated sections + that is critical for cross-platform operation. Designated sections must exist in the OpenStack gerrit namespace and have corresponding - tests. Code that meets the following criteria will not be considered + tests. Code that meets the following criteria will not be considered designated: provides vendor-specific functionality, are explicitly intended by the project maintainers to be replaceable, extend the project REST API in a new or different way, or code that is being @@ -70,9 +67,13 @@ Flagged Test field and it not required for vendor self-test. Guidelines - Output of the DefCore process detailing which sections and - capabilities are required. Guidelines will be approved on a regular - cadence and identified by the date of approval. + Output of the Interop Working Group process detailing which + Designated Sections and Capabilities are required. Guidelines will be + approved on a regular cadence and identified by the date of approval. + +Interop Working Group Process + The process used by the Interop Working Group to score Capabilities + and select Criteria. OpenStack Mark Right granted by the OpenStack Foundation to use the name and logo of @@ -82,8 +83,9 @@ Participant The subset of the Community that actively engages in creating components of OpenStack including, but not limited to, the code, documentation, training, product management and other materials. - For DefCore purposes, Participant is not limited to the community - members identified as "ATC" as per http://git.openstack.org/cgit/openstack/governance/tree/reference/charter.rst#n132 + For Interop Working Group purposes, Participant is not limited to + the community members identified as "ATC" as per + http://git.openstack.org/cgit/openstack/governance/tree/reference/charter.rst#n132 (see also Technical Leadership) Platform @@ -91,7 +93,10 @@ Platform Removed Capabilities that are no longer required and are not included in the - current guideline. + current Guideline. + +Required - Capabilities that are required to be exposed to end users to + satisfy the requirements of an Interoperability Guideline. Self-test Process by which a vendor runs tests against their product or service diff --git a/doc/source/process/PlatformCap.rst b/doc/source/process/PlatformCap.rst index 5b4be253..63757c92 100644 --- a/doc/source/process/PlatformCap.rst +++ b/doc/source/process/PlatformCap.rst @@ -9,13 +9,13 @@ Platform and Program Capabilities The following was approved by the OpenStack Board. -Extend the DefCore principles to allow for multiple levels: programs and +Extend the Interop WG principles to allow for multiple levels: programs and platforms. Programs represent subsections of the overall platform. In some cases, it is acceptable for a program identified without being included in the platform. New programs are added at Foundation recommendation via board approval. Programs are added to the platform via board approval. -The initial programs will be Compute & Object. The DefCore platform will +The initial programs will be Compute & Object. The Platform will require the Compute program, Object program and additional capabilities. Havana Approved: The Compute Program will consist of the following diff --git a/doc/source/process/ProcessCycles.rst b/doc/source/process/ProcessCycles.rst index b8b2f1f3..8c07be69 100644 --- a/doc/source/process/ProcessCycles.rst +++ b/doc/source/process/ProcessCycles.rst @@ -5,9 +5,9 @@ Defining OpenStack Core is a long term process and we are doing the work in progressive cycles. For reference, we have named the cycles. This helps describe concrete deliverables for a cycle while allowing discussion of the broader long term issues. For example, we may say that -"item X is important to DefCore but out of scope for Elephant." We have -found that this approach to breaking down the problem is necessary to -maintain community consensus because we are taking smaller bites of the +"item X is important to interoperability but out of scope for Elephant." +We have found that this approach to breaking down the problem is necessary +to maintain community consensus because we are taking smaller bites of the larger challenge (aka eating the elephant). Spider (Fall 2013) @@ -167,7 +167,7 @@ Meetings ~~~~~~~~ Meeting information for the Flag cycle will be archived here once the cycle is complete. During the cycle, please refer to the -`DefCore Committee wiki page `_. +`Interop WG wiki page `_. Future ------ diff --git a/doc/source/schema/1.6.rst b/doc/source/schema/1.6.rst index 543e1744..cd57e6c7 100644 --- a/doc/source/schema/1.6.rst +++ b/doc/source/schema/1.6.rst @@ -1,5 +1,5 @@ -DefCore Schema v1.6 Change Log -============================== +Interop WG Schema v1.6 Change Log +================================== Changes from v1.5 diff --git a/next.json b/next.json index 4a6ebb3e..64fe742a 100644 --- a/next.json +++ b/next.json @@ -3270,12 +3270,12 @@ "weight": 8 }, "stable": { - "Description": "Test is required stable for >2 releases because we don't want DefCore capabilities that do not have dependable APIs", + "Description": "Test is required stable for >2 releases because we don't want Capabilities that do not have dependable APIs", "name": "Stable", "weight": 9 }, "sticky": { - "Description": "A test that is a must-pass test should stay a must-pass test. This makes DefCore capabilities sticky release per release. Leaving Core is disruptive to the ecosystem", + "Description": "A test that is a must-pass test should stay a must-pass test. This makes Capabilities sticky release per release. Leaving Core is disruptive to the ecosystem", "name": "Core in Last Release", "weight": 9 }, diff --git a/setup.cfg b/setup.cfg index a724c07b..9126e4c5 100644 --- a/setup.cfg +++ b/setup.cfg @@ -1,11 +1,11 @@ [metadata] name = defcore -summary = OpenStack DefCore Process +summary = OpenStack Interop Working Group Process description-file = README.rst author = OpenStack -author-email = defcore-committee@lists.openstack.org -home-page = https://wiki.openstack.org/wiki/Governance/DefCoreCommittee +author-email = interop-wg@lists.openstack.org +home-page = https://wiki.openstack.org/wiki/Governance/InteropWG classifier = Environment :: OpenStack Intended Audience :: Information Technology diff --git a/tools/jsonToRst.py b/tools/jsonToRst.py index 7545ef02..4fc3d4cc 100755 --- a/tools/jsonToRst.py +++ b/tools/jsonToRst.py @@ -64,7 +64,7 @@ with open(outFileName, "w") as outFile: print 'Make sure there is a valid id' sys.exit(1) - line01 = "OpenStack DefCore %s" % data["id"] + line01 = "OpenStack Interoperability Guideline %s" % data["id"] outFile.write('=' * len(line01) + '\n') outFile.write(line01 + '\n') @@ -78,7 +78,7 @@ with open(outFileName, "w") as outFile: # Correct Source if data.get('source') != \ 'http://git.openstack.org/cgit/openstack/defcore/': - print "The expected DefCore source not found" + print "The expected interoperability guideline source not found" sys.exit(1) outFile.write(""" diff --git a/working_materials/additional_properties_waiver.rst b/working_materials/additional_properties_waiver.rst index 09356cd4..90c4d26a 100644 --- a/working_materials/additional_properties_waiver.rst +++ b/working_materials/additional_properties_waiver.rst @@ -15,14 +15,13 @@ Nova calls as part of tempest-lib. Prior to this change, clouds running the Nova 2.0 API could take advantage of a mechanism to add extensions to the Nova endpoint, and some also sent additional data back in their Nova responses. These clouds -passed interoperability testing when the DefCore interoperability testing -program was launched for the OpenStack Powered program. After strict -response checking was added to Tempest, these clouds failed interop -testing. +passed interoperability testing when the OpenStack Powered program was +launched. After strict response checking was added to Tempest, these +clouds failed interop testing. To address this issue, and the challenges vendors have in updating their products to match upstream API changes, this proposal offers a means for -vendors to pass the DefCore interoperability tests while proving +vendors to pass the interoperability tests while proving compatibility for required capabilities. There is a natural tension between the forward motion of upstream @@ -41,10 +40,10 @@ products include, but are not limited to: #. Remaining on the Nova v/2.0 API, which has been removed from the Newton release of OpenStack -This waiver program will cover the 2015.07, 2016.01, and 2016.08 DefCore -guidelines, and give downstream vendors a year to work internally -and within the ecosystem to update their products before re-verifying -their products. +This waiver program will cover the 2015.07, 2016.01, and 2016.08 +interoperability guidelines, and give downstream vendors a year +to work internally and within the ecosystem to update their products +before re-verifying their products. It's important to note that the Nova team has for two years been broadcasting their intentions[1][2][3], offering microversions as an @@ -61,21 +60,22 @@ Details of Waiver #. Products appyling for the OpenStack Powered Trademark in 2016 may request the waiver by submitting subunit data from their Tempest run that can be analyzed by the `find_additional_properties.py` script - from the DefCore repository. This script will identify tests that - failed because of additional properties. The vendor will then need - to modify tempest-lib[4] to remove additional checks on the impacted - APIs. Development is beginning within the refstack-client project[5] - to automate generation of a patch for tempest-lib. + from the Interop WG's git repository. This script will identify + tests that failed because of additional properties. The vendor will + then need to modify tempest-lib[4] to remove additional checks on + the impacted APIs. Development is beginning within the + refstack-client project[5] to automate generation of a patch for + tempest-lib. #. Products that use additional properties in Nova API responses will be clearly identified in the OpenStack Marketplace, with the product listing showing which APIs have included additional response data. Products using additional data will be restricted to the Nova 2.0 API. -#. Beginning with the 2017.01 release of the DefCore guidelines, this - waiver program will no longer be in force, unless 'additional +#. Beginning with the 2017.01 release of the Interoperability Guideline, + this waiver program will no longer be in force, unless 'additional properties' is listed as an acceptable implementation using the Nova - 2.0 API in the forthcoming DefCore capabilities. All other new + 2.0 API in the forthcoming capabilities list. All other new products must pass upstream testing. #. Aside from additional properties, no products may change the json API diff --git a/working_materials/DefCoreSpec.rst b/working_materials/interop_test_spec.rst similarity index 93% rename from working_materials/DefCoreSpec.rst rename to working_materials/interop_test_spec.rst index 605d471b..144510d0 100644 --- a/working_materials/DefCoreSpec.rst +++ b/working_materials/interop_test_spec.rst @@ -3,8 +3,9 @@ Feature Description ====================== This test specification describes interoperability testing as it is understood -by the DefCore Committee. An important goal of interoperability is that end users -of OpenStack should be able to expect certain APIs to be reliable across clouds. +by the Interop Working Group. An important goal of interoperability is that +end users of OpenStack should be able to expect certain APIs to be reliable +across clouds. The spec aims to define how interoperability should be tested for OpenStack. It provides guidelines for evaluating current tests and the need for new test @@ -76,4 +77,4 @@ Guidelines #. Test cases must setup any data they need to run and clean up after themselves. It is not acceptable to corrupt the user's environment - or leave any test data or test configuration behind. \ No newline at end of file + or leave any test data or test configuration behind. diff --git a/working_materials/scoring.txt b/working_materials/scoring.txt index d2a78ff5..1d664500 100644 --- a/working_materials/scoring.txt +++ b/working_materials/scoring.txt @@ -1,17 +1,17 @@ -DefCore Scoring Worksheet -========================= +Interop Working Group Scoring Worksheet +======================================== -This file is used to maintain capabilities scoring for DefCore. +This file is used to maintain capabilities scoring. It is considered "working materials" under the 2016A process and is not an authoritative, Board-approved source of information as to which capabilities are required for a particular Guideline. -Rather, it is a point-in-time listing used by DefCore Committee -members while determining which capabilities may be included in -a future Guideline. Each line takes the following form: +Rather, it is a point-in-time listing used by Interop +Working Group members while determining which capabilities may be +included in a future Guideline. Each line takes the following form: Capability Name: [Widely Deployed,Used By Tools,Used by Clients], [TC Future Direction, Complete, Stable], - [Discoverable, Doc'd, Core in Last Release], + [Discoverable, Doc'd, Required in Last Release], [Foundational, Atomic, Proximity], [Non-Admin] [Total] @@ -74,9 +74,10 @@ Notes: For example, there are several tests for network creation, but only one or few for deletion, listing, and updating. Combining these into a single "CRUD" group reduces the chance the chance of running into problems with - DefCore's 2015A process (specifically section C3(4)) in cases where there - is only a single test for a capability and the test needs to be flagged on - account of bugs or other valid reasons. The combined groups are: + the Interop Working Group's 2015A process (specifically section + C3(4)) in cases where there is only a single test for a capability and the + test needs to be flagged on account of bugs or other valid reasons. The + combined groups are: * networks-floating-ips-CRUD-and-associate * networks-floating-ips-associate * networks-floating-ips-create @@ -112,7 +113,7 @@ Notes: pools advisory now, they could become required in 2017.08--in which the earliest release in the Guideline will be Mitaka. Toolkits like jClouds, Fog, and libcloud do not yet support subnet pools that I see. - Subnet pools will be used by Neutron's upcoming get-me-a-network + Subnet pools will be used by Neutron's upcoming get-me-a-network capability and Kuryr appears to have plans to use them as part of Kubernets integration. * Extended port attribute tests were not considered because the capability @@ -129,6 +130,46 @@ compute-servers-suspend-resume: [1,1,1] [1,1,1] [1,1,0] [0,1,1] [1] [82]* compute-flavors-list: [1,1,1] [1,1,1] [1,1,0] [0,1,0] [1] [74]* compute-availability-zones-list: [1,1,1] [1,1,1] [1,1,0] [0,1,0] [1] [74]* +Notes: + * There has been some discussion that this capability is not widely + deployed [1]. However it appears that in most cases the API either: + * Is present and works, but only if a non-SSL endpoint is exposed. + SSL endpoints may have issues with redirects (which may actually + merely be configuration issues). + * Is accidentally blocked due to load balancer or API gateway + misconfiguration which is being remedied by affected providers. + Taking those situations into account, it is believed that it's actually + safe to call this "widely deployed". + * I'm having trouble finding evidence that tools actually support this + API (refer to [2] for information about what constitutes "tools"). + Unlike most of the CRUD-like API's, this one lists information about + the API versions available instead of on end user actionable resources + (such as instances or security groups), and the need to do so is newish + with the advent of microversioning. Hence, I think many tools may not + actually support this yet. Please feel free to provide evidence to the + contrary; I'd love to be proven wrong. + * Similarly, I'm having trouble finding evidence that clients [3] like + jClouds or Fog support this API (yet?) as they tend to focus on actions. + Please feel free to provide evidence to the contrary; I'd love to be + proven wrong. + * I'm tentatively scoring this API as Proximate [4]. Generally this applies + to CRUD operations (e.g. the delete API is considered proximate to the + create API). However I'm providing a 1 here on the grounds that due + to the shift to microversioning in the Nova API, this will be somewhat + proximate to everything going forward. + * The Interop Working Group is considering this capability for inclusion + in it's projected 2016.01 Guideline even though the test for it wasn't + developed until (narrowly) outside the normal window for identifying + new Capabilities on account of strong indications from the community + that such a Capability is important for API microversioning going + forward [5]. + +[1] https://bugs.launchpad.net/python-novaclient/+bug/1491579 +[2] http://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst#n44 +[3] http://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst#n48 +[4] http://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst#n90 +[5] http://eavesdrop.openstack.org/meetings/defcore_flag_14/2015/defcore_flag_14.2015-09-09-15.03.log.html#l-13 + Image ----- images-v1-delete: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58] @@ -155,7 +196,7 @@ Notes: * Scoring for v1 remains in place, but v2 is preferred interop standard (as reflected in worksheet). * images-v2-share inherently requires more than one credential, - and currently out of scope for DefCore capabilities. + and currently out of scope for required capabilities. * images-v2-remove has no test as of Auguest, 2016. Volume @@ -272,4 +313,3 @@ https://gist.github.com/notmyname/102e4aba7084598638f47cee47f62bb1#file-defcore_ objectstore-container-metadata used in Fog: https://github.com/fog/fog-openstack/blob/master/docs/storage.md#additional-parameters Also in jClouds: https://jclouds.apache.org/guides/openstack/#swift - diff --git a/working_materials/tabulate_scores.py b/working_materials/tabulate_scores.py index 1aeb26e8..cd3584ef 100755 --- a/working_materials/tabulate_scores.py +++ b/working_materials/tabulate_scores.py @@ -31,7 +31,7 @@ parser = argparse.ArgumentParser( description=textwrap.dedent("""\ Tabulate capability scores and write them to files. - This utility script tabulates scores from a DefCore scoring + This utility script tabulates scores from an Interop WG scoring worksheet based on the weights from a given Guideline JSON file. It writes the scores in three formats: