move release model into deliverable files

Because release models change from series to series, we want to track
the data in the releases repository instead of in the governance
repository. This change adds a new 'release-model' field to the
deliverable files to hold the new value.

The valid release models are documented as part of a new reference
section for the published documentation for the site.

The validator looks only at the deliverable file for the release model,
and no longer looks at the projects.yaml file in the governance
repository. For files in the _independent directory, the release model
is inferred from the filename to avoid having to set it redundantly.

Change-Id: I81198a86da98436dc2f65806aafc5f241340c57f
Signed-off-by: Doug Hellmann <doug@doughellmann.com>

README.rst

@@ -157,6 +157,7 @@ deliverable. For each release, we need to track:
 * the launchpad project name (such as ``oslo.config``)
 * the email list to receive release announcements
 * the series (Kilo, Liberty, etc.)
+* the release model being used
 * for each repository
 
   * the name (such as ``openstack/oslo.config``)
@@ -210,6 +211,11 @@ The top level of a deliverable file is a mapping with keys:
   announcement should include the link to the package on
   PyPI. Defaults to ``no``.
 
+``release-model``
+  Identify the release model used by the deliverable. See
+  the reference section of the documentation for descriptions
+  of the valid models.
+
 ``artifact-link-mode``
   Describe how to link to artifacts produced by the project. The
   default is ``tarball`. Valid values are:

doc/source/index.rst

@@ -130,3 +130,12 @@ Deliverables organized by the team that produces them.
    :glob:
 
    teams/*
+
+References
+==========
+
+.. toctree::
+   :maxdepth: 2
+   :glob:
+
+   reference/*

doc/source/reference/release_models.rst

@@ -0,0 +1,95 @@
+================
+ Release Models
+================
+
+Development in OpenStack is organized around 6-month cycles (like
+"kilo").  At the end of every 6-month cycle a number of projects
+release at the same time, providing a convenient reference point for
+downstream teams (stable branch maintenance, vulnerability management)
+and downstream users (in particular packagers of OpenStack
+distributions).
+
+This "final" release may be the only release of the development cycle,
+in which case the project publishes intermediary "development
+milestones" on a time-based schedule during the cycle. Or the project
+may release more often and make intermediary releases in the middle of
+the cycle. Other projects trail the main release deadline, waiting for
+the final releases of components on which they rely.
+
+A given deliverable can't have more than one model. It therefore must
+choose between one of the following models.
+
+.. _model-cycle-with-milestones:
+
+cycle-with-milestones
+=====================
+
+The "cycle-with-milestones" model describes projects thatproduce a
+single release at the end of the cycle, with development milestones
+published at predetermined times in the cycle schedule.
+
+* "cycle-with-milestones" projects commit to publish development
+  milestones following a predetermined schedule published by the Release
+  Management team before the start of the 6-month cycle.
+* "cycle-with-milestones" projects commit to produce a release to
+  match the end of the 6-month development cycle.
+* Release tags for deliverables using this tag are reviewed and
+  applied by the Release Management team.
+
+.. _model-cycle-with-intermediary:
+
+cycle-with-intermediary
+=======================
+
+The "cycle-with-intermediary" model describes projects that produce
+multiple full releases during the development cycle, with a final
+release to match the end of the cycle.
+
+* "cycle-with-intermediary" projects commit to produce a
+  release near the end of the 6-month development cycle to be used
+  with projects using the other cycle-based release models that are
+  required to produce a release at that time.
+* Release tags for deliverables using this tag are reviewed and
+  applied by the Release Management team.
+
+.. _model-cycle-trailing:
+
+cycle-trailing
+==============
+
+The "cycle-trailing" model describes projects that follow the release
+cycle, but because they rely on the other projects being completed may
+not always publish their final release at the same time as those
+projects. For example, projects related to packaging or deploying
+OpenStack components need the final releases of those components to be
+available before they can run their own final tests.
+
+.. warning::
+
+   This release model is not intended for use by components that have
+   programmatic dependencies on each other, such as one service that
+   calls another or a library used by multiple services. It is
+   intended for use by projects that truly cannot complete their work
+   without "final" versions of their dependencies.
+
+* "cycle-trailing" projects commit to produce a release no later than
+  2 weeks after the end of the 6-month development cycle.
+* Within the cycle, projects using this release model will produce
+  intermediate or milestone releases (adhering to the same regular
+  deadlines) leading up to their final release.
+* Release tags for deliverables using this tag are reviewed and
+  applied by the Release Management team.
+
+.. _model-independent:
+
+independent
+===========
+
+Some projects opt to completely bypass the 6-month cycle and release
+independently. For example, that is the case of projects that support
+the development infrastructure. The "independent" model describes such
+projects.
+
+* "independent" projects produce releases from time to time.
+* Release tags for deliverables using this tag are managed without
+  oversight from the Release Management team.

openstack_releases/cmds/validate.py

@@ -41,6 +41,13 @@ from openstack_releases import versionutils
 
 urllib3.disable_warnings()
 
+_VALID_MODELS = set([
+    'cycle-with-milestones',
+    'cycle-with-intermediary',
+    'cycle-trailing',
+    'independent',
+])
+
 
 def is_a_hash(val):
     "Return bool indicating if val looks like a valid hash."
@@ -73,14 +80,6 @@ def main():
     zuul_layout = project_config.get_zuul_layout_data()
 
     team_data = governance.get_team_data()
-    independent_repos = set(
-        r.name
-        for r in governance.get_repositories(
-            team_data,
-            tags=['release:independent'],
-        )
-    )
-    independent_checks = set()
 
     errors = []
     warnings = []
@@ -170,6 +169,21 @@ def main():
             os.path.dirname(filename)
         )
 
+        # Determine the release model. Don't require independent
+        # projects to redundantly specify that they are independent by
+        # including the value in their deliverablefile, but everyone
+        # else must provide a valid value.
+        is_independent = (series_name == '_independent')
+        if is_independent:
+            release_model = 'independent'
+        else:
+            release_model = deliverable_info.get('release-model', 'UNSPECIFIED')
+        if release_model not in _VALID_MODELS:
+            errors.append(
+                'Unknown release model %r for %s, must be one of %r' %
+                (release_model, filename, sorted(list(_VALID_MODELS)))
+            )
+
         # Remember which entries are new so we can verify that they
         # appear at the end of the file.
         new_releases = {}
@@ -180,11 +194,6 @@ def main():
         for release in deliverable_info['releases']:
 
             for project in release['projects']:
-                is_independent = (
-                    (series_name, project['repo']) in independent_checks or
-                    project['repo'] in independent_repos or
-                    series_name == '_independent'
-                )
 
                 # Check for release jobs (if we ship a tarball)
                 if link_mode != 'none':
@@ -208,7 +217,6 @@ def main():
                                                          filename)
                         print(msg)
                         warnings.append(msg)
-                    independent_checks.add((series_name, project['repo']))
 
                 # Check the SHA specified for the tag.
                 print('%s SHA %s ' % (project['repo'],

openstack_releases/governance.py

@@ -86,6 +86,13 @@ class Deliverable(object):
                 return t.partition(':')[-1]
         return 'unknown'
 
+    @property
+    def model(self):
+        for t in self.tags:
+            if t.startswith('release:'):
+                return t.partition(':')[-1]
+        return 'none'
+
     @property
     def tags(self):
         return set(self.data.get('tags', [])).union(self.team.tags)