openstack
/
docs-specs
Code Issues Proposed changes

Adds spec for migrating the documentation suite to project repos 

Currently the user-facing documentation belongs in the
openstack-manuals repo. Due to a severe decline in contributions,
the docs team has had to come up with a new way to move forward.

This spec proposes the new process for user-facing documentation
and how we are going to undertake the changes.

Co-Authored By: Doug Hellmann <doug@doughellmann.com>
Co-Authored By: Alexandra Settle <a.settle@outlook.com>
Change-Id: Ia750cb049c0f53a234ea70ce1f2bbbb7a2aa9454
This commit is contained in:
Alexandra Settle 2017-06-08 15:01:46 +01:00 committed by Doug Hellmann
parent 912ed2fd2f
commit f0fe465f69
1 changed files with 393 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

393
specs/pike/os-manuals-migration.rst Normal file
View File

@ -0,0 +1,393 @@
===================================
OpenStack manuals project migration
===================================
Problem description
~~~~~~~~~~~~~~~~~~~
The documentation team are rapidly losing key contributors and core reviewers.
We are not alone, this is happening across the board. It is making things
harder, but not impossible.
Since our inception in 2010, we’ve been climbing higher and higher trying to
achieve the best documentation we could, and uphold our high standards.
However, we now need to take a step back and realise that the amount of work
we are attempting to maintain is out of reach for the team size that we have.
At the moment we have 13 cores, of whom none are full time contributors or
reviewers.
Until this point, the documentation team has owned several manuals that
include content related to multiple projects, including an installation
guide, admin guide, configuration guide, networking guide, and security
guide. Because the team no longer has the resources to own that content,
we want to invert the relationship between the doc team and project teams,
so that we become liaisons to help with maintenance instead of asking for
project teams to provide liaisons to help with content. As a part of that
change, we plan to move the existing content out of the central manuals
repository, into repositories owned by the appropriate project teams.
Project teams will then own the content and the documentation team will
assist by managing the build tools, helping with writing guidelines and
style, but not writing the bulk of the text.
We currently have the infrastructure set up to empower project teams to
manage their own documentation in their own tree, and many do. As part of
this change, the rest of the existing content from the install guide and
admin guide will also move into project-owned repositories.
Proposed change
~~~~~~~~~~~~~~~
Combine all of the documentation builds for a given repository, so
that each repository has a single doc/source directory, a single
sphinx conf.py, and a single job for building and publishing the
content including developer, contributor, and user documentation. This
option would reduce the number of build jobs we have to run, and cut
down on the number of separate sphinx configurations in each
repository.
Move most of the content from various guides in openstack-manuals into
the same repository as the thing being documented -- the documentation
should live with the code. For example, the installation instructions
for glance move to the glance repository and the reference for using
the glance client command line tool move to the python-glanceclient
repository.
In order to make it easy to include links from the landing pages on
docs.openstack.org, we need to ensure a minimum level of consistency
in the organization of the docs directory. The sections are based on
the existing high-level groupings for which there are already landing
pages on docs.openstack.org that will need to be updated. Within each
top-level directory, project teams are free to organize their content
however seems most appropriate to them. This is the proposed layout
for phase 1:
* ``doc/source/``
* ``install/`` -- anything having to do with installation of the
project
* ``contributor/`` -- anything related to contributing to the
project or how the team is managed. Applies to some of the current
content under /developer, we are changing the name to emphasize
that not all contributors are developers and sometimes developers
are users but not contributors. Service projects should place
their automatically generated class documentation under this part
of the tree, e.g. in ``contributor/api`` or
``contributor/internals``.
* ``configuration/`` -- automatically generated configuration
reference information based on oslo.config's sphinx integration
(or manually written for projects not using
oslo.config). Step-by-step guides for doing things like enabling
cells or configuring a specific driver should be placed in the
``admin/`` section.
* ``cli/`` -- command line tool reference docs, similar to man pages
These may be automatically generated with cliff's sphinx
integration, or manually written when auto-generation is not
possible. Tutorials or other step-by-step guides *using* these
tools should go in either the ``user/`` or ``admin/`` sections,
depending on their audience. Because the documentation for each
project should live in the repository with the code, this
directory may not be present for all service repositories but will
be present for most of the client library repositories.
* ``admin/`` -- any content from the old admin guide or anything
else that discusses how to run or operate the software
* ``user/`` -- end-user content such as concept guides, advice,
tutorials, step-by-step instructions for using the CLI to perform
specific tasks, etc.
* ``reference/`` -- any reference information associated with a
project that is not covered by one of the above categories.
Library projects should place their automatically generated class
documentation here.
This layout is the *minimum* set. Projects are free and encouraged to
add whatever other docs they need beyond this list, but these items
are listed here explicitly because there are already links to most of
them from landing pages, and landing pages can be created for the
others.
During a later phase, we will merge the API reference and release notes builds
into the same job, along with the rest of the documentation for a project.
Both of those builds have custom considerations, though, and it is more
important to move the content that is no longer going to be maintained
by the documentation team.
* ``doc/source/``
* ``api/`` -- the REST API reference and Guide content, when it exists
* ``releasenotes/`` -- reno directions (the actual release notes
inputs will stay in /releasenotes/notes, where they are now)
.. note::
Further discussion of the layout of the ``api/`` and
``releasenotes/`` directories is deferred until we are farther
along with the initial migration work.
A new documentation build job will be set up to take the output produced from
``tox -e venv -- python setup.py build_sphinx`` (matching the existing
`Consistent Testing Interface"
<https://governance.openstack.org/tc/reference/project-testing-interface.html>`_)
and publish it to a new location under `<http://docs.openstack.org>`_
The results will go to:
* ``docs.openstack.org/$project-name/latest`` -- build from master
* ``docs.openstack.org/$project-name/$series`` -- build from
stable/$series
* ``docs.openstack.org/$project-name/latest/$lang`` -- build
translated version from master
* ``docs.openstack.org/$project-name/$series/$lang`` -- build
translated version from stable/$series
Because we plan to reuse the existing `doc/source` directory in each project,
some of the existing content will need to be rearranged as part of importing
the content from openstack-manuals.
Ultimately, this changes the way we publish results, and redirects will be
required to be setup from all of the existing locations to the new locations,
and move all of the existing documentation under the new structure. We will
retain landing pages for the high level categories such as the install guides,
the configuration reference, and contributor documentation. Those pages will
continue to be maintained by the documentation team, and will deep-link into
the project team documentation.
For example, we will keep pages like
https://docs.openstack.org/project-install-guide/ocata/ but they will
provide lists of links to URLs like
https://docs.openstack.org/nova/ocata/install, which will be part of
the single doc build for nova.
What is happening to each guide?
--------------------------------
* Installation Guide
* Most of the content will move from openstack-manuals into each project
tree.
* The chapters not directly related to specific OpenStack projects,
such as the parts related to installing ntp and RabbitMQ, will be
retained in openstack-manuals in a shared guide for setting up
common dependencies so that content does not need to be reproduced
several times.
* The current guide is actually built 3 separate times, to cover 3
separate deployment platforms. The new build will not support
that, so the migration for the installation guide will involve
breaking the content up into separate pages for each platform (as
needed). Patch https://review.openstack.org/473579 splits the
content up into separate patches, one per OS.
* Project Installation guides, already in tree
* We recommend any installation guides already in-tree also move to the new
organization.
* Administrator Guide
* This content will move from openstack-manuals into each project tree. No
part will be retained in openstack-manuals. This spec was already
approved:
https://review.openstack.org/#/c/439122/
* High Availability Guide
* This guide will remain in openstack-manuals and be managed by the HA team.
For more information: https://blueprints.launchpad.net/openstack-manuals/+spec/implement-ha-guide-todos
* Operations Guide
* This guide will eventually move from openstack-manuals into the wiki.
Nothing will be done with it until a volunteer is found to manage that
move.
* Security Guide
* This content will stay in openstack-manuals, and be managed by the
security team.
* A notice is being added to indicate the last time it was updated
and which release is relevant
(https://review.openstack.org/#/c/470059).
* Architecture Design Guide
* This content will stay in openstack-manuals, and be deprecated.
* A notice will be added to each page indicating that the guide is up to
date as of $RELEASE after the finalisation of the current set of goals.
For more information on those goals:
https://blueprints.launchpad.net/openstack-manuals/+spec/arch-design-pike
* Networking Guide
* This content will move from openstack-manuals to the neutron repository
under docs/source/admin.
* Configuration Reference
* A few pages will move from openstack-manuals to the user-facing
documentation in oslo.config.
* The remainder will be removed, and replaced with new pages in the
in-tree documentation built using oslo_config.sphinxext.
* For tracking purposes, please see:
https://blueprints.launchpad.net/openstack-manuals/+spec/automate-config-ref
* API Documentation
* No changes.
* End User Guide
* This content will be divided between the horizon repository and
python-openstackclient repository.
* Command-Line Reference
* This content will move the project-specific client documentation
trees under doc/source/cli. For example, the information about
using the ``glance`` command line tool would move to the
python-glanceclient repository.
* Virtual Machine Image Reference
* This content will stay in openstack-manuals.
Migration process
-----------------
We will need to parallelize the migration work as much as possible if we are
going to complete it by the end of the Pike cycle. We will therefore need
project teams to find volunteers to "pull" the content into their
repositories, instead of having the documentation team "push" it.
.. note::
Use the topic ``doc-migration`` for all patches.
.. note::
Repeat these steps for all server projects, clients, and other
libraries.
#. Move the existing contributor-focused content to fit the layout
above. Submit that change with ``Depends-On:
Ia750cb049c0f53a234ea70ce1f2bbbb7a2aa9454`` to tie it to this
spec.
#. If your project docs are not already building using
warning-is-error in setup.cfg, turn that on and fix any build
errors. Submit these as patches on top of the first patch.
#. Pull in the content being migrated, following the layout above.
* Go through the list of manuals in
https://etherpad.openstack.org/p/doc-migration-tracking and take
any actions needed to import content.
* Prepare one patch per manual (so one to import the install guide,
one to import the user guide, etc.). Submit these as patches on
top of any previous patches.
#. Ensure that there is a top-level index.rst in doc/source that
incorporates all of the documentation for the project by including
all of the subdirectories in a toctree.
#. Update the theme for the in-tree docs to use the openstackdocstheme
instead of oslosphinx.
#. Add auto-generated config reference section(s).
#. If pbr's autodoc feature is being used, update the ``api_doc_dir``
setting in the ``pbr`` section of ``setup.cfg`` to point to either
``reference/api`` (for libraries) or ``contributor/api`` (for other
types of projects).
#. Update project-config to have the doc build use the new jobs instead of the
old jobs by replacing 'openstack-server-publish-jobs' with
'openstack-unified-publish-jobs'.
Set ``Depends-On`` to the Change-Id from the patch created in
step 1. This ensures that we do not publish the old content to the
new location.
#. Add links to the reviews for individual TODO items below those
items in the sections dedicated to each manual. That way the docs
team will know when it is safe to start deleting content.
#. If the project has a service-type, add a 302 redirect rule from
``/$service-type`` to ``/$project-name`` by editing
``openstack-manuals/www/static/.htaccess``. For example,
``/compute`` should redirect to ``/nova``.
#. After project-specific install guides are moved into the doc tree,
set up a 302 redirect rule from /project-install-guide to the
new /install location by editing
``openstack-manuals/www/static/.htaccess``.
#. Update the main landing page(s) starting with docs.o.o/ (see notes above)
#. Remove the original copy of the content from the openstack-manuals repo
* This patch can be filed early with depends-on for the other related
patches.
Alternatives
------------
#. We could retain the existing trees for developer and API docs, and add a new
one for "user" documentation. The installation guide, configuration guide,
and admin guide would move here for all projects. Neutron's user
documentation would include the current networking guide as well. This
option would add 1 new build to each repository, but would allow us to
easily roll
out the change with less disruption in the way the site is organized and
published, so there would be less work in the short term.
#. We could move the content under separate repositories owned by the project
teams, rather than in-tree with the code. This would allow project teams to
delegate management of the documentation to a separate review
project-sub-team, but would complicate the process of landing code and
documentation updates together so that the docs are always up to date.
#. Do nothing, and watch the world burn.
We did consider using "service type" instead of "project name" for the
publishing URLs, but not all of the projects that need documentations
are services. We will have user-facing documentation coming from several
Oslo libraries, for example.
Implementation
~~~~~~~~~~~~~~
Assignee(s)
-----------
Primary assignee:
* Alexandra Settle (asettle)
* Doug Hellmann (dhellmann)
* Project teams
* Documentation team PTL for Queens
* Documentation team
Work items
----------
The task list is quite long, so rather than repeat it here we give a summary.
There is more detail in the tracking pad mentioned in step 3.
#. Define new doc build and gate jobs that work like the current job, using
"tox -e venv -- python setup.py build_sphinx`" in a repository, but publish
to the new location of docs.o.o/$project-name/latest (dhellmann)
* https://review.openstack.org/#/c/471881/
#. Define doc build jobs for stable branches that run the same command but
publish to docs.o.o/$project-name/$series (dhellmann)
* The same job will work for all branches.
#. In parallel, in each repository, perform the migration steps listed above to
copy the new content into the doc/source directory. Refer to
https://etherpad.openstack.org/p/doc-migration-tracking for details about
which pages go into which project trees.
#. Define new translation jobs based on the ones for the release notes build
but using the main doc build.
Dependencies
~~~~~~~~~~~~
- Project team(s) collaboration
- Infra team assistance
- Reviews from multiple sources
References
~~~~~~~~~~
* https://etherpad.openstack.org/p/doc-planning
* The list of all URLs and where the content will move can be found
in: https://etherpad.openstack.org/p/doc-migration-tracking
* Documentation Publishing future thread:
http://lists.openstack.org/pipermail/openstack-dev/2017-May/117162.html
* Operations Guide Future thread:
http://lists.openstack.org/pipermail/openstack-dev/2017-June/117799.html