diff --git a/.zuul.yaml b/.zuul.yaml index 33c7f94..fa30e6c 100644 --- a/.zuul.yaml +++ b/.zuul.yaml @@ -1,4 +1,6 @@ - project: + templates: + - build-openstack-docs-pti check: jobs: - gabbi-tempest diff --git a/README.rst b/README.rst index 893e810..3268af7 100644 --- a/README.rst +++ b/README.rst @@ -2,21 +2,19 @@ Gabbi + Tempest =============== -Gabbi-tempest is an experimental Tempest_ plugin_ that enables -testing the APIs of running OpenStack services, integrated with -tempest but without needing to write Python. Instead the YAML -format_ provided by gabbi_ is used to write and evaluate HTTP -requests and responses. +Gabbi-tempest is a Tempest_ plugin_ that enables testing the APIs of running +OpenStack services, integrated with tempest but without needing to write +Python. Instead the YAML format_ provided by gabbi_ is used to write and +evaluate HTTP requests and responses. -Tests are placed in YAML files in one or more directories. Those -directories are added to a ``GABBI_TEMPEST_PATH`` environment -variable. When that variable is passed into a tempest test -runner that is aware of the gabbi plugin, the files on that path -will be used to create tempests tests. +Tests are placed in YAML files in one or more directories. Those directories +are added to a ``GABBI_TEMPEST_PATH`` environment variable. When that variable +is passed into a tempest test runner that is aware of the gabbi plugin, the +files on that path will be used to create tempests tests. -The test harness sets a series of enviornment variables that can -be used in the YAML to reach the available services. The available -variables may be extended in two ways: +The test harness sets a series of enviornment variables that can be used in +the YAML to reach the available services. The available variables may be +extended in two ways: * Adding them to the environment that calls tempest if the values are known. @@ -29,55 +27,4 @@ For each service in the service catalog there are ``SERVICE_TOKEN``, ``IMAGE_REF``, ``FLAVOR_REF`` and ``FLAVOR_REF_ALT`` are also available. -For the time being the ``SERVICE_TOKEN`` is ``admin``. - -Trying It ---------- - -To experiment with this you need a working tempest installation and -configuration. One way to do that is to use devstack_ with the -following added to the local.conf:: - - enable_service tempest - INSTALL_TEMPEST=True - -in local.conf. - -Once tempest is confirmed to be working, gabbi-tempest must be -installed. Either install it from PyPI:: - - pip install gabbi-tempest - -Or make a clone of this repo_, cd into it, and do the equivalent of:: - - pip install -e . - -If you are using virtualenvs or need sudo, your form will be -different. - -Create some gabbi_ tests that exercise the OpenStack services. There -are sample files in the ``samples`` directory in the repo_. - -Go to the tempest directory (often ``/opt/stack/tempest``) and run -tempest as follows. Adding the ``regex`` will limit the test run -to just gabbi related tests:: - - GABBI_TEMPEST_PATH=/path/one:/path/two tempest run --regex gabbi - -This will run the tests described by the YAML files in -``/path/one`` and ``/path/two``. - -History -------- - -This code is based on the work of Mehdi Abaakouk who made a tempest -plugin for gnocchi_ that worked with gabbi_. He figured out the -details of the plugin structure. - -.. _devstack: https://docs.openstack.org/devstack/latest/ -.. _Tempest: https://docs.openstack.org/tempest/latest/ -.. _plugin: https://docs.openstack.org/tempest/latest/plugin.html -.. _gnocchi: https://review.openstack.org/#/c/301585/ -.. _gabbi: https://gabbi.readthedocs.org/ -.. _format: https://gabbi.readthedocs.io/en/latest/format.html -.. _repo: https://github.com/cdent/gabbi-tempest +Read the docs at https://gabbi-tempest.readthedocs.io/ diff --git a/doc/requirements.txt b/doc/requirements.txt new file mode 100644 index 0000000..6966869 --- /dev/null +++ b/doc/requirements.txt @@ -0,0 +1 @@ +sphinx diff --git a/doc/source/conf.py b/doc/source/conf.py new file mode 100644 index 0000000..2bd752d --- /dev/null +++ b/doc/source/conf.py @@ -0,0 +1,166 @@ +# -*- coding: utf-8 -*- +# +# Configuration file for the Sphinx documentation builder. +# +# This file does only contain a selection of the most common options. For a +# full list see the documentation: +# http://www.sphinx-doc.org/en/stable/config + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +# import os +# import sys +# sys.path.insert(0, os.path.abspath('.')) + + +# -- Project information ----------------------------------------------------- + +project = 'gabbi-tempest' +copyright = '2018, Chris Dent' +author = 'Chris Dent' + +# The short X.Y version +version = '' +# The full version, including alpha/beta/rc tags +release = '' + + +# -- General configuration --------------------------------------------------- + +# If your documentation needs a minimal Sphinx version, state it here. +# +# needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + 'sphinx.ext.autodoc', + 'sphinx.ext.todo', + 'sphinx.ext.viewcode', +] + +# Add any paths that contain templates here, relative to this directory. +templates_path = [] + +# The suffix(es) of source filenames. +# You can specify multiple suffix as a list of string: +# +# source_suffix = ['.rst', '.md'] +source_suffix = '.rst' + +# The master toctree document. +master_doc = 'index' + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +# +# This is also used if you do content translation via gettext catalogs. +# Usually you set "language" from the command line for these cases. +language = None + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path . +exclude_patterns = [] + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + + +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'alabaster' + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +# +# html_theme_options = {} + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = [] + +# Custom sidebar templates, must be a dictionary that maps document names +# to template names. +# +# The default sidebars (for documents that don't match any pattern) are +# defined by theme itself. Builtin themes are using these templates by +# default: ``['localtoc.html', 'relations.html', 'sourcelink.html', +# 'searchbox.html']``. +# +# html_sidebars = {} + + +# -- Options for HTMLHelp output --------------------------------------------- + +# Output file base name for HTML help builder. +htmlhelp_basename = 'gabbi-tempestdoc' + + +# -- Options for LaTeX output ------------------------------------------------ + +latex_elements = { + # The paper size ('letterpaper' or 'a4paper'). + # + # 'papersize': 'letterpaper', + + # The font size ('10pt', '11pt' or '12pt'). + # + # 'pointsize': '10pt', + + # Additional stuff for the LaTeX preamble. + # + # 'preamble': '', + + # Latex figure (float) alignment + # + # 'figure_align': 'htbp', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, +# author, documentclass [howto, manual, or own class]). +latex_documents = [ + (master_doc, 'gabbi-tempest.tex', 'gabbi-tempest Documentation', + 'Chris Dent', 'manual'), +] + + +# -- Options for manual page output ------------------------------------------ + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + (master_doc, 'gabbi-tempest', 'gabbi-tempest Documentation', + [author], 1) +] + + +# -- Options for Texinfo output ---------------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + (master_doc, 'gabbi-tempest', 'gabbi-tempest Documentation', + author, 'gabbi-tempest', 'One line description of project.', + 'Miscellaneous'), +] + + +# -- Extension configuration ------------------------------------------------- + +# -- Options for todo extension ---------------------------------------------- + +# If true, `todo` and `todoList` produce output, else they produce nothing. +todo_include_todos = True diff --git a/doc/source/index.rst b/doc/source/index.rst new file mode 100644 index 0000000..2162bdd --- /dev/null +++ b/doc/source/index.rst @@ -0,0 +1,94 @@ +.. gabbi-tempest documentation master file, created by + sphinx-quickstart on Mon Oct 29 11:32:20 2018. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +=============== +Gabbi + Tempest +=============== + +.. toctree:: + :maxdepth: 1 + :hidden: + + zuul + +Gabbi-tempest is a Tempest_ plugin_ that enables testing the APIs of running +OpenStack services, integrated with tempest but without needing to write +Python. Instead the YAML format_ provided by gabbi_ is used to write and +evaluate HTTP requests and responses. + +Tests are placed in YAML files in one or more directories. Those directories +are added to a ``GABBI_TEMPEST_PATH`` environment variable. When that variable +is passed into a tempest test runner that is aware of the gabbi plugin, the +files on that path will be used to create tempests tests. + +The test harness sets a series of enviornment variables that can be used in +the YAML to reach the available services. The available variables may be +extended in two ways: + +* Adding them to the environment that calls tempest if the values are + known. +* Setting them in a subclass of the plugin if the values need to + be calculated from what tempest knows. + +For each service in the service catalog there are +``_SERVICE`` and ``_BASE`` variables +(e.g., ``PLACEMENT_SERVICE`` and ``PLACEMENT_BASE``). A useful +``SERVICE_TOKEN``, ``IMAGE_REF``, ``FLAVOR_REF`` and ``FLAVOR_REF_ALT`` +are also available. + +For the time being the ``SERVICE_TOKEN`` is ``admin``. + +gabbi-tempest can be used with :doc:`zuul`. + +Trying It +--------- + +To experiment with this you need a working tempest installation and +configuration. One way to do that is to use devstack_ with the +following added to the local.conf:: + + enable_service tempest + INSTALL_TEMPEST=True + +in local.conf. + +Once tempest is confirmed to be working, gabbi-tempest must be +installed. Either install it from PyPI:: + + pip install gabbi-tempest + +Or make a clone of this repo_, cd into it, and do the equivalent of:: + + pip install -e . + +If you are using virtualenvs or need sudo, your form will be +different. + +Create some gabbi_ tests that exercise the OpenStack services. There +are sample files in the ``samples`` directory in the repo_. + +Go to the tempest directory (often ``/opt/stack/tempest``) and run +tempest as follows. Adding the ``regex`` will limit the test run +to just gabbi related tests:: + + GABBI_TEMPEST_PATH=/path/one:/path/two tempest run --regex gabbi + +This will run the tests described by the YAML files in +``/path/one`` and ``/path/two``. + +History +------- + +This code is based on the work of Mehdi Abaakouk who made a tempest +plugin for gnocchi_ that worked with gabbi_. He figured out the +details of the plugin structure. + +.. _devstack: https://docs.openstack.org/devstack/latest/ +.. _Tempest: https://docs.openstack.org/tempest/latest/ +.. _plugin: https://docs.openstack.org/tempest/latest/plugin.html +.. _gnocchi: https://review.openstack.org/#/c/301585/ +.. _gabbi: https://gabbi.readthedocs.org/ +.. _format: https://gabbi.readthedocs.io/en/latest/format.html +.. _repo: https://git.openstack.org/openstack/gabbi-tempest diff --git a/doc/source/zuul.rst b/doc/source/zuul.rst new file mode 100644 index 0000000..7e3ad4a --- /dev/null +++ b/doc/source/zuul.rst @@ -0,0 +1,40 @@ + +=============== +Using With Zuul +=============== + +gabbi-tempest is designed to work well with OpenStack `infrastructure`_. +A base zuul job, named 'gabbi-tempest', is provided. Children of this job need +to only define the path of where their gabbi YAML files are located and then +request that the job run. For example a job for nova where the YAML files live +in a directory named ``gate/gabbits`` from the root of the repository would +look like this:: + + - job: + name: nova-gabbi-tempest + parent: gabbi-tempest + timeout: 10800 + description: | + Test live nova using gabbi. + vars: + gabbi_tempest_path: "{{ devstack_base_dir|default('/opt/stack') }}/nova/gate/gabbits" + +Once that job is in place, further tests are added by adding more YAML files to +the ``gate/gabbits`` directory. + +More Detail +=========== + +The ``gabbi-tempest`` zuul job defines a small number of required settings, but +has its parent, ``devstack-tempest``, do most of the work. The settings: + +* Add ``openstack/gabbi-tempest`` to ``required-projects``, making sure the + latest version of the plugin is available. +* Chooses the ``all`` tox environment when running tempest. +* Passes ``gabbi`` as the test regular expression. +* Sets concurrency to 1 to make sure the gabbi tests run in order (otherwise + duplicate tests can run). +* Sets ``TEMPEST_PLUGINS``. +* Turns on Python 3.. + +.. _infrastructure: https://docs.openstack.org/infra/manual/zuulv3.html diff --git a/setup.cfg b/setup.cfg index 6bf97bf..67ac6bd 100644 --- a/setup.cfg +++ b/setup.cfg @@ -5,7 +5,7 @@ author-email = cdent@anticdent.org summary = Run Gabbi Tests as Tempest Plugin description-file = README.rst license = Apache-2 -home-page = https://github.com/cdent/gabbi-tempest +home-page = https://git.openstack.org/openstack/gabbi-tempest classifier = Intended Audience :: Developers Intended Audience :: Information Technology diff --git a/tox.ini b/tox.ini new file mode 100644 index 0000000..152921a --- /dev/null +++ b/tox.ini @@ -0,0 +1,16 @@ +[tox] +minversion = 2.1 +skipsdist = True + +[testenv] +basepython = python3 +usedevelop = True +whitelist_externals = rm + +[testenv:docs] +description = + Build documentation +deps = -r{toxinidir}/doc/requirements.txt +commands = + rm -rf doc/build + sphinx-build -W -b html -d doc/build/doctrees doc/source doc/build/html