docs: Add Patrole overview documentation
This patchset adds a Patrole overview documentation section which is sorely needed. It combines the previous usage documentation into it. Change-Id: Ia7412f2e99f33fbdfd2e60ba54ffdba757d1f886
This commit is contained in:
Felipe Monteiro
parent
f50f6f1384
commit
543f7b9b8a
26
README.rst
26
README.rst
|
|
@ -33,10 +33,22 @@ testing scope is confined to policies.
|
|||
* *Atomicity*. Patrole tests should be atomic: they should test policies in
|
||||
isolation. Unlike Tempest, a Patrole test strives to only call a single
|
||||
endpoint at a time.
|
||||
* *Holistic coverage*. Patrole strives for complete coverage of the OpenStack
|
||||
API. Additionally, Patrole strives to test the API-to-policy mapping
|
||||
contained in each project's policy in code documentation.
|
||||
* *Self-contained*. Patrole should attempt to clean up after itself; whenever
|
||||
* *Complete coverage*. Patrole should validate all policy in code defaults. For
|
||||
testing, Patrole uses the API-to-policy mapping contained in each project's
|
||||
`policy in code`_ documentation where applicable.
|
||||
|
||||
For example, Nova's policy in code documentation is located in the
|
||||
`Nova repository`_ under ``nova/policies``. Likewise, Keystone's policy in
|
||||
code documentation is located in the `Keystone repository`_ under
|
||||
``keystone/common/policies``. The other OpenStack services follow the same
|
||||
directory layout pattern with respect to policy in code.
|
||||
|
||||
.. note::
|
||||
|
||||
Realistically this is not always possible because some services have
|
||||
not yet moved to policy in code.
|
||||
|
||||
* *Self-cleaning*. Patrole should attempt to clean up after itself; whenever
|
||||
possible we should tear down resources when done.
|
||||
|
||||
.. note::
|
||||
|
|
@ -45,7 +57,11 @@ testing scope is confined to policies.
|
|||
pre-provisioned credentials. Work is currently underway to clean up
|
||||
modifications made to pre-provisioned credentials.
|
||||
|
||||
* *Self-tested*. Patrole should be self-tested.
|
||||
* *Self-testing*. Patrole should be self-testing.
|
||||
|
||||
.. _policy in code: https://specs.openstack.org/openstack/oslo-specs/specs/newton/policy-in-code.html
|
||||
.. _Nova repository: https://github.com/openstack/nova/tree/master/nova/policies
|
||||
.. _Keystone repository: https://github.com/openstack/keystone/tree/master/keystone/common/policies
|
||||
|
||||
Features
|
||||
--------
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
=======
|
||||
Hacking
|
||||
=======
|
||||
====================
|
||||
Patrole Coding Guide
|
||||
====================
|
||||
|
||||
.. include:: ../../HACKING.rst
|
||||
|
|
|
|||
|
|
@ -23,8 +23,19 @@ credentials, rather than using distinct credentials for setup/teardown
|
|||
and test execution, respectively. This is especially true when considering
|
||||
custom policy rule definitions, which can be arbitrarily complex.
|
||||
|
||||
Patrole, therefore, implicitly splits up each test into 3 stages: set up,
|
||||
test execution, and teardown.
|
||||
.. _role-overriding:
|
||||
|
||||
Role Overriding
|
||||
^^^^^^^^^^^^^^^
|
||||
|
||||
Role overriding is the way Patrole is able to create resources and delete
|
||||
resources -- including those that require admin credentials -- while still
|
||||
being able to exercise the same set of Tempest credentials to perform the API
|
||||
action that authorizes the policy under test, by manipulating the role of
|
||||
the Tempest credentials.
|
||||
|
||||
Patrole implicitly splits up each test into 3 stages: set up, test execution,
|
||||
and teardown.
|
||||
|
||||
The role workflow is as follows:
|
||||
|
||||
|
|
@ -43,7 +54,7 @@ The role workflow is as follows:
|
|||
Test Setup
|
||||
----------
|
||||
|
||||
Automatic role switch in background.
|
||||
Automatic role override in background.
|
||||
|
||||
Resources can be set up inside the ``resource_setup`` class method that Tempest
|
||||
provides. These resources are typically reserved for "expensive" resources
|
||||
|
|
@ -59,7 +70,7 @@ resources one needs, without having to worry about permissions.
|
|||
Test Execution
|
||||
--------------
|
||||
|
||||
Manual role switch required.
|
||||
Manual role override required.
|
||||
|
||||
"Test execution" here means calling the API endpoint that enforces the policy
|
||||
action expected by the ``rbac_rule_validation`` decorator. Test execution
|
||||
|
|
@ -152,7 +163,7 @@ everything extraneous outside.
|
|||
Test Cleanup
|
||||
------------
|
||||
|
||||
Automatic role switch in background.
|
||||
Automatic role override in background.
|
||||
|
||||
After the test -- no matter whether it ended successfully or in failure --
|
||||
the credentials are overridden with the admin role by the Patrole framework,
|
||||
|
|
|
|||
|
|
@ -2,6 +2,14 @@
|
|||
Patrole: Tempest Plugin for RBAC Testing
|
||||
========================================
|
||||
|
||||
Overview
|
||||
========
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
overview
|
||||
|
||||
User's Guide
|
||||
============
|
||||
|
||||
|
|
@ -12,7 +20,6 @@ Patrole Configuration Guide
|
|||
:maxdepth: 2
|
||||
|
||||
configuration
|
||||
usage
|
||||
|
||||
Patrole Installation Guide
|
||||
--------------------------
|
||||
|
|
|
|||
|
|
@ -0,0 +1,178 @@
|
|||
========================
|
||||
Team and repository tags
|
||||
========================
|
||||
|
||||
.. image:: https://governance.openstack.org/tc/badges/patrole.svg
|
||||
:target: https://governance.openstack.org/tc/reference/tags/index.html
|
||||
|
||||
Patrole - The OpenStack RBAC Validation Test Suite
|
||||
==================================================
|
||||
|
||||
The documentation for Patrole is officially hosted at:
|
||||
https://docs.openstack.org/patrole/latest/
|
||||
|
||||
This is a set of integration tests to be run against a live OpenStack
|
||||
cluster. Patrole has a battery of tests dedicated to validating the correctness
|
||||
and security of the cloud's RBAC implementation.
|
||||
|
||||
Design Principles
|
||||
-----------------
|
||||
|
||||
As a `Tempest plugin`_, Patrole borrows some `design principles`_ from Tempest,
|
||||
but not all, as its testing scope is confined to policies.
|
||||
|
||||
* Patrole uses OpenStack public interfaces. Tests in Patrole should only touch
|
||||
public OpenStack APIs.
|
||||
* Patrole tests should be atomic: they should test policies in isolation.
|
||||
Unlike Tempest, a Patrole test strives to only call a single endpoint at a
|
||||
time. This is because it is important to validate each policy is authorized
|
||||
correctly and the best way to do that is to validate the policy alone.
|
||||
* Patrole should validate all policy in code defaults. For testing, Patrole
|
||||
uses the API-to-policy mapping contained in each project's `policy in code`_
|
||||
documentation where applicable.
|
||||
|
||||
For example, Nova's policy in code documentation is located in the
|
||||
`Nova repository`_ under ``nova/policies``. Likewise, Keystone's policy in
|
||||
code documentation is located in the `Keystone repository`_ under
|
||||
``keystone/common/policies``. The other OpenStack services follow the same
|
||||
directory layout pattern with respect to policy in code.
|
||||
|
||||
.. note::
|
||||
|
||||
Realistically this is not always possible because some services have
|
||||
not yet moved to policy in code.
|
||||
|
||||
* Patrole should attempt to clean up after itself; whenever possible it should
|
||||
tear down resources when done.
|
||||
|
||||
.. note::
|
||||
|
||||
Patrole modifies roles dynamically in the background, which affects
|
||||
pre-provisioned credentials. Work is currently underway to clean up
|
||||
modifications made to pre-provisioned credentials.
|
||||
|
||||
* Patrole should be self-testing.
|
||||
|
||||
.. _Tempest plugin: https://docs.openstack.org/tempest/latest/plugin.html
|
||||
.. _design principles: https://docs.openstack.org/tempest/latest/overview.html#design-principles
|
||||
.. _policy in code: https://specs.openstack.org/openstack/oslo-specs/specs/newton/policy-in-code.html
|
||||
.. _Nova repository: https://github.com/openstack/nova/tree/master/nova/policies
|
||||
.. _Keystone repository: https://github.com/openstack/keystone/tree/master/keystone/common/policies
|
||||
|
||||
Quickstart
|
||||
----------
|
||||
|
||||
To run Patrole, you must first have `Tempest`_ installed and configured
|
||||
properly. Please reference Tempest's `Quickstart`_ guide to do so. Follow all
|
||||
the steps outlined therein. Afterward, proceed with the steps below.
|
||||
|
||||
#. You first need to install Patrole. This is done with pip after you check out
|
||||
the Patrole repo::
|
||||
|
||||
$ git clone https://git.openstack.org/openstack/patrole
|
||||
$ pip install patrole/
|
||||
|
||||
This can be done within a venv.
|
||||
|
||||
.. note::
|
||||
|
||||
You may also install Patrole from source code by running::
|
||||
|
||||
pip install -e patrole/
|
||||
|
||||
#. Next you must properly configure Patrole, which is relatively
|
||||
straightforward. For details on configuring Patrole refer to the
|
||||
:ref:`patrole-configuration`.
|
||||
|
||||
#. Once the configuration is done you're now ready to run Patrole. This can
|
||||
be done using the `tempest_run`_ command. This can be done by running::
|
||||
|
||||
$ tempest run --regex '^patrole_tempest_plugin\.tests\.api'
|
||||
|
||||
There is also the option to use testr directly, or any `testr`_ based test
|
||||
runner, like `ostestr`_. For example, from the workspace dir run::
|
||||
|
||||
$ stestr --regex '(?!.*\[.*\bslow\b.*\])(^patrole_tempest_plugin\.tests\.api))'
|
||||
|
||||
will run the same set of tests as the default gate jobs.
|
||||
|
||||
You can also run Patrole tests using `tox`_. To do so, ``cd`` into the
|
||||
**Tempest** directory and run::
|
||||
|
||||
$ tox -eall-plugin -- patrole_tempest_plugin.tests.api
|
||||
|
||||
.. note::
|
||||
|
||||
It is possible to run Patrole via ``tox -eall`` in order to run Patrole
|
||||
isolated from other plugins. This can be accomplished by including the
|
||||
installation of services that currently use policy in code -- for example,
|
||||
Nova and Keystone. For example::
|
||||
|
||||
$ tox -evenv-tempest -- pip install /opt/stack/patrole /opt/stack/keystone /opt/stack/nova
|
||||
$ tox -eall -- patrole_tempest_plugin.tests.api
|
||||
|
||||
#. Log information from tests is captured in ``tempest.log`` under the Tempest
|
||||
repository. Some Patrole debugging information is captured in that log
|
||||
related to expected test results and :ref:`role-overriding`.
|
||||
|
||||
More detailed RBAC testing log output is emitted to ``patrole.log`` under
|
||||
the Patrole repository. To configure Patrole's logging, see the
|
||||
:ref:`patrole-configuration` guide.
|
||||
|
||||
.. _Tempest: https://github.com/openstack/tempest
|
||||
.. _Quickstart: https://docs.openstack.org/tempest/latest/overview.html#quickstart
|
||||
.. _tempest_run: https://docs.openstack.org/tempest/latest/run.html
|
||||
.. _testr: https://testrepository.readthedocs.org/en/latest/MANUAL.html
|
||||
.. _ostestr: https://docs.openstack.org/os-testr/latest/
|
||||
.. _tox: https://tox.readthedocs.io/en/latest/
|
||||
|
||||
RBAC Tests
|
||||
----------
|
||||
|
||||
To change the role that the patrole tests are being run as, edit
|
||||
``rbac_test_role`` in the ``patrole`` section of tempest.conf: ::
|
||||
|
||||
[patrole]
|
||||
rbac_test_role = member
|
||||
...
|
||||
|
||||
.. note::
|
||||
|
||||
The ``rbac_test_role`` is service-specific. member, for example,
|
||||
is an arbitrary role, but by convention is used to designate the default
|
||||
non-admin role in the system. Most Patrole tests should be run with
|
||||
**admin** and **member** roles. However, other services may use entirely
|
||||
different roles.
|
||||
|
||||
For more information about the member role and its nomenclature,
|
||||
please see: `<https://ask.openstack.org/en/question/4759/member-vs-_member_/>`__.
|
||||
|
||||
Unit Tests
|
||||
----------
|
||||
|
||||
Patrole also has a set of unit tests which test the Patrole code itself. These
|
||||
tests can be run by specifying the test discovery path::
|
||||
|
||||
$ stestr --test-path ./patrole_tempest_plugin/tests/unit run
|
||||
|
||||
By setting ``--test-path`` option to ``./patrole_tempest_plugin/tests/unit``
|
||||
it specifies that test discovery should only be run on the unit test directory.
|
||||
|
||||
Alternatively, there are the py27 and py35 tox jobs which will run the unit
|
||||
tests with the corresponding version of Python.
|
||||
|
||||
One common activity is to just run a single test; you can do this with tox
|
||||
simply by specifying to just run py27 or py35 tests against a single test::
|
||||
|
||||
$ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils.RBACUtilsTest.test_override_role_with_missing_admin_role
|
||||
|
||||
Or all tests in the test_rbac_utils.py file::
|
||||
|
||||
$ tox -e py27 -- -n patrole_tempest_plugin.tests.unit.test_rbac_utils
|
||||
|
||||
You may also use regular expressions to run any matching tests::
|
||||
|
||||
$ tox -e py27 -- test_rbac_utils
|
||||
|
||||
For more information on these options and details about stestr, please see the
|
||||
`stestr documentation <http://stestr.readthedocs.io/en/latest/MANUAL.html>`_.
|
||||
|
|
@ -1,61 +0,0 @@
|
|||
.. _patrole-usage:
|
||||
|
||||
========
|
||||
Usage
|
||||
========
|
||||
|
||||
Patrole (API) Tests
|
||||
===================
|
||||
|
||||
If Patrole is installed correctly, then the RBAC tests can be executed
|
||||
from inside the tempest root directory as follows::
|
||||
|
||||
$ tox -eall-plugin -- patrole_tempest_plugin.tests.api
|
||||
|
||||
To execute patrole tests for a specific module, run::
|
||||
|
||||
$ tox -eall-plugin -- patrole_tempest_plugin.tests.api.compute
|
||||
|
||||
.. note::
|
||||
|
||||
It is possible to run Patrole via ``tox -eall`` in order to run Patrole
|
||||
isolated from other plugins. This can be accomplished by including the
|
||||
installation of services that currently use policy in code -- for example,
|
||||
Nova and Keystone. For example::
|
||||
|
||||
$ tox -evenv-tempest -- pip install /opt/stack/patrole /opt/stack/keystone /opt/stack/nova
|
||||
$ tox -eall -- patrole_tempest_plugin.tests.api
|
||||
..
|
||||
|
||||
To change the role that the patrole tests are being run as, edit
|
||||
``rbac_test_role`` in the ``patrole`` section of tempest.conf: ::
|
||||
|
||||
[patrole]
|
||||
rbac_test_role = Member
|
||||
...
|
||||
|
||||
.. note::
|
||||
|
||||
The ``rbac_test_role`` is service-specific. Member, for example,
|
||||
is an arbitrary role, but by convention is used to designate the default
|
||||
non-admin role in the system. Most patrole tests should be run with
|
||||
**admin** and **Member** roles. However, some services, like Heat, take
|
||||
advantage of a role called **heat_stack_user**, as it appears frequently
|
||||
in Heat's policy.json.
|
||||
|
||||
For more information about the Member role,
|
||||
please see: `<https://ask.openstack.org/en/question/4759/member-vs-_member_/>`__.
|
||||
|
||||
Unit Tests
|
||||
==========
|
||||
|
||||
Patrole includes unit tests for its RBAC framework. They can be run by
|
||||
executing::
|
||||
|
||||
$ tox -e py27
|
||||
|
||||
or::
|
||||
|
||||
$ tox -e py35
|
||||
|
||||
against the Python 3.5 interpreter.
|
||||
Loading…
Reference in New Issue