Add documentation about white box/black box testing to HACKING

This patch set adds documentation about white box vs block box
testing and their relationship in Patrole. This is so that
devs/test writers understand that Patrole is a bit different than
Tempest and requires digging a bit deeper in the internals of
the API implementation in order to properly test RBAC.

Also removes a misleading link in the README.rst section. The
discussion on member vs. _member_ role is very outdated and
so a link is provided to the RBAC overview section instead which
is concerned with documenting such information.

Change-Id: I0a014c2e917caeb058dd5b5294dd0af2e5e49132
This commit is contained in:
Felipe Monteiro 2018-07-29 13:44:10 -04:00
parent 05a73499ce
commit 4d4cb1e7d3
3 changed files with 37 additions and 2 deletions

View File

@ -118,3 +118,34 @@ The same `Tempest logic`_ regarding new tests for existing features or
policies also applies to Patrole.
.. _Tempest logic: https://docs.openstack.org/tempest/latest/HACKING.html#new-tests-for-existing-features
Black Box vs. White Box Testing
-------------------------------
Tempest is a `black box testing framework`_, meaning that it is concerned with
testing public API endpoints and doesn't concern itself with testing internal
implementation details. Patrole, as a Tempest plugin, also falls underneath
the category of black box testing. However, even with policy in code
documentation, some degree of white box testing is required in order to
correctly write RBAC tests.
This is because :ref:`policy-in-code` documentation, while useful in many
respects, is usually quite brief and its main purpose is to help operators
understand how to customize policy configuration rather than to help
developers understand complex policy authorization work flows. For example,
policy in code documentation doesn't make deriving
:ref:`multiple policies <multiple-policies>` easy. Such documentation also
doesn't usually mention that a specific parameter needs to be set, or that a
particular microversion must be enabled, or that a particular set of
prerequisite API or policy actions must be executed, in order for the policy
under test to be enforced by the server. This means that test writers must
account for the internal RBAC implementation in API code in order to correctly
understand the complete RBAC work flow within an API.
Besides, as mentioned :ref:`elsewhere <design-principles>` in this
documentation, not all services currently implement policy in code, making
some degree of white box testing a "necessary evil" for writing robust RBAC
tests.
.. _black box testing framework: https://docs.openstack.org/tempest/latest/HACKING.html#negative-tests

View File

@ -201,8 +201,10 @@ To change the role that the patrole tests are being run as, edit
**admin** and **member** roles. However, other services may use entirely
different roles.
For more information about the member role and its nomenclature,
please see: `<https://ask.openstack.org/en/question/4759/member-vs-_member_/>`__.
For more information about RBAC, reference the `rbac-overview`_
documentation page.
.. _rbac-overview: https://docs.openstack.org/patrole/latest/rbac-overview.html
Unit Tests
----------

View File

@ -1,3 +1,5 @@
.. _rbac-overview:
==================================
Role-Based Access Control Overview
==================================