From 4d4cb1e7d39fef5ce0d2489a830864a2ac829df0 Mon Sep 17 00:00:00 2001 From: Felipe Monteiro Date: Sun, 29 Jul 2018 13:44:10 -0400 Subject: [PATCH] Add documentation about white box/black box testing to HACKING This patch set adds documentation about white box vs block box testing and their relationship in Patrole. This is so that devs/test writers understand that Patrole is a bit different than Tempest and requires digging a bit deeper in the internals of the API implementation in order to properly test RBAC. Also removes a misleading link in the README.rst section. The discussion on member vs. _member_ role is very outdated and so a link is provided to the RBAC overview section instead which is concerned with documenting such information. Change-Id: I0a014c2e917caeb058dd5b5294dd0af2e5e49132 --- HACKING.rst | 31 +++++++++++++++++++++++++++++++ README.rst | 6 ++++-- doc/source/rbac-overview.rst | 2 ++ 3 files changed, 37 insertions(+), 2 deletions(-) diff --git a/HACKING.rst b/HACKING.rst index 28a977d6..68d0d918 100644 --- a/HACKING.rst +++ b/HACKING.rst @@ -118,3 +118,34 @@ The same `Tempest logic`_ regarding new tests for existing features or policies also applies to Patrole. .. _Tempest logic: https://docs.openstack.org/tempest/latest/HACKING.html#new-tests-for-existing-features + + +Black Box vs. White Box Testing +------------------------------- + +Tempest is a `black box testing framework`_, meaning that it is concerned with +testing public API endpoints and doesn't concern itself with testing internal +implementation details. Patrole, as a Tempest plugin, also falls underneath +the category of black box testing. However, even with policy in code +documentation, some degree of white box testing is required in order to +correctly write RBAC tests. + +This is because :ref:`policy-in-code` documentation, while useful in many +respects, is usually quite brief and its main purpose is to help operators +understand how to customize policy configuration rather than to help +developers understand complex policy authorization work flows. For example, +policy in code documentation doesn't make deriving +:ref:`multiple policies ` easy. Such documentation also +doesn't usually mention that a specific parameter needs to be set, or that a +particular microversion must be enabled, or that a particular set of +prerequisite API or policy actions must be executed, in order for the policy +under test to be enforced by the server. This means that test writers must +account for the internal RBAC implementation in API code in order to correctly +understand the complete RBAC work flow within an API. + +Besides, as mentioned :ref:`elsewhere ` in this +documentation, not all services currently implement policy in code, making +some degree of white box testing a "necessary evil" for writing robust RBAC +tests. + +.. _black box testing framework: https://docs.openstack.org/tempest/latest/HACKING.html#negative-tests diff --git a/README.rst b/README.rst index 2028536c..b07b1095 100644 --- a/README.rst +++ b/README.rst @@ -201,8 +201,10 @@ To change the role that the patrole tests are being run as, edit **admin** and **member** roles. However, other services may use entirely different roles. -For more information about the member role and its nomenclature, -please see: ``__. +For more information about RBAC, reference the `rbac-overview`_ +documentation page. + +.. _rbac-overview: https://docs.openstack.org/patrole/latest/rbac-overview.html Unit Tests ---------- diff --git a/doc/source/rbac-overview.rst b/doc/source/rbac-overview.rst index 5eefa5cc..4b2023ea 100644 --- a/doc/source/rbac-overview.rst +++ b/doc/source/rbac-overview.rst @@ -1,3 +1,5 @@ +.. _rbac-overview: + ================================== Role-Based Access Control Overview ==================================