refactor: Rename PluginRbacTest => ExtRbacTest

This patch set is a follow up on discussion in [0] which
concluded that Patrole should not test Neutron plugins.

The pertinent discussion from [0] is:

Patrole can test Neutron extensions but not plugins in tree
and that requires renaming the classes
(.*PluginRbacTest => .*ExtRbacTest) to make it clearer.

Having to support all the Neutron plugins in tree is out of scope
for Patrole. Clarifying documentation (which will be done in follow
up) can be added to Patrole documentation to make it clear that
it will only test "main" OpenStack projects, like Tempest.

[0] https://review.openstack.org/#/c/599869/

Change-Id: Iab029f2f875ce2268de12cc2a40e30f2f1a913fe
This commit is contained in:
Felipe Monteiro 2018-10-31 23:28:39 -04:00
parent 742b73767b
commit bbbdd9391d
17 changed files with 86 additions and 85 deletions

View File

@ -133,11 +133,14 @@
c-bak: false
- job:
name: patrole-plugin-base
name: patrole-extension-base
parent: patrole-base
description: |
Patrole plugin job for admin and member roles which
runs RBAC tests for neutron-tempest-plugin APIs (if the plugin is installed).
Patrole plugin job for admin and member roles which runs RBAC tests for
neutron-tempest-plugin APIs (if the plugin is installed).
Covers Neutron extension functionality only. Should not be used for
supporting Neutron plugins like fwaas.
required-projects:
- name: openstack/tempest
- name: openstack/patrole
@ -156,22 +159,22 @@
neutron-qos: true
- job:
name: patrole-plugin-member
parent: patrole-plugin-base
name: patrole-extension-member
parent: patrole-extension-base
voting: false
vars:
devstack_localrc:
RBAC_TEST_ROLES: member
tempest_test_regex: (?=.*PluginRbacTest)(^patrole_tempest_plugin\.tests\.api)
tempest_test_regex: (?=.*ExtRbacTest)(^patrole_tempest_plugin\.tests\.api)
- job:
name: patrole-plugin-admin
parent: patrole-plugin-base
name: patrole-extension-admin
parent: patrole-extension-base
voting: false
vars:
devstack_localrc:
RBAC_TEST_ROLES: admin
tempest_test_regex: (?=.*PluginRbacTest)(^patrole_tempest_plugin\.tests\.api)
tempest_test_regex: (?=.*ExtRbacTest)(^patrole_tempest_plugin\.tests\.api)
- project:
templates:
@ -193,8 +196,8 @@
- patrole-py35-member
- patrole-multinode-admin
- patrole-multinode-member
- patrole-plugin-admin
- patrole-plugin-member
- patrole-extension-admin
- patrole-extension-member
gate:
jobs:
- patrole-admin

View File

@ -39,9 +39,9 @@ The following are Patrole's specific Commandments:
- [P102] RBAC test class names must end in 'RbacTest'
- [P103] ``self.client`` must not be used as a client alias; this allows for
code that is more maintainable and easier to read
- [P104] RBAC `plugin test class`_ names must end in 'PluginRbacTest'
- [P104] RBAC `extension test class`_ names must end in 'ExtRbacTest'
.. _plugin test class: https://github.com/openstack/patrole/tree/master/patrole_tempest_plugin/tests/api/network#neutron-plugin-tests
.. _extension test class: https://github.com/openstack/patrole/tree/master/patrole_tempest_plugin/tests/api/network#neutron-extension-rbac-tests
Role Overriding
---------------

View File

@ -36,8 +36,8 @@ RBAC_CLASS_NAME_RE = re.compile(r'class .+RbacTest')
RULE_VALIDATION_DECORATOR = re.compile(
r'\s*@rbac_rule_validation.action\(.*')
IDEMPOTENT_ID_DECORATOR = re.compile(r'\s*@decorators\.idempotent_id\((.*)\)')
PLUGIN_RBAC_TEST = re.compile(
r"class .+\(.+PluginRbacTest\)|class .+PluginRbacTest\(.+\)")
EXT_RBAC_TEST = re.compile(
r"class .+\(.+ExtRbacTest\)|class .+ExtRbacTest\(.+\)")
have_rbac_decorator = False
@ -213,15 +213,15 @@ def no_client_alias_in_test_cases(logical_line, filename):
return 0, "Do not use 'self.client' as a service client alias"
def no_plugin_rbac_test_suffix_in_plugin_test_class_name(physical_line,
filename):
"""Check that Plugin RBAC class names end with "PluginRbacTest"
def no_extension_rbac_test_suffix_in_plugin_test_class_name(physical_line,
filename):
"""Check that Extension RBAC class names end with "ExtRbacTest"
P104
"""
suffix = "PluginRbacTest"
suffix = "ExtRbacTest"
if "patrole_tempest_plugin/tests/api" in filename:
if PLUGIN_RBAC_TEST.match(physical_line):
if EXT_RBAC_TEST.match(physical_line):
subclass, superclass = physical_line.split('(')
subclass = subclass.split('class')[1].strip()
superclass = superclass.split(')')[0].strip()
@ -238,16 +238,16 @@ def no_plugin_rbac_test_suffix_in_plugin_test_class_name(physical_line,
superclass.startswith("Base")):
return
# Case 1: Subclass of "BasePluginRbacTest" must end in `suffix`
# Case 1: Subclass of "BaseExtRbacTest" must end in `suffix`
# Case 2: Subclass that ends in `suffix` must inherit from base
# class ending in `suffix`.
if not subclass.endswith(suffix):
error = ("Plugin RBAC test subclasses must end in "
"'PluginRbacTest'")
"'ExtRbacTest'")
return len(subclass) - 1, error
elif not superclass.endswith(suffix):
error = ("Plugin RBAC test subclasses must inherit from a "
"'PluginRbacTest' base class")
"'ExtRbacTest' base class")
return len(superclass) - 1, error
@ -263,4 +263,4 @@ def factory(register):
register(no_rbac_rule_validation_decorator)
register(no_rbac_suffix_in_test_filename)
register(no_rbac_test_suffix_in_test_class_name)
register(no_plugin_rbac_test_suffix_in_plugin_test_class_name)
register(no_extension_rbac_test_suffix_in_plugin_test_class_name)

View File

@ -10,7 +10,7 @@ These tests are RBAC tests for Neutron and its associated plugins. They are
broken up into the following categories:
* :ref:`neutron-rbac-tests`
* :ref:`neutron-plugin-rbac-tests`
* :ref:`neutron-extension-rbac-tests`
.. _neutron-rbac-tests:
@ -22,18 +22,16 @@ test many of the Neutron policies found in the service's `policy.json file`_.
These tests are gated in many `Zuul jobs`_ (master, n-1, n-2) against many
roles (member, admin).
.. _neutron-plugin-rbac-tests:
.. _neutron-extension-rbac-tests:
Neutron plugin tests
^^^^^^^^^^^^^^^^^^^^
Neutron extension tests
^^^^^^^^^^^^^^^^^^^^^^^
The Neutron RBAC plugin tests focus on testing RBAC for various Neutron
extensions and plugins, or, stated differently:
extensions, or, stated differently: tests that rely on
`neutron-tempest-plugin`_.
* tests that rely on `neutron-tempest-plugin`_
* external Neutron plugins
These tests inherit from the base class ``BaseNetworkPluginRbacTest``. If an
These tests inherit from the base class ``BaseNetworkExtRbacTest``. If an
extension or plugin is not enabled in the cloud, the corresponding tests are
gracefully skipped.

View File

@ -27,7 +27,7 @@ class BaseNetworkRbacTest(rbac_utils.RbacUtilsMixin,
cls.setup_rbac_utils()
class BaseNetworkPluginRbacTest(BaseNetworkRbacTest):
class BaseNetworkExtRbacTest(BaseNetworkRbacTest):
"""Base class to be used with tests that require neutron-tempest-plugin.
"""
@ -35,14 +35,14 @@ class BaseNetworkPluginRbacTest(BaseNetworkRbacTest):
def get_auth_providers(cls):
"""Register auth_provider from neutron-tempest-plugin.
"""
providers = super(BaseNetworkPluginRbacTest, cls).get_auth_providers()
providers = super(BaseNetworkExtRbacTest, cls).get_auth_providers()
if cls.is_neutron_tempest_plugin_avaliable():
providers.append(cls.ntp_client.auth_provider)
return providers
@classmethod
def skip_checks(cls):
super(BaseNetworkPluginRbacTest, cls).skip_checks()
super(BaseNetworkExtRbacTest, cls).skip_checks()
if not cls.is_neutron_tempest_plugin_avaliable():
msg = ("neutron-tempest-plugin not installed.")
@ -59,7 +59,7 @@ class BaseNetworkPluginRbacTest(BaseNetworkRbacTest):
@classmethod
def get_client_manager(cls, credential_type=None, roles=None,
force_new=None):
manager = super(BaseNetworkPluginRbacTest, cls).get_client_manager(
manager = super(BaseNetworkExtRbacTest, cls).get_client_manager(
credential_type=credential_type,
roles=roles,
force_new=force_new

View File

@ -23,18 +23,18 @@ from patrole_tempest_plugin import rbac_rule_validation
from patrole_tempest_plugin.tests.api.network import rbac_base as base
class AddressScopePluginRbacTest(base.BaseNetworkPluginRbacTest):
class AddressScopeExtRbacTest(base.BaseNetworkExtRbacTest):
@classmethod
def skip_checks(cls):
super(AddressScopePluginRbacTest, cls).skip_checks()
super(AddressScopeExtRbacTest, cls).skip_checks()
if not utils.is_extension_enabled('address-scope', 'network'):
msg = "address-scope extension not enabled."
raise cls.skipException(msg)
@classmethod
def resource_setup(cls):
super(AddressScopePluginRbacTest, cls).resource_setup()
super(AddressScopeExtRbacTest, cls).resource_setup()
cls.network = cls.create_network()
def _create_address_scope(self, name=None, **kwargs):

View File

@ -238,18 +238,18 @@ class DHCPAgentSchedulersRbacTest(base.BaseNetworkRbacTest):
self.agent['id'], network_id=network_id)
class L3AgentsPluginRbacTest(base.BaseNetworkPluginRbacTest):
class L3AgentsExtRbacTest(base.BaseNetworkExtRbacTest):
@classmethod
def skip_checks(cls):
super(L3AgentsPluginRbacTest, cls).skip_checks()
super(L3AgentsExtRbacTest, cls).skip_checks()
if not utils.is_extension_enabled('l3_agent_scheduler', 'network'):
msg = "l3_agent_scheduler extension not enabled."
raise cls.skipException(msg)
@classmethod
def resource_setup(cls):
super(L3AgentsPluginRbacTest, cls).resource_setup()
super(L3AgentsExtRbacTest, cls).resource_setup()
name = data_utils.rand_name(cls.__name__ + '-Router')
cls.router = cls.ntp_client.create_router(name)['router']

View File

@ -20,11 +20,11 @@ from patrole_tempest_plugin import rbac_rule_validation
from patrole_tempest_plugin.tests.api.network import rbac_base as base
class AutoAllocationTopologyPluginRbacTest(base.BaseNetworkPluginRbacTest):
class AutoAllocationTopologyExtRbacTest(base.BaseNetworkExtRbacTest):
@classmethod
def skip_checks(cls):
super(AutoAllocationTopologyPluginRbacTest, cls).skip_checks()
super(AutoAllocationTopologyExtRbacTest, cls).skip_checks()
if not utils.is_extension_enabled('auto-allocated-topology',
'network'):
msg = "auto-allocated-topology extension not enabled."

View File

@ -22,18 +22,18 @@ from patrole_tempest_plugin import rbac_rule_validation
from patrole_tempest_plugin.tests.api.network import rbac_base as base
class DscpMarkingRulePluginRbacTest(base.BaseNetworkPluginRbacTest):
class DscpMarkingRuleExtRbacTest(base.BaseNetworkExtRbacTest):
@classmethod
def skip_checks(cls):
super(DscpMarkingRulePluginRbacTest, cls).skip_checks()
super(DscpMarkingRuleExtRbacTest, cls).skip_checks()
if not utils.is_extension_enabled('qos', 'network'):
msg = "qos extension not enabled."
raise cls.skipException(msg)
@classmethod
def resource_setup(cls):
super(DscpMarkingRulePluginRbacTest, cls).resource_setup()
super(DscpMarkingRuleExtRbacTest, cls).resource_setup()
name = data_utils.rand_name(cls.__class__.__name__ + '-qos')
cls.policy_id = cls.ntp_client.create_qos_policy(
name=name)["policy"]["id"]

View File

@ -23,11 +23,11 @@ from patrole_tempest_plugin import rbac_rule_validation
from patrole_tempest_plugin.tests.api.network import rbac_base as base
class FlavorsPluginRbacTest(base.BaseNetworkPluginRbacTest):
class FlavorsExtRbacTest(base.BaseNetworkExtRbacTest):
@classmethod
def resource_setup(cls):
super(FlavorsPluginRbacTest, cls).resource_setup()
super(FlavorsExtRbacTest, cls).resource_setup()
providers = cls.ntp_client.list_service_providers()
if not providers["service_providers"]:
raise cls.skipException("No service_providers available.")
@ -120,10 +120,10 @@ class FlavorsPluginRbacTest(base.BaseNetworkPluginRbacTest):
self.ntp_client.list_flavors()
class FlavorsServiceProfilePluginRbacTest(base.BaseNetworkPluginRbacTest):
class FlavorsServiceProfileExtRbacTest(base.BaseNetworkExtRbacTest):
@classmethod
def resource_setup(cls):
super(FlavorsServiceProfilePluginRbacTest, cls).resource_setup()
super(FlavorsServiceProfileExtRbacTest, cls).resource_setup()
providers = cls.ntp_client.list_service_providers()
if not providers["service_providers"]:
raise cls.skipException("No service_providers available.")

View File

@ -22,18 +22,18 @@ from patrole_tempest_plugin import rbac_rule_validation
from patrole_tempest_plugin.tests.api.network import rbac_base as base
class PolicyBandwidthLimitRulePluginRbacTest(base.BaseNetworkPluginRbacTest):
class PolicyBandwidthLimitRuleExtRbacTest(base.BaseNetworkExtRbacTest):
@classmethod
def skip_checks(cls):
super(PolicyBandwidthLimitRulePluginRbacTest, cls).skip_checks()
super(PolicyBandwidthLimitRuleExtRbacTest, cls).skip_checks()
if not utils.is_extension_enabled('qos', 'network'):
msg = "qos extension not enabled."
raise cls.skipException(msg)
@classmethod
def resource_setup(cls):
super(PolicyBandwidthLimitRulePluginRbacTest, cls).resource_setup()
super(PolicyBandwidthLimitRuleExtRbacTest, cls).resource_setup()
name = data_utils.rand_name(cls.__class__.__name__ + '-qos-policy')
cls.policy_id = cls.ntp_client.create_qos_policy(
name=name)["policy"]["id"]

View File

@ -22,18 +22,18 @@ from patrole_tempest_plugin import rbac_rule_validation
from patrole_tempest_plugin.tests.api.network import rbac_base as base
class PolicyMinimumBandwidthRulePluginRbacTest(base.BaseNetworkPluginRbacTest):
class PolicyMinimumBandwidthRuleExtRbacTest(base.BaseNetworkExtRbacTest):
@classmethod
def skip_checks(cls):
super(PolicyMinimumBandwidthRulePluginRbacTest, cls).skip_checks()
super(PolicyMinimumBandwidthRuleExtRbacTest, cls).skip_checks()
if not utils.is_extension_enabled('qos', 'network'):
msg = "qos extension not enabled."
raise cls.skipException(msg)
@classmethod
def resource_setup(cls):
super(PolicyMinimumBandwidthRulePluginRbacTest, cls).resource_setup()
super(PolicyMinimumBandwidthRuleExtRbacTest, cls).resource_setup()
name = data_utils.rand_name(cls.__class__.__name__ + '-qos')
cls.policy_id = cls.ntp_client.create_qos_policy(
name=name)["policy"]["id"]

View File

@ -22,18 +22,18 @@ from patrole_tempest_plugin import rbac_rule_validation
from patrole_tempest_plugin.tests.api.network import rbac_base as base
class QosPluginRbacTest(base.BaseNetworkPluginRbacTest):
class QosExtRbacTest(base.BaseNetworkExtRbacTest):
@classmethod
def skip_checks(cls):
super(QosPluginRbacTest, cls).skip_checks()
super(QosExtRbacTest, cls).skip_checks()
if not utils.is_extension_enabled('qos', 'network'):
msg = "qos extension not enabled."
raise cls.skipException(msg)
@classmethod
def resource_setup(cls):
super(QosPluginRbacTest, cls).resource_setup()
super(QosExtRbacTest, cls).resource_setup()
cls.network = cls.create_network()
def create_policy(self, name=None):

View File

@ -20,11 +20,11 @@ from patrole_tempest_plugin import rbac_rule_validation
from patrole_tempest_plugin.tests.api.network import rbac_base as base
class RbacPoliciesPluginRbacTest(base.BaseNetworkPluginRbacTest):
class RbacPoliciesExtRbacTest(base.BaseNetworkExtRbacTest):
@classmethod
def resource_setup(cls):
super(RbacPoliciesPluginRbacTest, cls).resource_setup()
super(RbacPoliciesExtRbacTest, cls).resource_setup()
cls.tenant_id = cls.os_primary.credentials.tenant_id
cls.network_id = cls.create_network()['id']

View File

@ -23,18 +23,18 @@ from patrole_tempest_plugin import rbac_rule_validation
from patrole_tempest_plugin.tests.api.network import rbac_base as base
class SegmentsPluginRbacTest(base.BaseNetworkPluginRbacTest):
class SegmentsExtRbacTest(base.BaseNetworkExtRbacTest):
@classmethod
def skip_checks(cls):
super(SegmentsPluginRbacTest, cls).skip_checks()
super(SegmentsExtRbacTest, cls).skip_checks()
if not utils.is_extension_enabled('segment', 'network'):
msg = "segment extension not enabled."
raise cls.skipException(msg)
@classmethod
def resource_setup(cls):
super(SegmentsPluginRbacTest, cls).resource_setup()
super(SegmentsExtRbacTest, cls).resource_setup()
cls.network = cls.create_network()
@classmethod

View File

@ -21,18 +21,18 @@ from patrole_tempest_plugin import rbac_rule_validation
from patrole_tempest_plugin.tests.api.network import rbac_base as base
class TrunksPluginRbacTest(base.BaseNetworkPluginRbacTest):
class TrunksExtRbacTest(base.BaseNetworkExtRbacTest):
@classmethod
def skip_checks(cls):
super(TrunksPluginRbacTest, cls).skip_checks()
super(TrunksExtRbacTest, cls).skip_checks()
if not utils.is_extension_enabled('trunk', 'network'):
msg = "trunk extension not enabled."
raise cls.skipException(msg)
@classmethod
def resource_setup(cls):
super(TrunksPluginRbacTest, cls).resource_setup()
super(TrunksExtRbacTest, cls).resource_setup()
cls.network = cls.create_network()
cls.port_id = cls.create_port(cls.network)["id"]

View File

@ -257,10 +257,10 @@ class RBACHackingTestCase(base.TestCase):
" cls.client",
"./patrole_tempest_plugin/tests/api/fake_test_rbac.py"))
def test_no_plugin_rbac_test_suffix_in_plugin_test_class_name(self):
check = checks.no_plugin_rbac_test_suffix_in_plugin_test_class_name
def no_extension_rbac_test_suffix_in_plugin_test_class_name(self):
check = checks.no_extension_rbac_test_suffix_in_plugin_test_class_name
# Passing cases: these do not inherit from "PluginRbacTest" base class.
# Passing cases: these do not inherit from "ExtRbacTest" base class.
self.assertFalse(check(
"class FakeRbacTest(BaseFakeRbacTest)",
"./patrole_tempest_plugin/tests/api/fake_test_rbac.py"))
@ -270,39 +270,39 @@ class RBACHackingTestCase(base.TestCase):
# Passing cases: these **do** end in correct test class suffix.
self.assertFalse(check(
"class FakePluginRbacTest(BaseFakePluginRbacTest)",
"class FakeExtRbacTest(BaseFakeExtRbacTest)",
"./patrole_tempest_plugin/tests/api/fake_test_rbac.py"))
self.assertFalse(check(
"class FakePluginRbacTest(base.BaseFakePluginRbacTest)",
"class FakeExtRbacTest(base.BaseFakeExtRbacTest)",
"./patrole_tempest_plugin/tests/api/fake_test_rbac.py"))
# Passing cases: plugin base class inherits from another base class.
self.assertFalse(check(
"class BaseFakePluginRbacTest(base.BaseFakeRbacTest)",
"class BaseFakeExtRbacTest(base.BaseFakeRbacTest)",
"./patrole_tempest_plugin/tests/api/fake_test_rbac.py"))
self.assertFalse(check(
"class BaseFakePluginRbacTest(BaseFakeRbacTest)",
"class BaseFakeExtRbacTest(BaseFakeRbacTest)",
"./patrole_tempest_plugin/tests/api/fake_test_rbac.py"))
# Failing cases: these **do not** end in correct test class suffix.
# Case 1: RbacTest subclass doesn't end in PluginRbacTest.
# Case 1: RbacTest subclass doesn't end in ExtRbacTest.
self.assertTrue(check(
"class FakeRbacTest(base.BaseFakePluginRbacTest)",
"class FakeRbacTest(base.BaseFakeExtRbacTest)",
"./patrole_tempest_plugin/tests/api/fake_test_rbac.py"))
self.assertTrue(check(
"class FakeRbacTest(BaseFakePluginRbacTest)",
"class FakeRbacTest(BaseFakeExtRbacTest)",
"./patrole_tempest_plugin/tests/api/fake_test_rbac.py"))
self.assertTrue(check(
"class FakeRbacTest(BaseFakeNetworkPluginRbacTest)",
"class FakeRbacTest(BaseFakeNetworkExtRbacTest)",
"./patrole_tempest_plugin/tests/api/network/fake_test_rbac.py"))
# Case 2: PluginRbacTest subclass doesn't inherit from
# BasePluginRbacTest.
# Case 2: ExtRbacTest subclass doesn't inherit from
# BaseExtRbacTest.
self.assertTrue(check(
"class FakePluginRbacTest(base.BaseFakeRbacTest)",
"class FakeExtRbacTest(base.BaseFakeRbacTest)",
"./patrole_tempest_plugin/tests/api/fake_test_rbac.py"))
self.assertTrue(check(
"class FakePluginRbacTest(BaseFakeRbacTest)",
"class FakeExtRbacTest(BaseFakeRbacTest)",
"./patrole_tempest_plugin/tests/api/fake_test_rbac.py"))
self.assertTrue(check(
"class FakeNeutronPluginRbacTest(BaseFakeNeutronRbacTest)",
"class FakeNeutronExtRbacTest(BaseFakeNeutronRbacTest)",
"./patrole_tempest_plugin/tests/api/fake_test_rbac.py"))