Patrole project is not active anymore and its gate is broken.
We waited for couple of cycle to see if there is any interest
in this project and anyone can maintain it. But we did not get any
new maintainers and current QA team does not have bandwidth/interest
to continue maintaining it.
This project was for RBAc testing which is moving towards unit/functional
tests on service side as well as tempest plugins tests.
In QA 2023.2 PTG, we decided to retire this project
- https://etherpad.opendev.org/p/qa-bobcat-ptg
Change-Id: I7721cf06104e5871ec27cdd87d4608dace60a8b7
This PS updates Tempest to 30.0.0. Due to
55414580c2
some refactoring was required around wait_for_interface_detach.
Additionally, the variables:
min_microversion
max_microversion
needed to be renamed to:
volume_min_microversion
volume_max_microversion
for volume related tests. See:
https://review.opendev.org/c/openstack/tempest/+/813676
Change-Id: Ie2183fdd2812d5d2fdfdc0815bf96e5c47a9f1e8
stable/stein is not suported in Patrole now, so
let's remove their jobs from master gate.
In order to pass the CI, the following changes are also made:
* Added skip for Nova policy
"os_compute_api:os-admin-actions:reset_network", which was removed in
https://review.opendev.org/c/openstack/nova/+/749315
* Removed openstack-tox-lower-constraints job for now until we have a
solution.
Change-Id: Id73342c24342637edc37104f2112235a2edcac39
Nova is moving to new policy defaults in ussuri[1] where
few polciies are made more granular to adopt the new defaults.
With granularity in few policies make change in policy name so
we have update the patrole tests to start checking against the
new policy names from ussuri onwards.
This commit updates the security group and server password
policy tests to move to new policies from ussuri onwards.
Also add the already fixed instance action policy in reno
Also fix the gate to parse the combining of deprecated rule
check_str with oslo policy parser instead of string processing.
Story: #2007585
Task: #39516
[1] https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/policy-defaults-refresh.html
Change-Id: If661299231d548ce40a2e340b1ddb9ebe8d3f964
Nova is moving to new policy defaults in ussuri[1] where
few polciies are made more granular to adopt the new defaults.
With granularity in few policies make change in policy name so
we have update the patrole tests to start checking against the
new policy names from ussuri onwards.
This commit updates the os-deferred_delete and os-attach-interfaces
tests to move to new policies from ussuri onwards.
[1] https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/policy-defaults-refresh.html
Change-Id: I399e9e2bf944cfbba4b47f05ba2f529cbc1b9ea1
Devstack added to create the shared network for tempest testing
which makes sure tempest and other testing part works fine when
testing env has shared network.
- 683454f319/lib/tempest (L259)
Now patrole started failing because of the above shared network.
Details: {u'message': u'Multiple possible networks found, use a
Network ID to be more specific.', u'code': 409}
http://logs.openstack.org/55/648255/1/check/patrole-admin/63b6639/testr_results.html.gz
When patrol tests call create_tesrt_server, network id is not passed
as kwargs which means it reply on tempest to add the network id for
creating the server[1]. Tempest fail to add the network id because it
relies on used credential to have the network created for them[2].
We should create the network for requested credential using
set_network_resources() method.
Following tests are skipped due to their failure due to bad request error:
- test_update_network_provider_network_type[id-d064ef96-662b-47b6-94b7-9106dcd7ba8c]
- test_update_network_provider_physical_network[id-e3a55660-f75c-494e-a1b1-a8b36cc789ef]
- test_update_network_provider_segmentation_id[id-f6164228-b670-45fd-9ff9-b101930318c7]
[1] 75ea3dbaf9/tempest/api/compute/base.py (L234)
[2] 3639f91c36/tempest/lib/common/fixed_network.py (L95)
Story: 2005369
Task: 30345
Change-Id: I1281aa090c167c1e3e401a1707d196a69e7b5cd2
The override_role function no longer needs test_obj parameter.
Also removing self.rbac_utils variable.
Story: 2002604
Task: 22223
Change-Id: I1ee95e3051d7bd27f73df818fa2b64caa07c1ed2
- When the URL refers to cloning or using git repositories, use the
cloning URL (https://git.openstack.org/<namespace>/<project>)
- When the URL refers to the browsable version of the repository, point to
the cgit frontend (https://git.openstack.org/cgit/<namespace>/<project>)
Change-Id: Iaeaa153a05aa85b9cf7451ae3c28aec56722222c
This patch set breaks up RbacMalformedException into the following
discrete exceptions:
* RbacMissingAttributeResponseBody
* RbacPartialResponseBody
* RbacEmptyResponseBody
Each of the exception classes deals with a different type of
failure related to a soft authorization failure [0] which
means that a failure occurred server side related to RBAC
authorization, but the result of which an incomplete, partial
or empty response body (with a 2xx status code).
* incomplete means that the response body (for show or list)
is missing certain attributes
* partial means that a list response only returned a subset of
the possible results available.
* empty means that the show or list response body is entirely
empty
Because RbacMalformedException is not part of a stable library
it is removed altogether; we do not need to deprecate it.
[0] http://git.openstack.org/cgit/openstack/patrole/tree/doc/source/rbac-overview.rst#n232
Story: 2003843
Task: 26633
Change-Id: I2c76c3c4d226e4877fc9d1e93707edfc230a1be4
This patch set replaces deprecated occurrences of rule with
rules and expected_error_code with expected_error_codes in
rbac_rule_validation.action decorator.
Along with removing the parameters from the decorator, all the
API tests have been changed to use the non-deprecated parameters
instead. Unit tests have also been updated.
Change-Id: I6485b6c57795b5fe75e2b339d5c9720da30be564
A new policy feature flag called
``[policy_feature_flag].removed_nova_policies_stein``
has been added to Patrole's config to handle Nova API
extension policies removed in Stein [0].
The policy feature flag is applied to tests that validate
response bodies for expected attributes previously returned
for the following policies that passed authorization:
- os_compute_api:os-config-drive
- os_compute_api:os-extended-availability-zone
- os_compute_api:os-extended-status
- os_compute_api:os-extended-volumes
- os_compute_api:os-keypairs
- os_compute_api:os-server-usage
- os_compute_api:os-flavor-rxtx
- os_compute_api:os-flavor-access (only from /flavors APIs)
- os_compute_api:image-size
Note that not all removed policies are included above because
test coverage is missing for them (like
os_compute_api:os-security-groups).
Also fixes test flows associated with image_size tests:
* endpoints are list images with details and show image (not
list image)
* both tests should check for OS-EXT-IMG-SIZE:size attribute
[0] https://review.openstack.org/#/c/586872/8
Story: 2003501
Change-Id: Ia6f8d255a540f7063beedd80a3ca1833f3987490
This patchset waits for interfaces to detach during
test clean up in MiscPolicyActionsNetworkRbacTest.
This is a follow up to [0], which missed clean up
for certain tests.
[0] I7b1a095bb73f4814b756f9493f9e353a8f98e7da
Change-Id: I041aaa7dce53f920b014c1cb586dc6bb3c10b489
This patchset moves VirtualInterfacesRbacTest into a separate
module as its presence in test_server_misc_policy_actions_rbac
was out of place, as the class just tests virtual interfaces.
Change-Id: I025e7cc318f4de149438bf838d28dae6e9ddeef7
Introduces foundational logic needed for multi-policy support
to rbac_rule_validation module. Patrole now offers support for
multiple policies. The ``rules`` argument has been added to the
``rbac_rule_validation.action`` decorator, which takes a list of
policy names which Patrole will use to determine the expected test
result. This allows Patrole to more accurately determine
whether RBAC is configured correctly, since some API endpoints
enforce multiple policies.
The new ``rules`` argument is implemented for
test_unlock_server_override test which corresponds to [0]
which enforces:
rules=["os_compute_api:os-lock-server:unlock:unlock",
"os_compute_api:os-lock-server:unlock:unlock_override"]
which is set for this test.
The ``rule`` argument in the ``rbac_rule_validation.action``
decorator has been deprecated in favor of ``rules``.
The following will be carried out in additional follow up patches:
* Renaming rule to rules
* Adding multi-policy support carefully for selected APIs
to be tracked via an etherpad
* Updating Patrole documentation with multi-policy support
details
[0] 0ab78890c1/nova/api/openstack/compute/lock_server.py (L42)
Partially Implements: bp rbac-testing-multiple-policies
Change-Id: Iec651aff1c1ef6acda19bcad2f57720f1dd3f8a0
In deployments with multiple networks, tests that create or attach an interface
will fail for the following reason:
"Multiple possible networks found, use a Network ID to be more specific."
To fix this, the tests should use the network created in the resource_setup
Change-Id: I10dc91252244fe05b513891ee8871965d398d351
'test_rescue_server' and 'test_unrescue_server' consistently cause
tearDownClass failures with the following failure message:
'Resource <server_uuid> failed to delete and is in ERROR status'
By adding waiters, this issue is mitigated.
Change-Id: Ia1d9e1e07370f5c1e4c86f14d96c342f9c41a817
While os_compute_api:os-server-usage policy in Nova is deprecated [0]
the current test in Patrole should be fixed since it is not
checking for expected attributes which are conditionally injected
following successful policy authorization. See [0] for details
on expected attributes.
[0] 15f1caf98a/nova/policies/server_usage.py (L32)
Change-Id: Ibc632cc084c2edb58c336f5ff56d2902bf2ccc96
The compute os-virtual-interfaces API is deprecated from the
Microversion 2.44, so we should set max_microversion = '2.43'
for compute virtual_interfaces tests.
This is based on work in Tempest [0].
[0] Ie1793802fa33898ffbdad16b58b085894e66d4d1
Partially Implements blueprint: clear-deprecated-api
Change-Id: Ib63bab15d4690d23f7ffbd8724ec1351929b94c5
Now that override_role has supplanted switch_role (which has
been deprecated) in [0], the RBAC tests need to switch to use
override_role.
This PS switches to override_role for the compute module. This
PS handles the last 11 modules for compute.
This PS also removes unnecessary indexing into response bodies.
[0] I670fba358bf321eae0d22d18cea6d2f530f00716
Partially Implements: blueprint rbac-utils-contextmanager
Change-Id: I0f8d249d58d7c6ad6f1d2fde2b52aefda10c64c4
This PS adds an RBAC test for unrescue compute server action to
validate that "os_compute_api:os-rescue" is enforced for the
endpoint.
Change-Id: I6c73d23dcd89addc0a364596c4f93511facb633f
This patch uses the decorators in tempest.common.utils instead
of the ones in tempest.test as they have been deprecated in [0].
[0] Ibd52153d00b8e60fb8c89e38d94e358ddc787251
Change-Id: Ib725e9d9fd087f75586daaafbfdb9aba49bdd5c6
This commit adds RBAC tests for extended server attributes
policies, the documentation for which can be found here:
https://github.com/openstack/nova/blob/master/nova/policies/extended_server_attributes.py
Tests for both APIs that enforce each policy were added.
Change-Id: I4150bcff934f1386ba8947d271289b790900ce2e
Implements: blueprint rbac-tests-for-extended-server-attributes
Adds tests for os-keypairs that looks for "key_name" in the
response body from a call to list or show server. Also adds a
skip exception to test_keypairs_rbac if os-keypairs extension
is not enabled.
Implements bp:rbac-tests-for-key-name-in-response
Change-Id: I2dc5332bfd59ea7c7a0e4a32b69d94ccd19ffaac
The multinode gate is reserved for tests that require multiple nodes
like server migration tests. It is also used for slow-running
tests that take a while to execute (20-30+ seconds) to better
parallelize tests in the gates.
However, 3 tests currently meet neither of these criteria and are
hence being moved to the voting gates:
* test_create_security_group_for_server [6.492102s]
* test_remove_security_group_from_server [6.525612s]
* test_volume_upload_public [4.558483s]
Change-Id: I24c65839692264f30a1494c051eec5fb72eb78a6
Source: http://logs.openstack.org/68/494368/12/check/gate-tempest-dsvm-patrole-multinode-member-ubuntu-xenial-nv/32ae39c/console.html
This commit adds RBAC tests for os-extended-volumes
policies, the documentation for which can be found here:
https://github.com/openstack/nova/blob/master/nova/policies/extended_volumes.py
Tests for both APIs that enforce each policy were added.
Change-Id: I6669f420743a3fee6470a7bb4ca5f1353a0e9696
Implements: blueprint rbac-tests-for-compute-extended-volumes
Depends-On: I1c14646dc8d102cd093be09833c23846781e5e73
This commit adds RBAC tests for extended availability zone
policies, the documentation for which can be found here:
https://github.com/openstack/nova/blob/master/nova/policies/extended_availability_zone.py
Tests for both APIs that enforce each policy were added.
Change-Id: I36aad1ea7ef3c1418cd23fda8357132ac8bde559
Depends-On: I1c14646dc8d102cd093be09833c23846781e5e73
Adds new exception and better explanations for failures due to
missing response body attributes and other unusual circumstances
that may lead to failures during testing.
Closes-Bug: #1699419
Closes-Bug: #1704684
Change-Id: I1c14646dc8d102cd093be09833c23846781e5e73
This commit adds RBAC tests for os-extended-status
policies, the documentation for which can be found here:
https://github.com/openstack/nova/blob/master/nova/policies/extended_status.py
Tests for both APIs that enforce each policy were added.
Change-Id: I2eb9c7f62d8adbff77dd36f39c3030751d21a894
Implements: blueprint rbac-tests-for-compute-extended-status
Depends-On: I1c14646dc8d102cd093be09833c23846781e5e73
Move instance actions tests into
test_server_misc_policy_actions_rbac.py to further increase gate
stability and decrease gate run time.
The number of calls to create_test_server() should be minimized
for RBAC testing because we don't do too much modification to the
resources that are created -- only what's necessary to trigger the
API action corresponding to the RBAC policy under test. Further,
minimizing such calls reduces the risk of spinning up too many servers
concurrently in our gates: the source of various gate failures as
limited resources lead to server faults being raised.
Change-Id: Ie01db6f58229f843684d9b4904defa6ed85b5f17
Partial-Bug: #1699415
This commit removes some excess code from the rbac_base classes,
including defining the auth_provider and setting credentials =
['primary', 'admin']. The credentials array should only be
populated with credentials needed by a test class -- but currently
admin is provided for rbac_utils, which is poor design.
This is accomplished by refactoring the constructor in RbacUtils
to instantiate an admin client manager using get_client_manager
which is available in the instance of tempest.test.BaseTestCase.
From there, it is easy to reference the admin roles client used
for switching roles. This is the only reason that admin was
provided in the credentials array above.
The following was changed:
- refactored RbacUtils constructor to remove need to add
'admin' to credentials array
- refactored rbac_utils functions to avoid using auth_provider
and to instead reference the auth_provider nested inside
os_primary
- adding doctring for RbacUtils
- refactored unit tests as needed
Change-Id: Id5588f2bf8947c314d46bd3cc0ef8b5c93874fc8
Tempest has moved their attr decorator to tempest.lib [0]. This
change updates all test.attr decorators to the new location,
removing deprecation warnings.
[0] https://review.openstack.org/#/c/456236/
Closes-Bug: #1683952
Change-Id: Ia0d47ab60f57ae0e1eee65527297f14afce378e0
This commit does 3 things:
1a) Corrects the "config_drive" tests by aligning the
tests with the nova policy documentation [0] which
states that "config_drive" is to added the response
body of showing a server or listing servers with details
if policy enforcement passes.
1b) Consequently, 2 tests are added: 1 for show and 1 for
list with details.
2) Moves both tests into the misc policy actions file to
increase gate stability and decrease gate run time.
The number of calls to create_test_server() should be minimized
for RBAC testing because we don't do too much modification to the
resources that are created -- only what's necessary to trigger the
API action corresponding to the RBAC policy under test. Further,
minimizing such calls reduces the risk of spinning up too many servers
concurrently in our gates: the source of various gate failures as
limited resources lead to server faults being raised.
[0] https://github.com/openstack/nova/blob/master/nova/policies/config_drive.py
Change-Id: If54fce795c2289b097b617cad7268dc9a3d9cf1c
Partial-Bug: #1699415
Move test_list_virtual_interfaces into
test_server_misc_policy_actions_rbac.py to further increase gate
stability and decrease gate run time.
The number of calls to create_test_server() should be minimized
for RBAC testing because we don't do too much modification to the
resources that are created -- only what's necessary to trigger the
API action corresponding to the RBAC policy under test. Further,
minimizing such calls reduces the risk of spinning up too many servers
concurrently in our gates: the source of various gate failures as
limited resources lead to server faults being raised.
Change-Id: I1ff0c14e741c8907f2f23a4dd63705713f06d337
Partial-Bug: #1699415
Group together tests that create a server and require network resources
so as to reduce overhead with creating a server and improving gate
stability. While these tests could also be grouped with
test_server_misc_policy_actions.py, those tests don't require network
resources, so it makes sense to separate out the network resource tests
into a separate class.
This commit groups together the tests from the following files:
* test_attach_interfaces_rbac
* test_ips_rbac
* test_multinic_rbac
into a shared class (MiscPolicyActionsNetworkRbacTest) inside
test_server_misc_policy_actions_rbac.py.
Closes-Bug: #1699421
Related-Bug: #1699415
Change-Id: I01c5e635028a6696331b1dc1401e274543db2716
Move tenant usage tests into test_server_misc_policy_actions_rbac.py
to further increase gate stability and decrease gate run time.
The number of calls to create_test_server() should be minimized
for RBAC testing because we don't do too much modification to the
resources that are created -- only what's necessary to trigger the
API action corresponding to the RBAC policy under test. Further,
minimizing such calls reduces the risk of spinning up too many servers
concurrently in our gates: the source of various gate failures as
limited resources lead to server faults being raised.
Change-Id: Ib104c08a5fa1708a829e5f91a587df6ba9c08ee0
Partial-Bug: #1699415
Move additional RBAC tests into test_server_misc_policy_actions_rbac.py
to further increase gate stability and decrease gate run time.
The number of calls to create_test_server() should be minimized
for RBAC testing because we don't do too much modification to the
resources that are created -- only what's necessary to trigger the
API action corresponding to the RBAC policy under test. Further,
minimizing such calls reduces the risk of spinning up too many servers
concurrently in our gates: the source of various gate failures as
limited resources lead to server faults being raised.
To that end, the following tests can be migrated to
test_server_misc_policy_actions_rbac.py:
- test_admin_password_rbac
- test_lock_server_rbac
- test_suspend_server_rbac
Additional migrations will be made in follow-up patches.
Change-Id: I7cf80d39b770afeb02d666266411fa2a91601eb9
Partial-Bug: #1699415
Minimize number of servers created for tests to avoid errors like
tempest.exceptions.BuildErrorException:
Server b5ebbe3e-6c30-4589-88b2-aaa52e107bee failed to build
and is in ERROR status
from happening [0].
This error is happening because too many servers are being created
concurrently in the gate, causing resource errors to be thrown.
This problem is only worsened with higher concurrency -- i.e. 4 -- as
the chances of creating many servers simultaneously across threads
are higher.
This commit:
Minimizes the number of servers that need to be created across classes
by consolidating test cases related to different policy "families" into
one class. This reduces the risk of running into `BuildErrorException`
errors being raised due to too many servers being created simultaneously
especially when higher concurrency is used.
Only applies to:
* policy "families" that require server creation
* small policy "families" -- i.e. containing one to three policies
[0] http://logs.openstack.org/56/475156/1/check/gate-tempest-dsvm-patrole-admin-ubuntu-xenial/df9f578/console.html#_2017-06-18_04_39_39_894737
Closes-Bug: #1698835
Change-Id: I3665aa5657dd5531a3b74edee42d6641bb6e6360
Ghanshyam Mann