From 9b0fec938f75e583a7e2b2fe14521f136ed72c42 Mon Sep 17 00:00:00 2001 From: Luz Cazares Date: Wed, 4 Oct 2017 20:50:01 +0000 Subject: [PATCH] Keystone scoring for 2018.01 guideline Scoring Keystone for guideline 2018.01 Changes: Update notes on the working materials folder. Items To Be Discussed: identity-v3-list-projects capability was marked required 2017.09. But the only TC available was flagged since it needs 2 users. Q. Should we move the capability back to advisory until TC is fixed? or until other suitable TCs are added? Change-Id: Ic84cbe834474e579345e22256f2e956ad2b4b897 Note: waiting for additional comments from PTL. --- working_materials/keystone_capabilities_info.csv | 13 ++++++------- working_materials/scoring.txt | 9 ++++----- working_materials/tabulated_scores.csv | 1 + 3 files changed, 11 insertions(+), 12 deletions(-) diff --git a/working_materials/keystone_capabilities_info.csv b/working_materials/keystone_capabilities_info.csv index 29039970..f37ef02e 100644 --- a/working_materials/keystone_capabilities_info.csv +++ b/working_materials/keystone_capabilities_info.csv @@ -1,7 +1,7 @@ Capability,Program,Status,Method,Endpoint,Test available?,interop relevant?,PTL Comments,From Defcore Discussion,Scorer Comments, -identity-v3-tokens-create,platform/compute/object,required,POST,/v3/auth/tokens,1,yes,The returned token value is in the X-Auth-Token header,,tempest.api.identity.v3.test_tokens{test_create_token}, This TC refers to API https://developer.openstack.org/api-ref/identity/v3/#password-authentication-with-unscoped-authorization. Should we add other test cases to tempest in order to validate API for: password-authentication-with-scoped-authorization and password-authentication-with-explicit-unscoped-authorization?, -identity-v3-api-discovery,platform/compute,required,,,3,yes,,make required,"tempest.api.identity.v3.test_api_discovery{test_api_version_resources, test_api_media_types, test_api_version_statuses}", -identity-v3-list-projects,platform/compute,advisory,GET,/v3/users/{user_id}/projects,1,yes,,,, +identity-v3-tokens-create,platform/compute/object,required,POST,/v3/auth/tokens,1,yes,The returned token value is in the X-Auth-Token header,,tempest.api.identity.v3.test_tokens{test_create_token}, This TC refers to API https://developer.openstack.org/api-ref/identity/v3/#password-authentication-with-unscoped-authorization. Should we add other test cases to tempest in order to validate API for: password-authentication-with-scoped-authorization and password-authentication-with-explicit-unscoped-authorization? +identity-v3-api-discovery,platform/compute,required,GET,/v3,3,yes,,make required,"tempest.api.identity.v3.test_api_discovery{test_api_version_resources, test_api_media_types, test_api_version_statuses}", +identity-v3-list-projects,platform/compute,required,GET,/v3/users/{user_id}/projects,1,yes,,,Flagged since require 2 set of user credentials., ,,,,,,,,,, identity-v3-create-ec2-credentials,,,POST,/v3/credentials,1,yes,,Should we make ec2 compatibility required? unclear,, identity-v3-list-ec2-credentials,,,GET,/v3/credentials,1,yes,,Should we make ec2 compatibility required? unclear,, @@ -9,8 +9,7 @@ identity-v3-show-ec2-credentials,,,GET,/v3/credentials/{credential_id},1,yes,,Sh identity-v3-delete-ec2-credentials,,,DELETE,/v3/credentials/{credential_id},1,yes,,Should we make ec2 compatibility required? unclear,, identity-v3-update-ec2-credentials,,,PATCH,/v3/credentials/{credential_id},,,,Should we make ec2 compatibility required? unclear,, identity-v3-catalog,(make sure it works on all supported releases),,,,,,returned with the token,,, -identity-v3-password-update,,,POST,/v3/users/{user_id}/password,1,yes,," -Untestable without changing user's password, security risk. Also password policies are very particular to different companies, making a test that would pass on all is near impossible.",tempest.api.identity.v3.test_users{test_update_own_password}, +identity-v3-password-update,,,POST,/v3/users/{user_id}/password,1,yes,,"Untestable without changing user's password, security risk. Also password policies are very particular to different companies, making a test that would pass on all is near impossible.",tempest.api.identity.v3.test_users{test_update_own_password}, ,,,,,,,,,, identity-v3-list-groups,platform/compute,,GET,/v3/users/{user_id}/groups,0,yes,,,no test available for this feature, identity-v3-get-project,platform/compute,,GET,/v3/projects/{project_id},0,yes,,,admin required, @@ -19,9 +18,9 @@ identity-v3-get-role,platform/compute,,GET,/v3/roles/{role_id},,no,,,admin requi identity-v3-list-domains,platform/compute,,GET,/v3/domains,,no,,,admin required, identity-v3-get-domain,platform/compute,,GET,/v3/domains/{domain_id},,no,,,admin required, ,,,,,,,,,, -identity-v3-tokens-validate,platform/compute,,GET,/v3/auth/tokens,,yes,Token to be validated is passed in the X-Subject-Token header,,,"This sounds backwards to me, need to check with steve, shouldn't it be POST for validating and GET for getting a token?" +identity-v3-tokens-validate,platform/compute,advisory,GET,/v3/auth/tokens,,yes,Token to be validated is passed in the X-Subject-Token header,,, identity-v3-revoke-token,platform/compute,,DELETE,/v3/auth/tokens,1,yes,Token to be revoked is passed in the X-Subject-Token header,keystone.keystone.tests.unit.test_revoke{test_revoke_by_user},, -identity-v3-get-catalog,platform/compute/object,,GET,/v3/auth/catalog,0,yes,,,"couldn't find a test specific for this, there are some tests related in keystone.tests.unit.test_v3_auth.py", +identity-v3-get-catalog,platform/compute/object,advisory,GET,/v3/auth/catalog,0,yes,,,"TC added in Tempest", identity-v3-get-auth-projects,platform/compute,,GET,/v3/auth/projects,0,yes,,,"equivalent as far as I can tell to identity-v3-list-projects. couldn't find a test specific for this, there are some tests related in keystone.tests.unit.test_v3_auth.py", ,,,,,,,,,, identity-v2-list-versions,,,GET,/,1,yes,,,Deprecated, diff --git a/working_materials/scoring.txt b/working_materials/scoring.txt index 52dd221b..33f0d4df 100644 --- a/working_materials/scoring.txt +++ b/working_materials/scoring.txt @@ -288,20 +288,19 @@ identity-v3-api-discovery: [1,0,1] [1,1,1] [1,1,1] [1,1,1] [1] [94]* identity-v3-catalog: [1,0,1] [1,1,1] [1,1,0] [1,1,1] [1] [85]* identity-v3-list-projects: [1,1,1] [1,1,1] [1,1,0] [0,1,0] [1] [74]* identity-v3-list-groups: [1,1,1] [1,1,1] [1,1,0] [0,1,0] [1] [74]* +identity-v3-tokens-create: [1,1,1] [1,1,1] [1,1,1] [1,1,0] [1] [92]* identity-v3-tokens-validate: [1,1,1] [1,1,1] [1,1,0] [0,1,0] [1] [74]* Notes: - * identity-v3-catalog is returned when the api for - identity-v3-tokens-create is called (GET /v3/auth/tokens). It is - important to consider it because end users may be relying on this - catalog for their apps (even though there are other API calls that - also show the catalog such as GET /v3/auth/catalog). * identity-v3-list-projects and identity-v3-list-groups didn't have usable tests in the past, but one was added for identity-v3-list-projects last year. Providers like Fog.io now actually use the /v3/users/{user_id}/[projects|groups] API's: https://git.io/vX9S6 https://git.io/vX9SP + Capability became required 2017.09 but the only TC available was flagged + since it requires two sets of credentials. Capability needs additional TCs + or existing test should be changed to require only one set of credentials. * identity-v3-change-password was considered here but it's applicability is a bit hard to gauge: many systems using third-party authentication (such as an LDAP/AD server, an external oauth system, etc) require password changes diff --git a/working_materials/tabulated_scores.csv b/working_materials/tabulated_scores.csv index 65ea91b2..5f97cdcd 100644 --- a/working_materials/tabulated_scores.csv +++ b/working_materials/tabulated_scores.csv @@ -105,6 +105,7 @@ identity-v3-api-discovery,1,0,1,1,1,1,1,1,1,1,1,1,1,94* identity-v3-catalog,1,0,1,1,1,1,1,1,0,1,1,1,1,85* identity-v3-list-projects,1,1,1,1,1,1,1,1,0,0,1,0,1,74* identity-v3-list-groups,1,1,1,1,1,1,1,1,0,0,1,0,1,74* +identity-v3-tokens-create,1,1,1,1,1,1,1,1,1,1,1,0,1,92* identity-v3-tokens-validate,1,1,1,1,1,1,1,1,0,0,1,0,1,74* objectstore-object-copy,1,1,1,1,1,1,1,1,1,1,1,1,1,100* objectstore-object-create,1,1,1,1,1,1,1,1,1,1,1,1,1,100*