openinfra
/
interop
Code Issues Proposed changes

Keystone scoring for 2018.01 guideline 

Scoring Keystone for guideline 2018.01
Changes: Update notes on the working materials folder.

Items To Be Discussed:
identity-v3-list-projects capability was marked required 2017.09.
But the only TC available was flagged since it needs 2 users.
Q. Should we move the capability back to advisory until TC is fixed?
   or until other suitable TCs are added?

Change-Id: Ic84cbe834474e579345e22256f2e956ad2b4b897
Note: waiting for additional comments from PTL.
This commit is contained in:
Luz Cazares 2017-10-04 20:50:01 +00:00
parent 02698b5411
commit 9b0fec938f
3 changed files with 11 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
working_materials/keystone_capabilities_info.csv
View File

@ -1,7 +1,7 @@
Capability,Program,Status,Method,Endpoint,Test available?,interop relevant?,PTL Comments,From Defcore Discussion,Scorer Comments,
identity-v3-tokens-create,platform/compute/object,required,POST,/v3/auth/tokens,1,yes,The returned token value is in the X-Auth-Token header,,tempest.api.identity.v3.test_tokens{test_create_token}, This TC refers to API https://developer.openstack.org/api-ref/identity/v3/#password-authentication-with-unscoped-authorization. Should we add other test cases to tempest in order to validate API for: password-authentication-with-scoped-authorization and password-authentication-with-explicit-unscoped-authorization?,
identity-v3-api-discovery,platform/compute,required,,,3,yes,,make required,"tempest.api.identity.v3.test_api_discovery{test_api_version_resources, test_api_media_types, test_api_version_statuses}",
identity-v3-list-projects,platform/compute,advisory,GET,/v3/users/{user_id}/projects,1,yes,,,,
identity-v3-tokens-create,platform/compute/object,required,POST,/v3/auth/tokens,1,yes,The returned token value is in the X-Auth-Token header,,tempest.api.identity.v3.test_tokens{test_create_token}, This TC refers to API https://developer.openstack.org/api-ref/identity/v3/#password-authentication-with-unscoped-authorization. Should we add other test cases to tempest in order to validate API for: password-authentication-with-scoped-authorization and password-authentication-with-explicit-unscoped-authorization?
identity-v3-api-discovery,platform/compute,required,GET,/v3,3,yes,,make required,"tempest.api.identity.v3.test_api_discovery{test_api_version_resources, test_api_media_types, test_api_version_statuses}",
identity-v3-list-projects,platform/compute,required,GET,/v3/users/{user_id}/projects,1,yes,,,Flagged since require 2 set of user credentials.,
,,,,,,,,,,
identity-v3-create-ec2-credentials,,,POST,/v3/credentials,1,yes,,Should we make ec2 compatibility required? unclear,,
identity-v3-list-ec2-credentials,,,GET,/v3/credentials,1,yes,,Should we make ec2 compatibility required? unclear,,
@ -9,8 +9,7 @@ identity-v3-show-ec2-credentials,,,GET,/v3/credentials/{credential_id},1,yes,,Sh
identity-v3-delete-ec2-credentials,,,DELETE,/v3/credentials/{credential_id},1,yes,,Should we make ec2 compatibility required? unclear,,
identity-v3-update-ec2-credentials,,,PATCH,/v3/credentials/{credential_id},,,,Should we make ec2 compatibility required? unclear,,
identity-v3-catalog,(make sure it works on all supported releases),,,,,,returned with the token,,,
identity-v3-password-update,,,POST,/v3/users/{user_id}/password,1,yes,,"
Untestable without changing user's password, security risk. Also password policies are very particular to different companies, making a test that would pass on all is near impossible.",tempest.api.identity.v3.test_users{test_update_own_password},
identity-v3-password-update,,,POST,/v3/users/{user_id}/password,1,yes,,"Untestable without changing user's password, security risk. Also password policies are very particular to different companies, making a test that would pass on all is near impossible.",tempest.api.identity.v3.test_users{test_update_own_password},
,,,,,,,,,,
identity-v3-list-groups,platform/compute,,GET,/v3/users/{user_id}/groups,0,yes,,,no test available for this feature,
identity-v3-get-project,platform/compute,,GET,/v3/projects/{project_id},0,yes,,,admin required,
@ -19,9 +18,9 @@ identity-v3-get-role,platform/compute,,GET,/v3/roles/{role_id},,no,,,admin requi
identity-v3-list-domains,platform/compute,,GET,/v3/domains,,no,,,admin required,
identity-v3-get-domain,platform/compute,,GET,/v3/domains/{domain_id},,no,,,admin required,
,,,,,,,,,,
identity-v3-tokens-validate,platform/compute,,GET,/v3/auth/tokens,,yes,Token to be validated is passed in the X-Subject-Token header,,,"This sounds backwards to me, need to check with steve, shouldn't it be POST for validating and GET for getting a token?"
identity-v3-tokens-validate,platform/compute,advisory,GET,/v3/auth/tokens,,yes,Token to be validated is passed in the X-Subject-Token header,,,
identity-v3-revoke-token,platform/compute,,DELETE,/v3/auth/tokens,1,yes,Token to be revoked is passed in the X-Subject-Token header,keystone.keystone.tests.unit.test_revoke{test_revoke_by_user},,
identity-v3-get-catalog,platform/compute/object,,GET,/v3/auth/catalog,0,yes,,,"couldn't find a test specific for this, there are some tests related in keystone.tests.unit.test_v3_auth.py",
identity-v3-get-catalog,platform/compute/object,advisory,GET,/v3/auth/catalog,0,yes,,,"TC added in Tempest",
identity-v3-get-auth-projects,platform/compute,,GET,/v3/auth/projects,0,yes,,,"equivalent as far as I can tell to identity-v3-list-projects. couldn't find a test specific for this, there are some tests related in keystone.tests.unit.test_v3_auth.py",
,,,,,,,,,,
identity-v2-list-versions,,,GET,/,1,yes,,,Deprecated,

Can't render this file because it has a wrong number of fields in line 2.

9
working_materials/scoring.txt
View File

@ -288,20 +288,19 @@ identity-v3-api-discovery: [1,0,1] [1,1,1] [1,1,1] [1,1,1] [1] [94]*
identity-v3-catalog: [1,0,1] [1,1,1] [1,1,0] [1,1,1] [1] [85]*
identity-v3-list-projects: [1,1,1] [1,1,1] [1,1,0] [0,1,0] [1] [74]*
identity-v3-list-groups: [1,1,1] [1,1,1] [1,1,0] [0,1,0] [1] [74]*
identity-v3-tokens-create: [1,1,1] [1,1,1] [1,1,1] [1,1,0] [1] [92]*
identity-v3-tokens-validate: [1,1,1] [1,1,1] [1,1,0] [0,1,0] [1] [74]*
Notes:
* identity-v3-catalog is returned when the api for
identity-v3-tokens-create is called (GET /v3/auth/tokens). It is
important to consider it because end users may be relying on this
catalog for their apps (even though there are other API calls that
also show the catalog such as GET /v3/auth/catalog).
* identity-v3-list-projects and identity-v3-list-groups didn't have usable
tests in the past, but one was added for identity-v3-list-projects last year.
Providers like Fog.io
now actually use the /v3/users/{user_id}/[projects|groups] API's:
https://git.io/vX9S6
https://git.io/vX9SP
Capability became required 2017.09 but the only TC available was flagged
since it requires two sets of credentials. Capability needs additional TCs
or existing test should be changed to require only one set of credentials.
* identity-v3-change-password was considered here but it's applicability is
a bit hard to gauge: many systems using third-party authentication (such as
an LDAP/AD server, an external oauth system, etc) require password changes

1
working_materials/tabulated_scores.csv
View File

@ -105,6 +105,7 @@ identity-v3-api-discovery,1,0,1,1,1,1,1,1,1,1,1,1,1,94*
identity-v3-catalog,1,0,1,1,1,1,1,1,0,1,1,1,1,85*
identity-v3-list-projects,1,1,1,1,1,1,1,1,0,0,1,0,1,74*
identity-v3-list-groups,1,1,1,1,1,1,1,1,0,0,1,0,1,74*
identity-v3-tokens-create,1,1,1,1,1,1,1,1,1,1,1,0,1,92*
identity-v3-tokens-validate,1,1,1,1,1,1,1,1,0,0,1,0,1,74*
objectstore-object-copy,1,1,1,1,1,1,1,1,1,1,1,1,1,100*
objectstore-object-create,1,1,1,1,1,1,1,1,1,1,1,1,1,100*

Can't render this file because it has a wrong number of fields in line 25.