Merge "Current version of DefCore capability scoring"

This commit is contained in:
Jenkins 2014-07-16 10:09:30 +00:00 committed by Gerrit Code Review
commit 8a62d1c6b7
3 changed files with 243 additions and 0 deletions

View File

@ -0,0 +1,127 @@
=========================================
2014-06-17 DefCore Capabilities Scoring
=========================================
The DefCore team has put together a scorecard_ combining many criteria
to evaluate which capabilities_ of OpenStack should be considered
core. They have asked the TC to help fill in the "Aligns with
Technical Direction" section, replacing the score in each cell with
either ``0`` (for "not needed") or ``1`` (for "required").
The table below includes the three columns from the "Aligns with
Technical Direction" of the original scorecard:
TC Future Direction
Does the Technical Committee plan to continue supporting this
feature?
Complete
Is the feature currently implemented and fully
working?
Stable
Is the API for the feature stable enough to support over a long
period?
..
The table below reproduces the "Aligns with Technical Direction"
section of the spreadsheet as it stands on 2014-06-17, based on
``defcore.csv``. A follow-up changeset will modify this resolution
to reflect the scores of the TC (separate patches will make it
easier to discuss the changes from the original values).
The ``0.5`` values are placeholders for the "undecided" scores the
DefCore committee needs us to resolve.
The other ``1`` and ``0`` scores were decided by the DefCore
committee, but as these are technical questions they are also up for
review. We should focus on filling in the missing values first, and
may want a second pass to update the other values.
The blank fields are for capabilities that won't be included based
on other criteria, so we can score them but don't have to during
this pass.
===== ================================= ===================== ========== ========
Row Candidate Capabilities TC Future Direction Complete Stable
===== ================================= ===================== ========== ========
4 compute-servers 1 1 1
5 volume 1 1 1
6 compute-volume 1 1 1
7 compute-quotas 1 1 1
8 compute-flavors 0.5 1 1
9 images-v1 1 1 1
10 compute-auth 0.5 1 1
11 images-v2 1 1 1
12 objectstore-object 1 1 1
13 compute-keypairs 1 1 1
14 compute-servers-metadata 1 1 1
15 objectstore-container 1 1 1
16 volume-snapshots 1 1 1
17 compute-images 0 1 1
18 compute-floating-ips 0.5 1 1
19 compute-instance-actions 1 1 1
20 compute-security-groups 0.5 1 1
21 compute-image-metadata 0 1 1
22 objectstore-container-quota 0.5 1 1
23 compute-virtual-interfaces 0.5 1 0
24 objectstore-container-acl 0 1 1
25 objectstore-acct-services 0 1 1
26 objectstore-container-staticweb 0 1 1
27 <identity-non-admin-roles> 1 1 0
28 compute-usage 0 1 1
29 compute-limits 1 1 1
30 networks-extensions 1 1 1
31 networks-l2 1 0 0
32 compute-ext-disk-config 0 1 0
33 networks-l3 1 0 0
34 compute-live-migration 1 0 0
35 compute-servers-personality 0 1 1
36 networks-floating-ips 1 0 0
37 networks-security-groups 1 0 0
38 networks-lbaas 1 0 0
39 orch-stacks 0 0 0
40 compute-multiple-create 0.5 0 0
41 networks-vpn 1 0 0
42 compute-attach-interface 0 0 0
43 networks-quotas 1 0 0
44 compute-auth-v3 1 0 0
45 compute-volume-proxy 0
46 compute-volume-ebs 0
47 compute-console-log 1 0 0
48 identity-admin-v3-roles 1 0 0
49 identity-admin-v3-endpoints 1 0 0
50 identity-admin-v3-credentials 1 0 0
51 identity-admin-v3-domains 1 0 0
52 identity-admin-v3-policies 1 0 0
53 identity-admin-v3-users 1 0 0
54 identity-admin-v3-services 1 0 0
55 identity-admin-v3-tokens 1 0 0
56 identity-admin-v3-projects 1 0 0
57 volume-multi-backend 1 1 1
58 identity-admin-users 0 1 1
59 identity-admin-roles 1 1 1
60 compute-admin-aggregates 0.5 1 1
61 objectstore-quotas 0.5 1 1
62 compute-admin-servers-pause 1 1 1
63 compute-admin-servers-suspend 1 1 1
64 identity-admin-tenants 0 1 0
65 compute-admin-avail-zone 0 1 1
66 identity-admin-services 1 1 1
67 volume-extra-specs 1 1 1
68 compute-admin-flavors 0.5 1 1
69 compute-admin-server 1 1 1
70 compute-admin-services 1 1 1
71 compute-admin-fixed-ips 0.5 1 1
72 compute-admin-quota 1 1 1
73 compute-admin-hypervisor 1 1 1
74 compute-admin-hosts 1 1 1
===== ================================= ===================== ========== ========
.. _scorecard: https://docs.google.com/a/dreamhost.com/spreadsheet/ccc?key=0Av62KoL8f9kAdFo4V1ZLUFM0OHlrRnFpQUkxSHJ5QWc&usp=drive_web#gid=6
.. _capabilities: https://github.com/stackforge/refstack/tree/master/defcore/havana

View File

@ -0,0 +1,75 @@
,,Criteria Weight,8.33,8.33,8.33,8.33,8.33,8.33,8.33,8.33,8.33,8.33,8.33,8.33,0,,
,,,Shows Proven Usage,,,Aligns with Technical Direction,,,Plays Well with Others,,,Takes a System View,,,,,
Candidate Capabilities,Count,Score,Widely Deployed,Used By Tools,Used By Clients,TC Future Direction,Complete,Stable,Discoverable,Doc'd,Core in Last Rel.,Foundational,Atomic,Proximity,Non-Admin,Description,xref
compute-servers,73,100,1,1,1,1,1,1,1,1,1,1,1,1,1,Basic Nova functions (launch instances),
volume,30,100,1,1,1,1,1,1,1,1,1,1,1,1,1,,
compute-volume,16,100,1,1,1,1,1,1,1,1,1,1,1,1,1,Attach/Detach functionality only,
compute-quotas,2,100,1,1,1,1,1,1,1,1,1,1,1,1,1,,
compute-flavors,15,96,1,1,1,0.5,1,1,1,1,1,1,1,1,1,,
images-v1,19,92,1,1,1,1,1,1,1,1,0,1,1,1,1,,
compute-auth,14,92,1,1,1,0.5,1,1,1,0.5,1,1,1,1,1,,
images-v2,5,92,1,1,1,1,1,1,1,1,0,1,1,1,1,,
objectstore-object,17,83,0.5,1,1,1,1,1,0.5,1,0,1,1,1,1,,
compute-keypairs,17,83,1,1,1,1,1,1,1,1,1,0,1,0,1,Injects keys into VMs,
compute-servers-metadata,16,83,1,1,1,1,1,1,1,1,1,1,0,0,1,Tagging,
objectstore-container,5,83,0.5,1,1,1,1,1,0.5,1,0,1,1,1,1,,
volume-snapshots,2,83,1,1,1,1,1,1,1,1,1,0,0,1,1,,
compute-images,37,79,0.5,1,1,0,1,1,1,1,1,0,1,1,1,,
compute-floating-ips,14,79,1,1,1,0.5,1,1,1,1,0,0,1,1,1,,
compute-instance-actions,4,79,1,1,1,1,1,1,1,0.5,0,1,1,0,1,,
compute-security-groups,27,71,1,1,1,0.5,1,1,0,1,0,0,1,1,1,,
compute-image-metadata,12,67,1,0,0,0,1,1,1,1,1,0,1,1,1,allows users add arbitrary keys and values on create,
objectstore-container-quota,3,67,1,0,0,0.5,1,1,0.5,1,1,1,1,0,1,,
compute-virtual-interfaces,2,67,0.5,1,1,0.5,1,0,1,1,0,1,1,0,1,There are COMPETITING VERSIONS that cause Interop pain,
objectstore-container-acl,11,62,1,0,0,0,1,1,0.5,1,1,1,1,0,1,,
objectstore-acct-services,8,62,1,0,0,0,1,1,0.5,1,1,1,1,0,1,,
objectstore-container-staticweb,2,62,1,0,0,0,1,1,0.5,1,1,1,1,0,1,,
<identity-non-admin-roles>,?,58,1,0,0,1,1,0,1,1,0,1,1,0,1,Used w/ middleware (LDAP or AD),
compute-usage,6,58,0,1,0,0,1,1,1,1,0,0,1,1,1,Used by Dashboard,
compute-limits,2,58,1,1,0,1,1,1,1,1,0,0,0,0,1,Works with rate limiting,
networks-extensions,1,58,0,0,0,1,1,1,1,1,0,1,1,0,1,,
networks-l2,14,50,0,0,0,1,0,0,1,1,0,1,1,1,1,,
compute-ext-disk-config,4,50,1,0,1,0,1,0,1,1,0,0,1,0,1,controls how the disk is partitioned when you manage a server,https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_disk_config.py
networks-l3,3,50,0,0,0,1,0,0,1,1,0,1,1,1,1,,
compute-live-migration,3,42,0,0,0,1,0,0,1,1,0,1,1,0,1,,
compute-servers-personality,3,42,1,0,0,0,1,1,1,1,0,0,0,0,1,,
networks-floating-ips,1,37,0,0,0,1,0,0,1,1,0,0.5,1,0,1,,http://docs.openstack.org/admin-guide-cloud/content/l3_workflow.html
networks-security-groups,5,33,0,0,0,1,0,0,1,1,0,0,0,1,1,,
networks-lbaas,12,25,0,0,0,1,0,0,1,1,0,0,0,0,1,,http://docs.openstack.org/admin-guide-cloud/content/fwaas_workflow.html
orch-stacks,20,21,0,0,0,0,0,0,0.5,1,0,1,0,0,1,,
compute-multiple-create,7,21,0,0,0,0.5,0,0,1,0,0,0,0,1,1,,
networks-vpn,3,17,0,0,0,1,0,0,1,0,0,0,0,0,1,,
compute-attach-interface,1,17,0,0,0,0,0,0,1,0,0,0,1,0,1,used to attach NIC to running VM,
networks-quotas,1,17,0,0,0,1,0,0,1,0,0,0,0,0,1,,
compute-auth-v3,1,8,0,0,0,1,0,0,0,0,0,0,0,0,1,,
compute-volume-proxy,?,0,,,,0,,,,,0,,,,,Maps to the old os-volume extensions,
compute-volume-ebs,?,0,,,,0,,,,,0,,,,,,
compute-console-log,1,-17,0,0,0,1,0,0,0,0,0,0,1,0,-1,,
identity-admin-v3-roles,5,-25,0,0,0,1,0,0,0,0,0,1,1,0,-1,,
identity-admin-v3-endpoints,3,-25,0,0,0,1,0,0,0,0,0,1,1,0,-1,,
identity-admin-v3-credentials,2,-25,0,0,0,1,0,0,0,0,0,1,1,0,-1,,
identity-admin-v3-domains,2,-25,0,0,0,1,0,0,0,0,0,1,1,0,-1,,
identity-admin-v3-policies,2,-25,0,0,0,1,0,0,0,0,0,1,1,0,-1,,
identity-admin-v3-users,2,-25,0,0,0,1,0,0,0,0,0,1,1,0,-1,,
identity-admin-v3-services,1,-25,0,0,0,1,0,0,0,0,0,1,1,0,-1,,
identity-admin-v3-tokens,1,-25,0,0,0,1,0,0,0,0,0,1,1,0,-1,,
identity-admin-v3-projects,15,-33,0,0,0,1,0,0,0,0,0,1,1,1,-1,,
volume-multi-backend,2,-42,0,0,0,1,1,1,0,1,0,0,1,0,-1,,
identity-admin-users,31,-54,1,0,0,0,1,1,1,0.5,0,0,1,1,-1,,
identity-admin-roles,23,-58,0,0,0,1,1,1,1,1,0,1,1,0,-1,,
compute-admin-aggregates,18,-62,1,0.5,0.5,0.5,1,1,0,1,0,0,1,1,-1,,https://blueprints.launchpad.net/nova/+spec/host-aggregates
objectstore-quotas,4,-67,1,0,0,0.5,1,1,0.5,1,1,1,1,0,-1,,
compute-admin-servers-pause,3,-67,1,1,0,1,1,1,1,0,0,0,1,1,-1,admin only,
compute-admin-servers-suspend,3,-67,1,1,0,1,1,1,1,0,0,0,1,1,-1,admin only,
identity-admin-tenants,20,-71,1,1,1,0,1,0,1,0.5,0,1,1,1,-1,,
compute-admin-avail-zone,4,-83,1,1,1,0,1,1,1,1,1,1,1,0,-1,,
identity-admin-services,2,-87,1,1,1,1,1,1,1,0.5,0,1,1,1,-1,,
volume-extra-specs,13,-92,1,1,1,1,1,1,0,1,1,1,1,1,-1,,
compute-admin-flavors,24,-96,1,1,1,0.5,1,1,1,1,1,1,1,1,-1,related to compute flavors,
compute-admin-server,11,-96,1,1,1,1,1,1,1,1,1,0.5,1,1,-1,,
compute-admin-services,9,-96,1,1,1,1,1,1,1,1,1,0.5,1,1,-1,,
compute-admin-fixed-ips,8,-96,1,1,1,0.5,1,1,1,1,1,1,1,1,-1,,
compute-admin-quota,9,-100,1,1,1,1,1,1,1,1,1,1,1,1,-1,,
compute-admin-hypervisor,8,-100,1,1,1,1,1,1,1,1,1,1,1,1,-1,,
compute-admin-hosts,7,-100,1,1,1,1,1,1,1,1,1,1,1,1,-1,,
,,,59.42%,48.55%,44.20%,71.13%,68.12%,62.32%,72.46%,71.74%,35.21%,64.49%,84.06%,50.72%,18.84%,,
1 Criteria Weight 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33 8.33 0
2 Shows Proven Usage Aligns with Technical Direction Plays Well with Others Takes a System View
3 Candidate Capabilities Count Score Widely Deployed Used By Tools Used By Clients TC Future Direction Complete Stable Discoverable Doc'd Core in Last Rel. Foundational Atomic Proximity Non-Admin Description xref
4 compute-servers 73 100 1 1 1 1 1 1 1 1 1 1 1 1 1 Basic Nova functions (launch instances)
5 volume 30 100 1 1 1 1 1 1 1 1 1 1 1 1 1
6 compute-volume 16 100 1 1 1 1 1 1 1 1 1 1 1 1 1 Attach/Detach functionality only
7 compute-quotas 2 100 1 1 1 1 1 1 1 1 1 1 1 1 1
8 compute-flavors 15 96 1 1 1 0.5 1 1 1 1 1 1 1 1 1
9 images-v1 19 92 1 1 1 1 1 1 1 1 0 1 1 1 1
10 compute-auth 14 92 1 1 1 0.5 1 1 1 0.5 1 1 1 1 1
11 images-v2 5 92 1 1 1 1 1 1 1 1 0 1 1 1 1
12 objectstore-object 17 83 0.5 1 1 1 1 1 0.5 1 0 1 1 1 1
13 compute-keypairs 17 83 1 1 1 1 1 1 1 1 1 0 1 0 1 Injects keys into VMs
14 compute-servers-metadata 16 83 1 1 1 1 1 1 1 1 1 1 0 0 1 Tagging
15 objectstore-container 5 83 0.5 1 1 1 1 1 0.5 1 0 1 1 1 1
16 volume-snapshots 2 83 1 1 1 1 1 1 1 1 1 0 0 1 1
17 compute-images 37 79 0.5 1 1 0 1 1 1 1 1 0 1 1 1
18 compute-floating-ips 14 79 1 1 1 0.5 1 1 1 1 0 0 1 1 1
19 compute-instance-actions 4 79 1 1 1 1 1 1 1 0.5 0 1 1 0 1
20 compute-security-groups 27 71 1 1 1 0.5 1 1 0 1 0 0 1 1 1
21 compute-image-metadata 12 67 1 0 0 0 1 1 1 1 1 0 1 1 1 allows users add arbitrary keys and values on create
22 objectstore-container-quota 3 67 1 0 0 0.5 1 1 0.5 1 1 1 1 0 1
23 compute-virtual-interfaces 2 67 0.5 1 1 0.5 1 0 1 1 0 1 1 0 1 There are COMPETITING VERSIONS that cause Interop pain
24 objectstore-container-acl 11 62 1 0 0 0 1 1 0.5 1 1 1 1 0 1
25 objectstore-acct-services 8 62 1 0 0 0 1 1 0.5 1 1 1 1 0 1
26 objectstore-container-staticweb 2 62 1 0 0 0 1 1 0.5 1 1 1 1 0 1
27 <identity-non-admin-roles> ? 58 1 0 0 1 1 0 1 1 0 1 1 0 1 Used w/ middleware (LDAP or AD)
28 compute-usage 6 58 0 1 0 0 1 1 1 1 0 0 1 1 1 Used by Dashboard
29 compute-limits 2 58 1 1 0 1 1 1 1 1 0 0 0 0 1 Works with rate limiting
30 networks-extensions 1 58 0 0 0 1 1 1 1 1 0 1 1 0 1
31 networks-l2 14 50 0 0 0 1 0 0 1 1 0 1 1 1 1
32 compute-ext-disk-config 4 50 1 0 1 0 1 0 1 1 0 0 1 0 1 controls how the disk is partitioned when you manage a server https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_disk_config.py
33 networks-l3 3 50 0 0 0 1 0 0 1 1 0 1 1 1 1
34 compute-live-migration 3 42 0 0 0 1 0 0 1 1 0 1 1 0 1
35 compute-servers-personality 3 42 1 0 0 0 1 1 1 1 0 0 0 0 1
36 networks-floating-ips 1 37 0 0 0 1 0 0 1 1 0 0.5 1 0 1 http://docs.openstack.org/admin-guide-cloud/content/l3_workflow.html
37 networks-security-groups 5 33 0 0 0 1 0 0 1 1 0 0 0 1 1
38 networks-lbaas 12 25 0 0 0 1 0 0 1 1 0 0 0 0 1 http://docs.openstack.org/admin-guide-cloud/content/fwaas_workflow.html
39 orch-stacks 20 21 0 0 0 0 0 0 0.5 1 0 1 0 0 1
40 compute-multiple-create 7 21 0 0 0 0.5 0 0 1 0 0 0 0 1 1
41 networks-vpn 3 17 0 0 0 1 0 0 1 0 0 0 0 0 1
42 compute-attach-interface 1 17 0 0 0 0 0 0 1 0 0 0 1 0 1 used to attach NIC to running VM
43 networks-quotas 1 17 0 0 0 1 0 0 1 0 0 0 0 0 1
44 compute-auth-v3 1 8 0 0 0 1 0 0 0 0 0 0 0 0 1
45 compute-volume-proxy ? 0 0 0 Maps to the old os-volume extensions
46 compute-volume-ebs ? 0 0 0
47 compute-console-log 1 -17 0 0 0 1 0 0 0 0 0 0 1 0 -1
48 identity-admin-v3-roles 5 -25 0 0 0 1 0 0 0 0 0 1 1 0 -1
49 identity-admin-v3-endpoints 3 -25 0 0 0 1 0 0 0 0 0 1 1 0 -1
50 identity-admin-v3-credentials 2 -25 0 0 0 1 0 0 0 0 0 1 1 0 -1
51 identity-admin-v3-domains 2 -25 0 0 0 1 0 0 0 0 0 1 1 0 -1
52 identity-admin-v3-policies 2 -25 0 0 0 1 0 0 0 0 0 1 1 0 -1
53 identity-admin-v3-users 2 -25 0 0 0 1 0 0 0 0 0 1 1 0 -1
54 identity-admin-v3-services 1 -25 0 0 0 1 0 0 0 0 0 1 1 0 -1
55 identity-admin-v3-tokens 1 -25 0 0 0 1 0 0 0 0 0 1 1 0 -1
56 identity-admin-v3-projects 15 -33 0 0 0 1 0 0 0 0 0 1 1 1 -1
57 volume-multi-backend 2 -42 0 0 0 1 1 1 0 1 0 0 1 0 -1
58 identity-admin-users 31 -54 1 0 0 0 1 1 1 0.5 0 0 1 1 -1
59 identity-admin-roles 23 -58 0 0 0 1 1 1 1 1 0 1 1 0 -1
60 compute-admin-aggregates 18 -62 1 0.5 0.5 0.5 1 1 0 1 0 0 1 1 -1 https://blueprints.launchpad.net/nova/+spec/host-aggregates
61 objectstore-quotas 4 -67 1 0 0 0.5 1 1 0.5 1 1 1 1 0 -1
62 compute-admin-servers-pause 3 -67 1 1 0 1 1 1 1 0 0 0 1 1 -1 admin only
63 compute-admin-servers-suspend 3 -67 1 1 0 1 1 1 1 0 0 0 1 1 -1 admin only
64 identity-admin-tenants 20 -71 1 1 1 0 1 0 1 0.5 0 1 1 1 -1
65 compute-admin-avail-zone 4 -83 1 1 1 0 1 1 1 1 1 1 1 0 -1
66 identity-admin-services 2 -87 1 1 1 1 1 1 1 0.5 0 1 1 1 -1
67 volume-extra-specs 13 -92 1 1 1 1 1 1 0 1 1 1 1 1 -1
68 compute-admin-flavors 24 -96 1 1 1 0.5 1 1 1 1 1 1 1 1 -1 related to compute flavors
69 compute-admin-server 11 -96 1 1 1 1 1 1 1 1 1 0.5 1 1 -1
70 compute-admin-services 9 -96 1 1 1 1 1 1 1 1 1 0.5 1 1 -1
71 compute-admin-fixed-ips 8 -96 1 1 1 0.5 1 1 1 1 1 1 1 1 -1
72 compute-admin-quota 9 -100 1 1 1 1 1 1 1 1 1 1 1 1 -1
73 compute-admin-hypervisor 8 -100 1 1 1 1 1 1 1 1 1 1 1 1 -1
74 compute-admin-hosts 7 -100 1 1 1 1 1 1 1 1 1 1 1 1 -1
75 59.42% 48.55% 44.20% 71.13% 68.12% 62.32% 72.46% 71.74% 35.21% 64.49% 84.06% 50.72% 18.84%

View File

@ -0,0 +1,41 @@
#!/usr/bin/env python
"""Extract relevant columns.
"""
import csv
import sys
with open('defcore.csv', 'r') as infile:
reader = csv.reader(infile)
read_iter = iter(reader)
# Read past the 2 header lines
read_iter.next()
read_iter.next()
rows = [[row[0]] + row[6:9] for row in read_iter]
# Ignore the percentages at the bottom
del rows[-1]
# Add the spreadsheet row numbers
for i, row in enumerate(rows, 3):
row.insert(0, str(i))
print rows[0]
rows[0][0] = 'Row'
# Candidate Capabilities,TC Future Direction,Complete,Stable
widths = []
for i in range(len(rows[0])):
widths.append(max(len(row[i])+2 for row in rows))
fmt = '{0:>%d} {1:<%d} {2:^%d} {3:^%d} {4:^%d}' % tuple(widths)
header = ' '.join('=' * wid for wid in widths)
print header
print fmt.format(*rows[0])
print header
for row in rows[1:]:
print fmt.format(*row)
print header