openstack
/
congress-specs
Code Issues Proposed changes

Switch to stestr 

According to Openstack summit session [1],
stestr is maintained project to which all Openstack projects should migrate.
Let's switch to stestr as other projects have already moved to it.

[1] https://etherpad.openstack.org/p/YVR-python-pti

Change-Id: I7daa364d2016e1b7951e6240b3254287361f9d0f
This commit is contained in:
Vu Cong Tuan 2018-07-09 14:09:58 +07:00 committed by Eric Kao
parent 2c423c83c2
commit 006a529286
7 changed files with 84 additions and 68 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -4,7 +4,7 @@
*.swp *.swp
*.swo *.swo
*.pyc *.pyc
.testrepository .stestr/
# Editors # Editors
*~ *~

3
.stestr.conf Normal file
View File

@ -0,0 +1,3 @@
[DEFAULT]
test_path=./tests
top_dir=./

4
.testr.conf
View File

@ -1,4 +0,0 @@
[DEFAULT]
test_command=OS_STDOUT_CAPTURE=1 OS_STDERR_CAPTURE=1 OS_TEST_TIMEOUT=60 ${PYTHON:-python} -m subunit.run discover -t ./ . $LISTOPT $IDOPTION
test_id_option=--load-list $IDFILE
test_list_option=--list

2
requirements.txt
View File

@ -1,5 +1,5 @@
oslosphinx oslosphinx
pbr>=0.6,!=0.7,<1.0 pbr>=0.6,!=0.7,<1.0
sphinx>=1.1.2,!=1.2.0,<1.3 sphinx>=1.1.2,!=1.2.0,<1.3
testrepository>=0.0.18 stestr>=2.0.0
testtools>=0.9.34 testtools>=0.9.34

73
specs/queens/configuration-files-validation.rst
View File

@ -13,8 +13,9 @@ https://blueprints.launchpad.net/congress/+spec/configuration-files-validation
Congress could be used by cloud operators to formalize the constraints between Congress could be used by cloud operators to formalize the constraints between
options defined in configuration files with the help of business rules written options defined in configuration files with the help of business rules written
in Datalog and verify the compliance of their deployments automatically. in Datalog and verify the compliance of their deployments automatically.
It is intended to be complementary to config management systems that should only It is intended to be complementary to config management systems
create valid configuration files and to integration tests such as RefStack. that should only create valid configuration files
and to integration tests such as RefStack.
Problem description Problem description
@ -41,13 +42,13 @@ library. Configuration options are not independent:
Constraints may be defined by different actors: Constraints may be defined by different actors:
* Service developers know the constraints on the options they define and usually * Service developers know the constraints on the options they define and
document them informally in the source code in the description field associated usually document them informally in the source code in the description field
to the option. associated to the option.
* Cloud integrators will discover additional constraints when they develop * Cloud integrators will discover additional constraints when they develop
deployment code. Most of the time those options are only implicitly defined in deployment code. Most of the time those options are only implicitly defined
the source code of the scripts. in the source code of the scripts.
* Administrators may want to enforce additional constraints reflecting the * Administrators may want to enforce additional constraints reflecting the
operational environment constraints. operational environment constraints.
@ -80,13 +81,15 @@ deployer impact' section).
The agent is responsible for conveying information on option values but also on The agent is responsible for conveying information on option values but also on
their meta-data: layout in groups, type, flags such as deprecation, secret their meta-data: layout in groups, type, flags such as deprecation, secret
status, etc. Meta-data are described in templates which are in fact a collection status, etc. Meta-data are described in templates which are in fact
of namespaces. Namespace contain the actual definition of meta-data. To avoid a collection of namespaces. Namespace contain the actual definition of
versionning problems, Meta-data must be obtained directly from the service. meta-data. To avoid versionning problems, Meta-data must be obtained directly
from the service.
To limit the amount of traffic between the driver and agents, template files are To limit the amount of traffic between the driver and agents, template files
hashed. Agents first reply to queries with hashes. The driver will request the are hashed. Agents first reply to queries with hashes. The driver will request
content only if the hash is unknown. The same process is used for namespaces. the content only if the hash is unknown.
The same process is used for namespaces.
The only processing performed by agents on the option files is the removal of The only processing performed by agents on the option files is the removal of
the values of secret options. the values of secret options.
@ -124,11 +127,11 @@ The definition of an option is split between three tables:
defined in a specialized table. The option id is used again as a key to refer defined in a specialized table. The option id is used again as a key to refer
to the option table. to the option table.
Regarding the uniqueness of configuration meta-data in the extensional database, Regarding the uniqueness of configuration meta-data in the extensional
the driver must ensure that the ids are deterministic. An option identified database, the driver must ensure that the ids are deterministic. An option
by the same name, same group name and same namespace name should always be identified by the same name, same group name and same namespace name should
given the same unique id. The use of the MD5 hash function (there is no always be given the same unique id. The use of the MD5 hash function (there is
cryptographic requirement) guarantees uniqueness and determinism. no cryptographic requirement) guarantees uniqueness and determinism.
Alternatives Alternatives
------------ ------------
@ -141,13 +144,14 @@ space beyond what is necessary.
The proposed change must be considered as a work in progress. It does not fully The proposed change must be considered as a work in progress. It does not fully
address the problem of the location and the management of constraints. Most address the problem of the location and the management of constraints. Most
constraints are known by the service developers and should be maintained in the constraints are known by the service developers and should be maintained in the
source code of services in a dedicated meta-data field of the option definition. source code of services in a dedicated meta-data field of
the option definition.
The use of an external agent to push a service configuration is not the only The use of an external agent to push a service configuration is not the only
solution. The oslo-config library could be modified to push the configuration solution. The oslo-config library could be modified to push the configuration
read by the service to the datasource driver. This could be done through the use read by the service to the datasource driver. This could be done through
of a hook in oslo-config. It would require additional configuration of the the use of a hook in oslo-config. It would require additional configuration of
services to identify the endpoint. the services to identify the endpoint.
Policy Policy
------ ------
@ -165,7 +169,8 @@ Z: lb-agt host
:: ::
warn('Multinode OVS and linuxbridge use incompatible UDP ports', 'vxlan_conflicting_ovs_lb_udp_ports') :- warn('Multinode OVS and linuxbridge use incompatible UDP ports',
'vxlan_conflicting_ovs_lb_udp_ports') :-
vxlan_conflicting_ovs_lb_udp_ports() vxlan_conflicting_ovs_lb_udp_ports()
vxlan_conflicting_ovs_lb_udp_ports(Y, Z) :- vxlan_conflicting_ovs_lb_udp_ports(Y, Z) :-
@ -232,8 +237,8 @@ Other end user impact
--------------------- ---------------------
None other than the usual management of the datasource and policy. None other than the usual management of the datasource and policy.
Eventually, we would like to feed the engine with rules that are coming from and Eventually, we would like to feed the engine with rules that are coming from
maintained in the services source code. and maintained in the services source code.
Performance impact Performance impact
------------------ ------------------
@ -252,16 +257,16 @@ the duplicated sending of files and templates, and to prevent overloading the
driver. When the driver is activated, it periodically notifies agents, over the driver. When the driver is activated, it periodically notifies agents, over the
communication bus requesting their data description. An agent send description communication bus requesting their data description. An agent send description
of the files it has been set to provide. The description contains hashes of of the files it has been set to provide. The description contains hashes of
namespaces, templates and configs. The driver then requests the resources, which namespaces, templates and configs. The driver then requests the resources,
hashes have not been recognized. which hashes have not been recognized.
We use the RPC server of the datasource associated DseNode. We use the RPC server of the datasource associated DseNode.
Other deployer impact Other deployer impact
--------------------- ---------------------
We add a dedicated group and options to configure an agent and the configuration We add a dedicated group and options to configure an agent and the
files to manage. configuration files to manage.
*validator.host* *validator.host*
@ -282,7 +287,8 @@ An dict option describing the OpenStack services activated on this node. The
values are also dictionaries. Keys are paths to config-files, values are also dictionaries. Keys are paths to config-files,
values are paths to the associated templates. For instance:: values are paths to the associated templates. For instance::
congress: { /etc/congress/congress.conf:/opt/stack/congress/etc/congress-config-generator.conf } congress: { /etc/congress/congress.conf:
/opt/stack/congress/etc/congress-config-generator.conf }
Example config : Example config :
@ -293,7 +299,8 @@ Example config :
transport_url = rabbit://..@control:5672/ transport_url = rabbit://..@control:5672/
[validator] [validator]
services = nova : { /etc/nova.conf:/opt/stack/nova/etc/nova/nova-config-generator.conf: services = nova : { /etc/nova.conf:
/opt/stack/nova/etc/nova/nova-config-generator.conf:
version = ocata version = ocata
host = node_A host = node_A
@ -349,8 +356,8 @@ sending of meta-data and files.
Documentation impact Documentation impact
==================== ====================
This feature introduces an agent component that requires separate configuration. This feature introduces an agent component that requires separate
It also defines new datasources. configuration. It also defines new datasources.
References References

66
specs/rocky/explicit-data-types.rst
View File

@ -37,9 +37,10 @@ policy engines), and backwards-compatible with the standard ``agnostic`` engine
and existing data source drivers. and existing data source drivers.
As an illustration of the need for extensibility and loose-coupling, consider As an illustration of the need for extensibility and loose-coupling, consider
that a SMT-based Z3 prover needs types that are as narrow as possible (e.g., enum{'ingress', 'egress'}), but another engine does not understand such narrow that a SMT-based Z3 prover needs types that are as narrow as possible
types. A data source driver then must specify its types in a way that (e.g., enum{'ingress', 'egress'}), but another engine does not understand
works well for both engines. such narrow types. A data source driver then must specify its types
in a way that works well for both engines.
Proposed change Proposed change
@ -57,9 +58,9 @@ we expect new ones to be introduced over time). To allow a policy engine to
handle custom data types it does not "understand", we require that every handle custom data types it does not "understand", we require that every
custom data type inherit from another data type, all of which ultimately custom data type inherit from another data type, all of which ultimately
inherit from one of the base types all policy engines must handle. That way, inherit from one of the base types all policy engines must handle. That way,
when a policy engine encounters a value of a custom type it does not understand, when a policy engine encounters a value of a custom type
the policy engine can fall back to handling the value according to an ancestor it does not understand, the policy engine can fall back to handling the value
type the engine does understand. according to an ancestor type the engine does understand.
What is a type? What is a type?
--------------- ---------------
@ -67,8 +68,8 @@ What is a type?
To define what a type is in this spec, we first establish four related To define what a type is in this spec, we first establish four related
concepts: concepts:
#. *source representation*: the representation used in the data received from an #. *source representation*: the representation used in the data received from
external source. Each external data source has its own source an external source. Each external data source has its own source
representation defined outside of Congress. In an IP address example, one representation defined outside of Congress. In an IP address example, one
source could use IPv4 dotted decimal string ``"1.0.0.1"`` while another source could use IPv4 dotted decimal string ``"1.0.0.1"`` while another
source could use IPv6 (short) hexadecimal string ``"::ffff:100:1"``. source could use IPv6 (short) hexadecimal string ``"::ffff:100:1"``.
@ -193,19 +194,26 @@ could be specified for the ``flavors`` table in the Nova data source driver.
'selector-type': 'DOT_SELECTOR', 'selector-type': 'DOT_SELECTOR',
'field-translators': 'field-translators':
({'fieldname': 'id', 'desc': 'ID of the flavor', ({'fieldname': 'id', 'desc': 'ID of the flavor',
'translator': {'translation-type': 'VALUE', 'data-type': CongressStr}}, 'translator': {'translation-type': 'VALUE',
'data-type': CongressStr}},
{'fieldname': 'name', 'desc': 'Name of the flavor', {'fieldname': 'name', 'desc': 'Name of the flavor',
'translator': {'translation-type': 'VALUE', 'data-type': CongressStr}}, 'translator': {'translation-type': 'VALUE',
'data-type': CongressStr}},
{'fieldname': 'vcpus', 'desc': 'Number of vcpus', {'fieldname': 'vcpus', 'desc': 'Number of vcpus',
'translator': {'translation-type': 'VALUE', 'data-type': CongressInt}}, 'translator': {'translation-type': 'VALUE',
'data-type': CongressInt}},
{'fieldname': 'ram', 'desc': 'Memory size in MB', {'fieldname': 'ram', 'desc': 'Memory size in MB',
'translator': {'translation-type': 'VALUE', 'data-type': CongressInt}}, 'translator': {'translation-type': 'VALUE',
'data-type': CongressInt}},
{'fieldname': 'disk', 'desc': 'Disk size in GB', {'fieldname': 'disk', 'desc': 'Disk size in GB',
'translator': {'translation-type': 'VALUE', 'data-type': CongressInt}}, 'translator': {'translation-type': 'VALUE',
'data-type': CongressInt}},
{'fieldname': 'ephemeral', 'desc': 'Ephemeral space size in GB', {'fieldname': 'ephemeral', 'desc': 'Ephemeral space size in GB',
'translator': {'translation-type': 'VALUE', 'data-type': CongressInt}}, 'translator': {'translation-type': 'VALUE',
'data-type': CongressInt}},
{'fieldname': 'rxtx_factor', 'desc': 'RX/TX factor', {'fieldname': 'rxtx_factor', 'desc': 'RX/TX factor',
'translator': {'translation-type': 'VALUE', 'data-type': CongressFloat}}) 'translator': {'translation-type': 'VALUE',
'data-type': CongressFloat}})
} }
@ -360,9 +368,9 @@ of an ancestor type. Ideally, every policy engine would recognize and support
The only exception is that values of the non-string types The only exception is that values of the non-string types
inheriting from CongressStr need to be converted to string to be interprable inheriting from CongressStr need to be converted to string to be interprable
as a value of CongressStr type. as a value of CongressStr type.
The CongressDataType abstract base class can include additional helper methods to The CongressDataType abstract base class can include additional helper methods
make the interpretation easy. Below is an expanded CongressDataType definition to make the interpretation easy. Below is an expanded
including the additional helper methods. CongressDataType definition including the additional helper methods.
.. code-block:: python .. code-block:: python
@ -386,7 +394,8 @@ including the additional helper methods.
this type among the types the data consumer supports. this type among the types the data consumer supports.
:param supported_types: iterable collection of types :param supported_types: iterable collection of types
:returns: the subclass of CongressDataType which is the least ancestor :returns: the subclass of CongressDataType
which is the least ancestor
''' '''
target_types = frozenset(target_types) target_types = frozenset(target_types)
current_class = cls current_class = cls
@ -399,15 +408,16 @@ including the additional helper methods.
@classmethod @classmethod
def convert_to_ancestor(cls, value, ancestor_type): def convert_to_ancestor(cls, value, ancestor_type):
'''Convert this type's exchange value to ancestor_type's exchange value '''Convert this type's exchange value
to ancestor_type's exchange value
Generally there is no actual conversion because descendant type value Generally there is no actual conversion because descendant type value
is directly interpretable as ancestor type value. The only exception is directly interpretable as ancestor type value. The only exception
is the conversion from non-string descendents to string. This is the conversion from non-string descendents to string. This
conversion is needed by Agnostic engine does not support boolean. conversion is needed by Agnostic engine does not support boolean.
.. warning:: undefined behavior if ancestor_type is not an ancestor of .. warning:: undefined behavior if ancestor_type is not
this type. an ancestor of this type.
''' '''
if ancestor_type == CongressStr: if ancestor_type == CongressStr:
return json.dumps(value) return json.dumps(value)
@ -427,7 +437,7 @@ including the additional helper methods.
raise cls.CongressDataTypeHierarchyError( raise cls.CongressDataTypeHierarchyError(
'More than one parent type found for {0}: {1}' 'More than one parent type found for {0}: {1}'
.format(cls, congress_parents)) .format(cls, congress_parents))
class CongressDataTypeNoParent(TypeError): class CongressDataTypeNoParent(TypeError):
pass pass
@ -543,11 +553,11 @@ No impact on REST API.
Security impact Security impact
--------------- ---------------
There is little new security impact. Because new/custom types include new/custom There is little new security impact. Because new/custom types include
data handling methods, it does theoretically increase the attack surface. Care new/custom data handling methods, it does theoretically increase the attack
needs to be taken to make sure the data handling methods are safe against surface. Care needs to be taken to make sure the data handling methods are safe
malformed or possibility malicious input. There are well-known best practices to against malformed or possibility malicious input. There are well-known best
minimize such risks. practices to minimize such risks.
Notifications impact Notifications impact
-------------------- --------------------

2
tox.ini
View File

@ -10,7 +10,7 @@ usedevelop = True
setenv = VIRTUAL_ENV={envdir} setenv = VIRTUAL_ENV={envdir}
install_command = pip install -U {opts} {packages} install_command = pip install -U {opts} {packages}
deps = -r{toxinidir}/requirements.txt deps = -r{toxinidir}/requirements.txt
commands = python setup.py testr --slowest --testr-args='{posargs}' commands = stestr run --slowest {posargs}
[testenv:venv] [testenv:venv]
commands = {posargs} commands = {posargs}