As Cyborg is switching to new policy, this is required
to avoid breaking the existing deployment using policy
file in json format and relying on default value of
'CONF.oslo_policy.policy_file'.
Default value of 'CONF.oslo_policy.policy_file' config option
has been changed from 'policy.json' to 'policy.yaml'. If new default
file 'policy.yaml' does not exist but old default 'policy.json' exist
then fallback to use old default file.
An upgrade checks is added to check the policy_file format and
fail upgrade checks if it is JSON formatted.
Added a warning in policy doc about JSON formatted file is deprecated,
also removed all the reference to policy.json file in doc as well as
in tests.
Related Blueprint: https://blueprints.launchpad.net/oslo.policy/+spec/policy-json-to-yaml
Change-Id: I865227e516dc7505c463ac279309169d95ea6a22
All these uwsgi invocations assume that the uwsgi binary is in the
same directory as their project binaries are installed into (probably
/usr/bin). That may not be correct -- for example if using a packaged
uwsgi on Fedora the binary will live in /usr/sbin/uwsgi (not /usr/bin
where the project files from pip are).
Switch invocations to just find it in the path.
Also fix the docs job
Change-Id: Ib56b4d3f4962466ea9e937b314e54014070e7adc
This patch removes OPAE driver package from cyborg-agent dependencies due to
the following reasons:
1. In the kolla cyborg-agent image we install OPAE, but OPAE is not
available for CentOS 8 for the moment. This will make the cyborg-agent image
unbuildable in Ussuri[0].
2. In devstack, due to the fact that OPAE packages depend on libjson0, which
is not available after Ubuntu 16.04, so cyborg can't be installed on Ubuntu
higher than 16.04 now(unless disable dependency manually). Moreover,from
cyborg's perspective, it does not need to contain any hardware driver
dependency, we can assume the admin should know about it and install the
correct version.
3. We had an agreement at Victoria PTG that we should remove.[1]
[0]http://lists.openstack.org/pipermail/openstack-discuss/2020-June/015428.html
[1]https://etherpad.opendev.org/p/cyborg-victoria-goals L261
Change-Id: If0c92ea6ed98f195f5645ae0260855ed50f4f2a8
Story: 2007810
Task: 40078
Right now, devstack/clean.sh always leaves an orphaned cyborg-api
service running. This makes us always stop it to avoid that.
Change-Id: I1c40804a7b20d54da3c09e613597c388e9f330f7
This removes the devstack plugin line which overrides the CONF.host in
the cyborg agent with the $LOCAL_HOSTNAME. The latter may be different
than what the nova-compute service generates (as the nova devstack
module does not override the host). Since these need to match, just
remove the override here so that both services use the hostname from
the service framework.
Closes Task: #38811
Change-Id: I393ccbf020cc4f80ce78b1ba4564a833abec02bf
This change adds support for installing the
python-cyborgclient from git. By default it is installed
but it can be disabled by setting CYBORG_CLIENT_INSTALL=False.
The version and repo can be set using CYBORG_CLIENT_REPO and
CYBORG_CLIENT_BRANCH.
Change-Id: I1dd616fe1e0c5b03118a23fc5afe2a4d3bb3686d
This change replaces the nested if used
in plugin.sh with nested case statements for
greater readablity.
This change removes an unnecessary check to determine
if services are enabled. By default all cyborg
service are enabled by the plugin. Since the plugin is
not loaded unless the user specifies the enable_plugin
line in there local.conf we can assume they want the plugin
to run and there for can remove the service check.
This change removes the install_agent function and renames
install_cyborg_in_controller function to
check_cyborg_service_deps to better reflect what the function
does.
Change-Id: Ia5ad1235ae4a5a76bcd498a743ca6be6b3634c5c
This change removes the non standard mechanism
for specifying the cyborg repo and branch
via CYBORG_REPO and CYBORG_BRANCH.
in-tree devstack modules use the
<PROJECT NAME>_REPO and <PROJECT NAME>_BRANCH
convention for specifying revisions of project to clone
however devstack plugins should only use that convention
for cloning other git repos.
A devstack plugin should never clone the repo that contains
the plugin itself as doing so will replace the code while it
is executing. This can lead to the code that is running and the
code on disk being different if the plugin line and CYBORG_*
options do not match.
This change updates the devstack_setup docs with an example of
testing an unmerged change.
This change removes some commented out code and unneeded checks in
devstack/lib/*
Change-Id: I473ef276197ad6578f64a708f4b66562a2ff1d49
All user settable options should be stored the
devstack/setting file so they are defiend when it is
sourced early in devstack to allow values to be shared
between plugins if required.
This change moves the cyborg settings from
devstack/lib/cyborg to devstack/settings to conform to
the standard plugin interface.
The name of the folder where devstack clones the plugin
is specified in the first argument ot the enable_plugin
function invocation in the local.conf.
This change makes updates CYBORG_DIR to respect that
and adds TODOs for other issues that will be adressed in
follow up patches.
Change-Id: I5b6879e5ddb86659b8c7eb87b8d26cee33ed4754
Code in grenade and elsewhere rely on the process/service name
when one runs "ps auxw" and they grep for example "grep -e cyborg-api"
to check if the service is running. with uwsgi, let us make sure
we use process name prefix so it is easier to spot the services
and be compatible with code elsewhere that relies on this.
Reference:
https://review.opendev.org/#/c/494531/
Change-Id: I7afa389160d3a2355e7a077fec2570423254c83d
According to nova's advice, api_servers is useless and
We should get rid of it[1].
[1]. https://review.opendev.org/#/c/682565/ PatchSet 16.
Co-Authored-By: zhurong <aaronzhu1121@gmail.com>
Change-Id: I84cb846561fba941b2b2cdff422a94928f9bfda1
1. Refer to nova's latest code, use get_sdk to create the client[1].
2. Move placement_client define from default to seperate placement file.
Note:
The new method for get nova and placement client has been tested in latest
cyborg devstack environment. And the test results are good.
Because now cyborg does not have unit tests for the common and conf directory,
unit tests related to this patch will be added in the future, but not in this patch.
Reference:
[1]. 3f019f2db3/nova/utils.py (L1013)
Change-Id: I2a335d47ee3cc24f78b4d49942f1cccd586a0dcc
On one hand, as discussed in this patch[0], rootwap was intented
to be replaced by privsep, especially for new project.
On the other hand, we need remove rootwrap to fix the following
cyborg-tempest job failure in patch [0]:
cannot stat '/opt/stack/cyborg/etc/cyborg/rootwrap.d/*.filters':
No such file or directory
[0]:https://review.opendev.org/#/c/673957/
Change-Id: I1aaf38da3a68c656616c7d701b8f815e17ed0152
The "enabled_drivers" is a configuration of agent section [1], but
it's set to DEFAULT section in devstack.
This patch try to fix it to make the "enabled_drviers" work.
[1] https://github.com/openstack/cyborg/blob/4a96835/cyborg/conf/agent.py#L27
Change-Id: I05b8143cc54f80ddd87561cd20a70810d64e225b
Currently, OPAE is so picky about the OS.
Not every developer need OPAE.
In order to make them happy, add knot for OPAE install.
Change-Id: Ief2c75c4bc80bb7b01a812fd6f18abd6a3acfcc5
1. missing glance config section
2. missing import keystoneauth1.exceptions
When call glance fail during download image, it will report:
NameError: global name 'ks_exc' is not defined
Change-Id: I17f25368f3361b4fc20a0eda9d6b2788beaa6362
With this patch, cyborg can report accelerators information into
placement. Then scheduer can find an appropriate host according to
user requirements.
This patch leverages Nova's code in order to avoid reinventing the wheel.
It is batter that nova can split the placement client to a separated lib.
So other projects can use it easily.
This new client can support provider trees, that is needed by cyborg's
different dirvers.
We can create a sub provider tree easily by the follow method.
class SchedulerReportClient(object):
def get_provider_tree_and_ensure_root(
self, context, rp_uuid, name=None,
parent_provider_uuid=None)
The patch include:
1. add get_ksa_adapter for placement client
2. add placement client
3. update placement config
Missing testcase for this patch.
Change-Id: I1ad9d525fa070dfa0f7cbf374003a74c50de17b4
1. amend keyston endpoint register info
Service type should acceleration instead of cyborg.
For cyborg is just the service name.
We do not support uwsgi at present, so we must indicate service
port explicitly.
2. Fix package import error
Add full path to import module, so that the reference can be
resolved.
Story: 2001887
Task: 14368
3. devstack broken
TypeError: launch_service() got an unexpected keyword argument 'restart_method'
When initialize cyborg api service launcher, we already pass restart_method.
So no need to pass restart_method to launcher.launch_service, and it has not
restart_method argument.
Co-Authored-By: jiapei <jiapei2@lenovo.com>
Co-Authored-By: Zhipeng Huang <huangzhipeng@huawei.com>
Change-Id: Ic69e76b7bf86bbd1382aff07694ea9fbafd0862a
The related bug was spotted when I tried to setup cyborg environment by devstack but failed to start cyborg.
It is result from the permission change in devstack/lib/cyborg.
Closes-bug: #1742643
Change-Id: I54426aea597119253f07acf243d6616cd27935ac
songwenping