OpenStackClient has a significant amount of startup overhead, which
adds a non-trivial amount of time to each devstack run because it makes
a lot of OSC calls. This change uses the OSC service from [0] to run
a persistent process that handles openstack calls. This removes most
of the startup overhead and in my local testing removes about three
minutes per devstack run.
Currently this is implemented as an opt-in feature. There are likely a
lot of edge cases in projects that use a devstack plugin so turning it
on universally is going to require boiling the ocean. I think getting
this in and enabled for some of the major projects should give us a lot
of the benefit without the enormous effort of making it 100% compatible
across all of OpenStack.
Depends-On: https://review.opendev.org/c/openstack/nova/+/918689
Depends-On: https://review.opendev.org/c/openstack/ironic/+/918690
Change-Id: I28e6159944746abe2d320369249b87f1c4b9e24e
0: http://lists.openstack.org/pipermail/openstack-dev/2016-April/092546.html
Make it a little more obvious what the difference between the two helper
functions is.
Change-Id: I07ec34ecfcd2b7925485145c4b4bf68eda385a32
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We'd like to move from configuring uWSGI with '.wsgi' files to
configuring with module paths. Do this for all in-tree services and log
a deprecation warning for anyone still passing a path.
Note that since 'basepath foo' returns 'foo', this is effectively a
no-op for the services being converted here.
Change-Id: Ia1ad5ff160a9821ceab97ff1c24bc48cd4bf1d6f
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
devstack doesn't do releases, so there should be no release notes,
either. Drop the one that was accidentally created to avoid confusion.
Change-Id: I75a295e50c36925a0137a5458444fb48bd5d9f8a
This is more likely how people will actually upload their images, but
it also prevents the "osc as a service" feature from working because
stdin isn't proxied (of course). So just convert our uses of "image
create" to use --file instead of stdin.
Change-Id: I7205eb0100ba7406650ed609cf517cba2c8d30aa
Running stack.sh on a python 3.12 system generates this
warning from worlddump.py:
DeprecationWarning: datetime.datetime.utcnow() is deprecated
Use datetime.now(timezone.utc) instead, which should be
backwards-compatible with older python versions.
TrivialFix
Change-Id: I11fe60f6b04842412045c6cb97f493f7fef66e1a
pbr's 'wsgi_scripts' entrypoint functionality is not long for this world
so we need to start working towards an alternative. We could start
packaging our own WSGI scripts in DevStack but using module paths seems
like a better option, particularly when it's supported by other WSGI
servers like gunicorn.
Currently only nova is migrated. We should switch additional projects as
they migrate and eventually remove the support for WSGI scripts
entirely.
Change-Id: I057dc635c01e54740ee04dfe7b39ef83db5dc180
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Depends-on: https://review.opendev.org/c/openstack/nova/+/902687/
The variable should be in quotes for the check to work
Testing the behavior in bash:
current behavior:
$ config_file=""
$ if [ -n ${config_file} ]; then echo a; fi
a
$ config_file="abc"
$ if [ -n ${config_file} ]; then echo a; fi
a
behavior with quotes:
$ config_file=""
$ if [ -n "$config_file" ]; then echo a; fi
$ config_file="abc"
$ if [ -n "$config_file" ]; then echo a; fi
a
Change-Id: Iba956d9d4f43b925848174a632aabe58999be74b
This patch removes a couple of tempest.conf settings that are being
overwrriten when Keystone is set to enforce scope.
These settings are already being set by the keystone devstack plugin [1]
and do not need to be overwritten here.
Keystone is changing the default admin credentials to be project-admin
instead of system-admin to address some failing tests in services that
require project-scoped admin for their admin APIs. [2] These overrides
are preventing that change from taking effect.
[1] https://opendev.org/openstack/keystone/src/branch/stable/2024.1/devstack/lib/scope.sh#L24-L25
[2] https://review.opendev.org/c/openstack/keystone/+/913999
Change-Id: I48edbcbaa993f2d1f35160c415986d21a15a4999
Updated clean.sh to remove Glance's Apache uWSGI config files in
APACHE_CONF_DIR, including /etc/apache2/sites-enabled/ on Ubuntu.
Test Plan:
- Run clean.sh.
- Confirm Glance uWSGI configs are removed from APACHE_CONF_DIR.
Closes-Bug: #2057999
Change-Id: I44475b8e084c4b20d7b7cb7f28574f797dbda7a2
The ubuntu-xenial labels are going to disappear from opendev as that
image is EOL and will we deleted. Clean up our zuul config.
Update some example reference as well.
Change-Id: Id04110f7c871caa1739ff2b62e9796be4fb9aa00
In the _setup_package_with_constraints_edit name of the package was
always discovered from the setup.cfg file. But as some projects
implements PEP-621 (see [1] for the SQLAlchemy for example) it is not
enough now.
This patch adds parsing pyproject.toml file also if name is not found in
the setup.cfg file.
[1] a8dbf8763a
Closes-Bug: #2052509
Change-Id: Iee9262079d09a8bd22cd05a8f17950a41a0d1f9d
Making newly introduced `centralized_db` driver as default cache
driver for glance so that it can be tested in available CI jobs.
New cache driver `centralized_db` needs `worker_self_reference_url`
in glance-api.conf file otherwise glance api service will fail to
start.
Related blueprint centralized-cache-db
Depends-On: https://review.opendev.org/c/openstack/glance/+/899871
Change-Id: I75267988b1c80ac9daa5843ce8462bbac49ffe27
The bashate tool has been very stable for a while and we rarely expect
changes which may break existing scripts.
This removes the current capping to avoid updating the upper limit when
when a new release is created in bashate.
Change-Id: Iae94811aebf58b491d6b2b2773db88ac50fdd737
zswap should only be enabled if ENABLE_ZSWAP is true.
The if condition was checking ENABLE_KSMTUNED.
That is now fixed.
Change-Id: I76ba139de69fb1710bcb96cc9f638260463e2032
This change add a new lib/host-mem file and moves the existing
ksm support to a new configure_ksm function.
Additional support for ksmtuned is added with a new flag
"ENABLE_KSMTUNED" which defaults to true.
This change also adds support for zswap. zswap is disabled
by default. When enabled on ubuntu lz4 will
be used as the default compressor and z3fold as the zpool.
On non debian distros the compressor and zpool are not set.
The default values should result in very low overhead although
the zstd compressor may provide better overall performance in ci
or with slow io due to the higher compression ratio.
Additionally memory and network sysctl tunings are optionally applied
to defer writes, prefer swapping and optimise tcp connection
startup and keepalive. The sysctl tunings are disabled by default
The base devstack job has been modifed to enable zram and sysctl
tuning.
Both ksm and zswap are wrapped by a tune_host function
which is now called very early in devstack to ensure
they are configured before any memory/network intensive
operations are executed.
The ci jobs do not enable this functionality by default.
To use this functionaltiy define
ENABLE_SYSCTL_MEM_TUNING: true
ENABLE_SYSCTL_NET_TUNING: true
ENABLE_ZSWAP: true
in the devstack_localrc section of the job vars.
Change-Id: Ia5202d5a9903492a4c18b50ea8d12bd91cc9f135
The Neutron OVN agent is a service that could run in any node. The
functionality will depend on the extensions configured. This new
agent is meant to be the replacement for the Neutron OVN metadata
agent once the "metadata" extension is implemented in this service
[1].
[1]https://review.opendev.org/c/openstack/neutron/+/898238
Related-Bug: #2017871
Change-Id: I8f82f0047e89aac122a67f59db84f03e1a6bf519
This preserved `ADITIONAL_VENV_PACKAGES` as an input for backwards
compatiblity, but takes `ADDITIONAL_VENV_PACKAGES` with priority.
Fixes spelling in comment.
Related-Bug: #2046936
Change-Id: I84151d8f71b12da134e8fb9dbf3ae30f2a171fe2
Looking at the recent failures in the tempest-integrated-compute
job, the reimage operation seems to be taking longer than our
expected time of 60 seconds (which was increased because of a similar
failure in the past, default is 20 seconds).
The main culprit for this failure is the image conversion from qcow2
to raw which is taking ~159 seconds.
Dec 05 13:29:59.709129 np0035951188 cinder-volume[77000]: DEBUG oslo_concurrency.processutils [req-5113eccb-05ba-486a-8130-a58898c8ad35 req-0edf972a-109a-465f-a771-ceb87ecbda3e tempest-ServerActionsV293TestJSON-1780705112 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -t none -f qcow2 /opt/stack/data/cinder/conversion/image_download_dbe01f18-1c90-4536-a09a-b49f0811c7a0_copod3cm /dev/mapper/stack--volumes--lvmdriver--1-volume--073a98e8--3c89--4734--9ae5--59af25f8914a" returned: 0 in 159.272s {{(pid=77000) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
The recent run took ~165 seconds on the cinder side but it failed
early since the nova operation timed out in 60 seconds hence
deleting the volume.
To be on the safer side, 180 seconds seems to be a sane time for
the operation to complete which this patch configures.
Closes-Bug: 2046252
Change-Id: I8a9628216038f6d363cab5dd8177274c9cfc17c2
Downloading an image can fail due to network issues, so let's
retry 5 times before giving up. We have seen issues in CI due
to network issues as described below and in the Related-Bug:-
Often times fetching Fedora image in FIPS jobs fails due to
"GnuTLS: One of the involved algorithms has insufficient security level."
This occurs when request to pull image is redirected to a mirror that's
incompatible with FIPS enabled system.
Making multiple attempts to download images could provide better chance of
pulling images from different mirrors and avoid failure of the job.
This will also save a few rechecks.
Related-Bug: #2045725
Change-Id: I7163aea4d121cb27620e4f2a083a543abfc286bf
Added new devstack variable `GLANCE_CACHE_DRIVER` default
to `sqlite` to set the cache driver for glance service.
Related blueprint centralized-cache-db
Change-Id: I76d064590356e2d65bfc6a3f57d1bdaeeb83a74a