Commit Graph

362 Commits

Author SHA1 Message Date
Yadnesh Kulkarni 6b0f055b4e Make multiple attempts to download image
Downloading an image can fail due to network issues, so let's
retry 5 times before giving up. We have seen issues in CI due
to network issues as described below and in the Related-Bug:-

Often times fetching Fedora image in FIPS jobs fails due to
"GnuTLS: One of the involved algorithms has insufficient security level."

This occurs when request to pull image is redirected to a mirror that's
incompatible with FIPS enabled system.

Making multiple attempts to download images could provide better chance of
pulling images from different mirrors and avoid failure of the job.
This will also save a few rechecks.

Related-Bug: #2045725
Change-Id: I7163aea4d121cb27620e4f2a083a543abfc286bf
2023-12-08 17:32:34 +05:30
Eliad Cohen fdfc14451a Clean up use of get_field
Openstack client can return the id field for create/show commands using
`-f value -c id`. Cleaned up the use of grep 'id' with get_field

Change-Id: I2f4338f30c11e5139cda51c92524782b86f0aacc
2022-08-16 15:27:45 -04:00
Stephen Finucane 970891a4ef Remove references to XenAPI driver
The XenAPI driver was removed during the Victoria release [1], while the
libvirt+xen driver has been removed in the Wallaby release [2]. Remove
references to Xen from DevStack since its all a no-op now.

[1] I42b302afbb1cfede7a0f7b16485a596cd70baf17
[2] I73305e82da5d8da548961b801a8e75fb0e8c4cf1

Change-Id: If7055feb88391f496a5e5e4c72008bf0050c5356
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-03-05 15:10:19 +00:00
Dan Smith 30d9bf9a6d Async task support
We have a *ton* of stuff in devstack that is very linear, specifically
the ten-ish minutes we spend loading osc to run a single API command
against something. We also generate configs, sync databases, and other
things that use one core of our worker and make our runtime longer
than it really needs to be.

The idea in this patch is to make it super simple to run some things
in the background and then wait for them to finish before proceeding
to something that will require them to be done. This avoids the
interleaving you would expect by redirecting the async tasks to a log
file, and then cat'ing that log file synchronously during the wait
operation. The per-task log file remains so it's easier to examine
it in isolation.

Multiple people have reported between 22-30% improvement in the
time it takes to stack with this. More can be done, but what is here
already makes a significant difference.

Change-Id: I270a910b531641b023c13f75dfedca057a1f1031
2021-02-09 15:57:04 -08:00
Dan Smith 36a575b036 Clean up create_disk() a little
The create_disk() helper had some redundant checks and dead code. This
refactors it to put all the stale cleanup at the top, and groups the
new actions together with more relevant comments to make it easier
to understand.

Change-Id: I1f6218a1994e66786ed9a8065e30bcceec7b8956
2021-01-07 11:29:01 -08:00
Dan Smith 6766f71d62 Make create_disk() persistent
Right now a system configured with the ceph plugin will not survive
a reboot because the backing disk we create and mount isn't mounted
at startup, preventing ceph from starting and the rest of nova/glance
from working.

This makes create_disk() idempotently write an fstab rule for the
disk we make, and adds a destroy_disk() handler for cleanup.

Change-Id: I50cd4234f51a335af25be756bd2459dca5aa343c
2020-11-09 14:59:23 -08:00
Dan Smith 33f8f6e684 Remove standalone requirement for glance import
As of the referenced patch in glance, we can do import in wsgi mode.
Also remove the enforcement that import methods are disabled.

Change-Id: I8da4b4ad6105bb64c4045ca80db9742591d01564
Depends-On: https://review.opendev.org/#/c/742065
2020-07-22 07:03:57 -07:00
Dan Smith 49ad4850c8 Only set hw_rng_model by default if we're on libvirt
This makes no sense to set on _every_ devstack deployment, only
if we are using libvirt qemu or kvm. Make it conditional.

Change-Id: I860e899274646ff73b8f084a0583325239aee9cc
2020-07-16 06:30:54 -07:00
Dan Smith 2614c1bba1 Fix ami/aki image create use of $img_property
This abstracts out the conversion of key=value,... property lists to a function
and makes both _upload_image() and the two ami/aki image create calls use it. The
move to bare key=value properties introduced a regression for ami/aki where
the --property flag stopped being passed to osc in that case.

Change-Id: Idf7fdfe3f5800f79f6c48f9d9606a7b53436a730
2020-07-16 00:26:42 +00:00
Dan Smith 09eea0b20b Make Glance standalone-ness a separate flag
Full Glance functionality requires Glance being run in a configuration
where it can spawn long-running task threads. The default uwsgi mode
does not allow this, and the current workaround is to set WSGI_MODE
to something other than uwsgi to get the devstack code to deploy
Glance as a standalone service. Since this affects the entire rest of
the deployment, this patch separates out a flag to control this behavior
specifically for Glance. When WSGI_MODE=uwsgi, control of the Glance
deployment mechanism is allowed via GLANCE_STANDALONE=True|False. If
WSGI_MODE!= uwsgi then we deploy standalone Glance anyway.

Change-Id: I79068ce0bd7414bc48ff534ee22f0de5d7b091cb
2020-07-14 08:07:07 -07:00
Abhishek Kekane 73ad9760a3 Use glance import workflow for creating image
Added new boolean option 'GLANCE_USE_IMPORT_WORKFLOW' default to False.
If this parameter set in local.conf as True then devstack will use the
new import workflow to create the image.

In order to use new import workflow of glance;
user need to set below options in local.conf

GLANCE_USE_IMPORT_WORKFLOW=True

Note that the import workflow does not work in uwsgi because of
some fundamental restrictions it has. Thus, devstack must be configured
with WSGI_MODE=mod_wsgi, otherwise glance will not be able to process
the imports. The new helper function will abort to avoid in that case
to avoid the image never being moved to "active" state by an import
task that will never be executed.

Co-Authored-By: Abhishek Kekane <akekane@redhat.com>
Co-Authored-By: Dan Smith <dansmith@redhat.com>
Needed-By: https://review.opendev.org/#/c/734184
Change-Id: I1306fe816b7a3eca1e2312c0c454be3d81118eca
2020-07-13 13:59:36 -07:00
Zuul e0f051ced7 Merge "Clean up remainders of USE_SYSTEMD and logging setup" 2020-06-04 10:36:47 +00:00
Bharat Kunwar f70cb70acb Support upload of qcow2.xz image format
Some images come in *.xz format [1] and this needs to be
uncompressed before being uploaded.

[1] https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/31.20200323.3.2/x86_64/fedora-coreos-31.20200323.3.2-openstack.x86_64.qcow2.xz

Change-Id: I11f8484ba939c4b2c0a98aa08ef7db730baf71e8
2020-04-22 15:54:58 +00:00
Dr. Jens Harbott 6808a3468d Clean up remainders of USE_SYSTEMD and logging setup
We dropped the ability to override USE_SYSTEMD to False when we deleted
screen support in [0], so we can also clean up any conditionals based
on it.

Also clean up the logging setup functions, dropping local vars for
parameters that we don't actually support anymore.

[0] I8c27182f60b0f5310b3a8bf5feb02beb7ffbb26a

Change-Id: I5cbce9f2c42e111761e8689447b3f8cbb7ea2eb5
2020-01-23 11:50:07 +00:00
Clark Boylan 7ddbece508 Enable libvirt virtio rng device
We've seen jobs where tests fail due to what appears to be rng
starvation. Enable virtio rng device to try and alleviate this.

Change-Id: I70d800cdc45b6008f775110f22c0000736421529
2019-12-05 07:58:23 -08:00
Matt Riedemann b14665f0dd Revert "Remove deprecated PostgreSQL database driver"
This reverts commit 168ca7f0a4.

Removing postgresql support from devstack was unnecessary
since it's not broken and not causing maintenance issues
as far as I know. The commit being reverted said that pg
support was deprecated in Pike but nothing in the docs or
commit message refer to official deprecation of postgres
support in devstack or openstack in general. Not to mention
that there are still postgres-based jobs that will no
longer work *and* the notification to the mailing list about
doing this happened *after* it was already done [1] leaving
stakeholders with no time to reply.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010196.html

Change-Id: Ie7036d37d79e6aba462b7c97f917e2e7aed108f9
2019-10-17 15:58:34 -04:00
Stephen Finucane 168ca7f0a4 Remove deprecated PostgreSQL database driver
This was deprecated for removal in Pike. It's probably time to drop it.
Note that the 'postgresql-devel'/'postgresql-server-dev-all' packages
are retained since some packages still include 'psycopg2' in their
general requirements.

Change-Id: I51e8354e99972757253ce259e6c03c91da24398c
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-08-26 10:16:06 +01:00
Eric Fried 2468ceaa72 Merge (don't overwrite) $NOVA_CPU_CONF
Per the referenced bug, $NOVA_CPU_CONF was previously being initialized
by copying $NOVA_CONF, thereby trashing any values already configured in
$NOVA_CPU_CONF.

With this commit, we merge the values from $NOVA_CPU_CONF in after the
copy.

Note that this makes use of the merge_config_file function, which is
defined in inc/meta-config, which wasn't being sourced from every code
path that hit start_nova_compute; so this commit also moves that import
from stack.sh to functions (next to the other imports from inc/, which
makes sense anyway).

Change-Id: Id3e2baa2221e13f512f8dcf1248e1e15b6a7597f
Closes-Bug: #1802143
2019-08-22 13:34:17 +00:00
Stephen Finucane 4b8cba77fe Remove n-cells, n-net and n-cauth
Remove nova cells v1 support, which also allows/necessitates removing
support for nova networks (which was only supported with cells v1) and
nova-consoleauth (which was required by cells v1 but is unnecessary
otherwise).

The Depends-On isn't really necessary, but it's here to make sure this
doesn't merge until we _really_ have killed cells v1.

I honestly expected this patch would be bigger.

Change-Id: I90316208d1af42c1659d3bee386f95e38aaf2c56
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Depends-On: Ib0e0b708c46e4330e51f8f8fdfbb02d45aaf0f44
2019-05-31 15:10:05 +01:00
Lee Yarwood 5d7d891380 Drop the deprecated and now removed barrier XFS mount options
Both barrier and nobarrier were deprecated with the 4.10 kernel [1] and
then removed [2] with the 4.19 kernel as now used by Fedora >= 28. Both
should be safe to drop at this point.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4cf4573
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1c02d50

Change-Id: I6871a7765e3e04122d8d546f43d36bb8415383fc
2019-01-04 20:53:29 +00:00
Maxim Nestratov c3a33fb691 Fix Virtuozzo CI
Don't specify hypervisor_type=vz property for hds images as nova
now reports only really connected hypervisors and not all supported
by a compute host

Change-Id: Ibec0856519ffe593d44d123f3b401eae19f7d95a
2018-07-04 22:07:20 +03:00
Prabhat Ranjan 6f38cf4ad8 Fix wait_for_compute to work for modified compute host name
When we change the name of compute host then devstack is breaking
because it is using default host name from host.

How to change compute host name in local.conf
[[post-config|$NOVA_CONF]]
[DEFAULT]
host = foo

Change-Id: I4d4392f1f58f0431b10764610668565af88d392f
Signed-off-by: Prabhat Ranjan <pranjank@in.ibm.com>
2018-07-02 05:49:52 +00:00
Dirk Mueller 6bab8321b4 Remove references to hardcoded file writing in /tmp
/tmp is a world writeable directory, so using hardcoded filenames
in there is just a bad coding style (susceptible to symlink attacks).
Avoid using it to not give a bad precedent.

Change-Id: Ia66763a0e4714f2226e98dbd85600b2035bd5088
2018-03-02 21:15:04 +01:00
Zuul a30f89b417 Merge "ceph plugin installation: umount correct device if it existed." 2018-02-27 09:16:25 +00:00
Chris Dent ac475bbb2a Correct configuration setting using NUMBER_FAKE_NOVA_COMPUTE
In Change-Id Ia3843818014f7c6c7526ef3aa9676bbddb8a85ca the 'host'
setting used for each of the fake compute hosts was accidentally named
'nhost' ('\nhost' was edited poorly), so the setting doesn't actually do
anything: you create multiple nova-compute processes they think they are
all on the same host and only one hypervisor and resource provider is
created.

With the correction in place, the wait_for_compute function needs to be
updated to be aware of the fact that the hostnames on the compute
services will have a numeric prefix when the fake virt driver is used.

Change-Id: I5e8430d170c0b1c4f195ebe510aff8be59e4a3bc
2018-02-07 19:06:36 +00:00
Ryota MIBU 842d54a299 use openstack command instead of nova command
In function 'get_instance_ip', 'nova' client command is used to get
instance information in order to retrive IP address of the instance.

There is no need to use the nova command, since 'openstack' client
already supports such basic operation.

Moreover, 'openstack' client has an option to get value of specified
column. That brings more accurate way of retriving IP address.

This patch replaces nova command in 'get_instance_ip' by 'openstack'
command. Note, this nova command is the only one in devstack tree.

Change-Id: Iee0b81a994a4da5b3f4572c2e8eb30514cd43f89
Signed-off-by: Ryota MIBU <r-mibu@cq.jp.nec.com>
2018-01-24 23:52:56 +09:00
liumk 38a23d901a ceph plugin installation: umount correct device if it existed.
The loopback device is created for ceph osd. If the directory
${storage_data_dir} has been mounted when create disk, we should
umount ${storage_data_dir} instead of ${storage_data_dir}/drives/sdb1.

Change-Id: Ie9fe81c820c485dab9f049cf5a81c02424925728
Closes-Bug: #1689089
2017-12-13 15:53:08 +08:00
Rafael Folco 7b8f1e7964 Set default disk driver to virtio-blk on Power
Reason is to be identical to the upstream KVM CI. Some Tempest
tests rely on vdX virtio-blk device naming. Others simply create
their own with a brand new image. Also, the scsi support on the
CirrOS image is limited, tests booting from volume fail.

Change-Id: I389147a58042aa6098a695e6dd32f3e697fbbbab
2017-10-17 17:57:37 +00:00
yuanke wei c652a498fb Fix partial download problem
(1) when checksum fails, better delete the broken files and try the second time;
(2) amazon s3 is not good in mainland China, better try one more time with wget

Change-Id: I24ee73f216b78bd80564863cd335e5d5a9b56360
2017-09-22 19:44:43 +08:00
Jenkins 87943ab88e Merge "Make list_images.sh emit the etcd3 tarball" 2017-09-10 17:13:15 +00:00
Monty Taylor d8bb220606 Make list_images.sh emit the etcd3 tarball
We need this for every devstack run now, so downloading it from github
every time isn't the most awesome thing in the world.

Add an extra variable EXTRA_CACHE_URLS which will be appended to the
output of tools/image_list.sh.  This way, these files will be
downloaded during the daily nodepool build, but they will not be in
the IMAGE_LIST and hence be considered as images to upload.

Add a function get_extra_file which echos the path to a file given the
URL.  It will first check the cache at $FILES, and if not present
download it.

Update the documentation in image_list.sh to reflect what's happening.

Move the defaults for etcd variables into stackrc, since it is a base
service now.

Change-Id: I86104824a29d973a6288df1f24b7891feb86267c
2017-09-05 09:40:40 +10:00
Sean Dague c2fe916fc7 Wait for compute service to check in
With cell v2, on initial bring up, discover hosts can't run unless all
the compute nodes have checked in. The documentation says that you
should run ``nova service-list --binary nova-compute`` and see all
your hosts before running discover hosts. This isn't really viable in
a multinode devstack because of how things are brought up in parts.

We can however know that stack.sh will not complete before the compute
node is up by waiting for the compute node to check in before
completing. This happens quite late in the stack.sh run, so shouldn't
add any extra time in most runs.

Cells v1 and Xenserver don't use real hostnames in the service table
(they encode complex data that is hostname like to provide more
topology information than just hostnames). They are exempted from this
check.

Related-Bug: #1708039

Change-Id: I32eb59b9d6c225a3e93992be3a3b9f4b251d7189
2017-08-02 09:40:25 -04:00
Sean Dague 01c0cc6d37 Revert "Wait for compute service to check in"
This reverts commit dcdf8c8e60.

Change-Id: Ib14016a3bc6f2714758ad0291396233218c593c6
2017-07-27 20:53:57 +00:00
Sean Dague dcdf8c8e60 Wait for compute service to check in
With cell v2, on initial bring up, discover hosts can't run unless all
the compute nodes have checked in. The documentation says that you
should run ``nova service-list --binary nova-compute`` and see all
your hosts before running discover hosts. This isn't really viable in
a multinode devstack because of how things are brought up in parts.

We can however know that stack.sh will not complete before the compute
node is up by waiting for the compute node to check in before moving
forward. This puts a few more seconds into the run, but ensures
everything is solid in multinode environments.

Change-Id: I667e6a9be3fee8bb5bfd73426eef567489e3d88d
2017-07-26 07:26:50 -04:00
Ian Wienand 07cbc44942 Put devstack-version info into separate file
As a follow on to I4c269a7f3d63ee9a976e7c3636fc3e5e8dab9ae3; the
quoting gets tricky when putting arbitrary command-substitution
strings into saved echo-able strings.  As they say, "the only winning
move is not to play" :)

An alternative proposal is to not write this into a script but just
dump info into a file.  To my mind, this has several advantages --
avoid getting involved in quoting, not dropping a script into the
global environment -- it's just as easy to "cat" -- and the plain-text
file can be collected as an artifact during log collection (also moved
git commit line to separate line for easier parsing during log search,
etc).

Change-Id: Ic7391dd087657c0daf74046e4a052c53f4eb6e1a
2017-07-03 11:56:16 +10:00
Manjeet Singh Bhatia 3415521d56 Don't dump commit message header to devstack-version
Dumping commit message to /usr/local/bin/devstack-version
has broken devstack build, because of bash error in formatting.
error log [1].

[1]. http://logs.openstack.org/20/418720/6/check/gate-tempest-dsvm-networking-odl-boron-snapshot-v2driver/ed9fd45/logs/devstacklog.txt.gz

Change-Id: I4c269a7f3d63ee9a976e7c3636fc3e5e8dab9ae3
2017-06-30 02:01:06 +00:00
Sean Dague 2c0faca038 Add devstack-version script
This will help in collecting information about the devstack version
being used.

Change-Id: I23a8ca023bda6d097051cd57ae45401806451ad5
2017-06-28 13:53:37 -04:00
Jenkins 805304f2ce Merge "Add a function to get an available random port" 2017-06-20 09:11:25 +00:00
Jenkins 4bca739576 Merge "Use correct argument order in truorfalse for USE_JOURNAL" 2017-06-06 13:24:32 +00:00
Jenkins 86f9bc7b0b Merge "Add global_request_id to systemd logs" 2017-05-31 12:29:52 +00:00
Matthew Treinish 309b99ebcf
Add a function to get an available random port
This commit adds a new function get_random_port to return a randomly
available port from the local port range.

Change-Id: Icaed180cc14602a74cdb3fd3456b690d8a4c729c
2017-05-28 10:04:53 -04:00
Sean Dague e123edeebf Add global_request_id to systemd logs
With cinder supporting this now, start logging global_request_id in
systemd logs. It will be None for all the services until the work
starts coming together, but it is safe to do.

Change-Id: Ic6ba1a42da88c03e43d89658b453f6a0b353e0db
2017-05-26 07:18:32 -04:00
Kirill Zaitsev c0644f39a4 Use correct argument order in truorfalse for USE_JOURNAL
truorfalse function from common-functions accepts default as the first
parameter. The arguments for USE_JOURNAL were mixed up and this commit
restores correct order.

Change-Id: Id3621b0e1910a625d6cfb8e81bd27bea82543ae9
2017-05-25 12:16:11 +00:00
Lucian Petrut 2715fd0b4a Do not upload vhdx images as vhd
This change ensures that when uploading vhdx images, we use the
proper format.

At the moment, vhdx images are uploaded as vhd, which can be
troublesome: first because this is misleading, second because the
actual image format may be checked, having the image rejected.

Change-Id: I9578be41ea9dc252404b7553679ac527e08a0ff6
2017-05-24 13:36:57 +03:00
Jenkins e19abda40a Merge "Restore log colors under devstack/systemd" 2017-05-17 20:10:30 +00:00
Eric Fried 8cd310d763 Restore log colors under devstack/systemd
One of the pending issues with the conversion to systemd was the loss of
log coloring.  It turns out that journalctl by default strips out
characters it considers "unprintable" - including the color codes
emitted by the old-style logging.  However, journalctl can be made to
print them by adding the `-a` flag.

This change makes devstack's log formatter conf settings include color
codes like the old screen-based setup used to

We also remove stackrc's setting of JOURNALCTL_F, whose usage was
removed via I6af6d1857effaf662a9d72bd394864934eacbe70.

Change-Id: I2401e267913a24d18dae355aa933072dbbdab1d8
2017-05-16 14:56:57 -05:00
Dan Smith 1f55d38911 Add systemd ulimit override function and set an override for NOFILE
This sets our default ulimit NOFILE to 2048, which is double what we
set things like mysql'd max_connections to.

Change-Id: I5126bed1e6b9f8c64db00eae4151ac61e47b1bf8
2017-05-16 09:23:28 -07:00
Angel Noam f24e29920b Add support for image name end with .raw
Change-Id: If3bd42825a29755fa8e68fa5661d068c6f0019d2
2017-05-11 15:13:29 +03:00
Sean Dague b2bfe5617a Turn off use_journal because of eventlet concerns
use_journal uses the systemd native path for logging, however there
are concerns that this might be negatively interacting with
eventlet. To be on the safe side fall back to stdout.

This introduces a USE_JOURNAL option which will let folks turn this
back on for testing.

This also adjusts the debug lines. When using the journal the pid
reported by systemd is correct. When using stdout, it will be the
parent process id, so we need to keep it to see which child each thing
is coming from.

Change-Id: Id7891c532bf99c099252e82d511a37a49506fea9
2017-05-03 18:35:54 +00:00
Sean Dague 5cd44dbe6c add use_journal if we are enabling systemd
This is going to be a new option in oslo.log, which we can start
setting early to make it take effect.

Change-Id: If0e5e4717a1810c759058f33608fbac7543f2d85
2017-04-05 07:24:13 -04:00