We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I1efdf4f86f5a9bd02fa99b0c1fb430cf621bb7e3
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
This is a mechanically generated patch to complete step 1 of moving
the zuul job settings out of project-config and into each project
repository.
Because there will be a separate patch on each branch, the branch
specifiers for branch-specific jobs have been removed.
Because this patch is generated by a script, there may be some
cosmetic changes to the layout of the YAML file(s) as the contents are
normalized.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: Id99b631d05c93d300a4fe2a4af9eea84fa7a903b
Story: #2002586
Task: #24341
We're migrating to zuul v3 layout, so let's run the new job.
Depends-On: Iaa85a8326e746604966a9c1bb4cc85f44e92299b
Change-Id: I65a52db87df1928c683052fabe65a0fee52905c4
Create new functional tox environment so that we can use it instead of
func environment to allow creation of cross-project
openstack-tox-functional job.
Depends-On: Ibebf9d0eeccabaefa2777a43592a72d19d064b6b
Needed-By: Ifa4c20cff72820a0016bef8dcdd36647ae7359d3
Change-Id: Ied08d83177b9d52b6eb8e4987f444230ac4ebfdb
Sourcing of environment files is a bit different than running scripts,
because they are imported into the current environment. That means
that if they are fiddling with "set -x" you can get odd things in the
output -- for example, if one environment file enables tracing then
all proceeding sourcing will have it on, but all the files sourced
before will not have output. This is super confusing.
It's also very helpful to see how things are being expanded as they
set environment variables.
Thus I think the best idea is for environment files to leave tracing
alone and dib-run-parts can a) give a useful output about what file it
is sourcing and b) enable tracing around the import, then restore it
to the previous state afterwards.
Change-Id: I29f7df1514aeb988222d1094e8269eddb485c2a0
In bash the only sensible way of returning a string from functions
or scripts is via stdout.
dib-run-parts not only 'returned' the output of the called scripts but
also some statistics / performance output.
Other parts of the script already used an 'output' function so
the statistics output was adapted to also use this kind of function.
In addition three simple test cases are implemented to check
the basic functions of the script.
As a result the log-output of the dib-run-parts is somewhat cleaner.
It also makes debugging of the called scripts easier.
This change is the precondition for simplification of functions like
diskimage-builder's eval_run_d.
Change-Id: Ie2f0332359a72ca7ab03f94b7573e073df714a97
Signed-off-by: Andreas Florath <andreas@florath.net>
By sourcing the environment files directly into the running process,
dib-run-parts can pollute its own environment. For example, if an
environment script enables tracing (set -x) then that enables tracing
for dib-run-parts; particularly annoying as it then spews a bunch of
stuff relating to the profiling.
Move to sourcing the environment files in a subshell with the scripts
to be executed to keep things separate.
Change-Id: I1e39822f218dc0322e2490a770f3dc867a55802c
This allows to run dib-run-parts on systems where python is not
installed, such as Debian docker image. Generally, awk is more likely to
be available than python.
Change-Id: I26d7ad55a56ca4785a779f576e2da50a69400563
dib-run-parts in dib-utils had several fixes missing that had been
committed to diskimage-builder. When diskimage-builder switched to using
dib-run-parts from dib-utils, we lost these fixes. The switch over
happened in Ia0a0df7784a14c49b5c47ac0b03e6c2602c84b3b
These are the fixes from diskimage-builder that this commit adds back:
I44cf98dfc80cfcaec54b88cc83be80a3dbf2cec3
I24fd76c2b4f66b8036010b5079db39ead729abee
Iad2d490c86dceab148ea9ab08f457c49a5d5352e
Ieb48faacb4c96b7b358771d70c17f2f22d0354f4
Ia68853505485fefbf0890313456e7edb6097666b
Ibfb562c5970b40598fc95da1e8d4beb9d51d7612
Change-Id: Ice4b6a4b3d36a02099a682ae8cda4721211d51d9
We use semantic versioning, not pbr's pre-version facility. This
line causes local package builds to have the wrong version, and
pbr will in future detect this and error.
Change-Id: I3f4504ac2e9c49fd1bba964999e789d7a6e918bf
When showing the timings of the executed scripts, sort the scripts in
the same way they were executed. This eases the comparison between runs
of dib-run-parts.
Change-Id: I171499c30e1a992d7225be8a9dc00e0f03ec46b0
Pass --tmpdir to mktemp so the new temporary directory is placed
within $TMPDIR, or falling back to /tmp. This way dib-run-parts can
use whatever the temporary directory is, instead of using an hardcoded
/tmp.
Change-Id: I6b55e3c6c1fe22108ea555d3766e0323695bd96d
Force LC_ALL to C to allow printf to work independently
from the locale. Gives invalid number error if run in non-english
locale.
Port to dib-utils of: Ia68853505485fefbf0890313456e7edb6097666b
Change-Id: Ibbfa7a4a91dca72b16bb8fcb9602b71eccf54008
The current glob match for environment files can source editor backup
files (foo.bash~) which will override the real changes you have made.
Other parts use the regex to avoid matching such files, so do
the same for environment file matching. Note this has to match "."
unlike the other regex, as most env files are "foo.bash"
Port to dib-utils of: I934486b3ff5884063d29c6d9b66fd9b11140464c
Change-Id: I11c030e8556d89c240e36233b249f0c1d9f80908
This is required to be able to build and release the project using
the standard infra tools. Since there currently aren't any unit
test or doc jobs defined for this project, only a very basic
tox.ini is being added. It can be filled out later as more jobs
are needed.
Change-Id: I121405d4ec82a4d8252098319fce0a9f5b5bfb4f
This change adjusts the "inline" python function used to profile
run-parts duration to make it run successfully on both py2 and py3
Change-Id: If1e6ac6fe528ca0225dcdbe5774545512be9d13a
Fixes problems found by set -eu and pipefail, including:
-Many unset variables
-Commands that can fail under normal circumstances, which breaks
with set -e. This change swallows those expected errors to allow
our existing error code to handle them.
-The dkms element was not finding Fedora kernel versions correctly.
This may be an issue for other distros too, but since Fedora was
working fine without this functionality I only changed it to print
a warning message rather than failing the build when it happens.
-The ramdisk init script will not be set -eu because if it fails
the result is a kernel panic, which can be tricky to debug.
However, in testing with set -e a few failing commands were found
and have been fixed in this patch.
Change-Id: I44cf98dfc80cfcaec54b88cc83be80a3dbf2cec3
Don't assume environment.d is always under /tmp/in_target.d. This makes
dib-run-parts more flexible, and less specific to diskimage-builder. For
instance, dib-run-parts is already used by os-refresh-config, but
/tmp/in_target.d/environment.d is never going to exist when it's called
by os-refresh-config.
This is useful b/c elements can now install os-refresh-config scripts
under /opt/stack/os-config-refresh/environment.d to influence how
further scripts are run. Specifically, the OpenSuse packages prepend the
OpenStack user accounts with "openstack-". Instead of checking for which
users exists with an if/else in all the os-refresh-config scripts that
need to do user related things, we can just update the scripts to use
$NOVA_USER, etc. The OpenSuse element can then install a file under
/opt/stack/os-config-refresh/environment.d to set NOVA_USER to
openstack-nova. We could have files that declare defaults as well within
each specific element, or just set a default variable at the top of each
os-refresh-config script.
Change-Id: Iadbfad995da657e2965fd55fc4ba3a88138b4cfc
Rather than using a script to mount the image using nbd to extract the
kernel and ramdisk, make a new element called baremetal, which contains
a cleanup.d script that will copy them out to <image name>.{vmlinuz,initrd}.
Closes-Bug: 1224669
Change-Id: I8f3569aa12148d18b1c8242b6fbbd8857894b26f
dib-run-parts should dereference symlinks, and if they point to an
executable file, execute that file as a script. This can be accomplished
by using the xtype predicate in the find command instead of the type
predicate.
This change is needed so that we can dynamically symlink hook scripts
into hook directories such as install.d at runtime to support different
install types.
Change-Id: I933e7f4b4dcf16956841d8c14aa63b0f9a18fc5d
To ignore swap files from getting into repository
currently the implemented ignore is *.swp
however vim adds more swap files if these files exists,
so improving this with *.sw?
Change-Id: Ic4a1a85dfdf21b7bd0abdf16a205446febab2fcd
Closes-Bug: #1255887
When LC_NUMERIC is set to a format that doesn't use a decimal point,
`printf` will fail.
Change-Id: Ie6c4d075928f47b17cc413d537fc31c9d0734bdb
Signed-off-by: Tomas Sedovic <tsedovic@redhat.com>
Our custom dib-run-parts now supports --list for better compatibility
with some OS-provided run-parts tools.
Change-Id: I2feb615b2094d9d6522bfe9b422362223bb8e652
Closes-Bug: #1190521
- Ensures /sbin and friends are in $PATH when invoked (without this,
various sudo invocations fail in exciting ways).
- Use dib-run-parts in lib/common-functions instead of run-parts
(neither SLES nor openSUSE ship run-parts).
- Ensure dib-run-parts doesn't descend into subdirectories (same
behaviour as run-parts).
- Move dib-run-parts from root.d to bin (cleaner, consistent with
other elements with separate bin scripts).
- Tested by building Ubuntu image on openSUSE 12.3.
- Note: this doesn't add support for creating SUSE images, it just
lets you run disk-image-create on SUSE-based distros.
Change-Id: I906c6bc3cf51cdf2c4415adeae1ca250faac25e1
Ramdisks are now built inside a chroot which is built by the normal
image build process. Doing so improves our independence of the
precise state of the build host.
This fixes bug 1194055.
Change-Id: Ibc254fbb9e7b404b5f38c1b35bcde8a4136e8e28
Qemu-nbd does not perform well with older versions of qemu due to
the lack of writeback caching mode. It also only builds qcow2 images
and there is a desire for raw image support. Finally, qemu-nbd makes
it very difficult to build images concurrently due to the somewhat
opaque nature of how it selects a /dev/nbd# device. losetup, on
the other hand, makes this process very straight forward.
Change-Id: I309fad8af4fd1e8d1720c17b65e1897a76d5e897
Co-Author: Clint Byrum <clint@fewbar.com>
It is valid to request os-refresh-config without putting any scripts in
the element root. This fixes failing builds of heat-cfntools.
Change-Id: I4f70b33e453399ff5baa48919557995a6eb27520
This is a necessary but not complete step towards supporting Fedora and Suse
distributions. Further work is needed (e.g. to quiesce daemons on
installation).
Change-Id: If3ea6093d41a21de755db52328226b84b5a3ede6
Per http://wiki.openstack.org/ProjectTestingInterface we enable all
of the commands except build_sphinx because we do not have any sphinx
documentation as of yet. Includes babel support though there are no
properly internationalized strings just yet.
Change-Id: Iae6e6b3f9e605106f4575196fa5527d2187255df
This adds a new optional file to the root of elements. The
file lists dependencies which will be added to the list
requested by the user during disk image creation.
Change-Id: Id71c3b333563604bbbaf90f9cf40e24fa9738fc8