On recent, enforcing systems, the update_yum.sh can't be executed,
because SELinux prevents container_t to open user_tmp_t:
type=AVC msg=audit(1674049913.380:22858): avc: denied { open } for
pid=70472 comm="bash" path="/tmp/yum_update.sh" dev="vda4" ino=218200014
scontext=system_u:system_r:container_t:s0:c65,c705
tcontext=unconfined_u:object_r:user_tmp_t:s0
tclass=file permissive=0
This patch ensures it gets properly relabelled when bind-mounted during
the image build. Using the "z" will also ensure it's still usable even
when running multiple builds at the same time.
Change-Id: I4085865965f48c9fa6a88cde7010a51cd8c653d8
With newer podman and container-selinux, we may face some issues
depending on where directories are located. The way we're running the
role is also important - being launched as root or as a user makes a big
difference, especially with the user's home content.
It also updates the setup.cfg content to match new format requirements
Change-Id: Ib2b7fd2f557d4743efd2eaca18474fb45b91cbcf
Tripleo-ci jobs are broken after latest release of setuptools 61.0
because of breaking changes which are not backwork compatible,
details in related bug and [1].
[1] https://github.com/pypa/setuptools/issues/3197
Related-Bug: #1966382
Change-Id: Id60e8bc882f6c7944f89333fb539f6800ef43725
Because there is no docker supported anymore, remove all custom
config of "container_build_tool", only "buildah" is supported.
Change-Id: I57720cc1d7da96f9277b62298cabce9ff7765f47
When a gating, component or delorean current repo
is present, containers should be updated with
the latest rpms.
This review checks the installed rpms and
greps for the repos where the updated rpms
are sourced.
Change-Id: Ie29c7c33c8d66bc3729c03c2d72cbdbf85ad443a
/etc/{{ pkg_mgr_suffix }}/vars exists on stream and
some other platforms but is missing from RHEL 8.x.
This patch checks that that directory exists first
before mounting it.
Change-Id: I048434b38eb1d6b1c83a89d90e01f71d467d7fb7
With [1] it used yum/vars but in CentOS8-Stream
that is not available, so detect and use dnf or yum
vars.
Closes-Bug: #1927302
Change-Id: Idda53d1b68b97e5bb65314b1e07d507736932531
Since repos may rely on yum vars, it's required
to mount yum/vars along with yum repos.
Container builds already mount yum/vars along
with yum yum. Molecule jobs also adding it with [1].
Since the repo is branchless and used in CentOS7 too,
so using yum/vars instead of dnf/vars.
[1] https://review.opendev.org/c/openstack/tripleo-ansible/+/787423
Change-Id: I36f175d97a86d4221b09dadf62f64a16b5c527e2
This change migrate c8 jobs/templates to content provider
jos so consumer jobs can use resources built by provider
jobs.
Change-Id: I17a40cbf6fb978bec17ab21fb2296ad3db46b6b3
Signed-off-by: Amol Kahat <amolkahat@gmail.com>
All the tripleo-ci jobs running on greater than train branch should run
on Centos8 only. Porting of this job was skipped earlier.
This change ports train jobs from centos7 to centos8.
Change-Id: I1eb2954ffcc582117b27fa3cf4d3fac4789d01b0
Signed-off-by: Bhagyashri Shewale <bshewale@redhat.com>
Currently, if the package was already installed, the script would return
2, or 6 or the package would be upgraded.
It causes the script to fail (we use set -e); so to avoid that we added
--replacepkgs --replacefiles and --oldpackage arguments to rpm.
- Install the packages even if some of them are already installed on
this system.
- Install the packages even if they replace files from other, already
installed, packages.
- Allow an upgrade to replace a newer package with an older one.
Change-Id: I8391e02456a304d4e29256cbbf5433879c6c42b1
We are seeing occasional failures to pull due to intermittent registry
errors downstream. A retry should help to avoid the whole update
failing because of this.
Change-Id: Ib9415e46a52cc6ad6459ec3f170d7e23aa9aca03
This change updates the copy task to assume the source is remote. This
will allow the module to use any path provided and not assume the file
is within the relative path from the role.
The become jinja used in the task "Modify image from {{ modify_dir_path }}"
was attempting to match on the full command executed, which would always
evaluate to `false`. This change corrects the condition so that it can
evaluate to true, when "buildah" is used.
Change-Id: I348dbf91b5f2e0dcdfeffd336782c24499d7a569
Signed-off-by: Kevin Carter <kecarter@redhat.com>
This is consistent with the other printf statements in the file. This
keeps the sorted data with the newlines as they are, and avoids
breaking down the data again. Otherwise, the sorting order can be
altered which breaks comm comparison and causes the "Run yum_update.sh"
task to fail with the following error:
"stderr_lines": [
"comm: file 2 is not in sorted order",
"error while running runtime: exit status 1"
],
This is particularly important for the $available variable which
contains the repoquery output, often very large and with unusual
characters. The error happens consistently when including RHEL 7 server
repos or RHEL 8 BaseOS repo and prevents the update from working at
all.
This patch also adds -u when sorting the $available output. Once the
versions are removed with cut, there can be a tremendous amount of
duplication in the data obtained from repoquery. That makes verbose
mode even more difficult to navigate when debugging.
Change-Id: Id64a8c2e395de28945d5d6b4370c44b80392e543
If the yum cache path exists and already mounted by someone,
do not attempt writing to it, use the overlay mode instead.
This still leaves a window of opportunity for another workers to
RW mount the cache after the ansible check has reported a stale
fact about there was no other mounts found. But this is unlikely
to happen.
Also, if it has to be retried in the rescue block, do not use the yum
cache for the maximum data safety and clean (a scratch) cache state
reasons.
This drastically reduces chances to have multiple writers for the
cache.
Closes-bug: #1860804
Change-Id: I19491a162e5bf6d6517fd343d675aff12bdc9719
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
The undercloud registry currently doesn't handle OCI formated images
correctly. We need to ensure that when buildah is run, we specify that
we want the docker format until we correctly support the OCI metadata.
Change-Id: Icf1a1c8f3a353239f2d244aa0bc811f8f86f6867
Related-Bug: #1860585
After implementing "yum localinstall", a critical part of the
process wasn't merged in the code which consists of copying
the actual RPMs through the Dockerfile-yum.j2 file.
Change-Id: Ifbcaa07bb80dd612e85f0d7cf8d99131fb739c84
Closes-bug: #1860184
This is needed if we want to only update installed packages and not
hit depenency issues encountered when updating packages with rpm_install.sh
Change-Id: I5095d7b04cb10fde1bd82afd1bc406445b7595fd
Closes-bug: #1858837
When updating RHEL7 containers from a RHEL8-based environment, the
update fails with the following messages:
"stderr": "/tmp/yum_update.sh: line 18: /bin/dnf: No such file or directory",
"stdout": "No packages were found for update...",
This patch reverts the PKG variable assignment to the way it used to be
so that the decision to use dnf or yum is made at runtime, within the
container.
Change-Id: I6a248b5277efd8707fc744292075949d94e3a05f
In order to make sure RHUI repos works with in a container,
PKI certs dirs needs to be mounted to container so that
RHUI repo solves and download the content.
Related-Bug: #1854685
Change-Id: Id09059559b5c207ef6f604e4bb999528118ae096
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
The dev_install.yaml script breaks when running python 3 because of
unsupported comparisions between 'AnsibleUnsafeText' and 'int' with the
'>' operator.
This is very similar to a bug opened against ansible upstream [0]. We
can fix the issue by using length explicitly, which ensures we're
comparing integers to integers since we're dealing with a list.
[0] https://github.com/ansible/ansible/issues/50388
Change-Id: I100639982b75c9d345269ba9bf10defb7e96d774
- avoids ansible-lint installation bug
- bumps linters to their current versions
- adds missing passenv which prevented run with SSL proxies
- removes tox environments to ease maintenance, user can use posargs
to run a single linter (hook).
Bug: https://bugs.launchpad.net/tripleo/+bug/1848512
Change-Id: I7657bb829a2928a8310c1758e6934b2f2ddef5c9
We want to try a best effort to remove the buildah containers but since
it's run with multiple processes we occassionally get layer conflicts.
Let's add a bit of a retry and ultimately skip the error since that was
the previous behavior.
Change-Id: I75a85745aed652a85f4c143c987cd5cccbf31cac
Related-Bug: #1846413
https://review.opendev.org/#/c/685466/ switched
to yum_update script to template but missed
adding tasks for detecting package manager.
Change-Id: I1acbbe418cba9e9f363ce4fdf4314985b52a2dd6
We currently map in the cache dir, but it only saves the metadata. The
improvements would come with the rpm saving as well. In order to have
that, we need to set keepcache=1 which is disabled by default. We can
enable this on the command line via --setopt so we don't have to touch
the container yum configuration.
Change-Id: Ia274dbbad9a220e52b2cfa554dede3bf40055f98
Currently in the yum update output, we see buildah trying to rmi the
image we were working with however it is currently in use. After we
commit our changes we need to cleanup our working container so we can
remove the image we were using (if not used by another process
elsewhere).
Change-Id: I54e37b43346b97be0a7cfab12e6cac9809537c83
Followup of [1] which switched to template for
yum_update.sh but missed updating it for
yum_update_docker.yml.
[1] https://review.opendev.org/#/c/683100/
Change-Id: Ibf5c9f0e07ff141a6c879fcda71be8119ae29b38
When yum_cache is set, that directory will be automatically
picked as either the source or destination for the containers
being updated as the following:
* when that host directory is missing (or empty), the container
under update will start populating it, while it gets updated.
That path going to become the lower overlay FS layer for future
use among other containers under concurrent yum update executions.
* when the yum_cache directory exists and is not empty, it will be
bind-mounted as an upper overlay FS layer for other containers under
update. So those can benefit from some of the already prefetched
contents in its yum cache without data races or conflicts when
concurrently accessing the cached data.
Overlaying ensures data safety as each container can only see the lower
layer of the overlay, while storing its local changes on top of it as
an ephemeral. The yum_cache directory existance & non-emptiness facts
act as a single mutex, which only grants a dedicated writing access to
the lower layer to a single "populating" container at a time. This
behavior may be forcefully reset via the force_purge_yum_cache flag.
The container update playbook invoked with it, instantly creates a
new populator and creates a fresh yum cache.
Note that the 100% saturation of the cache is only expected, when the
populating container finishes its execution.
The feature can be used only for buildah in yum update scenarios using
yum or dnf.
Change-Id: I30c6dd12454a0b1781803ab16ef79b5914178114
Related-bug: #1844446
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
The |exists is currently deprecated.
[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using
`result|exists` use `result is exists`. This feature will be removed in version
2.9. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
Change-Id: I0c32686062e79142aa5c664a4a42ac590263b64e
Drop compare_host_packages parameter and description from examples
This was dropped some time ago in
c9d085729f
Also highlight that only file repositories will be used in yum update
task
Change-Id: If9942c75ea786304ffe3aa6e28939708f8597ef3