Currently, if the package was already installed, the script would return
2, or 6 or the package would be upgraded.
It causes the script to fail (we use set -e); so to avoid that we added
--replacepkgs --replacefiles and --oldpackage arguments to rpm.
- Install the packages even if some of them are already installed on
this system.
- Install the packages even if they replace files from other, already
installed, packages.
- Allow an upgrade to replace a newer package with an older one.
Change-Id: I8391e02456a304d4e29256cbbf5433879c6c42b1
When yum_cache is set, that directory will be automatically
picked as either the source or destination for the containers
being updated as the following:
* when that host directory is missing (or empty), the container
under update will start populating it, while it gets updated.
That path going to become the lower overlay FS layer for future
use among other containers under concurrent yum update executions.
* when the yum_cache directory exists and is not empty, it will be
bind-mounted as an upper overlay FS layer for other containers under
update. So those can benefit from some of the already prefetched
contents in its yum cache without data races or conflicts when
concurrently accessing the cached data.
Overlaying ensures data safety as each container can only see the lower
layer of the overlay, while storing its local changes on top of it as
an ephemeral. The yum_cache directory existance & non-emptiness facts
act as a single mutex, which only grants a dedicated writing access to
the lower layer to a single "populating" container at a time. This
behavior may be forcefully reset via the force_purge_yum_cache flag.
The container update playbook invoked with it, instantly creates a
new populator and creates a fresh yum cache.
Note that the 100% saturation of the cache is only expected, when the
populating container finishes its execution.
The feature can be used only for buildah in yum update scenarios using
yum or dnf.
Change-Id: I30c6dd12454a0b1781803ab16ef79b5914178114
Related-bug: #1844446
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
Yum update works if you want to update existing packages, but sometimes
you want to install a package that's available via the yum repositories
that is extra. This implements a yum install action similiar to the yum
update action except it takes package names instead of repos to install.
Change-Id: Ia47a1ea9eb51a37f6d75d39c524e97fd4ec94fba
Fixed issue identified at https://review.opendev.org/#/c/664872/1 which
prevented full execution of yum_update.sh on dnf platforms because
`dnf repoquery` returned exit code 1 when repo did not exist.
Change-Id: I86918f292b6863d6cb0e4fd16e62409a7f9fa80a
This will allow dev to update their container images from a local Python
directory, example in /home/joe/git/openstack/heat.
The new parameter python_dir is a list of directories.
To use it, your playbook must be like:
- hosts: localhost
connection: local
tasks:
- name: dev install heat-api
import_role:
name: tripleo-modify-image
vars:
tasks_from: dev_install.yml
source_image: docker.io/tripleomaster/centos-binary-heat-api:current-tripleo
modified_append_tag: -devel
python_dir:
- /home/joe/git/openstack/heat
Change-Id: I182c3fa58dc9af870e0da9f51ae1e22aa90d03e5
repoquery is part of dnf so we can use 'dnf repoquery' directly rather
than install dnf-utils, which isn't installed by default on e.g. RHEL8.
When using yum, command -v repoquery will still be used as it was
before this update.
Change-Id: Iea8822084988b23652e6d3f8b5828c05734158af
Following the revert due to issue with repoquery in
Id12743c0cf83d054189df10cad2c8abec57961e8, re-submitting the two
unrelated changes. Follow-up patch to come for handling repoquery more
sanely.
* Use the --quiet flag with repoquery
This prevents notices such as subscription-related stderr warnings
from being added to the output and messing with the sorted order.
* Remove the : after else
It seems using else without a colon is the bash standard. This gets
rid of the following error on RHEL8:
"/tmp/yum_update.sh: line 34: else:: command not found"
Change-Id: I8e52f239fcf962c17543ba56d3653de0016ed47f
Containers no longer updated in CentOS7 now,
as yum repoquery is not a command.
This reverts commit c475c8394a.
Change-Id: Id12743c0cf83d054189df10cad2c8abec57961e8
* Use <yum|dnf> repoquery instead of repoquery directly
"repoquery" is part of <yum|dnf>-utils which may not be installed, in
which case no packages would end up updated.
* Use the --quiet flag with repoquery
This prevents notices such as subscription-related stderr warnings
from being added to the output and messing with the sorted order.
* Remove the : after else
It seems using else without a colon is the bash standard. This gets
rid of the following error on RHEL8:
"/tmp/yum_update.sh: line 34: else:: command not found"
Depends-On: https://review.opendev.org/662317
Change-Id: Id03405d49dc213d018941c44129521cca845e7d6
The commands contain package lists, so having "set -x" makes the
logged result very very verbose.
Change-Id: I8ffda90eb0633b6b46e29c5af7ed2ff3571c14a2
Related-Bug: #1813684
Doing a yum update with an explicit list of packages where many of
those packages are already up-to-date has a significant time penalty.
This change includes version comparison to remove packages from
packages_for_update where the installed and available versions are
identical.
This change also only installs yum-plugin-priorities if it is not
already installed, for some extra time saving.
For a specific example of time saved, I had a
centos-binary-rsyslog-base with the following timings.
Without this change
-------------------
packages passed to yum update: 87
packages actually updated: 11
time taken 35s
With this change
-------------------
packages passed to yum update: 11
packages actually updated: 11
time taken 10s
This time saved should add up over all of the container images during
image prepare in CI.
Change-Id: Ie282acd1cdc0de8d875959ec9578122fe34f98ac
It makes yum and pip installs work with py3 packages. The package manager
dnf/yum and pip binary are determined at run time from the OS.
Use dnf-plugins-core for dnf with respect to
yum-plugin-priorities.
Use pipefail with set -ex to fix linter errors.
Change-Id: I997509204e30abb8b21ef936ea44440fbaa5a0e4
Closes-Bug: #1813546
Executes all linters via pre-commit, which is much faster, guarantees
their version locking and allows upgrading them with a single command.
Before this change the only linter running via pre-commit was
ansible-lint.
Now we also run bashate, flake8 and yamllint via pre-commit.
For developer convenience we still keep the old tox environments
which allow running a single linter.
Added long_description_content_type to fix twine check failure
Change-Id: I037eae61921b2a84aa99838804f70e96ee8d8b13
The following playbook will produce a modified image with Python source
code installed via pip. To minimize dependencies within the container
we generate the sdist locally and then copy it into the resulting
container image as an sdist tarball to run pip install locally.
Useful for local ad-hoc testing of upstream patches without
having to build and RPM in those cases.
Change-Id: I93c0e86b2e421b0bcb777d536fcb0d30e0ee3b68
Now that the undercloud is containerized, there will be very few host
packages to compare to, so there is a high risk that required package
updates will be skipped.
This is a strategy inherited from container-update.py that was
intended to avoid unnecessary calls to yum update, however we now have
a better approach using the repoquery, so host package comparison is
no longer required, and probably causing some of the instances of bug
Change-Id: Iab7b9d6377494001d904bb84b058ea293d73110c
Partial-Bug: #1786764
This change builds the packages_for_update list by intersecting
installed packages with the packages available, which avoids passing
uninstalled packages to the yum update call.
For some reason doing a yum update with packages which are not
installed causes a delay for each uninstalled package.
For my test case this gave a speed improvement of 35s vs 45s for the
whole image build. This would accumulate to decent time savings for
all images in CI.
Change-Id: I338d9881d5a05dd7b1fbc5efea1145349f0997a6
Fix error:
"/tmp/yum_update.sh: line 17: [: too many arguments"
when upgrading packages in container
Change-Id: I159ee948b541adf281263696447608f3b6286505
When installing packages the package which was built by DLRN should
be installed even it has lower version. Let yum priorities to
ensure that.
Change-Id: Id1bb77550596c517e684a80a73cf0bee6d26e958
Currently the logic for this results in a full yum update when no
package updates are found in the provided repository. This can lead to
job timeouts when nothing was built in CI because it effectively does a
yum update on every container and applies other system packages rather
than ones that actually changed.
There is a larger issue in that we can still get out of sync with the
host OS.
Change-Id: Iaf41691ea3cb6e78186741ac5e15614fb73f89ff
Related-Bug: #1786764
This patch adds a new rpm_install tasks which can be used
to build a new container layer from a set of local RPMs.
This is useful for building contains locally with no network
connectivity.
As part of the change the existing Yum update Dockerfile.j2 was
renamed to Dockerfile-yum.j2 to make it more specific.
Change-Id: I951c976ebb84c28cecefd2f5753a8232f46ad4aa
A good chunk of container images try to be as slim as possible and as
a consequence do not have repoquery binary which is used to limit the
packages to a given repo.
This commit makes the playbook more against these containers and
installs all available updates when repoquery is not installed, the
same as when we don't restrict the list of repositories to get updates
from.
Change-Id: I81e5cd0edae56c1535a2e4dcc639ec71055b10d2