Commit Graph

76 Commits

Author SHA1 Message Date
James E. Blair 509880073e Remove command.warn usage
This is no longer present in Ansible 9.

Removing these upsets ansible-lint, so those errors are ignored.

The base roles job has bitrotted on centos-7 and bionic due to
a bad voluptuous release used in an stestr test.  That is fixed in
this change as well.

Change-Id: I67886d5ad82ab590979f82bd102d6f974b9d4421
2024-02-14 15:17:52 -08:00
Ian Wienand 6cac22ddab
configure-mirrors: fix typo in 9-stream enablement list
The debug repo for NFV wasn't configured to mirrors, this was a typo
in Ibfd88c137855c22b960099cc8d582b241c3b1db6.  Switch it back to
false.

Change-Id: If18d5fda42c68f4520a5c553b63c7b2d351ebd4f
2022-09-19 16:16:34 +10:00
Ian Wienand 500eba3634
configure-mirrors: make each compontent in 9-stream configurable
When we added 9-stream with I2da3a5e8a45598c6b3ded132ea61b86b4480e262
we had some components pointing to the mirrors and some not.  This is
basically because the OpenDev mirrors choose to mirror these parts of
the distribution.  It was an oversight to do things like this, as this
is nominally a generic configuration tool.

This proposes that we put each component behind a variable which can
be enabled or disabled.  This way users of the role can switch
on-or-off whatever their local mirrors carry, and fall-back to the
generic mirror metalink in other cases.

I imagine that with future distros like this we would have everything
disabled by default in zuul-jobs, and leave it up to sites to
explicitly opt-in the various components.  We could announce this via
our usual deprecation path and do similar for 9-stream; but for now
this keeps the defaults as it is currently written.

Change-Id: Ibfd88c137855c22b960099cc8d582b241c3b1db6
2022-09-16 10:32:29 +10:00
Zuul c3a0026fa7 Merge "Update gpg key file for extras-common in CS9" 2022-09-14 23:10:21 +00:00
Ian Wienand f76cfbab11 linters: fix spaces between filters
This fixes a number of places where we do not have spaces between
filters.  I think that this is a reasonable rule for readability (I
also think it probably was enforced, but maybe later versions got
better at detecting it?).

These are detected by a later version of Ansible lint; this change
should have no operational change to any roles but prepares us to
update in a follow-on change.

Change-Id: I07e1a109b87adce86f483d14d7e02fcecb8313d5
2022-07-27 17:13:39 +10:00
Clark Boylan baa7586095 Fix the disabling of deb src repos in configure-mirrors
The wrong flag was toggled to False. This revers the other flag back to
True and disables enable_deb_src_repos by setting it to false.

This was my mistake and also missed in review :(

Change-Id: Iffef2da4ae1ad4910be6d124e0cb0d019460fbb5
2022-05-13 10:49:52 -07:00
Clark Boylan 88cd70f90c Switch enable_src_repos to False in configure-mirrors
This should only be landed after we have announced this changing default
and enough time has passed for people to reasonably update their CI
systems if necessary.

Change-Id: Ibf2e2ca0579e180db074b7ab342dae15108dad27
2022-04-27 09:14:11 -07:00
Clark Boylan fc133bd165 Enable deb-src on Debian with a flag in configure-mirrors
This enables deb-src repos on Debian hosts with a flag. The flag
currently defaults to True to maintain backward compatible behavior with
the hardcoded deb-src repos that configure-mirrors previously set. We
intend on flipping this default to False after announcing this change.

The reasons for this are that source package repos are rarely needed
by CI systems, but these repos can consume large amounts of disk in
mirrors. To make it easier to people deploying mirrors we want to avoid
enabling source repos by default.

Change-Id: I7e9cd0ec1e3184c3c0561bbb7d3069feaf5f1ba5
2022-04-27 09:11:08 -07:00
Alfredo Moralejo 18021b8f0f Update gpg key file for extras-common in CS9
The actual signature for extras-common is the -sha512 file [1][2].

[1] https://gitlab.com/redhat/centos-stream/rpms/centos-release/-/blob/c9s/RPM-GPG-KEY-CentOS-SIG-Extras-SHA512
[2] https://gitlab.com/redhat/centos-stream/rpms/centos-release/-/blob/c9s/centos-addons.repo#L116

Change-Id: I6a8c3d737500696dafb66315aef38072a2d20d8d
2022-04-19 10:19:36 +02:00
Ian Wienand c20a2435a6 configure-mirrors: fix stripped newline
I74b9de7092f182c942a58ac7a46b9fbd791889de hit a common ansible gotcha
where it likes to strip the trailing newline after a {% endif %}.
This has resulted in invalid lines in our sources.list.

Unfortunately we miss this because it still exits with 0.  Add a
simple test looking for warning output.

Change-Id: I46d393a5e67d10a52c4dcca803176ff368a4b5bd
2022-03-01 11:10:30 +11:00
Jonathan Rosser 5d01b68574 Allow some configure-mirrors repositories to be disabled
Out-of-the-box installs of centos-8 do not enable the PowerTools
or HighAvailability repositories. Debian/Ubuntu do not enable
the backports repository by default.

Having these repos enabled by default in CI has led to merging
broken code for OpenStack deployment tooling which is attempting
to also manage the presence/absence of these repositories. It
is challenging to remove these repositories on the running node
because the repo URL (rather than just the name) is required as
input to the apt_repository and yum_repository ansible modules.

This patch adds a role default variable to configure-mirrors to
allow a job to opt out of these extra repositories. The default
is set to 'True' to allow existing jobs to work as before.

Change-Id: I74b9de7092f182c942a58ac7a46b9fbd791889de
2022-02-16 20:39:31 +00:00
Alfredo Moralejo 25f110c399 Add CentOS Stream 9 to configure-mirrors role
So that we use AFS mirrors for CS9 nodes.

Closes-Bug: #1959181
Change-Id: I2da3a5e8a45598c6b3ded132ea61b86b4480e262
2022-01-27 15:05:46 +01:00
Jeremy Stanley 750be2e2de Add new Debian security mirror suite pattern
Starting with Debian 11 (bullseye), security packages are in
bullseye-security as opposed to older releases like buster/updates.
List the last several stable releases in hopes nobody is trying to
use this role to configure platforms older than Debian 8 (jessie,
the current "oldoldstable").

A followup change demonstrates this works in the test-base-roles
job, but because the job matrices have to be updated in one fell
swoop, and many of those jobs won't work without this change already
merged (due to protected use in our base job), it's not tested
directly within this change.

Change-Id: I2d7712cbfd037a65b9025980a6c0cccd917f8947
2021-05-03 18:39:34 +00:00
Zuul d054d7ecc8 Merge "Fix CentOS wheel mirror URL" 2021-02-15 23:47:11 +00:00
Javier Pena cfada06486 Fix repo files for CentOS Stream
Some repo file names and ids do not match the files provided by
CentOS, which can cause duplicated repos [1].

[1] - https://8a51f3027f5f50a14ba7-fcfc85786424b7f5a3fcadff9da35f94.ssl.cf5.rackcdn.com/770771/4/check/packstack-centos8s-integration-scenario002/9dc0bd2/logs/etc/yum.repos.d/index.html

Change-Id: Ib89ad9f79cd5317b3178301a47462277004fa81a
2021-01-22 17:46:26 +01:00
Alfredo Moralejo 8beac88116 Rename config repos file config for CentOS Stream
Currently, config file is CentOS-8-stream.yaml but the value of
ansible_distribution_major_version fact in CentOS stream is 8, not
8-stream [1], so jobs running in centos-8-stream images are actually
using centos-8 repositories.

This patch is modyfing the list of config files to use ansible_lsb.id
fact wich is CentOSStream and renaming the file name.

[1] https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_792/770771/2/check/packstack-centos8s-integration-scenario002/792342c/zuul-info/host-info.centos8s.yaml

Change-Id: I7199acdd45151d2553ef17588909615b5a7c5b61
2021-01-21 09:00:36 +00:00
Gorka Eguileor 4284afc584 Fix CentOS wheel mirror URL
For CentOS we are currently using default values for "wheel_mirror" from
roles/configure-mirrors/defaults/main.yaml, which means that the URL
uses the full distribution version, which is wrong, we should be using
the major version instead.

As an example, centos-8 jobs are trying to use
  https://mirror.bhs1.ovh.opendev.org/wheel/centos-8.3-x86_64
When they should be using
  https://mirror.bhs1.ovh.opendev.org/wheel/centos-8-x86_64

Which makes some jobs fail to find alembic wheel files and then they
fail to build from source because the CI was expecting the wheel mirror
to have it.

This patch fixes this by creating a Centos specific "wheel_mirror"
variable.

Change-Id: I11199cf943a3070fd62bf486fe2b06381db4b04d
(cherry picked from commit c9cad38eeb)
2021-01-19 14:13:04 +01:00
yatinkarel 5cfc37b300 Rename CentOS8 repo files to CentOS-Linux
With CentOS8.3 release repo files are now
renamed to CentOS-Linux*, update in zuul-jobs
too to sync, also update repo names as per
latest CentOS8 minor release.

The change is needed now as dib images are updated
to reflect new repo files post [1].

[1] https://review.opendev.org/c/openstack/diskimage-builder/+/765963

Change-Id: I7ed9bd582043717ae5bae303ea9b32db7f73008e
2020-12-18 14:23:08 +05:30
Sorin Sbarnea 086d1a9284 More E208 fixes
Change-Id: Iddda539a24e03ae33ef5e37630f98a7268f18dbe
2020-11-03 10:04:27 +00:00
Carlos Goncalves 3f743e00fd Add CentOS 8 Stream testing
Add CentOS 8 Stream nodes to the testing regime.

Add repositories for CentOS 8 Stream to configure-mirrors

Depends-On: https://review.opendev.org/#/c/734788/
Change-Id: Ia2f2b22461b4f4eca19d294ae06f6e1c90ea8599
2020-10-07 07:11:35 +11:00
Clark Boylan d49893f894 Partial address ansible-lint E208
- replaces ignore with a warn, which displays issue without affecting
  the linting outcome (allowing gradual fixing)
- bumps linter to enable the warn_list feature
- fixes a set of issues, others will be fixed in follow-up

Change-Id: I7d6f8c156b06f68f681943e88860930968e7c9f9
2020-09-29 10:29:01 +01:00
Clark Boylan 43aa849174 Drop suse mirror types in our zypper repo configs
zypper wants to autodetect these values and it appears that upstream
changes repo types because we are having errors now. Dropping this
config manually on a held node seems to fix things.

Change-Id: I8ad28da7164d9a0955f43d4864ba24c14f0bd4a3
2020-08-06 13:59:27 -07:00
Zuul 0a02594e6f Merge "configure-mirrors: update include to include_tasks" 2020-05-26 15:57:56 +00:00
Albin Vass d01b4e1339 configure-mirrors: update include to include_tasks
Change-Id: I77ee216150e497b0b45f34c8778b45d89f63b20e
2020-05-26 10:12:12 +02:00
Clark Boylan 9471b8c42b Add option to prefer https/ssl in configure-mirrors
We should offer the option of https in addition to http in our
configure-mirrors role as users may want to consume mirrors using https.
This has become more viable in recent years with the releases of Debian
Buster and Ubuntu Bionic supporting it out of the box.

Change-Id: I747c1a379dfce9469e643d7fa199c8e8554f5289
2020-05-20 15:16:34 -07:00
Clark Boylan 6a2aaf7c80 Remove failovermethod from fedora dnf repo configs
The failovermethod option was removed in dnf and produces eye catching
warnings. Clean this up to clean up job output by configure-mirrors.

More details at: https://bugzilla.redhat.com/show_bug.cgi?id=1653831

Change-Id: Id646835e0fa9e5e99cde57382b3148caeb55e8bd
2020-05-20 15:16:34 -07:00
Andreas Jaeger 7e04128a8b Capitalize task names
We capitalize the first word in task names in general - and when you
follow a log and suddenly see a single lower-case one flying along...

Fix those names that I could find and that are not variables etc.

Change-Id: I9ccde577413270d49fd790c41767b859366dc2f6
2020-05-16 17:46:17 +02:00
Sorin Sbarnea 9baebe3684 yamlint: EOF newlines and comments indent
Fixed two rules which where temporary disabled during introduction
of the linter.

Change-Id: Icd1e1b40b1e8207ab5ff7088a48e8f0a800e3aa8
2020-05-14 08:42:17 +00:00
Sorin Sbarnea 4af438d136 Made sequence indent consistent
Change-Id: I5ea4232ca4fd6e03d5b5f72eb6704bee84d04ea5
2020-05-07 14:13:55 +01:00
Albin Vass bee0c6ae2f ansible-lint: use matchplay instead of matchtask
For some reason matchtask doesn't match includes, matchplay does so use
that instead.

Change-Id: I040f7f3394503e92d06c05e8ff671a43b14baebc
2020-05-05 20:42:38 +02:00
Albin Vass 9062289151 Check for loop_control in with_ type loops
Change-Id: I191265df7709a6262b44a428d78fe28ffaeb4b75
2020-05-01 13:45:34 +02:00
Alfredo Moralejo 5074bca828 Add CentOS8 High Availability repository
High Availability packages have been added as part of core CentOS
repositories in 8.1.

This patch add this repo and enables it as it provides corosync and
pacemaker based HA solution and some other packages required for it.

Change-Id: Idbddd81f251c1ade97892128e52f9214420bead7
2020-02-05 15:47:06 +01:00
Zuul e21bed0c48 Merge "Add pypi_fqdn to differentiate it package mirrors" 2020-01-06 16:36:58 +00:00
Clark Boylan b62c488eab Fix ansible use of filters and tests
Ansible 2.9 requires the use of "version" as a filter and
"version_compare" has been removed. Also tests cannot be used as
filters which means we cannot do when: foo | test and must do
when: foo is test instead. Make these fixes.

Both changes should be backward compatible to ansible 2.5:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_tests.html#version-comparison
https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.5.html#jinja-tests-used-as-filters

Change-Id: Id95f674a485877db2a7924994366d1c6c591a684
2019-12-17 10:49:57 -08:00
Bogdan Dobrelya 77bc616dc5 Add pypi_fqdn to differentiate it package mirrors
There is mirror_fqdn that, if defined, overrides mirrors used for
packages and also mirrors for pypi. This is generally incorrect,
if one wants to use different mirrors for packages vs pypi eggs.

Add pypi_fqdn that defaults to mirror_fqdn, which allows users
to go with a custom pypi_mirror. Make pypi trusting the given
pypi_fqdn value instead of generally unrelated mirror_fqdn.

Change-Id: I12975b57951699351cfc0d40beaeb7c703651dd0
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
2019-11-28 09:00:07 +01:00
Monty Taylor e79ca5ec3a Stop installing pydistutils.cfg
We install pydistutils.cfg to catch legacy cases where people
are doing things with easy_install. That code path is not
desirable and increasingly broken, as support for things like
allow-hosts is dropped upstream.

Stop installing the file. This will mean people using easy_install
won't get the benefit of the mirrors, but they shouldn't be
using easy_install anyway.

Remove pydistutils.cfg if it's there

Make sure we don't have any pydistutils.cfg around.

Change-Id: I24a05f456f87bd4cb57ebf89d4245477bf675f97
2019-11-24 12:03:39 -05:00
Sorin Sbarnea d3a52f108d configure-mirrors: Consolidate dnf/yum handler
Simplify logic by using a single handler for redhat systems.

Change-Id: I328f0abb602f0d71372812ff64d8ad3f3764fbd0
2019-10-15 18:21:44 +01:00
Ian Wienand 2e684a8da5 configure-mirrors: add CentOS 8
Add repositories for CentOS 8, add base test.

Co-Authored-By: Sorin Sbarnea <ssbarnea@redhat.com>
Change-Id: I24e194a2f4729046c8f521ffccd3b00055127516
2019-10-15 17:33:48 +11:00
Ian Wienand 2462d16322 configure-mirrors: make separate template directories for each platform
This keeps each platform's "/etc" template files in a separate
top-level directory.

Additionally, we add a distro major version match to the task import
and rename centos7 (in preparation for centos8).

Change-Id: If65d51a27e30311b1da20522afb6dbce7ee6cf35
2019-10-15 17:04:57 +11:00
Matthew Thode 4cfdae8188
notify needs a command
no action detected in task

Change-Id: I9bcb302b706c41ea5d2e792dc36108c924431c44
2018-09-20 22:02:55 -05:00
Matthew Thode 06e8c0fac1
update Gentoo cache
Change-Id: Icf0e8551236ae690b7997217723622031b7648b7
2018-09-20 02:24:22 -05:00
Xinliang Liu e330de0e09 Use debian-security mirror
debian-security has been mirrored by change[1], so we can use
it now.

[1] https://review.openstack.org/#/c/577316/

Change-Id: I2c88306564b04db3c6e061ab52545c5a63665e53
2018-06-27 11:03:34 +08:00
Paul Belanger 49711c6540
Update repo file for fedora-28
With the release of fedora-28, the default locations for
fedora-update.repo have changed. We still need to support fedora-27
until fedora-29 is released.

Change-Id: I8eacd659cf18a8dc571aea6531483610fcd91d1a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-05-02 12:28:36 -04:00
Paul Belanger df364a46e0
Switch to http://security.debian.org/ for debian
Today the mirror system in openstack-infra isn't properly setup to
mirror debian-security. Until we can fix this, switch to the upstream
URL.

Change-Id: Icfa7dc874d14de245927d461595f90b9dacc7825
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-04-18 13:38:23 -04:00
Ian Wienand 44bc52a539 Add arm64 support to mirror setup
Load the variables from an architecture specific file first --
ansible's ansible_architecture fact for arm64 is aarch64 (it's all
confusing, Ubuntu calls it arm64).  Add this file that sets up the
repo as "ubuntu-ports", which is our mirror of ports.ubuntu.com

Change-Id: Ieb50ae29c7f822e831ef1e32fdd03f74a93e298a
2018-03-07 13:49:26 +11:00
Zuul 979d6a74ac Merge "Allow updates from untrusted mirrors" 2018-03-05 16:02:04 +00:00
Jens Harbott 9861bc1640 Allow updates from untrusted mirrors
The default behaviour of apt has been changed, see [0]. When using
unsigned mirrors, we need to set the "trusted" option explicitly in
order to allow updates from these.

[0] https://manpages.debian.org/stretch/apt/apt-secure.8.en.html

Depends-On: https://review.openstack.org/#/c/536615/
Change-Id: I1a6818b022cd34c8899179c36cae962a6c8ec5ed
2018-03-03 06:49:27 +00:00
Dirk Mueller 294e9cbe1d Set wheels-mirror for openSUSE Tumbleweed
Unfortunately ansible_distribution is "openSUSE Tumbleweed" on
tumbleweed, and the whitespace in there is harmful. as there
isn't a version with tumbleweed, we can just replace
distribution/version with "opensuse-tumbleweed".

Change-Id: I21da4b5f851ec864dfb3cbf8ea6713bc12cda58f
2018-03-02 16:48:49 +01:00
Dirk Mueller 48099a5347 Fix Tumbleweed mirroring url path typo
Last revision of I700ba2279dc7aadc7051cad53898e1de9007a08d
of course had a typo compared to what was tested successfully before.

Change-Id: I6f1053d69650ee6235089e61e7f19f8927d8e8f5
2018-02-27 15:20:18 +01:00
Dirk Mueller 7172bd5dc8 Setup repositories for openSUSE Tumbleweed
Tumbleweed as a rolling release model distribution does not
have a sensible release version and is kept distinct from the
classing "release + update" distributions that are called "Leap".

Conditionalize the repository setup for tumbleweed properly,
the base path is different and it does not have a releaseversion
component (the release version is always changing, it is
the snapshot date in the form YYYYMMDD).

Change-Id: I700ba2279dc7aadc7051cad53898e1de9007a08d
2018-02-25 21:15:09 +01:00