dns-root-data has been demoted to a "Recommends" dependency of unbound,
which we don't install. Sadly the default unbound configuration is
broken without it.
Change-Id: I93e6928d30db8a90b45329ca00f066b4ec1b4ae7
Until now, whenever NetworkManager gets reloaded/restarted/lease
refresh, it would override the /etc/resolv.conf file with the
nameservers and related it gets from the network.
This patch ensures this won't happen ever again.
Note: this is a corrected version of
I92bc12b8f712e28962d24dd6474cfce22b81222c
that was reverted due to indentation + use of ConfigParser issues.
Change-Id: I48560641238911154cc9f353f707a9374613e51a
Bug in the inline Python's indentation levels. Please correct and
resubmit with more testing included where possible.
This reverts commit 368bb77ee3.
Change-Id: I20ed2c48bc223bd13d04c297e877f47c02141e4d
Until now, whenever NetworkManager gets reloaded/restarted/lease
refresh, it would override the /etc/resolv.conf file with the
nameservers and related it gets from the network.
This patch ensures this won't happen ever again.
Change-Id: I92bc12b8f712e28962d24dd6474cfce22b81222c
Change I316e9587b6e290cd421b47f506c91dbebe0975c0 had a rather
embarrasing oversight in that it copied the /usr/bindep-env/bin/pip
invocation for upgrading pip to all the other venv's.
i.e. we were upgrading hte bindep-env pip over and over, and not
actually the pip in the working venv. The os-testr install on older
platforms has now broken because it still tries to install with the
ancient inbuilt pip -- local testing has confirmed that it works with
the updated pip.
Change-Id: I22c549b5f9b9e3882fcd2340946d2850b0b2f86b
The previous patch[1] applied a partial context to the unbound.log file.
This patch applies a full context to resolve the "partial context" error, using semanage to make the file label persistent.
[1] https://review.opendev.org/c/openstack/project-config/+/841546
Change-Id: Ic15957fa4ef58355efd2e96f143386f393b0a59d
I4f3265c16320613d4ba74a02df1361c5d9cf2fb1 moved this file to
/var/lib/unbound on selinux systems, as it was getting permissions
errors trying to write into /var/log.
This turns out to make it harder to collect the logs from projects
like devstack. It's simpler if we just have the log file in a
consistent place. On selinux systems, set the context, and revert
things to just writing into /usr/log/unbound.log
Change-Id: I6bb58ef0d6bf4cbbb7fd4066e01b7a01d05009c3
The pip installed in the venv with "python3 -m venv" on Xenial is 8.X
-- this does not understand python_requires metadata on packages and
can thus pull in requirements that won't actually run inside the
virtualenv.
Avoid this by upgrading pip in the venv before installing.
While this is the immediate need; do the same on the other venv's we
create for general sanity.
Change-Id: I316e9587b6e290cd421b47f506c91dbebe0975c0
As noted inline, the /etc/init.d directory appears to have been
somehow remove/no longer created with a recent update. I've added
this manually and the image builds, and the rc-local.service still
runs. Do this for now to unblock other builds.
Change-Id: I0b0b2e38951bad656bcfdb47b6470e033564db59
This reverts commit 6f992efbc5.
Setuptools 50.1.0 has been released which reverts the breaking behavior.
https://review.opendev.org/#/c/749766/ tests that these fixes work in
other venvs that exhibited the same problems. That change looks happy so
I think this revert is ready to go.
Change-Id: I31b62be4f85f40f4d99e463cd961dec0a3542f47
Since all platforms have Python 3, use the new ensure-venv element
from the dependent change to install bindep and os-testr.
Since we are no longer using pip to install anything during the
builder, this drops the dependency on pip-and-virtualenv from
nodepool-base. Avoiding this element is our long-term goal, as it's
modification to system state are problematic in a number of ways. To
maintain the status-quo, the pip-and-virtualenv element is added
explicitly to each build's element list, with a note on it's future.
The current plan for backwards compatability is to replicate the
environment pip-and-virtualenv provides in a base role/job that can be
optionally included. To test this, provide a new node type
"ubuntu-bionic-plain" that will not include the pip-and-virtualenv
element. This is put on just one provider (rax) to minimise impact.
The dependent-change (and a dib release) is required before merge so
the ensure-venv element is available.
Depends-On: https://review.opendev.org/707513
Change-Id: I85438baf5bb31790a56fe5b38327361f0a2398e9
I was a little too hasty in I76acbd08acda20c88ff9fd9148e3918b78d6c6c6
removing the scripts/ directory. It has broken the puppeting of old
hosts.
Restore the directory with a blank file explaining the situation.
Also, we don't need to copy this in the nodepool elements; remove that.
Change-Id: I8b82950237ef69c4941866900cac9bda42f58ca2
--seeder=pip is breaking images where pip is not installed from source.
New virtualenv upstream release 20.0.2 has fixed the issue seen when
using sudo [1] by copying instead of symlinking and does not need this
fix anymore.
[1] f4fd6a0991
This reverts commit be9530ae16.
Change-Id: I799982d9c023141cf612901084d4ecbd4447e969
New virtualenv will by default use a common location for seed libs like
setuptools, wheel, and pip. Unfortunately this breaks if root installs
the virtualenv and other users are expected to use it because these
other users cannot access /root/.local (where the files are stashed).
We fix this by using --seeder=pip which will install all of those libs
into the virtualenv itself.
Change-Id: I4922ea50e31dceda96f545a0d409c0d7dc022e19
This removes trusty from the repo and thus from OpenDev.
Afterwards the AFS volume mirror.wheel.trustyx64 can be deleted.
Depends-On: https://review.opendev.org/702771
Depends-On: https://review.opendev.org/702818
Change-Id: I3fa4c26b0c8aeacf1af76f9046ea98edb2fcdbd0
On platforms such as CentOS 8 which are python3 first, "virtualenv"
and "pip" may not exist (removed to avoid any confusion over them
being v2 or v3 commands).
The referenced dib change exports new variables that should be correct
on all platforms for creating virtualenv's and pip installs.
(note will require DIB release to be active on builders)
Depends-On: https://review.opendev.org/684462
Change-Id: I3414fb9e503f94ff744b560eff9ec0f4afdbb50e
In openSUSE Tumbleweed, the SuSEfirewall2 package was removed in favor
of firewalld[1]. This commit updates the openSUSE nodeset to use plain
iptables rather than injecting iptables rules into the SuSEfirewall2
service. This will work on both Tumbleweed and Leap nodesets.
openSUSE provides no iptables-service package the way the RHEL family
does, so we can't fall back to that. Rather than try to convert iptables
rules to firewalld syntax, this change leverages init.d to ensure
iptables rules are loaded at boot. The 89-unbound script has been
coopted for this purpose since it already creates
/etc/init.d/boot.local. Switched from `dd` to `cat` which makes
conditionally composing the file more natural.
[1] https://lists.opensuse.org/opensuse-factory/2019-01/msg00490.html
Change-Id: Ia2b72e25078efa68019f1bf7c7a0b77e6ff702fd
CloudFlare's public recursive DNS resolvers are available at
multiple anycast addresses. For some reason 1.1.1.1 is unreachable
from parts of OVH's BHS1 region, but 1.0.0.1 seems to be
consistently reachable. Swap this for improved reliability.
Change-Id: I9a264282ea6c8239883d252f52e004deebca3edc
Use new locations following OpenDev changes.
Node: This changes the on-disk repo path to /opt/git/openstack/devstack
Change-Id: I7042913fefa64dcec4044779dbeb13f86daea858
This role now lives in opendev/base-jobs but we still have some usage
from ozj. We can clean that up by removing what appears to be the only
job doing that since it was tempoarary anyway.
Additioanlly point the nodepool element comments at the right role in
opendev/base-jobs
Change-Id: I1d73f543006d94a52fa1cfe38870391da959ae74
Ianw noticed problems on fedora29 with unbound. That resulted in a bug
filed upstream,
https://www.nlnetlabs.nl/bugs-script/show_bug.cgi?id=4226. In this bug
the helpful unbound maintainers point out that OpenDNS servers are
having trouble with RRSIG records which leads to not validating dnssec
which we require in our unbound config.
Address this by switching to CloudFlare DNS which is suppsoed to be
super localized (aka responsive), and not record queries against it.
Also if we want to we can update our config to do dns over tls against
these servers.
Change-Id: I08ef6a6fba2706803d2e9de6197e0ef8d695e313
We are seeing a problem on Fedora where it appears on hosts without
configured ipv6 unbound chooses to send queries via the ipv6
forwarders and then returns DNS failures.
An upstream issue has been filed [1], but it remains unclear exactly
why this happens on Fedora but not other platforms.
However, having ipv6 forwarders is not always correct. Not all our
platforms have glean support for ipv6 configuration, nor do all our
providers provide ipv6 transit.
Therefore, ipv4 is the lowest common denominator across all platforms.
Even those who are "ipv6 only" still provide ipv4 via NAT --
originally it was the unreliability of this NAT transit that lead to
unbound being used in the first place. It should be noted that in
most all jobs, the configure-unbound role [2] called from the base-job
will re-write the forwarding information and configure ipv4/6
correctly during the base job depending on the node & provider
support. Thus this only really affects some of the
openstack-zuul-jobs/system-config integration jobs, where we start out
without unbound configured because we're actually *testing* the
unbound configuration role.
An additional complication is that we want to keep backwards
compatability and populate the settings if
NODEPOOL_STATIC_NAMESERVER_V6 is explicitly set -- this is sometimes
required if you building infra-style images and are within a corporate
network that disallows outbound DNS queries for example.
Thus by default only populate ipv4 forwarders, unless explicitly asked
to add ipv6 with the new variable or the static v6 nameservers are
explicitly specified.
[1] https://www.nlnetlabs.nl/bugs-script/show_bug.cgi?id=4188
[2] http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/roles/configure-unbound
Change-Id: If060455e163266b2c3e72b4a2ac2838a61859496
On centos this seems to be part of the built in config, on ubuntu this
seems to not be configured but is unbounds default behavior there, on
gentoo it chroots per the default described in the manpage.
In order to force consistent behavior across all systems disable the
chroot when we configure our logfile path (so that the logfile's dir is
present and writeable) by setting chroot: "" across the board.
This should fix the unbound service on gentoo and be a noop for our
other platforms.
Change-Id: Ic1b66c1982c14759e8fd8370452df21d2b0d9510
Add rhel7 distrib in unbound and iptables script for nodepool-base, and
add haveged installation for infra-package-needs to ensure users could
use these elements to build rhel based image.
Change-Id: Ib0ad877369bafab64a1fd25cc331363d771d5753
This is applying the iptables rules during the build on the builder.
This will need to be converted into a script to run on boot or
similar.
Change-Id: Icc753013f218c3e4f04031c2fdbc7b32a534d887
As described in the inline comment, this ensures that on Bionic (and
any similar platform using systemd-resolved) we use a regular
/etc/resolv.conf file for nameserver configuration.
Depends-On: https://review.openstack.org/#/c/558362/
Change-Id: I1e1d285787f88370fec6e9e21701164cc232e153
This reverts commit da15523595.
systemd-resolved was not really involed in this. What seems to be
happening is that during the initial chroot creation there is no
/etc/resolv.conf file, and so the systemd postinst script creates one
pointing to the compatability files created by systemd-resolved.
This is not what we want, but dib doesn't really provide a way for us
to overwrite the file. That is covered in
Ie0e97d8072e2b21a54b053fa6fb07b62960c686d
We actually want systemd-resolved running -- it provides the dbus
nameserver api stuff that some tools may use. If /etc/resolv.conf is
a file, systemd-resolved leaves it alone and uses it for resolution as
you would expect. (we do, however, want to wait for the depends-on to
ensure the image has a correct /etc/resolv.conf before merging this).
Depends-On: https://review.openstack.org/557842/
Change-Id: Ie3cdc323bf8fb4fcee725f9a52bf53a4a6a01bbf
Like we do with ubuntu-trusty / ubuntu-xenial, pin our DIB builds for
ubuntu-bionic to our AFS mirrors. This will prevent issues with bionic
installing newer packages when we have mirroring issues with reprepro.
Change-Id: I036f90d25dfc6c244e1bd010a9f2db7e295a439f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
It seems with ubuntu-bionic, systemd-resolve boots by default. This is
then overwriting our /etc/resolv.conf file with its nameserver
address.
Change-Id: I3de217505cc0ba18a233e8712be49c5347a8e29b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
There is no jenkins, only zuul! With zuulv3 running in production,
there is no more need to add jenkins user or scripts to our images.
Move chmod of /opt/cache/files from
nodepool/elements/jenkins-slave/install.d/20-jenkins-slave to
nodepool/elements/zuul-worker/install.d/60-zuul-worker
We also leave the jenkins-slave elements for now, in case 3rd party CI
use them.
Change-Id: Ia9750877fbc1a17ec467ca4ac685afdb9c1627f8
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We now only have afs-slug.sh in jenkins/scripts/ and it is only
used by nodepool/scripts/configure_mirror.sh. Move the file into the
nodepool/scripts directory and adjust paths.
Remove copying of jenkins/scripts from nodepool, the directory does not
exist anymore.
Remove 89-jenkins-scripts, we don't need it anymore.
Depends-On: https://review.openstack.org/548393
Change-Id: I13cac791d70a2e5bcb0de1ba0adcd79e1b0aee9f
There is an issue with newer kernels (it seems to happen with 4.15.x)
that when conntrack is reloaded while a connection has packets in
flight, this connection going forward is neither considered INVALID
nor RELATED nor ESTABLISHED by the stateful tracking. While this is
certainly a bug somewhere in the kernel, we can be easily avoiding
this by just not using stateful filtering for ssh connections, as
we accept any connection from anywhere anyway.
Change-Id: I1b20644ce888930cd28d6eaf2c23787315e8199c
Remove using old location, we can switch now to the new location
everywhere.
Note: We can only merge this after new images are build with the new
location build in.
This needs a recheck once change 1 is in since project-config is
not self-testing. So, we can remove jenkins/data/bindep-fallback.txt in
this change.
Change-Id: I08f23e96defb152946ade1490628f37f0215300d
Depends-On: https://review.openstack.org/543141
Depends-On: https://review.openstack.org/543142
Let's get rid of jenkins in the name and use a new place to place
bindep-fallback.txt in.
Create new nodepool element to copy the file over.
This is a first of two changes for this repo, we also need to update
the few users in bindep and openstack-zuul-jobs before we can remove the
old location.
We can remove jenkins/data/bindep-fallback.txt only in the followup
change since project-config is not self-testing.
Change-Id: I5c94dcbbca25db7dd998bfbccfbb25c1c4342a2b