pkg_resources is part of setuptools which is no longer included in
python environments by default in python3.12. Importlib.resources can be
used instead and is included in stdlib starting with python3.7.
Unfortunately the ability to extract directories isn't available until
python3.12 in importlib.resources and we rely on this functionality. We
address these problems by trying to use pkg_resources first which should
work for anything python3.11 and older and if not available use
importlib.resources which should work for python3.12 and newer.
Change-Id: I97d9150f92960f0e7ab5efb60523a67d750c7646
It was pointed out in reviews of
Iac0760ed1dd33c06b81cdd2ea3885279d2aab878 (thanks clarkb) that a
single network is just a case of a multiple network list len()==1,
thus we can simplify this path.
This is a refactor to implement this, removing _write_rh_interfaces()
and combining it into write_redhat_interfaces(). I was probably a bit
deep into it when I wrote a lot of these comments, rework them a bit.
Change-Id: Iede6054d84cffa28e1365501918264a779e5db76
This adds a sample setup for vexxhost, which specifies DHCP and IPv6
SLAAC.
Honestly, I think this reveals that Debian/Ubuntu aren't handling this
combo quite right. They don't seem to be correctly specifying the
ipv6 autoconfig options for the interface. None of this path has been
touched in a long time, so I am inclined to leave well enough alone.
his at least provides a basis for tweaking in the future.
This has some slight reworking of things in the test generator. The
sample configs are all using the more modern stable-name "ens3"
instead of "eth0", so we need to parameterize this to some bits of the
testing. Also makes sure it sets up the ipv6 address as a hostname.
Change-Id: If87bb8f981cb2e7e12918919210d0d58349f17a2
This wires in support for ipv6_slaac metadata to the RedHat like
platforms. This is fairly straight-forward, but a full test for an
environment that provides this (vexxhost) will follow-on in
If87bb8f981cb2e7e12918919210d0d58349f17a2.
Change-Id: Ieca0706c0a27485a155cd35ab0200f22309c61bb
It turns out that prior to NetworkManager commit [1] you can not
specify default routes in the routes[6]-<interface> file. So
Iac0760ed1dd33c06b81cdd2ea3885279d2aab878 works OK for modern
releases, for maximum compatability we should walk the routes and put
the default ipv6 route in the ifcfg file directly, instead of in the
separate route file.
[1] https://cgit.freedesktop.org/NetworkManager/NetworkManager/commit/?id=c167e0140babcf1a045cee34ce4938f5087f8fe6&utm_source=anzwix
Change-Id: I41381bb65d980002c4ccfd2eca643ec51529360a
Many of the test fixture output comparision files are the same.
Updating them all when you change something can get quite annoying.
Symlink files that are expected to be the same together. The general
pattern is to use the most "generic" version -- so centos/fedora/rocky
symlink back to the "redhat" version, "ubuntu" symlinks back to the
"debian" version, etc.
Change-Id: I6d46cc076ed8571521a1edec06d4064c424c59f5
These config file are not ordered, so we don't need to add the ip info
between any particular header/footer section. Similar to
I09a79359bdc13973f67f0503eee9fa9fd7586ead I think this was mostly done
in I20bffabd333ea290d8712ec2a467f2b2d5678f3a to make the output fit
the extant test fixtures.
This will help make combining ipv6 information easier in the future.
This reorganises the output, but is intended to have no operational
effect.
Change-Id: I3f15ed78f75079e2f7cbc25629a6fff9f150cefa
This handling for "TYPE=Ethernet" came in with the suse split in
I20bffabd333ea290d8712ec2a467f2b2d5678f3a.
I think this is a case of over-fitting the test data incorrectly. If
"TYPE=Ethernet" is not specified, it is assumed (e.g. [1]). For
whatever reason this was not put in some of the test fixtures, which
have been copied over-and-over. When the referenced change moved to
more "generic" handling it was written to fit this pattern, but really
we can/should specify the type.
This is a cleanup to help make combining ipv6 information easier in
the future. This should have no operational effect, since this was
the default anyway.
[1] 6a68008e44/src/core/settings/plugins/ifcfg-rh/nms-ifcfg-rh-reader.c (L6446)
Change-Id: I09a79359bdc13973f67f0503eee9fa9fd7586ead
This is a follow-on to I0aac3674fd17ff2bfcf3542e27097e171036c837
Prior to this we are walking the list of networks and setting them up
one-by-one. As noted in the prior change, this fails when two
networks refer to the same interface (such as configuring ipv4 &
ipv6).
This builds a dict of interface -> [networks] mapping. For the most
part, interfaces will only have one entry in their networks. However,
some will have ipv4 & ipv6.
For now, we filter out the ipv6 networks; this will be addressed in a
follow-on. This means we are effectively keeping the status-quo;
which is to ignore ipv6. This change is intended to have no
operational changes.
Change-Id: I7eb3a4e33ae1d924a46e85035ac283a6755a74a4
We currently go through each network an assume that we're writing out
one ifcfg-<interface> file for each entry we see.
This fails when we have an ipv4 and and ipv6 entry for the same
interface. In this case, we need to combine the details into the same
ifcfg-<interface> config file.
This is a small refactor of write_redhat_interfaces to filter out the
interfaces list before we start writing the config files. This is the
first step in reworking things so an output function can take multiple
networks to configure.
This is intended to be only a refactor and have no functional change.
Change-Id: I0aac3674fd17ff2bfcf3542e27097e171036c837
This reverts commit 0908c99be9.
The config added and tested here is testing the override where we
ignore the metadata and just write out basic DHCP config.
OVH isn't configured like this and we haven't used this in production
coming up on 3 years since Id97aceb78019b7b71bc231778d7ea7e0f3964e0d.
I'm proposing removing this path to avoid us having to maintain this
as we move forward.
Change-Id: I2407dda8d126f2ef297de01817f8136db901f19d
It seems on CentOS 9 (maybe others?) instance boot with an eth0
interface that is renamed to something like ens3. When that renaming
happens it seems to short circuit eth0 udev event handling so glean
never sees an add event for eth0 (which wouldn't be correct anyway since
it has been renamed). Then the ens3 event is a move event and our udev
rule doesn't match that which means we never run glean for ens3 either.
Fix this by handling add|move in the ACTION list of our udev rule for
glean so that we run glean for ens3 when it has been moved.
Change-Id: I5369afb425dcf17e7539a4dd000231830862802b
Rename things to be slightly clearer by removing references to
glean.sh from various variable names and subsitutions.
Change-Id: Ib05220cd2de5a59b608c3e94e26ea44a89a77c49
Currently for the systemd/udev path, every device activated by udev
runs the "glean.sh" script, which attempts to mount the config drive
and set the ssh keys/hostname.
We should run an early service that mounts the config drive and does
this common setup. Then each interface activated by udev only needs
to configure it's own network settings by calling the glean tool
directly.
This modifies things to run a glean-early.service, which does the
mounting, etc. This runs the "glean-early.sh" script, which replaces
the no-longer necessary generic "glean.sh" script (an earlier change
moved legacy users depending on glean to iterate interfaces to use
"glean-legacy.sh").
Each of the udev-activated services is updated to depend on this early
configuration. These now call "python-glean", which is our small
wrapper to call the glean python tool under the interpreter it was
installed with.
Change-Id: I4b36e99ff8ee10e0b855733d97ec4ee12f941c11
This updates all the init scripts that call glean and expect it to
iterate the network interfaces to call a "glean-legacy.sh" script.
This is opposed to the udev path; where glean gets called for each
interface activated by the kernel. Follow-on changes will modify this
path to do less work for each invocation.
Change-Id: I789890b3d55838f3b70c65e519991fed0eb74e6e
Since Ic16f134fe34293bb68e7c632dd320f523366320d simple-init has
installed glean into a virtualenv. Despite this, it still needs to
setup a range of global symlinks (in /usr/local/bin) so glean can
blindly call "glean" or "glean.sh".
This means that glean installation isn't actually stand-alone. Unless
simple-init sets up these global symlinks the glean install doesn't
work. This makes it very annoying to try and update the way the
scripts are working, because we have to merge changes into simple-init
as well.
We can make the installation self-consistent by using the install
tool. The init scripts call glean.sh -- we can find the full path to
glean.sh using pkg_resources and write that into the files.
glean.sh wants to call the python tool "glean". This is slightly
harder, because at runtime the script doesn't really know how to
invoke this (i.e. in a virtualenv, it should call
/path/to/virtualenv/bin/glean). To allow for this, write a sibling
file next to "glean.sh" to invoke glean with the interpreter the
install is running under patched in. This way, glean.sh can call this
wrapper relative to itself and get the right thing. Add a __main__ to
allow glean to be called like this, and update the glean.sh script.
Change-Id: I1adfecf0d5c53648ee857be502216fd8d6cc5e16
We should not be using "is" to equate literals. It must have been
working by accident but breaks in recent Python 3. Appears to been
like this since initial commit with
Iaa17b6ce2444dd48a3f5fcf8eeb9f46ce6077bf7.
Change-Id: Iae85431e03d2db93376798e10cddb8f43ec67be2
Currently Glean rewrites resolve.conf and disables DNS handling in
NetworkManager even if there is no DNS information provided, making it
impossible to e.g. build DNS servers into the image. This behavior
also does not play way with the --no-dhcp-fallback flag.
Change-Id: I59cf2ece4e665d569d1db08d1df19b0892c6ba9d
The way we're using Glean in ironic, we'd rather have it not do anything
is there is no explicit configuration. This option allows it.
Change-Id: I4554ecf50ea03c1f56a4447befd498c5af5849e2
As described inline, we think that NetworkManager is starting before
glean because the udev events have not been processed and the glean
services have not started. Since there is nothing blocking it, the
network-pre.target is reached and systemd is free to start
NetworkManager.
Although it is not reccommended to wait on udev-settle because it
could result in long timeouts [1], I don't think we have a choice here
because the fundamental nature of this tool is to override settings
before NetworkManager goes off and does its own thing.
[1] https://www.freedesktop.org/software/systemd/man/systemd-udev-settle.service.html
Co-Authored-By: Dmitry Tantsur <dtantsur@protonmail.com>
Change-Id: I1c9c68d3eb5fbeb42901f2ed95860641cb2d676f
There's not need to open and parse this file multiple times. Do it
once and pass the json decoded variable to the various functions. The
functions are already probing for data and handling it if keys are not
there, so we pass a blank dictionary if there is no meta-data.
Change-Id: I28f5cd72004fcb1cdc5ba0a1255376da99e47140
Resource constrained environments like those used in testing may not be
happy running multiple python glean processes in succession. This can
take many seconds and cause race conditions.
Avoid this as much as possible by running glean once in glean.sh
depending on whether or not the config drive is present.
Change-Id: I8be30ebb3a3ef48beeeaebc4ff25a125fe4d946a
Some users add a newline to their key, strip the key when checking if
it's already added.
Stop using regex (that didn't work) when checking if we already added a
hosts entry. Use a simple startswith() instead.
Switch to use MagicMock as it has support for iteration.
Change-Id: Ibeabbf1c4448933cdf274e6667c5a9493f3e5e37
Signed-off-by: Matthew Thode <mthode@mthode.org>
When systemd-resolved is enabled dns info is only written to
/etc/systemd/resolved.conf. If systemd-resolved is not enabled dns info is
only written to /etc/resolv.conf. We do not write to /etc/resolv.conf
if systemd-resolved is enabled because /etc/resolv.conf is a symlink pointing
to ../run/systemd/resolve/stub-resolv.conf. This target file does not exist
before systemd-resolved is run so glean would fail with:
'FileNotFoundError: [Errno 2] No such file or directory: '/etc/resolv.conf'
Change-Id: I644e0b50cfb7bb00a108160b99c0c1359d6a9dd4
Signed-off-by: Matthew Thode <mthode@mthode.org>
1. When pulling VLAN info from config-drive, `vlan_link` not
referencing to any of the existent L2 interfaces could
cause previous interface properties to be applied to the next
interface or cause Glean to crash
2. System configs writing error could cause Glean to crash
3. When pulling VLAN info from config-drive, L2 addresses may
not be properly collected despite required `vlan_id`
field is properly present
Also, deprecated `log.warn` replaced with `log.warning`.
Change-Id: I8dd2edec148150ce735f347fc455323a6581131c
On SLE 15 SP1:
``distro.linux_distribution(full_distribution_name=False)``
returns ``('sles', '15.1', 'n/a')``.
"sles" should be considered equivalent to "suse" and any openSUSE
variant for the purpose of network configuration, so this change ensures
that _is_suse handles it.
Depends-on: https://review.opendev.org/696989
Change-Id: Ie123b8bd875b401d88235af6d241bad84415fc3d
The extant code is designed to loop over every device in
/sys/class/net and bringing the interface "up" to see if it valid and
something that should be configured.
As described inline, if we bring "up" an interface it can accept an RA
and get an ipv6 address assigned by the kernel. NetworkManager will
then refuse to further configure the interface, leaving the host
generally without ipv4 networking.
The per-interface loop only actually happens on older platforms that
don't use systemd. On systemd, glean is called for each interface
individually by udev rules. However, we fall into the old code path,
just with one interface to work with, rather than all of them.
Thus this initial hack detects that case (by noting we were passed the
interface explicitly) and short-circuits activity check; it just
assumes that if udev asked (and it's not of a device type we don't
support), then the interface should be configured. The interface will
*not* be put into the "up" state.
We should follow-on this change with a removal of this loop and
cleaning up the non udev/systemd activation paths. However, this
depends on a few longer term things:
- removing Trusty support (which still hangs on by the skin of its
teeth in OpenStack Infra, so we need to be not building nodes there)
- evaluating what Gentoo is doing in the non-systemd case.
- making sure bifrost doesnt' depend on this (likely only other user?)
In the mean time, this should fix the race conditions we've been
seeing on system+network-manager platforms.
Change-Id: I6ce51a8755e1892d3010eefd365fbad6bcec137b
Glean does not attempt to do os.fsync after writing files, which
is not safe in Python since Python does not guarantee the file
being written even if it is closed on Python level.
This might result in ifup complaining with unknown interface as
the file is still in cache.
Depends-On: https://review.opendev.org/677796
Change-Id: I05fd94662c409660857d4bc66b9f6354ac21496a
We need RAs when using dhcpv6-stateless interfaces but we had disabled
them because they didn't match the slaac type. Update this to handle the
dhcpv6-stateless case too.
Change-Id: Id394a7b0fb5a04fe09d9b8ed709adc75ab2f2a33
In I3d379d35e7b000f32c3f6cc197c8aaafebc683fb we found that just having
network-pre.target was not sufficient to get the interfaces to come
up, and found emperically using local-fs.target made things work in CI
and on our hosts [1]. However, it always seemed to be a race
condition; the same .qcow2 run in another cloud environment would
reliably configure its interfaces in one provider but not another.
However, we now have yet antoher new cloud environment where we are
still seeing races between updating the config files and starting
NetworkManager. It seems clear now that we also need to add
dependencies on network-pre.target
The key learning here might be that we required the
After=local-fs.target all along, to make sure we could write the files
to disk, but we should have kept the network-pre dependencies. So
restore these values.
[1] https://review.opendev.org/#/c/618964/9..17/glean/init/glean-nm%2540.service
Depends-On: https://review.opendev.org/670102
Change-Id: Ib650d4c4303535205a62600e2c8657eae45dc4a4
The Wireguard interfaces should not be managed by Glean as they
are usually configured manually (and they are mainly tunnel
interfaces).
They do present themselves with a permanent address however, which
means the only way to ignore them is by using the ignored list.
Change-Id: Ie0c2b56d78620f6ee562b42de6249b1efd37558e
Clark Boylan