Commit Graph

44 Commits

Author SHA1 Message Date
Dmitriy Rabotyagov 611fb2bf4c Deprecate OpenStack-Ansible nspawn repositories
Since nspawn support has been removed with [1] we now
start repositories deprecation process.

[1] https://review.opendev.org/c/openstack/openstack-ansible/+/782569

Depends-On: https://review.opendev.org/c/openstack/project-config/+/797723
Change-Id: I113f0f2b5befc4bca830c130d7f60e621c86798e
2021-06-23 17:22:41 +00:00
Dmitriy Rabotyagov 11594a85ae Use version test instead of version_compare
This test was changed to 'version' in ansible 2.5 [1].

[1] https://docs.ansible.com/ansible/2.8/user_guide/playbooks_tests.html#version-compariso

Change-Id: If27fde3a15e2b611cebdfd26af5b421475cb7488
2019-09-10 13:48:23 +03:00
Kevin Carter fae866ea72 Remove the private option from include_role
The private option on include role was never implemented and
will no longer be developed. This change removes the option
so ansible no longer raises a deprecation warning.

Change-Id: I1b7649481d2f9fbd25164a28cd10972a63143747
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2019-02-22 00:49:36 +00:00
Kevin Carter 2f76119d7a Set the dnsmasq aa profile to complain
If a stock aa profile exists for dnsmasq (SUSE, Ubuntu) the nspawn
deployment will fail due to the profile being strict. This change
sets the profile to complain, which allows the deployment to
succeed. The new tasks will detect a stock dnsmasq aa profile and
set it to complain if found.

Change-Id: Ie565b683d2f06e47f7a46497ce3c38d604a0fed6
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
2019-02-09 12:54:24 -06:00
Kevin Carter ef4aef4802
Add path variable for different distros
This change removes the assumption that ip and iptables are always in
the same location on any distro by setting the specific tool path per OS
package manager type.

This adapts the pattern set here:
fc9a2d03dc/tests/roles/bootstrap-host/defaults/main.yml (L106-L118)

Change-Id: If7ddd8f2afb8c990f2b5117947f103626663ad21
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2019-01-31 11:07:33 -06:00
Kevin Carter 144bf1ccd9 Change subvolume create to unmask errors
The subvolume creation task was running such that all errors were being
masked. This change uses the creates argument to give us an idempotent
task while also ensuring we're not masking real errors.

Change-Id: I3b87457e8871128c32b8829f440ea69e0623bfd8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2019-01-05 04:06:57 +00:00
Kevin Carter 26221a267d Use libeatmydata during nspawn cache prep
The nspawn cache prep is timing out regularly so use eatmydata to minimise
pressure on the filesystem during package installation.

dpkg calls fsync for each file installed from each package which can
result in very poor performance on some filesystems. This patch gives
around a 4x improvement in "slow" gate runs, and 2x on esxi hdd storage.

Change-Id: I80efd3f8044f377a497492f09689c459c6094742
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Co-authored-by: Jonathan Rosser <jonathan.rosser@rd.bbc.co.uk>
2019-01-02 12:02:54 -06:00
Kevin Carter fce12838ba Update the nspawn unit services
This change updates the unit file for systemd-nspawn to allow it to
better confine containers and have them reliabily start/stop on host
restart.

Change-Id: I3c7a07a94c94a81ac8380a4e336cf744615a6b5b
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-12-22 18:20:36 +00:00
Kevin Carter ca24797261 Correct host only routes
The host only route was adding the cidr twice which is wrong. This
change adds the missing network filter to correct the issue.

Change-Id: Ic280db301ce44b98ac35c8911999dec5fa263a29
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-10-09 02:05:35 +02:00
Kevin Carter 4f0837931b
Modify the network to ensure their more restrictive
The network rules used to be too broad and could result in conflict when
address space is more restricted. This change updates the network rule
set such that it will not only add address to a macvlan interface using
the `noprefixroute` option and setting the route to the local table
only. This limits the macvlan network scope to ensure we're not creating
conflicts while also not breaking inter host connectivity.

Change-Id: I9b27a006a5587150254b35288d8907ae32651b57
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-10-03 23:20:19 -05:00
Kevin Carter e873bda7d6
Add host only network option
The host only network function was adding a global network address and a
host only network address which would flip flop should networkd or the
nspawn-macvlan service be restarted. This change makes resolves the flip
flopping and adds an option to enable this functionality when required.

Change-Id: Ia856723580d345d01e0df06ac41adb9cb7fcdb24
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-09-30 10:14:53 -05:00
Nicolas Bock f54f58f57f
Add missing variable for nspawn networking
The variable `interface` is not defined in the `if` part of the
conditional, however, it is referenced later on.

Change-Id: I86ed0daabe369b984bf534d3bcf471c73c7795ce
2018-09-13 16:10:13 -06:00
Jesse Pretorius f3fe949711 Simplify inserting apt keys into nspawn image
Bionic requires a functioning gpg-agent to run apt-key add. This means
that gpg-agent must be working properly in the chroot when the nspawn
image preparation script runs.

Previous changes [1] have enabled apt-key to communicate with gpg-agent
during the nspawn_hosts role checks, however the cache prep fails almost
every time when nspawn_hosts is used within other role checks.

This is not a new issue, debian-installer is affected too [2].

This change adopts the same route as d-i, and simply copies the host
/etc/apt/trusted.gpg.d directory to the nspawn image, removing the
need for apt-key and in turn gpg-agent.

This is a re-implementation of https://review.openstack.org/588962
for nspawn.

[1] https://review.openstack.org/590431
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=851774

Change-Id: I3c56da445377d7ba27a623fb7ebe95c20d28a327
2018-08-25 21:12:51 +01:00
Kevin Carter 7ba42bc3d7 Add documentation for routed interfaces
Using the container_(extra)_networks hash a user can define a routed
interface for a network without an address. This change documents that
capability.

Change-Id: I93bdfde1aa5a9820d1853a779caf6670ee58d335
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-08-20 18:50:35 -05:00
Kevin Carter ca6fe7df8d Document adding "extra" networks
This change moves the hidden `container_extra_networks` options to the
defaults file and adds a blerb regarding how this functions.

Change-Id: I4d7a9f05cf8382d0b83dd0ce574f7f4ba3783a63
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-08-20 18:50:35 -05:00
Kevin Carter bf931fb29f
Document primary interfaces and add mv addresses
The macvlan primary interface will now allow deployers to run a fully
minimal network stack without any bridges or out-of-band configurations.
This capability has now been added to the defaults with documentation
regarding how its implemented.

Change-Id: I73e52ff9237dcc9c0d1bd156345d730454d28533
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-08-10 14:09:05 -05:00
Zuul 4d40aa71cc Merge "Tuneup prep and local routing when running w/out bridges" 2018-08-10 05:21:33 +00:00
Kevin Carter caf28c7663
Tuneup prep and local routing when running w/out bridges
When running without bridges the local routing needs to be scoped to the
host. This change ensures that is the care.

When re-preping an environment the GPG setup will require aceess to fd's
which may not exist at the time the base cache is re-prep'd.

nspawn mtu settings will now follow the primary interface.

Change-Id: I74e9301a98cf92161feb31e6808e9e02a07f662c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-08-09 23:51:34 -05:00
Zuul 1316735c81 Merge "use include_tasks instead of include" 2018-08-09 17:29:20 +00:00
Kevin Carter 4f6ed536ed
Add route when required on the local table
With the use of nspawn containers and macvlans, without an intermediary
bridge, there's no way to send traffic from the host back to the
container without configuring VEPA and having 802.1qg enabled
switches. To enhance the usability of nspawn a local route is added to
macvlan devices when an interface address is detected. This allows
the host to route back to the containers without issues and will not
interfere with the main host route. To ensure there's absolutely no
conflict with host routes a metric has added to all routes added to the
local table.

Change-Id: Id4326c9783f46884d3f39692e691775ba1118b38
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-08-02 21:19:36 -05:00
caoyuan 4f98651a93 use include_tasks instead of include
Change-Id: I49e0ed58b5d26af5c1309183f6b76c7a96cfe429
2018-07-24 21:00:53 +08:00
Jean-Philippe Evrard e9bb87e7a6 Fix usage of "|" for tests
With the more recent versions of ansible, we should now use
"is" instead of the "|" sign for the tests.

This should fix it.

Change-Id: I95b55ff5a5932a9d1214f357a32cf1e4393f82af
2018-07-12 15:21:03 +00:00
Kevin Carter cb466cd385
Update nspawn role correcting several issues
The tempfiles template was being setup on the host, this causes host
machines to have an issue with journald on reboot due to the directory
being part of systemd-tmpfiles.

Nspawn container journals were not being written to the host. This was
fixed by changing try-guest to try-host, which will fall back to the
guest in the event that host journalling is not possible.

Systemd-nspawn containers were not starting on boot due to them needing
to wait for networking to be online. The base template has been updated
to ensure networking is online before starting containers.

Change-Id: I6af3923bd10091172e75cfb16c9146cd47f827a4
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-07-03 23:56:07 -05:00
Kevin Carter 286e4c80c2
Add systemd tags to include role
There are times when a deployer will need to reconfigure parts of
an environment and having a general purpose tag to run said operation
will be important especicially should the deployer be needing to
reconfigure systemd unit files in a downtime event. This change adds
a general purpose systemd tag where include_role and systemd is found
which will assit operators with day2 operational tasks.

Change-Id: I8a007c62e110c96fce4fc687258af0b4ef7b3ab1
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-06-27 19:00:13 -05:00
Kevin Carter 9407b555b7
correct typo in service setup
Change-Id: Ia9782adaf0c36a73a94d27de8b252c04f34c68e6
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-06-27 18:04:57 -05:00
Kevin Carter 455c372b06
Correct networking setup for macvlan
The macvlan setup was duplicating IP addresses from the host using the mv
interface. This caused all kinds of unexpected brokeness. This change
corrects the config so that the role is not longer overreaching.

The test setup to pin the image download url elsewhere has been removed
as it's now causing problems connecting to the index.

Change-Id: Ida9751f6ad94bf5e4b0dcd15a506e503350fdd9d
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-06-25 13:26:35 -05:00
Kevin Carter 30477c0f89 tasks: Always create the machines.raw file
The machines.raw file is only created if the 'machinectl set-limit'
command returns non-zero exit code. However, this file is expected to
be present later on so we need to make sure that we create without
taking the 'machinectl' exit code into consideration.

Fixes the following problem:

Change-Id: If96c870d317bc7a024169ffd04e8f4c10003ba74
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Device /var/lib/machines.raw not found."}
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-06-21 06:00:31 +00:00
Kevin Carter cfb3f9e7cd
Set container create tasks to non-voting/fix pkg
There are some inter-dependencies here where the container tasks will
not pass until a few updates are in both here and elsewhere. This change
sets the container create tasks to non-voting until all of the required
updates get in.

Remove the aria2 requirement

While aria2 has some nice features, the get_url command can do most of
what we need and does so in a far more universal way. This change
removes the aria2 dependency and updates the aria2 tasks to use the
get_url module.

Change-Id: Iab422c718d789ef13b8ec55938fdb7e73e40061b
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-06-21 00:56:33 -05:00
Zuul 585a615f64 Merge "Add networkd prefix when creating networks" 2018-06-01 23:15:14 +00:00
Kevin Carter 7ca4224dda Add networkd prefix when creating networks
systemd networkd is a shared directory of networks that needs to be able
to co-exist with the host which may be using networkd on it's own. This
change adds an nspawn specific networkd prefix to the interface files to
ensure we're not creating interface file collisions.

Change-Id: I1a9e25f697b64e292261cf0abf8df61a97dd2730
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-06-01 17:24:41 -05:00
Kevin Carter 4552c56f2b Add a single variable for combined networks
The combined networks variable will allow deployers to set basic
container networks, default nspawn networks, and any "extra" networks a
deployer may need outside of a standard build.

Change-Id: I5346fdf81967d6eb7f69e61a187166c076ca4b60
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-06-01 22:08:55 +00:00
Kevin Carter d520e80462
Make host copying more selective
The host copy process was taking too much, this makes that process more
selective.

Change-Id: Ic25559d9b68b6208ffdbed020cc79bd6b3d52fdd
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-05-18 19:16:03 -05:00
Kevin Carter 61380b711f
Update qgroup messages
This change will provide additional insight to the deployer should an
error from qgroup settings be raised.

Change-Id: I16bb8c33ec83890e2e354822c94476bd97e3988c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-05-14 21:33:47 -05:00
Kevin Carter 556748bc4e
Document everything in config
Update container pkg list and use qgroups

Other roles within OSA require additional packages that were removed
from the base cache. This change puts those packages back.

Additionally this change implements qgroups which will resolve an issue
where the containers are spamming a reoccuring error message indicating
an issue with the loopback file system.

Change-Id: If658e5f332e4580cc51182ac92331a2ef91fbcb8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-05-14 20:42:33 -05:00
Kevin Carter bac2455583
Add task to regather facts once the role completes
In order to ensure the deployment host has the most up-to-date facts we
must regather them upon the completion of the role. This is done because
the role will run several tasks that can perform a state change on
various resources and we need to capture the most current state. This is
especially important when running in an environment that makes use of
cached facts.

Change-Id: I429485eba7e4e6643d10d0f3826471f47aab87fc
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-05-14 14:23:13 -05:00
Kevin Carter 4f4bc4aa54
Ensure interface ordering is consistent
This change adds a sort to the interface craetion process which will
key off the interface name and ensure we're not creating duplication
devices while also maintaining the sort order.

Change-Id: Iccedce916af0bcc6323b448229cd8958de40778c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-05-14 10:48:52 -05:00
Kevin Carter c2491c9e56
Add mount options for better machinectl performance
The machinectl default options, while functional, could be tuned for
better overall performance. This change adds several options which will
ensure container workloads are using the lest amount of storage with the
best possible performance.

For more information on the options being used see
 * https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs(5)#MOUNT_OPTIONS

Change-Id: I4f0a6f2ad5a96ff91b8efafba78236af653f5925
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-05-14 09:34:27 -05:00
Kevin Carter c0d1d3641b
Add service params to keep systems in sync
While the start order of these services shouldn't have any conflicts
this change ensures it by forcing the various services to be part of one
another and have specific ordering.

Change-Id: Idf5de7d018a92d6df31b3277e46473698088bfd8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-05-13 23:50:16 -05:00
Kevin Carter 50d7cc82c5
Update nspawn hosts
Use the common roles throught this role to remove all of the boilerplate
code we had. The common modules do most of the heavy lifing.

Update to fix the resolve.conf issue with the image cache prep.

Add legacy image support and use smaller upstream images by default.

Now that suse supports systemd-networkd we can enable suse support in
nspawn.

Change-Id: I5f6ceb928f5c0902adf2e34f96a5998840400777
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-05-13 14:28:16 -05:00
Kevin Carter 942531a8d8
add minimal functional tests
Change-Id: Ie76759e6c162270fd3e767d4719b885b57a7539b
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-02-23 11:45:59 -06:00
Kevin Carter 2758cbfc6a
add network ops
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-02-16 21:25:39 -06:00
Kevin Carter edf8e1238c
Enable less noisy journaling
Enable persistent journals and link the container journal to the host by
default.

Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-02-13 17:07:48 -06:00
Kevin Carter b485d3178a
Remove lock file if found
When the cache is created the lock file my be left over, this can cause
some conflicts later in the container provisioning process. This change
removes the file if found.

Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-02-09 18:54:40 -06:00
Kevin Carter e97c013f67
First commit
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-02-08 11:39:40 -06:00