Since we're converting plugins repo to collection, we make plugins
usable from collection. For that we change repo structure.
Symlinks to old destinations are created for backwards compatability
Change-Id: I15b4a9f50910794c11b57457f9f4fbb34f321654
The imp module is deprecated since version 3.4, so we should
use importlib wich is recommended replacement.
Change-Id: Ic725bf6e99af11bee6a4c4a5c8f409f0675ce8bf
Change from using variable manager to the var we already have for
the physical host. This saves nearly 50% time for re-running
playbooks/lxc-container-create.yml
Change-Id: Ic50763f9c74746ef94925a4dd52d16e1f313711b
This saves 25s from the previous runtime of 4m21s for
playbooks/lxc-containers-create on an already deployed AIO.
Change-Id: I8b315efed26a22ae669432856eb1e7d994f4ab3
Python 3 disallows implicit relative imports.
When skip_tags is provided as a single string, it will need to be cast
to an array to avoid being treated as a list of characters when compared
to task.tags.
Change-Id: Ib8e7b67b9400fb983aecaae95f50bb3ec35d4c26
inventory_hostname is not reliable to be used as container_name. When
delegating in 2.6+, the inventory_hostname of the delegating host is
passed to the connection plugin, additionally when a host doesn't have
its own container_name variable a cached inventory_hostname is used in
its place.
To get the connection plugin working with Ansible 2.6 this change
essentially allows the container_check to fall through when delegating
to physical hosts or hosts not in the inventory by:
- removing inventory_hostname as a fallback for container_name
- unsetting container_name when delegating to a known physical host
- unsetting container_name when delegating to a host not mentioned in
the inventory
container_name can no longer be used as a group variable based on
inventory_hostname, it will need to be an inventory/host variable for
each individual host.
Also, when container_user is used the remote_tmp path needs to be set to
a system writable directory instead of the root user's home.
Change-Id: If2eb4c16273e19599f6ec3f0cba6b3573912c6a0
The purpose of the 'physical_host_addrs' variable was to create a
mapping with all the physical hosts and their IP addresses. However,
the function was broken because it used the 'physical_host' task
variable to obtain the name of the physical host for the running task.
As such, the dictionary only contained a single entry. This breaks
delegation between hosts, since there was no entry in the dictionary for
the delegated host. So if a task targetting containerA on hostA was
delegated to containerB on hostB, the IP address of the hostB was not
in the dictionary. The connection plugin code then used the 'hostB'
hostname to connect to the host instead of its IP. However, if there
is no central DNS to translate hostB to an IP or if the deployer
did not add such mapping to /etc/hosts we observed the following
problem.
fatal: [compute00]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh:
Could not resolve hostname controller00: Name or service not known\r\n", "unreachable": true}
In order to fix that, we now examine the delegated host and create the
appropriate mappings if the task is delegated. This ensures that the
physical_host_addrs variable also contains information for the delegated
hosts instead of just the original one.
Change-Id: Ic84ad3ccdd8283a4ab3fb3130d4d5e520ab736de
Commit I76af71ffe19b2a8afb18ed731f1362a4e399c613 broke the ability
to define physical_host in host_vars and group_vars. The physical_host
is only loaded from the ini inventory.
Change-Id: I3798529cf2bf73afbff2d2ef0f3fe17d91123fbc
When running our plugins we're seeing a severe performance impact in
the ansible run-time. As it turns out, within the linear strategy we're
looping over all physical hosts in an environment and pulling address
information out to store as a task variable. This action causes all
tasks to reload all {group,host}_vars for all hosts and that repetitive
action results in a massive slowdown, especially when there are many
hosts within the inventory.
Results from a normal OSA playbook run on 50 hosts:
time openstack-ansible openstack-hosts-setup.yml --forks 15
real 37m44.263s
user 35m30.984s
sys 2m23.252s
Results with the strategy plugin disabled on 50 hosts:
time ANSIBLE_STRATEGY_PLUGINS=false openstack-ansible openstack-hosts-setup.yml --forks 15
real 5m8.641s
user 6m30.792s
sys 1m32.512s
This change scopes the physical host look up to the task and target host
and adds conditionals around the process to ensure we're not forcing a lookup
that has no chance of succeeding.
Results with this patch applied on 50 hosts and no forks:
time openstack-ansible openstack-hosts-setup.yml
real 6m40.573s
user 7m53.336s
sys 1m25.828s
Should anything like this come up again debug statements have been added
to the method so that we can enable verbose mode and know what is being
parsed.
Change-Id: I76af71ffe19b2a8afb18ed731f1362a4e399c613
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
In the scenario where the fallback group 'all' is used to find the
physical host address, there are typically containers present in this
group with ansible_host settings like "{{ container_name }}".
When this happens, you get physical_host_addrs lists like:
aiotest1 127.0.0.1
test1 1.1.1.1
test-1-zbx1 {{ container_name }}
test-1-cdns1 {{ container_name }}
test-1-lb1 {{ container_name }}
test-1-lb2 {{ container_name }}
fatal: [aiotest1]: FAILED! => {"msg": "'container_name' is undefined"}
To avoid this we need to run the physical_host_addr through a templar
loaded with the physical_host_vars:
aiotest1 127.0.0.1
test1 1.1.1.1
test-1-zbx1 test-1-zbx1
test-1-cdns1 test-1-cdns1
test-1-lb1 test-1-lb1
test-1-lb2 test-1-lb2
test-1-tpxy1 test-1-tpxy1
ok: [aiotest1]
Change-Id: I7a4e89c579a3e688decaeb91b9035500129c7b16
On ansible 2.4+, when 'ansible_host' is defined in the inventory
ini file, the physical host mapping in the strategy plugin works as
expected. However, if the ansible_host setting is in host_vars/<host>.yml,
then the mapping does not work.
Change-Id: I22c09c4de86233329f337b535b709cd5320c40d3
For 'physical_host_addrs' to be able to be looked up by the connection
plugin it will need to be included in the 'task_vars' dictionary which
is eventually passed to the task executor. Moreover, we remove the now
obsoleted[1] 'set_host_overrides' function.
If the 'container_name' variable is available, it should take precedence
over 'inventory_hostname' for determining a container's name.
The 'container_user' variable is now set through 'set_options' instead
of a host variable and, when it is available, the connection plugin's
shell is initialized with a system writable temp directory to avoid
requiring additional privileges for the user.
The container check has been updated to also check 'container_name'
against 'ansible_host'. If these match, the container is only known by
an IP address and will require a direct SSH connection.
[1]: 23b1dbacaf
Co-Authored-By: Markos Chandras <mchandras@suse.de>
Change-Id: I16bc76f17ae5d2da4803fd7cdcdcfe72e77f9df1
This exposes the container_user setting to an operator and allows them
to override it. This is helpful when you want to use 2 different
username for your physical_host user and container user. Today if the
physical_host user is root, the container user will always be root
too.
Change-Id: Ie8e32b0441a5d72fcbc3cbf9769d8a7dc7548f08
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Previously, the MAGIC_VARIABLE_MAPPING constant was updated to make
additional OSA required hostvars available to the play_context. In 2.5,
MAGIC_VARIABLE_MAPPING has a different location but updating it is no
longer necessary. Connection plugins are now able to access any hostvars
and options defined within their docstrings. The upstream connection/ssh
plugin's docstring has been copied here with the addition of OSA's
required options.
Change-Id: I6b1bcd4ec5e9d2701bd26e24e0f7645629ae4f3d
This change modifies our connection plugins to allow us to use nspawn
type containers. The change will pull in the container_type variable and
set the padding for commands accordingly.
While the module has been setup to default to LXC which ensures we've no
conflicts with existing deployments, the container_check function has been
set to ignore "nspawn" type containers until we have a suitable/reliable
method to connect to them without envoking ssh.
Change-Id: I8877c244af6458ca9840ea0519ab457a726feefd
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The inventory lookup to determine the map of physical_host ->
ansible_host vars was never locating ansible_host, and instead
always defaulted to physical_host's inventory_hostname.
This fixes the lookup so the inventory_hostname will only be used
as a fallback if ansible_host is not set on the physical_host.
Change-Id: Ibbadbaf2a6a5b7323ee6587da3773b8459fa2eb6
Tasks which delegate from a container to a container on a separate host
currently fail because the 'physical_host_addr' variable, which those
tasks use to connect to a physical host, is exclusively set as the
address of the originating container's physical host.
Assign a new host variable, 'physical_host_addrs', which contains a
mapping of each physical host and its address, so the tasks can now
lookup of the correct address to use depending on the target container.
Change-Id: If594914df53efacc6d5bba148f4f46280f5a117d
Within the strategy plugin set a host var, 'physical_host_addr', which
corresponds to the 'ansible_host' var of the host's 'physical_host'.
Use this within the connection plugin rather than the hostname of the
physical host, which may not exist in DNS or the deployment host's
hosts file.
Closes-Bug: 1695944
Change-Id: I75f9d0f55ecd875caa1bf608a77c92f950b679a1
The strategy plugin previously did not support container detection
when the delegate_to host was a container. This adds support for
delegating tasks to other containers, so the strategy will perform
the same container vars lookups so the SSH connection is toward the
physical host of the container, and the command is wrapped with
lxc-attach based on the container_name.
Co-Authored-By: Jimmy McCrory <jimmy.mccrory@gmail.com>
Change-Id: I56d8afddbccf01f2944d2fdd505b601a4b048374
Renames the "artifacting" strategy to "tagfilter" and implements
a more generic tag skipping implementation that can be defined
from within the ansible vars structure dynamically on a per-host,
per-group, or per-playbook basis.
To use the tagfilter strategy, run the playbook with
strategy: tagfilter
and define a hostvar named "skip_tags". Skip tags can be a string
or a list, and any tag CONTAINING a skip tag (an exact match is not
needed) will be skipped.
Also added is a check for a var called "skip_handlers", defaulted
to True. When skip_handlers is enabled, no handlers will be run
regardless of whether the task is skipped or not. If set to False,
handlers will be allowed to run as normal except when tasks are
skipped by the skip_tags.
Change-Id: I649707e6744e03763c4e786d203716ebf657ab48
Ideally Ansible should allow (from a playbook) to run only part
of a role thanks to its tags (like the --skip-tags or --tags from
the cli). But it's not possible now.
This code is a PoC code for introducing a kind of filtering,
which basically skips tasks containing the word "config".
If you are building lxc images ("artifacts") of a role, you
probably don't want to run all the tasks in the role, and don't
want to run the handler tasks (because they start software for
example). This also addresses it.
To use this work, just include strategy: artifacting in your
play.
Change-Id: I006bc640c6563c959ceb835ddf5bef8d25dd7517
In some situations it may be useful to execute a role or
set of tasks against a remote chroot.
This patch aims to add that capability.
Change-Id: Icebec389807aa8782f97a95f63687eb5606f7b22
This changes 'ansible_ssh_host' to 'ansible_host'. The 'ansible_ssh_host'
variable has been deprecated as noted here: [0].
[0] - http://docs.ansible.com/ansible/intro_inventory.html#hosts-and-groups
Change-Id: I39892435950eca6da773dd947894da2125eab147
Related-Bug: #1636606
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The physical_host detection in the linear strategy required
ansible_ssh_host be set on the physical host. Since ansible_ssh_host
is now deprecated, we will accept both the new var ansible_host and
the old var ansible_ssh_host.
Change-Id: I3504c1901cf8f9f6f60c5465b55269997955635e
The role base test use a much simplified inventory which this plugin
was not accounting for. This resolves the assumption.
Change-Id: I995b62cbf825f15a93e3bb44f48b3527c9e5ba48
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This change is creating an opportunistic Ansible execution strategy and
an update the ssh connection plugin so that it supports direct access to lxc
containers within ever having to ssh into them.
The intention of this change is to speed up execution time and reliability by
tuning the execution environment within Ansible to run faster while also attempting
to subvert transient ssh issues.
Change-Id: Ide34513bf82523257bdd2a8a68dff165f9927c56
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>