As a first step towards supporting multiple ansible versions we need
tooling to manage ansible installations. This moves the installation
of ansible from the requirements.txt into zuul. This is called as a
setup hook to install the ansible versions into
<prefix>/lib/zuul/ansible. Further this tooling abstracts knowledge
that the executor must know in order to actually run the correct
version of ansible.
The actual usage of multiple ansible versions will be done in
follow-ups.
For better maintainability the ansible plugins live in
zuul/ansible/base where plugins can be kept in different versions if
necessary. For each supported ansible version there is a specific
folder that symlinks the according plugins.
Change-Id: I5ce1385245c76818777aa34230786a9dbaf723e5
Depends-On: https://review.openstack.org/623927
The raw module had not been restricted to remote nodes so jobs could
run arbitrary code on the executor.
Change-Id: I1b37eac65ef59ca749f55117a678c38969e86ead
When using add_host on localhost (zuul-executor) we also need to add the
ssh known_host entry, otherwise when a play tries to use the new host
ansible will fail to connect since we try to validate SSH host keys by
default.
Change-Id: Ifc99f57085ab4e4ed022e411db77965673c6dbcf
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
There are a few missing commas in our sage_args, this fixes them.
Otherwise we get the following error from zuul:
Adding hosts ssh with ansible_password ansible_user to the inventory is prohibited
Change-Id: I6162bc6a223da54af26bedd2e950ed21a64908b0
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Since commit d07bc25fc2, it is possible
for an untrusted playbook to execute commands on the executor host.
This change restores the add_host restriction and white-lists the
intended use case.
Change-Id: I36cc604c62a50c95260d076a63a53f28b197792d
There's actually not anything unsafe about add_host. Doing CD from Zuul
requires being able to add hosts, so relax the restriction.
Change-Id: I7a5992808773722f3b81890fb4193da202cfea68
Our filtering for the synchronize ansible module didn't account for
using delegate_to. We now fix this and add some test coverage for
validation.
Change-Id: I33b1fe259454ec32e783d2ffe6aa5e6b73f6fea0
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Whitelist zuul_return to allow untrusted jobs to run the task on the
executor (localhost). Otherwise, only trusted jobs are only able to
use it.
Change-Id: I768394251d7a2ee102883694bfc93845254e8514
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Currently the lookup plugins are only permitted to read data
from the work dir. Some of them expect to read files based on
the CWD which Ansible sets to be the playbook directory. In the
case that the lookup plugins are being run by a playbook in a
project which is in the dependency chain, that will work fine.
But if the playbook project is not in the dependency chain, it
is checked out into the "trusted/" directory in the jobdir, which
is outside of the work dir. In this case, the lookup plugin will
fail.
Alter the lookup plugins to permit access to the "trusted/"
directory as well as "work/". This is safe because these plugins
only read files, not write. And the only content in the "trusted/"
directory is git repository checkouts which the user already has
access to.
Also allow the include_vars and synchronize action modules access
to read files from these directories.
The current tests are sufficient to show that current behavior is
not broken by this change. A followup change moves all playbook
project checkouts outside of the work directory, so that case will
be tested then.
Change-Id: Ie2e5b0d1c099d4f9cf59c1e67fb0603d7c5f757b
Ansible 2.4 adds a bunch more network action plugins. Prevent them all
from running.
It also adds three new lookup plugins, none of which are suitable.
2.4 also added a "telnet" action plugin. It just telnets somewhere, so
it doesn't seem like we need to do anything for it.
Change-Id: Id48d238f6bb3301597b52b841240bc3470182a94
Currently we don't check some modules for delegation to
localhost. This would make it possible to overwrite any data which is
writable within the bwrap context. Further the script module allows
arbitrary code execution when delegated to localhost.
The following modules are affected:
* assemble: add safe path check
* copy: add safe path check
* patch: add safe path check
* script: block completely
* template: add safe path check
* unarchive: add tests, fixed by safe path check of copy
Change-Id: I2360219f50e6a28bb134468ead08ec72148ad192
Story: 2001681
Currently the safe path checks mostly just work on the pure given
arguments. However ansible searches in several paths for the file and
uses this then. Thus the safe path check can validate a non existing
file (as it doesn't obey the modules search path) while the module
finds the file and follows any symlinks.
The solution for this is not to try to determine the correct paths
ourselves but hooking into the search routine (_find_needle) the
modules use for finding their local files. Using this approach we have
the correct paths and can validate them properly.
Change-Id: I2b3dbcfcfcf8a82e4e6df286cc3287aaa7fa2790
We want to block people from running get_mime on localhost. However,
sometimes people also explicitly want to not run get_mime. We should let
them express that.
Change-Id: I2b71d5532a3cd976610d6681611f03c41af7c833
The get_mime option may be used to abuse the file utility.
This change disables this module argument.
Change-Id: Idc3bf8d101a15f572841b504ef16335281079142
The normal action plugin was passing module_name when the code path
actually needed the module object itself.
Change-Id: Idb8968a7020ffb159d0d016b5bfab6bd88219c70
We greylist some modules in our action plugin blocking allowing them to
execute local code as long as it falls within safe constraints. Due to
the way ansible module loading works, a user could attack this by
creating a module in a local role or adjacent to a playbook that has the
same name as one of the modules we allow limited local execution. If
they did that it would allow them to execute arbitrary python code on
the executor.
Find the path of the module that will be executed in these cases and if
it is not within the ansible.modules package, disallow it. There are no
circumstances in which this is ok.
Change-Id: I7499e6b1091d745984ca36179de2793827c9f98f
The rtfd hook job just does an empty POST to a URI. There's no need to
allocate a node for that, we can just make REST calls from the executor.
Also, there is enough going on here that it needs to be documented. Add
a documentation section to the developer docs about what we're doing
with our ansible plugins. In support of that, add a simple sphinx domain
for ansible to allow us to easily link to upstream ansible documentation for
modules.
Change-Id: I9b0be1018388db7361aec10f30a70437de555615
python3 includes support for optional type annotations which can be used by
static analysis tools to perform type checking. The mypy tool is a
static type checking tool that can also infer type information in many
cases, but which will use explicit type information if it is present.
Add mypy to test-requirements and to the pep8 job so that our pep8 job
can do more analysis work and less with the code style.
To support this, there were a few places in the current codebase that
needed an explicit type hint. For variables/attributes in 3.5 this is done via
comments. There is a conditional import that was confusion that just got
marked with an 'ignore'.
Our ansible action and lookup plugins confuse mypi with the way they
import the ansible base classes. That's ok - they confuse us with that
too. The .pyi files are 'typeshed' files, which are a way that one can
provide static type annotations without putting the information into the
file itself. mypy will always prefer a .pyi file over a .py file (since
the point of them is to be external annotion/interface description) So
in order to get mypy to not barf on the ansible import weirdness, just
add a corresponding empty .pyi file. We could potentially actually put
interface descriptions in them - but I don't think there is very much
value in that.
It should be amusing to at least someone that we have to flake8: noqa
an import from typing that was done to provide a type hint in a comment.
Change-Id: I6c4ac3dcfc6fd990e6c6886749de147ad28389d1
We want to properly flex our bubblewrap implementation, this job does
so.
Change-Id: I6647d71434a8d8f6621d3fd34883683ef149775a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We incorrectly used the 'pull' argument from synchronize, the correct
setting is 'mode'.
Change-Id: I3cf53aa35f255a3cab68271d73d0414cd83b70f4
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We don't want people to rsync files from outside the work dir by
creating symlinks and then synchronizing them. Pass in --safe-links to
rsync. rsync's behavior is that if --safe-links is present in the
presence of other options to the contrary that --safe-links will win.
Change-Id: I0e9657412edf20f0138d6fb7da994f4804f77826
Ansible imports our module with the name ansible.plugins.action.normal
(regardless of its actual path). This line would then import the original
module, also under the same name, so it would overwrite ours. Correct
this by importing the original with a unique name. It's not actually
used anywhere anyway.
Change-Id: I5dd26e2f89399f991cebc23839a36e7aff47056c
We want to override the base action plugins. We can't do that if we try
to import non-existent locations in the zuul tree.
Change-Id: I6242d973d4ce3b42bcec2812ba261ac6f968fcf4
There are actions undertaken by action plugins in normal ansible that
allow for executing code on the host that ansible is executing on. We do
not want to allow that for untrusted code, so add a set of action
plugins that override the upstream ones and simply return errors.
Additionally, we can trap for attempts to execute local commands in the
normal action plugin by looking at remote_addr, connection and
delegate_to.
Change-Id: I57dbe5648a9dc6ec9147c8698ad46c4fa1326e5a
Tobias Henkel