As a first step towards supporting multiple ansible versions we need
tooling to manage ansible installations. This moves the installation
of ansible from the requirements.txt into zuul. This is called as a
setup hook to install the ansible versions into
<prefix>/lib/zuul/ansible. Further this tooling abstracts knowledge
that the executor must know in order to actually run the correct
version of ansible.
The actual usage of multiple ansible versions will be done in
follow-ups.
For better maintainability the ansible plugins live in
zuul/ansible/base where plugins can be kept in different versions if
necessary. For each supported ansible version there is a specific
folder that symlinks the according plugins.
Change-Id: I5ce1385245c76818777aa34230786a9dbaf723e5
Depends-On: https://review.openstack.org/623927
Currently the lookup plugins are only permitted to read data
from the work dir. Some of them expect to read files based on
the CWD which Ansible sets to be the playbook directory. In the
case that the lookup plugins are being run by a playbook in a
project which is in the dependency chain, that will work fine.
But if the playbook project is not in the dependency chain, it
is checked out into the "trusted/" directory in the jobdir, which
is outside of the work dir. In this case, the lookup plugin will
fail.
Alter the lookup plugins to permit access to the "trusted/"
directory as well as "work/". This is safe because these plugins
only read files, not write. And the only content in the "trusted/"
directory is git repository checkouts which the user already has
access to.
Also allow the include_vars and synchronize action modules access
to read files from these directories.
The current tests are sufficient to show that current behavior is
not broken by this change. A followup change moves all playbook
project checkouts outside of the work directory, so that case will
be tested then.
Change-Id: Ie2e5b0d1c099d4f9cf59c1e67fb0603d7c5f757b
Ansible 2.4 adds a bunch more network action plugins. Prevent them all
from running.
It also adds three new lookup plugins, none of which are suitable.
2.4 also added a "telnet" action plugin. It just telnets somewhere, so
it doesn't seem like we need to do anything for it.
Change-Id: Id48d238f6bb3301597b52b841240bc3470182a94
This is not complete, but it's a start.
While doing this it became clear that the upstream csvfile lookup plugin does
not work in python3. Work around that by putting in a simpler version of
the code into our copy.
Change-Id: Ic84d8265e6fd7e15a0e5d66c781409a087d761d7
python3 includes support for optional type annotations which can be used by
static analysis tools to perform type checking. The mypy tool is a
static type checking tool that can also infer type information in many
cases, but which will use explicit type information if it is present.
Add mypy to test-requirements and to the pep8 job so that our pep8 job
can do more analysis work and less with the code style.
To support this, there were a few places in the current codebase that
needed an explicit type hint. For variables/attributes in 3.5 this is done via
comments. There is a conditional import that was confusion that just got
marked with an 'ignore'.
Our ansible action and lookup plugins confuse mypi with the way they
import the ansible base classes. That's ok - they confuse us with that
too. The .pyi files are 'typeshed' files, which are a way that one can
provide static type annotations without putting the information into the
file itself. mypy will always prefer a .pyi file over a .py file (since
the point of them is to be external annotion/interface description) So
in order to get mypy to not barf on the ansible import weirdness, just
add a corresponding empty .pyi file. We could potentially actually put
interface descriptions in them - but I don't think there is very much
value in that.
It should be amusing to at least someone that we have to flake8: noqa
an import from typing that was done to provide a type hint in a comment.
Change-Id: I6c4ac3dcfc6fd990e6c6886749de147ad28389d1
Some of the lookup plugins access files on the executor host. Obviously
that's not what we want, so block them like we block action plugins.
password.py is banned, although it could be filtered. However, the
upstream code is fairly intense and slated for refactoring - so let's
wait until someone gets upset about it.
Change-Id: I6260b4658619a972b588c8bfba40ec33557bf2f6
Tobias Henkel