Several packages in calling "python setup.py --name ..." may return
warning message, e.g.:
...
_DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!
********************************************************************************
Requirements should be satisfied by a PEP 517 installer.
If you are using pip, you can try `pip install --use-pep517`.
********************************************************************************
!!
dist.fetch_build_eggs(dist.setup_requires)
WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section'
<PACKAGE_NAME>
and then this huge log places into the package_name variable. But
script expects that package_name will contain only package name. Because of this situation could not find siblings packages and
install them.
Change-Id: I5bf9a19233c48d1260b5ab17d749bfc58a8ef2fa
Tox 4 released today and is a complete rewrite with many backward
incompatible changes. We need to update a number of things to support
that in the zuul-jobs tox role and elsewhere. A possibly incomplete
list of what was changed in this commit to make this work:
* Don't run tox --showconfig with {{ tox_extra_args }} as -vv is
in tox_extra_args by default and results in interleaved debug
output in the ini output making it invalid ini content.
* Update the tox siblings tox config parser to look for renamed
environment dir locations.
* Stop using whitelist_externals and use allowlist_exteranls
because whitelist_externals is removed and external commands that
are not explicitly allowed produce errors.
* Make the tox version configurable in ensure-tox as some users
may not be able to easily upgrade to tox v4.
* Escape literal # chars in tox.ini as they are treated as comments
when in the command strings now.
https://github.com/tox-dev/tox/issues/2617
Change-Id: I38e13b4f13bb1b2d6fb7e5c70b708e9bb016a455
This commit in Ansible:
9142be2f6c
now allows Python modules to specify their interpreter with the shebang.
We expect our roles to use the discovered python interpreter on remote
nodes, and on the executor, we need them to use the virtualenv. Removing
the specific shebang accomplishes this under Ansible 6, and has no effect
under older versions of Ansible.
Without this, for example, the log upload roles would not have access to
their cloud libraries.
Also update our ansible/cli check in our module files. Many of our modules
can be run from the command line for ease of testing, but the check that we
perform to determine if the module is being invoked from the command line
or Ansible fails on Ansible 5. Update it to a check that should work in
all 4 versions of Ansible that Zuul uses.
Change-Id: I4e6e85156459cca032e6c3e1d8a9284be919ccca
I noticed this by accident when I ran ansible-lint over this repo from
an outside context; it didn't use the .yamllint in here and started
compalining about eof whitespace.
After scratching my head for a bit as to why this didn't fail here, I
realised we've allowed various newlines since the initial commit
I936fe2c997597972d884c5fc62655d28e8aaf8c5.
Remove this and just use the default eof rules, and fixup the
whitespace as required. This is fairly unimportant, but is nice for
consistency.
Change-Id: Idb46a1f39ba798b0bf70eaa27b4c6b4758ce3d26
This is preparation for a later version of ansbile-lint, which finds
missing names on blocks. This seems a reasonable rule, and the
Ansible manual says [1]
Names for blocks have been available since Ansible 2.3. We recommend
using names in all tasks, within blocks or elsewhere, for better
visibility into the tasks being executed when you run the playbook.
This simply adds a name tag for blocks that are missing it. This
should have no operational change, but allows us to update the linter
in a follow-on change.
[1] https://docs.ansible.com/ansible/latest/user_guide/playbooks_blocks.html
Change-Id: I92ed4616775650aced352bc9088a07e919f1a25f
This fixes a number of places where we do not have spaces between
filters. I think that this is a reasonable rule for readability (I
also think it probably was enforced, but maybe later versions got
better at detecting it?).
These are detected by a later version of Ansible lint; this change
should have no operational change to any roles but prepares us to
update in a follow-on change.
Change-Id: I07e1a109b87adce86f483d14d7e02fcecb8313d5
In 2021 there was a change [1] in tox role default values.
However, the readme was not updated as part of this change.
This commit adds the missed note.
[1] https://review.opendev.org/c/zuul/zuul-jobs/+/807702
Change-Id: Ifa5b37c854eddbcda677ec5cd67a59bc3384ba34
The below error is coming on mol-centos7 jobs while
installing sibling for python2 packeges on centos7
Error: TypeError: initial_value must be unicode or None, not str
This patch handled the TypeError while installing sibling
for python packeges for centos7.
Closes-Bug: #1946641
Change-Id: Ie8058cca92d099e50af19b95b4c417c5a665da0d
With the original verbose output support in change
Iafeb88eaf9a596603ad4d2134a4574345d5189ab we looked for lines from
tox --showconfig output starting with an opening '[' but verbosity
also causes output from pip install activity to be included if a
tox.requires entry causes it to install packages before continuing
and these lines are prefixed by a process ID number wrapped in
brackets with the command string after that, which prematurely
triggered our search for the start of the INI content. Add a
stipulation that the first INI output line also end in ']' in order
to skip over those additional prepended lines.
Change-Id: If29e5a9abe3b92a145d87f5efc1b93350ea3908a
This reverts commit 8b1cc73ee3.
Apparently tox.requires adds lines like this to verbose showconfig
output:
using tox-3.24.4 from /.../tox/__init__.py (pid 2919)
Which is confusing the current parser in the siblings module. Roll
this back while we work on a fix and regression test.
Change-Id: If3b1d48b36a5d32fddfdabd9c0ec1b81dd6453f2
When the tox role was introduced, a tox_extra_args rolevar was
included allowing the tox command line to be extended with arbitrary
options. When siblings functionality was added, tox_extra_args did
not get included in its separate tox invocations. If a project has a
particular situation where some aspect of tox's functionality must
be overridden in order to work, doing so through tox_extra_args
needs to apply to every tox invocation, including siblings installs.
Change-Id: Ibfe77f67e43135ae5af7588d6859b8b3dbd4c3ca
Unfortunately, when tox combines --showconfig with verbosity options
like -vv, some non-config output gets streamed to stdout before the
configuration is emitted. Filter this preamble in
tox_install_sibling_packages by discarding any initial lines of
output before the first section heading.
Also extend get_envlist() to deal with the fact that additional
verbosity adds a [tox] section in the --showconfig output, which it
was previously relying on to determine whether the config had been
filtered to a subset of env sections. Instead also check the
tox.args string to determine whether a -e option was passed on the
command line.
Change-Id: Iafeb88eaf9a596603ad4d2134a4574345d5189ab
In some situations, projects may not keep a tox.ini in the root
directory of their repository, or may even have multiple tox
configuration files. Allow the location and name of the config to be
overridden explicitly through the use of a new rolevar.
Change-Id: I1927142e6d9fa75e96902ae001c8ca98d69c7443
Avoid false-positive CI testing with tox where misconfigured tox
projects end-up skipping tests or running with different python
version than the required one.
While use of this option on development machine may be ok, when
executed in CI context, we never want to be relaxed about what
we test and which versions of python we use.
I seen projects running with wrong version of python for months
before someone discovered that a different version of python
was used on CI.
Change-Id: I5be9bce86833db11afd7072e477ccaf42658bf99
As of version 3.21.0, tox now throws an exception if an environment variable
used in config is missing. We now need to pass the variables also to the task
"Get tox envlist config".
Change-Id: I4073f55b6ba3456f3d2db40cafb692efbb478914
This allows projects, that use tox but may not have a setup.cfg file
still use tox siblings. We do this to allow non-python project, to use
tox as an entry point for testing, and still have depends-on
requirements work in zuul.
Change-Id: I9b37117b27ff6b7e436d456b6cbae39ccb9b968c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
There's a couple of places that call subprocess.check_output that
might raise and exception (I'm seeing one finding the "--name"); catch
the CalledProcessError and include it's output to give more info.
Change-Id: I53cfb9c1b37bd4dbf9e5bf04fbab83d3cdad795d
Setting tox_envlist to venv by default is unintuitive for
many users. Remove this behaviour and let default tox
behaviour be the same as running tox on the commandline.
Change-Id: I1b6d59ee4ebb7f6b3adcf4bd35d7148e83389008
siblings: python2.7 ConfigParser has no __getitem__
Constrain soupsieve to <2 for python2.7
Add python2-dev to bindep for subprocess32
Change-Id: If9d6a0ae1a62a94dcec11f6bf637ffee7f0f4fc9
Also we can parse the output from --showconfig to se which
testenvs we should act upon when installing siblings.
Change-Id: I438035c92d62102c80db9bed81e9f290e4c941b2
Since tox_envlist has a default value it cannot be undefined
so the fail task will never run. Instead handle the case when
tox_envlist is an empty string by getting the default configured
envlist from tox. Also handle the casewhen tox_envlist is 'ALL'.
This also updates tox_install_sibling_packages to correctly
handle multiple testenvs and uses configuration supplied by
'tox --showconfig -e <envlist>' instead of guessing where the
envdir and logdir are located.
We also cannot run tox inside python because it gets complicated
to know which tox_executable we should call during the python test
cases so run these commands in ansible and pass the output to
tox_install_sibling_packages.
Since role params have higher precedence than set_fact we set an
internal _tox_envlist fact that is a comma separated list of testenvs
that should be run.
Change-Id: I9e5a1b041f653cbcff7b8ed62e4a95a0a040fdd7
This reverts commit 53f2444dbb.
Revert to avoid regression:
https://review.opendev.org/710208 just gave a lot of new linting warnings that look new "py38: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()"
Change-Id: I26c862255cce6b6c99a69d9e3f5f4354db90f953
This is needed to be able to parse output from ansible-lint.
Removes a testcase with a faulty assumption that if the column is
not present the output must be a warning and should not be returned.
It is probably better to allow the user to configure their linting
output themselves.
Also we don't need a separate matcher for sphinx.
Change-Id: I6d06dce921348d3c6fb5a56cdc1e4df675d685c2
Adds yamllint to the linters with a minimal configuration, some
rules are disabled to allow us to fix them in follow-ups, if
we agree on them.
Fixes invalid YAML file containing characters inside block.
Fixes few minor linting issues.
Change-Id: I936fe2c997597972d884c5fc62655d28e8aaf8c5
The sphinx regex is picking up python warnings. We can disambiguate
the two because sphinx doesn't emit a space before the message but
the python warnings module does. This adjusts the regex to do that
and adds a test file that includes warnings output.
If we need/want to make this more robust, we could also simply filter
the sphinx output for messages that start with " ?\w+Warning:".
We could also decide that we want to include python warnings.
Change-Id: Ib58cc1a2d4f673a0ce3d2a9cae306004559f5b52
We get many false positives on the .tox directory, ignore it.
Even better would be to only handle files under version control but that
is more involved, so use the simple heuristic for .tox for now since
that catches 90+ per cent of the cases.
Change-Id: Ibb17958454a4d6bf156020fe4d9f588f3c666cc7
After stripping the work dir there should not be any absolute path
left that can be related to the repo. Thus filter them so they don't
produce mapping warnings in zuul.
Change-Id: Iadeac9b1faea63aa6009e2cfbc215853695abbe6
Roman Kuznecov