In 67645a46eb the requirement on
setuptools was removed in favor of the version implementation in
python-jenkins. However, other code (regsitry.py, xml_config.py) is
still using pkg_resources, which leads to errors like this:
% jenkins-jobs
ERROR:stevedore.extension:Could not load 'delete': No module named 'pkg_resources'
ERROR:stevedore.extension:Could not load 'delete-all': No module named 'pkg_resources'
ERROR:stevedore.extension:Could not load 'get-plugins-info': No module named 'pkg_resources'
ERROR:stevedore.extension:Could not load 'list': No module named 'pkg_resources'
ERROR:stevedore.extension:Could not load 'test': No module named 'pkg_resources'
ERROR:stevedore.extension:Could not load 'update': No module named 'pkg_resources'
Traceback (most recent call last):
File "/tmp/tmp.qQ9GsQCSeH/venv/bin/jenkins-jobs", line 8, in <module>
sys.exit(main())
^^^^^^
File "/tmp/tmp.qQ9GsQCSeH/venv/lib64/python3.12/site-packages/jenkins_jobs/cli/entry.py", line 177, in main
jjb = JenkinsJobs(argv)
^^^^^^^^^^^^^^^^^
File "/tmp/tmp.qQ9GsQCSeH/venv/lib64/python3.12/site-packages/jenkins_jobs/cli/entry.py", line 60, in __init__
self.parser = create_parser()
^^^^^^^^^^^^^^^
File "/tmp/tmp.qQ9GsQCSeH/venv/lib64/python3.12/site-packages/jenkins_jobs/cli/parser.py", line 112, in create_parser
extension_manager.map(parse_subcommand_args, subparser)
File "/tmp/tmp.qQ9GsQCSeH/venv/lib64/python3.12/site-packages/stevedore/extension.py", line 276, in map
raise NoMatches('No %s extensions found' % self.namespace)
stevedore.exception.NoMatches: No jjb.cli.subcommands extensions found
I could only reproduce this on Python 3.12, as on 3.11 and earlier a
`python -m venv` installs setuptools already (to bootstrap pip?).
Fixes: 67645a46eb
Change-Id: I679bbfd4b178094585587b5f47d56da37c663b5c
Signed-off-by: Evgeni Golov <evgeni@golov.de>
LegacyVersion class is removed from newer setuptools package. But
support for legacy versions is added in python-jenkins 1.8.2.
Switch to that implementation.
Fix broken plugin version comparison for legacy versions.
Assume latest plugin version if no plugin version is found.
Story: 2010990
Story: 2009943
Story: 2009819
Story: 2010842
Task: 49236
Task: 44852
Task: 44396
Task: 48448
Change-Id: Id7f0be1c42357454bd9bedcdee3fefb174943d81
It lools like versions prior to 3.13 do not support
Python 3.7 nor Python 3.8.
Compatibility with PyYAML versions 3.13 and 6.0 are
checked with Python 3.7 and 3.8 manually (locally)
with `tox -e py37/py38 -r`.
Change-Id: Ie53d82a88498cc5650118d275496970ad6b4a6eb
Starting from version 66.0.0 setuptools has dropped support for
LegacyVersion. But many Jenkins plugins still provide versions in
legacy format.
Default setuptools version is changed recently on OpenStack
infrastructure, so we need to fix it now.
Change-Id: I3c6029e89a44d2c552f58336859249ed5fe958f5
PyYAML v4.1 was released which breaks the `!!python/tuple` syntax with the
following error:
yaml.constructor.ConstructorError: could not determine a constructor for the tag
'tag:yaml.org,2002:python/tuple'
Change-Id: Iaf838b111e6dfad5dc5c1f904a482b525e99a911
Bumps python-jenkins version to avoid few
remarkable issues fixed in it:
- allow to wait for jenkins to enter normal operation
- remove cap on pbr version
- more py3 enablement
- other, see its changelog
Change-Id: I508b412bc2d17561e7a7ceef015ea018615596ad
Allows specifying a folder attribute for each job generated, which in
turn is used when creating or uploading to place the job under the
requested folder.
The job name is expanded after defaults are applied, to support the
attribute being defined within a set of defaults applied to a number of
jobs.
This in turn allows for multiple jobs with the same basename to exist,
provided they are targeted at different folders.
Does not support creating the folders if they do not exist.
Change-Id: I8c2157c4c81087cc972a048d1b88d5f08ac65361
This template included using !include-jinja2:
"""
{{ my_var }}
"""
is rendered the same as an existing template that looks like this:
"""
{my_var}
"""
This also allows the use of Jinja2's richer syntax:
"""
{% for test_environment in configuration.get("envs", ["py35"]) %}
tox -e {{ test_environment }}
{% endfor %}
"""
Story: 2001135
Change-Id: Ia3ee21822d6e9237f5ea46796bc8810ecac61e2c
When a jjb run is thrown when another jjb is already running, it can
cause corruption of cache. Start using a fasteners to ensure this
won't be happening and run securely on automated systems.
Ensure unlock is called from only the destructor, so that it is only
called when the JJB process is guaranteed to be finished using it.
Make it obvious that _lock is intended to be internal to the cache
storage implementation.
Potentially we may not need to call it at all, as python might unlock
it for us on exit by closing the file when no longer needed. However
better to make it explicit.
Change-Id: I53a1f92cf2bfbbe87c9ea205c377f93869353620
This is not necessary - jenkins-job-builder does not follow the
requirements sync process. That said - these are likely all safe bets
for being solid versions of these dependencies - and some of them were
kinda stale.
Change-Id: Id1941ff4c9736100f9bdd5508ec3c0377e51fdbe
If the stevedore is limited to 1.8.0 and the version of stevedore
in host is higher than 1.8.0, it will raise an 'VersionConflict'
error when executing 'jenkins-jobs update` command.
Change-Id: I9e83fddd82049be0d21ec4106f7b846184f3af8d
Closes-Bug: #1614435
This commit intentionally introduces a number of important API
breakages. Specifically, the jenkins_jobs.cmd module has been pared
down to some of its most difficult-to-refactor elements.
* Create jenkins_jobs.cli.entry.JenkinsJobs class to organize command
line parsing and execution.
* Remove references to ConfigParser object in test code, hidden as an
implementation detail of JenkinsJobs command line parsing. This will
be necessary in the next stage of JJB 2.0 code which will be to
create a JJBConfig object that handles logic and presentation of
configuration from various sources--defaults, command line
arguments, configuration file, and maybe environment variables in
the future.
* Remove references to Namespace object produced by argparse module.
Required rewrite of multipath & recursive path tests with a new
MatchesDir testtools Matcher class that validates the expected
output for a run of JJB against a given set of yamldirs with the
specified command line arguments.
* Use stevedore to dynamically load subcommand parsers.
* Move configuration loading/testing to its own test file. Also fix
the global vs home directory JJB config file test.
Change-Id: If62280418ba7319c313033ab387af4284237747e
python 2.6 is no longer supported and adding a marker
to requirements.txt for installing ordereddict only in py2.6
envs was not an option because markers are not supported in
older pip versions.
So remove python 2.6 support completly.
Change-Id: Iebdd999b469c9a9681b0d7e9f50cc488a8820953
Add separate linkcheck tox environment to be used in non-voting Jenkins
job to watch for stale and broken links in the sphinx generated html
documentation. Include a small addition to make it easier for
contributors to select individual builders when executing the docs tox
environment.
Add minimum dependency of pbr 1.0.0, as that was the first version to
allow passing of sphinx builders on the command line to override the
internal defaults or those set in the configuration file.
This will make it easier to spot problems with just the links, and also
ensure that when the sphinx project releases with proper warniserror fix
for the linkcheck builder, that html and man page documentation can
continue to be built even if a previous valid link has become stale or
is temporarily unreachable.
Change-Id: I80446abe96824a444a04cc526df6cc38632a4607
delete-all command was deleting one job at a time using the jenkins
doDelete rest endpoint which is very slow. Change Ia4fbfca970165
allows us to execute a groovy script on the jenkins server to
do the same thing. The only difference is that running the script
on the server will delete all jobs much faster.
As a data point:
I added all (~5,000) openstack jobs to my jenkins server.
The previous delete_all took many hours to delete all the
jobs while the script method did it in a matter of seconds.
Change-Id: Id7f7dfb567997e042fe0f783e54680482fefdc15
depends-on: Ia4fbfca970165d890d7e076f47ddcde7633afa9b
This patch aims to pin pbr>=0.8.2,<2.0 instead of pbr>=0.8.2,<1.0.
Some distros ship 1.0.1 so it conflicts with the version in
requirements.
Depends-on: I3620c5fe72e49e226f2f5d1e6c36179eb053f424
Change-Id: I9cd70d20f28a6da66795d3d85504bc6dab2dd9bd
Change I1cae480a9a341ec2f6062904c962530dfce95057 introduced a
dependency on the jenkins.Jenkins.get_plugins_info method, which was
not added to python-jenkins until 0.4.1.
Change-Id: Iedbe85d2dadfbb5a87a5f704649c7d30031a21a7
Convert to use idioms that work for both python 3 and python 2.6+ and
ensure that a suitable version of dependencies is included for python 3
compatibility.
Update python-jenkins to 0.3.4 as the earliest version that supports
python 3 without any known regressions. Add an extra parser check for
missing 'command' due to changes in how argparse works under python 3.
To access the first element of a dict in both python 2 and 3,
'next(iter(dict.items()))' is used as the standard idiom to replace
'dict.items()[0]' as 'items()' returns an iterator in python 3 which
cannot be indexed. Using 'next(iter(..))' allows for both lists and
iterators to be passed in without unnecessary conversion of iterators to
lists which would be true of 'list(dict.items())[0]'.
Original change which was reverted due to breaking use of job-groups is
If4b35e2ceee8239379700e22eb79a3eaa04d6f0f. This replaces the previous
conversion of 'dict.items()[0]' to 'dict.popitem()', which would result
in removing a job-group when first called, thus defeating the benefit of
being able to reference the group mulitple times. This usage has been
replaced with 'next(iter(dict.items()))' as a non-modifying alternative
that still avoids creating unnecessary copies of data while working for
all supported versions of python.
Change-Id: I37e3b67c043dadddb54e16ee584bde3f79e6a770
Need ordereddict in the requirements as it is used at run time by the
yaml parsing code not just a test time.
Change-Id: Ic41e7adb5a06ed319f6201a13b9f7391fc245853
This reverts commit 1d7647fa85.
This change altered the xml output (by forcing it to fail) which a
backward compatible change should not do. Revert it in order to get this
compat change in without breaking that output and test.
Change-Id: I20f66fb1bd9c70a0debbdd5eebacf6ec5d0f5df9
Convert to use idioms that work for both python 3 and python 2.6+ and
ensure that a suitable version of dependencies is included for python 3
compatibility.
Update python-jenkins to 0.3.3 as the earliest version that supports
python 3 without any known regressions. Add an extra parser check for
missing 'command' due to changes in how argparse works under python 3.
Where contents should be retained, to access the first element of a dict
in both python 2 and 3, 'next(iter(dict.items()))' is used as the
standard idiom to replace 'dict.items()[0]' as 'items()' returns an
iterator in python 3 which cannot be indexed. Using 'next(iter(..))'
allows for both lists and iterators to be passed in without unnecessary
conversion of iterators to lists which would be true of
'list(dict.items())[0]'.
Alternatively, where further access to the data is not required,
'dict.popitem()' is used.
Change-Id: If4b35e2ceee8239379700e22eb79a3eaa04d6f0f
I found out pbr to be a bit nicer when it comes to edit files. AFAIK
most OpenStack project are using it, so I guess Jenkins Job Builder can
be pbr based as well.o
Basically:
* moved everything from setup.py to setup.cfg
* sorted entries while at it
* require pbr module using the same version limits used by Zuul
* enable pbr.warnerrors
* testenv.usedevelop = True or build_sphinx can not find entry points
* move requirement files to root of repository (we had them in /tools/)
Change-Id: I44ee8910ad4fe4eebb7337951efa31baff4281fe