Commit Graph

28 Commits

Author SHA1 Message Date
Vsevolod Fedorov 67645a46eb Fix legacy plugin version comparison; Remove cap on setuptools version
LegacyVersion class is removed from newer setuptools package. But
support for legacy versions is added in python-jenkins 1.8.2.
Switch to that implementation.

Fix broken plugin version comparison for legacy versions.

Assume latest plugin version if no plugin version is found.

Story: 2010990
Story: 2009943
Story: 2009819
Story: 2010842
Task: 49236
Task: 44852
Task: 44396
Task: 48448

Change-Id: Id7f0be1c42357454bd9bedcdee3fefb174943d81
2023-12-19 12:10:43 +03:00
Vsevolod Fedorov c011568aa0 Requirements: Update PyYAML version
It lools like versions prior to 3.13 do not support
Python 3.7 nor Python 3.8.
Compatibility with PyYAML versions 3.13 and 6.0 are
checked with Python 3.7 and 3.8 manually (locally)
with `tox -e py37/py38 -r`.

Change-Id: Ie53d82a88498cc5650118d275496970ad6b4a6eb
2023-02-28 20:16:57 +03:00
Vsevolod Fedorov 571df29fd7 Requirements: Fix setuptools version
Starting from version 66.0.0 setuptools has dropped support for
LegacyVersion. But many Jenkins plugins still provide versions in
legacy format.
Default setuptools version is changed recently on OpenStack
infrastructure, so we need to fix it now.

Change-Id: I3c6029e89a44d2c552f58336859249ed5fe958f5
2023-02-09 13:59:44 +03:00
Pau Espin Pedrol 76f417da09 requirements.txt: Avoid depending on PyYAML<6
Some distros like Archlinux already ship with PyYAML 6, which makes it
impossible to build jenkins-job-builder.

I tried manually dropping the "PyYAML < 6" dependency and I could update
some jenkins jobs without problem. Let's not block newer versions from
now on in order o avoid blocking downstream.

Related: https://storyboard.openstack.org/#!/story/2009723
Related: https://aur.archlinux.org/packages/python-jenkins-job-builder#comment-860221
Change-Id: I2308020ed226615afd2652ff74d6766b5ecfab43
2022-04-07 18:40:37 +02:00
Evgeni Golov 9c7dbb6392 force pre 2.0 stevedore on Python 2
stevedore 2.0 dropped Python 2 support, but JJB still supports it.

Change-Id: I35eb127fc528b3e4fc84a4e0503b08ef4f56509f
2020-09-08 09:28:29 +02:00
Thomas Bechtold 23079af777 Allow pyyaml >= 5
Current latest version is 5.1 so allow this version, too.

Change-Id: Ic349545720bf9a3d973fe863c7068d1e590732c3
2019-04-08 13:29:43 +02:00
Vicky Chijwani 0deb58146a Constrain PyYAML to v3.x.
PyYAML v4.1 was released which breaks the `!!python/tuple` syntax with the
following error:

yaml.constructor.ConstructorError: could not determine a constructor for the tag
'tag:yaml.org,2002:python/tuple'

Change-Id: Iaf838b111e6dfad5dc5c1f904a482b525e99a911
2018-06-27 18:14:53 +05:30
Sorin Sbarnea 705051658f bump python-jenkins version
Bumps python-jenkins version to avoid few
remarkable issues fixed in it:
- allow to wait for jenkins to enter normal operation
- remove cap on pbr version
- more py3 enablement
- other, see its changelog

Change-Id: I508b412bc2d17561e7a7ceef015ea018615596ad
2018-01-12 18:18:58 +00:00
Darragh Bailey af9d984baa
Basic folder support
Allows specifying a folder attribute for each job generated, which in
turn is used when creating or uploading to place the job under the
requested folder.

The job name is expanded after defaults are applied, to support the
attribute being defined within a set of defaults applied to a number of
jobs.

This in turn allows for multiple jobs with the same basename to exist,
provided they are targeted at different folders.

Does not support creating the folders if they do not exist.

Change-Id: I8c2157c4c81087cc972a048d1b88d5f08ac65361
2017-10-23 13:29:21 -04:00
Daniel Watkins b95f194612 Add !include-jinja2 for rendering templates with Jinja2
This template included using !include-jinja2:

"""
{{ my_var }}
"""

is rendered the same as an existing template that looks like this:

"""
{my_var}
"""

This also allows the use of Jinja2's richer syntax:

"""
{% for test_environment in configuration.get("envs", ["py35"]) %}
tox -e {{ test_environment }}
{% endfor %}
"""

Story: 2001135

Change-Id: Ia3ee21822d6e9237f5ea46796bc8810ecac61e2c
2017-08-09 10:11:42 -04:00
Darragh Bailey 6e237c4369 Allow using lockfile per jenkins master
When a jjb run is thrown when another jjb is already running, it can
cause corruption of cache. Start using a fasteners to ensure this
won't be happening and run securely on automated systems.

Ensure unlock is called from only the destructor, so that it is only
called when the JJB process is guaranteed to be finished using it.

Make it obvious that _lock is intended to be internal to the cache
storage implementation.

Potentially we may not need to call it at all, as python might unlock
it for us on exit by closing the file when no longer needed. However
better to make it explicit.

Change-Id: I53a1f92cf2bfbbe87c9ea205c377f93869353620
2017-06-27 15:02:04 +00:00
Monty Taylor 503fd8dcd4
Sync requirements with openstack/requirements
This is not necessary - jenkins-job-builder does not follow the
requirements sync process. That said - these are likely all safe bets
for being solid versions of these dependencies - and some of them were
kinda stale.

Change-Id: Id1941ff4c9736100f9bdd5508ec3c0377e51fdbe
2016-12-10 09:42:57 -06:00
liyuanzhen e70e1f2bf4 Not limit stevedore to 1.8.0
If the stevedore is limited to 1.8.0 and the version of stevedore
in host is higher than 1.8.0, it will raise an 'VersionConflict'
error when executing 'jenkins-jobs update` command.

Change-Id: I9e83fddd82049be0d21ec4106f7b846184f3af8d
Closes-Bug: #1614435
2016-08-19 07:38:19 +00:00
Wayne fc73cedb45 Introduce modular implementation of subcommands.
This commit intentionally introduces a number of important API
breakages. Specifically, the jenkins_jobs.cmd module has been pared
down to some of its most difficult-to-refactor elements.

* Create jenkins_jobs.cli.entry.JenkinsJobs class to organize command
  line parsing and execution.
* Remove references to ConfigParser object in test code, hidden as an
  implementation detail of JenkinsJobs command line parsing. This will
  be necessary in the next stage of JJB 2.0 code which will be to
  create a JJBConfig object that handles logic and presentation of
  configuration from various sources--defaults, command line
  arguments, configuration file, and maybe environment variables in
  the future.
* Remove references to Namespace object produced by argparse module.
  Required rewrite of multipath & recursive path tests with a new
  MatchesDir testtools Matcher class that validates the expected
  output for a run of JJB against a given set of yamldirs with the
  specified command line arguments.
* Use stevedore to dynamically load subcommand parsers.
* Move configuration loading/testing to its own test file. Also fix
  the global vs home directory JJB config file test.

Change-Id: If62280418ba7319c313033ab387af4284237747e
2016-07-08 09:55:44 -07:00
Thomas Bechtold 389afe422c Remove ordereddict support from py26
python 2.6 is no longer supported and adding a marker
to requirements.txt for installing ordereddict only in py2.6
envs was not an option because markers are not supported in
older pip versions.
So remove python 2.6 support completly.

Change-Id: Iebdd999b469c9a9681b0d7e9f50cc488a8820953
2016-02-09 17:22:42 +01:00
Darragh Bailey 3faecfd76a Add separate linkcheck env and allow dev to select builder
Add separate linkcheck tox environment to be used in non-voting Jenkins
job to watch for stale and broken links in the sphinx generated html
documentation. Include a small addition to make it easier for
contributors to select individual builders when executing the docs tox
environment.

Add minimum dependency of pbr 1.0.0, as that was the first version to
allow passing of sphinx builders on the command line to override the
internal defaults or those set in the configuration file.

This will make it easier to spot problems with just the links, and also
ensure that when the sphinx project releases with proper warniserror fix
for the linkcheck builder, that html and man page documentation can
continue to be built even if a previous valid link has become stale or
is temporarily unreachable.

Change-Id: I80446abe96824a444a04cc526df6cc38632a4607
2016-01-11 13:06:17 +00:00
Jenkins b8eb60239f Merge "Improve delete all job performance" 2015-09-18 17:35:02 +00:00
Ken Dreyer 34570607e2 rm argparse from requirements.txt
argparse is built into Python 2.7. Remove it from requirements.txt.

Change-Id: Ia55308e2f8d8a4cab67794816a3cd5d50acebc3f
2015-08-25 18:41:47 +01:00
Khai Do 67de07ceed Improve delete all job performance
delete-all command was deleting one job at a time using the jenkins
doDelete rest endpoint which is very slow.  Change Ia4fbfca970165
allows us to execute a groovy script on the jenkins server to
do the same thing.  The only difference is that running the script
on the server will delete all jobs much faster.

As a data point:
I added all (~5,000) openstack jobs to my jenkins server.
The previous delete_all took many hours to delete all the
jobs while the script method did it in a matter of seconds.

Change-Id: Id7f7dfb567997e042fe0f783e54680482fefdc15
depends-on: Ia4fbfca970165d890d7e076f47ddcde7633afa9b
2015-07-31 14:48:30 -07:00
Emilien Macchi 4f146bc3e8 requirements: pin pbr>=0.8.2,<2.0
This patch aims to pin pbr>=0.8.2,<2.0 instead of pbr>=0.8.2,<1.0.

Some distros ship 1.0.1 so it conflicts with the version in
requirements.

Depends-on: I3620c5fe72e49e226f2f5d1e6c36179eb053f424
Change-Id: I9cd70d20f28a6da66795d3d85504bc6dab2dd9bd
2015-06-09 17:40:26 -04:00
Jeremy Stanley 520f3c5459 Depend on python-jenkins 0.4.1 or later
Change I1cae480a9a341ec2f6062904c962530dfce95057 introduced a
dependency on the jenkins.Jenkins.get_plugins_info method, which was
not added to python-jenkins until 0.4.1.

Change-Id: Iedbe85d2dadfbb5a87a5f704649c7d30031a21a7
2015-02-06 14:26:56 +00:00
Marc Abramowitz 64e217f885 Make JJB python 3 compatible
Convert to use idioms that work for both python 3 and python 2.6+ and
ensure that a suitable version of dependencies is included for python 3
compatibility.

Update python-jenkins to 0.3.4 as the earliest version that supports
python 3 without any known regressions. Add an extra parser check for
missing 'command' due to changes in how argparse works under python 3.

To access the first element of a dict in both python 2 and 3,
'next(iter(dict.items()))' is used as the standard idiom to replace
'dict.items()[0]' as 'items()' returns an iterator in python 3 which
cannot be indexed. Using 'next(iter(..))' allows for both lists and
iterators to be passed in without unnecessary conversion of iterators to
lists which would be true of 'list(dict.items())[0]'.

Original change which was reverted due to breaking use of job-groups is
If4b35e2ceee8239379700e22eb79a3eaa04d6f0f. This replaces the previous
conversion of 'dict.items()[0]' to 'dict.popitem()', which would result
in removing a job-group when first called, thus defeating the benefit of
being able to reference the group mulitple times. This usage has been
replaced with 'next(iter(dict.items()))' as a non-modifying alternative
that still avoids creating unnecessary copies of data while working for
all supported versions of python.

Change-Id: I37e3b67c043dadddb54e16ee584bde3f79e6a770
2014-10-06 15:24:16 +01:00
Darragh Bailey 2d74b16620 Move ordereddict to requirements
Need ordereddict in the requirements as it is used at run time by the
yaml parsing code not just a test time.

Change-Id: Ic41e7adb5a06ed319f6201a13b9f7391fc245853
2014-09-04 14:58:20 +01:00
Clark Boylan 405ff049ff Revert "Some tweaks to get closer to Python 3 compat"
This reverts commit 1d7647fa85.

This change altered the xml output (by forcing it to fail) which a
backward compatible change should not do. Revert it in order to get this
compat change in without breaking that output and test.

Change-Id: I20f66fb1bd9c70a0debbdd5eebacf6ec5d0f5df9
2014-09-02 15:38:49 -07:00
Marc Abramowitz 1d7647fa85 Some tweaks to get closer to Python 3 compat
Convert to use idioms that work for both python 3 and python 2.6+ and
ensure that a suitable version of dependencies is included for python 3
compatibility.

Update python-jenkins to 0.3.3 as the earliest version that supports
python 3 without any known regressions. Add an extra parser check for
missing 'command' due to changes in how argparse works under python 3.

Where contents should be retained, to access the first element of a dict
in both python 2 and 3, 'next(iter(dict.items()))' is used as the
standard idiom to replace 'dict.items()[0]' as 'items()' returns an
iterator in python 3 which cannot be indexed. Using 'next(iter(..))'
allows for both lists and iterators to be passed in without unnecessary
conversion of iterators to lists which would be true of
'list(dict.items())[0]'.

Alternatively, where further access to the data is not required,
'dict.popitem()' is used.

Change-Id: If4b35e2ceee8239379700e22eb79a3eaa04d6f0f
2014-09-01 15:59:19 +01:00
Darragh Bailey fd82f48a84 Argparse is required for JJB to parse arguments
Move argparse requirement from tests to install requirements as it is
required when using JJB.

Change-Id: If503ebfe790c904f77ef45b59ea909a2577db5cd
2014-07-27 00:49:51 +01:00
Marc Abramowitz b33ac74828 Add tox "coverage" target
Change-Id: I577d62e016a550ea30671837a0a426165984e982
2014-05-27 16:50:34 -07:00
Antoine Musso d2ff8c098f Migrate to pbr
I found out pbr to be a bit nicer when it comes to edit files. AFAIK
most OpenStack project are using it, so I guess Jenkins Job Builder can
be pbr based as well.o

Basically:
* moved everything from setup.py to setup.cfg
* sorted entries while at it
* require pbr module using the same version limits used by Zuul
* enable pbr.warnerrors
* testenv.usedevelop = True or build_sphinx can not find entry points
* move requirement files to root of repository (we had them in /tools/)

Change-Id: I44ee8910ad4fe4eebb7337951efa31baff4281fe
2014-03-14 22:25:46 +01:00