Commit Graph

63 Commits

Author SHA1 Message Date
Vsevolod Fedorov f4d64f9f66 Move tests to pytest
Pytest makes each scenario into individual selectable test.
To be able to run each scenario separately is very useful for development.

Change-Id: I4b1c990a1fd839ce327cd7faa27159a9b9632fed
2022-12-20 10:44:43 +03:00
Jiri Podivin 352b93d586 Bumping black to 22.3.0
Setting desired version of black to 22.3.0 prevents issues with
the click dependency described in the #1966879.

Introduction of black 22.3.0 forced style changes in following files:
- doc/source/conf.py
- jenkins_jobs/local_yaml.py
- tests/base.py

These changes do not modify behavior of the programs concerned.

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: I42a46f9ed31ea0f404d698eec73d30975e2e6f4c
2022-03-31 09:27:41 +02:00
Eric Ball a833d1fde6
Feat: Add GitHub Organization job type
This adds a job type "githuborg", which creates multibranch pipelines
for all repos in the org that contain the specified Jenkinsfile.

Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Change-Id: I148e2300f3b1ee148b51d25a2a805ed0b5a97862
2021-07-09 16:46:11 -07:00
Jiri Tyr 0f9dd4e805 Adding view for the Delivery Pipeline Plugin
Change-Id: I8a9a71267562734b078b7260c79d299db7fb4e2b
Signed-off-by: Jiri Tyr <jiri.tyr@gmail.com>
2020-02-07 17:35:13 +00:00
Jan Zerebecki 35d4da29d3 Support sectioned views
Change-Id: Iaf352e8cc62644fb05afaae21630b9bec2cd30c0
2019-10-28 23:37:27 +01:00
Jan Zerebecki 587740ec75 Support nested views
Change-Id: I13532a16efc6e970ab5a7c021ec4d77be98d3de8
2019-10-24 14:27:34 +02:00
Thanh Ha 4d90c187a9 Auto-generated output from python-black
Please review the following patch containing the code changes in
the repo. This patch is a transition patch and is the auto-generated
output of the python-black tool.

Change-Id: I2d2de71da8a105fb62b561899ae78441ddab4032
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
2019-09-09 19:23:24 +01:00
Philip Roche 1e901d921d Avoid ResourceWarning by closing file handlers when finished
Warnings appear when running local_yaml unit tests.
This commit fixes the issues causing these warnings

```
 ResourceWarning: unclosed file <_io.TextIOWrapper
name='jenkins-job-builder/tests/yamlparser/fixtures/include_path001.conf'
mode='r' encoding='utf-8'>
  jjb_config = JJBConfig(self.conf_filename)
```

Change-Id: I8fec288cce8c9f6c7b928886d7c70f7a35d43b4a
2019-07-29 16:40:33 +01:00
Thanh Ha 8c99e569b5
Add support for All view-type
Change-Id: I49774d6d177382f30f117dab3836619a40d0e895
Signed-off-by: Thanh Ha <zxiiro@linux.com>
2018-11-12 16:55:39 +08:00
Sorin Sbarnea 4953756c27
adopt pre-commit hooks
Change-Id: I98bc0b5717c3921b35e74f53c55f896427880a84
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
2018-07-12 13:40:38 +01:00
Sorin Sbarnea 14011df16e
Symlink folders001.xml to deduplicate the test xml
Change-Id: Id03b1a147303cc377a48be25ba93f97cab856a8c
Co-Authored-By: Thanh Ha <thanh.ha@linuxfoundation.org>
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
2018-06-29 09:17:50 -04:00
Joost van der Griendt efc5fbe0fd Add multibranch project-type
This work was based on original work done by
Joshua Harlow on top of older code from:
https://github.com/abnamrocoesd/jenkins-job-builder/

Credit is due to `Joost van der Griendt` for doing this
work. This builds on his work in that repo, and adjusts it
so that it can get merged into upstream and released
as a fully supported job type.

SCM implementation is different than than normal SCM module,
supporting: BitBucket, git, and GitHub in this initial
patch.

Change-Id: If50a54d282dd7d901c16edb9fe04874bdd83c9ef
Co-Authored-By: Joshua Harlow <jxharlow@godaddy.com>
Co-Authored-By: Sorin Sbarnea <ssbarnea@redhat.com>
Co-Authored-By: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
Signed-off-by: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
2018-03-15 16:51:32 +00:00
Tomas Janousek f30179db4a tests/base: Use plugins_info in SingleJobTestCase as well
This is a preparation for the next commit which adds a testcase that
requires a plugins_info with newer version of hipchat plugin.

Change-Id: Ia8a7b226a375f407d4e0fb067737118efd0ab845
2018-02-01 11:43:51 +01:00
Tomas Janousek fdd6180107 tests/base: Drop "is not None"
A harmless (in this particular case, where self.plugins_info_filename is
a string) refactoring.

Change-Id: I25a4b3c6b80d9c1e7ac358e655db70121add2d3c
2018-02-01 11:41:30 +01:00
Sorin Sbarnea d8ace6802d docs: sphinx upgrade and warnings as erros
Change-Id: I455ac0d79cca3883d3a758a2709c8c0255f4cc7d
2018-01-26 23:57:53 +00:00
Sorin Sbarnea c0b3b021dc Improve test resiliency
Avoids occasional test executions failures
caused by leftover files on dev machine or
by preexisting configuration files.

Change-Id: Ia0ba29e2dda58c37410a6926cae3039e4ba3473e
2018-01-18 18:08:56 +00:00
Darragh Bailey af9d984baa
Basic folder support
Allows specifying a folder attribute for each job generated, which in
turn is used when creating or uploading to place the job under the
requested folder.

The job name is expanded after defaults are applied, to support the
attribute being defined within a set of defaults applied to a number of
jobs.

This in turn allows for multiple jobs with the same basename to exist,
provided they are targeted at different folders.

Does not support creating the folders if they do not exist.

Change-Id: I8c2157c4c81087cc972a048d1b88d5f08ac65361
2017-10-23 13:29:21 -04:00
Sorin Sbarnea 6a9d5736d6 use alphanum sorting for jobs and views
This improves UX by using a alphanum sorting which
is more natural to humans: [a1, a2, a10] instead of
[a1, a10, a2].

Change-Id: Ice9c3de282f08e85c8f8a34a27d1b6c0502ca9cb
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
2017-08-09 17:51:00 +01:00
Jenkins e47f9629bc Merge "Add view management functionality" 2016-11-10 16:35:23 +00:00
Thanh Ha 1deb3aff4c
Add view management functionality
- Adds the ability for JJB to work with views
- Views can be created, updated, and deleted.
- New modules for List view and Build Pipeline view are added
- New tests for testing the deletion of views

Example View configuration:

    - view:
        name: MyView
        view-type: list

Change-Id: Idb29a4407bcc14593e10a4d951036cb04e8e6c27
Co-Authored-By: Brandon Leonard <brandon.leonard@rackspace.com>
Co-Authored-By: Joao Vale <jpvale@gmail.com>
Co-Authored-By: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
2016-10-14 09:14:21 -04:00
Darragh Bailey 64537f5125 Improve logger output for expanding templates
Output the variable inputs used that trigger an error when expanding
template names to the error logger channel in a sensible format.

Ensures that when indented variable inputs for templates result in
exceptions when expanding a template name, that the project, template
name and variables that failed to be iterated over are outputted in a
log error message along with the original set of inputs from the
project definition to make it easier for end users to find where the
error has been made in a JJB definition.

Add code to allow dumping of variables stored in OrderedDict
transparently to match the input format used in JJB definitions and
hide the implementation detail of using OrderedDict to be within the
localyaml library.

Change-Id: I660bb0ca3b109e1a861948d6a867f185047b90ae
2016-09-14 15:15:04 +01:00
Darragh Bailey 4573b3a25d Refactor base test classes inheritance for reuse
Refactor base test class inheritance to allow for BaseTest class to be
reused without needing to also import the TestCase class everywhere in
order to avoid having the common test function executed on base classes.

This makes it easier to build base testing classes and then simplifies
the subclassing of these for actual tests.

Change-Id: I89809e8082469f814f245db4a9ab7658aac8a405
2016-09-14 14:58:57 +01:00
Wayne Warren aaae83c623
Factor XmlJobGenerator out of YamlParser.
Separate XML generation from Yaml parsing/interpreting. The goal here
is to allow different sources to provide data for XML generation,
including external API users writing job definitions in pure Python or
JJB developers who would like to work on alternative Yaml parsing code
since the current YamlParser has very likely reached the limits of
what can be reasonably done with one giant expandYaml procedure.

Change-Id: I9da848acac7e944c0e07286b7399b2e1956a58a5
2016-08-18 22:42:24 -04:00
Wayne Warren ae1fb60f16
Disentangle YamlParser and ModuleRegistry classes
Create the ModuleRegistry anywhere other than inside the YamlParser
class. This will make it slightly easier to factor a XmlGenerator out
of YamlParser, but I also want to work toward eliminating the circular
references between YamlParser and ModuleRegistry which have been
making it difficult to understand overall program flow.

This commit also replaces all YamlParser instances being passed to
Jenkins job config generating functions with a ModuleRegistry. Mostly
it seems like the parser was only needed to call the ModuleRegistry's
'dispatch' method which to be honest I don't fully understand. This is
where the circular references mentioned in previously come in...it
seems like the "dispatch" function needs access to the (mostly) raw
data contained by the parser, so it took that as a parameter.

The need for the YamlParser's job data can be satisfied by assigning
it to a property on the ModuleRegistry object before Yaml expansion or
XML generation begins; by doing this, we allow the ModuleRegistry to
avoid referencing the parser.

Change-Id: I4b571299b81e708540392ad963163fe092acf1d9
2016-08-18 22:42:24 -04:00
Wayne Warren 4f04de1f9a Use JJBConfig in YamlParser.
This commit sees JJBConfig start to take the form it ought to have,
namely using attributes to represent different logical sections of
configuration that target specific subsystems of JJB.

It also moves ConfigParser data retrieval from
jenkins_jobs.modules.helpers.get_value_from_yaml_or_config_file() to
jenkins_jobs.config.JJBConfig.get_module_config()

TODO: Add JJBConfig tests to validate the behavior of JJBConfig in
specific circumstances.

Change-Id: I053d165559f5325a2f40b239117a86e6d0f3ef37
2016-07-22 17:33:25 +01:00
Wayne Warren 2cf11f014c Use JJBConfig for arg/config initialization.
* Create jenkins_jobs.config module with JJBConfig class.
* Move DEFAULT_CONF from jenkins_jobs.cmd into jenkins_jobs.config
* Move configuration initialization into JJBConfig
* Create method, "do_magical_things" to handle arbitration between
  config file and arguments as well as setting default values for
  config file settings if it doesn't contain the expected keys.
* Move JenkinsJobs.create_parser into its own module,
  jenkins_jobs.cli.parser, it can be used to provide default settings
  in the JJBConfig class when an argparse namespace object is not
  provided; this is primarily necessary because most of the original
  configuration initialization code relies on this being a namespace
  object (simple descendant of the object class).

At this point JJBConfig isn't much more than an object-oriented
version of the way configuration handling happened previously. Its
current form, however, is more amenable to the ultimate goal of the
2.0 refactorings--namely, being able to pass a single config object
around rather than breaking it up into apparently arbitrary settings
necessary to instantiate the Builder class and its delegate objects.

Change-Id: Ic0147e1dccbe620aaaba039a434e7cea6c670054
2016-07-19 18:57:34 +01:00
Darragh Bailey bafc09598f Compatibility support for older maven project plugins
Enable testing of project modules to receive plugin info and ensure
that versions of the Maven project plugin older than 2.0.1 continue to
receive the required XML setting.

Change-Id: I2669052d0ef31c0b766e9cde24c842136c172b6c
2016-05-03 18:45:49 +01:00
Darragh Bailey 8805a34c0f Reorder imports to match hacking guidelines
Ensure that the imports follow the standard OpenStack hacking
guidelines.

Change-Id: Iaa4326aef118ddfd807dd006934f1d9ca80a1cfa
2015-12-23 15:23:42 -08:00
Darragh Bailey 9bfbffa559 Capture logs with FakeLogging Fixture
Capture logs to attach them on exceptions to simplify the debugging of
errors in tests when they occur.

Change-Id: I5c1b5c306aba37789a8d41eabf65e0ac628af838
2015-09-30 16:25:22 +01:00
Jenkins 5ce1b32e02 Merge "Allow whitespace to be significant" 2015-08-26 06:59:48 +00:00
Jenkins 6fd8c4ec0e Merge "Ensure tests run with ConfigParser object" 2015-08-26 06:53:17 +00:00
Jenkins 2854ad8942 Merge "Document that jjb supports json job definitions" 2015-08-12 19:48:02 +00:00
Darragh Bailey e5738d0852 Allow whitespace to be significant
Remove the test options suppressing ignoring differences in whitespace
outputted in the XML to allow capturing of issues where the code
incorrectly adds or removes significant whitespace.

Add a helper function to deal with the edge case of parsing XML directly
that contains non significant whitespace to avoid accidentally adding
excess whitespace when minidom outputs the resulting document.

Change-Id: I9936042cd82c204ba2b3c19f575703e33564f7fd
2015-08-11 18:31:48 +01:00
Darragh Bailey dc464d943f Ensure tests run with ConfigParser object
Should always use a ConfigParser object when testing to allow unit tests
to check that calls behave as expected.

Read in the default configuration to align with expected usage, and fix
some issues caused by usage of dicts resulting in the option name
requested being taken as the default value.

Since '.get()' is a valid method for a dictionary object, take care to
ensure that we do not use a dummy dict object as this will not throw the
expected errors if a config file is not provided and will also treat the
option name as a default value resulting in the wrong data appearing in
the outputted XML.

Change-Id: I6abb85f0363a48f220914c52b74e5a5fb65b0faa
2015-08-11 16:36:08 +01:00
Jenkins 8c46ff8b10 Merge "Replace open() with io.open() and force 'utf-8'" 2015-08-10 18:14:11 +00:00
Somay Jain 92b7f446f9 Support for externaljob project-type for monitoring external jobs
Change-Id: I8904725f75d27180d44c55768605db7d67e79586
2015-08-01 12:45:40 -07:00
Khai Do e52bb31fa1 Document that jjb supports json job definitions
Add documentation to let users know that they can also define jobs
in JSON.  Create a new test suite to verify that jjb can correctly
generate jenkins xml from json definitions.

Change-Id: Idb5216b0b56837d0f4dd2298986149be66bd217e
2015-07-28 17:20:29 -07:00
Darragh Bailey 0e74bddb56 Replace open() with io.open() and force 'utf-8'
Use io.open() to allow reading and writing of files in 'utf-8' format
irrespective of the terminal encoding selected.

Change-Id: Ie952617a34c0719efc59a7729d698beafaa477b0
2015-07-25 14:16:12 -07:00
Jenkins f6a8cfede5 Merge "Remove YamlParser from jenkins_jobs.builder" 2015-04-23 17:07:59 +00:00
Wayne 4a8b93b8c2 Remove YamlParser from jenkins_jobs.builder
The goal of this patch is simply to move some classes out of
jenkins_jobs.builder into more appropriately-named modules. This started with
simply moving YamlParser into jenkins_jobs.parser but led to other moves in
order to avoid cyclic imports since YamlParser uses other classes previously
defined in jenkins_jobs.builder.

That said, this patch doesn't intend to address all of the clutter in
jenkins_jobs.builder, mostly just what is necessary to get started working on
YamlParser independent of other classes in that module.

Change-Id: Ie88bf683e495033eb0b670fe29c256a70282735f
2015-04-22 11:59:07 -07:00
Darragh Bailey 2338167e4e Treat non-existant output files as empty files
Replace requiring empty files for tests that produce no output and treat
any non-existant output file candidate as though the output is blank.

This simplifies tests that are intended to trigger exceptions and will
not generate an output.

Change-Id: I978fdabb488bc3e4bc6f22db294d21ca5d68913b
2015-03-23 16:33:41 +00:00
Darragh Bailey 1e70128d30 Add tests for yaml anchor behaviour
Adds two tests to ensure correct behaviour with referencing yaml
anchors and a third test to verify the expansion internally of yaml
anchors and aliases.

* test that anchors are not carried across subsequent top level
  invocations of yaml.load(). This will be used subsequently to ensure
  that where anchors are allowed across included files they are
  correctly reset on each top level call.
* test that where anchors are defined in a top level file and
  subsequently included files, duplicate anchors raise exceptions as
  though they were defined within the same file.
* test that data returned from yaml loading contains the additional data
  specified by the alias. Uses json to force conversion so that the
  outputted yaml contains the results of the anchors and aliases instead
  of them.

Update documentation to contain details of the use of anchors and
aliases including a refernce to a simple generic example from the
specification as well as a JJB specific example.

Change-Id: I0f2b55e1e2f2bad09f65b1b981baa0159372ee10
2015-03-10 14:27:30 +00:00
Wayne b854bef68e Implement recursive search for test fixtures.
This allows hierarchical organization of test scenarios which I consider an
improvement over the current setup.

Change-Id: I079bfa984832e991712099e0ed353d7c2470dc5b
2015-02-05 14:01:29 -08:00
Wayne b4e5aa9b77 Add plugins_info to module registry object.
This makes it available for use by any module that has version-specific behavior
differences since modules always have access to the yamlparser which in turn
contains a module registry.

Change-Id: I1cae480a9a341ec2f6062904c962530dfce95057
2015-02-04 05:02:14 -08:00
Jeremy Stanley fbd432a0ad Use mock in a Python 3.4 compatible way
Related to https://code.google.com/p/mock/issues/detail?id=234

Change-Id: I85549fdba7b9cb575850a12d52e1ffb0d2dba8c0
2015-01-29 23:07:53 +00:00
Joshua Hesketh 24a252c734 Start libification of jjb
Allow the expansion of the jjb template language yaml into a list before
generating xml. This will allow others to consume the template parts of
jjb without forcing them to be tied to xml.

Change-Id: Ifd99151f8246eb817194adc2180189f739ccf742
2014-10-16 17:24:00 +11:00
Marc Abramowitz 64e217f885 Make JJB python 3 compatible
Convert to use idioms that work for both python 3 and python 2.6+ and
ensure that a suitable version of dependencies is included for python 3
compatibility.

Update python-jenkins to 0.3.4 as the earliest version that supports
python 3 without any known regressions. Add an extra parser check for
missing 'command' due to changes in how argparse works under python 3.

To access the first element of a dict in both python 2 and 3,
'next(iter(dict.items()))' is used as the standard idiom to replace
'dict.items()[0]' as 'items()' returns an iterator in python 3 which
cannot be indexed. Using 'next(iter(..))' allows for both lists and
iterators to be passed in without unnecessary conversion of iterators to
lists which would be true of 'list(dict.items())[0]'.

Original change which was reverted due to breaking use of job-groups is
If4b35e2ceee8239379700e22eb79a3eaa04d6f0f. This replaces the previous
conversion of 'dict.items()[0]' to 'dict.popitem()', which would result
in removing a job-group when first called, thus defeating the benefit of
being able to reference the group mulitple times. This usage has been
replaced with 'next(iter(dict.items()))' as a non-modifying alternative
that still avoids creating unnecessary copies of data while working for
all supported versions of python.

Change-Id: I37e3b67c043dadddb54e16ee584bde3f79e6a770
2014-10-06 15:24:16 +01:00
Jenkins 182a189640 Merge "Fix method name call when files cannot be found" 2014-09-03 16:41:30 +00:00
Clark Boylan 405ff049ff Revert "Some tweaks to get closer to Python 3 compat"
This reverts commit 1d7647fa85.

This change altered the xml output (by forcing it to fail) which a
backward compatible change should not do. Revert it in order to get this
compat change in without breaking that output and test.

Change-Id: I20f66fb1bd9c70a0debbdd5eebacf6ec5d0f5df9
2014-09-02 15:38:49 -07:00
Marc Abramowitz 1d7647fa85 Some tweaks to get closer to Python 3 compat
Convert to use idioms that work for both python 3 and python 2.6+ and
ensure that a suitable version of dependencies is included for python 3
compatibility.

Update python-jenkins to 0.3.3 as the earliest version that supports
python 3 without any known regressions. Add an extra parser check for
missing 'command' due to changes in how argparse works under python 3.

Where contents should be retained, to access the first element of a dict
in both python 2 and 3, 'next(iter(dict.items()))' is used as the
standard idiom to replace 'dict.items()[0]' as 'items()' returns an
iterator in python 3 which cannot be indexed. Using 'next(iter(..))'
allows for both lists and iterators to be passed in without unnecessary
conversion of iterators to lists which would be true of
'list(dict.items())[0]'.

Alternatively, where further access to the data is not required,
'dict.popitem()' is used.

Change-Id: If4b35e2ceee8239379700e22eb79a3eaa04d6f0f
2014-09-01 15:59:19 +01:00