Pytest makes each scenario into individual selectable test.
To be able to run each scenario separately is very useful for development.
Change-Id: I4b1c990a1fd839ce327cd7faa27159a9b9632fed
Setting desired version of black to 22.3.0 prevents issues with
the click dependency described in the #1966879.
Introduction of black 22.3.0 forced style changes in following files:
- doc/source/conf.py
- jenkins_jobs/local_yaml.py
- tests/base.py
These changes do not modify behavior of the programs concerned.
Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: I42a46f9ed31ea0f404d698eec73d30975e2e6f4c
This adds a job type "githuborg", which creates multibranch pipelines
for all repos in the org that contain the specified Jenkinsfile.
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Change-Id: I148e2300f3b1ee148b51d25a2a805ed0b5a97862
Please review the following patch containing the code changes in
the repo. This patch is a transition patch and is the auto-generated
output of the python-black tool.
Change-Id: I2d2de71da8a105fb62b561899ae78441ddab4032
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
This work was based on original work done by
Joshua Harlow on top of older code from:
https://github.com/abnamrocoesd/jenkins-job-builder/
Credit is due to `Joost van der Griendt` for doing this
work. This builds on his work in that repo, and adjusts it
so that it can get merged into upstream and released
as a fully supported job type.
SCM implementation is different than than normal SCM module,
supporting: BitBucket, git, and GitHub in this initial
patch.
Change-Id: If50a54d282dd7d901c16edb9fe04874bdd83c9ef
Co-Authored-By: Joshua Harlow <jxharlow@godaddy.com>
Co-Authored-By: Sorin Sbarnea <ssbarnea@redhat.com>
Co-Authored-By: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
Signed-off-by: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
This is a preparation for the next commit which adds a testcase that
requires a plugins_info with newer version of hipchat plugin.
Change-Id: Ia8a7b226a375f407d4e0fb067737118efd0ab845
Avoids occasional test executions failures
caused by leftover files on dev machine or
by preexisting configuration files.
Change-Id: Ia0ba29e2dda58c37410a6926cae3039e4ba3473e
Allows specifying a folder attribute for each job generated, which in
turn is used when creating or uploading to place the job under the
requested folder.
The job name is expanded after defaults are applied, to support the
attribute being defined within a set of defaults applied to a number of
jobs.
This in turn allows for multiple jobs with the same basename to exist,
provided they are targeted at different folders.
Does not support creating the folders if they do not exist.
Change-Id: I8c2157c4c81087cc972a048d1b88d5f08ac65361
This improves UX by using a alphanum sorting which
is more natural to humans: [a1, a2, a10] instead of
[a1, a10, a2].
Change-Id: Ice9c3de282f08e85c8f8a34a27d1b6c0502ca9cb
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
- Adds the ability for JJB to work with views
- Views can be created, updated, and deleted.
- New modules for List view and Build Pipeline view are added
- New tests for testing the deletion of views
Example View configuration:
- view:
name: MyView
view-type: list
Change-Id: Idb29a4407bcc14593e10a4d951036cb04e8e6c27
Co-Authored-By: Brandon Leonard <brandon.leonard@rackspace.com>
Co-Authored-By: Joao Vale <jpvale@gmail.com>
Co-Authored-By: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
Output the variable inputs used that trigger an error when expanding
template names to the error logger channel in a sensible format.
Ensures that when indented variable inputs for templates result in
exceptions when expanding a template name, that the project, template
name and variables that failed to be iterated over are outputted in a
log error message along with the original set of inputs from the
project definition to make it easier for end users to find where the
error has been made in a JJB definition.
Add code to allow dumping of variables stored in OrderedDict
transparently to match the input format used in JJB definitions and
hide the implementation detail of using OrderedDict to be within the
localyaml library.
Change-Id: I660bb0ca3b109e1a861948d6a867f185047b90ae
Refactor base test class inheritance to allow for BaseTest class to be
reused without needing to also import the TestCase class everywhere in
order to avoid having the common test function executed on base classes.
This makes it easier to build base testing classes and then simplifies
the subclassing of these for actual tests.
Change-Id: I89809e8082469f814f245db4a9ab7658aac8a405
Separate XML generation from Yaml parsing/interpreting. The goal here
is to allow different sources to provide data for XML generation,
including external API users writing job definitions in pure Python or
JJB developers who would like to work on alternative Yaml parsing code
since the current YamlParser has very likely reached the limits of
what can be reasonably done with one giant expandYaml procedure.
Change-Id: I9da848acac7e944c0e07286b7399b2e1956a58a5
Create the ModuleRegistry anywhere other than inside the YamlParser
class. This will make it slightly easier to factor a XmlGenerator out
of YamlParser, but I also want to work toward eliminating the circular
references between YamlParser and ModuleRegistry which have been
making it difficult to understand overall program flow.
This commit also replaces all YamlParser instances being passed to
Jenkins job config generating functions with a ModuleRegistry. Mostly
it seems like the parser was only needed to call the ModuleRegistry's
'dispatch' method which to be honest I don't fully understand. This is
where the circular references mentioned in previously come in...it
seems like the "dispatch" function needs access to the (mostly) raw
data contained by the parser, so it took that as a parameter.
The need for the YamlParser's job data can be satisfied by assigning
it to a property on the ModuleRegistry object before Yaml expansion or
XML generation begins; by doing this, we allow the ModuleRegistry to
avoid referencing the parser.
Change-Id: I4b571299b81e708540392ad963163fe092acf1d9
This commit sees JJBConfig start to take the form it ought to have,
namely using attributes to represent different logical sections of
configuration that target specific subsystems of JJB.
It also moves ConfigParser data retrieval from
jenkins_jobs.modules.helpers.get_value_from_yaml_or_config_file() to
jenkins_jobs.config.JJBConfig.get_module_config()
TODO: Add JJBConfig tests to validate the behavior of JJBConfig in
specific circumstances.
Change-Id: I053d165559f5325a2f40b239117a86e6d0f3ef37
* Create jenkins_jobs.config module with JJBConfig class.
* Move DEFAULT_CONF from jenkins_jobs.cmd into jenkins_jobs.config
* Move configuration initialization into JJBConfig
* Create method, "do_magical_things" to handle arbitration between
config file and arguments as well as setting default values for
config file settings if it doesn't contain the expected keys.
* Move JenkinsJobs.create_parser into its own module,
jenkins_jobs.cli.parser, it can be used to provide default settings
in the JJBConfig class when an argparse namespace object is not
provided; this is primarily necessary because most of the original
configuration initialization code relies on this being a namespace
object (simple descendant of the object class).
At this point JJBConfig isn't much more than an object-oriented
version of the way configuration handling happened previously. Its
current form, however, is more amenable to the ultimate goal of the
2.0 refactorings--namely, being able to pass a single config object
around rather than breaking it up into apparently arbitrary settings
necessary to instantiate the Builder class and its delegate objects.
Change-Id: Ic0147e1dccbe620aaaba039a434e7cea6c670054
Enable testing of project modules to receive plugin info and ensure
that versions of the Maven project plugin older than 2.0.1 continue to
receive the required XML setting.
Change-Id: I2669052d0ef31c0b766e9cde24c842136c172b6c
Capture logs to attach them on exceptions to simplify the debugging of
errors in tests when they occur.
Change-Id: I5c1b5c306aba37789a8d41eabf65e0ac628af838
Remove the test options suppressing ignoring differences in whitespace
outputted in the XML to allow capturing of issues where the code
incorrectly adds or removes significant whitespace.
Add a helper function to deal with the edge case of parsing XML directly
that contains non significant whitespace to avoid accidentally adding
excess whitespace when minidom outputs the resulting document.
Change-Id: I9936042cd82c204ba2b3c19f575703e33564f7fd
Should always use a ConfigParser object when testing to allow unit tests
to check that calls behave as expected.
Read in the default configuration to align with expected usage, and fix
some issues caused by usage of dicts resulting in the option name
requested being taken as the default value.
Since '.get()' is a valid method for a dictionary object, take care to
ensure that we do not use a dummy dict object as this will not throw the
expected errors if a config file is not provided and will also treat the
option name as a default value resulting in the wrong data appearing in
the outputted XML.
Change-Id: I6abb85f0363a48f220914c52b74e5a5fb65b0faa
Add documentation to let users know that they can also define jobs
in JSON. Create a new test suite to verify that jjb can correctly
generate jenkins xml from json definitions.
Change-Id: Idb5216b0b56837d0f4dd2298986149be66bd217e
Use io.open() to allow reading and writing of files in 'utf-8' format
irrespective of the terminal encoding selected.
Change-Id: Ie952617a34c0719efc59a7729d698beafaa477b0
The goal of this patch is simply to move some classes out of
jenkins_jobs.builder into more appropriately-named modules. This started with
simply moving YamlParser into jenkins_jobs.parser but led to other moves in
order to avoid cyclic imports since YamlParser uses other classes previously
defined in jenkins_jobs.builder.
That said, this patch doesn't intend to address all of the clutter in
jenkins_jobs.builder, mostly just what is necessary to get started working on
YamlParser independent of other classes in that module.
Change-Id: Ie88bf683e495033eb0b670fe29c256a70282735f
Replace requiring empty files for tests that produce no output and treat
any non-existant output file candidate as though the output is blank.
This simplifies tests that are intended to trigger exceptions and will
not generate an output.
Change-Id: I978fdabb488bc3e4bc6f22db294d21ca5d68913b
Adds two tests to ensure correct behaviour with referencing yaml
anchors and a third test to verify the expansion internally of yaml
anchors and aliases.
* test that anchors are not carried across subsequent top level
invocations of yaml.load(). This will be used subsequently to ensure
that where anchors are allowed across included files they are
correctly reset on each top level call.
* test that where anchors are defined in a top level file and
subsequently included files, duplicate anchors raise exceptions as
though they were defined within the same file.
* test that data returned from yaml loading contains the additional data
specified by the alias. Uses json to force conversion so that the
outputted yaml contains the results of the anchors and aliases instead
of them.
Update documentation to contain details of the use of anchors and
aliases including a refernce to a simple generic example from the
specification as well as a JJB specific example.
Change-Id: I0f2b55e1e2f2bad09f65b1b981baa0159372ee10
This allows hierarchical organization of test scenarios which I consider an
improvement over the current setup.
Change-Id: I079bfa984832e991712099e0ed353d7c2470dc5b
This makes it available for use by any module that has version-specific behavior
differences since modules always have access to the yamlparser which in turn
contains a module registry.
Change-Id: I1cae480a9a341ec2f6062904c962530dfce95057
Allow the expansion of the jjb template language yaml into a list before
generating xml. This will allow others to consume the template parts of
jjb without forcing them to be tied to xml.
Change-Id: Ifd99151f8246eb817194adc2180189f739ccf742
Convert to use idioms that work for both python 3 and python 2.6+ and
ensure that a suitable version of dependencies is included for python 3
compatibility.
Update python-jenkins to 0.3.4 as the earliest version that supports
python 3 without any known regressions. Add an extra parser check for
missing 'command' due to changes in how argparse works under python 3.
To access the first element of a dict in both python 2 and 3,
'next(iter(dict.items()))' is used as the standard idiom to replace
'dict.items()[0]' as 'items()' returns an iterator in python 3 which
cannot be indexed. Using 'next(iter(..))' allows for both lists and
iterators to be passed in without unnecessary conversion of iterators to
lists which would be true of 'list(dict.items())[0]'.
Original change which was reverted due to breaking use of job-groups is
If4b35e2ceee8239379700e22eb79a3eaa04d6f0f. This replaces the previous
conversion of 'dict.items()[0]' to 'dict.popitem()', which would result
in removing a job-group when first called, thus defeating the benefit of
being able to reference the group mulitple times. This usage has been
replaced with 'next(iter(dict.items()))' as a non-modifying alternative
that still avoids creating unnecessary copies of data while working for
all supported versions of python.
Change-Id: I37e3b67c043dadddb54e16ee584bde3f79e6a770
This reverts commit 1d7647fa85.
This change altered the xml output (by forcing it to fail) which a
backward compatible change should not do. Revert it in order to get this
compat change in without breaking that output and test.
Change-Id: I20f66fb1bd9c70a0debbdd5eebacf6ec5d0f5df9
Convert to use idioms that work for both python 3 and python 2.6+ and
ensure that a suitable version of dependencies is included for python 3
compatibility.
Update python-jenkins to 0.3.3 as the earliest version that supports
python 3 without any known regressions. Add an extra parser check for
missing 'command' due to changes in how argparse works under python 3.
Where contents should be retained, to access the first element of a dict
in both python 2 and 3, 'next(iter(dict.items()))' is used as the
standard idiom to replace 'dict.items()[0]' as 'items()' returns an
iterator in python 3 which cannot be indexed. Using 'next(iter(..))'
allows for both lists and iterators to be passed in without unnecessary
conversion of iterators to lists which would be true of
'list(dict.items())[0]'.
Alternatively, where further access to the data is not required,
'dict.popitem()' is used.
Change-Id: If4b35e2ceee8239379700e22eb79a3eaa04d6f0f