Commit 5e61fbaf00 added a new --enabled-only option but that
doesn't work correctly because the xml_jobs_filtered variable isn't
filled with all the jobs in the end. So some jobs are not updated even
if the disabled status was not found. That's fixed now.
Change-Id: I915116cf875e9e20c10ca43175c924b9e4307cca
Rewrite YAML parser, YAML objects and parameters expansion logic to
enable better control over expansion logic.
Broken backward compatilibity:
* More agressive parameter expansion. This may lead to parameters
expanded in places where they were not expanded before.
* Top-level elements, which is not known to parser (such as 'job',
'view', 'project' etc), are now lead to parse failures.
Prepend them with underscore to be ignored by parser.
* Files included using '!include-raw:' elements and having formatting in
it's path ('lazy-loaded' in previous implementation) are now expanded
too.
Use '!include-raw-escape:' for them instead.
See changes in these tests for examples:
tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml
tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml
tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml
* Parameters with template value using itself were substituted as is.
For example: "timer: '{timer}'" was expanded to "{timer}". Now it
leads to recursive parameter error.
See changes in this test for example:
tests/yamlparser/job_fixtures/parameter_name_reuse_default.*
->
tests/yamlparser/error_fixtures/parameter_name_reuse_default.*
* When job group includes a job which was never declared, it was just
ignored. Now it fails: job is missing.
See changes in this test for example:
tests/yamlparser/job_fixtures/job_group_includes_missing_job.*
->
tests/yamlparser/error_fixtures/job_group_includes_missing_job.*
Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
That option can be used when calling the update subcommand. If the
option is set, only jobs which are enabled will be updated.
That is useful of there are a lot of disabled jobs which are not
required.
Change-Id: Ie2f268e2356a3b7e0a334a331696ca81e09790aa
Pytest makes each scenario into individual selectable test.
To be able to run each scenario separately is very useful for development.
Change-Id: I4b1c990a1fd839ce327cd7faa27159a9b9632fed
The 'mock' package is imported in 'tests.base'. Whenever possible
'tests.base' prefers 'unittest.mock' over 'mock' (which is the case on
Python 3.3+). Importing 'mock' together with 'unittest.mock' causes
unexpected behavior that results in exceptions when running tests, at
least on Python 3.6.12, like:
> File "/usr/lib/python3.6/unittest/mock.py", line 1196, in patched
> patching.__exit__(*exc_info)
>
> File "/home/zuul/src/opendev.org/jjb/jenkins-job-builder/.tox/cover/lib/python3.6/site-packages/mock/mock.py", line 1545, in __exit__
> return exit_stack.__exit__(*exc_info)
>
> File "/usr/lib/python3.6/contextlib.py", line 339, in __exit__
> received_exc = exc_details[0] is not None
>
> IndexError: tuple index out of range
From now on, to avoid similar issues in the future, the 'mock' package
is only installed for Python 2.x.
Change-Id: Ib3044312f0770735d88d8657d1f953e5a33e771f
Resolves flake8 BLK related issues.
"BLK100 Black would make changes."
"BLK999 Unexpected exception: ..."
Change-Id: Ic6d4265e3d02b2f2a15ab851eaed98288c62730c
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
In the current implementation, jjb creates an empty
combinationFilter tag whenever there is `executionStrategy`.
This empty tag should not appear.
Change-Id: I2bf25ae58d12904ce0a00e0a5cfefbf27d9451ad
Signed-off-by: Eren ATAS <eatas.contractor@libertyglobal.com>
In the new version of PyYAML the API changed to be more explicit. Now
the default value for the Loader is None, see:
https://github.com/yaml/pyyaml/blob/5.1/lib3/yaml/__init__.py#L103
The load is unsafe. It's better to use safe_load function.
Change-Id: Ia1cd16f2ff970ca220a266c99b6554dd4254ebd9
Please review the following patch containing the code changes in
the repo. This patch is a transition patch and is the auto-generated
output of the python-black tool.
Change-Id: I2d2de71da8a105fb62b561899ae78441ddab4032
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Add options --jobs-only and --views-only to job_builder
section in config file.
By default JJB updates both jobs and views. Some cases
non-admin users are not allowed to update views on Jenkins
which requires explicity using -j flag.
Allow users to set a config option 'update=jobs|views|all'
in the 'job_builder' section implicitly use --jobs-only or
--views-only and control these flags from the config file.
Note: CLI options takes precedence over the config file.
Change-Id: I2a94e5a2d671ccbfc505de2f19b578ecfef9e9d7
Co-Authored-By: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Anil Belur <askb23@gmail.com>
Signed-off-by: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Anil Belur <askb23@gmail.com>
Preparatory move for adopting pre-commit hooks.
Shoudl only remove newlines and spaces at end of files.
Change-Id: Ia33679467c533722b9308f7e7b74be395c926428
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
- upgrades hacking to current version
- sorts new linting issues
- sorts bug with sys.reload on py3
Change-Id: I4a18abc93116667a2733e8aec619ac59ea73d630
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
At the moment, if you don't know that you have configuration for the
given plugin in your config file, there's nothing that actually suggests
that's where you should start looking to fix this.
Change-Id: I32481eb9b1a4feafc8b8ca2cec63e4b9a0b29441
Story: 2001735
Task: 12090
Add a list subcommand that allows listing of all the jobs defined in
yaml configuration, or on the Jenkins master, with the option of taking
glob parameters to allow matching some job names.
Also supports the recursive and exclude path control options.
jenkins-jobs list 'wikimedia-fundraising*'
jenkins-jobs list -r -p configs/ 'wikimedia-fundraising*'
Co-Authored-By: Darragh Bailey <dbailey@hpe.com>
Co-Authored-By: Sorin Sbarnea <ssbarnea@redhat.com>
Change-Id: I897b9ed35561e455dc6b89c3bacec74b54777903
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
Running a test with input yaml will result in output XML being sent
to the stdout. Use mock on the console to capture this so that running
the tests via testtools directly doesn't output unnecessarily
Change-Id: I4ccbfcb464d40a77c04913cb8281a0167504e1a4
Use a simple object to ensure matching against any default value is
possible to distinguish been not being set by a user versus use of
a negative default value passed.
This prevents triggering an incorrect notification to the user when
the code is querying for plugin options that could override the
default behaviour and the compatibility code attempts to look up an non
existing section. It would fail to distinguish between no section
present and current value to return matching the default value
suggested.
Change-Id: I5597c2628ccb5a4282a97a4ce5d3bbe41bd9eebb
Allows specifying a folder attribute for each job generated, which in
turn is used when creating or uploading to place the job under the
requested folder.
The job name is expanded after defaults are applied, to support the
attribute being defined within a set of defaults applied to a number of
jobs.
This in turn allows for multiple jobs with the same basename to exist,
provided they are targeted at different folders.
Does not support creating the folders if they do not exist.
Change-Id: I8c2157c4c81087cc972a048d1b88d5f08ac65361
Before, `jenkins-job-builder test -o outdir` would write
outdir/job_name
With the new option `--config-xml`, it writes
outdir/job_name/config.xml
to match the layout that Jenkins uses internally. This is more useful
for diffing against existing Jenkins installations or uploading job
definitions by other means.
Also, it seems better to give the output files an .xml extension.
Change-Id: I78fc7aff531925ee071b8e19f40f766b4af0b5ad
- Added failOnError argument for pre-scm-buildstep
- Introduced buildsteps argument and moved list of build steps to match xml configuration of the preSCMbuildstep plugin.
- Updated existing test cases to match the new argument structure
- Added test case for pre-scm-buildstep to verify creation of failOnError tag
- Added verification on version to maintain retro-compatibility
- Updated documentation to fetch example from testcase definition
Change-Id: I25c19cd0552cf2de97b9c391492d95779c1360be
Task: 3163
Story: 2000690
using python's "input" rather than "raw_input" raise exception
since the user's input is interpreted.
In python 3.x raw_input = input, so the interpertation bug
only exists in python 2.x
Change-Id: I28781a8742d6e03bc850fd0178f317474603a9d2
- Adds the ability for JJB to work with views
- Views can be created, updated, and deleted.
- New modules for List view and Build Pipeline view are added
- New tests for testing the deletion of views
Example View configuration:
- view:
name: MyView
view-type: list
Change-Id: Idb29a4407bcc14593e10a4d951036cb04e8e6c27
Co-Authored-By: Brandon Leonard <brandon.leonard@rackspace.com>
Co-Authored-By: Joao Vale <jpvale@gmail.com>
Co-Authored-By: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
The confirm() util incorrectly returns true only when the answer is
false. This patch reverses the logic so that it properly returns true
when the user inputs a true statement.
Include test case for delete-all command usage to accept or abort.
Mocking the input builtin ensures that the behaviour resulting from the
user inputting 'y' or 'n' will remain tested for delete-all.
Change-Id: I742804a8efe7da8ca6aa70f20059d02758910deb
Co-Authored-By: Darragh Bailey <dbailey@hpe.com>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
Ensure that we explicitly check that there were no extra calls in
addition to the expected set of calls to the delete jobs API, when
populating keep_jobs with the list of jobs updated and checking that
the unmanaged job is ignored.
Correct the list of jobs returned by get_jobs call.
Prevent issue introduced by: https://review.openstack.org/352060
Change-Id: I5b7452d593707db7085e7d305893f05a77be82a1
Refactor base test class inheritance to allow for BaseTest class to be
reused without needing to also import the TestCase class everywhere in
order to avoid having the common test function executed on base classes.
This makes it easier to build base testing classes and then simplifies
the subclassing of these for actual tests.
Change-Id: I89809e8082469f814f245db4a9ab7658aac8a405
And rename it to JobCache. The word "Cache" already implies "Storage"
so it makes more sense to name it after the things that it
stores...Jobs!
Change-Id: If7a7a56e8f6a3637f62e285eaa7ba989f67002dc
Since both of these classes essentially build on basic functionality
provided by jenkins.Jenkins, merge them into a single class. Name this
class "JenkinsManager" to avoid confusion with jenkins.Jenkins.
Also moves some tests for update timeout configuration into the
tests.cmd.test_config module from tests.cmd.subcommand.test_update
module, replacing those tests with skipped stubs as reminders to
figure out a better way later.
Change-Id: I13c17bc90e29e702e5e02992e93cf3cdc689731d
Tests that if no encoding is defined on the output that JJB does not
accidentally trigger a recursion issue when iterating over a large
number of jobs and attempting to wrap the output stream.
Inspired by commit message from: https://review.openstack.org/283441
Change-Id: Ie69c3af0beb9fd7c589a436e8c6d2862210f8a4c
Co-Authored-By: Hao Liu <hliu@redhat.com>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>