Commit Graph

98 Commits

Author SHA1 Message Date
Vsevolod Fedorov 1ead727a2d Story 2011049: Add --force flag to update-all command
Without it, using this command in scripts is impossible.

Change-Id: Ifd44fe0171a7753c9a09e9b05047ce2965c183f3
2024-03-25 13:08:25 +03:00
Vsevolod Fedorov eb08e60765 Story 2011049: Delete also old views when --delete-old option is specified
Improve 'update' command to handle old views also, not just jobs.

Task: 49601

Change-Id: I60431f662d5a69c084b2698d08104a1524d02767
2024-03-25 12:47:40 +03:00
Vsevolod Fedorov 1f320ad9f9 Do not delete unmanaged jobs when --views-only option specified
Change-Id: I54e4d62db56824bfa020bbc1a711b95a5b15fcc5
2024-03-25 12:20:43 +03:00
Vsevolod Fedorov d95a677d85 Add tests for 'update' command
Cover views and --jobs-only and --views-only options.

Change-Id: Ib325a09180162d46e2556e05eb6f982daea1dd21
2024-03-25 11:34:06 +03:00
Zuul f3d42c2fbb Merge "Rewrite YAML parser" 2023-03-02 08:49:49 +00:00
Thomas Bechtold f0b52ec41a Fix --enabled-only option when updating jobs
Commit 5e61fbaf00 added a new --enabled-only option but that
doesn't work correctly because the xml_jobs_filtered variable isn't
filled with all the jobs in the end. So some jobs are not updated even
if the disabled status was not found. That's fixed now.

Change-Id: I915116cf875e9e20c10ca43175c924b9e4307cca
2023-03-01 12:50:47 +01:00
Vsevolod Fedorov af9e03ec08 Rewrite YAML parser
Rewrite YAML parser, YAML objects and parameters expansion logic to
enable better control over expansion logic.
Broken backward compatilibity:
* More agressive parameter expansion. This may lead to parameters
  expanded in places where they were not expanded before.
* Top-level elements, which is not known to parser (such as 'job',
  'view', 'project' etc), are now lead to parse failures.
  Prepend them with underscore to be ignored by parser.
* Files included using '!include-raw:' elements and having formatting in
  it's path ('lazy-loaded' in previous implementation) are now expanded
  too.
  Use '!include-raw-escape:' for them instead.
  See changes in these tests for examples:
    tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml
    tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml
    tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml
* Parameters with template value using itself were substituted as is.
  For example: "timer: '{timer}'" was expanded to "{timer}". Now it
  leads to recursive parameter error.
  See changes in this test for example:
    tests/yamlparser/job_fixtures/parameter_name_reuse_default.*
    ->
    tests/yamlparser/error_fixtures/parameter_name_reuse_default.*
* When job group includes a job which was never declared, it was just
  ignored. Now it fails: job is missing.
  See changes in this test for example:
    tests/yamlparser/job_fixtures/job_group_includes_missing_job.*
    ->
    tests/yamlparser/error_fixtures/job_group_includes_missing_job.*

Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
2023-02-28 20:16:57 +03:00
Thomas Bechtold 5e61fbaf00 Add config option --enabled-only
That option can be used when calling the update subcommand. If the
option is set, only jobs which are enabled will be updated.
That is useful of there are a lot of disabled jobs which are not
required.

Change-Id: Ie2f268e2356a3b7e0a334a331696ca81e09790aa
2023-02-22 09:11:00 +01:00
Vsevolod Fedorov f4d64f9f66 Move tests to pytest
Pytest makes each scenario into individual selectable test.
To be able to run each scenario separately is very useful for development.

Change-Id: I4b1c990a1fd839ce327cd7faa27159a9b9632fed
2022-12-20 10:44:43 +03:00
Vsevolod Fedorov e51d4ec473 Run tox linters on zuul check
It will prevent committing code rejected by linters.
Commit changes performed by black.

Change-Id: I5abe7f8aaec51c890bff355d0a544edae2992d5d
2022-06-09 11:13:40 +03:00
wangdonghui f6556f5636 ListSubCommand: return job fullname
Change-Id: Id1e53d05383efad191ad254ff0f17e3c573aeb65
2022-04-25 19:42:05 +08:00
Adam Romanek 118ac858c4 tests: fix mixed imports of 'unittest.mock' and 'mock'
The 'mock' package is imported in 'tests.base'. Whenever possible
'tests.base' prefers 'unittest.mock' over 'mock' (which is the case on
Python 3.3+). Importing 'mock' together with 'unittest.mock' causes
unexpected behavior that results in exceptions when running tests, at
least on Python 3.6.12, like:

>  File "/usr/lib/python3.6/unittest/mock.py", line 1196, in patched
>    patching.__exit__(*exc_info)
>
>  File "/home/zuul/src/opendev.org/jjb/jenkins-job-builder/.tox/cover/lib/python3.6/site-packages/mock/mock.py", line 1545, in __exit__
>    return exit_stack.__exit__(*exc_info)
>
>  File "/usr/lib/python3.6/contextlib.py", line 339, in __exit__
>    received_exc = exc_details[0] is not None
>
>  IndexError: tuple index out of range

From now on, to avoid similar issues in the future, the 'mock' package
is only installed for Python 2.x.

Change-Id: Ib3044312f0770735d88d8657d1f953e5a33e771f
2021-01-01 12:58:28 +01:00
Thanh Ha 5a98ebb8bb
Fix flake8 BLK100 and BLK999 issues
Resolves flake8 BLK related issues.

"BLK100 Black would make changes."
"BLK999 Unexpected exception: ..."

Change-Id: Ic6d4265e3d02b2f2a15ab851eaed98288c62730c
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
2020-08-31 10:54:19 -04:00
Eren ATAS 2e84bdf5b6 Removed the unnecessary empty combinationFilter Tag
In the current implementation, jjb creates an empty
combinationFilter tag whenever there is `executionStrategy`.
This empty tag should not appear.

Change-Id: I2bf25ae58d12904ce0a00e0a5cfefbf27d9451ad
Signed-off-by: Eren ATAS <eatas.contractor@libertyglobal.com>
2020-07-07 11:32:51 +02:00
Daniel Bengtsson 0b4ed38ca6 Fix the yaml load warning.
In the new version of PyYAML the API changed to be more explicit. Now
the default value for the Loader is None, see:

https://github.com/yaml/pyyaml/blob/5.1/lib3/yaml/__init__.py#L103

The load is unsafe. It's better to use safe_load function.

Change-Id: Ia1cd16f2ff970ca220a266c99b6554dd4254ebd9
2019-12-10 09:52:40 +01:00
Thanh Ha 4d90c187a9 Auto-generated output from python-black
Please review the following patch containing the code changes in
the repo. This patch is a transition patch and is the auto-generated
output of the python-black tool.

Change-Id: I2d2de71da8a105fb62b561899ae78441ddab4032
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
2019-09-09 19:23:24 +01:00
Sorin Sbarnea 71f1c89b65
remove support of old plugin config format
Remove code refering to old plugin config format.

Change-Id: Iba0922d3bfe066444e5e982c303968d15e1853a1
2019-04-16 09:08:05 +08:00
Anil Belur 9399b1f986
Add config options --jobs-only and --views-only
Add options --jobs-only and --views-only to job_builder
section in config file.

By default JJB updates both jobs and views. Some cases
non-admin users are not allowed to update views on Jenkins
which requires explicity using -j flag.

Allow users to set a config option 'update=jobs|views|all'
in the 'job_builder' section implicitly use --jobs-only or
--views-only and control these flags from the config file.

Note: CLI options takes precedence over the config file.

Change-Id: I2a94e5a2d671ccbfc505de2f19b578ecfef9e9d7
Co-Authored-By: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Anil Belur <askb23@gmail.com>
Signed-off-by: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Anil Belur <askb23@gmail.com>
2018-09-19 02:24:05 +05:30
Anil Belur 81f69c6f0d
Fix typo in test docstring
Change-Id: I28f7b19319fdbbb8f5b2d582596643d85fc3efb9
Signed-off-by: Anil Belur <askb23@gmail.com>
2018-07-17 20:38:53 +10:00
Sorin Sbarnea c2781e670f
Normalize newlines at end of file
Preparatory move for adopting pre-commit hooks.
Shoudl only remove newlines and spaces at end of files.

Change-Id: Ia33679467c533722b9308f7e7b74be395c926428
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
2018-06-29 16:07:06 -04:00
Sorin Sbarnea 4f7b6ee568
upgrade hacking module
- upgrades hacking to current version
- sorts new linting issues
- sorts bug with sys.reload on py3

Change-Id: I4a18abc93116667a2733e8aec619ac59ea73d630
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
2018-06-19 18:04:37 +01:00
Zuul 1614f4f992 Merge "Fixes delete-all command when no option is provided" 2018-06-01 00:23:58 +00:00
Truc Nguyen c051dd8502
Fixes delete-all command when no option is provided
Change-Id: Ibc06209c2ec19045e12a145083a64a179d57d731
Implements: fix for delete-all command without option
Closes-Bug: #2001207
2018-05-31 19:07:12 +01:00
Daniel Watkins 3e72803eec Make the deprecated plugin config messaging clearer
At the moment, if you don't know that you have configuration for the
given plugin in your config file, there's nothing that actually suggests
that's where you should start looking to fix this.

Change-Id: I32481eb9b1a4feafc8b8ca2cec63e4b9a0b29441
Story: 2001735
Task: 12090
2018-05-30 16:27:07 -04:00
Marc Abramowitz 8d4671e3a7
Add a "list" subcommand
Add a list subcommand that allows listing of all the jobs defined in
yaml configuration, or on the Jenkins master, with the option of taking
glob parameters to allow matching some job names.

Also supports the recursive and exclude path control options.

  jenkins-jobs list 'wikimedia-fundraising*'

  jenkins-jobs list -r -p configs/ 'wikimedia-fundraising*'

Co-Authored-By: Darragh Bailey <dbailey@hpe.com>
Co-Authored-By: Sorin Sbarnea <ssbarnea@redhat.com>
Change-Id: I897b9ed35561e455dc6b89c3bacec74b54777903
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
2018-03-09 10:58:21 -05:00
Zuul 5e023d0301 Merge "Capture XML output from test" 2018-03-07 15:23:08 +00:00
Darragh Bailey 3fe85e5bbd Capture XML output from test
Running a test with input yaml will result in output XML being sent
to the stdout. Use mock on the console to capture this so that running
the tests via testtools directly doesn't output unnecessarily

Change-Id: I4ccbfcb464d40a77c04913cb8281a0167504e1a4
2018-03-07 14:14:38 +00:00
Zuul 96aa1c756f Merge "re-raise the original Exception instead of raising a new one" 2018-02-23 16:36:23 +00:00
Darragh Bailey 630fc6e88a No warn on old plugin conf format not in use
Use a simple object to ensure matching against any default value is
possible to distinguish been not being set by a user versus use of
a negative default value passed.

This prevents triggering an incorrect notification to the user when
the code is querying for plugin options that could override the
default behaviour and the compatibility code attempts to look up an non
existing section. It would fail to distinguish between no section
present and current value to return matching the default value
suggested.

Change-Id: I5597c2628ccb5a4282a97a4ce5d3bbe41bd9eebb
2017-12-15 14:18:11 +00:00
Evgeni Golov f2c4453df3 re-raise the original Exception instead of raising a new one
this preserves the error traceback and makes debugging easier

Change-Id: Ia513ac009b81d35e3df11939771aa29e40700c08
2017-11-06 13:36:17 +01:00
Darragh Bailey af9d984baa
Basic folder support
Allows specifying a folder attribute for each job generated, which in
turn is used when creating or uploading to place the job under the
requested folder.

The job name is expanded after defaults are applied, to support the
attribute being defined within a set of defaults applied to a number of
jobs.

This in turn allows for multiple jobs with the same basename to exist,
provided they are targeted at different folders.

Does not support creating the folders if they do not exist.

Change-Id: I8c2157c4c81087cc972a048d1b88d5f08ac65361
2017-10-23 13:29:21 -04:00
zhangyangyang 8744b5b1a5 change assert(Not)Equals to assert(Not)Equal
According to http://docs.python.org/2/library/unittest.html
assert(Not)Equals is a deprecated alias of assert(Not)Equal.

Change-Id: Ibadf2b2c6cb09db6f5cc191240aad2ac8cb4ea33
Closes-Bug: #1329757
2017-09-13 17:24:13 +08:00
Peter Eisentraut 919052714c Add option to make test output file layout match Jenkins
Before, `jenkins-job-builder test -o outdir` would write

    outdir/job_name

With the new option `--config-xml`, it writes

    outdir/job_name/config.xml

to match the layout that Jenkins uses internally.  This is more useful
for diffing against existing Jenkins installations or uploading job
definitions by other means.

Also, it seems better to give the output files an .xml extension.

Change-Id: I78fc7aff531925ee071b8e19f40f766b4af0b5ad
2017-05-27 12:22:06 +01:00
Jenkins 4f77324fea Merge "Wrapper pre-scm-buildstep: added support for failOnError parameter" 2017-03-29 09:55:34 +00:00
Sandro Gattuso 12c2ca3396 Wrapper pre-scm-buildstep: added support for failOnError parameter
- Added failOnError argument for pre-scm-buildstep
- Introduced buildsteps argument and moved list of build steps to match xml configuration of the preSCMbuildstep plugin.
- Updated existing test cases to match the new argument structure
- Added test case for pre-scm-buildstep to verify creation of failOnError tag
- Added verification on version to maintain retro-compatibility
- Updated documentation to fetch example from testcase definition

Change-Id: I25c19cd0552cf2de97b9c391492d95779c1360be
Task: 3163
Story: 2000690
2017-03-28 17:10:29 +00:00
Sorin Sbarnea 8a7d77c180 Removed some deprecation warnings and enabled py35 on tox.
Change-Id: Icc5b28c4636b542a10502b7b1d2d2dc2028cc166
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
2017-03-12 19:43:33 +00:00
Jenkins 8a1c91327e Merge "Test stream no encoding bug" 2016-12-01 06:58:59 +00:00
Jenkins e47f9629bc Merge "Add view management functionality" 2016-11-10 16:35:23 +00:00
Jenkins f0c99b70a7 Merge "user's input should not be interpreted" 2016-10-27 20:59:03 +00:00
Isaac Beckman 4b63891dd7 user's input should not be interpreted
using python's "input" rather than "raw_input" raise exception
since the user's input is interpreted.
In python 3.x raw_input = input, so the interpertation bug
only exists in python 2.x

Change-Id: I28781a8742d6e03bc850fd0178f317474603a9d2
2016-10-26 13:03:30 +03:00
Jenkins 771a94227c Merge "Regression test for update command with --delete-old" 2016-10-25 18:08:09 +00:00
Thanh Ha 1deb3aff4c
Add view management functionality
- Adds the ability for JJB to work with views
- Views can be created, updated, and deleted.
- New modules for List view and Build Pipeline view are added
- New tests for testing the deletion of views

Example View configuration:

    - view:
        name: MyView
        view-type: list

Change-Id: Idb29a4407bcc14593e10a4d951036cb04e8e6c27
Co-Authored-By: Brandon Leonard <brandon.leonard@rackspace.com>
Co-Authored-By: Joao Vale <jpvale@gmail.com>
Co-Authored-By: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
2016-10-14 09:14:21 -04:00
Jenkins 02ae9dc8ed Merge "Fix confirm() util true state" 2016-09-30 16:32:47 +00:00
Thanh Ha fd01e9e916
Fix confirm() util true state
The confirm() util incorrectly returns true only when the answer is
false. This patch reverses the logic so that it properly returns true
when the user inputs a true statement.

Include test case for delete-all command usage to accept or abort.
Mocking the input builtin ensures that the behaviour resulting from the
user inputting 'y' or 'n' will remain tested for delete-all.

Change-Id: I742804a8efe7da8ca6aa70f20059d02758910deb
Co-Authored-By: Darragh Bailey <dbailey@hpe.com>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
2016-09-20 22:16:29 -04:00
Darragh Bailey d75f1436cd Regression test for update command with --delete-old
Ensure that we explicitly check that there were no extra calls in
addition to the expected set of calls to the delete jobs API, when
populating keep_jobs with the list of jobs updated and checking that
the unmanaged job is ignored.

Correct the list of jobs returned by get_jobs call.

Prevent issue introduced by: https://review.openstack.org/352060

Change-Id: I5b7452d593707db7085e7d305893f05a77be82a1
2016-09-16 16:45:19 +01:00
Darragh Bailey 4573b3a25d Refactor base test classes inheritance for reuse
Refactor base test class inheritance to allow for BaseTest class to be
reused without needing to also import the TestCase class everywhere in
order to avoid having the common test function executed on base classes.

This makes it easier to build base testing classes and then simplifies
the subclassing of these for actual tests.

Change-Id: I89809e8082469f814f245db4a9ab7658aac8a405
2016-09-14 14:58:57 +01:00
Wayne Warren e9a295b1fb Move CacheStorage to its own module.
And rename it to JobCache. The word "Cache" already implies "Storage"
so it makes more sense to name it after the things that it
stores...Jobs!

Change-Id: If7a7a56e8f6a3637f62e285eaa7ba989f67002dc
2016-09-02 08:58:09 -07:00
Wayne Warren e75062d103 Merge builder.Builder and builder.Jenkins
Since both of these classes essentially build on basic functionality
provided by jenkins.Jenkins, merge them into a single class. Name this
class "JenkinsManager" to avoid confusion with jenkins.Jenkins.

Also moves some tests for update timeout configuration into the
tests.cmd.test_config module from tests.cmd.subcommand.test_update
module, replacing those tests with skipped stubs as reminders to
figure out a better way later.

Change-Id: I13c17bc90e29e702e5e02992e93cf3cdc689731d
2016-09-02 08:58:09 -07:00
Darragh Bailey a949a274e3
Test stream no encoding bug
Tests that if no encoding is defined on the output that JJB does not
accidentally trigger a recursion issue when iterating over a large
number of jobs and attempting to wrap the output stream.

Inspired by commit message from: https://review.openstack.org/283441

Change-Id: Ie69c3af0beb9fd7c589a436e8c6d2862210f8a4c
Co-Authored-By: Hao Liu <hliu@redhat.com>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
2016-08-29 23:43:35 -04:00
Wayne Warren 2236a00c53 Rename Builder.delete_job to Builder.delete_jobs.
Change-Id: If57a98a00e9577fdf7a3d7585905a2102d8c61cd
2016-08-20 11:28:00 -07:00