Commit 5e61fbaf00 added a new --enabled-only option but that
doesn't work correctly because the xml_jobs_filtered variable isn't
filled with all the jobs in the end. So some jobs are not updated even
if the disabled status was not found. That's fixed now.
Change-Id: I915116cf875e9e20c10ca43175c924b9e4307cca
Rewrite YAML parser, YAML objects and parameters expansion logic to
enable better control over expansion logic.
Broken backward compatilibity:
* More agressive parameter expansion. This may lead to parameters
expanded in places where they were not expanded before.
* Top-level elements, which is not known to parser (such as 'job',
'view', 'project' etc), are now lead to parse failures.
Prepend them with underscore to be ignored by parser.
* Files included using '!include-raw:' elements and having formatting in
it's path ('lazy-loaded' in previous implementation) are now expanded
too.
Use '!include-raw-escape:' for them instead.
See changes in these tests for examples:
tests/yamlparser/job_fixtures/lazy-load-jobs-multi001.yaml
tests/yamlparser/job_fixtures/lazy-load-jobs-multi002.yaml
tests/yamlparser/job_fixtures/lazy-load-jobs001.yaml
* Parameters with template value using itself were substituted as is.
For example: "timer: '{timer}'" was expanded to "{timer}". Now it
leads to recursive parameter error.
See changes in this test for example:
tests/yamlparser/job_fixtures/parameter_name_reuse_default.*
->
tests/yamlparser/error_fixtures/parameter_name_reuse_default.*
* When job group includes a job which was never declared, it was just
ignored. Now it fails: job is missing.
See changes in this test for example:
tests/yamlparser/job_fixtures/job_group_includes_missing_job.*
->
tests/yamlparser/error_fixtures/job_group_includes_missing_job.*
Change-Id: Ief4e515f065a1b9e0f74fe06d7e94fa77d69f273
That option can be used when calling the update subcommand. If the
option is set, only jobs which are enabled will be updated.
That is useful of there are a lot of disabled jobs which are not
required.
Change-Id: Ie2f268e2356a3b7e0a334a331696ca81e09790aa
Please review the following patch containing the code changes in
the repo. This patch is a transition patch and is the auto-generated
output of the python-black tool.
Change-Id: I2d2de71da8a105fb62b561899ae78441ddab4032
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Add options --jobs-only and --views-only to job_builder
section in config file.
By default JJB updates both jobs and views. Some cases
non-admin users are not allowed to update views on Jenkins
which requires explicity using -j flag.
Allow users to set a config option 'update=jobs|views|all'
in the 'job_builder' section implicitly use --jobs-only or
--views-only and control these flags from the config file.
Note: CLI options takes precedence over the config file.
Change-Id: I2a94e5a2d671ccbfc505de2f19b578ecfef9e9d7
Co-Authored-By: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Anil Belur <askb23@gmail.com>
Signed-off-by: Thanh Ha <zxiiro@linux.com>
Signed-off-by: Anil Belur <askb23@gmail.com>
Additional argument for the update subcommand to allow user to update
only jobs currently installed on the Jenkins server and managed by
this tool.
Change-Id: I4b0e997df01057f89896b8caa27dc00b5ca077df
JJB 2.0 update does not allow updating only views
or jobs which would be useful, which is implemented with
delete and delete-all. This patch add's thoses flags
for the update option.
Change-Id: I3712c8552ea83a4c29c05461894c4464aa41230f
Signed-off-by: Anil Belur <askb23@gmail.com>
Getting plugin info requires Administrator permissions. It seems the
test command allows providing the plugin_info details via a yaml
configuration file. This patch allows the same command to be passed to
the update command to allow us to configure the plugin versions manually
and not require administrator permissions.
Additionally this patch adds a new command called get-plugins-info
which can create the plugins_info.yaml file to pass to the
plugin_info_path.
See:
http://lists.openstack.org/pipermail/openstack-infra/2016-June/004388.html
Change-Id: I5a34979407d863a84f34afbf8f565081ec52190a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
- Adds the ability for JJB to work with views
- Views can be created, updated, and deleted.
- New modules for List view and Build Pipeline view are added
- New tests for testing the deletion of views
Example View configuration:
- view:
name: MyView
view-type: list
Change-Id: Idb29a4407bcc14593e10a4d951036cb04e8e6c27
Co-Authored-By: Brandon Leonard <brandon.leonard@rackspace.com>
Co-Authored-By: Joao Vale <jpvale@gmail.com>
Co-Authored-By: Lucas Dutra Nunes <ldnunes@ossystems.com.br>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
Since both of these classes essentially build on basic functionality
provided by jenkins.Jenkins, merge them into a single class. Name this
class "JenkinsManager" to avoid confusion with jenkins.Jenkins.
Also moves some tests for update timeout configuration into the
tests.cmd.test_config module from tests.cmd.subcommand.test_update
module, replacing those tests with skipped stubs as reminders to
figure out a better way later.
Change-Id: I13c17bc90e29e702e5e02992e93cf3cdc689731d
Separate XML generation from Yaml parsing/interpreting. The goal here
is to allow different sources to provide data for XML generation,
including external API users writing job definitions in pure Python or
JJB developers who would like to work on alternative Yaml parsing code
since the current YamlParser has very likely reached the limits of
what can be reasonably done with one giant expandYaml procedure.
Change-Id: I9da848acac7e944c0e07286b7399b2e1956a58a5
Create the ModuleRegistry anywhere other than inside the YamlParser
class. This will make it slightly easier to factor a XmlGenerator out
of YamlParser, but I also want to work toward eliminating the circular
references between YamlParser and ModuleRegistry which have been
making it difficult to understand overall program flow.
This commit also replaces all YamlParser instances being passed to
Jenkins job config generating functions with a ModuleRegistry. Mostly
it seems like the parser was only needed to call the ModuleRegistry's
'dispatch' method which to be honest I don't fully understand. This is
where the circular references mentioned in previously come in...it
seems like the "dispatch" function needs access to the (mostly) raw
data contained by the parser, so it took that as a parameter.
The need for the YamlParser's job data can be satisfied by assigning
it to a property on the ModuleRegistry object before Yaml expansion or
XML generation begins; by doing this, we allow the ModuleRegistry to
avoid referencing the parser.
Change-Id: I4b571299b81e708540392ad963163fe092acf1d9
Fixes a regression where using --delete-old will delete jobs that are
not updated for example in the case of caching and jenkins-jobs
determines a job does not need to be updated.
jobs returns a list of jobs that were updated via jenkins-jobs while
xml_jobs returns a list of all jobs generated jobs regardless of if they
were updated or not.
Change-Id: I8f9d0b609d1a7cc0a526d479b4f0a2448c25aba3
Fixes bug introduced by change
Id465078da2a7016fd59407516fe124602abd016c causing all jobs to be
unintentionally deleted on update.
The builder.delete_old_managed()'s keep parameter was expected a
string list of job names but we were mistakenly passing in a list
of XmlJob objects.
Change-Id: I4051925337691d4b59796d21f56cacf4152de1d4
Co-Authored-By: Kien Ha <kienha9922@gmail.com>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
Sharing code between these two classes will make it easier to work on
extracting an XmlGenerator class from the YamlParser.
Change-Id: If93227d0b4fca4d7bb80de2341d1e83f1e3ddc79
This commit intentionally introduces a number of important API
breakages. Specifically, the jenkins_jobs.cmd module has been pared
down to some of its most difficult-to-refactor elements.
* Create jenkins_jobs.cli.entry.JenkinsJobs class to organize command
line parsing and execution.
* Remove references to ConfigParser object in test code, hidden as an
implementation detail of JenkinsJobs command line parsing. This will
be necessary in the next stage of JJB 2.0 code which will be to
create a JJBConfig object that handles logic and presentation of
configuration from various sources--defaults, command line
arguments, configuration file, and maybe environment variables in
the future.
* Remove references to Namespace object produced by argparse module.
Required rewrite of multipath & recursive path tests with a new
MatchesDir testtools Matcher class that validates the expected
output for a run of JJB against a given set of yamldirs with the
specified command line arguments.
* Use stevedore to dynamically load subcommand parsers.
* Move configuration loading/testing to its own test file. Also fix
the global vs home directory JJB config file test.
Change-Id: If62280418ba7319c313033ab387af4284237747e
Vsevolod Fedorov