new function get_build_artifact_as_bytes to replace get_build_artifact (incompatible with artifacts not json-formatted)
+ use a stream to download binary artifacts
all artifacts are returned as bytes to avoid encoding issues
and an exception NotFoundException is raised in case of missing artifact
Closes-Bug: #1973243
Change-Id: I24ce4ecd854f8a19ed4d760404adb7d1ac6b5509
urllib3 1.x used to accept socket._GLOBAL_DEFAULT_TIMEOUT as a sentinal
object to mean "no configured timeout".
In urllib3 2.x, urllib3 uses its own _DEFAULT_TIMEOUT sentinal object,
and it rejects socket._GLOBAL_DEFAULT_TIMEOUT.
Assign our own DEFAULT_TIMEOUT constant to the newer object if it
exists, and fall back to the old behavior on old urllib3 versions.
Co-authored-by: Vsevolod Fedorov <vsevolod.fedorov@gmail.com>
Closes-Bug: #2018567
Change-Id: Ic626ba0e8ed79eec3a63ffab6cc02f91aa545ab1
Use full job name in `get_job_info_regex` instead of just the name, so
that matches will work in folders as well.
Closes-Bug: 1685575
Change-Id: I97ba8ca004387b376a511ba52fc9fdd2aeb98a63
When retreiving data from builds, the build number can have a string portion.
This is for matrix jobs that define a label to retrieve one sub-build of the
matrix job.
The URL structure is:
https://<jenkins_server>/job/>job_name>/<number>/label=<configuration>/
Allowing a string for build number allows for it to be specifed as, e.g.:
1/label=amd64
And whilst being somewhat a workaround, works without introducing complexity
and modifying function calls for extra parameters.
Change-Id: Ic409df6e20e85d6a417c6490472b3729f1f20cd2
Enable calls to retreive artifacts from Jenkins and also to pull
the stages data from the wfapi endpoint.
Change-Id: I59eb403d54416ed45a918d98fc2f43c3c02441d0
Pipeline jobs in in Jenkins 2.x appear as placeholder tasks for their
entire lifetime when queried via the API. The fix for #1659787
introduced completly ignoring placeholder tasks. Thus pipeline jobs are
never included in the list returned by get_running_builds.
This commit will change the behaviour of get_running_builds as follows:
- Placeholder tasks will be included: So running pipeline jobs
will be included.
- Builds which do not have a build number yet will be ignored:
This prevents the exception described in #1659787.
Change-Id: I30351ec78d6186b0674e3c08be00076b0ce2333e
Closes-Bug: #1750372
Related-Bug: #1659787
Previously, reconfig_credential() passed empty `data` with
`Content-Length: 0` to the credentials configuration rather
than the credentials data.
This patch fixes the issue by passing the config_xml data to
the configuration API to properly update the credential.
Change-Id: Idef50f5a31d55991698b6217f55f15a9308b8526
The repo uses docs-on-read-the-docs, remove extra
build-openstack-sphinx gate job.
Without this change, two docs jobs would be building in gate, tox-docs
and build-sphinx. One is enough.
Also, use templates instead of individual jobs.
Import other templates from project-config, use the failing docs
template since readthedocs is not setup yet.
Remove publish-to-pypi, this should stay in project-config.
Fix sphinx requirements for python 3.
Fix decoding in failing test.
Change-Id: Ic4f715b45c1284ea49339e96c2aa87b2e81fcbea
Make sure to use fullName from the API results to correctly
compute the URL for the additional API request.
Change-Id: I7ca1f2e15a6c40883ba376b182c2c991ca76e8b2
Closes-Bug:#1828051
Other specific requests may need a body. Thus remove the general check
for an empty body and add a more specific one where needed.
I noticed this because when updating a Jenkins job with Jenkins Job
Builder I got an EmptyResponseException even though updating worked
correctly.
Change-Id: Ie144e6fb0b922a681e6c65ffbe5120da72a495f0
Now that jobs_in_folder isn't reused in all other get_(all_)jobs tests
there's no more any reason to keep jobs_in_folder_named_job a separate
test.
Change-Id: Ic51a4b2cac3105cf9406d2a7864943a7df1248be
On our Jenkins instance with almost a hundred folders, JJB update
stalls for quite a while because it calls get_all_jobs. When invoked
locally at the Jenkins master, it's a matter of seconds, on a fast
broadband link and VPN, it's 2 minutes, and on a train it's easily 10
minutes.
But there's trick!
curl \
--show-error --silent --fail \
--user : --negotiate \
--get \
--data-urlencode \
tree=jobs\[url\,name\,jobs\[url\,name\,jobs\[url\,name\,\
jobs\[url\,name\,jobs\[url\,name\,jobs\[url\,name\,\
jobs\[url\,name\,jobs\[url\,name\,jobs\[url\,name\,\
jobs\[url\,name\,jobs\]\]\]\]\]\]\]\]\]\]
https://jenkins.example.com/api/json
This returns almost instantly.
And it gets better: if we fail to correctly guess the nesting level
necessary, Jenkins returns
…, "jobs": [{}, {}, …], …
so we can easily detect that we need to recurse deeper.
Change-Id: I7268259149e4bc8939c512a112c7e6ec1908224f
This abstraction did more harm than good -- pretended to create correct
JSON responses from a simpler structure, but created responses that were
very far from what Jenkins would return:
* "url" fields didn't contain folders, but nobody noticed as these
weren't used anyway
* "jobs" fields contained the string "null" (not JSON null, an actual
JSON string!), whereas real Jenkins returns a list of almost empty
objects (just a "class" field, if Jenkins is new enough, otherwise
nothing)
* "color" field is absent from folders
The usage of deepcopy, insert and append totally obstructed the real
contents of those responses from most readers. Let's just be a bit
verbose here, please.
Change-Id: Ida2cfa3662e491e4178228096dc622cdd859202f
Avoid repeatedly serializing and splitting the root path component of
jobs being added to the list to be processed and returned.
Attempting to split a string path using 'job' introduces a bug if a job
ever happens to also use the name 'job' as part of its name. This could
be fixed by using root.split('/job/') but it still seems hackish.
Maintain the root path to the folder in the hierarchy as a list of path
components, add to it when descending and store the result of joining
it with '/job/' as a separate variable to avoid needing to re-parse.
Change-Id: I074ad74ad935c50716141a79822ca84999734c3f
This would've caught that the info parameter is ignored in get_info and
I'm going to refactor/improve get_all_jobs so I need stricter tests
anyway.
Also, this makes test_unsafe_chars useful. Before it was a totally
meaningless test that didn't test anything at all. :-(
(Actually the get_info problem was caught by this test, but I'm
committing the fix first to not break git bisect.)
Change-Id: Ifd06bbbd9969a2739322e36ae83f521490a6eaa0
In some network setups Jenkins may be hidden behind complex reverse
proxy setups that require additional custom headers to be set on
each request in order to pass them trough
Allow providing such headers using JENKINS_API_EXTRA_HEADERS environment
variable
Change-Id: If071c5c707f916ba5f4f2c371ec600b7476bf723
This, I think, would've caused get_all_jobs to loop indefinitely if the
Jenkins instance has folders. Am I really the first to have caught this?
Change-Id: I4158b0d068135de08a3337486d8a4b41d8b7cda6
Detects if API url is redirected and corrects it in order
to avoid doubling number of requests and even failing
POST ones because of having them silently converted by
requests from POST to GET.
Displays a warning to the user when redirection occurs
because almost always this means that they are using an
old endpoint, like a non secured one.
Change-Id: I7387bf150dad307342f9a6a91afbae32859bc82e
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
If the keep_alive is not available because of using reverse
proxy, the header 'Content-Lenght' may not be available,
causing the exception:
Error communicating with server[...]: empty response
However, if 'Location' header is present, the response is valid.
Add unit tests:
- build_job must pass even if no 'content-lenght'
in response header
- build_job must fail if no 'location' in response header
Change-Id: I4da6dd19f9d8302a76652a3686a9377f9a2503a6
Closes-Bug:#1775047
Resolve regression caused by I5369a0d35be4bf8b3b197a51e60aba21b5742cc7
preventing run_scripts() from successfully running against the remote
Jenkins instance.
Switching data to a dictionary and passing the script string without
quote() appears to allow the script to successfully run.
Add a magic string ')]}.' as an indicator of if the groovy script
successfully passed or failed. In theory if the groovy script failed
perhaps due to syntax error or some other reason the magic string
will not be printed and we can assume failure.
Change-Id: Ibaffb768ea82c76b44ec5a6cccde8563afe1783f
Signed-off-by: Thanh Ha <zxiiro@linux.com>
To ensure consistent test behaviour need to mock the import of
requests_kerberos to be None only for the test classes based
off of the default JenkinsTestBase
Change-Id: I79903a6b3e4a1677fa1a1980af9f73c84b385828
Depends-On: I56c9580be9a90f5cda7349a148d467e6ff4e9270
Some API endpoints require the use of a filter or they will respond
with HTTP error code 418 I'm a teapot.
This was seen on CloudBees Jenkins Enterprise 2.107.2.1-rolling.
Adding a depth filter will ensure the API calls will succeed.
Change-Id: Ib4d6a251bf3a024a76081b2fc83baa7839ad4015
The test for kerberos was dropped in
34cca0c4d9, as it was not needed for the
simpler implementation with requests. Let's drop the requests-kerberos
from test dependencies as well, as it has only caused troubles in other
tests - at least for me.
One had to set jenkins.requests_kerberos to None, as was done in the
tests.base.JenkinsTestBase. In places where it was not set to None, the
jenkins.Jenkins.jenkins_open tried the kerberos authentication in
addition to anonymous authentication on the /api/json endpoint.
This was the case for
tests.test_jenkins_sockets.JenkinsRequestTimeoutTests.test_jenkins_open_timeout
where it has resulted in a different exception than was expected:
Traceback (most recent call last):
File "tests/test_jenkins_sockets.py", line 31, in test_jenkins_open_timeout
j.jenkins_open(request, add_crumb=False)
File "jenkins/__init__.py", line 533, in jenkins_open
return self.jenkins_request(req, add_crumb, resolve_auth).text
File "jenkins/__init__.py", line 547, in jenkins_request
self._maybe_add_auth()
File "jenkins/__init__.py", line 384, in _maybe_add_auth
% '\n'.join(failures))
jenkins.JenkinsException: Unable to authenticate with any scheme:
auth(kerberos) Error in request: HTTPConnectionPool(host='127.0.0.1', port=33921): Read timed out. (read timeout=0.1)
auth(anonymous) Error in request: HTTPConnectionPool(host='127.0.0.1', port=33921): Read timed out. (read timeout=0.1)
Change-Id: I45cbb3a666ee6a9ecbe628ca8b7f0b07dcfcb0eb
If tests results are added to builds, or environment variables are injected
into a job, there are separate API calls to retrieve that data. Add functions
to do so.
Change-Id: Ie0a51839d527db6d440719eeb44cd1cc89644d06
Account for situations where no Content-Length or Transfer-Encoding
headers are present in a response.
The length of the response.content could still indicate a response was
given. The earlier raise_for_status() should catch bad HTTP responses
that return content.
Closes-bug: #1766660
Change-Id: I6739e26a039a898982f5cdc96d19d6992ed37737
Pass in a queue ID number to discover a job's status, and possibly a job
URL.
Change-Id: I20541ec49cc30e5c74a6c596e02b3f42b2567fa5
Closes-Bug: #1724932
Since Jenkins version 1.519 (released 2013/06/17), build_job() would
always return an empty string (the empty HTTP body from the POST
request).
Read the HTTP response's Location header and return the queue ID number.
Clients can use this number to query server for the the upcoming job's
status.
Change-Id: I2a1ef3abb7e675e0ad37dd8eb4a377af22f27a03
Closes-Bug: #1724932
This feature will allow build_job to accept build parameters as a list
of two membered tuples - [('parameter key', 'parameter value', ), ...].
Currently, build_job expects to receive a dictionary of parameters but
this limits the build_job_url function (which is called from
build_job()) to build jenkins url for jobs with multi-select
parameter(s) because dictionaries cannot have duplicate keys (in case,
when there are several values to one parameter key). urlencode is
capable of converting lists with two membered tuples to url as well.
Change-Id: I134a5f8b69ba91acd89afa791cb1cf0b5e63f5f5
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
Support for deleting individual builds for a Jenkins job.
Change-Id: I35c98855b0883986a85cbe7212ec52e6f67e5638
Signed-off-by: Sorin Sbarnea <ssbarnea@redhat.com>
Guillaume DeMengin