This adds a new database table, zuul_buildset_event, which stores
event information related to the buildset. At the moment, the
only event information stored is a description of the trigger event
which originally caused the item to be enqueued. Later changes may
add other events such as the reason the item was dequeued or
superceded, or the reason a buildset was canceled due to a gate reset.
A particular goal is to be able to inform users who see unsolicited
reports on their changes (which can happen due to a different change
in a dependency cycle being enqueued) understand why the report was
made.
The new event information is exposed via the rest api and a later
change will add it to the web ui.
Change-Id: I9bcbb64faa3d499a26b90d20932009b6ce226061
This adds support for excluding locked (read-only) branches. This is
currently only supported by the Github driver.
Change-Id: I360edeb04c9734189396e8c5ddbed17e7f7464a8
The only way to find out if a branch is locked is by using a graphql
query for branch protection rules. This change adds a such a query.
Because neither our graphql client nor our fake graphql server fully
implemented pagination, and we expect to routinely have more than 100
branches, this change also handles pagination.
And since this in graphql, it handles nested pagination.
The word "Fake" is removed from the fake graphql server classes because
graphene introspects them to determine their object type names. Using
their real names makes the object types match which is now important
since one of our new queries specifies an object type.
The github connection class is updated to use this query for protected
branches. Locking support will come in a future change.
Change-Id: I2633332396b79280984f0ebfa64a955d24fb7bae
The current branch cache is hyper-optimized to support exactly two
types of branch queries: all branches for a project, or unprotected
branches for a project. GitHub provides another axis: "locked"
branches. Cconceivably, other code review systems could as well, and
there may be even more axes in the future.
In order to support locked branches in a future change, we must first
refactor the branch cache to support more than two queries. This
change implements that with the following scheme:
The branch cache will be a dictionary of project_name -> ProjectInfo,
and ProjectInfo will hold general information about the project such
as supported merge modes and default branch, as well as another
dictionary branch_name -> BranchInfo. The BranchInfo record will hold
boolean flags indicating whether the branch is protected or locked.
Additionally, the project_info record will hold a set of which queries
for these flags have been performed and whether they were successful
or not. This allows us to determine whether the branch_info flags
are valid or not. For example, if we have only performed a query to
get all the branches, and a caller requests a list of protected
branches, we know that the BranchInfo.protected bool is not valid,
so we return a LookupEerror to the caller which will trigger another
query to get the protected branch list which will then be used to
update the branch cache, setting the protected bool to true where
appropriate on existing BranchInfo objects and setting the protected
query flag on ProjectInfo. The result matches the current behavior,
but is extensible to support more flags.
In order to minimize the size of the branch cache in ZooKeeper, the
BranchInfo object is serialized as a simple integer with a bitmap of
the associated booleans. Likewise, the several queries are stored in
the serialized ProjectInfo as two bitmaps (one for success, and one
for failed).
This change stubs out the "locked" flag and query in some places, just
to demonstrate sufficiency for future use, but it does not implement
support for locked branches yet. A future change will do that.
As long as we don't actually add any locked branches, we can still
serialize to the old branch cache data structure, so this change does
so to enable rolling upgrades. Tests of the upgrade path and
continued operation on only the old data path are included.
Change-Id: I8841e675295f15e5d6dd004f9e34836b8bbbdb63
This adds a framework for upgrade testing where we split a
functional test in half, running the first half on the previous
commit and the second half on the current commit. This may allow
us to catch upgrade errors which are otherwise difficult to find
in tests because they require data generated by old/removed code.
This does not run the test, since it operates on the current and
prior commits, only a commit that follows this one can run the
job successfully.
Change-Id: I9d4d4af42fb1f684a88ec5a7e747b132423696f1
This adds methods to save and restore the test change database.
This can be used in a future change to perform an upgrade test
where we save and restore the fake gerrit (and other drivers)
state across an upgrade.
Some minor changes are made to FakeGerritChange to avoid storing
non-pickleable values. Similarly, the Gerrit reporter was using
ZuulConfigKey objects as review categories. That's fine since
they behave like strings everywhere we use them, but they aren't
pickleable, and we don't need the extra line context information
after loading the config, so we discard it and convert them to
strings.
Change-Id: Ifa404203e414932f50811291ad56a661f2875af0
This change is merely a reorganization to move the fake gerrit and
gitlab classes into their own files to match github and gitlab.
The Fake*Connection classes for all 4 drivers are also moved into
their respective files. This is accomplished by moving some symbols
from base.py into a new tests/util.py to resolve the import cycle
(which is likely why they were not there in the first place).
Change-Id: I274b9e5abf6086656f8ceb5a16dab2f8393deead
When api-root authentication is configured, access of zuul/api/tenants
would fail with 401 before authentication is done. Before this fix,
however, the browser would continuesly sending the request before
it has a chance to redirect to the authenticate page, which results
in a endless loop in some browser (as tested in Firefox), or loops for
randomly a few seconds and finally redirected to the authentication
page (as tested in Chrome).
It is fixed that it calls fetchTenantsIfNeeded() only when authentication
is done or not needed (api-root authencation is not configured).
Change-Id: I2cc67c791d694f329cd48c09d81cdda452eff12c
test_pre_run_failure_retry has been hitting timeouts semi regularly.
This test was checking that after all retry attempts other things
continue on. Rather than increasing the timeout time to avoid timeouts
we reduce the number of retry attempts from 3 to 2 which should make
things run faster and within the timeout. This should be safe from a
testing perspective because we're still doing at least one retry which
ensures that previous code paths are still exercised.
Change-Id: If78c7cdfac63c30f9e52a1a5984a662ab969c2ee
Adds a new type of trigger to the Gerrit driver that only triggers if
the approval value was changed by the user in the comment. This is
useful if Zuul is configured to allow many different scores to trigger a
pipeline (with an additional requirement on all of them), but arbitrary
comments made while the scores are present should _not_ trigger (or
potentially re-trigger) the pipeline. This can happen because Gerrit
sends all approvals by a user on all comments, regardless of if they
were changed by the comment.
The new `approval-change` trigger requirement inspects the `oldValue`
field in the Gerrit event. The pipeline will only trigger if this value
is present and not equal to the new approval value (thus, only when the
user actually changed it).
`oldValue` has been present since at least Gerrit 3.4
Change-Id: I88cf840ae8b4e63c77f10ee68b6901e85f7c5fb1
We have mostly managed to ignore the /COMMIT_MSG in files matchers
(because it is unintuitive that it would be considered), but the recent
change to allow negated regexes in irrelevant-files exposed the fact
that it can still have an effect in that case. The new feature hasn't
been used yet, so we could silently correct this, but a very similar
construction would be possible with the deprecated style of regex with
a negative lookahead. Just in case someone wrote that, this change
includes a release note letting them know they can drop the /COMMIT_MSG
from their regex.
Change-Id: Ide04ed01224b5c0c48ab8d3c15ea7aef324cc42d
The merger starts every operation by resetting the repository.
That means clearing out any failed previous merges and updating
and restoring the branch state to match the upstream origin
repo.
For repos with a very large number of branches (10k), this can take
some time (minutes). This is mostly due to the inefficiency of
looking up the origin ref one at a time (gitpython reads the
packed-refs file for each lookup, ironically negating the benefit
of packed-refs). To bypass this, use our previously developed
method for getting all the refs efficiently and do that once at
the start of the reset method.
Change-Id: If21245cd562c6499378c4f3353332d87c4ca4b47
Given two PRs with B depending on A which are enqueued in gate, A is
closed and then immediately reopened.
This sequence of events will currently dequeue A and then immediately
enqueue it behind B. Since the check for whether a dependency is already
in the queue doesn't care if it's ahead or behind the current change,
we'll not dequeue B and the content of builds executed by B will not
include A.
This change updates the check to determine if a change is already in
the queue to only check for changes ahead of it. This causes B to
be correctly dequeued in the next pipeline pass.
This behavior is correct, but isn't always intuitive or consistent.
If the time between closing and reopening a change is long enough for
a pipeline process, then both changes will be enqueued by the reopening
(because we check for changes needing enqueued changes and enqueue them
behind). But if both events are processed in a single pipeline run,
then the removal of B happens after the re-enqueue of A which means that
it won't be re-added.
To correct this, whenever we remove abandoned changes, we will also remove
changes behind them that depend on the removed abandoned changes at the
same time. This means that in our scenario above, the re-enqueue happens
under the same conditions as the original enqueue, and both A and B are
re-enqueued.
Co-Authored-By: James E. Blair <jim@acmegating.com>
Change-Id: Ia1d79bccb9ea39e486483283611601aa23903000
On BTRFS, f_files and f_ffree are always 0. For now, assume there is
no limit by setting files_percent_avail to 100%.
Change-Id: I53455e46101130596ae178a5933fe51ebaee206f
It looks like urllib3 version 2.1.0 causes problems when connecting to
Windows nodes.
Fixed in 2.2.0, but ibm-cos-sdk is preventing that from installing, so
exclude 2.1.0 for now.
https://github.com/urllib3/urllib3/pull/3326
Change-Id: I5d4a33c477d6872389c1d4197e926991b70f06ec
This facilitaties the creation of a Zuul system with a running config
(that will be kept up to date as long as it receives events) but does
not run any jobs or make any reports. This can be used in conjunction
with zuul-web to serve REST API requests for introspection, or to create
a standby Zuul system with a warmed config cache, or to support other
debugging techniques.
This change adds an extra assertion to the wait-for-init test since it
would be too similar otherwise. It also adds some documentation for
wait-for-init (so that both similar options are documented) and support
for setting both options by environment variables for ease of use
in k8s environments.
Change-Id: I3ee83b08c8280066cfa0744f2e30e41edd0f364c
Gitlab does not include files lists with merge request events, so
we always have Zuul perform that calculation itself. But if a
merge request is updated after the fileschanges job completes, we
would overwrite our files list with None again.
Instead, recognize that for a given change+patchset, files are
immutable and don't update it if it's set already.
This matches the logic in Github, which, although it does provide
files with PR events, it only returns the first 300, so this logic
is needed in those cases.
Change-Id: I115c69b97e17cd4b01e3fa6c70140add6254283d
Approximately 40% of all Gerrit events that OpenDev processes are
ref-updated events for refs/changes/.../meta refs. This is likely
due to the increased use of notedb for storing data in Gerrit. Since
Zuul users are not likely to need to trigger off of ref-updates to
the meta ref, let's avoid enqueing them into Zuul's event queue.
This will reduce ZK traffic.
Change-Id: I724f5b20790d1ad32e72b1ce642355c2257026c1
These attributes are documented to support the new Zuul-standard
regex format (str or {pattern: str, negate: bool}) but actually only
accepted strings. Update them to include negation, and add tests.
Change-Id: Ie8a4f816afc5277006b61406b7b0bd725947dd78
This variable is mostly intended for retroactive debugging of jobs.
There is currently no way to determine for which specific refs a
deduplicated job is being run. This records that information in
the inventory file for the job.
This change also corrects the docs to indicate that change_url is
always present.
Change-Id: I990ecfe2a4b455fa800906ed0e542a3581bb8b29
So far, when getting parent jobs recursively we've kept a set of tuples
with the job UUID and the dependency type (hard/soft) that needed to be
processed.
This worked as long as all relevant jobs used the same type of
dependency to a job. However, when hard/soft dependencies are mixed this
could lead to a job being processed twice. Zuul wrongly detected this as
a job dependency cycle.
This problem also lead to exceptions in pipeline processing as some
schedulers detected a dependency cycle whereas other did not. The reason
here is the non-deterministic odering of the set data type.
Since we don't need the information about the dependency type, we'll
just keep a set of job UUIDs to be processed, which prevents Zuul from
processing a job with a different type of dependency twice.
Note: The provided regression test did not fail consistently due to the
set non-determinism mentioned above.
Change-Id: I6459cbc90bf745ddda2da9dbc25bcee97005d933
Specify dependent jobs are made available with artifacts.
Specify job.provides/requires is ignored for non-change items.
This documenting implementation made in [1].
Specify examples of passing artifacts for parent/child jobs in
while using job.provides/requires.
[1] https://review.opendev.org/c/zuul/zuul/+/915188
Change-Id: Ibb36ce2fdfa8b07ae399ae69ebf63de663c71fb3
Since squash merges will not create a merge commit Zuul is creating one
separately. However, this step can fail when there are no changes to
commit. This might for example happen in change-triggered post
pipelines.
To fix this issue we will restore the behavior used prior to
I3379b19d77badbe2a2ec8347ddacc50a2551e505 and allow creating empty
merge commits.
Change-Id: Ic18cb98f12260338799e963b8dc915c8be1d421b
Fixes the way that missing labels are handled in Gerrit. The intention
is that labels provided by Zuul are removed from the set of missing
labels on the change (and thus ignored). The original code was using the
">" set comparison operator to do this, but this operator is actually
"issuperset()". This means that if there was any disjoint members in the
allow_needs set (that is allow_needs had labels that were not missing),
the comparison would be False, and any actual missing labels would be
ignored.
The fix is to use set difference to calculate the missing labels and
remove the allow_needs set. If any labels are left after this, they are
actually missing and the change cannot be merged
Change-Id: Ibdb5df44e80d75198493f8287443ed19bcf269f1
When a change that is part of a circular dependency is abandoned we'd
set the item status to dequeued needing change. This will set all builds
as skipped, overwriting exiting builds.
This means that when the item was removed, we did not cancel any of the
builds. For normal builds this mainly waste resources, but if there are
paused builds, those will be leaked and continue running until the
executor is force-restarted.
The fix here is to cancel the jobs before setting it as dequeued needing
change.
Change-Id: If111fe1a21a1c944abcf460a6601293c255376d6
Zuul