It has been migrated to the Jenkins community:
https://github.com/jenkinsci/gearman-plugin/
Depends-On: Ib6010d7ce85a934501c50a53e9ac78dcf74bc403
Change-Id: I0c84db2ad3fbb4d9f0eff793a0159c6ed3a8e25c
The registerJobs method of an executorworker can be invoked by an
external caller before the worker is completely initialized by
its run method. We protected against that by checking one instance
variable, but there's still a race condition involving another.
Add a check for that variable as well.
Change-Id: I8e2cfffb54aa8a4cf8b1e61e9a9184b091054462
Previously, a node would register the ability to run a job under
all labels associated with the project if the node matched at
least one of them. What we really want to do is, if the node
matches any project label, register the generalized job and then
also register it for each label in the _intersection_ of project
labels and node labels.
Change-Id: Ie41af7b1a2f26a4ca2aac6c7263817d5021cca9b
Fixed registration for slaves that have the 'usage' configuration set to
"Leave this machine for tied jobs only"
Change-Id: Ieeb16ae9070915b3461379af9e50ae6fd69f22d7
Closes-Bug: #1253429
This undoes some of the previous change. In particular, calling
something that could affect a NodeAvailabilityMonitor's lock
(eg in canTake or the init methods in createExecutorWorkersOnNode)
can deadlock on trying to obtain the Queue lock while holding the
GearmanProxy lock.
Instead, lock on just the handle lists where necessary, and try
to do serious work outside of that lock.
Change-Id: I4c1ed6c7a4f5586c1034651a649bbbd420eacdd7
There was at least one error, likely a race condition, with the
previous code which could cause more than one ExecutorWorkerThread for
a node to be spawned. In particular, I think the bogus comparison in
ComputerListenerImpl may have a large part in that (it checked to see
if a _Computer_ object was in a list of _Thread_ objects).
To improve reliability around adding and removing nodes, all related
functionality is moved to the GearmanProxy class. Any methods (most
of them) that have to do with starting or stopping worker threads are
synchronized on the GearmanProxy monitor (the important parts of most
threads were already synchronized on the worker list before, so this
should not be much of a performance change).
The methods that start management and executor threads now do their
own checking to verify that such threads do not already exist, making
it so that calling them is more idempotent. Existing checks external
to the class have been removed (these were likely somewhat racy).
To avoid keeping redundant data structures, the node availability list
is removed, and instead if we need to find an availability object, we
walk the list of worker threads and compare to their nodes. Because
we do this so much, the list of worker and management threads are
changed to use those explicit classes instead of
AbstractWorkerThreads.
The accessor methods for the internal lists of worker threads is
removed to ensure that they are only managed through GearmanProxy.
This changed some unit tests and required the removal of one complete
test (which was not doing much more than verifying the addition
operator).
Also, when stopping ExecutorWorkerThreads, stop all of the ones
associated with a node.
When a computer goes offline, Computer.getNode() returns null, so we
can't know which workers should be deleted. Instead of using Nodes as
keys for our workers, use Computers instead and change everything to
use Computer (most functions needed Computer rather than Node anyway),
and in the few remaining places where a Node is needed, convert the
other direction.
Change-Id: Ia5084579317f972400069cc3e84db4e0b6560a80
maven builds intermittently fail due to something in the code.
essentially its the same issue as in bug 1201035.
This patch makes it so the build ignores the error and just logs
it so that we can have a record of the error. Hopefully we
can use logstash to query for a pattern to determine the
root cause of this problem.
The error on this build:
java.lang.NullPointerException
at hudson.plugins.gearman.ExecutorWorkerThread.registerJobs(ExecutorWorkerThread.java:150)
at hudson.plugins.gearman.SaveableListenerImpl.onChange(SaveableListenerImpl.java:68)
at hudson.model.listeners.SaveableListener.fireOnChange(SaveableListener.java:78)
at hudson.model.AbstractItem.save(AbstractItem.java:474)
at hudson.model.Job.save(Job.java:154)
at hudson.model.AbstractProject.save(AbstractProject.java:273)
at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:915)
at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:622)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:586)
at hudson.model.Run.execute(Run.java:1575)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:477)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:241)
Change-Id: I16fb0261312fe0aef55d2f2b6c83652333aed05a
Every node (slave or master) gets an AvailabilityMonitor that
handles mutually exclusive access to scheduling builds on that
node. If Jenkins wants to run a build on the node, it will only
be able to do so if we are not waiting for a response to a
GRAB_JOB packet from Gearman. Likewise, immediately before
sending a GRAB_JOB, we lock the monitor and only unlock it if
we either get a NO_JOB response, or after the job we were just
assigned starts building.
(As an exception to the above rule, since Jenkins will apply the
same scheduling veto logic to the build that we request via Gearman,
(while we still hold the lock) we tell the monitor to expect a request
for that build from Jenkins and we permit Jenkins to build it even
if the lock is held.)
Change-Id: Iae03932aef4b503c69699b99d38a6fc2691fb02e
This rearranges a bit of the previous change to move the WaitBool
functionality into an AvailabilityChecker class.
It implements the new feature of not grabbing jobs while not in the
quiet mode for shutdown by using a busy wait. There doesn't seem to
be an event framework for that, and well, it doesn't happen very
often, so a slow busy wait probably isn't terrible.
This only applies to Executor workers, not Management workers
(so jobs can still be stopped and descriptions set).
Removed the default-name constructor of AbstractWorkerThread because
it is not used anywhere now (removed its test as well).
Change-Id: I6d5e1cd3cb47c8876ceb909d205cb66445388992
Jobs can be triggered by non-gearman sources, sadly. This makes the gearman
plugin aware of when executors are busy and it will refrain from grabbing a
job from gearman in those circumstances.
It's far from perfect, but should at least handle the most likely cases where
a job is already running an an executor.
Change-Id: If993c6d6bc63ed89b385d2e5bb41762ef84a429f
If an unhandled exception happens during the work method, shut
down the worker and start again.
Also, name worker threads for easier debugging.
Change-Id: Icc8e4c060bca4ab624bffc2e1bef18655d50d6b2
The "stop:" function should be qualified by the name of the jenkins master
to make it globally unique (eg "stop:jenkins.example.com").
Supply the name of the master in the WORK_DATA that is sent to the client
when the job starts building so that the client can direct a subsequent
"stop:" request to the right worker.
Change-Id: I0112b84ae614ce4faaed880ea3d1073674dfe5fe
Add a package local implementation of something like the GearmanWorker from
java-gearman (based on GearmanWorkerImpl).
It is much simpler than the existing GearmanWorkerImpl and is more suited to
the way we need to use it in the Jenkins plugin. It assumes jobs are always
changed in batches, and only changes jobs at the top of the event loop (not
when a job is running).
The worker threads are updated to only request job changes when there is an
actual difference.
WORK_STATUS events are sent every 10 seconds while a job is running.
run-fast is updated to only remove the gearman plugin from the work directory,
preserving any other plugins that may be installed.
This isn't very elegant, but is a start and broadly demonstrates what we need
the plugin to do.
Change-Id: I26df504534ec50f03c9e0ef772a709046cf88a23
This commit adds tests to verify registering of gearman jobs.
Most of the single label scenarios are tested, however due to
weirdness with the HudsonTestCase class the multiple label
scenarios don't work the same as in normal Jenkins operation so
I could not create automated test for it.
Change-Id: I2e2ac5c51efd13d560c575310da4b8f8fb58f05b
This checkin does the following:
1. Makes the plugin more thread safe.
2. Exposes the node object in the ExecutorWorkerThread
to allow gearman function updates to be more efficient and makes it
simpler to track new slaves vs existing slaves in the system.
3. Treat slave disconnect the same as a slave deletion. Disconnect will
stop the associated executor worker thread. Connect will spawn a new
thread for the slave. This more closely matches how jenkins
treats slave states.
src/main/java/hudson/plugins/gearman/ComputerListenerImpl.java
src/main/java/hudson/plugins/gearman/ExecutorWorkerThread.java
src/main/java/hudson/plugins/gearman/GearmanPluginConfig.java
Expose the node to listeners for updating gearman functions
Treat slave disconnect the same as a slave deletion
src/main/java/hudson/plugins/gearman/GearmanProxy.java
Made lists into syncronized list
src/main/java/hudson/plugins/gearman/ProjectListener.java
Syncronized iteration of lists
Change-Id: I7d7a40ec7783e3e7f21c9974dad9159beee6b985
Checkin to change README from txt to rst format.
Jenkins seems to hijack logging so i've removed all of the
logging specific bindings that were added previously.
I also added "----" to begining of this plugin's logging
messages so i could easily keep track of them.
Change-Id: Ibd8c56af5b9ad18152bcb0d3ff0c41168a6d2fd1
This change is to create a new object to store Gearman
objects and state info.
src/main/java/hudson/plugins/gearman/GearmanProxy.java
created to keep Gearman state info.
src/main/java/hudson/plugins/gearman/GearmanPluginConfig.java
simplied this class by removing the core gearman stuff out to
a GearmanProxy.java class
src/main/java/hudson/plugins/gearman/Constants.java
Use one logger instead of two. updated logger reference in all
of the other files in this checkin
src/main/java/hudson/plugins/gearman/ProjectListener.java
src/main/java/hudson/plugins/gearman/StartJobWorker.java
src/main/java/hudson/plugins/gearman/StopJobWorker.java
src/main/java/hudson/plugins/gearman/ComputerListenerImpl.java
update references to changed class and methods
Change-Id: I879cdb8839c8b5437bccf6d7e1602c33eff434a6
Initial checkin for dynamic gearman function registration.
This is not thread safe but does work for most of the use cases
where we want to update gearman worker threads and functions.
functions are getting updated for the following scenarios:
1. project name change
2. project label change
3. project delete
4. project create
5. slave creation - spawns a new worker thread
6. slave deletion - stops an existing worker thread
7. slave state change online/offline
8. slave connection change connect/disconnect
Not working:
1. project state change (disable/enable)
2. master and slave node label change
3. master and slave node executor change
Change-Id: I8e79519198224fc4c93fd16c741fb7de568077b9
This checkin adds an id to each gearman executor. Not sure what we
wanted to use for the UUID at this time so I've made it a UUID. Also
added a constructor that doesn't require user to specify a name for the
thread. By default name will be set to 'anonymous' if not passed in.
Created getters and setters for object variables.
AbstractWorkerThread - auto generate a random id for each executor and
set name to 'anonymous' if user doesn't specify.
ExecutorWorkerThread - added constructor
ManagementWorkerThread - added constructor
Change-Id: I82958d259ca951ed08cccebc369fa62c44ab532f
Jenkins.getProjects() wasn't returning maven and multi-config projects.
This change now attempts to register all projects. The only caveat
is that the multi-config projects don't get registered quite right
because the labels are treated differently in that project. I don't
plan to fix multi-config project registration because openstack
doesn't have any of those projects setup in jenkins.
Register an additional function for each executor worker. We register
a generic "build:$projectName" function on each executor
that can run a build on $projectName.
Minor change, set gearman-plugin version to 'SNAPSHOT'. The maven
release plugin will convert to a non-SNAPSHOT version on release.
Change-Id: I49a0f73d297e61657fb6f31f26e87c181a644115
Also added functionality to wait until a StartJobWorker can service a build request. This change eliminates
putting builds on the jenkins queue. Now jobs are either running or it's not. The only cancel that
makes sense is an abort (currently running jobs).
AbstractWorkerThread.java - add comments, set worker id to name instead of random uuid
ExectorWorkerThread.java - create thread of each jenkins executor
GearmanPlugin.java - refactor to spawn a thread for every executor
NodeAssignmentAction.java - provide access to label name StartJobWorker.java - make thread block execution until there is an available
jenkins executor to run the job. Also set the gearman job return parameters.
StopJobWorker.java - Set gearman job return parameters.
Change-Id: I30cec8ca3900eb7976c38077383505ea73e744dd
The gearman function registration was registering functions on nodes
multiple times. Added a check to make sure nodes on the label
matched the node in the executor before registering the function.
Also added better documentation and helper functions.
Change-Id: Id13bbfe0c003a82e1e311736c5ff7a682e5efca1
Gearman functions to start jenkins jobs and abort jenkins jobs are now registered when
'launchWorker' checkbox is selected in the Jenkins global config page. There is a
stop:$host function to stop a builds. There are build:$project:$label functions
to start jenkins builds. The functions are registed and plumbed thru but don't
actually do anything yet. I also had to fake UUID functionality due to
gearman-java issue: https://bugs.launchpad.net/gearman-java/+bug/1098816
Change-Id: I7cb772c7119fa289d17edff5d81a041cb01031ae
Antoine Musso