It has been migrated to the Jenkins community:
https://github.com/jenkinsci/gearman-plugin/
Depends-On: Ib6010d7ce85a934501c50a53e9ac78dcf74bc403
Change-Id: I0c84db2ad3fbb4d9f0eff793a0159c6ed3a8e25c
This fixes a bug that causes the result object to be left off of the
returned data packet after a job finishes executing. This only occurs
when the job request sets the node to temporary offline (i.e.
"OFFLINE_NODE_WHEN_COMPLETE":"true"). The problem is that the
ComputerListener.onTemporaryOffline event was stopping the
gearman worker which causes a disconnect to the gearman client thus
not being able to send back the results for the job. The worker should
not stop on a temporary offline, only on disconnect and delete of a node.
The fix is to only re-register the gearman functions instead of stopping
the workers on a temporary online/offline of a node.
Change-Id: Icf672acf059d6cb0e16a6685d637824c678ea5be
This change updates the gearman-plugin to build against Jenkins LTS
ver 1.565.3. This is required to support later versions of Jenkins
because there have been changes to Jenkins events in core.
This change adds the ItemListener to re-register gearman
functions on changes to jenkins projects. Using the ItemListener is
better because it provides more details on the item that's been
changed and the events are more grandular. There was also a change
to ComputerListener events (onTemporaryOffline and onTemporaryOnline)
that needed to be handled seperately in this change. The SaveableListener
is still needed due to a bug in ItemListener.
The most notable change to function registration is that the gearman
plugin will no longer register functions containing a node's self label.
For example:
Assume you have the following setup:
a node: named 'trusty-slave1' and labeled 'trusty'
a job: named 'my-job' that targets the 'trusty' label
The gearman plugin used to register the following functions:
'build:my-job', 'build:my-job:trusty' and
'build:my-job:trusty-slave1'
With this update the gearman plugin will only register
'build:my-job' and 'build:my-job:trusty'. It will no
longer register functions containing the implicit
node name (trusty-slave1).
If your gearman client has been using explicit labels
to execute builds then this change will not affect
your workflow.
Closes-Bug: #1353891
Change-Id: I9e57ec9f24bf303989a4df1fc4f1a0c4b6d001bc
There was at least one error, likely a race condition, with the
previous code which could cause more than one ExecutorWorkerThread for
a node to be spawned. In particular, I think the bogus comparison in
ComputerListenerImpl may have a large part in that (it checked to see
if a _Computer_ object was in a list of _Thread_ objects).
To improve reliability around adding and removing nodes, all related
functionality is moved to the GearmanProxy class. Any methods (most
of them) that have to do with starting or stopping worker threads are
synchronized on the GearmanProxy monitor (the important parts of most
threads were already synchronized on the worker list before, so this
should not be much of a performance change).
The methods that start management and executor threads now do their
own checking to verify that such threads do not already exist, making
it so that calling them is more idempotent. Existing checks external
to the class have been removed (these were likely somewhat racy).
To avoid keeping redundant data structures, the node availability list
is removed, and instead if we need to find an availability object, we
walk the list of worker threads and compare to their nodes. Because
we do this so much, the list of worker and management threads are
changed to use those explicit classes instead of
AbstractWorkerThreads.
The accessor methods for the internal lists of worker threads is
removed to ensure that they are only managed through GearmanProxy.
This changed some unit tests and required the removal of one complete
test (which was not doing much more than verifying the addition
operator).
Also, when stopping ExecutorWorkerThreads, stop all of the ones
associated with a node.
When a computer goes offline, Computer.getNode() returns null, so we
can't know which workers should be deleted. Instead of using Nodes as
keys for our workers, use Computers instead and change everything to
use Computer (most functions needed Computer rather than Node anyway),
and in the few remaining places where a Node is needed, convert the
other direction.
Change-Id: Ia5084579317f972400069cc3e84db4e0b6560a80
This commit is a fix to bug 1185952.
While the jenkins host is running its master node may
be in a disabled state (master has 0 executors) however the
host should still be able to run a management worker and
communicate with a Gearman Server. This commit makes
sure that the manaegement worker runs regardless
the master node's state.
This commit also makes sure that we are setting the management
worker's name to something.
Change-Id: I871019fadde47787cfe20bec666e24d242728f30
fix for bug 1181569
Refactored how stop was handled. Removing thread and stopping thread
in the same syncronized loop was causing concurrent modification error.
Needed to seperate the actions for it to work.
Change-Id: I99b676f377842826f9e6a847cfeff76be91c8299
Users no longer need to disable the plugin to connect to
a different gearman server.
Along with this commit I changed the 'launchWorker' plugin config name to
'enablePlugin' because enabling plugin is distinct from running gearman workers.
Change-Id: Ia81d78da3dbdc83fd46dd7f5d40ccb9aca3af97f
Refactor GearmanProxy to be a singleton instead of using static fields
and methods to manage thread states.
Change-Id: I8d84778f0e0d7020bb08087be29808f46aff8786
src/main/java/hudson/plugins/gearman/AbstractWorkerThread.java
Removed Id field, it was initially added because I thought it was the plugin's
responsibility to cancel jobs that are on the gearman queue. We've decided that
it will be the client (zuul or otherwise) responsibility to cancel jobs from the gearman
queue. The gearman plugin will cancel jobs that have already been put on the
jenkins queue.
src/main/java/hudson/plugins/gearman/ComputerListenerImpl.java
src/main/java/hudson/plugins/gearman/ExecutorWorkerThread.java
src/main/java/hudson/plugins/gearman/GearmanPluginConfig.java
src/main/java/hudson/plugins/gearman/GearmanProxy.java
src/main/java/hudson/plugins/gearman/ManagementWorkerThread.java
src/main/java/hudson/plugins/gearman/StartJobWorker.java
Refactor to reduce code duplication. Consolidated creation of management worker and
executor workers. Added a fix so that executors spawned on master node
is named 'master-manager' for the manager and 'master-exec-#' for executors
src/test/java/hudson/plugins/gearman/ManagementWorkerThreadTest.java
Added test to make sure worker name is set correctly
src/main/java/hudson/plugins/gearman/GearmanPluginUtil.java
src/test/java/hudson/plugins/gearman/GearmanPluginUtilTest.java
Useful utils for the gearman plugin with tests
Change-Id: I96e097dc0dbf5cd78e5e82af584976085aee61b3
This checkin enables the gearman plugin to update gearman
functions on a node label change. We use a SaveableListener to
listen for changes to nodes and projects. The SaveableListener
is a catch all for any event that requires a save. If any change
happens then we re-register all gearman functions in every
executor worker thread. This listener essentially replaces the
ProjectListener and some of the ComputerListener events.
We needed to use the SaveableListener as a workaround
for the ComputerListener onConfigurationChange method due to
a bug in the onConfigurationChange method. The jenkins
config doesn't get saved until after the onConfigurationChange
is called.
This seems to work ok but probably not a good solution because
the SaveableListener fires for every instance of a jenkins node.
So if there are 3 nodes in jenkins it will fire 3 time even though
there was only a change to one of the nodes. This will slow down
the system as you have more executors.
This is also not a great solution because
I'm sensing a race condition that may conflict
between SaveablListner and ComputerListener. On a delete or
disconnect of a node the disconnect may happen during an
gearman function update which may cause a NPE during the update.
This checkin also enables updating gearman functions on
a project state change (disable/enable).
I believe the only event left that requires gearman funciton
updates are adding and removing of jenkins executors.
Change-Id: I67747d4d895a2474477d2a8ec01c3226e1d5456a
This checkin does the following:
1. Makes the plugin more thread safe.
2. Exposes the node object in the ExecutorWorkerThread
to allow gearman function updates to be more efficient and makes it
simpler to track new slaves vs existing slaves in the system.
3. Treat slave disconnect the same as a slave deletion. Disconnect will
stop the associated executor worker thread. Connect will spawn a new
thread for the slave. This more closely matches how jenkins
treats slave states.
src/main/java/hudson/plugins/gearman/ComputerListenerImpl.java
src/main/java/hudson/plugins/gearman/ExecutorWorkerThread.java
src/main/java/hudson/plugins/gearman/GearmanPluginConfig.java
Expose the node to listeners for updating gearman functions
Treat slave disconnect the same as a slave deletion
src/main/java/hudson/plugins/gearman/GearmanProxy.java
Made lists into syncronized list
src/main/java/hudson/plugins/gearman/ProjectListener.java
Syncronized iteration of lists
Change-Id: I7d7a40ec7783e3e7f21c9974dad9159beee6b985
This checkin will enable the gearman-plugin to be in the same state
as it was configured to before jenkins shutdown. The gearman plugin will
now startup in ether enabled or disabled state. Enabled means that
the plugin will automatically spawn all the threads and register all the
functions depending on the state of the slaves, projects and labels.
Disabled means that no workers are spawned and no functions are registers.
When in diabled state user can manually enable the gearman-plugin from the
jenkins configuration page. If the plugin is enabled on startup the user can
disable it from the jenkins global configuration page, which will stop
all of the workers.
src/main/java/hudson/plugins/gearman/ComputerListenerImpl.java
automate the spawning of executors when master comes online.
add license info
src/main/java/hudson/plugins/gearman/GearmanPluginConfig.java
remove setting to put gearman-plugin in disable state on startup.
src/main/java/hudson/plugins/gearman/GearmanProxy.java
Add method to get the list of management executors
changed thead list from Stack to ArrayList object
Change-Id: Iea20dd244b322de0bf75eff3b31a4f70ed688282
This checkin attempts to make gearman plugin objects thread safe.
src/main/java/hudson/plugins/gearman/SaveableListenerImpl.java
Decided that this was not needed.
src/main/java/hudson/plugins/gearman/GearmanPluginConfig.java
Added method to allow other objects to access gearman server
launchWorker, host, and port settings
src/main/java/hudson/plugins/gearman/GearmanPluginUtil.java
Useful utilities for this plugin
src/main/java/hudson/plugins/gearman/AbstractWorkerThread.java
src/main/java/hudson/plugins/gearman/ComputerListenerImpl.java
src/main/java/hudson/plugins/gearman/ProjectListener.java
src/main/java/hudson/plugins/gearman/GearmanProxy.java
Made object private and added syncronized setters and getters
Change-Id: I04ca6a275ba8184bd18cf8954d07b94d02b2a47d
This change is to create a new object to store Gearman
objects and state info.
src/main/java/hudson/plugins/gearman/GearmanProxy.java
created to keep Gearman state info.
src/main/java/hudson/plugins/gearman/GearmanPluginConfig.java
simplied this class by removing the core gearman stuff out to
a GearmanProxy.java class
src/main/java/hudson/plugins/gearman/Constants.java
Use one logger instead of two. updated logger reference in all
of the other files in this checkin
src/main/java/hudson/plugins/gearman/ProjectListener.java
src/main/java/hudson/plugins/gearman/StartJobWorker.java
src/main/java/hudson/plugins/gearman/StopJobWorker.java
src/main/java/hudson/plugins/gearman/ComputerListenerImpl.java
update references to changed class and methods
Change-Id: I879cdb8839c8b5437bccf6d7e1602c33eff434a6
Initial checkin for dynamic gearman function registration.
This is not thread safe but does work for most of the use cases
where we want to update gearman worker threads and functions.
functions are getting updated for the following scenarios:
1. project name change
2. project label change
3. project delete
4. project create
5. slave creation - spawns a new worker thread
6. slave deletion - stops an existing worker thread
7. slave state change online/offline
8. slave connection change connect/disconnect
Not working:
1. project state change (disable/enable)
2. master and slave node label change
3. master and slave node executor change
Change-Id: I8e79519198224fc4c93fd16c741fb7de568077b9
Antoine Musso