x
/
gearman-plugin
Code Issues Proposed changes

misc doc and logging updates 

Checkin to change README from txt to rst format.
Jenkins seems to hijack logging so i've removed all of the
logging specific bindings that were added previously.
I also added "----" to begining of this plugin's logging
messages so i could easily keep track of them.

Change-Id: Ibd8c56af5b9ad18152bcb0d3ff0c41168a6d2fd1
This commit is contained in:
zaro0508 2013-03-12 10:51:24 -07:00
parent 174d783623
commit c461e204f2
11 changed files with 68 additions and 123 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

47
README.rst Normal file
View File

@ -0,0 +1,47 @@
---------------------------------------------------------------------
Copyright 2013 Hewlett-Packard Development Company, L.P.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---------------------------------------------------------------------
Installation
============
The plugin can be installed like any other Jenkins plugin. Please refer to the Jenkins documentation on installing plugins.
Configuration
=============
Configuration of the plugin can be done on the Jenkins configure page (i.e. http://localhost:8080/configure). Click on the help if needed.
Developers
==========
Project site:
* https://wiki.jenkins-ci.org/display/JENKINS/Gearman+Plugin
Bug report:
* https://bugs.launchpad.net/~gearman-plugin-core
Cloning:
* https://github.com/openstack-infra/gearman-plugin.git
Patches are submitted via Gerrit at:
* https://review.openstack.org/
More details on how you can contribute is available on our wiki at:
* http://wiki.openstack.org/HowToContribute

61
README.txt
View File

@ -1,61 +0,0 @@
---------------------------------------------------------------------
Copyright 2013 Hewlett-Packard Development Company, L.P.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
---------------------------------------------------------------------
Jenkins does not support multiple masters. You can setup multiple Jenkins
masters but there is no coordination between them.
One problem with scheduling builds on Jenkins master (“MasterA”) server
is that MasterA only knows about its connected slaves. If all slaves on
MasterA are busy then MasterA will just put the next scheduled build on
its queue. Now MasterA needs to wait for an available slave to run
the build. This will be very in-efficient if your builds take a long
time to run. So..what if there is another Jenkins master (“MasterB”)
that has free slaves to service the next scheduled build? Your probably
saying “Then slaves on MasterB should run the build”. However MasterB
will never service the builds on MasterA's queue. The client that schedules
the builds must know about MasterB and then schedule builds on MasterB.
This is what we mean by lack of coordination between masters.
The gearman-plugin attempts to fill this gap.
This plugin integrates Gearman with Jenkins and will make it so that
any Jenkins slave on any Jenkins master can service a job in the queue.
It will essentially replace the Jenkins build queue with the Gearman
job queue. The job should stay in the gearman queue until there is a
Jenkins node that can run that job.
This is the typical workflow:
1) On a 'Launch Workers', we spawn a Gearman worker for each Jenkins
executor. We'll call these "executor worker threads".
Each executor worker thread is associated 1:1 with a Jenkins node (slave or master)
2) Now we register jobs for each Gearman executor depending on
projects and nodes. View the image to see the mapping.
3) On a 'Launch Workers', we spawn one more thread to be a Gearman
worker to handle job management for this Jenkins master. We'll call
it the "management worker thread" and register the following function:
stop:$hostname
4) Any Gearman client can connect to the Gearman server and send a
request to build a Jenkins project or cancel a project. View the
examples to see how this can be done.
5) The Gearman workers will service any client request that come
through to start/cancel a Jenkins build.

22
pom.xml
View File

@ -135,8 +135,6 @@
<taglist-maven-plugin.version>2.4</taglist-maven-plugin.version>
<versions-maven-plugin.version>1.2</versions-maven-plugin.version>
<xml-maven-plugin.version>1.0-beta-3</xml-maven-plugin.version>
<slf4j.version>1.7.2</slf4j.version>
<logback.version>1.0.9</logback.version>
<gson.version>2.2.2</gson.version>
<gearman.version>0.6-SNAPSHOT</gearman.version>
</properties>
@ -333,26 +331,6 @@
</plugins>
</reporting>
<dependencies>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>${logback.version}</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<version>${logback.version}</version>
</dependency>
<dependency>
<groupId>org.gearman</groupId>
<artifactId>gearman-java</artifactId>

8
src/main/java/hudson/plugins/gearman/AbstractWorkerThread.java
View File

@ -101,7 +101,7 @@ public abstract class AbstractWorkerThread implements Runnable {
*/
public void registerJobs() {
logger.info("----- AbstractorWorker registerJobs function ----");
logger.info("---- AbstractorWorker registerJobs function ----");
}
@ -122,10 +122,10 @@ public abstract class AbstractWorkerThread implements Runnable {
if (worker.isRunning()) {
try {
logger.info("Stopping " + getName() + ":" + getId().toString() +
logger.info("---- Stopping " + getName() + ":" + getId().toString() +
" (" + new Date().toString() + ")");
worker.unregisterAll();
worker.stop();
worker.shutdown();
} catch (Exception e) {
e.printStackTrace();
}
@ -154,7 +154,7 @@ public abstract class AbstractWorkerThread implements Runnable {
public void run() {
if (!worker.isRunning()) {
logger.info("Starting Worker "+ getName() + ":" + getId().toString() +
logger.info("---- Starting Worker "+ getName() + ":" + getId().toString() +
" ("+new Date().toString()+")");
worker.setWorkerID(name);
worker.setJobUniqueIdRequired(true);

6
src/main/java/hudson/plugins/gearman/ExecutorWorkerThread.java
View File

@ -135,7 +135,7 @@ public class ExecutorWorkerThread extends AbstractWorkerThread{
@Override
public void registerJobs() {
logger.info("----- Registering executor jobs on " + name + " ----");
logger.info("---- Registering executor jobs on " + name + " ----");
/*
* We start with an empty worker.
@ -161,7 +161,7 @@ public class ExecutorWorkerThread extends AbstractWorkerThread{
if (label == null) { // project has no label -> so register
// "build:projectName" on all nodes
String jobFunctionName = "build:" + projectName;
logger.info("Registering job " + jobFunctionName + " on "
logger.info("---- Registering job " + jobFunctionName + " on "
+ name);
worker.registerFunctionFactory(new CustomGearmanFunctionFactory(
jobFunctionName, StartJobWorker.class.getName(),
@ -180,7 +180,7 @@ public class ExecutorWorkerThread extends AbstractWorkerThread{
if (projectLabelNodes.contains(this.node)) {
String jobFunctionName = "build:" + projectName
+ ":" + projectLabel;
logger.info("Registering job " + jobFunctionName
logger.info("---- Registering job " + jobFunctionName
+ " on " + this.node.getNodeName());
// register with label (i.e. "build:$projectName:$projectLabel")
worker.registerFunctionFactory(new CustomGearmanFunctionFactory(

4
src/main/java/hudson/plugins/gearman/GearmanPluginConfig.java
View File

@ -53,7 +53,7 @@ public class GearmanPluginConfig extends GlobalConfiguration {
* Constructor.
*/
public GearmanPluginConfig() {
logger.info("--- GearmanPluginConfig Constructor ---");
logger.info("---- GearmanPluginConfig Constructor ---");
gearmanProxy = new GearmanProxy();
load();
@ -96,7 +96,7 @@ public class GearmanPluginConfig extends GlobalConfiguration {
if (launchWorker) {
// check for a valid connection to gearman server
logger.info("--- Check connection to Gearman Server " + host + ":"
logger.info("---- Check connection to Gearman Server " + host + ":"
+ port);
if (!GearmanPluginUtil.connectionIsAvailable(host, port, 5000)) {
launchWorker = false;

4
src/main/java/hudson/plugins/gearman/GearmanPluginUtil.java
View File

@ -61,10 +61,10 @@ public class GearmanPluginUtil {
} else {
try {
socket.connect(endPoint, timeout);
logger.info("Connection Success: " + endPoint);
logger.info("---- Connection Success: " + endPoint);
return true;
} catch (Exception e) {
logger.info("Connection Failure: " + endPoint + " message: "
logger.info("---- Connection Failure: " + endPoint + " message: "
+ e.getClass().getSimpleName() + " - " + e.getMessage());
} finally {
if (socket != null) {

10
src/main/java/hudson/plugins/gearman/GearmanProxy.java
View File

@ -50,7 +50,7 @@ public class GearmanProxy {
// constructor
public GearmanProxy() {
logger.info("--- GearmanProxy Constructor ---");
logger.info("---- GearmanProxy Constructor ---");
gewtHandles = new ArrayList<AbstractWorkerThread>();
gmwtHandles = new ArrayList<AbstractWorkerThread>();
@ -94,9 +94,9 @@ public class GearmanProxy {
try {
masterNode = Computer.currentComputer().getNode();
} catch (NullPointerException npe) {
logger.info("--- Master is offline");
logger.info("---- Master is offline");
} catch (Exception e) {
logger.info("--- Can't get Master");
logger.info("---- Can't get Master");
e.printStackTrace();
}
@ -140,7 +140,7 @@ public class GearmanProxy {
}
}
logger.info("--- Num of executors running = " + getNumExecutors());
logger.info("---- Num of executors running = " + getNumExecutors());
}
@ -160,7 +160,7 @@ public class GearmanProxy {
gmwtHandles.clear();
numWorkerNodes = 0;
logger.info("--- Num of executors running = " + getNumExecutors());
logger.info("---- Num of executors running = " + getNumExecutors());
}
/*

4
src/main/java/hudson/plugins/gearman/StartJobWorker.java
View File

@ -75,7 +75,7 @@ public class StartJobWorker extends AbstractGearmanFunction {
*/
@Override
public GearmanJobResult executeFunction() {
logger.info("----- Running executeFunction in " + name + " ----");
logger.info("---- Running executeFunction in " + name + " ----");
// decode the uniqueId from the client
String decodedUniqueId = null;
@ -125,7 +125,7 @@ public class StartJobWorker extends AbstractGearmanFunction {
Action [] actions = {runNode, params};
// schedule jenkins to build project
logger.info("Scheduling "+project.getName()+" build #" +
logger.info("---- Scheduling "+project.getName()+" build #" +
project.getNextBuildNumber()+" on " + runNodeName
+ " with UUID " + decodedUniqueId + " and build params " + buildParams);
Future<?> future = project.scheduleBuild2(0, new Cause.UserIdCause(), actions);

6
src/main/java/hudson/plugins/gearman/StopJobWorker.java
View File

@ -71,7 +71,7 @@ public class StopJobWorker extends AbstractGearmanFunction {
String jobResultMsg = "";
if (decodedUniqueId.isEmpty() || decodedUniqueId == null){
logger.info("Client passed in an invalid UUID");
logger.info("---- Client passed in an invalid UUID");
jobFailureMsg = "I need the job Id please";
jobResult = false;
} else {
@ -131,7 +131,7 @@ public class StopJobWorker extends AbstractGearmanFunction {
if (param.getUuid().equals(uuid)) {
logger.info("Aborting build : "+buildNum+": "+buildId+" on " + runNodeName
logger.info("---- Aborting build : "+buildNum+": "+buildId+" on " + runNodeName
+" with UUID " + uuid + " and build params " + buildParams);
// abort the running jenkins build
@ -174,7 +174,7 @@ public class StopJobWorker extends AbstractGearmanFunction {
if (param.getUuid().equals(uuid)) {
logger.info("Aborting build : "+buildNum+": "+buildId+" on " + runNodeName
logger.info("---- Aborting build : "+buildNum+": "+buildId+" on " + runNodeName
+" with UUID " + uuid + " and build params " + buildParams);
// abort the running jenkins build

19
src/main/resources/logback.xml
View File

@ -1,19 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="fileAppender" class="ch.qos.logback.core.FileAppender">
<file>GearmanPlugin.log</file>
<append>true</append>
<encoder>
<pattern>%d [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
</appender>
<appender name="consoleAppender" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
</appender>
<root level="TRACE">
<appender-ref ref="fileAppender" />
<appender-ref ref="consoleAppender" />
</root>
</configuration>