opendev
/
publications
Code Issues Proposed changes
publications/overview/index.html

915 lines
24 KiB
HTML
Raw Blame History

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en-US">
<head>
<meta name="generator" content=
"HTML Tidy for Linux/x86 (vers 1st November 2003), see www.w3.org" />
<title>Scaling OpenStack Development with Git, Gerrit, and Jenkins</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="copyright" content=
"Copyright &#169; 2005-2010 W3C (MIT, ERCIM, Keio)" />
<meta name="duration" content="40" />
<meta name="font-size-adjustment" content="0" />
<link rel="stylesheet" href="styles/slidy.css" type="text/css" />
<link rel="stylesheet" href="styles/openstack.css" type="text/css" />
<script src="scripts/slidy.js" charset="utf-8" type="text/javascript"></script>
<script src="scripts/jquery-1.7.2.min.js" charset="utf-8" type="text/javascript">
</script>
<script src="scripts/jquery-watch.js" charset="utf-8" type="text/javascript">
</script>
<script src="scripts/raphael-min.js" type="text/javascript" charset="utf-8"></script>
<style>
div.slide ul {
font-size: 120%
}
</style>
</head>
<body>
<div class="background"><img alt="" id="head-icon"
src="graphics/open-stack-cloud-computing-logo-2.png" /></div>
<div class="slide cover title">
<!-- hidden style graphics to ensure they are saved with other content -->
<img class="hidden" src="graphics/bullet.png" alt="" />
<img class="hidden" src="graphics/fold.gif" alt="" />
<img class="hidden" src="graphics/unfold.gif" alt="" />
<img class="hidden" src="graphics/fold-dim.gif" alt="" />
<img class="hidden" src="graphics/nofold-dim.gif" alt="" />
<img class="hidden" src="graphics/unfold-dim.gif" alt="" />
<img class="hidden" src="graphics/bullet-fold.gif" alt="" />
<img class="hidden" src="graphics/bullet-unfold.gif" alt="" />
<img class="hidden" src="graphics/bullet-fold-dim.gif" alt="" />
<img class="hidden" src="graphics/bullet-nofold-dim.gif" alt="" />
<img class="hidden" src="graphics/bullet-unfold-dim.gif" alt="" />
<img src="graphics/openstack-cloud-software-vertical-large.png" alt="OpenStack logo"
class="cover" /><br clear="all" />
<h1>Scaling OpenStack Development with Git, Gerrit, and Jenkins</h1>
<p>
Monty Taylor
&lt;<a href="mailto:mordred@inaugust.com">mordred@inaugust.com</a>&gt;<br />
James E. Blair
&lt;<a href="mailto:corvus@inaugust.com">corvus@inaugust.com</a>&gt;<br />
</div>
<div class="slide">
<h1>Systems</h1>
<ul>
<li>https://jenkins.openstack.org</li>
<li>https://review.openstack.org</li>
<li>http://ci.openstack.org (docs)</li>
<li>https://jenkins.stackforge.org</li>
</div>
<div class="slide">
<h1>Projects</h1>
<small>
<ul>
<li>nova (compute)</li>
<li>swift (object storge)</li>
<li>glance (image service)</li>
<li>keystone (identity service)</li>
<li>quantum (network service)</li>
<li>horizon (dashboard)</li>
<li>cinder (volume service)</li>
<li>python-novaclient</li>
<li>python-swiftclient</li>
<li>python-glanceclient</li>
<li>python-keystoneclient</li>
<li>python-quantumclient</li>
<li>python-cinderclient</li>
<li>python-openstackclient</li>
</ul>
</small>
</div>
<div class="slide">
<h1>Contributors</h1>
<ul>
<li>Individual Contributors</li>
<li>Commercial Entities</li>
<li>Number, quality, and area of contributions can change daily</li>
</ul>
</div>
<div class="slide">
<h1>Release Management</h1>
<ul>
<li>Time Based Releases</li>
<li>Six Month Cadence<ul>
<li>Tied to Ubuntu Releases</li>
</ul><li>Design summits each cycle</li>
<li>Continuously Open Trunk<ul>
<li>Develop directly on master</li>
</ul><li>One Month Milestone Releases</li>
<li>Post-release Stable Branches</li>
</ul>
</div>
<div class="slide">
<h1>Vision</h1>
<ul>
<li>Consistent Tooling</li>
<li>Consistent Process</li>
<li>Consistent Product</li>
<li>Multiplier Effect</li>
</ul>
</div>
<div class="slide">
<h1>Consistent Tooling</h1>
<ul>
<li>Minimize meta-development</li>
<li>Process divergence == wasted developer time</li>
<li>Lowers onboarding time</li>
<li>Consolidate tool development</li>
<li>Minimize project-specific weird build crud</li>
</ul>
</div>
<div class="slide">
<h1>Development Infrastructure Systems</h1>
<ul>
<li>Gerrit (code review, git) <a href="http://review.openstack.org">http://review.openstack.org</a></li>
<li>Jenkins (CI, automation) <a href="http://jenkins.openstack.org">http://jenkins.openstack.org</a><ul>
<li>pre-merge testing</li>
<li>post-merge testing</li>
<li>post-merge artifact management</li>
</ul>
<li>Launchpad (bugs, blueprints, releases, translations, SSO) <a href="http://launchpad.net/openstack">http://launchpad.net/openstack</a></li>
<li>Documentation servers</li>
<li>IRC bots</li>
<li>planet.openstack.org</li>
<li>Repository mirrors</li>
<li>Etherpad</li>
</ul>
</div>
<div class="slide">
<h1>OpenID SSO Integration</h1>
<img src="images/gerrit-sso.png"/>
</div>
<div class="slide">
<h1>Environment</h1>
<ul>
<li>Ubuntu</li>
<li>Python<ul>
<li>pep8 standards</li>
<li>openstack.common</li>
</ul><li>virtualenv/pip</li>
<li>IRC (#openstack-dev, #openstack-meeting)</li>
<li>devstack</li>
<li>gated trunk based on master</li>
</ul>
</div>
<div class="slide">
<h1>Gated Trunk</h1>
<ul>
<li>Ensures Code Quality</li>
<li>Protects developers<ul>
<li>Devs always start from working code</li>
</ul><li>Protects tree<ul>
<li>Bad code doesn't land</li>
</ul><li>Egalitarian<ul>
<li>Process is the same for everyone</li>
<li>Process is transparent</li>
<li>Process is automated</li>
</ul></ul>
</ul>
</div>
<div class="slide">
<h1>Everything Is Automated</h1>
<img src="images/jenkins-gate.png" />
</div>
<div class="slide">
<h1>Process Flow</h1>
<ul>
<li>Code is written and locally tested in a virtualenv</li>
<li>Code is submitted for code review to gerrit</li>
<li>Code is run through patch-uploaded automated checks</li>
<li>Code is peer-reviewed</li>
<li>Code is accepted or rejected by core team</li>
<li>Code is run through pre-merge automated checks</li>
<li>Code is merged or rejected</li>
<li>Code is run through post-merge automated checks</li>
</ul>
</div>
<div class="slide">
<h1>Gerrit</h1>
<ul>
<li>Developed by Google for Android</li>
<li>Stand-alone patch review system</li>
<li>Integration points: hooks, JSON queries, event-stream</li>
<li>Extensible review categories, default: Verified, Code-Review</li>
</ul>
</div>
<div class="slide">
<h1>Bug Integration - Gerrit</h1>
<img src="images/gerrit-bug.png"/>
</div>
<div class="slide">
<h1>Bug Integration - Launchpad</h1>
<img src="images/lp-bug.png"/>
</div>
<div class="slide">
<h1>Blueprints - Gerrit</h1>
<img src="images/gerrit-bp.png"/>
</div>
<div class="slide">
<h1>Blueprints - Launchpad</h1>
<img src="images/lp-bp.png"/>
</div>
<div class="slide">
<h1>Blueprints - Gerrit Topics</h1>
<img src="images/gerrit-bp-topic.png"/>
</div>
<div class="slide">
<h1>Pre-merge Check</h1>
<img src="images/gerrit-jenkins.png"/>
</div>
<div class="slide">
<h1>States of a Patch</h1>
<ul>
<li>Code Submitted</li>
<li>Code Verified</li>
<li>Code Reviewed</li>
<li>Code Accepted</li>
<li>Code Merged</li>
</ul>
</div>
<div class="slide">
<h1>Approved Reviews</h1>
<img src="images/gerrit-approved.png"/>
</div>
<div class="slide">
<h1>Types of Gerrit Triggers</h1>
<ul>
<li> Patchset uploaded </li>
<li> Change merged </li>
<li> Comment added (review state) </li>
</ul>
</div>
<div class="slide">
<h1>Git Review</h1>
<ul>
<li>External Git subcommand</li>
<li>Developers can easily incorporate code review into git workflow</li>
<li>Zero-configuration</li>
<li>Can be used for any project, being adopted by other projects</li>
</ul>
<pre>
corvus@shiprock:~/rs/github/quantum$ git commit -a
[new-versionpy ddf1dce] Base version.py on glance.
3 files changed, 28 insertions(+), 107 deletions(-)
delete mode 100644 version.py
corvus@shiprock:~/rs/github/quantum$ git review
remote: Resolving deltas: 0% (0/3)
remote:
remote: New Changes:
remote: https://review.openstack.org/3072
remote:
To ssh://corvus@review.openstack.org:29418/openstack/quantum.git
* [new branch] HEAD -> refs/for/master/bug/916018
</pre>
</div>
<div class="slide">
<h1>Types of Tests</h1>
<ul>
<li> Unit tests </li>
<li> Integration tests
<ul>
<li> May be able to run on virtual servers, should run on real servers </li>
<li> Difficult or impossible for a developer to run </li>
</ul>
</li>
</ul>
</div>
<div class="slide">
<h1>Specific Challenges/Solutions</h1>
<ul>
<li>Testing effect of merging change</li>
<li>Using cloud builders</li>
<li>Large numbers of similar projects</li>
<li>Disparate hardware configurations</li>
</ul>
</div>
<div class="slide">
<h1>Gerrit Git Prep</h1>
<ul>
<li>Test the result of the change, not the change</li>
<li>https://github.com/openstack/openstack-ci-puppet/blob/master/modules/jenkins_slave/files/slave_scripts/gerrit-git-prep.sh</li>
<li>For each gerrit change:
<ul>
<li>grabs target branch</li>
<li>cleans tree</li>
<li>merges change to be tested</li>
</ul>
</li>
</ul>
</div>
<div class="slide">
<h1>Interrelated Integration Testing</h1>
<ul>
<li>devstack-gate
<ul>
<li>spin up fresh node</li>
<li>run devstack</li>
<li>run integration tests</li>
</ul>
<li>Gate proposed change against current state of other projects</li>
</ul>
</div>
<div class="slide">
<h1>Devstack-Gate Problems</h1>
<ul>
<li>Tests are slow</li>
<li>Clouds suck</li>
<li>github sucks</li>
<li>pypi sucks</li>
<li>Ubuntu sucks</li>
<li><b><em>ALL NETWORK ACCESS IS FAILURE</em></b></li>
</ul>
</div>
<div class="slide">
<h1>Devstack-Gate Solutions</h1>
<ul>
<li>Create a new node</li>
<li>pre-fetch all needed packages, repos</li>
<li>snapshot to cloud image</li>
<li>maintain a pool of cloud nodes</li>
<li>slave can only be used for one test run</li>
<li>set of python and shell scripts triggered by jenkins</li>
</ul>
</div>
<div class="slide">
<h1>Jclouds-Plugin</h1>
<ul>
<li>ec2 plugin useless to us (sorry)</li>
<li>Adrian Cole and Cloudsoft wrote jclouds-plugin</li>
<li>Provisions on any provider via jclouds</li>
<li>Image pre-creation and pooling features</li>
<li>single use slaves</li>
<li>Slave provisioning as build step</li>
</ul>
</div>
<div class="slide">
<h1>Zuul</h1>
<ul>
<li>A general purpose trunk gating system</li>
<li>Interfaces with Gerrit and Jenkins</li>
<li>Flexible configuration allows for many kinds of project automation</li>
<li>Allows parallel testing of serialized changes</li>
</ul>
</div>
<div class="slide">
<h1>Bottlenecking</h1>
<ul>
<li>Serialize changes across all projects</li>
<li>Speculative execution of tests</li>
<li>Run in parallel in order triggered</li>
<li>Assume success</li>
<li>Start over on failure</li>
</ul>
</div>
<div class="slide" id="zuul">
<h1>Zuul Simulation</h1>
<script>
var green = "#00bb44";
var red = "#bb0000";
var light_green = "#8dbb9e";
var light_red = "#bb7777";
function Change(paper, name, number, x, y, delay)
{
this.set = paper.set();
this.x = x;
this.y = y;
this.text = paper.text(x, y, name+'\n#'+number);
this.set.push(this.text);
this.text.attr({"font-size": 16});
width = this.text.getBBox().width+16;
height = this.text.getBBox().height+12;
this.left = x-width/2;
this.right = x+width/2;
this.top = y-height/2;
this.bottom = y+height/2;
this.rect = paper.rect(this.left, this.top, width, height, 2);
this.set.push(this.rect);
this.rect.attr({fill: "#ccc", stroke: "#bbb"});
this.rect.toBack();
this.set.attr({opacity: 0});
this.set.animate(Raphael.animation({opacity: 1}, 500, 'linear').delay(delay));
this.hide = function ()
{
var set = this.set;
var remove = function () { set.remove(); };
this.set.animate({opacity: 0}, 1000, 'linear', remove);
}
this.merge = function(x, y)
{
this.set.toFront();
this.text.animate(Raphael.animation({opacity: 0}, 500, 'linear'));
this.rect.animate(Raphael.animation(
{width: 18, height: 18, r: 9, x: x, y: y, stroke: green},
1000).delay(500));
}
this.provisionalResult = function(success)
{
if (success)
{
color = light_green;
} else {
color = light_red;
}
this.rect.animate({fill: color}, 500);
}
this.result = function(success)
{
if (success)
{
color = green;
} else {
color = red;
}
this.rect.animate({fill: color}, 500);
}
this.reset = function(success)
{
this.rect.animate({fill: "#ccc"}, 500);
}
}
function linePath(x1, y1, x2, y2) {
return "M"+x1+","+y1+"L"+x2+","+y2;
}
function Dependency(paper, source, target, delay)
{
this.set = paper.set();
this.source = source;
this.target = target;
startx = this.source.right + 8;
starty = this.source.y;
tipx = this.target.left - 8;
tipy = this.target.y;
path = "M" + (startx) + "," + starty;
path = path + "L" + (tipx-15) + "," + tipy;
this.line = paper.path(path);
this.set.push(this.line);
this.line.attr("stroke-width", "2");
this.line.attr("fill", "#000");
this.line.attr("stroke", "#000");
path = "M" + (tipx - 20) + "," + (tipy - 10);
path = path + "L" + (tipx) + "," + (tipy);
path = path + "L" + (tipx - 20) + "," + (tipy + 10);
path = path + "Q" + (tipx - 12) + "," + (tipy);
path = path + "," + (tipx - 20) + "," + (tipy - 10);
path = path + "Z";
this.arrow = paper.path(path);
this.set.push(this.arrow);
this.arrow.attr("fill", "#000");
this.arrow.attr("stroke", "#000");
this.set.attr({opacity: 0});
this.set.animate(Raphael.animation({opacity: 1}, 500, 'linear').delay(delay));
this.hide = function ()
{
var set = this.set;
var remove = function () { set.remove(); };
this.set.animate({opacity: 0}, 1000, 'linear', remove);
}
}
function TestSeries(paper, change, tests)
{
this.x = change.x - 20;
this.y = change.bottom + 6;
offset = 40; // the "merge" test length
height = 0;
for (test in tests)
{
len = tests[test][0];
if (len > height)
{
height = len;
}
}
this.width = (1+10*tests.length)
this.height = offset + height + 10;
this.set = paper.set();
startpath = "M" + this.x + "," + this.y;
line = paper.path(startpath+"l0," + offset);
this.set.push(line);
startpath = "M" + this.x + "," + (this.y + offset);
line = paper.path(startpath+"l0," + (this.height - offset));
this.set.push(line);
line.attr("stroke-dasharray", "-");
line.attr("stroke", green);
line = paper.path(startpath + "l" + (1+10*tests.length) + ",0");
this.set.push(line);
var global_success = true;
curx = this.x;
cury = this.y + offset;
for (var test in tests)
{
len = tests[test][0];
success = tests[test][1];
curx = curx + 10;
line = paper.path("M" + curx + "," + cury + "l0," + len);
this.set.push(line);
line = paper.path("M" + curx + "," + cury + "m0," + len +
"l0," + (this.height-offset-len));
this.set.push(line);
line.attr("stroke-dasharray", "-");
if (success) {
line.attr("stroke", green);
} else {
line.attr("stroke", red);
global_success = false;
}
}
var complete = function ()
{
change.provisionalResult(global_success);
}
this.rect = paper.rect(this.x-2, this.y-2, this.width+4, this.height+2);
this.rect.attr({fill: "#fff", stroke: "#fff"})
this.rect.animate({y: this.y+this.height+2, height: 0}, 36*this.height, //36
'linear', complete);
this.hide = function ()
{
var set = this.set;
var remove = function () { set.remove(); };
this.set.animate({opacity: 0}, 2000, 'linear', remove);
}
}
var zuul_simulation = {
paper: null,
state: 0,
finished: false,
setup: function()
{
this.paper = Raphael(200, 200, 760, 400);
this.finished = false;
this.state = 0;
},
teardown: function()
{
if (this.paper != null) {
this.paper.remove();
this.paper = null;
}
this.finished = true;
},
next: function()
{
this.state = this.state + 1;
if (this.state == 1)
{
head_label = this.paper.text(520, 385, "HEAD");
head_label.attr({"font-size": 16});
nova_label = this.paper.text(520, 300, "nova");
nova_label.attr({"font-size": 16});
nova_label.attr({x:this.paper.width-(nova_label.getBBox().width/2)-10});
nova_end = [(this.paper.width-nova_label.getBBox().width-16), 300];
nova_line = this.paper.path(linePath(520, 300, nova_end[0], nova_end[1]))
nova_line.attr({"stroke-width": 2, fill: "#000", stroke: "#000"});
nova_head = this.paper.circle(520, 300, 9);
nova_head.attr({fill: "#5069c9", stroke:"#5069c9"});
keystone_label = this.paper.text(520, 360, "keystone");
keystone_label.attr({"font-size": 16});
keystone_label.attr({x:this.paper.width-(keystone_label.getBBox().width/2)-10});
keystone_end = [(this.paper.width-keystone_label.getBBox().width-16), 360];
keystone_line = this.paper.path(linePath(520, 360, keystone_end[0], keystone_end[1]));
keystone_line.attr({"stroke-width": 2, fill: "#000", stroke: "#000"});
keystone_head = this.paper.circle(520, 360, 9);
keystone_head.attr({fill: "#eead5f", stroke:"#eead5f"});
}
if (this.state == 2)
{
change1 = new Change(this.paper, 'nova', '1', 480, 40, 0);
change2 = new Change(this.paper, 'nova', '2', 340, 40, 1000);
dep21 = new Dependency(this.paper, change2, change1, 1800);
change3 = new Change(this.paper, 'keystone', '3', 180, 40, 2000);
dep32 = new Dependency(this.paper, change3, change2, 2800);
change4 = new Change(this.paper, 'nova', '4', 40, 40, 3000);
dep43 = new Dependency(this.paper, change4, change3, 3800);
}
if (this.state == 3) {
change1_test = new TestSeries(this.paper, change1, [[80, 1], [60, 1], [95, 1]]);
change2_test = new TestSeries(this.paper, change2, [[80, 1], [60, 1], [90, 1]]);
change3_test = new TestSeries(this.paper, change3, [[50, 1], [30, 0]]);
change4_test = new TestSeries(this.paper, change4, [[80, 1], [60, 1], [85, 0]]);
}
if (this.state == 4) {
change1_test.hide();
change1.result(1);
}
if (this.state == 5) {
change1.merge(480, 290);
dep21.hide();
setTimeout(function() {
nova_line.attr({path: linePath(490, 300, nova_end[0], nova_end[1])})
} , 2000);
}
if (this.state == 6) {
change2_test.hide();
change2.result(1);
}
if (this.state == 7) {
change2.merge(450, 290);
dep32.hide();
setTimeout(function() {
nova_line.attr({path: linePath(460, 300, nova_end[0], nova_end[1])})
} , 2000);
}
if (this.state == 8) {
change3_test.hide();
change3.result(0);
}
if (this.state == 9) {
change4.reset();
change4_test.hide();
change3.hide();
dep43.hide();
}
if (this.state == 10) {
change4_test = new TestSeries(this.paper, change4, [[80, 1], [60, 1], [85, 1]]);
}
if (this.state == 11) {
change4_test.hide();
change4.result(1);
}
if (this.state == 12) {
change4.merge(420, 290);
setTimeout(function() {
nova_line.attr({path: linePath(430, 300, nova_end[0], nova_end[1])})
} , 2000);
}
if (this.state == 13) {
this.finished = true;
}
},
}; /* zuul_simulation */
$(document).ready(function() {
$("#zuul").bind("DOMSubtreeModified", function() {
if ($(this).hasClass("hidden")) {
$("#zuul").unbind('click');
w3c_slidy.mouse_click_enabled = true;
zuul_simulation.teardown();
} else {
zuul_simulation.setup();
zuul_simulation.next();
w3c_slidy.mouse_click_enabled = false;
$("#zuul").bind('click', function() {
zuul_simulation.next();
if (zuul_simulation.finished) {
$("#zuul").unbind('click');
w3c_slidy.mouse_click_enabled = true;
}
});
}
}); /* dom modified */
}); /* document ready */
</script>
</div>
<div class="slide">
<h1>Zuul Check Queue</h1>
<pre>
queues:
- name: check
manager: IndependentQueueManager
trigger:
- event: patchset-created
success:
verified: 1
failure:
verified: -1
</pre>
</div>
<div class="slide">
<h1>Zuul Gate Queue</h1>
<pre>
queues:
- name: gate
manager: DependentQueueManager
trigger:
- event: comment-added
approval:
- approved: 1
start:
verified: 0
success:
verified: 2
submit: true
failure:
verified: -2
</pre>
</div>
<div class="slide">
<h1>Zuul Post-Merge Queue</h1>
<pre>
queues:
- name: post
manager: IndependentQueueManager
trigger:
- event: ref-updated
ref: ^(?!refs/).*$
</pre>
</div>
<div class="slide">
<h1>Zuul Silent Queue</h1>
<pre>
queues:
- name: silent
manager: IndependentQueueManager
trigger:
- event: patchset-created
</pre>
</div>
<div class="slide">
<h1>Zuul Project Configuration</h1>
<pre>
projects:
- name: openstack/nova
check:
- gate-nova-merge:
- gate-nova-pep8
- gate-nova-python26
- gate-nova-python27
- gate-integration-tests-devstack-vm
gate:
- gate-nova-merge:
- gate-nova-pep8
- gate-nova-python26
- gate-nova-python27
- gate-integration-tests-devstack-vm
post:
- nova-tarball
silent:
- gate-tempest-devstack-vm
</pre>
</div>
<div class="slide">
<h1>Templated Jobs</h1>
<ul>
<li>TONS of consistent projects</li>
<li>Manage everything through git and code review</li>
<li><a href="https://github.com/openstack/openstack-ci-puppet/tree/master/modules/jenkins_jobs">https://github.com/openstack/openstack-ci-puppet/tree/master/modules/jenkins_jobs</a></li>
<li>Manage jobs as yaml files in git</li>
<li>Jobs updated via Jenkins API</li>
<li>groupings of jobs supported "nova: python_jobs"</li>
</ul>
</div>
<div class="slide">
<h1>Example Job</h1>
<pre>
main:
name: gate-ceilometer-python26-essex
review_site: review.stackforge.org
github_org: stackforge
project: ceilometer
authenticatedBuild: true
concurrent: true
assigned_node: oneiric
triggers:
- zuul
builders:
- gerrit_git_prep
- python26_essex
</pre>
</div>
<div class="slide">
<h1>Example Template</h1>
<pre>
project:
template: python_jobs
values:
name: python-novaclient
disabled: false
github_org: openstack
review_site: review.openstack.org
publisher_site: nova.openstack.org
tarball_project: nova
doc_publisher_site: docs.openstack.org
node: precise
</pre>
</div>
<div class="slide">
<h1>Scaling Hardware Needs</h1>
<ul>
<li>Different vendors have different combinations of configurations</li>
<li><a
href="http://ci.openstack.org/third_party.html#the-jenkins-gerrit-trigger-plugin-way">http://ci.openstack.org/third_party.html#the-jenkins-gerrit-trigger-plugin-way</a></li>
<li>Run their own jenkins</li>
<li>Consume gerrit event stream</li>
<li>Their jenkins votes VRFY +1/-1</li>
<li>Our jenkins votes VRFY +2/-2</li>
<li>They can run their lab - we don't have to know anything about it</li>
</ul>
</div>
<div class="slide">
<h1>Thanks!</h1>
<p><img src="images/stack-o-pancakes-150x150.png"/>
<p>
These slides available at: <a href="https://github.com/openstack-ci/publications">https://github.com/openstack-ci/publications</a>
</p>
</div>
</body>
</html>
View Git Blame Copy Permalink