346 lines
8.0 KiB
HTML
346 lines
8.0 KiB
HTML
<?xml version="1.0" encoding="utf-8"?>
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
|
|
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en-US">
|
|
<head>
|
|
<meta name="generator" content=
|
|
"HTML Tidy for Linux/x86 (vers 1st November 2003), see www.w3.org" />
|
|
<title>OpenStack Project Continuous Integration</title>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
|
<meta name="copyright" content=
|
|
"Copyright © 2005-2010 W3C (MIT, ERCIM, Keio)" />
|
|
<meta name="duration" content="40" />
|
|
<meta name="font-size-adjustment" content="0" />
|
|
<link rel="stylesheet" href="styles/slidy.css" type="text/css" />
|
|
<link rel="stylesheet" href="styles/openstack.css" type="text/css" />
|
|
<script src="scripts/slidy.js" charset="utf-8" type="text/javascript">
|
|
</script>
|
|
<style>
|
|
div.slide ul {
|
|
font-size: 120%
|
|
}
|
|
</style>
|
|
</head>
|
|
<body>
|
|
<div class="background"><img alt="" id="head-icon"
|
|
src="graphics/open-stack-cloud-computing-logo-2.png" /></div>
|
|
|
|
<div class="slide cover title">
|
|
<!-- hidden style graphics to ensure they are saved with other content -->
|
|
<img class="hidden" src="graphics/bullet.png" alt="" />
|
|
<img class="hidden" src="graphics/fold.gif" alt="" />
|
|
<img class="hidden" src="graphics/unfold.gif" alt="" />
|
|
<img class="hidden" src="graphics/fold-dim.gif" alt="" />
|
|
<img class="hidden" src="graphics/nofold-dim.gif" alt="" />
|
|
<img class="hidden" src="graphics/unfold-dim.gif" alt="" />
|
|
<img class="hidden" src="graphics/bullet-fold.gif" alt="" />
|
|
<img class="hidden" src="graphics/bullet-unfold.gif" alt="" />
|
|
<img class="hidden" src="graphics/bullet-fold-dim.gif" alt="" />
|
|
<img class="hidden" src="graphics/bullet-nofold-dim.gif" alt="" />
|
|
<img class="hidden" src="graphics/bullet-unfold-dim.gif" alt="" />
|
|
|
|
<img src="graphics/openstack-cloud-software-vertical-large.png" alt="OpenStack logo"
|
|
class="cover" /><br clear="all" />
|
|
<h1>OpenStack Project Continuous Integration</h1>
|
|
<p>
|
|
Monty Taylor
|
|
<<a href="mailto:mordred@inaugust.com">mordred@inaugust.com</a>><br />
|
|
|
|
</div>
|
|
|
|
|
|
<div class="slide">
|
|
<h1>OpenStack CI</h1>
|
|
<ul>
|
|
<li>Run by OpenStack Infrastructure Team</li>
|
|
<li>OpenStack Infrastructure Team is an open team</li>
|
|
<li>OpenStack Infrastructure Team would love you to join</li>
|
|
<li>All repos are at: <a href="https://github.com/openstack-infra">https://github.com/openstack-infra</a></li>
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Scale</h1>
|
|
<ul>
|
|
<li>Over 500 Active Technical Contributors (and growing)</li>
|
|
<li>As many as 200 events an hour (and growing)</li>
|
|
<li>18 (integarted) projects (and growing)</li>
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Automation</h1>
|
|
<ul>
|
|
|
|
|
|
<li>Automate Away all non-creative tasks</li>
|
|
<li>Humans get repeatable processes wrong</li>
|
|
<li>We don't have enough humans</li>
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Following from that</h1>
|
|
<ul>
|
|
|
|
<li>Consistency</li>
|
|
<li>Repeatability</li>
|
|
<li>Availability</li>
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Consistency</h1>
|
|
<ul>
|
|
|
|
<li>To automate 18 projects, they need to act the same</li>
|
|
<li>Inteface for devs for 18 projects needs to be the same</li>
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Repeatability</h1>
|
|
<ul>
|
|
|
|
<li>Everything must be able to run 500x a day without failing</li>
|
|
<li>Everything must be able to run 500x a day without failing</li>
|
|
<li>Everything must be able to run 500x a day without failing</li>
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Availability</h1>
|
|
<ul>
|
|
|
|
<li>Once everything is automated, the projects stops if the automation does</li>
|
|
<li>Access to manage needs to be egalitarian</li>
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Gated Trunk</h1>
|
|
<ul>
|
|
<li>Ensures Code Quality</li>
|
|
<li>Protects developers<ul>
|
|
<li>Devs always start from working code</li>
|
|
</ul><li>Protects tree<ul>
|
|
<li>Bad code doesn't land</li>
|
|
</ul><li>Egalitarian<ul>
|
|
<li>Process is the same for everyone</li>
|
|
<li>Process is transparent</li>
|
|
<li>Process is automated</li>
|
|
</ul></ul>
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Gating Implemenation</h1>
|
|
<ul>
|
|
|
|
<li>Test everything we can on every commit before it lands</li>
|
|
<li>Test change when they are proposed</li>
|
|
<li>Peer Code review ALL changes</li>
|
|
<li>Test the resulting state of the tree <em>before</em> it becomes the resulting state</li>
|
|
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Test Everything</h1>
|
|
<ul>
|
|
|
|
<li>unittests</li>
|
|
<li>style tests</li>
|
|
<li>integration tests</li>
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Gating Impl</h1>
|
|
<ul>
|
|
|
|
<li>Gerrit
|
|
<ul>
|
|
<li>Repos</li>
|
|
<li>Code Review</li>
|
|
<li>Event Stream
|
|
</ul>
|
|
</li>
|
|
<li>Zuul
|
|
<ul>
|
|
<li>Takes actions based on gerrit events</li>
|
|
</ul>
|
|
</li>
|
|
<li>Jenkins
|
|
<ul>
|
|
<li>Runs tests</li>
|
|
</ul>
|
|
</li>
|
|
<li>devstack-gate
|
|
<ul>
|
|
<li>Prepped pool of slaves</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Gerrit</h1>
|
|
<ul>
|
|
<li>Developed by Google for Android</li>
|
|
<li>Stand-alone patch review system</li>
|
|
<li>Integration points: hooks, JSON queries, event-stream</li>
|
|
<li>Extensible review categories, default: Verified, Code-Review</li>
|
|
</ul>
|
|
</div>
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>devstack-gate</h1>
|
|
<ul>
|
|
|
|
<li>Clouds can't create machines consistently enough
|
|
<ul>
|
|
<li>Make a pool of test machines</li>
|
|
</ul></li>
|
|
<li>Internet breaks often
|
|
<ul>
|
|
<li>Prefetch all network needs of devstack</li>
|
|
</ul></li>
|
|
<li>configure, run, and test result of devstack
|
|
<ul>
|
|
<li>Install every service in openstack using devstack</li>
|
|
<li>Test result cloud using tempest</li>
|
|
<li>Change to a project is tested against tip of all others</li>
|
|
</ul></li>
|
|
<li>Devstack does evil things to machines
|
|
<ul>
|
|
<li>Throw away machines after each test run</li>
|
|
</ul></li>
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>System Management</h1>
|
|
<ul>
|
|
|
|
<li>Puppet</li>
|
|
<li>Jenkins Job Builder</li>
|
|
<li>jeepyb</li>
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>jenkins-job-builder</h1>
|
|
<ul>
|
|
|
|
<li>Availability of jenkins config to everyone</li>
|
|
<li>Set of yaml files describing our jenkins jobs</li>
|
|
<li>Macro-based for composition</li>
|
|
<li>yaml files are in gerrit</li>
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>projects.yaml</h1>
|
|
<ul>
|
|
|
|
<li>Availability of project config to everyone</li>
|
|
<li>Drives config of repos in gerrit and github and zuul</li>
|
|
<li>Drives pypi mirror</li>
|
|
<li>Source of info for pull request closer</li>
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>pypi mirror</h1>
|
|
<ul>
|
|
|
|
<li>Availability of pip commands</li>
|
|
<li>Common set of requirements</li>
|
|
<li>PyPI isn't available enough</li>
|
|
<li>pip download reqs for all projects in projects.yaml</li>
|
|
<li>turn download cache into static pypi mirror</li>
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Docmentation</h1>
|
|
<ul>
|
|
|
|
<li>build sphinx docs on each commit</li>
|
|
<li>upload to openstack.org</li>
|
|
<li>optionally trigger rtfd</li>
|
|
<li>why not gate on it?</li>
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Translations</h1>
|
|
<ul>
|
|
|
|
<li>apparently some people don't speak english</li>
|
|
<li>transifex for translations management</li>
|
|
<li>On each commit to trunk, upload latest strings to transifex</li>
|
|
<li>Periodicly, fetch translated strings from transifex and propose change to gerrit</li>
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Tag Based Releasing</h1>
|
|
<ul>
|
|
|
|
<li>Triggered by git tag</li>
|
|
<li>If tag exists, that's the version</li>
|
|
<li>If tag doesn't, version is either:
|
|
<ul>
|
|
<li>$latesttag.$revcount.g$gitsha</li>
|
|
<li>$versionarg.a$revcount.g$gitsha</li>
|
|
</ul></li>
|
|
<li>On tag:
|
|
<ul>
|
|
<li>Upload sdist tarball to tarballs.o.o</li>
|
|
<li>If release tag (no letters) upload to pypi too</li>
|
|
</ul></li>
|
|
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Things we're working on</h1>
|
|
<ul>
|
|
|
|
<li>Jenkins Gearman Plugin</li>
|
|
<li>Upstream branch tracking</li>
|
|
<li>Logstash for devstack jobs</li>
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="slide">
|
|
<h1>Thanks!</h1>
|
|
|
|
<p>All repos are at: <a href="https://github.com/openstack-infra">https://github.com/openstack-infra</a></p>
|
|
|
|
<p>
|
|
These slides available at: <a href="https://openstack-ci.github.com/publications">https://openstack-ci.github.com/publications</a>
|
|
</p>
|
|
|
|
</div>
|
|
|
|
</body>
|
|
</html>
|