x
/
turbo-hipster
Code Issues Proposed changes
A generic zuul worker using the gearman protocol.
309 Commits 1 Branch 0 Tags 1.9 MiB
Python 83%
Shell 17%
Go to file
Joshua Hesketh 8e166e0dc6 Add support to upgrade via Juno and Kilo 
Rather than stepping through the migrations just in master (which
works fine assuming they are no different to previous releases)
checkout the correct stable branch and walk through that.

Change-Id: Iff79684990cbbaa8c6ed2474aca77a2f73d64b96
 		2015-05-01 11:52:00 +10:00
datasets Tweak timeouts based on recent failures 2014-02-20 13:54:51 +11:00
doc Update config.yaml to define jobs rather than plugins 2014-12-02 13:24:30 +11:00
etc Git origin can't have a trailing slash 2014-05-23 15:59:17 +10:00
tests Force migrate flavour data 2015-04-30 09:57:12 +10:00
tools Fix the zuul_enqueue tool 2014-08-14 12:54:42 +10:00
turbo_hipster Add support to upgrade via Juno and Kilo 2015-05-01 11:52:00 +10:00
.gitignore Check the logs for migrations that take too long 2013-12-12 13:39:07 +11:00
.gitreview Add .gitreview file and fix documentation build 2013-11-21 10:31:33 +11:00
.testr.conf Add in tests for ZuulManager and ZuulClient 2014-03-28 15:53:57 +11:00
LICENSE add in license 2013-08-12 23:25:35 +10:00
README.md More worker build documentation. 2013-12-29 22:02:20 +11:00
requirements.txt Pin python-daemon<2.0 2015-01-12 11:29:43 +13:00
setup.cfg Move extra commands into folder 2014-01-09 13:56:07 +08:00
setup.py update install requirements 2013-11-21 10:37:27 +11:00
test-requirements.txt Improve how jobs log 2014-12-02 13:23:23 +11:00
tox.ini tox >= 1.6 seems to be needed, specify that 2014-11-26 09:12:22 -05:00

README.md

turbo-hipster

A set of CI tools.

worker_server.py is a worker server that loads and runs task_plugins.

Each task_plugin is a zuul gearman worker that implements, handles, executes a job, uploads/post-processes the logs and sends back the results to zuul.

Plugins

gate_real_db_upgrade: Runs the db_sync migrations on each dataset available in the datasets subdir.

Installation

  • boot a fresh Ubuntu image
  • setup ssh authentication for your admin team
  • apt-get update; apt-get dist-upgrade
  • adduser th
  • apt-get install vim git python-pip python-setuptools python-keystoneclient virtualenvwrapper python-eventlet python-numpy python-mysqldb python-git python-gitdb python-netaddr python-pkg-resources libxml2-dev libxml2-utils libxslt-dev git-review libxml2-dev libxml2-utils libxslt-dev libmysqlclient-dev pep8 postgresql-server-dev-9.1 python2.7-dev python-coverage python-netaddr
  • pip install -U pip
  • apt-get purge python-pip
  • cd /home/th; git clone http://github.com/stackforge/turbo-hipster
  • apply any patches you need
  • python setup.py install
  • cp turbo_hipster/task_plugins/gate_real_db_upgrade/*.sh /usr/local/lib/python2.7/dist-packages/turbo_hipster/task_plugins/gate_real_db_upgrade/
  • cp -R etc/* /etc/
  • mkdir /var/lib/turbo-hipster
  • chown -R th.th /var/lib/turbo-hipster
  • mkdir /var/log/turbo-hipster
  • chown -R th.th /var/log/turbo-hipster
  • install your chosen MySQL-like database engine (percona, maria, mysql)
  • mysql -u root --password=$1 -e "create user 'nova'@'localhost' identified by 'tester';"
  • mysql -u root --password=$1 -e "create user 'nova'@'172.16.0.2' identified by 'tester';"
  • mysql -u root --password=$1 -e "grant all privileges on . to 'nova'@'localhost' with grant option;"
  • mysql -u root --password=$1 -e "grant all privileges on . to 'nova'@'172.16.0.2' with grant option;"
  • /etc/rc.local
  • rsync the datasets from the master
  • logrotate -f /etc/logrotate.conf
  • chmod -R ugo+r /var/log/*
  • chmod ugo+rx /var/log/mysql
  • mkdir /var/cache/pip
  • chmod -R ugo+rwx /var/cache/pip
  • touch /var/log/mysql/slow-queries.log
  • /etc/init.d/turbo-hipster start