A generic zuul worker using the gearman protocol.
Go to file
OpenDev Sysadmins 2a26998726 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:50:52 +00:00
datasets Tweak timeouts based on recent failures 2014-02-20 13:54:51 +11:00
doc Replace ci.o.o links with docs.o.o/infra 2015-05-14 21:46:42 +00:00
etc Replace openstack.org git:// URLs with https:// 2019-03-24 20:36:22 +00:00
tests Replace openstack.org git:// URLs with https:// 2019-03-24 20:36:22 +00:00
tools Use account id with new gerrit 2015-12-17 17:14:57 +11:00
turbo_hipster Replace openstack.org git:// URLs with https:// 2019-03-24 20:36:22 +00:00
.gitignore Check the logs for migrations that take too long 2013-12-12 13:39:07 +11:00
.gitreview OpenDev Migration Patch 2019-04-19 19:50:52 +00:00
.testr.conf Add in tests for ZuulManager and ZuulClient 2014-03-28 15:53:57 +11:00
LICENSE add in license 2013-08-12 23:25:35 +10:00
README.md More worker build documentation. 2013-12-29 22:02:20 +11:00
requirements.txt Remove argparse from requirements 2016-01-20 19:26:31 +01:00
setup.cfg remove python 2.6 trove classifier 2015-12-23 01:31:17 +00:00
setup.py update install requirements 2013-11-21 10:37:27 +11:00
test-requirements.txt Fix hacking and requirements 2015-06-04 14:13:46 +10:00
tox.ini Deprecated tox -downloadcache option removed 2015-12-11 23:41:46 +01:00

README.md

turbo-hipster

A set of CI tools.

worker_server.py is a worker server that loads and runs task_plugins.

Each task_plugin is a zuul gearman worker that implements, handles, executes a job, uploads/post-processes the logs and sends back the results to zuul.

Plugins

gate_real_db_upgrade: Runs the db_sync migrations on each dataset available in the datasets subdir.

Installation

  • boot a fresh Ubuntu image
  • setup ssh authentication for your admin team
  • apt-get update; apt-get dist-upgrade
  • adduser th
  • apt-get install vim git python-pip python-setuptools python-keystoneclient virtualenvwrapper python-eventlet python-numpy python-mysqldb python-git python-gitdb python-netaddr python-pkg-resources libxml2-dev libxml2-utils libxslt-dev git-review libxml2-dev libxml2-utils libxslt-dev libmysqlclient-dev pep8 postgresql-server-dev-9.1 python2.7-dev python-coverage python-netaddr
  • pip install -U pip
  • apt-get purge python-pip
  • cd /home/th; git clone http://github.com/stackforge/turbo-hipster
  • apply any patches you need
  • python setup.py install
  • cp turbo_hipster/task_plugins/gate_real_db_upgrade/*.sh /usr/local/lib/python2.7/dist-packages/turbo_hipster/task_plugins/gate_real_db_upgrade/
  • cp -R etc/* /etc/
  • mkdir /var/lib/turbo-hipster
  • chown -R th.th /var/lib/turbo-hipster
  • mkdir /var/log/turbo-hipster
  • chown -R th.th /var/log/turbo-hipster
  • install your chosen MySQL-like database engine (percona, maria, mysql)
  • mysql -u root --password=$1 -e "create user 'nova'@'localhost' identified by 'tester';"
  • mysql -u root --password=$1 -e "create user 'nova'@'172.16.0.2' identified by 'tester';"
  • mysql -u root --password=$1 -e "grant all privileges on . to 'nova'@'localhost' with grant option;"
  • mysql -u root --password=$1 -e "grant all privileges on . to 'nova'@'172.16.0.2' with grant option;"
  • /etc/rc.local
  • rsync the datasets from the master
  • logrotate -f /etc/logrotate.conf
  • chmod -R ugo+r /var/log/*
  • chmod ugo+rx /var/log/mysql
  • mkdir /var/cache/pip
  • chmod -R ugo+rwx /var/cache/pip
  • touch /var/log/mysql/slow-queries.log
  • /etc/init.d/turbo-hipster start