834733bbe5
This patch exports a new env var for the script, the $TH_RESULT_FILE. If the launched script writes something into the file name specified in that env var and if the content of that file doesn't appear to indicate a successful build, the content of the file is used as summary message sent back to Zuul. Originally, this patch attempted to do a similar thing through reading back stuff from the shell's output log and reporting the last line from that output, but the two major disadvantages were the inclusion of timestamps in the log output, and Python's enrichment of the log line with timestamps. I pondered writing a custom Python's logging.Handler which simply remembers the last message, but the required changes in utils.execute_to_log appeared rather messy to my untrained eye. The main driver behind this change is a need to differentiate from hard build failures (where we don't get any build artifacts because the build simply failed) from "mere" test failures. Yes, our (KDE) tests are still sloppy, and we have a fair amount of broken test still around, which is why this difference matters to us. The result field in Zuul is more or less free-form, so this patch does not impose any restrictions about its format, except the "cannot start with 'SUCCESS'" because that one is indeed very, very special. It can be seen in action at http://ci-logs.kde.flaska.net/matrix.html . Change-Id: I48c29d2566da12b02dcf27a551a058ecc4a303d4 |
||
|---|---|---|
| datasets | ||
| doc | ||
| etc | ||
| tests | ||
| tools | ||
| turbo_hipster | ||
| .gitignore | ||
| .gitreview | ||
| .testr.conf | ||
| LICENSE | ||
| README.md | ||
| requirements.txt | ||
| setup.cfg | ||
| setup.py | ||
| test-requirements.txt | ||
| tox.ini | ||
README.md
turbo-hipster
A set of CI tools.
worker_server.py is a worker server that loads and runs task_plugins.
Each task_plugin is a zuul gearman worker that implements, handles, executes a job, uploads/post-processes the logs and sends back the results to zuul.
Plugins
gate_real_db_upgrade: Runs the db_sync migrations on each dataset available in the datasets subdir.
Installation
- boot a fresh Ubuntu image
- setup ssh authentication for your admin team
- apt-get update; apt-get dist-upgrade
- adduser th
- apt-get install vim git python-pip python-setuptools python-keystoneclient virtualenvwrapper python-eventlet python-numpy python-mysqldb python-git python-gitdb python-netaddr python-pkg-resources libxml2-dev libxml2-utils libxslt-dev git-review libxml2-dev libxml2-utils libxslt-dev libmysqlclient-dev pep8 postgresql-server-dev-9.1 python2.7-dev python-coverage python-netaddr
- pip install -U pip
- apt-get purge python-pip
- cd /home/th; git clone http://github.com/stackforge/turbo-hipster
- apply any patches you need
- python setup.py install
- cp turbo_hipster/task_plugins/gate_real_db_upgrade/*.sh /usr/local/lib/python2.7/dist-packages/turbo_hipster/task_plugins/gate_real_db_upgrade/
- cp -R etc/* /etc/
- mkdir /var/lib/turbo-hipster
- chown -R th.th /var/lib/turbo-hipster
- mkdir /var/log/turbo-hipster
- chown -R th.th /var/log/turbo-hipster
- install your chosen MySQL-like database engine (percona, maria, mysql)
- mysql -u root --password=$1 -e "create user 'nova'@'localhost' identified by 'tester';"
- mysql -u root --password=$1 -e "create user 'nova'@'172.16.0.2' identified by 'tester';"
- mysql -u root --password=$1 -e "grant all privileges on . to 'nova'@'localhost' with grant option;"
- mysql -u root --password=$1 -e "grant all privileges on . to 'nova'@'172.16.0.2' with grant option;"
- /etc/rc.local
- rsync the datasets from the master
- logrotate -f /etc/logrotate.conf
- chmod -R ugo+r /var/log/*
- chmod ugo+rx /var/log/mysql
- mkdir /var/cache/pip
- chmod -R ugo+rwx /var/cache/pip
- touch /var/log/mysql/slow-queries.log
- /etc/init.d/turbo-hipster start