The exec to install subunit2sql tries to compare the output of pip list
versus pip search to determine whether it is on the latest version.
Without this patch, the pip list command produces a string that has too
many spaces on the end and therefore doesn't match the result of the pip
search command. This means that it will always try to reinstall
subunit2sql even when it doesn't need to, which breaks idempotency
tests. This patch fixes the regex group to exclude trailing spaces.
Change-Id: I92f614db2da936d68d8f2305b348c65ba773b01c
The os-performance-tools library currently caps its statsd dependency at
<3.0[1]. If we try to install latest, puppet will succeed at installing
it in the first place, but upon the next run it will see that it is not
using the latest and try to upgrade and fail due to the cap. Add the pin
so that it doesn't try and fail to install something it can't.
[1] http://git.openstack.org/cgit/openstack/os-performance-tools/tree/requirements.txt?id=d96d5145fa5facdfd7a4e5ea2811948f0efd465e#n7
Change-Id: I37303d3b23cf5c387511292a8e128faa1f9df9ae
PyYAML is already a dependency of subunit2sql, so the distro package
doesn't need to be installed explicitly. If it is installed by the
distro, then on a system with pip 10 the subunit2sql installation will
fail.
Change-Id: Ie7be5cf01fcf9a630c7bf282549a862515c159d9
When testing to see whether the installed subunit2sql package
matches the version available on PyPI, the first argument can be
empty if the package has not yet been installed. Make sure these
arguments are quoted so that the test command gets a syntax it can
parse (rather than a string starting with a binary operator).
Change-Id: Ia2e7165f851085328e00431a739cce17d2e494b2
In at least some circumstances, our pip is installed into
/usr/local/bin to make sure that's added to the environment and
switch to dynamic path resolution when calling pip.
Change-Id: I4e53ce2aba1b8335025e180bd03a5f5e93461814
When installing or upgrading subunit2sql, override pip's default
upgrade strategy to avoid updating preinstalled dependencies if they
already satisfy the requirements. This in particular avoids
attempting to replace the distro package of python-netifaces by
rebuilding a newer sdist (it is not distributed as a wheel on PyPI
and needs C extensions compiled, which in turn have a lot of
build-time deps).
This implementation is copied from the puppet-zuul module.
Change-Id: I885a61527ca9938200fb965ea97ff957fed3c480
In order to make sure pip installation of subunit2sql does not
attempt to pip install (and subsequently build from sdist)
netifaces, set a dependency ensuring the distro package for
python-netifaces is installed before that.
Change-Id: I01118c2771aa4e4fa553fb04b4a0885bf977a738
Since subunit2sql requires oslo.db which requires oslo.utils which
requires netifaces which builds C extensions and does not provide
prebuilt wheels for any platform other than Windows, install
Ubuntu's python-netifaces package to avoid needing an entire build
toolchain. We're moving the servers to Xenial, which provides a
version compatible with the current oslo.utils requirements list.
Change-Id: I4c701f08cf11bc068f6ebb10534f7e6f865970bb
The paho-mqtt package is only ever needed on the worker because that is
what emits mqtt. However by putting it in init this also gets installed
on the server node. Additionally this commit adds a duplication check
around the package because it's possible that something else in another
module will conflict with paho-mqtt as we start adding mqtt to more
services.
Change-Id: Id0308d2d4d1843fcca73f459cffa2ae944bebd0c
This commit adds support to the gearman worker for publishing an mqtt
message when processing a gearman job succeeds or fails. By default this
is disabled since it requires extra configuration to tell the worker how
to talk to the mqtt broker.
Right now the payload of the message is just the build_uuid and whether
it was written to the db or not. Eventually some details about the
subunit2sql db entry will be added to the payload. But this requires
changes to either subunit2sql or how the worker calls the subunit2sql
api before it is feasible.
Change-Id: Ibd13b737eccf52863a69d20843cb7d50242f7bb9
This will register itself as a plugin and use the statsd configuration
options added to send any counters seen in attachments to statsd.
Change-Id: If93c84781fde46852d65bac88ab500563f7978cc
In anticipation of puppet 4, start trying to deal with puppet 4 things
that can be helpfully predicted by puppet lint plugins. Also fix lint
errors caught by the puppet-lint-absolute_classname-check gem as well
as arrow alignment errors caught by the now bundle-installed
puppet-lint.
Change-Id: I6975c605e238046b3812b2fea20e65db8c890e42
This commit fixes the path for the gearman-worker script so it can
actually find the file. The script is located in the subunit2sql
module and not the log_processor module.
Change-Id: Ie3228d33ed3c8bdf50b6b2bdc46770fb38742b69
This commit moves all the pieces from worker.pp which are used by all
the worker process into init.pp. This was not done before because the
package definitions would conflict with those in logstash.o.o's
manifest. However with the if not defined checks we can have them in
init.pp, so let's move everything in there.
Change-Id: I5048604ce64bb52f463397c6a37b4a19a0bab8bb
This adds a new gearman worker to process the subunit files from
the gate job runs. It will use subunit2sql to connect to a sql
server and process the data from the subunit file. The
log-gearman-client is modified to allow for pushing subunit jobs
to gearman, and the worker model for processsing logs is borrowed
to process the subunit files.
Change-Id: I83103eb6afc22d91f916583c36c0e956c23a64b3
Clark Boylan