In a recent elastic-recheck change we updated the output of the
uncategorized command to split between the integrated gate failues
and everything else. In the future we'll add a configurable views
system where we can construct multiple different groups. The days
of a single uncategorized.html file have disappeared. This commit
adds the necessary support to the cron job so it can handle an
multiple number of html files.
Change-Id: I12690cecd3b5d27fa3e508f7ea4daa43696423b8
Order and intendation of those parameters are changed
to follow Puppet Style Guide recommendation [0].
Moreover, it will allow to an user to find much faster
a variable in a list of variables.
[0]. https://docs.puppetlabs.com/guides/style_guide.html
Change-Id: I36851bcdbef1157fbc339834a1cc8dc8344397ce
Elastic recheck commands pull the logstash url
from the provided configuration file. Allow this
url to be set.
Change-Id: I5fd2bc05f1cd6647fa43aaa307c47935c64b5f36
Depends-on: I282745796a40f10955e0c9893e817779b2d4d55a
pyelasticseach>=1.0 defaults the port to 9200 if not specified.
Fix it to port 80, so that when pyelasticsearch is upgraded, the
correct port (80) will still be used.
Change-Id: I50107e90449f8d78b70985e6f2b808a951310eb3
Allow custom command line options for the elastic-recheck-uncategorized,
graph_all_cmd, and graph_gate_cmd.
This can include other current or future options e.g. --db_uri --es_url.
Remove these commands from init.pp since they are only used by the cron.pp
class (since Ibf5dad1071f65f8d676e8b9a0ec4331bada9943f).
Change-Id: I4588ea4e40ae643070f8da4954cca984ec6a3d96
In anticipation of puppet 4, start trying to deal with puppet 4 things
that can be helpfully predicted by puppet lint plugins. This patch also
corrects lint errors caught by the puppet-lint-absolute_classname-check
and puppet-lint-empty_string-check gems as well as arrow alignment
which wasn't being caught under the system version of puppet-lint.
Change-Id: I5552777ba63d07039f9b2bed9c4100a44e6a0255
Puppet does not like cron list syntax to be made as a normal string.
Instead you must give puppet cron resources a puppet list that it will
convert to a cron list...
Change-Id: Ia0839be9b7d8ae772065f782adf61f831e75b92e
Decrease the time between e-r batch jobs to half an hour between
consecutive runs of the same job to reduce the latency on the updates of
these reports. Offset each of the three batch jobs from each other by
ten minutes. The run times look like this now:
minute: 0 10 20 30 40 50
job: 1 2 3 1 2 3
Change-Id: Ic4176b019e6da71e8c1e33ca89f53b6700309a91
Currently elastic-recheck is hammering the ElasticSearch cluster every
15 minutes, basically bringing everything else to a halt while these
queries are processed. Spread elastic-recheck periodic queries out, run
each once an hour and 20 minutes after the previous set of queries.
Also, remove the auto triggering when files change, instead we wait for
the hour to roll over and give us new data.
Change-Id: Ibf5dad1071f65f8d676e8b9a0ec4331bada9943f
We want to display the hits for all queues and for just the gate queue.
A parameter was added to elastic-recheck-graph to support this, and a
second patch updates the web page to give the gate queue only option.
This patch updates the usage of elastic-recheck-gate.
Related elastic-recheck patchs:
* I9217a2ceedf86ffe04851084df78238384fccd51
* I23ed8d945c9517221622e1ca215c80539bb07bbd
Change-Id: Iac51aeaf152f50f8b9a064cd249b0920bef824df
puppet exec doesn't run under a real shell, so running cd doesn't
work because it's not a command. Instead use cwd properly to
actually run these commands.
Also actually call the correct command to generate the uncategorized
jobs page.
Change-Id: I84ed19bbdc16d185316e3967efbd798e06a521d1
this adds the new elastic-recheck-unclassified report, which
generates an html page of all the unclassified gate failures.
it also runs this command, and the elastic-recheck-graph command
not only on cron, but also on updates to the repository. In order
to do this safely, we create a new er_safe_run.sh in /usr/local
that provides a flock front end to ensure we won't race on our
html writes.
Change-Id: I624f2629e2ac19a1394866f73f1aa9ba18e968ce
Also, normalize some parameters to the status manifest to make it
more readable (since they follow the same pattern). And make the
ssh config for elastic-recheck bot match reviewday.
Change-Id: I2417f121e7b3685aab9540504cdd4c6db1754e67
* modules/elastic_recheck/manifests/bot.pp: The subscribe on the
install_elastic-recheck exec in ff1e7bc2 specified an explicit
scope, but exec resources are not scoped. Fix it.
Change-Id: I16ee575a24c9abe1db13f4c2164f8c49fa95980f
The installation of elastic-recheck is done on multiple machines but the
irc bot is only run on one machine. Therefore don't notify the bot
service from the install exec instead subscribe to the exec from the
service.
Change-Id: I11f3d32e40408a02d948d0533f5a4c45c983adae
this provides the changes needed to handle the new elastic-recheck
web console installed from the elastic-recheck package in the
share tree.
factor out bot code to elastic_recheck::bot
this change moves elastic_recheck bot starting code out into
a separate module, which lets us include all the elastic_recheck
base module into the static site config.
Change-Id: If53523754494a746c152c5d1384db5426b096fc1
this provides the changes needed to handle the new elastic-recheck
web console installed from the elastic-recheck package in the
share tree.
Change-Id: I8c8b76cb53b2f2eea026eb0f321e48903df71636
in I86317fcf6f1886ab5b6c0ee154b29e71865c52b7 we want to move to
a directory of queries instead of a single file. In order to
support this we need to support linking in that directory as well.
This needs to land prior to that change.
Change-Id: I3af42cff1050754c6844db994004fd203063bb3d
The pip vs. easy_install interactions continually get the systems
into a weird state. pip install $path_to_repo, on the other hand,
works like a charm.
Left out two graphite installs because they are doing path manipulation
and I want to do further testing before touching that.
Change-Id: I373b29aca03f1ebd20e533ebaaf5de9ef2db017e
this ensures we notify the er daemon to restart after we've updated
the code, making it so we deploy instantly into the environment.
Change-Id: Ifa266c7c0248ce9d0431f44e76de87958503f401
* manifests/site.pp: Expect the SSH key in /home rather than in
/var/run.
* modules/elastic_recheck/manifests/init.pp: Change the recheck
shell account for elastic-recheck to use a homedir in /home.
Change-Id: I43f4a8cdcb803328388b7e92732e5cb9d6acb2a8
* modules/elastic_recheck/files/elastic-recheck.init: Minor
whitespace cleanup for space-before-tab indent on a comment line.
* modules/elastic_recheck/files/logging.config: New logging
configuration file for elastic-recheck IRC bot.
* modules/elastic_recheck/manifests/init.pp: Install the new logging
configuration file.
* modules/elastic_recheck/templates/elastic-recheck.conf.erb: Tell
the IRC bot to use the new logging configuration file.
Change-Id: I47cabb01df209c9f020846bcb272b9a2f796b0e5
* modules/elastic_recheck/files/elastic-recheck.init: Update the
daemon invocation now that it has an executable entrypoints wrapper.
* modules/elastic_recheck/manifests/init.pp: Correct the Puppet
dependency tree sufficiently to be able to apply the manifest
successfully.
Change-Id: Ib018564e66c29d73b30aaf50c9306dc6d67ba2b2
This elastic recheck thing that classifies failures based on logstash
queries is awesome. We should run it on the OpenStack developer
infrastructure. Deploy it on logstash.o.o to keep related services
together.
Change-Id: I16ae7a32c2c48474454df17931779d0f60c3ed85
Sorin Sbarnea