Commit Graph

16 Commits

Author SHA1 Message Date
Clark Boylan 678e461ca7 flock lockfile path comes before the command to lock
When I added the timeout I didn't move the lock file path properly. Fix
this by putting the lock file path before the timeout command to flock.

Change-Id: Iacf8d55f9cfe5b2c92569929bbdadebbe2f8de09
2019-03-20 15:16:49 -07:00
Clark Boylan 014bc3b72e Timeout generation of e-r graphs
Add a four hour timeout to the generation of e-r graphs as these scripts
occasiaonlly run for days without exiting. We'd rather fail then try
again later to ensure the graphs are mostly up to date.

Change-Id: I7ee1c94c41edb9d6b5aeedebac64aeded1cf0e97
2019-03-20 11:32:01 -07:00
Jens Harbott a1f5230d2b Talk about Zuul instead of jenkins
This was already changed in elastic-recheck in [0], so let's also use
this in our running config.

[0] https://review.openstack.org/330684

Change-Id: If9911f1ad2fa45afd6cfde1216f6178e87891028
2018-05-09 11:30:19 +00:00
Ihar Hrachyshka f43cc3d760 Enable recheck irc bot for neutron
Change-Id: I93057432e4e68e69e4fe047851c875ea2b5d280e
2017-02-15 20:17:42 +00:00
Sean Dague c9cd39aaac put a 1hr timeout on flock blocking
From time to time the flock calls on the er system back up for an
unknown reason. However, when fixing another bug I found that flock
has a wait timeout. Set this to an hour so that if we fail to flock
after an hour we don't back up forever.

Change-Id: Ie574b994d2b5cc38f559d3d7f8dd980148cf0890
2015-11-17 12:14:32 -05:00
Ramy Asselin f8b1c57f9a Allow elastic-recheck daemon args to be specified
For example, allow passing in --noirc or --nocomment.

Change-Id: I90d3e3bf9dfce70b2de379edee4acfbb68f2ec09
2015-10-16 18:29:26 +00:00
Jeremy Stanley c80b87ca5c Workflow documentation is now in infra-manual
Replace URLs for workflow documentation to appropriate parts of the
OpenStack Project Infrastructure Manual.

Change-Id: I8aa32f5617a0331c116a703c45ae7f644f5b409a
2015-01-08 20:31:20 +00:00
Jeremy Stanley 49c28acc8a Stop the elastic-recheck bot more gracefully
* modules/elastic_recheck/files/elastic-recheck.init: It has in the
past been observed that the elastic-recheck IRC/Gerrit-commenting
bot does not always terminate cleanly. Use a retry with escalation
of kill signals in the initscript to help prevent additional old
processes from hanging around when restarts take place.

Change-Id: I9498eeeb935cb609c2c4c099c3552bed337fbe99
2014-10-03 17:23:44 +00:00
Sean Dague 294bd6865d convert to two level recheckwatchbot.yaml
As of I7353af4c3f141d4bd617d6fd388d7957e0586ba8 we'll introduce
the ability to have a 2 layer config which includes the messages
that we'll send back to the user on gerrit outside of the
elastic-recheck code base. This makes it easier to make changes
to the UX consistent with what we want to change with the rest
of the config.

Change-Id: I9208123a4cb3be02c272cd8a6eba460f4130a960
Depends-On: I7353af4c3f141d4bd617d6fd388d7957e0586ba8
2014-07-22 10:50:51 -04:00
James E. Blair 4f24fbee80 Log recheckwatchbot messages
The qualname was wrong for this logger.

Change-Id: Idc20113fcb96af8b979d1b6552155f2a1e8c5c66
2014-03-03 08:08:18 -08:00
Matthew Treinish 0013a700fb Add projects section to elastic recheck bot yaml
The elastic-recheck change: Ic3dd76bad94213c7152c29a99c00ed23a2c01a31
will add a new section to the recheckwatchbot yaml file to specify
which bug targets get reported to which channels. This commit preempts
that change by adding the new section to the -qa channel to preserve
the existing functionality.

Change-Id: I452040883027b260c0480b9d7ac2e2ca476b72b3
2014-01-23 14:52:39 -05:00
Sean Dague ff2b7a4ba9 add in elastic-recheck-unclassified report
this adds the new elastic-recheck-unclassified report, which
generates an html page of all the unclassified gate failures.

it also runs this command, and the elastic-recheck-graph command
not only on cron, but also on updates to the repository. In order
to do this safely, we create a new er_safe_run.sh in /usr/local
that provides a flock front end to ensure we won't race on our
html writes.

Change-Id: I624f2629e2ac19a1394866f73f1aa9ba18e968ce
2014-01-21 18:29:30 -05:00
James E. Blair 838368798f Increase verbosity in elastic-recheck debug log
Change-Id: I5c326d821cba2e4c66b7eaf0fdfeba43ae7cd238
2013-09-30 15:13:53 -07:00
Jeremy Stanley 53165e490b Turn on logging for elastic-recheck
* modules/elastic_recheck/files/elastic-recheck.init: Minor
whitespace cleanup for space-before-tab indent on a comment line.

* modules/elastic_recheck/files/logging.config: New logging
configuration file for elastic-recheck IRC bot.

* modules/elastic_recheck/manifests/init.pp: Install the new logging
configuration file.

* modules/elastic_recheck/templates/elastic-recheck.conf.erb: Tell
the IRC bot to use the new logging configuration file.

Change-Id: I47cabb01df209c9f020846bcb272b9a2f796b0e5
2013-09-30 14:27:21 -07:00
Jeremy Stanley 9caae512f2 Correct elastic-recheck Puppet dependencies
* modules/elastic_recheck/files/elastic-recheck.init: Update the
daemon invocation now that it has an executable entrypoints wrapper.

* modules/elastic_recheck/manifests/init.pp: Correct the Puppet
dependency tree sufficiently to be able to apply the manifest
successfully.

Change-Id: Ib018564e66c29d73b30aaf50c9306dc6d67ba2b2
2013-09-30 20:22:51 +00:00
Clark Boylan 9f6762830c Deploy elastic-recheck on logstash.openstack.org.
This elastic recheck thing that classifies failures based on logstash
queries is awesome. We should run it on the OpenStack developer
infrastructure. Deploy it on logstash.o.o to keep related services
together.

Change-Id: I16ae7a32c2c48474454df17931779d0f60c3ed85
2013-09-30 18:56:23 +00:00