The opendev project has been moving away from puppet and this is one of
the puppet modules that is no longer used. To simplify things for us we
are taking the extra step of retiring this repo.
Change-Id: Ibbd66faf35447c5e8587c6996cd5011fcede688a
These stats.timers make up the bulk of our graphite data and are filling
the disk of the server. Reduce retention for them specifically to cut
back on disk demands.
Change-Id: Iba85c361fd70a8511c6fbb9da5123c650b11dec1
The HTTP Strict Transport Security (HSTS) mechanism defined in IETF
RFC 6797 allows us to indicate to clients that the site to which
they are connecting should only every be reached over an encrypted
HTTPS connection, in an effort to thwart protocol downgrade attacks
which could convince a client to fall back to plaintext HTTP. Set
such a policy header for the SSL vhost, valid for one year, and
indicate that this policy also applies to any subdomains of the
hostname with which the site is served (even though it's unlikely
that there would ever be any in this case, this is useful for
consistency with inclusion in other vhost templates in the future).
While HSTS policy can't prevent downgrade attacks the very first
time a client connects to this site, thereafter their browser would
be wary of connecting over plain HTTP for subsequent connections for
a full year.
Change-Id: If5c2f3b70e7f7646bf6168e8942aee0ecb7c2ec8
I think I chose a bad example to cargo-cult copy from in
Ic133e3abc09343541210c061af544f7b37480f27; the variables are not being
found. Use scope.lookupvar() (the once place where I did use this is
working).
Change-Id: If3af4eb6a7d29ddde16f08cb7f55e64ad058d12d
If passed the new ssl file parameters, configure and install a SSL
version of the apache config.
For graphite.opendev.org it is intended to use the letsencrypt
certificates provisioned by the base ansible run for this.
Change-Id: Ic133e3abc09343541210c061af544f7b37480f27
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.
This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.
This update should result in no functional change.
For more information see the thread at
http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html
Change-Id: I4e4e5811438aae3ea57917ec1843c50c97c149b5
Django 1.8 in Xenial hits changes to the app loading made in 1.7 [1].
On Xenail and above, ship a slightly different wsgi loader that uses
the new-style loading.
[1] https://docs.djangoproject.com/en/dev/releases/1.7/#app-loading-changes
Change-Id: Ia741af25391603fe81c0185b586b5841910f714a
When bootstrapping a new server it's noticed that pip wasn't installed
before trying to use it. Add it as a dependency of the pip installs.
It was installed by the time the whisper install ran, but there no gcc
required for some of the dependencies. Add build-essential.
Change-Id: I29af4665a1ef7c6f78d9fae29db066ea64743d70
An extra comma snuck in with
I1fec03b32441a1962190f9bd8e24e8bec2318ab5; also use non-printing
statements to tidy it up a bit.
Change-Id: I955eef2ca658d5375ed2e38279ecd58db1ed4968
By default we're not listening on an ipv6 address; so any stats sent
there just go missing. Set up by default to listen to '::' (and add
required ipv6 argument) which, thanks to dual-stack, means ipv4 & 6
packets are all accounted for.
Change-Id: I1fec03b32441a1962190f9bd8e24e8bec2318ab5
The logic in the Gemfile was relying on Zuulv2 variables to find out
whether the spec helper gem was already available on disk, and since
Zuulv3 has changed things it was failing to find it and downloading the
master version instead. This patch ensures the Gemfile looks for the gem
in the right place when running in CI.
Change-Id: Ifd7d38b683aeacd85906ebd26aba2db5c4d1bfac
Turns out this is a bit of a pain, and we found out via a full disk.
Firstly, carbon wants to rotate it's own log files, but doesn't
compress or cleanup. Disable this with ENABLE_LOGROTATION=False,
which has been around for a while. This is copied from the upstream
example config.
Secondly, we were missing the console.log file. Change the rotate to
just one for "*.log", and also add "sharedscripts" so the post-rotate
is only run once per rotation of all files.
Thirdly, copytruncate doesn't work [2]. It needs to be "nocreate";
the file is moved and carbon starts a new one (as suggested by the
config comment). This is updated in the combined rotate section.
[1] https://github.com/graphite-project/carbon/pull/68
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=733856
Change-Id: I946bb728971db79a2f4d08f2787c0768b958fb53
Instead of keeping a local copy of spec_helper_acceptance.rb and
requiring updates to all modules for any change, we can move it into the
common helper gem and require it from there. This will make it easier to
create and review changes that affect all puppet modules. Also change
the Gemfile to look for the gem in the local workspace if running in a
zuul environment.
Change-Id: I2c1adf57f6d025d6e7fe9bc1dcb721b9961aac5f
Since the beaker jobs are being run on xenial, we need a special nodeset
for it, otherwise beaker gives an error:
beaker-hostgenerator was not able to use this value as input.
Exiting with an Error.
We also want to install puppet from the Ubuntu repos rather than from
puppetlabs, since puppetlabs doesn't support puppet 3 for Xenial. For
centos we can keep the install process the same.
Also run django.setup() since django >=1.7 now requires it[1].
Finally, correct the cron check string in the spec tests.
[1] http://django.readthedocs.io/en/latest/releases/1.7.html#backwards-incompatible-changes-in-1-7
Change-Id: Ifd2244ae9dd212b2475f9cd6adb994bc058a4769
Depends-On: I02729bc2d49f10a37e9314632b229fbbe72d0b0b
Bindep is a tool for checking the presence of binary packages needed
to use an application / library. It started life as a way to make it
easier to set up a development environment for OpenStack projects.
Change-Id: I02729bc2d49f10a37e9314632b229fbbe72d0b0b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Order and intendation of those parameters are changed
to follow Puppet Style Guide recommendation [0].
Moreover, it will allow to an user to find much faster
a variable in a list of variables.
[0]. https://docs.puppetlabs.com/guides/style_guide.html
Change-Id: I80c9ab9dc4b305d6ad800cc58cc94e9648b940b0
When running in apache 2.4, the following sort of error was observed
when trying to access the graphite Web app:
AH01630: client denied by server configuration:
/var/lib/graphite/webapp/content/js/composer.js, referer: httpcomposer/?
This change conditionally adds the appropriate permissions to allow
static web content to be served.
Change-Id: I7826a2fdb1b47d39eb7a4a43c6896a16fdbb32c8
The python-django package for Ubuntu 14.04 is installing version 1.6.1
whereas the graphite version running on the existing precise servers only
works with django<1.3. This update conditionally sets the graphite
version to 0.9.15 which requires at least django 1.4.
Change-Id: I348f15ab200b57d3cd9018eb5ba9ebd9e68d56d5
The directory /var/lib/graphite/webapp/content is not created in the manifest,
requiring it to be copied manually from /opt/graphite-web/webapp/content when
setting up a new graphite node.
Change-Id: I385e22b9b4d4ddc0afa48b1e4e2693f8e5e97fe1
In the default configuration, statsd emits 0 values for every known
counter, timer, and guage for every interval. In our system, we
have a lot of sparsely updated timers and counters -- those
associated with a job -- so the longer statsd runs, the more 0
metrics it constantly emits.
Many of these metrics are null much of the time anyway -- each time
statsd restarts it forgets all of these metrics and so the databases
will have nulls until it sees a metric anyway.
This change tells statsd not to send 0 values for timecs and counters
that have not reported during the update interval.
Guages will still report their last value.
Change-Id: I87c85f82f6d38506977bc9bf26d34f6e66746b01
When graphite aggregates data from a high resolution database into
a lower resolution database, this value determines how many samples
must be present in order for data to be aggregated. A value of
0.3 means 30% of the possible high-resolution values must be present
in order for data to be aggregated; if less than 30% are present,
then a null value will be stored instead.
Because so much of our data, particularly related to individual jobs,
providers, etc, are so sporadic, set this value to 0 globally. This
means that all min, max, and average values will be available in all
databases.
Change-Id: I5f416e798e7abedfde776c9571b6fc8cea5f3a33
If we configure pip to install data to our real data location, then
pip helpfully tries to back it up before doing an operation. Problem
is, pip isn't actually installing data to that location, and we have
a lot of data. Misdirect pip so that it thinks an empty dir is what
it should manage.
Change-Id: I28cf2b7ce69023b7d30b9faf2a88185905ecbfee
Add acceptance tests for puppet-graphite module so that once the module
is applied we check if files were created, packages were installed and
services were started.
Change-Id: I54cf2e8e0cc484a8212d234efbf1d67df87583fd
Co-Authored-By: Danilo Ramalho <dramalho@thoughtworks.com>
Use same target directory for zuul-cloner and
the regular git command.
Change-Id: I542a8bd9bffb716a9bc18d5d886eefd52f29a1c7
Co-Authored-By: Fabien Boucher <fabien.boucher@enovance.com>
In anticipation of puppet 4, start trying to deal with puppet 4 things
that can be helpfully predicted by puppet lint plugins. Also fix lint
errors caught by the puppet-lint-absolute_classname-check gem.
Change-Id: I40b3eb70669133414681ab9c9a4272d46c4e83e2
Values for the graphite_admin_username, graphite_admin_email, and
graphite_admin_password parameters must be provided by the user. The
empty string is not a sane default and will cause broken deployments if
used. This change makes these parameters required. Infra is providing
these parameters in system-config so this change will not break Infra.
If downstream users are not providing these parameters their
deployments will not be functional, so this change should be safe for
users already using this class correctly.
Change-Id: I7aed54bb09e54dc8c0827188adf971b76649f94d
If mod_wsgi is not installed, the puppet-graphite module will fail to
start apache because of the WSGI commands in the vhost template. This
patch adds the httpd::mod::wsgi class to take care of it. There is
nothing else in the graphite.openstack.org site.pp node that installs
mod_wsgi so the package resource in the class should not conflict with
anything.
Change-Id: Iaba16663c99a9a3aa6b69ac9b5af6dba22cd3fd2
puppet-httpd is the openstack-infra version of puppetlabs-apache
(0.0.4) release.
This patchset will remove the puppetlabs-apache namespace from -infra
allowing for possible future patchsets to use newer puppetlabs-apache
modules.
Change-Id: Iba588ffb565e3c27615f444efeb5a3586cafbd82
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
carbon creates 3 different logs that it appends to. We should log rotate
them, so that they don't take up all the disk. Also, the carbon-cache-a
dir needs to be writable by www-data so that carbon can write the logs
in the first place.
Change-Id: I804ead080fa72053ecc6a46d962c7d30b2a165fb
The http://ci.openstack.org/ documentation site has been deprecated,
replaced by redirects to corresponding paths within
http://docs.openstack.org/infra/ where other Project Infrastructure
documentation already resides.
Change-Id: I7269339391c3735322c6a61561c79086b75da8c8
Because we are adding CORS authorization to the header, we need to
ensure the apache 'headers' module is enabled.
Change-Id: I5ed0c2c413cb6ea1408d76e094ff897b8ec6a1c2
Graphite needs to enable simple CORS requests so that third-party
services that want to pull data via the API to build things like
interactive graphs and visualization dashboards can do that.
Change-Id: I34a422c787118425aa484fa980aadd47cbf197e2