Commit Graph

72 Commits

Author SHA1 Message Date
Clark Boylan 8f55da407f Retire this repo
The opendev project has been moving away from puppet and this is one of
the puppet modules that is no longer used. To simplify things for us we
are taking the extra step of retiring this repo.

Change-Id: Ibbd66faf35447c5e8587c6996cd5011fcede688a
2022-02-17 11:42:09 -08:00
Clark Boylan c5003bfd36 Reduce retention of stats.timers
These stats.timers make up the bulk of our graphite data and are filling
the disk of the server. Reduce retention for them specifically to cut
back on disk demands.

Change-Id: Iba85c361fd70a8511c6fbb9da5123c650b11dec1
2019-10-24 10:56:47 -07:00
Zuul 7ee0c28e0c Merge "Enable RFC 6797 HSTS signaling" 2019-05-21 00:39:09 +00:00
OpenDev Sysadmins 7d55d46618 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:25:37 +00:00
Jeremy Stanley 2658fb41ba Enable RFC 6797 HSTS signaling
The HTTP Strict Transport Security (HSTS) mechanism defined in IETF
RFC 6797 allows us to indicate to clients that the site to which
they are connecting should only every be reached over an encrypted
HTTPS connection, in an effort to thwart protocol downgrade attacks
which could convince a client to fall back to plaintext HTTP. Set
such a policy header for the SSL vhost, valid for one year, and
indicate that this policy also applies to any subdomains of the
hostname with which the site is served (even though it's unlikely
that there would ever be any in this case, this is useful for
consistency with inclusion in other vhost templates in the future).
While HSTS policy can't prevent downgrade attacks the very first
time a client connects to this site, thereafter their browser would
be wary of connecting over plain HTTP for subsequent connections for
a full year.

Change-Id: If5c2f3b70e7f7646bf6168e8942aee0ecb7c2ec8
2019-04-14 14:56:38 +00:00
Ian Wienand 8a0e111327 Fix ssl lookups
I think I chose a bad example to cargo-cult copy from in
Ic133e3abc09343541210c061af544f7b37480f27; the variables are not being
found.  Use scope.lookupvar() (the once place where I did use this is
working).

Change-Id: If3af4eb6a7d29ddde16f08cb7f55e64ad058d12d
2019-04-12 11:37:24 +10:00
Ian Wienand d6368cf248 Add a httpd SSL template
If passed the new ssl file parameters, configure and install a SSL
version of the apache config.

For graphite.opendev.org it is intended to use the letsencrypt
certificates provisioned by the base ansible run for this.

Change-Id: Ic133e3abc09343541210c061af544f7b37480f27
2019-04-12 08:00:16 +10:00
Ian Wienand 8bd634047e Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I4e4e5811438aae3ea57917ec1843c50c97c149b5
2019-03-24 20:35:39 +00:00
Ian Wienand 78494074a8 Fix django wsgi typo
Introduced with Ia741af25391603fe81c0185b586b5841910f714a

Change-Id: I2c0f1e37edcb806556c6f50998e95abdece6f999
2019-03-01 12:33:55 +11:00
Ian Wienand 977f065a3f graphite.wsgi : update for django 1.7
Django 1.8 in Xenial hits changes to the app loading made in 1.7 [1].
On Xenail and above, ship a slightly different wsgi loader that uses
the new-style loading.

[1] https://docs.djangoproject.com/en/dev/releases/1.7/#app-loading-changes

Change-Id: Ia741af25391603fe81c0185b586b5841910f714a
2019-03-01 09:19:32 +11:00
Ian Wienand 2aa06a80e1 Fix pip ordering and build requirements
When bootstrapping a new server it's noticed that pip wasn't installed
before trying to use it.  Add it as a dependency of the pip installs.

It was installed by the time the whisper install ran, but there no gcc
required for some of the dependencies.  Add build-essential.

Change-Id: I29af4665a1ef7c6f78d9fae29db066ea64743d70
2019-02-28 15:07:54 +11:00
Ian Wienand ae9a92e707 Fix config for ipv6
An extra comma snuck in with
I1fec03b32441a1962190f9bd8e24e8bec2318ab5; also use non-printing
statements to tidy it up a bit.

Change-Id: I955eef2ca658d5375ed2e38279ecd58db1ed4968
2018-09-25 15:33:28 +10:00
Ian Wienand fb26d09101 Have statsd listen on ipv6 by default
By default we're not listening on an ipv6 address; so any stats sent
there just go missing.  Set up by default to listen to '::' (and add
required ipv6 argument) which, thanks to dual-stack, means ipv4 & 6
packets are all accounted for.

Change-Id: I1fec03b32441a1962190f9bd8e24e8bec2318ab5
2018-09-14 14:29:45 +10:00
Colleen Murphy 7ed3967361 Update Gemfile for Zuulv3
The logic in the Gemfile was relying on Zuulv2 variables to find out
whether the spec helper gem was already available on disk, and since
Zuulv3 has changed things it was failing to find it and downloading the
master version instead. This patch ensures the Gemfile looks for the gem
in the right place when running in CI.

Change-Id: Ifd7d38b683aeacd85906ebd26aba2db5c4d1bfac
2018-07-12 09:57:44 +02:00
Ian Wienand dfc94e4fd9 Fix up log rotation
Turns out this is a bit of a pain, and we found out via a full disk.

Firstly, carbon wants to rotate it's own log files, but doesn't
compress or cleanup.  Disable this with ENABLE_LOGROTATION=False,
which has been around for a while.  This is copied from the upstream
example config.

Secondly, we were missing the console.log file.  Change the rotate to
just one for "*.log", and also add "sharedscripts" so the post-rotate
is only run once per rotation of all files.

Thirdly, copytruncate doesn't work [2].  It needs to be "nocreate";
the file is moved and carbon starts a new one (as suggested by the
config comment).  This is updated in the combined rotate section.

[1] https://github.com/graphite-project/carbon/pull/68
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=733856

Change-Id: I946bb728971db79a2f4d08f2787c0768b958fb53
2018-02-07 14:15:59 +11:00
Colleen Murphy 4a30897c05 Depend on helper gem for spec_helper_acceptance
Instead of keeping a local copy of spec_helper_acceptance.rb and
requiring updates to all modules for any change, we can move it into the
common helper gem and require it from there. This will make it easier to
create and review changes that affect all puppet modules. Also change
the Gemfile to look for the gem in the local workspace if running in a
zuul environment.

Change-Id: I2c1adf57f6d025d6e7fe9bc1dcb721b9961aac5f
2017-08-18 10:41:43 +02:00
Jenkins 4799d1a472 Merge "Fix beaker on xenial" 2017-06-14 23:57:12 +00:00
Colleen Murphy 17bff7d7b4 Fix beaker on xenial
Since the beaker jobs are being run on xenial, we need a special nodeset
for it, otherwise beaker gives an error:

 beaker-hostgenerator was not able to use this value as input.
 Exiting with an Error.

We also want to install puppet from the Ubuntu repos rather than from
puppetlabs, since puppetlabs doesn't support puppet 3 for Xenial. For
centos we can keep the install process the same.

Also run django.setup() since django >=1.7 now requires it[1].

Finally, correct the cron check string in the spec tests.

[1] http://django.readthedocs.io/en/latest/releases/1.7.html#backwards-incompatible-changes-in-1-7

Change-Id: Ifd2244ae9dd212b2475f9cd6adb994bc058a4769
Depends-On: I02729bc2d49f10a37e9314632b229fbbe72d0b0b
2017-06-10 20:09:16 +02:00
Paul Belanger 7a5c496d48
Add bindep.txt file
Bindep is a tool for checking the presence of binary packages needed
to use an application / library. It started life as a way to make it
easier to set up a development environment for OpenStack projects.

Change-Id: I02729bc2d49f10a37e9314632b229fbbe72d0b0b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-05-01 21:09:09 -04:00
Spencer Krum 16dbc67667 Use new infra_spec_helper for gem dependencies
Change-Id: Iba49b989f65e9be3a4b537c09e5f3b1bc2cdccae
2016-06-21 18:38:25 -07:00
Spencer Krum 2103aa22fd Pin google-api-client; sanitize Gemfile
Change-Id: I64cb74ff6a3a113d536f092ff69b616bc9b576da
2016-04-14 15:19:16 -07:00
Andrey Nikitin 4bf656f4a8 Order of the class parameters is refactored
Order and intendation of those parameters are changed
to follow Puppet Style Guide recommendation [0].
Moreover, it will allow to an user to find much faster
a variable in a list of variables.

[0]. https://docs.puppetlabs.com/guides/style_guide.html

Change-Id: I80c9ab9dc4b305d6ad800cc58cc94e9648b940b0
2016-03-21 11:57:03 +03:00
John Warren 1cbe0c8fa6 Add python-tz package
Version 0.9.15 requires the python-tz package.

Change-Id: I1bd04aceb0c7b5a5ce269dcd0d857c229ff6bf27
2015-12-07 11:35:59 -05:00
John Warren 2efd63e6e2 Add docroot permissions for apache >= 2.4
When running in apache 2.4, the following sort of error was observed
when trying to access the graphite Web app:

AH01630: client denied by server configuration:
/var/lib/graphite/webapp/content/js/composer.js, referer: httpcomposer/?

This change conditionally adds the appropriate permissions to allow
static web content to be served.

Change-Id: I7826a2fdb1b47d39eb7a4a43c6896a16fdbb32c8
2015-12-07 11:34:48 -05:00
John Warren a25d50cfb7 Add support for django >= 1.4
The python-django package for Ubuntu 14.04 is installing version 1.6.1
whereas the graphite version running on the existing precise servers only
works with django<1.3.  This update conditionally sets the graphite
version to 0.9.15 which requires at least django 1.4.

Change-Id: I348f15ab200b57d3cd9018eb5ba9ebd9e68d56d5
2015-12-03 15:11:05 -05:00
John Warren c9091c5dbd Copy content to vhost document root
The directory /var/lib/graphite/webapp/content is not created in the manifest,
requiring it to be copied manually from /opt/graphite-web/webapp/content when
setting up a new graphite node.

Change-Id: I385e22b9b4d4ddc0afa48b1e4e2693f8e5e97fe1
2015-12-01 12:04:30 -05:00
Jenkins d69c62d46f Merge "Add support for apache >= 2.4" 2015-11-25 13:40:36 +00:00
Jenkins d20e40f906 Merge "Prevent possible http_mod headers declaration conflict" 2015-11-23 12:36:07 +00:00
James E. Blair 933b580fd6 Set statsd to delete idle counters/timers
In the default configuration, statsd emits 0 values for every known
counter, timer, and guage for every interval.  In our system, we
have a lot of sparsely updated timers and counters -- those
associated with a job -- so the longer statsd runs, the more 0
metrics it constantly emits.

Many of these metrics are null much of the time anyway -- each time
statsd restarts it forgets all of these metrics and so the databases
will have nulls until it sees a metric anyway.

This change tells statsd not to send 0 values for timecs and counters
that have not reported during the update interval.

Guages will still report their last value.

Change-Id: I87c85f82f6d38506977bc9bf26d34f6e66746b01
2015-11-17 13:36:59 -08:00
James E. Blair 395b5ad315 Set xFilesFactor to 0
When graphite aggregates data from a high resolution database into
a lower resolution database, this value determines how many samples
must be present in order for data to be aggregated.  A value of
0.3 means 30% of the possible high-resolution values must be present
in order for data to be aggregated; if less than 30% are present,
then a null value will be stored instead.

Because so much of our data, particularly related to individual jobs,
providers, etc, are so sporadic, set this value to 0 globally.  This
means that all min, max, and average values will be available in all
databases.

Change-Id: I5f416e798e7abedfde776c9571b6fc8cea5f3a33
2015-11-16 14:05:18 -08:00
John Warren 60cd705940 Prevent possible http_mod headers declaration conflict
Making a minor tweak that allows graphite to be deployed on a node
that also has other resources that need to use the headers http_mod,
for instance jenkins.  See:
http://git.openstack.org/cgit/openstack-infra/puppet-jenkins/tree/manifests/master.pp#n66
for an example where the same approach is used.

Change-Id: Ia7495f7640d356f130381fbb021f1cae05e72c00
2015-11-13 12:02:48 -05:00
Monty Taylor b49441ef1c Install pip data to a non-production location
If we configure pip to install data to our real data location, then
pip helpfully tries to back it up before doing an operation. Problem
is, pip isn't actually installing data to that location, and we have
a lot of data. Misdirect pip so that it thinks an empty dir is what
it should manage.

Change-Id: I28cf2b7ce69023b7d30b9faf2a88185905ecbfee
2015-11-08 09:01:37 -05:00
Yolanda Robla 3ed819160d Add support for apache >= 2.4
Graphite is failing for trusty, because of the
Require All granted need. Add that to the vhost.

Change-Id: I8b0c49929581439a3bce2b04915ee9124ec51acb
2015-11-06 09:32:25 +01:00
Bruno Tavares c9a9d607c2 Add acceptance tests for puppet-graphite.
Add acceptance tests for puppet-graphite module so that once the module
is applied we check if files were created, packages were installed and
services were started.

Change-Id: I54cf2e8e0cc484a8212d234efbf1d67df87583fd
Co-Authored-By: Danilo Ramalho <dramalho@thoughtworks.com>
2015-09-24 19:12:23 -03:00
Spencer Krum 6455398176 Fix target path for regular git clone during tests
Use same target directory for zuul-cloner and
the regular git command.

Change-Id: I542a8bd9bffb716a9bc18d5d886eefd52f29a1c7
Co-Authored-By: Fabien Boucher <fabien.boucher@enovance.com>
2015-08-19 16:36:01 -07:00
Jenkins b44f64e7a0 Merge "Add Gemfile and puppet 4 checks" 2015-08-09 02:58:12 +00:00
Spencer Krum 970926a0f6 Boilerplate beaker-rspec files
Change-Id: I4133133b627ade5df61b94af72ec5846ae3a41f6
2015-07-28 02:04:27 -07:00
Jenkins c7bec6c4b5 Merge "Make username/email/password params required" 2015-07-28 07:48:51 +00:00
Colleen Murphy ce47a1494d Add Gemfile and puppet 4 checks
In anticipation of puppet 4, start trying to deal with puppet 4 things
that can be helpfully predicted by puppet lint plugins. Also fix lint
errors caught by the puppet-lint-absolute_classname-check gem.

Change-Id: I40b3eb70669133414681ab9c9a4272d46c4e83e2
2015-07-23 18:34:36 -07:00
Colleen Murphy 5bc4dbc6c6 Make username/email/password params required
Values for the graphite_admin_username, graphite_admin_email, and
graphite_admin_password parameters must be provided by the user. The
empty string is not a sane default and will cause broken deployments if
used. This change makes these parameters required. Infra is providing
these parameters in system-config so this change will not break Infra.
If downstream users are not providing these parameters their
deployments will not be functional, so this change should be safe for
users already using this class correctly.

Change-Id: I7aed54bb09e54dc8c0827188adf971b76649f94d
2015-07-23 18:34:21 -07:00
Colleen Murphy ea59f65f75 Include mod_wsgi for graphite
If mod_wsgi is not installed, the puppet-graphite module will fail to
start apache because of the WSGI commands in the vhost template. This
patch adds the httpd::mod::wsgi class to take care of it. There is
nothing else in the graphite.openstack.org site.pp node that installs
mod_wsgi so the package resource in the class should not conflict with
anything.

Change-Id: Iaba16663c99a9a3aa6b69ac9b5af6dba22cd3fd2
2015-07-23 18:34:01 -07:00
Paul Belanger fa747a6175 Migrate to puppet-httpd module
puppet-httpd is the openstack-infra version of puppetlabs-apache
(0.0.4) release.

This patchset will remove the puppetlabs-apache namespace from -infra
allowing for possible future patchsets to use newer puppetlabs-apache
modules.

Change-Id: Iba588ffb565e3c27615f444efeb5a3586cafbd82
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2015-07-16 15:45:33 -04:00
Paul Belanger 77d2d1b7ba Fix variable access warnings
Change-Id: Ib7487f61e9835b8b6c57204958aa15b41371c31f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2015-07-09 16:52:01 -04:00
Monty Taylor f4213b2175 Rotate carbon-cache logs
carbon creates 3 different logs that it appends to. We should log rotate
them, so that they don't take up all the disk. Also, the carbon-cache-a
dir needs to be writable by www-data so that carbon can write the logs
in the first place.

Change-Id: I804ead080fa72053ecc6a46d962c7d30b2a165fb
2015-06-15 13:41:37 +00:00
Jeremy Stanley 7fc763ee13 Replace ci.o.o links with docs.o.o/infra
The http://ci.openstack.org/ documentation site has been deprecated,
replaced by redirects to corresponding paths within
http://docs.openstack.org/infra/ where other Project Infrastructure
documentation already resides.

Change-Id: I7269339391c3735322c6a61561c79086b75da8c8
2015-05-14 21:38:17 +00:00
Ramy Asselin 3db569b877 Rename openstackci to openstackinfra
Change-Id: Ieae4587f40befe613465362af020cf5ffd0b94f7
2015-04-20 13:44:06 -07:00
Timothy Chavez 56cc4b39d1 Ensure the 'headers' module is enabled
Because we are adding CORS authorization to the header, we need to
ensure the apache 'headers' module is enabled.

Change-Id: I5ed0c2c413cb6ea1408d76e094ff897b8ec6a1c2
2015-04-03 13:55:44 -05:00
Timothy Chavez fe1b569e5b Enable simple CORS requests
Graphite needs to enable simple CORS requests so that third-party
services that want to pull data via the API to build things like
interactive graphs and visualization dashboards can do that.

Change-Id: I34a422c787118425aa484fa980aadd47cbf197e2
2015-04-03 11:30:59 -05:00
Yolanda Robla 0e3d87fbb3 Add ability to configure retention policy
Create new parameters on manifest, then apply
them to storage template.

Change-Id: Ia8fc17bb970d28bdfa80ed19cee6b44364dfc9c0
2015-03-19 10:45:04 +01:00
Yolanda Robla cfa9755e5a Add missing storage subfolders
Create storage/log, storage/rrd and storage/whisper folders
and ensure ownership.

Change-Id: I61727a37ea3edb5f76bb88dc4f90a5b97ad53b7b
2015-03-05 11:53:02 +01:00