Commit Graph

45 Commits

Author SHA1 Message Date
Clark Boylan 3caf0cf70d Retire this project
We've shutdown the kibana service and don't need to manage it with
puppet any more.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/839235
Change-Id: Iad395dfc91b9c7523b735b31da4dec2721acc42c
2022-04-25 09:43:47 -07:00
OpenDev Sysadmins ed143a4383 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:26:09 +00:00
Ian Wienand 7d24a8209d Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I27d612e447f6b8495e96c5b307e2a7160581a2fa
2019-03-24 20:35:40 +00:00
Clark Boylan 073ef439a7 Use full lookup path for serveradmin in template
The last change was incomplete bceause we evaluate the apache conf
template in the context of the httpd class meaning any vars outside of
that class must be fully qualified lookups. Make serveradmin a fully
qualified lookup.

Change-Id: I709af6a78316cd1ca7e489d62f13e988997f3157
2018-12-14 15:40:03 -08:00
Clark Boylan c6e070db6c Set server admin var so that vhost works
Using the puppet 4 future parser it is far more particular about
variables and their scoping. We don't set a serveradmin in this class
but then attempt to use it in the vhost template. This was fine under
puppet 3 as it would just look for this out of the current scope but
breaks under puppet 4.

Make this trivially happy on puppet 4 by setting a default that is sane.

Change-Id: Id415056cdfa693fd199e0b4a0c6b09e20faf23ef
2018-12-14 12:47:52 -08:00
Zuul db7e1887c8 Merge "Fix template variables" 2018-07-23 07:34:04 +00:00
Colleen Murphy 74e6af7fd7 Fix template variables
When serveraliases is not defined it should evaluate to nil, not ''.
Check for both to cover our bases.

Change-Id: Ie69baf01963f33cf695a0de763af75dc33b67daa
2018-07-12 14:34:59 +02:00
Colleen Murphy b60e5b9893 Update Gemfile for Zuulv3
The logic in the Gemfile was relying on Zuulv2 variables to find out
whether the spec helper gem was already available on disk, and since
Zuulv3 has changed things it was failing to find it and downloading the
master version instead. This patch ensures the Gemfile looks for the gem
in the right place when running in CI.

Change-Id: Ie3b3ae2d157b52f0b1ab36a8906642abd2a85ec3
2018-07-12 09:57:44 +02:00
Zuul 4902600dd1 Merge "Fix scope of vhost template variables" 2018-06-12 22:33:33 +00:00
Colleen Murphy de1b7966d4 Fix scope of vhost template variables
Dynamic scoping for variables in ERB templates was removed in puppet
4[1] which means that the variables defined in the
kibana::js class cannot be found when it is referenced in
the httpd::vhost defined type and will be evaluated as nil when
puppet runs. Use scope.lookupvar instead to be explicit about the
variable's source.

[1] https://puppet.com/docs/puppet/4.10/lang_updating_manifests.html#dynamic-scoping-in-erb

Change-Id: I27a63e74bda27269145b58e6d237c440085f0709
2018-04-27 19:50:28 +02:00
Colleen Murphy efba033351 Use instance variables in puppet template
Using local ruby variables in puppet ERB templates was deprecated in
puppet 3 and doesn't work any more in puppet 4. Access these values as
ruby instance variables instead.

Change-Id: I06bdd6f5b1f13c4970c5e3e9dc8be4cf8d919b03
2018-04-26 23:05:46 +02:00
Colleen Murphy 65192133b7 Depend on helper gem for spec_helper_acceptance
Instead of keeping a local copy of spec_helper_acceptance.rb and
requiring updates to all modules for any change, we can move it into the
common helper gem and require it from there. This will make it easier to
create and review changes that affect all puppet modules. Also change
the Gemfile to look for the gem in the local workspace if running in a
zuul environment.

Change-Id: I5b2adaf4f13eed7b7074ddf82dc5cb2da342f8c0
2017-08-18 10:41:43 +02:00
Colleen Murphy 7b6d9468c1 Fix beaker on xenial
Add a xenial nodeset and update the spec helper to install puppet 3 from
the Ubuntu repos instead of from puppetlabs.

Change-Id: I19ec4ea48b440d0014395ebf81c04c8232649a65
2017-06-24 15:50:23 +02:00
Spencer Krum 9ece2bf61d Use new infra_spec_helper for gem dependencies
Change-Id: Iaacf6e1cba36007ba2add504db0579fa21dfaa84
2016-06-21 18:38:50 -07:00
Clark Boylan 0a62118c0e Support Apache 2.4 ACLs
We need to use Require all granted in our directory that hosts the
Kibana files otherwise we get 403 forbidden when trying to access the
content.

Change-Id: I2e14ba113705f3fabaa4920704589da0f9abe978
2016-05-26 15:15:42 -07:00
Clark Boylan 7904825abb Configure apache when we use it
The kibana apache vhost file makes use of mod rewrite and mod proxy but
we don't enable those modules explicitly here. We happen to enable them
in other places that use this puppet module but that leads to races and
isn't explicit enough. Go ahead and explicitly enable the deps in this
puppet module so that it is self sufficient.

Change-Id: Ic05f26d61f2ac4b5adffee98782d1a02931d5731
2016-05-26 11:06:06 -07:00
Jenkins 586bb3818b Merge "Remove support for Kibana2" 2016-05-06 03:51:58 +00:00
Spencer Krum e0dbc71c64 Pin google-api-client; sanitize Gemfile
Change-Id: I74e18f0879df40d5b3a8c7f145028ccf0d44c754
2016-04-14 15:20:10 -07:00
K Jonathan Harker 3906807caf Remove support for Kibana2
The ruby version of Kibana reached its end-of-life in May 2015, remove
support for it from this puppet module.

Change-Id: I164ab9228e8ac8ada95d1ce2c0ee809cc01c9aa3
2016-04-13 12:09:46 -07:00
Andrey Nikitin 0c7f9c5e48 Order of the classes parameters is refactored
Order and intendation of those parameters are changed
to follow Puppet Style Guide recommendation [0].
Moreover, it will allow to an user to find much faster
a variable in a list of variables.

[0]. https://docs.puppetlabs.com/guides/style_guide.html

Change-Id: I7bd6abf2bc6b10da52fb4dad0c238b7b59cec54b
2016-03-21 13:03:26 +03:00
Spencer Krum 183dc2e05b Remove conflicting ownership resources
/opt/kibana was being chowned to 'kibana' on every run, but
/opt/kibana/<git_version> was a vcsrepo that was being checked out as
www-data. This caused a changed file resource for every file in the git
repo for every puppet run, causing a bunch of noise in the logs and
eventually running the puppetmaster out of disk space.

Also changed directory mode to 755 because 644 doesn't make a ton of
sense.

Change-Id: I3cc511c4a5ae561ec30478d96d9745611d4b5b65
2015-12-30 15:14:55 -08:00
Clark Boylan df57bd6f9e Encode query uri component properly
Previously we used encodeURI() to encode a single component of a URI,
this meant that characters like = were not encoded. This isn't correct
bceause we are just encoding a single component and need = to be
encoded. Fix this by using encodeURIComponent which should do the right
thing.

Change-Id: Ib320c1e646ccbd2233bd9d7a2be9df29ff7bd1ed
2015-12-15 09:50:36 -08:00
Clark Boylan 47f7ddf78f Vendor dashLoader.js with Share patch
In order to make Share work without relying on elasticsearch write perms
we patch dashLoader.js to construct the urls based on the logstash
dashboard since it is deterministic anyways. This is a massive hack but
maybe it will be less terrible once ES is upgraded and we can use
kibana4.

Change-Id: I64cc693e5e188c68e134b4769ccc519f6191f36c
2015-10-16 12:44:59 -07:00
Clark Boylan 304a9e6e81 Remove trailing / from config.js url
We are sensitive to trailing slashes in the url for the elasticsearch
api due to the way kibana3 constructs its urls and how we proxy ES api.
Basically if ther config has a trailing / then we double up /'s and the
mod rewrite rules for rewriting the ES api breaks.

Simple fix is to have config.js remove and /s for us so that puppet
doesn't have to be too smart.

Fix here adapted from suggested at
https://stackoverflow.com/questions/6680825/return-string-without-trailing-slash

Change-Id: Ica46003f8a3706c88e9e2bf58e0d3c8ea7c4a50d
2015-10-16 09:14:07 -07:00
Clark Boylan 86fb79e091 Use custom logstash dashboard
We update the default logstash dashboard to remove excessively short
refresh period options, default the query result listing output to
be timestamp and message, default to a 15 minute query instead
of a 24 hour query, and default to a blank query instead of '*" because
the '*' forces you to delete it to add any new queries but they are
functionally equivalent.

Change-Id: Id68a1f1e496f5d729b21aa2be30a5e2b73ab65fd
2015-10-16 09:13:35 -07:00
K Jonathan Harker c051797294 Use httpd::vhost instead of apache::vhost
Change-Id: I429e1977ed70d315adb5003b03c4b8682d43335c
2015-10-14 19:14:31 -07:00
K Jonathan Harker 8c66da99df Add kibana3 support
Add support for the javascript version of kibana.

By default, this is done by serving both the kibana and also an
elasticsearch proxy from the same location. Allowed GETs and POSTs
for read-only access to elasticsearch are passed to a proxy and all
other requests are served from the kibana source directory.

An optional prefix, such as 'elasticsearch/', can be specified in which
case the reverse proxy to elasticsearch will be served from this
sub-path.

Change-Id: I13f9dff0bbd6498a36dc75b026c9042a9bb05e8f
2015-10-14 12:18:31 -07:00
K Jonathan Harker 3e95d92cfc Refactor kibana-ruby specific bits to kibana::ruby
Move all of the resources specific to the ruby version of kibana to
kibana::ruby.

Change-Id: I93a4b0f103b4259f356e93fe025e0a649c79b75b
2015-08-24 15:09:28 -07:00
Spencer Krum 8423c1ae96 Fix target path for regular git clone during tests
Use same target directory for zuul-cloner and
the regular git command.

Change-Id: I894950857079adae442af77cfce82371d6ca768d
Co-Authored-By: Fabien Boucher <fabien.boucher@enovance.com>
2015-08-19 16:36:50 -07:00
Colleen Murphy 257a7d00be Add Gemfile and puppet 4 checks
In anticipation of puppet 4, start trying to deal with puppet 4 things
that can be helpfully predicted by puppet lint plugins. Also fix
arrow alignment errors not caught before.

Change-Id: Ie4df1a6316f10989af653d74aef5306bb79a8870
2015-08-03 18:44:50 -07:00
Spencer Krum 89d3945873 Boilerplate beaker-rspec files
Change-Id: I92e764113d21ef7c8bef673e7ebb2a6e4e7e282a
2015-07-28 02:04:45 -07:00
K Jonathan Harker fd84794d71 Add some missing dependencies
Change-Id: I3c85199a2ffce6e91531e1761601483181e67743
2015-06-03 13:18:28 -07:00
Jeremy Stanley 8ec2ed9d90 Replace ci.o.o links with docs.o.o/infra
The http://ci.openstack.org/ documentation site has been deprecated,
replaced by redirects to corresponding paths within
http://docs.openstack.org/infra/ where other Project Infrastructure
documentation already resides.

Change-Id: Iea20a2ff00ad8ef27f0d01e50e8854919b7dee91
2015-05-14 21:38:18 +00:00
Ramy Asselin a5be3eb33a Rename openstackci to openstackinfra
Change-Id: Id3319313f3209feeb3e02a1270ef6f8da6bc38c5
2015-04-20 13:44:48 -07:00
Yolanda Robla b78dfa4507 Update kibana project url
Version 0.2.0 doesn't exist on that repository.
It looks like the old kibana repo was renamed to kibana2 and then the
new kibana3 repo was renamed kibana.

Change-Id: I2c703fab5e802584eaea5e6dc51d5ea3cf8d84ff
2015-03-12 13:41:22 -07:00
Jeremy Stanley f46b543e95 Add missing LICENSE file
The content of this project is Apache 2 licensed, but we should
include a standard LICENSE file just to be clear about that.

Change-Id: Iee6320b9d7e35fbe8d3b0a9794f3e485c18ef2c8
2015-01-29 22:58:09 +00:00
Mikhail S Medvedev 04de99338d Add standard puppet module files and .gitreview
This module has been split out from system-config, and additional
files are required to create a proper puppet module.

Some of the files are described in the spec:
http://specs.openstack.org/openstack-infra/infra-specs/specs/puppet-modules.html

Change-Id: I215b6cad7638a3477faa3cc046f02c869f9d976e
2014-11-20 07:48:55 -06:00
Davanum Srinivas a0c7195d36 Bump up results per page in Kibana
Default is 100 results, bumping it up to 500

Change-Id: I45eb6084c42c9d1243b22a9f827d44cd91edf670
2013-12-06 15:17:28 -05:00
Clark Boylan d69b469b11 Upgrade Logstash to 1.2.1.
* modules/logstash/manifests/init.pp: Download and install Logstash
1.2.1.

* modules/openstack_project/files/logstash/log-gearman-client.py:
Logstash 1.2.1 comes with a new schema. Update the job data sent to log
push workers to better accomodate the new schema.

* modules/openstack_project/files/logstash/log-gearman-worker.py: Push
Logstash 1.2.1 schema compliant JSON to the Logstash TCP input.

* modules/openstack_project/templates/logstash/indexer.conf.erb:
Logstash 1.2.1 comes with a new schema and many input and filter
changes. Use the newly supported features like conditionals to keep the
config up to date.

* modules/kibana/templates/config.rb.erb: Change the default field for
kibana to 'message'. It was @message which is deprecated in the new
logstash schema.

Change-Id: Id19fc05bcce8d42c5c0cf33df3da7e95f5794107
2013-10-21 10:17:48 -07:00
Clark Boylan c7932eaf20 Better elasticsearch cluster settings.
Use mutliple discover nodes to determine elasticsearch cluster
membership. Put a timeout on recovery starting instead of the default
to recovery immediately. Describe cluster topology in elasticsearch yaml
config so that it can make smarter decisions. Round robin kibana
requests across each discover node.

Change-Id: I08ef9dd158ddf6a6ce01dfb2050626f543d45b10
Reviewed-on: https://review.openstack.org/34106
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-07-01 17:49:09 +00:00
Clark Boylan ae73e18646 Add support for elasticsearch cluster.
We need to expand our elasticsearch install base. Update puppet to make
this possible.

Change-Id: Id0dae839b12ebf47715cf40a363832e0f661a94f
Reviewed-on: https://review.openstack.org/33910
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
2013-07-01 17:48:16 +00:00
Clark Boylan ce79dcb720 Fix elasticsearch connectivity on logstash.o.o.
The new elasticsearch server needs to be able to connect back to
logstash over the elasticsearch ports. Also make kibana talk to the new
elasticsearch server.

Change-Id: I88af40c7885b263fcef5ff16210f073cd0a696a5
Reviewed-on: https://review.openstack.org/30354
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
2013-05-23 23:46:30 +00:00
Clark Boylan 54fb22d626 Set number of faceting indexes to 1 in Kibana.
* modules/kibana/files/config.rb: Kibana uses ElasticSearch's faceting
feature to collect statistics. It defaults to performing this over all
indexes but this is expensive so limit it to 1 index.

Change-Id: I324099f88a9855062aba4905380c9dc8540555a5
Reviewed-on: https://review.openstack.org/28202
Reviewed-by: K Jonathan Harker <jesusaurus@mail.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-05-06 18:23:44 +00:00
Clark Boylan 41163e7071 Set more precise Kibana timestamp format.
* modules/kibana/files/config.rb: The Kibana timestamp format we were
using did not include milliseconds. Change the format to
'yyyy-mm-dd\'T\'HH:MM:ss.l' to include milliseconds in the time field.

Change-Id: I6d812d5ee3f2b778a1adccf950f5423584ad69cf
Reviewed-on: https://review.openstack.org/27752
Reviewed-by: matthew-wagoner <matthew.wagoner@hp.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2013-04-30 17:05:38 +00:00
K Jonathan Harker 06c170ea0c Use kibana as the web frontend to logstash.
Add a kibana module that installs kibana and flat configuration files,
and also disables the previous boring bland web frontend.

Change-Id: I772c66f07d2bc3a88128e101074be9d4162e3f8a
Reviewed-on: https://review.openstack.org/27089
Reviewed-by: Jesse Keating <jesse.keating@rackspace.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
2013-04-25 20:45:19 +00:00