Commit Graph

67 Commits

Author SHA1 Message Date
Clark Boylan 66cf2dd8ef Retire this project
We've shutdown the elasticsearch service and don't need to manage it
with puppet any more.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/839235
Change-Id: I7b94d0fc8f49b3900ab0012196ff665e3e54266d
2022-04-25 09:46:00 -07:00
OpenDev Sysadmins 4627e95891 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:25:38 +00:00
Ian Wienand 86e4eb1ff5 Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I40158b08c789c9134a609c2be55a6421733010a2
2019-03-24 20:35:38 +00:00
Colleen Murphy c7f4ea9e3b Update Gemfile for Zuulv3
The logic in the Gemfile was relying on Zuulv2 variables to find out
whether the spec helper gem was already available on disk, and since
Zuulv3 has changed things it was failing to find it and downloading the
master version instead. This patch ensures the Gemfile looks for the gem
in the right place when running in CI.

Change-Id: I6923d679c1e499e1e1451bc250173603211ec133
2018-07-12 09:57:44 +02:00
Paul Belanger 0d9347f41e
Stop managing file permissions under /var/lib/elasticsearch
There shouldn't be a need to do this, as the contents are generated
directly from elasticsearch.

Change-Id: I99ec3cfcc146477ee011f6dd926191c65495e90f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-09-20 14:47:53 -04:00
Colleen Murphy 32b7f9ee61 Depend on helper gem for spec_helper_acceptance
Instead of keeping a local copy of spec_helper_acceptance.rb and
requiring updates to all modules for any change, we can move it into the
common helper gem and require it from there. This will make it easier to
create and review changes that affect all puppet modules. Also change
the Gemfile to look for the gem in the local workspace if running in a
zuul environment.

Change-Id: I090e8ed3c8dbaf5f1a23cf09806d9dbc03c5f143
2017-08-22 22:40:50 +02:00
Jenkins b5d7ae0ea9 Merge "Fix beaker on xenial" 2017-06-14 23:57:01 +00:00
Colleen Murphy f79ea7c853 Fix beaker on xenial
Since the beaker jobs are being run on xenial, we need a special nodeset
for it, otherwise beaker gives an error:

 beaker-hostgenerator was not able to use this value as input.
 Exiting with an Error.

We also want to install puppet from the Ubuntu repos rather than from
puppetlabs, since puppetlabs doesn't support puppet 3 for Xenial. For
centos we can keep the install process the same.

Additionally, this patch updates params.pp to distinguish between JRE
packages based on the Ubuntu release so that these module will work on
Xenial. We still keep support for trusty.

Finally, remove spec assertions that are not useful, including
verifications that packages are installed and config files existing. It
is enough to assume that if the puppet ran without errors that it was
able to also successfully find and install packages and lay down config
files. We remove these tests instead of updating the package check for
the new JRE package.

Change-Id: Ifd2244ae9dd212b2475f9cd6adb994bc058a4769
Depends-On: I053d437ceb7895fe2b1e2c46bf9cf19f73bdb20c
2017-06-10 20:11:50 +02:00
Paul Belanger beb8b5dc97
Add bindep.txt file
Bindep is a tool for checking the presence of binary packages needed
to use an application / library. It started life as a way to make it
easier to set up a development environment for OpenStack projects.

Change-Id: I053d437ceb7895fe2b1e2c46bf9cf19f73bdb20c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-05-01 21:07:13 -04:00
Alex Monk d4fcbbff1c Access es_template_config with @
To prevent this on every puppet run:
(warning): Variable access via 'es_template_config' is deprecated. Use
'@es_template_config' instead.
template[/etc/puppet/modules/elasticsearch/templates/elasticsearch.yml.erb]:33

Change-Id: Ie6c685d0748ce26c2bd2d227ce3b4e0578a95018
2016-08-07 00:13:30 +00:00
Spencer Krum 4f6c91e3f0 Use new infra_spec_helper for gem dependencies
Change-Id: Idd617dbe475ed6490ad06a862b273d3ec1b282d4
2016-06-21 18:38:06 -07:00
Paul Belanger 20ee6430cb
Set permissions on /var/lib/elasticsearch
It is possible when we move volumes from one server to another that
our UID permissions are not currect. Have puppet ensure they are setup
properly.

Change-Id: I8f15b60507d1d52235d2db49753df144bea046fe
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2016-05-27 11:45:11 -04:00
Clark Boylan 28896065c2 Use current elasticsearch url
Elasticsearch now hosts at elastic.co. The old name still functions but
update this before upstream can break us.

Change-Id: Ia39471881f2946eaab6e37dea8a19ed5d36d0662
2016-05-04 12:05:42 -07:00
Spencer Krum 4a05a695c9 Pin google-api-client; sanitize Gemfile
Change-Id: Iba0d903e50cb813dcd4000630f678763575c3e08
2016-04-14 15:18:21 -07:00
Jenkins 43f0701f76 Merge "Make total_shards_per_node configurable" 2016-04-12 00:47:42 +00:00
Andrey Nikitin d1429d37d2 Order of the class parameters is refactored
Order and intendation of those parameters are changed
to follow Puppet Style Guide recommendation [0].
Moreover, it will allow to an user to find much faster
a variable in a list of variables.

[0]. https://docs.puppetlabs.com/guides/style_guide.html

Change-Id: Ie516f4a5aafc724d8f290c6d12f5ee2f6fe99a94
2016-03-21 11:40:23 +03:00
K Jonathan Harker 0733bdc394 Make total_shards_per_node configurable
Add support for index.routing.allocation.total_shards_per_node in the
elasticsearch.yml.erb template. This value sets the total number of
shards across all indices to store on any single node. A value of -1
specifies unlimited shards.

Change-Id: I8a7472fb8daf97015d47ab33476b78779e762dbe
2016-03-11 12:33:39 -08:00
jesusaurus a3745ab6ee Make network.publish_host configurable
The network publish host value defaults to the ip address of the
first non-loopback interface, but this is not always an appropriate
value.

In some cases this value may need to be set to something other than
the ip address of the local interface, such as when connections should
be made to a floating ip address.

Change-Id: I9ace9bc80607e8d60478e869ef13feea8c49e10a
2016-02-12 12:22:20 -05:00
James E. Blair 31c222aca1 Stop logging actions at DEBUG level
They can be very verbose, and on the controller, include things
like full tracebacks for a simple inability to finish a query.
Such instances are still logged at WARN.

Change-Id: I32844cbd7fdd57881ae83d9a78f23e3e8eaa2587
2015-11-30 13:01:35 -08:00
James E. Blair 57f13f58b8 Add logging configuration
This is the file as currently installed.

Change-Id: I53f03523204306ab0d97ee4ba1b0ae2277949547
2015-11-30 13:01:25 -08:00
Spencer Krum 00ec398586 Ensure gem utility is on the host
The gem utility is needed by the archive module to install the faraday
gem so that archive works.

Change-Id: I914a6c6490e17ba02b4a6af4ed20c9f33931a33f
2015-11-27 12:21:55 -08:00
Clark Boylan f6a724c575 Add indices.breaker.fielddata.limit to config
The indices.breaker.fielddata.limit config option specifies how much of
the heap can be used before breaking out and failing to prevent OOMing.
By default this is set to 60% of heap which may be too conservative if
you have larger datasets.

Change-Id: I46c0cb5157aae40a0029ff1b425ecc663d171768
2015-11-13 12:06:43 -08:00
Clark Boylan bbd742c992 Add config options for cors config
With newer ES we need to set CORS controls properly because it seems to
care about that now. This change allows you to enable the use of CORS
and specify which origins are allowed.

By default ES doesn't allow any cross origin requests so you have to
explicitly enable it with these options if you want to allow it.

Change-Id: I0aa8d5167c770c1024b7596da582d6cc089b1b47
2015-11-12 09:56:08 -08:00
Bruno Tavares a111864fd2 Add acceptance tests for puppet-elasticsearch.
Add acceptance tests for puppet-elasticsearch module so that once the
module is applied we check if files were created, packages were
installed and services were started.

Co-Authored-By: Danilo Ramalho <dramalho@thoughtworks.com>
Change-Id: Id902337b8f6ca3210644f9401002755d9f6b3e0b
2015-09-24 19:39:14 -03:00
Spencer Krum 1985eb7f05 Fix target path for regular git clone during tests
Use same target directory for zuul-cloner and
the regular git command.

Change-Id: Iddd6ca6b4e1d8d2ec90d7aa7013068f172928f1c
Co-Authored-By: Fabien Boucher <fabien.boucher@enovance.com>
2015-08-19 16:35:14 -07:00
Jenkins 2e9a3ea1ce Merge "Add Gemfile and puppet 4 checks" 2015-08-06 03:33:41 +00:00
Spencer Krum f7d9c124a8 Boilerplate beaker-rspec files
Change-Id: I80f4060e22a968b8afae9c86e25f6df193aaa7c8
2015-07-28 02:04:10 -07:00
Colleen Murphy d5e54f7316 Add Gemfile and puppet 4 checks
In anticipation of puppet 4, start trying to deal with puppet 4 things
that can be helpfully predicted by puppet lint plugins.

Change-Id: I4bb194d3eedea05f4bbfa9b70920793e2b30d10c
2015-07-21 18:50:32 -07:00
Jenkins 9b03dcd9ef Merge "Fix variable access warnings" 2015-07-14 19:12:14 +00:00
Jenkins 1bc472e79c Merge "Add template support for the path.data" 2015-07-14 18:56:13 +00:00
Jenkins 90a9f83ee6 Merge "Add a cluster name option" 2015-07-14 18:56:11 +00:00
Jenkins 3f33a3169e Merge "Use archive to fetch/checksum instead of wget." 2015-07-14 18:16:00 +00:00
Paul Belanger 113d29ffeb Fix variable access warnings
Change-Id: I70403b12ba86f6bd8106acd342bfa3f1806a3632
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2015-07-09 16:49:38 -04:00
K Jonathan Harker 57c667dd6b Add template support for the path.data
Also ensure the directory exists and is owned by elasticsearch if
specified.

Change-Id: I501d5995acc018d8e27a0a216609b4d4ed138864
2015-05-26 10:45:54 -07:00
K Jonathan Harker b40f1165d1 Add a cluster name option
This is useful if you want to run multiple clusters, or if you just want
to change the default value to something custom.

Change-Id: Id1731a209b43057090852a3bc484f61529717bd3
2015-05-20 16:51:09 -07:00
Jenkins b990d859d2 Merge "Cleanup old ES logs" 2015-05-15 00:41:44 +00:00
Clark Boylan 0c638194c0 Cleanup old ES logs
Keep elasticsearch from fillings its disk with logs by deleting logs
older than two weeks. Note we do not use logrotate because elasticsearch
expects to be doing the log rotation itself, it just doesn't delete
older logs. Instead we setup a cron with find to delete the older logs.

Change-Id: Ia2611cbf6e4aa1945ec8b3914d7ba042d2a7b0c1
2015-05-14 16:10:07 -07:00
Jeremy Stanley 5cce1adb33 Replace ci.o.o links with docs.o.o/infra
The http://ci.openstack.org/ documentation site has been deprecated,
replaced by redirects to corresponding paths within
http://docs.openstack.org/infra/ where other Project Infrastructure
documentation already resides.

Change-Id: I81e2e8010c2fd7f7a12185da9d3c87f04e5fbd39
2015-05-14 21:38:17 +00:00
Ramy Asselin da4dbde4fe Rename openstackci to openstackinfra
Change-Id: I54558900901b9d2a7f8f5a4cbe99593378a90164
2015-04-20 13:43:33 -07:00
Nan Liu ee1fe5cf6b Use archive to fetch/checksum instead of wget.
Change-Id: Iab11edf7fa8030864bf52253a991db3d85c0a02e
Depends-On: I002951a492ef25d86b3bfd5df01429edd3da08fd
2015-03-31 16:29:12 -07:00
K Jonathan Harker c6ac0c47f3 Make the number of shards and replicas tunable
The shard count and replica count are some of the primary ways to tune
an elasticsearch cluster and should be tunable through this template.

Change-Id: Id6dd65ed3ca75e464f294774282eb36f6c001c67
2015-02-10 16:29:45 -08:00
Jeremy Stanley 9ec09cf603 Add missing LICENSE file
The content of this project is Apache 2 licensed, but we should
include a standard LICENSE file just to be clear about that.

Change-Id: Iee6320b9d7e35fbe8d3b0a9794f3e485c18ef2c8
2015-01-29 22:56:59 +00:00
Mikhail S Medvedev 7ba9b2a265 Add standard puppet module files and .gitreview
This module has been split out from system-config, and additional
    files are added to create a proper puppet module.

    The spec:
    http://specs.openstack.org/openstack-infra/infra-specs/specs/puppet-modules.html

Change-Id: Ifc65c4d7b5a4b7111d01cb2f954020f7c7ccc3de
2015-01-18 11:38:31 -06:00
Clark Boylan 9222b1ffd8 Use local ES balancers on logstash workers
Set up small non master non data elasticsearch daemons on logstash
workers to act as local load balancers for the elasticsearch http
protocol.

Change-Id: Ie3729f851ebef3331a6b69f718e57d663209bfc2
2014-07-07 17:01:57 -07:00
James E. Blair 99663c48ad Revert "Fixing deprecation warnings"
This reverts commit 82b9b59522928863ddadeacfec819e287303ef20.

Change-Id: I746d7ae57802dc76618db9024a0cf94c43774c02
2014-07-02 17:35:02 +00:00
Spencer Krum f039738b9f Fixing deprecation warnings
Non instance variable representation is deprecated
so needs to be changed. This change changes varibles
to their instance variable representation.

See more details see:
http://docs.puppetlabs.com/guides/templating.html

Change-Id: Ib77827e01011ef6c0380c9ec7a9d147eafd8ce2f
2014-06-19 22:41:42 -07:00
Monty Taylor 295bcb7ee0 Fix two typos in updated elasticsearch module
The resource has _deb in it, which was left out.
Also, the java package name resource was replaced in only
one of two locations.

Change-Id: Ia47beeeff80f2cba1e7dfff0538156bd6464b9a1
2014-04-18 16:02:37 -07:00
Monty Taylor fce067853a Silence successful elasticsearch runs
java7-runtime-headless is an empty package that can never be
installed, so it attempts to install every time. Replace it with
openjdk-7-jre-headless, which is the real package.

Also, add a refreshonly on the exec for checking the diff, because
it runs every time as well.

Change-Id: I5195d27ae90cd7f9f3e20d92aca43d7b8d0e504a
2014-04-18 15:38:09 -07:00
Clark Boylan 9268dfc1d4 Install curl on elasticsearch nodes.
We rely on curl to query the elasticsearch api for cluster state,
deleting indexes and so on. Ensure it is installed on the elasticsearch
nodes.

Change-Id: I843c6be65e5750856cb0b68f68eb7fac47e04e29
2014-03-03 10:30:27 -08:00
Jeremy Stanley ba7791374f Don't ensure elasticsearch service is running
The package maintainer scripts start elasticsearch when it gets
installed, but it seems to return control before the pidfile is
created which allows puppet to race for it and ultimately launch a
second daemon resulting in an OOM condition. It's configured to
start at boot anyway, so the puppet ensure running isn't buying us
much.

Change-Id: I1b374f90c198b9cb73d412fe63b439623f434d9a
2014-02-26 01:52:46 +00:00