Commit Graph

58 Commits

Author SHA1 Message Date
Andreas Jaeger 16f5f4cf44 Retire repository
Fuel (from openstack namespace) and fuel-ccp (in x namespace)
repositories are unused and ready to retire.

This change removes all content from the repository and adds the usual
README file to point out that the repository is retired following the
process from
https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project

See also
http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011647.html

Depends-On: https://review.opendev.org/699362
Change-Id: I6b38110f2d230006cd9cce1da5d2cf76cf470d35
2019-12-18 09:54:55 +01:00
Peter Razumovsky bd90fc114a Remove elasticsearch code from stacklight repo
Elasticsearch moved to fuel-ccp-elasticsearch repo, due
to collisions between similar docker image names, remove
elasticsearch code from here.

Change-Id: Ia8c74a335ffe9355e4c033d0998080ce56fb1d8f
Depends-On: Ic39eb474f42b25e55772cb95edd362e4be5623c3
2017-03-16 11:20:01 +00:00
Andrey Pavlov e0c865b5a2 Fix influxdb without grafana
Change-Id: I50a780bf98f1df8a89f62ab45268a9308630ed67
2017-01-25 05:44:12 +00:00
Bartosz Kupidura 0d1024c807 Start using grafana from fuel-ccp-grafana
Change-Id: Ic8b059b95c0956a92abfc9fc754071a60c66649e
Depends-on: I89540f8dd656dd930c8b4d999aa195a295592ba4
2017-01-11 16:08:56 +01:00
Proskurin Kirill 9bc92a39dc Removed unneded probes. Merry Xmas!
Change-Id: Iff4ddf85a53d5c26b2f0127842c6f77e09836562
2016-12-26 13:12:33 +00:00
Andrey Pavlov 5458369061 Use "address" func when it's possible
Change-Id: I8de5b5edcb4a2c78b8fd5d64bb2782ee807f6948
2016-12-05 08:54:36 +00:00
Andrey Pavlov c50a942322 Adding minimal dsl version for services
Change-Id: I4a0c00b40b2af4f213daf7e43419dd46951fc4bc
Depends-On: Iddb02f88e47e53f8f7233f76ec1afe161c185a86
2016-11-21 12:31:29 +00:00
Andrey Pavlov 5d9cbfb7c7 Ports configuration refactoring
Ports configuration has been changed:

before:

  port: 123:3333

after:

  port:
    cont: 123
    node: 3333

Change-Id: I76642c407fd4ca00f711212b916f8cc4fbc285e2
Depends-On: I54ab8dd02fc88b821a1f0d05c08f98b618730150
2016-10-26 17:37:57 +00:00
Aleksandr Mogylchenko 50b30be50c Migrate liveness check to new format
Change-Id: Ic8f7b67cc7188fd4a2898dd5c58789132d41afa0
Depends-On: I920029d4cb14981427e7f978778506fb0b69c435
2016-10-26 15:01:05 +02:00
Andrey Pavlov d1cb178c91 Changing "daemonset" flag in service def to "kind" param
We want to alling with k8s definitions and it would be more convenient
to have one param instead of several flags.

Change-Id: I378f7fd3e89ac12e9f6d16fca3591d09ff33d4f9
2016-10-25 10:21:22 +00:00
Yuriy Taraday bcb332548d Convert all stacklight configs to nested
Change-Id: Ie12b0fdcc890c71893826f52560b5ab5117ca132
2016-10-07 18:25:30 +03:00
Andrey 3f8ea9410b Using "node_name" instead of custom env vars
Now we have "node_name" variable to render config files

Change-Id: I6ff108cf769da846fd878a3f0fb221df2854917f
Depends-On: I8ebbbd94803ccb9a8d13eede2db7db8b13673937
2016-09-29 07:17:13 +00:00
Olivier Bourdon 4b0617cd24 Implement alarm-manager
alarm-manager is responsible for watching a configurable
location within the filesystem where the user can put
a YAML file which defines alarms. When change/creation is
detected, the YAML file is checked for proper contents
and if verification is successfull, LUA code is generated
as well as LUA configuration files. Hindsight will pick
up those changes after a certain period of time and
provides proper alarming to the platform.

Change-Id: I7b2b98f379c49bdbf23177a038bdca9433d1c6e5
2016-09-21 19:54:39 +02:00
Olivier Bourdon 6fcfb2d349 Fix YAML content for proper CI acceptance
Due to the fact that yamllint utity is now run
on each YAML file, this change fix the current
error for line being too long

Change-Id: If35a7fc823c5d13a0dbdade6900be6ee51b1a28f
2016-09-21 11:23:59 +00:00
Sergey Lukjanov 1e3b0d8053 Yaml linter execution and config improved
Change-Id: Id06bb20d5e6c8803b5890788e803f0fa3d893d75
2016-09-16 19:56:25 -07:00
Jenkins 710e0c01bb Merge "Add Lua code for alarming" 2016-09-16 10:25:27 +00:00
Jenkins c13be0d39d Merge "Add cronjob for elasticsearch indices cleanup" 2016-09-15 22:46:37 +00:00
Proskurin Kirill 89567d8d14 Add cronjob for elasticsearch indices cleanup
Elasticsearch pod now has a cron container to run
some jobs for it.

Change-Id: Ia299e662681174512b0ad70b932caf66448b4cea
2016-09-15 14:06:53 +00:00
Jenkins ba90f98fbf Merge "Unhardcode elasticsearch and kibana version" 2016-09-15 12:42:28 +00:00
Éric Lemoine ed5934cd36 Add Lua code for alarming
This commits adds Lua code for generating AFD (Anomaly and
Fault Detection) metrics based on the evaluation of alarms.
The Lua code was copied from the lma_collector Fuel plugin
[*], with changes to accomodate Hindsight and the versions
of lua_sandbox and lua_sandbox_extensions we rely on.

In the future we plan to move this Lua code in its own Git
repository. And the Hindsight Dockerfile will install the
Lua code in the image using Debian packages.

The afd_node_default_cpu_alarms.lua and
hindsight_afd_node_default_cpu_alarms.cfg.j2 files will be
removed. Instead the operator will configure alarms through
a YAML file, and we will use a sidecar container for
generating Lua tables including alarm definitions and
corresponding plugin configuration files.

[*] https://github.com/openstack/fuel-plugin-lma-collector/

Change-Id: If182c3a6453f7bf8b72f03af56a14ace109eaa68
2016-09-15 12:13:27 +00:00
Proskurin Kirill d865e94bb1 Unhardcode elasticsearch and kibana version
Plus update them to the recent ones.

Change-Id: Ieee08bd4c261f17b3cf30782861b840124a9575f
2016-09-15 10:46:43 +00:00
Jenkins 103d426993 Merge "Add support for multi-value metrics" 2016-09-14 06:54:11 +00:00
Éric Lemoine a6f365dfa5 Add support for multi-value metrics
This commit adds support for metrics with multiple values.
Multi-value metrics will for example be needed for alarming.

Change-Id: I496fa1925c389f2638cf9b99243fbf45d7d2dad7
2016-09-13 13:21:31 +00:00
Dmitriy Ukhlov 1227114cd7 Adapt to new fuel-ccp rendering approach
Change-Id: I11e2e20546f82d289d7689fd6ddbb3f921e3f273
2016-09-13 16:12:49 +03:00
Olivier Bourdon 0e7c59917e Add SMART monitoring
Note that we can not gather SMART infos from the host within
VMs
See https://www.smartmontools.org/wiki/FAQ
and more particularly the
DosmartctlandsmartdrunonavirtualmachineguestOS
sub-entry

Change-Id: Idee7d48e45a5a388061d196d1e07c55404780085
2016-09-13 06:33:40 +02:00
Jenkins 22b493e88e Merge "Remove "passthru" Snap plugin" 2016-09-08 17:26:14 +00:00
Olivier Bourdon b21cc701a0 Use snap binaries to build image
Until we find a better official and publicly available
location for hosting these binaries, they will be hosted on
bintray.com

This should change when/if Intel will be providing access
to nightly binary builds. Please note that the binaries
have been produced using Intel's build scripts

The snap task file has been updated to take into account
the fact that cpu metrics are now dynamic and that due to
snap framework issue #1144 you can not request a specific
instance of dynamic metrics

The Grafana system dashboard has been updated to comply with
the snap task change above

Change-Id: I76a2eac0497c8e2024234aab5e117d173e136049
2016-09-07 16:00:05 +02:00
Éric Lemoine 78b13b7209 Set correct host name in metrics
This commit fixes a bug where hostnames were not correct in
metrics collected by Snap and Hindsight. It relies on
Kubernetes' downward API and the spec.nodeName field [1].
The latter is only supported by Kubernetes 1.4 and higher,
and the deployment of stacklight-collector pods will fail if
Kubernetes 1.3 or lower is used.

[1] <https://github.com/kubernetes/kubernetes/pull/27880>

Change-Id: I73cd35803a2201a09144bf925753156e47489cff
Depends-On: I293bb3aa113883c02f2e738f9d74291bf2f23d95
Partial-Bug: #1614484
2016-09-07 11:48:03 +02:00
Éric Lemoine 1b12557fe7 Collect kubelet stats
This commit adds an input Hindsight plugin that scrapes the
Kubelet stats API at a regular time interval. This is to
collect system metrics (CPU usage, etc.) relative to pods
running on the cluster. The metrics created by the plugin
are injected into the Hindsight pipeline, and then read by
the InfluxDB plugin which sends them to InfluxDB for
storage.

Change-Id: I0b39d416ebc4e8090a959267d6fc813ddab2674a
2016-09-06 17:19:05 +02:00
Éric Lemoine 0fe1681f10 Use packages for Hindsight
Change-Id: I44594235ff723ee5b50cbe212530e57aaf704180
2016-09-02 16:17:25 +02:00
Éric Lemoine 07ff4d8544 Use a "local" command for configuring Grafana
Currently the grafana-configure.sh script is executed in
a job. So if the Grafana Pod is re-created all the
pre-configured Grafana dashboards are lost. This commit
fixes the bug by using a "local" post command instead of
a "single" post command for the execution of
grafana-configure.sh.

Change-Id: I9368b010da684ac7f2c352b920b7df1785fd0e78
2016-08-30 15:14:08 +02:00
Éric Lemoine ed89c626e0 Remove "passthru" Snap plugin
Having a "process" node in a "collect" node is not
mandatory. This commit removes the "process" node
and places the "publish" node in "collect" node
directly.

Change-Id: I482b0ea37f80dd7f188ef38467f9ab41c7e5221e
2016-08-29 14:56:25 +02:00
Jenkins b60fb5f901 Merge "Change Grafana datasource name to "CCP InfluxDB"" 2016-08-24 13:16:39 +00:00
Éric Lemoine f816463113 Change Grafana datasource name to "CCP InfluxDB"
Change-Id: I3049d4d67a854adfd393955eaea526938fdbafaa
2016-08-24 13:24:35 +02:00
Jenkins 71ccd9445e Merge "All Jinja2 template files must end in .j2" 2016-08-22 16:13:08 +00:00
Olivier Bourdon 4fef089ff4 All Jinja2 template files must end in .j2
Change-Id: I5cbfb798b3804511c91545c69c12f89520499b75
2016-08-22 17:13:18 +02:00
Jenkins 6ae60e0ce7 Merge "Added cron container, for logs rotating" 2016-08-22 13:14:16 +00:00
Proskurin Kirill acdd220ded Added cron container, for logs rotating
Closes-Bug: 1615598
Change-Id: I9ffaf4c2429f80821e5efbabd4d2a70ed2650d35
2016-08-22 14:19:48 +02:00
Olivier Bourdon 8bed964353 Reduce the max nb of consecutive failures
Note that this require the snap framework commit ID
to be updated to a later version so that this effectively
work properly

Change-Id: Ie3963834d875c40ff0e34760d82e908365feaae0
2016-08-19 15:28:53 +02:00
Jenkins b78237a53e Merge "Remove variables from volumes of rbmq and mariadb" 2016-08-18 17:08:51 +00:00
Jenkins 9ce6ff679f Merge "Add support for collecting apache logs" 2016-08-18 17:06:48 +00:00
Proskurin Kirill cc7b3899f7 Remove variables from volumes of rbmq and mariadb
Change-Id: I8e3af19876e2c904aa3ac630fe98f4563fddf2aa
2016-08-18 18:17:20 +03:00
Proskurin Kirill 422bfe32d0 Add support for collecting apache logs
Depends-On: Ib62ace1db43fb2c9125a890575fb901646b76e53
Depends-On: Iac338a9e3183428c07357d02d3a597d78249b155

Change-Id: I662a12cfcc195cf457f85cdd2fee66f754a2acf2
2016-08-18 18:09:30 +03:00
Éric Lemoine 733ab45774 Rename finish-grafana-setup to grafana-configure
This is to be consistent with the names used for the
"keystone-db-create" pre job and the others of the same
type.

Change-Id: I0c3ef71b77b82589e903923c7d0ccaa9bde0bfe8
2016-08-18 16:52:19 +02:00
Éric Lemoine 67922c0d38 Update Hindsight
This commit updates Hindsight, lua_sandbox and
lua_sandbox_extensions.

For lua_sandbox a tag/version is now used (v1.0.3).

Change-Id: Ie5eaf8ecbeb1bfe77600fe553c904c8031895ede
2016-08-17 15:52:34 +02:00
Jenkins e32c56ed87 Merge "Add ovs logs collecting support for heka" 2016-08-12 07:07:04 +00:00
Proskurin Kirill bb776dd623 Add ovs logs collecting support for heka
Since ovs logging to stderr too and it's not OS service, I have to
create a multidecoder for dockerlogs. Using it, I can try different
decoders on each log string to find the right one.

It's the only option available for us right now, since Dockerlogs
plugin doesnt support docker containers filtering.

Change-Id: I53ab2beb49e5847c3e17e443a7838b0448cb066f
2016-08-11 15:47:15 +02:00
Éric Lemoine d846447209 Add monitoring agent service
This commit adds Snap and Hindsight Dockerfiles, and an
application definition to deploy Snap and Hindsight in pods
controlled by a DaemonSet.

This commit covers the "collector" part of the
"introduce-monitoring" specification [*].

[*] <https://review.openstack.org/gitweb?p=openstack/fuel-ccp-specs.git;a=blob;f=specs/introduce-monitoring.rst;h=9e50a1a91cbd851ba6d5b5a30ae2c6a2388dc1ba;hb=c8c111f10ab104d0f508d17906b6bd30e94bdfd6>

Change-Id: Ib65499d266c8c19c93efc288294a2f56e71485d8
2016-08-10 15:34:01 +02:00
Proskurin Kirill 54c8a5709d Change mariadb.log to mysql.log in heka parser
I think it's better to always use "mysql" as a name for mysql logs
for all mysql forks(Mariadb, Percona, etc), since we could use any of
this avalible forks.

Change-Id: Ia6dc753908ee9986904a2255b13766835013f208
2016-08-10 11:39:56 +02:00
Jenkins 5bf3334f8f Merge "Added rabbitmq and mariadb logs collection" 2016-08-10 08:35:04 +00:00