Add contrib/ directory
The contrib directory contains scripts to deploy ElasticSearch, InfluxDB and the visualization dashboards as Docker containers. Ultimately they will be superseded by the elasticsearch-kibana and influxdb-grafana plugins but for the short term, they are handy to validate the LMA collector plugin itself. Change-Id: If61b5193f12688974aa74994d967bc95e785d457
This commit is contained in:
parent
6854302bd6
commit
830ad25364
|
@ -0,0 +1,31 @@
|
||||||
|
The scripts in this directory can be used to deploy the additional systems that
|
||||||
|
are running along with the Logging, Monitoring and Alerting collector.
|
||||||
|
|
||||||
|
# Pre-requisites
|
||||||
|
|
||||||
|
The scripts require the [Docker](https://www.docker.com/) runtime.
|
||||||
|
|
||||||
|
# ElasticSearch
|
||||||
|
|
||||||
|
[ElasticSearch](http://www.elasticsearch.org/overview/elasticsearch) is used to
|
||||||
|
store and index the logs and notifications gathered by the LMA collector.
|
||||||
|
|
||||||
|
To install the ElasticSearch stack, see (elasticsearch/README.md).
|
||||||
|
|
||||||
|
# InfluxDB
|
||||||
|
|
||||||
|
[InfluxDB](http://influxdb.com/) is used to store the metrics reported by the
|
||||||
|
LMA collector.
|
||||||
|
|
||||||
|
To install the InfluxDB stack, see (influxdb/README.md).
|
||||||
|
|
||||||
|
# LMA dashboards
|
||||||
|
|
||||||
|
The LMA dashboards are based on:
|
||||||
|
|
||||||
|
* [Kibana](http://www.elasticsearch.org/overview/kibana) for displaying and
|
||||||
|
querying data in ElasticSearch.
|
||||||
|
|
||||||
|
* [Grafana](http://grafana.org/) for displaying and querying data in InfluxDB.
|
||||||
|
|
||||||
|
To install the dahsboards, see (ui/README.md).
|
|
@ -0,0 +1,35 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# Copyright 2015 Mirantis, Inc.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
function docker_pull_image {
|
||||||
|
if [[ "$( docker images -q ${1})" == "" ]]; then
|
||||||
|
docker pull ${1}
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
function docker_get_id {
|
||||||
|
echo $(docker inspect --format="{{ .Id }}" ${1} 2>/dev/null)
|
||||||
|
}
|
||||||
|
|
||||||
|
function docker_is_running {
|
||||||
|
local IS_RUNNING=$(docker inspect --format="{{ .State.Running }}" ${1} 2>/dev/null)
|
||||||
|
[[ "${IS_RUNNING}" == "true" ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
function docker_shorten_id {
|
||||||
|
echo ${1} | cut -c 1-12
|
||||||
|
}
|
|
@ -0,0 +1,64 @@
|
||||||
|
# Description
|
||||||
|
|
||||||
|
Scripts and tools for running an ElasticSearch server to be used with the LMA
|
||||||
|
collector.
|
||||||
|
|
||||||
|
# Requirements
|
||||||
|
|
||||||
|
To run ElasticSearch, the host should have at least 1GB of free RAM. You also
|
||||||
|
need sufficient free disk space for storing the data. The exact amount of disk
|
||||||
|
depends highly on your environment and retention policy but 20GB is probably a
|
||||||
|
sane minimum.
|
||||||
|
|
||||||
|
To store and query the data, clients need to be able to connect to the IP
|
||||||
|
address of the container's host on the TCP port 9200.
|
||||||
|
|
||||||
|
# Running
|
||||||
|
|
||||||
|
Simply:
|
||||||
|
|
||||||
|
```
|
||||||
|
$ ./run_container.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Use environment variables to override the default configuration:
|
||||||
|
|
||||||
|
```
|
||||||
|
$ ES_MEMORY=2 ./run_container.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Supported environment variables for configuration:
|
||||||
|
|
||||||
|
* ES_LISTEN_ADDRESS: listen address on the container host (default=127.0.0.1)
|
||||||
|
|
||||||
|
* ES_DATA: directory where to store the ES data and logs (default=~/es_volume)
|
||||||
|
|
||||||
|
* ES_MEMORY: amount of memory (in GB) allocated to the JVM (default=16)
|
||||||
|
|
||||||
|
# Testing
|
||||||
|
|
||||||
|
You can check that ElasticSearch is working using `curl`:
|
||||||
|
|
||||||
|
```
|
||||||
|
curl http://$HOST:9200/
|
||||||
|
```
|
||||||
|
|
||||||
|
Where `HOST` is the IP address or the name of the container's host.
|
||||||
|
|
||||||
|
The expected output is something like this:
|
||||||
|
|
||||||
|
```
|
||||||
|
{
|
||||||
|
"status" : 200,
|
||||||
|
"name" : "fuel.domain.tld",
|
||||||
|
"cluster_name" : "elasticsearch",
|
||||||
|
"version" : {
|
||||||
|
"number" : "1.4.2",
|
||||||
|
"build_hash" : "927caff6f05403e936c20bf4529f144f0c89fd8c",
|
||||||
|
"build_timestamp" : "2014-12-16T14:11:12Z",
|
||||||
|
"build_snapshot" : false,
|
||||||
|
"lucene_version" : "4.10.2"
|
||||||
|
},
|
||||||
|
"tagline" : "You Know, for Search"
|
||||||
|
}
|
||||||
|
```
|
|
@ -0,0 +1,52 @@
|
||||||
|
{
|
||||||
|
"mappings": {
|
||||||
|
"message": {
|
||||||
|
"properties": {
|
||||||
|
"Logger": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"programname": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"Hostname": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"deployment_mode": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"fuel_environment": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"os_region": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"os_release": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"request_id": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"tenant_id": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"user_id": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"settings": {
|
||||||
|
"number_of_shards": 3
|
||||||
|
},
|
||||||
|
"template": "log-*"
|
||||||
|
}
|
|
@ -0,0 +1,52 @@
|
||||||
|
{
|
||||||
|
"mappings": {
|
||||||
|
"message": {
|
||||||
|
"properties": {
|
||||||
|
"Logger": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"Hostname": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"fuel_environment": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"os_region": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"os_release": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"event_type": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"instance_id": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"instance_name": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"tenant_id": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"user_id": {
|
||||||
|
"index": "not_analyzed",
|
||||||
|
"type": "string"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"settings": {
|
||||||
|
"number_of_shards": 3
|
||||||
|
},
|
||||||
|
"template": "notification-*"
|
||||||
|
}
|
|
@ -0,0 +1,91 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# Copyright 2015 Mirantis, Inc.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
#
|
||||||
|
# Start a container running ElasticSearch
|
||||||
|
#
|
||||||
|
# You can modify the following environment variables to tweak the configuration
|
||||||
|
# - ES_DATA: directory where to store the ES data and logs (default=~/es_volume)
|
||||||
|
# - ES_MEMORY: amount of memory (in Gb) allocated to the JVM (default=16)
|
||||||
|
# - ES_LISTEN_ADDRESS: listen address on the container host (default=127.0.0.1)
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
CURRENT_DIR=$(dirname $(readlink -f $0))
|
||||||
|
[[ -f ${CURRENT_DIR}/../common/functions.sh ]] && . ${CURRENT_DIR}/../common/functions.sh
|
||||||
|
|
||||||
|
DOCKER_NAME=elasticsearch
|
||||||
|
DOCKER_IMAGE="dockerfile/elasticsearch"
|
||||||
|
RUN_TIMEOUT=60
|
||||||
|
ES_DATA=${ES_DATA:-~/es_volume}
|
||||||
|
ES_MEMORY=${ES_MEMORY:-16}
|
||||||
|
# We don't want to accidently expose the ES ports on the Internet so we only
|
||||||
|
# publish ports on the host's loopback address. To access ElasticSearch from
|
||||||
|
# another host, you can SSH to the Docker host with port forwarding (eg
|
||||||
|
# '-L 9200:127.0.0.1:9200').
|
||||||
|
# Or you can override the ES_LISTEN_ADDRESS variable...
|
||||||
|
ES_LISTEN_ADDRESS=${ES_LISTEN_ADDRESS:-127.0.0.1}
|
||||||
|
ES_HTTP_PORT=${ES_HTTP_PORT:-9200}
|
||||||
|
ES_TRANSPORT_PORT=${ES_TRANSPORT_PORT:-9300}
|
||||||
|
ES_URL="http://${ES_LISTEN_ADDRESS}:${ES_HTTP_PORT}"
|
||||||
|
|
||||||
|
if [[ "$(docker_get_id $DOCKER_NAME)" != "" ]]; then
|
||||||
|
echo "Docker container '${DOCKER_NAME}' already exists! Please remove it first."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -d $ES_DATA ]]; then
|
||||||
|
mkdir $ES_DATA
|
||||||
|
elif [[ -f ${ES_DATA}/elasticsearch.yml ]]; then
|
||||||
|
echo "Warning: ${ES_DATA}/elasticsearch.yaml already exists."
|
||||||
|
fi
|
||||||
|
|
||||||
|
cat <<EOF > $ES_DATA/elasticsearch.yml
|
||||||
|
#cluster.name: elastic_lma
|
||||||
|
node.name: $(hostname)
|
||||||
|
bootstrap.mlockall: true
|
||||||
|
path:
|
||||||
|
logs: /data/log
|
||||||
|
data: /data/data
|
||||||
|
# This is required for Kibana 3.x
|
||||||
|
http.cors.enabled: true
|
||||||
|
EOF
|
||||||
|
|
||||||
|
docker_pull_image ${DOCKER_IMAGE}
|
||||||
|
|
||||||
|
DOCKER_ID=$(timeout $RUN_TIMEOUT docker run -d -e ES_HEAP_SIZE=${ES_MEMORY}g -p ${ES_LISTEN_ADDRESS}:${ES_HTTP_PORT}:9200 -p ${ES_LISTEN_ADDRESS}:${ES_TRANSPORT_PORT}:9300 --name ${DOCKER_NAME} -v $ES_DATA:/data ${DOCKER_IMAGE} /elasticsearch/bin/elasticsearch -Des.config=/data/elasticsearch.yml)
|
||||||
|
SHORT_ID=$(docker_shorten_id $DOCKER_ID)
|
||||||
|
|
||||||
|
echo -n "Waiting for ElasticSearch to start"
|
||||||
|
while ! curl http://${ES_LISTEN_ADDRESS}:${ES_HTTP_PORT} 1>/dev/null 2>&1; do
|
||||||
|
echo -n '.'
|
||||||
|
IS_RUNNING=$(docker inspect --format="{{ .State.Running }}" ${DOCKER_ID})
|
||||||
|
if [[ "${IS_RUNNING}" == "false" ]]; then
|
||||||
|
echo ''
|
||||||
|
echo "Container '${DOCKER_NAME}/${SHORT_ID}' failed to start!"
|
||||||
|
docker logs $DOCKER_ID
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
echo
|
||||||
|
|
||||||
|
echo "Container '${DOCKER_NAME}/${SHORT_ID}' started successfully"
|
||||||
|
|
||||||
|
# Configure template for 'log-*' and 'notification-*' indices
|
||||||
|
curl -s -XDELETE ${ES_URL}/_template/log 1>/dev/null
|
||||||
|
curl -s -XPUT -d @log_index_template.json ${ES_URL}/_template/log 1>/dev/null
|
||||||
|
curl -s -XPUT -d @notification_index_template.json ${ES_URL}/_template/notification 1>/dev/null
|
||||||
|
|
||||||
|
echo "ElasticSearch API avaiable at ${ES_URL}"
|
|
@ -0,0 +1,30 @@
|
||||||
|
# Description
|
||||||
|
|
||||||
|
Scripts and tools for running an InfluxDB server to be used with the LMA
|
||||||
|
toolchain.
|
||||||
|
|
||||||
|
# Running
|
||||||
|
|
||||||
|
Simply:
|
||||||
|
|
||||||
|
```
|
||||||
|
$ run_container.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Use environment variables to override the default configuration:
|
||||||
|
|
||||||
|
```
|
||||||
|
$ LISTEN_ADDRESS=192.169.0.1 run_container.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Supported environment variables for configuration:
|
||||||
|
|
||||||
|
* LISTEN_ADDRESS: listen address on the container host (default=127.0.0.1)
|
||||||
|
|
||||||
|
* LMA_DB: name of the LMA database (default=lma)
|
||||||
|
|
||||||
|
* LMA_USER: username for the LMA db (default=lma)
|
||||||
|
|
||||||
|
* LMA_PASSWORD: password for the LMA user (default=lmapass)
|
||||||
|
|
||||||
|
* ROOT_PASSWORD: password for the admin user (default=supersecret)
|
|
@ -0,0 +1,75 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# Copyright 2015 Mirantis, Inc.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
#
|
||||||
|
# Start a container running InfluxDB
|
||||||
|
#
|
||||||
|
# You can modify the following environment variables to tweak the configuration
|
||||||
|
# - LISTEN_ADDRESS: listen address on the container host (default=127.0.0.1)
|
||||||
|
# - LMA_DB: name of the LMA database (default=lma)
|
||||||
|
# - LMA_USER: username for the LMA db (default=lma)
|
||||||
|
# - LMA_PASSWORD: password for the LMA user (default=lmapass)
|
||||||
|
# - ROOT_PASSWORD: password for the admin user (default=supersecret)
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
CURRENT_DIR=$(dirname $(readlink -f $0))
|
||||||
|
[[ -f ${CURRENT_DIR}/../common/functions.sh ]] && . ${CURRENT_DIR}/../common/functions.sh
|
||||||
|
|
||||||
|
DOCKER_NAME=influxdb
|
||||||
|
DOCKER_IMAGE="tutum/influxdb"
|
||||||
|
RUN_TIMEOUT=30
|
||||||
|
INFLUXDB_LISTEN_ADDRESS=${LISTEN_ADDRESS:-127.0.0.1}
|
||||||
|
INFLUXDB_GRAFANA_DB=grafana
|
||||||
|
INFLUXDB_LMA_DB=${LMA_DB:-lma}
|
||||||
|
INFLUXDB_LMA_USER=${LMA_USER:-lma}
|
||||||
|
INFLUXDB_LMA_PASSWORD=${LMA_PASSWORD:-lmapass}
|
||||||
|
INFLUXDB_ROOT_PASSWORD=${ROOT_PASSWORD:-supersecret}
|
||||||
|
INFLUXDB_URL="http://${INFLUXDB_LISTEN_ADDRESS}:8086"
|
||||||
|
|
||||||
|
if [[ "$(docker_get_id $DOCKER_NAME)" != "" ]]; then
|
||||||
|
echo "Docker container '${DOCKER_NAME}' already exists! Please remove it first."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
docker_pull_image $DOCKER_IMAGE
|
||||||
|
|
||||||
|
DOCKER_ID=$(timeout $RUN_TIMEOUT docker run -d -p ${INFLUXDB_LISTEN_ADDRESS}:8083:8083 -p ${INFLUXDB_LISTEN_ADDRESS}:8086:8086 --expose 8090 --expose 8099 --name ${DOCKER_NAME} ${DOCKER_IMAGE})
|
||||||
|
SHORT_ID=$(docker_shorten_id $DOCKER_ID)
|
||||||
|
|
||||||
|
echo -n "Waiting for InfluxDB to be up"
|
||||||
|
while ! curl ${INFLUXDB_URL} 1>/dev/null 2>&1; do
|
||||||
|
echo -n '.'
|
||||||
|
if ! docker_is_running $DOCKER_ID; then
|
||||||
|
echo ''
|
||||||
|
echo "Container '${DOCKER_NAME}/${SHORT_ID}' failed to start!"
|
||||||
|
docker logs $DOCKER_ID
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
echo
|
||||||
|
echo "Container '${DOCKER_NAME}/${SHORT_ID}' started successfully"
|
||||||
|
|
||||||
|
curl -X POST "${INFLUXDB_URL}/cluster_admins/root?u=root&p=root" -d '{"password": "'${INFLUXDB_ROOT_PASSWORD}'"}'
|
||||||
|
|
||||||
|
curl -X POST "${INFLUXDB_URL}/db?u=root&p=${INFLUXDB_ROOT_PASSWORD}" -d '{"name": "'${INFLUXDB_LMA_DB}'"}'
|
||||||
|
curl -X POST "${INFLUXDB_URL}/db/${INFLUXDB_LMA_DB}/users?u=root&p=${INFLUXDB_ROOT_PASSWORD}" -d '{"name": "'${INFLUXDB_LMA_USER}'", "password": "'${INFLUXDB_LMA_PASSWORD}'"}'
|
||||||
|
echo "InfluxDB provisioned with db=${INFLUXDB_LMA_DB}, user=${INFLUXDB_LMA_USER}, pass=${INFLUXDB_LMA_PASSWORD}"
|
||||||
|
|
||||||
|
curl -X POST "${INFLUXDB_URL}/db?u=root&p=${INFLUXDB_ROOT_PASSWORD}" -d '{"name": "'${INFLUXDB_GRAFANA_DB}'"}'
|
||||||
|
curl -X POST "${INFLUXDB_URL}/db/${INFLUXDB_GRAFANA_DB}/users?u=root&p=${INFLUXDB_ROOT_PASSWORD}" -d '{"name": "'${INFLUXDB_LMA_USER}'", "password": "'${INFLUXDB_LMA_PASSWORD}'"}'
|
||||||
|
echo "InfluxDB provisioned with db=${INFLUXDB_GRAFANA_DB}, user=${INFLUXDB_LMA_USER}, pass=${INFLUXDB_LMA_PASSWORD}"
|
||||||
|
|
||||||
|
echo "InfluxDB API available at ${INFLUXDB_URL}"
|
|
@ -0,0 +1,146 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# Copyright 2015 Mirantis, Inc.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
#
|
||||||
|
# This script is intended to run on the Fuel master node. It will deploy all
|
||||||
|
# the pieces needed to run the LMA collector.
|
||||||
|
#
|
||||||
|
# It performs the following operations:
|
||||||
|
# * Build the LMA collector plugin.
|
||||||
|
# * Install the LMA collector plugin.
|
||||||
|
# * Deploy an ElasticSearch container
|
||||||
|
# * Deploy an InfluxDB container
|
||||||
|
# * Deploy a container for running the LMA dashboards.
|
||||||
|
#
|
||||||
|
# TODO: add script parameters to provide LMA password
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
function fail {
|
||||||
|
echo $1
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
function info {
|
||||||
|
echo $1
|
||||||
|
}
|
||||||
|
|
||||||
|
# Run pre-installation checks
|
||||||
|
if [[ "$(id -u)" != "0" ]]; then
|
||||||
|
fail "This script needs to be run as root."
|
||||||
|
fi
|
||||||
|
|
||||||
|
if ! which docker 1>/dev/null; then
|
||||||
|
fail "Couldn't find the docker binary."
|
||||||
|
fi
|
||||||
|
|
||||||
|
FUEL_VERSION_FILE="/etc/fuel/version.yaml"
|
||||||
|
if [ ! -f ${FUEL_VERSION_FILE} ]; then
|
||||||
|
fail "This script needs to be run on the Fuel node."
|
||||||
|
fi
|
||||||
|
|
||||||
|
cat <<EOF | python -
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
release_re = "6\."
|
||||||
|
ok = False
|
||||||
|
|
||||||
|
with open("/etc/fuel/version.yaml") as f:
|
||||||
|
data = yaml.load(f.read())
|
||||||
|
ok = 'VERSION' in data and \
|
||||||
|
'release' in data['VERSION'] and \
|
||||||
|
re.match(release_re, data['VERSION']['release'])
|
||||||
|
if ok:
|
||||||
|
sys.exit(0)
|
||||||
|
else:
|
||||||
|
print "Fuel version not supported."
|
||||||
|
sys.exit(1)
|
||||||
|
EOF
|
||||||
|
|
||||||
|
CURRENT_DIR=$(dirname $(readlink -f $0))
|
||||||
|
|
||||||
|
UI_PORT=8081
|
||||||
|
PRIMARY_IP_ADDRESS=$(hostname --ip-address)
|
||||||
|
ES_DIR=/var/elasticsearch
|
||||||
|
MIN_MEM=$(( 1 * 1024 ))
|
||||||
|
MAX_MEM=$(( 32 * 1024 ))
|
||||||
|
|
||||||
|
FREE_MEM=$(free -m | egrep '^Mem:' | awk '{print $4}')
|
||||||
|
|
||||||
|
if [[ "$FREE_MEM" == "" ]]; then
|
||||||
|
fail "Couldn't determine the amount of free RAM."
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ $FREE_MEM -lt $MIN_MEM ]]; then
|
||||||
|
fail "Need at least ${MIN_MEM}MB of free RAM while only ${FREE_MEM}MB are available."
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Don't eat up all the free memory unless it is absolutely necessary
|
||||||
|
if [[ $FREE_MEM -gt $(( 2 * $MIN_MEM )) ]]; then
|
||||||
|
FREE_MEM=$(( $FREE_MEM - $MIN_MEM ))
|
||||||
|
fi
|
||||||
|
|
||||||
|
# There is no point of allocating more than 32GB of RAM to the JVM
|
||||||
|
if [[ $FREE_MEM -gt $MAX_MEM ]]; then
|
||||||
|
FREE_MEM=$MAX_MEM
|
||||||
|
fi
|
||||||
|
ES_MEMORY=$(( $FREE_MEM / 1024 ))
|
||||||
|
|
||||||
|
info "Starting the installation of the LMA collector plugin..."
|
||||||
|
|
||||||
|
yum install -y python-pip
|
||||||
|
pip install -U fuel-plugin-builder
|
||||||
|
if ! rpm -q epel-release-6-8.noarch; then
|
||||||
|
rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
|
||||||
|
fi
|
||||||
|
yum install -y createrepo rpm dpkg-devel
|
||||||
|
|
||||||
|
info "Building the Fuel plugin..."
|
||||||
|
rm -f ../lma_collector*fp
|
||||||
|
if ! (cd ${CURRENT_DIR}/.. && fuel-plugin-builder --build ./); then
|
||||||
|
fail "Failed to build the Fuel plugin."
|
||||||
|
fi
|
||||||
|
|
||||||
|
info "Installing the Fuel plugin..."
|
||||||
|
if ! (cd ${CURRENT_DIR}/.. && fuel plugins --force --install lma_collector*.fp); then
|
||||||
|
fail "Failed to install the Fuel plugin."
|
||||||
|
fi
|
||||||
|
|
||||||
|
info "Building the documentation"
|
||||||
|
pip install Sphinx
|
||||||
|
if ! (cd ${CURRENT_DIR}/../doc && make html); then
|
||||||
|
info "Couldn't build the documentation."
|
||||||
|
fi
|
||||||
|
|
||||||
|
info "Starting the ElasticSearch container..."
|
||||||
|
mkdir -p $ES_DIR
|
||||||
|
if ! (cd ${CURRENT_DIR}/elasticsearch && ES_MEMORY=$ES_MEMORY ES_DATA=$ES_DIR ES_LISTEN_ADDRESS=$PRIMARY_IP_ADDRESS ./run_container.sh); then
|
||||||
|
fail "Failed to start the ElasticSearch container."
|
||||||
|
fi
|
||||||
|
|
||||||
|
info "Starting the InfluxDB container..."
|
||||||
|
if ! (cd ${CURRENT_DIR}/influxdb && LISTEN_ADDRESS=$PRIMARY_IP_ADDRESS ./run_container.sh); then
|
||||||
|
fail "Failed to start the InfluxDB container."
|
||||||
|
fi
|
||||||
|
|
||||||
|
info "Starting the LMA UI container..."
|
||||||
|
if ! (cd ${CURRENT_DIR}/ui && docker build -t lma_ui . && docker run -d -p ${UI_PORT}:80 --name lma_ui lma_ui); then
|
||||||
|
fail "Failed to start the LMA UI container."
|
||||||
|
fi
|
||||||
|
info "Kibana dashboard available at http://${PRIMARY_IP_ADDRESS}:${UI_PORT}/kibana/"
|
||||||
|
info "Grafana dashboard available at http://${PRIMARY_IP_ADDRESS}:${UI_PORT}/grafana/"
|
||||||
|
|
||||||
|
info "The LMA collector storage and dashboard services are ready."
|
|
@ -0,0 +1,12 @@
|
||||||
|
FROM nginx:latest
|
||||||
|
|
||||||
|
ADD https://download.elasticsearch.org/kibana/kibana/kibana-3.1.2.tar.gz /tmp/kibana.tar.gz
|
||||||
|
ADD http://grafanarel.s3.amazonaws.com/grafana-1.9.1.tar.gz /tmp/grafana.tar.gz
|
||||||
|
ADD run.sh /usr/local/bin/run
|
||||||
|
|
||||||
|
RUN tar zxf /tmp/kibana.tar.gz && mv kibana-3.1.2 /usr/share/nginx/html/kibana && rm -rf /tmp/kibana*
|
||||||
|
RUN tar zxvf /tmp/grafana.tar.gz && mv grafana-1.9.1 /usr/share/nginx/html/grafana && rm -rf /tmp/grafana*
|
||||||
|
|
||||||
|
EXPOSE 80
|
||||||
|
|
||||||
|
CMD ["/usr/local/bin/run"]
|
|
@ -0,0 +1,52 @@
|
||||||
|
# LMA user interface
|
||||||
|
|
||||||
|
Docker container for running the LMA dashboards (Kibana and Grafana).
|
||||||
|
|
||||||
|
## Build the image
|
||||||
|
|
||||||
|
From this directory:
|
||||||
|
|
||||||
|
`docker build -t lma_ui .`
|
||||||
|
|
||||||
|
## Run the image
|
||||||
|
|
||||||
|
`docker run -d -p 80:80 --name lma_ui lma_ui`
|
||||||
|
|
||||||
|
You can pass environment variables to the ``docker run`` command to override
|
||||||
|
the default parameters:
|
||||||
|
|
||||||
|
* ``KIBANA_ENABLED``, whether or not to enable the Kibana dashboard (default:
|
||||||
|
"yes").
|
||||||
|
|
||||||
|
* ``GRAFANA_ENABLED``, whether or not to enable the Grafana dashboard (default:
|
||||||
|
"yes").
|
||||||
|
|
||||||
|
* ``ES_HOST``, the address of the ElasticSearch server (default: same host).
|
||||||
|
|
||||||
|
* ``INFLUXDB_HOST``, the address of the InfluxDB server (default: same host).
|
||||||
|
|
||||||
|
* ``INFLUXDB_DBNAME``, the name of the InfluxDB database storing the metrics
|
||||||
|
(default: "lma").
|
||||||
|
|
||||||
|
* ``INFLUXDB_USER``, the username for connecting to the InfluxDB databases
|
||||||
|
(default: "lma").
|
||||||
|
|
||||||
|
* ``INFLUXDB_PASS``, the password for connecting to the InfluxDB databases
|
||||||
|
(default: "lmapass").
|
||||||
|
|
||||||
|
If you want to save the Grafana dashboards into InfluxDB, you also need to
|
||||||
|
create a database named 'grafana' on the InfluxDB server. This database needs
|
||||||
|
to be accessible to the InfluxDB LMA user.
|
||||||
|
|
||||||
|
## Accessing the UI
|
||||||
|
|
||||||
|
The dashboards are available at the following URLs:
|
||||||
|
|
||||||
|
* http://<<span></span>container host>:<<span></span>public port>/kibana/
|
||||||
|
* http://<<span></span>container host>:<<span></span>public port>/grafana/
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
If the dashboards fail to display the data or are unresponsive, run the
|
||||||
|
``docker logs lma_ui`` command and check that the ElasticSearch and InfluxDB
|
||||||
|
servers are reachable from the machine running the web browser.
|
|
@ -0,0 +1,44 @@
|
||||||
|
// Sample configuration for the LMA dashboard
|
||||||
|
// Copy this file to your Grafana source directory
|
||||||
|
define(['settings'], function(Settings) {
|
||||||
|
|
||||||
|
return new Settings({
|
||||||
|
datasources: {
|
||||||
|
influxdb: {
|
||||||
|
type: 'influxdb',
|
||||||
|
url: "http://" + window.location.hostname + ":8086/db/lma",
|
||||||
|
username: 'lma',
|
||||||
|
password: 'lmapass',
|
||||||
|
},
|
||||||
|
grafana: {
|
||||||
|
type: 'influxdb',
|
||||||
|
url: "http://" + window.location.hostname + ":8086/db/grafana",
|
||||||
|
username: 'lma',
|
||||||
|
password: 'lmapass',
|
||||||
|
grafanaDB: true
|
||||||
|
},
|
||||||
|
},
|
||||||
|
|
||||||
|
search: {
|
||||||
|
max_results: 100
|
||||||
|
},
|
||||||
|
|
||||||
|
default_route: '/dashboard/file/default.json',
|
||||||
|
|
||||||
|
unsaved_changes_warning: true,
|
||||||
|
|
||||||
|
playlist_timespan: "1m",
|
||||||
|
|
||||||
|
admin: {
|
||||||
|
password: ''
|
||||||
|
},
|
||||||
|
|
||||||
|
window_title_prefix: 'LMA - ',
|
||||||
|
|
||||||
|
plugins: {
|
||||||
|
panels: [],
|
||||||
|
dependencies: [],
|
||||||
|
}
|
||||||
|
|
||||||
|
});
|
||||||
|
});
|
|
@ -0,0 +1,27 @@
|
||||||
|
// Sample configuration for the LMA dashboard
|
||||||
|
// Copy this file to your Kibana source directory
|
||||||
|
define(['settings'],
|
||||||
|
function (Settings) {
|
||||||
|
return new Settings({
|
||||||
|
elasticsearch: "http://" + window.location.hostname +":9200",
|
||||||
|
default_route : '/dashboard/file/default.json',
|
||||||
|
kibana_index: "kibana-int",
|
||||||
|
panel_names: [
|
||||||
|
'histogram',
|
||||||
|
'map',
|
||||||
|
'goal',
|
||||||
|
'table',
|
||||||
|
'filtering',
|
||||||
|
'timepicker',
|
||||||
|
'text',
|
||||||
|
'hits',
|
||||||
|
'column',
|
||||||
|
'trends',
|
||||||
|
'bettermap',
|
||||||
|
'query',
|
||||||
|
'terms',
|
||||||
|
'stats',
|
||||||
|
'sparklines'
|
||||||
|
]
|
||||||
|
});
|
||||||
|
});
|
|
@ -0,0 +1,132 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# Copyright 2015 Mirantis, Inc.
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
function convert_yes_no {
|
||||||
|
if [[ ${1^^} == "YES" || ${1^} == "Y" || ${1} == "1" ]]; then
|
||||||
|
echo "yes"
|
||||||
|
else
|
||||||
|
echo "no"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
KIBANA_ENABLED=$(convert_yes_no ${KIBANA_ENABLED:-yes})
|
||||||
|
GRAFANA_ENABLED=$(convert_yes_no ${GRAFANA_ENABLED:-yes})
|
||||||
|
|
||||||
|
if [[ ${KIBANA_ENABLED} == "yes" ]]; then
|
||||||
|
ES_HOST=${ES_HOST:-\"+window.location.hostname+\"}
|
||||||
|
ES_PORT=9200
|
||||||
|
ES_URL="http://${ES_HOST}:${ES_PORT}"
|
||||||
|
echo "ElasticSearch URL is ${ES_URL}"
|
||||||
|
|
||||||
|
cat <<EOF > /usr/share/nginx/html/kibana/config.js
|
||||||
|
define(['settings'],
|
||||||
|
function (Settings) {
|
||||||
|
return new Settings({
|
||||||
|
elasticsearch: "${ES_URL}",
|
||||||
|
default_route : '/dashboard/file/log.json',
|
||||||
|
kibana_index: "kibana-int",
|
||||||
|
panel_names: [
|
||||||
|
'histogram',
|
||||||
|
'map',
|
||||||
|
'goal',
|
||||||
|
'table',
|
||||||
|
'filtering',
|
||||||
|
'timepicker',
|
||||||
|
'text',
|
||||||
|
'hits',
|
||||||
|
'column',
|
||||||
|
'trends',
|
||||||
|
'bettermap',
|
||||||
|
'query',
|
||||||
|
'terms',
|
||||||
|
'stats',
|
||||||
|
'sparklines'
|
||||||
|
]
|
||||||
|
});
|
||||||
|
});
|
||||||
|
EOF
|
||||||
|
else
|
||||||
|
echo "Kibana dashboard is disabled."
|
||||||
|
rm -rf /usr/share/nginx/html/kibana
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ${GRAFANA_ENABLED} == "yes" ]]; then
|
||||||
|
INFLUXDB_HOST=${INFLUXDB_HOST:-\"+window.location.hostname+\"}
|
||||||
|
INFLUXDB_PORT=8086
|
||||||
|
INFLUXDB_DBNAME=${INFLUXDB_DBNAME:-lma}
|
||||||
|
INFLUXDB_USER=${INFLUXDB_USER:-lma}
|
||||||
|
INFLUXDB_PASS=${INFLUXDB_PASS:-lmapass}
|
||||||
|
INFLUXDB_URL="http://${INFLUXDB_HOST}:${INFLUXDB_PORT}"
|
||||||
|
echo "InfluxDB URL is ${INFLUXDB_URL}"
|
||||||
|
echo "InfluxDB database for metrics is ${INFLUXDB_DBNAME}"
|
||||||
|
|
||||||
|
cat <<EOF > /usr/share/nginx/html/grafana/config.js
|
||||||
|
define(['settings'], function(Settings) {
|
||||||
|
|
||||||
|
return new Settings({
|
||||||
|
datasources: {
|
||||||
|
lma: {
|
||||||
|
type: 'influxdb',
|
||||||
|
url: "${INFLUXDB_URL}/db/${INFLUXDB_DBNAME}",
|
||||||
|
username: "${INFLUXDB_USER}",
|
||||||
|
password: "${INFLUXDB_PASS}"
|
||||||
|
},
|
||||||
|
grafana: {
|
||||||
|
type: 'influxdb',
|
||||||
|
url: "${INFLUXDB_URL}/db/grafana",
|
||||||
|
username: "${INFLUXDB_USER}",
|
||||||
|
password: "${INFLUXDB_PASS}",
|
||||||
|
grafanaDB: true
|
||||||
|
},
|
||||||
|
},
|
||||||
|
|
||||||
|
search: {
|
||||||
|
max_results: 100
|
||||||
|
},
|
||||||
|
|
||||||
|
default_route: '/dashboard/file/lma.json',
|
||||||
|
|
||||||
|
unsaved_changes_warning: true,
|
||||||
|
|
||||||
|
playlist_timespan: "1m",
|
||||||
|
|
||||||
|
admin: {
|
||||||
|
password: ''
|
||||||
|
},
|
||||||
|
|
||||||
|
window_title_prefix: 'LMA - ',
|
||||||
|
|
||||||
|
plugins: {
|
||||||
|
panels: [],
|
||||||
|
dependencies: [],
|
||||||
|
}
|
||||||
|
|
||||||
|
});
|
||||||
|
});
|
||||||
|
EOF
|
||||||
|
else
|
||||||
|
echo "Grafana dashboard is disabled."
|
||||||
|
rm -rf /usr/share/nginx/html/grafana
|
||||||
|
fi
|
||||||
|
|
||||||
|
# disable ipv6 support
|
||||||
|
if [ ! -f /proc/net/if_inet6 ]; then
|
||||||
|
sed -e '/listen \[::\]:80/ s/^#*/#/' -i /etc/nginx/sites-enabled/*
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Starting nginx..."
|
||||||
|
nginx -c /etc/nginx/nginx.conf -g 'daemon off;'
|
Loading…
Reference in New Issue