summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJenkins <jenkins@review.openstack.org>2016-07-22 08:02:15 +0000
committerGerrit Code Review <review@openstack.org>2016-07-22 08:02:15 +0000
commitbb6c1122089dfc9d7bee53af502aba6e12424219 (patch)
tree235eea754900f1593a49080d093ec32c72849d6a
parentf20a5e1c7e1c6a98d2ee4b8d9c9c311663bc5dfd (diff)
parentdd402357e349870872a3da022eb066d53879c997 (diff)
Merge "[docs] Restructuring Kibana 0.10.0 User guide"
-rw-r--r--doc/source/advanced_operations.rst128
-rw-r--r--doc/source/appendix.rst8
-rw-r--r--doc/source/conf.py4
-rw-r--r--doc/source/configure.rst138
-rw-r--r--doc/source/definitions.rst21
-rw-r--r--doc/source/deploy_with_network_templates.rst46
-rw-r--r--doc/source/index.rst48
-rw-r--r--doc/source/install.rst146
-rw-r--r--doc/source/installation.rst100
-rw-r--r--doc/source/intro.rst20
-rw-r--r--doc/source/licenses.rst18
-rw-r--r--doc/source/limitations.rst16
-rw-r--r--doc/source/overview.rst73
-rw-r--r--doc/source/references.rst8
-rw-r--r--doc/source/release_notes.rst54
-rw-r--r--doc/source/releases.rst46
-rw-r--r--doc/source/requirements.rst35
-rw-r--r--doc/source/troubleshooting.rst66
-rw-r--r--doc/source/user.rst357
-rw-r--r--doc/source/verify.rst100
20 files changed, 794 insertions, 638 deletions
diff --git a/doc/source/advanced_operations.rst b/doc/source/advanced_operations.rst
index 1c488d6..6e5ff30 100644
--- a/doc/source/advanced_operations.rst
+++ b/doc/source/advanced_operations.rst
@@ -1,59 +1,67 @@
1.. _advanced_user_guide: 1.. _advanced_user_guide:
2 2
3Advanced Operations 3Advanced operations
4=================== 4===================
5 5
6 .. _cluster_operations: 6This section describes advanced operations that you can apply to your
7Elasticsearch cluster using the StackLight Elasticsearch-Kibana Fuel plugin.
7 8
8Cluster Operations 9.. _cluster_operations:
10
11Cluster operations
9------------------ 12------------------
10 13
11Because of certain limitations in the current implementation of the Fuel plugin, it is necessary to 14Because of limitations in the current implementation of the plugin, manual
12perform some manual operations after the Elasticsearch cluster is scaled up or scaled down. 15operations are required after the Elasticsearch cluster is scaled up or scaled down. Using these operations, you can adjust the replication factor of the
13Those operations are needed to adjust the replication factor of the Elasticsearch indices, 16Elasticsearch indices that are based on the new number of nodes on the cluster.
14based on the new number of nodes in the cluster.
15There are three types of indices used by the plugin:
16 17
17 * The log indices named *log-%{+YYYY.MM.dd}* which are created on a daily basis. 18The plugin uses three types of indices:
18 * The notification indices named *notification-%{+YYYY.MM.dd}* which are also created on a daily
19 basis.
20 * The Kibana index named *kibana-int* which is created once at installation time to store the
21 templates of the Kibana dashboards.
22 19
23Adjusting the replication factor for the *kibana-int* index is 20* The log indices named *log-%{+YYYY.MM.dd}* which are created on a daily basis.
24performed automatically by the plugin and therefore there is no need to do any manual operation 21* The notification indices named *notification-%{+YYYY.MM.dd}* which are
25for that index when the cluster is scaled up or down. 22 created on a daily basis.
23* The Kibana index named *kibana-int* which is created once during the
24 installation to store the templates of the Kibana dashboards.
26 25
27This is not the case for the replication factor of the other two indices which needs to 26Adjusting the replication factor for the *kibana-int* index is performed
28be updated manualy as described in the 27automatically by the plugin. Therefore, no manual operation is not required
29`official documentation <https://www.elastic.co/guide/en/elasticsearch/reference/1.7/indices-update-settings.html>`_. 28for this index when the cluster is scaled up or down. But this is not the case
29for the replication factor of other two indices that you should manually
30update as described in the
31`Elasticsearch official documentation <https://www.elastic.co/guide/en/elasticsearch/reference/1.7/indices-update-settings.html>`_.
30 32
31The following sections provide more details, describing what do when scaling up/down the 33The following sections describe in detail how to scale up and scale down the
32Elasticsearch cluster. Scaling up from one node to three nodes, and scaling down from three nodes 34Elasticsearch cluster. Scaling up from one node to three nodes and scaling
33to one node, are used as examples. Your mileage may vary but the 35down from three nodes to one node are used as examples. Your mileage may vary,
34principal of (re)configuring the replication factor of the indices should remain the same. 36but the principal of (re)configuring the replication factor of the indices
37should remain the same.
35 38
36Scaling Up 39Scaling Up
37^^^^^^^^^^ 40~~~~~~~~~~
41
42The problem that the manual operation aims to address is that the replication
43factor for the old indices is not updated automatically by the plugin when
44a new node is added in the cluster. If you want the old indices to be
45replicated on the new node(s), you need to adjust the *number_of_replicas*
46parameter to the current size of the cluster for those indices as shown below.
38 47
39The problem the manual operation aims to address is that the replication factor for the old 48The output below shows that the replication factor of the indices created
40indices is not updated automatically by the plugin when a new node is added in the cluster. If you 49before the scale-up is zero. Here, a scale-up was performed on the 3rd of
41want the old indices to be replicated on the new node(s), you need to adjust the 50February, so the indices created after that date (*log-2016.02.04* in this
42*number_of_replicas* parameter to the current size of the cluster for those indices as shown below. 51example) are automatically updated with the correct number of replicas
52(number of cluster nodes - 1).
43 53
44The output below shows that the replication factor of the indices created before the scale-up is 54.. code-block:: console
45zero. Here, a scale-up was performed on the 3rd of February, so the indices created after that date
46(*log-2016.02.04* here) are automatically updated with the correct number of replicas
47(number of cluster nodes - 1). ::
48 55
49 [root@node-1 ~]# curl <VIP>:9200/_cat/indices?v 56 [root@node-1 ~]# curl <VIP>:9200/_cat/indices?v
50 health status index pri rep docs.count docs.deleted .... 57 health status index pri rep docs.count docs.deleted ....
51 green open log-2016.02.03 5 0 270405 0 .... 58 green open log-2016.02.03 5 0 270405 0 ....
52 green open log-2016.02.04 5 2 1934581 0 .... 59 green open log-2016.02.04 5 2 1934581 0 ....
53 60
61If you want the *log-2016.02.03* index to be replicated, update the
62*number_of_replicas* parameter of that index as shown below:
54 63
55Then, if you want the *log-2016.02.03* index to be replicated, you need to update the 64.. code-block:: console
56*number_of_replicas* parameter of that index as shown below::
57 65
58 [root@node-1 ~]# curl -XPUT <VIP>:9200/log-2016.02.03/_settings \ 66 [root@node-1 ~]# curl -XPUT <VIP>:9200/log-2016.02.03/_settings \
59 -d '{ "index": {"number_of_replicas": 2}}' 67 -d '{ "index": {"number_of_replicas": 2}}'
@@ -64,16 +72,17 @@ Then, if you want the *log-2016.02.03* index to be replicated, you need to updat
64 green open log-2016.02.03 5 2 270405 0 .... 72 green open log-2016.02.03 5 2 270405 0 ....
65 green open log-2016.02.04 5 2 1934581 0 .... 73 green open log-2016.02.04 5 2 1934581 0 ....
66 74
67 75.. caution:: Replicating the old indices on the new node(s) will increase the
68Note that replicating the old indices on the new node(s) will increase the load on the 76 load on the cluster as well as the size required to store the data.
69cluster as well as the size required to store the data.
70 77
71Scaling down 78Scaling down
72^^^^^^^^^^^^ 79~~~~~~~~~~~~
73 80
74Similarly, after a scale-down the *number_of_replicas* of all indices must be 81Similarly, after a scale-down the *number_of_replicas* of all indices must be
75aligned with the new size of the cluster. Not doing so will be reported by LMA as a critical 82aligned with the new size of the cluster. Not doing so will be reported by LMA
76status for the Elasticsearch cluster:: 83as a critical status for the Elasticsearch cluster:
84
85.. code-block:: console
77 86
78 [root@node-1 ~]# # the current index health is 'red' after the scale-down 87 [root@node-1 ~]# # the current index health is 'red' after the scale-down
79 [root@node-1 ~]# curl <VIP>:9200/_cat/indices?v 88 [root@node-1 ~]# curl <VIP>:9200/_cat/indices?v
@@ -88,42 +97,3 @@ status for the Elasticsearch cluster::
88 [root@node-1 ~]# curl <VIP>:9200/_cat/indices?v 97 [root@node-1 ~]# curl <VIP>:9200/_cat/indices?v
89 health status index pri rep docs.count .... 98 health status index pri rep docs.count ....
90 green open log-2016.02.04 5 2 1934581 .... 99 green open log-2016.02.04 5 2 1934581 ....
91
92.. _network_templates:
93
94Deployment using network templates
95----------------------------------
96
97By default, the Elasticsearch-Kibana cluster will be deployed on the Fuel management
98network. If this default configuration doesn't meet your requirements, you can leverage the
99Fuel `network templates
100<https://docs.mirantis.com/openstack/fuel/fuel-8.0/operations.html#using-networking-templates>`_
101capability to change that default configuration to use a dedicated network instead.
102
103Below is a network template example to define a new network named `monitoring`.
104
105.. literalinclude:: ./network_template.yaml
106
107You can use this configuration example as a starting point and adapt it to your requirements.
108
109The deployment of the environment should work as described in the :ref:`User Guide
110<user_guide>` excepted that before deploying the environment, you will have to:
111
112* Upload the network template::
113
114 $ fuel2 network-template upload -f ./network_template <ENVIRONMENT_ID>
115
116* Allocate an IP subnet for the `monitoring` network::
117
118 $ fuel2 network-group create -N <ENVIRONMENT_ID> -C 10.109.5.0/24 monitoring
119
120* Adjust the IP range through the Fuel user interface (optional).
121
122 .. image:: ../images/network_group_configuration.png
123 :width: 800
124 :align: center
125
126* Deploy the environment.
127
128For more details, please refer to the official `Fuel documentation
129<https://docs.mirantis.com/openstack/fuel/fuel-8.0/operations.html#using-networking-templates>`_.
diff --git a/doc/source/appendix.rst b/doc/source/appendix.rst
deleted file mode 100644
index f46fbcc..0000000
--- a/doc/source/appendix.rst
+++ /dev/null
@@ -1,8 +0,0 @@
1.. _user_appendix:
2
3Appendix
4========
5
6* The `Elasticsearch-Kibana plugin <https://github.com/openstack/fuel-plugin-elasticsearch-kibana>`_ project at GitHub.
7* The official `Kibana documentation <https://www.elastic.co/guide/en/kibana/3.0/index.html>`_.
8* The official `Elasticsearch documentation <https://www.elastic.co/guide/en/elasticsearch/reference/1.4/index.html>`_.
diff --git a/doc/source/conf.py b/doc/source/conf.py
index 8de7831..27fdbc9 100644
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -6,7 +6,7 @@ source_suffix = '.rst'
6master_doc = 'index' 6master_doc = 'index'
7 7
8project = u'The StackLight Elasticsearch-Kibana Plugin for Fuel' 8project = u'The StackLight Elasticsearch-Kibana Plugin for Fuel'
9copyright = u'2015, Mirantis Inc.' 9copyright = u'2016, Mirantis Inc.'
10 10
11version = '0.10' 11version = '0.10'
12release = '0.10.0' 12release = '0.10.0'
@@ -19,7 +19,7 @@ html_theme = 'default'
19html_static_path = ['_static'] 19html_static_path = ['_static']
20 20
21latex_documents = [ 21latex_documents = [
22 ('index', 'ElasticsearchKibana.tex', u'The StackLight Elasticsearch-Kibana Plugin for Fuel Documentation', 22 ('index', 'ElasticsearchKibana.tex', u'Fuel StackLight Elasticsearch-Kibana Plugin Guide',
23 u'Mirantis Inc.', 'manual'), 23 u'Mirantis Inc.', 'manual'),
24] 24]
25 25
diff --git a/doc/source/configure.rst b/doc/source/configure.rst
new file mode 100644
index 0000000..ef7776b
--- /dev/null
+++ b/doc/source/configure.rst
@@ -0,0 +1,138 @@
1.. _plugin_configuration:
2
3Configure the plugin during an environment deployment
4=====================================================
5
6To configure the StackLight Elasticsearch-Kibana plugin during an environment
7deployment:
8
9#. Using the Fuel web UI,
10 `Create a new environment <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/create-environment/start-create-env.html>`_.
11
12#. In the Fuel web UI, click the :guilabel:`Settings` tab and select the
13 :guilabel:`Other` category.
14
15#. Scroll down through the settings to find the
16 :guilabel:`StackLight Elasticsearch-Kibana Plugin` section.
17
18#. Select :guilabel:`StackLight Infrastructure Alerting Plugin` and fill in
19 the required fields as follows:
20
21 .. image:: ../images/elastic_kibana_settings.png
22 :width: 800
23 :align: center
24
25 #. Specify the number of days to retain your data.
26 #. Specify the :guilabel:`JVM heap size` for Elastisearch. Use the tips
27 below:
28
29 * By default, 1 GB of heap memory is allocated to the Elasticsearch
30 process. This value is enough to run Elasticsearch for local testing
31 only.
32 * To run Elasticsearch in production, allocate minimum 4 GB of memory.
33 But we recommend allocating 50% of the available memory up to 32 GB
34 maximum.
35 * If you set a value greater than the memory size, Elasticsearch will
36 not start.
37 * Reserve enough memory for operating system and other services.
38
39 #. Select and edit :guilabel:`Advanced settings` if Elasticsearch and
40 Kibana are installed on a cluster of nodes.
41
42#. Select :guilabel:`Enable TLS for Kibana` if you want to encrypt your
43 Kibana credentials (username, password). Then, fill in the required fields
44 as follows:
45
46 .. image:: ../images/tls_settings.png
47 :width: 800
48 :align: center
49
50 #. Specify the DNS name of the Kibana server. This parameter is used
51 to create a link in the Fuel dashboard to the Kibana server.
52 #. Specify the location of a PEM file that contains the certificate
53 and the private key of the Kibana server that will be used in TLS handchecks
54 with the client.
55
56#. If you want to authenticate through LDAP to Kibana, select
57 :guilabel:`Use LDAP for Kibana authentication`. Then, fill in the required
58 fields as follows:
59
60 .. image:: ../images/ldap_auth.png
61 :width: 800
62 :align: center
63
64 #. Select :guilabel:`LDAPS` if you want to enable LDAP authentication
65 over SSL.
66 #. Specify one or several :guilabel:`LDAP servers` addresses separated by
67 space. Those addresses must be accessible from the node where Kibana
68 is installed.
69 The addresses that are external to the *management network* are not
70 routable by default (see more details in step 7).
71 #. Specify the LDAP server :guilabel:`Port` number or leave it empty to
72 use the defaults.
73 #. Specify the :guilabel:`Bind DN` of a user who has search privileges on
74 the LDAP server.
75 #. Specify the password of the user identified by the :guilabel:`Bind DN`
76 selected in the above field.
77 #. Specify the :guilabel:`User search base DN` in the Directory
78 Information Tree (DIT) from where to search for users.
79 #. Specify a valid attribute to search for users, for example, ``uid``.
80 The search should return a unique user entry.
81 #. Specify a valid search filter to search for users, for example,
82 ``(objectClass=*)``
83
84 You can further restrict access to Kibana to those users who
85 are members of a specific LDAP group:
86
87 #. Select :guilabel:`Enable group-based authorization`.
88 #. Specify the :guilabel:`LDAP attribute` in the user entry that identifies
89 the LDAP group membership, for example, ``memberUid``.
90 #. Specify the DN of the LDAP group that will be mapped to the *admin role*.
91 #. Specify the DN of the LDAP group that will be mapped to the *viewer role*.
92
93 Users who have the *admin role* can modify the Kibana dashboards
94 or create new ones. Users who have the *viewer role* can only
95 view the Kibana dashboards.
96
97#. In the Fuel web UI,
98 `configure your environment <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment.html>`_.
99
100 .. caution:: By default, StackLight is configured to use the
101 *management network* of the so-called
102 `Default Node Network Group <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/network-settings.html>`_.
103 While this default setup may be appropriate for small deployments or
104 evaluation purposes, we recommend not to use this network
105 for StackLight in production. Instead, create a network
106 dedicated to StackLight to improve performance and reduce the monitoring
107 footprint. It will also facilitate access to the Kibana UI after
108 deployment.
109
110#. Click the :guilabel:`Nodes` tab and assign the *Elasticsearch_Kibana* role
111 to the node(s) where you want to install the plugin.
112
113 The example below shows that the *Elasticsearch_Kibana* role is assigned
114 to three nodes alongside with the *Alerting_Infrastructure* and the
115 *InfluxDB_Grafana* roles. The three plugins of the LMA toolchain back-end
116 servers are installed on the same nodes.
117
118 .. image:: ../images/elastic_kibana_role.png
119 :width: 800
120 :align: center
121
122 .. note:: The Elasticsearch clustering for high availability requires
123 that you assign the *Elasticsearch_Kibana* role to at least three nodes,
124 but you can assign the *Elasticsearch_Kibana* role up to five nodes.
125 You can also add or remove a node with the *Elasticsearch_Kibana*
126 role after deployment.
127
128#. If required, `adjust the disk partitioning <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/customize-partitions.html>`_.
129
130 By default, the Elasticsearch-Kibana plugin allocates:
131
132 * 20% of the first available disk for the operating system by honoring
133 a range of 15 GB minimum and 50 GB maximum.
134 * 10 GB for ``/var/log``.
135 * At least 30 GB for the Elasticsearch database in ``/opt/es-data``.
136
13710. `Deploy your environment
138 <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/deploy-environment.html>`_.
diff --git a/doc/source/definitions.rst b/doc/source/definitions.rst
new file mode 100644
index 0000000..97a02ee
--- /dev/null
+++ b/doc/source/definitions.rst
@@ -0,0 +1,21 @@
1Key terms
2=========
3
4The table below lists the key terms that are used in this document.
5
6+---------------------+--------------------------------------------------------------------------------------+
7| **Term** | **Definition** |
8+=====================+======================================================================================+
9| StackLight Collector| A smart monitoring agent running on every node which collects and processes the logs |
10| | and the notifications of your OpenStack environment. |
11+---------------------+--------------------------------------------------------------------------------------+
12| Elasticsearch | An open-source (Apache-licensed) application based on the Lucene search engine |
13| | that makes data-like log messages easy to explore and correlate. |
14| | |
15| | Elasticsearch is written in Java and uses Lucene internally for all of its indexing |
16| | and searching, but it aims to make full-text search easy by hiding the complexities |
17| | of Lucene behind a simple, coherent RESTful API. |
18+---------------------+--------------------------------------------------------------------------------------+
19| Kibana | An open-source (Apache-licensed) browser-based analytics and search dashboard for |
20| | Elasticsearch. Kibana is easy to setup and use. |
21+---------------------+--------------------------------------------------------------------------------------+
diff --git a/doc/source/deploy_with_network_templates.rst b/doc/source/deploy_with_network_templates.rst
new file mode 100644
index 0000000..7c059ed
--- /dev/null
+++ b/doc/source/deploy_with_network_templates.rst
@@ -0,0 +1,46 @@
1.. _network_templates:
2
3Deploy an OpenStack environment using networking templates
4==========================================================
5
6By default, the Elasticsearch-Kibana cluster will be deployed on the Fuel
7management network. If this default configuration does not meet your
8requirements, you can leverage the Fuel
9`networking templates' <https://docs.mirantis.com/openstack/fuel/fuel-9.0/operations.html#using-networking-templates>`_
10capability to change that default configuration to use a dedicated network
11instead.
12
13Below is a networking template example to define a new network named
14``monitoring``. You can use this configuration example as a starting point
15and adapt it to your requirements.
16
17.. literalinclude:: ./network_template.yaml
18
19**To deploy an environment using networking templates:**
20
21#. Upload the networking template:
22
23 .. code-block:: console
24
25 $ fuel2 network-template upload -f ./network_template <ENVIRONMENT_ID>
26
27#. Allocate an IP subnet for the ``monitoring`` network:
28
29 .. code-block:: console
30
31 $ fuel2 network-group create -N <ENVIRONMENT_ID> -C 10.109.5.0/24 monitoring
32
33#. (Optional) Using the Fuel web UI, adjust the IP range:
34
35 .. image:: ../images/network_group_configuration.png
36 :width: 800
37 :align: center
38
39#. Proceed to :ref:`Configure the plugin during an environment deployment <plugin_configuration>`.
40
41For details on networking templates, see
42`Mirantis Operations Guide <https://docs.mirantis.com/openstack/fuel/fuel-9.0/operations.html#using-networking-templates>`_.
43
44.. raw:: latex
45
46 \pagebreak \ No newline at end of file
diff --git a/doc/source/index.rst b/doc/source/index.rst
index b1db35f..406267e 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -1,22 +1,40 @@
1==================================================================== 1=========================================================================
2Welcome to the StackLight Elasticsearch-Kibana Plugin Documentation! 2Welcome to the Fuel StackLight Elasticsearch-Kibana plugin documentation!
3==================================================================== 3=========================================================================
4 4
5User documentation 5Overview
6================== 6========
7 7
8.. toctree:: 8.. toctree::
9 :maxdepth: 2 9 :maxdepth: 1
10 10
11 overview 11 intro
12 releases 12 definitions
13 installation 13 requirements
14 user 14 limitations
15 advanced_operations 15 release_notes
16 licenses 16 licenses
17 appendix 17 references
18
19Installing and configuring StackLight Elasticsearch-Kibana plugin for Fuel
20==========================================================================
21
22.. toctree::
23 :maxdepth: 1
18 24
19Indices and Tables 25 install
20================== 26 configure
27 deploy_with_network_templates
28 verify
29
21 30
22* :ref:`search` 31Using StackLight Elasticsearch-Kibana plugin for Fuel
32=====================================================
33
34.. toctree::
35 :maxdepth: 1
36
37 user
38 troubleshooting
39 advanced_operations
40
diff --git a/doc/source/install.rst b/doc/source/install.rst
new file mode 100644
index 0000000..3aa5294
--- /dev/null
+++ b/doc/source/install.rst
@@ -0,0 +1,146 @@
1.. _install:
2
3Install the plugin
4==================
5
6Introduction
7------------
8
9You can install the StackLight Elasticsearch-Kibana Fuel plugin using one of
10the following options:
11
12* Install using the RPM file
13* Install from source
14
15The following is a list of software components installed by the StackLight
16Elasticsearch-Kibana Fuel plugin:
17
18+---------------+---------------------------------------------+
19| Components | Version |
20+===============+=============================================+
21| Elasticsearch | v2.3.3 for Ubuntu (64-bit) |
22+---------------+---------------------------------------------+
23| Kibana | v4.5 |
24+---------------+---------------------------------------------+
25| Apache | Version coming with the Ubuntu distribution |
26+---------------+---------------------------------------------+
27
28Install using the RPM file
29--------------------------
30
31To install the StackLight Elasticsearch-Kibana Fuel plugin using the RPM file
32from the Fuel plugins' catalog:
33
34#. Go to the
35 `Fuel plugins' Catalog <https://www.mirantis.com/validated-solution-integrations/fuel-plugins>`_.
36
37#. From the :guilabel:`Filter` drop-down menu, select the Mirantis OpenStack
38 version you are using and the :guilabel:`MONITORING` category.
39
40#. Download the RPM file.
41
42#. Copy the RPM file to the Fuel Master node:
43
44 .. code-block:: console
45
46 [root@home ~]# scp elasticsearch_kibana-0.10-0.10.0-0.noarch.rpm \
47 root@<Fuel Master node IP address>:
48
49#. Install the plugin using the `Fuel Plugins CLI
50 <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/cli/cli_plugins.html>`_:
51
52 .. code-block:: console
53
54 [root@fuel ~]# fuel plugins --install elasticsearch_kibana-0.10-0.10.0-0.noarch.rpm
55
56#. Verify that the plugin is installed correctly:
57
58 .. code-block:: console
59
60 [root@fuel ~]# fuel plugins --list
61 id | name | version | package_version
62 ---|----------------------|----------|----------------
63 1 | elasticsearch_kibana | 0.10.0 | 4.0.0
64
65
66Install from source
67-------------------
68
69Alternatively, you can build the RPM file of the plugin from source if, for
70example, you want to test the latest features of the master branch or
71customize the plugin.
72
73.. caution:: Running a Fuel plugin that you built from source is at your
74 own risk and is not supported.
75
76To install the StackLight Elasticsearch-Kibana plugin from source, at first
77prepare an environment to build the RPM file. We recommend building the RPM
78file directly on the Fuel Master node not to copy that file later on.
79
80**To prepare an environment for building the plugin on the Fuel Master node:**
81
82#. Install the standard Linux development tools:
83
84 .. code-block:: console
85
86 [root@home ~] yum install createrepo rpm rpm-build dpkg-devel
87
88#. Install ``pip``:
89
90 .. code-block:: console
91
92 [root@home ~] easy_install pip
93
94#. Install the Fuel Plugin Builder (the ``fpb`` command line) using ``pip``:
95
96 .. code-block:: console
97
98 [root@home ~] pip install fuel-plugin-builder
99
100 .. note:: You may also need to build the Fuel Plugin Builder if the
101 package version of the plugin is higher than the package version supported
102 by the Fuel Plugin Builder you get from ``pypi``. For instructions on how
103 to build the Fuel Plugin Builder, see the
104 *Install Fuel Plugin Builder* section of the
105 `Fuel Plugin SDK Guide <http://docs.openstack.org/developer/fuel-docs/plugindocs/fuel-plugin-sdk-guide/create-plugin/install-plugin-builder.html>`_.
106
107#. Clone the plugin repository:
108
109 .. code-block:: console
110
111 [root@home ~] git clone \
112 https://github.com/openstack/fuel-plugin-elasticsearch-kibana.git
113
114#. Verify that the plugin is valid:
115
116 .. code-block:: console
117
118 [root@home ~] fpb --check ./fuel-plugin-elasticsearch-kibana
119
120#. Build the plugin:
121
122 .. code-block:: console
123
124 [root@home ~] fpb --build ./fuel-plugin-elasticsearch-kibana
125
126**To install the plugin:**
127
128#. Once you create the RPM file, install the plugin:
129
130 .. code-block:: console
131
132 [root@fuel ~] fuel plugins --install \
133 ./fuel-plugin-elasticsearch-kibana/*.noarch.rpm
134
135#. Verify that the plugin is installed correctly:
136
137 .. code-block:: console
138
139 [root@fuel ~]# fuel plugins --list
140 id | name | version | package_version
141 ---|----------------------|----------|----------------
142 1 | elasticsearch_kibana | 0.10.0 | 4.0.0
143
144.. raw:: latex
145
146 \pagebreak \ No newline at end of file
diff --git a/doc/source/installation.rst b/doc/source/installation.rst
deleted file mode 100644
index 15c3f0d..0000000
--- a/doc/source/installation.rst
+++ /dev/null
@@ -1,100 +0,0 @@
1.. _user_installation:
2
3Installation Guide
4==================
5
6StackLight Elasticsearch-Kibana Plugin installation using the RPM file of the Fuel Plugins Catalog
7--------------------------------------------------------------------------------------------------
8
9To install the StackLight Elasticsearch-Kibana Fuel Plugin using the RPM file of the Fuel Plugins
10Catalog, you need to follow these steps:
11
12
131. Select, using the MONITORING category and Mirantis OpenStack version you are using,
14 the RPM file you want to download from the `Fuel Plugins Catalog
15 <https://www.mirantis.com/validated-solution-integrations/fuel-plugins>`_.
16
172. Copy the RPM file to the Fuel Master node::
18
19 [root@home ~]# scp elasticsearch_kibana-0.10-0.10.0-0.noarch.rpm \
20 root@<Fuel Master node IP address>:
21
223. Install the plugin using the `Fuel CLI
23 <http://docs.mirantis.com/openstack/fuel/fuel-8.0/user-guide.html#using-fuel-cli>`_::
24
25 [root@fuel ~]# fuel plugins --install elasticsearch_kibana-0.10-0.10.0-0.noarch.rpm
26
274. Verify that the plugin is installed correctly::
28
29 [root@fuel ~]# fuel plugins --list
30 id | name | version | package_version
31 ---|----------------------|----------|----------------
32 1 | elasticsearch_kibana | 0.10.0 | 4.0.0
33
34StackLight Elasticsearch-Kibana Fuel Plugin installation from source
35--------------------------------------------------------------------
36
37Alternatively, you may want to build the RPM file of the plugin from source if,
38for example, you want to test the latest features of the master branch or customize the plugin.
39
40.. note:: Be aware that running a Fuel plugin that you built yourself is at your
41 own risk and will not be supported.
42
43To install StacLight Elasticsearch-Kibana Plugin from source,
44you first need to prepare an environment to build the RPM file.
45The recommended approach is to build the RPM file directly onto the Fuel Master
46node so that you won't have to copy that file later on.
47
48**Prepare an environment for building the plugin on the Fuel Master Node**
49
501. Install the standard Linux development tools::
51
52 [root@home ~] yum install createrepo rpm rpm-build dpkg-devel
53
542. Install the Fuel Plugin Builder. To do that, you should first get pip::
55
56 [root@home ~] easy_install pip
57
583. Then install the Fuel Plugin Builder (the `fpb` command line) with `pip`::
59
60 [root@home ~] pip install fuel-plugin-builder
61
62.. note:: You may also need to build the Fuel Plugin Builder if the package version of the
63 plugin is higher than the package version supported by the Fuel Plugin Builder you get from `pypi`.
64 In this case, please refer to the section "Preparing an environment for plugin development"
65 of the `Fuel Plugins wiki <https://wiki.openstack.org/wiki/Fuel/Plugins>`_,
66 if you need further instructions about how to build the Fuel Plugin Builder.
67
684. Clone the plugin git repository::
69
70 [root@home ~] git clone \
71 https://github.com/openstack/fuel-plugin-elasticsearch-kibana.git
72
735. Check that the plugin is valid::
74
75 [root@home ~] fpb --check ./fuel-plugin-elasticsearch-kibana
76
776. And finally, build the plugin::
78
79 [root@home ~] fpb --build ./fuel-plugin-elasticsearch-kibana
80
817. Now that you have created the RPM file, you can install the plugin using the `fuel plugins --install` command::
82
83 [root@fuel ~] fuel plugins --install \
84 ./fuel-plugin-elasticsearch-kibana/*.noarch.rpm
85
86
87StackLight Elasticsearch-Kibana Fuel Plugin software components
88---------------------------------------------------------------
89
90List of software components installed by the plugin
91
92+---------------+--------------------------------------------------------+
93| Components | Version |
94+===============+========================================================+
95| Elasticsearch | v2.3.3 for Ubuntu (64-bit) |
96+---------------+--------------------------------------------------------+
97| Kibana | v4.5 |
98+---------------+--------------------------------------------------------+
99| Apache | Version coming by default with the Ubuntu distribution |
100+---------------+--------------------------------------------------------+
diff --git a/doc/source/intro.rst b/doc/source/intro.rst
new file mode 100644
index 0000000..c501ae0
--- /dev/null
+++ b/doc/source/intro.rst
@@ -0,0 +1,20 @@
1.. _intro:
2
3Introduction
4============
5
6The **StackLight Elasticsearch-Kibana plugin** is used to install and configure
7Elasticsearch and Kibana components that collectively provide access to the
8logs and notifications analytics of the so-called Logging, Monitoring, and
9Alerting (LMA) Toolchain of Mirantis OpenStack.
10
11These analytics can be used to search and correlate service-affecting
12events which may occur on your OpenStack environment. It is an indispensable
13tool to troubleshoot problems.
14
15Elasticsearch and Kibana are key components of the
16`LMA Toolchain project <https://launchpad.net/lma-toolchain>`_, also known as
17StackLight:
18
19.. image:: ../images/toolchain_map.png
20 :align: center
diff --git a/doc/source/licenses.rst b/doc/source/licenses.rst
index 6b0e226..3870c3c 100644
--- a/doc/source/licenses.rst
+++ b/doc/source/licenses.rst
@@ -3,15 +3,15 @@
3Licenses 3Licenses
4======== 4========
5 5
6Third Party Components 6Third-party components
7---------------------- 7----------------------
8 8
9+------------------------+-------------------------------------------------------------+------------+ 9+------------------------+-------------------------------------------------------------+------------+
10| Name | Project Web Site | License | 10| Name | Project Web Site | License |
11+========================+=============================================================+============+ 11+========================+=============================================================+============+
12| Elasticsearch | https://www.elastic.co/products/elasticsearch | Apache V2 | 12| Elasticsearch | https://www.elastic.co/products/elasticsearch | Apache v2 |
13+------------------------+-------------------------------------------------------------+------------+ 13+------------------------+-------------------------------------------------------------+------------+
14| Kibana | https://www.elastic.co/products/kibana | Apache V2 | 14| Kibana | https://www.elastic.co/products/kibana | Apache v2 |
15+------------------------+-------------------------------------------------------------+------------+ 15+------------------------+-------------------------------------------------------------+------------+
16 16
17Puppet modules 17Puppet modules
@@ -20,15 +20,15 @@ Puppet modules
20+------------------------+-------------------------------------------------------------+------------+ 20+------------------------+-------------------------------------------------------------+------------+
21| Name | Project Web Site | License | 21| Name | Project Web Site | License |
22+========================+=============================================================+============+ 22+========================+=============================================================+============+
23| Elasticsearch | https://forge.puppetlabs.com/elasticsearch/elasticsearch | Apache V2 | 23| Elasticsearch | https://forge.puppetlabs.com/elasticsearch/elasticsearch | Apache v2 |
24+------------------------+-------------------------------------------------------------+------------+ 24+------------------------+-------------------------------------------------------------+------------+
25| Concat | https://github.com/puppetlabs/puppetlabs-concat | Apache V2 | 25| Concat | https://github.com/puppetlabs/puppetlabs-concat | Apache v2 |
26+------------------------+-------------------------------------------------------------+------------+ 26+------------------------+-------------------------------------------------------------+------------+
27| Stdlib | https://github.com/puppetlabs/puppetlabs-stdlib | Apache V2 | 27| Stdlib | https://github.com/puppetlabs/puppetlabs-stdlib | Apache v2 |
28+------------------------+-------------------------------------------------------------+------------+ 28+------------------------+-------------------------------------------------------------+------------+
29| Apache | https://github.com/puppetlabs/puppetlabs-apache | Apache V2 | 29| Apache | https://github.com/puppetlabs/puppetlabs-apache | Apache v2 |
30+------------------------+-------------------------------------------------------------+------------+ 30+------------------------+-------------------------------------------------------------+------------+
31| Firewall | https://github.com/puppetlabs/puppetlabs-firewall | Apache V2 | 31| Firewall | https://github.com/puppetlabs/puppetlabs-firewall | Apache v2 |
32+------------------------+-------------------------------------------------------------+------------+ 32+------------------------+-------------------------------------------------------------+------------+
33| Datacat | https://github.com/richardc/puppet-datacat | Apache V2 | 33| Datacat | https://github.com/richardc/puppet-datacat | Apache v2 |
34+------------------------+-------------------------------------------------------------+------------+ 34+------------------------+-------------------------------------------------------------+------------+
diff --git a/doc/source/limitations.rst b/doc/source/limitations.rst
new file mode 100644
index 0000000..8d94a4c
--- /dev/null
+++ b/doc/source/limitations.rst
@@ -0,0 +1,16 @@
1.. _limitations:
2
3Limitations
4===========
5
6The StackLight Elasticsearch-Kibana plugin 0.10.0 has the following
7limitations:
8
9* Currently, the maximum size of an Elasticsearch cluster that can be
10 installed by Fuel is limited to five nodes. But each node of an Elasticsearch
11 cluster is configured as *master candidate* and a *storage node*. This means
12 that each node of the Elasticsearch cluster can be selected as master, and
13 all nodes will store data.
14
15* The :ref:`cluster operations <cluster_operations>` may require manual
16 operations.
diff --git a/doc/source/overview.rst b/doc/source/overview.rst
deleted file mode 100644
index 70be0c5..0000000
--- a/doc/source/overview.rst
+++ /dev/null
@@ -1,73 +0,0 @@
1.. _user_overview:
2
3Overview
4========
5
6The **StackLight Elasticsearch-Kibana Plugin** is used to install and configure
7Elasticsearch and Kibana which collectively provide access to the logs and
8notifications analytics of the so-called Logging, Monitoring and Alerting (LMA)
9Toolchain of Mirantis OpenStack.
10These analytics can be used to search and correlate service-affecting
11events which occurred in your OpenStack environment. It is an indispensable
12tool to troubleshooting problems.
13
14Elasticsearch and Kibana are key components
15of the `LMA Toolchain project <https://launchpad.net/lma-toolchain>`_
16as shown in the figure below.
17
18.. image:: ../images/toolchain_map.png
19 :align: center
20
21.. _plugin_requirements:
22
23Requirements
24------------
25
26+------------------------+------------------------------------------------------------------------------------------+
27| **Requirement** | **Version/Comment** |
28+========================+==========================================================================================+
29| Disk space | The plugin's specification requires to provision at least 15GB of disk space for the |
30| | system, 10GB for the logs and 30GB for the database. As a result, the installation |
31| | of the plugin will fail if there is less than 55GB of disk space available on the node. |
32+------------------------+------------------------------------------------------------------------------------------+
33| Mirantis OpenStack | 8.0, 9.0 |
34+------------------------+------------------------------------------------------------------------------------------+
35| Hardware configuration | The hardware configuration (RAM, CPU, disk) required by this plugin depends on the size |
36| | of your cloud environment and other parameters like the retention period and log level. |
37| | |
38| | A typical setup would at least require a quad-core server with 8GB of RAM and fast disks |
39| | (ideally, SSDs). The actual disk space you need to run the plugin depends on several |
40| | factors including the size of your OpenStack environment, the retention period, the |
41| | logging level and workload. The more of the above, the more disk space you will need to |
42| | run the Elaticsearch-Kibana Plugin. It is also highly recommended to use dedicated |
43| | disk(s) for your data storage. |
44+------------------------+------------------------------------------------------------------------------------------+
45
46Limitations
47-----------
48
49Currently, the maximum size of an Elasticsearch cluster that can be installed by Fuel is limited to five nodes.
50Each node of an Elasticsearch cluster is configured as *master candidate* and a *storage node*.
51This means, that each node of the Elasticsearch cluster can be elected as a master and all nodes will store data.
52
53The :ref:`cluster operations <cluster_operations>` can require some manual operations some times.
54
55Key terms, acronyms and abbreviations
56-------------------------------------
57
58+----------------------------+--------------------------------------------------------------------------------------+
59| **Terms & acronyms** | **Definition** |
60+============================+======================================================================================+
61| The Collector | The StackLight Collector is a smart monitoring agent running on every node which |
62| | collects and processes the logs and the notifications of your OpenStack environment. |
63+----------------------------+--------------------------------------------------------------------------------------+
64| Elasticsearch | An open source (Apache Licensed) application based on the Luceneā„¢ search engine |
65| | that makes data like log messages easy to explore and correlate. |
66| | |
67| | Elasticsearch is written in Java and uses Lucene internally for all of its indexing |
68| | and searching, but it aims to make full-text search easy by hiding the complexities |
69| | of Lucene behind a simple, coherent, RESTful API. |
70+----------------------------+--------------------------------------------------------------------------------------+
71| Kibana | An open source (Apache Licensed), browser based analytics and search dashboard for |
72| | Elasticsearch. Kibana is easy to setup and start using. |
73+----------------------------+--------------------------------------------------------------------------------------+
diff --git a/doc/source/references.rst b/doc/source/references.rst
new file mode 100644
index 0000000..e57fccf
--- /dev/null
+++ b/doc/source/references.rst
@@ -0,0 +1,8 @@
1.. _refs:
2
3References
4==========
5
6* `GitHub project <https://github.com/openstack/fuel-plugin-elasticsearch-kibana>`_
7* `Official Kibana documentation <https://www.elastic.co/guide/en/kibana/3.0/index.html>`_
8* `Official Elasticsearch documentation <https://www.elastic.co/guide/en/elasticsearch/reference/1.4/index.html>`_
diff --git a/doc/source/release_notes.rst b/doc/source/release_notes.rst
new file mode 100644
index 0000000..7a8f80a
--- /dev/null
+++ b/doc/source/release_notes.rst
@@ -0,0 +1,54 @@
1.. _releases:
2
3Release notes
4=============
5
6Version 0.10.0
7--------------
8
9The StackLight Elasticsearch-Kibana plugin 0.10.0 contains the following
10updates:
11
12* Added support for the LDAP(S) authentication to access the Kibana UI.
13* Added support for the TLS encryption to access the Kibana UI.
14
15 To configure the TLS termination, update the plugin settings with a PEM
16 file obtained by concatenating the SSL certificate with the private key.
17
18* Upgraded to Elasticsearch v2.3.3.
19* Upgraded to Kibana v4.5.
20* Fixed the issue in logs and notifications being dropped during a long
21 Elasticsearch outage. See
22 `#1566748 <https://bugs.launchpad.net/lma-toolchain/+bug/1566748>`_.
23
24Version 0.9.0
25-------------
26
27The StackLight Elasticsearch-Kibana plugin 0.9.0 contains the following
28updates:
29
30* Added support for Elasticsearch and Kibana clustering for scale-out and
31 high availability of those services.
32* Upgraded to Elasticsearch 1.7.4.
33* Upgraded to Kibana 3.1.3.
34
35Version 0.8.0
36-------------
37
38The StackLight Elasticsearch-Kibana plugin 0.8.0 contains the following
39updates:
40
41* Added support for the ``elasticsearch_kibana`` Fuel plugin role instead of
42 the ``base-os`` role which had several limitations.
43* Added support for the retention policy configuration with
44 `Elastic Curator <https://github.com/elastic/curator>`_.
45* Upgraded to Elasticsearch 1.4.5.
46
47Version 0.7.0
48-------------
49
50The initial release of the plugin (beta version).
51
52.. raw:: latex
53
54 \pagebreak
diff --git a/doc/source/releases.rst b/doc/source/releases.rst
deleted file mode 100644
index 23b52a5..0000000
--- a/doc/source/releases.rst
+++ /dev/null
@@ -1,46 +0,0 @@
1.. _releases:
2
3Release Notes
4=============
5
6Version 0.10.0
7--------------
8
9* Changes
10
11 * Add support for LDAP(S) authentication to access the Kibana UI.
12 * Add support for TLS encryption to access the Kibana UI.
13 A PEM file (obtained by concatenating the SSL certificate with the private key)
14 must be provided in the settings of the plugin to configure the TLS termination.
15 * Upgrade to Elasticsearch v2.3.3.
16 * Upgrade to Kibana v4.5.
17
18* Bug Fixes
19
20 * Logs and notifications are dropped during a "long" Elasticsearch outage (`#1566748
21 <https://bugs.launchpad.net/lma-toolchain/+bug/1566748>`_).
22
23Version 0.9.0
24-------------
25
26* Support Elasticsearch and Kibana clustering for scale-out and high
27 availability of those services.
28
29* Upgrade to Elasticsearch 1.7.4.
30
31* Upgrade to Kibana 3.1.3.
32
33Version 0.8.0
34-------------
35
36* Add support for the "elasticsearch_kibana" Fuel Plugin role instead of
37 the "base-os" role which had several limitations.
38
39* Add support for retention policy configuration with `Elastic Curator <https://github.com/elastic/curator>`_.
40
41* Upgrade to Elasticsearch 1.4.5.
42
43Version 0.7.0
44-------------
45
46* Initial release of the plugin. This is a beta version.
diff --git a/doc/source/requirements.rst b/doc/source/requirements.rst
new file mode 100644
index 0000000..8e0726a
--- /dev/null
+++ b/doc/source/requirements.rst
@@ -0,0 +1,35 @@
1.. _plugin_requirements:
2
3Requirements
4============
5
6The StackLight Elasticsearch-Kibana Plugin 0.10.0 has the following
7requirements:
8
9+------------------------+------------------------------------------------------------------------------------------+
10| **Requirement** | **Version/Comment** |
11+========================+==========================================================================================+
12| Disk space | The plugin's specification requires: |
13| | |
14| | * provisioning at least 15 GB of disk space for the system |
15| | * 10 GB for the logs |
16| | * 30 GB for the database |
17| | |
18| | As a result, the installation of the plugin will fail if there is less than **55 GB** |
19| | of disk space available on the node. |
20+------------------------+------------------------------------------------------------------------------------------+
21| Mirantis OpenStack | 8.0, 9.0 |
22+------------------------+------------------------------------------------------------------------------------------+
23| Hardware configuration | The hardware configuration (RAM, CPU, disk) depends on the size of your cloud environment|
24| | of your cloud environment and other parameters such as the retention period and log |
25| | level. |
26| | |
27| | A typical setup at least requires a quad-core server with 8 GB of RAM and fast disks |
28| | (ideally, SSDs). The actual disk space you need to run the plugin on depends on several |
29| | factors including the size of your OpenStack environment, the retention period, the |
30| | logging level, and workload. The more of the above, the more disk space you need to |
31| | run the Elaticsearch-Kibana plugin. We also highly recommend using dedicated |
32| | disk(s) for your data storage. |
33+------------------------+------------------------------------------------------------------------------------------+
34
35
diff --git a/doc/source/troubleshooting.rst b/doc/source/troubleshooting.rst
new file mode 100644
index 0000000..cf00109
--- /dev/null
+++ b/doc/source/troubleshooting.rst
@@ -0,0 +1,66 @@
1Troubleshooting
2===============
3
4If you cannot access the Kibana Dashboard or you get no data in the Dashboard,
5use the following troubleshooting tips:
6
7#. Verify that the StackLight Collector is running properly. For details, see
8 the `StackLight Collector <http://fuel-plugin-lma-collector.readthedocs.io/>`_
9 troubleshooting instructions.
10
11#. Verify that the nodes can connect to the Elasticsearch cluster
12 through the virtual IP address on port ``9200`` as described in the
13 :ref:`verify-elastic` section.
14
15#. On any of the *Elasticsearch_Kibana* role nodes, check the status of the
16 virtual IP address and HAProxy resources on the Pacemaker cluster:
17
18 .. code-block:: console
19
20 root@node-1:~# crm resource status vip__es_vip_mgmt
21 resource vip__es_vip_mgmt is running on: node-1.test.domain.local
22
23 root@node-1:~# crm resource status p_haproxy
24 resource p_haproxy is running on: node-1.test.domain.local
25
26#. If the virtual IP or HAProxy resources are down, restart them:
27
28 .. code-block:: console
29
30 root@node-1:~# crm resource start vip__es_vip_mgmt
31 root@node-1:~# crm resource start p_haproxy
32
33#. Verify that the Elasticsearch server is up and running on both CentOS and
34 Ubuntu:
35
36 .. code-block:: console
37
38 [root@node-1 ~]# /etc/init.d/elasticsearch-es-01 status
39
40#. If Elasticsearch is down, restart it on both CentOS and Ubuntu:
41
42 .. code-block:: console
43
44 [root@node-1 ~]# /etc/init.d/elasticsearch-es-01 start
45
46#. Verify that Apache is up and running on both CentOS and Ubuntu:
47
48 .. code-block:: console
49
50 [root@node-1 ~]# /etc/init.d/apache2 status
51
52#. If Apache is down, restart it on both CentOS and Ubuntu:
53
54 .. code-block:: console
55
56 [root@node-1 ~]# /etc/init.d/apache2 start
57
58#. Look for errors in the Elasticsearch log files located at
59 ``/var/log/elasticsearch/es-01/``.
60
61#. Look for errors in the Apache log files located at ``/var/log/apache2/``.
62
63.. raw:: latex
64
65 \pagebreak
66
diff --git a/doc/source/user.rst b/doc/source/user.rst
index 08e11f4..9fbd198 100644
--- a/doc/source/user.rst
+++ b/doc/source/user.rst
@@ -1,230 +1,22 @@
1.. _user_guide: 1.. _user:
2 2
3User Guide 3Use the plugin
4========== 4==============
5
6.. _plugin_configuration:
7
8Plugin configuration
9--------------------
10
11To configure the **StackLight Elasticsearch-Kibana Plugin**, you need to follow these steps:
12
131. `Create a new environment
14 <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/create-environment/start-create-env.html>`_.
15
162. Click on the *Settings* tab of the Fuel web UI and select the *Other* category.
17
183. Scroll down through the settings until you find the *StackLight Elasticsearch-Kibana
19 Plugin* section.
20
214. Tick the *StackLight Infrastructure Alerting Plugin* box and fill-in the required
22 fields as indicated below.
23
24 .. image:: ../images/elastic_kibana_settings.png
25 :width: 800
26 :align: center
27
28 a. Specify the number of days of retention for your data.
29 #. Specify the JVM heap size for Elastisearch. See the configuration recommendations below.
30
31 .. note:: By default, 1GB of heap memory is allocated to the Elasticsearch process.
32 This value is too small to run Elasticsearch for anything else than local testing.
33 To run Elasticsearch in production you need to allocate at least 4 GB of memory
34 but it is recommended to allocate 50% of the available memory up to 32 GB maximum.
35 If you set a value that is greater than the memory size, Elasticsearch won't start.
36 Keep in mind also to reserve enough memory for the operating system and the other services.
37
38 #. At this point, you can choose to either edit the *Advanced settings* or let the plugin
39 decide the defaults for you. The advanced settings are used to specify advanced settings
40 when Elasticsearch and Kibana are installed on a cluster of nodes.
41 To manually configure those advanced settings, check the *Advanced settings* box and fill-in
42 the required parameters.
43
445. Tick the *Enable TLS for Kibana* box if you want to encrypt your
45 Kibana credentials (username, password). Then, fill-in the required
46 fields as indicated below.
47
48 .. image:: ../images/tls_settings.png
49 :width: 800
50 :align: center
51
52 a. Specify the DNS name of the Kibana server. This parameter is used
53 to create a link in the Fuel dashboard to the Kibana server.
54 #. Specify the location of a PEM file that contains the certificate
55 and the private key of the Kibana server that will be used in TLS handchecks
56 with the client.
57
586. Tick the *Use LDAP for Kibana Authentication* box if you want to authenticate
59 via LDAP to Kibana. Then, fill-in the required fields as indicated below.
60
61 .. image:: ../images/ldap_auth.png
62 :width: 800
63 :align: center
64
65 a. Select the *LDAPS* button if you want to enable LDAP authentication
66 over SSL.
67 #. Specify one or several LDAP server addresses separated by a space. Those
68 addresses must be accessible from the node where Kibana is installed.
69 Note that addresses external to the *management network* are not routable
70 by default (see the note below).
71 #. Specify the LDAP server port number or leave it empty to use the defaults.
72 #. Specify the *Bind DN* of a user who has search priviliges on the LDAP server.
73 #. Specify the password of the user identified by the *Bind DN* above.
74 #. Specify the *Base DN* in the Directory Information Tree (DIT) from where
75 to search for users.
76 #. Specify a valid attribute (ex. 'uid') to search for users. The search should
77 return a unique user entry.
78 #. Specify a valid search filter (ex. '(objectClass=*') to search for users.
79
80 You can further restrict access to Kibana to those users who
81 are member of a specific LDAP group.
82
83 a. Tick the *Enable group-based authorization*.
84 #. Specify the LDAP attribute (ex. memberUid) in the user entry
85 that identifies the LDAP group membership.
86 #. Specify the DN of the LDAP group that will be mapped to the *admin role*
87 #. Specify the DN of the LDAP group that will be mapped to the *viewer role*
88
89 Users who have the *admin role* can modify the Kibana dashboards
90 or create new ones. Users who have the *Viewer role* can only
91 visualise the Kibana dashboards.
92
937. `Configure your environment
94 <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment.html>`_.
95
96 .. note:: By default, StackLight is configured to use the *management network*,
97 of the so-called `Default Node Network Group
98 <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/network-settings.html>`_.
99 While this default setup may be appropriate for small deployments or
100 evaluation purposes, it is recommended not to use this network
101 for StackLight in production. It is instead recommended to create a network
102 dedicated to StackLight. Using a dedicated network for StackLight should
103 improve performances and reduce the monitoring footprint.
104 It will also facilitate access to the Kibana UI after deployment.
105
1068. Click the *Nodes* tab and assign the *Elasticsearch_Kibana* role
107 to the node(s) where you want to install the plugin.
108
109 You can see in the example below that the *Elasticsearch_Kibana*
110 role is assigned to three nodes along side with the
111 *Alerting_Infrastructure* and the *InfluxDB_Grafana* roles.
112 Here, the three plugins of the LMA toolchain backend servers are
113 installed on the same nodes.
114
115 .. image:: ../images/elastic_kibana_role.png
116 :width: 800
117 :align: center
118
119 .. note:: The Elasticsearch clustering for high availability requires
120 that you assign the *Elasticsearch_Kibana* role to at least three nodes,
121 but you can assign the *Elasticsearch_Kibana* role up to five nodes.
122 Note also that is possible to add or remove a node with the *Elasticsearch_Kibana*
123 role after deployment.
124
1259. `Adjust the disk partitioning if necessary
126 <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/customize-partitions.html>`_.
127
128 By default, the Elasticsearch-Kibana Plugin allocates:
129
130 * 20% of the first available disk for the operating system by honoring a range of 15GB minimum and 50GB maximum.
131 * 10GB for */var/log*.
132 * At least 30 GB for the Elasticsearch database in */opt/es-data*.
133
13410. `Deploy your environment
135 <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/deploy-environment.html>`_.
136
137.. _plugin_install_verification:
138
139Plugin verification
140-------------------
141
142Be aware, that depending on the number of nodes and deployment setup,
143deploying a Mirantis OpenStack environment can typically take anything
144from 20 minutes to several hours. But once your deployment is complete,
145you should see a deployment success notification message with two
146links to Kibana as shown in the figure below:
147
148.. image:: ../images/deploy_notif.png
149 :align: center
150 :width: 800
151
152.. note:: For technical reasons, it was necessary to create two different ports
153 to enforce the access authorization to Kibana. One port (80) for users with the
154 *admin role* and one port (81) for users with the *viewer role*.
155 Be also aware that if Kibana is installed on the *management network*,
156 you may not have direct access to the UI. Some extra network
157 configuration may be required to create an SSH tunnel to the *management network*.
158
159Verifying Elasticsearch
160~~~~~~~~~~~~~~~~~~~~~~~
161
162You should verify that the Elasticsearch cluster is running properly.
163To do that, you need first to retrieve the Elasticsearch cluster VIP address.
164Here is how to proceed.
165
166#. On the Fuel Master node, find the IP address of a node where the Elasticsearch
167 server is installed using the following command::
168
169 [root@fuel ~]# fuel nodes
170 id | status | name | cluster | ip | mac | roles |
171 ---|----------|------------------|---------|-----|----------------------------|
172 1 | ready | Untitled (fa:87) | 1 | ... | ... | elasticsearch_kibana |
173 2 | ready | Untitled (12:aa) | 1 | ... | ... | elasticsearch_kibana |
174 3 | ready | Untitled (4e:6e) | 1 | ... | ... | elasticsearch_kibana |
175
176
177#. Then `ssh` to anyone of these nodes (ex. *node-1*) and type the command::
178
179 root@node-1:~# hiera lma::elasticsearch::vip
180 10.109.1.5
181
182 This tells you that the VIP address of your Elasticsearch cluster is *10.109.1.5*.
183
184#. With that VIP address type the command::
185
186 curl http://10.109.1.5:9200/
187
188 The output should look like this::
189
190 {
191 "status" : 200,
192 "name" : "node-3.test.domain.local_es-01",
193 "cluster_name" : "lma",
194 "version" : {
195 "number" : "1.7.4",
196 "build_hash" : "0d3159b9fc8bc8e367c5c40c09c2a57c0032b32e",
197 "build_timestamp" : "2015-12-15T11:25:18Z",
198 "build_snapshot" : false,
199 "lucene_version" : "4.10.4"
200 },
201 "tagline" : "You Know, for Search"
202 }
203
204Verifying Kibana
205~~~~~~~~~~~~~~~~
206
207From the Fuel dashboard, click on the *Kibana (Admin role)* link
208(or enter the IP address and port number if your DNS is not setup),
209then enter your credentials. You should be redirected to a Kibana 4
210**Logs Anaytics Dashboard** as shown in the figure below.
211
212.. image:: ../images/kibana_logs_dash.png
213 :align: center
214 :width: 800
215 5
216Dashboards management 6Dashboards management
217--------------------- 7---------------------
218 8
219The StackLight Elasticsearch-Kibana Plugin comes with two built-in dashboards: 9The StackLight Elasticsearch-Kibana plugin contains two built-in dashboards:
220 10
221 * The Logs Analytics Dashboard that is used to visualize and search the logs. 11 * The :guilabel:`Logs` Analytics Dashboard is used to visualize and
222 * The Notifications Analytics Dashboard that is used to visualize and 12 search the logs.
223 search the OpenStack notifications if you enabled the feature in the 13 * The :guilabel:`Notifications` Analytics Dashboard is used to visualize
14 and search the OpenStack notifications if you enabled the feature in the
224 Collector settings. 15 Collector settings.
225 16
226You can switch from one dashboard to another by clicking on the top-right *Load* 17You can switch from one dashboard to another by clicking on the top-right
227icon in the toolbar to select the requested dashboard from the list, as shown below. 18:guilabel:`Load` icon on the toolbar to select the requested dashboard from
19the list, as shown below:
228 20
229.. image:: ../images/kibana_dash.png 21.. image:: ../images/kibana_dash.png
230 :align: center 22 :align: center
@@ -232,11 +24,11 @@ icon in the toolbar to select the requested dashboard from the list, as shown be
232 24
233Each dashboard provides a single pane of glass for visualizing and searching 25Each dashboard provides a single pane of glass for visualizing and searching
234all the logs and the notifications of your OpenStack environment. 26all the logs and the notifications of your OpenStack environment.
235Note that in the Collector settings it is possible to tag the logs by
236environment name so that you can distinguish which logs (and notifications)
237belong to what environment.
238 27
239As you can see, the Kibana dashboard for logs is divided in several sections: 28In the Collector settings, you can tag the logs by an environment name to
29distinguish which logs (and notifications) belong to what environment.
30
31The Kibana Dashboard for logs is divided into several sections.
240 32
241.. image:: ../images/kibana_logs_sections_1.png 33.. image:: ../images/kibana_logs_sections_1.png
242 :align: center 34 :align: center
@@ -245,111 +37,64 @@ As you can see, the Kibana dashboard for logs is divided in several sections:
2451. A time-picker control that lets you choose the time period you want 371. A time-picker control that lets you choose the time period you want
246 to select and refresh frequency. 38 to select and refresh frequency.
247 39
2482. A text-box to enter search queries. 402. A text box to enter search queries.
249 41
2503. Various logs analytics with six different panels: 423. Various logs analytics with six different panels:
251 43
252 a. A stack graph showing all the logs per source. 44 a. A stack graph showing all the logs per source.
253 b. A stack graph showing all the logs per severity. 45 b. A stack graph showing all the logs per severity.
254 c. A stack graph showing all logs top 10 sources. 46 c. A stack graph showing all logs for top 10 sources.
255 d. A stack graph showing all the logs top 10 programs. 47 d. A stack graph showing all the logs for top 10 programs.
256 e. A stack graph showing all logs top 10 hosts. 48 e. A stack graph showing all logs for top 10 hosts.
257 f. A graph showing the number of logs per severity. 49 f. A graph showing the number of logs per severity.
258 g. A graph showing the number of logs per role. 50 g. A graph showing the number of logs per role.
259 51
2604. A table of log messages sorted in reverse chronological order. 524. A table of log messages sorted in reverse chronological order.
261 53
262.. image:: ../images/kibana_logs_sections_2.png 54 .. image:: ../images/kibana_logs_sections_2.png
263 :align: center 55 :align: center
264 :width: 800 56 :width: 800
265 57
266Filters and queries 58Filters and queries
267------------------- 59-------------------
268 60
269Filters and queries have similar syntax but they are used for different purposes. 61Filters and queries have similar syntax but they are used for different
62purposes:
270 63
271 * The filters are used to restrict what is displayed in the dashboard. 64* The filters are used to restrict what is displayed in the dashboard.
272 * The queries are used for free-text search. 65* The queries are used for free-text search.
273 66
274You can also combine multiple queries and compare their results. 67You can combine multiple queries and compare their results.
275To further filter the log messages to, for example, select the *deployment_id*, 68You can also further filter the log messages. For example, to select
276you need to expand a log entry and then select the *deployment_id* field 69:guilabel:`deployment_id`:
277by clicking on the magnifying glass icon as shown below.
278 70
279.. image:: ../images/kibana_logs_filter1.png 71#. Expand a log entry.
280 :align: center 72#. Select the :guilabel:`deployment_id` field by clicking on the magnifying
281 :width: 800 73 glass icon as shown below:
282
283This will apply a new filter in the dashboard.
284 74
285.. image:: ../images/kibana_logs_filter2.png 75 .. image:: ../images/kibana_logs_filter1.png
286 :align: center 76 :align: center
287 :width: 800 77 :width: 800
288
289Filtering will work for any field that has been indexed for the log entries that
290are in the dashboard.
291 78
292Filters and queries can also use wildcards that can be combined with *field names* like in:: 79 This will apply a new filter in the Dashboard:
293 80
294 programname: <name>* 81 .. image:: ../images/kibana_logs_filter2.png
82 :align: center
83 :width: 800
295 84
296For example, to display only the Nova logs you could enter:: 85Filtering works for any field that has been indexed for the log entries that
86are in the Dashboard.
297 87
298 programname:nova* 88Filters and queries can also use wildcards that can be combined with the
89*field names* like in ``programname: <name>*``
299 90
300in the query textbox as shown below. 91For example, to display only the Nova logs, enter ``programname:nova*`` in
92the query text box as shown below:
301 93
302.. image:: ../images/kibana_logs_query1.png 94.. image:: ../images/kibana_logs_query1.png
303 :align: center 95 :align: center
304 :width: 800 96 :width: 800
305 97
306Troubleshooting 98.. raw:: latex
307---------------
308
309If you cannot access the Kibana dashboard or you get no data in the dashboard,
310follow these troubleshooting tips.
311
3121. First, check that the StackLight Collector is running properly by following the
313 `StackLight Collector troubleshooting instructions
314 <http://fuel-plugin-lma-collector.readthedocs.io/>`_.
315
316#. Check that the nodes are able to connect to the Elasticsearch cluster via the VIP address
317 on port *9200* as explained in the `Verifying Elasticsearch` section above.
318
319#. On any of the *Elasticsearch_Kibana* role nodes, check the status of the VIP address
320 and HAProxy resources in the Pacemaker cluster::
321
322 root@node-1:~# crm resource status vip__es_vip_mgmt
323 resource vip__es_vip_mgmt is running on: node-1.test.domain.local
324
325 root@node-1:~# crm resource status p_haproxy
326 resource p_haproxy is running on: node-1.test.domain.local
327
328#. If the VIP or HAProxy resources are down, restart them::
329
330 root@node-1:~# crm resource start vip__es_vip_mgmt
331 root@node-1:~# crm resource start p_haproxy
332
333#. Check that the Elasticsearch server is up and running::
334
335 # On both CentOS and Ubuntu
336 [root@node-1 ~]# /etc/init.d/elasticsearch-es-01 status
337
338#. If Elasticsearch is down, restart it::
339
340 # On both CentOS and Ubuntu
341 [root@node-1 ~]# /etc/init.d/elasticsearch-es-01 start
342
343#. Check the apache is up and running::
344
345 # On both CentOS and Ubuntu
346 [root@node-1 ~]# /etc/init.d/apache2 status
347
348#. If apache is down, restart it::
349
350 # On both CentOS and Ubuntu
351 [root@node-1 ~]# /etc/init.d/apache2 start
352
353#. Look for errors in the Elasticsearch log files (located at /var/log/elasticsearch/es-01/).
354 99
355#. Look for errors in the apache log files (located at /var/log/apache2/). 100 \pagebreak \ No newline at end of file
diff --git a/doc/source/verify.rst b/doc/source/verify.rst
new file mode 100644
index 0000000..e91fb18
--- /dev/null
+++ b/doc/source/verify.rst
@@ -0,0 +1,100 @@
1.. _verification:
2
3Verify the plugin after deployment
4==================================
5
6Depending on the number of nodes and deployment setup, deploying a
7Mirantis OpenStack environment can typically take from 20 minutes
8to several hours. But once your deployment is complete, you should see a
9deployment success notification message with two links to Kibana as shown
10in the picture below:
11
12.. image:: ../images/deploy_notif.png
13 :align: center
14 :width: 800
15
16.. note:: For technical reasons, it was necessary to create two different ports
17 to enforce the access authorization to Kibana:
18
19 * One port (80) for users with the *admin role*
20 * One port (81) for users with the *viewer role*.
21
22 If Kibana is installed on the *management network*, you may not have
23 direct access to the Kibana web UI. Some extra network configuration may
24 be required to create an SSH tunnel to the *management network*.
25
26.. _verify-elastic:
27
28Verifying Elasticsearch
29-----------------------
30
31To verify that the Elasticsearch cluster is running properly, first retrieve
32the Elasticsearch cluster VIP address:
33
34#. On the Fuel Master node, find the IP address of a node where the Elasticsearch
35 server is installed using the :command:`fuel nodes` command. For example:
36
37 .. code-block:: console
38
39 [root@fuel ~]# fuel nodes
40 id|status|name |cluster|ip |mac |roles |
41 --|------|----------------|-------|----|-------------------------|
42 1 |ready |Untitled (fa:87)| 1 |... |... |elasticsearch_kibana|
43 2 |ready |Untitled (12:aa)| 1 |... |... |elasticsearch_kibana|
44 3 |ready |Untitled (4e:6e)| 1 |... |... |elasticsearch_kibana|
45
46
47#. Log in to any of these nodes using SSH, for example, to ``node-1``.
48#. Run the following command:
49
50 .. code-block:: console
51
52 root@node-1:~# hiera lma::elasticsearch::vip
53 10.109.1.5
54
55 Where ``10.109.1.5`` is the virtual IP address of your Elasticsearch cluster.
56
57#. With that virtual IP address, run the following command:
58
59 .. code-block:: console
60
61 curl http://10.109.1.5:9200/
62
63 The output should look as follows:
64
65 .. code-block:: console
66
67 {
68 "status" : 200,
69 "name" : "node-3.test.domain.local_es-01",
70 "cluster_name" : "lma",
71 "version" : {
72 "number" : "1.7.4",
73 "build_hash" : "0d3159b9fc8bc8e367c5c40c09c2a57c0032b32e",
74 "build_timestamp" : "2015-12-15T11:25:18Z",
75 "build_snapshot" : false,
76 "lucene_version" : "4.10.4"
77 },
78 "tagline" : "You Know, for Search"
79 }
80
81.. raw:: latex
82
83 \pagebreak
84
85Verifying Kibana
86----------------
87
88To verify the Kibana Dashboard:
89
90#. Log in to the Fuel web UI.
91#. Click on the :guilabel:`Kibana (Admin role)` link.
92 If your DNS is not setup, enter the IP address and the port number.
93#. Enter your credentials.
94
95 You should be redirected to the Kibana **Logs Anaytics Dashboard** with four
96 logs' sections as follows:
97
98 .. image:: ../images/kibana_logs_dash.png
99 :align: center
100 :width: 800