From dd402357e349870872a3da022eb066d53879c997 Mon Sep 17 00:00:00 2001 From: Olena Logvinova Date: Wed, 20 Jul 2016 14:31:09 +0300 Subject: [PATCH] [docs] Restructuring Kibana 0.10.0 User guide This patch: - modifies the structure for a better readability, - fixes noticed grammar and formatting mistakes. PDF build: https://drive.google.com/a/mirantis.com/file/d/0B2pEhXPCoNIIS0hwNEdtNUFBaVk/view?usp=sharing Change-Id: Ic0cbe6aa4d150e7750a836ee404c59edc504a27c --- doc/source/advanced_operations.rst | 128 +++---- doc/source/appendix.rst | 8 - doc/source/conf.py | 4 +- doc/source/configure.rst | 138 +++++++ doc/source/definitions.rst | 21 ++ doc/source/deploy_with_network_templates.rst | 46 +++ doc/source/index.rst | 48 ++- doc/source/install.rst | 146 ++++++++ doc/source/installation.rst | 100 ------ doc/source/intro.rst | 20 ++ doc/source/licenses.rst | 18 +- doc/source/limitations.rst | 16 + doc/source/overview.rst | 73 ---- doc/source/references.rst | 8 + doc/source/release_notes.rst | 54 +++ doc/source/releases.rst | 46 --- doc/source/requirements.rst | 35 ++ doc/source/troubleshooting.rst | 66 ++++ doc/source/user.rst | 357 +++---------------- doc/source/verify.rst | 100 ++++++ 20 files changed, 794 insertions(+), 638 deletions(-) delete mode 100644 doc/source/appendix.rst create mode 100644 doc/source/configure.rst create mode 100644 doc/source/definitions.rst create mode 100644 doc/source/deploy_with_network_templates.rst create mode 100644 doc/source/install.rst delete mode 100644 doc/source/installation.rst create mode 100644 doc/source/intro.rst create mode 100644 doc/source/limitations.rst delete mode 100644 doc/source/overview.rst create mode 100644 doc/source/references.rst create mode 100644 doc/source/release_notes.rst delete mode 100644 doc/source/releases.rst create mode 100644 doc/source/requirements.rst create mode 100644 doc/source/troubleshooting.rst create mode 100644 doc/source/verify.rst diff --git a/doc/source/advanced_operations.rst b/doc/source/advanced_operations.rst index 1c488d6..6e5ff30 100644 --- a/doc/source/advanced_operations.rst +++ b/doc/source/advanced_operations.rst @@ -1,59 +1,67 @@ .. _advanced_user_guide: -Advanced Operations +Advanced operations =================== - .. _cluster_operations: +This section describes advanced operations that you can apply to your +Elasticsearch cluster using the StackLight Elasticsearch-Kibana Fuel plugin. -Cluster Operations +.. _cluster_operations: + +Cluster operations ------------------ -Because of certain limitations in the current implementation of the Fuel plugin, it is necessary to -perform some manual operations after the Elasticsearch cluster is scaled up or scaled down. -Those operations are needed to adjust the replication factor of the Elasticsearch indices, -based on the new number of nodes in the cluster. -There are three types of indices used by the plugin: +Because of limitations in the current implementation of the plugin, manual +operations are required after the Elasticsearch cluster is scaled up or scaled down. Using these operations, you can adjust the replication factor of the +Elasticsearch indices that are based on the new number of nodes on the cluster. - * The log indices named *log-%{+YYYY.MM.dd}* which are created on a daily basis. - * The notification indices named *notification-%{+YYYY.MM.dd}* which are also created on a daily - basis. - * The Kibana index named *kibana-int* which is created once at installation time to store the - templates of the Kibana dashboards. +The plugin uses three types of indices: -Adjusting the replication factor for the *kibana-int* index is -performed automatically by the plugin and therefore there is no need to do any manual operation -for that index when the cluster is scaled up or down. +* The log indices named *log-%{+YYYY.MM.dd}* which are created on a daily basis. +* The notification indices named *notification-%{+YYYY.MM.dd}* which are + created on a daily basis. +* The Kibana index named *kibana-int* which is created once during the + installation to store the templates of the Kibana dashboards. -This is not the case for the replication factor of the other two indices which needs to -be updated manualy as described in the -`official documentation `_. +Adjusting the replication factor for the *kibana-int* index is performed +automatically by the plugin. Therefore, no manual operation is not required +for this index when the cluster is scaled up or down. But this is not the case +for the replication factor of other two indices that you should manually +update as described in the +`Elasticsearch official documentation `_. -The following sections provide more details, describing what do when scaling up/down the -Elasticsearch cluster. Scaling up from one node to three nodes, and scaling down from three nodes -to one node, are used as examples. Your mileage may vary but the -principal of (re)configuring the replication factor of the indices should remain the same. +The following sections describe in detail how to scale up and scale down the +Elasticsearch cluster. Scaling up from one node to three nodes and scaling +down from three nodes to one node are used as examples. Your mileage may vary, +but the principal of (re)configuring the replication factor of the indices +should remain the same. Scaling Up -^^^^^^^^^^ +~~~~~~~~~~ -The problem the manual operation aims to address is that the replication factor for the old -indices is not updated automatically by the plugin when a new node is added in the cluster. If you -want the old indices to be replicated on the new node(s), you need to adjust the -*number_of_replicas* parameter to the current size of the cluster for those indices as shown below. +The problem that the manual operation aims to address is that the replication +factor for the old indices is not updated automatically by the plugin when +a new node is added in the cluster. If you want the old indices to be +replicated on the new node(s), you need to adjust the *number_of_replicas* +parameter to the current size of the cluster for those indices as shown below. -The output below shows that the replication factor of the indices created before the scale-up is -zero. Here, a scale-up was performed on the 3rd of February, so the indices created after that date -(*log-2016.02.04* here) are automatically updated with the correct number of replicas -(number of cluster nodes - 1). :: +The output below shows that the replication factor of the indices created +before the scale-up is zero. Here, a scale-up was performed on the 3rd of +February, so the indices created after that date (*log-2016.02.04* in this +example) are automatically updated with the correct number of replicas +(number of cluster nodes - 1). + +.. code-block:: console [root@node-1 ~]# curl :9200/_cat/indices?v health status index pri rep docs.count docs.deleted .... green open log-2016.02.03 5 0 270405 0 .... green open log-2016.02.04 5 2 1934581 0 .... +If you want the *log-2016.02.03* index to be replicated, update the +*number_of_replicas* parameter of that index as shown below: -Then, if you want the *log-2016.02.03* index to be replicated, you need to update the -*number_of_replicas* parameter of that index as shown below:: +.. code-block:: console [root@node-1 ~]# curl -XPUT :9200/log-2016.02.03/_settings \ -d '{ "index": {"number_of_replicas": 2}}' @@ -64,16 +72,17 @@ Then, if you want the *log-2016.02.03* index to be replicated, you need to updat green open log-2016.02.03 5 2 270405 0 .... green open log-2016.02.04 5 2 1934581 0 .... - -Note that replicating the old indices on the new node(s) will increase the load on the -cluster as well as the size required to store the data. +.. caution:: Replicating the old indices on the new node(s) will increase the + load on the cluster as well as the size required to store the data. Scaling down -^^^^^^^^^^^^ +~~~~~~~~~~~~ Similarly, after a scale-down the *number_of_replicas* of all indices must be -aligned with the new size of the cluster. Not doing so will be reported by LMA as a critical -status for the Elasticsearch cluster:: +aligned with the new size of the cluster. Not doing so will be reported by LMA +as a critical status for the Elasticsearch cluster: + +.. code-block:: console [root@node-1 ~]# # the current index health is 'red' after the scale-down [root@node-1 ~]# curl :9200/_cat/indices?v @@ -88,42 +97,3 @@ status for the Elasticsearch cluster:: [root@node-1 ~]# curl :9200/_cat/indices?v health status index pri rep docs.count .... green open log-2016.02.04 5 2 1934581 .... - -.. _network_templates: - -Deployment using network templates ----------------------------------- - -By default, the Elasticsearch-Kibana cluster will be deployed on the Fuel management -network. If this default configuration doesn't meet your requirements, you can leverage the -Fuel `network templates -`_ -capability to change that default configuration to use a dedicated network instead. - -Below is a network template example to define a new network named `monitoring`. - -.. literalinclude:: ./network_template.yaml - -You can use this configuration example as a starting point and adapt it to your requirements. - -The deployment of the environment should work as described in the :ref:`User Guide -` excepted that before deploying the environment, you will have to: - -* Upload the network template:: - - $ fuel2 network-template upload -f ./network_template - -* Allocate an IP subnet for the `monitoring` network:: - - $ fuel2 network-group create -N -C 10.109.5.0/24 monitoring - -* Adjust the IP range through the Fuel user interface (optional). - - .. image:: ../images/network_group_configuration.png - :width: 800 - :align: center - -* Deploy the environment. - -For more details, please refer to the official `Fuel documentation -`_. diff --git a/doc/source/appendix.rst b/doc/source/appendix.rst deleted file mode 100644 index f46fbcc..0000000 --- a/doc/source/appendix.rst +++ /dev/null @@ -1,8 +0,0 @@ -.. _user_appendix: - -Appendix -======== - -* The `Elasticsearch-Kibana plugin `_ project at GitHub. -* The official `Kibana documentation `_. -* The official `Elasticsearch documentation `_. diff --git a/doc/source/conf.py b/doc/source/conf.py index 8de7831..27fdbc9 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -6,7 +6,7 @@ source_suffix = '.rst' master_doc = 'index' project = u'The StackLight Elasticsearch-Kibana Plugin for Fuel' -copyright = u'2015, Mirantis Inc.' +copyright = u'2016, Mirantis Inc.' version = '0.10' release = '0.10.0' @@ -19,7 +19,7 @@ html_theme = 'default' html_static_path = ['_static'] latex_documents = [ - ('index', 'ElasticsearchKibana.tex', u'The StackLight Elasticsearch-Kibana Plugin for Fuel Documentation', + ('index', 'ElasticsearchKibana.tex', u'Fuel StackLight Elasticsearch-Kibana Plugin Guide', u'Mirantis Inc.', 'manual'), ] diff --git a/doc/source/configure.rst b/doc/source/configure.rst new file mode 100644 index 0000000..ef7776b --- /dev/null +++ b/doc/source/configure.rst @@ -0,0 +1,138 @@ +.. _plugin_configuration: + +Configure the plugin during an environment deployment +===================================================== + +To configure the StackLight Elasticsearch-Kibana plugin during an environment +deployment: + +#. Using the Fuel web UI, + `Create a new environment `_. + +#. In the Fuel web UI, click the :guilabel:`Settings` tab and select the + :guilabel:`Other` category. + +#. Scroll down through the settings to find the + :guilabel:`StackLight Elasticsearch-Kibana Plugin` section. + +#. Select :guilabel:`StackLight Infrastructure Alerting Plugin` and fill in + the required fields as follows: + + .. image:: ../images/elastic_kibana_settings.png + :width: 800 + :align: center + + #. Specify the number of days to retain your data. + #. Specify the :guilabel:`JVM heap size` for Elastisearch. Use the tips + below: + + * By default, 1 GB of heap memory is allocated to the Elasticsearch + process. This value is enough to run Elasticsearch for local testing + only. + * To run Elasticsearch in production, allocate minimum 4 GB of memory. + But we recommend allocating 50% of the available memory up to 32 GB + maximum. + * If you set a value greater than the memory size, Elasticsearch will + not start. + * Reserve enough memory for operating system and other services. + + #. Select and edit :guilabel:`Advanced settings` if Elasticsearch and + Kibana are installed on a cluster of nodes. + +#. Select :guilabel:`Enable TLS for Kibana` if you want to encrypt your + Kibana credentials (username, password). Then, fill in the required fields + as follows: + + .. image:: ../images/tls_settings.png + :width: 800 + :align: center + + #. Specify the DNS name of the Kibana server. This parameter is used + to create a link in the Fuel dashboard to the Kibana server. + #. Specify the location of a PEM file that contains the certificate + and the private key of the Kibana server that will be used in TLS handchecks + with the client. + +#. If you want to authenticate through LDAP to Kibana, select + :guilabel:`Use LDAP for Kibana authentication`. Then, fill in the required + fields as follows: + + .. image:: ../images/ldap_auth.png + :width: 800 + :align: center + + #. Select :guilabel:`LDAPS` if you want to enable LDAP authentication + over SSL. + #. Specify one or several :guilabel:`LDAP servers` addresses separated by + space. Those addresses must be accessible from the node where Kibana + is installed. + The addresses that are external to the *management network* are not + routable by default (see more details in step 7). + #. Specify the LDAP server :guilabel:`Port` number or leave it empty to + use the defaults. + #. Specify the :guilabel:`Bind DN` of a user who has search privileges on + the LDAP server. + #. Specify the password of the user identified by the :guilabel:`Bind DN` + selected in the above field. + #. Specify the :guilabel:`User search base DN` in the Directory + Information Tree (DIT) from where to search for users. + #. Specify a valid attribute to search for users, for example, ``uid``. + The search should return a unique user entry. + #. Specify a valid search filter to search for users, for example, + ``(objectClass=*)`` + + You can further restrict access to Kibana to those users who + are members of a specific LDAP group: + + #. Select :guilabel:`Enable group-based authorization`. + #. Specify the :guilabel:`LDAP attribute` in the user entry that identifies + the LDAP group membership, for example, ``memberUid``. + #. Specify the DN of the LDAP group that will be mapped to the *admin role*. + #. Specify the DN of the LDAP group that will be mapped to the *viewer role*. + + Users who have the *admin role* can modify the Kibana dashboards + or create new ones. Users who have the *viewer role* can only + view the Kibana dashboards. + +#. In the Fuel web UI, + `configure your environment `_. + + .. caution:: By default, StackLight is configured to use the + *management network* of the so-called + `Default Node Network Group `_. + While this default setup may be appropriate for small deployments or + evaluation purposes, we recommend not to use this network + for StackLight in production. Instead, create a network + dedicated to StackLight to improve performance and reduce the monitoring + footprint. It will also facilitate access to the Kibana UI after + deployment. + +#. Click the :guilabel:`Nodes` tab and assign the *Elasticsearch_Kibana* role + to the node(s) where you want to install the plugin. + + The example below shows that the *Elasticsearch_Kibana* role is assigned + to three nodes alongside with the *Alerting_Infrastructure* and the + *InfluxDB_Grafana* roles. The three plugins of the LMA toolchain back-end + servers are installed on the same nodes. + + .. image:: ../images/elastic_kibana_role.png + :width: 800 + :align: center + + .. note:: The Elasticsearch clustering for high availability requires + that you assign the *Elasticsearch_Kibana* role to at least three nodes, + but you can assign the *Elasticsearch_Kibana* role up to five nodes. + You can also add or remove a node with the *Elasticsearch_Kibana* + role after deployment. + +#. If required, `adjust the disk partitioning `_. + + By default, the Elasticsearch-Kibana plugin allocates: + + * 20% of the first available disk for the operating system by honoring + a range of 15 GB minimum and 50 GB maximum. + * 10 GB for ``/var/log``. + * At least 30 GB for the Elasticsearch database in ``/opt/es-data``. + +10. `Deploy your environment + `_. diff --git a/doc/source/definitions.rst b/doc/source/definitions.rst new file mode 100644 index 0000000..97a02ee --- /dev/null +++ b/doc/source/definitions.rst @@ -0,0 +1,21 @@ +Key terms +========= + +The table below lists the key terms that are used in this document. + ++---------------------+--------------------------------------------------------------------------------------+ +| **Term** | **Definition** | ++=====================+======================================================================================+ +| StackLight Collector| A smart monitoring agent running on every node which collects and processes the logs | +| | and the notifications of your OpenStack environment. | ++---------------------+--------------------------------------------------------------------------------------+ +| Elasticsearch | An open-source (Apache-licensed) application based on the Lucene search engine | +| | that makes data-like log messages easy to explore and correlate. | +| | | +| | Elasticsearch is written in Java and uses Lucene internally for all of its indexing | +| | and searching, but it aims to make full-text search easy by hiding the complexities | +| | of Lucene behind a simple, coherent RESTful API. | ++---------------------+--------------------------------------------------------------------------------------+ +| Kibana | An open-source (Apache-licensed) browser-based analytics and search dashboard for | +| | Elasticsearch. Kibana is easy to setup and use. | ++---------------------+--------------------------------------------------------------------------------------+ diff --git a/doc/source/deploy_with_network_templates.rst b/doc/source/deploy_with_network_templates.rst new file mode 100644 index 0000000..7c059ed --- /dev/null +++ b/doc/source/deploy_with_network_templates.rst @@ -0,0 +1,46 @@ +.. _network_templates: + +Deploy an OpenStack environment using networking templates +========================================================== + +By default, the Elasticsearch-Kibana cluster will be deployed on the Fuel +management network. If this default configuration does not meet your +requirements, you can leverage the Fuel +`networking templates' `_ +capability to change that default configuration to use a dedicated network +instead. + +Below is a networking template example to define a new network named +``monitoring``. You can use this configuration example as a starting point +and adapt it to your requirements. + +.. literalinclude:: ./network_template.yaml + +**To deploy an environment using networking templates:** + +#. Upload the networking template: + + .. code-block:: console + + $ fuel2 network-template upload -f ./network_template + +#. Allocate an IP subnet for the ``monitoring`` network: + + .. code-block:: console + + $ fuel2 network-group create -N -C 10.109.5.0/24 monitoring + +#. (Optional) Using the Fuel web UI, adjust the IP range: + + .. image:: ../images/network_group_configuration.png + :width: 800 + :align: center + +#. Proceed to :ref:`Configure the plugin during an environment deployment `. + +For details on networking templates, see +`Mirantis Operations Guide `_. + +.. raw:: latex + + \pagebreak \ No newline at end of file diff --git a/doc/source/index.rst b/doc/source/index.rst index b1db35f..406267e 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -1,22 +1,40 @@ -==================================================================== -Welcome to the StackLight Elasticsearch-Kibana Plugin Documentation! -==================================================================== +========================================================================= +Welcome to the Fuel StackLight Elasticsearch-Kibana plugin documentation! +========================================================================= -User documentation -================== +Overview +======== .. toctree:: - :maxdepth: 2 + :maxdepth: 1 - overview - releases - installation - user - advanced_operations + intro + definitions + requirements + limitations + release_notes licenses - appendix + references -Indices and Tables -================== +Installing and configuring StackLight Elasticsearch-Kibana plugin for Fuel +========================================================================== -* :ref:`search` +.. toctree:: + :maxdepth: 1 + + install + configure + deploy_with_network_templates + verify + + +Using StackLight Elasticsearch-Kibana plugin for Fuel +===================================================== + +.. toctree:: + :maxdepth: 1 + + user + troubleshooting + advanced_operations + diff --git a/doc/source/install.rst b/doc/source/install.rst new file mode 100644 index 0000000..3aa5294 --- /dev/null +++ b/doc/source/install.rst @@ -0,0 +1,146 @@ +.. _install: + +Install the plugin +================== + +Introduction +------------ + +You can install the StackLight Elasticsearch-Kibana Fuel plugin using one of +the following options: + +* Install using the RPM file +* Install from source + +The following is a list of software components installed by the StackLight +Elasticsearch-Kibana Fuel plugin: + ++---------------+---------------------------------------------+ +| Components | Version | ++===============+=============================================+ +| Elasticsearch | v2.3.3 for Ubuntu (64-bit) | ++---------------+---------------------------------------------+ +| Kibana | v4.5 | ++---------------+---------------------------------------------+ +| Apache | Version coming with the Ubuntu distribution | ++---------------+---------------------------------------------+ + +Install using the RPM file +-------------------------- + +To install the StackLight Elasticsearch-Kibana Fuel plugin using the RPM file +from the Fuel plugins' catalog: + +#. Go to the + `Fuel plugins' Catalog `_. + +#. From the :guilabel:`Filter` drop-down menu, select the Mirantis OpenStack + version you are using and the :guilabel:`MONITORING` category. + +#. Download the RPM file. + +#. Copy the RPM file to the Fuel Master node: + + .. code-block:: console + + [root@home ~]# scp elasticsearch_kibana-0.10-0.10.0-0.noarch.rpm \ + root@: + +#. Install the plugin using the `Fuel Plugins CLI + `_: + + .. code-block:: console + + [root@fuel ~]# fuel plugins --install elasticsearch_kibana-0.10-0.10.0-0.noarch.rpm + +#. Verify that the plugin is installed correctly: + + .. code-block:: console + + [root@fuel ~]# fuel plugins --list + id | name | version | package_version + ---|----------------------|----------|---------------- + 1 | elasticsearch_kibana | 0.10.0 | 4.0.0 + + +Install from source +------------------- + +Alternatively, you can build the RPM file of the plugin from source if, for +example, you want to test the latest features of the master branch or +customize the plugin. + +.. caution:: Running a Fuel plugin that you built from source is at your + own risk and is not supported. + +To install the StackLight Elasticsearch-Kibana plugin from source, at first +prepare an environment to build the RPM file. We recommend building the RPM +file directly on the Fuel Master node not to copy that file later on. + +**To prepare an environment for building the plugin on the Fuel Master node:** + +#. Install the standard Linux development tools: + + .. code-block:: console + + [root@home ~] yum install createrepo rpm rpm-build dpkg-devel + +#. Install ``pip``: + + .. code-block:: console + + [root@home ~] easy_install pip + +#. Install the Fuel Plugin Builder (the ``fpb`` command line) using ``pip``: + + .. code-block:: console + + [root@home ~] pip install fuel-plugin-builder + + .. note:: You may also need to build the Fuel Plugin Builder if the + package version of the plugin is higher than the package version supported + by the Fuel Plugin Builder you get from ``pypi``. For instructions on how + to build the Fuel Plugin Builder, see the + *Install Fuel Plugin Builder* section of the + `Fuel Plugin SDK Guide `_. + +#. Clone the plugin repository: + + .. code-block:: console + + [root@home ~] git clone \ + https://github.com/openstack/fuel-plugin-elasticsearch-kibana.git + +#. Verify that the plugin is valid: + + .. code-block:: console + + [root@home ~] fpb --check ./fuel-plugin-elasticsearch-kibana + +#. Build the plugin: + + .. code-block:: console + + [root@home ~] fpb --build ./fuel-plugin-elasticsearch-kibana + +**To install the plugin:** + +#. Once you create the RPM file, install the plugin: + + .. code-block:: console + + [root@fuel ~] fuel plugins --install \ + ./fuel-plugin-elasticsearch-kibana/*.noarch.rpm + +#. Verify that the plugin is installed correctly: + + .. code-block:: console + + [root@fuel ~]# fuel plugins --list + id | name | version | package_version + ---|----------------------|----------|---------------- + 1 | elasticsearch_kibana | 0.10.0 | 4.0.0 + +.. raw:: latex + + \pagebreak \ No newline at end of file diff --git a/doc/source/installation.rst b/doc/source/installation.rst deleted file mode 100644 index 15c3f0d..0000000 --- a/doc/source/installation.rst +++ /dev/null @@ -1,100 +0,0 @@ -.. _user_installation: - -Installation Guide -================== - -StackLight Elasticsearch-Kibana Plugin installation using the RPM file of the Fuel Plugins Catalog --------------------------------------------------------------------------------------------------- - -To install the StackLight Elasticsearch-Kibana Fuel Plugin using the RPM file of the Fuel Plugins -Catalog, you need to follow these steps: - - -1. Select, using the MONITORING category and Mirantis OpenStack version you are using, - the RPM file you want to download from the `Fuel Plugins Catalog - `_. - -2. Copy the RPM file to the Fuel Master node:: - - [root@home ~]# scp elasticsearch_kibana-0.10-0.10.0-0.noarch.rpm \ - root@: - -3. Install the plugin using the `Fuel CLI - `_:: - - [root@fuel ~]# fuel plugins --install elasticsearch_kibana-0.10-0.10.0-0.noarch.rpm - -4. Verify that the plugin is installed correctly:: - - [root@fuel ~]# fuel plugins --list - id | name | version | package_version - ---|----------------------|----------|---------------- - 1 | elasticsearch_kibana | 0.10.0 | 4.0.0 - -StackLight Elasticsearch-Kibana Fuel Plugin installation from source --------------------------------------------------------------------- - -Alternatively, you may want to build the RPM file of the plugin from source if, -for example, you want to test the latest features of the master branch or customize the plugin. - -.. note:: Be aware that running a Fuel plugin that you built yourself is at your - own risk and will not be supported. - -To install StacLight Elasticsearch-Kibana Plugin from source, -you first need to prepare an environment to build the RPM file. -The recommended approach is to build the RPM file directly onto the Fuel Master -node so that you won't have to copy that file later on. - -**Prepare an environment for building the plugin on the Fuel Master Node** - -1. Install the standard Linux development tools:: - - [root@home ~] yum install createrepo rpm rpm-build dpkg-devel - -2. Install the Fuel Plugin Builder. To do that, you should first get pip:: - - [root@home ~] easy_install pip - -3. Then install the Fuel Plugin Builder (the `fpb` command line) with `pip`:: - - [root@home ~] pip install fuel-plugin-builder - -.. note:: You may also need to build the Fuel Plugin Builder if the package version of the - plugin is higher than the package version supported by the Fuel Plugin Builder you get from `pypi`. - In this case, please refer to the section "Preparing an environment for plugin development" - of the `Fuel Plugins wiki `_, - if you need further instructions about how to build the Fuel Plugin Builder. - -4. Clone the plugin git repository:: - - [root@home ~] git clone \ - https://github.com/openstack/fuel-plugin-elasticsearch-kibana.git - -5. Check that the plugin is valid:: - - [root@home ~] fpb --check ./fuel-plugin-elasticsearch-kibana - -6. And finally, build the plugin:: - - [root@home ~] fpb --build ./fuel-plugin-elasticsearch-kibana - -7. Now that you have created the RPM file, you can install the plugin using the `fuel plugins --install` command:: - - [root@fuel ~] fuel plugins --install \ - ./fuel-plugin-elasticsearch-kibana/*.noarch.rpm - - -StackLight Elasticsearch-Kibana Fuel Plugin software components ---------------------------------------------------------------- - -List of software components installed by the plugin - -+---------------+--------------------------------------------------------+ -| Components | Version | -+===============+========================================================+ -| Elasticsearch | v2.3.3 for Ubuntu (64-bit) | -+---------------+--------------------------------------------------------+ -| Kibana | v4.5 | -+---------------+--------------------------------------------------------+ -| Apache | Version coming by default with the Ubuntu distribution | -+---------------+--------------------------------------------------------+ diff --git a/doc/source/intro.rst b/doc/source/intro.rst new file mode 100644 index 0000000..c501ae0 --- /dev/null +++ b/doc/source/intro.rst @@ -0,0 +1,20 @@ +.. _intro: + +Introduction +============ + +The **StackLight Elasticsearch-Kibana plugin** is used to install and configure +Elasticsearch and Kibana components that collectively provide access to the +logs and notifications analytics of the so-called Logging, Monitoring, and +Alerting (LMA) Toolchain of Mirantis OpenStack. + +These analytics can be used to search and correlate service-affecting +events which may occur on your OpenStack environment. It is an indispensable +tool to troubleshoot problems. + +Elasticsearch and Kibana are key components of the +`LMA Toolchain project `_, also known as +StackLight: + +.. image:: ../images/toolchain_map.png + :align: center diff --git a/doc/source/licenses.rst b/doc/source/licenses.rst index 6b0e226..3870c3c 100644 --- a/doc/source/licenses.rst +++ b/doc/source/licenses.rst @@ -3,15 +3,15 @@ Licenses ======== -Third Party Components +Third-party components ---------------------- +------------------------+-------------------------------------------------------------+------------+ | Name | Project Web Site | License | +========================+=============================================================+============+ -| Elasticsearch | https://www.elastic.co/products/elasticsearch | Apache V2 | +| Elasticsearch | https://www.elastic.co/products/elasticsearch | Apache v2 | +------------------------+-------------------------------------------------------------+------------+ -| Kibana | https://www.elastic.co/products/kibana | Apache V2 | +| Kibana | https://www.elastic.co/products/kibana | Apache v2 | +------------------------+-------------------------------------------------------------+------------+ Puppet modules @@ -20,15 +20,15 @@ Puppet modules +------------------------+-------------------------------------------------------------+------------+ | Name | Project Web Site | License | +========================+=============================================================+============+ -| Elasticsearch | https://forge.puppetlabs.com/elasticsearch/elasticsearch | Apache V2 | +| Elasticsearch | https://forge.puppetlabs.com/elasticsearch/elasticsearch | Apache v2 | +------------------------+-------------------------------------------------------------+------------+ -| Concat | https://github.com/puppetlabs/puppetlabs-concat | Apache V2 | +| Concat | https://github.com/puppetlabs/puppetlabs-concat | Apache v2 | +------------------------+-------------------------------------------------------------+------------+ -| Stdlib | https://github.com/puppetlabs/puppetlabs-stdlib | Apache V2 | +| Stdlib | https://github.com/puppetlabs/puppetlabs-stdlib | Apache v2 | +------------------------+-------------------------------------------------------------+------------+ -| Apache | https://github.com/puppetlabs/puppetlabs-apache | Apache V2 | +| Apache | https://github.com/puppetlabs/puppetlabs-apache | Apache v2 | +------------------------+-------------------------------------------------------------+------------+ -| Firewall | https://github.com/puppetlabs/puppetlabs-firewall | Apache V2 | +| Firewall | https://github.com/puppetlabs/puppetlabs-firewall | Apache v2 | +------------------------+-------------------------------------------------------------+------------+ -| Datacat | https://github.com/richardc/puppet-datacat | Apache V2 | +| Datacat | https://github.com/richardc/puppet-datacat | Apache v2 | +------------------------+-------------------------------------------------------------+------------+ diff --git a/doc/source/limitations.rst b/doc/source/limitations.rst new file mode 100644 index 0000000..8d94a4c --- /dev/null +++ b/doc/source/limitations.rst @@ -0,0 +1,16 @@ +.. _limitations: + +Limitations +=========== + +The StackLight Elasticsearch-Kibana plugin 0.10.0 has the following +limitations: + +* Currently, the maximum size of an Elasticsearch cluster that can be + installed by Fuel is limited to five nodes. But each node of an Elasticsearch + cluster is configured as *master candidate* and a *storage node*. This means + that each node of the Elasticsearch cluster can be selected as master, and + all nodes will store data. + +* The :ref:`cluster operations ` may require manual + operations. diff --git a/doc/source/overview.rst b/doc/source/overview.rst deleted file mode 100644 index 70be0c5..0000000 --- a/doc/source/overview.rst +++ /dev/null @@ -1,73 +0,0 @@ -.. _user_overview: - -Overview -======== - -The **StackLight Elasticsearch-Kibana Plugin** is used to install and configure -Elasticsearch and Kibana which collectively provide access to the logs and -notifications analytics of the so-called Logging, Monitoring and Alerting (LMA) -Toolchain of Mirantis OpenStack. -These analytics can be used to search and correlate service-affecting -events which occurred in your OpenStack environment. It is an indispensable -tool to troubleshooting problems. - -Elasticsearch and Kibana are key components -of the `LMA Toolchain project `_ -as shown in the figure below. - -.. image:: ../images/toolchain_map.png - :align: center - -.. _plugin_requirements: - -Requirements ------------- - -+------------------------+------------------------------------------------------------------------------------------+ -| **Requirement** | **Version/Comment** | -+========================+==========================================================================================+ -| Disk space | The plugin's specification requires to provision at least 15GB of disk space for the | -| | system, 10GB for the logs and 30GB for the database. As a result, the installation | -| | of the plugin will fail if there is less than 55GB of disk space available on the node. | -+------------------------+------------------------------------------------------------------------------------------+ -| Mirantis OpenStack | 8.0, 9.0 | -+------------------------+------------------------------------------------------------------------------------------+ -| Hardware configuration | The hardware configuration (RAM, CPU, disk) required by this plugin depends on the size | -| | of your cloud environment and other parameters like the retention period and log level. | -| | | -| | A typical setup would at least require a quad-core server with 8GB of RAM and fast disks | -| | (ideally, SSDs). The actual disk space you need to run the plugin depends on several | -| | factors including the size of your OpenStack environment, the retention period, the | -| | logging level and workload. The more of the above, the more disk space you will need to | -| | run the Elaticsearch-Kibana Plugin. It is also highly recommended to use dedicated | -| | disk(s) for your data storage. | -+------------------------+------------------------------------------------------------------------------------------+ - -Limitations ------------ - -Currently, the maximum size of an Elasticsearch cluster that can be installed by Fuel is limited to five nodes. -Each node of an Elasticsearch cluster is configured as *master candidate* and a *storage node*. -This means, that each node of the Elasticsearch cluster can be elected as a master and all nodes will store data. - -The :ref:`cluster operations ` can require some manual operations some times. - -Key terms, acronyms and abbreviations -------------------------------------- - -+----------------------------+--------------------------------------------------------------------------------------+ -| **Terms & acronyms** | **Definition** | -+============================+======================================================================================+ -| The Collector | The StackLight Collector is a smart monitoring agent running on every node which | -| | collects and processes the logs and the notifications of your OpenStack environment. | -+----------------------------+--------------------------------------------------------------------------------------+ -| Elasticsearch | An open source (Apache Licensed) application based on the Lucene™ search engine | -| | that makes data like log messages easy to explore and correlate. | -| | | -| | Elasticsearch is written in Java and uses Lucene internally for all of its indexing | -| | and searching, but it aims to make full-text search easy by hiding the complexities | -| | of Lucene behind a simple, coherent, RESTful API. | -+----------------------------+--------------------------------------------------------------------------------------+ -| Kibana | An open source (Apache Licensed), browser based analytics and search dashboard for | -| | Elasticsearch. Kibana is easy to setup and start using. | -+----------------------------+--------------------------------------------------------------------------------------+ diff --git a/doc/source/references.rst b/doc/source/references.rst new file mode 100644 index 0000000..e57fccf --- /dev/null +++ b/doc/source/references.rst @@ -0,0 +1,8 @@ +.. _refs: + +References +========== + +* `GitHub project `_ +* `Official Kibana documentation `_ +* `Official Elasticsearch documentation `_ diff --git a/doc/source/release_notes.rst b/doc/source/release_notes.rst new file mode 100644 index 0000000..7a8f80a --- /dev/null +++ b/doc/source/release_notes.rst @@ -0,0 +1,54 @@ +.. _releases: + +Release notes +============= + +Version 0.10.0 +-------------- + +The StackLight Elasticsearch-Kibana plugin 0.10.0 contains the following +updates: + +* Added support for the LDAP(S) authentication to access the Kibana UI. +* Added support for the TLS encryption to access the Kibana UI. + + To configure the TLS termination, update the plugin settings with a PEM + file obtained by concatenating the SSL certificate with the private key. + +* Upgraded to Elasticsearch v2.3.3. +* Upgraded to Kibana v4.5. +* Fixed the issue in logs and notifications being dropped during a long + Elasticsearch outage. See + `#1566748 `_. + +Version 0.9.0 +------------- + +The StackLight Elasticsearch-Kibana plugin 0.9.0 contains the following +updates: + +* Added support for Elasticsearch and Kibana clustering for scale-out and + high availability of those services. +* Upgraded to Elasticsearch 1.7.4. +* Upgraded to Kibana 3.1.3. + +Version 0.8.0 +------------- + +The StackLight Elasticsearch-Kibana plugin 0.8.0 contains the following +updates: + +* Added support for the ``elasticsearch_kibana`` Fuel plugin role instead of + the ``base-os`` role which had several limitations. +* Added support for the retention policy configuration with + `Elastic Curator `_. +* Upgraded to Elasticsearch 1.4.5. + +Version 0.7.0 +------------- + +The initial release of the plugin (beta version). + +.. raw:: latex + + \pagebreak diff --git a/doc/source/releases.rst b/doc/source/releases.rst deleted file mode 100644 index 23b52a5..0000000 --- a/doc/source/releases.rst +++ /dev/null @@ -1,46 +0,0 @@ -.. _releases: - -Release Notes -============= - -Version 0.10.0 --------------- - -* Changes - - * Add support for LDAP(S) authentication to access the Kibana UI. - * Add support for TLS encryption to access the Kibana UI. - A PEM file (obtained by concatenating the SSL certificate with the private key) - must be provided in the settings of the plugin to configure the TLS termination. - * Upgrade to Elasticsearch v2.3.3. - * Upgrade to Kibana v4.5. - -* Bug Fixes - - * Logs and notifications are dropped during a "long" Elasticsearch outage (`#1566748 - `_). - -Version 0.9.0 -------------- - -* Support Elasticsearch and Kibana clustering for scale-out and high - availability of those services. - -* Upgrade to Elasticsearch 1.7.4. - -* Upgrade to Kibana 3.1.3. - -Version 0.8.0 -------------- - -* Add support for the "elasticsearch_kibana" Fuel Plugin role instead of - the "base-os" role which had several limitations. - -* Add support for retention policy configuration with `Elastic Curator `_. - -* Upgrade to Elasticsearch 1.4.5. - -Version 0.7.0 -------------- - -* Initial release of the plugin. This is a beta version. diff --git a/doc/source/requirements.rst b/doc/source/requirements.rst new file mode 100644 index 0000000..8e0726a --- /dev/null +++ b/doc/source/requirements.rst @@ -0,0 +1,35 @@ +.. _plugin_requirements: + +Requirements +============ + +The StackLight Elasticsearch-Kibana Plugin 0.10.0 has the following +requirements: + ++------------------------+------------------------------------------------------------------------------------------+ +| **Requirement** | **Version/Comment** | ++========================+==========================================================================================+ +| Disk space | The plugin's specification requires: | +| | | +| | * provisioning at least 15 GB of disk space for the system | +| | * 10 GB for the logs | +| | * 30 GB for the database | +| | | +| | As a result, the installation of the plugin will fail if there is less than **55 GB** | +| | of disk space available on the node. | ++------------------------+------------------------------------------------------------------------------------------+ +| Mirantis OpenStack | 8.0, 9.0 | ++------------------------+------------------------------------------------------------------------------------------+ +| Hardware configuration | The hardware configuration (RAM, CPU, disk) depends on the size of your cloud environment| +| | of your cloud environment and other parameters such as the retention period and log | +| | level. | +| | | +| | A typical setup at least requires a quad-core server with 8 GB of RAM and fast disks | +| | (ideally, SSDs). The actual disk space you need to run the plugin on depends on several | +| | factors including the size of your OpenStack environment, the retention period, the | +| | logging level, and workload. The more of the above, the more disk space you need to | +| | run the Elaticsearch-Kibana plugin. We also highly recommend using dedicated | +| | disk(s) for your data storage. | ++------------------------+------------------------------------------------------------------------------------------+ + + diff --git a/doc/source/troubleshooting.rst b/doc/source/troubleshooting.rst new file mode 100644 index 0000000..cf00109 --- /dev/null +++ b/doc/source/troubleshooting.rst @@ -0,0 +1,66 @@ +Troubleshooting +=============== + +If you cannot access the Kibana Dashboard or you get no data in the Dashboard, +use the following troubleshooting tips: + +#. Verify that the StackLight Collector is running properly. For details, see + the `StackLight Collector `_ + troubleshooting instructions. + +#. Verify that the nodes can connect to the Elasticsearch cluster + through the virtual IP address on port ``9200`` as described in the + :ref:`verify-elastic` section. + +#. On any of the *Elasticsearch_Kibana* role nodes, check the status of the + virtual IP address and HAProxy resources on the Pacemaker cluster: + + .. code-block:: console + + root@node-1:~# crm resource status vip__es_vip_mgmt + resource vip__es_vip_mgmt is running on: node-1.test.domain.local + + root@node-1:~# crm resource status p_haproxy + resource p_haproxy is running on: node-1.test.domain.local + +#. If the virtual IP or HAProxy resources are down, restart them: + + .. code-block:: console + + root@node-1:~# crm resource start vip__es_vip_mgmt + root@node-1:~# crm resource start p_haproxy + +#. Verify that the Elasticsearch server is up and running on both CentOS and + Ubuntu: + + .. code-block:: console + + [root@node-1 ~]# /etc/init.d/elasticsearch-es-01 status + +#. If Elasticsearch is down, restart it on both CentOS and Ubuntu: + + .. code-block:: console + + [root@node-1 ~]# /etc/init.d/elasticsearch-es-01 start + +#. Verify that Apache is up and running on both CentOS and Ubuntu: + + .. code-block:: console + + [root@node-1 ~]# /etc/init.d/apache2 status + +#. If Apache is down, restart it on both CentOS and Ubuntu: + + .. code-block:: console + + [root@node-1 ~]# /etc/init.d/apache2 start + +#. Look for errors in the Elasticsearch log files located at + ``/var/log/elasticsearch/es-01/``. + +#. Look for errors in the Apache log files located at ``/var/log/apache2/``. + +.. raw:: latex + + \pagebreak + diff --git a/doc/source/user.rst b/doc/source/user.rst index 08e11f4..9fbd198 100644 --- a/doc/source/user.rst +++ b/doc/source/user.rst @@ -1,230 +1,22 @@ -.. _user_guide: +.. _user: -User Guide -========== - -.. _plugin_configuration: - -Plugin configuration --------------------- - -To configure the **StackLight Elasticsearch-Kibana Plugin**, you need to follow these steps: - -1. `Create a new environment - `_. - -2. Click on the *Settings* tab of the Fuel web UI and select the *Other* category. - -3. Scroll down through the settings until you find the *StackLight Elasticsearch-Kibana - Plugin* section. - -4. Tick the *StackLight Infrastructure Alerting Plugin* box and fill-in the required - fields as indicated below. - - .. image:: ../images/elastic_kibana_settings.png - :width: 800 - :align: center - - a. Specify the number of days of retention for your data. - #. Specify the JVM heap size for Elastisearch. See the configuration recommendations below. - - .. note:: By default, 1GB of heap memory is allocated to the Elasticsearch process. - This value is too small to run Elasticsearch for anything else than local testing. - To run Elasticsearch in production you need to allocate at least 4 GB of memory - but it is recommended to allocate 50% of the available memory up to 32 GB maximum. - If you set a value that is greater than the memory size, Elasticsearch won't start. - Keep in mind also to reserve enough memory for the operating system and the other services. - - #. At this point, you can choose to either edit the *Advanced settings* or let the plugin - decide the defaults for you. The advanced settings are used to specify advanced settings - when Elasticsearch and Kibana are installed on a cluster of nodes. - To manually configure those advanced settings, check the *Advanced settings* box and fill-in - the required parameters. - -5. Tick the *Enable TLS for Kibana* box if you want to encrypt your - Kibana credentials (username, password). Then, fill-in the required - fields as indicated below. - - .. image:: ../images/tls_settings.png - :width: 800 - :align: center - - a. Specify the DNS name of the Kibana server. This parameter is used - to create a link in the Fuel dashboard to the Kibana server. - #. Specify the location of a PEM file that contains the certificate - and the private key of the Kibana server that will be used in TLS handchecks - with the client. - -6. Tick the *Use LDAP for Kibana Authentication* box if you want to authenticate - via LDAP to Kibana. Then, fill-in the required fields as indicated below. - - .. image:: ../images/ldap_auth.png - :width: 800 - :align: center - - a. Select the *LDAPS* button if you want to enable LDAP authentication - over SSL. - #. Specify one or several LDAP server addresses separated by a space. Those - addresses must be accessible from the node where Kibana is installed. - Note that addresses external to the *management network* are not routable - by default (see the note below). - #. Specify the LDAP server port number or leave it empty to use the defaults. - #. Specify the *Bind DN* of a user who has search priviliges on the LDAP server. - #. Specify the password of the user identified by the *Bind DN* above. - #. Specify the *Base DN* in the Directory Information Tree (DIT) from where - to search for users. - #. Specify a valid attribute (ex. 'uid') to search for users. The search should - return a unique user entry. - #. Specify a valid search filter (ex. '(objectClass=*') to search for users. - - You can further restrict access to Kibana to those users who - are member of a specific LDAP group. - - a. Tick the *Enable group-based authorization*. - #. Specify the LDAP attribute (ex. memberUid) in the user entry - that identifies the LDAP group membership. - #. Specify the DN of the LDAP group that will be mapped to the *admin role* - #. Specify the DN of the LDAP group that will be mapped to the *viewer role* - - Users who have the *admin role* can modify the Kibana dashboards - or create new ones. Users who have the *Viewer role* can only - visualise the Kibana dashboards. - -7. `Configure your environment - `_. - - .. note:: By default, StackLight is configured to use the *management network*, - of the so-called `Default Node Network Group - `_. - While this default setup may be appropriate for small deployments or - evaluation purposes, it is recommended not to use this network - for StackLight in production. It is instead recommended to create a network - dedicated to StackLight. Using a dedicated network for StackLight should - improve performances and reduce the monitoring footprint. - It will also facilitate access to the Kibana UI after deployment. - -8. Click the *Nodes* tab and assign the *Elasticsearch_Kibana* role - to the node(s) where you want to install the plugin. - - You can see in the example below that the *Elasticsearch_Kibana* - role is assigned to three nodes along side with the - *Alerting_Infrastructure* and the *InfluxDB_Grafana* roles. - Here, the three plugins of the LMA toolchain backend servers are - installed on the same nodes. - - .. image:: ../images/elastic_kibana_role.png - :width: 800 - :align: center - - .. note:: The Elasticsearch clustering for high availability requires - that you assign the *Elasticsearch_Kibana* role to at least three nodes, - but you can assign the *Elasticsearch_Kibana* role up to five nodes. - Note also that is possible to add or remove a node with the *Elasticsearch_Kibana* - role after deployment. - -9. `Adjust the disk partitioning if necessary - `_. - - By default, the Elasticsearch-Kibana Plugin allocates: - - * 20% of the first available disk for the operating system by honoring a range of 15GB minimum and 50GB maximum. - * 10GB for */var/log*. - * At least 30 GB for the Elasticsearch database in */opt/es-data*. - -10. `Deploy your environment - `_. - -.. _plugin_install_verification: - -Plugin verification -------------------- - -Be aware, that depending on the number of nodes and deployment setup, -deploying a Mirantis OpenStack environment can typically take anything -from 20 minutes to several hours. But once your deployment is complete, -you should see a deployment success notification message with two -links to Kibana as shown in the figure below: - -.. image:: ../images/deploy_notif.png - :align: center - :width: 800 - -.. note:: For technical reasons, it was necessary to create two different ports - to enforce the access authorization to Kibana. One port (80) for users with the - *admin role* and one port (81) for users with the *viewer role*. - Be also aware that if Kibana is installed on the *management network*, - you may not have direct access to the UI. Some extra network - configuration may be required to create an SSH tunnel to the *management network*. - -Verifying Elasticsearch -~~~~~~~~~~~~~~~~~~~~~~~ - -You should verify that the Elasticsearch cluster is running properly. -To do that, you need first to retrieve the Elasticsearch cluster VIP address. -Here is how to proceed. - -#. On the Fuel Master node, find the IP address of a node where the Elasticsearch - server is installed using the following command:: - - [root@fuel ~]# fuel nodes - id | status | name | cluster | ip | mac | roles | - ---|----------|------------------|---------|-----|----------------------------| - 1 | ready | Untitled (fa:87) | 1 | ... | ... | elasticsearch_kibana | - 2 | ready | Untitled (12:aa) | 1 | ... | ... | elasticsearch_kibana | - 3 | ready | Untitled (4e:6e) | 1 | ... | ... | elasticsearch_kibana | - - -#. Then `ssh` to anyone of these nodes (ex. *node-1*) and type the command:: - - root@node-1:~# hiera lma::elasticsearch::vip - 10.109.1.5 - - This tells you that the VIP address of your Elasticsearch cluster is *10.109.1.5*. - -#. With that VIP address type the command:: - - curl http://10.109.1.5:9200/ - - The output should look like this:: - - { - "status" : 200, - "name" : "node-3.test.domain.local_es-01", - "cluster_name" : "lma", - "version" : { - "number" : "1.7.4", - "build_hash" : "0d3159b9fc8bc8e367c5c40c09c2a57c0032b32e", - "build_timestamp" : "2015-12-15T11:25:18Z", - "build_snapshot" : false, - "lucene_version" : "4.10.4" - }, - "tagline" : "You Know, for Search" - } - -Verifying Kibana -~~~~~~~~~~~~~~~~ - -From the Fuel dashboard, click on the *Kibana (Admin role)* link -(or enter the IP address and port number if your DNS is not setup), -then enter your credentials. You should be redirected to a Kibana 4 -**Logs Anaytics Dashboard** as shown in the figure below. - -.. image:: ../images/kibana_logs_dash.png - :align: center - :width: 800 +Use the plugin +============== Dashboards management --------------------- -The StackLight Elasticsearch-Kibana Plugin comes with two built-in dashboards: +The StackLight Elasticsearch-Kibana plugin contains two built-in dashboards: - * The Logs Analytics Dashboard that is used to visualize and search the logs. - * The Notifications Analytics Dashboard that is used to visualize and - search the OpenStack notifications if you enabled the feature in the + * The :guilabel:`Logs` Analytics Dashboard is used to visualize and + search the logs. + * The :guilabel:`Notifications` Analytics Dashboard is used to visualize + and search the OpenStack notifications if you enabled the feature in the Collector settings. -You can switch from one dashboard to another by clicking on the top-right *Load* -icon in the toolbar to select the requested dashboard from the list, as shown below. +You can switch from one dashboard to another by clicking on the top-right +:guilabel:`Load` icon on the toolbar to select the requested dashboard from +the list, as shown below: .. image:: ../images/kibana_dash.png :align: center @@ -232,11 +24,11 @@ icon in the toolbar to select the requested dashboard from the list, as shown be Each dashboard provides a single pane of glass for visualizing and searching all the logs and the notifications of your OpenStack environment. -Note that in the Collector settings it is possible to tag the logs by -environment name so that you can distinguish which logs (and notifications) -belong to what environment. -As you can see, the Kibana dashboard for logs is divided in several sections: +In the Collector settings, you can tag the logs by an environment name to +distinguish which logs (and notifications) belong to what environment. + +The Kibana Dashboard for logs is divided into several sections. .. image:: ../images/kibana_logs_sections_1.png :align: center @@ -245,111 +37,64 @@ As you can see, the Kibana dashboard for logs is divided in several sections: 1. A time-picker control that lets you choose the time period you want to select and refresh frequency. -2. A text-box to enter search queries. +2. A text box to enter search queries. 3. Various logs analytics with six different panels: - a. A stack graph showing all the logs per source. - b. A stack graph showing all the logs per severity. - c. A stack graph showing all logs top 10 sources. - d. A stack graph showing all the logs top 10 programs. - e. A stack graph showing all logs top 10 hosts. - f. A graph showing the number of logs per severity. - g. A graph showing the number of logs per role. + a. A stack graph showing all the logs per source. + b. A stack graph showing all the logs per severity. + c. A stack graph showing all logs for top 10 sources. + d. A stack graph showing all the logs for top 10 programs. + e. A stack graph showing all logs for top 10 hosts. + f. A graph showing the number of logs per severity. + g. A graph showing the number of logs per role. 4. A table of log messages sorted in reverse chronological order. -.. image:: ../images/kibana_logs_sections_2.png - :align: center - :width: 800 + .. image:: ../images/kibana_logs_sections_2.png + :align: center + :width: 800 Filters and queries ------------------- -Filters and queries have similar syntax but they are used for different purposes. +Filters and queries have similar syntax but they are used for different +purposes: - * The filters are used to restrict what is displayed in the dashboard. - * The queries are used for free-text search. +* The filters are used to restrict what is displayed in the dashboard. +* The queries are used for free-text search. -You can also combine multiple queries and compare their results. -To further filter the log messages to, for example, select the *deployment_id*, -you need to expand a log entry and then select the *deployment_id* field -by clicking on the magnifying glass icon as shown below. +You can combine multiple queries and compare their results. +You can also further filter the log messages. For example, to select +:guilabel:`deployment_id`: -.. image:: ../images/kibana_logs_filter1.png - :align: center - :width: 800 +#. Expand a log entry. +#. Select the :guilabel:`deployment_id` field by clicking on the magnifying + glass icon as shown below: -This will apply a new filter in the dashboard. + .. image:: ../images/kibana_logs_filter1.png + :align: center + :width: 800 -.. image:: ../images/kibana_logs_filter2.png - :align: center - :width: 800 + This will apply a new filter in the Dashboard: -Filtering will work for any field that has been indexed for the log entries that -are in the dashboard. + .. image:: ../images/kibana_logs_filter2.png + :align: center + :width: 800 -Filters and queries can also use wildcards that can be combined with *field names* like in:: +Filtering works for any field that has been indexed for the log entries that +are in the Dashboard. - programname: * +Filters and queries can also use wildcards that can be combined with the +*field names* like in ``programname: *`` -For example, to display only the Nova logs you could enter:: - - programname:nova* - -in the query textbox as shown below. +For example, to display only the Nova logs, enter ``programname:nova*`` in +the query text box as shown below: .. image:: ../images/kibana_logs_query1.png :align: center :width: 800 -Troubleshooting ---------------- +.. raw:: latex -If you cannot access the Kibana dashboard or you get no data in the dashboard, -follow these troubleshooting tips. - -1. First, check that the StackLight Collector is running properly by following the - `StackLight Collector troubleshooting instructions - `_. - -#. Check that the nodes are able to connect to the Elasticsearch cluster via the VIP address - on port *9200* as explained in the `Verifying Elasticsearch` section above. - -#. On any of the *Elasticsearch_Kibana* role nodes, check the status of the VIP address - and HAProxy resources in the Pacemaker cluster:: - - root@node-1:~# crm resource status vip__es_vip_mgmt - resource vip__es_vip_mgmt is running on: node-1.test.domain.local - - root@node-1:~# crm resource status p_haproxy - resource p_haproxy is running on: node-1.test.domain.local - -#. If the VIP or HAProxy resources are down, restart them:: - - root@node-1:~# crm resource start vip__es_vip_mgmt - root@node-1:~# crm resource start p_haproxy - -#. Check that the Elasticsearch server is up and running:: - - # On both CentOS and Ubuntu - [root@node-1 ~]# /etc/init.d/elasticsearch-es-01 status - -#. If Elasticsearch is down, restart it:: - - # On both CentOS and Ubuntu - [root@node-1 ~]# /etc/init.d/elasticsearch-es-01 start - -#. Check the apache is up and running:: - - # On both CentOS and Ubuntu - [root@node-1 ~]# /etc/init.d/apache2 status - -#. If apache is down, restart it:: - - # On both CentOS and Ubuntu - [root@node-1 ~]# /etc/init.d/apache2 start - -#. Look for errors in the Elasticsearch log files (located at /var/log/elasticsearch/es-01/). - -#. Look for errors in the apache log files (located at /var/log/apache2/). + \pagebreak \ No newline at end of file diff --git a/doc/source/verify.rst b/doc/source/verify.rst new file mode 100644 index 0000000..e91fb18 --- /dev/null +++ b/doc/source/verify.rst @@ -0,0 +1,100 @@ +.. _verification: + +Verify the plugin after deployment +================================== + +Depending on the number of nodes and deployment setup, deploying a +Mirantis OpenStack environment can typically take from 20 minutes +to several hours. But once your deployment is complete, you should see a +deployment success notification message with two links to Kibana as shown +in the picture below: + +.. image:: ../images/deploy_notif.png + :align: center + :width: 800 + +.. note:: For technical reasons, it was necessary to create two different ports + to enforce the access authorization to Kibana: + + * One port (80) for users with the *admin role* + * One port (81) for users with the *viewer role*. + + If Kibana is installed on the *management network*, you may not have + direct access to the Kibana web UI. Some extra network configuration may + be required to create an SSH tunnel to the *management network*. + +.. _verify-elastic: + +Verifying Elasticsearch +----------------------- + +To verify that the Elasticsearch cluster is running properly, first retrieve +the Elasticsearch cluster VIP address: + +#. On the Fuel Master node, find the IP address of a node where the Elasticsearch + server is installed using the :command:`fuel nodes` command. For example: + + .. code-block:: console + + [root@fuel ~]# fuel nodes + id|status|name |cluster|ip |mac |roles | + --|------|----------------|-------|----|-------------------------| + 1 |ready |Untitled (fa:87)| 1 |... |... |elasticsearch_kibana| + 2 |ready |Untitled (12:aa)| 1 |... |... |elasticsearch_kibana| + 3 |ready |Untitled (4e:6e)| 1 |... |... |elasticsearch_kibana| + + +#. Log in to any of these nodes using SSH, for example, to ``node-1``. +#. Run the following command: + + .. code-block:: console + + root@node-1:~# hiera lma::elasticsearch::vip + 10.109.1.5 + + Where ``10.109.1.5`` is the virtual IP address of your Elasticsearch cluster. + +#. With that virtual IP address, run the following command: + + .. code-block:: console + + curl http://10.109.1.5:9200/ + + The output should look as follows: + + .. code-block:: console + + { + "status" : 200, + "name" : "node-3.test.domain.local_es-01", + "cluster_name" : "lma", + "version" : { + "number" : "1.7.4", + "build_hash" : "0d3159b9fc8bc8e367c5c40c09c2a57c0032b32e", + "build_timestamp" : "2015-12-15T11:25:18Z", + "build_snapshot" : false, + "lucene_version" : "4.10.4" + }, + "tagline" : "You Know, for Search" + } + +.. raw:: latex + + \pagebreak + +Verifying Kibana +---------------- + +To verify the Kibana Dashboard: + +#. Log in to the Fuel web UI. +#. Click on the :guilabel:`Kibana (Admin role)` link. + If your DNS is not setup, enter the IP address and the port number. +#. Enter your credentials. + + You should be redirected to the Kibana **Logs Anaytics Dashboard** with four + logs' sections as follows: + + .. image:: ../images/kibana_logs_dash.png + :align: center + :width: 800