Merge "[docs] Restructuring Kibana 0.10.0 User guide"

This commit is contained in:
Jenkins 2016-07-22 08:02:15 +00:00 committed by Gerrit Code Review
commit bb6c112208
20 changed files with 794 additions and 638 deletions

View File

@ -1,59 +1,67 @@
.. _advanced_user_guide:
Advanced Operations
Advanced operations
===================
.. _cluster_operations:
This section describes advanced operations that you can apply to your
Elasticsearch cluster using the StackLight Elasticsearch-Kibana Fuel plugin.
Cluster Operations
.. _cluster_operations:
Cluster operations
------------------
Because of certain limitations in the current implementation of the Fuel plugin, it is necessary to
perform some manual operations after the Elasticsearch cluster is scaled up or scaled down.
Those operations are needed to adjust the replication factor of the Elasticsearch indices,
based on the new number of nodes in the cluster.
There are three types of indices used by the plugin:
Because of limitations in the current implementation of the plugin, manual
operations are required after the Elasticsearch cluster is scaled up or scaled down. Using these operations, you can adjust the replication factor of the
Elasticsearch indices that are based on the new number of nodes on the cluster.
* The log indices named *log-%{+YYYY.MM.dd}* which are created on a daily basis.
* The notification indices named *notification-%{+YYYY.MM.dd}* which are also created on a daily
basis.
* The Kibana index named *kibana-int* which is created once at installation time to store the
templates of the Kibana dashboards.
The plugin uses three types of indices:
Adjusting the replication factor for the *kibana-int* index is
performed automatically by the plugin and therefore there is no need to do any manual operation
for that index when the cluster is scaled up or down.
* The log indices named *log-%{+YYYY.MM.dd}* which are created on a daily basis.
* The notification indices named *notification-%{+YYYY.MM.dd}* which are
created on a daily basis.
* The Kibana index named *kibana-int* which is created once during the
installation to store the templates of the Kibana dashboards.
This is not the case for the replication factor of the other two indices which needs to
be updated manualy as described in the
`official documentation <https://www.elastic.co/guide/en/elasticsearch/reference/1.7/indices-update-settings.html>`_.
Adjusting the replication factor for the *kibana-int* index is performed
automatically by the plugin. Therefore, no manual operation is not required
for this index when the cluster is scaled up or down. But this is not the case
for the replication factor of other two indices that you should manually
update as described in the
`Elasticsearch official documentation <https://www.elastic.co/guide/en/elasticsearch/reference/1.7/indices-update-settings.html>`_.
The following sections provide more details, describing what do when scaling up/down the
Elasticsearch cluster. Scaling up from one node to three nodes, and scaling down from three nodes
to one node, are used as examples. Your mileage may vary but the
principal of (re)configuring the replication factor of the indices should remain the same.
The following sections describe in detail how to scale up and scale down the
Elasticsearch cluster. Scaling up from one node to three nodes and scaling
down from three nodes to one node are used as examples. Your mileage may vary,
but the principal of (re)configuring the replication factor of the indices
should remain the same.
Scaling Up
^^^^^^^^^^
~~~~~~~~~~
The problem the manual operation aims to address is that the replication factor for the old
indices is not updated automatically by the plugin when a new node is added in the cluster. If you
want the old indices to be replicated on the new node(s), you need to adjust the
*number_of_replicas* parameter to the current size of the cluster for those indices as shown below.
The problem that the manual operation aims to address is that the replication
factor for the old indices is not updated automatically by the plugin when
a new node is added in the cluster. If you want the old indices to be
replicated on the new node(s), you need to adjust the *number_of_replicas*
parameter to the current size of the cluster for those indices as shown below.
The output below shows that the replication factor of the indices created before the scale-up is
zero. Here, a scale-up was performed on the 3rd of February, so the indices created after that date
(*log-2016.02.04* here) are automatically updated with the correct number of replicas
(number of cluster nodes - 1). ::
The output below shows that the replication factor of the indices created
before the scale-up is zero. Here, a scale-up was performed on the 3rd of
February, so the indices created after that date (*log-2016.02.04* in this
example) are automatically updated with the correct number of replicas
(number of cluster nodes - 1).
.. code-block:: console
[root@node-1 ~]# curl <VIP>:9200/_cat/indices?v
health status index pri rep docs.count docs.deleted ....
green open log-2016.02.03 5 0 270405 0 ....
green open log-2016.02.04 5 2 1934581 0 ....
If you want the *log-2016.02.03* index to be replicated, update the
*number_of_replicas* parameter of that index as shown below:
Then, if you want the *log-2016.02.03* index to be replicated, you need to update the
*number_of_replicas* parameter of that index as shown below::
.. code-block:: console
[root@node-1 ~]# curl -XPUT <VIP>:9200/log-2016.02.03/_settings \
-d '{ "index": {"number_of_replicas": 2}}'
@ -64,16 +72,17 @@ Then, if you want the *log-2016.02.03* index to be replicated, you need to updat
green open log-2016.02.03 5 2 270405 0 ....
green open log-2016.02.04 5 2 1934581 0 ....
Note that replicating the old indices on the new node(s) will increase the load on the
cluster as well as the size required to store the data.
.. caution:: Replicating the old indices on the new node(s) will increase the
load on the cluster as well as the size required to store the data.
Scaling down
^^^^^^^^^^^^
~~~~~~~~~~~~
Similarly, after a scale-down the *number_of_replicas* of all indices must be
aligned with the new size of the cluster. Not doing so will be reported by LMA as a critical
status for the Elasticsearch cluster::
aligned with the new size of the cluster. Not doing so will be reported by LMA
as a critical status for the Elasticsearch cluster:
.. code-block:: console
[root@node-1 ~]# # the current index health is 'red' after the scale-down
[root@node-1 ~]# curl <VIP>:9200/_cat/indices?v
@ -88,42 +97,3 @@ status for the Elasticsearch cluster::
[root@node-1 ~]# curl <VIP>:9200/_cat/indices?v
health status index pri rep docs.count ....
green open log-2016.02.04 5 2 1934581 ....
.. _network_templates:
Deployment using network templates
----------------------------------
By default, the Elasticsearch-Kibana cluster will be deployed on the Fuel management
network. If this default configuration doesn't meet your requirements, you can leverage the
Fuel `network templates
<https://docs.mirantis.com/openstack/fuel/fuel-8.0/operations.html#using-networking-templates>`_
capability to change that default configuration to use a dedicated network instead.
Below is a network template example to define a new network named `monitoring`.
.. literalinclude:: ./network_template.yaml
You can use this configuration example as a starting point and adapt it to your requirements.
The deployment of the environment should work as described in the :ref:`User Guide
<user_guide>` excepted that before deploying the environment, you will have to:
* Upload the network template::
$ fuel2 network-template upload -f ./network_template <ENVIRONMENT_ID>
* Allocate an IP subnet for the `monitoring` network::
$ fuel2 network-group create -N <ENVIRONMENT_ID> -C 10.109.5.0/24 monitoring
* Adjust the IP range through the Fuel user interface (optional).
.. image:: ../images/network_group_configuration.png
:width: 800
:align: center
* Deploy the environment.
For more details, please refer to the official `Fuel documentation
<https://docs.mirantis.com/openstack/fuel/fuel-8.0/operations.html#using-networking-templates>`_.

View File

@ -1,8 +0,0 @@
.. _user_appendix:
Appendix
========
* The `Elasticsearch-Kibana plugin <https://github.com/openstack/fuel-plugin-elasticsearch-kibana>`_ project at GitHub.
* The official `Kibana documentation <https://www.elastic.co/guide/en/kibana/3.0/index.html>`_.
* The official `Elasticsearch documentation <https://www.elastic.co/guide/en/elasticsearch/reference/1.4/index.html>`_.

View File

@ -6,7 +6,7 @@ source_suffix = '.rst'
master_doc = 'index'
project = u'The StackLight Elasticsearch-Kibana Plugin for Fuel'
copyright = u'2015, Mirantis Inc.'
copyright = u'2016, Mirantis Inc.'
version = '0.10'
release = '0.10.0'
@ -19,7 +19,7 @@ html_theme = 'default'
html_static_path = ['_static']
latex_documents = [
('index', 'ElasticsearchKibana.tex', u'The StackLight Elasticsearch-Kibana Plugin for Fuel Documentation',
('index', 'ElasticsearchKibana.tex', u'Fuel StackLight Elasticsearch-Kibana Plugin Guide',
u'Mirantis Inc.', 'manual'),
]

138
doc/source/configure.rst Normal file
View File

@ -0,0 +1,138 @@
.. _plugin_configuration:
Configure the plugin during an environment deployment
=====================================================
To configure the StackLight Elasticsearch-Kibana plugin during an environment
deployment:
#. Using the Fuel web UI,
`Create a new environment <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/create-environment/start-create-env.html>`_.
#. In the Fuel web UI, click the :guilabel:`Settings` tab and select the
:guilabel:`Other` category.
#. Scroll down through the settings to find the
:guilabel:`StackLight Elasticsearch-Kibana Plugin` section.
#. Select :guilabel:`StackLight Infrastructure Alerting Plugin` and fill in
the required fields as follows:
.. image:: ../images/elastic_kibana_settings.png
:width: 800
:align: center
#. Specify the number of days to retain your data.
#. Specify the :guilabel:`JVM heap size` for Elastisearch. Use the tips
below:
* By default, 1 GB of heap memory is allocated to the Elasticsearch
process. This value is enough to run Elasticsearch for local testing
only.
* To run Elasticsearch in production, allocate minimum 4 GB of memory.
But we recommend allocating 50% of the available memory up to 32 GB
maximum.
* If you set a value greater than the memory size, Elasticsearch will
not start.
* Reserve enough memory for operating system and other services.
#. Select and edit :guilabel:`Advanced settings` if Elasticsearch and
Kibana are installed on a cluster of nodes.
#. Select :guilabel:`Enable TLS for Kibana` if you want to encrypt your
Kibana credentials (username, password). Then, fill in the required fields
as follows:
.. image:: ../images/tls_settings.png
:width: 800
:align: center
#. Specify the DNS name of the Kibana server. This parameter is used
to create a link in the Fuel dashboard to the Kibana server.
#. Specify the location of a PEM file that contains the certificate
and the private key of the Kibana server that will be used in TLS handchecks
with the client.
#. If you want to authenticate through LDAP to Kibana, select
:guilabel:`Use LDAP for Kibana authentication`. Then, fill in the required
fields as follows:
.. image:: ../images/ldap_auth.png
:width: 800
:align: center
#. Select :guilabel:`LDAPS` if you want to enable LDAP authentication
over SSL.
#. Specify one or several :guilabel:`LDAP servers` addresses separated by
space. Those addresses must be accessible from the node where Kibana
is installed.
The addresses that are external to the *management network* are not
routable by default (see more details in step 7).
#. Specify the LDAP server :guilabel:`Port` number or leave it empty to
use the defaults.
#. Specify the :guilabel:`Bind DN` of a user who has search privileges on
the LDAP server.
#. Specify the password of the user identified by the :guilabel:`Bind DN`
selected in the above field.
#. Specify the :guilabel:`User search base DN` in the Directory
Information Tree (DIT) from where to search for users.
#. Specify a valid attribute to search for users, for example, ``uid``.
The search should return a unique user entry.
#. Specify a valid search filter to search for users, for example,
``(objectClass=*)``
You can further restrict access to Kibana to those users who
are members of a specific LDAP group:
#. Select :guilabel:`Enable group-based authorization`.
#. Specify the :guilabel:`LDAP attribute` in the user entry that identifies
the LDAP group membership, for example, ``memberUid``.
#. Specify the DN of the LDAP group that will be mapped to the *admin role*.
#. Specify the DN of the LDAP group that will be mapped to the *viewer role*.
Users who have the *admin role* can modify the Kibana dashboards
or create new ones. Users who have the *viewer role* can only
view the Kibana dashboards.
#. In the Fuel web UI,
`configure your environment <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment.html>`_.
.. caution:: By default, StackLight is configured to use the
*management network* of the so-called
`Default Node Network Group <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/network-settings.html>`_.
While this default setup may be appropriate for small deployments or
evaluation purposes, we recommend not to use this network
for StackLight in production. Instead, create a network
dedicated to StackLight to improve performance and reduce the monitoring
footprint. It will also facilitate access to the Kibana UI after
deployment.
#. Click the :guilabel:`Nodes` tab and assign the *Elasticsearch_Kibana* role
to the node(s) where you want to install the plugin.
The example below shows that the *Elasticsearch_Kibana* role is assigned
to three nodes alongside with the *Alerting_Infrastructure* and the
*InfluxDB_Grafana* roles. The three plugins of the LMA toolchain back-end
servers are installed on the same nodes.
.. image:: ../images/elastic_kibana_role.png
:width: 800
:align: center
.. note:: The Elasticsearch clustering for high availability requires
that you assign the *Elasticsearch_Kibana* role to at least three nodes,
but you can assign the *Elasticsearch_Kibana* role up to five nodes.
You can also add or remove a node with the *Elasticsearch_Kibana*
role after deployment.
#. If required, `adjust the disk partitioning <http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/customize-partitions.html>`_.
By default, the Elasticsearch-Kibana plugin allocates:
* 20% of the first available disk for the operating system by honoring
a range of 15 GB minimum and 50 GB maximum.
* 10 GB for ``/var/log``.
* At least 30 GB for the Elasticsearch database in ``/opt/es-data``.
10. `Deploy your environment
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/deploy-environment.html>`_.

View File

@ -0,0 +1,21 @@
Key terms
=========
The table below lists the key terms that are used in this document.
+---------------------+--------------------------------------------------------------------------------------+
| **Term** | **Definition** |
+=====================+======================================================================================+
| StackLight Collector| A smart monitoring agent running on every node which collects and processes the logs |
| | and the notifications of your OpenStack environment. |
+---------------------+--------------------------------------------------------------------------------------+
| Elasticsearch | An open-source (Apache-licensed) application based on the Lucene search engine |
| | that makes data-like log messages easy to explore and correlate. |
| | |
| | Elasticsearch is written in Java and uses Lucene internally for all of its indexing |
| | and searching, but it aims to make full-text search easy by hiding the complexities |
| | of Lucene behind a simple, coherent RESTful API. |
+---------------------+--------------------------------------------------------------------------------------+
| Kibana | An open-source (Apache-licensed) browser-based analytics and search dashboard for |
| | Elasticsearch. Kibana is easy to setup and use. |
+---------------------+--------------------------------------------------------------------------------------+

View File

@ -0,0 +1,46 @@
.. _network_templates:
Deploy an OpenStack environment using networking templates
==========================================================
By default, the Elasticsearch-Kibana cluster will be deployed on the Fuel
management network. If this default configuration does not meet your
requirements, you can leverage the Fuel
`networking templates' <https://docs.mirantis.com/openstack/fuel/fuel-9.0/operations.html#using-networking-templates>`_
capability to change that default configuration to use a dedicated network
instead.
Below is a networking template example to define a new network named
``monitoring``. You can use this configuration example as a starting point
and adapt it to your requirements.
.. literalinclude:: ./network_template.yaml
**To deploy an environment using networking templates:**
#. Upload the networking template:
.. code-block:: console
$ fuel2 network-template upload -f ./network_template <ENVIRONMENT_ID>
#. Allocate an IP subnet for the ``monitoring`` network:
.. code-block:: console
$ fuel2 network-group create -N <ENVIRONMENT_ID> -C 10.109.5.0/24 monitoring
#. (Optional) Using the Fuel web UI, adjust the IP range:
.. image:: ../images/network_group_configuration.png
:width: 800
:align: center
#. Proceed to :ref:`Configure the plugin during an environment deployment <plugin_configuration>`.
For details on networking templates, see
`Mirantis Operations Guide <https://docs.mirantis.com/openstack/fuel/fuel-9.0/operations.html#using-networking-templates>`_.
.. raw:: latex
\pagebreak

View File

@ -1,22 +1,40 @@
====================================================================
Welcome to the StackLight Elasticsearch-Kibana Plugin Documentation!
====================================================================
=========================================================================
Welcome to the Fuel StackLight Elasticsearch-Kibana plugin documentation!
=========================================================================
User documentation
==================
Overview
========
.. toctree::
:maxdepth: 2
:maxdepth: 1
overview
releases
installation
user
advanced_operations
intro
definitions
requirements
limitations
release_notes
licenses
appendix
references
Indices and Tables
==================
Installing and configuring StackLight Elasticsearch-Kibana plugin for Fuel
==========================================================================
* :ref:`search`
.. toctree::
:maxdepth: 1
install
configure
deploy_with_network_templates
verify
Using StackLight Elasticsearch-Kibana plugin for Fuel
=====================================================
.. toctree::
:maxdepth: 1
user
troubleshooting
advanced_operations

146
doc/source/install.rst Normal file
View File

@ -0,0 +1,146 @@
.. _install:
Install the plugin
==================
Introduction
------------
You can install the StackLight Elasticsearch-Kibana Fuel plugin using one of
the following options:
* Install using the RPM file
* Install from source
The following is a list of software components installed by the StackLight
Elasticsearch-Kibana Fuel plugin:
+---------------+---------------------------------------------+
| Components | Version |
+===============+=============================================+
| Elasticsearch | v2.3.3 for Ubuntu (64-bit) |
+---------------+---------------------------------------------+
| Kibana | v4.5 |
+---------------+---------------------------------------------+
| Apache | Version coming with the Ubuntu distribution |
+---------------+---------------------------------------------+
Install using the RPM file
--------------------------
To install the StackLight Elasticsearch-Kibana Fuel plugin using the RPM file
from the Fuel plugins' catalog:
#. Go to the
`Fuel plugins' Catalog <https://www.mirantis.com/validated-solution-integrations/fuel-plugins>`_.
#. From the :guilabel:`Filter` drop-down menu, select the Mirantis OpenStack
version you are using and the :guilabel:`MONITORING` category.
#. Download the RPM file.
#. Copy the RPM file to the Fuel Master node:
.. code-block:: console
[root@home ~]# scp elasticsearch_kibana-0.10-0.10.0-0.noarch.rpm \
root@<Fuel Master node IP address>:
#. Install the plugin using the `Fuel Plugins CLI
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/cli/cli_plugins.html>`_:
.. code-block:: console
[root@fuel ~]# fuel plugins --install elasticsearch_kibana-0.10-0.10.0-0.noarch.rpm
#. Verify that the plugin is installed correctly:
.. code-block:: console
[root@fuel ~]# fuel plugins --list
id | name | version | package_version
---|----------------------|----------|----------------
1 | elasticsearch_kibana | 0.10.0 | 4.0.0
Install from source
-------------------
Alternatively, you can build the RPM file of the plugin from source if, for
example, you want to test the latest features of the master branch or
customize the plugin.
.. caution:: Running a Fuel plugin that you built from source is at your
own risk and is not supported.
To install the StackLight Elasticsearch-Kibana plugin from source, at first
prepare an environment to build the RPM file. We recommend building the RPM
file directly on the Fuel Master node not to copy that file later on.
**To prepare an environment for building the plugin on the Fuel Master node:**
#. Install the standard Linux development tools:
.. code-block:: console
[root@home ~] yum install createrepo rpm rpm-build dpkg-devel
#. Install ``pip``:
.. code-block:: console
[root@home ~] easy_install pip
#. Install the Fuel Plugin Builder (the ``fpb`` command line) using ``pip``:
.. code-block:: console
[root@home ~] pip install fuel-plugin-builder
.. note:: You may also need to build the Fuel Plugin Builder if the
package version of the plugin is higher than the package version supported
by the Fuel Plugin Builder you get from ``pypi``. For instructions on how
to build the Fuel Plugin Builder, see the
*Install Fuel Plugin Builder* section of the
`Fuel Plugin SDK Guide <http://docs.openstack.org/developer/fuel-docs/plugindocs/fuel-plugin-sdk-guide/create-plugin/install-plugin-builder.html>`_.
#. Clone the plugin repository:
.. code-block:: console
[root@home ~] git clone \
https://github.com/openstack/fuel-plugin-elasticsearch-kibana.git
#. Verify that the plugin is valid:
.. code-block:: console
[root@home ~] fpb --check ./fuel-plugin-elasticsearch-kibana
#. Build the plugin:
.. code-block:: console
[root@home ~] fpb --build ./fuel-plugin-elasticsearch-kibana
**To install the plugin:**
#. Once you create the RPM file, install the plugin:
.. code-block:: console
[root@fuel ~] fuel plugins --install \
./fuel-plugin-elasticsearch-kibana/*.noarch.rpm
#. Verify that the plugin is installed correctly:
.. code-block:: console
[root@fuel ~]# fuel plugins --list
id | name | version | package_version
---|----------------------|----------|----------------
1 | elasticsearch_kibana | 0.10.0 | 4.0.0
.. raw:: latex
\pagebreak

View File

@ -1,100 +0,0 @@
.. _user_installation:
Installation Guide
==================
StackLight Elasticsearch-Kibana Plugin installation using the RPM file of the Fuel Plugins Catalog
--------------------------------------------------------------------------------------------------
To install the StackLight Elasticsearch-Kibana Fuel Plugin using the RPM file of the Fuel Plugins
Catalog, you need to follow these steps:
1. Select, using the MONITORING category and Mirantis OpenStack version you are using,
the RPM file you want to download from the `Fuel Plugins Catalog
<https://www.mirantis.com/validated-solution-integrations/fuel-plugins>`_.
2. Copy the RPM file to the Fuel Master node::
[root@home ~]# scp elasticsearch_kibana-0.10-0.10.0-0.noarch.rpm \
root@<Fuel Master node IP address>:
3. Install the plugin using the `Fuel CLI
<http://docs.mirantis.com/openstack/fuel/fuel-8.0/user-guide.html#using-fuel-cli>`_::
[root@fuel ~]# fuel plugins --install elasticsearch_kibana-0.10-0.10.0-0.noarch.rpm
4. Verify that the plugin is installed correctly::
[root@fuel ~]# fuel plugins --list
id | name | version | package_version
---|----------------------|----------|----------------
1 | elasticsearch_kibana | 0.10.0 | 4.0.0
StackLight Elasticsearch-Kibana Fuel Plugin installation from source
--------------------------------------------------------------------
Alternatively, you may want to build the RPM file of the plugin from source if,
for example, you want to test the latest features of the master branch or customize the plugin.
.. note:: Be aware that running a Fuel plugin that you built yourself is at your
own risk and will not be supported.
To install StacLight Elasticsearch-Kibana Plugin from source,
you first need to prepare an environment to build the RPM file.
The recommended approach is to build the RPM file directly onto the Fuel Master
node so that you won't have to copy that file later on.
**Prepare an environment for building the plugin on the Fuel Master Node**
1. Install the standard Linux development tools::
[root@home ~] yum install createrepo rpm rpm-build dpkg-devel
2. Install the Fuel Plugin Builder. To do that, you should first get pip::
[root@home ~] easy_install pip
3. Then install the Fuel Plugin Builder (the `fpb` command line) with `pip`::
[root@home ~] pip install fuel-plugin-builder
.. note:: You may also need to build the Fuel Plugin Builder if the package version of the
plugin is higher than the package version supported by the Fuel Plugin Builder you get from `pypi`.
In this case, please refer to the section "Preparing an environment for plugin development"
of the `Fuel Plugins wiki <https://wiki.openstack.org/wiki/Fuel/Plugins>`_,
if you need further instructions about how to build the Fuel Plugin Builder.
4. Clone the plugin git repository::
[root@home ~] git clone \
https://github.com/openstack/fuel-plugin-elasticsearch-kibana.git
5. Check that the plugin is valid::
[root@home ~] fpb --check ./fuel-plugin-elasticsearch-kibana
6. And finally, build the plugin::
[root@home ~] fpb --build ./fuel-plugin-elasticsearch-kibana
7. Now that you have created the RPM file, you can install the plugin using the `fuel plugins --install` command::
[root@fuel ~] fuel plugins --install \
./fuel-plugin-elasticsearch-kibana/*.noarch.rpm
StackLight Elasticsearch-Kibana Fuel Plugin software components
---------------------------------------------------------------
List of software components installed by the plugin
+---------------+--------------------------------------------------------+
| Components | Version |
+===============+========================================================+
| Elasticsearch | v2.3.3 for Ubuntu (64-bit) |
+---------------+--------------------------------------------------------+
| Kibana | v4.5 |
+---------------+--------------------------------------------------------+
| Apache | Version coming by default with the Ubuntu distribution |
+---------------+--------------------------------------------------------+

20
doc/source/intro.rst Normal file
View File

@ -0,0 +1,20 @@
.. _intro:
Introduction
============
The **StackLight Elasticsearch-Kibana plugin** is used to install and configure
Elasticsearch and Kibana components that collectively provide access to the
logs and notifications analytics of the so-called Logging, Monitoring, and
Alerting (LMA) Toolchain of Mirantis OpenStack.
These analytics can be used to search and correlate service-affecting
events which may occur on your OpenStack environment. It is an indispensable
tool to troubleshoot problems.
Elasticsearch and Kibana are key components of the
`LMA Toolchain project <https://launchpad.net/lma-toolchain>`_, also known as
StackLight:
.. image:: ../images/toolchain_map.png
:align: center

View File

@ -3,15 +3,15 @@
Licenses
========
Third Party Components
Third-party components
----------------------
+------------------------+-------------------------------------------------------------+------------+
| Name | Project Web Site | License |
+========================+=============================================================+============+
| Elasticsearch | https://www.elastic.co/products/elasticsearch | Apache V2 |
| Elasticsearch | https://www.elastic.co/products/elasticsearch | Apache v2 |
+------------------------+-------------------------------------------------------------+------------+
| Kibana | https://www.elastic.co/products/kibana | Apache V2 |
| Kibana | https://www.elastic.co/products/kibana | Apache v2 |
+------------------------+-------------------------------------------------------------+------------+
Puppet modules
@ -20,15 +20,15 @@ Puppet modules
+------------------------+-------------------------------------------------------------+------------+
| Name | Project Web Site | License |
+========================+=============================================================+============+
| Elasticsearch | https://forge.puppetlabs.com/elasticsearch/elasticsearch | Apache V2 |
| Elasticsearch | https://forge.puppetlabs.com/elasticsearch/elasticsearch | Apache v2 |
+------------------------+-------------------------------------------------------------+------------+
| Concat | https://github.com/puppetlabs/puppetlabs-concat | Apache V2 |
| Concat | https://github.com/puppetlabs/puppetlabs-concat | Apache v2 |
+------------------------+-------------------------------------------------------------+------------+
| Stdlib | https://github.com/puppetlabs/puppetlabs-stdlib | Apache V2 |
| Stdlib | https://github.com/puppetlabs/puppetlabs-stdlib | Apache v2 |
+------------------------+-------------------------------------------------------------+------------+
| Apache | https://github.com/puppetlabs/puppetlabs-apache | Apache V2 |
| Apache | https://github.com/puppetlabs/puppetlabs-apache | Apache v2 |
+------------------------+-------------------------------------------------------------+------------+
| Firewall | https://github.com/puppetlabs/puppetlabs-firewall | Apache V2 |
| Firewall | https://github.com/puppetlabs/puppetlabs-firewall | Apache v2 |
+------------------------+-------------------------------------------------------------+------------+
| Datacat | https://github.com/richardc/puppet-datacat | Apache V2 |
| Datacat | https://github.com/richardc/puppet-datacat | Apache v2 |
+------------------------+-------------------------------------------------------------+------------+

View File

@ -0,0 +1,16 @@
.. _limitations:
Limitations
===========
The StackLight Elasticsearch-Kibana plugin 0.10.0 has the following
limitations:
* Currently, the maximum size of an Elasticsearch cluster that can be
installed by Fuel is limited to five nodes. But each node of an Elasticsearch
cluster is configured as *master candidate* and a *storage node*. This means
that each node of the Elasticsearch cluster can be selected as master, and
all nodes will store data.
* The :ref:`cluster operations <cluster_operations>` may require manual
operations.

View File

@ -1,73 +0,0 @@
.. _user_overview:
Overview
========
The **StackLight Elasticsearch-Kibana Plugin** is used to install and configure
Elasticsearch and Kibana which collectively provide access to the logs and
notifications analytics of the so-called Logging, Monitoring and Alerting (LMA)
Toolchain of Mirantis OpenStack.
These analytics can be used to search and correlate service-affecting
events which occurred in your OpenStack environment. It is an indispensable
tool to troubleshooting problems.
Elasticsearch and Kibana are key components
of the `LMA Toolchain project <https://launchpad.net/lma-toolchain>`_
as shown in the figure below.
.. image:: ../images/toolchain_map.png
:align: center
.. _plugin_requirements:
Requirements
------------
+------------------------+------------------------------------------------------------------------------------------+
| **Requirement** | **Version/Comment** |
+========================+==========================================================================================+
| Disk space | The plugin's specification requires to provision at least 15GB of disk space for the |
| | system, 10GB for the logs and 30GB for the database. As a result, the installation |
| | of the plugin will fail if there is less than 55GB of disk space available on the node. |
+------------------------+------------------------------------------------------------------------------------------+
| Mirantis OpenStack | 8.0, 9.0 |
+------------------------+------------------------------------------------------------------------------------------+
| Hardware configuration | The hardware configuration (RAM, CPU, disk) required by this plugin depends on the size |
| | of your cloud environment and other parameters like the retention period and log level. |
| | |
| | A typical setup would at least require a quad-core server with 8GB of RAM and fast disks |
| | (ideally, SSDs). The actual disk space you need to run the plugin depends on several |
| | factors including the size of your OpenStack environment, the retention period, the |
| | logging level and workload. The more of the above, the more disk space you will need to |
| | run the Elaticsearch-Kibana Plugin. It is also highly recommended to use dedicated |
| | disk(s) for your data storage. |
+------------------------+------------------------------------------------------------------------------------------+
Limitations
-----------
Currently, the maximum size of an Elasticsearch cluster that can be installed by Fuel is limited to five nodes.
Each node of an Elasticsearch cluster is configured as *master candidate* and a *storage node*.
This means, that each node of the Elasticsearch cluster can be elected as a master and all nodes will store data.
The :ref:`cluster operations <cluster_operations>` can require some manual operations some times.
Key terms, acronyms and abbreviations
-------------------------------------
+----------------------------+--------------------------------------------------------------------------------------+
| **Terms & acronyms** | **Definition** |
+============================+======================================================================================+
| The Collector | The StackLight Collector is a smart monitoring agent running on every node which |
| | collects and processes the logs and the notifications of your OpenStack environment. |
+----------------------------+--------------------------------------------------------------------------------------+
| Elasticsearch | An open source (Apache Licensed) application based on the Lucene™ search engine |
| | that makes data like log messages easy to explore and correlate. |
| | |
| | Elasticsearch is written in Java and uses Lucene internally for all of its indexing |
| | and searching, but it aims to make full-text search easy by hiding the complexities |
| | of Lucene behind a simple, coherent, RESTful API. |
+----------------------------+--------------------------------------------------------------------------------------+
| Kibana | An open source (Apache Licensed), browser based analytics and search dashboard for |
| | Elasticsearch. Kibana is easy to setup and start using. |
+----------------------------+--------------------------------------------------------------------------------------+

View File

@ -0,0 +1,8 @@
.. _refs:
References
==========
* `GitHub project <https://github.com/openstack/fuel-plugin-elasticsearch-kibana>`_
* `Official Kibana documentation <https://www.elastic.co/guide/en/kibana/3.0/index.html>`_
* `Official Elasticsearch documentation <https://www.elastic.co/guide/en/elasticsearch/reference/1.4/index.html>`_

View File

@ -0,0 +1,54 @@
.. _releases:
Release notes
=============
Version 0.10.0
--------------
The StackLight Elasticsearch-Kibana plugin 0.10.0 contains the following
updates:
* Added support for the LDAP(S) authentication to access the Kibana UI.
* Added support for the TLS encryption to access the Kibana UI.
To configure the TLS termination, update the plugin settings with a PEM
file obtained by concatenating the SSL certificate with the private key.
* Upgraded to Elasticsearch v2.3.3.
* Upgraded to Kibana v4.5.
* Fixed the issue in logs and notifications being dropped during a long
Elasticsearch outage. See
`#1566748 <https://bugs.launchpad.net/lma-toolchain/+bug/1566748>`_.
Version 0.9.0
-------------
The StackLight Elasticsearch-Kibana plugin 0.9.0 contains the following
updates:
* Added support for Elasticsearch and Kibana clustering for scale-out and
high availability of those services.
* Upgraded to Elasticsearch 1.7.4.
* Upgraded to Kibana 3.1.3.
Version 0.8.0
-------------
The StackLight Elasticsearch-Kibana plugin 0.8.0 contains the following
updates:
* Added support for the ``elasticsearch_kibana`` Fuel plugin role instead of
the ``base-os`` role which had several limitations.
* Added support for the retention policy configuration with
`Elastic Curator <https://github.com/elastic/curator>`_.
* Upgraded to Elasticsearch 1.4.5.
Version 0.7.0
-------------
The initial release of the plugin (beta version).
.. raw:: latex
\pagebreak

View File

@ -1,46 +0,0 @@
.. _releases:
Release Notes
=============
Version 0.10.0
--------------
* Changes
* Add support for LDAP(S) authentication to access the Kibana UI.
* Add support for TLS encryption to access the Kibana UI.
A PEM file (obtained by concatenating the SSL certificate with the private key)
must be provided in the settings of the plugin to configure the TLS termination.
* Upgrade to Elasticsearch v2.3.3.
* Upgrade to Kibana v4.5.
* Bug Fixes
* Logs and notifications are dropped during a "long" Elasticsearch outage (`#1566748
<https://bugs.launchpad.net/lma-toolchain/+bug/1566748>`_).
Version 0.9.0
-------------
* Support Elasticsearch and Kibana clustering for scale-out and high
availability of those services.
* Upgrade to Elasticsearch 1.7.4.
* Upgrade to Kibana 3.1.3.
Version 0.8.0
-------------
* Add support for the "elasticsearch_kibana" Fuel Plugin role instead of
the "base-os" role which had several limitations.
* Add support for retention policy configuration with `Elastic Curator <https://github.com/elastic/curator>`_.
* Upgrade to Elasticsearch 1.4.5.
Version 0.7.0
-------------
* Initial release of the plugin. This is a beta version.

View File

@ -0,0 +1,35 @@
.. _plugin_requirements:
Requirements
============
The StackLight Elasticsearch-Kibana Plugin 0.10.0 has the following
requirements:
+------------------------+------------------------------------------------------------------------------------------+
| **Requirement** | **Version/Comment** |
+========================+==========================================================================================+
| Disk space | The plugin's specification requires: |
| | |
| | * provisioning at least 15 GB of disk space for the system |
| | * 10 GB for the logs |
| | * 30 GB for the database |
| | |
| | As a result, the installation of the plugin will fail if there is less than **55 GB** |
| | of disk space available on the node. |
+------------------------+------------------------------------------------------------------------------------------+
| Mirantis OpenStack | 8.0, 9.0 |
+------------------------+------------------------------------------------------------------------------------------+
| Hardware configuration | The hardware configuration (RAM, CPU, disk) depends on the size of your cloud environment|
| | of your cloud environment and other parameters such as the retention period and log |
| | level. |
| | |
| | A typical setup at least requires a quad-core server with 8 GB of RAM and fast disks |
| | (ideally, SSDs). The actual disk space you need to run the plugin on depends on several |
| | factors including the size of your OpenStack environment, the retention period, the |
| | logging level, and workload. The more of the above, the more disk space you need to |
| | run the Elaticsearch-Kibana plugin. We also highly recommend using dedicated |
| | disk(s) for your data storage. |
+------------------------+------------------------------------------------------------------------------------------+

View File

@ -0,0 +1,66 @@
Troubleshooting
===============
If you cannot access the Kibana Dashboard or you get no data in the Dashboard,
use the following troubleshooting tips:
#. Verify that the StackLight Collector is running properly. For details, see
the `StackLight Collector <http://fuel-plugin-lma-collector.readthedocs.io/>`_
troubleshooting instructions.
#. Verify that the nodes can connect to the Elasticsearch cluster
through the virtual IP address on port ``9200`` as described in the
:ref:`verify-elastic` section.
#. On any of the *Elasticsearch_Kibana* role nodes, check the status of the
virtual IP address and HAProxy resources on the Pacemaker cluster:
.. code-block:: console
root@node-1:~# crm resource status vip__es_vip_mgmt
resource vip__es_vip_mgmt is running on: node-1.test.domain.local
root@node-1:~# crm resource status p_haproxy
resource p_haproxy is running on: node-1.test.domain.local
#. If the virtual IP or HAProxy resources are down, restart them:
.. code-block:: console
root@node-1:~# crm resource start vip__es_vip_mgmt
root@node-1:~# crm resource start p_haproxy
#. Verify that the Elasticsearch server is up and running on both CentOS and
Ubuntu:
.. code-block:: console
[root@node-1 ~]# /etc/init.d/elasticsearch-es-01 status
#. If Elasticsearch is down, restart it on both CentOS and Ubuntu:
.. code-block:: console
[root@node-1 ~]# /etc/init.d/elasticsearch-es-01 start
#. Verify that Apache is up and running on both CentOS and Ubuntu:
.. code-block:: console
[root@node-1 ~]# /etc/init.d/apache2 status
#. If Apache is down, restart it on both CentOS and Ubuntu:
.. code-block:: console
[root@node-1 ~]# /etc/init.d/apache2 start
#. Look for errors in the Elasticsearch log files located at
``/var/log/elasticsearch/es-01/``.
#. Look for errors in the Apache log files located at ``/var/log/apache2/``.
.. raw:: latex
\pagebreak

View File

@ -1,230 +1,22 @@
.. _user_guide:
.. _user:
User Guide
==========
.. _plugin_configuration:
Plugin configuration
--------------------
To configure the **StackLight Elasticsearch-Kibana Plugin**, you need to follow these steps:
1. `Create a new environment
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/create-environment/start-create-env.html>`_.
2. Click on the *Settings* tab of the Fuel web UI and select the *Other* category.
3. Scroll down through the settings until you find the *StackLight Elasticsearch-Kibana
Plugin* section.
4. Tick the *StackLight Infrastructure Alerting Plugin* box and fill-in the required
fields as indicated below.
.. image:: ../images/elastic_kibana_settings.png
:width: 800
:align: center
a. Specify the number of days of retention for your data.
#. Specify the JVM heap size for Elastisearch. See the configuration recommendations below.
.. note:: By default, 1GB of heap memory is allocated to the Elasticsearch process.
This value is too small to run Elasticsearch for anything else than local testing.
To run Elasticsearch in production you need to allocate at least 4 GB of memory
but it is recommended to allocate 50% of the available memory up to 32 GB maximum.
If you set a value that is greater than the memory size, Elasticsearch won't start.
Keep in mind also to reserve enough memory for the operating system and the other services.
#. At this point, you can choose to either edit the *Advanced settings* or let the plugin
decide the defaults for you. The advanced settings are used to specify advanced settings
when Elasticsearch and Kibana are installed on a cluster of nodes.
To manually configure those advanced settings, check the *Advanced settings* box and fill-in
the required parameters.
5. Tick the *Enable TLS for Kibana* box if you want to encrypt your
Kibana credentials (username, password). Then, fill-in the required
fields as indicated below.
.. image:: ../images/tls_settings.png
:width: 800
:align: center
a. Specify the DNS name of the Kibana server. This parameter is used
to create a link in the Fuel dashboard to the Kibana server.
#. Specify the location of a PEM file that contains the certificate
and the private key of the Kibana server that will be used in TLS handchecks
with the client.
6. Tick the *Use LDAP for Kibana Authentication* box if you want to authenticate
via LDAP to Kibana. Then, fill-in the required fields as indicated below.
.. image:: ../images/ldap_auth.png
:width: 800
:align: center
a. Select the *LDAPS* button if you want to enable LDAP authentication
over SSL.
#. Specify one or several LDAP server addresses separated by a space. Those
addresses must be accessible from the node where Kibana is installed.
Note that addresses external to the *management network* are not routable
by default (see the note below).
#. Specify the LDAP server port number or leave it empty to use the defaults.
#. Specify the *Bind DN* of a user who has search priviliges on the LDAP server.
#. Specify the password of the user identified by the *Bind DN* above.
#. Specify the *Base DN* in the Directory Information Tree (DIT) from where
to search for users.
#. Specify a valid attribute (ex. 'uid') to search for users. The search should
return a unique user entry.
#. Specify a valid search filter (ex. '(objectClass=*') to search for users.
You can further restrict access to Kibana to those users who
are member of a specific LDAP group.
a. Tick the *Enable group-based authorization*.
#. Specify the LDAP attribute (ex. memberUid) in the user entry
that identifies the LDAP group membership.
#. Specify the DN of the LDAP group that will be mapped to the *admin role*
#. Specify the DN of the LDAP group that will be mapped to the *viewer role*
Users who have the *admin role* can modify the Kibana dashboards
or create new ones. Users who have the *Viewer role* can only
visualise the Kibana dashboards.
7. `Configure your environment
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment.html>`_.
.. note:: By default, StackLight is configured to use the *management network*,
of the so-called `Default Node Network Group
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/network-settings.html>`_.
While this default setup may be appropriate for small deployments or
evaluation purposes, it is recommended not to use this network
for StackLight in production. It is instead recommended to create a network
dedicated to StackLight. Using a dedicated network for StackLight should
improve performances and reduce the monitoring footprint.
It will also facilitate access to the Kibana UI after deployment.
8. Click the *Nodes* tab and assign the *Elasticsearch_Kibana* role
to the node(s) where you want to install the plugin.
You can see in the example below that the *Elasticsearch_Kibana*
role is assigned to three nodes along side with the
*Alerting_Infrastructure* and the *InfluxDB_Grafana* roles.
Here, the three plugins of the LMA toolchain backend servers are
installed on the same nodes.
.. image:: ../images/elastic_kibana_role.png
:width: 800
:align: center
.. note:: The Elasticsearch clustering for high availability requires
that you assign the *Elasticsearch_Kibana* role to at least three nodes,
but you can assign the *Elasticsearch_Kibana* role up to five nodes.
Note also that is possible to add or remove a node with the *Elasticsearch_Kibana*
role after deployment.
9. `Adjust the disk partitioning if necessary
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/customize-partitions.html>`_.
By default, the Elasticsearch-Kibana Plugin allocates:
* 20% of the first available disk for the operating system by honoring a range of 15GB minimum and 50GB maximum.
* 10GB for */var/log*.
* At least 30 GB for the Elasticsearch database in */opt/es-data*.
10. `Deploy your environment
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/deploy-environment.html>`_.
.. _plugin_install_verification:
Plugin verification
-------------------
Be aware, that depending on the number of nodes and deployment setup,
deploying a Mirantis OpenStack environment can typically take anything
from 20 minutes to several hours. But once your deployment is complete,
you should see a deployment success notification message with two
links to Kibana as shown in the figure below:
.. image:: ../images/deploy_notif.png
:align: center
:width: 800
.. note:: For technical reasons, it was necessary to create two different ports
to enforce the access authorization to Kibana. One port (80) for users with the
*admin role* and one port (81) for users with the *viewer role*.
Be also aware that if Kibana is installed on the *management network*,
you may not have direct access to the UI. Some extra network
configuration may be required to create an SSH tunnel to the *management network*.
Verifying Elasticsearch
~~~~~~~~~~~~~~~~~~~~~~~
You should verify that the Elasticsearch cluster is running properly.
To do that, you need first to retrieve the Elasticsearch cluster VIP address.
Here is how to proceed.
#. On the Fuel Master node, find the IP address of a node where the Elasticsearch
server is installed using the following command::
[root@fuel ~]# fuel nodes
id | status | name | cluster | ip | mac | roles |
---|----------|------------------|---------|-----|----------------------------|
1 | ready | Untitled (fa:87) | 1 | ... | ... | elasticsearch_kibana |
2 | ready | Untitled (12:aa) | 1 | ... | ... | elasticsearch_kibana |
3 | ready | Untitled (4e:6e) | 1 | ... | ... | elasticsearch_kibana |
#. Then `ssh` to anyone of these nodes (ex. *node-1*) and type the command::
root@node-1:~# hiera lma::elasticsearch::vip
10.109.1.5
This tells you that the VIP address of your Elasticsearch cluster is *10.109.1.5*.
#. With that VIP address type the command::
curl http://10.109.1.5:9200/
The output should look like this::
{
"status" : 200,
"name" : "node-3.test.domain.local_es-01",
"cluster_name" : "lma",
"version" : {
"number" : "1.7.4",
"build_hash" : "0d3159b9fc8bc8e367c5c40c09c2a57c0032b32e",
"build_timestamp" : "2015-12-15T11:25:18Z",
"build_snapshot" : false,
"lucene_version" : "4.10.4"
},
"tagline" : "You Know, for Search"
}
Verifying Kibana
~~~~~~~~~~~~~~~~
From the Fuel dashboard, click on the *Kibana (Admin role)* link
(or enter the IP address and port number if your DNS is not setup),
then enter your credentials. You should be redirected to a Kibana 4
**Logs Anaytics Dashboard** as shown in the figure below.
.. image:: ../images/kibana_logs_dash.png
:align: center
:width: 800
Use the plugin
==============
Dashboards management
---------------------
The StackLight Elasticsearch-Kibana Plugin comes with two built-in dashboards:
The StackLight Elasticsearch-Kibana plugin contains two built-in dashboards:
* The Logs Analytics Dashboard that is used to visualize and search the logs.
* The Notifications Analytics Dashboard that is used to visualize and
search the OpenStack notifications if you enabled the feature in the
* The :guilabel:`Logs` Analytics Dashboard is used to visualize and
search the logs.
* The :guilabel:`Notifications` Analytics Dashboard is used to visualize
and search the OpenStack notifications if you enabled the feature in the
Collector settings.
You can switch from one dashboard to another by clicking on the top-right *Load*
icon in the toolbar to select the requested dashboard from the list, as shown below.
You can switch from one dashboard to another by clicking on the top-right
:guilabel:`Load` icon on the toolbar to select the requested dashboard from
the list, as shown below:
.. image:: ../images/kibana_dash.png
:align: center
@ -232,11 +24,11 @@ icon in the toolbar to select the requested dashboard from the list, as shown be
Each dashboard provides a single pane of glass for visualizing and searching
all the logs and the notifications of your OpenStack environment.
Note that in the Collector settings it is possible to tag the logs by
environment name so that you can distinguish which logs (and notifications)
belong to what environment.
As you can see, the Kibana dashboard for logs is divided in several sections:
In the Collector settings, you can tag the logs by an environment name to
distinguish which logs (and notifications) belong to what environment.
The Kibana Dashboard for logs is divided into several sections.
.. image:: ../images/kibana_logs_sections_1.png
:align: center
@ -245,111 +37,64 @@ As you can see, the Kibana dashboard for logs is divided in several sections:
1. A time-picker control that lets you choose the time period you want
to select and refresh frequency.
2. A text-box to enter search queries.
2. A text box to enter search queries.
3. Various logs analytics with six different panels:
a. A stack graph showing all the logs per source.
b. A stack graph showing all the logs per severity.
c. A stack graph showing all logs top 10 sources.
d. A stack graph showing all the logs top 10 programs.
e. A stack graph showing all logs top 10 hosts.
f. A graph showing the number of logs per severity.
g. A graph showing the number of logs per role.
a. A stack graph showing all the logs per source.
b. A stack graph showing all the logs per severity.
c. A stack graph showing all logs for top 10 sources.
d. A stack graph showing all the logs for top 10 programs.
e. A stack graph showing all logs for top 10 hosts.
f. A graph showing the number of logs per severity.
g. A graph showing the number of logs per role.
4. A table of log messages sorted in reverse chronological order.
.. image:: ../images/kibana_logs_sections_2.png
:align: center
:width: 800
.. image:: ../images/kibana_logs_sections_2.png
:align: center
:width: 800
Filters and queries
-------------------
Filters and queries have similar syntax but they are used for different purposes.
Filters and queries have similar syntax but they are used for different
purposes:
* The filters are used to restrict what is displayed in the dashboard.
* The queries are used for free-text search.
* The filters are used to restrict what is displayed in the dashboard.
* The queries are used for free-text search.
You can also combine multiple queries and compare their results.
To further filter the log messages to, for example, select the *deployment_id*,
you need to expand a log entry and then select the *deployment_id* field
by clicking on the magnifying glass icon as shown below.
You can combine multiple queries and compare their results.
You can also further filter the log messages. For example, to select
:guilabel:`deployment_id`:
.. image:: ../images/kibana_logs_filter1.png
:align: center
:width: 800
#. Expand a log entry.
#. Select the :guilabel:`deployment_id` field by clicking on the magnifying
glass icon as shown below:
This will apply a new filter in the dashboard.
.. image:: ../images/kibana_logs_filter1.png
:align: center
:width: 800
.. image:: ../images/kibana_logs_filter2.png
:align: center
:width: 800
This will apply a new filter in the Dashboard:
Filtering will work for any field that has been indexed for the log entries that
are in the dashboard.
.. image:: ../images/kibana_logs_filter2.png
:align: center
:width: 800
Filters and queries can also use wildcards that can be combined with *field names* like in::
Filtering works for any field that has been indexed for the log entries that
are in the Dashboard.
programname: <name>*
Filters and queries can also use wildcards that can be combined with the
*field names* like in ``programname: <name>*``
For example, to display only the Nova logs you could enter::
programname:nova*
in the query textbox as shown below.
For example, to display only the Nova logs, enter ``programname:nova*`` in
the query text box as shown below:
.. image:: ../images/kibana_logs_query1.png
:align: center
:width: 800
Troubleshooting
---------------
.. raw:: latex
If you cannot access the Kibana dashboard or you get no data in the dashboard,
follow these troubleshooting tips.
1. First, check that the StackLight Collector is running properly by following the
`StackLight Collector troubleshooting instructions
<http://fuel-plugin-lma-collector.readthedocs.io/>`_.
#. Check that the nodes are able to connect to the Elasticsearch cluster via the VIP address
on port *9200* as explained in the `Verifying Elasticsearch` section above.
#. On any of the *Elasticsearch_Kibana* role nodes, check the status of the VIP address
and HAProxy resources in the Pacemaker cluster::
root@node-1:~# crm resource status vip__es_vip_mgmt
resource vip__es_vip_mgmt is running on: node-1.test.domain.local
root@node-1:~# crm resource status p_haproxy
resource p_haproxy is running on: node-1.test.domain.local
#. If the VIP or HAProxy resources are down, restart them::
root@node-1:~# crm resource start vip__es_vip_mgmt
root@node-1:~# crm resource start p_haproxy
#. Check that the Elasticsearch server is up and running::
# On both CentOS and Ubuntu
[root@node-1 ~]# /etc/init.d/elasticsearch-es-01 status
#. If Elasticsearch is down, restart it::
# On both CentOS and Ubuntu
[root@node-1 ~]# /etc/init.d/elasticsearch-es-01 start
#. Check the apache is up and running::
# On both CentOS and Ubuntu
[root@node-1 ~]# /etc/init.d/apache2 status
#. If apache is down, restart it::
# On both CentOS and Ubuntu
[root@node-1 ~]# /etc/init.d/apache2 start
#. Look for errors in the Elasticsearch log files (located at /var/log/elasticsearch/es-01/).
#. Look for errors in the apache log files (located at /var/log/apache2/).
\pagebreak

100
doc/source/verify.rst Normal file
View File

@ -0,0 +1,100 @@
.. _verification:
Verify the plugin after deployment
==================================
Depending on the number of nodes and deployment setup, deploying a
Mirantis OpenStack environment can typically take from 20 minutes
to several hours. But once your deployment is complete, you should see a
deployment success notification message with two links to Kibana as shown
in the picture below:
.. image:: ../images/deploy_notif.png
:align: center
:width: 800
.. note:: For technical reasons, it was necessary to create two different ports
to enforce the access authorization to Kibana:
* One port (80) for users with the *admin role*
* One port (81) for users with the *viewer role*.
If Kibana is installed on the *management network*, you may not have
direct access to the Kibana web UI. Some extra network configuration may
be required to create an SSH tunnel to the *management network*.
.. _verify-elastic:
Verifying Elasticsearch
-----------------------
To verify that the Elasticsearch cluster is running properly, first retrieve
the Elasticsearch cluster VIP address:
#. On the Fuel Master node, find the IP address of a node where the Elasticsearch
server is installed using the :command:`fuel nodes` command. For example:
.. code-block:: console
[root@fuel ~]# fuel nodes
id|status|name |cluster|ip |mac |roles |
--|------|----------------|-------|----|-------------------------|
1 |ready |Untitled (fa:87)| 1 |... |... |elasticsearch_kibana|
2 |ready |Untitled (12:aa)| 1 |... |... |elasticsearch_kibana|
3 |ready |Untitled (4e:6e)| 1 |... |... |elasticsearch_kibana|
#. Log in to any of these nodes using SSH, for example, to ``node-1``.
#. Run the following command:
.. code-block:: console
root@node-1:~# hiera lma::elasticsearch::vip
10.109.1.5
Where ``10.109.1.5`` is the virtual IP address of your Elasticsearch cluster.
#. With that virtual IP address, run the following command:
.. code-block:: console
curl http://10.109.1.5:9200/
The output should look as follows:
.. code-block:: console
{
"status" : 200,
"name" : "node-3.test.domain.local_es-01",
"cluster_name" : "lma",
"version" : {
"number" : "1.7.4",
"build_hash" : "0d3159b9fc8bc8e367c5c40c09c2a57c0032b32e",
"build_timestamp" : "2015-12-15T11:25:18Z",
"build_snapshot" : false,
"lucene_version" : "4.10.4"
},
"tagline" : "You Know, for Search"
}
.. raw:: latex
\pagebreak
Verifying Kibana
----------------
To verify the Kibana Dashboard:
#. Log in to the Fuel web UI.
#. Click on the :guilabel:`Kibana (Admin role)` link.
If your DNS is not setup, enter the IP address and the port number.
#. Enter your credentials.
You should be redirected to the Kibana **Logs Anaytics Dashboard** with four
logs' sections as follows:
.. image:: ../images/kibana_logs_dash.png
:align: center
:width: 800