x
/
fuel-plugin-kafka
Code Issues Proposed changes

[Docs] Edits the plugin guide structure 

Edits the table of contents and overall structure of the
Kafka Cluster Plugin documentation.
Text edits will be performed in a separate commit.

Change-Id: Ic0d5e06bef2923e5b0740b47ff254f634f58d9be
This commit is contained in:
Maria Zlatkova 2016-08-31 18:55:55 +03:00 committed by Nadya Shakhat
parent f828aa6b34
commit 84d91c9a78
11 changed files with 177 additions and 136 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/source/conf.py
View File

@ -6,7 +6,7 @@ source_suffix = '.rst'
master_doc = 'index'
project = u'The Kafka Cluster Plugin'
copyright = u'2016, Mirantis Inc.'
version = '0.1'
version = '1.0'
release = '0.1.0'
exclude_patterns = [
]

45
docs/source/configure.rst Normal file
View File

@ -0,0 +1,45 @@
.. _configuration:
Configure the plugin
~~~~~~~~~~~~~~~~~~~~
Once the *Kafka Plugin* is installed following the instructions of
the :ref:`installation`, you can add Kafka nodes to new or
existing Mirantis OpenStack (MOS) environment.
To use the *Kafka Plugin*, you need to add nodes with the **Kafka** role
`Add a node to an OpenStack environment
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/add-nodes.html>`_.
1. Make sure that the plugin is properly installed on the Fuel Master node.
Go to the *Plugins* tab. You should see the following:
.. image:: images/plugins-list.png
:width: 100%
2. Enable the plugin. You can configure additional settings on this step.
Go to the *Environments* tab and select the *The Apache Kafka Message
Broker Plugin* checkbox:
.. image:: images/settings.png
:width: 100%
3. Add nodes to your environment and assign the **Kafka** role.
.. note:: When `adding nodes
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/add-nodes.html>`_
to the environment and `assign or change a role
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/change-roles.html>`_,
do not forget to use an odd number of nodes as recommended in the :ref:`introduction` section.
.. image:: images/assign-role.png
:width: 100%
4. `Verify your network configuration
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/verify-networks.html>`_.
5. `Deploy your changes
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/deploy-environment.html>`_
once you are done with the configuration of your environment.

39
docs/source/description.rst
View File

@ -1,39 +0,0 @@
.. _overview:
Overview
========
The *Kafka Plugin* installs `Apache Kafka <http://kafka.apache.org/>`_ and
`Apache ZooKeeper <https://zookeeper.apache.org/>`_ in a
Mirantis OpenStack (MOS) environment deployed by Fuel.
Apache Kafka is publish-subscribe messaging system. It is fast,
scalable and durable.
The *Kafka Plugin* is created for exchanging messages between various components of StackLight and Ceilometer, but it is generic enough to accommodate other usages.
Plugin provides fuel role *kafka*. Maximum node number is 5.
Recommended minimum is 3 nodes, odd number of nodes required for leader election.
Please refer to the `Kafka 0.10.0 documentation <http://kafka.apache.org/documentation.html>`_
for more information.
Requirements
------------
======================= ================
Requirements Version/Comment
======================= ================
Mirantis OpenStack 9.x
======================= ================
.. _limitations:
Limitations
-----------
* Kafka supports authentication, encryption and authorization. Current version of the
plugin doesn't support any form of security, meaning that the Kafka cluster will be
“open” on the management network.
* Kafka Plugin will not expose configuration properties for all the broker configuration parameters.
This means that the Kafka broker configuration set by the plugin will not be appropriate for every
usage.

76
docs/source/guide.rst
View File

@ -1,76 +0,0 @@
User Guide
==========
Once the *Kafka Plugin* is installed following the instructions of
the :ref:`Installation Guide`, you can add Kafka nodes to new or
existing Mirantis OpenStack (MOS) environment.
Plugin Configuration
--------------------
To use the *Kafka Plugin*, you need to add nodes with Kafka role `Add a node to an OpenStack environment
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/add-nodes.html>`_.
1. Make sure that the plugin is properly installed on the Fuel Master node.
Go to the *Plugins* tab. You should see the following:
.. image:: images/plugins-list.png
:width: 100%
2. Enable the plugin. You can configure additional setting on this step.
Go to the *Environments* tab and select the *The Apache Kafka Message Broker Plugin* checkbox:
.. image:: images/settings.png
:width: 100%
3. Add nodes to your environment and assign the **Kafka** role.
.. note:: When `adding nodes
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/add-nodes.html>`_
to the environment and `assign or change a role
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/change-roles.html>`_,
do not forget to use an odd number of nodes as recommended in :ref:`overview` section.
.. image:: images/assign-role.png
:width: 100%
4. `Verify your network configuration
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/configure-environment/verify-networks.html>`_.
5. `Deploy your changes
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-user-guide/deploy-environment.html>`_
once you are done with the configuration of your environment.
Plugin Verification
-------------------
#. On the Fuel Master node, find the IP address of a node where the
Kafka is installed using the :command:`fuel nodes` command:
.. code-block:: console
[root@fuel ~]# fuel nodes
id|status|name |cluster|ip |mac |roles |
--|------|----------------|-------|----|-------------------------|
1 |ready |Untitled (fa:87)| 1 |... |... |kafka |
2 |ready |Untitled (12:aa)| 1 |... |... |kafka |
3 |ready |Untitled (4e:6e)| 1 |... |... |kafka |
#. Log in to any of these nodes using SSH, for example, to ``node-1``.
#. Run the following command:
.. code-block:: console
root@node-1:~# netstat -ntpl | grep java
tcp6 0 0 :::9092 :::* LISTEN 14702/java
tcp6 0 0 :::2181 :::* LISTEN 9710/java
tcp6 0 0 :::9990 :::* LISTEN 14702/java
You will see that Kafka and Zookeeper are running and listening theirs ports:
2181 - Zookeeper, 9092 and 9990 - Kafka.
#. Additionally you can test sending/receiving messages with instructions on `Quick Start Guide (Step 3 - Step 5) <http://kafka.apache.org/documentation.html#quickstart>`_

31
docs/source/index.rst
View File

@ -1,15 +1,24 @@
=====================================================
Welcome to the Kafka Cluster Plugin Documentation!
=====================================================
==================================================
Welcome to the Kafka Cluster Plugin documentation!
==================================================
Overview
========
.. toctree::
:maxdepth: 1
intro
requirements
limitations
references
Installing and configuring the Kafka Cluster Plugin for Fuel
============================================================
.. toctree::
:maxdepth: 2
description
installation
guide
Indices and Tables
==================
* :ref:`search`
install
configure
verify

15
docs/source/installation.rst → docs/source/install.rst
View File

@ -1,10 +1,7 @@
.. _installation guide:
.. _installation:
Installation Guide
==================
Install the Plugin
------------------
Install the plugin
~~~~~~~~~~~~~~~~~~
To install the *Kafka Plugin*, you need to follow these steps.
@ -18,18 +15,18 @@ To install the *Kafka Plugin*, you need to follow these steps.
<http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-install-guide/intro/intro_fuel_intro.html>`_
with secure copy (scp)::
# scp fuel-plugin-ceilometer-redis/kafka-0.1-0.1.0-1.noarch.rpm /
# scp fuel-plugin-ceilometer-redis/kafka-1.0-0.1.0-1.noarch.rpm /
root@:<the_Fuel_Master_node_IP address>:/tmp
#. Log into the Fuel Master node and install the plugin::
# ssh root@:<the_Fuel_Master_node_IP address>
[root@fuel-master ~]# cd /tmp
[root@fuel-master ~]# fuel plugins --install kafka-0.1-0.1.0-1.noarch.rpm
[root@fuel-master ~]# fuel plugins --install kafka-1.0-0.1.0-1.noarch.rpm
#. Verify that the plugin is installed correctly::
[root@fuel-master ~]# fuel plugins list
id | name | version | package_version | releases
---+-------+---------+-----------------+--------------------
1 | kafka | 0.1.0 | 4.0.0 | ubuntu (mitaka-9.0)
1 | kafka | 1.0.0 | 4.0.0 | ubuntu (mitaka-9.0)

21
docs/source/intro.rst Normal file
View File

@ -0,0 +1,21 @@
.. _introduction:
Introduction
~~~~~~~~~~~~
The *Kafka Plugin* installs `Apache Kafka <http://kafka.apache.org/>`_ and
`Apache ZooKeeper <https://zookeeper.apache.org/>`_ in a
Mirantis OpenStack (MOS) environment deployed by Fuel.
Apache Kafka is a publish-subscribe messaging system. It is fast,
scalable and durable.
The *Kafka Plugin* is created for exchanging messages between various
components of StackLight and Ceilometer, but it is generic enough to
accommodate other usages.
The Plugin implements the *Kafka* Fuel role. The maximum number of nodes for
the *Kafka* Fuel role is 5. To support high-availability, the minimum number
of nodes is 3. You need to have an initial odd number of nodes to comply with
the leader election process constraints.
Please refer to the `Kafka 0.10.0 documentation <http://kafka.apache.org/documentation.html>`_
for more information.

12
docs/source/limitations.rst Normal file
View File

@ -0,0 +1,12 @@
.. _limitations:
Limitations
~~~~~~~~~~~
* Kafka supports authentication, encryption and authorization. Current version of the
plugin doesn't support any form of security, meaning that the Kafka cluster will be
“open” on the management network.
* Kafka Plugin will not expose configuration properties for all the broker configuration parameters.
This means that the Kafka broker configuration set by the plugin will not be appropriate for every
usage.

9
docs/source/references.rst Normal file
View File

@ -0,0 +1,9 @@
.. _references:
References
~~~~~~~~~~
For more information about Kafka, see:
* `Kafka Cluster Plugin project <https://github.com/openstack/fuel-plugin-kafka>`__ at GitHub
* Official `Kafka documentation <http://kafka.apache.org/documentation.html>`__

29
docs/source/requirements.rst Normal file
View File

@ -0,0 +1,29 @@
.. _requirements:
Requirements
~~~~~~~~~~~~
.. tabularcolumns:: |p{4cm}|p{12.5cm}|
+----------------------+-------------------------------------------------------+
|**Requirement** |**Version/Comment** |
+======================+=======================================================+
|Mirantis OpenStack |9.x |
+----------------------+-------------------------------------------------------+
|Disk space |Provide at least 50 GB of disk space. The disk space |
| |depends on the environment size, replication settings, |
| |and data retention. |
+----------------------+-------------------------------------------------------+
|Hardware requirements |The hardware configuration (RAM, CPU, disk(s)) required|
| |by this plugin depends on the size of your cloud |
| |environment and other factors like the replication |
| |factor and retention policy for Kafka queues. |
| |The JVM Heap size is configurable through the plugin |
| |UI. Provide at least 8 GB for the JVM heap size of |
| |Kafka. For an intensive workload, you should provide |
| |16GB of RAM for the JVM heap size of Kafka and 5 GB of |
| |RAM for the JVM heap size of Zookeeper. An average |
| |setup would require a quad-core server with at least |
| |8 GB of RAM for Kafka and 3GB for Zookeeper and access |
| |to a 500+ IOPS disk. |
+----------------------+-------------------------------------------------------+

34
docs/source/verify.rst Normal file
View File

@ -0,0 +1,34 @@
.. _verify_plugin:
Verify the plugin
~~~~~~~~~~~~~~~~~
#. On the Fuel Master node, find the IP address of a node where Kafka is
installed using the :command:`fuel nodes` command:
.. code-block:: console
[root@fuel ~]# fuel nodes
id|status|name |cluster|ip |mac |roles |
--|------|----------------|-------|----|-------------------------|
1 |ready |Untitled (fa:87)| 1 |... |... |kafka |
2 |ready |Untitled (12:aa)| 1 |... |... |kafka |
3 |ready |Untitled (4e:6e)| 1 |... |... |kafka |
#. Log in to any of these nodes using SSH, for example, to ``node-1``.
#. Run the following command:
.. code-block:: console
root@node-1:~# netstat -ntpl | grep java
tcp6 0 0 :::9092 :::* LISTEN 14702/java
tcp6 0 0 :::2181 :::* LISTEN 9710/java
tcp6 0 0 :::9990 :::* LISTEN 14702/java
You will see Zookeeper running and listening on port 2181 and Kafka on ports
9092 and 9990 accordingly.
#. To perform additional verification tests, for example, to try send and
receive messages, see the instructions provided in the
`Quick Start Guide (Step 3 - Step 5) <http://kafka.apache.org/documentation.html#quickstart>`_.