x
/
kloudbuster
Code Issues Proposed changes

Reorganize documentation and add container related information 

Change-Id: Ib1f498f40ded7648168e9cb0d6bfe93742ee47cf
This commit is contained in:
ahothan 2016-05-12 09:05:15 -07:00
parent d5c0fff830
commit a6d20e1aef
14 changed files with 1196 additions and 793 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.rst
View File

@ -17,7 +17,7 @@ CLI/REST or Web User Interface.. you pick what works best for you.
Read the full documentation with feature list, snapshots and diagrams, scale
test design, how-to and installation instructions:
`KloudBuster Documentation <http://kloudbuster.readthedocs.org>`_
`KloudBuster Documentation <http://kloudbuster.readthedocs.io>`_
Licensing

127
doc/source/adv_features.rst Normal file
View File

@ -0,0 +1,127 @@
=================
Advanced Features
=================
Control the VM Placement
------------------------
By default, VMs are placed by NOVA using its own scheduling logic. However,
traffic can be shaped precisely to fill the appropriate network links by using
specific configuration settings. KloudBuster can change that behavior, and
force NOVA to place VMs on desired hypervisors as we defined by supplying
the topology file.
The format of the topology file is relatively simple, and group into two
sections. See file "cfg.topo.yaml" for an example.
The "servers_rack" section contains the hypervisors that the server side VMs
will be spawned on, and the "clients_rack" section contains the hypervisors
that the client side VMs will be spawned on. The hypervisor names can be
obtained from Horizon dashboard, or via "*nova hypervisor-list*". Note that
the name in the config files must exactly match the name shown in Horizon
dashboard or NOVA API output.
A typical use case is to place all server VMs on one rack, and all client VMs
on the other rack to test Rack-to-Rack performance. Similarly, all server VMs
on one host, and all client VMs on the other host to test the Host-to-Host
performance.
To use this feature, just pass *-t <path_to_topo_file>* to the kloudbuster
command line.
.. note:: Admin access is required to use this feature.
Running KloudBuster without admin access
----------------------------------------
When there is no admin access to the cloud under test, KloudBuster does
support to run and reused the existing tenant and user for running tests.
You have to ask the cloud admin one time to create the resources in advance,
and KloudBuster will create the resources using the pre-created tenant/user.
When running under the tenant/user reusing mode:
* Only one tenant will be used for hosting both server cloud and client
cloud resources;
* Only two users will be used for creating resources, and each cloud has
its own user;
And also there are some limitations that you should aware:
* The VM placement feature will not be supported;
* The flavor configs will be ignored, and the KloudBuster will
automatically pick the closest flavor settings from the existing list;
* KloudBuster will not automatically adjust the tenant quota, and give
warnings when quota exceeded;
See file "cfg.tenants.yaml" for an example. Modify the settings to match your
cloud.
To use this feature, just pass *-l <path_to_tenants_file>* to the kloudbuster
command line.
Displaying the Results
----------------------
Results can be saved in a file in json format or in HTML format. The json format
is more appropriate for usage by any post-processing tool or script while the
HTML file is more adapted for human usage.
The KloudBuster Web UI will display the results using charts and tables when the
test is finished running. The KloudBuster CLI provides an option to generate
the HTML file from the results (*--html* option). It can also generate the HTML
file from the JSON results (*--charts-from-json* option).
Examples of running KloudBuster
-------------------------------
Assuming the OpenStack RC file is stored at ~/admin_openrc.sh, and the
password is "admin". Running the program is relatively easy, some examples
are given to help get started quickly.
.. note::
Before going to large scale test, it is strongly recommended to start with
a small scale. The default config is a good point to start with. It will
make sure KloudBuster is talking to the clouds well.
Example 1: Single-cloud Mode
""""""""""""""""""""""""""""
Kloudbuster will create both server VMs and client VMs in the same cloud if
only one RC file is provided::
$ kloudbuster --tested-rc ~/admin_openrc.sh --tested-passwd admin
Example 2: Dual-cloud Mode, Save results
""""""""""""""""""""""""""""""""""""""""
Assume the cloud for server VMs is ~/admin_openrc1.sh, and the cloud for
client VMs is ~/admin_openrc2.sh. The password for both clouds is "admin".
Also save the results to a JSON file once the run is finished::
$ kloudbuster --tested-rc ~/admin_openrc1.sh --tested-passwd admin --testing-rc ~/admin_openrc2.sh --testing-passwd admin --json result.json
Example 3: Single-cloud Mode, Customized VM placements
""""""""""""""""""""""""""""""""""""""""""""""""""""""
.. code::
$ kloudbuster --tested-rc ~/admin_openrc.sh --tested-passwd admin -t cfg.topo.yaml
Example 4: Single-cloud Mode, Running storage test, Save results to JSON
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
.. code::
$ kloudbuster --tested-rc ~/aio-openrc.sh --tested-passwd lab --storage --json aio.json

5
doc/source/cleanup.rst
View File

@ -134,8 +134,9 @@ Installation and Dependencies
The script is available in the OpenStack KloudBuster repository under
kloudbuster/force_cleanup.py. If you need to run the script outside of the usual
KloudBuster installation, the script requires the usual OpenStack python client
libraries and credentials.py (from the kloudbuster module). Otherwise, pick one
of the kloudbuster installation method to install the script.
libraries and credentials.py (from the kloudbuster module).
Otherwise, pick one of the kloudbuster installation method to install the script
(the KloudBuster docker container looks to be the simplest).
Known Issues and Limitations
----------------------------

222
doc/source/configuration.rst Normal file
View File

@ -0,0 +1,222 @@
=====================
Configuration Options
=====================
.. _default_http_scale:
Default HTTP Scale Test
-----------------------
The default HTTP scale test will run on a single cloud and perform the following steps:
* Create 2 tenants, 2 users, and 2 routers;
* Create 1 shared network for both servers and clients tenants
* Create 1 VM running the Redis server (for orchestration)
* Create 1 VM running as an HTTP server
* Create 1 VM running the HTTP traffic generator (defaults to 1000 connections,
1000 requests per second, and 30 seconds duration)
* Measure/aggregate throughput and latency
* Bring down and cleanup
.. _default_storage_scale:
Default Storage Scale Test
--------------------------
The default storage scale test will use the following settings:
* Create 1 tenant
* Create 1 router
* Create 1 private network
* Create 1 VM and attach a 10 GB Cinder volume to it
* Perform the default storage workload sequence:
* random access 4KB block size, IO depth 4, 100 IOPs for 30 seconds each
* 100% read
* 100% write
* 70% read, 30% write
* sequential access 64KB block size, IO depth 64, 60 MB/sec for 30 seconds
* 100% read
* 100% write
* 70% read, 30% write
* Measure/aggregate IOPs, throughput and latency
* Bring down and cleanup
The run should take a few minutes (depending on how fast the cloud can
create the resources) and you should see the actions taken by KloudBuster
displayed on the console. Once this minimal scale test passes, you can tackle
more elaborate scale testing by increasing the scale numbers or providing
various traffic shaping options. See below sections for more details about
configuring KloudBuster.
KloudBuster Configuration File
------------------------------
To create a custom scale test configuration, make a copy of the default
configuration (this can be obtained by redirecting the output of
*--show-config* to a new file, as described in the quick start guide)
and modify that file to satisfy our own needs.
The configuration file follows the yaml syntax and contains options
that are documented using yaml comments.
.. note::
The default configuration is always loaded by KloudBuster and
any default option can be overridden by providing a custom configuration
file that only contains modified options. So you can delete all the lines
in the configuration file that you do not intend to change
Once modified, you can pass the configuration file to KloudBuster using the
--config option.
General Configuration Options
-----------------------------
Each item in the configuration file is well documented. Below is
just a quick-start on some important config items that need to be paid more
attention to.
* **vm_creation_concurrency**
This controls the level of concurrency when creating VMs. There is no
recommended values, as it really varies and up to the cloud performance.
On a well-deployed cloud, you may able to push the values to more than 50.
The default value of 5 concurrent VM creations should be OK for most deployments.
.. note::
For deployment prior to Kilo release, you may hit this
`bug <https://bugs.launchpad.net/neutron/+bug/1194579>`_ if the
concurrency level is too high. Try to lower down the value if
you are hitting this issue.
* **server:number_tenants, server:routers_per_tenant,
server:networks_per_router, server:vms_per_network**
These are the four key values which controls the scale of the cloud you are
going to create. Depends on how you want the VM to be created, sets these values
differently. For example, if we want to create 180 Server VMs, we could do
either of the following settings:
(1) 30 tenants, 1 router per tenant, 2 networks per router, and 3 VMs per
network (so-called 30*1*2*3);
(2) 20 tenants, 3 routers per tenant, 3 networks per router, and 1 VMs per
network (so-called 20*3*3*1);
* **server:secgroups_per_network**
Reference Neutron router implementation is using IPTABLES to perform
security controls, which should be OK for small scale networks. This
setting for now is to investigate the upper limit capacity that Neutron
can handle. Keep the default to 1 if you don't have the concerns on
this part yet.
* **client:progression**
KloudBuster will give multiple runs (progression) on the cloud under this mode.
If enabled, KloudBuster will start with certain amount of VMs, and put more VMs
into the testing for every iteration. The increment of the VM count is specified
by *vm_multiple*. The iteration will continue until it reaches the scale defined
in the upper sections, or the stop limit.
The stop limit is used for KloudBuster to determine when to stop the
progression, and do the cleanup if needed earlier.
In the case of HTTP testing:
It is defines as: [number_of_err_packets,
percentile_of_packet_not_timeout(%)]. For example: [50, 99.99] means,
KloudBuster will continue the progression run only if **ALL** below
conditions are satisfied:
(1) The error count of packets are less or equal than 50;
(2) 99.99% of the packets are within the timeout range;
In the case of Storage testing:
It is a single integer indicating the degrading percentile. In the mode of
random read and random write, this value indicates the percentile of
degrading on IOPS, while in the mode of sequential read and sequential
write, this value indicates the percentile of degrading on throughput.
Assume the IOPS or throughput per VM is a fixed value, usually we are
expecting higher values when the VM count grows. At certain point where the
capacity of storage is reached, the overall performance will start to
degrade.
e.g. In the random read and random write mode, for example the IOPS is limited to
100 IOPS/VM. In the iteration of 10 VMs, the requested IOPS for the system
is 100 * 10 = 1000. However, the measured IOPS is degraded to only 800 IOPS.
So the degraded percentile is calculated as 800/1000=20% for this set of
data.
HTTP Test Specific Options
""""""""""""""""""""""""""
* **client:http_tool_configs**
This section controls how the HTTP traffic will be generated. Below are the two
values which determine the traffic::
# Connections to be kept concurrently per VM
connections: 1000
# Rate limit in RPS per client (0 for unlimited)
rate_limit: 1000
Each testing VM will have its targeting HTTP server for sending the requests.
Simply to say, connections determines the how many concurrent users that the
tool is emulating, and rate_limit determines how fast the HTTP request will be
sent. If the connections are more than the capacity of the cloud can handle,
socket errors or timeouts will occur; if the requests are sending too fast, you
will likely to have lots of requests responded very slow (will be reflected in
the latency distribution spectrum generated by KloudBuster).
Different cloud has different capacity to handle data plane traffics. The best
practice is to have an estimate first, and get started. In a typical 10GE VLAN
deployment, the line rate is about 9Gbps, or 1.125 GB/s. For pure HTTP traffic,
the effective rate minus the overhead is approximately 80% of the line rate,
which is about 920 MB/s. Each HTTP request will consume 32KB traffic for loading
the HTML page (HTML payload size is configurable), so the cloud capacity is
about 30,000 req/sec. If you are staging a cloud with 20 testing pairs, the
rate_limit for each VM settings will be about (30000 / 20 = 1500).
The capacity for handling connections varies among factors including kernel
tuning, server software, server configs, etc. and hard to have an estimate. It
is simple to start with the same count as the rate_limit to have (1
request/connection) for each VM, and we can adjust it later to find out the
maximum value. If you see socket errors or timeouts, means the scale you are
testing is more than the cloud capacity.
Some other values which are self-explained, and you can change them as needed.
Storage Test Specific Options
"""""""""""""""""""""""""""""
* **client:storage_stage_configs**
This section defines the storage specific configs in the staging phase::
# The number of VMs for running storage tests
vm_count: 1
# KloudBuster supports to run storage tests on Cinder Volumes or Ephemeral
# Disks. Available options to be configured: ['volume', 'ephemeral'].
target: 'volume'
# Volumes size in GB for each VM
disk_size: 10
# The size of the test file for running IO tests in GB. Must be less or
# equal than disk_size.
io_file_size: 1
* **client:storage_tool_configs**
This section controls how the Storage tests will be performed. All the fields
are self-explained, and you can create your own test case with customized
parameters.

28
doc/source/contributing.rst
View File

@ -69,25 +69,33 @@ Bugs should be filed on Launchpad, not GitHub:
https://bugs.launchpad.net/kloudbuster
Build KloudBuster Docker Image
-----------------------
Build the KloudBuster Docker Image
----------------------------------
Two files are used to build the Docker image: *Dockerfile* and *.dockerignore*. The former provides all the build instructions while the latter provides the list of files/directories that should not be copied to the Docker image.
In order to make the Docker image clean, remove all auto generated files from the root of your workspace first. Specify the image name and the tag, and feed them to docker build. Examples to build the image with name "$USER/koudbuster", tag "latest"::
In order to make the Docker image clean, remove all auto generated files from the root of your workspace first. Specify the image name and the tag, and feed them to docker build. Examples to build the image with name "berrypatch/kloudbuster", tag "6.0.4"::
$ cd <koudbuster-ws-root>
$ sudo docker build --tag=$USER/koudbuster:latest .
$ cd <kloudbuster-ws-root>
$ sudo docker build --tag=berrypatch/kloudbuster:6.0.4 .
The images should be available for use::
$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
xiyu3/kloudbuster latest 0f17ae788c69 8 minutes ago 443.1 MB
$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
berrypatch/kloudbuster 6.0.4 0f17ae788c69 8 minutes ago 443.1 MB
For exchanging purposes, the image could be saved to a tar archive. You can distribute the KloudBuster Docker image among your servers easily with this feature::
To push the new image to the KloudBuster Docker Hub repository (berrypatch), you need to login to
Docker Hub (sudo docker login) and you need to have write access to the berrypatch/kloudbuster repository
before you can push the new container::
$ sudo docker save -o <IMAGE_FILE> <IMAGE_ID>
sudo docker login
sudo docker push berrypatch/kloudbuster:6.0.4
It is also good practice to build and override the latest tag::
sudo docker build --tag=berrypatch/kloudbuster:latest .
sudo docker push berrypatch/kloudbuster:latest
.. _developer_guide_of_openstack:

93
doc/source/development.rst
View File

@ -4,46 +4,19 @@ Development
.. _build_vm_image:
Build KloudBuster Image
------------------------
Build the KloudBuster VM Image
------------------------------
The official pre-built image from OpenStack App Catalog should satisfy most of
use cases. This option is just for development purposes, and for whatever
reason the pre-built version does not work for you.
This section describes how to rebuild:
Build on MacOSX
^^^^^^^^^^^^^^^
You need to install first:
* `Virtualbox <https://www.virtualbox.org/wiki/Downloads>`_
* `Vagrant <https://www.vagrantup.com/downloads.html>`_
And build the image with below commands:
.. code-block:: bash
$ # Clone the kloudbuster repository if you have not done so
$ git clone https://github.com/openstack/kloudbuster.git
$ # Go to the dib directory
$ cd kloudbuster/kb_dib
$ # Run vagrant and start building the image
$ vagrant up
After a few minutes (depending on virtualbox overhead), the qcow2 image will
be built and available in the same directory. You can then upload it to
OpenStack using the glance CLI, destroy the vagrant VM ("vagrant destroy") and
dispose of the kloudbuster directory (if no longer needed).
- the KloudButer VM Image from the git repository source code
- the KloudBuster Docker image
Build on Linux
^^^^^^^^^^^^^^
A generally faster build method than with MacOSX/Vagrant. Your Linux server
must have python, git and qemu utilities installed. You can skip the package
installation if you've installed KloudBuster using the GitHub/OpenStack
repository based installation.
Your Linux server must have python, git and qemu utilities installed.
Ubuntu/Debian based:
@ -76,9 +49,6 @@ After a few minutes, the qcow2 image will be built and available in the same
directory. You can then upload it to OpenStack using the glance CLI.
Trouble-shooting
^^^^^^^^^^^^^^^^
If you get an error message saying that import yaml fails (seems to happen
only on Ubuntu)::
@ -94,6 +64,53 @@ You need to comment out the secure_path option in your /etc/sudoers file (use
#Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
.. _build_web_ui:
.. include:: ../../kb_server/public/README.rst
Build on MacOSX
^^^^^^^^^^^^^^^
You need to install first:
* `Virtualbox <https://www.virtualbox.org/wiki/Downloads>`_
* `Vagrant <https://www.vagrantup.com/downloads.html>`_
And build the image with below commands:
.. code-block:: bash
$ # Clone the kloudbuster repository if you have not done so
$ git clone https://github.com/openstack/kloudbuster.git
$ # Go to the dib directory
$ cd kloudbuster/kb_dib
$ # Run vagrant and start building the image
$ vagrant up
After a few minutes (depending on virtualbox overhead), the qcow2 image will
be built and available in the same directory. You can then upload it to
OpenStack using the glance CLI, destroy the vagrant VM ("vagrant destroy") and
dispose of the kloudbuster directory (if no longer needed).
Build the KloudBuster Docker Container Image
--------------------------------------------
The KloudBuster Docker images are published in the DockerHub berrypatch repository:
`<https://hub.docker.com/r/berrypatch/kloudbuster/>`_
The Dockerfile at the root of the git repository can be used to build a new container based on Ubuntu 14.04.
To build for tag 6.0.3 (replace as needed with the real tag), go to the root of the repository then execute the docker build command:
.. code-block:: bash
sudo docker build --tag=berrypatch/kloudbuster:6.0.3 .
To publish you need to be a member of the berrypatch kloudbuster team. After the login (requires your DockerHub username and password), push the appropriate version to berrypatch:
.. code-block:: bash
sudo docker login
sudo docker push berrypatch/kloudbuster:6.0.3

4
doc/source/index.rst
View File

@ -14,8 +14,10 @@ Contents:
readme
architecture
gallery
installation
usage
configuration
std_scale_profile
adv_features
cleanup
development
contributing

220
doc/source/installation.rst
View File

@ -1,220 +0,0 @@
============
Installation
============
KloudBuster is already pre-installed as a web service in the KloudBuster VM image
available from the `OpenStack Community App Catalog <https://apps.openstack.org>`_.
So if you just need to use the KloudBuster Web user interface you can instantiate
that VM image and point your browser to its public address as described in :ref:`run_server`.
There are two alternative ways to install and run KloudBuster tool.
Users of KloudBuster who prefers to use the CLI or who prefer to run KloudBuster
locally on their workstation or laptop should use the PyPI based installation
(pip install).
Developers of KloudBuster should use the GitHub/OpenStack Repository based installation
(git clone).
Web Service and PyPI based installation will satisfy most use cases
and are the 2 recommended ways for running KloudBuster under production environments,
or through an automated or scheduled job.
.. note:: Installation from PyPI will only have the latest stable version.
PyPI based Installation
-----------------------
This is the recommended way to install KloudBuster for non-development use if CLI is preferred
or if you prefer to run KloudBuster locally.
KloudBuster is available in the Python Package Index (PyPI)
`KloudBuster PyPI <https://pypi.python.org/pypi/KloudBuster>`_
and can be installed on any system that has python 2.7.
Step 1: Install pip and the python virtualenv (if not installed already)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You will need to have python 2.7, pip, and some dependencies installed
before installing KloudBuster depending on the operating system at the installation site.
These pre-requisites can be skipped if the corresponding dependencies are already installed.
Ubuntu/Debian based:
.. code-block:: bash
$ sudo apt-get install python-dev python-pip python-virtualenv libyaml-dev
RHEL/Fedora/CentOS based:
.. code-block:: bash
$ sudo yum install gcc python-devel python-pip python-virtualenv libyaml-devel
MacOSX:
.. code-block:: bash
$ # Download the XCode command line tools from Apple App Store
$ xcode-select --install
$ sudo easy_install pip
$ sudo pip install virtualenv
$
$ # If you need to run KloudBuster Web UI from PyPI installation,
$ # coreutils needs to be installed using Homebrew.
$ # Refer here for the steps to install Homebrew on Mac:
$ # http://brew.sh/
$ brew install coreutils
Step 2: Install KloudBuster in a virtual environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Create a virtual environment for Python, and install KloudBuster:
.. code-block:: bash
$ virtualenv vkb
$ source vkb/bin/activate
$ pip install kloudbuster
Alternatively, if you have
`virtualenvwrapper <https://virtualenvwrapper.readthedocs.org>`_ installed:
.. code-block:: bash
$ mkvirtualenv kloudbuster
$ pip install kloudbuster
.. note::
"A Virtual Environment is a tool to keep the dependencies required by
different projects in separate places, by creating virtual Python
environments for them." It is optional but recommended. We could use::
$ sudo pip install kloudbuster
instead if isolation among multiple Python projects is not needed.
To verify kloudbuster is installed, just type::
kloudbuster --help
.. _git_installation:
GitHub/OpenStack Repository based Installation
----------------------------------------------
It is recommended to run KloudBuster inside a virtual environment. However,
it can be skipped if installed in a dedicated VM.
Quick installation on Ubuntu/Debian
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: bash
$ sudo apt-get install python-dev python-virtualenv git git-review qemu-utils
$ # create a virtual environment
$ virtualenv ./vkb
$ source ./vkb/bin/activate
$ git clone https://github.com/openstack/kloudbuster.git
$ cd kloudbuster
$ pip install -r requirements-dev.txt
Quick installation on RHEL/Fedora/CentOS
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: bash
$ sudo yum install python-devel python-virtualenv git qemu-img
$ # create a virtual environment
$ virtualenv ./vkb
$ source ./vkb/bin/activate
$ git clone https://github.com/openstack/kloudbuster.git
$ cd kloudbuster
$ pip install -r requirements-dev.txt
Quick installation on MacOSX
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KloudBuster can run natively on MacOSX.
First, download XCode from App Store, then execute below commands:
.. code-block:: bash
$ # Download the XCode command line tools
$ xcode-select --install
$ # Install pip
$ sudo easy_install pip
$ # Install python virtualenv
$ sudo pip install virtualenv
$ # create a virtual environment
$ virtualenv ./vkb
$ source ./vkb/bin/activate
$ git clone https://github.com/openstack/kloudbuster.git
$ cd kloudbuster
$ pip install -r requirements-dev.txt
Verify installation
^^^^^^^^^^^^^^^^^^^
To verify kloudbuster is installed, just type:
.. code-block:: bash
$ python kloudbuster/kloudbuster.py --help
.. _upload_kb_image:
Upload KloudBuster Image
-------------------------
.. note::
If your OpenStack Glance is able to access the Internet and you only use
the CLI to launch KloudBuster, you can skip this section (KloudBuster CLI
will request Glance to download the image from the OpenStack App Catalog when
it is not present in Glance).
In the cloud under test, KloudBuster needs one "universal" test VM image
(referred to as "KloudBuster image") that contains the necessary test software.
The KloudBuster image is
then instantiated in potentially large number of VMs by the KloudBuster
application using the appropriate role (HTTP server, HTTP traffic generator,
etc.).
Pre-built images are available for download from the
`OpenStack App Catalog <http://apps.openstack.org>`_ (preferred method).
.. note::
The same KloudBuster VM image can be instantiated for running the test functions
(HTTP servers and HTTP traffic generators) and for running KloudBuster as a web service.
Manual upload of the KloudBuster VM image
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In order to upload the KloudBuster Image to the cloud under test, the image
must be downloaded from the OpenStack App Catalog either directly from
the OpenStack App Catalog (if you have direct access to the Internet)
or through an intermediate location such as a laptop that has Internet access
or a jump host (a jump host has access
to both Internet and the cloud under
test and can be used to download the image from the App Catalog
and upload to Glance using either a Glance CLI command or via Horizon
dashboard).
KloudBuster VM images are qcow2 images named "kloudbuster_v<version>.qcow2"
(e.g. "kloudbuster_v6.qcow2"). The image can be downloaded from
`<http://apps.openstack.org/#tab=glance-images>`_. Look for an image named
with the "kloudbuster_v" prefix and download the latest version from the list.
The name of the image in Glance must match exactly the image name in the App
Catalog (without the .qcow2 extension), for example to upload the image from
a local copy of that image using the Glance CLI:
.. code-block:: bash
$ glance image-create --file kloudbuster_v6.qcow2 --disk-format qcow2 --container-format bare --is-public True --name kloudbuster_v6

135
doc/source/quickstart_docker.rst Normal file
View File

@ -0,0 +1,135 @@
==============================================
KloudBuster Docker Container Quick Start Guide
==============================================
The KloudBuster Docker container provides a quick way to use KloudBuster
if you are already familiar with Docker.
Prerequisites
-------------
This quick start guide assumes you have already installed Docker.
All command line examples below are based on Linux (which could be
either native or in a VM) and require Internet access to Docker Hub.
1. Pull latest Docker container image
-------------------------------------
KloudBuster is available as a container in Docker Hub at
`berrypatch/kloudbuster <https://hub.docker.com/r/berrypatch/kloudbuster/>`_
.. code-block:: bash
$ docker pull berrypatch/kloudbuster
2. Get the openrc file from your OpenStack Horizon dashboard
------------------------------------------------------------
Using the Horizon dashboard, download the openrc file (Project|Compute|API Access then click on "Download OpenStack RC File").
It is best to use the admin user to run KloudBuster as much as possible (otherwise there are restrictions on what you can do).
Instructions below assume a copy of that file is saved under the local directory with the name "admin-openrc.sh"
3. Upload the KloudBuster VM image to the cloud under test
----------------------------------------------------------
If your OpenStack cloud has full access to the Internet, you can skip this step as KloudBuster will
instruct Glance to download the KloudBuster VM inage directly from the OpenStack (skip to next step).
Otherwise, :ref:`download the latest kloudbuster image <upload_kb_image>` from the OpenStack App Catalog.
In addition to the method described to upload the image using the Horizon dashboard or the glance CLI,
you can also use the glance CLI that is already available in the KloudBuster container.
Start a bash shell in the container and map the local directory to '/opt/kb' in the container
so that you have access to the image and the RC file:
.. code-block:: bash
docker run -v $PWD:/opt/kb --rm -it berrypatch/kloudbuster bash
Then from inside the container bash prompt, source the openrc file, invoke the glance
CLI to upload the VM image (should take a few minutes) then exit and terminate the container:
.. code-block:: bash
source /opt/kb/admin-openrc.sh
glance image-create --name "kloudbuster_v6" --visibility public --disk-format qcow2 --container-format bare --file /opt/kb/kloudbuster_v6.qcow2
exit
Now you should be back to the host and should see the kloudbuster image in the current directory.
4. Running the KloudBuster CLI
------------------------------
If you do not really need a Web UI or REST interface, you can simply run
KloudBuster scale test straight from CLI in the container.
.. code-block:: bash
docker run -v $PWD:/opt/kb --rm -t berrypatch/kloudbuster kloudbuster -h
We assume in the below example that you have an openrc file available called "admin-openrc.sh" in the local directory
and that the corresponding OpenStack password is "admin"
Run the default HTTP data plane scale test
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The default HTTP scale test is described :ref:`here <default_http_scale>`
.. code-block:: bash
docker run --rm -t -v $PWD:/opt/kb berrypatch/kloudbuster kloudbuster --tested-rc /opt/kb/admin-openrc.sh --tested-passwd admin
Run the default storage scale test
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The default storage scale test is described :ref:`here <default_storage_scale>`
.. code-block:: bash
docker run --rm -t -v $PWD:/opt/kb berrypatch/kloudbuster kloudbuster --tested-rc /opt/kb/admin-openrc.sh --tested-passwd admin --storage
Run KloudBuster with a custom configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To get a copy of the default KloudBuster configuration and store it to a file called "kb.cfg"::
.. code-block:: bash
docker run --rm berrypatch/kloudbuster kloudbuster --show-config >/opt/kb/kb.cfg
less kb.cfg
You can then edit kb.cfg and modify it appropriately. To run KloudBuster with the custom configuration, simply pass it to container after mapping the host local directory to "/opt/kb" (for example)::
.. code-block:: bash
docker run --rm -t -v $PWD:/opt/kb berrypatch/kloudbuster kloudbuster --tested-rc /opt/kb/admin-openrc.sh --tested-passwd admin --config /opt/kb/kb.cfg
5. Running KloudBuster as a WebUI/REST Server
---------------------------------------------
By default KloudbBuster will listen on port 8080 in the container. This port must be mapped to a host level port using the -p argument. For example, to use the same port number at the host level:
.. code-block:: bash
docker run -p 8080:8080 --rm berrypatch/kloudbuster kb_start_server&
The first port number is the host listen port (any port of your choice) while the second one after the column is the container listen port (always 8080 for KloudBuster). For example, to use port 9090 on the host and map it to the KloudBuster port in the container, you would use -p 9090:8080
To stop the KloudBuster container, you can use the "docker kill <id>" command.
Assuming the host port used is 8080, the Web UI URL to use from any browser is::
http://<host_ip>:8080
The KloudBuster REST base URL is the above URL with "/api" appended::
http://<host_ip>:8080/api
ref:`How to use the Web UI <webui_usage>`
ref:`How to use the REST interface <rest_usage>`

187
doc/source/quickstart_git.rst Normal file
View File

@ -0,0 +1,187 @@
=================================
KloudBuster Git Quick Start Guide
=================================
.. _git_installation:
This is the default installation method for code development.
It is recommended to run KloudBuster inside a virtual environment. However,
it can be skipped if installed in a dedicated VM.
1. Install Dependencies and Clone Git Repository
------------------------------------------------
Quick installation on Ubuntu/Debian
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: bash
$ sudo apt-get install python-dev python-virtualenv git git-review qemu-utils
$ # create a virtual environment
$ virtualenv ./vkb
$ source ./vkb/bin/activate
$ git clone https://github.com/openstack/kloudbuster.git
$ cd kloudbuster
$ pip install -r requirements-dev.txt
Quick installation on RHEL/Fedora/CentOS
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: bash
$ sudo yum install python-devel python-virtualenv git qemu-img
$ # create a virtual environment
$ virtualenv ./vkb
$ source ./vkb/bin/activate
$ git clone https://github.com/openstack/kloudbuster.git
$ cd kloudbuster
$ pip install -r requirements-dev.txt
Quick installation on MacOSX
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KloudBuster can run natively on MacOSX.
First, download XCode from App Store, then execute below commands:
.. code-block:: bash
$ # Download the XCode command line tools
$ xcode-select --install
$ # Install pip
$ sudo easy_install pip
$ # Install python virtualenv
$ sudo pip install virtualenv
$ # create a virtual environment
$ virtualenv ./vkb
$ source ./vkb/bin/activate
$ git clone https://github.com/openstack/kloudbuster.git
$ cd kloudbuster
$ pip install -r requirements-dev.txt
If you need to run the KloudBuster Web UI you need to install coreutils
(you can skip this step if you do not run the KloudBuster Web server):
.. code-block:: bash
$
$ # If you need to run KloudBuster Web UI,
$ # coreutils needs to be installed using Homebrew.
$ # Refer here for the steps to install Homebrew on Mac:
$ # http://brew.sh/
$ brew install coreutils
$ # Follow instructions to add the new directory in the path
$ export PATH="/usr/local/opt/coreutils/libexec/gnubin:$PATH"
$ # Verify that you have stdbuf working
$ stdbuf --help
Usage: stdbuf OPTION... COMMAND
Run COMMAND, with modified buffering operations for its standard streams.
Mandatory arguments to long options are mandatory for short options too.
-i, --input=MODE adjust standard input stream buffering
-o, --output=MODE adjust standard output stream buffering
-e, --error=MODE adjust standard error stream buffering
--help display this help and exit
--version output version information and exit
If MODE is 'L' the corresponding stream will be line buffered.
This option is invalid with standard input.
If MODE is '0' the corresponding stream will be unbuffered.
Otherwise MODE is a number which may be followed by one of the following:
KB 1000, K 1024, MB 1000*1000, M 1024*1024, and so on for G, T, P, E, Z, Y.
In this case the corresponding stream will be fully buffered with the buffer
size set to MODE bytes.
NOTE: If COMMAND adjusts the buffering of its standard streams ('tee' does
for example) then that will override corresponding changes by 'stdbuf'.
Also some filters (like 'dd' and 'cat' etc.) don't use streams for I/O,
and are thus unaffected by 'stdbuf' settings.
GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Full documentation at: <http://www.gnu.org/software/coreutils/stdbuf>
or available locally via: info '(coreutils) stdbuf invocation'
Verify installation
^^^^^^^^^^^^^^^^^^^
To verify kloudbuster is installed, from the root of the kloudbuster repository type:
.. code-block:: bash
$ python kloudbuster/kloudbuster.py --help
2. Upload the KloudBuster VM image
----------------------------------
Follow the :ref:`steps <upload_kb_image>` to upload the KloudBuster VM image
to the OpenStack cloud under test.
3. Download the openrc file
---------------------------
Using the Horizon dashboard, download the openrc file (Project|Compute|API Access then click on "Download OpenStack RC File").
It is best to use the admin user to run KloudBuster as much as possible (otherwise there are restrictions on what you can do).
The examples below assume the openrc file is saved at the root of the kloudbuster git repository with the name "admin-openrc.sh"
and the password is "admin".
4. Running the KloudBuster CLI
------------------------------
Run the default HTTP data plane scale test
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The default HTTP scale test is described :ref:`here <default_http_scale>`
.. code-block:: bash
python kloudbuster/kloudbuster.py --tested-rc admin-openrc.sh --tested-passwd admin
Run the default storage scale test
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The default storage scale test is described :ref:`here <default_storage_scale>`
.. code-block:: bash
python kloudbuster/kloudbuster.py --tested-rc admin-openrc.sh --tested-passwd admin --storage
Run KloudBuster with a custom configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The default KloudBuster configuration file is in cfg.scale.yaml. You can make a copy of it in "kb.cfg"::
.. code-block:: bash
cp kloudbuster/cfg.scale.yaml kb.cfg
You can then edit kb.cfg and modify it appropriately. To run KloudBuster with the custom configuration::
.. code-block:: bash
python kloudbuster/kloudbuster.py --tested-rc admin-openrc.sh --tested-passwd admin --config kb.cfg
5. Running KloudBuster as a WebUI/REST Server
---------------------------------------------
.. code-block:: bash
python kloudbuster/start_server.py&
You should see a message similar to the one below, which indicates the server
is up running::
Starting server in PID 27873
serving on 0.0.0.0:8080, view at http://127.0.0.1:8080
By default KloudbBuster will listen on port 8080.
ref:`How to use the Web UI <webui_usage>`
ref:`How to use the REST interface <rest_usage>`
To terminate the server, simply use the kill command on the server pid.

182
doc/source/quickstart_pip.rst Normal file
View File

@ -0,0 +1,182 @@
=========================================
KloudBuster Pip Install Quick Start Guide
=========================================
KloudBuster is available in the Python Package Index (PyPI)
`KloudBuster PyPI <https://pypi.python.org/pypi/KloudBuster>`_
and can be installed on any system that has python 2.7.
1. Install pip and the python virtualenv (if not installed already)
-------------------------------------------------------------------
You will need to have python 2.7, pip, and some dependencies installed
before installing KloudBuster depending on the operating system at the installation site.
These pre-requisites can be skipped if the corresponding dependencies are already installed.
Ubuntu/Debian based:
.. code-block:: bash
$ sudo apt-get install python-dev python-pip python-virtualenv libyaml-dev
RHEL/Fedora/CentOS based:
.. code-block:: bash
$ sudo yum install gcc python-devel python-pip python-virtualenv libyaml-devel
MacOSX:
.. code-block:: bash
$ # Download the XCode command line tools from Apple App Store
$ xcode-select --install
$ sudo easy_install pip
$ sudo pip install virtualenv
$
$ # If you need to run KloudBuster Web UI from PyPI installation,
$ # coreutils needs to be installed using Homebrew.
$ # Refer here for the steps to install Homebrew on Mac:
$ # http://brew.sh/
$ brew install coreutils
$ # Follow instructions to add the new directory in the path
$ export PATH="/usr/local/opt/coreutils/libexec/gnubin:$PATH"
$ # Verify that you have stdbuf working
$ stdbuf --help
Usage: stdbuf OPTION... COMMAND
Run COMMAND, with modified buffering operations for its standard streams.
Mandatory arguments to long options are mandatory for short options too.
-i, --input=MODE adjust standard input stream buffering
-o, --output=MODE adjust standard output stream buffering
-e, --error=MODE adjust standard error stream buffering
--help display this help and exit
--version output version information and exit
If MODE is 'L' the corresponding stream will be line buffered.
This option is invalid with standard input.
If MODE is '0' the corresponding stream will be unbuffered.
Otherwise MODE is a number which may be followed by one of the following:
KB 1000, K 1024, MB 1000*1000, M 1024*1024, and so on for G, T, P, E, Z, Y.
In this case the corresponding stream will be fully buffered with the buffer
size set to MODE bytes.
NOTE: If COMMAND adjusts the buffering of its standard streams ('tee' does
for example) then that will override corresponding changes by 'stdbuf'.
Also some filters (like 'dd' and 'cat' etc.) don't use streams for I/O,
and are thus unaffected by 'stdbuf' settings.
GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
Full documentation at: <http://www.gnu.org/software/coreutils/stdbuf>
or available locally via: info '(coreutils) stdbuf invocation'
2. Install KloudBuster in a virtual environment
-----------------------------------------------
Create a virtual environment for Python, and install KloudBuster:
.. code-block:: bash
$ virtualenv vkb
$ source vkb/bin/activate
$ pip install kloudbuster
Alternatively, if you have
`virtualenvwrapper <https://virtualenvwrapper.readthedocs.org>`_ installed:
.. code-block:: bash
$ mkvirtualenv kloudbuster
$ pip install kloudbuster
.. note::
"A Virtual Environment is a tool to keep the dependencies required by
different projects in separate places, by creating virtual Python
environments for them." It is optional but recommended. We could use::
$ sudo pip install kloudbuster
instead if isolation among multiple Python projects is not needed.
To verify kloudbuster is installed properly, just type::
kloudbuster --help
3. Upload the KloudBuster VM image
----------------------------------
Follow the :ref:`steps <upload_kb_image>` to upload the KloudBuster VM image
to the OpenStack cloud under test.
4. Download the openrc file
---------------------------
Using the Horizon dashboard, download the openrc file (Project|Compute|API Access then click on "Download OpenStack RC File").
It is best to use the admin user to run KloudBuster as much as possible (otherwise there are restrictions on what you can do).
5. Running the KloudBuster CLI
------------------------------
Run the default HTTP data plane scale test
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The default HTTP scale test is described :ref:`here <default_http_scale>`
.. code-block:: bash
kloudbuster --tested-rc admin-openrc.sh --tested-passwd admin
Run the default storage scale test
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The default storage scale test is described :ref:`here <default_storage_scale>`
.. code-block:: bash
kloudbuster --tested-rc admin-openrc.sh --tested-passwd admin --storage
Run KloudBuster with a custom configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To get a copy of the default KloudBuster configuration and store it to a file called "kb.cfg"::
.. code-block:: bash
kloudbuster --show-config >kb.cfg
less kb.cfg
You can then edit kb.cfg and modify it appropriately. To run KloudBuster with the custom configuration::
.. code-block:: bash
kloudbuster --tested-rc admin-openrc.sh --tested-passwd admin --config kb.cfg
6. Running KloudBuster as a WebUI/REST Server
---------------------------------------------
.. code-block:: bash
kb_start_server&
You should see a message similar to the one below, which indicates the server
is up running::
Starting server in PID 27873
serving on 0.0.0.0:8080, view at http://127.0.0.1:8080
By default KloudbBuster will listen on port 8080.
The KloudBuster Web UI URL to use from any browser is::
http://<host_ip>:8080
The KloudBuster REST base URL is the above URL with "/api" appended::
http://<host_ip>:8080/api
ref:`How to use the Web UI <webui_usage>`
ref:`How to use the REST interface <rest_usage>`

67
doc/source/quickstart_vmapp.rst Normal file
View File

@ -0,0 +1,67 @@
============================================
KloudBuster VM Application Quick Start Guide
============================================
The pre-built KloudBuster qcow2 VM image contains the KloudBuster Web server
and is ready to service HTTP and REST requests once up and running. To get the
KloudBuster Web server running in any OpenStack cloud:
1. Upload the KloudBuster VM image
----------------------------------
Follow the :ref:`steps <upload_kb_image>` to upload the KloudBuster VM image
to the OpenStack cloud that will host your KloudBuster web server
.. note::
The KloudBuster web server can run in the cloud under test or in another
OpenStack cloud.
2. Create a Neutron tenant router and network
---------------------------------------------
If necessary, and as for any VM-based web server application bring up, create
and configure the Neutron router and network where the KloudBuster web server
VM instance will be attached.
You can also reuse an existing tenant network and router.
3. Create a Security Group
--------------------------
Create or reuse a security group which allows ingress TCP traffic on port 8080
4. Launch the KloudBuster VM Application
----------------------------------------
Launch an instance using the KloudBuster image with the proper security
group, and connect to the appropriate network. Leave the Key Pair as blank,
as we don't need the SSH access to this VM
5. Associate a floating IP
--------------------------
Associate a floating IP to the newly created VM instance so that it can be
accessible from an external browser
6. Connect to the web UI with a browser
---------------------------------------
The Web UI URL to use from any browser is::
http://<floating_ip>:8080
The base URL to use for REST access is::
http://<floating_ip>:8080/api
7. Download the openrc file
---------------------------
Using the Horizon dashboard, download the openrc file (Project|Compute|API Access then click on "Download OpenStack RC File").
It is best to use the admin user to run KloudBuster as much as possible (otherwise there are restrictions on what you can do).
8. Login to KloudBuster
-----------------------
Follow ref:`instructions <webui_usage>` on how to use the web UI.

98
doc/source/std_scale_profile.rst Normal file
View File

@ -0,0 +1,98 @@
==================================
KloudBuster Standard Scale Profile
==================================
Standard scale profile definition
----------------------------------
Multiple factors can impact data plane scale numbers measured by KloudBuster: VM
count, number of connections per VM, number of requests per seconds per VM,
timeout, etc... To help obtaining quick and easy results without having to
tweak too many parameters, KloudBuster defines an off the shelf *default scale
profile*.
In the default scale profile for running HTTP load:
- The number of connections per VM is set to 1000;
- The number of requests per seconds per VM is set to 1000;
- The HTTP request timeout is set to 5 seconds;
- The stop limit for progression runs will be error packets greater than 50;
- The size of the HTML page in the server VMs will be 32768 Bytes;
As a reference, KloudBuster can run approximately 21 VMs (with 21,000
connections and 21,000 HTTP requests/sec) and achieve approximately 5 Gbps of
HTTP throughput on a typical multi-node Kilo OpenStack deployment (LinuxBridge +
VLAN, 10GE NIC card).
In the default scale profile for running Storage load:
- A standard set of 6 test cases (random read/write/mixed, sequential
read/write/mixed);
- The IOPS limit per VM is set to 100 for random read/write/mixed test cases,
and Rate limit per VM is set to 60MB/s for sequential read/write/mixed test
cases;
- Block size is set to 4K for random read/write/mixed test cases, and 64K for
sequential read/write/mixed test cases;
- IO Depth is set to 4 for random read/write/mixed test cases, and 64 for
sequential read/write/mixed test cases;
- The stop limit for progression runs is degrading more than 20% of the target;
Note that it is hard to give a reference on storage testing since the
performance varies a lot on different hardware or solutions.
How to run the standard scale profile
-------------------------------------
In order to perform a run using the default scale profile, set the max VM counts
for the test, enable progression run and leave everything else with their
default values. KloudBuster will start the iteration until reaching the stop
limit or the max scale. Eventually, once the KloudBuster run is finished, the
cloud performance can be told by looking at how many VMs KloudBuster can run to
and by looking at the latency charts.
Steps:
1. Enable progression runs:
Running from CLI: Edit the config file, and set
**client:progression:enabled** to True
Running from Web UI: Navigate to "Interactive Mode" from the top menu
bar, unfold the left panel for detail settings, under "Progression Test"
section, and check the "Progression Test" checkbox.
2. Set up the max scale:
The max scale basically means the max VM counts that KloudBuster will try to
reach. In the case of HTTP testing, for a typical 10GE NIC card with VLAN
encapsulation, 25 will be a good value; in the case of Storage testing,
depends on the solution the deployment is using, pick a number from 10 to 25
would usually be fine. Remember you can always adjust it to a more
reasonable value based on your deployment details.
Running from CLI: Edit the config file, and set **server:vms_per_network**
to a proper value.
Running from Web UI: Navigate to "Interactive Mode" from the top menu
bar, unfold the left panel for detail settings, under "Staging Settings"
section, and set "VMs/Network" to a proper value.
Interpret the results
---------------------
From the CLI, check the log and find the warning that KloudBuster gave, similar
to this::
WARNING KloudBuster is stopping the iteration because the result reaches the stop limit.
One line before is the json output of last successful run, which has the number
in the "total_server_vms" field.
From the Web UI, in the "Interactive Mode" tab, you will see how many sets of
data are you getting. The second last set of data shows the last successful run,
which has the number in the "Server VMs" column.

619
doc/source/usage.rst
View File

@ -1,127 +1,81 @@
=====
Usage
=====
============================
Usage and Quick Start Guides
============================
There are three ways for running KloudBuster, the easiest being the **Web UI**.
KloudBuster User Interfaces
---------------------------
KloudBuster provides 3 interfaces, the easiest being the **Web User Interface**.
It offers the most user friendly interface and needs the least learning to get
started. **CLI** is the traditional way to run applications. It has the most
comprehensive feature sets when compared to the other two ways. **Rest API**
comprehensive feature sets when compared to the other methods. **REST API**
gives another way to access and control KloudBuster from another application.
The built-in Web UI is fully implemented on top of the REST API.
The default scale settings of KloudBuster is at minimal scale, which is
generally safe to run on any cloud, small or large. It should also work on an
all-in-one devstack cloud installation as well. The minimal pre-requisites to
run KloudBuster:
The Web UI is fully implemented on top of the REST API.
OpenStack Cloud Pre-Requisites
------------------------------
OpenStack cloud pre-requisites to run KloudBuster:
* Neutron networking
* Admin access to the cloud under test (non-admin might work with some
tweaks and limitations)
* 3 available floating IPs if running the HTTP data plane scale test,
* 2 available floating IPs if running the Storage scale test
Regardless of the way you launch KloudBuster, you will need the access info and
the credentials to the cloud under test. This information can be downloaded
from the Horizon dashboard (Project|Access&Security|Api Access|Download OpenStack
RC File). Save it to your local file system for future use.
KloudBuster Installation Options
--------------------------------
.. _run_server:
There are 4 different ways to install KloudBuster:
Running KloudBuster as a Web/REST Server
----------------------------------------
- use a pre-built Docker container (recommended if you already use Docker)
- use a pre-built VM image (if you prefer to run the KloudBuster application in a VM and do not need CLI)
- install from PyPI (if you prefer to use pip install)
- install directly from GitHub (git clone, for code development or if you want to browse the code)
Starting the KloudBuster Server from a VM Image
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Users of KloudBuster who prefer to use the CLI or who prefer to run KloudBuster
locally on their workstation or laptop can use the PyPI based installation
(pip install) or the Docker container.
The easiest way to use KloudBuster is to run it as a web server application
running in a VM. The pre-built KloudBuster qcow2 image contains the Web server
and is ready to service HTTP and REST requests once up and running. To get the
KloudBuster Web server running in any OpenStack cloud:
1. Follow the steps :ref:`here <upload_kb_image>` to upload the KloudBuster
image to the OpenStack cloud that will host your KloudBuster web server
.. note::
This could be the same as the cloud under test or a different cloud.
2. If necessary, and as for any VM-based web server application bring up, create
and configure the Neutron router and network where the KloudBuster web server
VM instance will be attached
3. Create or reuse a security group which allows ingress TCP traffic on port
8080
4. Launch an instance using the KloudBuster image with the proper security
group, and connect to the appropriate network. Leave the Key Pair as blank,
as we don't need the SSH access to this VM
5. Associate a floating IP to the newly created VM instance so that it can be
accessible from an external browser
The base URL to use for REST access is::
http://<floating_ip>:8080
The Web UI URL to use from any browser is::
http://<floating_ip>:8080/ui/index.html
For your convenience, the server will redirect the root of the address to the
Web UI by default. i.e. http://<floating_ip>:8080/ will get you into the UI
directly.
Docker container, Web Service and PyPI based installation will satisfy most use cases
and are the recommended ways for running KloudBuster under production environments
or through an automated or scheduled job.
Starting the KloudBuster Server from PyPI installation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Quick Start Guides
------------------
.. toctree::
:maxdepth: 2
From the virtual environment which hosts KloudBuster, run::
$ kb_start_server
You should see a message similar to the one below, which indicates the server
is up running::
Starting server in PID 27873
serving on 0.0.0.0:8080, view at http://127.0.0.1:8080
Starting the KloudBuster Server from a git clone
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you use git clone, you can bring up the KloudBuster Web/REST server from the
CLI. KloudBuster uses the `Pecan <http://www.pecanpy.org/>`_ web server to host
both the KloudBuster REST server and the KloudBuster front-end website (which
listens to port 8080 by default).
From the root of the KloudBuster repository, go to the kb_server directory. If
this is the first time you start the server, run the below command once to setup the
environment::
$ python setup.py develop
Then start the server by doing::
$ pecan serve config.py
You should see a message similar to the one below, which indicates the server
is up running::
Starting server in PID 26431
serving on 0.0.0.0:8080, view at http://127.0.0.1:8080
quickstart_docker
quickstart_pip
quickstart_vmapp
quickstart_git
.. _webui_usage:
Using the KloudBuster Web UI
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
----------------------------
Using any browser, point to the provided URL at port 8080. You will get a Login
page where you will need to enter:
Using any browser, point to the provided URL. You will get a Login page where you will need to enter:
* The type of scale test (HTTP data plane or storage)
* The location of the openrc file of the cloud under test
* The password for the cloud under test
* The location of the openrc file for the cloud under test and the corresponding
OpenStack password
You could modify the scale test configuration options or simply start the scale test with the default scale configuration.
Click on Stage button to instruct KloudBuster to stage all the OpenStack resources. This can take time
depending on how many VMs are requested and how fast is the cloud under test.
Once staging is done, click on the Run button to run the scale test.
Interacting with the KloudBuster Server REST Interface
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. _rest_usage:
Interacting with the KloudBuster REST Interface
-----------------------------------------------
REST Documentation
^^^^^^^^^^^^^^^^^^
Once the server is started, you can use different HTTP methods
(GET/PUT/POST/DELETE) to interact with the KloudBuster REST interface using the
@ -130,452 +84,75 @@ provided URL at port 8080.
* `KloudBuster REST API Documentation Preview <https://htmlpreview.github.io/?https://github.com/openstack/kloudbuster/blob/master/doc/source/_static/kloudbuster-swagger.html>`_
* `REST API Documentation (Swagger yaml) <https://github.com/openstack/kloudbuster/blob/master/kb_server/kloudbuster-swagger.yaml>`_
Running the KloudBuster from CLI
--------------------------------
If you do not really need a Web UI or REST interface, you can simply run
KloudBuster scale test straight from CLI.
KloudBuster is ready to run with the default configuration, which can be
displayed from the command line using *--show-config* option.
By default, KloudBuster will run on a single cloud and run the default HTTP data plane scale
test:
* Create 2 tenants, 2 users, and 2 routers;
* Create 1 shared network for both servers and clients tenants
* Create 1 VM running as an HTTP server
* Create 1 VM running the Redis server (for orchestration)
* Create 1 VM running the HTTP traffic generator (default to 1000 connections,
1000 requests per second, and 30 seconds duration
* Measure/aggregate throughput and latency
* Bring down and cleanup
Run KloudBuster with the following options::
kloudbuster --tested-rc <path_to_the_admin_rc_file> --tested-passwd <admin_password>
If you installed KloudBuster using git clone you need to explicitly call the python interpreter
since the "kloudbuster" wrapper is only installed with pip install)::
python kloudbuster.py --tested-rc <path_to_the_admin_rc_file> --tested-passwd <admin_password>
If you want to run the default storage scale test, simply adding *--storage*::
kloudbuster --storage --tested-rc <path_to_the_admin_rc_file> --tested-passwd <admin_password>
The default storage scale test will use the following settings:
* Create 1 tenant
* Create 1 router
* Create 1 private network
* Create 1 VM and attach a 10 GB Cinder volume to it
* Perform the default storage workload sequence
* random access 4KB block size, IO depth 4, 100 IOPs for 30 seconds each
* 100% read
* 100% write
* 70% read, 30% write
* sequential access 64KB block size, IO depth 64, 60 MB/sec for 30 seconds
* 100% read
* 100% write
* 70% read, 30% write
* Measure/aggregate IOPs, throughput and latency
* Bring down and cleanup
The run should take a few minutes (depending on how fast the cloud can
create the resources) and you should see the actions taken by KloudBuster
displayed on the console. Once this minimal scale test passes, you can tackle
more elaborate scale testing by increasing the scale numbers or providing
various traffic shaping options. See below sections for more details about
configuring KloudBuster.
KloudBuster Configuration
Examples of REST requests
^^^^^^^^^^^^^^^^^^^^^^^^^
To create a custom scale test configuration, make a copy of the default
configuration and modify that file to satisfy our own needs. A copy of the
default configuration can be obtained by redirecting the output of
*--show-config* to a new file. For example:
.. _upload_kb_image:
kloudbuster --show-config >scale1.cfg
KloudBuster VM Image Upload
---------------------------
Once done, provide that custom configuration
file to the KloudBuster command line using the *--config <file>* option.
Before you can use KloudBuster you must upload the KloudBuster VM image to your
OpenStack cloud under test.
KloudBuster needs one "universal" test VM image
(referred to as "KloudBuster image") that contains the necessary test software.
The KloudBuster image is then instantiated by the KloudBuster application
in potentially large number of VMs using the appropriate role (HTTP server,
HTTP traffic generator...).
Pre-built VM images are available for download from the
`OpenStack App Catalog <http://apps.openstack.org/#tab=glance-images>`_.
.. note::
The default configuration is always loaded by KloudBuster and
any default option can be overridden by providing a custom configuration
file that only contains modified options. So you can delete all the lines
in the configuration file that you do not intend to change
General Options
"""""""""""""""
Each item in cfg.scale.yaml is well documented and self-explained. Below is
just a quick-start on some important config items that need to be paid more
attention to.
* **vm_creation_concurrency**
This controls the level of concurrency when creating VMs. There is no
recommended values, as it really varies and up to the cloud performance.
On a well-deployed cloud, you may able to push the values to more than 50.
Safely to say, 5 would be OK for most deployments.
The same KloudBuster VM image can be instantiated for running the test functions
(HTTP servers, HTTP traffic generators, file access tools) and for running KloudBuster as a web service.
.. note::
For deployment prior to Kilo release, you may hit this
`bug <https://bugs.launchpad.net/neutron/+bug/1194579>`_ if the
concurrency level is too high. Try to lower down the value if
you are hitting this issue.
If your OpenStack Glance is able to access the Internet and you only use
the CLI to launch KloudBuster, you can skip this section (KloudBuster CLI
will request Glance to download the image from the OpenStack App Catalog when
it is not present in Glance).
* **server:number_tenants, server:routers_per_tenant,
server:networks_per_router, server:vms_per_network**
Download the KloudBuster VM image to the local directory
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
These are the four key values which controls the scale of the cloud you are
going to create. Depends on how you want the VM to be created, sets these values
differently. For example, if we want to create 180 Server VMs, we could do
either of the following settings:
You must download a local copy if your OpenStack cloud does not have dirct access
to the Internet.
Download the latest image directly from `OpenStack App Catalog <http://apps.openstack.org/#tab=glance-images>`_
using your favorite browser (search for "kloudbuster") or using wget.
KloudBuster VM images are qcow2 images named "kloudbuster_v<version>.qcow2"
(e.g. "kloudbuster_v6.qcow2"). Look for an image named
with the "kloudbuster_v" prefix and download the latest version from the list.
(1) 30 tenants, 1 router per tenant, 2 networks per router, and 3 VMs per
network (so-called 30*1*2*3);
Example for downloading the v6 image using wget:
(2) 20 tenants, 3 routers per tenant, 3 networks per router, and 1 VMs per
network (so-called 20*3*3*1);
.. code-block:: bash
* **server:secgroups_per_network**
wget http://storage.apps.openstack.org/images/kloudbuster_v6.qcow2
Reference Neutron router implementation is using IPTABLES to perform
security controls, which should be OK for small scale networks. This
setting for now is to investigate the upper limit capacity that Neutron
can handle. Keep the default to 1 if you don't have the concerns on
this part yet.
* **client:progression**
KloudBuster will give multiple runs (progression) on the cloud under this mode.
Upload the KloudBuster VM image using the Horizon Dashboard
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If enabled, KloudBuster will start with certain amount of VMs, and put more VMs
into the testing for every iteration. The increment of the VM count is specified
by *vm_multiple*. The iteration will continue until it reaches the scale defined
in the upper sections, or the stop limit.
From the dashboard, create a new image and select either "Image File" if you want to
uplaod from the local copy of the image or "Image Location" if you want to
upload directly from the OpenStack App Catalog (you will need the complete URL of the image).
The stop limit is used for KloudBuster to determine when to stop the
progression, and do the cleanup if needed earlier.
The name of the image in Glance *must* match exactly the image name in the App
Catalog (without the .qcow2 extension, e.g. "kloudbuster_v6").
In the case of HTTP testing:
Upload the KloudBuster VM image using the Glance CLI
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
It is defines as: [number_of_err_packets,
percentile_of_packet_not_timeout(%)]. For example: [50, 99.99] means,
KloudBuster will continue the progression run only if **ALL** below
conditions are satisfied:
This assumes that you have installed the OpenStack Glance API and have sourced
the appropriate openrc file.
To upload the image from a local copy of that image using the Glance CLI:
(1) The error count of packets are less or equal than 50;
.. code-block:: bash
(2) 99.99% of the packets are within the timeout range;
glance image-create --file kloudbuster_v6.qcow2 --disk-format qcow2 --container-format bare --visibility public --name kloudbuster_v6
In the case of Storage testing:
It is a single integer indicating the degrading percentile. In the mode of
random read and random write, this value indicates the percentile of
degrading on IOPS, while in the mode of sequential read and sequential
write, this value indicates the percentile of degrading on throughput.
Assume the IOPS or throughput per VM is a fixed value, usually we are
expecting higher values when the VM count grows. At certain point where the
capacity of storage is reached, the overall performance will start to
degrade.
e.g. In the random read and random write mode, for example the IOPS is limited to
100 IOPS/VM. In the iteration of 10 VMs, the requested IOPS for the system
is 100 * 10 = 1000. However, the measured IOPS is degraded to only 800 IOPS.
So the degraded percentile is calculated as 800/1000=20% for this set of
data.
HTTP Testing Specific Options
"""""""""""""""""""""""""""""
* **client:http_tool_configs**
This section controls how the HTTP traffic will be generated. Below are the two
values which determine the traffic::
# Connections to be kept concurrently per VM
connections: 1000
# Rate limit in RPS per client (0 for unlimited)
rate_limit: 1000
Each testing VM will have its targeting HTTP server for sending the requests.
Simply to say, connections determines the how many concurrent users that the
tool is emulating, and rate_limit determines how fast the HTTP request will be
sent. If the connections are more than the capacity of the cloud can handle,
socket errors or timeouts will occur; if the requests are sending too fast, you
will likely to have lots of requests responded very slow (will be reflected in
the latency distribution spectrum generated by KloudBuster).
Different cloud has different capacity to handle data plane traffics. The best
practice is to have an estimate first, and get started. In a typical 10GE VLAN
deployment, the line rate is about 9Gbps, or 1.125 GB/s. For pure HTTP traffic,
the effective rate minus the overhead is approximately 80% of the line rate,
which is about 920 MB/s. Each HTTP request will consume 32KB traffic for loading
the HTML page (HTML payload size is configurable), so the cloud capacity is
about 30,000 req/sec. If you are staging a cloud with 20 testing pairs, the
rate_limit for each VM settings will be about (30000 / 20 = 1500).
The capacity for handling connections varies among factors including kernel
tuning, server software, server configs, etc. and hard to have an estimate. It
is simple to start with the same count as the rate_limit to have (1
request/connection) for each VM, and we can adjust it later to find out the
maximum value. If you see socket errors or timeouts, means the scale you are
testing is more than the cloud capacity.
Some other values which are self-explained, and you can change them as needed.
Storage Testing Specific Options
""""""""""""""""""""""""""""""""
* **client:storage_stage_configs**
This section defines the storage specific configs in the staging phase::
# The number of VMs for running storage tests
vm_count: 1
# KloudBuster supports to run storage tests on Cinder Volumes or Ephemeral
# Disks. Available options to be configured: ['volume', 'ephemeral'].
target: 'volume'
# Volumes size in GB for each VM
disk_size: 10
# The size of the test file for running IO tests in GB. Must be less or
# equal than disk_size.
io_file_size: 1
* **client:storage_tool_configs**
This section controls how the Storage tests will be performed. All the fields
are self-explained, and you can create your own test case with customized
parameters.
Advanced Features
^^^^^^^^^^^^^^^^^
Control the VM Placement
""""""""""""""""""""""""
By default, VMs are placed by NOVA using its own scheduling logic. However,
traffic can be shaped precisely to fill the appropriate network links by using
specific configuration settings. KloudBuster can change that behavior, and
force NOVA to place VMs on desired hypervisors as we defined by supplying
the topology file.
The format of the topology file is relatively simple, and group into two
sections. See file "cfg.topo.yaml" for an example.
The "servers_rack" section contains the hypervisors that the server side VMs
will be spawned on, and the "clients_rack" section contains the hypervisors
that the client side VMs will be spawned on. The hypervisor names can be
obtained from Horizon dashboard, or via "*nova hypervisor-list*". Note that
the name in the config files must exactly match the name shown in Horizon
dashboard or NOVA API output.
A typical use case is to place all server VMs on one rack, and all client VMs
on the other rack to test Rack-to-Rack performance. Similarly, all server VMs
on one host, and all client VMs on the other host to test the Host-to-Host
performance.
To use this feature, just pass *-t <path_to_topo_file>* to the kloudbuster
command line.
.. note:: Admin access is required to use this feature.
Running KloudBuster without admin access
""""""""""""""""""""""""""""""""""""""""
When there is no admin access to the cloud under test, KloudBuster does
support to run and reused the existing tenant and user for running tests.
You have to ask the cloud admin one time to create the resources in advance,
and KloudBuster will create the resources using the pre-created tenant/user.
When running under the tenant/user reusing mode:
* Only one tenant will be used for hosting both server cloud and client
cloud resources;
* Only two users will be used for creating resources, and each cloud has
its own user;
And also there are some limitations that you should aware:
* The VM placement feature will not be supported;
* The flavor configs will be ignored, and the KloudBuster will
automatically pick the closest flavor settings from the existing list;
* KloudBuster will not automatically adjust the tenant quota, and give
warnings when quota exceeded;
See file "cfg.tenants.yaml" for an example. Modify the settings to match your
cloud.
To use this feature, just pass *-l <path_to_tenants_file>* to the kloudbuster
command line.
Displaying the Results
^^^^^^^^^^^^^^^^^^^^^^
Results can be saved in a file in json format or in HTML format. The json format
is more appropriate for usage by any post-processing tool or script while the
HTML file is more adapted for human usage.
The KloudBuster Web UI will display the results using charts and tables when the
test is finished running. The KloudBuster CLI provides an option to generate
the HTML file from the results (*--html* option). It can also generate the HTML
file from the JSON results (*--charts-from-json* option).
Examples of running KloudBuster
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Assuming the OpenStack RC file is stored at ~/admin_openrc.sh, and the
password is "admin". Running the program is relatively easy, some examples
are given to help get started quickly.
.. note::
Before going to large scale test, it is strongly recommended to start with
a small scale. The default config is a good point to start with. It will
make sure KloudBuster is talking to the clouds well.
Example 1: Single-cloud Mode
""""""""""""""""""""""""""""
Kloudbuster will create both server VMs and client VMs in the same cloud if
only one RC file is provided::
$ kloudbuster --tested-rc ~/admin_openrc.sh --tested-passwd admin
Example 2: Dual-cloud Mode, Save results
""""""""""""""""""""""""""""""""""""""""
Assume the cloud for server VMs is ~/admin_openrc1.sh, and the cloud for
client VMs is ~/admin_openrc2.sh. The password for both clouds is "admin".
Also save the results to a JSON file once the run is finished::
$ kloudbuster --tested-rc ~/admin_openrc1.sh --tested-passwd admin --testing-rc ~/admin_openrc2.sh --testing-passwd admin --json result.json
Example 3: Single-cloud Mode, Customized VM placements
""""""""""""""""""""""""""""""""""""""""""""""""""""""
.. code::
$ kloudbuster --tested-rc ~/admin_openrc.sh --tested-passwd admin -t cfg.topo.yaml
Example 4: Single-cloud Mode, Running storage test, Save results to JSON
""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
.. code::
$ kloudbuster --tested-rc ~/aio-openrc.sh --tested-passwd lab --storage --json aio.json
KloudBuster Standard Scale Profile
----------------------------------
Multiple factors can impact data plane scale numbers measured by KloudBuster: VM
count, number of connections per VM, number of requests per seconds per VM,
timeout, etc... To help obtaining quick and easy results without having to
tweak too many parameters, KloudBuster defines an off the shelf *default scale
profile*.
In the default scale profile for running HTTP load:
- The number of connections per VM is set to 1000;
- The number of requests per seconds per VM is set to 1000;
- The HTTP request timeout is set to 5 seconds;
- The stop limit for progression runs will be error packets greater than 50;
- The size of the HTML page in the server VMs will be 32768 Bytes;
As a reference, KloudBuster can run approximately 21 VMs (with 21,000
connections and 21,000 HTTP requests/sec) and achieve approximately 5 Gbps of
HTTP throughput on a typical multi-node Kilo OpenStack deployment (LinuxBridge +
VLAN, 10GE NIC card).
In the default scale profile for running Storage load:
- A standard set of 6 test cases (random read/write/mixed, sequential
read/write/mixed);
- The IOPS limit per VM is set to 100 for random read/write/mixed test cases,
and Rate limit per VM is set to 60MB/s for sequential read/write/mixed test
cases;
- Block size is set to 4K for random read/write/mixed test cases, and 64K for
sequential read/write/mixed test cases;
- IO Depth is set to 4 for random read/write/mixed test cases, and 64 for
sequential read/write/mixed test cases;
- The stop limit for progression runs is degrading more than 20% of the target;
Note that it is hard to give a reference on storage testing since the
performance varies a lot on different hardware or solutions.
In order to perform a run using the default scale profile, set the max VM counts
for the test, enable progression run and leave everything else with their
default values. KloudBuster will start the iteration until reaching the stop
limit or the max scale. Eventually, once the KloudBuster run is finished, the
cloud performance can be told by looking at how many VMs KloudBuster can run to
and by looking at the latency charts.
How-to
^^^^^^
In order to run KloudBuster Standard Scale Profile, you have to set up below
configurations:
1. Enable progression runs:
Running from CLI: Edit the config file, and set
**client:progression:enabled** to True
Running from Web UI: Navigate to "Interactive Mode" from the top menu
bar, unfold the left panel for detail settings, under "Progression Test"
section, and check the "Progression Test" checkbox.
2. Set up the max scale:
The max scale basically means the max VM counts that KloudBuster will try to
reach. In the case of HTTP testing, for a typical 10GE NIC card with VLAN
encapsulation, 25 will be a good value; in the case of Storage testing,
depends on the solution the deployment is using, pick a number from 10 to 25
would usually be fine. Remember you can always adjust it to a more
reasonable value based on your deployment details.
Running from CLI: Edit the config file, and set **server:vms_per_network**
to a proper value.
Running from Web UI: Navigate to "Interactive Mode" from the top menu
bar, unfold the left panel for detail settings, under "Staging Settings"
section, and set "VMs/Network" to a proper value.
Interpret the results
^^^^^^^^^^^^^^^^^^^^^
From the CLI, check the log and find the warning that KloudBuster gave, similar
to this::
WARNING KloudBuster is stopping the iteration because the result reaches the stop limit.
One line before is the json output of last successful run, which has the number
in the "total_server_vms" field.
From the Web UI, in ihe "Interactive Mode" tab, you will see how many sets of
data are you getting. The second last set of data shows the last successful run,
which has the number in the "Server VMs" column.