Kafka scenario for omsimulator

Change-Id: I024be7212b196fdc305ce0107adc4e99a7debdf9
This commit is contained in:
Ilya Shakhat 2016-04-19 15:58:04 +03:00
parent c5d91c68ab
commit 40e3a6f41c
2 changed files with 125 additions and 0 deletions

View File

@ -0,0 +1,73 @@
Oslo.messaging simulator report
-------------------------------
This report contains results of `message_queue_performance`_ execution
with `Oslo.messaging Simulator`_ with Kafka driver.
Test Case: Notification Throughput Test
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Message processing
~~~~~~~~~~~~~~~~~~
Messages are collected at 2 points: ``sent`` - messages sent by the client
and ``received`` - messages received by the server. Also the number of lost
messages is calculated. Sizes of messages is based on the distribution of
messages collected on the 100-node cloud.
{{'''
title: NOTIFY Message count
axes:
x: threads
y: sent, msg
y2: received, msg
y3: lost, msg
chart: line
pipeline:
- { $match: { task: omsimulator, mode: notify }}
- { $group: { _id: { threads: { $multiply: [ "$threads", "$host_count", "$processes" ] } },
sent: { $sum: "$client.count" },
received: { $sum: "$server.count" },
lost: { $sum: { $subtract: ["$client.count", "$server.count"] }}
}}
- { $project: { x: "$_id.threads",
y: "$sent",
y2: "$received",
y3: "$lost"
}}
- { $sort: { x: 1 }}
''' | chart_and_table
}}
The throughput and latency
~~~~~~~~~~~~~~~~~~~~~~~~~~
The chart shows the throughput and latency depending on number of concurrent threads.
{{'''
title: NOTIFY throughput and latency depending on thread count
axes:
x: threads
y: throughput, msg/sec
y3: latency, ms
chart: line
pipeline:
- $match: { task: omsimulator, mode: notify }
- $group:
_id: { threads: { $multiply: [ "$threads", "$host_count", "$processes" ] } }
throughput: { $sum: { $divide: ["$client.count", "$client.duration"] }}
latency: { $avg: "$server.latency" }
- $project:
x: "$_id.threads"
y: "$throughput"
y3: { $multiply: [ "$latency", 1000 ] }
''' | chart_and_table
}}
.. references:
.. _message_queue_performance: http://docs.openstack.org/developer/performance-docs/test_plans/mq/plan.html
.. _Oslo.messaging Simulator: https://github.com/openstack/oslo.messaging/blob/master/tools/simulator.py

View File

@ -0,0 +1,52 @@
title: Oslo.messaging notifications with Kafka driver
description:
This scenario oslo.messaging is tested with Kafka driver
parameters:
tester_hosts: List of hosts were omsimulator will be executed
kafka_url: Kafka address
client_url: Override Kafka address for the client
server_url: Override Kafka address for the server
setup:
-
hosts: {{ tester_hosts }}
tasks:
- apt: name=git
become: yes
- name: installing omsimulator
git: repo=git://git.openstack.org/openstack/oslo.messaging
dest=/tmp/performa/oslo.messaging
- apt: name=python-dev
become: yes
- apt: name=python-pip
become: yes
- pip: name=virtualenv
become: yes
- pip: requirements=/tmp/performa/oslo.messaging/requirements.txt virtualenv=/tmp/performa/oslo.messaging/.venv
- pip: name=eventlet virtualenv=/tmp/performa/oslo.messaging/.venv
- pip: name="kafka-python<1.0.0,>=0.9.5" virtualenv=/tmp/performa/oslo.messaging/.venv
- command: /tmp/performa/oslo.messaging/.venv/bin/python setup.py install
args:
chdir: /tmp/performa/oslo.messaging
execution:
-
hosts: {{ tester_hosts }}
matrix:
processes: [ 1, 2, 3, 4, 5, 6 ]
tasks:
- omsimulator:
mode: notify
duration: 100
threads: 10
url: {{ kafka_url }}
client_url: {{ client_url }}
server_url: {{ server_url }}
unique_topic_per_pair: yes
report:
template: kafka.rst