openstack
/
performance-docs
Code Issues Proposed changes

Neutron control plane plan + results 

Change-Id: I04a26909a3472a6abc18a50266131e3be7c71e26
This commit is contained in:
Dina Belova 2016-09-27 21:38:38 -07:00
parent 1ef9268e1a
commit 596fde7b74
16 changed files with 4275 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

221
doc/source/test_plans/neutron_control_plane/plan.rst Normal file
View File

@ -0,0 +1,221 @@
.. _openstack_neutron_control_plane_performance_test_plan:
==================================================================
OpenStack Networking (Neutron) control plane performance test plan
==================================================================
:status: **ready**
:version: 1.0
:Abstract:
This test plan aims to provide set of tests to identify OpenStack Networking
(aka Neutron) Control Plane performance against given OpenStack cloud using
Rally tests.
Test Plan
=========
This test plan describes several Rally test cases that can cover almost
all most important in terms of networking performance basic cloud
operations e.g. routers, security groups and other objects management.
Test Environment
----------------
Preparation
^^^^^^^^^^^
This test plan is performed either against existing OpenStack cloud with
pre-installed Rally framework or can be executed via Rally from very beginning
including deployment of the OpenStack cloud. As an option verification (Tempest
testing) can be run prior the benchmarking (scenarios to be described in this
document).
Environment description
^^^^^^^^^^^^^^^^^^^^^^^
The environment description includes hardware specification of servers,
network parameters, operation system and OpenStack deployment characteristics.
Hardware
~~~~~~~~
This section contains list of all types of hardware nodes.
+-----------+-------+----------------------------------------------------+
| Parameter | Value | Comments |
+-----------+-------+----------------------------------------------------+
| model | | e.g. Supermicro X9SRD-F |
+-----------+-------+----------------------------------------------------+
| CPU | | e.g. 6 x Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz |
+-----------+-------+----------------------------------------------------+
| role | | e.g. compute or network |
+-----------+-------+----------------------------------------------------+
Network
~~~~~~~
This section contains list of interfaces and network parameters.
For complicated cases this section may include topology diagram and switch
parameters.
+------------------+-------+-------------------------+
| Parameter | Value | Comments |
+------------------+-------+-------------------------+
| network role | | e.g. provider or public |
+------------------+-------+-------------------------+
| card model | | e.g. Intel |
+------------------+-------+-------------------------+
| driver | | e.g. ixgbe |
+------------------+-------+-------------------------+
| speed | | e.g. 10G or 1G |
+------------------+-------+-------------------------+
| MTU | | e.g. 9000 |
+------------------+-------+-------------------------+
| offloading modes | | e.g. default |
+------------------+-------+-------------------------+
Software
~~~~~~~~
This section describes installed software.
+-----------------+-------+---------------------------+
| Parameter | Value | Comments |
+-----------------+-------+---------------------------+
| OS | | e.g. Ubuntu 14.04.3 |
+-----------------+-------+---------------------------+
| OpenStack | | e.g. Mitaka |
+-----------------+-------+---------------------------+
| Hypervisor | | e.g. KVM |
+-----------------+-------+---------------------------+
| Neutron plugin | | e.g. ML2 + OVS |
+-----------------+-------+---------------------------+
| L2 segmentation | | e.g. VLAN / VxLAN / GRE |
+-----------------+-------+---------------------------+
| virtual routers | | e.g. HA / DVR |
+-----------------+-------+---------------------------+
Test tool
---------
**Rally** is a benchmarking tool that was designed specifically for OpenStack
API testing. To make this possible, **Rally** automates and unifies multi-node
OpenStack deployment, cloud verification, benchmarking & profiling. This is a
simple way to check cloud workability and performance of control plane
operations running on it.
Test Case 1: Basic Neutron test suite
-------------------------------------
Description
^^^^^^^^^^^
This test suite is combined from default Rally Neutron test cases with `default
configuration`_. It is most useful for validating cloud operability. The
following Rally test scenarios need to be executed:
* create-and-list-floating-ips
* create-and-list-networks
* create-and-list-ports
* create-and-list-routers
* create-and-list-security-groups
* create-and-list-subnets
* create-and-delete-floating-ips
* create-and-delete-networks
* create-and-delete-ports
* create-and-delete-routers
* create-and-delete-security-groups
* create-and-delete-subnets
* create-and-update-networks
* create-and-update-ports
* create-and-update-routers
* create-and-update-security-groups
* create-and-update-subnets
.. _default configuration: https://github.com/openstack/rally/tree/master/samples/tasks/scenarios/neutron
List of performance metrics
^^^^^^^^^^^^^^^^^^^^^^^^^^^
+----------+-------+-------------------+---------------------------+
| Priority | Value | Measurement Units | Description |
+==========+=======+===================+===========================+
| 1 | | sec | Time of atomic operations |
+----------+-------+-------------------+---------------------------+
Test Case 2: Stressful Neutron test suite
-----------------------------------------
Description
^^^^^^^^^^^
This test case the same set of scenarios that were mentioned in
`Test Case 1: Basic Neutron test suite`_ can be used, the difference is in
increased number of iterations and concurrency that create sufficient load on
Neutron control plane. To stress OpenStack networking control plane 50-100
concurrency can be used with 2000-5000 iterations in total. We can advice to
focus on the following Rally test cases (that cover most interesting and most
stressed parts of OpenStack Networking):
* create-and-list-networks
* create-and-list-ports
* create-and-list-routers
* create-and-list-security-groups
* create-and-list-subnets
* boot-and-list-server
* boot-and-delete-server-with-secgroups
* boot-runcommand-delete
List of performance metrics
^^^^^^^^^^^^^^^^^^^^^^^^^^^
+----------+-------+-------------------+---------------------------+
| Priority | Value | Measurement Units | Description |
+==========+=======+===================+===========================+
| 1 | | sec | Time of atomic operations |
+----------+-------+-------------------+---------------------------+
Test case 3: Neutron scalability test with many networks
--------------------------------------------------------
Description
^^^^^^^^^^^
The aim of this test is to create a large number of networks, subnets, routers
and security groups with rules per tenant. Each network has a single VM. For
example 100 networks (each with a subnet, router and a VM) can be created per
each iteration (up to 20 iterations in total).
List of performance metrics
^^^^^^^^^^^^^^^^^^^^^^^^^^^
+----------+-------+-------------------+---------------------------+
| Priority | Value | Measurement Units | Description |
+==========+=======+===================+===========================+
| 1 | | sec | Time of atomic operations |
+----------+-------+-------------------+---------------------------+
Test case 4: Neutron scalability test with many servers
-------------------------------------------------------
The outline of this test is almost the same as of
`Test case 3: Neutron scalability test with many networks`_. The main
difference is that during each scenario iteration this test creates huge enough
number of VMs (e.g. 100) per a single network, hence it is possible to check
the case with many number of ports per subnet.
List of performance metrics
^^^^^^^^^^^^^^^^^^^^^^^^^^^
+----------+-------+-------------------+---------------------------+
| Priority | Value | Measurement Units | Description |
+==========+=======+===================+===========================+
| 1 | | sec | Time of atomic operations |
+----------+-------+-------------------+---------------------------+
Reports
=======
Test plan execution reports:
* :ref:`openstack_neutron_control_plane_performance_report`

BIN
doc/source/test_results/neutron_control_plane/configs/Network_Scheme.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 12 KiB

571
doc/source/test_results/neutron_control_plane/index.rst Normal file
View File

@ -0,0 +1,571 @@
.. _openstack_neutron_control_plane_performance_report:
OpenStack Networking (Neutron) control plane performance report for 400 nodes
*****************************************************************************
:Abstract:
This document includes OpenStack Networking (aka Neutron) control plane
performance test results against two OpenStack environments: 200 nodes and
378 nodes. All tests have been performed regarding
:ref:`openstack_neutron_control_plane_performance_test_plan`
Environment description
=======================
Lab A (200 nodes)
-----------------
3 controllers, 196 computes, 1 node for Grafana/Prometheus
Hardware configuration of each server
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. table:: Description of controller servers
+-------+----------------+---------------------------------+
|server |vendor,model |Supermicro MBD-X10DRI |
+-------+----------------+---------------------------------+
|CPU |vendor,model |Intel Xeon E5-2650v3 |
| +----------------+---------------------------------+
| |processor_count |2 |
| +----------------+---------------------------------+
| |core_count |10 |
| +----------------+---------------------------------+
| |frequency_MHz |2300 |
+-------+----------------+---------------------------------+
|RAM |vendor,model |8x Samsung M393A2G40DB0-CPB |
| +----------------+---------------------------------+
| |amount_MB |2097152 |
+-------+----------------+---------------------------------+
|NETWORK|vendor,model |Intel,I350 Dual Port |
| +----------------+---------------------------------+
| |bandwidth |1G |
| +----------------+---------------------------------+
| |vendor,model |Intel,82599ES Dual Port |
| +----------------+---------------------------------+
| |bandwidth |10G |
+-------+----------------+---------------------------------+
|STORAGE|vendor,model |Intel SSD DC S3500 Series |
| +----------------+---------------------------------+
| |SSD/HDD |SSD |
| +----------------+---------------------------------+
| |size |240GB |
| +----------------+---------------------------------+
| |vendor,model |2x WD WD5003AZEX |
| +----------------+---------------------------------+
| |SSD/HDD |HDD |
| +----------------+---------------------------------+
| |size |500GB |
+-------+----------------+---------------------------------+
.. table:: Description of compute servers
+-------+----------------+---------------------------------+
|server |vendor,model |SUPERMICRO 5037MR-H8TRF |
+-------+----------------+---------------------------------+
|CPU |vendor,model |INTEL XEON Ivy Bridge 6C E5-2620 |
| +----------------+---------------------------------+
| |processor_count |1 |
| +----------------+---------------------------------+
| |core_count |6 |
| +----------------+---------------------------------+
| |frequency_MHz |2100 |
+-------+----------------+---------------------------------+
|RAM |vendor,model |4x Samsung DDRIII 8GB DDR3-1866 |
| +----------------+---------------------------------+
| |amount_MB |32768 |
+-------+----------------+---------------------------------+
|NETWORK|vendor,model |AOC-STGN-i2S - 2-port |
| +----------------+---------------------------------+
| |bandwidth |10G |
+-------+----------------+---------------------------------+
|STORAGE|vendor,model |Intel SSD DC S3500 Series |
| +----------------+---------------------------------+
| |SSD/HDD |SSD |
| +----------------+---------------------------------+
| |size |80GB |
| +----------------+---------------------------------+
| |vendor,model |1x WD Scorpio Black BP WD7500BPKT|
| +----------------+---------------------------------+
| |SSD/HDD |HDD |
| +----------------+---------------------------------+
| |size |750GB |
+-------+----------------+---------------------------------+
Lab B (378 nodes)
-----------------
Environment contains 4 types of servers:
- rally node
- controller node
- compute-osd node
- compute node
.. table:: Amount of servers each role (node types are listed below)
+------------+--------------+------+
|Role |Servers count |Type |
+============+==============+======+
|rally |1 |1 or 2|
+------------+--------------+------+
|controller |3 |1 or 2|
+------------+--------------+------+
|compute |291 |1 or 2|
+------------+--------------+------+
|compute-osd |34 |3 |
+------------+--------------+------+
|compute-osd |49 |1 |
+------------+--------------+------+
Hardware configuration of each server
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All servers have 3 types of configuration describing in table below
.. table:: Description of servers hardware type 1
+-------+----------------+---------------------------------+
|server |vendor,model |Dell PowerEdge R630 |
+-------+----------------+---------------------------------+
|CPU |vendor,model |Intel,E5-2680 v3 |
| +----------------+---------------------------------+
| |processor_count |2 |
| +----------------+---------------------------------+
| |core_count |12 |
| +----------------+---------------------------------+
| |frequency_MHz |2500 |
+-------+----------------+---------------------------------+
|RAM |vendor,model |Samsung, M393A2G40DB0-CPB |
| +----------------+---------------------------------+
| |amount_MB |262144 |
+-------+----------------+---------------------------------+
|NETWORK|interface_name s|eno1, eno2 |
| +----------------+---------------------------------+
| |vendor,model |Intel,X710 Dual Port |
| +----------------+---------------------------------+
| |bandwidth |10G |
| +----------------+---------------------------------+
| |interface_names |enp3s0f0, enp3s0f1 |
| +----------------+---------------------------------+
| |vendor,model |Intel,X710 Dual Port |
| +----------------+---------------------------------+
| |bandwidth |10G |
+-------+----------------+---------------------------------+
|STORAGE|dev_name |/dev/sda |
| +----------------+---------------------------------+
| |vendor,model | | raid1 - Dell, PERC H730P Mini |
| | | | 2 disks Intel S3610 |
| +----------------+---------------------------------+
| |SSD/HDD |SSD |
| +----------------+---------------------------------+
| |size | 3,6TB |
+-------+----------------+---------------------------------+
.. table:: Description of servers hardware type 2
+-------+----------------+-------------------------------+
|server |vendor,model |Lenovo ThinkServer RD550 |
+-------+----------------+-------------------------------+
|CPU |vendor,model |Intel,E5-2680 v3 |
| +----------------+-------------------------------+
| |processor_count |2 |
| +----------------+-------------------------------+
| |core_count |12 |
| +----------------+-------------------------------+
| |frequency_MHz |2500 |
+-------+----------------+-------------------------------+
|RAM |vendor,model |Samsung, M393A2G40DB0-CPB |
| +----------------+-------------------------------+
| |amount_MB |262144 |
+-------+----------------+-------------------------------+
|NETWORK|interface_name s|enp3s0f0, enp3s0f1 |
| +----------------+-------------------------------+
| |vendor,model |Intel,X710 Dual Port |
| +----------------+-------------------------------+
| |bandwidth |10G |
| +----------------+-------------------------------+
| |interface_names |ens2f0, ens2f1 |
| +----------------+-------------------------------+
| |vendor,model |Intel,X710 Dual Port |
| +----------------+-------------------------------+
| |bandwidth |10G |
+-------+----------------+-------------------------------+
|STORAGE|dev_name |/dev/sda |
| +----------------+-------------------------------+
| |vendor,model | | raid1 - Lenovo 720ix |
| | | | 2 disks Intel S3610 |
| +----------------+-------------------------------+
| |SSD/HDD |SSD |
| +----------------+-------------------------------+
| |size |799GB |
+-------+----------------+-------------------------------+
.. table:: Description of servers hardware type 3
+-------+----------------+-------------------------------+
|server |vendor,model |Lenovo ThinkServer RD650 |
+-------+----------------+-------------------------------+
|CPU |vendor,model |Intel,E5-2670 v3 |
| +----------------+-------------------------------+
| |processor_count |2 |
| +----------------+-------------------------------+
| |core_count |12 |
| +----------------+-------------------------------+
| |frequency_MHz |2500 |
+-------+----------------+-------------------------------+
|RAM |vendor,model |Samsung, M393A2G40DB0-CPB |
| +----------------+-------------------------------+
| |amount_MB |131916 |
+-------+----------------+-------------------------------+
|NETWORK|interface_name s|enp3s0f0, enp3s0f1 |
| +----------------+-------------------------------+
| |vendor,model |Intel,X710 Dual Port |
| +----------------+-------------------------------+
| |bandwidth |10G |
| +----------------+-------------------------------+
| |interface_names |ens2f0, ens2f1 |
| +----------------+-------------------------------+
| |vendor,model |Intel,X710 Dual Port |
| +----------------+-------------------------------+
| |bandwidth |10G |
+-------+----------------+-------------------------------+
|STORAGE|dev_name |/dev/sda |
| +----------------+-------------------------------+
| |vendor,model | | raid1 - Lenovo 720ix |
| | | | 2 disks Intel S3610 |
| +----------------+-------------------------------+
| |SSD/HDD |SSD |
| +----------------+-------------------------------+
| |size |799GB |
| +----------------+-------------------------------+
| |dev_name |/dev/sdb |
| +----------------+-------------------------------+
| |vendor,model | | raid10 - Lenovo 720ix |
| | | | 10 disks 2T |
| +----------------+-------------------------------+
| |SSD/HDD |HDD |
| +----------------+-------------------------------+
| |size |9999GB |
+-------+----------------+-------------------------------+
Network configuration of each server
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
All servers have same network configuration:
.. image:: configs/Network_Scheme.png
:alt: Network Scheme of the environment
:width: 650px
Software configuration on servers with controller, compute and compute-osd roles
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. table:: Services on servers by role
+------------+----------------------------+
|Role |Service name |
+============+============================+
|controller || horizon |
| || keystone |
| || nova-api |
| || nava-scheduler |
| || nova-cert |
| || nova-conductor |
| || nova-consoleauth |
| || nova-consoleproxy |
| || cinder-api |
| || cinder-backup |
| || cinder-scheduler |
| || cinder-volume |
| || glance-api |
| || glance-glare |
| || glance-registry |
| || neutron-dhcp-agent |
| || neutron-l3-agent |
| || neutron-metadata-agent |
| || neutron-openvswitch-agent |
| || neutron-server |
| || heat-api |
| || heat-api-cfn |
| || heat-api-cloudwatch |
| || ceph-mon |
| || rados-gw |
| || memcached |
| || rabbitmq_server |
| || mysqld |
| || galera |
| || corosync |
| || pacemaker |
| || haproxy |
+------------+----------------------------+
|compute || nova-compute |
| || neutron-l3-agent |
| || neutron-metadata-agent |
| || neutron-openvswitch-agent |
+------------+----------------------------+
|compute-osd || nova-compute |
| || neutron-l3-agent |
| || neutron-metadata-agent |
| || neutron-openvswitch-agent |
| || ceph-osd |
+------------+----------------------------+
.. table:: Software version on servers with controller, compute and compute-osd roles
+------------+-------------------+
|Software |Version |
+============+===================+
|OpenStack |Mitaka |
+------------+-------------------+
|Ceph |Hammer |
+------------+-------------------+
|Ubuntu |Ubuntu 14.04.3 LTS |
+------------+-------------------+
You can find outputs of some commands and /etc folder in the following archives:
| :download:`controller-1.tar.gz <configs/controller-1.tar.gz>`
| :download:`controller-2.tar.gz <configs/controller-2.tar.gz>`
| :download:`controller-3.tar.gz <configs/controller-3.tar.gz>`
| :download:`compute-1.tar.gz <configs/compute-1.tar.gz>`
| :download:`compute-osd-1.tar.gz <configs/compute-osd-1.tar.gz>`
Software configuration on servers with Rally role
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
On this server should be installed Rally. How to do it you can find in
`Rally installation documentation`_
.. table:: Software version on server with Rally role
+------------+-------------------+
|Software |Version |
+============+===================+
|Rally |0.5.0 |
+------------+-------------------+
|Ubuntu |Ubuntu 14.04.3 LTS |
+------------+-------------------+
Test results
============
Test Case 1: Basic Neutron test suite
-------------------------------------
The following list of tests were run with the `default configuration`_ against
`Lab A (200 nodes)`_:
* create-and-list-floating-ips
* create-and-list-networks
* create-and-list-ports
* create-and-list-routers
* create-and-list-security-groups
* create-and-list-subnets
* create-and-delete-floating-ips
* create-and-delete-networks
* create-and-delete-ports
* create-and-delete-routers
* create-and-delete-security-groups
* create-and-delete-subnets
* create-and-update-networks
* create-and-update-ports
* create-and-update-routers
* create-and-update-security-groups
* create-and-update-subnets
The time needed for each scenario can be comparatively presented using the
following chart:
.. image:: reports/basic_neutron.png
:alt: Basic Neutron scenarios timings comparison
:width: 650px
To overview extended information please download the following report:
:download:`basic_neutron.html <reports/basic_neutron.html>`
.. _default configuration: https://github.com/openstack/rally/tree/master/samples/tasks/scenarios/neutron
Test Case 2: Stressful Neutron test suite
-----------------------------------------
The following list of tests were run against both `Lab A (200 nodes)`_ and
`Lab B (378 nodes)`_:
* create-and-list-networks
* create-and-list-ports
* create-and-list-routers
* create-and-list-security-groups
* create-and-list-subnets
* boot-and-list-server
* boot-and-delete-server-with-secgroups
* boot-runcommand-delete
Here is short representation of the collected results:
+--------------------------------------+-----------------------+-------------------------+---------------------------------------------+
|Scenario |Iterations/concurrency |Time, sec |Errors |
| +-----------+-----------+------------+------------+----------------------+----------------------+
| |Lab A |Lab B |Lab A |Lab B |Lab A |Lab B |
+======================================+===========+===========+============+============+======================+======================+
|create-and-list-networks |3000/50 |5000/50 |avg 2.375 |avg 3.654 |1 |6 |
| | | |max 7.904 |max 11.669 |Internal server error |Internal server error |
| | | | | |while processing your |while processing your |
| | | | | |request |request |
+--------------------------------------+-----------+-----------+------------+------------+----------------------+----------------------+
|create-and-list-ports |1000/50 |2000/50 |avg 123.97 |avg 99.274 |1 |0 |
| | | |max 277.977 |max 270.84 |Internal server error | |
| | | | | |while processing your | |
| | | | | |request | |
+--------------------------------------+-----------+-----------+------------+------------+----------------------+----------------------+
|create-and-list-routers |2000/50 |2000/50 |avg 15.59 |avg 12.94 |0 |0 |
| | | |max 29.006 |max 19.398 | | |
+--------------------------------------+-----------+-----------+------------+------------+----------------------+----------------------+
|create-and-list-security-groups |50/1 |1000/50 |avg 210.706 |avg 68.712 |0 |0 |
| | | |max 210.706 |max 169.315 | | |
+--------------------------------------+-----------+-----------+------------+------------+----------------------+----------------------+
|create-and-list-subnets |2000/50 |2000/50 |avg 25.973 |avg 17.415 |1 |0 |
| | | |max 64.553 |max 50.415 |Internal server error | |
| | | | | |while processing your | |
| | | | | |request | |
+--------------------------------------+-----------+-----------+------------+------------+----------------------+----------------------+
|boot-and-list-server |4975/50 |1000/50 |avg 21.445 |avg 14.375 |0 |0 |
| | | |max 40.736 |max 25.21 | | |
+--------------------------------------+-----------+-----------+------------+------------+----------------------+----------------------+
|boot-and-delete-server-with-secgroups |4975/200 |1000/100 |avg 190.772 |avg 65.651 |394 |0 |
| | | |max 443.518 |max 95.651 |Server has ERROR | |
| | | | | |status; The server | |
| | | | | |didn't respond in | |
| | | | | |time. | |
+--------------------------------------+-----------+-----------+------------+------------+----------------------+----------------------+
|boot-runcommand-delete |2000/15 |3000/50 |avg 28.39 |avg 28.587 |34 |1 |
| | | |max 35.756 |max 85.659 |Rally tired waiting |Resource <Server: |
| | | | | |for Host ip:<ip> to |s_rally_b58e9bd |
| | | | | |become ('ICMP UP'), |e_Y369JdPf> has ERROR |
| | | | | |current status |status. Deadlock |
| | | | | |('ICMP DOWN') |found when trying to |
| | | | | | |get lock. |
+--------------------------------------+-----------+-----------+------------+------------+----------------------+----------------------+
During execution of Rally were filed and fixed bugs affecting
`boot-and-delete-server-with-secgroups` and `boot-runcommand-delete` scenarios
on Lab A:
* `Bug LP #1610303 l2pop mech fails to update_port_postcommit on a loaded cluster`_ ,
fix - https://review.openstack.org/353835
* `Bug LP #1614452 Port create time grows at scale due to dvr arp update`_ ,
fix - https://review.openstack.org/357052
With these fixes applied on Lab B mentioned Rally scenarios passed
*successfully*.
Other bugs that were faced:
* `Bug LP #1595837 Neutron failed to create network`_
* `Bug LP #1610153 nova list can sometimes return 404`_
Observed trends
~~~~~~~~~~~~~~~
Create and list networks: the total time spent on each iteration grows linearly
.. image:: reports/create_and_list_networks.png
:alt: Create and list networks trend
:width: 650px
Create and list routers: router list operation time gradually grows from 0.12
to 1.5 sec (2000 iterations).
.. image:: reports/create_and_list_routers_list.png
:alt: Create and list routers trend (list operation)
:width: 650px
Create and list routers: total load duration remains line-rate
.. image:: reports/create_and_list_routers.png
:alt: Create and list routers trend (total time)
:width: 650px
Create and list subnets: subnet list operation time increases ~ after 1750
iterations (4.5 sec at 1700th iteration to 10.48 at 1800th iteration).
.. image:: reports/create_and_list_subnets_list.png
:alt: Create and list subnets trend (list operation)
:width: 650px
Create and list subnets: creating subnets has time peaks after 1750 iterations
.. image:: reports/create_and_list_subnets_create.png
:alt: Create and list subnets trend (create operation)
:width: 650px
Create and list security groups: secgroup list operation exposes the most rapid
growth rate with time increasing from 0.548 sec in first iteration to over 10
sec in last iterations
.. image:: reports/create_and_list_secgroup_list.png
:alt: Create and list security groups trend (list operation)
:width: 650px
More details can be found in original Rally report:
:download:`stress_neutron.html <reports/stress_neutron.html>`
Test case 3: Neutron scalability test with many networks
--------------------------------------------------------
In our tests 100 networks (each with a subnet, router and a VM) were created
per each iteration.
+-----------------------+--------------+--------------+-------------------------------+
|Iterations/concurrency |Avg time, sec |Max time, sec |Errors |
+=======================+==============+==============+===============================+
|10/1 |1237.389 |1294.549 |0 |
+-----------------------+--------------+--------------+-------------------------------+
|20/3 |1298.611 |1425.878 |1 |
| | | |HTTPConnectionPool Read time |
| | | |out |
+-----------------------+--------------+--------------+-------------------------------+
Load graph for run with 20 iterations/concurrency 3:
.. image:: reports/scale_neutron_networks.png
:alt: Neutron scalability test with many networks trend
:width: 650px
More details can be found in original Rally report:
:download:`scale_neutron_networks.html <reports/scale_neutron_networks.html>`
Test case 4: Neutron scalability test with many servers
-------------------------------------------------------
During each iteration this test creates huge number of VMs (100 in our case)
per a single network, hence it is possible to check the case with many number
of ports per subnet.
+-----------------------+--------------+--------------+-------------------------------+
|Iterations/concurrency |Avg time, sec |Max time, sec |Errors |
+=======================+==============+==============+===============================+
|10/1 |100.422 |104.315 |0 |
+-----------------------+--------------+--------------+-------------------------------+
|20/3 |119.767 |147.107 |0 |
+-----------------------+--------------+--------------+-------------------------------+
Load graph for run with 20 iterations/concurrency 3:
.. image:: reports/scale_neutron_servers.png
:alt: Neutron scalability test with many servers trend
:width: 650px
More details can be found in original Rally report:
:download:`scale_neutron_servers.html <reports/scale_neutron_servers.html>`
.. references:
.. _Rally installation documentation: https://rally.readthedocs.io/en/latest/install.html
.. _Bug LP #1610303 l2pop mech fails to update_port_postcommit on a loaded cluster: https://bugs.launchpad.net/neutron/+bug/1610303
.. _Bug LP #1614452 Port create time grows at scale due to dvr arp update: https://bugs.launchpad.net/neutron/+bug/1614452
.. _Bug LP #1595837 Neutron failed to create network: https://bugs.launchpad.net/mos/+bug/1595837
.. _Bug LP #1610153 nova list can sometimes return 404: https://bugs.launchpad.net/nova/+bug/1610153

864
doc/source/test_results/neutron_control_plane/reports/basic_neutron.html Normal file
View File

File diff suppressed because one or more lines are too long

BIN
doc/source/test_results/neutron_control_plane/reports/basic_neutron.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 124 KiB

BIN
doc/source/test_results/neutron_control_plane/reports/create_and_list_networks.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 123 KiB

BIN
doc/source/test_results/neutron_control_plane/reports/create_and_list_routers.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 133 KiB

BIN
doc/source/test_results/neutron_control_plane/reports/create_and_list_routers_list.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 150 KiB

BIN
doc/source/test_results/neutron_control_plane/reports/create_and_list_secgroup_list.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 162 KiB

BIN
doc/source/test_results/neutron_control_plane/reports/create_and_list_subnets_create.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 162 KiB

BIN
doc/source/test_results/neutron_control_plane/reports/create_and_list_subnets_list.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 141 KiB

873
doc/source/test_results/neutron_control_plane/reports/scale_neutron_networks.html Normal file
View File

File diff suppressed because one or more lines are too long

BIN
doc/source/test_results/neutron_control_plane/reports/scale_neutron_networks.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 82 KiB

873
doc/source/test_results/neutron_control_plane/reports/scale_neutron_servers.html Normal file
View File

File diff suppressed because one or more lines are too long

BIN
doc/source/test_results/neutron_control_plane/reports/scale_neutron_servers.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 84 KiB

873
doc/source/test_results/neutron_control_plane/reports/stress_neutron.html Normal file
View File

File diff suppressed because one or more lines are too long