diff --git a/deploy-guide/source/app-nova-cells.rst b/deploy-guide/source/app-nova-cells.rst new file mode 100644 index 0000000..723d94f --- /dev/null +++ b/deploy-guide/source/app-nova-cells.rst @@ -0,0 +1,245 @@ +Appendix G: Additional Nova Cells +================================= + +Overview +++++++++ + +Since Pike all OpenStack charm deployments have two cells: cell0 which is +used to store instances which failed to get scheduled and cell1 which is an +active compute cell. As of the 18.11 charm release multiple nova cells can be +created when deploying a cloud or an existing deployment can be extended to +add additional cells. + +Nova Cells v2 Topology +++++++++++++++++++++++ + +Nova cells v2 is entirely distinct from Nova cells v1. V1 has been deprecated +and special care should be taken when reading nova documentation to ensure that +it is v2 specific. Also, this is a *Nova* cell, other services such as + +.. note:: + + This is a *Nova* cell, other services such as Neutron, Glance etc are not + cell aware. + +The Super Conductor +~~~~~~~~~~~~~~~~~~~ + +For a deployment with multiple cells a dedicated conductor must be run for the +API-level services. This conductor will have access to the API database and a +dedicated message queue. This is called the super conductor to distinguish its +place and purpose from the per-cell conductor nodes +(See `Cells Layout `_.). The existing *nova-cloud-controller* charm already performs +the role of the super conductor for the whole cloud and, in addition, acts as a +cell conductor for cell1. + + +nova-cell-controller charm +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A new *nova-cell-controller* charm has been created. This charm just runs the +*nova-conductor* service and proxies some charm information, such as ssh keys, +between the cell compute nodes and the super-conductor. An instance of +*nova-cell-controller* is required per additional cell. + +Using application names to distinguish between cells +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Each cell requires the following charms: + +* nova-cell-controller +* rabbitmq-server +* percona-cluster +* nova-compute + +To allow each application to be configured differently between cells and to +be able to distinguish which instance of an application goes with which cell it +is useful to add the cell name to the service name when deploying the +components. + +.. code:: bash + + juju deploy nova-compute nova-compute-cell4 + +Rabbit for Neutron +~~~~~~~~~~~~~~~~~~ + +One of the motivations for using cells is to help with scaling to a large +number of compute nodes. While not a cells feature (nor a new feature) it is +worth pointing out that the charms support breaking Neutron message traffic +out onto a separate broker. + +.. code:: bash + + juju add-relation neutron-api:amqp rabbitmq-server-neutron:amqp + juju add-relation neutron-gateway:amqp rabbitmq-server-neutron:amqp + juju add-relation neutron-openvswitch:amqp rabbitmq-server-neutron:amqp + juju add-relation ceilometer:amqp-listener rabbitmq-server-neutron:amqp + +Adding A New Cell to an Existing Deployment ++++++++++++++++++++++++++++++++++++++++++++ + +In an existing deployment cell1 already exists as do the services to support +it. To add 'cell2' to this deployment the new cell applications need to be +deployed and related to the existing applications. + +Add the four cell applications: + +.. code:: bash + + juju deploy cs:percona-cluster mysql-cell2 + juju deploy cs:rabbitmq-server rabbitmq-server-nova-cell2 + juju deploy cs:nova-compute nova-compute-cell2 + juju deploy cs:nova-cell-controller --config cell-name='cell2' nova-cell-controller-cell2 + +Relate the new cell applications to each other: + +.. code:: bash + + juju add-relation nova-compute-cell2:amqp rabbitmq-server-nova-cell2:amqp + juju add-relation nova-cell-controller-cell2:amqp rabbitmq-server-nova-cell2:amqp + juju add-relation nova-cell-controller-cell2:shared-db mysql-cell2:shared-db + juju add-relation nova-cell-controller-cell2:cloud-compute nova-compute-cell2:cloud-compute + +Relate the super conductor to the new cell: + +.. code:: bash + + juju add-relation nova-cloud-controller:nova-cell-api nova-cell-controller-cell2:nova-cell-compute + juju add-relation nova-cloud-controller:amqp-cell rabbitmq-server-nova-cell2:amqp + juju add-relation nova-cloud-controller:shared-db-cell mysql-cell2:shared-db + + +Relate the new cell to network, image and identity services: + +.. code:: bash + + juju add-relation nova-compute-cell2:neutron-plugin neutron-openvswitch:neutron-plugin + juju add-relation nova-compute-cell2:image-service glance:image-service + juju add-relation nova-compute-cell2:cloud-credentials keystone:identity-credentials + +Relate the new cell to telemetry services. + +.. note:: + + The ceilometer charm has an *amqp* and an *amqp-listerner* interface. + ceilometer will listen and post messages to the broker related to the + *amqp* interface. It will only listen to messages posted to the broker(s) + related to the *amqp-listener*. Therefore services that consume messages + from ceilometer, such as aodh, should be related to the broker associated + with ceilometers *amqp* interface. + +.. code:: bash + + juju add-relation ceilometer:amqp-listener rabbitmq-server-nova-cell2:amqp + juju add-relation ceilometer-agent:nova-ceilometer nova-compute-cell2:nova-ceilometer + +New Deployments ++++++++++++++++ + +For all cell deploymends ensure the following: + +* Application naming scheme such that the cell an application belongs to is + clear. +* Naming the central message broker such that its purpose is clear + eg rabbitmq-server-general + +If cells is being used primarly to help with a large scale out of compute +resources then in addition: + +* Do not relate compute nodes to the *nova-cloud-controller* +* Have a separate message broker for Neutron. + +Below is an example of an overlay which can be used when doing a fresh deploy +to add a second cell: + +.. code:: yaml + + applications: + mysql-cell2: + charm: cs:percona-cluster + series: bionic + num_units: 1 + options: + max-connections: 1000 + nova-cell-controller-cell2: + charm: cs:nova-cell-controller + series: bionic + num_units: 1 + options: + cell-name: "cell2" + nova-compute-cell2: + charm: cs:nova-compute + series: bionic + num_units: 1 + constraints: mem=4G + options: + config-flags: default_ephemeral_format=ext4 + enable-live-migration: true + enable-resize: true + migration-auth-type: ssh + rabbitmq-server-nova-cell2: + charm: cs:rabbitmq-server + num_units: 1 + relations: + - - nova-compute-cell2:neutron-plugin + - neutron-openvswitch:neutron-plugin + - - nova-cloud-controller:amqp-cell + - rabbitmq-server-nova-cell2:amqp + - - ceilometer:amqp-listener + - rabbitmq-server-nova-cell2 + - - ceilometer-agent + - nova-compute-cell2 + - - nova-cloud-controller:nova-cell-api + - nova-cell-controller-cell2:nova-cell-compute + - - nova-cloud-controller:shared-db-cell + - mysql-cell2:shared-db + - - nova-cloud-controller:amqp-cell + - rabbitmq-server-nova-cell2:amqp + - - nova-compute-cell2:amqp + - rabbitmq-server-nova-cell2:amqp + - - nova-cell-controller-cell2:cloud-compute + - nova-compute-cell2:cloud-compute + - - nova-compute-cell2:image-service + - glance:image-service + - - nova-cell-controller-cell2:amqp + - rabbitmq-server-nova-cell2:amqp + - - nova-cell-controller-cell2:shared-db + - mysql-cell2:shared-db + - - nova-compute-cell2:cloud-credentials + - keystone:identity-credentials + +Targeting instances at a cell +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Instances can be targetted at a specific cell by manually maintaining host +aggregates and corresponding flavors which target those host aggregates. For +example, assume *cell2* has one compute host *juju-250b86-prod-19*. Create a +host aggregate for *cell2* and add the compute host into it. + +.. code:: bash + + openstack aggregate create --property cell=cell2 ag_cell2 + openstack aggregate add host ag_cell2 juju-250b86-prod-19 + + +Now create a flavor that targets that cell. + +.. code:: bash + + openstack flavor create --id 5 --ram 2048 --disk 10 --ephemeral 0 --vcpus 1 --public --property cell=cell2 m1.cell2.small + +Finally, enable the *AggregateInstanceExtraSpecsFilter* + +.. code:: bash + + FILTERS=$(juju config nova-cloud-controller scheduler-default-filters) + juju config nova-cloud-controller scheduler-default-filters="${FILTERS},AggregateInstanceExtraSpecsFilter" + +Now instances that use the *m1.cell2.small* filter will land on cell2 compute +hosts. + +.. note:: + + These host aggregates need to be manually updated when compute nodes are + added to the cell. diff --git a/deploy-guide/source/app.rst b/deploy-guide/source/app.rst index 53a4208..e7e16bc 100644 --- a/deploy-guide/source/app.rst +++ b/deploy-guide/source/app.rst @@ -11,3 +11,4 @@ Appendices app-encryption-at-rest.rst app-certificate-management.rst app-series-upgrade.rst + app-nova-cells.rst