openstack-ansible/doc/source/user/ceph/full-deploy.rst

Ceph production example

This section describes an example production environment for a working OpenStack-Ansible (OSA) deployment with high availability services and using the Ceph backend for images, volumes, and instances.

This example environment has the following characteristics:

• Three infrastructure (control plane) hosts with ceph-mon containers
• Two compute hosts
• Three Ceph OSD storage hosts
• One log aggregation host
• Multiple Network Interface Cards (NIC) configured as bonded pairs for each host
• Full compute kit with the Telemetry service (ceilometer) included, with Ceph configured as a storage back end for the Image (glance), and Block Storage (cinder) services
• Internet access via the router address 172.29.236.1 on the Management Network

Integration with Ceph

OpenStack-Ansible allows Ceph storage cluster integration in two ways:

• connecting to your own ceph cluster by pointing to its information in user_variables.yml
• deploying a ceph cluster by using the roles maintained by the Ceph-Ansible project. Deployers can enable the ceph-install playbook by adding hosts to the ceph-mon_hosts and ceph-osd_hosts groups in openstack_user_config.yml, and then configuring Ceph-Ansible specific vars in the OpenStack-Ansible user_variables.yml file.

This example will focus on the deployment of both OpenStack-Ansible and its Ceph cluster.

Network configuration

Network CIDR/VLAN assignments

The following CIDR and VLAN assignments are used for this environment.

Network	CIDR	VLAN
Management Network	172.29.236.0/22	10
Tunnel (VXLAN) Network	172.29.240.0/22	30
Storage Network	172.29.244.0/22	20

IP assignments

The following host name and IP address assignments are used for this environment.

Host name	Management IP	Tunnel (VxLAN) IP	Storage IP
lb_vip_address	172.29.236.9
infra1	172.29.236.11	172.29.240.11
infra2	172.29.236.12	172.29.240.12
infra3	172.29.236.13	172.29.240.13
log1	172.29.236.14
compute1	172.29.236.16	172.29.240.16	172.29.244.16
compute2	172.29.236.17	172.29.240.17	172.29.244.17
osd1	172.29.236.18	172.29.240.18	172.29.244.18
osd2	172.29.236.19	172.29.240.19	172.29.244.19
osd3	172.29.236.20	172.29.240.20	172.29.244.20

Host network configuration

Each host will require the correct network bridges to be implemented. The following is the /etc/network/interfaces file for infra1.

Note

If your environment does not have eth0, but instead has p1p1 or some other interface name, ensure that all references to eth0 in all configuration files are replaced with the appropriate name. The same applies to additional network interfaces.

../../../../etc/network/interfaces.d/openstack_interface.cfg.prod.example

Deployment configuration

Environment layout

The /etc/openstack_deploy/openstack_user_config.yml file defines the environment layout.

The following configuration describes the layout for this environment.

../../../../etc/openstack_deploy/openstack_user_config.yml.prod-ceph.example

Environment customizations

The optionally deployed files in /etc/openstack_deploy/env.d allow the customization of Ansible groups. This allows the deployer to set whether the services will run in a container (the default), or on the host (on metal).

For a ceph environment, you can run the cinder-volume in a container. To do this you will need to create a /etc/openstack_deploy/env.d/cinder.yml file with the following content:

../../../../etc/openstack_deploy/env.d/cinder-volume.yml.container.example

User variables

The /etc/openstack_deploy/user_variables.yml file defines the global overrides for the default variables.

For this example environment, we configure a HA load balancer. We implement the load balancer (HAProxy) with an HA layer (keepalived) on the infrastructure hosts. Your /etc/openstack_deploy/user_variables.yml must have the following content to configure haproxy, keepalived and ceph:

../../../../etc/openstack_deploy/user_variables.yml.prod-ceph.example