.. ko.py.readme.rst ============================== "ko.py" - Kubernetes OpenStack ============================== Purpose ======= This is a tool to deploy OpenStack on a Kubernetes Cluster using Kolla images and Kolla-Kubernetes on bare metal servers or virtual machines. It sticks to the methods outlined in the kolla-kubernetes Bare Metal Deployment Guide: https://docs.openstack.org/developer/kolla-kubernetes/deployment-guide.html Features ======== 1. Supports both Centos and Ubuntu natively. 2. Requires just a VM with two NIC's, low congnitive overhead. 3. Simplicity to run: 'ko.py int1 int2' 4. Options to change the versions of all the tools, like helm, kubernetes etc. 5. Option to change the version of OpenStack as needed. 6. Easy on the eye output, with optional verbose mode for more information. 7. Contains a demo mode that walks the user through each step with additional information and instruction. 8. Verifies it's completeness by generating a VM in the OpenStack Cluster. 9. Leaves the user with a working OpenStack Cluster with all the basic services. 10. Lots of options to customize - even edit globals.yaml and cloud.yaml before deploying. Host machine requirements ========================= The host machine must satisfy the following minimum requirements: - 2 network interfaces - 8GB min, 16GB preferred RAM - 40G min, 80GB preferred disk space - 2 CPU's Min, 4 preferred CPU's - Root access to the deployment host machine Prerequisites ============= Verify the state of network interfaces. If using a VM spawned on OpenStack as the host machine, the state of the second interface will be DOWN on booting the VM:: ip addr show Bring up the second network interface if it is down:: ip link set ens4 up However as this interface will be used for Neutron External, this Interface should not have an IP Address. Verify this with:: ip addr show Mandatory Inputs ================ 1. mgmt_int (network_interface):: Name of the interface to be used for management operations. The `network_interface` variable is the interface to which Kolla binds API services. For example, when starting Mariadb, it will bind to the IP on the interface list in the ``network_interface`` variable. 2. neutron_int (neutron_external_interface):: Name of the interface to be used for Neutron operations. The `neutron_external_interface` variable is the interface that will be used for the external bridge in Neutron. Without this bridge the deployment instance traffic will be unable to access the rest of the Internet. To create two interfaces like this in Ubuntu, for example:: Edit /etc/network/interfaces: # The primary network interface auto eth0 iface eth0 inet dhcp # Neutron network interface (up but no ip address) auto eth1 iface eth1 inet manual ifconfig eth1 up TODO ==== 1. Convert to using https://github.com/kubernetes-incubator/client-python 2. Add option to use a CNI other than canal 3. Note there are various todo's scattered inline as well. Recomendations ============== 1. Due to the length the script can run for, recomend disabling sudo timeout:: sudo visudo Add: 'Defaults timestamp_timeout=-1' 2. Due to the length of time the script can run for, I recommend using nohup:: E.g. nohup python -u k8s.py eth0 eth1 Then in another window: tail -f nohup.out OUTPUT ====== An example of ko.py running from beginning to end:: ubuntu@ip-10-0-0-241:~/os$ ../k8s/ko.py ens3 ens4 -cn [sudo] password for ubuntu: [sudo] password for ubuntu: [sudo] password for ubuntu: ******************************************* Kubernetes - Bring up a Kubernetes Cluster: ******************************************* Linux info: ('Ubuntu', '16.04', 'xenial') Networking: Management Int: ens3 Neutron Int: ens4 Management IP: 10.0.0.241 VIP Keepalive: 10.0.0.56 Versions: Docker version : 1.12.6 Openstack version: ocata(4.0.0) Helm version: 2.6.2 K8s version: 1.8.2 Ansible version: 2.2.0.0 Jinja2 version: 2.8.1 Base version: centos (01/15) Kubernetes - Installing base tools (02/15) Kubernetes - Setup NTP (03/15) Kubernetes - Turn off firewall and ISCSID (04/15) Kubernetes - Creating Kubernetes repo, installing Kubernetes packages (05/15) Kubernetes - Start docker and setup the DNS server with the service CIDR (06/15) Kubernetes - Reload the hand-modified service files (07/15) Kubernetes - Enable and start kubelet (08/15) Kubernetes - Fix iptables to enable bridging (09/15) Kubernetes - Deploying Kubernetes with kubeadm (Slow!) You can now join any number of machines by running the following on each node as root: kubeadm join --token 5e76b0.5ac8cc93b3f53bbf 10.0.0.241:6443 --discovery-token-ca-cert-hash sha256:46665415bd9c77d9eb08af4d427b0925d8036278700894da412514cdd4c45195 (10/15) Kubernetes - Load kubeadm credentials into the system Note "kubectl get pods --all-namespaces" should work now (11/15) Kubernetes - Wait for basic Kubernetes (6 pods) infrastructure *Running pod(s) status after 20 seconds 2:6* *Running pod(s) status after 50 seconds 3:6* *Running pod(s) status after 60 seconds 4:6* *Running pod(s) status after 70 seconds 5:6* *All pods 6/6 are started, continuing* (12/15) Kubernetes - Add API Server (13/15) Kubernetes - Deploy pod network SDN using Canal CNI Wait for all pods to be in Running state: *01 pod(s) are not in Running state* *All pods are in Running state* (14/15) Kubernetes - Mark master node as schedulable by untainting the node (15/15) Kubernetes - Test 'nslookup kubernetes' - bring up test pod Wait for all pods to be in Running state: *01 pod(s) are not in Running state* *All pods are in Running state* ************************************ Kubernetes Cluster is up and running ************************************ ************************** Kolla - install OpenStack: ************************** (01/45) Kolla - Overide default RBAC settings (02/45) Kolla - Install and deploy Helm version 2.6.2 - Tiller pod Wait for all pods to be in Running state: *01 pod(s) are not in Running state* *All pods are in Running state* (03/45) Kolla - Helm successfully installed (04/45) Kolla - Clone kolla-ansible (05/45) Kolla - Clone kolla-kubernetes (06/45) Kolla - Install kolla-ansible and kolla-kubernetes (07/45) Kolla - Copy default kolla-ansible configuration to /etc (08/45) Kolla - Copy default kolla-kubernetes configuration to /etc (09/45) Kolla - Setup Loopback LVM for Cinder (Slow!) (10/45) Kolla - Install Python Openstack Client (11/45) Kolla - Generate default passwords via SPRNG (12/45) Kolla - Create a Kubernetes namespace "kolla" to isolate this Kolla deployment (13/45) Kolla - Label Nodes: Label the AIO node as 'kolla_compute' Label the AIO node as 'kolla_controller' (14/45) Kolla - Modify global.yml to setup network_interface and neutron_interface (15/45) Kolla - Add default config to globals.yml (16/45) Kolla - Enable qemu (17/45) Kolla - Generate the default configuration (18/45) Kolla - Generate the Kubernetes secrets and register them with Kubernetes (19/45) Kolla - Create and register the Kolla config maps (20/45) Kolla - Build all Helm microcharts, service charts, and metacharts (Slow!) (21/45) Kolla - Verify number of helm images 195 Helm images created (22/45) Kolla - Create a version 4 cloud.yaml (23/45) Kolla - Helm Install service chart: \--'openvswitch'--/ Wait for all pods to be in Running state: *02 pod(s) are not in Running state* *01 pod(s) are not in Running state* *All pods are in Running state* (24/45) Kolla - Helm Install service chart: \--'mariadb'--/ Wait for all pods to be in Running state: *02 pod(s) are not in Running state* *01 pod(s) are not in Running state* *All pods are in Running state* (25/45) Kolla - Helm Install service chart: \--'rabbitmq'--/ (26/45) Kolla - Helm Install service chart: \--'memcached'--/ (27/45) Kolla - Helm Install service chart: \--'keystone'--/ (28/45) Kolla - Helm Install service chart: \--'glance'--/ (29/45) Kolla - Helm Install service chart: \--'cinder-control'--/ (30/45) Kolla - Helm Install service chart: \--'cinder-volume-lvm'--/ (31/45) Kolla - Helm Install service chart: \--'horizon'--/ (32/45) Kolla - Helm Install service chart: \--'neutron'--/ Wait for all pods to be in Running state: *45 pod(s) are not in Running state* *44 pod(s) are not in Running state* *43 pod(s) are not in Running state* *42 pod(s) are not in Running state* *40 pod(s) are not in Running state* *39 pod(s) are not in Running state* *37 pod(s) are not in Running state* *35 pod(s) are not in Running state* *34 pod(s) are not in Running state* *31 pod(s) are not in Running state* *30 pod(s) are not in Running state* *29 pod(s) are not in Running state* *28 pod(s) are not in Running state* *27 pod(s) are not in Running state* *26 pod(s) are not in Running state* *25 pod(s) are not in Running state* *23 pod(s) are not in Running state* *20 pod(s) are not in Running state* *19 pod(s) are not in Running state* *16 pod(s) are not in Running state* *15 pod(s) are not in Running state* *14 pod(s) are not in Running state* *13 pod(s) are not in Running state* *11 pod(s) are not in Running state* *10 pod(s) are not in Running state* *08 pod(s) are not in Running state* *06 pod(s) are not in Running state* *05 pod(s) are not in Running state* *04 pod(s) are not in Running state* *03 pod(s) are not in Running state* *02 pod(s) are not in Running state* *01 pod(s) are not in Running state* *All pods are in Running state* (33/45) Kolla - Helm Install service chart: \--'nova-control'--/ (34/45) Kolla - Helm Install service chart: \--'nova-compute'--/ Wait for all pods to be in Running state: *23 pod(s) are not in Running state* *22 pod(s) are not in Running state* *21 pod(s) are not in Running state* *19 pod(s) are not in Running state* *18 pod(s) are not in Running state* *17 pod(s) are not in Running state* *16 pod(s) are not in Running state* *15 pod(s) are not in Running state* *14 pod(s) are not in Running state* *13 pod(s) are not in Running state* *12 pod(s) are not in Running state* *11 pod(s) are not in Running state* *10 pod(s) are not in Running state* *09 pod(s) are not in Running state* *08 pod(s) are not in Running state* *01 pod(s) are not in Running state* *All pods are in Running state* (35/45) Kolla - Final Kolla Kubernetes OpenStack pods for namespace kube-system: NAME READY STATUS RESTARTS AGE canal-46w8r 3/3 Running 0 14m etcd-ip-10-0-0-241 1/1 Running 0 14m kube-apiserver-ip-10-0-0-241 1/1 Running 0 14m kube-controller-manager-ip-10-0-0-241 1/1 Running 0 14m kube-dns-545bc4bfd4-gnrlv 3/3 Running 0 15m kube-proxy-6c65v 1/1 Running 0 15m kube-scheduler-ip-10-0-0-241 1/1 Running 0 14m tiller-deploy-cffb976df-thwlt 1/1 Running 0 13m (36/45) Kolla - Final Kolla Kubernetes OpenStack pods for namespace kolla: NAME READY STATUS RESTARTS AGE cinder-api-649bc7654d-5gf6g 3/3 Running 0 6m cinder-scheduler-0 1/1 Running 0 6m cinder-volume-4n6rg 1/1 Running 3 5m glance-api-7f5b759667-95g4v 1/1 Running 0 6m glance-registry-74cc4c977d-956l4 3/3 Running 0 6m horizon-7966fccff7-dbn2s 1/1 Running 0 5m iscsid-xxmn4 1/1 Running 0 5m keystone-55d7f5c7c-kjrg9 1/1 Running 0 6m mariadb-0 1/1 Running 0 6m memcached-5b858fb696-4fmf6 2/2 Running 0 6m neutron-dhcp-agent-4xj76 1/1 Running 0 5m neutron-l3-agent-network-9j978 1/1 Running 0 5m neutron-metadata-agent-network-nlpvd 1/1 Running 0 5m neutron-openvswitch-agent-network-8cc2x 1/1 Running 0 5m neutron-server-68d97c559f-xwwjl 3/3 Running 0 5m nova-api-69876b658f-pmf4f 3/3 Running 0 2m nova-api-create-cell-sm6bj 1/1 Running 0 2m nova-compute-s9rbl 1/1 Running 0 2m nova-conductor-0 1/1 Running 0 2m nova-consoleauth-0 1/1 Running 0 2m nova-libvirt-zjw6h 1/1 Running 0 2m nova-novncproxy-58fb468d4b-4l57m 3/3 Running 0 2m nova-scheduler-0 1/1 Running 0 2m openvswitch-ovsdb-network-j4gbm 1/1 Running 0 6m openvswitch-vswitchd-network-6q8lw 1/1 Running 0 6m placement-api-697b85cf9-6twdf 1/1 Running 0 2m rabbitmq-0 1/1 Running 0 6m tgtd-wblfn 1/1 Running 0 5m (37/45) Kolla - Create a keystone admin account and source in to it (38/45) Kolla - Allow Ingress by changing neutron rules (39/45) Kolla - Configure Neutron, pull images Enter passphrase (empty for no passphrase): Enter same passphrase again: (40/45) Kolla - Create a demo VM in our OpenStack cluster To create a demo image VM do: . ~/keystonerc_admin; openstack server create --image cirros --flavor m1.tiny --key-name mykey --nic net-id=c7df4092-b88b-4596-bbad-6c1a2888ee82 test Kubernetes - Wait for VM demo1 to be in running state: *Kubernetes - VM demo1 is not Running yet - wait 15s* *Kubernetes - VM demo1 is not Running yet - wait 15s* *Kubernetes - VM demo1 is not Running yet - wait 15s* *Kubernetes - VM demo1 is Running* (41/45) Kolla - Create floating ip (42/45) Kolla - nova list to see floating IP and demo VM +--------------------------------------+-------+--------+------------+-------------+-------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+-------+--------+------------+-------------+-------------------+ | 1bd09c59-85a0-4d8f-9fc2-2949ca01192d | demo1 | ACTIVE | - | Running | public1=10.0.0.60 | +--------------------------------------+-------+--------+------------+-------------+-------------------+ (43/45) Kolla - To Access Horizon: Point your browser to: 10.3.3.189 OS_PASSWORD=oUfo1H4hSLxkJJbEmtieN7UN2sqwZfRGpaR8U6lW OS_USERNAME=admin ************************************************************************** Successfully deployed Kolla-Kubernetes. OpenStack Cluster is ready for use **************************************************************************