openstack
/
kolla-kubernetes
Code Issues Proposed changes
kolla-kubernetes/contrib/orchestration/ko.py.readme.rst

16 KiB
Raw Blame History

"ko.py" - Kubernetes OpenStack

Purpose

This is a tool to deploy OpenStack on a Kubernetes Cluster using Kolla images and Kolla-Kubernetes on bare metal servers or virtual machines.

It sticks to the methods outlined in the kolla-kubernetes Bare Metal Deployment Guide:

https://docs.openstack.org/developer/kolla-kubernetes/deployment-guide.html

Features

  1. Supports both Centos and Ubuntu natively.
  2. Requires just a VM with two NIC's, low congnitive overhead.
  3. Simplicity to run: 'ko.py int1 int2'
  4. Options to change the versions of all the tools, like helm, kubernetes etc.
  5. Option to change the version of OpenStack as needed.
  6. Easy on the eye output, with optional verbose mode for more information.
  7. Contains a demo mode that walks the user through each step with additional information and instruction.
  8. Verifies it's completeness by generating a VM in the OpenStack Cluster.
  9. Leaves the user with a working OpenStack Cluster with all the basic services.
  10. Lots of options to customize - even edit globals.yaml and cloud.yaml before deploying.

Host machine requirements

The host machine must satisfy the following minimum requirements:

  • 2 network interfaces
  • 8GB min, 16GB preferred RAM
  • 40G min, 80GB preferred disk space
  • 2 CPU's Min, 4 preferred CPU's
  • Root access to the deployment host machine

Prerequisites

Verify the state of network interfaces. If using a VM spawned on OpenStack as the host machine, the state of the second interface will be DOWN on booting the VM:

ip addr show

Bring up the second network interface if it is down:

ip link set ens4 up

However as this interface will be used for Neutron External, this Interface should not have an IP Address. Verify this with:

ip addr show

Mandatory Inputs

  1. mgmt_int (network_interface):: Name of the interface to be used for management operations.

The network_interface variable is the interface to which Kolla binds API services. For example, when starting Mariadb, it will bind to the IP on the interface list in the network_interface variable.

  1. neutron_int (neutron_external_interface)::

    Name of the interface to be used for Neutron operations.

The neutron_external_interface variable is the interface that will be used for the external bridge in Neutron. Without this bridge the deployment instance traffic will be unable to access the rest of the Internet.

To create two interfaces like this in Ubuntu, for example:

Edit /etc/network/interfaces:

# The primary network interface
auto eth0
iface eth0 inet dhcp

# Neutron network interface (up but no ip address)
auto eth1
iface eth1 inet manual
ifconfig eth1 up

TODO

  1. Convert to using https://github.com/kubernetes-incubator/client-python
  2. Add option to use a CNI other than canal
  3. Note there are various todo's scattered inline as well.

Recomendations

  1. Due to the length the script can run for, recomend disabling sudo timeout:

    sudo visudo
Add: 'Defaults    timestamp_timeout=-1'

  2. Due to the length of time the script can run for, I recommend using nohup:

    E.g. nohup python -u k8s.py eth0 eth1

Then in another window:

tail -f nohup.out

OUTPUT

An example of ko.py running from beginning to end:

ubuntu@ip-10-0-0-241:~/os$ ../k8s/ko.py ens3 ens4 -cn
[sudo] password for ubuntu:
[sudo] password for ubuntu:
[sudo] password for ubuntu:


*******************************************
Kubernetes - Bring up a Kubernetes Cluster:
*******************************************

Linux info:        ('Ubuntu', '16.04', 'xenial')

Networking:
Management Int:  ens3
Neutron Int:     ens4
Management IP:   10.0.0.241
VIP Keepalive:   10.0.0.56

Versions:
Docker version  :  1.12.6
Openstack version: ocata(4.0.0)
Helm version:      2.6.2
K8s version:       1.8.2
Ansible version:   2.2.0.0
Jinja2 version:    2.8.1
Base version:      centos


(01/15) Kubernetes - Installing base tools
(02/15) Kubernetes - Setup NTP
(03/15) Kubernetes - Turn off firewall and ISCSID
(04/15) Kubernetes - Creating Kubernetes repo, installing Kubernetes packages
(05/15) Kubernetes - Start docker and setup the DNS server with the service CIDR
(06/15) Kubernetes - Reload the hand-modified service files
(07/15) Kubernetes - Enable and start kubelet
(08/15) Kubernetes - Fix iptables to enable bridging
(09/15) Kubernetes - Deploying Kubernetes with kubeadm (Slow!)
You can now join any number of machines by running the following on each node as root:
kubeadm join --token 5e76b0.5ac8cc93b3f53bbf 10.0.0.241:6443 --discovery-token-ca-cert-hash sha256:46665415bd9c77d9eb08af4d427b0925d8036278700894da412514cdd4c45195
(10/15) Kubernetes - Load kubeadm credentials into the system
Note "kubectl get pods --all-namespaces" should work now
(11/15) Kubernetes - Wait for basic Kubernetes (6 pods) infrastructure
*Running pod(s) status after 20 seconds 2:6*
*Running pod(s) status after 50 seconds 3:6*
*Running pod(s) status after 60 seconds 4:6*
*Running pod(s) status after 70 seconds 5:6*
*All pods 6/6 are started, continuing*
(12/15) Kubernetes - Add API Server
(13/15) Kubernetes - Deploy pod network SDN using Canal CNI
Wait for all pods to be in Running state:
  *01 pod(s) are not in Running state*
  *All pods are in Running state*
  (14/15) Kubernetes - Mark master node as schedulable by untainting the node
  (15/15) Kubernetes - Test 'nslookup kubernetes' - bring up test pod
Wait for all pods to be in Running state:
  *01 pod(s) are not in Running state*
  *All pods are in Running state*


  ************************************
  Kubernetes Cluster is up and running
  ************************************

  **************************
  Kolla - install OpenStack:
  **************************

  (01/45) Kolla - Overide default RBAC settings
  (02/45) Kolla - Install and deploy Helm version 2.6.2 - Tiller pod
  Wait for all pods to be in Running state:
  *01 pod(s) are not in Running state*
  *All pods are in Running state*
  (03/45) Kolla - Helm successfully installed
  (04/45) Kolla - Clone kolla-ansible
  (05/45) Kolla - Clone kolla-kubernetes
  (06/45) Kolla - Install kolla-ansible and kolla-kubernetes
  (07/45) Kolla - Copy default kolla-ansible configuration to /etc
  (08/45) Kolla - Copy default kolla-kubernetes configuration to /etc
  (09/45) Kolla - Setup Loopback LVM for Cinder (Slow!)
  (10/45) Kolla - Install Python Openstack Client
  (11/45) Kolla - Generate default passwords via SPRNG
  (12/45) Kolla - Create a Kubernetes namespace "kolla" to isolate this Kolla deployment
  (13/45) Kolla - Label Nodes:
  Label the AIO node as 'kolla_compute'
  Label the AIO node as 'kolla_controller'
  (14/45) Kolla - Modify global.yml to setup network_interface and neutron_interface
  (15/45) Kolla - Add default config to globals.yml
  (16/45) Kolla - Enable qemu
  (17/45) Kolla - Generate the default configuration
  (18/45) Kolla - Generate the Kubernetes secrets and register them with Kubernetes
  (19/45) Kolla - Create and register the Kolla config maps
  (20/45) Kolla - Build all Helm microcharts, service charts, and metacharts (Slow!)
  (21/45) Kolla - Verify number of helm images
  195 Helm images created
  (22/45) Kolla - Create a version 4 cloud.yaml
  (23/45) Kolla - Helm Install service chart: \--'openvswitch'--/
Wait for all pods to be in Running state:
  *02 pod(s) are not in Running state*
  *01 pod(s) are not in Running state*
  *All pods are in Running state*
  (24/45) Kolla - Helm Install service chart: \--'mariadb'--/
Wait for all pods to be in Running state:
  *02 pod(s) are not in Running state*
  *01 pod(s) are not in Running state*
  *All pods are in Running state*
  (25/45) Kolla - Helm Install service chart: \--'rabbitmq'--/
  (26/45) Kolla - Helm Install service chart: \--'memcached'--/
  (27/45) Kolla - Helm Install service chart: \--'keystone'--/
  (28/45) Kolla - Helm Install service chart: \--'glance'--/
  (29/45) Kolla - Helm Install service chart: \--'cinder-control'--/
  (30/45) Kolla - Helm Install service chart: \--'cinder-volume-lvm'--/
  (31/45) Kolla - Helm Install service chart: \--'horizon'--/
  (32/45) Kolla - Helm Install service chart: \--'neutron'--/
Wait for all pods to be in Running state:
  *45 pod(s) are not in Running state*
  *44 pod(s) are not in Running state*
  *43 pod(s) are not in Running state*
  *42 pod(s) are not in Running state*
  *40 pod(s) are not in Running state*
  *39 pod(s) are not in Running state*
  *37 pod(s) are not in Running state*
  *35 pod(s) are not in Running state*
  *34 pod(s) are not in Running state*
  *31 pod(s) are not in Running state*
  *30 pod(s) are not in Running state*
  *29 pod(s) are not in Running state*
  *28 pod(s) are not in Running state*
  *27 pod(s) are not in Running state*
  *26 pod(s) are not in Running state*
  *25 pod(s) are not in Running state*
  *23 pod(s) are not in Running state*
  *20 pod(s) are not in Running state*
  *19 pod(s) are not in Running state*
  *16 pod(s) are not in Running state*
  *15 pod(s) are not in Running state*
  *14 pod(s) are not in Running state*
  *13 pod(s) are not in Running state*
  *11 pod(s) are not in Running state*
  *10 pod(s) are not in Running state*
  *08 pod(s) are not in Running state*
  *06 pod(s) are not in Running state*
  *05 pod(s) are not in Running state*
  *04 pod(s) are not in Running state*
  *03 pod(s) are not in Running state*
  *02 pod(s) are not in Running state*
  *01 pod(s) are not in Running state*
  *All pods are in Running state*
  (33/45) Kolla - Helm Install service chart: \--'nova-control'--/
  (34/45) Kolla - Helm Install service chart: \--'nova-compute'--/
Wait for all pods to be in Running state:
  *23 pod(s) are not in Running state*
  *22 pod(s) are not in Running state*
  *21 pod(s) are not in Running state*
  *19 pod(s) are not in Running state*
  *18 pod(s) are not in Running state*
  *17 pod(s) are not in Running state*
  *16 pod(s) are not in Running state*
  *15 pod(s) are not in Running state*
  *14 pod(s) are not in Running state*
  *13 pod(s) are not in Running state*
  *12 pod(s) are not in Running state*
  *11 pod(s) are not in Running state*
  *10 pod(s) are not in Running state*
  *09 pod(s) are not in Running state*
  *08 pod(s) are not in Running state*
  *01 pod(s) are not in Running state*
  *All pods are in Running state*
  (35/45) Kolla - Final Kolla Kubernetes OpenStack pods for namespace kube-system:
  NAME                                    READY     STATUS    RESTARTS   AGE
  canal-46w8r                             3/3       Running   0          14m
  etcd-ip-10-0-0-241                      1/1       Running   0          14m
  kube-apiserver-ip-10-0-0-241            1/1       Running   0          14m
  kube-controller-manager-ip-10-0-0-241   1/1       Running   0          14m
  kube-dns-545bc4bfd4-gnrlv               3/3       Running   0          15m
  kube-proxy-6c65v                        1/1       Running   0          15m
  kube-scheduler-ip-10-0-0-241            1/1       Running   0          14m
  tiller-deploy-cffb976df-thwlt           1/1       Running   0          13m
  (36/45) Kolla - Final Kolla Kubernetes OpenStack pods for namespace kolla:
  NAME                                      READY     STATUS    RESTARTS   AGE
  cinder-api-649bc7654d-5gf6g               3/3       Running   0          6m
  cinder-scheduler-0                        1/1       Running   0          6m
  cinder-volume-4n6rg                       1/1       Running   3          5m
  glance-api-7f5b759667-95g4v               1/1       Running   0          6m
  glance-registry-74cc4c977d-956l4          3/3       Running   0          6m
  horizon-7966fccff7-dbn2s                  1/1       Running   0          5m
  iscsid-xxmn4                              1/1       Running   0          5m
  keystone-55d7f5c7c-kjrg9                  1/1       Running   0          6m
  mariadb-0                                 1/1       Running   0          6m
  memcached-5b858fb696-4fmf6                2/2       Running   0          6m
  neutron-dhcp-agent-4xj76                  1/1       Running   0          5m
  neutron-l3-agent-network-9j978            1/1       Running   0          5m
  neutron-metadata-agent-network-nlpvd      1/1       Running   0          5m
  neutron-openvswitch-agent-network-8cc2x   1/1       Running   0          5m
  neutron-server-68d97c559f-xwwjl           3/3       Running   0          5m
  nova-api-69876b658f-pmf4f                 3/3       Running   0          2m
  nova-api-create-cell-sm6bj                1/1       Running   0          2m
  nova-compute-s9rbl                        1/1       Running   0          2m
  nova-conductor-0                          1/1       Running   0          2m
  nova-consoleauth-0                        1/1       Running   0          2m
  nova-libvirt-zjw6h                        1/1       Running   0          2m
  nova-novncproxy-58fb468d4b-4l57m          3/3       Running   0          2m
  nova-scheduler-0                          1/1       Running   0          2m
  openvswitch-ovsdb-network-j4gbm           1/1       Running   0          6m
  openvswitch-vswitchd-network-6q8lw        1/1       Running   0          6m
  placement-api-697b85cf9-6twdf             1/1       Running   0          2m
  rabbitmq-0                                1/1       Running   0          6m
  tgtd-wblfn                                1/1       Running   0          5m
  (37/45) Kolla - Create a keystone admin account and source in to it
  (38/45) Kolla - Allow Ingress by changing neutron rules
  (39/45) Kolla - Configure Neutron, pull images
  Enter passphrase (empty for no passphrase):
  Enter same passphrase again:
  (40/45) Kolla - Create a demo VM in our OpenStack cluster
  To create a demo image VM do:
  .  ~/keystonerc_admin; openstack server create --image cirros --flavor m1.tiny --key-name mykey --nic net-id=c7df4092-b88b-4596-bbad-6c1a2888ee82 test
Kubernetes - Wait for VM demo1 to be in running state:
  *Kubernetes - VM demo1 is not Running yet - wait 15s*
  *Kubernetes - VM demo1 is not Running yet - wait 15s*
  *Kubernetes - VM demo1 is not Running yet - wait 15s*
  *Kubernetes - VM demo1 is Running*
  (41/45) Kolla - Create floating ip
  (42/45) Kolla - nova list to see floating IP and demo VM
  +--------------------------------------+-------+--------+------------+-------------+-------------------+
  | ID                                   | Name  | Status | Task State | Power State | Networks          |
  +--------------------------------------+-------+--------+------------+-------------+-------------------+
  | 1bd09c59-85a0-4d8f-9fc2-2949ca01192d | demo1 | ACTIVE | -          | Running     | public1=10.0.0.60 |
  +--------------------------------------+-------+--------+------------+-------------+-------------------+
  (43/45) Kolla - To Access Horizon:
  Point your browser to: 10.3.3.189
OS_PASSWORD=oUfo1H4hSLxkJJbEmtieN7UN2sqwZfRGpaR8U6lW
OS_USERNAME=admin


**************************************************************************
Successfully deployed Kolla-Kubernetes. OpenStack Cluster is ready for use
**************************************************************************