With this change each node will be labeled with the following:
* --node-labels=magnum.openstack.org/role=${NODEGROUP_ROLE}
* --node-labels=magnum.openstack.org/nodegroup=${NODEGROUP_NAME}
Change-Id: Ic410a059b19a1252cdf6eed786964c5c7b03d01c
Removes the role heat param from all templates. Instead and only for
k8s templates adds the master_role and worker_role params. The new
worker_only condition should be true for all roles except for master.
Finally, adds the missing is_cluster_stack param to all templates.
Change-Id: Ie0799373fe492c2e0a0cad903ed6e8c93e6266b5
* Fedora CoreOS need the key to be passed as
a string.
* We can adopt in all drivers so that users in
the same project can do cluster resize.
story: 2005201
task: 36934
Change-Id: I9a18ce4dcbd74f0dcd23274baed7c8c3d2029d50
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
We currently have a lot of duplicate functions across our drivers
which use Kubernetes. This takes them and brings them into a
common class called KubernetesDriver and cleans up the subclasses.
Change-Id: I6f880cb03ed43ec3bc9d3d9e5a7b87eaceda40e9
To enable the rolling upgrade ability of Kubernetes Cluster, this
patch is proposing a new API /upgrade to support upgrade the
base operating system of nodes and the version of Kubernetes, even
add-ons running on the k8s cluster:
POST <ClusterID>/actions/upgrade
And the post body will be:
{
"cluster_template": 'dd9cc5ed-3a2b-11e9-9233-fa163e46bcc2',
"max_batch_size": 1,
"nodegroup": "production_group"
}
Co-Authored-By: Feilong Wang <flwang@catalyst.net.nz>
Task: 30168
Story: 2002210
Change-Id: Ia168877778aa0d473383eb06b1c8a16dc06b0576
Using comma delimited ipv4 address list to specify multi dns server
"8.8.8.8,114.114.114.114".
Task: 29465
Story: 2004994
Change-Id: I031247b0cc2ae417f18b2a5b9b3832e78ed9dafd
- Never allocate floating IP for etcd service.
- Introduce a new label `master_lb_floating_ip_enabled` which controls
if Magnum allocates floating IP for the master load balancer. This
label only takes effect when the `master_lb_enabled` is set. The
default value is the same with `floating_ip_enabled`.
- The `floating_ip_enabled` property now only controls if Magnum
should allocate the floating IPs for the master and worker nodes.
Change-Id: I0a232406deaf112b0cb9e445735d7b49206c676d
Story: #2005153
Task: #29868
Return the nova instance UUID of worker nodes in kubeminion
templates. We will be able to remove resources from the
ResourceGroups based on nova instance uuid.
Backstory:
In heat a ResourceGroup creates a stack of depth 2. ResourceGroups
support removal policies to declare which resources must be removed.
This can be done by passing the index of the resource or the stack_id
of the nested stack. If a stack update call receives a list of
indices (eg [0, 5, 3]) or nested stack uuid (eg [uuidA, uuidB]), it
will remove the corresponding nested stacks.
In magnum's heat templates, a nested stack logically represents a
nova compute instance which is a cluster node. Using composition in
heat, we can change the way a resources group references the nested
stacks. This proposes to use the nova instance uuid as
'OS::stack_id'.
With this change, an external consumer of the stack (the cluster
autoscaler or an actual user) can remove resources from the
ResourceGroup using the nova instance uuid or resource index. Without
this change, a user or system (which typically knows the name,
server uuid or ip) would have to find in which nested stack a
kubernetes node belongs too. Resulting multiple call to heat.
The end result of this patch can be verified like this:
nested_stack_id=$(openstack stack resource show <STACK_ID_OR_NAME> kube_minions -c physical_resource_id -f value)
openstack stack show "${nested_stack_id}"
Task: 29664
Story: 2005054
Change-Id: I6d776f62d640c72b3228460392b92df94fe56fe6
Now Magnums onlys has one server group for all master and worker nodes
per cluster, which is not very flexible for small cloud scale. For a
3+ master clusters, it's easily meeting the capacity when using hard
anti-affinity policy. This patch is proposing one server group for each
master and worker nodes group to have better flexibility.
story: 2004195
Change-Id: If11ba863a2aa538efe1e3e850084bdd33afd27d2
There are 2 changes included in this patch:
1. Using cluster ip instead of fixed ip for grafana service to
make sure the address is reachable.
2. Move node exporter to prometheus-monitoring namespace and
make it as a DaemonSet to collect metrics from master node.
Task: 28468
Story: 2004590
Change-Id: I9090c6dc4b38e1a1466c4c3a6a827d95c089fb41
A user may not rely on nova-keypairs to access their cluster
such as a preconfigured SSSD.
story: 2004402
task: 28035
Change-Id: I77fbdc174d3dddfd312fb8dac20516314d4c182e
Scripts are the core of Magnum for COE deployment. To be more
clear and consistent, two changes proposed in this patch:
1. Rename network related script to xxx-flannel-xxx given they
are all for flannel and now we have calico driver.
2. Adding .sh for some scripts to be consistent with others.
Change-Id: I97f3e53b4b43648a4896193fb4ce469dbf42c611
In the OpenStack deployment with Octavia service enabled, the octavia
service should be used not only for master nodes high availability, but
also for k8s LoadBalancer type service implementation as well.
Change-Id: Ib61f59507510253794a4780a91e49aa6682c8039
Closes-Bug: #1770133
Due to bug #1746510, the kubernetes scale manager needs to be
disabled to not break the scale down command completely. Note, that when
magnum scales down the cluster will pick the nodes to scale randomly.
Related-Bug: #1746510
Change-Id: I8c3505ec6d155323288217e0c8ea54adabdff1c3
Currently, the default k8s version in Magnum is v1.7.4, but based on the
deprecation policy of k8s. It will be deprecated at March 2018, see
https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
So it would be nice to change the default k8s version to latest.
Closes-Bug: #1750549
Change-Id: I053e50ac879b031c8438a2587a99de44e0360c47
Due to a few several small connected patches for the
fedora atomic driver, this patch includes 4 smaller patches.
Patch 1:
k8s: Do not start kubelet and kube-proxy on master
Patch [1], misses the removal of kubelet and kube-proxy from
enable-services-master.sh and therefore they are started if they
exist in the image or the script will fail.
https://review.openstack.org/#/c/533593/
Closes-Bug: #1726482
Patch 2:
k8s: Set require-kubeconfig when needed
From kubernetes 1.8 [1] --require-kubeconfig is deprecated and
in kubernetes 1.9 it is removed.
Add --require-kubeconfig only for k8s <= 1.8.
[1] https://github.com/kubernetes/kubernetes/issues/36745
Closes-Bug: #1718926https://review.openstack.org/#/c/534309/
Patch 3:
k8s_fedora: Add RBAC configuration
* Make certificates and kubeconfigs compatible
with NodeAuthorizer [1].
* Add CoreDNS roles and rolebindings.
* Create the system:kube-apiserver-to-kubelet ClusterRole.
* Bind the system:kube-apiserver-to-kubelet ClusterRole to
the kubernetes user.
* remove creation of kube-system namespaces, it is created
by default
* update client cert generation in the conductor with
kubernetes' requirements
* Add --insecure-bind-address=127.0.0.1 to work on
multi-master too. The controller manager on each
node needs to contact the apiserver (on the same node)
on 127.0.0.1:8080
[1] https://kubernetes.io/docs/admin/authorization/node/
Closes-Bug: #1742420
Depends-On: If43c3d0a0d83c42ff1fceffe4bcc333b31dbdaab
https://review.openstack.org/#/c/527103/
Patch 4:
k8s_fedora: Update coredns config to pass e2e
To pass the e2e conformance tests, coredns needs to
be configured with POD-MODE verified. Otherwise, pods
won't be resolvable [1].
[1] https://github.com/coredns/coredns/tree/master/plugin/kuberneteshttps://review.openstack.org/#/c/528566/
Closes-Bug: #1738633
Change-Id: Ibd5245ca0f5a11e1d67a2514cebb2ffe8aa5e7de
Currently, there is no guarantee to make sure all nodes of one cluster are
created on different compute hosts. So it would be nice if we can create
a server group and set it with anti-affinity policy to get a better HA
for cluster. This patch is proposing to create a server group for master
and minion nodes with soft-anti-affinity policy by default.
Closes-Bug: #1737802
Change-Id: Icc7a73ef55296a58bf00719ca4d1cdcc304fab86
In the drivers section of magnum.conf add openstack_ca_file.
This file is expected to be a CA Certificate OR CA bundle
which will be passed on every node and it will be installed
on the host's CA bundle.
Update devstack plugin to use the ssl bundle if tls-proxy is
enabled.
Install the CA for drivers:
k8s_coreos_v1
k8s_fedora_atomic_v1
k8s_fedora_ironic_v1
mesos_ubuntu_v1
swarm_fedora_atomic_v1
swarm_fedora_atomic_v2
Add doc in troubleshooting-guide.
Add release notes.
Closes-Bug: #1580704
Partially-Implements: blueprint heat-agent
Change-Id: Id48fbea187da667a5e7334694c3ec17c8e2504db
Use the heat-container-agent from a system container.
It means that the docker daemon can be started later.
Pass as a software deployment with the heat-agent the following
software-configurations:
* prometheus-monitoring
** pin prometheus to v1.8.2 since its config is not 2.0.0
compatible
Add heat-container-agent container image.
Implements: blueprint heat-agent
Related-Bug: #1680900
Change-Id: I084b7fe51eddb7b36c74f9fe76cda37e8b48f646
Allow any value to be passed on the docker_storage_driver field by turning it
into a StringField (was EnumField), and remove the constraints limiting the
values to 'devicemapper' and 'overlay'.
Change the docker storage setup to have a generic setup for all drivers with
the exception of 'devicemapper', which keeps its own specific storage config
function. For all others, do the same we already did for overlay (with two
cases for usage of a cinder volume or not) and simply set the storage driver
in the docker configuration to the value provided in the cluster template.
Change-Id: I9aa8f232ce64ece4d439c0a476f463820a499617
Closes-Bug: #1722522
Added configuration parameter, verify_ca, to magnum.conf with default
value of True. This parameter is passed to the heat templates to
indicate whether the cluster nodes validate the Certificate Authority
when making requests to the OpenStack APIs (Keystone, Magnum, Heat).
This configuration parameter can be set to False to disable CA
validation.
Co-Authored-By: Vijendar Komalla <vijendar.komalla@rackspace.com>
Change-Id: Iab02cb1338b811dac0c147378dbd0e63c83f0413
Partial-Bug: #1663757
Since 1.6 --apiservers is deprecated and it is removed in
1.8. Add the server parameter in kubeconfig and remove
--apiservers.
Change-Id: Ie766ec0797fdc86a93e7f70a321d39332a73b552
Closes-Bug: #1718926
Add a label to prefix all container image use by magnum:
* kubernetes components
* coredns
* node-exporter
* kubernetes-dashboard
Using this label all containers will be pulled from the specified
registry and group in the registry.
TODO:
* grafana
* prometheus
Closes-Bug: #1712810
Change-Id: Iefe02f5ebc97787ee80431e0f16f73ae8444bdc0
1. It will fail to create cluster if there is chinese in tenant name
2. TENANT_NAME is unnecessary after changing to trustee
this patch is for k8s_fedora_atomic and k8s_fedora_ironic
Change-Id: Ie072f183110ae95861fb3694a913a3a4526549fb
Close-Bug: #1711308
Separate the tag from which to pull from the kubernetes version.
With the current state the tag and the version happen to be the
the same. But, it is not decided yet in the fedoraproject how the
images are going to be tag. Finally, operators might want to try
their own container images with custom tags.
Depends-On: Icddb8ed1598f2ba1f782622f86fb6083953c3b3f
Implements: blueprint run-kube-as-container
Change-Id: I4c4bc055d7df5e65aede93464bff51e6d5971504
Use system containers based on fedora rawhide from
projectatomic [1]. Until the fedoraproject updated
the tags properly we mirror our containers in [2].
System containers are meant to be drop in replacements
of the fedora kubernetes binaries.
Update k8s to 1.7.4 to match the version in the containers.
[1] https://github.com/projectatomic/atomic-system-containers
[2] https://hub.docker.com/r/openstackmagnum/
Implements: blueprint run-kube-as-container
Change-Id: I22918c0b06ca34d96ee68ac43fabcd5c0b281950
The instance type of servers at the moment can become quite long
due to the Heat autogenerated names. This patch cleans up the names
so that they are shorter yet contain all the info needed to be able
to know where they belong to.
Change-Id: I5bcbe73f08844242d049b8408221da40d22cd3dc
Previously the master's private IP address was not pushed through to the
minion configuration when the load balancer is disabled as the heat
templates were not wired up in this case. This change resolves that
issue and makes it possible for security groups to be applied to the
master and minion ports.
Change-Id: If85a5434f014c5a09b54dda710d13739e9bff928
Related-Bug: #1544195
Allow setting the size of a volume for etcd storage.
Default is 0 which matches the current behavior - no persistency.
Related-Bug: #1697655
Change-Id: I8a30df63684133a902ae209ba6c124da2a567d3f
* add docker_volume_type for the cinder volumes which are
used for docker storage.
Related-Bug: #1678153
Change-Id: I55418a667cc8af043c61130aa61138d700fdc4ca
Same fix as CoreOS for Fedora which enable multimaster with
TLS and ETCD Load balancer.
Closes-Bug: #1679724
Change-Id: I45b62a20f0a89ebd1494ad61021384fc7a416e8e
diskimage-builder/bin/disk-image-create no longer exists.
So update steps for creating disk images.
Change-Id: Ieae16ac32641f7525feb6caa0c55caa29615df9e
Closes-Bug: #1675285
kube-ui [2] is deprecated and not actively maintained since long time.
Instead kubernetes dashboard [1] has lot of features and is actively
managed.
With this patch kube-ui is removed and kubernetes dashboard is added
and enabled in k8s cluster by default.
The kubernetes dashboard is enabled by default. To disable it, set the
label 'kube_dashboard_enabled' to False
Reference:
[1] https://github.com/kubernetes/dashboard
[2] https://github.com/kubernetes/kube-ui
Change-Id: I8864c097a3da6a602e0f25d3ff8ade788aa134a9
Implements: blueprint add-kube-dashboard
In a default nova deployment user-data for software deployments
goes into nova's db. That field is 64KB, so we are contraint by
space and we need to pass only what is needed.
Change-Id: I84bb59dbedc6e97b3bb985db5621daf34b6e17ee
Closes-Bug: #1680900
Closes-Bug: #1650538
New release of Fedora Atomic [1].
The new release of Fedora Ironic includes the same
packages.
Main changes:
Kubernetes 1.5.3
etcd 3.1.3
Plus several fixes and version bumps.
Add :Z when mounting certs in the swarm containers to set
selinux labels properly.
[1] http://www.projectatomic.io/blog/2017/03/fedora_atomic_mar28/
Closes-Bug: #1677664
Change-Id: I2539ae83401db5b34716ebd4bbdfbe288f5c768b
Profit from the default cAdvisor deployed by k8s to deploy the
remaining monitoring stack on top, made of node-exporter,
Prometheus and Grafana.
Node-exporter is ran as a normal pod through a manifest, while
Prometheus and Grafana are deployments with 1 replica.
Prometheus has compliance with Kubernetes, so the discovery of
the nodes and other k8s components is configured directly in
Prometheus configuration.
Change-Id: If2cab996b9458580a55b5212ab298c909622e7f3
Partially-Implements: blueprint container-monitoring
If nothing is specified a set of recommended default plugins is used,
which includes the ServiceAccount one.
Change-Id: I1383aae09ba68f8e83b07e3eaae40ab071f7be94
Closes-Bug: #1646489
Make Kubernetes' kube-controller-manager and kube-scheduler
health checks configurable as a parameter to the cluster-template
(label).
Set their value higher for all deployments. And set their value
to a high number for tests, for the CI.
Change-Id: I65e2da12487c513419125f0525a4e21bac22210e
Closes-Bug: 1648826
Michal Nasiadka
yatin