This patch moves k8s-coreos specific templates and
template_definition class to the new drivers folder.
It also deletes the /magnum/templates folder
as everything has been moved to the drivers directory.
Change-Id: I6b2ca49e4d7d5fcfb96d0abc373d6476fd907358
Paritially-Implements: blueprint bay-drivers
This patch adds an environment file and a couple of template resources
to allow the LBaaS resources to be conditionally enabled/disabled.
Change-Id: I40ef0839dca84f398efb02022fa7c1de821fb1a3
Partially-Implements: blueprint decouple-lbaas
Partially-Implements: blueprint bay-with-no-floating-ips
Moves templates and template_definitions to the new
directory structure.
Change-Id: I42e4d2bd056f3d8082ef51ef599d917f2fe82960
Paritially-Implements: blueprint bay-drivers
Moved all the swarm templates and template_definition code
to the magnum/drivers folder.
Moved base template_definition classes to drivers/common
folder
Change-Id: Ieff57f0f47835c35d9f17c3d7d1b7e6a40907462
Partially-Implements: blueprint bay-drivers
Co-Authored-by: Spyros Trigazis <strigazi@gmail.com>
Currently when we create a Bay, magnum is not enough to prevent bay creation failure,
due to Invalid discovery url can be referenced and there're no check toward it. We
should check the discovery url before creating bay, a 400 Invalid will be raised if
the discovery url is invalid:
1.the discovery url should be a correct url of etcd cluster.
2.the discovery url should have suitable cluster size.
Change-Id: Ib75f9f2aade18b16dd46035efe139311faf93e1e
Add a parameter 'slaves_to_remove' that lists resources to be
removed on stack update. The value will be passed to
removal_policies of the 'mesos_slaves' resource group.
Change-Id: I18fcd0be962dd7414329dcfcad8bb4c79958ee9a
Partially-Implements: blueprint mesos-smart-bay-scale-down
1.when self.get_output_value() return None, K8sApiAddressOutputMapping's
set_output() and SwarmApiAddressOutputMapping's set_output() should
checks whether output_value is None at first.
2.'output_value = "%(protocol)s://%(address)s:%(port)s" % params'
certainly not is None, So 'if output_value is not None' is useless.
Change-Id: I637ee0793e966ef46bfee6e51fbda2a15032979e
To setup magnum easier, name based configuration is important.
This patch makes config file support trustee_domain_name and
trustee_domain_admin_name in trust section.
If name and id of trustee domain are provided by user, both
values are passed through into keystone.
Closes-Bug: #1581372
Change-Id: Ia691aca7c29a471f6ba36a1a371ec1edf830b365
* This parameter will be optional and users will be able to select
a supported driver, otherwise the default configuration will be
used.
* Add docker storage driver enum field to baymodel
* Add db upgrade file
* Update heat templates for kubernetes and swarm allowing only
devicemapper and overlay as docker_storage_driver values.
* Add configuration for OverlayFS on Fedora Atomic, if overlay is
incompatible bay creation will result a CREATE_FAILED status.
* Factor out configuration of docker storage drivers
* Update tests
* Add Release Notes
Partially-Implements: blueprint support-for-different-docker-storage-driver
Change-Id: Ib58cb734c4e9c90d5d83574852213d2e97359e92
This path adds supporting to using insecure registry for k8s COE when
deploy a k8s COE.
Partially-Implements: blueprint support-insecure-registry
Change-Id: I384358b1419085ed42d96239f97292fa32db9aed
mesos_slave_executor_env_variables is mesos executor
environment variables, and it should be a json form.
(1) validate mesos_slave_executor_env_variables
if matching the json form.
(2) modify the mesos slave configure file
(3) unit test
Partially-Implements:blueprint mesos-slave-flags
Change-Id: I35e56515f321b0df52b3ba0e8aba48d7dd421d58
* Add cloud-init to generate TLS certificates in each node.
* Modify coreos fragments to point to the path of certificates.
* Add support for "--tls-disabled" to turn off TLS.
* Use Keystone trust to retrieve TLS certificates.
Implements: blueprint tls-for-coreos-bay
Change-Id: I66842f9374abe5a9bbf275fa130c0eae3e6065ab
Add docker registry support for swarm in heat template. After this
patch is merged, we can use docker registry in swarm bay.
Change-Id: I5161a4c0259a2df89dfc8591453aebc6f037d40d
Partially-Implements: blueprint registryv2-in-master
The Mesos slave has many different flags and different user have
different requirement for different parameters, add isolation,
work_dir, image_providers parameters for slave in mesos bay to
give end user more choice. for example Set the parameters to
support Container Images in Mesos Containerizer
Note: support Container Images in Mesos Containerizer needs mesos
version >= 0.28
Partially-Implements:blueprint mesos-slave-flags
Change-Id: Ib4fd7076704a3266f0cf2addf08896729ec6062f
After this patch is merged, docker registry will be available.
We can push docker images into local repo on a bay node.
Change-Id: I1245bde7ef3173226617e60e6436759f433eb464
Partially-Implements: blueprint registryv2-in-master
Currently, we use the users auth token, which expires after a while.
We need to use a trust instead.
Remove user_token at the same time.
Change-Id: Id1d34c59eccd70be24c5b9e00cd921b5a9d59860
Partially-Implements: blueprint use-trust-for-tls-cert-generation
Previously,the kubernetes bay type did not support the Magnum Container
Volume Model. This patch adds support for volume through the following:
1. Add volume_driver, region_name to
kubernetes bay type.
2. Update kubernetes unit tests to support container volume.
3. Update kubernetes heat templates for container volume model.
Note: The Container Volume Model of Kubernetes need the
kubernetes Version >= 1.1.1 and docker version >= 1.8.3
Change-Id: I8f568087f8e8254cb14a81edb526e596da33abcc
Partially-Implements: blueprint magnum-integrate-with-cinder
Allow configuring Flannel with 3 different backends
Magnum deploys k8s/swarm over a dedicated neutron private network,
possibly using flannel. Flannel's `host-gw` backend gives the best
performance in this topopolgy (private layer2): no packet processing
overhead, no reduction to MTU, scales to many hosts as well as the
alternatives. The performance difference is significant, see bug for
performance numbers for the 3 backend options.
Note that part of this change involves relaxing the minion IP spoofing
rules to allow traffic from all dynamically-allocated flannel subnets.
This is morally equivalent to what we were doing previously with
encapsulation - only now neutron is able to see the inner IP header
directly.
This patch repurposes the label "flannel_use_vxlan" when the network
driver is flannel.
1. Rename the label flannel_use_vxlan to flannel_backend
2. Redefine the value of this label from "yes/no"
to "udp/vxlan/host-gw"
For example, to create a bay model with flannel as network driver:
--network-driver flannel --labels flannel_backend=host-gw
Other backend options are udp and vxlan.
Co-Authored-By: Ton Ngo <ton@us.ibm.com>
Partial-Bug: #1518605
Closes-Bug: #1516789
Change-Id: I6d2441664ad1baaca14d0e6ff4bcddbe75bee094
Url for keystone is needed by trust and other services, such as k8s
and docker registry.
Change-Id: I269332b5736b6c5a9bc85d843f0d03f1a4d059ee
Partially-Implements: blueprint create-trustee-user-for-each-bay
Previously,the mesos bay type did not support the Magnum Container
Volume Model. This patch adds support for volume through the following:
1. Add username, tenant_name, preempt, region_name, domain_name to
mesos bay type.
2. Add get_parameter to mesos.
3. Update mesos unit tests to support container volume.
4. Update mesos heat templates for container volume model
Change-Id: Icf7cb9fc38c7facb2d49904b6e52bbce974948f7
Partially-Implements: blueprint magnum-integrate-with-cinder
Add trust info into heat params. If any service wants to use the
trust, it can get the info from the heat params.
Change-Id: I406a80d0d4fc92c6045b902945198a0d765f1123
Partially-Implements: blueprint create-trustee-user-for-each-bay
Originally BayModel#fixed_network was used for setting which
fixed_network was set to Bay [1].
But this attribute changed to setting fixed_network_cidr [2].
This patch removes mapping for fixed_network_cidr.
And also, fixed_network_cidr attribute will be add to
baymodel [3].
Magnum user want to specify fixed_network for bay, so I'll add
a mapping to template definition to set fixed_network
next patch.
[1]: bc3bc6190d/magnum/conductor/handlers/bay_k8s_heat.py (L101)
[2]: I624b95930f9eb506abfb2e29beea8d83878ea142
[3]: Ia74ed924b3058b4891775f34f1b9624774c02ddb
Change-Id: Ia9820213bcc0e2a451450dd014e774843b881fd1
Closes-Bug: #1536484
The CoreOS template has been fixed [1]. This patch is for updating
the CoreOS template definition.
[1] https://review.openstack.org/#/c/262628/
Partial-Implements: blueprint coreos-k8s-bay
Change-Id: I19660bfa702f8349090fd66d098bd037f2316f0d
If discovery endpoint is not accessible, magnum returns "ERROR: Timed
out waiting for a reply to message ID a74fe7ec63b34f298c46c9f6659257d7
(HTTP 500)" when we create a bay. The response is misleading.
So we should raise exception when magnum fails to get discovery_url
and make the response easy to understand.
Change-Id: Ia649e55de0878160f42fecf1647d9da7fc954d72
Closes-Bug: #1529200
Swarm agent has been moved out of master node in the depending patch,
so it is appropriate to separate the flavor now
Co-Authored-By: Hongbin Lu <hongbin.lu@huawei.com>
Closes-Bug: #1498570
Change-Id: I6f5f15274a10f503a2554ec554e5c822f47a39dc
Object "stack" returned by Heat doesn't necessary have the "outputs"
attribute. This could happen when a stack is in failure state. Magnum
needs to handle this case robustly.
Closes-Bug: #1525678
Change-Id: I288af63bd9e2704f9869f3eaf8d8d4c6495a7973
number_of_masters, number_of_minions and number_of_nodes
should be number, not string.
Change-Id: Iac20023f0448aab313302c174e3743b95cd42d7a
Closes-Bug: #1524236
To create a HA mesos bay, users need to specify at least 3 master
nodes. For example:
magnum bay-create --name mesosbay --baymodel mesosbaymodel \
--master-count 3
Change-Id: I85c2113fec0fd743cc3b142e6a490bdfb3b896e6
Implements: blueprint mesos-multi-master-node
Move get_discovery_url to BaseTemplateDefinition so swarm and k8s template
can share it.
Behavior changes
Change swarm's discovery member number from 1 to bay.master_count
Partially implements: blueprint swarm-high-availability
Change-Id: I2617bc81daf68b45148987190a2f04820ad18bfd
Would like to make swarm much more similar with k8s.
This patch is the preparation of supporting Swarm HA mode:
1. Refactor swarm.yaml to swarmcluster.yaml and swarmmaster.yaml
2. Add api_pool, LB in front of swarm master node.
3. Add etcd_pool
After this change:
Swarm bay will update the fileds of 'master_addresses'.
P.S. notes Swarm HA is not supported yet, master_addresses will be only 1
ip address.
Partially implements: blueprint swarm-high-availability
Change-Id: Ib6346bfd5a7ad0ef2226a6e6bc98b0ad46e577cb
Move Mesos master into its own nested template. This will make the
style consistent with k8s templates.
Change-Id: Icfb0df1b486febacc08f34ca86a58a80dc83f812
Partial-Implements: blueprint heat-network-refactor
Previously, Swarm leveraged Docker's public discovery mechanism
for bootstrapping a cluster. Etcd bootstrapping is supported by
Swarm and is preferred for production use for the following reasons:
1. Required for HA.
2. Is more secure.
3. Required for the Flannel network-driver.
Partially-Implements: blueprint extend-baymodel-net-attributes
Partially-Implements: blueprint conductor-template-net-update
Change-Id: Iab844c03ed7cf8bbee69b72ff71c219f0a5ab1dd
Previously, the swarm bay type did not support using a cinder
volume for docker.
Implements: blueprint swarm-cinder
Change-Id: I8acaf60caab52a5e5a234e4c5f89ce3dd155759c
Rename heat-kubernetes to kubernetes, heat-mesos to mesos,
docker-swarm to swarm in templates. We use heat templates and
no other methods, so I think it is unnecessary to add heat before
coe. kubernetes, mesos, swarm are better than
heat-kubernetes, heat-mesos, docker-swarm.
Change-Id: I257b35c1c4ef55d3172095736f550f2c55c8d81f
Closes-Bug: #1514682
api_address is a API address instead of master node's IP address.
This patch fix it by adding a new OutputMapping, it will help to handle
the mapping from master address to api_address.
Also test cases added.
Closes-Bug: #1514311
Change-Id: I47baf92dd466150adbd75746d0cb804d108f7d5f
After this commit, all Heat templates have five common outputs:
* api_address: API endpoint of specific COE.
* kube_masters/swarm_master/mesos_master: Public IP address(es) of
master node(s).
* kube_masters_private/swarm_master_private/mesos_master_private:
Private IP address(es) of master node(s).
* kube_minions/swarm_nodes/mesos_slaves: List of public IP addresses
of worker nodes.
* kube_minions_private/swarm_nodes_private/mesos_slaves_private:
List of private IP addresses of worker nodes.
Change-Id: Ie44136dafa326db598a5f17978d89adce8e69801
Closes-Bug: #1514252
murali allada