Commit Graph

249 Commits

Author SHA1 Message Date
Jake Yip 8a30ad3462 Add feature to specify driver explicitly
Allow ClusterTemplate to explicitly specify a driver to use for creating
Clusters.

This is initially sourced from the image property 'magnum_driver', but
may be improved to be specified via client in the future.

Falls back to old driver discovery using (coe, server_type, os) tuple to
keep existing behaviour.

Change-Id: I9e206b589951a02360d3cef0282a9538236ef53b
2024-02-26 14:50:18 +11:00
Michal Nasiadka 68c8acba39 Remove execution bit on unnecessary files
Change-Id: Ia41b843fdf20154750b129a8ab5dd42f5c3989fb
2024-02-19 00:30:21 +00:00
Michal Nasiadka bc79012f46 Drop Swarm support
Label validator function has been left behind, although it's not
checking for anything right now - might be useful in future.

Change-Id: I74c744dc957d73aef7556aff00837611dadbada7
2024-01-24 13:20:21 +13:00
guilhermesteinmuller 6e8657912c Drop Mesos code
The coe mesos has not been maintained for quite some
time and hasn't got much attention from the community
in general. As discussed in the mailing list [1] we
are dropping for now.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026230.html

Co-Authored-By: jake.yip@ardc.edu.au

Change-Id: Ie7de9a37bfad6214f52d0605409e74d62a51df47
2023-03-07 14:02:05 +11:00
Jake Yip cb40fb3685 Add back pep8 test
In Change I523a4a85867f82d234ba1f3e6fad8b8cd2291182, the pep8 test was
accidentally dropped.

Fix up code so that pep8 passes.

In addition to that following change has been added here to unbreak CI:

Add WebTest as an indirect test dependency

Pecan has made webtest an optional dependency for testing only [1].
Since it is still used for testing we need to add it to our
test-requirements.txt.

[1]: https://github.com/pecan/pecan/pull/140

Change-Id: I2f85adb4ef29a43389897c201e6152fd4c7be9d6
2022-07-19 12:35:14 +00:00
Stavros Moiras 0792885a1b Optimize cluster list api
Up till now, cluster api controller cluster_template_id was a
property field loading the id from the DB every time. With this
change the field becomes of text type and mandatory, so wsme fwk
guarantees that the field is provided when needed.

Cluster objects will not load the cluster template on creation.
Instead cluster templates will be loaded when they are actually
needed.

story: 2006693
task: 36989

Co-Authored-By: Stavros Moiras <stavros.moiras@cern.ch>

Change-Id: I2313c6a8b647e521cfa476f9cec65ab286fa5a23
2021-07-01 07:42:08 +00:00
Feilong Wang 16344a5a95 Add separated CA cert for etcd and front-proxy
Support creating different for k8s, etcd and front-proxy for
security hardening. We're following some best practices[1][2] but
adjusted based on the current Magnum deployment approach.

[1] https://kubernetes.io/docs/setup/best-practices/certificates/
[2] https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/

Task: 40687
Story: 2008031

Change-Id: I523a4a85867f82d234ba1f3e6fad8b8cd2291182
2021-04-01 17:31:34 +00:00
Diogo Guerra 332e2b6fe4 Add CT tags field to the database and API
We noticed that from the user perspective it is hard
to know when a cluster_template provided by the cloud
admin is mature enough for a production release.
This field will allow the administrator to add an
annotation to the cluster template like
{deprecated, recommended, testing} giving further
usefull information to the end user about the
template's life cycle

This patch adds the necessary database column and
API objects to handle the new argument.

story: 2007857
task: 40160

Change-Id: I5d1c4221f089bc5cd12b25f620aa01771a029df9
Signed-off-by: Diogo Guerra <diogo.filipe.tomas.guerra@cern.ch>
2021-02-05 22:45:47 +00:00
Theodoros Tsioutsias f46923cc5e Allow nodegroups with node_count equal to 0
This change allows users to create clusters and nodegroups with
node_count equal to 0. Also adds support for resizing existing
nodegroups to 0.

Change-Id: Id63459d0fe9836e678bb7569f23d29eabc225e9e
story: 2007851
task: 40145
Signed-off-by: Diogo Guerra <diogo.filipe.tomas.guerra@cern.ch>
2021-02-04 13:07:18 +00:00
Feilong Wang 946c1d67c7 Add master_lb_enabled to cluster
Adding the master_lb_enabled option when creating a cluster,
which will benefit both the cloud provider side and the end
user side. For cloud prodiver, they don't have to maintain
separate cluster templates with or w/o master_lb_enabled enabled.
For end user, they can easily use one single template to create
different clusters with different configs.

Task: 39680
Story: 2007634

Change-Id: I0b586f05168ece84fd340ef7493a56688191053d
2020-07-21 11:07:33 +12:00
Feilong Wang d224999cb0 Improve log of k8s health status check
When the k8s cluster is created in a private network without
FIP and without a LB FIP, the cluster is not reachable by Magnum
control plane. For this case, the cluster health status is unknown
for Magnum and Magnum should not connect the cluster to avoid
unnecessary logs.

Task: 36187
Story: 2006383

Change-Id: Ibb589ec51f91f05a334c907560ef5fe57cbfbffb
2019-10-01 18:29:16 +00:00
Theodoros Tsioutsias cbe05aa97d ng-6: Add new fields to nodegroup objects
Since each nodegroup will be one independent stack, we have to add
more fields to the table and object in order to track each stack
contained in the cluster. This adds the stack_id, version, status,
status_reason and version fields to the nodegroup object.

Change-Id: I6d36b2d3bc6476efbef6a9f702ffc73cfa0fab8c
2019-09-25 12:26:00 +00:00
Feilong Wang 32989b4f7b Allow setting network, subnet and FIP when creating cluster
When using a public cluster template, user still need the capability
to reuse their existing network/subnet, and they also need to be
able to turn of/off the floatingip to overwrite the setting in the
public template. This patch supports that by adding those three
items as parameters when creating cluster.

Story: 2006208
Task: 35797

Change-Id: I11579ff6b83d133c71c2cbf49ee4b20996dfb918
2019-07-31 20:41:20 +12:00
Theodoros Tsioutsias 3f80cbab06 ng-4: Adapt cluster object
This commit removes the fields node_addresses, master_addresses,
node_count and master_count from the cluster object since this info
will be stored in the nodegroups. At the same time, provides the way
to adapt existing clusters to the new schema.

story: 2005266

Change-Id: Iaf2cef3cc50b956c9b6d7bae13dbb716ae54eaf7
2019-03-29 10:31:48 +00:00
Theodoros Tsioutsias 18c77a288d ng-2: Adapt existing cluster APIs and conductor
This changes the existing cluster APIs and the cluster conductor to
take into consideration nodegroups:

* create: now creates the default nodegroups for the cluster
* update: updates the default nodegroups of the cluster
* delete: deletes also the nodegroups that belong to the cluster
* cluster_resize: takes into account the nodegroup provided by the API

story: 2005266

Change-Id: I5478c83ca316f8f09625607d5ae9d9f3c02eb65a
2019-03-28 10:31:01 +00:00
Theodoros Tsioutsias 0607c7a9d6 ng-1: Add nodegroup representation
This adds the object and db schema changes needed for supporting
nodegroups.

story: 2005266

Change-Id: Ibf10277a52aa94c4b217cf3b364844b04baab1e0
2019-03-21 16:19:56 +00:00
Ricardo Rocha 6aee864954 Add hidden flag to cluster template
Add a new hidden flag to cluster templates. This allows an operator to
keep a cluster public (accessible to all users) while not showing them
in cluster template listing.

Story: 2004941
Task: 29342

Change-Id: Ia2717ca960041753f6e772bf2d41c7f5a196dae6
2019-02-12 11:38:15 +01:00
Feilong Wang d80febb384 Add health_status and health_status_reason to cluster
APIImpact
Related blueprint cluster-healing

Change-Id: I78d4d14fb064996de77bdd6a381d2df53a9488b8
2018-08-27 14:28:46 +12:00
Clenimar Filemon ec950be894 federation api: api endpoints
this commit introduces a new '/federations'
endpoint to Magnum API, as well as its controllers,
entities and conductor handlers.

this corresponds to the first phase of the
federation-api spec. please refer to [1] for more
details.

[1] https://review.openstack.org/#/c/489609/

Change-Id: I662ac2d6ddec07b50712109541486fd26c5d21de
Partially-Implements: blueprint federation-api
2018-02-09 00:59:31 -03:00
Ricardo Rocha 84006f63d7 Allow flavor_id on cluster create
Add flavor_id as an option during cluster create. If not given,
the default is taken from the cluster template.

Add flavor_id in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1699245
Change-Id: Ib60c05cce1cf2639ca4740abdd264403033433f9
2017-12-15 16:32:40 +00:00
Ricardo Rocha 28fff8006a Make docker_storage_driver a str instead of enum
Allow any value to be passed on the docker_storage_driver field by turning it
into a StringField (was EnumField), and remove the constraints limiting the
values to 'devicemapper' and 'overlay'.

Change the docker storage setup to have a generic setup for all drivers with
the exception of 'devicemapper', which keeps its own specific storage config
function. For all others, do the same we already did for overlay (with two
cases for usage of a cinder volume or not) and simply set the storage driver
in the docker configuration to the value provided in the cluster template.

Change-Id: I9aa8f232ce64ece4d439c0a476f463820a499617
Closes-Bug: #1722522
2017-12-14 14:41:09 +00:00
Ricardo Rocha 446b159349 Allow master_flavor_id on cluster create
Add master_flavor_id as an option during cluster create. If not given,
the default is taken from the cluster template.

Add master_flavor_id in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1699247
Change-Id: Id1d973167b381538121583a0a9691304b39e98de
2017-09-27 09:30:18 +00:00
Ricardo Rocha 3d04ed4cbb Allow labels on cluster create
Add labels as an option during cluster create. If not given,
the default is taken from the cluster template.

Add labels in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1697651
Implements: blueprint flatten-attributes
Change-Id: I8990c78433dcbbca5bc4aa121678b02636346802
2017-08-21 12:27:52 +00:00
Spyros Trigazis b4386f83ad Add swarm-mode driver
* add swarm-mode config
* remove etcd, flannel, discovery_url, swift_registry
* add swarm-mode COE obj
* add functional test, create cluster, create/remove
  service and delete cluster.

Co-Authored-By: ArchiFleKs <kevin.lefevre@osones.io>
Implements: blueprint swarm-mode-support
Change-Id: Iba177be167cb3a3866441d5f42670171f26c5a86
2017-07-18 10:47:23 +02:00
Ricardo Rocha 79039bb419 Allow docker_volume_size on cluster create
Add docker_volume_size as an option during cluster create. If not given,
the default is taken from the cluster template.

Add docker_volume_size in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1697648
Implements: blueprint flatten-attributes
Change-Id: Ic6d77e6fdf5b068fa5319b238f4fd98b4d499be4
2017-06-23 11:31:27 +00:00
jinzhenguo 4377e0d5ab fix the function named get_count_all
To achieve a unified style in the cluster.py,add function annotation
and rename the third parameter

Change-Id: I271910695d32c06c46e9cf01e072fb756a33cd76
2017-06-08 09:02:42 +08:00
fengbeihong 977f3af83f Support dcos installation on centos vm cluster
This patch provides support for installing dcos on centos using magnum.
A new coe type(dcos) is added. This provides support for vm.
Design spec and steps on how to test can be found in
contrib/drivers/dcos_centos_v1/README.md.

Public agent nodes are not supported now.

Co-Authored-By: fengbeihong (fengbeihong@gmail.com)
Co-Authored-By: vmud213 (vinay50muddu@yahoo.com)

Change-Id: I58b378b4bd6df34fd43307e4252cfbbd9bf593d8
Partially-Implements: blueprint mesos-dcos
2017-04-10 12:05:45 +08:00
Vijendar Komalla 206d17f8ca Resource Quota - Limit clusters per project
Currently there is no limit on the number of clusters that can
be created in a project. This change limits number of clusters
in a project by checking cluster quota on cluster-create.

Change-Id: Ifa17d12692751fc6929e62be8bb59d481a2fd205
Partially-Implements: blueprint resource-quota
2017-01-25 08:55:45 -06:00
Vijendar Komalla aa56874bfb Resource Quota - Adding quota API
Change-Id: I7d2da1f86edae002531a529c4ffb469ce9f1777b
Partially-Implements: blueprint resource-quota
2017-01-23 11:00:11 -06:00
Jenkins 942fa495b6 Merge "Magnum stats API" 2017-01-19 07:24:33 +00:00
Vijendar Komalla 51e833137b Magnum stats API
This change introduces a new /stats REST endpoint that
provide the following basic information;
1) Total number of clusters and nodes for the given tenant.
2) Total number of clusters and nodes across all the tenants.
Follow-up patches include more stats.

Change-Id: Iac0bf9343549de31654545d5b1fd7601e56142a7
Partially Implements blueprint magnum-stats-api
2017-01-17 09:48:54 -06:00
Hongbin Lu 900c2af483 Remove unused context variable in db api
Change-Id: I78d7ddccddcc3630ddaf8fc6ddfc62dcff352bcd
2017-01-15 04:26:39 +00:00
Feng Shengqin 85b1babb00 Remove the usage of MagnumObjectDictCompat from magnum_service
Nova has removed the usage of NovaObjectDictCompat:
https://blueprints.launchpad.net/nova/+spec/rm-object-dict-compat.
magnum needs a similar cleanup.

Change-Id: I07ce6db76aaca517aadff6fba7fb51ff0972ccca
Partially-Implements: blueprint rm-object-dict-compat
2017-01-06 03:25:28 +00:00
Jenkins ec5857e793 Merge "Remove the usage of MagnumObjectDictCompat from x509keypair" 2017-01-04 06:44:00 +00:00
Jenkins 1d1096ce54 Merge "Remove the usage of MagnumObjectDictCompat from certificate" 2016-12-28 04:43:58 +00:00
Cao Xuan Hoang 2bc67d43a1 Removes unnecessary utf-8 encoding
The following file(s) added utf-8 encoding but never used. So we can
remove them at all.

magnum/objects/certificate.py
magnum/objects/cluster.py

Change-Id: I359bc96e21bedf60c62181c9aa0c5e65a07ba5e9
2016-12-22 10:56:03 +07:00
Feng Shengqin cea125b9a1 Remove the usage of MagnumObjectDictCompat from certificate
Nova has removed the usage of NovaObjectDictCompat:
https://blueprints.launchpad.net/nova/+spec/rm-object-dict-compat.
magnum needs a similar cleanup.

Change-Id: I04b221febaa4a31b0ecc9454a978c3a2ec4fa9c8
Partially-Implements: blueprint rm-object-dict-compat
2016-12-20 16:58:58 +08:00
Feng Shengqin 3dc50e2ad1 Remove the usage of MagnumObjectDictCompat from x509keypair
Nova has removed the usage of NovaObjectDictCompat:
https://blueprints.launchpad.net/nova/+spec/rm-object-dict-compat.
magnum needs a similar cleanup.

Change-Id: I69dc2d5bec54b6d3228dcb6ba834e5cc515c6a3f
Partially-Implements: blueprint rm-object-dict-compat
2016-12-20 01:12:09 +08:00
Eli Qiao 6033d0ef6f Add RESUME_FAILED to cluster's status field
Heat stack may return RESUME_FAILED, sync this status to cluster object.

Closes-Bug: #1646324
Change-Id: If48fc62d91fddeb579048a4ff5c4b238d3c5324c
2016-12-01 11:38:59 +08:00
Jaycen Grant 06f056f606 Allow keypair to be added during cluster create
Keypair can now be added during Cluster create command. This allows
Clusters to be created from the same ClusterTemplate but have
different keypair values when created. If not specified on create
then the value from the ClusterTemplate will be used. Adds keypair_id
to Cluster object and uses of keypair_id will use the value from Cluster
instead of ClusterTemplate.

Added release note for new features in magnum and magnum CLI.

Change-Id: I177a5aa06f881156944a9f74c9ccc3cd2abac492
Implements: blueprint keypair-override-on-create
2016-10-06 17:25:09 +00:00
yatin 745188b5ef Restrict server type only to vm/bm
Currently server type can be set to any String, but it should
be restricted to only allowed types: vm/bm.
vm: virtual machine
bm: baremetal
With this patch it is restricted only to allowed types.

Change-Id: Ifbeff8621b26702e54ad088195fddcb7c35424b2
Closes-Bug: #1622432
2016-09-16 14:48:44 +05:30
Jenkins 5f78e2f33a Merge "Restrict magnum service name" 2016-09-16 08:09:15 +00:00
Jaycen Grant 729c2d0ab4 Rename Bay DB, Object, and internal usage to Cluster
This is patch 3 of 3 to change the internal usage of the terms
Bay and BayModel.  This patch updates Bay to Cluster in DB and
Object as well as all the usages.  No functionality should be
changed by this patch, just naming and db updates.

Change-Id: Ife04b0f944ded03ca932d70e09e6766d09cf5d9f
Implements: blueprint rename-bay-to-cluster
2016-09-08 13:01:12 -07:00
Jaycen Grant 0b7c6401dd Rename BayModel DB, Object, and internal usage to ClusterTemplate
This patch is the first of 3 patches to change the internal
usage of the terms Bay and BayModel. This patch updates
BayModel to ClusterTemplate. No functionality should be
changed by this patch, just naming and db updates.

Change-Id: I0803e81be6482962be2878a8ea2c7480f89111ac
Implements: blueprint rename-bay-to-cluster
2016-09-01 14:47:39 -07:00
Jaycen Grant 8e0de76aff Rename bay to cluster in certificate object and references
This is patch #2 of 3 to rename the term bay to cluster within
the internal references and objects of magnum. This patch changes
all references to the certificate objects bay_uuid field to
cluster_uuid.  Certifcate does not have a db table so no db
changes were made. No functionality is changed by this patch,
just internal naming.

Change-Id: I68a3b87b75b49de43a7855355807b50a4ae695f3
Implements: blueprint rename-bay-to-cluster
2016-09-01 08:48:24 -07:00
Vijendar Komalla 50bc376c4d Include version info in bay/cluster show operation
Currently bay-show operation does not return bay/cluster
version information. This change contain changes to return
bay/cluster version and container version info.

Change-Id: Ie12b6583e6d85faa3607f87295c04d72698034a5
Closes-Bug: #1613413
2016-08-31 09:39:42 -05:00
Rajiv Kumar fd05c590a5 Restrict magnum service name
Magnum service binary name can be only conductor. Currently any
string having length less than 256 is allowed.

Change-Id: I03dad7a36a01e22e5fc63917f5e2919636f6ab2f
Closes-bug: #1615221
2016-08-22 09:30:31 +05:30
OTSUKA, Yuanying 26370a2509 Add floating_ip_enabled field to baymodel
This patch adds floating_ip_enabled field to baymodel to specify
whether floating ip is used or not.

Change-Id: I99677221250480b43a4b95ebf460c43bc77090ad
Partially-Implements: blueprint bay-with-no-floating-ips
2016-08-16 16:21:35 -05:00
Wenzhi Yu 63b5c21c8d Rollback bay on update failure
There is a rollback mechanism in heat after the stack
update failed. There should be a rollback mechanism in
magnum after bay update failed.

This patch add new microversion 1.3 to add rollback
support for Magnum bay, user can enable rollback on bay
update failure by specifying microversion 1.3 in header(
{'OpenStack-API-Version': 'container-infra 1.3'}) and
passing 'rollback=True'(http://XXX/v1/bays/XXX/?rollback=True)
when issuing bay update reqeust.

Change-Id: Idd02769f98078702404a11dc9f7a3339ce4e22eb
Partially-Implements: blueprint bay-rollback-on-update-failure
2016-08-15 17:09:23 +08:00
Spyros Trigazis e6a29fb252 Remove ReplicationController object
Following the removal of service [1], pod [2] and container [3], remove
COE specific object ReplicationController.

This change also removes k8s_conductor.

[1] I4f06bb779caa0ad369a2b96b4714e1bf2db8acc6
[2] I8c2499ccb97aae39d80868ce02fbef292d762c10
[3] I288fa7a9717519b1ae8195820975676d99b4d6d2

Change-Id: Ica100c8d2dfdd7dc709feb1f5cdc5a3f3d6c7318
Partially-Implements: blueprint delete-container-endpoint
Partially-Implements: blueprint bay-drivers
2016-08-10 17:56:48 +02:00