Commit Graph

114 Commits

Author SHA1 Message Date
Michal Nasiadka d544698fae Remove use of autocommit
Change-Id: I6e973935edf5e754ba054e724b6963b6d5fd795a
2024-02-29 11:54:28 +00:00
Jake Yip 8a30ad3462 Add feature to specify driver explicitly
Allow ClusterTemplate to explicitly specify a driver to use for creating
Clusters.

This is initially sourced from the image property 'magnum_driver', but
may be improved to be specified via client in the future.

Falls back to old driver discovery using (coe, server_type, os) tuple to
keep existing behaviour.

Change-Id: I9e206b589951a02360d3cef0282a9538236ef53b
2024-02-26 14:50:18 +11:00
Takashi Kajinami 206d19ec52 Remove six from db module
This is part of the steps to remove usage of six library, which is no
longer needed since python 2 support was removed.

Change-Id: I529759167f960714ebf93be7b89ee849040361b2
2024-02-19 10:43:16 +00:00
Michal Nasiadka 021293d8f8 SQL Alchemy 2.x: Stop using deprecated API
This patch fixes magnum/db/sqlalchemy/models.py:99: MovedIn20Warning:
Deprecated API features detected! errors.

See [1].

[1]: https://docs.sqlalchemy.org/en/20/changelog/migration_20.html#migration-orm-configuration

Change-Id: I397fc59fefab86149a023ff8102e0823fa5ab6a7
2024-02-01 12:10:07 +00:00
Michal Nasiadka d18aa8ffa2 sqlalchemy: Import String from sqlalchemy directly
With dropping of MySQL NDB cluster support in oslo.db the String
subclass that implemented specific options is gone [1]

[1]: Ia8b4ed8cd755d283bb773e55293457190b34c482

Change-Id: I4ae0943a9cfdf6ed96bcd9ca54d98d008f6a5573
2023-09-27 12:48:55 +02:00
Diogo Guerra 11bcc17568 Drop bay and baymodel from magnum
- Drop bay and baymodel tests
- Drop bay and baymodel from controllers

Depends-On: Ib85e4fda8e4ac467bd49590dc72ba5913bb9a19d

Story: 2009104
Task: 42957
Task: 42959

Signed-off-by: Diogo Guerra <diogo.filipe.tomas.guerra@cern.ch>
Change-Id: Ida2e42c86400438951d9804e3ce122c56a46b94f
2023-05-09 13:59:57 +00:00
Feilong Wang 16344a5a95 Add separated CA cert for etcd and front-proxy
Support creating different for k8s, etcd and front-proxy for
security hardening. We're following some best practices[1][2] but
adjusted based on the current Magnum deployment approach.

[1] https://kubernetes.io/docs/setup/best-practices/certificates/
[2] https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/

Task: 40687
Story: 2008031

Change-Id: I523a4a85867f82d234ba1f3e6fad8b8cd2291182
2021-04-01 17:31:34 +00:00
Diogo Guerra 332e2b6fe4 Add CT tags field to the database and API
We noticed that from the user perspective it is hard
to know when a cluster_template provided by the cloud
admin is mature enough for a production release.
This field will allow the administrator to add an
annotation to the cluster template like
{deprecated, recommended, testing} giving further
usefull information to the end user about the
template's life cycle

This patch adds the necessary database column and
API objects to handle the new argument.

story: 2007857
task: 40160

Change-Id: I5d1c4221f089bc5cd12b25f620aa01771a029df9
Signed-off-by: Diogo Guerra <diogo.filipe.tomas.guerra@cern.ch>
2021-02-05 22:45:47 +00:00
Feilong Wang 946c1d67c7 Add master_lb_enabled to cluster
Adding the master_lb_enabled option when creating a cluster,
which will benefit both the cloud provider side and the end
user side. For cloud prodiver, they don't have to maintain
separate cluster templates with or w/o master_lb_enabled enabled.
For end user, they can easily use one single template to create
different clusters with different configs.

Task: 39680
Story: 2007634

Change-Id: I0b586f05168ece84fd340ef7493a56688191053d
2020-07-21 11:07:33 +12:00
Andreas Jaeger ae228bb5cc Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found.

Update local hacking checks for new flake8.

Remove hacking and friends from lower-constraints, those are not needed
for co-installing.

Change-Id: I926efaef501f190e78da9cab40c1e94203277258
2020-03-31 20:09:46 +02:00
Theodoros Tsioutsias cbe05aa97d ng-6: Add new fields to nodegroup objects
Since each nodegroup will be one independent stack, we have to add
more fields to the table and object in order to track each stack
contained in the cluster. This adds the stack_id, version, status,
status_reason and version fields to the nodegroup object.

Change-Id: I6d36b2d3bc6476efbef6a9f702ffc73cfa0fab8c
2019-09-25 12:26:00 +00:00
Feilong Wang 32989b4f7b Allow setting network, subnet and FIP when creating cluster
When using a public cluster template, user still need the capability
to reuse their existing network/subnet, and they also need to be
able to turn of/off the floatingip to overwrite the setting in the
public template. This patch supports that by adding those three
items as parameters when creating cluster.

Story: 2006208
Task: 35797

Change-Id: I11579ff6b83d133c71c2cbf49ee4b20996dfb918
2019-07-31 20:41:20 +12:00
Theodoros Tsioutsias 3f80cbab06 ng-4: Adapt cluster object
This commit removes the fields node_addresses, master_addresses,
node_count and master_count from the cluster object since this info
will be stored in the nodegroups. At the same time, provides the way
to adapt existing clusters to the new schema.

story: 2005266

Change-Id: Iaf2cef3cc50b956c9b6d7bae13dbb716ae54eaf7
2019-03-29 10:31:48 +00:00
Theodoros Tsioutsias 0607c7a9d6 ng-1: Add nodegroup representation
This adds the object and db schema changes needed for supporting
nodegroups.

story: 2005266

Change-Id: Ibf10277a52aa94c4b217cf3b364844b04baab1e0
2019-03-21 16:19:56 +00:00
Ricardo Rocha 6aee864954 Add hidden flag to cluster template
Add a new hidden flag to cluster templates. This allows an operator to
keep a cluster public (accessible to all users) while not showing them
in cluster template listing.

Story: 2004941
Task: 29342

Change-Id: Ia2717ca960041753f6e772bf2d41c7f5a196dae6
2019-02-12 11:38:15 +01:00
Erik Olof Gunnar Andersson daf34d9df8 Use oslo_serialization instead of the json module directly
* Always use oslo jsonutils.
* Consistently import jsonutils as-is.
* Use dump_as_bytes instead of dumps.
https://wiki.openstack.org/wiki/Python3#Serialization:_base64.2C_JSON.2C_etc.

Change-Id: I2b65faa7df43a1d58205e8ff106ff62f73d78198
2019-01-21 16:28:03 -08:00
Feilong Wang d80febb384 Add health_status and health_status_reason to cluster
APIImpact
Related blueprint cluster-healing

Change-Id: I78d4d14fb064996de77bdd6a381d2df53a9488b8
2018-08-27 14:28:46 +12:00
oorgeron 0e6e27eb7a Enables MySQL Cluster Support for Magnum
This enables the usage of the mysql_enable_ndb setting
in oslo_db from bug 1564110. This allows operators to
select from MySQL (InnoDB) or MySQL Cluster (NDB)
as the storage engine backend. There are only two
tables that need to have their columns auto adjusted
to fit with in the 14k row width limit of NDB, cluster
and cluster_templates. The modifications here will
have zero affect on users of MySQL (InnoDB). Additionally,
this fix will auto adjust those tables in models.py
module to support this enhancement.

Closes-Bug: 1691514
Change-Id: Ia0584059f998baf3db4a44bccbb6f6ff8a058ae1
Depends-On: I7f9c830073bf9a30abce0aa4bb55b5c9cf661afe
2018-02-19 12:15:58 +00:00
Clenimar Filemon 34754f36f0 federation api: federation table and db layer
this commit introduces a new `Federation` table to
Magnum database, as well as the necessary DB layer
APIs to access and manage it.

this belongs to the first phase of the implementation
of the federation api. check [1] for more details.

[1] https://review.openstack.org/#/c/489609/

Change-Id: Ie8a68cd3198c8fc7930069fd2e55f1cad55b6c9b
Partially-Implements: blueprint federation-api
2018-01-15 14:58:04 -03:00
Ricardo Rocha 84006f63d7 Allow flavor_id on cluster create
Add flavor_id as an option during cluster create. If not given,
the default is taken from the cluster template.

Add flavor_id in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1699245
Change-Id: Ib60c05cce1cf2639ca4740abdd264403033433f9
2017-12-15 16:32:40 +00:00
Ricardo Rocha 446b159349 Allow master_flavor_id on cluster create
Add master_flavor_id as an option during cluster create. If not given,
the default is taken from the cluster template.

Add master_flavor_id in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1699247
Change-Id: Id1d973167b381538121583a0a9691304b39e98de
2017-09-27 09:30:18 +00:00
Ricardo Rocha 3d04ed4cbb Allow labels on cluster create
Add labels as an option during cluster create. If not given,
the default is taken from the cluster template.

Add labels in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1697651
Implements: blueprint flatten-attributes
Change-Id: I8990c78433dcbbca5bc4aa121678b02636346802
2017-08-21 12:27:52 +00:00
Ricardo Rocha 79039bb419 Allow docker_volume_size on cluster create
Add docker_volume_size as an option during cluster create. If not given,
the default is taken from the cluster template.

Add docker_volume_size in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1697648
Implements: blueprint flatten-attributes
Change-Id: Ic6d77e6fdf5b068fa5319b238f4fd98b4d499be4
2017-06-23 11:31:27 +00:00
yuanpeng 71d25456d2 Remove log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.

See:
http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html
http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html

Change-Id: If1f4bd2f6be967368f52fb367c5a428d3eb58a9d
Closes-Bug:#1674551
2017-03-30 17:05:10 +08:00
Jaycen Grant 06f056f606 Allow keypair to be added during cluster create
Keypair can now be added during Cluster create command. This allows
Clusters to be created from the same ClusterTemplate but have
different keypair values when created. If not specified on create
then the value from the ClusterTemplate will be used. Adds keypair_id
to Cluster object and uses of keypair_id will use the value from Cluster
instead of ClusterTemplate.

Added release note for new features in magnum and magnum CLI.

Change-Id: I177a5aa06f881156944a9f74c9ccc3cd2abac492
Implements: blueprint keypair-override-on-create
2016-10-06 17:25:09 +00:00
Hieu LE d86b5735cd Centralize config option: database section
Centralize config option of database section.
Replace oslo_conf cfg to magnum.conf.

Change-Id: Id12bbf3ad8d3342450cd64cf23761a60d49ee46a
Implements: blueprint centralize-config-magnum
2016-09-22 05:57:01 +00:00
Jaycen Grant 729c2d0ab4 Rename Bay DB, Object, and internal usage to Cluster
This is patch 3 of 3 to change the internal usage of the terms
Bay and BayModel.  This patch updates Bay to Cluster in DB and
Object as well as all the usages.  No functionality should be
changed by this patch, just naming and db updates.

Change-Id: Ife04b0f944ded03ca932d70e09e6766d09cf5d9f
Implements: blueprint rename-bay-to-cluster
2016-09-08 13:01:12 -07:00
Jaycen Grant 0b7c6401dd Rename BayModel DB, Object, and internal usage to ClusterTemplate
This patch is the first of 3 patches to change the internal
usage of the terms Bay and BayModel. This patch updates
BayModel to ClusterTemplate. No functionality should be
changed by this patch, just naming and db updates.

Change-Id: I0803e81be6482962be2878a8ea2c7480f89111ac
Implements: blueprint rename-bay-to-cluster
2016-09-01 14:47:39 -07:00
Vijendar Komalla 50bc376c4d Include version info in bay/cluster show operation
Currently bay-show operation does not return bay/cluster
version information. This change contain changes to return
bay/cluster version and container version info.

Change-Id: Ie12b6583e6d85faa3607f87295c04d72698034a5
Closes-Bug: #1613413
2016-08-31 09:39:42 -05:00
OTSUKA, Yuanying 26370a2509 Add floating_ip_enabled field to baymodel
This patch adds floating_ip_enabled field to baymodel to specify
whether floating ip is used or not.

Change-Id: I99677221250480b43a4b95ebf460c43bc77090ad
Partially-Implements: blueprint bay-with-no-floating-ips
2016-08-16 16:21:35 -05:00
Spyros Trigazis e6a29fb252 Remove ReplicationController object
Following the removal of service [1], pod [2] and container [3], remove
COE specific object ReplicationController.

This change also removes k8s_conductor.

[1] I4f06bb779caa0ad369a2b96b4714e1bf2db8acc6
[2] I8c2499ccb97aae39d80868ce02fbef292d762c10
[3] I288fa7a9717519b1ae8195820975676d99b4d6d2

Change-Id: Ica100c8d2dfdd7dc709feb1f5cdc5a3f3d6c7318
Partially-Implements: blueprint delete-container-endpoint
Partially-Implements: blueprint bay-drivers
2016-08-10 17:56:48 +02:00
Tom Cammann 40aa6550f1 Remove container object
Following on from removing the k8s specific APIs in
I1f6f04a35dfbb39f217487fea104ded035b75569 the objects associated with
these APIs need removal.

Remove the container object, drop the db table and remove references to
the container object. The docker_conductor has also been removed as this
was used for managing containers using Magnum objects.

Change-Id: I288fa7a9717519b1ae8195820975676d99b4d6d2
Partially-Implements: blueprint delete-container-endpoint
Co-Authored-By: Spyros Trigazis <strigazi@gmail.com>
2016-08-01 16:16:20 +02:00
PanFengyun f3f0f9fef6 Add i18n support for some ERROR message
There are some ERROR message miss i18n support, this patch adds i18n
support for then and changes some marks in message.

Change-Id: I33ce248b5bfc4e09680a4c6ea1f1c0e1ef02ed5d
Closes-Bug: #1603759
2016-07-17 20:55:49 +08:00
OTSUKA, Yuanying 62c75e5883 Add fixed_subnet field to baymodel
Ironic template needs to specify fixed subnet name/ID.
And also user sometimes want to build his cluster on
his network which he built.
This patch adds fixed_subnet field to baymodel to specify
fixed subnet which bay will build on.

Change-Id: Ia21ad6e3e1472a0d078d08369ef62601034fcc17
Partial-Implements: blueprint magnum-baremetal-full-support
2016-06-30 15:16:45 +09:00
Drago Rosson 6d10dbf35d Add master_lb_enabled field to baymodel
master_lb_enabled is a boolean field that determines if a bay's master
nodes should be load balanced. Defaults to False.

Change-Id: Icf8317432dec70c99c29a3af9edb0499bfd2439d
Partially-Implements: blueprint decouple-lbaas
2016-06-28 08:41:49 +00:00
Tom Cammann 3e02840628 Remove service object
Following on from removing the k8s specific APIs in
I1f6f04a35dfbb39f217487fea104ded035b75569 the objects associated with
these APIs need removal.

Remove the service object, drop the db table and remove references to
the service object.

Change-Id: I4f06bb779caa0ad369a2b96b4714e1bf2db8acc6
Partially-Implements: blueprint delete-container-endpoint
2016-06-02 10:46:13 +01:00
Spyros Trigazis dec85b538f Add docker-storage-driver attribute to baymodel
* This parameter will be optional and users will be able to select
  a supported driver, otherwise the default configuration will be
  used.
* Add docker storage driver enum field to baymodel
* Add db upgrade file
* Update heat templates for kubernetes and swarm allowing only
  devicemapper and overlay as docker_storage_driver values.
* Add configuration for OverlayFS on Fedora Atomic, if overlay is
  incompatible bay creation will result a CREATE_FAILED status.
* Factor out configuration of docker storage drivers
* Update tests
* Add Release Notes

Partially-Implements: blueprint support-for-different-docker-storage-driver
Change-Id: Ib58cb734c4e9c90d5d83574852213d2e97359e92
2016-06-01 10:42:19 +02:00
Madhuri Kumari 303d14dde0 X509keypair cleanup
This patch does following:

* Removes X509keypair controller as there is already Certificate
  controller for same purpose.
* Removes X509keypair conductor.
* Removes name, ca_cert and bay_uuid from x509keypair model as
  Bay model already holds certificate references.
* Add intermediates and private_key_passphrase to x509keypair
  model.
* Remove related tests and changes.

Change-Id: I9271221cd1d07c672c4a380a4ae3593237fca66a
Partially-Implements: blueprint barbican-alternative-storeX
2016-05-30 12:02:26 +05:30
Tom Cammann d20e5ef715 Remove pod object
Following on from removing the k8s specific APIs in
I1f6f04a35dfbb39f217487fea104ded035b75569 the objects associated with
these APIs need removal.

Remove the pod object, drop the db table and remove references to the
pod object.

Change-Id: I8c2499ccb97aae39d80868ce02fbef292d762c10
Partially-Implements: blueprint delete-container-endpoint
2016-05-27 11:34:09 +01:00
Eli Qiao 7b1e0ecb2e Add insecure_registry column to baymoddel
Adding new column of `insecure_registry`, it indicates using insecure
registry when deploying a bay, it should be a validate url with port.

Partially-Implements: blueprint support-insecure-registry
Change-Id: Iaa28d6409264d1d2acd992a7fa14630bb6394852
2016-05-23 09:02:22 +08:00
Hongbin Lu c64e8eb502 Remove unused attribute "ssh_authorized_key"
This attribute was used in CoreOS template before. The usage of this
attribute has been removed [1][2]. This patch remove it from Magnum.
[1] https://review.openstack.org/#/c/262628/
[2] https://review.openstack.org/#/c/262926/

Change-Id: I7ff80fd1421bf2ea61e951e87f83983fb2883f79
Closes-Bug: #1540134
Partial-Implements: blueprint coreos-k8s-bay
2016-02-28 15:33:56 -05:00
Vilobh Meshram faca94607a Resource Quota - Introduce Quota Table
Introduce Quota Table and Quota list/create API's. With
this concept it will be possible to set quota on a
resource within a project. Fox example X number
of bay creation within a project and if the request
exceeds X bay creation will not be allowed. This change
only introduces the db layer changes.

Change-Id: I8990052df48bdbf6eee426e88ed6c9c2f8cfd344
Partially-Implements: bp resource-quota
2016-02-27 03:25:47 -08:00
Hua Wang 725bd5c99d Create a trustee user for each bay
Docker registry, k8s load balancer and volume driver have a similar
need to use trust, so we need to create a trustee for each bay.

Change-Id: If034e74ce2ea80a7faa886d4edf789e576c30eb5
Partially-Implements: blueprint create-trustee-user-for-each-bay
2016-02-18 18:16:35 +08:00
Hua Wang 342e83f033 Remove node object from Magnum
The node object represents either a bare metal or virtual machine
node that is provisioned with an OS to run the containers, or
alternatively, run kubernetes. Magnum use Heat to deploy the nodes,
so it is unnecessary to maintain node object in Magnum. Heat can do
the work for us. The code about node object is useless now, so let's
remove it from Magnum.

Closes-Bug: #1540790
Change-Id: If8761b06a364127683099afb4dc51ea551be6f89
2016-02-04 14:02:49 +08:00
wangqun 6dc4816521 Adds volume_driver in db api
Adds volume_driver to the following:
1. api baymodel attribute
2. a new column to the baymodel db
3. objects.Note: Updates baymodel object version
4. Unit tests

Co-Authored By: Kai Qiang Wu(Kennan)<wkqwu@cn.ibm.com>
Co-Authored By: Ton Ngo<ton@us.ibm.com>
Partially-Implements: blueprint magnum-integrate-with-cinder

Change-Id: I25506bd39ab4be513ef99c39f8c52a502b300d92
2016-01-26 05:42:11 +00:00
Hua Wang 41b7bccc02 Remove baylock
bay_lock was used for horizontal scalability perivously,
and it is deprecated now. The discussions are in [1].
Let's remove it.

[1] https://blueprints.launchpad.net/magnum/+spec/horizontal-scale

Change-Id: Ia2b33a96c0e831d8c85a2a18d701575958be6dbd
Closes-Bug: #1526581
2015-12-18 09:47:46 +08:00
Hua Wang 8074f6f4ce Create trust_id for bay
All bays use the same trustee_user and different trust. A trust is
created for a bay when the bay is created, and is deleted when the
bay is deleted.

Partially-Implements: blueprint registryv2-in-master
Change-Id: Iab2037677f683fe4c562915b98303da02c59c299
2015-12-15 11:28:23 +08:00
Yang Hongyang f0749cc56c Add missing bay_create_timeout to bay object
Currently bay-show command always shows "0" in bay_create_timeout, this
is confusing for users, store bay_create_timeout into db so that bay-show
will return correct bay_create_timeout value.

Change-Id: Icaea11d7d298bd6245a50c464bff6ff20292e092
Closes-Bug: #1521146
2015-12-02 22:46:59 +08:00
Xian Chaobo c5317ebf03 Add support to set env to a container
Docker run support an option to set environment variables
to a container. container-create in Magnum also needs this.
This will help for the user to create a container with some
environment variables. Also, swarm use some special environments
to scheduler. This patch add support to do this.

Implements: blueprint support-to-set-env-variables

Change-Id: Ifa46eb6d0e528eadca44d265e906c92ee752af43
2015-11-28 11:53:00 +08:00
Vilobh Meshram 798fc51c47 Create BayModel with provided(VM/BM) server type
In its current state of art, the Baymodel is created
with only "VM" server type. This value is hardcoded in the
code and a tech debt bug was filed to clean it off. Also
this feature can be useful to allow bay model creation with
various server type like virtual machines, baremetal etc.

Client side changes will be provided as part of seperate
patch.

Partial-Bug: #1491003

Change-Id: I8981031d097fc41c94ad509c7b090f0a57a90525
2015-11-11 22:07:00 +00:00