Commit Graph

35 Commits

Author SHA1 Message Date
Michal Nasiadka bc79012f46 Drop Swarm support
Label validator function has been left behind, although it's not
checking for anything right now - might be useful in future.

Change-Id: I74c744dc957d73aef7556aff00837611dadbada7
2024-01-24 13:20:21 +13:00
Feilong Wang 946c1d67c7 Add master_lb_enabled to cluster
Adding the master_lb_enabled option when creating a cluster,
which will benefit both the cloud provider side and the end
user side. For cloud prodiver, they don't have to maintain
separate cluster templates with or w/o master_lb_enabled enabled.
For end user, they can easily use one single template to create
different clusters with different configs.

Task: 39680
Story: 2007634

Change-Id: I0b586f05168ece84fd340ef7493a56688191053d
2020-07-21 11:07:33 +12:00
Hervé Beraud f1f4979844 Use unittest.mock instead of mock
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.

Change-Id: Ib3ca91e93db260e081c43d236d5d6fae5a2adfd7
2020-06-09 20:57:54 +02:00
Sean McGinnis 45fd25c572 Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.

Change-Id: I6245a207ec65df0ef11a798b4cffe3157abccb73
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Co-Authored-By: Spyros Trigazis <spyridon.trigazis@cern.ch>
2020-04-29 17:10:23 +03:00
Theodoros Tsioutsias 44631afbbf ng-10: Fix cluster template conditions
Removes the role heat param from all templates. Instead and only for
k8s templates adds the master_role and worker_role params. The new
worker_only condition should be true for all roles except for master.
Finally, adds the missing is_cluster_stack param to all templates.

Change-Id: Ie0799373fe492c2e0a0cad903ed6e8c93e6266b5
2019-10-16 11:53:44 +00:00
Spyros Trigazis 2f72fdfbf6 Pass ssh public key as string
* Fedora CoreOS need the key to be passed as
  a string.
* We can adopt in all drivers so that users in
  the same project can do cluster resize.

story: 2005201
task: 36934

Change-Id: I9a18ce4dcbd74f0dcd23274baed7c8c3d2029d50
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
2019-10-08 07:56:52 +00:00
Theodoros Tsioutsias d4a52719f1 ng-7: Adapt parameter and output mappings
With this change parameter and output mappings can support multiple
stacks.

Change-Id: I45cf765977c7f5a92f28ae12c469b98435763163
2019-09-26 08:45:51 +00:00
Feilong Wang 32989b4f7b Allow setting network, subnet and FIP when creating cluster
When using a public cluster template, user still need the capability
to reuse their existing network/subnet, and they also need to be
able to turn of/off the floatingip to overwrite the setting in the
public template. This patch supports that by adding those three
items as parameters when creating cluster.

Story: 2006208
Task: 35797

Change-Id: I11579ff6b83d133c71c2cbf49ee4b20996dfb918
2019-07-31 20:41:20 +12:00
Theodoros Tsioutsias 3f80cbab06 ng-4: Adapt cluster object
This commit removes the fields node_addresses, master_addresses,
node_count and master_count from the cluster object since this info
will be stored in the nodegroups. At the same time, provides the way
to adapt existing clusters to the new schema.

story: 2005266

Change-Id: Iaf2cef3cc50b956c9b6d7bae13dbb716ae54eaf7
2019-03-29 10:31:48 +00:00
Theodoros Tsioutsias ea95b0dc5c ng-3: Adapt existing drivers
The existing drivers are adapted to get node_count and master_count
information from the cluster's nodegroups. At the same time the
output mappings were updated to reflect the changes in the stack to
the nodegroups.

story: 2005266

Change-Id: I725413e77f5a7bdb48131e8a10e5dc884b5e066a
2019-03-28 10:31:01 +00:00
Lingxian Kong 3c8edd4d88 Add support for Octavia resources in Heat
Octavia is already an official LBaaS solution for Openstack
(https://governance.openstack.org/tc/reference/projects/octavia.html) and
will deprecate the neutron-lbaas extension starting from Queens release.

For deployment support Octavia service for load balancing functionlity,
Octavia related resources instead of LBaaS should be used in Heat template.

Tested in my DevStack environment.

Change-Id: Icc45e0a126c648fbcba4ebcd1bb258d60957f2d6
Closes-Bug: #1748577
2018-02-16 11:35:46 +13:00
Ricardo Rocha 53d386dc01 Add label availability_zone
Add a new label 'availability_zone' allowing users to specify the AZ
the nodes should be deployed in. Only one AZ can be passed for this
first implementation.

Change-Id: I9e55d7631191fffa6cc6b9bebbeb4faf2497815b
Partially-Implements: blueprint magnum-availability-zones
2018-02-05 15:03:59 +00:00
Feilong Wang be0609ce88 Support soft-anti-affinity policy for nodes
Currently, there is no guarantee to make sure all nodes of one cluster are
created on different compute hosts. So it would be nice if we can create
a server group and set it with anti-affinity policy to get a better HA
for cluster. This patch is proposing to create a server group for master
and minion nodes with soft-anti-affinity policy by default.

Closes-Bug: #1737802

Change-Id: Icc7a73ef55296a58bf00719ca4d1cdcc304fab86
2018-01-24 07:13:48 +13:00
Spyros Trigazis 65dfb2009f Add openstack_ca_file configuration option
In the drivers section of magnum.conf add openstack_ca_file.
This file is expected to be a CA Certificate OR CA bundle
which will be passed on every node and it will be installed
on the host's CA bundle.

Update devstack plugin to use the ssl bundle if tls-proxy is
enabled.

Install the CA for drivers:
k8s_coreos_v1
k8s_fedora_atomic_v1
k8s_fedora_ironic_v1
mesos_ubuntu_v1
swarm_fedora_atomic_v1
swarm_fedora_atomic_v2

Add doc in troubleshooting-guide.

Add release notes.

Closes-Bug: #1580704
Partially-Implements: blueprint heat-agent
Change-Id: Id48fbea187da667a5e7334694c3ec17c8e2504db
2018-01-17 14:58:56 +00:00
Ricardo Rocha 84006f63d7 Allow flavor_id on cluster create
Add flavor_id as an option during cluster create. If not given,
the default is taken from the cluster template.

Add flavor_id in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1699245
Change-Id: Ib60c05cce1cf2639ca4740abdd264403033433f9
2017-12-15 16:32:40 +00:00
Kirsten G b07b6f34d5 Add verify_ca configuration parameter
Added configuration parameter, verify_ca, to magnum.conf with default
value of True. This parameter is passed to the heat templates to
indicate whether the cluster nodes validate the Certificate Authority
when making requests to the OpenStack APIs (Keystone, Magnum, Heat).
This configuration parameter can be set to False to disable CA
validation.

Co-Authored-By: Vijendar Komalla <vijendar.komalla@rackspace.com>

Change-Id: Iab02cb1338b811dac0c147378dbd0e63c83f0413
Partial-Bug: #1663757
2017-11-21 10:25:32 -08:00
Ricardo Rocha 446b159349 Allow master_flavor_id on cluster create
Add master_flavor_id as an option during cluster create. If not given,
the default is taken from the cluster template.

Add master_flavor_id in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1699247
Change-Id: Id1d973167b381538121583a0a9691304b39e98de
2017-09-27 09:30:18 +00:00
Ricardo Rocha 3d04ed4cbb Allow labels on cluster create
Add labels as an option during cluster create. If not given,
the default is taken from the cluster template.

Add labels in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1697651
Implements: blueprint flatten-attributes
Change-Id: I8990c78433dcbbca5bc4aa121678b02636346802
2017-08-21 12:27:52 +00:00
Ricardo Rocha 79039bb419 Allow docker_volume_size on cluster create
Add docker_volume_size as an option during cluster create. If not given,
the default is taken from the cluster template.

Add docker_volume_size in the Cluster object and use that instead
of the one from ClusterTemplate.

Update both magnum and magnum cli documentation to reflect the above changes.

Partial-Bug: #1697648
Implements: blueprint flatten-attributes
Change-Id: Ic6d77e6fdf5b068fa5319b238f4fd98b4d499be4
2017-06-23 11:31:27 +00:00
Spyros Trigazis 295ef945f6 swarm: Add docker_volume_type label
* add docker_volume_type for the cinder volumes which are
  used for docker storage.
* add default_docker_volume_type configuration option

Related-Bug: #1678153
Change-Id: Ie18096acf24873ef91a904df4f1a84694a2bb644
2017-06-01 16:17:54 +00:00
Kevin Lefevre 4c241a683f Enable custom keystone endpoint_type in templates
Allow to specify a custom AUTH_URL for the templates in case instances
cannot reach internalURL which is the case in mose deployment.

A new variable in trust section: trustee_keystone_interface which
default to public is introduced.

Change-Id: I2a908c0752387e4ff4ad2b0fdf0c1025a73ce806
Closes-Bug: #1643197
2017-05-01 18:15:58 +02:00
Mathieu Velten c0787b4e94 Fix usage of the trustee user in K8S Cinder plugin
Closes-Bug: #1672667
Change-Id: I702818777ea4664ecd560c4b7a02431c86988e17
2017-04-07 14:15:52 +02:00
Johannes Grassler e93d82e8b3 Fix CVE-2016-7404
This commit addresses multiple potential vulnerabilities in
Magnum. It makes the following changes:

* Permissions for /etc/sysconfig/heat-params inside Magnum
  created instances are tightened to 0600 (used to be 0755).
* Certificate retrieval is modified to work without the need
  for a Keystone trust.
* The cluster's Keystone trust id is only passed into
  instances for clusters where that is actually needed. This
  prevents the trustee user from consuming the trust in cases
  where it is not needed.
* The configuration setting trust/cluster_user_trust (False by
  default) is introduced. It needs to be explicitely enabled
  by the cloud operator to allow clusters that need the
  trust_id to be passed into instances to work. Without this
  setting, attempts to create such clusters will fail.

Please note, that none of these changes apply to existing
clusters. They will have to be deleted and rebuilt to benefit
from these changes.

Change-Id: I643d408cde0d6e30812cf6429fb7118184793400
2017-02-09 16:44:27 +01:00
Mathieu Velten 22fb89a6e1 Make private network optional
If a fixed_network and fixed_subnet is specified no private network
is created by the templates and the specified network is
used instead for VMs provisioning, like in the Ironic driver.

Currently missing is the code to handle the use case where you
specify a fixed_network but not a fixed_subnet, this will come
in a following patch.

Partially Implements: blueprint decouple-private-network
Change-Id: I2003eb709b22b905063d846eb71570fc5e033618
2017-01-09 20:14:44 +01:00
Randall Burt c40413518a Use correct context synching status
Use the trust context when synching cluster status
with orchestration status.

Change-Id: I8ae0d1b92c3adce83032bb6c5f269d8d23c20c5e
Partial-Blueprint: bp-driver-consolodation
Closes-Bug: #1651243
2016-12-20 17:02:00 +00:00
Randall Burt 759c1b3b2b Move cluster status updates into driver
This is an alternative implementation to:

https://review.openstack.org/#/c/397961

This version implements an earlier proposal from the
spec that adds a driver method for synchronizing
cluster state. This method is optional so that drivers
that do not wish to leverage the existing periodic
synchronization task can do so in whatever manner
they wish and Magnum will not force them to do anything
unnecessarily.

1. add an update_cluster_status method to the driver
   interface
2. implment update_cluster_status for Heat drivers
   using the existing tested logic
3. Remove cluster status updates from the cluster conductor
   in favor of the periodic sync_cluster_status task - this
   should avoid timeouts and race conditions possible in the
   previous implementation
4. Update the periodic sync_cluster_status method to use
   the driver to update cluster status rather than calling
   Heat directly

Change-Id: Iae0ec7af2542343cc51e85f0efd21086d693e540
Partial-Blueprint: bp-driver-consolodation
2016-12-01 19:52:06 -06:00
Randall Burt 7890725c52 Refactor driver interface (pt 1)
Refactor driver interface to encapsulate the orchestration
strategy. This first patch only refactors the main driver
operations. A follow-on will handle the state synchronization
and removing the poller from the conductor.

1. Make driver interface abstract
2. Move external cluster operations into driver interface
3. Make Heat-based driver abstract and update based on
   driver interface changes
4. Move Heat driver code into its own module
5. Update existing Heat drivers based on interface changes

Change-Id: Icfa72e27dc496862d950ac608885567c911f47f2
Partial-Blueprint: bp-driver-consolodation
2016-12-01 09:23:46 -06:00
Spyros Trigazis f82749457c Make cinder volume optional
In the swarm_atomic and k8s_atomic drivers container images are
stored in a dedicated cinder volume per cluster node. It is
proven that this architecture can be a scalability bottleneck.

Make the use of cinder volumes for container images and opt-in
option.  If docker-volume-size is not specified no cinder
volumes will be created. Before, if docker-volume-size wasn't
specified the default value was 25.

To use cinder volumes for container storage the user will
interact with magnum as before, (meaning the valid values are
integers starting from 1).

Closes-Bug: #1638006
Change-Id: I3394c62a43bbf950b7cf0b86a71b1d9b0481d68f
2016-11-10 11:13:09 +01:00
yatin 2d160ecfcb Support scheduler strategy for swarm cluster
Swarm cluster can be created by specifying any of the scheduler
strategy supported by swarm. The strategy can be specified
while creating cluster template using labels parameter, Ex:-
--labels swarm_strategy=spread
Supported values for swarm_strategy=spread, binpack, random

Change-Id: If471f10a3b1f955638a77d5afe462aebdeb4277c
Implements: blueprint add-support-different-strategy-in-swarmbay
2016-11-03 12:00:57 +05:30
Hieu LE 08a48895c4 Centralize config option: docker_registry section
Centralize config option of docker_registry section.
Replace oslo_conf cfg to magnum.conf.

Change-Id: I43d3ce068bb6638f71ea14577f34c1df3d7c9d8c
Implements: blueprint centralize-config-magnum
2016-10-11 11:05:40 +07:00
Jenkins ba11900215 Merge "Cluster Drivers" 2016-10-10 17:18:26 +00:00
Jaycen Grant 06f056f606 Allow keypair to be added during cluster create
Keypair can now be added during Cluster create command. This allows
Clusters to be created from the same ClusterTemplate but have
different keypair values when created. If not specified on create
then the value from the ClusterTemplate will be used. Adds keypair_id
to Cluster object and uses of keypair_id will use the value from Cluster
instead of ClusterTemplate.

Added release note for new features in magnum and magnum CLI.

Change-Id: I177a5aa06f881156944a9f74c9ccc3cd2abac492
Implements: blueprint keypair-override-on-create
2016-10-06 17:25:09 +00:00
murali allada 104501cfe6 Cluster Drivers
- Dynamically load drivers using stevedore
- Changed the entry points to reference drivers instead of
  template definitions
- Implement Create and update driver operations

Change-Id: I5c3259404c796e1935c872cf3109ffecae3cee02
Partially-Implements: blueprint bay-drivers
2016-10-06 08:57:52 -05:00
Hieu LE 8f9eeb801a Centralize config option: cluster_heat section
Centralize config option of Cluster Heat section.
Replace oslo_conf cfg to magnum.conf.

Change-Id: I9118eeb17061a0aa26269ea9deaba28e79f28b76
Implements: blueprint centralize-config-magnum
2016-09-22 05:56:43 +00:00
Jaycen Grant 729c2d0ab4 Rename Bay DB, Object, and internal usage to Cluster
This is patch 3 of 3 to change the internal usage of the terms
Bay and BayModel.  This patch updates Bay to Cluster in DB and
Object as well as all the usages.  No functionality should be
changed by this patch, just naming and db updates.

Change-Id: Ife04b0f944ded03ca932d70e09e6766d09cf5d9f
Implements: blueprint rename-bay-to-cluster
2016-09-08 13:01:12 -07:00