Commit Graph

81 Commits

Author SHA1 Message Date
Michal Nasiadka bc79012f46 Drop Swarm support
Label validator function has been left behind, although it's not
checking for anything right now - might be useful in future.

Change-Id: I74c744dc957d73aef7556aff00837611dadbada7
2024-01-24 13:20:21 +13:00
Ghanshyam Mann 3b7a33eb64 [goal] Deprecate the JSON formatted policy file
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:

1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.

2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.

Also replace policy.json to policy.yaml ref from doc.

[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: Icfd9e2a75d8fdfb24cbd1c850f498aadee91f543
2021-03-08 15:11:01 +00:00
Feilong Wang 001b9c6101 Fix label fixed_network_cidr
Now the label `fixed_network_cidr` is not handled correctly, no matter
if the label is set, the default value '10.0.0.0/24' is used for
fixed network anyway. This patch fixes it and renamed it as
`fixed_subnet_cidr` to make less confusion. The new behaviour will be:
1. If the label `fixed_subnet_cidr` is set but no fixed subnet passed
   in, then a new subnet will be created with the given CIDR.
2. If a fixed subnet is passed in by user, then label `fixed_subnet_cidr`
   will be override with the CIDR from the given subnet.

Task: 39847
Story: 2007712

Change-Id: Id05e36696bf85297a556fcd959ed897fe47b7354
2020-06-11 13:54:59 +12:00
Theodoros Tsioutsias 44631afbbf ng-10: Fix cluster template conditions
Removes the role heat param from all templates. Instead and only for
k8s templates adds the master_role and worker_role params. The new
worker_only condition should be true for all roles except for master.
Finally, adds the missing is_cluster_stack param to all templates.

Change-Id: Ie0799373fe492c2e0a0cad903ed6e8c93e6266b5
2019-10-16 11:53:44 +00:00
Peter Pouliot 7dc4c7d904 Fixing typos and spelling errors in driver template files
* Changing the reference to 'Atomic' in the k8s_coreos_v1 driver
  to 'Container Linux'
* Changing a misspelled 'mater' to 'master'
  in swarm_fedora_atomic_v1/templates/swarmmaster.yaml description
* Changing a misspelled 'mater' to 'master' in
  swarm_fedora_atomic_v1/templates/swarmmaster.yaml description

Change-Id: I69a31d2b91fed48f07f649fa876e208e268fb339
2019-10-10 22:18:58 -04:00
Spyros Trigazis 2f72fdfbf6 Pass ssh public key as string
* Fedora CoreOS need the key to be passed as
  a string.
* We can adopt in all drivers so that users in
  the same project can do cluster resize.

story: 2005201
task: 36934

Change-Id: I9a18ce4dcbd74f0dcd23274baed7c8c3d2029d50
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
2019-10-08 07:56:52 +00:00
Theodoros Tsioutsias d4a52719f1 ng-7: Adapt parameter and output mappings
With this change parameter and output mappings can support multiple
stacks.

Change-Id: I45cf765977c7f5a92f28ae12c469b98435763163
2019-09-26 08:45:51 +00:00
Spyros Trigazis (strigazi) 9b1bd5da54 Add cluster upgrade to the API
To enable the rolling upgrade ability of Kubernetes Cluster, this
patch is proposing a new API /upgrade to support upgrade the
base operating system of nodes and the version of Kubernetes, even
add-ons running on the k8s cluster:

POST <ClusterID>/actions/upgrade

And the post body will be:

{
    "cluster_template": 'dd9cc5ed-3a2b-11e9-9233-fa163e46bcc2',
    "max_batch_size": 1,
    "nodegroup": "production_group"
}

Co-Authored-By: Feilong Wang <flwang@catalyst.net.nz>

Task: 30168
Story: 2002210

Change-Id: Ia168877778aa0d473383eb06b1c8a16dc06b0576
2019-06-07 12:01:10 +12:00
huang.xiangdong 3cb6226ff0 Support multi DNS server
Using comma delimited ipv4 address list to specify multi dns server
"8.8.8.8,114.114.114.114".

Task: 29465
Story: 2004994

Change-Id: I031247b0cc2ae417f18b2a5b9b3832e78ed9dafd
2019-04-08 23:08:45 +00:00
Lingxian Kong c47fde0cbe Improve floating IP allocation
- Never allocate floating IP for etcd service.
- Introduce a new label `master_lb_floating_ip_enabled` which controls
  if Magnum allocates floating IP for the master load balancer. This
  label only takes effect when the `master_lb_enabled` is set. The
  default value is the same with `floating_ip_enabled`.
- The `floating_ip_enabled` property now only controls if Magnum
  should allocate the floating IPs for the master and worker nodes.

Change-Id: I0a232406deaf112b0cb9e445735d7b49206c676d
Story: #2005153
Task: #29868
2019-03-20 18:44:45 +13:00
Feilong Wang 4f84c849f6 Add server group for cluster worker nodes
Now Magnums onlys has one server group for all master and worker nodes
per cluster, which is not very flexible for small cloud scale. For a
3+ master clusters, it's easily meeting the capacity when using hard
anti-affinity policy. This patch is proposing one server group for each
master and worker nodes group to have better flexibility.

story: 2004195

Change-Id: If11ba863a2aa538efe1e3e850084bdd33afd27d2
2019-02-27 09:09:20 +00:00
Jim Bach 8f4643d85c Make providing a keypair optional
A user may not rely on nova-keypairs to access their cluster
such as a preconfigured SSSD.

story: 2004402
task: 28035

Change-Id: I77fbdc174d3dddfd312fb8dac20516314d4c182e
2018-11-21 16:17:46 +00:00
Feilong Wang be0609ce88 Support soft-anti-affinity policy for nodes
Currently, there is no guarantee to make sure all nodes of one cluster are
created on different compute hosts. So it would be nice if we can create
a server group and set it with anti-affinity policy to get a better HA
for cluster. This patch is proposing to create a server group for master
and minion nodes with soft-anti-affinity policy by default.

Closes-Bug: #1737802

Change-Id: Icc7a73ef55296a58bf00719ca4d1cdcc304fab86
2018-01-24 07:13:48 +13:00
Spyros Trigazis 65dfb2009f Add openstack_ca_file configuration option
In the drivers section of magnum.conf add openstack_ca_file.
This file is expected to be a CA Certificate OR CA bundle
which will be passed on every node and it will be installed
on the host's CA bundle.

Update devstack plugin to use the ssl bundle if tls-proxy is
enabled.

Install the CA for drivers:
k8s_coreos_v1
k8s_fedora_atomic_v1
k8s_fedora_ironic_v1
mesos_ubuntu_v1
swarm_fedora_atomic_v1
swarm_fedora_atomic_v2

Add doc in troubleshooting-guide.

Add release notes.

Closes-Bug: #1580704
Partially-Implements: blueprint heat-agent
Change-Id: Id48fbea187da667a5e7334694c3ec17c8e2504db
2018-01-17 14:58:56 +00:00
Ricardo Rocha 28fff8006a Make docker_storage_driver a str instead of enum
Allow any value to be passed on the docker_storage_driver field by turning it
into a StringField (was EnumField), and remove the constraints limiting the
values to 'devicemapper' and 'overlay'.

Change the docker storage setup to have a generic setup for all drivers with
the exception of 'devicemapper', which keeps its own specific storage config
function. For all others, do the same we already did for overlay (with two
cases for usage of a cinder volume or not) and simply set the storage driver
in the docker configuration to the value provided in the cluster template.

Change-Id: I9aa8f232ce64ece4d439c0a476f463820a499617
Closes-Bug: #1722522
2017-12-14 14:41:09 +00:00
Kirsten G b07b6f34d5 Add verify_ca configuration parameter
Added configuration parameter, verify_ca, to magnum.conf with default
value of True. This parameter is passed to the heat templates to
indicate whether the cluster nodes validate the Certificate Authority
when making requests to the OpenStack APIs (Keystone, Magnum, Heat).
This configuration parameter can be set to False to disable CA
validation.

Co-Authored-By: Vijendar Komalla <vijendar.komalla@rackspace.com>

Change-Id: Iab02cb1338b811dac0c147378dbd0e63c83f0413
Partial-Bug: #1663757
2017-11-21 10:25:32 -08:00
Javier Castillo Alcíbar 8f74906435 [swarm-fedora-atomic] fix cluster etcd_lb protocol definition
In file drivers/swarm_fedora_atomic_v1/templates/cluster.yaml,
etcd_lb protocol is hardcoded:

  etcd_lb:
    type: ../../common/templates/lb.yaml
    properties:
      fixed_subnet: {get_attr: [network, fixed_subnet]}
      external_network: {get_param: external_network}
      protocol: HTTP
      port: 2379

but protocol should be: {get_param: loadbalancing_protocol}

Change-Id: Ib36ff0a99e219db9c0e0f35267faf317f5a14554
Closes-Bug: #1712054
2017-09-27 13:06:39 +02:00
Jenkins 62afc249d1 Merge "Clean-up server names in drivers" 2017-08-28 09:49:55 +00:00
Michael Tupitsyn 35dc923d66 Fix no_proxy evaluation for Swarm clusters
At the moment, no_proxy variable is evaluated separately for docker
daemon and for swarm-manager container running in docker. Evaluated
value for swarm-manager is not getting into cloud-init script, because
$NODE_PROXY token is getting replaced by Heat str_replace function.
This commit is intended to unify NO_PROXY evaluation and also fix the
issue with swarm-manager.

Related-Bug: #1647815
Related-Bug: #1632698
Related-Bug: #1660562
Change-Id: I336024265008b6cae308bf7b614476b71b81fa01
2017-08-23 08:26:19 +00:00
Mohammed Naser 5153d912b2
Clean-up server names in drivers
The instance type of servers at the moment can become quite long
due to the Heat autogenerated names.  This patch cleans up the names
so that they are shorter yet contain all the info needed to be able
to know where they belong to.

Change-Id: I5bcbe73f08844242d049b8408221da40d22cd3dc
2017-08-13 14:03:55 -04:00
coldmoment ba8ad5e37f Add a hacking rule for string interpolation at logging
String interpolation should be delayed to be handled
by the logging code, rather than being done at the point
of the logging call.
See the oslo i18n guideline
* https://docs.openstack.org/oslo.i18n/latest/user/guidelines.html#adding-variables-to-log-messages
and
* https://github.com/openstack-dev/hacking/blob/master/hacking/checks/other.py#L39

Change-Id: I8a4f5f896865aebbff88ee894f0081e58cfce9ef
2017-07-15 14:49:45 +08:00
Mark Goddard e4d691f48f Swarm: simplify heat WC signalling with $WAIT_CURL
This change uses the curl_cli attribute of heat's waitconditions in
the swarm driver which provides a preconstructed curl command which
can be used for signalling the waitcondition. This pattern has been
used elsewhere in magnum and simplifies the process of using wait
conditions.

Change-Id: I8e5f63e6d905266cc43d4957ce95e53659d01321
2017-06-19 12:19:57 +00:00
Spyros Trigazis 295ef945f6 swarm: Add docker_volume_type label
* add docker_volume_type for the cinder volumes which are
  used for docker storage.
* add default_docker_volume_type configuration option

Related-Bug: #1678153
Change-Id: Ie18096acf24873ef91a904df4f1a84694a2bb644
2017-06-01 16:17:54 +00:00
yuanpeng 71d25456d2 Remove log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.

See:
http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html
http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html

Change-Id: If1f4bd2f6be967368f52fb367c5a428d3eb58a9d
Closes-Bug:#1674551
2017-03-30 17:05:10 +08:00
Spyros Trigazis d2532a3af2 Upgrade to Fedora 25
Atomic image contains:
kubernetes-1.5.2-2.fc25.x86_64
docker-1.12.6-5.git037a2f5.fc25.x86_64
flannel-0.5.5-8.fc25.x86_64
etcd-3.0.15-1.fc25.x86_64

The ironic image contains exactly the same packages.

* For this upgrade the upstream image is used, which is
  uploaded here [1].
* Minor changes for flannel and docker-storage-setup
  were needed.
* The image will be built in the CI and uploaded to
  tarballs.openstack.org as soon as possible.
* Ironic image [2].

Notes:
* docker-storage-setup config changes were needed because in
  the previous images it was disabled and it was started by us.
* We can have selinux enables in containers since the images
  have kernel 4.9.x.

[1] https://fedorapeople.org/groups/magnum/fedora-atomic-25-latest.qcow2
[2] https://fedorapeople.org/groups/magnum/fedora-25-kubernetes-ironic.tar.gz

Change-Id: Iac6e30c530821a49a5c3978e335e0b1d56a576e0
2017-01-26 15:40:34 +01:00
Spyros Trigazis f997a332da Move monitors at driver level
Following changes for cluster-drivers, move coe specific monitors
at driver level. This change is needed to add the driver field
properly.

Change-Id: Id4658b8f7400bf3c86c8ff81756fb33d1211a0b3
Implements: blueprint bay-drivers
2017-01-23 13:33:37 +01:00
Mathieu Velten 22fb89a6e1 Make private network optional
If a fixed_network and fixed_subnet is specified no private network
is created by the templates and the specified network is
used instead for VMs provisioning, like in the Ironic driver.

Currently missing is the code to handle the use case where you
specify a fixed_network but not a fixed_subnet, this will come
in a following patch.

Partially Implements: blueprint decouple-private-network
Change-Id: I2003eb709b22b905063d846eb71570fc5e033618
2017-01-09 20:14:44 +01:00
Mathieu Velten 472af7fd8e Update Swarm version to 1.2.5
We should upgrade swarm because 1.0.0 is more than
one year old and lack features and bug fixes.

Change-Id: Ia236e473ce3152a6bf3115a3bfc62fbf0af100d7
Closes-Bug: 1644620
2016-12-16 10:29:52 +01:00
Jenkins d3277af787 Merge "Improve security for swarm" 2016-12-15 21:56:53 +00:00
Jenkins 43dabfff37 Merge "Consolidate heat network resources" 2016-12-15 16:53:16 +00:00
Spyros Trigazis 4dd178e7ca Add docker-d options in sysconfig/docker
Remove custom docker unit file and pass the necessary options
through /etc/sysconfig/docker file.

Change-Id: I6bf91843b9120b700d13aad54cef38342ae1f8bd
Closes-Bug: #1646123
2016-12-12 17:55:35 +01:00
Rajiv Kumar 10e85ee6ce Consolidate heat network resources
Currently for each driver has following code
  1) Create a fixed Network.
  2) Create a fixed subnet in the network created at step 1.
  3) Create a router
  4) Attach subnet(created at step2) to router(created at step 3)

A new resource is created for above tasks in network.yaml file.
New resource does the above tasks and output the fixed network ID
and fixed subnet id, which is used by other parts of the heat
template.

Change-Id: Ib347ce5c54c6566300a43e05b277bf80351a2256
Closes-Bug: #1606912
2016-12-09 15:34:06 +01:00
Jenkins 5488f668e1 Merge "Refactor driver interface (pt 1)" 2016-12-09 12:20:29 +00:00
Jenkins e600ee82fb Merge "Add a SELinux policy to relabel files in /usr/local/bin as bin_t" 2016-12-07 18:45:31 +00:00
Mathieu Velten 9c34f928e6 Add a SELinux policy to relabel files in /usr/local/bin as bin_t
There is a default policy for that in Fedora, however it doesn't
work in Atomic since /usr/local is a symlink to /var/usrlocal

Closes-Bug: 1646421
Change-Id: I4c5b836f4f76ff93a2c55f85ff6ff0cbe990bcff
2016-12-07 11:49:46 +01:00
Mathieu Velten ce3c68424a Factorize load balancer code into its own template
Heat types moved to lb.yaml:

Neutron::LBaaS::LoadBalancer
Neutron::LBaaS::Listener
Neutron::LBaaS::Pool
Neutron::LBaaS::HealthMonitor
FloatingIP on LBaaS

Closes-Bug: 1644454
Change-Id: I513b92c205ed97334033ba38ddf1d890ed61ff84
2016-12-06 11:18:27 +01:00
Randall Burt 7890725c52 Refactor driver interface (pt 1)
Refactor driver interface to encapsulate the orchestration
strategy. This first patch only refactors the main driver
operations. A follow-on will handle the state synchronization
and removing the poller from the conductor.

1. Make driver interface abstract
2. Move external cluster operations into driver interface
3. Make Heat-based driver abstract and update based on
   driver interface changes
4. Move Heat driver code into its own module
5. Update existing Heat drivers based on interface changes

Change-Id: Icfa72e27dc496862d950ac608885567c911f47f2
Partial-Blueprint: bp-driver-consolodation
2016-12-01 09:23:46 -06:00
Mathieu Velten 004e0cd742 Remove underscores from Nova server names
It creates a mismatch between the generated Nova name
and its hostname which can lead to weird problems.

Closes-Bug: 1645730
Change-Id: I1c7dd459caefacaf41dd77e59c1a6e1df3ef0d42
2016-11-29 16:54:30 +01:00
Rajiv Kumar 5b02a6090d Improve security for swarm
All traffic was allowed for swarm manager. With this patch
following secgroup is created for restricted access.

Security Group: secgroup_swarm_manager

  1) Allow TCP 22, 2376 ports for everyone.
  2) Allow all the ports to subnet created.
  3) Allow UDP 53 port for everyone.

Change-Id: Ie1aa4fffeb6317dc200a764319ac93e18d414a4b
Depends-On: I9ad6e0577918e811e9dd051b56aa69bfe2c391a0
Closes-bug: #1501050
2016-11-23 13:38:36 +05:30
Spyros Trigazis f82749457c Make cinder volume optional
In the swarm_atomic and k8s_atomic drivers container images are
stored in a dedicated cinder volume per cluster node. It is
proven that this architecture can be a scalability bottleneck.

Make the use of cinder volumes for container images and opt-in
option.  If docker-volume-size is not specified no cinder
volumes will be created. Before, if docker-volume-size wasn't
specified the default value was 25.

To use cinder volumes for container storage the user will
interact with magnum as before, (meaning the valid values are
integers starting from 1).

Closes-Bug: #1638006
Change-Id: I3394c62a43bbf950b7cf0b86a71b1d9b0481d68f
2016-11-10 11:13:09 +01:00
yatin 2d160ecfcb Support scheduler strategy for swarm cluster
Swarm cluster can be created by specifying any of the scheduler
strategy supported by swarm. The strategy can be specified
while creating cluster template using labels parameter, Ex:-
--labels swarm_strategy=spread
Supported values for swarm_strategy=spread, binpack, random

Change-Id: If471f10a3b1f955638a77d5afe462aebdeb4277c
Implements: blueprint add-support-different-strategy-in-swarmbay
2016-11-03 12:00:57 +05:30
Hieu LE 08a48895c4 Centralize config option: docker_registry section
Centralize config option of docker_registry section.
Replace oslo_conf cfg to magnum.conf.

Change-Id: I43d3ce068bb6638f71ea14577f34c1df3d7c9d8c
Implements: blueprint centralize-config-magnum
2016-10-11 11:05:40 +07:00
murali allada 104501cfe6 Cluster Drivers
- Dynamically load drivers using stevedore
- Changed the entry points to reference drivers instead of
  template definitions
- Implement Create and update driver operations

Change-Id: I5c3259404c796e1935c872cf3109ffecae3cee02
Partially-Implements: blueprint bay-drivers
2016-10-06 08:57:52 -05:00
Madhuri Kumari 9493a81d4c Split swarm atomic template
This patch splits the swarm atomic template to support
both swarm vm and bm drivers.

Change-Id: Ib03e1d6cb441230a17df2c47e1ed79052f3394bf
Partially-Implements: blueprint magnum-baremetal-full-support
2016-09-21 14:17:25 +05:30
Hua Wang 77e6c3160f Add Support of LBaaS v2 API
LBaaS v1 api is completely removed by neutron, so it
cannot be used now. Added Support of LBaaS v2 API.
Now all COE's uses LBaaS v2.

Co-Authored-By: yatin karel <yatin.karel@nectechnologies.in>
Change-Id: Idbccbe1065857449fc8e158115b7833b68c2da9f
Partially-Implements: blueprint magnum-lbaasv2-support
2016-09-09 10:01:30 +05:30
Jenkins 4548b310e5 Merge "Rename Bay DB, Object, and internal usage to Cluster" 2016-09-09 00:23:25 +00:00
Jaycen Grant 729c2d0ab4 Rename Bay DB, Object, and internal usage to Cluster
This is patch 3 of 3 to change the internal usage of the terms
Bay and BayModel.  This patch updates Bay to Cluster in DB and
Object as well as all the usages.  No functionality should be
changed by this patch, just naming and db updates.

Change-Id: Ife04b0f944ded03ca932d70e09e6766d09cf5d9f
Implements: blueprint rename-bay-to-cluster
2016-09-08 13:01:12 -07:00
Jenkins d0fd2c6be6 Merge "Add support for overlay networks in Swarm." 2016-09-08 01:25:40 +00:00
Mathieu Velten 7c0be2d384 Add support for overlay networks in Swarm.
Built-in in docker >= 1.9, it justs need a K-V store.
This patch reuses the existing etcd supporting Swarm.

Change-Id: Iae2e5aa1d8836d344f16ec911c766c426b1b558a
2016-09-07 12:09:51 +02:00
Jenkins 3837da3b06 Merge "Consolidate enable docker registery fragments" 2016-09-06 21:11:42 +00:00