Commit Graph

40 Commits

Author SHA1 Message Date
Michal Nasiadka bc79012f46 Drop Swarm support
Label validator function has been left behind, although it's not
checking for anything right now - might be useful in future.

Change-Id: I74c744dc957d73aef7556aff00837611dadbada7
2024-01-24 13:20:21 +13:00
ricolin eca79453c0 Fix Trust token scope for drivers
This fix driver token scope to make sure we use correct token
scope from Trust.

Change-Id: If5b31951959c7a141dc1cae5fefcabe4ebf438b3
2023-07-25 17:00:40 +08:00
ricolin 6169eb26ed Fix pep8 gate
This fix propose two parts:
* introduce timeout (60s) to requests calls
* remove `file` scheme support for requests calls.

Change-Id: Ide2c2915ba5d6ff03933160b74f7206492276968
2023-03-14 09:17:54 +08:00
Erik Olof Gunnar Andersson daf34d9df8 Use oslo_serialization instead of the json module directly
* Always use oslo jsonutils.
* Consistently import jsonutils as-is.
* Use dump_as_bytes instead of dumps.
https://wiki.openstack.org/wiki/Python3#Serialization:_base64.2C_JSON.2C_etc.

Change-Id: I2b65faa7df43a1d58205e8ff106ff62f73d78198
2019-01-21 16:28:03 -08:00
Mohammed Naser cf5f78e5be Add iptables -P FORWARD ACCEPT unit
On node reboot, kubelet and kube-proxy set
iptables -P FORWARD DROP which doesn't work with
flannel in the way we use it.
Add a systemd unit to set the rule to ACCEPT after
flannel,docker,kubelet,kube-proxy.

Change-Id: I7f6200a4966fda1cc701749bf1f37ddc492390c5
Co-Authored-By: Spyros Trigazis <spyridon.trigazis@cern.ch>
2018-11-22 16:50:21 -05:00
Tobias Urdin 095b49e6f5 [swarm-mode] Remove --live-restore from Docker daemon options
Ensure the --live-restore is not in the Docker daemon OPTIONS.

Some images has this option by default which will cause the node
not being able to perform it swarm init process.

Change-Id: I287a5274143903fad5d4476e9d1640b26bdb46d4
Story: 2004095
Task: 27497
2018-10-16 17:22:41 +00:00
Bharat Kunwar b7bfee5d27 Support disabling floating IPs in swarm mode
We use the same technique that is used for kubernetes clusters, with a
custom heat resource that provides either a floating IP, or
OS::Heat::None when disabled. We also add coverage tests for swarm-mode.

Change-Id: I3b5877bcd89fc2436776f49e479ffadf72c00ea3
Story: 1772433
Task: 21662
Task: 22102
Co-authored-by: Mark Goddard <mark@stackhpc.com>
2018-07-10 09:27:07 +00:00
Kirsten G b07b6f34d5 Add verify_ca configuration parameter
Added configuration parameter, verify_ca, to magnum.conf with default
value of True. This parameter is passed to the heat templates to
indicate whether the cluster nodes validate the Certificate Authority
when making requests to the OpenStack APIs (Keystone, Magnum, Heat).
This configuration parameter can be set to False to disable CA
validation.

Co-Authored-By: Vijendar Komalla <vijendar.komalla@rackspace.com>

Change-Id: Iab02cb1338b811dac0c147378dbd0e63c83f0413
Partial-Bug: #1663757
2017-11-21 10:25:32 -08:00
Javier Castillo Alcíbar 010a2673d4 Swarm: Incorrect reference to Flannel variables
Swarm network-config-service makes reference to
wrong variables:
- FLANNEL_ETCD instead of FLANNEL_ETCD_ENDPOINTS
- FLANNEL_ETCD_KEY instead of FLANNEL_ETCD_PREFIX

Change-Id: Ie9f1913177f4a5fd803bcf8de1de66384395ecdd
Closes-Bug: #1679301
2017-09-27 13:13:55 +02:00
Michael Tupitsyn 35dc923d66 Fix no_proxy evaluation for Swarm clusters
At the moment, no_proxy variable is evaluated separately for docker
daemon and for swarm-manager container running in docker. Evaluated
value for swarm-manager is not getting into cloud-init script, because
$NODE_PROXY token is getting replaced by Heat str_replace function.
This commit is intended to unify NO_PROXY evaluation and also fix the
issue with swarm-manager.

Related-Bug: #1647815
Related-Bug: #1632698
Related-Bug: #1660562
Change-Id: I336024265008b6cae308bf7b614476b71b81fa01
2017-08-23 08:26:19 +00:00
Spyros Trigazis 4fb91cc109 tests: Use swarm-mode for api tests
* Swarm-mode is the fastest cluster to deploy since it doesn't
  require to pull anything from outside.
* Add the output nodes for swarm-mode too.
* Disable copy logs (I think a better practice is to copy logs
  on demand).
* Don't run test_create_list_sign_delete_clusters, because it is
  very unstable on the CI.

Partially-Implements: blueprint swarm-mode-support

2nd commit message:

Update to Fedora Atomic 26

This patch moves the current master to test against Fedora Atomic 26,
in addition, it switches to downloading from Fedora mirrors.

2nd-Change-Id: I9a97c0eb78b2c9d10e8be1501babb19e73ee70c1

3rd commit message:

Set default iptables FORWARD policy to ACCEPT

With the release of Docker 1.13 which is available in Fedora
Atomic 26, it no longer sets the policy of the FORWARD chain
to ACCEPT[1].  Therefore, CNI networking such as Flannel will
cease to work.

This patch sets the policy to ACCEPT so that traffic can work
once again for deployments which are based on Docker versions
which are newer than 1.13

[1]: https://github.com/moby/moby/pull/28257

3rd-Change-Id: I1457602748619f38f87542fc01a2996ee80e58b7
Closes-Bug: #1708454

Co-Authored-By: Mohammed Naser <mnaser@vexxhost.com>
Change-Id: I86d4dcc94fff622be4ee2acc8dd60ed81bc5d433
2017-08-21 05:30:09 +00:00
Mark Goddard e4d691f48f Swarm: simplify heat WC signalling with $WAIT_CURL
This change uses the curl_cli attribute of heat's waitconditions in
the swarm driver which provides a preconstructed curl command which
can be used for signalling the waitcondition. This pattern has been
used elsewhere in magnum and simplifies the process of using wait
conditions.

Change-Id: I8e5f63e6d905266cc43d4957ce95e53659d01321
2017-06-19 12:19:57 +00:00
Mark Goddard 0dc463e391 Use lowercase keys for swarm waitcondition signal
The heat waitcondition signal API accepts status, reason, data and id
fields in a JSON object supplied as POST data. Missing fields will be
filled with defaults. Previously, the swarm script fragments used a
capitalised form of these keys (Status, Reason, Data, Id) which was
not being recognised by heat. This caused failures to not be reported.

This change uses the correct lowercase names for these fields and also
fixes some quoting and incorrect use of UUIDs provided as the id field.

Change-Id: I9bfe36e5dd956280eaa42d1c3f1620c4ec27bc0c
Closes-Bug: #1504059
2017-06-16 10:54:50 +01:00
Jenkins 0dee921e6e Merge "Enable custom keystone endpoint_type in templates" 2017-05-10 16:40:04 +00:00
Mathieu Velten ad94578a2f Fix rexray systemd unit
* remove existing rexray containers in ExecStartPre
* set volume tag to rshared
* fix indentation

Closes-Bug: #1686421
Change-Id: I71ffd708baac0403dae7d8f38a073240c44e0434
2017-05-09 12:23:39 +02:00
Kevin Lefevre 4c241a683f Enable custom keystone endpoint_type in templates
Allow to specify a custom AUTH_URL for the templates in case instances
cannot reach internalURL which is the case in mose deployment.

A new variable in trust section: trustee_keystone_interface which
default to public is introduced.

Change-Id: I2a908c0752387e4ff4ad2b0fdf0c1025a73ce806
Closes-Bug: #1643197
2017-05-01 18:15:58 +02:00
Spyros Trigazis c5a4a18104 Update Fedora images
New release of Fedora Atomic [1].
The new release of Fedora Ironic includes the same
packages.

Main changes:
Kubernetes 1.5.3
etcd 3.1.3
Plus several fixes and version bumps.

Add :Z when mounting certs in the swarm containers to set
selinux labels properly.

[1] http://www.projectatomic.io/blog/2017/03/fedora_atomic_mar28/

Closes-Bug: #1677664

Change-Id: I2539ae83401db5b34716ebd4bbdfbe288f5c768b
2017-04-03 08:52:12 +02:00
Jenkins ed173776ca Merge "Fix CVE-2016-7404" 2017-02-22 20:58:44 +00:00
Johannes Grassler e93d82e8b3 Fix CVE-2016-7404
This commit addresses multiple potential vulnerabilities in
Magnum. It makes the following changes:

* Permissions for /etc/sysconfig/heat-params inside Magnum
  created instances are tightened to 0600 (used to be 0755).
* Certificate retrieval is modified to work without the need
  for a Keystone trust.
* The cluster's Keystone trust id is only passed into
  instances for clusters where that is actually needed. This
  prevents the trustee user from consuming the trust in cases
  where it is not needed.
* The configuration setting trust/cluster_user_trust (False by
  default) is introduced. It needs to be explicitely enabled
  by the cloud operator to allow clusters that need the
  trust_id to be passed into instances to work. Without this
  setting, attempts to create such clusters will fail.

Please note, that none of these changes apply to existing
clusters. They will have to be deleted and rebuilt to benefit
from these changes.

Change-Id: I643d408cde0d6e30812cf6429fb7118184793400
2017-02-09 16:44:27 +01:00
Kevin Lefevre a24de0a536 Use right no proxy settings for swarm master and agent
Change-Id: I6a920a189264bf4c306be44399caa25537d4a0ce
Closes-Bug: 1660562
2017-01-31 10:21:20 +01:00
Jenkins 25aa5b06ff Merge "Use UUID instead of "00000" for UniqueId" 2017-01-30 15:24:40 +00:00
Jason Dunsmore fbfdbec60d Pass OpenStack-API-Version header in make-cert scripts
Otherwise, the magnum certificates API will return a 406 Not
Acceptable error.

Change-Id: I0d59bf71b62bdd4204cd32d26ef3f2fc30f8f180
Closes-Bug: #1659423
2017-01-26 20:27:22 +00:00
Spyros Trigazis d2532a3af2 Upgrade to Fedora 25
Atomic image contains:
kubernetes-1.5.2-2.fc25.x86_64
docker-1.12.6-5.git037a2f5.fc25.x86_64
flannel-0.5.5-8.fc25.x86_64
etcd-3.0.15-1.fc25.x86_64

The ironic image contains exactly the same packages.

* For this upgrade the upstream image is used, which is
  uploaded here [1].
* Minor changes for flannel and docker-storage-setup
  were needed.
* The image will be built in the CI and uploaded to
  tarballs.openstack.org as soon as possible.
* Ironic image [2].

Notes:
* docker-storage-setup config changes were needed because in
  the previous images it was disabled and it was started by us.
* We can have selinux enables in containers since the images
  have kernel 4.9.x.

[1] https://fedorapeople.org/groups/magnum/fedora-atomic-25-latest.qcow2
[2] https://fedorapeople.org/groups/magnum/fedora-25-kubernetes-ironic.tar.gz

Change-Id: Iac6e30c530821a49a5c3978e335e0b1d56a576e0
2017-01-26 15:40:34 +01:00
Spyros Trigazis 7c4ef12fd4 [swarm] Fix cert filename in swarm-agent service
In swarm nodes the docker certs are named server.crt and server.key.
Replace filenames in swarm-agent service from client to server.

Change-Id: Ic3bc228d98c3829b583403156d8ad3ad4939037a
PArtially-Implements: blueprint secure-etcd-cluster-coe
2017-01-15 07:42:16 +00:00
yatin ffb751d638 [swarm] Enable TLS in Etcd cluster
With this patch following are done:-
- Configure Etcd with TLS support

Configure Following to commuicate with TLS enabled Etcd:-
- Swarm manager
- Swarm agent
- Docker
- Flannel

Etcd also listens at http://127.0.0.1:2379,
so on master nodes etcdctl can be used without certificates.

if TLS_DISABLED="True" then no TLS is enabled for etcd.

Change-Id: I6cadfebcfaaaf7ac7a7660b377b7d96748f0f9f0
Partially-Implements: blueprint secure-etcd-cluster-coe
2017-01-11 21:33:38 +05:30
ArchiFleKs 8d7bc9c99e Make Docker proxy configuration consistent across template
Closes-Bug: #1647815
Related-Bug: #1632698
Change-Id: Ic5e5cadd3d912f01072eff427a1095309c4f6e9b
2016-12-20 10:08:42 +01:00
PanFengyun c489a5d47e Use UUID instead of "00000" for UniqueId
The swarm bay should pass specified "UniqueId" to the resource of
OS::Heat::WaitConditionHandle, but the "UniqueId" is "00000" in
the templates of swarm. So let's use UUID instead of "00000".
In addition, "UniqueID" seems to be obsolete, Use "Id" instead.

Change-Id: I86739db4a2e6faf93d55fe4998bada110de118c6
Closes-Bug: #1606486
2016-12-16 20:06:48 +08:00
Spyros Trigazis 4dd178e7ca Add docker-d options in sysconfig/docker
Remove custom docker unit file and pass the necessary options
through /etc/sysconfig/docker file.

Change-Id: I6bf91843b9120b700d13aad54cef38342ae1f8bd
Closes-Bug: #1646123
2016-12-12 17:55:35 +01:00
Jenkins e600ee82fb Merge "Add a SELinux policy to relabel files in /usr/local/bin as bin_t" 2016-12-07 18:45:31 +00:00
Dirk Mueller 80fc5a2d42 Add bashate checks to pep8 step
Similarly to pep8 checks, this allows enforcing a consistent
style of the shell scripts accross modfications. For now
only the indentation is enforced to reduce code churn.

Closes-Bug: 1648099
Change-Id: Ie66cbe1aea4bd01a8bba8833ef6cbd2cff6a7c6a
2016-12-07 15:25:41 +01:00
Mathieu Velten 9c34f928e6 Add a SELinux policy to relabel files in /usr/local/bin as bin_t
There is a default policy for that in Fedora, however it doesn't
work in Atomic since /usr/local is a symlink to /var/usrlocal

Closes-Bug: 1646421
Change-Id: I4c5b836f4f76ff93a2c55f85ff6ff0cbe990bcff
2016-12-07 11:49:46 +01:00
Jenkins c47bc5fcae Merge "Add docker daemon systemd proxy variables" 2016-11-18 18:02:29 +00:00
Jenkins 60b7724609 Merge "Using sys.exit(main()) instead of main()" 2016-11-18 04:45:38 +00:00
Spyros Trigazis f82749457c Make cinder volume optional
In the swarm_atomic and k8s_atomic drivers container images are
stored in a dedicated cinder volume per cluster node. It is
proven that this architecture can be a scalability bottleneck.

Make the use of cinder volumes for container images and opt-in
option.  If docker-volume-size is not specified no cinder
volumes will be created. Before, if docker-volume-size wasn't
specified the default value was 25.

To use cinder volumes for container storage the user will
interact with magnum as before, (meaning the valid values are
integers starting from 1).

Closes-Bug: #1638006
Change-Id: I3394c62a43bbf950b7cf0b86a71b1d9b0481d68f
2016-11-10 11:13:09 +01:00
Vijendar Komalla 490deb256e Restart swarm infra containers if deleted
Currently a user can accidentally delete swarm infra
conatiners (swarm-manager, swarm-agent). This change is
to restart infra containers if they were deleted/killed.

Change-Id: I4640dfb3dbb4bb6684da86998424936d3128eade
Closes-Bug: #1640312
2016-11-08 15:49:44 -06:00
yatin 2d160ecfcb Support scheduler strategy for swarm cluster
Swarm cluster can be created by specifying any of the scheduler
strategy supported by swarm. The strategy can be specified
while creating cluster template using labels parameter, Ex:-
--labels swarm_strategy=spread
Supported values for swarm_strategy=spread, binpack, random

Change-Id: If471f10a3b1f955638a77d5afe462aebdeb4277c
Implements: blueprint add-support-different-strategy-in-swarmbay
2016-11-03 12:00:57 +05:30
ArchiFleKs 2e9c364eec Add docker daemon systemd proxy variables
add HTTPS_PROXY and NO_PROXY to docker.service.d

Change-Id: I9ab02773695fef14256ca79e68a3d3d1e52c3ff7
Closes-Bug: #1632698
2016-10-13 14:38:22 +02:00
Abhishek Chanda 4c0850731d Disable cert checks while talking to endpoints
A lot of deployments use self signed certs. Curl breaks in those
cases trying to validate certs against known set of CAs

Change-Id: Ib36f9a99a91ce2c4d2141421ab7295303ead716f
2016-10-06 19:31:45 -07:00
Luong Anh Tuan 3418767adc Using sys.exit(main()) instead of main()
TrivialFix: Similar [1] in Kolla project
As we known, Exceptions are raised by the sys.exit() function. When they
are not handled, no stack traceback is printed in the Python interpreter.
Therefore, when using sys.exit(main()) instead of main()
may be more readable and reasonable.

[1] https://review.openstack.org/#/c/349353/

Change-Id: Iad395100505c70da11c825ff8f3f5787db07ca44
2016-09-26 10:37:38 +07:00
Madhuri Kumari 9493a81d4c Split swarm atomic template
This patch splits the swarm atomic template to support
both swarm vm and bm drivers.

Change-Id: Ib03e1d6cb441230a17df2c47e1ed79052f3394bf
Partially-Implements: blueprint magnum-baremetal-full-support
2016-09-21 14:17:25 +05:30