Commit Graph

32 Commits

Author SHA1 Message Date
Michal Nasiadka 4eef6e8290 Add newer cluster-autoscaler versions to Docker Hub
Updated golang version since it's required for 1.26+
Removing builds for 1.24 and below - since those are long time
end of life
Bumping build job timeout to 3600s since it's timing out now

Change-Id: I1a40f10d438281c2cbd9a0c73c24d311f7c57b3f
2023-11-13 14:48:42 +00:00
Bartosz Bezak b73c6d25eb Update cluster autoscaler build to v1.23
* Add v1.23
* Update the go version in the build Dockerfile

Change-Id: Id11e5c4830a4538fd3e1869dddd56c977241aac3
2022-02-25 12:03:33 +01:00
Thomas George Hartland 6468aad3d1 Update cluster autoscaler build for v1.22
* Add v1.21/v1.22
* Update v1.18/v1.19 to newest point releases
* Drop v1.16/v1.17
* Update the go version in the build Dockerfile

Change-Id: If31d8027fa3541f6124777392539200dd68ae7af
2021-08-11 16:28:58 +02:00
Bharat Kunwar 1de7a6af5e [hca] Use fedora:rawhide now that greenlet 0.4.16 is released
In Change-Id: Ice75ae880925cd15c096eb6d1cdabf7f802bccde, we pinned
fedora:32 but now that greenlet 0.4.16 is released with support for
Python 3.9, we can unpin this.

Story: 2007264
Task: 39967

Change-Id: I841c78a942e87a25ef6ae04b24a1f7c73c48f096
2020-06-05 08:40:23 +00:00
Bharat Kunwar 2e1811861b [hca] Join threads before closing file descriptor
In some environments, heat container agent is erroring because of file
descriptor closing before threads have joined.

Change-Id: I1de5042ea5f4229518e96b985b1832dcacb052db
Story: 2007264
Task: 39788
2020-06-03 08:57:08 +00:00
Bharat Kunwar 81d0699c4c [hca] Pin fedora to 32 until new greenlet release
Eventlet used by many openstack packages depends on greenlet which does
not have a pip release supported by Python 3.9 (default Python version
on Fedora 33). Therefore, pin Fedora to version 32 until new greenlet
release is cut which includes the required fix [0].

Also update default heat_container_agent_tag to victoria-dev.

[0] https://github.com/python-greenlet/greenlet/pull/161

Change-Id: Ice75ae880925cd15c096eb6d1cdabf7f802bccde
Story: 2007264
Task: 39941
2020-06-03 08:55:30 +00:00
Zuul 07103112a1 Merge "[k8s] Build helm-client containers v2.16.6 and v3.2.0" 2020-04-29 11:48:55 +00:00
Bharat Kunwar 8a35cba25c [k8s] Build helm-client containers v2.16.6 and v3.2.0
Use buster-slim base image which is the latest stable [0].

[0] https://www.debian.org/releases/index.en.html

Story: 2007514
Task: 39525

Change-Id: I1ff8224cf064b7138f8868b2ac17710014f1e988
2020-04-29 10:10:21 +00:00
Spyros Trigazis b0de93d629 hca: Add hostname command
hostname is 28k and has no dependencies.
Add it as a convenient command.

Change-Id: I979431e849d208d257731b275d716dbffdb0845f
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
2020-04-24 19:12:54 +02:00
Bharat Kunwar 39ccad9cc4 Build new autoscaler containers
Versions: 1.16.5, 1.17.2 and 1.18.1

Story: 2007465
Task: 39152

Change-Id: Ie70e5ba38799c3da17ec9a34f6990f8c2d853732
2020-04-14 09:05:00 +00:00
Bharat Kunwar 3b94cd141b
[hca] Restore deploy_{stdout,stderr,status_code}
In I5504c00efce89105d403722d583bb75f7bdea714, we removed
deploy_stderr from the output and instead piped everything into
deploy_stdout. Turns out that this is not backward compatible and
removes it from the ouptut of:

    openstack software deployment output show <output_id> --long --all

This PS uses threading to write stdout and stderr live to a file and
correctly return deploy_status_code instead of None.

Story: 2007264
Task: 38983

Change-Id: I174e80c6982317f52150a4b255f3d1c592d9caaf
Signed-off-by: Bharat Kunwar <brtknr@bath.edu>
2020-03-10 12:06:23 +00:00
Bharat Kunwar 69c9b19516 [hca] Live log for SoftwareDeployment scripts
At present, when heat-container-agent is executing SoftwareDeployment
scripts, the output of this is not visible to the cluster administrator
until the execution is complete. This is an unhelpful behaviour, as it
is far more useful to see what is happening in real time. This change
logs output to files under /var/log/heat-config/heat-config-script/.

Also removes duplication of prefix for heat-container-agent container.

Story: 2007264
Task: 38632

Change-Id: I5504c00efce89105d403722d583bb75f7bdea714
Signed-off-by: Bharat Kunwar <brtknr@bath.edu>
2020-02-20 16:01:24 +00:00
Xinliang Liu 36cb39110a Fix heat-container-agent image building error on arm64
Fix bellow building error on arm64.
----------
  build/temp.linux-aarch64-3.8/_openssl.c:498:10: fatal error: openssl/opensslv.h: No such file or directory
    498 | #include <openssl/opensslv.h>
        |          ^~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for cryptography
  Running setup.py clean for cryptography
  Failed to build cryptography
  ERROR: Could not build wheels for cryptography
  which use PEP 517 and cannot be installed directly
----------

Additionally, add an ARCH ARG to pass other architecure value when
building image.
E.g: $ docker build --build-arg ARCH=`uname -m` -t TAGNAME .

task: 37823
story: 2007026

Change-Id: I7f62b882fa812beb74e38bbc5916d9df5afbd481
2019-12-17 07:34:04 +00:00
Bharat Kunwar 8f2a324984 Use --containerized flag to support 1.{13,14,15}.x in Atomic
The flag has been removed in Kubernetes version 1.16.x for which users
should use Podman but to continue to use Fedora Atomic without
use_podman=true which means using Docker 1.13.x, ServiceAccount tokens
cannot be propagated without using the --containerized flag when
use_podman=false.

This flag should not have been removed in
I3efd4e55e885b95721f13279b44dc1246e2fd2e4.

Story: 2006846
Task: 37434

Change-Id: I5ccef63de928ff01d10dc4cc500d0e1583eb0378
2019-11-21 20:45:55 +00:00
Spyros Trigazis 66ebe442c2 heat-agent: Check if scripts exists
When we start or restart the heat-agent, we run
configure_container_agent.sh which writes a few scripts. Make sure that
the scipts do not exist before writing to avoid overwriting any values
created on runtime.

When the heat-agent starts, /etc/os-collect-config.conf includes only
the reference to the os-refresh-config command. After the agent
bootstap, this file contains the credentials to check for software
deployments in the [heat] section. Before this patch, when the agent
restarted /etc/os-collect-config.conf was cleared resulting the agent to
stop working. I have the survive restarts, skiping only
os-collect-config.conf should be enough, but it is better to not touch
files on just service restart.

Additionally, fix file permissions for /etc/os-collect-config.conf.

Change heat-container-agent tag to ussuri-dev.

Change-Id: I3efd4e55e885b95721f13279b44dc1246e2fd2e4
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
2019-10-23 09:57:44 +02:00
Thomas Hartland 25c0f46169 Build cluster autoscaler container images
Add Dockerfile and CI config for building cluster autoscaler
container images specifically for magnum.
The autoscaler is built with the build tag "magnum" so that
only the magnum provider is included in the binary. This cuts
the size of the image in half compared to building with all
cloud providers.
The container-build job in .zuul.yaml has to have its timeout
increased, as the build time was already close to the timeout.

Change-Id: Iecbae5866278afe1687a4533b71af60fce537a4a
2019-10-16 08:05:23 +00:00
Andreas Jaeger 997497d8f4 Remove unneeded Zuul branch matcher
We have implicit branch matchers, so there's no need to add a check for
not-ocata, this job is only run for the branch it's on - like master
now.

Remove it to not confuse Zuul when multiple branches matches and the job
is different.

Also fix heat-container-agent to make linking of /usr/bin/python to python3
conditional on whether it already exists.

Change-Id: I1fec9d19cd14cf2fe2473b7610870e8d669021b9
2019-09-13 10:10:00 +00:00
Bharat Kunwar e47d6f6cfe Readable heat-container-agent log
The output of heat-container-agent has become unreadable. This patch
aims to address this in order to make debugging easier.

Additionally, this patch also adds missing dependencies in the most
recent fedora:rawhide (32) image.

Task: 36392
Story: 2006463

Change-Id: I54180b96357f6fa6d4044d818740ae70e036e435
2019-08-28 13:32:27 +00:00
Bharat Kunwar 06f78353f4 Fix heat-container-agent by setting LC_ALL=C
At the moment, the Python locale module expects `en_US.utf8` to be
present.  More recent fedora rawhides only come with `C`, `C.utf8` and
`POSIX` locale options unlike the older rawhides. The workaround is to
build the Dockerfile with environment variable `LC_ALL` set to `C`.  See
https://storyboard.openstack.org/#!/story/2006381#comment-141003 for a
longer description of the problem.

Change-Id: I412dd84f09dc217f2c9d974fe203c296b0710ef0
Story: 2006381
Task: 36184
2019-08-22 08:02:46 +00:00
Bharat Kunwar b99673f87c Disable gpg check in fedora:rawhide image
The heat-container-agent is currently failing to build due to misconfigured
upstream fedora:rawhide image. We can revert this change later.

Change-Id: I66723ae4329985c84a4549e44a4a7624927b3045
Story: 2006381
Task: 36184
2019-08-22 08:02:46 +00:00
Feilong Wang bd23e58825 Fix py3 issue of heat-container-agent
Now the default python version of rawhide has been upgraded to
python 3.7.4, this patch fixes those py2->py3 issues which followed
from the origin heat-agent commits[1][2].

[1] 25cd394bbe
[2] 73e2125532

Task: 35989
Story: 2006283

Change-Id: I23056513dcc6f0cb0c7d41aa529f6a2e77679db9
2019-07-30 12:07:00 +12:00
Spyros Trigazis 451358a57c heat-agent: Do not use absolute path
In a recent version of fedora base and
python/pip that comes with it, pip installed
pkgs put their binaries in /usr/local/bin
instead of /usr/bin. Since the lanuch script is
a bash script we do not have to use the absolute
path.

Change-Id: I9f31d047fe538114136d2199b447e3fe8248aa8c
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
2019-07-25 12:02:56 +02:00
Theodoros Tsioutsias 810e81195e Fix kubernetes systemd service templates
There are cases where systemd fails to restart kubernetes services.
This change fixes that by inceasing the restart interval in the
service templates.

Change-Id: I305644a78cee80db43b561e71a532feb414d8322
story: 2006265
task: 35956
2019-07-23 14:44:29 +00:00
Spyros Trigazis c9262419fd ci: Fix ADD_ALLOW_PRIV build-arg
In [0] we added a build-arg for the deprecaeted
--allow-priv option. This arg needs to be defined
after the FROM line in the dockerfile.

Note, other systems like podman can use the
arg even before the FROM statement. Docker needs
it after.

[0] I2935d34ace08800c805028f1673bc515f2f577e6

story: 2005124

Change-Id: I34af2451e92962b835ac0f1a1e49dfcbfd477830
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
2019-07-12 16:04:19 +02:00
Zuul 764cfe041d Merge "Add build-arg for --allow-privileged" 2019-07-05 08:18:09 +00:00
Jake Yip 35542713aa Make kubernetes apiserver start after network
Without this, the service fails to start with

 runc[1150]: error: unable to find suitable network address.error='no
 default routes found in "/proc/net/route" or "/proc/net/ipv6_route"'.
 Try to set the AdvertiseAddress directly or provide a valid BindAddress
 to fix this

Story: 2005698
Task: 31019
Change-Id: I0da4381cdc06f15815bd5ae51417aaf93d36b2c1
2019-06-24 22:20:48 +00:00
Spyros Trigazis fe0f0efa72 Add build-arg for --allow-privileged
https://github.com/kubernetes/kubernetes/pull/77820
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.15.md#node

story: 2005124

Change-Id: I2935d34ace08800c805028f1673bc515f2f577e6
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
2019-06-21 11:47:50 +02:00
Spyros Trigazis 0b5f4260d9 k8s_fedora: Deploy tiller
Add enable_tiller  label to install tiller in k8s_fedora_atomic
clusters. Defaults to false.

Add tiller_tag label to select the version of tiller. If the
tag is not set the tag that matches the helm client version in
the heat-agent will be picked.  The tiller image can be stored
in a private registry and the cluster can pull it using the
container_infra_prefix label.

Install tiller securely using helper container.

TODO:

*add instructions on how RBAC is designed
https://docs.helm.sh/using_helm/#example-deploy-tiller-in-a-namespace-restricted-to-deploying-resources-in-another-namespace
* add docs on how to install addon in the cluster using this tiller
* how users can get the creds to talk to tiller

NOTE:
The main goal of this tiller is internal usage!
Users can still deploy other tillers in other namespaces.

story: 2003902
task: 26780

Change-Id: I99d3a78085ba10030200f12bbfe58a72964e2326
Signed-off-by: dioguerra <dy090.guerra@gmail.com>
2019-02-11 11:18:08 +01:00
Spyros Trigazis e7efb49aac heat-agent: Add openssh-clients
Add openssh-client to the heat-agent to act on the
host.

story: 2002210
task: 29142

Change-Id: I6e52291e4fc750418c70a22cc386034fa729d765
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
2019-01-30 23:08:27 +00:00
Spyros Trigazis 60f9799374 ci: Rebuild kubernetes v1.11.6 containers
Change-Id: I531bf1fef6a87d8fbbf8f16c22c3db6a80797b69
2019-01-28 11:56:57 +01:00
Spyros Trigazis 60c996d2b4 Changes in container builder
* do not log the login command
* change dockerhub creds
* fix reference of tag in the push images task
* add retries
* remove repository parameter
* pull fedora from docker.io
* name docker_image tasks for each kubernetes and magnum images
* drop async logs

Change-Id: Iead202bdf9d7d42d0b7e21bea73a298678be714b
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
2018-12-15 22:28:49 +01:00
Spyros Trigazis 20d965c864 Build images in the ci
Use docker.io/openstackmagnumtest initially.

Change-Id: Idfb5ba636df2928e2e3e5248fdcaabd63160a4a4
Signed-off-by: Spyros Trigazis <spyridon.trigazis@cern.ch>
2018-12-14 16:56:40 +01:00