Commit Graph

43 Commits

Author SHA1 Message Date
Sergiy Markin d21e3434f6 Airflow stable 2.8.2
This PS bumps up Airflow version to the latest
2.8.2 and also bumps up openstack dependences to
Antelope 2023.1

Change-Id: Iae72c6da9406749cf157437495f31dc3b9f6ba2c
2024-02-26 21:01:52 +00:00
Sergiy Markin 1a0ea41f28 Airflow stable 2.8.1
Change-Id: I49093c3c5bbdf076ef9f2ffa39769f9a2ec55c5c
2024-02-22 21:09:21 +00:00
Sergiy Markin 69a74590e7 Airflow stable 2.6.2
This PS updates python modules and code to match Airflow 2.6.2:

- bionic py36 gates  were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfoemed based on
  airflow-2.6.2 constraints

Change-Id: I9c3e139b3437414a61af7e7c0b7d7e533fadefda
2023-08-29 21:12:11 +00:00
Sergiy Markin 2f2455f0d1 Restored ubuntu_bionic image build
This PS restores image build for ubuntu_bionic and adds appropriate
gates to keep it tested by appropriate functional and integrational
tests.

Change-Id: I8ef524a1f66f1a88593a18334b8c37c4db0175c6
2023-05-25 22:53:15 +00:00
Sergiy Markin 45bcba068e Promenade timeout adjustment
This PS set read/connection timeout to None for http requests.

Change-Id: Ia2d8b59a5fa55bac24575149c8c1e8a67707c29d
2023-05-12 16:19:33 +00:00
Sergiy Markin 2e8f6642f4 Deckhand module uplift
This PS has most recent commitid of Deckhand project

Change-Id: Ie77a6f365b99c6f7d154811e63e722d3f63e0f9e
2023-05-09 20:26:08 +00:00
Sergiy Markin 97e3721a03 Sync requirements with shipyard
This PS adjusts list of dependences needed to get syncronized with Shipyard project:

- lock sphinx with 3.3.1 version for doc generation
- updated deckhand reference
- adjusted other python dependences

Change-Id: I5b0a60a2c0709a37d65cb8258bf8c79631c94f00
2023-04-27 19:47:55 +00:00
Sergiy Markin 32ad8a96b0 [focal] Python modules sync with Airship project
- uplifted/downgraded some python modules
- fixed falcon.API deprecation - -> falcon.App
- uplifted deckhand reference for python deps
- fixed formatting style  using yapf linter
- added bindep role and bindep.txt file with required deps
- fixed quai docker image publishing
- re-enabled openstack-tox-py38 gate job

Change-Id: I0e248182efad75630721a1291bc86a5edc79c22a
2023-04-21 06:09:14 +00:00
SPEARS, DUSTIN (ds443n) 27a8b0d798 k8s upgrade to 1.26.0
upgrades kubernetes client to v1.26.0
remove installation of containerd during genesis.sh to prevent containerd downgrade
update bitnami kubectl image to image with curl installed for readiness check

Change-Id: I3afd5a7e7211bae3f52263167a62a012da0619a0
2023-03-20 13:16:48 -04:00
Wahlstedt, Walter (ww229g) 8ce937a9f7 updates for focal
add focal dockerfile
update zuul jobs for focal
update tox for tox4 changes
update all requirements to latest and match deckhand
update cfssl from R1.2 to v1.6.3
fixed local gates for focal
updated examples promenade manifests to run on focal

Change-Id: I2af4043784766d36588c6f738053ad66e7b89a90
2023-02-27 12:11:07 -05:00
Ruslan Aliev c10165c144 K8S upgrade 1.24
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: Iaa0c5f57ac621f2b91f525da423db0acd9d8ea99
2022-09-14 19:34:02 -05:00
Ruslan Aliev e207bbe966 k8s upgrade to v1.23.7
Address changes and deprecations in Kubernetes v1.21=>v1.23

controller-manager:
* --authorization-kubeconfig and --authentication-kubeconfig must be set
* liveness/readiness probes must use HTTPS
* the default port has been changed to 10257

kubelet:
* --dynamic-config-dir has been deprecated, will not move to GA
* --cni-bin-dir has been deprecated, will be removed with dockershim
* --cni-conf-dir has been deprecated, will be removed with dockershim
* --network-plugin has been deprecated, will be removed with dockershim

https: //github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md#deprecation
https: //kubernetes.io/docs/tasks/administer-cluster/reconfigure-kubelet/
https: //github.com/kubernetes/enhancements/tree/master/keps/sig-node/281-dynamic-kubelet-configuration
Change-Id: Ia996d7c14d81d1d8b8067f11c02ffb4ce90eb49a
2022-06-29 00:21:45 -05:00
Phil Sphicas 59e1f8af83 Update Python kubernetes-client version
Change-Id: Ia7a26f1086af0882c9ef666e759d87252a4331db
2022-03-21 08:32:39 -07:00
Phil Sphicas 7365da32e5 Use jsonschema 3.2.0
Change-Id: I4d61ad393445cc708d6aa6f8e39201e4e59ad416
2021-09-09 14:29:01 +00:00
Phil Sphicas c7e72942a9 Remove hyperkube extraction functionality
The extraction of the monolithic hyperkube binary from its container
image to be used as kubelet was last relevant in Kubernetes 1.16. Since
then, the hyperkube image has been deprecated, the structure of the
image has been changed, and it has ultimately been eliminated in
Kubernetes 1.19.

This change cleans up promenade accordingly.

Reverts the following commits:
* 886007b New CLI option to extract hyperkube
* 32a6c15 hyperkube image in promenade init
* 955deed New source for hyperkube binary definition

Change-Id: Ib62ecdf1af13abe8202a4ba4f86c39b9042ed13f
2021-02-11 17:23:32 +00:00
Rick Bartra eb748a75f3 Update setuptools to resolve dependency conflict
When using pip 20.3, pip package conflicts must be resolved. This is in
regards to the following conflicts:

google-auth 1.22.1 requires setuptools>=40.3.0, but you'll have setuptools 20.7.0 which is incompatible.
kubernetes 11.0.0 requires setuptools>=21.0.0, but you'll have setuptools 20.7.0 which is incompatible.

Change-Id: Ia0a99626b14773c3ed8981aef2ae46ffe5a4b632
2020-10-15 17:57:25 -04:00
Rick Bartra 8513b46153 Update pip package versions in preparation of pip 20.3
When pip is upgraded to 20.3, the pip dependency resolver is much more
strict and will no longer install a combination of packages that is mutually
inconsistent[0].

These changes account for the fact that Shipyard imports Armada, Drydock,
Promenade, and Deckhand. Having said that, with pip 20.3, the pip
packages amongst those projects cannot conflict. A follow-up change may
be needed if more conflicts are found.

Change-Id: Ie6effbdae759158e19b0b0adb2bdac0396eab047
2020-09-28 16:34:04 -04:00
Phil Sphicas b41c3151fb Accelerate YAML operations with LibYAML
Patch PyYAML (via the pylibyaml library) to automatically enable the
LibYAML parser and emitter, which are faster than the Python versions.

https://pypi.org/project/pylibyaml/

Change-Id: Iad54bfd21083b24cad5429bd8ecf794a9ead513e
2020-09-24 04:27:29 +00:00
Mahmoudi, Ahmad (am495p) c302a083a6 Upgrade k8s from v1.17.3 to v1.18.6
This ps makes following changes to upgrade kubernetes from v1.17.3
to v1.18.6.
  - Updated all references to k8s images to 1.18.6
  - Updated command options and api object and versions based on
    k8s 1.18 release notes:
      https://kubernetes.io/docs/setup/release/notes/
  - Uplifted uwsgi to 2.0.19.1 to align with other airship
    components, and to bring in fixes and improvements.
  - Added build-essentials and python3-dev packages to pass the zull
    gate, which was looking for a c compiler.

Change-Id: I1160d1e6e2f02a0524043641b9296ea39edb301e
2020-08-19 15:56:45 +00:00
Alexander Hughes b1f00852e3 Uplift python deps to match deckhand
Pegleg is a consumer of both Promenade and Deckhand, these projects
have conflicting versions of multiple Python requirements.

This patch:
- Uplifts Promenade to more recent versions of these dependencies to
match Deckhand.
- Applies `tox -e freeze` to uplift requirements-frozen.txt

Locally these changes can be tested to ensure compatability with
Pegleg by following these steps:
1. Checkout this patch locally
2. Clone Pegleg to local disc
3. Update Pegleg's Pipfile, specifically changing line 30:
   promenade = {file = "/your/path/to/local/promenade"}
4. Run tox -e update-requirements from Pegleg

If these steps are successful you will see the following output:
  update-requirements: commands succeeded
  congratulations :)

Change-Id: Ifdc74c4f80f599058e8b56008e234324a6d89e49
Signed-off-by: Alexander Hughes <Alexander.Hughes@pm.me>
2020-06-08 14:50:51 -04:00
Ian H Pittwood adc5146ffb Upgrade mismatch dependencies for pegleg
Upgrades requests 2.20->2.22 to fix a security vulnerability.

Upgrades PyYAML 3.12->~5.1 to fix incompatibility with the version used
by pegleg. This upgrade should not affect any yaml references currently
used in promenade.

Upgrades deckhand to match newer version used by pegleg.

Change-Id: Ie0ccfb36c4942ce0c782c3d2ffac70919c4c24f6
2019-09-20 15:23:21 +00:00
Ian H. Pittwood 73704e4456 Pull requirements-direct into requirements
Promenade currently does not define a requirements.txt, meaning that any
pip installations of Promenade do not install any of Promenade's
dependencies. This change adds a link from requirements.txt to the
contents of requirements-direct.txt. This allows requirements.txt to be
defined without immediately altering Promenade's current method of
dependency management.

In the future, Promenade could remove requirements-direct.txt and
requirements-frozen.txt to only utilize a carefully pinned
requirements.txt as many other Airship projects do.

Change-Id: If7e80f868322cd0285b146b5627a8ddf3fde9663
2019-08-26 13:52:04 +00:00
HUGHES, ALEXANDER (ah8742) 6fd3b955a0 Update packages related to requests
Pegleg Promenade Deckhand and Shipyard should all use same versions
of packages when able.  Requests currently is giving a warning
in Pegleg:
ERROR: deckhand 0.0.1.dev657 has requirement urllib3==1.24, but
you'll have urllib3 1.24.3 which is incompatible.

Change-Id: Icdd5c687bb7072bd479107fa9a6f80a0fe1284e9
2019-07-31 15:20:41 +00:00
Sean Eagan 754b543107 Uplift deckhand dependency
Uplift deckhand dependency to include support for v2 schemas [0].

[0]: https://review.opendev.org/#/c/666659/

Change-Id: I0e4cc96a03b9e58a946cb3570e9027e85507f970
2019-07-01 15:29:47 -05:00
Egorov, Stanislav (se6518) 955deeda41 New source for hyperkube binary definition
Now it's possible to use hyperkube Docker image to extract hyperkube binary.
Use case for this feature is kubelet/kubectl delivery in one binary(hyperkube)
which is built into Docker image. Promenade will extract hyperkube from Docker image,
create symlinks for kubelet/kubectl pointed to hyperkube. To do so promenade container
need to be configured to use Docker on the host where this container will be created.
This is happening only for script generation for genesis node. Later when promenade
will be started as a service pod inside ucp cluster it will generate scripts for joining nodes
by using cached hyperkube from /tmp.

Old way to delivery kubelet from tarball is still supported.

Configuration for the new method.

Need to export environment variables to properly configure Docker in Docker.
Docker socket should be provided as a mounted file inside promenade.
Also need to set temporary permissions for this socket during the build scripts stage.

Example:
DOCKER_SOCK="/var/run/docker.sock"
sudo chmod o+rw $DOCKER_SOCK
export DOCKER_HOST="unix:/${DOCKER_SOCK}"
export PROMENADE_TMP="abs_path_tmp_dir_on_host"
export PROMENADE_TMP_LOCAL="tmp_dir_inside_container"

After genesis scripts generation Docker socket permission should be turned back:
sudo chmod o-rw $DOCKER_SOCK

Change-Id: Ida22ea934fc551fec34df162d8147c8b9e630330
2019-06-06 10:30:29 -07:00
Crank, Daniel (dc6350) 9b62a49eae Update Deckhand version
A recent Promenade change [0] introduced user-context tracing support in
Promenade, which requires the passing of an additional parameter to the
Deckhand API client. The Deckhand client was updated to handle the
additional parameter [1], but the Deckhand version Promenade uses was
not updated. This change uplifts the version of Deckhand in
requirements.txt for Promenade to a version that supports the
aforementioned, additional, user-context parameter.

[0] https://review.openstack.org/634071
[1] https://review.openstack.org/634068

Change-Id: Iaea328b32f95262e9e81be957893b7a50ac295fa
2019-04-18 11:28:13 -05:00
Crank, Daniel (dc6350) d8c87f3f7b Log client-id in UCP API endpoints
Adds functionality to read context marker and end-user
from request headers and log that information where
available, to aid in tracing transactions that span
multiple Airship components.

Change-Id: I31c26b269ef6db9ebf44eb381eb6745ce40fc4ea
2019-04-05 09:55:15 +00:00
Bryan Strassner 6359f7138c Newer Deckhand engine
Updates Promenade to use a newer Deckhand version

Change-Id: I240b12093a9ef6da26d04eaffb79d834f513e537
2018-11-06 14:20:49 -06:00
Scott Hussey fec9b14376 Uplift deckhand dependency
- Uplift deckhand dependency to gain support of recursive document
  substitution.

Change-Id: Iccaee6d750c2b351ddcb0dcc5321d628778a8e98
2018-10-23 09:43:04 -05:00
Mark Burnett 28c6cbbab7 Uplift Deckhand version
Change-Id: Ia006d45fefb9e02ded4fbea6582596a9fe5579eb
2018-09-21 14:25:12 -05:00
Scott Hussey 358cd882ad (fix) Update deckhand dependency
Resolve an issue on Deckhand schema validation

Change-Id: I271f07e704dd2ee61482ad1c86f7b465b7f0c32c
2018-07-17 13:57:02 -05:00
Mark Burnett 9cb2c5a5ca Add local file cache for Builder tarball fetch
Change-Id: I4741b50c4a4b505f4f70ff2cbc5e9689b9f28b7b
2018-06-25 07:51:44 -05:00
Scott Hussey a1640349d7 Update Deckhand commit
- Uplift deckhand commit to include fix for document replacement
- Add pod status polling to genesis bootstrap

Change-Id: I32d22110749b334d1fbf19f910e41ab0b7ff3a16
2018-05-25 11:28:13 -05:00
Mark Burnett 27ead3e613 Bump Deckhand version
Change-Id: I2399ca7cd95fd4cf2e9ae81c3ded63ad0a50d96e
2018-04-05 11:37:45 -05:00
Mark Burnett ee16d7b411 Bump Deckhand version
This new version contains a number of bug fixes.

Change-Id: I179189e22f7c10878d5de5c59d9d29139a9142b3
2018-03-08 11:10:09 -06:00
Mark Burnett 3205668e2f Pin Deckhand version
Change-Id: Ief46ce19de61b7327f2ad2c42c7e6c4fe7955bbe
2018-03-06 10:07:20 -06:00
Scott Hussey 7917237ae0 Migrate to DH-managed config files
Use the Deckhand engine module directly to manage local configuration
files during CLI usage.

Note: not doing document validation as DH currently requires schemas to
be sourced from the database. Simple schema validation in place.

- Layering/substitution
- Schema validation based on DataSchema documents in payload
- Add deckhand to requirements

A few tooling updates

- concatenate test & schema yaml files into a single file to avoid name
  conflicts
- make nginx directory in build-scripts stage

Change-Id: I2d56244f01c58052f14331bc09fd5843d4c95292
2018-01-26 13:43:18 -05:00
Mark Burnett eeee591f8e Add deckhand design_ref support
* Add ability to fetch design from Deckhand
* Add functional testing for Deckhand design_ref integration
* Update complete example to work with changes to Ceph chart

Change-Id: Ice25a27b340e68a8ab38a23021cd91e032ca537b
2018-01-22 08:28:19 -06:00
Mark Burnett 527b2ad096 Add keystone integration test
* Adds "g2" gate for testing keystone integration
* Adds policy enforcement for join-scripts endpoint
* Updates ceph to luminous

Change-Id: Id52c1e51b567afc5a16d46c72145e21316c00a49
2017-12-19 16:00:20 -06:00
Mark Burnett b4d9596468 Add join-scripts endpoint
* Adds initial join-scripts API
* Updates resiliency test to leverage API for joining

Change-Id: Ibe0d42b8f4f4a3e1f6f102dee85a22cb8f78f8ec
2017-11-15 15:03:53 -06:00
Mark Burnett 6caf7fb54d Add initial chart for Promenade API
Co-author: Mark Burnett <mark.m.burnett@gmail.com>
Co-author: Samantha Blanco <spblanco.1@gmail.com>

Change-Id: I2e6af00b7905d9070f79b8c536385ebdae877d50
2017-11-15 14:19:43 -06:00
Samantha Blanco 9eb6f9c686 Inital API Commit
Creates necessary API files and implements health API route.

Change-Id: Id545d65949fcc48a05565f39b08180d4aa86006f
2017-11-03 14:48:47 -04:00
Mark Burnett ecbe862a24 Avoid directly installing non-frozen dependencies
Currently the Dockerfile specifies running `pip install -e ...`, which
will indirectly install dependencies from `requirements.txt`.  This is
generally safe, but should be avoided, since we are exclusively using
frozen dependencies.

Change-Id: Ie368ddb9f1229cc248c8d8804c71889c4339aa85
2017-10-20 10:54:10 -05:00