Commit Graph

38 Commits

Author SHA1 Message Date
Sergiy Markin d21e3434f6 Airflow stable 2.8.2
This PS bumps up Airflow version to the latest
2.8.2 and also bumps up openstack dependences to
Antelope 2023.1

Change-Id: Iae72c6da9406749cf157437495f31dc3b9f6ba2c
2024-02-26 21:01:52 +00:00
Sergiy Markin 1a0ea41f28 Airflow stable 2.8.1
Change-Id: I49093c3c5bbdf076ef9f2ffa39769f9a2ec55c5c
2024-02-22 21:09:21 +00:00
Sergiy Markin 69a74590e7 Airflow stable 2.6.2
This PS updates python modules and code to match Airflow 2.6.2:

- bionic py36 gates  were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfoemed based on
  airflow-2.6.2 constraints

Change-Id: I9c3e139b3437414a61af7e7c0b7d7e533fadefda
2023-08-29 21:12:11 +00:00
Sergiy Markin 2f2455f0d1 Restored ubuntu_bionic image build
This PS restores image build for ubuntu_bionic and adds appropriate
gates to keep it tested by appropriate functional and integrational
tests.

Change-Id: I8ef524a1f66f1a88593a18334b8c37c4db0175c6
2023-05-25 22:53:15 +00:00
Sergiy Markin 32ad8a96b0 [focal] Python modules sync with Airship project
- uplifted/downgraded some python modules
- fixed falcon.API deprecation - -> falcon.App
- uplifted deckhand reference for python deps
- fixed formatting style  using yapf linter
- added bindep role and bindep.txt file with required deps
- fixed quai docker image publishing
- re-enabled openstack-tox-py38 gate job

Change-Id: I0e248182efad75630721a1291bc86a5edc79c22a
2023-04-21 06:09:14 +00:00
SPEARS, DUSTIN (ds443n) 27a8b0d798 k8s upgrade to 1.26.0
upgrades kubernetes client to v1.26.0
remove installation of containerd during genesis.sh to prevent containerd downgrade
update bitnami kubectl image to image with curl installed for readiness check

Change-Id: I3afd5a7e7211bae3f52263167a62a012da0619a0
2023-03-20 13:16:48 -04:00
Wahlstedt, Walter (ww229g) 8ce937a9f7 updates for focal
add focal dockerfile
update zuul jobs for focal
update tox for tox4 changes
update all requirements to latest and match deckhand
update cfssl from R1.2 to v1.6.3
fixed local gates for focal
updated examples promenade manifests to run on focal

Change-Id: I2af4043784766d36588c6f738053ad66e7b89a90
2023-02-27 12:11:07 -05:00
Vladimir Maliaev d0d0e8be36 Adjust plugin cri parameters
Switch docker repo to use docker-ce package

See:
https://www.docker.com/blog/changes-dockerproject-org-apt-yum-repositories/

Change format of parameters for plugin cri configuration
to be able to use version = 2

See:
https://github.com/containerd/cri/blob/master/docs/config.md

Use python3.6 instead of python3.5 in tests

Change-Id: Ie56c0a7a344ea78c9f348fbf0e77617399815a7e
2020-03-26 05:02:37 +00:00
Roman Gorshunov d12927a156 Fix: Promenade Exceptions docs rendering on RTD
Readthedocs failed to render Promenade exceptions with error:
> WARNING: autodoc: failed to import exception 'xxx' from module
> 'promenade'; the following exception was raised: No module
> named 'falcon'

Trying to add Promenade requirements to the installed requirements list,
so that Readthedocs has all modules, including those needed for the
Promenade itself.

Unify docs building by utilizing Zuul docs-on-readthedocs template job.

Cosmetic readability changes:
1. combined all Makefile .PHONY targets into one
2. merged multiple LABEL instructions in Dockerfile into one

Change-Id: I731ee3426a631fa765f13ba7091dcb4b9ebd0353
2019-08-27 22:57:15 +02:00
Pradeep Kumar b3308e22a6 Moving documentation only requirements into doc/requirements.txt
Change-Id: I2bc7fd90a7ccc7a96edaf09ec7d4cdffe987c8c0
2019-02-11 12:11:02 +00:00
Felipe Monteiro 9d1e27f040 Treat warnings as errors in docs and fix warnings
This patch set fixes Promenade's failing documenation job (it
is currently issuing a number of warnings that should be treated
as errors):

/home/zuul/src/git.openstack.org/openstack/airship-promenade/doc/source/configuration/index.rst:10: WARNING: toctree contains reference to nonexisting document 'configuration/encryption-policy'
WARNING: autodoc: failed to import exception 'exceptions.ApiError' from module 'promenade'; the following exception was raised:
No module named 'falcon'

The first one indicates that the file has the wrong extension
and the second one that autodoc is failing to import something
(first case => rename file, second case => add requirement to
test-requirements.txt)

.. http://logs.openstack.org/18/605818/1/gate/airship-promenade-unit-py35/c733138/ara-report/

Change-Id: I974d5929aaac8a6e27d4196eed5f8bc97703e61d
2018-10-04 14:25:22 +00:00
Mark Burnett 3ac95edc60 Fix: re-add incorrectly removed format test
This was removed in Iccf6228ab9e6d621d3047994b3adc192d67273c9 but should
not have been as it has allowed for code format drift.

This also

* Pins the version of yapf to 0.24.0
* Fixes some drift
* Updates formatting to the version of yapf being used

Change-Id: Ie3d9fd6344a29d8ddb76a36d4a31d001a4c8b7c6
2018-09-12 10:40:45 -05:00
Andreas Jaeger 5c145f1ba5 Add venv tox environment
Some OpenStack CI jobs need the venv tox environment to run, add it.

Change-Id: I4022c360aa8b64544014708bc55927e75dbc8164
2018-08-24 21:33:40 +02:00
Gage Hugo f60f6d7a85 Consolidate pep8/bandit zuul gating
This change adds the global zuul pep8 tox job, which runs both
bandit and pep8 using tox. This also removes the two other airship
specific lint-pep8 and bandit zuul jobs since they are both covered
by the default openstack global one.

Also cleaned up the tox.ini by moving the requirements into the
test-requirements.txt file.

Change-Id: Iccf6228ab9e6d621d3047994b3adc192d67273c9
2018-08-21 12:57:02 -05:00
Mark Burnett 0e5009277a Fix broken validatedesign endpoint
This endpoint did not correctly return a response body.

Change-Id: Iec23083402d92069ab34698f31aeee4dce035aa6
2018-07-19 13:29:18 -05:00
Vu Cong Tuan cb0baea330 Add py36 testenv
Python 3.6 is installed by default in Ubuntu 18.04 LTS.
Therefore, according to Transition Plan [1],
it'll be handy to have py36 testenv.

For more details, please check Python2 Deprecation Timeline [2]
and Python3-first Goal - Completion Criteria [3].

[1] https://wiki.ubuntu.com/Python/Python36Transition
[2] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html
[3] https://review.openstack.org/#/c/575933/8/goals/stein/python3-first.rst@42

Change-Id: Ied2445269d56090770558a765a38aeff549f8dae
2018-06-28 08:48:47 +07:00
Roman Gorshunov 46e406ba71 Unifying proxy variables for tox
This patch adds proxy-related variables to tox command.

Change-Id: Ifa202936f5cd240978893e2ee280eb4f27fb0744
2018-06-22 14:54:35 +02:00
Felipe Monteiro ed65d983f7 [trivial] Rename tox jobs for zuul
This patchset renames the tox.ini jobs:

* coverage => cover
* lint => pep8
* unit => py35

to comply with OpenStack standards [0].

[0] e.g. 04469a5181/tox.ini (L119)

Change-Id: I1a542c5e36f29d3788df8a5ebdce3cbe49ab4046
2018-06-06 15:12:54 +00:00
Felipe Monteiro 13bc298f60 Rename docs to doc to align with OpenStack standard
This patchset updates docs to doc to align with OpenStack
standard. Follow-up patchset will be needed to publish
documentation to OpenStack [0].

[0] https://docs.openstack.org/doc-contrib-guide/project-guides.html

Change-Id: I0f2346801bc3749e74fcc3104cc2258626747185
2018-06-04 15:10:35 -04:00
Zuul a48dcacecc Merge "Update Deckhand commit" 2018-05-29 13:53:29 +00:00
Scott Hussey a1640349d7 Update Deckhand commit
- Uplift deckhand commit to include fix for document replacement
- Add pod status polling to genesis bootstrap

Change-Id: I32d22110749b334d1fbf19f910e41ab0b7ff3a16
2018-05-25 11:28:13 -05:00
Mark Burnett 57466a94bf Add docker image publishing
Change-Id: I6a09b73ffd0af6d2d23963b99c436b43410b1a46
2018-05-24 08:59:21 -05:00
Mark Burnett 38ac4bff79 Speed up image build
* Ignore FIO output file when building docker
* Removes PBR and removes .git from docker context

Change-Id: I58d4fa7e9d538035c52b1826d1488f72e74beea8
2018-04-25 12:00:06 -05:00
Krysta 31c2fc8346 Documentation for exceptions
Adds documentation for promenade exceptions.

Change-Id: I26f817f7294df686eeb6f29099dfe69e0b64bbea
2018-02-23 10:35:18 -05:00
Mark Burnett b4449434a5 Fix DNS name list for kube services in certs
* Also adds liveness and readiness probes for Prom deployment

Change-Id: Id65d1e555e14478f2439c14dd6d6d7952411256d
2018-02-14 15:33:41 -06:00
Felipe Monteiro eea5aaf0f6 Add basepython=python3.5 to [testenv] in tox.ini
Running tox -e lint in an environment with python2.7 for example
as the default python interpreter causes the following errors
to be emitted:

lint runtests: commands[0] | yapf -rd /opt/stack/ucp/promenade/promenade /opt/stack/ucp/promenade/tests
lint runtests: commands[1] | flake8 /opt/stack/ucp/promenade/promenade
/opt/stack/ucp/promenade/promenade/renderer.py:18:39: E999 SyntaxError: invalid syntax
/opt/stack/ucp/promenade/promenade/config.py:17:19: E999 SyntaxError: invalid syntax
/opt/stack/ucp/promenade/promenade/tar_bundler.py:17:20: E999 SyntaxError: invalid syntax
/opt/stack/ucp/promenade/promenade/logging.py:76:12: E999 SyntaxError: invalid syntax
/opt/stack/ucp/promenade/promenade/generator.py:11:33: E999 SyntaxError: invalid syntax
/opt/stack/ucp/promenade/promenade/builder.py:15:33: E999 SyntaxError: invalid syntax
/opt/stack/ucp/promenade/promenade/pki.py:59:43: E999 SyntaxError: invalid syntax
/opt/stack/ucp/promenade/promenade/cli.py:13:16: E999 SyntaxError: invalid syntax

This PS just adds basepython=python3.5 to the base [testenv] category
from which jobs like tox -e lint inherit causing tox to use the python3.5
interpreter. Since Promenade only supports Python 3.5+ (since it isn't
tested against anything else), this won't break anything.

Change-Id: I549a02dbfa5f3b53bab26b87ad3042192e315530
2018-02-01 05:51:28 +00:00
Mark Burnett eeee591f8e Add deckhand design_ref support
* Add ability to fetch design from Deckhand
* Add functional testing for Deckhand design_ref integration
* Update complete example to work with changes to Ceph chart

Change-Id: Ice25a27b340e68a8ab38a23021cd91e032ca537b
2018-01-22 08:28:19 -06:00
Mark Burnett b9dceb2a9a Minor testing-related cleanup
* Move docs dependencies out of test-requirements.txt
* Eliminate python version specific test (py35)
* Re-order default envlist to shorten feedback cycle
* Minor whitespace changes
* Simplify versions unit test
* Add /.cache to .gitignore
* Include tests directory in basic format requirements, but not flake8
* Fix sample noauth-api-paste.ini config

Change-Id: I440f5c2d988098287f9914449f7103735f832192
2018-01-02 10:14:10 -06:00
Samantha Blanco fd31aae8e8 Add API unit tests
Add unit tests for Promenade API endpoints. Also adds validatedesign
API endpoint.

Change-Id: I4c6a5da1f521f913c94ae8a07c8bc43cf4114f75
2017-12-19 16:45:37 -05:00
Mark Burnett c3a5410619 Docs: Add design doc
This adds an initial description of Promenade's design.

Change-Id: I76060bcacf67ef2422c7d7514dcdc72fcd49d0f0
2017-11-16 13:50:46 -05:00
Mark Burnett 2fd461d0e8 Freeze tox lint deps
Varying versions of yapf were reformatting the code in slightly
different ways.  Freezing all lint deps to avoid related issues.

Change-Id: I2ae889c883fb8954588dabc7a2b02b110d000f33
2017-11-16 09:03:21 -06:00
Mark Burnett e82fd04ff1 Gate: Add publishing metadata for JUNit files
Includes:

* JSON-schema to validate gate manifests.
* `tox -e gate-lint` job to check gate manifests.
* More robust handling of conformance test results.
* Bufix in `tools/g2/lib/kube.sh` where a missing `fail` method was
  called.

Change-Id: I90740f659a691cb6ee92144205976ef97133f7bf
2017-10-31 10:38:23 -05:00
Mark Burnett 597c1a3929 Security: Add bandit job to tox
Change-Id: Idcf6efade852e7de0c636184c21f35a03fe0d980
2017-10-30 13:34:23 -04:00
Mark Burnett 918ee0500d Tox: always recreate the virtualenv for `freeze`
Change-Id: I7d9434609818d29b517797f20e00b85a0e221c57
2017-10-27 08:17:43 -05:00
Mark Burnett 22e2196b7c Add lint and fmt tox jobs
Change-Id: I487a4738bc19de52b094bb2eadbcd083c4e2d538
2017-10-25 11:01:18 -04:00
Mark Burnett ecbe862a24 Avoid directly installing non-frozen dependencies
Currently the Dockerfile specifies running `pip install -e ...`, which
will indirectly install dependencies from `requirements.txt`.  This is
generally safe, but should be avoided, since we are exclusively using
frozen dependencies.

Change-Id: Ie368ddb9f1229cc248c8d8804c71889c4339aa85
2017-10-20 10:54:10 -05:00
Mark Burnett 95643147c5 Migrate to self hosted using charts
This change includes several interconnected features:

* Migration to Deckhand-based configuration.  This is integrated here,
  because new configuration data were needed, so it would have been
  wasted effort to either implement it in the old format or to update
  the old configuration data to Dechkand format.
* Failing faster with stronger validation.  Migration to Deckhand
  configuration was a good opportunity to add schema validation, which
  is a requirement in the near term anyway.  Additionally, rendering
  all templates up front adds an additional layer of "fail-fast".
* Separation of certificate generation and configuration assembly into
  different commands.  Combined with Deckhand substitution, this creates
  a much clearer distinction between Promenade configuration and
  deployable secrets.
* Migration of components to charts.  This is a key step that will
  enable support for dynamic node management.  Additionally, this paves
  the way for significant configurability in component deployment.
* Version of kubelet is configurable & controlled via download url.
* Restructuring templates to be more intuitive.  Many of the templates
  require changes or deletion due to the migration to charts.
* Installation of pre-configured useful tools on hosts, including calicoctl.
* DNS is now provided by coredns, which is highly configurable.

Change-Id: I9f2d8da6346f4308be5083a54764ce6035a2e10c
2017-10-17 13:29:46 -05:00
Felipe Monteiro b3de5e990a Promenade documentation via build_sphinx.
This PS adds tooling and automation to automatically generate
Promenade's documentation into feature-rich HTML pages that can
be hosted.

To run the documentation job, simply execute:

    tox -e docs

Change-Id: I5a925c82544c14c34c1796da5804cf12b45ae575
2017-10-06 15:06:45 -04:00