Commit Graph

58 Commits

Author SHA1 Message Date
Sergiy Markin e85f0ccdeb Airflow stable 2.8.2
This PS bumps up Airflow version to the latest
2.8.2 and also bumps up openstack dependences to
Antelope 2023.1

Change-Id: I5c7fb6ae950ea0f906106fc2184d5c6eb7858f5a
2024-02-26 21:00:58 +00:00
Sergiy Markin b3b8d05fe7 Airflow stable 2.8.1
Change-Id: I46c97108d50304fd4e3417f6057d999c891b0661
2024-02-21 22:24:46 +00:00
Sergiy Markin cdcdbd292d Airflow stable 2.6.2
This PS updates python modules and code to match Airflow 2.6.2:

- bionic py36 gates  were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfoemed based on
  airflow-2.6.2 constraints
- airskiff deploy pipeline was aligned with latest in treasuremap v1.9

Change-Id: If6f57325339995216d2553c7a5ff56e7673b5acc
2023-08-19 00:37:37 +00:00
Sergiy Markin 966742ec74 Restored ubuntu_bionic image build
This PS restores image build for ubuntu_bionic and adds appropriate
gates to keep it tested by appropriate functional and integratinal
tests.

Also the latest osh-infra commit was used with kubernetes 1.27.1 or
integration tests.

Make images section of Makefile was enriched with run_armada step in
order to quicktest the freshly built images - this is vital for focal
and bionic images differences.

Change-Id: Ida43eebd4aa51fff6261ac9cf5f8259359bc99a9
2023-05-19 13:52:30 +00:00
Sergiy Markin a8d9b7214c Armada updates
This PS delivers the following updates:
- fixed sample config and policy files generation process

Change-Id: I6393abc6aac0d79539820dc5768a7094147190a6
2023-04-28 18:56:45 +00:00
Sergiy Markin 386a686e69 [focal] Python modules sync with Airship project
- armada-airskiff-deploy is voting gate again
- fixed falcon.API deprecation - -> falcon.App
- fixed collections.abc.defaultdict not found error
- fixed tox4 requirements
- implemented requirements-frozen.txt approach to make allike as other
  Airship projects
- uplifted docker version in the image building and publishing gate

Change-Id: I337ec07cd6d082acabd9ad65dd9eefb728a43b12
2023-04-21 23:49:14 +00:00
SPEARS, DUSTIN (ds443n) 0e9d828fe6 Update armada to focal
Update kubernetes client to v26.1.0
Updating armada to focal base image
Remove xenial and opensuse dockerfiles
Update tox python from py35 to py38
Add apparmor for docker build
Uplift HTK chart version 0.2.52
Bumping up some python dependencies to get in sync with shipyard
Added clear-firewall role for airskiff-deploy playbook

Change-Id: If06a3f60466702d05a21c24a7cb8041bed41507a
2023-03-20 11:30:37 -04:00
SPEARS, DUSTIN (ds443n) 099de8aaf4 Bump k8s client to v25.3.0
Bumping k8s client to v25.3.0
Cronjob batch v1beta1 no longer available in k8s 1.25
Update tox.ini file to be compatible with v4

Change-Id: Iac79c52c97c9ef1223ae8d502da1572ef8d068fa
2023-01-18 11:25:05 -05:00
Sean Eagan a5730f8db8 Remove Tiller
For now we leave the tiller status enpdpoint, until
Shipyard has had a release to stop depending on it [0].

[0]: https://review.opendev.org/c/airship/shipyard/+/802718

Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: If8a02d7118f6840fdbbe088b4086aee9a18ababb
2021-10-05 02:41:32 +00:00
HUGHES, ALEXANDER (ah8742) b787c418e3 Standardize Armada code with YAPF
From recently merged document updates in [0] there is a desire to
standardize the Airship project python codebase.  This is the effort
to do so for the Armada project.

[0] https://review.opendev.org/#/c/671291/

Change-Id: I4fe916d6e330618ea3a1fccfa4bdfdfabb9ffcb2
2019-07-31 10:16:15 -05:00
Hervé Beraud e0a91426af Fix bandit errors
Bandit 1.6.0 changes the behavior of the '-x' option so that it now
supports glob patterns. Update our tox file to use to correctly exclude
test code from bandit scans. This requires bumping our minimum bandit version.

As an aside, changing the behavior of an option like this in a minor
version if bad form :(

Change-Id: I11bee26373ea9894b47d7f964eac88ef70859732
2019-05-13 15:31:52 +02:00
Drew Walters 163cacd973 style: Pin YAPF to 0.26.0
CI jobs are failing due to new YAPF style guidelines [0] that are not
PEP8 compliant. This commit pins YAPF to 0.26.0 to prevent non-compliant
regressions in the gate.

[0] https://github.com/google/yapf/blob/master/CHANGELOG

Change-Id: I030031f3748cddc5fce3020e7bcfe324b666cbd8
2019-04-08 19:21:39 +00:00
Andreas Jaeger cb737354f0 Fix Flake8 3.6.0 errors
Flake8 3.6.0 now warns about both line break after and *before* binary
operator, you have to choose whether you use W503 or W504. Disable the
newer W504.

Fix "F841 local variable 'e' is assigned to but never used".

Handle warnings about invalid escape sequence in regex.

Handle invalid escape sequence in string.

Change-Id: I68efbde4e9dd2e6e9455d91313eb45c9c79d35ce
2018-10-26 09:23:42 -04:00
Felipe Monteiro 0aa0935c7c chore: Combine pep8 with whitespace linter
This patch set combines whitespace linter and pep8 together
(as both do code linting) into the same job by dropping the
.zuul.yaml whitespace-linter job and rolling it up into the
same pep8 tox.ini check, just like Deckhand [0].

[0] 0462b7b929/tox.ini (L106)

Change-Id: I31ed60adfd431157ffc96a0f4d5d77d54af37352
2018-10-14 17:52:32 -04:00
anthony.bellino 63e63a3c30 Fix for yapf v0.24.0
This PS fixes yapf v0.24.0 errors. Also updates tox.ini
and test-requirements.txt accordingly.

Change-Id: If1dd0e51d328ad976bf0a7bfd512425c4da4ac0a
2018-09-11 20:39:25 +00:00
Sean Eagan c6dbf3344d Pin yapf version
The output with yapf version 0.24.0 has some changes which are in
conflict with the flake8 E126 rule. Hence pinning to <=0.23.0 for now.

Change-Id: I80b4094673eeca6dddc4fde0c324e0f5d9ec5d99
2018-09-10 15:28:56 -05:00
Andreas Jaeger 785242eb2a Add venv tox environment
Some OpenStack CI jobs need the venv tox environment to run, add it.

Change-Id: Id044c887f7b79c31391cf27a124a3f065f48b052
2018-08-24 21:23:29 +02:00
Tin Lam 11d941c918 chore(py3): update tox targets
This patch set updates the various tox targets to use python 3 first
per Stein goal outlined in [0]. Also, added minversion to tox.

[0] https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: I60ad774e089df248f3b1993a815dc6ac7232c3fc
Signed-off-by: Tin Lam <tin@irrational.io>
2018-08-19 16:23:10 +00:00
Vu Cong Tuan b12f2e1dfd Add py36 testenv
Python 3.6 is installed by default in Ubuntu 18.04 LTS.
Therefore, according to Transition Plan [1],
it'll be handy to have py36 testenv.

For more details, please check Python2 Deprecation Timeline [2]
and Python3-first Goal - Completion Criteria [3].

[1] https://wiki.ubuntu.com/Python/Python36Transition
[2] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html
[3] https://review.openstack.org/#/c/575933/8/goals/stein/python3-first.rst@42

Change-Id: I275d0ffe7b03cf564d02f575f92dd5bb395c7b48
2018-06-27 15:25:19 +07:00
Marshall Margenau f235512d57 Adding yapf config, plus formatted code.
- Adding yapf diff to pep8 target
- Adding yapf tox target to do actual format

** The rest of this PS contains formatted code only, no other changes

Change-Id: Idfef60f53565add2d0cf65bb8e5b91072cf0aded
2018-06-22 14:56:04 -05:00
Tin Lam d5249d0f05 chore(tox): update tox.ini
This patch set adds command to clean releasenote prior to building
releasenotes. This also clean up unnecessary whitelist command.

Change-Id: Ib9439235a24132111a56a9e35f0c9640d2892f5e
Signed-off-by: Tin Lam <tin@irrational.io>
2018-06-13 23:35:26 -05:00
huang.zhiping 99ced61d3e fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I3f2720a6e8e6c685d2e5ca288008be20785a8309
2018-06-10 15:38:40 +00:00
Felipe Monteiro 354cc14c59 Use stestr as the test runner for unit tests
This patchset changes Armada to use stestr instead of pytest
for running unit tests as it is the officially sanctioned
test runner in OpenStack [0].

[0] https://openstack.nimeyo.com/119685/openstack-dev-recent-changes-in-testr-and-migrating-stestr

Change-Id: I4f1320968990d9d9cc4d50cac0f2eecc3a0d331c
2018-06-08 21:50:03 +00:00
Tin Lam c39d21eec8 fix(coverage): add coverage rc file
The tox coverage refers to a non-existing rc file. This patch set
redresses this by putting in a proper .coveragerc file similar to
all the other projects.

This patchset also makes the tox coverage job compatible w/ Zuul.

Change-Id: I96a3d13482053b591f7d01e1c5c1e906d84f8fb7
Depends-On: If3cba68492ba9c7a0e78258c7b669d1b39fcdeba
Signed-off-by: Tin Lam <tin@irrational.io>
2018-06-08 06:42:57 +00:00
Tin Lam 333f4ba284 chore(bandit): add bandit
This patch set puts the bandit check as part of the pep8 gating job
for airship-armada and remove the separate bandit gate.

Change-Id: I3871bb20f04aedb80a1ffca4038e94a156156a14
Signed-off-by: Tin Lam <tin@irrational.io>
2018-06-03 20:31:24 -05:00
Tin Lam 6f025d1d27 feat(reno): add reno
This patchset adds releasenotes to Armada.

Change-Id: Ia89459a8900bfb94daeeac65e377ef9e79fc9793
Signed-off-by: Tin Lam <tin@irrational.io>
2018-06-01 21:58:18 +00:00
Felipe Monteiro 1ac92cc7f6 [trivial] Rename tox job coverage to cover
This patchset renames the tox.ini coverage job to cover to
comply with OpenStack standards [0]. This is needed so that [1]
works seamlessly (it will enable py35 and cover jobs for Armada).

[0] e.g. 04469a5181/tox.ini (L119)
[1] I174d5df008f6e000da1a3878afe75919312ea7aa

Change-Id: I353a60313bef2f8aa0319a4e89e659cd23c8a4a9
2018-05-31 21:55:08 +00:00
Tin Lam 10207be7b9 docs(requirements): update requirements
This change migrates the docs building requirements as outlined in
the Project Testing Interface[0]. One benefit of this that we no longer
load all the doc-generating modules when we run tox unittest. See [1] for
more details.

[0] https://governance.openstack.org/tc/reference/project-testing-interface.html#documentation
[1] http://lists.openstack.org/pipermail/openstack-dev/2017-November/124815.html

Change-Id: I0f4920df6ea68c289c82005d30ecd0efe0e29ca1
Signed-off-by: Tin Lam <tin@irrational.io>
2018-05-22 18:28:00 -05:00
Felipe Monteiro 085d9887e0 Rename docs to doc to align with OpenStack standard
This patchset updates docs to doc to align with OpenStack
standard. Follow-up patchset will be needed to publish
documentation to OpenStack [0].

TODO: Update Armada documentation job to align with [1].

[0] https://docs.openstack.org/doc-contrib-guide/project-guides.html
[1] http://lists.openstack.org/pipermail/openstack-dev/2018-March/128594.html

Change-Id: I100541611ddfcd5c42f09da631346cb8ef3de5e7
2018-05-17 21:39:01 +00:00
Tin Lam 8d1521e96c style(pep8): remove E722 exclusion
This patch set removes E722 pep8 exclusion that allows for "bare"
except: statement.

Change-Id: Icdce885366541b88aabbef35166cf196a588676b
Signed-off-by: Tin Lam <tin@irrational.io>
2018-05-08 23:11:18 -05:00
Felipe Monteiro 950c6012ab Fix chartbuilder get_files raising UnicodeDecodeError
This adds better exception handling ang logging to
_append_file_to_result helper in get_files. When reading
arbitrary file data and attempting to encode to utf-8
this can cause UnicodeDecodeError to be raised.

However, Armada will not skip over such files; it will
raise an exception with appropriate details instead.

Closes #195
Closes #196

Change-Id: Id7c32c17e351d1ffe042e3755c116c36b6380223
2018-03-22 17:13:58 -05:00
One-Fine-Day ec252e7069 Updated Makefile
ARMADA-29: CI/CD Armada Testing Framework Entrypoints defined

It does not appear the the docker run command to call tox tests/unit
is necessary.  The Makefile already includes several testing checks.

Added a few additional targets as specified in the upc-integration
code-convenstions to satisfy the clcp Makefile standards.

Change-Id: Ia920a856ed6d607fcf38d5f85dbb9a0c8aae4fa6
2018-03-15 12:40:19 -04:00
Marshall Margenau e1b64d200b style(armada): quality of life and cleanup
- adding .editorconfig file
- minor cleanup in various files related to .editorconfig
- typos, whitespace, etc.
- other general housekeeping items on the codebase

Change-Id: I104f8dcb06aafb180da12f7ee4c0ded41fc07b9d
2018-02-12 10:27:11 -05:00
Felipe Monteiro bfff1f4ed3 tests(git): Unskip git tests and add conditional skip test.
This PS unskips the last remaining git test in test_source which
had an unconditional unittests.skip() decorator applied to it.
The source_cleanup test now issues a warning if the repo path
isn't found and the associated test (passing in a bad path)
simply asserts that the warning was called.

This also adds a conditional skip to the git clone unit tests
in the event that the user doesn't have network connectivity.

Change-Id: Ic300c48a4e0c361e37ec15cd3675415f8170a817
2018-01-22 21:09:26 -05:00
Felipe Monteiro 23433b0808 bug(tox): Support regular expressions via tox -e py35
This PS simply allows tox -e py35 to take in regular expressions
for better testing experience. For example:

  tox -e py35  # runs all unit tests
  tox -e py35 -- test_armada_controller  # only run unit tests that match

Change-Id: Iab5a70fa0d51e1572a31943d8f38bc9b0d6a7c18
2018-01-22 23:07:57 +00:00
Roadrunner2058 d383e772fd feat(yaml): Support document references
This allows the user to apply and/or validate a manifest using
either a filepath (as before) or URL.

Addition by sh8121att:

- Create a general document resolver class to handle local paths
  and URIs
- Allow multiple filenames and combine them into a single document
  set
- Change API to allow for passing document reference URIs to be
  resolved server-side rather
- Update validation API to conform to UCP specification
- Dockerfile updates to speed up build
- Fix unit tests

Closes #96

Change-Id: I5a57779f10d1b63ffc161a14afec851a34ae9efe
2018-01-08 13:39:26 -06:00
Felipe Monteiro 48c7e540ca fix(tox): Add whitelist externals to tox.ini to avoid warnings
This PS adds whitelist externals to Armada's tox.ini to avoid
warnings that are issued while running tox -e py35 or tox -e pep8.
Example:

WARNING:test command found but not installed in testenv
  cmd: /usr/bin/find
  env: /opt/stack/armada/.tox/py35

This PS also resolves a critical error while running
tox -e coverage, because tox defaults to python2.7 for
the coverage job, and Armada's hapi module is not compatible
with python2.7. The fix involves setting the testenv's
basepython to python3.5.

Change-Id: Ice0074c3453c80290886f27a4518fbed01722572
2017-12-19 11:46:17 -05:00
daeseong.kim 92d7a3a563 fix(tox): add {posargs} in tox command
This PS can help the developer run just one unittest.
like below.
tox -e py35 armada/tests/unit/api/test_api.py::TestAPI::test_armada_apply

Change-Id: I846c52bb4f3b133942f7c201f750dc15607f93ea
2017-12-18 19:34:26 -05:00
gardlt 2cf7f17a2c feat(armada): generic-exception
- generic exception in errors

The goal of this is to move all exceptions into one file and generic format.
This aligns with the UCP projects.

Change-Id: I1862f08409182c77b03eaeb9e6bd3209d72c5cee
2017-11-29 21:42:56 +00:00
Scott Hussey 9c73661c8b Change job deletion logic
Update handler for chart pre-upgrade Jobs deletion
to rely on Kubernetes propagationPolicy for deleting
child Pods so that more generic labels can exist in
an Armada manifest without impacting job-unrelated
pods

- Update K8s API integration to use propagationPolicy for job delete
- Make default propagationPolicy 'Foreground'
- Update documents to clarify structure of specifying pre-upgrade hooks
- Fix tox file to support running unit tests behind a HTTP proxy

Change-Id: I650543cfe05cc6a9661ab375e831bb425b7eeeab
2017-11-22 15:38:22 -06:00
gardlt 0663a308d9 feat(tiller): updating-helm-version-2.6.0
- updated hapi lib
- implemented wait-resource-is-ready

Change-Id: Ia547bec0c83e5dca19c87a99dd2cdbe413d78c06
2017-10-27 16:08:47 +00:00
Mark Burnett 083ba20f50 fix(git): allow specifying git refs for sources
This allows, e.g., pointing at a particular Gerrit Patch Set using
natural configuration, e.g.:

    ---
    schema: armada/Chart/v1
    metadata:
      schema: metadata/Document/v1
      name: ceph-config
    data:
      chart_name: ceph-config
      release: ceph-config
      namespace: ucp
      timeout: 3600
      install:
        no_hooks: true
      upgrade:
        no_hooks: true
      values: {}
      source:
        type: git
        location: https://github.com/openstack/openstack-helm
        subpath: ceph
        reference: refs/changes/54/457754/73
      dependencies: []

Change-Id: Ib6af0fec2bbfa1fa4e523d839df44af047697522
2017-10-18 14:17:17 -05:00
gardlt cba78d1d03 style(armada): converting py2 to py3
- Format code to python 3.5
- using absoulute paths

Change-Id: I7414b5de915429c2c7f85b99f2ab91f395c62121
2017-10-10 19:55:20 +00:00
Pete Birley 746cbd0bd8 Fix(linting): Make Armada pep8 compliant
This patch set makes Armada pep8 compliant. Note the hapi/** is
autogenerated and therefore should be excluded from linting.

Change-Id: I123eefb543f9bd9cf0bc6bd98ed95646d8d72cc3
2017-09-29 11:46:58 -04:00
gardlt bb26131ce2 feat(api): policy enforcement and api standard
- enhanced logging
- created base structure
- updated docs
- PasteDeploy auth
- Oslo Policy

Closes #107

Change-Id: I805863c57f17fcfb26dac5d03efb165e4be49a4e
2017-09-20 21:54:39 +00:00
Kochetov, Mikhail (mk899x) e9790db48d Fix tox/bandit tests
Change-Id: I9dc199ef0361aa4ba08b438f4eca2769a79c2a44
2017-09-14 14:19:49 -04:00
Alexis Rivera De La Torre d17485f8b9 [feat] adding-tox-testing
- tox commands
- docs update

Change-Id: Ie6caa498d6017822e095ee5b001124074e2755aa
2017-08-11 16:48:57 +00:00
Tim Heyer ea6ad23672 Migrate from pygit2 to GitPython
Change-Id: Ib477b945e922fd0a1767bd6643c0a45751599fad
2017-08-09 21:20:25 +00:00
gardlt afb7fe83ab [feat] adding standard armada manifest
- adding tmeplate armada manifest
- create Armada Manifest
- updated validation for new documents
- updated testing
- updated docs
2017-07-26 14:44:05 -05:00
drewwalters96 f4303037cc [Bug] Adds required Oslo configuration to API and unit tests
- Register Oslo logging configurations in server.py, test_armada.py, test_chartbuilder.py, and test_tiller.py
- Add mock to test-requirements.txt
- Update .travis.yml
- Add mock case for tiller IP
2017-06-28 18:03:27 -05:00