Update helm toolkit ref so that ingress will be created with a pathType
of Prefix.
https: //review.opendev.org/c/openstack/openstack-helm-infra/+/905757
Change-Id: I261603e3c68337347955996f77b86bd60a2f2af5
- adjusted .gitignore to keep fresh egg-info and omit build artifacts
- fresh egg-info data is needed for promenade that depends on Deckhand
- restored deckhand-functional-uwsgi-py38 gate
- restored deckhand-integration-uwsgi-py38 gate
- made deckhand-airskiff-deployment gate voting ( treasuremap project
has been updated)
- removed bionic gates
- updated focal dockerfile
- added more binary deps into bindep.txt
- updated deckhand chart values to latest images - focal and wallaby
- fixed python code to compy with CVE's found by fresh version of bandit
- implemented pip freeze approach
- added tox -e freeze profile to manage it
- requirements-frozen.txt is now main file with requirements
- requirements-direct.txt is the file to control deps
- updated setup.cfg to adjust to newer version of setuptools
- fixed airskiff-deploy gate
- fixed docker-image-build playbook to restore Quay repo image publish
- updated other playbooks to include roles from zuul/base-jobs in order
to setup build hosts properly
- removed workaround with hardcoded dns resolver ip 10.96.0.10 as it
became obsolette due to recent fix in openstack-helm-infra
- adjusted tools/whitespace-linter.sh script
- tox.ini has been brought to compliance with tox4 requirements
- replaced str() calls with six.text_type() according to D325 Deckhand specific
commandment from Hacking.rst
- locked python-barbicanclient version with 5.2.0 because of breaking
changes in the upper versions
Change-Id: I1cd3c97e83569c4db7e958b3400bdd4b7ea5e668
update dockerfile for python deckhand install
add deckhand version to chart 1.0
add chart version 0.2.0
update all packages to latest in requirements.txt
update zuul jobs for focal and python 3.8
remove zuul job functional-uwsgi-py38 in favor of functional-docker-py38
update tox config
typecast to string in re.sub() function
add stestr to test-requirements.txt
add SQLAlchemy jsonpickle sphinx-rtd-theme stestr to requirements.txt
deprecated function: BarbicanException -> BarbicanClientException
fix mock import using unittest
fix import collections to collections.abc
fix for collections modules for older than python 3.10 versions.
deprecated function: json -> to_json
deprecated function: werkzeug.contrib.profiler ->
werkzeug.middleware.profiler
deprecated function: falcon.AIP -> falcon.App
deprecation warning: switch from resp.body to resp.text
rename fixtures to dh_fixtures because there is an imported module
fixtures
switch from stream.read to bounded_stream.read
deprecated function: falcon process_response needed additional parameter
deprecated function: falcon default_exception_handler changed parameter
order
move from MagicMock object to falcon test generated object to fix
incompatability with upgraded Falcon module.
Adjust gabbi tests to fix incompatability with upgraded DeepDiff module
update Makefile to execute ubuntu_focal
update HTK (helmtoolkit)
unpin barbican to pass integration tests
Use helm 3 in chart build.
`helm serve` is removed in helm 3 so this moves
to using local `file://` dependencies [0] instead.
Change-Id: I180416f480edea1b8968d80c993b3e1fcc95c08d
When performing substitutions, there are occasions when the source value
does not exactly match the format required by the destination document
(e.g. the values.yaml structure of an Armada chart).
This change provides the ability extract a substring of the source
value, and substitute that into the destination document.
Two optional fields are added to `src` under `metadata.substitutions`:
* `pattern`: a regular expression, with optional capture groups
* `match_group`: the number of the desired capture group
The canonical use case is a chart that requires an image with the repo
name and tag in separate fields, while the substitution source has the
full image path as a single value.
For example, assuming that the source document "software-versions" has:
data:
images:
hello: docker.io/library/hello-world:latest
Then the following set of substitutions would put the repo and tag in
the applicable values in the destination document:
metadata:
substitutions:
- src:
schema: pegleg/SoftwareVersions/v1
name: software-versions
path: .images.hello
pattern: '^(.*):(.*)'
match_group: 1
dest:
path: .values.images.hello.repo
- src:
schema: pegleg/SoftwareVersions/v1
name: software-versions
path: .images.hello
pattern: '^(.*):(.*)'
match_group: 2
dest:
path: .values.images.hello.tag
data:
values:
images:
hello:
repo: # docker.io/library/hello-world
tag: # latest
Change-Id: I2fcb0d2b8e2fe3d85479ac2bad0b7b90f434eb77
Update tox configuration and Zuul config to use Python 3.6, and make the
xenial and opensuse gates non-voting.
Change-Id: Iab98ad6cf269d57acf0598ae431b2f661673b234
Unpin python3-six for Opensuse image build.
Update helm-toolkit stable commit to merge of this change:
https://review.opendev.org/#/c/803654/
Update the helm installation script to download and install v2.17.0
Fix integration tests by pinning Barbican to stable commit.
Pinn jsonschema to 4.0.0a2 to fix tox tests
Change-Id: I2badd0e2f6c934098f0c9f5ef7e52354756c12e0
When pip is upgraded to 20.3, the pip dependency resolver is much more
strict and will no longer install a combination of packages that is mutually
inconsistent[0].
These changes account for the fact that Shipyard imports Armada, Drydock,
Promenade, and Deckhand. Having said that, with pip 20.3, the pip
packages amongst those projects cannot conflict. A follow-up change may
be needed if more conflicts are found.
[0] https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-2-2020
Change-Id: Id75acea82ddf5d915a8b8805e076dac49cab800f
Policy validation in Deckhand was not implemented completely. Refer link
below:
https://airshipit.readthedocs.io/projects/deckhand/en/latest/users/validation.html#policy-validations
This PS removes some of the code related to the feature which was being
used in a code path when a set of documents are uploaded to Deckhand.
In standard Airship deployments the number of documents could be quite
high and this leads to significant delay (more than 300seconds in some
cases). As there are no plans to implement the policy validation feature,
it makes sense to remove it from code path which could cause delay and
sometimes timeouts while uploading documents.
This has been tested on a Baremetal lab: GF and BF.
Change-Id: I2ff3f40a7fe37bed5a589fab00d829db726604fe
- Added ability to build deckhand docker image, using either the
Ubuntu xenial or the Ubuntu bionic base image.
- Made the bionic base image the default base image for deckhand
docker image build.
Change-Id: I26657de34a233ee3223a7f93fc667e734ac9140b
Apache-airflow release 1.10.3 and later use Flask 1.10.1,
which needs at least Werkzeug 0.15.1.
https://issues.apache.org/jira/browse/AIRFLOW-4900
Airflow depends on deckhand for document validation. But,
deckhand locks the Werkzeug package version to 0.14.1,
which breaks airflow.
This PS lets deckhand use the latest Werkzeug available release.
Change-Id: I4385a69c41571010c426dfaee1082109914c7d2a
Adding DISTRO parameter for makefile to invoke distribution specific
Dockerfile and build image accordingly.
Updated logic for existing jobs to have distro specific logic
for building and publishing images.
Added multiple distro specific document in operator section.
Change-Id: I415ab28b06ea17b21e76d28ccb3e284041c8072d
Remove manual generation of UML diagrams; they will be built by Sphinx
itself.
Require to install PlantUML and Graphviz via bindep in
openstack-tox-docs job; ReadTheDocs service already has both PlantUML
and Graphviz installed for documentation builds.
Change-Id: I21ab124a80e7768bc9edd891b975b4e0f8a6e50d
Story: 2004747
This patch set adds use cases for each of the mutation
operations used by Deckhand:
* substitution
* layering
* replacement
To help document authors decide which operations they should use.
Change-Id: I8bc470c7ddc2f3c0aa9d4372a2323fa84632e369
This patch set adds additional documentation and unit tests
to validate further replacement scenarios.
In particular this commit adds an additional document check that
looks for documents exisitng in different layers that contain the
same name and same schema without any of them having `replacement: true`
Change-Id: I7c033d32a6755f36e609789a748cbc6d4af06bc2
This PS adds configuration documentation that includes
a literalinclude of the config file as well as some
information on each of the cache config options as
these are important on performance.
Change-Id: I3b06012b8843b7bfbd46307f81397172a41d3675
This package is used for generation autodoc documentation
automatically which can be linked to by Deckhand
documentation from other places. This is to make autodoc
generation work in RTD.
More info: https://pypi.org/project/sphinxcontrib-apidoc/
Change-Id: I43aac82728e5935a5a2626f2fd29d7a7188d19f9
This patch set adds `./entrypoint.sh alembic upgrade head` command
to manual install section in Getting Started guide, without which
Deckhand server cannot start correctly.
Change-Id: Id486cc98b8fd93271a8571364adfb9fb30bf4bcc
This patch set adds documentation related to data redaction in
order to capture the purpose behind it and how it adds an
added layer of security to Deckhand.
Change-Id: Icb28970684a9026fda200273a14d9ba421f627d7
- If a document has a storage policy of encrypted
- Redacts (sha256) the data section.
- Redacts (sha256) the substition paths.
- Uses the same /documents endpoint, adds a new query parameter
?cleartext-secrets=true to show the non-redacted values.
Change-Id: I42808901b97c667a1148c00fbb7717a0847c9981
This patch set pins down package requirements for the following
reasons:
* Inconsistencies between ranges and direct pins
* Deckhand isn't gated against requirements repo so changes to
some packages isn't actively tested
* Other Airship components use pins, so Deckhand might break them
when package requirements go out of sync between components
* Deckhand as of yet has no stable release so Airship relies on
master for stability so it is better to ensure stability over
anything else
Change-Id: I20ef4be3e01cd267771098d33447ccb61aee20b4
This patchset elaborates on document layering in the documentation
to provide much greater clarity into what layering is and its
associated concepts, including: layer, layer order, layering policy,
layering definition, document abstraction, parent selection,
layering actions, etc.
Change-Id: I584e67b7984fa4035cef481a116ae3b8a3eb2906
This patch set reorganizes Deckhand's documentation structure
for better organization into 3 distinct categories:
* developer's guide
* operator's guide
* user's guide
This means that the RTD navigation menu on the left-hand side
will have fewer links (see list above) making navigation much
easier. This is similar to how Armada organizes its documentation
too.
This patch set also updates README section with a better
overview and trims some fat from it (remove testing
documentation as it doesn't really belong there -- there
is a dedicated page for that already).
Finally, this patch set changes the exceptions page to
render as a basic list of autoexception classes because
the current tabularized view is not rendering correctly
on RTD [0].
[0] https://airship-deckhand.readthedocs.io/en/latest/exceptions.html
Change-Id: I162383bf8e3bbd5004603c979ac7b0d760a210c4
1) UCP -> Airship
2) readthedocs.org -> readthedocs.io (there is redirect)
3) http -> https
4) attcomdev -> airshipit (repo on quay.io)
5) att-comdev -> openstack/airship-* (repo on github/openstack git)
6) many URLs have been verified and adjusted to be current
7) no need for 'en/latest/' path in URL of the RTD
8) added more info to some setup.cfg and setup.py files
9) ucp-integration docs are now in airship-in-a-bottle
10) various other minor fixes
Change-Id: I12b2fa8fbec37a483a0ad50382e08f51ed97533a
GET /revisions/{{revision_id}}/deepdiff/{{comparison_revision_id}}
- Added deepdiff api for generating diff between
two rendered documents.
- Deep diffing for data and metadata
- Refactor diff functions
- Client update
- Added unit testcases
- Added funtional testcases
- Doc update
Change-Id: Ib60fa60a3b33e9125a1595a999272ca595721b38
Patterns may now be replaced recursively. This can be
achieved by using specifying a ``pattern`` value and
``recurse`` (with a required ``depth`` argument).
Example:
substitutions:
- dest:
path: .
pattern: REGEX
recurse:
depth: -1
src:
schema: deckhand/Passphrase/v1
name: example-password
path: .
NOTE:
Recursive selection of patterns will only consider
matching patterns. Non-matching patterns will be ignored.
Thus, even if recursion can "pass over" non-matching patterns,
they will be silently ignored.
This is useful for reducing the number of substitution
stanzas that are required for the purposes of performing
all the string pattern replacements that are required.
Best practice is to limit the scope of the recursion
as much as possible: e.g. avoid passing in "$" as the
``jsonpath``, but rather a JSON path that lives closer
to the nested strings in question.
Included in this patch set:
* recursive implementation for pattern replacement
* unit tests for most scenarios and edge cases
* docstring updates
* documentation updates
* schema updates for validation
Change-Id: I85048349097ed696667fae80f1180808d264bbcf
This patch set updates document types documentation
which includes expounding on control documents, layering policy
documents, and dataschema documents.
Change-Id: Id31fcb6b68ca30fdf681dec8348c2fc4237cd48d
This renames some instances of ucp to airship in the documentation
and makes some trivial documentation fixes.
Change-Id: I9a4a81d15bfc13b4fe089b7d65f0df43eeade9fb