Adding DISTRO parameter for makefile to invoke distribution specific
Dockerfile and build image accordingly.
Also qualifying image job name to distinguish ubuntu vs opensuse
check jobs and passing distro specific variable.
Similar logic for pushing distro specific images.
Added new document section for multiple distro support.
Change-Id: I215f8a107cab8770181c0472e0cc29053b2b978e
Change to use ubuntu base image instead of python
Refactor Shipyard Dockerfile to reduce image size significantly
BREAKING CHANGE: The `make images` PYTHON_BASE_IMAGE arg is now renamed to BASE_IMAGE.
Change-Id: I3338dfbbb91b5514fa4fd205bdfc4136d0abc2e5
Changes to make the docker image build to include the workflows from
Shipyard, rather than adding them to the container during Helm install
of Shipyard. This also removes the "prod" switch, as it is now always
built the same way, with the workflows in place.
Change-Id: I4acd6195cbec32193e15621e75ccaeb9879455f5
Updates to Airflow 1.10.1; See (1), (2) for some notes
Related, and additionally: configures Airflow to restore logging of
workflow steps to a console/sdtout logger, supporting the desired
ability to attach logging and monitoring to standard container
mechanisms. This does not change the behavior of also logging to the
airflow-arranged log files for steps and DAG runs.
A side effect of updating to 1.10.1 includes a major decrease in
resource usage by the Airflow scheudler process (reducing from ~ 1 core
fully consumed to less than 5% of a core consumed YMMV, but significant)
Additional adjustment downward of resources allocated, threads produced,
and frequency of polling leads to an overall significant reduction in
resource usage.
Airship note:
Because Airflow 1.10.0 and 1.10.1 use compatible versions of celery and
dag_run information, updating from 1.10.0 - 1.10.1 in place is possible
if airflow-worker pods are allowed to continue to run.
(1) https://github.com/apache/incubator-airflow/blob/master/UPDATING.md
(2) https://github.com/apache/incubator-airflow/releases/tag/1.10.1
Change-Id: I9b024e3996c528c7b74e2888191d48c7a45a1f04
This PS enables shipyard to use PBR as rest
of the other OpenStack project.
Currently if shipyard is used as a requirement in another
project, the dependencies for shipyard doesn't get installed.
This PS fixes this bug.
Change-Id: I34d452649a886af9d865d4ad51eeab70399fe395
Updates the image building to Airflow 1.10, including necessary
configuration changes and a general update of dependencies.
Airflow 1.10 includes many enhancements and bugfixes since 1.9 [0]
This change introduces many "unused" configuration parameters to satsify
Airflow's expectations[1]. An ugly, but likely harmless change to the
log output with interleaved newline characters from Airflow steps[2].
Changes to the chart and other dependendencies have also been introduced
to match this update.
[0] https://github.com/apache/incubator-airflow/blob/master/CHANGELOG.txt
[1] https://issues.apache.org/jira/browse/AIRFLOW-3099
[2] https://issues.apache.org/jira/browse/AIRFLOW-1917
Change-Id: I179dcf1f0369650b8c4519f704abb7fb495f4248
1) Use OCI Image Specs for labels instead of custom 'commit-id=xxxxx'
or legacy "Label Schema"
2) Fix missing git commit id labels on images (.revision)
3) Add human-readable title (.title) of the image, URL (.url), and
a few other properties (annotations) according to the latest Specs
Change-Id: Iedd64fa70016880d5df241d27d8dad101e10130b
l is to let user customize the base image of the component
by passing FROM=myimage during the build process. This would let any
project leveraging Airship ensure that the base image is matching the
security requirements for that project and still use the same Dockerfile.
This will also ease the control of the /etc/apt/source.list
and thereby the result of apt-get update/upgrade procedure.
2. The above goal is achievable by using docker-ce feature such as:
ARG FROM="defaultbaseimage:xx"
FROM ${FROM}
For this reason, the installation of docker.io in the Zuul gating is beeing
replaced by docker-ce.
3. Third Goal is to bring consistency with the other compoenents leveraging
Helm such as the openstack-helm and potentially use bindep the same way
the LOCI images are to ensure
4. The new syntax in the Dockerfile is still commented out until the associated
image builder have been updated to use docker-ce as they have been for the LOCI
images.
Change-Id: Ife7e1be53c7c139bdc42dee42f0798e83f4fd271
Updates the Shipyard/Airflow workflow for deploy_site and
update_site to use the deployment group/deployment strategy
information from the design.
This allows for baremetal nodes to be deployed in a design-
specified order, with criticality and success criteria driving
the success and failure of deployment.
Includes refactoring of service endpoints to reduce the need
for so much data passing.
Change-Id: Ib5e9fca535ca74d1819fe46959695acfed5b65c2
This reduces the number of layers built by the Dockerfile and puts
more of hte dependency management into a common place.
Change-Id: I416ac11749f3fe8803919994e807f46d64493b5a
As mentioned in [0], the best practice is NOT to use ENV for
DEBIAN_FRONTEND as it will persists in the final image. It
appears that the common consensus is to use ARG instead so
that noninteractive DEBIAN_FRONTEND is set only during build.
[0] https://github.com/moby/moby/issues/4032
Change-Id: I6980e140ee5a51596a2f9ba4e1471e2c98599846
This patch set is meant to create a workflow that will allow us
to upgrade the airflow worker without causing disruption to the
current running workflow.
Note that we will set the update strategy for airflow worker
to 'OnDelete'. The 'OnDelete' update strategy implements the legacy
(1.6 and prior) behavior. When we select this update strategy, the
statefulSet controller will not automatically update Pods when a
modification is made to the StatefulSet’s '.spec.template field'.
This strategy can be selected by setting the '.spec.template.updateStrategy.type'
to 'OnDelete'. Refer to [0] for more information.
[0] https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#creating-a-statefulset
Change-Id: I1f6c3564b7fba6abe422b86e36818eb2cd3454ea
The one that we are currently point to is missing critical
fixes [0]. Hence we will need to pin to the current latest
deckhand merged commit which contains the fix for the deckhand
client.
[0] https://review.gerrithub.io/#/c/402182/
Change-Id: Ic523f675e527dc1b129131b75e0c0716d086f40e
We need a side car container to perform log rotation
on the log files. Logs shall be retained for 30 days.
This is the default setting and can be changed by updating
values.yaml
Also cleaned up README.md
Change-Id: I39a7797e96abd349160d753f8917f7f78f7d8797
The story requires Shipyard to get the rendered document from
Deckhand before proceeding with the rest of the workflow.
This is to ensure that the schema validation at Deckhand is
good and to fail early if need be.
This p.s. attempts to make use of the deckhand client to get
the list of rendered documents. Note that we will need to install
the DeckHand client libraries in the Airflow image in order for
this to happen.
Change-Id: I22f4165be101edfa4cef9bead6dd3d3765146c6e
Set up and do a rudimentary test of the images created
by the shipyard project to see if the images are okish.
Change-Id: I1f1c8fbcfeeafff66764ae99a176ff6a6766edce
This PS migrates the Shipyard and Airflow Dockerfiles
into this repo and adds a Makefile
We will run the following command from the root directory
to build the airflow and shipyard images
$ sudo make build_airflow
$ sudo make build_shipyard
Change-Id: I9a9fb761ce193b1c5b9c5d9589982366eb73e396
Arun Kant