Commit Graph

32 Commits

Author SHA1 Message Date
Sergiy Markin 73d22a488f Restored ubuntu_bionic image build
This PS restores image build for ubuntu_bionic and adds appropriate
gates to keep it tested by appropriate functional and integrational
tests.

Change-Id: I695fb42efff29aeda737f2b9c1eaeb84b95aa57c
2023-05-24 18:37:21 +00:00
Sergiy Markin d00eaf0303 Drydock focal related upgrades
This PS implements the following changes:
- switches freeze approach to requirements-direct.txt and
  requirements-frozen.txt files
- adjusts code tabulation style according to  yapf recommendations
- replaces deprecated usage of responce.body attribute with
  responce.text
- fixes integration tests in controlled by Makefile + tox
- uplifts Helm to v3.9.4

Change-Id: I751db72eb8f670825382f11a36657112faeb169a
2023-04-26 22:32:49 +00:00
Sergiy Markin 415a8b52c5 [focal] Python modules sync with Airship project
- uplifted some python modules
- fixed tox4 requirements
- added focal build node as a default one
- added bindep.txt and bindep role to playbooks and docker image build process
- changes Makefile to reflect GoLang and dependency management changes
- upgraded Helm to v3 for chart build process
- uplifted postgresql version to 14.6
- fixed deprecated falcon.API - replaced with falcon.APP
- fixed upstream docker image publishing process

Change-Id: I307d72bb7680f6f5c71e42ad30666cf786420460
2023-04-08 08:20:22 +00:00
Ruslan Aliev 389db640e4 Change MAAS api endpoints according to v3.0.0
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I9daa8b5b8d76f2da939ce0dffdb3471ba7cd8947
2023-03-10 01:44:09 -06:00
Ahmad Mahmoudi 2a02bc9b3f Add support for ubuntu bionic base image
Adds support to build drydock image using either a xenial or a bionic base
image. Currently only xenial base is supported.
The default base image is set to ubuntu bionic.

Change-Id: I93672cf35879d8525d28e870ea83e5512c1043f9
2020-01-17 11:38:56 -06:00
Ahmad Mahmoudi 54675558fd Update image build script to run in a docker
Updated Makefile to run the build baclient package for go on the
host instead of as a docker container, to allow the Makefile be
called from another container. Reason being, in a docker-in-docker,
volume mapping requires knowledge of host filesystem path instead
of the docker daemon filesystem path.

Corrected proxy configuration in the scripts to use the USE_PROXY,
PROXY and NO_PROXY environment variables.

Updated Dockerfile to add multi-stage build, to avoid including the
golang-go package in the docker image. Stage one creates the
baclient Go library, and stage two creates the drydock image, and
copies the baclient from stage one image.

Change-Id: I29a30e870da8f44279dcd62bb1173165fa939d43
2020-01-14 17:39:27 -06:00
Roman Gorshunov 161326fe06 Fix: Drydock Exceptions docs rendering on RTD
Readthedocs failed to render Drydock exceptions with error:
> WARNING: autodoc: failed to import exception xxx from module
> 'drydock_provisioner'; the following exception was raised: No module
> named 'drydock_provisioner'

Trying to add Drydock requirements to the installed requirements list,
so that Readthedocs has all modules, including those needed for the
Drydock itself.

Unify docs building by utilizing Zuul docs-on-readthedocs template job.

Cosmetic readability changes:
1. combined all Makefile .PHONY targets into one
2. merged multiple LABEL instructions in Dockerfile into one

Change-Id: I6a9b47cffc66d739968fa886c51e25b1e09ef124
2019-08-27 22:56:32 +02:00
Drew Walters ac997fd6ec images: Allow base image substitution for Drydock
Currently, Drydock images are built using the Ubuntu:16.04 base image.
This change allows users to specify different base images using the
UBUNTU_BASE_IMAGE build argument.

Change-Id: I9ddaa89eb5262571703a3dbf7ebb6deed1505842
2019-03-07 12:09:52 -06:00
Zuul 4431661986 Merge "Fix: adding back the possibility to add arbitrary labels" 2018-11-08 20:10:03 +00:00
Roman Gorshunov 2c09054c4b Fix: adding back the possibility to add arbitrary labels
Arbitrary labels could be added as `make` parameter `LABEL=`, which is used in
att-comdev/cicd Jenkins pipelines.

Sample parameter:
'LABEL=org.label-schema.vcs-url=${GERRIT_CHANGE_URL} \
--label org.label-schema.base-image=${base_sha256}'

Sample usage:
See Jenkinsfile files under images/ directory in att-comdev/cicd repo.

In addition to that, if `COMMIT` variable is undefined when invoking `make`, we
use result of `git rev-parse HEAD` command, which should output latest git
commit ID.

[0] https://github.com/att-comdev/cicd

Change-Id: I9193ca03cb3970e3a3e1cb64ff643a4d24543636
2018-11-08 19:07:51 +01:00
Dimitrios Markou a995d86b37 [Bug fix] Remove "clean" from "chart" target
In Makefile there is a "clean" target as prerequisite to "chart"
target. That causes problems because it removes the "build" directory
that was created at the begining of the file and the helm_install.sh
script cannot write to the "build" directory in subsequent step [0].

[0]: https://github.com/openstack/airship-drydock/blob/master/tools/helm_install.sh#L30

Change-Id: Ic608630c38ff3ad7caf3a4d42e0b318d0d9d4d7e
Signed-off-by: Dimitrios Markou <dm844v@att.com>
2018-11-02 13:38:14 -05:00
Zuul 9625febab6 Merge "Fix: git commit id labels on images" 2018-10-03 00:14:27 +00:00
Roman Gorshunov 34f7d760e8 Fix: git commit id labels on images
1) Use OCI Image Specs for labels instead of custom 'commit-id=xxxxx'
   or legacy "Label Schema"
2) Fix missing git commit id labels on images (.revision)
3) Add human-readable title (.title) of the image, URL (.url), and
   a few other properties (annotations) according to the latest Specs

Change-Id: I78ae651cdcba69e3a9a7a27bafa9a3a13c8eb557
2018-09-21 03:31:12 +02:00
Scott Hussey 70e5f095d6 (fix) Add nodeSelector to Drydock Pod
Add ingress support

- Add nodeSelectors to the PodSpec for the Drydock API
  pod

- Physical nodes bootstrapping need to access the Drydock
  API to retrieve the bootactions that should be executed during
  deployment. This moves that access from a nodeport to ingress.

Change-Id: I3db41932c567cc85e89ad003389b7a019a10715f
2018-09-17 12:45:24 -05:00
Scott Hussey ecb3e01527 Bootaction client initial implementation
This is the implementation of a standalone bootaction
signal API client to be used on deployed nodes for signaling
details and results of bootactions.

Change-Id: Icc3d39253a02457a76f79d7a7c06333ae494d735
2018-09-06 16:33:15 -05:00
Scott Hussey 0f39a55942 Refactor build to include Go
Adding the baclient code to Drydock requires a refactor
of the build automation to support multiple languages
and multiple artifacts included in a single Docker image

NOTE: the go source here is a placeholder 'hello world' sample

Change-Id: I1b4883f018b33b3d4fcd7cbcb6cba660fcdc93de
2018-09-04 13:13:21 -05:00
Scott Hussey 812cef9335 (zuul) Improve image building
Improve image building to get closer to
support zuul package mirrors

Change-Id: I57285b242ddc50e8e902ad2820982a2d4b351bc3
2018-08-10 21:09:50 +00:00
Scott Hussey 246775da42 Add build data access to Drydock client
- Add CLI actions to output the build data for a node or
  a task
- Add API methods to access the Drydock API to retrieve node
  or task build data

Change-Id: I0ee01bd4b165b93c2bc0e3050554514ba40f152a
2018-07-30 10:08:22 -05:00
Scott Hussey f57301fae9 Add build data for MAAS logs
- Collect logs from MAAS when failures happen
  during deployment
- Save the logs to build data so it is available via
  API
- Add postgres integration test

Change-Id: Ied2d8539fe02a75f1f175a421b897b4f8ce07c8d
2018-07-30 10:08:22 -05:00
Roman Gorshunov 7613f1c624 Unifying proxy variables for tox and docker build
This patch adds proxy-related variables to tox and 'docker build'
commands.

Change-Id: I29fee630bd4c98406797ffec93f8f80bf1a3a043
2018-06-22 14:45:51 +02:00
Scott Hussey cc77125953 [411387] Schema update to support repo
- Support one or more repo specifications for a site
- Add object model for repository
- Add testing for repository parsing

Update freeze job with make target

- Update the requirements freeze job
  to have a mark target that rebuilds the tox
  virtualenv each run
- Update Dockerfile to create a valid /etc/protocols file

Change-Id: I9d09b7dd7226827995e23756ff968b36eaa4d16c
2018-06-07 23:08:20 +00:00
Zuul def51688e4 Merge "Update docs for developer overview" 2018-06-01 20:47:09 +00:00
Scott Hussey 20873ad4f9 Update docs for developer overview
- New diagrams and documents for developer overview
- Update conf.py for docs to work w/ readthedocs.io
- Add policy and config gen to `make docs`
- Update zuul-linter to support checked in images
- Last fix to document publishing

Change-Id: I4faa1b87032ae5b0e786aa0fd998f809124b7987
2018-06-01 14:50:05 -05:00
Felipe Monteiro e1463c6b79 [trivial] Rename tox jobs for zuul
This patchset renames the tox.ini jobs:

* coverage => cover
* lint => pep8
* unit => py35

to comply with OpenStack standards [0]. This is needed so that [1]
works seamlessly (it will enable py35 and cover jobs for Drydock).

[0] e.g. 04469a5181/tox.ini (L119)
[1] I174d5df008f6e000da1a3878afe75919312ea7aa

Change-Id: I9ead2156cde9b1bbbec8374845653825158dec19
2018-05-31 17:00:32 +01:00
Scott Hussey 53f6ccbfb7 Clean up gates
- Remove OSH-based gate as it is not relevant at this point
- Add entries for unit testing, linting and security scans
- Some trivial changes to trigger all gates

Change-Id: Icb2731573b45f3824cde8cb44ef64e9d04c017bc
2018-05-25 16:36:45 -05:00
Scott Hussey dbad75775b Libvirt OOB driver
- Create a driver to support OOB actions via libvirt API
- Update Makefile with external dependency target
- Update Makefile and tooling to support new chart pipeline
   - Add 'drydock' make target for chart building
   - Add step to install helm binary

Change-Id: I8a3984d8fd70f99a82a954b7a869eab8e30145b4
2018-05-25 11:13:53 -05:00
Scott Hussey 97f9fbd12b [Chart] SSH private key support
- Support optionally mounting a SSH private key
  to allow Drydock to interact with remote hosts via
  ssh (e.g. virsh)

Change-Id: Ib83bc53a46497af6d05f4d87595f1000d3178ec0
2018-05-01 21:18:13 +00:00
Scott Hussey b628a1bfce [390773] Support SRIOV fields in definition profiles
- Add hugepages and cpu_sets stanzas to HardwareProfile as the
  size and count of hugepages and the exact CPUs to pin for SRIOV
  are dependent on hardware.
- Add sriov stanza to a node interface to specify vf_count and
  trustedmode. These will be passthrough values as Drydock doesn't
  configure SRIOV.
- Add sriov information to the bootaction context so it can be written
  to disk on a deployed node if needed
- Allow an interface configuration to be skipped when an interface has
  no defined network_link for things like SR-IOV interfaces.
- Add kernel parameter reference support to access hardware profile
  information
- Add unit tests
- Update topology documentation for usage of HardwareProfile
  and kernel parameter references

Change-Id: Iefd326f5c6fad19dbd21300ee249019a3dfd4848
2018-03-14 14:12:51 -05:00
Stacey Fletcher e7f630651f Update Makefile
This ps updates the build commands to allow the same
format to be used in att-comdev projects

Change-Id: Ibc5be98636b2d0f29654a66261df27367ded49a6
2018-02-25 20:08:04 -05:00
Stacey Fletcher 455c3215db Add labels to Docker image
This ps allows labels to be added to the docker image at build
time

Change-Id: Iff40db4b34a549c976b0ac27f4ed0cba3bcb235c
2018-02-17 15:20:36 -06:00
Scott Hussey f9c6a34a23 Add optional image push to Makefile
- When building an image, optionally also push
  that image if PUSH_IMAGE=true
- Default value to false

Change-Id: I206e0261624b8a286d83e86bdb493e003887c8e6
2018-01-19 10:09:11 -06:00
Scott Hussey ae87cd1714 Update image and chart mgmt
NOTE: This has become a monolithic commit to get gate
      settings/scripts in place for CI

- Add Makefile with UCP standard entrypoints
- Move Dockerfile into images/drydock per UCP standards
- Add values.yaml entries for uWSGI threads and workers
- Add environment variables to chart Deployment manifest
  for uWSGI thread and workers
- Add threads and workers specification to uWSGI commandline
  in entrypoint
- Test that the Drydock API is responding
- Test that the Drydock API rejects noauth requests
- Fix Makefile utility script to work behind a proxy

Correct task success voting

Some tasks were incorrectly considered partial_success even when
no failure occurred.

- Network configuration erroneously marked messages as errors
- Update result propagation logic to only use the latest retry

The deploy_nodes task ended as incomplete due to a missing
subtask assignment

Also added a node check step to prepare_nodes so that nodes that
are already under provisioner control (MaaS) are not IPMI-rebooted.

Tangential changes:
- added config item to for leadership claim interval
- added some debug logging to bootaction_report task
- fix tasks list API endpoint to generate valid JSON

Improve task concurrency

When tasks are started with a scope of multiple nodes,
split the main task so each node is managed independently
to de-link the progression of nodes.

- Split the prepare_nodes task
- Begin reducing cyclomatic complexity to allow for
  better unit testing
- Improved tox testing to include coverage by default
- Include postgresql integration tests in coverage

Closes #73

Change-Id: I600c2a4db74dd42e809bc3ee499fb945ebdf31f6
2017-12-15 15:33:14 -06:00