Commit Graph

131 Commits

Author SHA1 Message Date
Vladimir Kozhukalov a72d9a9b89 Use divingbell-single-node nodeset
Change-Id: I7704225267a006ec687613dbda290ca4284f80ba
2024-02-21 12:01:32 -06:00
Mosher, Jaymes (jm616v) 502a74064c Add optional pre/post install commands to divingbell-apt
Change-Id: I3fdee4b128bfba80bd827fb6a64b800652cdee2f
2023-11-30 10:59:46 -07:00
Anselme, Schubert (sa246v) 78315ae509 Deprecating the Ingress Class Annotation
Upgrading htk to version 0.2.55, which deprecates the ingress class
annotation (kubernetes.io/ingress.class) with .spec.ingressClassName

https://review.opendev.org/c/openstack/openstack-helm-infra/+/891720

Change-Id: I03f3c5a33f21079492505550c9a5d42570d8506a
Signed-off-by: Anselme, Schubert (sa246v) <sa246v@att.com>
2023-10-05 22:24:14 +00:00
Ruslan Aliev 9ef6046f33 Add whitelist of packages to bypass verification
Change-Id: I459f4a241496cf98bd0bb00f3843f2b58bb397c1
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2023-05-16 18:23:27 -05:00
Ruslan Aliev 234248c272 Add readiness probe to divingbell-exec
Also add dist-upgrade verification.

Change-Id: I0716ee878e9a2fa9a557debe543996691c0540ce
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2023-04-30 23:03:17 -05:00
SPEARS, DUSTIN (ds443n) 7d533d65c3 Adding readiness/liveness probes to apt
This adds readiness and liveness probes to set daemonset to a non-ready status during dpkg usage

Change-Id: I5b9d029f1f8f696b4132a27ea29a77465babc29c
2022-10-19 15:09:04 -04:00
SPEARS, DUSTIN (ds443n) ebf0e22964 Add checks for dpkg availability
Check that dpkg is available before continuing to prevent unwanted pod restarts.

Change-Id: I6925cd074b88d10a858f044da21c7e20a7a238e5
2022-09-30 10:47:30 -04:00
Markin, Sergiy (sm515x) 0ba6181058 [zuul] Zuul gates fixes and Helm version upgrade to 3.x
Add firewall flush rules to zuul pre-update gates.

Wrap gate scripts by run-gates.sh script in order to preserve the scripts execution contexts.

Also migrated chart building process to Helm v3.x.

Fixed 020-test-divingbell.sh script.

Change-Id: I6295d55338a6a75ac43b54c092704670d61854d9
2022-09-30 01:17:39 +00:00
Walter Wahlstedt 229bbe75b0 Create option to turn on verbose logging.
Change-Id: I1ad71a603a92e44ee93e0663c7b2db216a1811ff
2022-01-19 16:34:26 -05:00
Phil Sphicas 1858d0ef37 perm: Optionally ignore missing files
The default behavior of divingbell-perm is to fail when trying to assign
permissions to non-existent files.

This change adds an option to values.yaml to skip any missing files and
proceed with the rest of the assignments.

    conf:
      perm:
        ignore_missing: true   # default is false

This may be useful in cases where files will never exist on a node, or
cases where the file does not exist yet, but will exist later. Note that
with this option enabled, a run in which files are skipped is considered
successful, so the rerun_policy and rerun_interval will determine if and
when another attempt will be made.

Change-Id: I15505d6292dda66942c66eea5a4d0666bd6bdfa7
2021-09-07 20:32:12 +00:00
Phil Sphicas 3007010064 perm: Various fixes (values hash, revert)
The hash used by divingbell-perms to decide whether or not to rerun the
permissions script was being generated incorrectly, using a fixed value
instead of actually looking at the values passed to the chart.

This change updates the hash to reflect conf.divingbell.perms, and will
rerun the script if the hash changes.

Also fixes the logic to revert permissions.

Change-Id: I74f056f69a1b7f0eb9223915b1671e1e18091483
2021-09-07 20:30:59 +00:00
Phil Sphicas c8eba1688c Update helm installation script
Updates the helm installation script to download and install v2.17.0
from get.helm.sh (instead of v2.16.9 from storage.googleapis.com).

Change-Id: I805bf95abcc97dc5dacfb6b2b0f1b671404df2cd
2021-09-07 17:20:52 +00:00
Phil Sphicas d657f7968c apt: Remove /var/lib/apt/lists before update
When divingbell-apt is managing the apt sources list, remove the
contents of /var/lib/apt/lists before running apt-get update.

Change-Id: I379af0b1a887bc81bc76f57289f35bae64e146c6
2021-03-14 06:46:08 +00:00
Phil Sphicas 918da6d055 Avoid rbd unmap failure; use HostToContainer mountPropagation
The divingbell pods use a hostPath volume for the root filesystem.
Because this mount includes /var/lib/kubelet, the pod holds a reference
to every volume mounted by every pod on the same host.

The most visible case where this causes a problem is the termination of
a pod that uses a ceph-backed PVCs. When kubelet tries to unmap the rbd
device, it is unable to do so, manifesting in the kubelet logs as:
    rbd: unmap failed: (16) Device or resource busy

This change sets the mountPropagation to HostToContainer for the rootfs
volume, so that the divingbell pods will not prevent kubelet from
releasing these devices.

https://kubernetes.io/docs/concepts/storage/volumes/#mount-propagation

Change-Id: I6e91fb9b9d7cbe852c5e6dc8b7224d6085175590
2020-11-24 23:57:54 +00:00
Phil Sphicas 55ba4cb61c Allow node selector configuration per module
This change adds the ability to configure node selectors per module. The
default node selector is 'kubernetes.io/os=linux'. For example:

    labels:
      apt:
        node_selector_key=divingbell-apt
        node_selector_value=enabled

Will result in a node selector of 'divingbell-apt=enabled'.

Change-Id: I7150c5f998afa30dce22f505be4d0d164254214f
2020-10-03 01:30:56 +00:00
Andrii Ostapenko 0779e2b468
Change helm-toolkit dependency version to ">= 0.1.0"
Since we introduced chart version check in gates, requirements are not
satisfied with strict check of 0.1.0

Change-Id: I9a9cfd54cd14c9624c20b6e4399137bd32b85c33
2020-09-24 19:42:40 -05:00
KAVVA, JAGAN MOHAN REDDY (jk330k) f9de95a6cc Update divingbell to use Ubuntu 18.04
Updated divingbell to use Ubuntu 18.04.

Change-Id: I721ffddbe8d8939303a1d38266462e751feca2f0
2020-09-23 08:57:02 -05:00
KAVVA, JAGAN MOHAN REDDY (jk330k) 634421a2e9 Move Tiller version to 2.16.9
Update Helm chart for Divingbell to use Tiller version 2.16.9.
Updated package reno>=2.5.0 to reno>=3.2.0.

Change-Id: Id6340c629986e9c6d92359cedd8839c803e0425f
2020-09-16 17:04:32 +00:00
Crank, Daniel c39963341f Fix problems with test script
1. OSH jobs now require gate_scripts_relative_path
variable to be explicitly defined.

2. Strict-mode test cases require a test package
that does not have to install dependencies, or
the test case will fail (since strict mode will
uninstall the dependency package and thus the
originally requested package).

3. Reduce redundant logging of the entire pod log
every time the pod status is checked; this was
causing long test cases (e.g. apt strict mode) to
fail.

4. Add a helper function to dump the pod log for
debugging failed test cases, since we will no
longer have the redundant logs above.


Change-Id: I7d2f6d2d161689a8744275b3d07571c83862a89c
2020-09-14 16:39:20 +00:00
Prateek Dodda 30200a54d9 Implement Security Context for Divingbell
Change-Id: Ibc93ccac6d6015faff3491211f5f8cb752a0328f
2020-03-30 23:04:50 +00:00
Anderson, Craig (ca846m) 32da2fbd4b Add ability to disable package uninstalls
Allow users to disable auto-uninstall functionality for packages.

Change-Id: Ib59ff175fc474a592118374c23974c6a9439cd72
2020-03-23 10:23:20 -07:00
Zuul db4f382b59 Merge "Update dpkg commands to be non-interactive" 2020-03-20 20:37:00 +00:00
Michael Beaver b98efc4f29 Update dpkg commands to be non-interactive
The current `dpkg --configure -a` command does not always work if the
package that needs to be configured has a modified conffile which can
require user input to resolve. This change adds flags to make these
lines work as intended in that scenario.

Change-Id: I8f459b0c1c2fc7ecbe1ff478bdb77fd9af31dc90
2020-03-19 14:10:44 +00:00
Crank, Daniel f0eb0b7582 [ad-hoc] Fix test case exit conditions
While working on another change, I discovered conditions
in many test cases that echoed fail messages but did not
actually exit, so the gate could succeed even though some
tests failed. This patchset aims to fix those problems, and
then fix the problems masked by those problems:

1) fix bug in revert function of file permissions module
preventing permissions from being reverted.
2) fix various syntax and logic problems in test script
3) add wait_for_tiller_ready function to avoid race condition
with test script using helm too early
4) add install for ethtool in test script
5) ignore ethtool pod failures (see note #1 in [0])
6) make logging of test results more uniform
7) Fix error message logic in perm.sh
8) Fix case in _shcommon.tpl where error message was not
logged, causing test script to unnecessarily wait for
container timeout

[0]: https://review.opendev.org/676010

Change-Id: I22182d35250c37c96e73d9f5f49abfb2246f2a35
2020-03-12 15:25:30 +00:00
Drew Walters 7d968220c7 Add SECURITY.md
All Airship projects are moving to GitHub issues. This change adds a
GitHub security policy that links to the official Airship vulnerability
management process [0]. When users on GitHub click "New Issue" on this
GitHub repository, they will see an option to report a security
vulnerability, which will direct them to our official policy.

[0] https://airship-docs.readthedocs.io/en/latest/security/vulnerabilities.html

Change-Id: Iaf060dd0085c21f0c4f18f100e3e053b5ceedbed
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-02-20 17:07:16 -06:00
KAVVA, JAGAN MOHAN REDDY (jk330k) 37594c8d16 Add Docker default AppArmor profile to divingbell
This adds default AppArmor profile to divingbell.

Also, update to gate script to install ethtool if it is not present.

Change-Id: I7abb13a533b596f4db5fe65fdae5eb7fc57ec00a
2020-02-13 14:43:44 -08:00
Michael Beaver fe0a034ec7 Add --no-install-recommends to apt install
This change adds the --no-install-recommends flag to the apt-get
install command portion of _apt.sh.tpl. This will modify Divingbell
to only install direct dependencies of packages instead of following
the default apt behavior, which is to also install recommended packages

Change-Id: I118a72e1e591101b0e2878e088e9fbaa96067d2c
2020-01-29 18:29:06 -06:00
Drew Walters fe270ec595 apt: Add whitelist for strict mode
This change adds a whitelist of packages that will be ignored when using
strict mode.

Change-Id: I9138f35a72618100e6094575271f6160336332f4
Signed-off-by: Drew Walters <andrew.walters@att.com>
2020-01-27 21:23:27 +00:00
Crank, Daniel 3cc1620319 Remove 'autoremove' from strict mode apt purge
This patchset makes two changes for strict mode only:

1) Removes the --autoremove flag from the apt-get purge
   command line
2) Causes the install stage to call apt-get install on
   all packages regardless of whether they're already
   installed. This will have the effect of marking all
   requested packages as manually installed if they
   were previously auto-installed.

Change-Id: Ic1a39205c941973af9d82685180d28457ea2011f
2020-01-25 13:15:46 -06:00
Crank, Daniel 44525162a5 Add "strict" mode for apt package removal
Currently, divingbell-apt will only remove packages that aren't
on the current requested package list when they were previously
installed by divingbell-apt. This patchset adds a "strict" mode
which causes it to remove packages not on the requested package
list regardless of whether divingbell installed them (i.e., it
can remove unwanted packages that were part of the host's base
image).

Change-Id: Ie2ba5d47646bfaaf030cb54673e644ab0e917fd4
2020-01-24 12:19:22 -06:00
Schiefelbein, Andrew (as3525) as3525@att.com ac357b9bff This is to allow for ganged install of packages instead of single
package installations with apt

Change-Id: Ifd268e7eca212fb5686b30213c1c7c1e47f5eb25
2020-01-17 16:03:03 -06:00
Phil Sphicas 788501e806 apt: chart update: allow conf.apt.packages as map
This change allows conf.apt.packages to be defined as a map of lists,
allowing for logical grouping and easier substitution when values.yaml
is being assembled from multiple sources.

The existing format (conf.apt.packages as a list) is still supported.

Change-Id: I4d4c09723b2e9ac1f0ecf847e786d991cc6e669a
2020-01-07 12:31:53 -08:00
Phil Sphicas 524c1b1e32 Fix airship-divingbell-ubuntu zuul gate
Fixes the airship-divingbell-ubuntu zuul gate.

Change-Id: I83642d43f4a4ae8a4882b120e965fcacd166700a
2020-01-07 12:31:53 -08:00
Drew Walters 66e9241d37 docs: Update copyright footer
During the recent Airship Working Committee meeting, the committee
addressed feedback from the Airship confirmation review [0]. One such
item was concerned with copyright footers mistakenly claiming rights to
all Airship documentation.

This change updates the footer to attribute documentation to the
Divingbell authors.

[0] https://etherpad.openstack.org/p/airship-wc-meeting-2019-12-09

Change-Id: I954141c18175a263973d4288c7d559c0419e08dc
Signed-off-by: Drew Walters <andrew.walters@att.com>
2019-12-09 22:05:56 +00:00
Sphicas, Phil (ps3910) 0576ecde4b doc update for blacklistpkgs
blacklistpkgs supports a list of package names only.

This updates the documentation to match the current functionality.

Change-Id: Ic6f586aa89773ea22e9bf54610ea968243583ac5
2019-11-26 15:58:50 -08:00
Zuul 010b5c6c03 Merge "apt: Add allow-downgrades option per package" 2019-10-17 18:26:23 +00:00
anthony.bellino d917166a73 apt: Add allow-downgrades option per package
This change adds the ability to include the --allow-downgrades
option per package install.

Change-Id: I2e0c6f11a51c1b78994e77084e3b2046c179d888
2019-10-17 03:11:19 +00:00
Evgeny L 9be717e860 Allow to configure service network policy
The patch introduces network policy configuration similar
to openstack-helm services. It allows users to configure
policies depending on the environment.

* Network policies are disabled by default.
* When enabled default policies allow all ingress and
  egress traffic (i.e. policy set to {}), this may be
  changed in future patch-sets.

Change-Id: I2adb5e652c1da0a1982ab18c498f033910a47cd8
2019-09-27 20:48:09 +00:00
Anderson, Craig (ca846m) e541ec72b7 Fix gate script trackback printout
Change-Id: I9111cddf89ad57641b163309e5d2202a44cd36eb
2019-08-26 13:53:58 -07:00
Drew Walters 2e5ffaccca apt: Add full-system upgrade feature
Currently, the APT daemonset allows the installation of new packages or
upgrade of existing packages to a newer version. Sometimes, it may be
desirable to trigger an update for all packages. This change introduces
the ability to trigger a full-system upgrade using the .conf.apt.upgrade
chart value. The new option is disabled by default.

Change-Id: I611422c2093b9dbbae4e2d7cc05ebd726e895c88
Signed-off-by: Drew Walters <andrew.walters@att.com>
2019-08-21 16:07:54 +00:00
Zuul c503961841 Merge "ignore ethtool failures, other gate improvements" 2019-08-16 20:17:27 +00:00
Anderson, Craig (ca846m) 34c6d930e4 ignore ethtool failures, other gate improvements
Gate enhancements:

1. On certain opendev hardware, it's not possible to change
   ethtool tunables, or the expected tunables are unavailable.
   Until we have a mechanism to schedule to the right hardware,
   we will issue a warning whenever these tests fail instead of
   failing the gate.
2. Add a check so that gate script will not run until there are
   no other instances of the gate script running on the same node,
   as this can cause spurious gate failures.
3. Print gate script tracebacks in the event of gate script faliure
4. Increase check interval for two exec tests that were seen to fail
   on one ocassion due to insufficient wait time.

Change-Id: Ifdbb203a1b14242e3801ba10ef7e932931771878
2019-08-16 00:30:26 -07:00
Zuul 8e523b029c Merge "Change DaemonSet apiVersion to apps/v1" 2019-08-12 21:36:19 +00:00
Roman Gorshunov 1504533fb1 Change DaemonSet apiVersion to apps/v1
DaemonSet apiVersion: extensions/v1beta1 is deprecated starting from
Kubernetes v1.8.0-alpha.3 [0].

DaemonSet uses apiVersion: apps/v1 starting from v1.9.0 [1].

We run Kubernetes v1.13.4 and up at the moment.

[0] -
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.8.md
[1] -
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.9.md

Change-Id: Ic286e208836cf17be09fa78ba4d0f45084ae47fb
2019-08-01 20:25:43 +00:00
Roman Gorshunov 13ce2562b7 Fix: read the docs project name, webhook trigger
The docs-on-readthedocs template job requires rtd_project_name
parameter, because it's different from the project name.

Change-Id: Ibb2610c9bf997e77803bf10fdb1ee1c5423c6c96
2019-08-01 20:24:36 +00:00
Zuul d3b1a5c985 Merge "Various gate fixes to make gate green" 2019-08-01 19:46:37 +00:00
Zuul f727f6adf1 Merge "Add release uuid annotation to POD spec" 2019-07-30 19:04:16 +00:00
Anderson, Craig (ca846m) c68a3ff61f Various gate fixes to make gate green
1. There is an ocassional timing issue when container logs are
   unavailabile at certain points in the crash loop at the same
   time the gate script tries to request them. The gate will now retry
   this operation, instead of terminating right away with failure.
2. Re-enable uamlite security context so that useradd operations would
   succeed.
3. Change apt pinning tests to use a version of the package that is
   available in the apt repo. Upstream repos change, so we should not
   pin to an explicit version that will be removed in the future and
   break the gate.
4. Update helm version to 2.14.1 to sync with openstack-helm-infra
5. Fix divingbell build script: git --depth=1 incompatible with explicit
   non-master commit checkout
6. Enhance overrides test case #7 to test for the issue identified in
   [0].
7. Change hostname scheduling to match minikube hostname now configured
   by OSH gate, instead of using the node's actual hostname
8. Re-enable gate voting

[0] https://storyboard.openstack.org/#!/story/2005936

Depends-On: https://review.opendev.org/671875/
Change-Id: Iad983ce363711e16ccd54e663c23d30a4a6a1177
2019-07-29 14:42:18 -07:00
Zuul 49fc3ccc7e Merge "Update uamlite.sh to handle empty user_sshkeys arrays" 2019-07-24 15:48:58 +00:00
Kumar, Nishant(nk613n) d5a65962fe Add release uuid annotation to POD spec
Change-Id: I6158af07b15dbc098ae4e67c949b00c293b30894
2019-07-24 14:50:25 +00:00