Commit Graph

99 Commits

Author SHA1 Message Date
astebenkova 2216cbfec4 Add compute-kit job with DPDK enabled
+ add role for enabling hugepages

Change-Id: I89d3c09ea3bedcba6cb51178c8d1ac482a57af01
Depends-On: I2f9d954258451f64eb87d03affc079b71b00f7bd
2024-02-14 13:23:34 -06:00
Vladimir Kozhukalov cf2cdd7821 Fix prevent trailing whitespace lint command
Recently we added a jpg file to OSH documentation
but the lint job didn't run due to the job configuration.

But then for the next PR link job did run and failed
due to trailing whitespace in the jpg file.

Change-Id: I9abf8f93a4566411076190965f282375846dc5db
2024-02-02 14:12:26 -06:00
Zuul ff552d5969 Merge "Create osh-bandit role" 2023-10-27 22:12:27 +00:00
Vladimir Kozhukalov f9f487ce4a Fix deploy-env role
The role tried to include non-existing file
which was forgotten while we moved the role to this repo.
This inclusion is only actual for cases when we
consume images from a buildset registry.

Change-Id: I1510edf7bdc78f9c61f7722e2c7848e152edf892
2023-10-27 13:59:19 -05:00
Vladimir Kozhukalov 45b209ac79 Create osh-bandit role
The motivation is to reduce the code base and get rid
of unnecessary duplications. This PR is moves bandit
tasks from the osh-infra-bandit.yaml playbook
to the osh-bandit role. Then we can use this role for the
same job in OSH.

Change-Id: I9489a8c414e6679186e6c399243a7c0838df812a
2023-10-25 17:59:56 -05:00
Sergiy Markin f9b0360418 Mount extra 80Gb volume
This PS mounts extra 80Gb volume if available and mounts it to
/opt/ext_vol. It also alters docker and containerd configs to move their
root folder to that extra volume. This helps zuul gates to succeed when
a node with 40Gb volume is assigned to a zuul gate.

Change-Id: I1c91b13c233bac5ebfe6e3cb16d4288df2c2fe80
2023-10-24 18:39:51 +00:00
Vladimir Kozhukalov ae91cf3fc3 Use deploy-env role for all deployment jobs
To make it easier to maintain the jobs all experimental
jobs (those which are not run in check and gate pipelines)
are moved to a separate file. They will be revised later
to use the same deploy-env role.

Also many charts use Openstack images for testing this
PR adds 2023.1 Ubuntu Focal overrides for all these charts.

Change-Id: I4a6fb998c7eb1026b3c05ddd69f62531137b6e51
2023-09-22 15:02:07 -05:00
Mohammed Naser 70f2bc42bd Start using ensure-helm role for publishing
At the moment, our publishing scripts have their own little
way of deploying Helm so it's using the old version of Helm
which is failing linting.

This updates it so that it matches how we're running it inside
our lint playbook, and it will also fix the builds since it's
also failing because of the old version of Helm.

Change-Id: I719c2dadc3ca87912234ac13d87d63e8c7b779a7
2022-08-09 16:30:21 -04:00
Mohammed Naser 0224fb0f2c Fix chart builds
We broke the chart builds when we dropped the 'package' target in the
Makefile.   This fixes it also removes/drops the need to run the sed
locally since we point to the local folders anyways.

Change-Id: Iac8924f14c8fbb8e07b96b51790539c3ce1618e6
2022-08-09 10:46:56 -04:00
Gage Hugo 690258b221 Add clear-firewall role to bandit playbook
This change adds the clear-firewall role to the osh-infra-bandit
playbook to resolve an issue with coredns not coming up when
this job is ran on an ubuntu focal node.

Change-Id: I189ceff30271f3a478aff697b84709b19d0b09fc
2022-05-03 14:55:46 -05:00
Gage Hugo fe3e47aff0 Update Makefile for helm v3 scripts
With the removal of helm v2, the Makefile still points to the old
playbooks that try to deploy helm v2. Since we have moved to using
shell scripts to deploy helm v3, this change updates the Makefile
to point to the new scripts.

Change-Id: I6e363f3ce92a228da1301a45c754e68a859325ea
2022-04-04 20:37:07 +00:00
Stephen Taylor 4296b7d486 Use bandit 1.7.1 to avoid Python version issues
The following error is appearing when the bandit playbook is used:
bandit requires Python '>=3.7' but the running Python is 3.6.9

This change specifies bandit 1.7.1 in the playbook, which is
compatible with Python 3.5+

Change-Id: I3b43ed6de3a90af49cfc7124fdee542831f73f40
2022-02-04 11:57:04 -07:00
Gage Hugo ddb377df6d Test linting osh on helm-toolkit changes
Since most of the charts in both openstack-helm and
this repo use helm-toolkit, changes in helm-toolkit
have the possibility of impacting charts in the
openstack-helm repo and will not be caught in testing
here.

This change adds a conditional linter to lint the
charts in the openstack-helm repo if any changes
to helm-toolkit are made.

Change-Id: I0f6a935eca53d966c01e0902e546ea132a636a9d
2021-11-02 22:46:43 +00:00
Phil Sphicas a934f83187 Lint job: Install Helm before chart-testing
Ensure that Helm is installed before running ct.

Change-Id: Id8a12a0d08ad930d6052af21071cba87c127dadd
2021-10-25 19:02:48 -07:00
Gage Hugo fca6ec0277 Re-enable chart testing
A previous change to move the linting job to helm3 removed the
chart testing role. This change adds it back.

Change-Id: Ifb8b1885b4dbe8d964f46347c8c510c743af91f4
2021-10-22 17:29:10 -05:00
Gage Hugo 41e60f065c Update lint job to use helm v3
This change updates the lint job to use helm v3. This
is part of the effort to migrate from helm v2 to v3 and
to ensure each chart is compatible with helm v3.

Change-Id: Ibc8ba5d8fe8efc3637d64df61305602385e644e4
2021-10-11 16:09:52 +00:00
Gage Hugo 9030ff05da Remove unused jobs and related files
This change removes a bunch of unused and unmaintained files
and job declarations related to deploying osh-infra with armada.

Change-Id: I158a255132cd6b02607b6e1e77b8b9525cc8a3d5
2021-09-06 22:23:33 -05:00
Gage Hugo 21ada44f59 Add base helm3 job
This change adds a new script and job to deploy minikube with
helm3. This job will be improved upon in later changes as
part of the movement to helm3.

Change-Id: Ia7ef30a4e2af77508ad95191e5241d2c1b83a7c4
2021-09-02 04:54:52 +00:00
Gage Hugo 00334a7f8a Enforce release notes updates
This change removes the ignore-errors parameter for the
linter in order to enforce release note updates per chart
change.

Change-Id: I15f538be3c4ad253d5c4f64c21c0b1df10e7e5d2
2021-03-01 22:55:14 +00:00
Gage Hugo dc1f3df145 Update reno script to work in other osh repos
This change updates the releasenotes checking script in the linting
gate to be able to be ran from other osh repos.

Change-Id: Id897c7e10d0792b6df01c8a5e82e68cff2a99b0e
2021-02-22 19:46:19 +00:00
Gage Hugo b243a1e829 Track release note changes on chart change
This change adds in the script to eventually enforce release note
updates when a chart version is bumped. It will currently be set
to ignore if it fails, however once all of the charts are updated,
this should be removed to hard-enforce.

Change-Id: If3b3b26619f7288b723c0d4e6e1b97d6cfe1cf74
2021-02-08 22:36:50 +00:00
Zuul 91df918e87 Merge "Update build-chart playbook" 2020-12-07 22:27:13 +00:00
Zuul e4683420d7 Merge "Revert "Don't use opendev docker proxy"" 2020-12-07 14:51:47 +00:00
Gage Hugo 4047ff0fd4 Update build-chart playbook
The build-chart playbook task to point each chart to helm-toolkit
has a find command that when used with another repo, will
include the charts for osh-infra as well.

This change modifies the playbook to only modify requirements
in charts in the repo being published.

Change-Id: I493b4c64fe2525bac0acae06bd40c3896c918e20
2020-12-04 17:31:18 -06:00
Gage Hugo 7fdf282271 Revert "Don't use opendev docker proxy"
This reverts commit 42f3b3eaf5.

Reason for revert: dockerhub now sets a hard limit on daily pulls, lets switch back to using the opendev docker proxy.

Change-Id: I87e399c89d5736f39d7bdba2011655e5f5766180
2020-12-03 19:42:47 +00:00
Mohammed Naser ca60e1d875 Make publish jobs more generic
This will help in allowing the openstack-helm repo cleanly
publish to the seperate folder.

Change-Id: I2651c2f81191802a8f30314c4eebffdf0c2a53af
2020-11-11 19:22:55 -05:00
Andrii Ostapenko 42f3b3eaf5
Don't use opendev docker proxy
Look like using docker proxy is slower and less stable than pulling from
dockerhub directly and contributes to some part of unstable builds.

This reverts commit e3f14aaff3.

Change-Id: I9735ad35ce9240f610479a56eaa38715defa2e04
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-10-27 10:33:40 -05:00
Tin Lam 62b10c7d49 chore(pkg): updates the chart packaging
Part 2. This patch set adjusts the url once the initial packages are
make available.

Change-Id: Idfb69146d606b43c98c552d1d2c5680ccd503282
Signed-off-by: Tin Lam <tin@irrational.io>
2020-10-21 00:58:16 -05:00
Tin Lam 738c89b342 fix(job): fixes the post job
This corrects the ability to sync artifacts to tarballs.o.o.

Change-Id: Icb2b6653f263aaab173d1479d05c0209e7390c50
Signed-off-by: Tin Lam <tin@irrational.io>
2020-10-20 22:43:10 -05:00
Tin Lam da81705a47 fix(post): fixes publish job
This fixes a typo of the publish job.

Change-Id: I077feb29a8764a0b3031b34b462779c911baaee3
Signed-off-by: Tin Lam <tin@irrational.io>
2020-10-19 11:53:46 -05:00
Tin Lam e5c776e5c4 chore(pkg): updates the chart packaging
This patch set updates the ability to package (and subsequent publish)
of the charts in the OpenStack-Helm-Infra repository.

Change-Id: I6175325b0e7a668c22a7ec3ab08cae51ad4f9ab8
Signed-off-by: Tin Lam <tin@irrational.io>
2020-10-17 08:42:53 +00:00
Tin Lam 3a2d0f83b4 chore(charts): addresses issues with chart publish
This changes attempts to address the chart publish issue. Also makes
the job periodic.

Change-Id: I806da82a7eb07ce8e83ae8c023a014fa3b917193
Signed-off-by: Tin Lam <tin@irrational.io>
2020-10-16 15:15:35 +00:00
Andrii Ostapenko 7a0558bd78 Override images provided as artifacts from buildset registry
Switch from using images from defined in docker_images to provided as
zuul artifacts. Currently to be used in conjunction with [0] in
openstack-helm-images pipelines.

[0] https://review.opendev.org/741551

Change-Id: I43dbd38906e8854c87a361f2e5e479f57850252f
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-09-28 17:38:29 +00:00
Mohammed Naser 52093576aa publish: fix missing task for creating new index
The task was missed which was causing the post pipeline to fail, this
patch should fix it by adding the missing task.

Change-Id: I13955b1c9ac3899325f7397da6bf5379b3991241
2020-09-18 16:16:50 -04:00
Zuul 87535287f3 Merge "lint: enable running on changed charts only" 2020-09-18 19:21:17 +00:00
Zuul 487c50a40a Merge "Publish Helm charts into tarballs.openstack.org" 2020-09-18 17:08:07 +00:00
Mohammed Naser d4e2228f03 lint: enable running on changed charts only
At the moment, we are using --all which means run the linting on all of
the charts.  However, the problem with using --all is that it disables
version checking which means we can't enforce version changes on Helm
charts.

This patch drops it which means the chart-testing logic will go over the
changed files and make sure that it lints those charts which have
undergone changes.

Because we use a mdoel of 1 commit per merge within Gerrit, this should
still give us the exact coverage that we need without potentially
missing any linting changes.

Change-Id: I64c7896b25c1f3daaa4f61723de8a6c722aaf3a6
2020-09-17 13:50:26 -04:00
Mohammed Naser 887327aff8 Publish Helm charts into tarballs.openstack.org
This will start publishing all of the charts into tarballs.openstack.org
which should allow for easier public consumption of these charts.  This
patch covers adding publishing for openstack-helm-infra first.

Change-Id: Iaa14629c0d0c36b98c2295119af3008f14c0cd39
2020-09-11 19:13:22 +00:00
Mohammed Naser c7a45f166f Run chart-testing on all charts
Added chart lint in zuul CI to enhance the stability for charts.
Fixed some lint errors in the current charts.

Change-Id: I9df4024c7ccf8b3510e665fc07ba0f38871fcbdb
2020-09-11 18:02:38 +03:00
Andrii Ostapenko fdcbd03784 Allow parallelization in gate runner
Change-Id: I393a22cfcaecb00d14d8ac643bd4b7ffbba03b12
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-08-06 00:51:11 +00:00
Andrii Ostapenko e3f14aaff3
Use zuul docker mirror for functional jobs
Change-Id: I4899a69ef2c961094d808821b8a14d58ed02d621
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-22 09:06:27 -05:00
Andrii Ostapenko e8caaac6b4
Add ensure-pip role to functional jobs
Change-Id: Idcead2d24fd30d950130a6f1f121beac039e656f
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-16 10:52:40 -05:00
Andrii Ostapenko 0ff44fc155
Support images overrides
Adds ability to override images in osh and osh-i repositories
if docker_images dict is provided with tags following zuul-jobs
upload-docker-image naming convention.

This allows to inherit osh job from osh-images project providing
required vars.

Change-Id: I5e7acb0dd0edd6838b6f7ddb5111db0fa34912d4
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-12 11:41:49 -05:00
Andrii Ostapenko 2b4cf6a2d9 Completely switch to python3 for developers installation
This addresses an issue with using py2 as interpreter while
installing required dependencies with py3.

Also switch kubeadm-aio image to bionic.

Change-Id: I5a9e6678c45fad8288aa6971f57988b46001c665
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-07-03 09:47:43 +00:00
Gage Hugo 16676b5b63 Remove duplicate lint job entry and script
osh-infra currently has a duplicate linter playbook that is not
being used, since the other is used for both osh and osh-infra.
This change removes the duplicate entry and playbook.

Change-Id: If7040243a45f2166973dc5f0c8cd793431916942
2020-06-22 12:31:25 -05:00
Gage Hugo 26350f37aa Add new python roles to playbooks
With the latest infra update, the images used no longer contain
python by default and projects are expected to use the new
ensure roles to use packages as needed.

This change adds some of the ensure roles to a few playbooks,
additional cleanup can be done using these in future changes.

Change-Id: Ie14ab297e71195d4fee070af253edf4d25ee5d27
2020-06-19 18:07:13 +00:00
Ahmad Mahmoudi b7d3311649 (fix) Changed pip to pip3
Changed pip to pip3 to address zuul gate issues.

Change-Id: Id265b405b293af5b51b4774ca4c7465c9e9457b6
2020-06-17 18:15:07 +00:00
Andrii Ostapenko 8f24a74bc7 Introduces templates linting
This commit rewrites lint job to make template linting available.
Currently yamllint is run in warning mode against all templates
rendered with default values. Duplicates detected and issues will be
addressed in subsequent commits.

Also all y*ml files are added for linting and corresponding code changes
are made. For non-templates warning rules are disabled to improve
readability. Chart and requirements yamls are also modified in the name
of consistency.

Change-Id: Ife6727c5721a00c65902340d95b7edb0a9c77365
2020-06-11 23:29:42 -05:00
Andrii Ostapenko 6779ff041c Add yamllint check to lint job
Initial commit with bootstrapping non-voting configuration
for yamllint. Yamllint checks will be switched from 'warning'
to 'enabled' in subsequent commits together with code adjustments.

Change-Id: Ie372cb9fefb310bd044b4b03064e183f0c8c003b
2020-05-20 12:42:28 -05:00
Gage Hugo d14d826b26 Remove OSH Authors copyright
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.

This change removes all references to this copyright by the
non-existent group and any blank lines underneath.

Change-Id: I1882738cf9757c5350a8533876fd37b5920b5235
2020-05-07 02:11:15 +00:00