StarlingX stopped supporting CentOS builds in the after release 7.0.
This update will strip CentOS from our code base. It will also remove
references to the failed OpenSUSE feature as well.
Story: 2011110
Task: 49958
Change-Id: I7fe10de6fd71701f142c8416b91cac3a338bf740
Signed-off-by: Scott Little <scott.little@windriver.com>
Updating the rsa ssh host key based on:
https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/
Note: In the future, StarlingX should have a zuul job and
secret setup for all repos so we do not need to do this
for every repo.
Needed to rename the secret, because zuul fails if like-named
secrets have diffent values in different branches of the same
repo.
Partial-Bug: #2015246
Change-Id: I37b44a091fbd9c820c6047627906d05dadbcdd23
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
This change will allow this repo to pass zuul now
that this has merged:
https://review.opendev.org/c/zuul/zuul-jobs/+/866943
Tox 4 deprecated whitelist_externals.
Replace whitelist_externals with allowlist_externals
Partial-Bug: #2000399
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: Icf70dcde638cb86683e9dd6e6a96e0d2a2309438
Add missing {centos,debian}_helm.inc files used by scripts in
build-tools. The centos file used to reside in starlingx/config repo,
but it really belongs here.
TESTS
======================================
Build packages, then run build-helm-charts.sh and make sure
"stx-monitor-helm" tarball is generated.
Story: 2010226
Task: 46419
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: Idc888781963b5387f397c0645db3080ca0582df3
Helm chart DEB packages: change target arch from "all" to "any" (ie
arch-independent). This is required for starlingx debian tools to work
correctly.
Story: 2010226
Task: 46405
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
Change-Id: I83fa678f0195442b97010408b60fba11fccc2647
Use debian method to apply local patches instead of patching in rules.
Add "dl_path" and "src_files" in meta_data.yaml to download tarball and
copy local files.
No need dl_hook
Test Plan:
Pass: successfully build monitor-helm-elastic.
Pass: No difference comparing with the result of dl_hook
Story: 2009101
Task: 43838
Signed-off-by: Yue Tao <Yue.Tao@windriver.com>
Change-Id: I068f3ae0cbf9e0661abc12adcec005be145f2462
Use debian method to apply local patches instead of patching in rules.
Add "dl_path" and "src_files" in meta_data.yaml to download tarball and
copy local files.
No need dl_hook
Test Plan:
Pass: successfully build monitor-helm.
Pass: No difference comparing with the result of dl_hook
Story: 2009101
Task: 43838
Signed-off-by: Yue Tao <Yue.Tao@windriver.com>
Change-Id: I441540b54e27f63d8a84d3a83f65679dc6dfaf4b
Update remaining StarlingX packages with hardcoded TIS_PATCH_VER to
use PKG_GITREVCOUNT where possible, with offsets as needed to ensure
the version is incremented above the hardcoded version.
Story: 2008455
Task: 41462
Signed-off-by: Don Penney <don.penney@windriver.com>
Change-Id: I63186d0561e47cb55f40856a4e01ff02f34406f3
Newer tarballs generated from github contain a different top-level
directory than what are currently contained in the CENGN mirror cache.
By parsing the first entry in the tarball we can figure out the
proper directory.
Tested by building both the new and old tarballs and verifing the
RPM builds correctly and the log file has correct directory.
Change-Id: Ie484cb2f5c615917afd320749ab3e68c8f7a8daa
Closes-Bug: 1857049
Signed-off-by: Saul Wold <sgw@linux.intel.com>
This adds support for Helm v3.
- 'helm init' and initialization is no longer required
- 'chartmuseum' is used as a drop-in replacement for 'helm serv'
- all Charts require the tag: apiVersion: v1 (or v2)
This updates nginx-ingress chart to specify apiVersion.
Change-Id: I6d084dafccbbc192ba5e1f5a998540e2728c9e22
Story: 2007000
Task: 39328
Depends-On: https://review.opendev.org/719962
Signed-off-by: Jim Gauld <james.gauld@windriver.com>
Add support for configuring logstash ingress rules
for the logstash service.
Story: 2007221
Task: 39560
Change-Id: If78635667e1acfe171e48aa106d5f31ff3cdb9b2
Signed-off-by: Matt Peters <matt.peters@windriver.com>
Logstash should not be using a custom port for collectd
input from the K8s NodePort range since it might cause a
conflict if the port is allocated to another service.
Therefore, logstash will use a proper NodePort value
reserved by the nginx-ingress service.
Remove hostNetworking from Logstash since it is no longer
required.
Add support for specifying a helm nodePort value for TCP/UDP
services exposed for the nginx-ingress controller.
Story: 2007221
Task: 39544
Change-Id: Ifdb750a578ad6a5e7b96019e7caf6fe1231fa1be
Signed-off-by: Matt Peters <matt.peters@windriver.com>
The readiness probe endpoint was changed in the 7.6.0 chart update. This
prevents the application from being reapplied after being removed since
elasticsearch stores its state in the persistent volume, meaning that
when it is reapplied it will try and fail to achieve quorum. Changing
the readiness probe to make a local healthcheck lets it advance to the
next master pod, instead of getting stuck on the first master pod.
Story: 2007221
Task: 39491
Change-Id: I720cec5a7a50a0207669258b97497aba9c99efee
Signed-off-by: Simon Cousineau <Simon.Cousineau@windriver.com>
This allows restart when elasticsearch configmap changes.
Change-Id: Ic16bc98fc3fa3d39d8448965a3b3463c03372d27
Signed-off-by: Simon Cousineau <Simon.Cousineau@windriver.com>
This change enables Logstash to receive collectd logs.
Change-Id: Id25138f7f153b38ef6db362f0390325ad6f58729
Signed-off-by: Simon Cousineau <Simon.Cousineau@windriver.com>
This change allows us to define additional parameters for the
metricbeat and filebeat updateStrategy configs.
Change-Id: Ie586ad21311f743a3ee7b2da911b0d01361daca5
Signed-off-by: Simon Cousineau <Simon.Cousineau@windriver.com>
Container logs were being collected directly from the 'log' input.
Container logs are now being collected using Filebeat's 'container'
input, which adds the Kubernetes meta-data to the logs.
Change-Id: I8f1e01442ee4d52c2882ae4bfd3529cc2891f075
Signed-off-by: Simon Cousineau <Simon.Cousineau@windriver.com>
Add support for different nodeSelector and tolerations configs for
metricbeat daemonset and deployment.
Story: 2007221
Task: 39335
Change-Id: I1e96dffac42bf746451618be3da6dcfb21f389c2
Signed-off-by: Simon Cousineau <Simon.Cousineau@windriver.com>
Updated stx-monitor to use official Elastic 7.6.0 helm-charts for Kibana,
Filebeat, Metricbeat and Logstash.
This change included updating the helm-elastic tarball sha to the newest
release and updating helm-elastic patches accordingly. Patched the
charts to support previous stx-monitor functionality and updated the
manifest to conform to the new config format.
Change-Id: I03b356b2b7acda0fc8336704784bd870252dd5b8
Default to using the same image as the main container
in logstash extra init containers so that the same docker
registry as the main container image can be used.
Closes-Bug: 1869256
Change-Id: I0555c31208f25146ba74b1a7b2f926abaf31e060
Signed-off-by: Kevin Smith <kevin.smith@windriver.com>
Modify the readiness probe in the elasticsearch chart so that
2 sets of cluster health parameters can be set. One set of cluster
health parameters will be used when no connection to the master
is possible or there are less than 2 elasticsearch data nodes in the
cluster. The other will be used when connection to the master is
possible and 2 or more elasticsearch data nodes are present in the
cluster.
This is necessary as in various cluster recovery scenarios and AIO-SX
only local node health should be checked, whereas when modifying
elasticsearch data nodes we need to ensure no recovery is in progress
before advancing to updating the next data node.
Change-Id: I8125c3c5b87e081a00907c519e3d513c74031c70
Closes-Bug: 1869001
Signed-off-by: Kevin Smith <kevin.smith@windriver.com>
The metricbeat pod requests a hostPath volume which mounts the
rootfs of the host into the container. This is required for
metricbeat to read the filesystem usage from within the container.
The rootfs mount is also bind mounting all subpath mounts within the
container which includes the DRBD filesystems. When SM attempts
to unmount the DRBD filesystems as part of the swact operation, the
metricbeat container is still holding on to the mounted filesystems,
preventing the DRBD devices to be shutdown.
To address this, the metricbeat container will specify the mount
propagation so that any changes to the host filesystem are also
reflected inside the container. This will ensure that when the host
performs the unmount, the container will also release the mount
reference.
In the future, security policies should be added to ensure other
application containers do not perform host mount operations without
having the necessary privileges to access the host filesystem.
Change-Id: I43c6dcbe6a438b357532263be5b04f122b587c70
Closes-Bug: 1866099
Signed-off-by: Matt Peters <matt.peters@windriver.com>
Clean up the Add-imagePullSecrets.patch to remove redundant
imagePullSecrets.
Update manifest with correct imagePullSecrets overrides in
order to allow stx-monitor to pull images from local docker
registry on each node.
stx-monitor app applied successfully and verified that each
running pod has the following in its spec:
imagePullSecrets:
- name: default-registry-key
Change-Id: I330004bd44b255a46e2d1fb29ada63dbcd46e881
Closes-Bug: 1860132
Signed-off-by: Angie Wang <angie.wang@windriver.com>
Move the beats setup scripts to the values files so they can be
overridden.
Additionally, set the maxUnavailable parameter for the elasticsearch
charts so that a maximum of one pod at a time will be down
when the application is re-applied or upgraded.
Change-Id: I1362788dae30424f1e7afe452a3ca3b5a306b94c
Closes-Bug: 1860696
Signed-off-by: Kevin Smith <kevin.smith@windriver.com>
Metricbeat couldn't collect state_replicaset due to the missing RBAC role definition
so replicasets was needed in apps apigroup.
This change is based on:
122647b905
Change-Id: Id0be124971820ccf91735d3f40cdc75d96fa57d4
Closes-Bug: 1860532
Signed-off-by: Abdelfattah Saafan <abdelfattah.saafan@windriver.com>
Remove all references to images that are not used by the app
to prevent downloading them. Removals include images for
testing kibana and enabling logstash exporter.
Also explicitly add a reference to the nginx defaultbackend image
in the manifest as it fails to appear in the image download
list in the application images yaml file. This is possibly an
image parsing issue to be looked at later.
Change-Id: Id6ec19e6c8e5df4aa543a3e4f6b3c36b954d0971
Partial-Bug: 1856197
Signed-off-by: Kevin Smith <kevin.smith@windriver.com>
In order to improve recovery from scenarios such as Dead Office
Recovery, set clusterHealthCheckParams 'local=true'
Verified also with mtce actions host-lock/unlock, swact.
Change-Id: I67324ab1db87b67dd3538b0160138aee333bc61f
Story: 2006990
Task: 37767
Signed-off-by: John Kung <john.kung@windriver.com>
The commit https://review.opendev.org/#/c/696123/ has
updated to handle various image formats for application
to support pulling images from local registry.
This commit updates the manifest to remove the unnecessary
image references and specify "null" for the images that
are not required for the app.
Change-Id: I6517ba3cb7173c70b8c00807dcb58f73ec426c62
Depends-On: https://review.opendev.org/#/c/696123/
Closes-Bug: 1851294
Signed-off-by: Angie Wang <angie.wang@windriver.com>
Add an initcontainer to elasticsearch-curator that will
poll elasticsearch for the number of data nodes. If the number
of data nodes found is 1, indicating either a locked controller
node or AIO-SX, write a file with a LIMIT_FACTOR of 2. This
file will be sourced and used to adjust the existing index
limits accordingly before curator is run in the main container.
As the curator image does not include curl, use the filebeat
image, which also requires mounting a dummy directory:
/usr/share/filebeat.
Change-Id: I96ac6ee02e1446ca251a4c0cce5397669e399cea
Closes-Bug: 1852472
Partial-Bug: 1852462
Signed-off-by: Kevin Smith <kevin.smith@windriver.com>
Update manifest with imagePullSecrets or serviceaccount in order to
allow stx-monitor to pull images from local docker image repository.
The armada manifest has registry.local included until sysinv is updated
to be able to handle image overrides with the various elastic formats.
Verified that 'kubectl describe pods -n monitor' Image: are
pulled from registry.local
Verified elastic-curator and distributed cloud configuration with
stx-monitor.
Change-Id: Ia63e593bef96ea011cffbcca49ccf5a3a0f2f4d7
Partial-Bug: 1851294
Signed-off-by: John Kung <john.kung@windriver.com>
This update adds a preStop lifecycle script to be run on
elasticsearch-data pods when they are also functioning
as master capable, as will be the case for AIO-DX without
worker hosts. The script will check if the data node is
master capable, and if so sleep for a number of seconds
configured by an environment variable, defaulted to 100.
The elasticsearch-data termination delay is desired when
the single master capable only pod is on the standby
controller and that controller host is locked. The delay
in termination will leave the 2 elasticsearch-data pods up
to maintain master quorum until the elasticsearch-master
pod has migrated over to the active controller.
This update also removes the check for green cluster
health status in the elasticsearch readiness probe.
Story: 2005733
Task: 37321
Change-Id: I6991ffff4e94342f9190896e92790cdaa8de9dea
Signed-off-by: Kevin Smith <kevin.smith@windriver.com>
The metadata "maintain_user_overrides" is used to determine whether
the user overrides will be reused or not when updating the application
to a new version.
Set it true for stx-monitor.
Change-Id: Icfb5963f6396d099a61ca9d76963dc63531595b5
Partial-Bug: 1849486
Signed-off-by: Angie Wang <angie.wang@windriver.com>
For the statefulsets being launched, we want to wait until
all pods in the statefulset go ready before moving on to
creating/updating other pods.
Story: 2005733
Task: 37303
Change-Id: I06b6c63574778f83944bbf96c4fadc4312b8682b
Signed-off-by: Kevin Smith <kevin.smith@windriver.com>
Update the init containers to have access to the same set of
environment variables as the runtime containers. Specifically
NODE_NAME is required to ensure the name parameter is set
properly.
Change-Id: I6fa2494f75a7cd88f8f8f141f44ce34928a24956
Story: 2005733
Task: 36656
Signed-off-by: Matt Peters <matt.peters@windriver.com>
Modify the podManagementPolicy for elasticsearch pods
to OrderedReady so no outage is incurred during application-updates.
This setting will also affect initial application-apply, so the elasticsearch
helm chart is also modified to set the initial_master_nodes to
just the first master pod so it will go ready and the apply can proceed.
Change-Id: I87e73f6000430f81f2dc08d37ecd523682377ce3
Story: 2005733
Task: 37290
Signed-off-by: Kevin Smith <kevin.smith@windriver.com>
Add a root filesystem mount point for metricbeat to
enable access to the host file systems for collecting
filesystem specific metrics.
The filesystems are mounted under /hostfs within the
beat Pod, therefore the system.filesystem.mount_point
value must be filtered by Logstash to rewrite the path
to the original mount path on the host.
Change-Id: I7d98560eb00928e39a1c97ebc3e6b5dda8f5b278
Story: 2005733
Task: 37291
Signed-off-by: Matt Peters <matt.peters@windriver.com>
Scott Little