This commit removes the mgmt_ip field from the database i_host table.
The mgmt_ip is necessary for MTCE and other calls, this value is
returned by the database now, using the address name and network type
by the function: get_address_by_host_networktype.
Test Plan:
PASS: AIO-SX installation
PASS: AIO-SX mgmt reconfiguration
PASS: AIO-DX installation
PASS: Standard installation
PASS: DC install
PASS: DC AIO-SX mgmt reconfiguration
PASS: Sanity for AIO-SX, AIO-DX, DC
Story: 2010722
Task: 49835
Signed-off-by: Teresa Ho <teresa.ho@windriver.com>
Change-Id: I577fa51df261846292f2e9dff2aeba5fff431848
Introduce an additional check to block Kubernetes upgrades on the
kube-upgrade-start step if applied apps marked to be post updated are
incompatible with the target k8s version. This will prevent
incompatibility gaps in the upgrade process, i.e. avoiding that the
post-update stage is reached while running an incompatible app.
Application developers can leverage the auto update feature and the
pre-update step in case incompatible apps need to be updated before
deploying a new Kubernetes version.
Test Plan:
PASS: build-pkgs -a && build-image.
PASS: Deploy platform with Kubernetes 1.24.
Apply a metrics-server version containing
supported_k8s_version:minimum: 1.24 and
supported_k8s_version:maximum: 1.27.
Add a new metrics-server bundle containing
supported_k8s_version:minimum: 1.28 and
supported_k8s_version:maximum: 1.29.
Start Kubernetes upgrade to 1.28.
Confirm that the upgrade was blocked.
PASS: Deploy platform with Kubernetes 1.24.
Apply a metrics-server version containing
supported_k8s_version:minimum: 1.24 and
supported_k8s_version:maximum: 1.28.
Add a new metrics-server bundle containing
supported_k8s_version:minimum: 1.28 and
supported_k8s_version:maximum: 1.29.
Start Kubernetes upgrade to 1.28.
Confirm that metrics-server is updated during post-update step.
Closes-Bug: 2066042
Change-Id: Ie5d7e872ae49f833bf1cf1a4eb80a7f50dad8920
Signed-off-by: Igor Soares <Igor.PiresSoares@windriver.com>
Starting on stx-11, patching will be managed by the
USM feature, so all patch related health-checks and
information should be gathered via USM APIs.
This commit changes the patch current health checks
to point to USM APIs. However, there is not an endpoint
that returns per-host data, and deploy host-list only
returns data if there is a deployment in progress, so
there will be a followup commit to change this logic
again if an endpoint is created to return data as the
legacy sw-patch query-hosts command.
This commit does not affect legacy upgrades, since
on legacy upgrade the health-check code runs on the
FROM release code base.
Test Plan
PASS: run 'deploy precheck' and verify the output
PASS: run 'deploy precheck' on a system with [Fail]
patch current health check, apply the fix and
verify the output
Story: 2010676
Task: 50107
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
Change-Id: I69c630468543ac99d8ba79f21646573a91140616
This commit creates the update_users, create_roles
and create_projects methods in
openstack_config_endpoints.py, which is required in [1]
in order to set the required services, roles and users
during keystone bootstrap.
[1]: https://review.opendev.org/c/starlingx/ansible-playbooks/+/915284
Test plan:
Note that all of the test cases were performed with the changes from
[1].
1. PASS: Verify the services, roles and users were created after
the keystone bootstrap.
2. PASS: Validate that the admin and sysinv users have the
ignore_lockout_failure_attempts set to true.
3. PASS: Validate the sql dump of the keystone database generated in
a subcloud deployment in relation to the one generated before the
changes from [1].
Story: 2011035
Task: 49966
Change-Id: I5be50bec1174a451d11e4dbc2eff0b01fc182576
Signed-off-by: Raphael Lima <Raphael.Lima@windriver.com>
Some of the k8s Opaque type secrets data has None values, this
is failing the "system k8s-certificate-list" command.This fix
addresses this issue by checking the "data" field in the
secret
Test Cases:
Pass: Have a secret with no data, run "system k8s-certificate-
list" comamnd and verify it is listing the certificates
Closes-bug: 2065926
Change-Id: I85ea4341d71016c6064b22860f761b3f77f1619c
Signed-off-by: amantri <ayyappa.mantri@windriver.com>
Script introduced by [1] broke USM deploy start due to not
parsing the fourth parameter passed to the scripts during
the data migration.
This commit fixes the issue.
[1] https://review.opendev.org/c/starlingx/config/+/919624
Test Plan
PASS: execute deploy start successfully
Closes-bug: 2065910
Change-Id: I03a9e055bc487b423c385ee7fb69fc31396ac734
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
This change added ipsec-auth client invocation in controller_config,
worker_config and storage_config init scripts that will run during
first reboot after installation, to configure and enable IPsec for the
node.
Note that IPsec for the first controller is configured and enabled by
bootstrap ansible playbook. So the invocation of ipsec-client is
skipped in controller_config.
Test Plan:
PASS: DX system, install controller-0, bootstrap and unlock, verify
IPsec is configured and enabled.
PASS: Install controller-1, verify IPsec is configured and enabled
after first reboot, SAs are established, and controller-1 is
online.
PASS: Install a worker node, verify IPsec is configured and enabled
after first reboot, SAs are establishe, and the worker node is
online.
PASS: After controller-1 and worker hosts are unlocked, verify SAs are
established among all hosts, and all nodes are in unlocked,
enabled and available states.
PASS: DC system with SX subcloud, verify System Controller and subcloud
are deployed successfully. In central cloud, SAs are established
among all hosts, all nodes are in unlocked, enabled and available
states.
Verify subcloud are online, managed, and all resource are in
in-sync states.
Verfiy user can ssh to subcloud.
Story: 2010940
Task: 50021
Depends-On: https://review.opendev.org/c/starlingx/ansible-playbooks/+/917868
Signed-off-by: Andy Ning <andy.ning@windriver.com>
Change-Id: I5572b4b50238c0c5e76cc04cabd24078e9defa5b
This commit updates various codes related to the "protecting mgmt
network with IPsec" feature, to support stx 9 to stx 10 upgrade. With
this changeset, the IPsec feature will not break current upgrade. There
will be follow-up changes to fully support stx 9 to stx 10 upgrade with
the IPsec feature.
Test Plan: (AIO DX system)
PASS: Packages build, image build.
PASS: Without IPsec feature enabled, upgrade from stx 9 to stx 10,
verify all steps are successful.
PASS: With Ipsec feature enabled, upgrade from stx 9 to stx 10, verify
all steps are successful.
(IPsec is not configured and enabled during upgrade)
Story: 2010940
Task: 50095
Change-Id: Id708f540f39ed98a9e05cc12a71a5a4fa3d5bfa5
Signed-off-by: Andy Ning <andy.ning@windriver.com>
This commit fixes a bug that was causing container images provided via
user overrides not to be downloaded when applying StarlingX
applications. The problem was caused by two issues:
* The way the chart override filename was assembled in the image
download method got outdated due to a previous commit [1]. The
global namespace once present in the filename is not mandatory
anymore as it represents the namespace of FluxCD resources and not
necessarily the namespace of all underlying resources. In order to
fix this the download method now uses a helper function, created on
the same referenced commit, which aims to standardize how override
filenames are assembled.
* The implementation assumed that override files loaded using ruamel
always have a "data:values" section, which is not the case for
every supported app. This was causing the overrides parsing
function to receive an empty dictionary, thus returning no images
to override. That assumption was removed and the whole file is now
being parsed given that image tags can be present in different
subsections of the yaml file.
[1] https://review.opendev.org/c/starlingx/config/+/887430
Test plan:
PASS: build-pkgs -a && build-image
PASS: AIO-SX fresh install
PASS: Override vault-manager image using "system helm-override-update".
Apply vault app.
Confirm that sysinv attempted to download the image provided via
user overrides.
PASS: Apply vault app.
Confirm that default images were correctly downloaded.
PASS: Override ceph_config_helper image using
"system helm-override-update".
Apply platform-integ-apps.
Confirm that sysinv attempted to download the image provided via
user overrides.
PASS: Apply platform-integ-apps app.
Confirm that default images were correctly downloaded.
Closes-bug: 2065699
Change-Id: I9cc6b8522aaf2624377814b2249e3c00da9ec424
Signed-off-by: Igor Soares <Igor.PiresSoares@windriver.com>
As it was before, sysinv placed a gateway address in every interface
config file for which the corresponding address pool had a gateway
specified. This allowed multiple default gateways to exist
simultaneously in the system, causing network problems.
This change improves the hieradata generation logic so that only one
interface has default gateways, one for each address family. The
chosen interface config follows a precedence order which depends on
the network type: OAM > Management > Admin.
Test plan
System: AIO-SX subcloud IPv4
Initial setup:
- oam0: ethernet, assigned to oam network, with gateway address
- mgmt0: ethernet, assigned to mgmt network, no gateway address
- admin0: ethernet, assigned to admin network, no gateway address
[PASS] Add gateway to mgmt's address pool, check that oam's gateway
remains the default
[PASS] Add gateway to admin's address pool, check that oam's gateway
remains the default
[PASS] Remove gateway from oam's address pool, check that mgmt's
gateway becomes the default
[PASS] Remove gateway from mgmt's address pool, check that admin's
gateway becomes the default
Closes-Bug: 2065715
Change-Id: I28eb5de0c34db2cef089c83cf568ba61fa2d4e42
Signed-off-by: Lucas Ratusznei Fonseca <lucas.ratuszneifonseca@windriver.com>
This commit fix the issue to add a new host by adding a try except
avoiding error.
Test Plan:
PASS: Add a new host with "system host-update 2 personality=controller"
cmd.
Closes-Bug: 2065636
Change-Id: Ie658058ab6cdbab042d8428f77a81499eb8fbc82
Signed-off-by: Luis Eduardo Bonatti <LuizEduardo.Bonatti@windriver.com>
During dual-stack (IPv4 and IPv6) network testing, it was observed
that traffic on the primary address family (e.g., IPv4) was
interrupted when the secondary address pool (e.g., IPv6) was
configured in Linux.
This issue stemmed from how StarlingX manipulated ifcfg files. When an
interface configuration file contained only one address family, the
final file used lacked a label. However, for dual-stack
configurations, a separate labeled file was generated for the same
interface.
This behavior caused problems when apply_network_config.h script
executed. It compares the contents of /etc/network/interface.d/ with
the configuration provided by Puppet. Since the files differed due to
the missing label in the single-address case, the script triggered an
unnecessary ifdown operation on the entire interface, not just the
labels, leading to traffic interruption.
PXE boot interfaces are an exception to the labeling requirement as
MTCE relies on the filename during boot to extract information.
Therefore, when a PXE boot interface is the only network configured
on an interface, no label is generated. This is acceptable because
PXE boot typically uses IPv4 (single-stack) and doesn't encounter
the dual-stack labeling issue.
Test Plan
[PASS] Install AIO-SX in single-stack and then add dual-stack config
for OAM network in runtime and observe that there is no traffic
interruption as the secondary address is added
[PASS] Install AIO-DX in single-stack with the following variants:
- ethernet port with {mgmt, cluster-host, pxeboot} networks
- ethernet port with pxeboot and vlan with {mgmt, cluster-host}
networks
- bonding port with {mgmt, cluster-host, pxeboot} networks
- bonding port with pxeboot and vlan with {mgmt, cluster-host}
networks
Story: 2011027
Task: 50054
Change-Id: I8df423a428c7a853b65f7b448f4c0740f7e72321
Signed-off-by: Andre Kantek <andrefernandozanella.kantek@windriver.com>
The new system service-parameter will allow a user to specify the
number of sysinv api workers for horizontal scaling
The maximum value is capped at the number of physical cores.
The values
- service -> platform
- section -> config
- name -> sysinv_api_workers
- value -> [1 .. n ]
- personality -> None
- resource -> ::platform::sysinv::params::sysinv_api_workers
CGTS-CLIENT
- add --format arg to cli
- update service-parameter Unit Tests
SYSINV REST API
- add validation for sysinv_api_workers parameter
( min 1, max core count)
SYSINV Conductor
- update config to include sysinv_api_workers in puppet config hierdata
Test plan:
PASS - Unit testing
PASS - AIO-SX:
verify default sysinv_api_workers_value
verify system service-parameter-add
verify system service-parameter-modify
verify system service-parameter-delete
PASS - AIO-DX:
verify default sysinv_api_workers_value
verify system service-parameter-add
verify system service-parameter-modify
verify system service-parameter-delete
PASS - AIO-SX system controller:
verify default sysinv_api_workers value
PASS - AIO-SX/DX:
verify max sysinv_api_workers <= physical cores
compare to system host-cpu-list <controller>
----------------
Example error:
----------------
Parameter 'sysinv_api_workers' must be between 1 and 6.
Story: 2011106
Task: 50065
Change-Id: I7fa0407a747def9bf66060a1b6622029a00ba5a8
Signed-off-by: Kyale, Eliud <Eliud.Kyale@windriver.com>
Previously the function was basing a decision whether
or not Armada is required soley on whether or not
the folder /opt/platform/armada exists or not. There
might be the condition where an Armada application was
installed on a lower version of STX, then upgraded to
FluxCD, and the old armada manifests left over. This
change seeks to make the determination about Armada
being required solely upon the helm2 cli showing a release
and if a pod existing in the armada namespace existing with
the "application=armada" label.
Test Plan:
PASS: Upgrade activation step between stx9.0 and future
stx10.0 and 76 script passes.
PASS: Force install an old armada application to check
if the 76 script leaves armada in place. In this
case the Armada pod is detected and the script
exit and activation fails as it should.
Closes-Bug:2065320
Change-Id: I50dabe843549f7f84522c2a61056560c5c084da5
Signed-off-by: Joshua Reed <joshua.reed@windriver.com>
The 'sysinv-helm create-fluxcd-app-overrides <app_name>
<namespace>' is a command that allows a user to generate helm
override values for a helm chart independently of an application
upload/apply. This command is useful for testing.
It was broken due to a cache cleaning done in apps_metadata_dict
that mistakenly added a different key to the one being used.
The command correction is done by changing the dictionary's key
set to the correct key.
Test Plan:
PASS: Run "sysinv-helm create-fluxcd-app-overrides
/home/sysadmin oidc-auth-apps kube-system"
commamd and generate files with success.
PASS: Run "sysinv-helm create-fluxcd-app-overrides
/home/sysadmin/ cert-manager cert-manager" command and
generate files with success.
Closes-Bug: 2060864
Change-Id: If5aa2bc96577811182ed0fd326c55b229410c4ff
Signed-off-by: David Bastos <david.barbosabastos@windriver.com>
This commit adds the code to raise the error for the following command,
if the hardware does not have 4940 or 4942 QAT devices.
Command: "helm-chart-attribute-modify --enabled true
intel-device-plugins-operator intel-device-plugins-qat
intel-device-plugins-operator"
TEST CASES:
PASSED: Build is success.
PASSED: Bootstrap is success.
PASSED: Upload the package using command system application-upload.
PASSED: Check chart enabled status using command
"system helm-override-list intel-device-plugins-operator --long"
PASSED: Enable the QAT chart using command "system
helm-chart-attribute-modify", raises error in non QAT system.
PASSED: Enable the QAT chart using command "system
helm-chart-attribute-modify", it should success on QAT system.
PASSED: Disable the QAT chart using command "system
helm-chart-attribute-modify", it should success in both QAT and
non QAT system.
PASSED: Apply app intel-device-plugins-operator on both QAT and non QAT
system is successful.
Story: 2010604
Task: 50027
Change-Id: Ied634cf35b53421bcaa2f8307e76a0fc87d3bb1f
Signed-off-by: Md Irshad Sheikh <mdirshad.sheikh@windriver.com>
This change adds the variable public_secondary_ip_address to
platform::haproxy::params filled with the secondary OAM address pool
floating address value, in a similar way that is done for the primary
address pool. This will be used in HAproxy to bind the necessary L4
public ports to the secondary address.
Test plan
[PASS] Install and add a secondary pool via CLI and, then, after
lock/unlock, check that all public endpoints (openstack
endpoint list) are available in the primary and secondary
addresses, on the following setups:
- AIO-SX (prim:IPv4, sec:IPv6)
- AIO-SX (prim:IPv6, sec:IPv4)
- AIO-DX (prim:IPv4, sec:IPv6) with system-controller role
- AIO-DX (prim:IPv6, sec:IPv4) with system-controller role
[PASS] Access the public APIs on both protocols using curl.
Story: 2011027
task: 49996
Change-Id: I1b79f4e462ab34ab2aa7187d92460202fa15ae7e
Signed-off-by: Andre Kantek <andrefernandozanella.kantek@windriver.com>
This commit is to replace the direct request to db API to get
the upgrade state with a new proxy API.
The proxy API will firstly direct the request to USM REST API
if the USM endpoint is available. If not, the request will be directed
to legacy db API.
Test Plan:
PASS: run the upgrade with USM available
PASS: run the upgrade with legacy upgrade method
Depends-on: https://review.opendev.org/c/starlingx/update/+/916597
Task: 49798
Story: 2010676
Change-Id: If64c5fd6585ce7a96bee84393205194bd2fd92a4
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
Skip recovery for applications that have update_failure_no_rollback
set to 'true' and that eventually fail to pass lifecycle semantic
checks during updates.
Triggering the recovery of an app that does not support rollbacks can
result in a broken state. This aims to standardize the behavior of the
application update process by equalizing how we handle lifecycle
semantic checks failures to other update errors such as apply failures.
Test Plan:
PASS: build-pkgs && build-image
PASS: AIO-SX fresh install
PASS: Create a modified version of cert-manager setting the
'update_failure_no_rollback' option to 'true'.
Update cert-manager to the modified version.
Confirm that the update succeeded.
PASS: Create a modified version of cert-manager setting the
'update_failure_no_rollback' option to 'true'.
Force an exception when running lifecycle semantic checks.
Update cert-manager to the modified version.
Confirm that the update failed with a descriptive error message
informing that the skip recovery feature is enabled.
Fix the code and reapply the app.
Confirm that the app was successfully applied.
Closes-Bug: 2064737
Change-Id: Ie90c5c3c3a79d8502eb9cc1aa11222963ba13621
Signed-off-by: Igor Soares <Igor.PiresSoares@windriver.com>
This commits changes IPSec client to use hostname instead of IP Address
on swanctl configuration parameter local addr in worker nodes.
Test Plan:
PASS: In a DX system with IPsec enabled and security association
established in both controllers, add a worker node and observe
that IPSec will be enabled and security association will
established in the three nodes without manually intervention.
Story: 2010940
Task: 50039
Change-Id: Idba336e3870f33db840846578441984e11b0d574
Signed-off-by: Leonardo Mendes <Leonardo.MendesSantana@windriver.com>
When user executes "system health-query" commands, it logs the
full certificate snapshot to sysinv.log, this is happening
because the CertAlarmAudit is imported to health.py module
to check for any expiry/expired alarms before upgrade activity.
This fix addresses this issue by changing the "info" log to
"debug".
Test Cases:
PASS: Run "system health-query", "system health-query-kube-upg
rade" , "system health-query-upgrade" and verify that
cert snapshot is logging only in the debug mode.
Closes-bug: 2064925
Change-Id: Ia0482a557931afdef89a6fa88017ea488a6dca59
Signed-off-by: amantri <ayyappa.mantri@windriver.com>
Modify USM software to use fqdn host name for management network
Test Plan:
PASS: Install DC subcloud, ensure it is in managed state,
and execute software commands (Eg. software list)
Closes-Bug: 2063460
Change-Id: I1782d02d58dfe3c8a08048f6d807e3e62532b292
Signed-off-by: Joseph Vazhappilly <joseph.vazhappillypaily@windriver.com>
This commit adds and installs ipsec-config script executed during the
execution of the sm-service. The ipsec-config service has the goal to
create a symbolic link between swanctl.conf file and different .conf
files depending on which personality the controller node is assuming,
swanctl_active.conf or swanctl_standby.conf.
This script implements 5 actions: start, stop, status, meta-data and
monitor.
1) The start action creates a symbolic link between swanctl.conf and
swanctl_active.conf file, as the active controller has ipsec-config
service on enabled-active status.
2) The stop action creates a symbolic link between swanctl.conf and
swanctl_standby.conf file, as the stand-by controller has ipsec-
config service on disabled status.
3) The status action reports the current service status based on the
symbolic link associated with swanctl.conf file.
4) The meta-data action reports ipsec-config's meta-data info.
5) The monitor action indicates ipsec-config service is working as
expected. This action is performed on a specific interval to check
in-service status.
Test Plan:
PASS: Build a debian iso containing the changes.
PASS: Bootstrap, install and unlock a DX system w/ IPsec enabled. Wait
until system reboots and verify unlocked enable available status.
On controller-0, manually execute ipsec-config's start action and
observe that a symbolic link is created between swanctl.conf and
swanctl_active.conf.
/etc/swanctl/swanctl.conf -> /etc/swanctl/swanctl_active.conf
PASS: Bootstrap, install and unlock a DX system w/ IPsec enabled. Wait
until system reboots and verify unlocked enable available status.
On controller-1, manually execute ipsec-config's stop action and
observe that a symbolic link is created between swanctl.conf and
swanctl_standby.conf.
/etc/swanctl/swanctl.conf -> /etc/swanctl/swanctl_standby.conf
PASS: Manually execute ipsec-config's status action and observe status
report output. Observe that the output matches with the symbolic
link associated with /etc/swanctl/swanctl.conf.
PASS: Manually execute ipsec-config's monitor action. Observe that the
output matches with the symbolic link associated with
/etc/swanctl/swanctl.conf. It is expected that controller's
floating IP is addressed on system-local-nodes configuration for
an active controller. In return, controller's floating IP is not
expected on swanctl configuration for a stand-by controller.
Story: 2010940
Task: 49990
Change-Id: I45f06ad41f3240d4149a688cef130cd7c9ae7019
Signed-off-by: Manoel Benedito Neto <Manoel.BeneditoNeto@windriver.com>
This commit updated ipsec-client to generate two copies of swanctl
configurtion files for controller nodes, one for when the node is active
controller (swanctl_active.conf), and one for when the node is standby
controller (swanctl_standby.conf). A symlink (swanctl.conf) is created
pointing to one of the two config files based on the role of the node.
When controller swact, the symlink will be updated by a SM service.
Test Plan (IPv4 and IPv6 DX system):
PASS: controller-0 bootstrap, verify swanctl configuration files and
symlink are created in /etc/swanctl directory:
/etc/swanctl/swanctl_standby.conf
/etc/swanctl/swanctl_active.conf
/etc/swanctl/swanctl.conf -> /etc/swanctl/swanctl_active.conf
PASS: controller-1 installation, after installed, verify swanctl
configuration files and symlink are created in /etc/swanctl
directory:
/etc/swanctl/swanctl_standby.conf
/etc/swanctl/swanctl_active.conf
/etc/swanctl/swanctl.conf -> /etc/swanctl/swanctl_standby.conf
PASS: controller-1 unlock, after controller-1 is unlocked, verfiy that
during drbd synchronization there is no uncontrolled swact, and
controller-1 comes up in "enabled" and "available" state after
drbd is fully synced.
Story: 2010940
Task: 49927
Change-Id: Ic4b3d8a8368e87b2c9f875d5f9cdf555be25a682
Signed-off-by: Andy Ning <andy.ning@windriver.com>
Zuul