Include some improvements in the playbook:
- CAs cert/keys are verified first, then installed at the end of
the playbook (this makes the playbook work if the system-local-ca
secret was deleted - not having the secret prevented installing
the RCA as trusted in the early steps of playbook).
- Not deleting oidc-auth-apps-certificate Certificate unless the
application is applied in the system (Certificate is recreated in
this case).
- Only wait for 'system-openldap-local-certificate' when the
playbook created it (only standalone or SystemController).
- Included step to reapply old 'system-local-ca' secret if the
playbook fails in a state where the secret was already deleted.
Test Plan:
PASS: Run update_platform_certificates playbook in DC + SX subcloud.
PASS: Remove system-local-ca secret.
Run upgrade_platform_certificates playbook.
PASS: Provide wrong field in inventory file.
Run upgrade_platform_certificates playbook, observe that it
fails.
Fix the inventory file.
Run upgrade_platform_certificates plabook.
PASS: Issue oidc-auth-apps-certificate Certificate, using
system-local-ca ClusterIssuer.
Without oidc auth apps applied in the system, run
upgrade_platform_certificates plabook.
Observe that the certificate is not deleted.
Story: 2009811
Task: 50080
Change-Id: Ic0213ea739dbb116536f9e4a85d16da0b55cf6ca
Signed-off-by: Marcelo Loebens <Marcelo.DeCastroLoebens@windriver.com>
Integrating Hashcorp vault backup procedure into platform backup, so it
can be backed up alongside platform optionally. Also contains amendments
to vault backup/restore playbook to accomodate for platform integration.
The vault backup playbook now will create a tarball containing both the
snapshot tarball and the metadata. The vault subdir will be treated as
tempdir and deleted at the end.
The vault restore playbook now requires the tarball created above in the
backup procedure, instead of vault subdir and its parent dir. It will
follow the same convention as the platform restore playbook.
The restore playbook also has extra validation procedures, to
automatically attempt to fix the sealed vault pods.
Test Plan:
PASS Validate platform backup with backup_hc_vault enabled
PASS Validate new hashcorp vault backup playbook
PASS Validate new hashcorp vault restore playbook
PASS Validate vault sanity after restore
PASS Vault is restored to correct status produced by backup
PASS Unit test
Story: 2011073
Task: 49841
Change-Id: I1cba38893d9191bdd3902ef02abdf89d0ec943ed
Signed-off-by: Tae Park <tae.park@windriver.com>
This commit will be updating default password occurrences on
ansible-playbooks files to comply with new password rules, that will be:
- Minimum 12 characters
- At least 1 Uppercase letter
- At least 1 number
- At least 1 special character
- Cannot reuse past 5 passwords
- Default password expiry period should be set to 90 days.
The default passwords are updated as follows:
St8rlingX* -> St8rlingXCloud*
Boot5trap*1234 -> Boot5trapCloud*
Test Plan:
PASS: Run build-pkgs -c -p playbookconfig
Task: 50001
Story: 2011084
Change-Id: Ib6c1fd96f335bfb53e71da48966baa4246649a1f
Signed-off-by: Karla Felix <karla.karolinenogueirafelix@windriver.com>
This commit creates the Barbican user, service, and
endpoints before the 'update sysinv db' task. This
enables the creation of the registry secrets to be
used to create service parameters for the registries.
Test Plan:
PASS: Perform a complete deploy in a DC environment.
PASS: Check that all service parameters for Docker are created.
PASS: Successfully apply the platform-integ-apps.
Closes-Bug: 2065317
Change-Id: I259e176a4a6309ca8748aef37e137e0c6e0894b9
Signed-off-by: Hugo Brito <hugo.brito@windriver.com>
This commit will be updating default password occurrences on
ansible-playbooks files to comply with new password rules, that will be:
- Minimum 12 characters
- At least 1 Uppercase letter
- At least 1 number
- At least 1 special character
- Cannot reuse past 5 passwords
- Default password expiry period should be set to 90 days.
The default passwords are updated as follows:
St8rlingX* -> St8rlingX*1234
Boot5trap* -> Boot5trap*1234
Test Plan:
PASS: Run a full deploy successfully.
Story: 2011084
Task: 49824
Change-Id: If1b7acdde2adc749a3113c0d4a923fd7e92912c0
Signed-off-by: Karla Felix <karla.karolinenogueirafelix@windriver.com>
Add FluxCD images from helm-controller v0.27.0 and
source-controller v0.32.1 to support upgrade from
stx9.0 to stx10.0
Test Case:
PASS: Perform an upgrade from stx9.0 to stx10.0 and
after running upgrade playbook verify that FluxCD
pods are successfully running.
Closes-Bug: 2064525
Change-Id: I0a5c957fd7d2ed1ea7c49f2a7ad983c841ae880e
Signed-off-by: Reed, Joshua <Joshua.Reed@windriver.com>
Changed initial configurations to bootstrap the system w/
HTTPS endpoints. This will change current behavior, that is to
perform the change during the first unlock of c0.
Test plan:
PASS: Deploy AIO-SX - Verify HTTPS endpoints
PASS: Deploy DC + SX subcloud - Verify HTTPS endpoints
Story: 2009811
Task: 50010
Change-Id: Ie0a187838b1da080d81fa3e28607a56a1f9fbf50
Signed-off-by: Marcelo Loebens <Marcelo.DeCastroLoebens@windriver.com>
Included code to avoid repeating the system_local_ca_cert in case
the ca.crt cannot be retrieved.
Filling this field with a cert that it's not a RCA can cause problems when renewing certificates signed by 'system-local-ca' issuer, while
having the field as an empty string doesn't pose a problem for
renewal.
Test plan:
PASS: Bootstrap AIO-SX (fresh install).
PASS: Bootstrap DC + SX subcloud (fresh install).
PASS: Perform upgrade from stx 9.0 (AIO-SX).
Story: 2009811
Task: 50018
Change-Id: I1757b5c0438aba9ca8a782b3f05c160cdabec134
Signed-off-by: Marcelo Loebens <Marcelo.DeCastroLoebens@windriver.com>
Switch to using "stage1" and "stage2" symlinks under
/var/lib/kubernetes to select versions for kubeadm, kubelet,
and kubectl.
We have been using bind mounts to select K8s versions, but they are not
well supported by Puppet and suffer from fragility since you cannot
remove a bind mount while an executable is still running from it. They
also need to be re-created when creating an OSTree hotfix.
Symlinks suffer from no such issues, they just need to be created in
a filesystem that is not managed by OSTree.
NOTE: This needs to go in at the same time as its two dependencies or
else things will break.
Depends-On: https://review.opendev.org/c/starlingx/integ/+/916337
Depends-On: https://review.opendev.org/c/starlingx/stx-puppet/+/916338
Story: 2011047
Task: 49914
TEST PLAN:
See integ repo commit for test plan.
Change-Id: Ia092228fc4afef081b9a95cb09f13b7f5fe729b0
Signed-off-by: Chris Friesen <chris.friesen@windriver.com>
Use the 'gpg' linux command to encrypt/decrypt a file. The encryption
method is "--symmetric", with a user supplied passphrase.
See also man 'gpg' for description of the command options.
Ansible variable options are described in roles/encrypt/vars/main.yml
and roles/decrypt/vars/main.yml
Story: 2011073
Task: 49929
Test Plan:
pass ansible-lint
pass Unit test
Change-Id: Ibc4fc574733b321e3f8e309417cfd5ec7fc91071
Signed-off-by: Michel Thebeau <michel.thebeau@windriver.com>
Use the 'shred' linux command to securely remove files. See also
'man shred' for description of the command options.
Ansible variable options are described in roles/shred/vars/main.yml
Story: 2011073
Task: 49925
Test Plan:
PASS ansible-lint
PASS Unit test
Change-Id: I54f6f1c93a7fe9f9b9fbfb70d455e789680d7b6c
Signed-off-by: Michel Thebeau <michel.thebeau@windriver.com>
This commit fixes the solution introduced in
https://review.opendev.org/c/starlingx/ansible-playbooks/+/912317.
Test Plan:
PASS: Deploy a DC environment with one SX and one DX subcloud
and backup both subclouds. Restore the subclouds backup and
verify that both operations completes successfully.
Story: 2011035
Task: 49694
Signed-off-by: Gustavo Pereira <gustavo.lyrapereira@windriver.com>
Change-Id: I9f84328d15fba6acf867e6a322e97e4dd3b2a6df
Add cert-manager images from v1.7.1 to v1.11.5 to support upgrade
from stx9.0 to stx10.0
Test Cases:
PASS: Perform an upgrade from stx9.0 to stx10.0 and after
running upgrade playbook verify that cert-manager app
is successfully running, perform upgrade activate
and notice that app is upgraded.
Closes-Bug: 2063372
Change-Id: I30fc44bb3e76375c0590233708a8cc23b6e1141c
Signed-off-by: amantri <ayyappa.mantri@windriver.com>
Previously, L4 ports had default values defined in Puppet classes for
bootstrap and backup/restore scenarios.
These defaults were removed to ensure all ports are managed by the
firewall. The change is:
https://review.opendev.org/c/starlingx/stx-puppet/+/885586
While this functions well for fresh installations, it caused an issue
during DX subcloud backup and restore. Specifically, the Ansible
playbook wasn't configuring L4 ports during subcloud restore.
Test Plan:
IPv4 DC with subcloud AIO-DX fresh install
IPv4 AIO-DX fresh install
IPv4 AIO-SX fresh install
IPv4 Subcloud AIO-DX Backup and Restore
IPv4 AIO-DX Backup and Restore
IPv4 AIO-SX Backup and Restore
Closes-Bug: 2056054
Signed-off-by: Fabiano Correa Mercer <fabiano.correamercer@windriver.com>
Change-Id: I91b0d0e714aff1a2a0dbfbb1031975d010872c81
Creating new ansible playbooks vault_backup and vault_restore that
creates a vault snapshot for backup and uses it to restore vault
respectively. Each playbook invokes the vault backup/restore script to
access vault REST API.
The vault_backup playbook has one required option and one optional option:
required:
--initial_backup_dir: the path to the directory, where the vault
subdirectory will be created. The vault_backup playbook will place the
resulting backup tarball in the subdur.
optional:
--encrypt_hc_vault_secret: a string that will be used as a secret key
for encrypting the backup tarball
The vault_restore playbook, in addition to the options for vault_backup,
has one additional required option:
--backup_filename: the filename of the backup tarball that will be used
to restore the vault application. This file must be in the vault
subdirectory of the initial_backup_dir directory
Test Plan:
PASS vault backup then vault restore
PASS vault backup/restore with custom encryption secret key
PASS backup, rekey vault, lose the new key shards, restore from
backup
PASS backup, delete the vault namespace and recreate the cluster,
restore
Story: 2011073
Task: 49841
Change-Id: I3824450ae8bb0c602c44cddd19dd10f5b307e8d6
Signed-off-by: Tae Park <tae.park@windriver.com>
The CNI system images for the last version of the old release
and the first version of the new release should be the same.
Testing:
- Build successful
- All kube-system pods came up
- Manual K8s upgrade
Story: 2010639
Task: 49900
Change-Id: Id28ba013c3470c3656ca36745e09a53924ad6dcf
Signed-off-by: Mohammad Issa <mohammad.issa@windriver.com>
Included a default entries for the fields:
- 'commonName' - default now is <cert_short_name>
- 'localities' - default now is <region>
- 'organization' - default now is 'starlingx'
Where:
<region> is the region name
<cert_short_name> is an internal proper name used for each of the
platform certs.
These fields can still be overridden by the user during bootstrap / CA
update. The override 'subject_prefix' is now removed.
Modified update_platform_certificates.yml playbook to delete/recreate
the leaf certificates instead of re-configuring it. In some cases,
just re-configuring would not change nested values in the Certificate
spec entries. Also, waited for the local OpenLDAP cert to be ready
before progressing, avoiding issues with remaining tasks caused by
delays in cert-manager.
Test plan:
PASS: Bootstrap system without overriding 'subject_L', 'subject_O'
or 'subject_CN'.
Verify that the default fields are included.
PASS: W/ default values, test Horizon access.
PASS: W/ default values, test access through remote CLI.
PASS: W/ default values, test pulling images from the local
registry externally (outside the system).
PASS: Update platform certificates overriding all 'subject_*' fields.
Verify that the overridden values are included in the
respective fields.
Story: 2009811
Task: 49831
Change-Id: I208c30a6eb2c60397d50e6ea411ee5994fa27f9a
Signed-off-by: Marcelo Loebens <Marcelo.DeCastroLoebens@windriver.com>
This is to update the health check in backup to match the new output
that includes alarm info related to certs.
Presently, it is possible to create backups when expired certs are
present. After this change that will no longer be possible.
TEST PLAN
PASS: AIO-SX backup fails when expired certs present
PASS: AIO-SX backup fails when mgmt affecting alarms present
PASS: AIO-SX backup works when no alarms present
PASS: AIO-SX backup works when only minor alarms present
Closes-Bug: 2062087
Change-Id: I5a66fc4b59c619623b9da8c688d576e67f262d33
Signed-off-by: Joshua Kraitberg <joshua.kraitberg@windriver.com>
Added new ca certificate install commands in the playbook
Testcases:
PASS: Bootstrap the system with changes and verify that system is
installed successfully
PASS: Run update_platform_certificates and verify it
is successful
PASS: Bootstrap systemcontroller and verify that system is installed
successfully, bootstrap subcloud from systemcontroller and
verify subcloud is installed fine.
Story: 2010848
Task: 48473
Change-Id: I4151e1be84e2cc9d65f5740a9280408a202c1765
Signed-off-by: amantri <ayyappa.mantri@windriver.com>
Depends-on: https://review.opendev.org/c/starlingx/config/+/893799
This commits enables user defined applications to be uploaded
and applied in parallel using ansible async module.
Test plan:
PASS: Deploy one controller with two applications defined in
localhost.yml file. Bootstrap the controller and verify that
the upload and apply tasks were executed in parallel
PASS: Deploy a system controller with all applications that
can be applied by the user. Verify that all applications were
deployed in parallel.
PASS: Deploy a system controller without user applications
added to localhost.yml. Verify that the bootstrap finishes
successfully.
PASS: Deploy a subcloud with two applications defined in
overrides file. Replay the subcloud bootstrap adding an
application in overrides. Verify that all applications
are applied.
PASS: Deploy a subcloud with two applications defined in
subcloud overrides file. Force an application failure case
and verify that the bootstrap fails successfuly.
e.g.
Add portieris to overrides without a caCert.yaml file.
Story: 2011035
Task: 49584
Signed-off-by: Gustavo Pereira <gustavo.lyrapereira@windriver.com>
Change-Id: Ic9140aaf3c9b1a60c11c441f745d8b9206413d41
Add sysinv_bootstrap task file to apply_bootstrap_manifest role
to reduce bootstrap time. The corresponding sysinv bootstrap
implementation in puppet will be removed.
Changes include:
- Create a template for sysinv.conf and sysinv/api-paste.ini files
- Ensure the installation of sysinv packages
- Ensure the execution of sysinv-api, sysinv-conductor
and sysinv-agent services
Test plan:
1. PASS: Deploy a DC system with one system controller and two subclouds
and ensure the subclouds can be managed
2. PASS: Deploy an AIO-SX system and verify the host unlocks
3. PASS: Perform bootstrap replay and ensure the host unlocks after
re-execution
4. PASS: Verify the openstack user, role, service and endpoints
configuration for sysinv after bootstrap for each deployment type
5. PASS: Verify the sysinv.conf and api-paste.ini file for each
deployment type
6. PASS: Validate the sql dump of the keystone database generated in
a subcloud deployment in relation to the one generated before the
changes
Depends-On: https://review.opendev.org/c/starlingx/config/+/915365
Story: 2011035
Task: 49764
Change-Id: I7cc9b7d45b770b454178da3f6c974bdbf7fc1e57
Signed-off-by: Raphael Lima <Raphael.Lima@windriver.com>
This change removes the conditional statement around the section that
enables the firewall for IPv6 in Calico. The IPv6 firewall will be
permanently enabled regardless of the setup, so that even if IPv6 is
unused, traffic will be blocked.
Test plan
=========
The tests for https://review.opendev.org/c/starlingx/config/+/915508
also cover this change.
Story: 2011027
Task: 49816
Depends-On: https://review.opendev.org/c/starlingx/stx-puppet/+/915509
Change-Id: I986f361493b29596851e781632c782c92ec22546
Signed-off-by: Lucas Ratusznei Fonseca <lucas.ratuszneifonseca@windriver.com>
This commit update IPSec certificates, including trusted CA
certificates when system-local-ca is updated in the system.
Test plan:
PASS: In a DX system with IPsec Initial Auth configured on each
host and SAs established. Run "ansible-playbook /usr/share/
ansible/stx-ansible/playbooks/update_platform_certificates.yml
-i inventory.yaml --extra-vars "target_list=localhost mode=update
ignore_alarms=yes" following documentation to create inventory
file. After execution, run "swanctl --list-certs" and observe
strongswan have all CA certificates, including Root CA if it's
an intermediate CA, and SAs are still established and it's
possible to ping all nodes.
PASS: In a DC system with a central DX and a subcloud DX with IPsec
Initial Auth configured on each host and SAs established.
Run "ansible-playbook /usr/share/ansible/stx-ansible/playbooks/
update_platform_certificates.yml -i inventory.yaml --extra-vars
"target_list=localhost,all_online_subclouds mode=update
ignore_alarms=yes" following documentation to create inventory
file. After execution, run "swanctl --list-certs" and observe
strongswan have all CA certificates, including Root CA if it's
an intermediate CA, and SAs are still established and it's
possible to ping all nodes.
Story: 2010940
Task: 49823
Depends-On: https://review.opendev.org/c/starlingx/config/+/914969
Change-Id: Ie18990fde89b92c98a013782454919eddf3f8fdf
Signed-off-by: Leonardo Mendes <Leonardo.MendesSantana@windriver.com>
The change made in review [1] introduced the creation of the
snapshot resources in all installations during the bootstrap.
An issue was identified during the restore process (uses
bootstrap playbook) in a specific scenario: systems with
more than one host where the volume snapshot controller pod
was not running on controller-0 when the backup was performed.
During the restore process, within the snapshot-controller role,
it checks if the snapshot-controller pod is running, resulting
in a failure in the above scenario, since the assigned node
will not be ready, as only controller-0 is ready to run pods during
restore.
Therefore, the fix involves skipping the execution of the role
during restore. It's important to note that if the
snapshot-controller pod was created before the backup, it will
later be restored and will operate correctly after completing
the restore process regardless of the node that was attached.
[1]: https://review.opendev.org/c/starlingx/ansible-playbooks/+/904360
Test Plan:
PASS: Successful backup and restore on an AIO-DX whose
snapshot-controller pod was running on controller-0
during backup
PASS: Successful backup and restore on an AIO-DX whose
snapshot-controller pod was running on controller-1
during backup
PASS: Successful backup and restore on a Standard (2+1) whose
snapshot-controller pod was running on controller-1
during backup
PASS: AIO-SX | AIO-DX fresh install + Check if the CRDs
and snapshot-controller were created during bootstrap
Closes-bug: 2060675
Change-Id: Ia2f69fafba4854236ea2d6c26932e99e63059ff8
Signed-off-by: Gabriel de Araújo Cabral <gabriel.cabral@windriver.com>
This code will be integrated with ansible playbook(s) for backup and
restore of the Vault application data. The integration will follow with
commits for ansible role for vault.
Depends-on: Id786105aa8ddba2e77085b3897c0c8efd7e98c9b
Test Plan:
PASS unit test
PASS bashate
PASS backup and restore of vault using the scripts
Change-Id: I324b270ec738f864410068c4ac661301ca8176fd
Signed-off-by: Michel Thebeau <Michel.Thebeau@windriver.com>
Experiencing issues related to the
initcontainer: "delete-multus-conf".
Which was initially added as a workaround.
Avoid using upstream "apline" image inside the container.
For now, revert back to multus v3.9.3.
Testing:
- All kube-system pods came up
- Multus conf file was generated properly
- Was able to deploy pods with multiple interfaces
Story: 2010639
Task: 49830
Change-Id: I4d4f420784cf49316ae9146f2b8bcc4f29f748f6
Signed-off-by: Mohammad Issa <mohammad.issa@windriver.com>
Zuul