Commit Graph

51 Commits

Author SHA1 Message Date
Riccardo Pittau 40b277b7bd Collect lshw output in json format
Add lshw json output to test logs

Change-Id: Id0689f7111719d166a52f45ac911923646439297
2024-01-22 12:41:56 +01:00
Maksim Malchuk e3fb8c54d0 Create the log file for the disk-image-create command
Change-Id: Iab88b3a28378cc22f767204bef84128bc9627210
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
2023-05-09 09:03:07 +00:00
Steve Baker ea99b85883 Copy shim and grub into tftp and http directories
In preparation for grub network boot, copy the shim and grub efi files
to serving directories.

Change-Id: If181b31c0c69201a1f6d6424f7f4c1322d61eea3
2023-01-19 10:31:00 +13:00
Erik Berg 4a17e35adc Switching netstat to ss in report
Probably should have happened a long time ago [1]. netstat doesn't
come out-of-the-box with some distributions these days.

[1] https://lwn.net/Articles/710533/

Change-Id: Idcd2099cca715837a3ad9bd595fa8f08531ef863
2022-10-24 18:20:03 +02:00
Dmitry Tantsur 2fe879dd0d Fix the CI on CentOS 9
* Prefer modern SSH key algorithms
* Generate an ECDSA key if Cirros is used
* Always resolve symlinks when copying logs

Remove all previous work-arounds.

Change-Id: Ie9248584c71f7d326d5839710d6b9bdf6d779749
2022-03-04 14:46:41 +01:00
Zuul f3061e0a42 Merge "Generate an environment file during bifrost-cli install" 2022-01-25 17:10:15 +00:00
Zuul 5c83d1a7dc Merge "Use the combined Ironic service instead of API+conductor" 2022-01-25 12:20:23 +00:00
Dmitry Tantsur 3cb96f1b67 Tighten permissions for PXE directories
Make the HTTP directory not world readable by default.
Images may contain secrets, so regular users should not read them.

Add nginx and dnsmasq to the ironic group so that they can read ironic
files that are group accessible.

Change-Id: Iaa8585fb48e5db6c0d5063dca0d84c9d2300f0c9
2022-01-14 12:56:36 +01:00
Dmitry Tantsur 9b83665d83 Generate an environment file during bifrost-cli install
It is recommended to pass the same variables (e.g. testing) to other
commands and playbooks, the new file makes it possible.

Change-Id: I2adc4e2dcfd8555b2c9e6c3bf0f512f603367091
2022-01-06 15:36:47 +01:00
Dmitry Tantsur 779e4d8a1c Move /httpboot to /var/lib/ironic
There is no guarantee that /var is not on a separate filesystem. Also
we should not pollute the root directory with our stuff.

The TFTP directory /tftpboot is left where it is, since it needs to be
accessed by dnsmasq. We may consider moving it later on.

Removes the wildly outdated statement about purging the image cache
from the documentation.

Change-Id: Ib1e46e7a9c5eec193082858614026ca4c9f537ac
2022-01-05 16:07:15 +01:00
Dmitry Tantsur ea2d2a37a4 Use the combined Ironic service instead of API+conductor
Change-Id: I9faecfe6ece6d3c35396e3378c1e3930a487e130
Depends-On: https://review.opendev.org/c/openstack/ironic/+/819620
Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/820513
2021-12-06 11:39:54 +01:00
Zuul f7a36512a9 Merge "Revert "Explicitly trap on ERR" and fix exit code hanlding" 2021-11-22 22:52:22 +00:00
Dmitry Tantsur 3dbc7347ec Revert "Explicitly trap on ERR" and fix exit code hanlding
Trapping on ERR does not do what is implied here and is redundant when
trapping on EXIT is used together with `set -e`. The real problem is
that in the CI scripts we pipe test-bifrost.sh into tee with a log file
without setting -o pipefail in bash. This change fixes it.

This reverts commit b31bc66726.

Change-Id: Ifca4049f5967c0b2153ecffb4c153a96b4a62cc7
2021-11-22 10:35:29 +01:00
Dmitry Tantsur da084360e5 Remove the keystone admin endpoint
It's no longer needed and was deprecated in Xena.

Change-Id: Idff1097c13f483f8290bc800388cff3746324dbd
2021-10-04 14:46:27 +02:00
Zuul dc9ac16ea1 Merge "Add uWSGI role and use systemd instead of emperor mode" 2021-09-20 10:52:37 +00:00
Zuul 0b5d61b3f1 Merge "CI: collect keystone information" 2021-09-13 16:05:02 +00:00
Dmitry Tantsur 5c1881282c Add uWSGI role and use systemd instead of emperor mode
Using the emperor mode of uWSGI is redundant now that we depend on
systemd. This change switches uWSGI Keystone to systemd template services
based on the recipe from the uWSGI documentation [1].

A new role is created to encapsulate the uWSGI login in anticipation of
switching Ironic API to uWSGI as well. Reduce repetition in Keystone.

[1] https://uwsgi-docs.readthedocs.io/en/latest/Systemd.html

Change-Id: I50bc3f7a4faf14c36b92e7656b7149e9c833c85e
2021-09-04 15:57:04 +02:00
Dmitry Tantsur d4ddc05399 CI: collect keystone information
Change-Id: Icaff9cb99990e499565a4401f206a15650cefaf1
2021-08-26 16:14:55 +02:00
Dmitry Tantsur 7e1dbbd0de CI: copy bifrost logs to the log directory
Using them is more convenient than scrolling the whole output,
especially in case of upgrades.

Change-Id: Ib086f106a0e78d247bac5fa63801c2fecb40efa8
2021-08-26 16:13:43 +02:00
Riccardo Pittau d751208aca Collect explicit list of installed packages
Change-Id: Ib8e1b002afc053617aa1f39c342bd1809d0154a4
2021-07-08 11:15:09 +02:00
Riccardo Pittau 422bafd6ea Exit on all errors in bash scripts when possible
Except for the collect-test-info script, the other bash scripts should
just exit on error.

Change-Id: Ia68bef56edd7edec14f678155220b6e4e15e57f6
2021-02-09 12:05:11 +01:00
Riccardo Pittau 70101f6a2d Collect firewalld info if present
Change-Id: I90c7ba15a10faaab24712a8cdc8b6c67457b51a6
2021-02-04 08:22:43 +00:00
Dmitry Tantsur 02fb11fcc7 Support redfish emulation and run the keystone jobs with redfish
Prepare for UEFI support that will be finished in follow-up patches.

Change-Id: I18d034239edd3d49ceedf281b48a650e6011bab4
2020-08-07 15:20:54 +02:00
Dmitry Tantsur 33a63bc0da Make ironic logging more in line with other services.
Currently ironic logs into files, other services - into journald. Ramdisk
logs are for some reason stored with data rather than in /var/log.

This change makes all services log into journald and changes to standard paths
for ramdisk logs (making them available in the CI as a side effect).

Change-Id: I8b3c7750596602c2609798ffbb7e274e2c18b40f
2020-07-29 14:32:41 +02:00
Dmitry Tantsur 1606aad1a4 Store inspector ramdisk logs by default
They have not been stored previously because the logs collector was not
enabled. Also use the standard location for logs and save them in the CI.

Change-Id: I034a5fffb982a848ceda3db6635da841f869111e
2020-07-24 16:08:36 +02:00
Dmitry Tantsur 0b7fd20e9e Stop installing openstackclient if keystone is not enabled
We have a standalone command now, let's use it.

Change-Id: I446d7b490d4a14ede01384fd72926f85660ea6e7
2020-05-25 12:38:33 +02:00
Riccardo Pittau 1d2ab2b841 Fix and add logs and info
This patch fixes collection of logs adding sudo to commands that
need that.
Also adds collection of some more useful info for libvirt
and qemu, for example increasing the log verbosity of libvirt.

Reordered commands to divide them by section.

Change-Id: I86ae0dac8de63449405012a99ddf94f915b304b0
2020-03-11 10:37:07 +01:00
Mark Goddard 6513e5a6dc Run vbmcd as a Systemd service
Virtualbmc 2.0 dropped the daemon autostart feature. This means we need
to explicitly start it. Running it as a Systemd unit, code adapted from
Tenks.

In order to allow Systemd to execute the daemon on systems with SELinux
enabled, it must be installed to the system rather than a virtualenv.

Change-Id: I66e8e5f5a9c16ac90e132074c826d508458ae4d1
2020-03-09 13:42:44 +00:00
Georgy Karataev c8c4b8c90f Fix "unbound variable" error in scripts
If user runs scripts without VENV defined, then an error
"unbound variable" will be received. This patch fixes this issue

Change-Id: I22b817ad7b945509bb2e89d0467df53975434313
2020-02-05 21:04:52 +05:00
Dmitry Tantsur 8bf1d8dff1 Collect output of pip freeze in the CI
Grealy simplifies debugging package conflicts.

Change-Id: I6937800c73a914f4a2225cdea2c41d1f9ad8cec8
2020-01-23 13:03:19 +01:00
Dmitry Tantsur 4bb71fc481 Migrate to Zuul v3 native jobs
This corrects the path handling in the CI and removes a lot of
redundancy in the CI definitions. Also adds fixed log collection
from https://review.opendev.org/#/c/649526/ to make the CI pass.

Removes handling of ZUUL_CHANGES. It is not present in new jobs,
and its use case in Bifrost is completely unclear.

Eventually, we should rewrite the whole test-bifrost.sh in ansible,
but that's a much bigger task.

Change-Id: Id893e816a7806e7d75282628817288d8a34ab8af
2019-10-30 14:26:26 +01:00
Riccardo Pittau 906cec71b8 Fixing ipxe file not found in Centos job
The ipxe file can have different extensions based on the distro.
Changing from .pxe to generic extension should cover them all.

Change-Id: I6da5bf86a022315861d140029ce25321a5b57469
2019-04-17 11:33:22 +02:00
Riccardo Pittau 4b63bf7686 Improve iptables test info collection
Adding collection of saved iptables rules and info from all
tables.

Change-Id: I5ef1482d464f2ec837a04740bfb1cacf224768cd
2019-04-09 10:29:24 +02:00
Riccardo Pittau c7547ae68d Improving test vm data collection
Collecting xml file with the vm data.

Change-Id: I2ba245a5ff545a23c6c5e943412e133357207feb
2019-04-05 17:41:36 +02:00
Markos Chandras ec88ca2c86 Add openSUSE Leap 15 jobs
We make the following changes in order to support openSUSE Leap 15.0

- Updates the bindep.txt file to match the SUSE family.
- The netstat binary has been moved to net-tools-deprecated
  package.
- We fix a bug when checking the status to extra dependencies. The
  variable is an array, but the code was only checking the first
  entry of the array.
- Replace spaces with '_' on ansible_distribution since openSUSE Leap
  15.0 is being reported as 'openSUSE Leap' by Ansible.

Change-Id: I800331e29e0d47a3121229cd66a5814277e34b84
2018-11-07 09:25:26 +00:00
Julia Kreger d2fa3a6354 Sudo is needed to copy logs
We're not capturing the logs as we likely can't
copy the contents of the folder. So lets use
sudo to exec the cp command, and for good measure
we'll pull a list in case there just is nothing
present to be uploaded for logs

Change-Id: I73ad58037518e6d334efe0bad99458a2fabfc047
2018-11-06 11:51:08 -08:00
Olivier Bourdon 632a5367b1 Fix CI for CentOS
Change-Id: I7d8c2b29ece07ead5f6e23706a5e51d4c1b65ad8
2018-10-17 14:18:44 +02:00
Markos Chandras 6db1bee4c3 scripts: collect-test-info: Fix cp command when collecting VM logs
Fixes: 431b768929 ("scripts: Fix shell expansion when collecting VM logs")
Change-Id: I4edf4eeb9118df8c2bd219d23bde85a74e1375c5
2018-10-16 09:30:28 +01:00
Markos Chandras d1c9bd8101 scripts: collect-test-info.sh: Collect baremetal information
Collect baremetal node information which can help debug issues.

Change-Id: I923ed22ea16c8692f803ac7656f5d105f9a32032
2018-10-05 14:49:52 +01:00
Markos Chandras 2be2c38399 scripts: collect-test-info.sh: Collect PXE scripts
Collect PXE boot scripts which can help us identify VM boot problems.

Change-Id: I83866b7650a38a885c94c0b7933443f30bef02c4
2018-10-05 14:44:40 +01:00
Markos Chandras 431b768929 scripts: Fix shell expansion when collecting VM logs
The [[:digit:]] does not work correctly when using sudo

 2018-10-04 14:31:33.319558 | opensuse-423 | + sudo cp '/var/log/libvirt/baremetal_logs/testvm[[:digit:]]_console.log' /home/zuul/workspace/logs
 2018-10-04 14:31:33.328756 | opensuse-423 | cp: cannot stat '/var/log/libvirt/baremetal_logs/testvm[[:digit:]]_console.log': No such file or directory

As we can see [[:digit:]] is not expanded so no logs are collected. This
is because the expansion happens by the user but since he does not have
access to the directory the command fails. We can fix this by opening a
subshell so it gets run with sudo privileges.

Change-Id: Ibdf8606a3b72c48faffdf2b58b122b690008ba98
2018-10-05 14:34:15 +01:00
Pavlo Shchelokovskyy 2676f0fda6 Collect ironic-inspector logs
Change-Id: I564f03b219afb2c8f8ccf0858fd1b42e92f8afe3
2018-02-13 12:35:41 +00:00
Julia Kreger 0790a87914 Add collection of IPA failure logs
Sometimes we need to troubleshoot CI a little more,

So lets preserve IPA failure logs so we can investigate further.

Change-Id: I47e7f45e65997a61e41c73e072e3c8c31ef9a244
2017-11-29 10:01:29 -05:00
Markos Chandras fc5e688df6 scripts: Collect logs even if deployment fails.
Always exit with success code when collecting various logs. This allows
this script to be used to collect partial logs in case a deployment
fails since it may help with debugging the root cause of the failure.

Change-Id: I4d5a2c71303e9ed466b5fd5dbc63469677f2380f
2017-02-14 09:37:41 +00:00
Markos Chandras 7fb32dd599 Fix unbound variable error in scripts/collect-test-info.sh
The scripts/collect-test-info.sh script is executed with '-u' which means
that unset variables will trigger an error like the following one:

scripts/collect-test-info.sh: line 16: TEST_VM_NODE_NAMES: unbound variable

We handle this case by simply setting an alternative value when the variable is set
in order to treat the unset variable gracefully.

Change-Id: Ic4b450fe8654568e30eacb00aeff128d675ed616
2016-09-07 16:33:44 +01:00
Fatih Degirmenci a3706f0e49 Fix log collection when VMs are created with non-default names
The change I4c9eec82882d94649ee0e6bdb7e2ce033c29e44a introduced the
ability to create VMs with user defined names. This change aligns
the VM log collection accordingly.

Change-Id: I8c4895a8f49749b6845a804b02ad64bc227ec3ec
Signed-off-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com>
2016-08-31 21:10:24 +02:00
Julia Kreger ddb3965d30 Fix testing script permission settings for logs
Somewhere along the way we lost the ability to read the
conductor, api, and baremetal log files that we retained
for storage as CI test results. This revision resets
the permissions so any user can read the file since
we can't make assumptions about the webserver user.

Change-Id: Idc36f736f9e54a69348045988483559b88e51980
2016-06-27 09:01:19 -04:00
Julia Kreger bdab0447c2 Collect all test VM console logs
Presently the post-test information collection script is only
collecting a single test virtual machine's console log.

As the DHCP test operates with multiple VMs, we need to collect
all console logs to help identify potential test failures.

Change-Id: I8ddfb7870ea57b51f6340db4ba388c13cc35c059
2016-03-21 11:52:04 -04:00
Julia Kreger 917983e5ce Fix and improve test job data collection
Corrected the netstat data collection as the --version flag causes
a non-zero exit code with netstat.

Added interface counter data collection to allow for additional
troubleshooting context.

Change-Id: I7e0e26cc735d6c0ad9052547bc0eefe3ed3877ae
2016-03-16 18:54:56 -04:00
stephane c785d947f6 Improve log collection
Logs from test runs on systemd systems were not being
collected properly. Allow logs to be collected from
systemd if available and fall back to the upstart
location. Also, only collect information from netstat
and iptables if present.

Change-Id: I000c3d0c7d576eeb4ab051f68a6fef8a83dc06fb
Closes-Bug: #1470128
2016-02-23 17:28:57 -08:00