Commit Graph

120 Commits

Author SHA1 Message Date
Zuul b6075156b3 Merge "USB device discovery" 2024-03-28 21:22:53 +00:00
Damien Rannou 3fd68c0848 USB device discovery
The idea is to retreive USB devices informations via 'lshw' and
return the list to ironic in order to be able to create introspection
rules based on USB devices.

Change-Id: I39d60cb467614fca7a7f701dbe576154213580a5
2024-02-19 14:49:52 +01:00
Zuul 6d35c1e949 Merge "Make inspection URL optional if the collectors are provided" 2024-02-07 23:06:34 +00:00
Zuul 1e107bd625 Merge "Add support for reporting CPU socket number" 2024-01-22 11:52:06 +00:00
Kaifeng Wang 9cafe76225 Add support for reporting CPU socket number
IPA reports a few cpu fields including cores, arch, flags etc.
There is a need that user wants to utilize the physical number in
a baremetal since cores are just a logical representation of the
compute resource.
The socket number is more suitable for the quota control in some
use cases.

Change-Id: I94be86d6b12a3a7e7ca1041d948427a073412a31
2024-01-19 21:24:37 +00:00
Dmitry Tantsur 6cd36a750f
Make inspection URL optional if the collectors are provided
With the new in-band inspection, we can derive the callback URL from
the Ironic URL, there is no need to duplicate it. This change uses
the presence of collectors as a sign to run inspection.

The previous approach of setting an inspection URL, with or without
explicitly setting collectors, still works for compatibility with
ironic-inspector.

Change-Id: Ie4279ee6d2995c9686f1dcdef1d6e5dc1dd20871
2024-01-10 08:55:42 +01:00
Zuul d298e06b49 Merge "[codespell] Fix spelling issues in IPA" 2024-01-08 17:22:02 +00:00
Jay Faulkner 36e5993a04 [codespell] Fix spelling issues in IPA
This fixes several spelling issues identified by codepsell. In some
cases, I may have manually modified a line to make the output more clear
or to correct grammatical issues which were obvious in the codespell
output.

Later changes in this chain will provide the codespell config used to
generate this, as well as adding this commit's SHA, once landed, to a
.git-blame-ignore-revs file to ensure it will not pollute git historys
for modern clients.

Related-Bug: 2047654
Change-Id: I240cf8484865c9b748ceb51f3c7b9fd973cb5ada
2023-12-28 10:54:46 -08:00
Dmitry Tantsur 91b7ae96c9
Reformat and update the section on injecting root credentials
Change-Id: I49ad9979daad11bf7a54069564c6b7919de0ea7c
2023-12-15 12:34:31 +01:00
Michal Nasiadka c23c913fc2 docs: improve rootpwd password generation command
Currently the command is a bit misleading because you need to escape
dollar ($) signs.
Command copied from DIB dynamic-log element docs [1].

[1]: https://docs.openstack.org/diskimage-builder/latest/elements/dynamic-login/README.html

Change-Id: I7d5dc60aec373372f8faae4242a79f18d8a26d14
2023-11-15 07:52:08 +00:00
Julia Kreger eb95273ffb Add get_service_steps logic to the agent
Initial code patches for service steps have merged in
ironic, and it is now time to add support into the
agent which allows service steps to be raised to
the service.

Updates the default hardware manager version to 1.2,
which has *rarely* been incremented due to oversight.

Change-Id: Iabd2c6c551389ec3c24e94b71245b1250345f7a7
2023-08-31 06:22:22 -07:00
Zuul 4845fd04ba Merge "Follow-up Add documentation for MellanoxDeviceHardwareManager" 2023-05-25 15:03:19 +00:00
waleedm 406c844aac Follow-up Add documentation for MellanoxDeviceHardwareManager
Add a follow-up documentation for
"update NVIDIA NIC firmware images and settings by ironic-python-agent"
Icfaffd7c58c3c73c3fa28cfc2a6c954d2c93c16e

Change-Id: I481cdd622f360cbba3312c6f3d4af45383bb7e1b
2023-05-25 10:55:11 +00:00
Jay Faulkner 6098747ec5 Ironic (and IPA) use launchpad now
Correct links to point to launchpad bug tracker, correct docs config

Change-Id: I5d46af2a9d94f3b2e05e4f937e0619a89fe04d4c
2023-05-17 15:38:57 -07:00
Dmitry Tantsur 9ed232e77e Add network interface speed to the inventory
This is another fact that Metal3's baremetal-operator is currently
consuming from extra-hardware.

Change-Id: I2ec9d5e9369f5508e7583a4e13c2083f5c8b28ba
2023-05-03 12:20:35 +02:00
Zuul f37ea85a27 Merge "Disable MD5 image checksums" 2023-05-02 06:41:25 +00:00
Dmitry Tantsur 3e05a03f7c Deprecate LLDP in inventory in favour of a new collector
Binary LLDP data is bloating inventory causing us to disable its collection
by default. For other similar low-level information, such as PCI devices
or DMI data, we already use inspection collectors instead. Now that the
inventory format is shared with out-of-band inspection, having LLDP
there makes even less sense.

This change adds a new collector ``lldp`` to replace the now-deprecated
inventory field.

Change-Id: I56be06a7d1db28407e1128c198c12bea0809d3a3
2023-04-26 19:33:51 +00:00
Julia Kreger 32df26a22a Disable MD5 image checksums
MD5 image checksums have long been supersceeded by the use of a
``os_hash_algo`` and ``os_hash_value`` field as part of the
properties of an image.

In the process of doing this, we determined that checksum via
URL usage was non-trivial and determined that an appropriate
path was to allow the checksum type to be determined as needed.

Change-Id: I26ba8f8c37d663096f558e83028ff463d31bd4e6
2023-04-24 16:54:42 -07:00
Dmitry Tantsur 0304c73c0e Report system firmware information in the inventory
Change-Id: I5b6ceb9cdcf4baa97a6f0482d1030d14f3f2ecff
2023-03-31 14:28:32 +02:00
Dmitry Tantsur 2ddb693491 Trivial: formatting issue in the inventory docs
Double ticks don't work if followed by a symbol without space.

Change-Id: Ia455650b5e601dadb2b0ab91f71e1d9286d26071
2023-03-30 13:33:39 +02:00
liuyuanfeng 1846d6f776 modify error word node
Change-Id: Ie5c9fa7489eb891ef1bbe57c7d51ecb64e1c0db8
2022-12-30 01:18:36 -08:00
Jakub Jelinek a99bf274e4 SoftwareRAID: Enable skipping RAIDS
Extend the ability to skip disks to RAID devices
This allows users to specify the volume name of
a logical device in the skip list which is then not cleaned
or created again during the create/apply configuration phase
The volume name can be specified in target raid config provided
the change https://review.opendev.org/c/openstack/ironic-python-agent/+/853182/
passes

Story: 2010233

Change-Id: Ib9290a97519bc48e585e1bafb0b60cc14e621e0f
2022-09-05 20:43:51 +00:00
niuke 4bf88b204f remove unicode prefix from code
Change-Id: I70f0112f1ee3066ffd9316d10b84b9ea5b7fc306
2022-08-23 19:44:10 +08:00
Jakub Jelinek 1ac61e1dbd Improve function list_block_devices_check_skip_list
Fix minor issues suggested by dtantsur
Add an example of skip list specification to the documentation

A follow-up patch to I3bdad3cca8acb3e0a69ebb218216e8c8419e9d65

Change-Id: Ic94a33b7bc0572a1cc8f92b330474ec63a173e81
2022-08-16 15:17:15 +00:00
Jakub Jelinek 0212337bd5 Enable skipping disks for cleaning
Introduce a field skip_block_devices in properties - this is a list of dictionaries
Create a helper function list_block_devices_check_skip_list
Update tests of erase_devices_express to use node when calling _list_erasable_devices
Add tests covering various options of the skip list definition
Use the helper function in get_os_install_device when node is cached

Story: 2009914

Change-Id: I3bdad3cca8acb3e0a69ebb218216e8c8419e9d65
2022-08-11 09:30:00 +00:00
Julia Kreger beb7484858 Guard shared device/cluster filesystems
Certain filesystems are sometimes used in specialty computing
environments where a shared storage infrastructure or fabric exists.
These filesystems allow for multi-host shared concurrent read/write
access to the underlying block device by *not* locking the entire
device for exclusive use. Generally ranges of the disk are reserved
for each interacting node to write to, and locking schemes are used
to prevent collissions.

These filesystems are common for use cases where high availability
is required or ability for individual computers to collaborate on a
given workload is critical, such as a group of hypervisors supporting
virtual machines because it can allow for nearly seamless transfer
of workload from one machine to another.

Similar technologies are also used for cluster quorum and cluster
durable state sharing, however that is not specifically considered
in scope.

Where things get difficult is becuase the entire device is not
exclusively locked with the storage fabrics, and in some cases locking
is handled by a Distributed Lock Manager on the network, or via special
sector interactions amongst the cluster members which understand
and support the filesystem.

As a reult of this IO/Interaction model, an Ironic-Python-Agent
performing cleaning can effectively destroy the cluster just by
attempting to clean storage which it percieves as attached locally.
This is not IPA's fault, often this case occurs when a Storage
Administrator forgot to update LUN masking or volume settings on
a SAN as it relates to an individual host in the overall
computing environment. The net result of one node cleaning the
shared volume may include restoration from snapshot, backup
storage, or may ultimately cause permenant data loss, depending
on the environment and the usage of that environment.

Included in this patch:
- IBM GPFS - Can be used on a shared block device... apparently according
             to IBM's documentation. The standard use of GPFS is more Ceph
             like in design... however GPFS is also a specially licensed
             commercial offering, so it is a red flag if this is
             encountered, and should be investigated by the environment's
             systems operator.
- Red Hat GFS2 - Is used with shared common block devices in clusters.
- VMware VMFS - Is used with shared SAN block devices, as well as
                local block devices. With shared block devices,
                ranges of the disk are locked instead of the whole
                disk, and the ranges are mapped to virtual machine
                disk interfaces.
                It is unknown, due to lack of information, if this
                will detect and prevent erasure of VMFS logical
                extent volumes.

Co-Authored-by: Jay Faulkner <jay@jvf.cc>
Change-Id: Ic8cade008577516e696893fdbdabf70999c06a5b
Story: 2009978
Task: 44985
2022-07-19 13:24:03 -07:00
waleedm eb07839bd4 Fix passing kwargs in clean steps
Pass kwargs to dispatch_to_managers method in execute_clean_step

Change-Id: Ida4ed4646659b2ee3f8f92b0a4d73c0266dd5a99
Story: 2010123
Task: 45705
2022-07-01 23:03:55 +00:00
Arne Wiebalck cacdd9bab3 Burn-in: Add network step
Add a clean step for network burn-in via fio. Get basic
run parameters from the node's driver_info.

Story: #2007523
Task: #42385

Change-Id: I2861696740b2de9ec38f7e9fc2c5e448c009d0bf
2021-07-13 11:36:31 +02:00
Arne Wiebalck 20c5894bc2 Burn-in: Add disk step
Add a clean step for disk burn-in via fio. Get basic
run parameters from the node's driver_info.

Story: #2007523
Task: #42384

Change-Id: I5f5e336bd629846b3d779fd0fc7a2060b385b035
2021-05-21 16:33:11 +02:00
Arne Wiebalck 5c222560f0 Burn-in: Add memory step
Add a clean step for memory burn-in via stress-ng. Get basic
run parameters from the node's driver_info.

Story: #2007523
Task: #42383

Change-Id: I33a83968c9f87cf795ec7ec922bce98b52c5181c
2021-05-01 10:36:58 +02:00
Arne Wiebalck 6702fcaa43 Burn-in: Add CPU step
Add a clean step for CPU burn-in via stress-ng. Get basic
run parameters from the node's driver_info.

Story: #2007523
Task: #42382

Change-Id: I14fd4164991fb94263757244f716b6bfe8edf875
2021-05-01 10:36:20 +02:00
Jay Faulkner de726d4acf Do not permit IPA standalone to be enabled by conf
IPA standalone mode is a developer-only option, and if enabled
accidentally on a production agent could cause undesired behavior.

Developers who need this behavior should build a purpose-built agent,
with standalone hardcoded to True in cmd/agent.py.

Change-Id: Icc67dbe15acbbf6fee886f274d2169a0769a5053
2021-03-25 12:45:28 +01:00
Mohammed Naser 2220aaae57 Added comment about IPA logs being uploaded to Ironic
Change-Id: I983ad3bd6fff539e877844e54788f63689ce8a84
2021-03-01 11:40:51 -05:00
Dmitry Tantsur 59cb08fd28 New deploy step for injecting arbitrary files
This change adds a deploy step inject_files that adds a flexible
way to inject files into the instance.

Change-Id: I0e70a2cbc13744195c9493a48662e465ec010dbe
Story: #2008611
Task: #41794
2021-02-16 16:56:52 +01:00
Zuul 4762aca077 Merge "Add clean step 'erase_pstore'" 2020-11-18 17:38:00 +00:00
Arne Wiebalck 92e26b01e9 Add clean step 'erase_pstore'
Add an automatic clean step to clean the Linux kernel's pstore.
The step is disabled by default.

Story: #2008317
Task: #41214

Change-Id: Ie1a42dfff4c7e1c7abeaf39feca956bb9e2ea497
2020-11-17 18:00:16 +01:00
Vladyslav Drok c7858d3cc8 Add UUID to BlockDevice object
It'd allow for example custom ansible playbooks to use UUIDs of the
introspected node's disks. In future it might also enable agent
to use UUID (or by_path value) to refer to a device instead of
name, as it happens currently.

Change-Id: Id00437d2295c39fb12f3c25a92b30b56a58eef13
2020-11-11 17:25:59 +00:00
Dmitry Tantsur 565d596dae Document ramdisk TLS and update existing TLS docs
Story: #2007214
Task: #40945
Change-Id: I1a930a0e52ab860edcd597df4d95a4e4eb51da96
2020-09-23 15:07:49 +02:00
Dmitry Tantsur 8f4975763b Documentation: fix incorrect step names
I can never remember if it's "device" or "devices".

Change-Id: Ib1388f0425ec4c02fbeacb26ad55bc90b0f3483c
2020-09-21 18:16:01 +02:00
Arne Wiebalck 9dd968fa30 [doc] Clarify custom hardware manager inheritance
Change-Id: I94657967763f058fccd4f19656925d02cd8973b8
2020-09-15 11:34:11 +02:00
Dmitry Tantsur ffacb713e5 Document in-band deploy steps and add more docs for custom steps
Change-Id: I304a460f88f3f8ee33cf642355f0e659184db724
Story: #2006963
Task: #40727
2020-08-24 17:39:54 +02:00
Dmitry Tantsur 3faf13bbb4 Refactor and fix documentation
Currently the index page contains a brief introduction and 3 links.
Expand the table of contents to show more subheadings.

Clean up the install, admin and contributing docs of unnecessary
and wrong content and redundant headings.

Update URLs throughout the text.

Change-Id: I03279ffc9faf387b08f727dee3b8898a69918460
2020-08-24 15:27:45 +02:00
Dmitry Tantsur 7fb098aa0b Import example hardware managers from ipa-example-hardware-managers
They're not easily discoverable there, let's keep them in tree.

The examples have been restructured to have two different projects
ready to be copied and adjusted. PEP8 failures have been fixed.

Change-Id: I2af04f4b7f9a2109fe83ec517e716159331a48bb
Co-Authored-By: Jay Faulkner <jay@jvf.cc>
2020-07-31 13:35:34 +02:00
Julia Kreger cf870ae510 Fix apidoc build
Change-Id: Id7c5686cdbdc79812e06eb4a5f87eddc75005cc3
2020-06-15 11:30:02 -07:00
Jay Faulkner 3dd929eb92 Document how to output journal to console
Having all logs from the device visible on console may be useful for
deployers who do not enable login access to IPA ramdisks as a matter of
policy.

Change-Id: If9d29dd00a8c35fe679e1f54d1574eed0a79470c
2020-06-09 14:30:18 +00:00
Andreas Jaeger f108b7a2d0 Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems

Update Sphinx version as well.

Set openstackdocs_pdf_link to link to PDF file. Note that
the link to the published document only works on docs.openstack.org
where the PDF file is placed in the top-level html directory. The
site-preview places the PDF in a pdf directory.

Disable openstackdocs_auto_name to use 'project' variable as name.

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.

openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.
This repo uses storyboard, configure the proper variables.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: Id084bfc9d5ae7a95d3cba22a00b4c03a35967ee4
2020-05-23 18:29:40 +02:00
Arne Wiebalck b5e25760af [doc] How to pause the IPA for debugging
This patch adds two handy methods to pause the IPA when it needs
to be debugged.

Change-Id: I6ab4d6707fce340ee447355c717b19a12f3fe1ec
2020-02-28 11:03:25 +01:00
Dmitry Tantsur 31b73b4984 Expose collector and hardware manager names via introspection data
This change adds a new introspection data field 'configuration'
with two lists: managers and collectors.

Change-Id: Ice0d7e6ecff3f319bc3a4f41617059fd6914e31c
2020-01-22 11:15:38 +01:00
Dmitry Tantsur 4b602771e9 Document introspection data and collectors
Change-Id: Ie51ec8fbae3848ad8a913b05cc3f0f80f33b5f95
2020-01-21 15:40:40 +01:00
Madhuri Kumari 22ab827796 Remove deprecated ironic-agent element
ironic-agent is deprecated. Also replace disk-image-create
with ironic-python-agent-builder.

Change-Id: I6a3b5a31eda66b122a310119539fad6f8600ed80
2019-12-27 10:13:51 +05:30