pytz will be removed from RHEL/CentOS 10 because of the built-in
zoneinfo[1].
Because the current usage of pytz can be very easily replaced, this
removes the dependency on pytz.
[1] https://issues.redhat.com/browse/RHEL-219
Change-Id: Iafcaf2f1095cd7c738dac391a9af10622806e932
Primarily remove the workaround added in
Ia6d512ff2ae417bab938cb095fbb0884d195010a which added
continued use of autocommit, which is incompatible with
SQLAlchemy 2.0.
Also set the environment for unit tests to report compatability
warnings, although it appears none are being reported at this time.
Also cuts out the db upgrade cruft to only use the online database
migration code through oslo_db's enginefacade, which has the smarts
to handle online or offline migrations.
And then, retools unit/functional test data storage to utlize sqlite,
and in that re-tooled the queries to prevent locking conditions
which could exist with queries, and some additional refactoring/cleanup.
Also, don't mock and test time.sleep().
Additionally, it looks like we have discovered the root cause of the
memory/connection leakage issue which has been observed, due to the
way lists of nodes are processed/returned.
This change was based upon the work in
I506da42a9891a245831f325e34bec92e0a3f33f0 which is included in
this commit as the entire database structure and interaction
has been modified for ironic-inspector.
Co-Authored-By: aarefiev <aarefiev@mirantis.com>
Story: 2009727
Task: 44132
Change-Id: Ic88eb9dec5fddc924a72d9a23c17a304954ebf46
This change allows uses to enable the healthcheck middleware from
oslo.middleware in API pipeline, by setting the [healthcheck]/enabled
option. This middleware provides an API endpoint at /healthcheck path
which allows load balancers or monitoring applications to determine
a service is up using HTTP requests.
This change basically follows the same change merged in ironic[1] repo.
[1] 6f439414bdcef9fc02f844f475ec798d48d42558
Change-Id: Ic2ee2bca74ea2a5a0723ef54b10c531f77ea7b8d
this is a follow-up to Icd3de82877c6a53d32b4c9fd3e500d3cd9d7fb17
one more place was identified as trying to use uuid attr of a Node
object from openstacksdk instead of id attr.
In tests, use stricter mocking with spec_set (that actually fails to
access or set an atribute of a mock that is absent from the spec class
or instance) to guard against future possible changes.
Change-Id: I83c420d0e91e00f583a28833f4f710cf70b01fa8
Story: 2008379
Task: 41300
This change implements an alternative middleware which supports the
same deferred auth mechanism as the keystone auth middleware.
When auth fails the header X-Identity-Status is set to Invalid, which
only becomes an Unauthorized response when the path is not public.
Without this change, the paths /, /v1 and /v1/continue
incorrectly require authentication when using basic auth.
Change-Id: I780151870f851ad5dcd45610aacedcca23607a71
Story: 2007656
Task: 39826
When the config option ``auth_strategy`` is set to ``http_basic`` then
non-public API calls require a valid HTTP Basic authentication header to be
set. The config option ``http_basic_auth_user_file`` defaults to
``/etc/ironic-inspector/htpasswd`` and points to a file which supports the
Apache htpasswd syntax[1]. This file is read for every request, so no
service restart is required when changes are made.
The only password digest supported is bcrypt, and the ``bcrypt``
python library is used for password checks since it supports ``$2y$``
prefixed bcrypt passwords as generated by the Apache htpasswd utility.
To try basic authentication, the following can be done:
* Set ``/etc/ironic-inspector/inspector.conf`` ``DEFAULT`` ``auth_strategy``
to ``http_basic``
* Populate the htpasswd file with entries, for example:
``htpasswd -nbB myName myPassword >> /etc/ironic-inspector/htpasswd``
* Make basic authenticated HTTP requests, for example:
``curl --user myName:myPassword http://localhost:6385/v1/introspection``
[1] https://httpd.apache.org/docs/current/misc/password_encryptions.html
Change-Id: If50dfbfc18445ad9fe27e17cb0ee1b317ff25a0b
Depends-On: https://review.opendev.org/729070
Story: 2007656
Task: 39826
This patch splits API and conductor services for ironic-inspector.
Previous patch utilized lock from tooz coordinator, this patch adds
a coordinator wrapper for easier usage and further introduces group
interfaces.
Each conductor service will join a predefined group to mark it's
availability, on each request, API service will query members from
the group and randomly choose on of them, create desiginated topic
and deliver request to it.
The feature is tested with the memcached, file backend of tooz.
Other backends are not fully tested but may work as well, please
refer to tooz documentation for driver compatibilities[1].
[1] https://docs.openstack.org/tooz/latest/user/compatibility.html
Story: 2001842
Task: 30376
Change-Id: I419176cd6d44d74c066db275ef008fe8bb6ef37a
Configurable introspection data storage backend [1] is proposed
to provide flexible extension of introspection data storage
instead of the single support of Swift storage backend.
This patch adds plugin mechanism for loading introspection
storage, creates database backend and moves Swift storage
into a plugin.
[1] http://specs.openstack.org/openstack/ironic-inspector-specs/specs/configurable-introspection-data-backends.html
Story: 1726713
Task: 11373
Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com>
Change-Id: Ie4d09dc0afc441b20a1e5e3bd8e742b1df918954
Configurable introspection data storage backend [1] is
proposed to provide flexible extension of introspection
data storage instead of the single support of Swift storage
backend.
This patch adds database support for using ironic-inspector
database as the storage backend.
A table named ``introspection_data`` is created to serve as
the storage for introspected data.
[1] http://specs.openstack.org/openstack/ironic-inspector-specs/specs/configurable-introspection-data-backends.html
Change-Id: I8b29b7b86d90823d29b921ebf64acddbcd2d8d0d
Story: 1726713
Task: 11373
this patch introduces an oslo.policy-based API access policy
enforcement engine to ironic-inspector.
As part of implementation, a proper oslo.context-based request
context is also generated and assigned to each request.
Short overview of changes:
- added custom RequestContext class
- extends oslo.context to handle of "is_public_api" flag
(False by default)
- added context to request in each API route
- '/continue' api sets the "is_public_api" flag to True
- added documented definitions for API access policies and their
defaults
- added enforcement of these policies on API requests
- added oslo.policy-specific entry points to setup.cfg
- added autogenerated policy sample file with defaults
- added documentation with autogenerated policies
Change-Id: Iff6f98fa9950d78608f0a7c325d132c11a1383b3
Closes-Bug: #1719812
It became apparent in https://review.openstack.org/#/c/480679/ that these are
correct values for ipmitool to return.
Change-Id: Ied18a81dc899d8fc5290a2756f412e5075e923c4
The i18n team has decided not to translate the logs because it seems
like it not very useful.
This are the files as of now do not have merge conflicts.
Change-Id: I082f6302f120a8c0c81482b73be301e83fe3a3a8
Partial-Bug: #1674374
Mostly removes old authentication options and support for [discoverd].
Also update example.conf to the latest version.
Change-Id: Ided8705c4345a1170c211d926d916cec2173ccb9
There is a demand to use introspection on diskless nodes to figure out
what is possible to figure out.
We might need more changes to properly support diskless nodes, this
change is just to allow people to play with it.
The property ``local_gb == 0`` for a diskless node.
Change-Id: I21b2f2c069fdbf767367ec3d1fbf77bab6292b25
Partial-Bug: #1554243
Currently, state of a node introspection isn't kept in the database.
This change introduces:
* a new database column to keep the node introspection state
* an automaton to manage the node introspection state
* a decorator to declare a function performing an introspection state
transition
* a version_id column is added, to enhance database consistency, that
is consulted whenever node_info is committed
This change is part of the HA_Inspector effort[1]
[1] https://specs.openstack.org/openstack/ironic-inspector-specs/specs/HA_inspector.html
Closes-Bug: #1618835
Partial-Bug: #1525218
Change-Id: I18cb45f0d1194414715ccbe826d8a95610ec718d
Enhance the introspection status with the fields:
* uuid
* started_at
* finished_at
Change-Id: I36caa7d954a9bfb029d3f849fdf5e73f06f3da74
Partial-Bug: #1525238
From now on only rely on the IPA inventory and 2 additional fields:
boot_interface and root_device.
Also updated unit tests to use one inventory example.
Also removed duplicating unit tests and checks in test_process.
Also removed devstack support for the old ramdisk.
Change-Id: Ib382328295fc2c1b9143171b1047304febadcaca
Currently we are using only the resulting MAC(s) when doing a node lookup.
In many cases it is the MAC of the PXE-booting NIC. However, it's not necessary
the MAC that people used for enrolling the Ironic node, which will lead to
lookup failures on the virtual environment. This change makes the lookup
procedure use all of the valid MAC's.
Similarly, the enroll node_not_found_hook now checks all MAC's before creating
a node.
Code in the validate_interfaces hook was reordered to ensure we only keep
interfaces with valid MAC's even in the "all_interfaces" list.
Change-Id: Ie7df05d9a7855716fb835c90cfb0ac7fc4cd66df
For now, all Error exceptions logged with error level, but some exceptions
may be expected as correct flow and they shouldn't be logged as errors.
NotFoundInCacheError, for example, is raised when there isn't info in cache
about introspected node, this case may be handled by not_found_hook, so
this wouldn't be error anymore.
Change-Id: Ie537ccaef0035b2ef839c34fad0a5e6c9ba8f064
A green thread is now used instead of spawn_n for running asynchronous
operations during introspection, processing and aborting.
The existing periodic tasks are now run using Futurist PeriodicWorker.
Main shut down procedure was split into a separate function for convenience.
Also updated the example.conf to the latest versions (some pending updates from
3rdparty libraries included).
Change-Id: Id0efa31aee68a80ec55e4136c53189484b452559
Add new node_not_found_hook - enroll_node_not_found hook,
which allows to enroll unknown nodes to Ironic automatically.
Change-Id: If1528688504e4be4b2369b985bc576544d96868d
Related-Bug: #1524753
Currently, utils:processing_logger_prefix() assumes node_info.uuid to be a
string. However, a (broken) test case could encounter a TypeError logging:
File "ironic_inspector/utils.py", line 103, in processing_logger_prefix
return _('[node: %s]') % ' '.join(parts)
TypeError: sequence item 0: expected string, MagicMock found
The patch purpose is to apply str(node_info.uuid) as a precaution. This may
also help track where the real issue is with a broken test case.
Change-Id: I689f67c44a6304f7b296829fef3339872a4f6d36
Currently our logging in processing is very inconsistent:
some log strings mention node UUID, some - node BMC IP, some nothing.
This change introduces a common prefix for all processing logs
based on as much information as possible.
Only code that actually have some context about the node (either
NodeInfo or introspection data) is updated.
Also logging BMC addresses can be disabled now.
Updates example.conf (a lot of updated comments from oslo).
Change-Id: Ib20f2acdc60bfaceed7a33467557b92857c32798
Deprecated since liberty. We used to need it because ironic didn't have
'manageable' and 'enroll' states when ironic-inspector was started.
Change-Id: Ia38c40806836820219e86fe277ffd0b2fbd9ec58
Closes-Bug: #1506347
Conditions:
* in-net: checks if address is in a network
Actions:
* set-capability: sets a capability
* extend-attribute: append value to a list attibute
Helper method NodeInfo.replace_field is added to simplify writing
similar action plugins.
Implements: blueprint rules
Change-Id: I7e47b3500624df1f2cb15445d05e1c9bca6dc9ae
When a node is behind an IPMI bridge the IPMI address stored in ironic
is not a trust worthy identifier for recognising a node returning its
introspection data, as it can be the same or multiple nodes. It will
also cause unique constraint errors as the database will not allow
multiple bmc_address attribute entries with the same value. This patch
adds detection for ipmi bridging to the get_ipmi_address function, so
if ipmi bridging is enabled then it returns None, and the node is added
without any bmc_address listed.
Change-Id: I09d17dffcde0f4a023bb03e1bce88bcf725cdf0a
Closes-Bug: #1488501
* Recommend using ENROLL for setting IPMI credentials
* Deprecate using maintenance mode, will be dropped once we stop
supporting Ironic Kilo
* Drop bits related to Ironic Juno, we no longer support it
* Clarify error message about wrong provision state, stop mentioning
maintenance mode there
Change-Id: I3a9d3ba24a32c7844cd6fd5e4f8ec4b15b2d9d20
Closes-Bug: #1479331
This is Kilo, so make it clear in README that we actually still
support it (only stand alone case).
Change-Id: I9a5bd97a2a33a12311626bb3fae878bfb5c9d53c
Currently, Ironic Inspector talks with Ironic using public API endpoint.
This patch fix this by making endpoint type configurable and also
make default value "internalURL".
Change-Id: I11f8016a69fabe450989174b846cf84eacf83652
Partial-Bug: #1470565
Takashi Kajinami