Commit Graph

14 Commits

Author SHA1 Message Date
Anton Arefiev 3fe42b53fd SQLAlchemy 2.0 Support
Primarily remove the workaround added in
Ia6d512ff2ae417bab938cb095fbb0884d195010a which added
continued use of autocommit, which is incompatible with
SQLAlchemy 2.0.

Also set the environment for unit tests to report compatability
warnings, although it appears none are being reported at this time.

Also cuts out the db upgrade cruft to only use the online database
migration code through oslo_db's enginefacade, which has the smarts
to handle online or offline migrations.

And then, retools unit/functional test data storage to utlize sqlite,
and in that re-tooled the queries to prevent locking conditions
which could exist with queries, and some additional refactoring/cleanup.

Also, don't mock and test time.sleep().

Additionally, it looks like we have discovered the root cause of the
memory/connection leakage issue which has been observed, due to the
way lists of nodes are processed/returned.

This change was based upon the work in
I506da42a9891a245831f325e34bec92e0a3f33f0 which is included in
this commit as the entire database structure and interaction
has been modified for ironic-inspector.

Co-Authored-By: aarefiev <aarefiev@mirantis.com>
Story: 2009727
Task: 44132
Change-Id: Ic88eb9dec5fddc924a72d9a23c17a304954ebf46
2022-12-15 09:28:55 -08:00
Dmitry Tantsur 0d20064f44 Add leader election for periodic sync with ironic
Periodically elect a leader to do the cleanup sync with
Ironic (to avoid unnecessary calls to  Ironic in a
multi-inspector setup).
Add config option 'leader_election_interval' to control
the period in which the leaders is elected.

Co-authored-by: Arne Wiebalck <arne.wiebalck@cern.ch>

Story: #2007801
Change-Id: I6f60aea41865f09f9e53e5d91b1e887f0b39efbd
2020-07-02 12:17:28 +02:00
Dmitry Tantsur 6e38efffea Stop aborting the process on periodic task failures
It's nice to know when a periodic task fails, but aborting the whole
process makes it fragile in the events of e.g. problems with accessing
ironic, even if transient.

Simplify the overly complex unit tests and split the dual-purpose task.

Story: #2007622
Task: #39653
Change-Id: I64f5ca6e377a4cdfa08319ef9973b591cc552121
2020-05-06 09:24:29 +02:00
Dmitry Tantsur cee5922674 Hacking: enforce usage of autospec=True in tests
Using autospec ensures that mocked functions are called with correct
arguments, so it's highly desired to have it.

Change-Id: I9c8395adf852495d2ef6db732d727990e8abd5d7
2020-04-28 12:27:41 +02:00
Sean McGinnis a9f7f67de5
Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.

Change-Id: Ic67f09a223ae2d0cb460771a10a4122307afa05b
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-18 16:15:28 -05:00
Riccardo Pittau 3accdfbbc6 Use openstacksdk for ironic module
This patches removes the ironic-client dependency from the
ironic module in favor of openstacksdk.

Increase minimum required version of openstacksdk to use
recent added features.

Change-Id: I31964179835ad454e8205a59f483b419de4fb62d
2020-03-23 14:28:09 +01:00
Kaifeng Wang 293b0c7c15 Split API and conductor services
This patch splits API and conductor services for ironic-inspector.
Previous patch utilized lock from tooz coordinator, this patch adds
a coordinator wrapper for easier usage and further introduces group
interfaces.

Each conductor service will join a predefined group to mark it's
availability, on each request, API service will query members from
the group and randomly choose on of them, create desiginated topic
and deliver request to it.

The feature is tested with the memcached, file backend of tooz.
Other backends are not fully tested but may work as well, please
refer to tooz documentation for driver compatibilities[1].

[1] https://docs.openstack.org/tooz/latest/user/compatibility.html

Story: 2001842
Task: 30376

Change-Id: I419176cd6d44d74c066db275ef008fe8bb6ef37a
2019-08-12 15:29:55 +08:00
Kaifeng Wang 1bed475409 Adds tooz locking driver
Adds tooz requirements, configuration options and locking support.

Story: 2001842
Task: 30376

Change-Id: I78ffdeeb502cbed0550152b19efbb224f47c65af
2019-08-02 09:58:10 +08:00
Dmitry Tantsur 258d7388a4 Expose baremetal-introspection endpoint via mdns
This change adds an option to publish the endpoint via mDNS on start
up and clean it up on tear down.

Story: #2005393
Task: #30384
Change-Id: Ia9407cb065979aac6761d3e4122d3884e45b559d
2019-05-22 13:58:55 +02:00
Kaifeng Wang 29d8515f50 Support reapply with supplied introspection data
This patch adds support to provide unprocessed introspection data
to reapply a node. The provided introspection data will be save to
current introspection storage backend.

Change-Id: I969ae9c32f53f89c006a64a006388ddea9542aa5
Story: 1564863
Task: 11344
2019-04-15 15:04:33 +00:00
space d278bb6f77 introspection data backend: plugin layer
Configurable introspection data storage backend [1] is proposed
to provide flexible extension of introspection data storage
instead of the single support of Swift storage backend.

This patch adds plugin mechanism for loading introspection
storage, creates database backend and moves Swift storage
into a plugin.

[1] http://specs.openstack.org/openstack/ironic-inspector-specs/specs/configurable-introspection-data-backends.html

Story: 1726713
Task: 11373

Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com>
Change-Id: Ie4d09dc0afc441b20a1e5e3bd8e742b1df918954
2019-01-07 17:31:15 +08:00
Kaifeng Wang a228912827 Wrap rpc server into oslo.service
This patch is part of work splitting inspector api and worker.
The rpc server is wrapped into oslo.service, and launched from
api process.

Now api and worker belongs to separate thread, functional tests
use the fake messaging driver, while devstack jobs use rabbitmq.

Change-Id: Ie03d16f9d115c3561723463dea6a57d2a763bcc6
Story: #2001842
Task: #23109
2018-10-23 09:17:42 +08:00
Dmitry Tantsur e7c3218f71 Add manage_boot parameter to introspection API
Adds a new node field "manage_boot" to store this value. When it is set
to False, neither boot device nor power state are touched for this node.
Instead, we expect a 3rd party to handle them.

We still manage the PXE filter because the node may need DHCP.

Change-Id: Id3585bd32138a069dfcfc0ab04ee4f5f10f0a5ea
Story: #1528920
Task: #11338
2018-06-25 12:09:17 +02:00
Kaifeng Wang 6469a1fc0f Introduce oslo.messaging and sync rpc call
Adds oslo.messaging to ironic-inspector, and convert
inspect, abort and reapply to synchronized rpc calls.

This is the first step of API and worker seperation.

Change-Id: I15e86d7feb623b6b2889891b9700e5de6b3164cd
Story: #2001842
Task: # 12609
2018-05-06 21:07:25 +08:00