Primarily remove the workaround added in
Ia6d512ff2ae417bab938cb095fbb0884d195010a which added
continued use of autocommit, which is incompatible with
SQLAlchemy 2.0.
Also set the environment for unit tests to report compatability
warnings, although it appears none are being reported at this time.
Also cuts out the db upgrade cruft to only use the online database
migration code through oslo_db's enginefacade, which has the smarts
to handle online or offline migrations.
And then, retools unit/functional test data storage to utlize sqlite,
and in that re-tooled the queries to prevent locking conditions
which could exist with queries, and some additional refactoring/cleanup.
Also, don't mock and test time.sleep().
Additionally, it looks like we have discovered the root cause of the
memory/connection leakage issue which has been observed, due to the
way lists of nodes are processed/returned.
This change was based upon the work in
I506da42a9891a245831f325e34bec92e0a3f33f0 which is included in
this commit as the entire database structure and interaction
has been modified for ironic-inspector.
Co-Authored-By: aarefiev <aarefiev@mirantis.com>
Story: 2009727
Task: 44132
Change-Id: Ic88eb9dec5fddc924a72d9a23c17a304954ebf46
oslo.db 12.1.0 has changed the default value for the 'autocommit'
parameter of 'LegacyEngineFacade' from 'True' to 'False'. This is a
necessary step to ensure compatibility with SQLAlchemy 2.0. However, we
are currently relying on the autocommit behavior and need changes to
explicitly manage sessions. Until that happens, we need to override the
default.
Change-Id: Ia6d512ff2ae417bab938cb095fbb0884d195010a
Co-authored-by: Stephen Finucane <stephenfin@redhat.com>
Added 'scope' property to IntrospectionRule and logic to check if a node
falls in the same scope.This allows introspection rules to be applied on
selected nodes instead of every one of them.
Story: 2006995
Task: 37763
Change-Id: I77034f032ea0ec16886afdd928546eb801f7a90a
When using store_data=database with sqlalchemy, running introspection results
in a DBDuplicateEntry error. This happens because the query when adding an
entry uses both the primary key (uuid) and the processed flag, but the
processed flag is not a key. This change makes it a key so that both
unprocessed and processed data can be stored in the table.
Note - since the previous migration hasn't been released yet this fixes it
without creating a new one.
Change-Id: I052594d529ae363fce50b1726169d86583bb1439
Story: #2004992
Task: #29463
Configurable introspection data storage backend [1] is
proposed to provide flexible extension of introspection
data storage instead of the single support of Swift storage
backend.
This patch adds database support for using ironic-inspector
database as the storage backend.
A table named ``introspection_data`` is created to serve as
the storage for introspected data.
[1] http://specs.openstack.org/openstack/ironic-inspector-specs/specs/configurable-introspection-data-backends.html
Change-Id: I8b29b7b86d90823d29b921ebf64acddbcd2d8d0d
Story: 1726713
Task: 11373
Adds a new node field "manage_boot" to store this value. When it is set
to False, neither boot device nor power state are touched for this node.
Instead, we expect a 3rd party to handle them.
We still manage the PXE filter because the node may need DHCP.
Change-Id: Id3585bd32138a069dfcfc0ab04ee4f5f10f0a5ea
Story: #1528920
Task: #11338
Legacy EngineFacade was deprecated a while ago, using new
EngineFacade system is the recommended way to work with
db engine and session.
The new system has a lot of notable changes[1]:
* Thread-safe initialization of EngineFacade;
* Declarative reader/writer transactions separation;
* Transaction scope defined by developer-specified context.
[1] http://specs.openstack.org/openstack/oslo-specs/specs/kilo/make-enginefacade-a-facade.html
Closes-Bug: #1475182
Change-Id: Ia03f35a1f3b22b2eda74263e319469cf2391e7b5
When using the Inspector rules API to query existing introspection
rules, inspector does not return 'invert' or 'multiple' attributes of
conditions associated with the rules.
Change-Id: I08606cea676ecf57bbb3b73077c4832240fbe0d2
Closes-Bug: #1670372
The option sqlite_db was deprecated and removed by
Id269d921e40edf95eb977b011f1753f633b79d18
As we passed it as positional argument, another max_pool_size
var was set automatically.
This patch removes dropped sqlite_db option.
Depends-On: Icdd89453ec3ebecd8375a5f2efd1a0d4bf6b4064
Change-Id: I6c1cc55d73a1d9cc0728be93b3b134d5312f783b
Closes-Bug: #1676809
This patch modifies current attributes matching from a single
name-value->node hit to a best-match score.
Also using an UUID as the attributes table primary key to allow exposing
attributes in API later (bug 1525231).
Change-Id: I205e31652b21b9e030b9530149e533b29c52a387
Closes-Bug: 1651719
Partial-Bug: 1525231
Mostly removes old authentication options and support for [discoverd].
Also update example.conf to the latest version.
Change-Id: Ided8705c4345a1170c211d926d916cec2173ccb9
Db column started_at and finished_at are defined as float type, but
float time doesn't fit into default db Float size. This change migrates
columns type to DateTime.
Closes-Bug: 1658975
Change-Id: I13bbb3c9a3e6e8bc744473437a378453ebc81318
Currently, state of a node introspection isn't kept in the database.
This change introduces:
* a new database column to keep the node introspection state
* an automaton to manage the node introspection state
* a decorator to declare a function performing an introspection state
transition
* a version_id column is added, to enhance database consistency, that
is consulted whenever node_info is committed
This change is part of the HA_Inspector effort[1]
[1] https://specs.openstack.org/openstack/ironic-inspector-specs/specs/HA_inspector.html
Closes-Bug: #1618835
Partial-Bug: #1525218
Change-Id: I18cb45f0d1194414715ccbe826d8a95610ec718d
Some conditions do not have a native counterparts, so we need a way
to invert them (aka NOT operation). This patch adds a new generic
parameter "invert", defaulting to False.
Change-Id: I50342689ba52346a5a4fbf362536b629fc688986
These changes allow a developer to generate migrations using the
autogenerate function without the need to pass in a config file that
includes sql connection information.
Change-Id: I6b3942f7747e8f73e52925c24340e20daeb78911
OpenStack CI uses MyISAM by default, which does not even support
foreign keys. This change essentially mirrors what ironic is doing.
Also sets the default charset to UTF-8.
Closes-Bug: #1506160
Change-Id: I7caf0a2122f98c59f121f003ac092ecb0ce123c4
MySQL can't use TEXT fields for primary keys. This change switches all them
to VARCHAR(255). This change should not be breaking for SQLite, as it
does not distinguish between these two data types.
README is updated with up-to-date information about the connection option.
Change-Id: I0153855c1827b55067a7c04310bfad7eb71f35fe
Closes-Bug: #1501746
This patch adds a new command ironic-inspector-dbsync which can be used
to sync the ironic inspector database using alembic migrations. It adds
a migration to match the current required db schema.
Change-Id: I21188b3f5003c8ab43d82903473e2a6ef7f755a0
Closes-Bug: #1495620
This patch introduces a simple JSON-based DSL to run on introspected
data. Conditions and actions are provided via new plugin entry points.
This patch is missing PUT operation on a rule, this can be added later.
Also not all planned conditions and actions are added in this patch,
will also follow up.
Implements: blueprint rules
Change-Id: If4d17b5f1462d03879cb4c2ff4e5cb3ea364b697
We'll have database code not related to node_cache, i.e. table
for rules.
Change-Id: I58a98d9250e7ca136b3d79e44177faa401f5c29e
Implements: blueprint rules
Anton Arefiev