Primarily remove the workaround added in
Ia6d512ff2ae417bab938cb095fbb0884d195010a which added
continued use of autocommit, which is incompatible with
SQLAlchemy 2.0.
Also set the environment for unit tests to report compatability
warnings, although it appears none are being reported at this time.
Also cuts out the db upgrade cruft to only use the online database
migration code through oslo_db's enginefacade, which has the smarts
to handle online or offline migrations.
And then, retools unit/functional test data storage to utlize sqlite,
and in that re-tooled the queries to prevent locking conditions
which could exist with queries, and some additional refactoring/cleanup.
Also, don't mock and test time.sleep().
Additionally, it looks like we have discovered the root cause of the
memory/connection leakage issue which has been observed, due to the
way lists of nodes are processed/returned.
This change was based upon the work in
I506da42a9891a245831f325e34bec92e0a3f33f0 which is included in
this commit as the entire database structure and interaction
has been modified for ironic-inspector.
Co-Authored-By: aarefiev <aarefiev@mirantis.com>
Story: 2009727
Task: 44132
Change-Id: Ic88eb9dec5fddc924a72d9a23c17a304954ebf46
This is a follow up to comments on https://review.opendev.org/720010.
Adding autospect=True requires an update to ironic_inspector.test.\
base.BaseTest.assertCalledWithPatch.
Also add autospec=True in several mocks where it was not used.
Change-Id: Ia340d7a8ecde1c8cf191555de87b11b19b30f06c
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: Ic67f09a223ae2d0cb460771a10a4122307afa05b
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This patches removes the ironic-client dependency from the
ironic module in favor of openstacksdk.
Increase minimum required version of openstacksdk to use
recent added features.
Change-Id: I31964179835ad454e8205a59f483b419de4fb62d
The devstack example configuration is synchronized with
https://docs.openstack.org/ironic/latest/contributor/dev-quickstart.html
Also remove setting IRONIC_VM_LOGS_DIR from the example local.conf,
as it does not work, at least not on CentOS.
Change-Id: Ife51165742a8b762e84d5ac7ad807480669bfedb
Consolidate all config options under ``conf`` directory.
New config modules should give a better picture of the configuration
options provided by the inspector.
Change-Id: I501ed0787ff4e1d91462f936e1a54de2c7abb35c
Related-Bug: #1561100
Co-Authored-By: Anton Arefiev <aarefiev@mirantis.com>
this patch introduces an oslo.policy-based API access policy
enforcement engine to ironic-inspector.
As part of implementation, a proper oslo.context-based request
context is also generated and assigned to each request.
Short overview of changes:
- added custom RequestContext class
- extends oslo.context to handle of "is_public_api" flag
(False by default)
- added context to request in each API route
- '/continue' api sets the "is_public_api" flag to True
- added documented definitions for API access policies and their
defaults
- added enforcement of these policies on API requests
- added oslo.policy-specific entry points to setup.cfg
- added autogenerated policy sample file with defaults
- added documentation with autogenerated policies
Change-Id: Iff6f98fa9950d78608f0a7c325d132c11a1383b3
Closes-Bug: #1719812
upcoming policy-related changes need a properly initialized
global oslo_config's CONF instance, which inspector's unit tests
currently lack.
As a side effect of implementing this, the 'dbsync' module was
changed to not register it's CLI options right on its import.
Also, setting default log levels was moved to a function which
is already registered as the one providing defaults to options
in oslo_config's entrypoints. Config sample is updated accordingly.
Change-Id: I20bc537605062900d00fcc0343172774c1cd1363
Legacy EngineFacade was deprecated a while ago, using new
EngineFacade system is the recommended way to work with
db engine and session.
The new system has a lot of notable changes[1]:
* Thread-safe initialization of EngineFacade;
* Declarative reader/writer transactions separation;
* Transaction scope defined by developer-specified context.
[1] http://specs.openstack.org/openstack/oslo-specs/specs/kilo/make-enginefacade-a-facade.html
Closes-Bug: #1475182
Change-Id: Ia03f35a1f3b22b2eda74263e319469cf2391e7b5
oslo.config is becoming more strict with type enforcement [1], so we
cannot use False as slave_connection string.
[1] https://review.openstack.org/328692
Partial-Bug: #1683732
Change-Id: I6c6182df3a54d68737f8bbc77f1a2890ffcb0da2
Set it to True for the PXE-booting port, to False for all the others.
Create an extended functional tests covering various operations with ports.
Change-Id: I435a5c04884b6c4da70cb7260b305fbde23eebc0
Closes-Bug: #1667472
The i18n team has decided not to translate the logs because it seems
like it not very useful.
This are the files as of now do not have merge conflicts.
Change-Id: I082f6302f120a8c0c81482b73be301e83fe3a3a8
Partial-Bug: #1674374
InfiniBand is computer-networking communications standard
used in high-performance computing, features very high
throughput and very low latency.
This patch allow ironic-inspector to add the client_id
to ironic port extra. The client_id option allow pxe boot
from InfiniBand interface.
Closes-Bug: #1532534
Depends-On: Ifad453977e5d3be64b34e544f269835a72b4d73f
Change-Id: I479d54c29bcacb6bd5c1ab20033ae6e428b0e744
This adds a plugin to process the raw LLDP TLVs stored in Swift for
the Basic Mgmt, 802.1, and 802.3 data sets and store the parsed data
back in Swift. It implements the TLV processing as described in the
specification:
http://specs.openstack.org/openstack/ironic-inspector-specs/specs/lldp-reporting.html
Change-Id: I854826787ff045ffb2807970deaba8b77cbe277d
Closes-Bug: 1647515
Related-Bug: 1626253
Db column started_at and finished_at are defined as float type, but
float time doesn't fit into default db Float size. This change migrates
columns type to DateTime.
Closes-Bug: 1658975
Change-Id: I13bbb3c9a3e6e8bc744473437a378453ebc81318
Currently, state of a node introspection isn't kept in the database.
This change introduces:
* a new database column to keep the node introspection state
* an automaton to manage the node introspection state
* a decorator to declare a function performing an introspection state
transition
* a version_id column is added, to enhance database consistency, that
is consulted whenever node_info is committed
This change is part of the HA_Inspector effort[1]
[1] https://specs.openstack.org/openstack/ironic-inspector-specs/specs/HA_inspector.html
Closes-Bug: #1618835
Partial-Bug: #1525218
Change-Id: I18cb45f0d1194414715ccbe826d8a95610ec718d
Oslotest base class should be used as the base for unit test classes.
In addtion the oslotest BaseTestCase inherits from testtools.testCase,
according to fixtures docs "testtools has it’s own implementation of
useFixture so there is no need to use fixtures.TestWithFixtures with
testtools.TestCase."
Co-Authored-By: moshe levi <moshele@mellanox.com>
Change-Id: I7296d59ee30230ec2de6d55649c9f57a33534435
For Ironic multi-tenant networking support, we need to be able to
discover the nodes local connectivity. To do this IPA can try to pull
LLDP packets for every NIC.
This patch adds a processing hook to handle the data from these packets
in ironic-inspector so that we can populate the correct fields fields in
Ironic. The generic lldp hook only handles the mandatory fields port id
and chassis id, set on port_id and switch_id in local_link_connection.
Further LLDP fields should be handled by additional vendor specific LLDP
processing hooks, that populate the switch_info field in a non-generic
way.
Change-Id: I884eaaa9cc54cd08c21147da438b1dabc10d3a40
Related-Bug: #1526403
Depends-On: Ie655fd59b06de7b84fba3b438d5e4c2ecd8075c3
Depends-On: Idae9b1ede1797029da1bd521501b121957ca1f1a
From now on only rely on the IPA inventory and 2 additional fields:
boot_interface and root_device.
Also updated unit tests to use one inventory example.
Also removed duplicating unit tests and checks in test_process.
Also removed devstack support for the old ramdisk.
Change-Id: Ib382328295fc2c1b9143171b1047304febadcaca
Fix calls from .dict() to to_dict() in
IntrospectionRule.check_conditions and corresponding unit test.
Co-Authored-By: Nathan Harrison <nathan.harrison@sap.com>
Related-Bug: #1552781
Change-Id: I9fe7d400edcd7573132081a196e74eb09bc88cb7
A green thread is now used instead of spawn_n for running asynchronous
operations during introspection, processing and aborting.
The existing periodic tasks are now run using Futurist PeriodicWorker.
Main shut down procedure was split into a separate function for convenience.
Also updated the example.conf to the latest versions (some pending updates from
3rdparty libraries included).
Change-Id: Id0efa31aee68a80ec55e4136c53189484b452559
Conditions: ``field`` supports new format, it allow to comparison
data both from inspection and node info:
{'field': 'node://ironic/style/path', 'op': 'eq', 'value': 'val'}
{'field': 'data://introspection/path', 'op': 'eq', 'value': 'val'}
Actions: ``value`` supports fetching data from introspection, it's
using python string formatting notation:
{'action': 'set-attribute', 'path': '/driver_info/ipmi_address',
'value': '{data[inventory][bmc_address]}'}
Related-Bug: #1524753
Change-Id: Ie05f82e7a29fba2f743217f0893c085fd843cd5b
not doing so introduces implicit (ordering) dependency issues between test
cases. For instance, in python client functional test[1] renaming
`TestSimplePythonAPI.test_introspect_get_status` to
`TestSimplePythonAPI.test_01_introspect_get_status` makes
`TestSimplePythonAPI.test_api_version` fail.
Notes:
I use this command to check for the cases where fixed uuids are being used:
`git grep -En "\<[^\s,\.(]*uuid[^\s,\.]*\s*=\s*['\"]"`
Links:
[1] https://github.com/openstack/python-ironic-inspector-client/blob/
master/ironic_inspector_client/test/functional.py#L137
Change-Id: Icd4a2a25824c6372d10fc40d67d8f275016b201c
Now a NodeInfo object has a lock instance attached, which is associated
with a node UUID. New calls acquire_lock and release_lock are added.
NodeInfo.find_node now always acquire a lock, get_node does it optionally.
NodeInfo.finished now always release a lock.
Note that this change does not make NodeInfo thread-safe. Rather it's
preventing simultaneous operations on the same node.
Change-Id: I0c0bd2f4e0622fd7086660b0874c5b8dca82b4d2
Implements: blueprint node-state
These changes allow a developer to generate migrations using the
autogenerate function without the need to pass in a config file that
includes sql connection information.
Change-Id: I6b3942f7747e8f73e52925c24340e20daeb78911
This approach was an optimization, but it proved to bring more
troubles than benefits - see LP issue for details.
Precautions are made to avoid breakage. Still, some 3rdparty
hooks might get broken by this change.
Change-Id: I5a84512758e92c87091d6306c8d5baa944955e55
Closes-Bug: #1492946
Allows for much cleaner and complete testing, also simplifies future
changes.
Change-Id: Ia586c21ff8e6e6449c6048d4764d1d12259ab0ba
Partial-Bug: #1492946
We'll have database code not related to node_cache, i.e. table
for rules.
Change-Id: I58a98d9250e7ca136b3d79e44177faa401f5c29e
Implements: blueprint rules
Currently, Ironic-inspector uses SQLite as database.
We should migrate to oslo.db.
In this patch, make database connect operations to oslo_db.
Change-Id: Ibd0f787b570bc101eab9fbd0b59d8a775a95e2e0
Implements: blueprint migrate-to-oslodb
This is a backward incompatible change of hooks interfaces.
The goal is to make these interfaces easier extensible, and to give
hooks access to NodeInfo instance.
Change-Id: I5b00a71e86a5a57111c25b0e16c662107074c21f
Implements: blueprint plugin-interface-v2
As agreed on the summit I'm renaming the python modules
and doing some adjustments:
* This is a breaking change, so version is bumped to 2.0.0
* Used this chance to split conf options over proper sections
* RELEASES.rst is gone; it's too hard to keep it up-to-date;
anyway git does better job at doing history
* Dropped deprecated option ports_for_inactive_interfaces
* Dropped old /v1/discover endpoint and associated client call
* No longer set on_discovery and newly_discovered in Node.extra
(deprecated since 1.0.0, superseded by the get status API)
* Default firewall chain name is "ironic-inspector" and
is configurable
Notes:
* Some links will be updated after real move.
* Stable branches will probably use the old name.
* Some usage of discovery word is left in context of
"discovered data"
* DIB element will probably be deprecated, so leaving it
alone for now.
* Some usages of word "discovery" in the README will be updated
later to make this patch a bit smaller
* Ramdisk code will be moved to IPA, so not touching it too much
Change-Id: I59f1f5bfb1248ab69973dab845aa028df493054e
Anton Arefiev