Commit Graph

200 Commits

Author SHA1 Message Date
Takashi Kajinami ea9ad2891c Remove dependency on pytz
pytz will be removed from RHEL/CentOS 10 because of the built-in
zoneinfo[1].

Because the current usage of pytz can be very easily replaced, this
removes the dependency on pytz.

[1] https://issues.redhat.com/browse/RHEL-219

Change-Id: Iafcaf2f1095cd7c738dac391a9af10622806e932
2024-01-27 10:42:39 +00:00
Tobias Urdin 84b5f8cc1b Use new get_rpc_client API from oslo.messaging
Use the new API that is consistent with
the existing API instead of instantiating the client
class directly.

This was introduced in release 14.1.0 here [1] and
added into oslo.messaging here [2]

[1] https://review.opendev.org/c/openstack/requirements/+/869340
[2] https://review.opendev.org/c/openstack/oslo.messaging/+/862419

Change-Id: I66bef4d339b5ee2462dc46f626fd1b2c6182b2cd
2023-01-19 21:05:00 +00:00
Anton Arefiev 3fe42b53fd SQLAlchemy 2.0 Support
Primarily remove the workaround added in
Ia6d512ff2ae417bab938cb095fbb0884d195010a which added
continued use of autocommit, which is incompatible with
SQLAlchemy 2.0.

Also set the environment for unit tests to report compatability
warnings, although it appears none are being reported at this time.

Also cuts out the db upgrade cruft to only use the online database
migration code through oslo_db's enginefacade, which has the smarts
to handle online or offline migrations.

And then, retools unit/functional test data storage to utlize sqlite,
and in that re-tooled the queries to prevent locking conditions
which could exist with queries, and some additional refactoring/cleanup.

Also, don't mock and test time.sleep().

Additionally, it looks like we have discovered the root cause of the
memory/connection leakage issue which has been observed, due to the
way lists of nodes are processed/returned.

This change was based upon the work in
I506da42a9891a245831f325e34bec92e0a3f33f0 which is included in
this commit as the entire database structure and interaction
has been modified for ironic-inspector.

Co-Authored-By: aarefiev <aarefiev@mirantis.com>
Story: 2009727
Task: 44132
Change-Id: Ic88eb9dec5fddc924a72d9a23c17a304954ebf46
2022-12-15 09:28:55 -08:00
Riccardo Pittau 6513d3db37 Fix CI
Sync with ironic configuration to fix grenade job
Drop lower-constraints.txt and its testing (was [1])
Fix ironic-inspector-tempest-managed-non-standalone sine we're here

[1] https://review.opendev.org/c/openstack/ironic-inspector/+/840075

Change-Id: I9b08c6e61c39a0e5616e51256f76a19dc5fd00c0
2022-05-09 09:31:55 +02:00
Riccardo Pittau 611a543ecd Update requirements
Update minimum required versions of python packages.
The updates is based on crosscheck requirements from required
packages:

oslo-policy 3.7.0 depends on oslo.context>=2.22.0

increase oslo-service version to be able to use eventlet 0.26.0
oslo-service 1.24.0 depends on eventlet!=0.18.3, !=0.20.1, <0.21.0 and >=0.18.2

increase oslo-log version to fix error:
AttributeError: type object 'deprecated' has no attribute 'WALLABY'

oslo-log 4.3.0 depends on oslo.serialization>=2.25.0
oslo-log 4.3.0 depends on pbr>=3.1.1
oslo-log 4.3.0 depends on oslo.i18n>=3.20.0

Change-Id: I20a5a20b94ffd27ea34f23ac89ca2826eb548b1a
2021-07-13 18:11:31 +02:00
Ghanshyam Mann 6a4899be86 Fix & suppress oslo policy warnings
This fix the below warning for DeprecatedRule:

Since 3.7.0, oslo policy started the DeprecationWarning[1] if
deprecated_reason and deprecated_since param are not passed
in DeprecatedRule or they are passed in RuleDefault object.

Andf suppress the policy deprecation and default change warnings

Oslo policy log warnings if defaults for policies are changed.
With new RBAC change every policy rules' default is changed,
which end up lot of warnings in logs. We can suppress these for now
until we are enforcing new defaults.

- https://zuul.opendev.org/t/openstack/build/5cefaef6d02a4b7abe3c449491b81e68/log/job-output.txt#879

[1] https://github.com/openstack/oslo.policy/blob/3.7.0/oslo_policy/policy.py#L1538

Change-Id: If481a5afc3b23d1d196ffd7576d0784a9702da59
2021-07-05 09:00:06 -05:00
Julia Kreger 901c816b23 Add upgrade check, and json2yaml policy handling
Adds the ironic-inspector-status command as well with the
requried upgrade check.

Mostly based upon https://review.opendev.org/#/c/763262/
which is based upon https://review.opendev.org/#/c/748059/

Note: Also had to update the version of eventlet because
existing minimum requirement was seemingly incompatible with
prior versions and would prevent lower constraints testing
to proceed due to issues with ``os`` being patched by eventlet.

Change-Id: I1f479f834f1d79e9eeb591c58a52b6ab80c24534
2021-01-04 13:56:28 -08:00
Iury Gregory Melo Ferreira 9a89c30479 Fix l-c and mysql GRANT syntex for Ubuntu Focal
Also fixing user creation with GRANT in MySQL 8.0(Ubuntu Focal)
Ubuntu Focal (20.04) has mysql 8.0 and with mysql 8.0 there
is no implicit user creation with GRANT. We need to
create the user first before using GRANT command.
Change-Id: I4c0469628e82e86b4023a9b011d09f20f9b66aac
2020-09-11 14:36:56 -05:00
Zuul 0b9b175666 Merge "Identify accelerator devices during introspection" 2020-09-07 15:07:22 +00:00
Kaifeng Wang de2a27ad8b Identify accelerator devices during introspection
Identify acclerator devices by processing pci devices and update to
ironic node when found. Currently only Tesla T4 from NVIDIA is
supported.

Change-Id: Id702cb04cb2445d544965821680cd0cc5cfd37e5
Story: 2007971
Task: 40473
2020-08-20 14:03:37 +08:00
Riccardo Pittau f4648facf7 Replace retrying with tenacity
We are replacing all usages of the 'retrying' package with
'tenacity' as the author of retrying is not actively maintaining
the project. Tenacity is a fork of retrying, but has improved the
interface and extensibility (see [1] for more details). Our end
goal here is removing the retrying package from our requirements.

[1] https://github.com/jd/tenacity

Change-Id: I660192549918b9cd9738810cbfaf3dc4456f97fa
Story: #1635390
Task: #40647
2020-08-18 15:11:16 +02:00
Riccardo Pittau 523c52d8f0 Fix requirements job
The jsonpath-rw library has no cap anymore.

Change-Id: I1a692a32b5731c33f56ddc53ac5e4f0089c464f0
2020-08-11 17:06:17 +02:00
Julia Kreger 6e2cb60e77 Respond so Apache HTTPd doesn't think the request failed
When sending a literal empty response, Flask does not include a
ContentType in the response. While in many cases, we don't need
need a ContentType nor expect one on the API client, Apache
webserver can treat this as an error and generate an Error
indicating a Bad Gateway. When doing this, we also now include
an empty JSON body in the response for 202 messages. For 204
message errors, the message body is expected to be empty.

However, when this Bad Gateway error occurs, the API/Conductor
were proceeding like there was no issue. The API client on the
other hand thinks that a hard failure has occured.

Also adds some additional catches to provide additional logging
which turned out not to be needed in this case, but it would be
useful for others.

Change-Id: If2e7697e3fde58ab0a4193787e29d3acdca81ebf
2020-07-22 10:18:00 -07:00
Steve Baker 2532be9e05 Document [ironic] auth options for inspector
This change documents configuring inspector for client-side
authentication with a standalone Ironic API service.

Since this documention refers to a keystone auth feature which is
about to be released, requirements and lower-constraints are updated
to reflect the required release (see https://review.opendev.org/737365 ).

Change-Id: I567fc8c7f2147339856563ad880334791f93d99b
2020-06-23 13:35:12 +12:00
Steve Baker 196c019771 Enable Basic HTTP authentication middleware.
When the config option ``auth_strategy`` is set to ``http_basic`` then
non-public API calls require a valid HTTP Basic authentication header to be
set. The config option ``http_basic_auth_user_file`` defaults to
``/etc/ironic-inspector/htpasswd`` and points to a file which supports the
Apache htpasswd syntax[1]. This file is read for every request, so no
service restart is required when changes are made.

The only password digest supported is bcrypt, and the ``bcrypt``
python library is used for password checks since it supports ``$2y$``
prefixed bcrypt passwords as generated by the Apache htpasswd utility.

To try basic authentication, the following can be done:

* Set ``/etc/ironic-inspector/inspector.conf`` ``DEFAULT`` ``auth_strategy``
  to ``http_basic``
* Populate the htpasswd file with entries, for example:
  ``htpasswd -nbB myName myPassword >> /etc/ironic-inspector/htpasswd``
* Make basic authenticated HTTP requests, for example:
  ``curl --user myName:myPassword http://localhost:6385/v1/introspection``

[1] https://httpd.apache.org/docs/current/misc/password_encryptions.html

Change-Id: If50dfbfc18445ad9fe27e17cb0ee1b317ff25a0b
Depends-On: https://review.opendev.org/729070
Story: 2007656
Task: 39826
2020-06-05 01:28:40 +12:00
Hervé Beraud 5960cde402 Cap jsonschema 3.2.0 as the minimal version
Previous versions of jsonschema (<3.2.0) doesn't support python 3.8 [1].
Python 3.8 is part of the victoria supported runtimes [2] so we now force
to use jsonschema version 3.2.0 to avoid issues, remove ambiguity and ensure
that everything works with python 3 in general.

[1] https://github.com/Julian/jsonschema/pull/627
[2] https://governance.openstack.org/tc/reference/runtimes/victoria.html#python-runtimes-for-victoria

Change-Id: I944e60f8c1962bfa3b6478b14e682eeca7948098
2020-05-26 21:50:42 +02:00
Iury Gregory Melo Ferreira b9f1746643 Stop configuring install_command in tox.
Currently, we are overriding 'install_command' to use 'pip'. This is
considered poor behavior and 'python -m pip' should be used instead:

https://snarky.ca/why-you-should-use-python-m-pip/

It turns out that this is the the default value provided by tox:

https://tox.readthedocs.io/en/latest/config.html#conf-install_command

So we can remove the line and simply use the default value.

openstack-discuss thread
http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014237.html

Bumping requirements and l-c with:
keystoneauth1 3.18.0
alembic 0.9.6
pika 0.10.0 (necessary for installation)
Change-Id: I849342bd077ab2f8bbda4e0a228c308eb7a4908b
2020-05-14 14:56:26 +02:00
Riccardo Pittau 721bd814a6 Use latest version of python construct
Also increasing minimum version to 2.9.39 for String class name
change to PaddedString.

Change-Id: Ie7b80f4698a77208c89797b82f7e41fcd8dbf168
2020-05-14 10:37:07 +02:00
Dmitry Tantsur 1613a6b7ae Move Babel to test-requirements
It's not a runtime requirement, it's only used to extract i18n catalogs.

Change-Id: I29e624c894f84390b23a2d2d7f547e488c82d7e8
2020-04-06 14:55:44 +02:00
Riccardo Pittau 4eefb42550 Finalize ironicclient removal
Completely remove dependency from ironicclient in favor of
openstacksdk.

Change-Id: I9b98ddb05fe867a0d3884f736a2edd7b4d61b00c
2020-03-25 10:08:38 +01:00
Riccardo Pittau 3accdfbbc6 Use openstacksdk for ironic module
This patches removes the ironic-client dependency from the
ironic module in favor of openstacksdk.

Increase minimum required version of openstacksdk to use
recent added features.

Change-Id: I31964179835ad454e8205a59f483b419de4fb62d
2020-03-23 14:28:09 +01:00
Riccardo Pittau ba8ecd3cf5 Stop supporting Flask < 1.0
Latest version of Werkzeug (1.0.0 at the moment) broke
compatibility with older Flask, actually causing issues when
running lower-constraints tests, since Flask will install
highest version of Werkzueg after 0.7.
With this patch we require Flask to be at least version 1.0
to re-establish compatibility.

See failed test https://review.opendev.org/706701 or run
lower-constraints test locally for confirmation.

Change-Id: Id05b4a205379fdf1b26cb239757bd370f3fe88e3
2020-02-10 10:59:43 +01:00
Riccardo Pittau 9b1450398c Stop using six library
Since we've dropped support for Python 2.7, it's time to look at
the bright future that Python 3.x will bring and stop forcing
compatibility with older versions.
This patch removes the six library from requirements, not
looking back.

Change-Id: Ic443c7e4d5a5a849c4dc220207f8957e4c90bf53
2019-12-17 09:23:01 +01:00
Riccardo Pittau 96c5e6516c Limit lower-constraints.txt to only direct dependencies
Also increasing lower version of keystoneauth1 to a recommended
version for openstacksdk >= 0.30.0

Change-Id: Ifb9bbb8d091f28a8d65444caaeba301e2f707a14
2019-11-22 12:30:36 +01:00
Riccardo Pittau d3aee899a1 Raising minimum version of oslo.db
oslo.db was not compatible with Python 3.7 until version 4.40.0
because Python 3.7 makes "async" a keyword [1].
To prevent bad surprises, this patch increases the minumum version
required for oslo.db to 4.40.0.

[1] https://review.opendev.org/574833

Change-Id: I6b915a0b0f78690defa962be96c8ae9d35d69345
2019-10-07 14:46:26 +02:00
Dmitry Tantsur d257171497 Exclude python-ironicclient 3.0.0 from requirements.txt
It breaks no-auth mode and functional tests.

Change-Id: Ieea919ed50ab70c9f257d1bf3bd8241a713f5f7e
Depends-On: https://review.opendev.org/#/c/684376/
2019-09-24 17:37:30 +02:00
Kaifeng Wang 1bed475409 Adds tooz locking driver
Adds tooz requirements, configuration options and locking support.

Story: 2001842
Task: 30376

Change-Id: I78ffdeeb502cbed0550152b19efbb224f47c65af
2019-08-02 09:58:10 +08:00
Kaifeng Wang f979456c16 testenv does not install requirements
Fix testenv which only installs test-requirement, also updated
upper constraint url.

mock minimum version is bumped to 3.0.0,
keystonemiddleware minimum version is bumped to 4.18.0.

Change-Id: I162bcf372598c2268c9c5e30947e49eedb0147e3
2019-07-16 10:38:14 +08:00
Dmitry Tantsur 85a3186d50 Use openstacksdk for accessing Swift
This change replaces the swiftclient dependency with openstacksdk.
The ultimate goal is to use only openstacksdk for all cross-service
interactions.

The configuration option max_retires has been broken for several
releases since commit 7e8cdc0d0f,
this change deprecates it.

Change-Id: I635519f88e90f4267ff7a9cb063a697ff39e4710
2019-06-19 14:49:03 +02:00
Zuul bd2c386cd7 Merge "Expose baremetal-introspection endpoint via mdns" 2019-05-29 10:19:51 +00:00
Riccardo Pittau 3c219475e6 Update sphinx and uncap jsonschema
Following recent changes to global requirements [1], we need
to update sphinx requirements locally.
Also uncapping jsonschema following this [2] change.

[1] If558f184c959e4b63b56dec3ca1571d1034cfe5c
[2] I328cbd8216ec4591e08414dafc9ea099391efd86

Change-Id: If628a93b95cf73cb5af19b847758e9e50c5bf7a8
2019-05-23 12:12:47 +02:00
Dmitry Tantsur 258d7388a4 Expose baremetal-introspection endpoint via mdns
This change adds an option to publish the endpoint via mDNS on start
up and clean it up on tear down.

Story: #2005393
Task: #30384
Change-Id: Ia9407cb065979aac6761d3e4122d3884e45b559d
2019-05-22 13:58:55 +02:00
Kaifeng Wang edd6810c3d Wrap Flask into oslo.service
This patch is part of inspector HA work, which wraps inspector api into
oslo service.

oslo.service has also provided support to signal processing like SIGHUP or
SIGTERM, so these code were removed in this patch.

Deprecated current SSL cert/key options used by ironic-inspector, code
manually creates ssl context were removed. These options will be fed
from [ssl] section.

Change-Id: Ia5e16fcb9104556d62c90f5507f17b41f73a5208
Story: #2001842
Task: #12609
2018-10-23 09:20:16 +08:00
Chuck Short 47d7fb2025 Sync up Flask version with global-requirements
As Flask version was updated we need to update our own requirements.txt
to avoid reqirements-check job to fail on every patch that changes
anything related to requirements.

Change-Id: I22f471534eded46a8deecccdb07db71d0705226b
Signed-off-by: Chuck Short <chucks@redhat.com>
2018-07-11 12:54:11 -04:00
Kaifeng Wang 6469a1fc0f Introduce oslo.messaging and sync rpc call
Adds oslo.messaging to ironic-inspector, and convert
inspect, abort and reapply to synchronized rpc calls.

This is the first step of API and worker seperation.

Change-Id: I15e86d7feb623b6b2889891b9700e5de6b3164cd
Story: #2001842
Task: # 12609
2018-05-06 21:07:25 +08:00
Doug Hellmann 3b3a3ef20b Fix lower constraints and uncap eventlet
Fix the lower constraints settings to match the expected values.

We will manage the eventlet version using constraints now. See the
thread starting at
http://lists.openstack.org/pipermail/openstack-dev/2018-April/129096.html
for more details.

Change-Id: Ic44a750002943920c2f517f22df670989460dbc9
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-04-23 10:53:07 -04:00
OpenStack Proposal Bot 3946a94447 Updated from global requirements
Change-Id: I9f0bae0f2d5279c401c6d6698e5cdc2302839751
2018-03-26 08:04:42 +00:00
OpenStack Proposal Bot f440bcba11 Updated from global requirements
Change-Id: I64b9eb17a8bf2e121c06d578d5926a5e5c01c811
2018-03-14 05:31:06 +00:00
OpenStack Proposal Bot 1292657bb9 Updated from global requirements
Change-Id: I0e122923bb4d2736e7d52d123ab4a40b4923c164
2018-03-13 06:54:53 +00:00
OpenStack Proposal Bot 11c26d2f4a Updated from global requirements
Change-Id: Icc6ba6292e015c91fa2ef7ef053a25fab50b75bb
2018-02-17 09:32:11 +00:00
Dmitry Tantsur 3237511cc6 Retry port lists on failure in PXE filter periodic sync
These calls are subject to transient network problems, we should
not abort ironic-inspector process in this case. Also due to
bug 1748893 the port listing API can sometimes return HTTP 400.

This change retries port listing 5 times with 1 second break
before aborting the periodic task and thus the process.

This change introduces a dependency on the retrying library,
which is already widely used in OpenStack (including ironic).

Change-Id: I92fd70ca5692ce9f6798eedf9e540d5aa7c6f1af
Closes-Bug: #1748893
2018-02-14 15:10:39 +01:00
OpenStack Proposal Bot dc254b5659 Updated from global requirements
Change-Id: Ibc1719a9e91c3f99b2394b0d8514ff5b346aa947
2018-01-29 00:09:48 +00:00
OpenStack Proposal Bot ee7a761812 Updated from global requirements
Change-Id: I312c19b8306f542ff03a38e059030976d4017001
2018-01-16 12:29:24 +00:00
OpenStack Proposal Bot 23278737c2 Updated from global requirements
Change-Id: I7f5a6cdd8ec991b77a022562cc580f720e43d7b6
2017-12-19 01:18:26 +00:00
OpenStack Proposal Bot b22559455b Updated from global requirements
Change-Id: I8444c1c0285b1e3a576e926cff53de9c6d62595a
2017-12-07 13:24:32 +00:00
OpenStack Proposal Bot 3a854d7a2a Updated from global requirements
Change-Id: Ifae893e5cd31cc288357362b5381f1c53de8eed8
2017-12-05 03:06:00 +00:00
OpenStack Proposal Bot dbc7e85980 Updated from global requirements
Change-Id: I1269b8c61b53fcdda88eaf1ef57a17710c348755
2017-11-29 08:57:12 +00:00
OpenStack Proposal Bot 8de3bdd97a Updated from global requirements
Change-Id: I82cdde1293a6c3cc89f9869627d37df478b49bf2
2017-11-16 11:05:41 +00:00
OpenStack Proposal Bot 8104e33366 Updated from global requirements
Change-Id: I8fdb22ea4dff5e321e257688617d3283809342fa
2017-11-13 10:19:29 +00:00
OpenStack Proposal Bot 54c049bd6f Updated from global requirements
Change-Id: I9c8b5a889564a5f1a5ffe44c8c94b5945df7d9bb
2017-11-03 13:41:32 +00:00