This is the first in a series of commits to add support for codespell. This is continuning the process completed in ironic-python-agent.
Future Commits will add a Tox Target, CI support and potentially a git-blame-ignore-revs file if their are lots of spelling mistakes that could clutter git blame.
Change-Id: Icb4e66359c3e79618c916b35793ea7b5f79b030d
Flask 2.3 removed this method, with recommended way of running
startup actions explicitly as part of app initialization.
Change-Id: I5dea1320c9aa63c1ef1742382cb46ea7cc56b10b
What hit us was a regression in 2.2.0. The hack we landed only fixes one
URL (e.g. /v1/rules/ is still broken) and leaves some redundant code in place
around path handling. Werkzeug 2.2.1 fixes our problems.
Reverts commit 97f4e98d0b.
Story: #2010193
Task: #45904
Depends-On: https://review.opendev.org/c/openstack/requirements/+/851500
Change-Id: Ice7e9499fbb2585d353ece7c5fa30e425e92d362
Werkzeug 2.2.0 included a major rewrite[0] of the rule matching logic
which was to improve performance. Unfortunately it necessitates a
few minor changes to our logic.
This is sort of similar to the sushy-tools[1] change, except in this
case ironic-inspector utilizes it's own internal decorator to register
URLs. In this case, we needed to make it a little smarter to handle
the possible version path interaction.
[0]: https://github.com/pallets/werkzeug/pull/2433/files
[1]: https://review.opendev.org/c/openstack/sushy-tools/+/851162
Change-Id: Ia0f7ec9b4ce01967c06b64dc29e25c2e43e8a8b9
Story: 2010190
Task: 45892
It turns out that eventlet has been injecting a
``Transfer-Encoding`` header as of recent into WSGI application
response headers. The result of this ultimately depends on how
the HTTP client which is passing the request to the server is
written to handle data.
Apache, for example, will return that an invalid response was
received. In part because it sees the request end, with an HTTP
204 response code, but also an encoding indicating there is
a multipart body encoding inbound. Which is confusing.
Other C based HTTP clients can have any number of reactions up to
and including disconnecting sessions. Curl, depending on the
headers present either returns success but notes body weirdness
or actually returns return code 18.
Python-Requests kind of has it a little worse, and we see this
with clients. With it, it tries to prepare a respones content
body based upon the presence of the header indicating there is
a body. But it blows up thinking there is more data to read on
the socket when there is not more data to read.
Regardless, all of this is an RFC7230 violation.
Neither Content-Length nor Transfer-Encoding should be on an HTTP
204 response. However, Content-Length is the lesser evil, and we
have a similar endpoing in Ironic which *does* explicitly get
returned with a zero length content-length, and does not
demonstrate such issues.
As such, in the interest of the lesser evils until Eventlet's evil
ways of header injection are remedied, we're explicitly going to
force a Content-Length header to be sent indicating a zero length
response.
For more information, please see: https://github.com/eventlet/eventlet/issues/746
Change-Id: I014cc65c79222f4d4d7c2b6ff11a76e56659340c
This commit add support for state selector to the list introspection.
* ``GET /v1/introspection?state=[starting, waiting, processing,
finished, error, reapplying,
enrolling]``
Story: 1625183
Task: 11350
Change-Id: I2c5222110487a08a4e7b1efbcbc5dc3d552fae3e
This change allows uses to enable the healthcheck middleware from
oslo.middleware in API pipeline, by setting the [healthcheck]/enabled
option. This middleware provides an API endpoint at /healthcheck path
which allows load balancers or monitoring applications to determine
a service is up using HTTP requests.
This change basically follows the same change merged in ironic[1] repo.
[1] 6f439414bdcef9fc02f844f475ec798d48d42558
Change-Id: Ic2ee2bca74ea2a5a0723ef54b10c531f77ea7b8d
According to the openstacksdk docs[1] the Node uuid is stored in the
id attribute. This change removes the get_node shim which adds the
uuid attribute, and replaces any calls to Node.uuid with Node.id. This
will stop the many log debug warnings about this attribute:
DEBUG openstack.resource [-] Attribute [uuid] not found in
[<openstack.resource._ComponentManager object at 0x7f417e43aa20>]:
'uuid'. __getattribute__ /usr/lib/python3.6/site-packages/openstack/resource.py:623
Calls to NodeInfo.uuid or db.Node.uuid remain unchanged.
Change-Id: Icd3de82877c6a53d32b4c9fd3e500d3cd9d7fb17
Story: 2008379
Task: 41300
SCRIPT_NAME should be used to correctly construct the returned links.
This patch also adds ProxyFix from werkzeug to allow using inspector
behind SSL terminating proxy while still keeping the links correct.
Change-Id: I4f40e9266a55d237f7aa68324b11f59b667cc940
When sending a literal empty response, Flask does not include a
ContentType in the response. While in many cases, we don't need
need a ContentType nor expect one on the API client, Apache
webserver can treat this as an error and generate an Error
indicating a Bad Gateway. When doing this, we also now include
an empty JSON body in the response for 202 messages. For 204
message errors, the message body is expected to be empty.
However, when this Bad Gateway error occurs, the API/Conductor
were proceeding like there was no issue. The API client on the
other hand thinks that a hard failure has occured.
Also adds some additional catches to provide additional logging
which turned out not to be needed in this case, but it would be
useful for others.
Change-Id: If2e7697e3fde58ab0a4193787e29d3acdca81ebf
When the config option ``auth_strategy`` is set to ``http_basic`` then
non-public API calls require a valid HTTP Basic authentication header to be
set. The config option ``http_basic_auth_user_file`` defaults to
``/etc/ironic-inspector/htpasswd`` and points to a file which supports the
Apache htpasswd syntax[1]. This file is read for every request, so no
service restart is required when changes are made.
The only password digest supported is bcrypt, and the ``bcrypt``
python library is used for password checks since it supports ``$2y$``
prefixed bcrypt passwords as generated by the Apache htpasswd utility.
To try basic authentication, the following can be done:
* Set ``/etc/ironic-inspector/inspector.conf`` ``DEFAULT`` ``auth_strategy``
to ``http_basic``
* Populate the htpasswd file with entries, for example:
``htpasswd -nbB myName myPassword >> /etc/ironic-inspector/htpasswd``
* Make basic authenticated HTTP requests, for example:
``curl --user myName:myPassword http://localhost:6385/v1/introspection``
[1] https://httpd.apache.org/docs/current/misc/password_encryptions.html
Change-Id: If50dfbfc18445ad9fe27e17cb0ee1b317ff25a0b
Depends-On: https://review.opendev.org/729070
Story: 2007656
Task: 39826
We've been historically using endpoints without trailing slashes in
our API. Apparently, some libraries (like gophercloud) are quite
opinionated about it (see the story), so let's handle both.
The implementation could be simpler if we just added trailing slash
to all routes, but it would cause redirects for current users.
Change-Id: Icbd971a8e792f93f9c3fa66ba29bec055dcdee32
Story: #2007660
Task: #39749
Added 'scope' property to IntrospectionRule and logic to check if a node
falls in the same scope.This allows introspection rules to be applied on
selected nodes instead of every one of them.
Story: 2006995
Task: 37763
Change-Id: I77034f032ea0ec16886afdd928546eb801f7a90a
Since we've dropped support for Python 2.7, it's time to look at
the bright future that Python 3.x will bring and stop forcing
compatibility with older versions.
This patch removes the six library from requirements, not
looking back.
Change-Id: Ic443c7e4d5a5a849c4dc220207f8957e4c90bf53
This patch splits API and conductor services for ironic-inspector.
Previous patch utilized lock from tooz coordinator, this patch adds
a coordinator wrapper for easier usage and further introduces group
interfaces.
Each conductor service will join a predefined group to mark it's
availability, on each request, API service will query members from
the group and randomly choose on of them, create desiginated topic
and deliver request to it.
The feature is tested with the memcached, file backend of tooz.
Other backends are not fully tested but may work as well, please
refer to tooz documentation for driver compatibilities[1].
[1] https://docs.openstack.org/tooz/latest/user/compatibility.html
Story: 2001842
Task: 30376
Change-Id: I419176cd6d44d74c066db275ef008fe8bb6ef37a
This patch adds support to provide unprocessed introspection data
to reapply a node. The provided introspection data will be save to
current introspection storage backend.
Change-Id: I969ae9c32f53f89c006a64a006388ddea9542aa5
Story: 1564863
Task: 11344
In https://review.openstack.org/#/c/637673, the get introspection
interface is narrowed down to only accept node uuid, which previously
accepts both uuid and name. But the name to uuid conversion is
missing in the reapply api, which causes feature regression, and
this is the fix :(
Story: 1726713
Task: 11373
Change-Id: I6912853deab77f1365f665ca1e52c13063d2cdf1
Configurable introspection data storage backend [1] is proposed
to provide flexible extension of introspection data storage
instead of the single support of Swift storage backend.
This patch adds plugin mechanism for loading introspection
storage, creates database backend and moves Swift storage
into a plugin.
[1] http://specs.openstack.org/openstack/ironic-inspector-specs/specs/configurable-introspection-data-backends.html
Story: 1726713
Task: 11373
Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com>
Change-Id: Ie4d09dc0afc441b20a1e5e3bd8e742b1df918954
Adds support to use latest as the microversion value. When set to
latest, the maximum version is assumed by ironic-inspector.
Story: 1672400
Task: 11363
Change-Id: I35be1034697a7d69ed30af9542d9711fb2f65bd0
Modify introspection rules to allow formatting to be applied to strings
nested in dicts and lists in the actions.
Change-Id: Ia53e0de98438f7789e9b9136dcd85c1b1274b713
Story: #1670768
Task: #11362
Adds a new node field "manage_boot" to store this value. When it is set
to False, neither boot device nor power state are touched for this node.
Instead, we expect a 3rd party to handle them.
We still manage the PXE filter because the node may need DHCP.
Change-Id: Id3585bd32138a069dfcfc0ab04ee4f5f10f0a5ea
Story: #1528920
Task: #11338
This fixes the warning/errors except for the C901 Function is
too complex in the ironic_inspector/rules.py's create method.
Bump's max-complexity to 18 in tox.ini to workaround the C901.
Story: 2001985
Task: 19604
Change-Id: I6c76a43353b1beb572dbde78dba4b4a839d45ea7
Adds oslo.messaging to ironic-inspector, and convert
inspect, abort and reapply to synchronized rpc calls.
This is the first step of API and worker seperation.
Change-Id: I15e86d7feb623b6b2889891b9700e5de6b3164cd
Story: #2001842
Task: # 12609
In Python 2.7, functools.wraps() does not provide the '__wrapped__'
attribute. This attribute is used by
oslo_utils.reflection.get_signature() when getting the signature of a
function. If a function is decorated without the '__wrapped__'
attribute then the signature will be of the decorator rather than the
underlying function.
From the six documentation for six.wraps():
This is exactly the functools.wraps() decorator, but it sets the
__wrapped__ attribute on what it decorates as functools.wraps()
does on Python versions after 3.2.
Change-Id: I11bf2fa945d36bfbc89ec8239e7c9259e3e12496
Consolidate all config options under ``conf`` directory.
New config modules should give a better picture of the configuration
options provided by the inspector.
Change-Id: I501ed0787ff4e1d91462f936e1a54de2c7abb35c
Related-Bug: #1561100
Co-Authored-By: Anton Arefiev <aarefiev@mirantis.com>
this patch introduces an oslo.policy-based API access policy
enforcement engine to ironic-inspector.
As part of implementation, a proper oslo.context-based request
context is also generated and assigned to each request.
Short overview of changes:
- added custom RequestContext class
- extends oslo.context to handle of "is_public_api" flag
(False by default)
- added context to request in each API route
- '/continue' api sets the "is_public_api" flag to True
- added documented definitions for API access policies and their
defaults
- added enforcement of these policies on API requests
- added oslo.policy-specific entry points to setup.cfg
- added autogenerated policy sample file with defaults
- added documentation with autogenerated policies
Change-Id: Iff6f98fa9950d78608f0a7c325d132c11a1383b3
Closes-Bug: #1719812
This experimental feature was deprecated in the Ocata release,
as it was found unstable, untested and dangerous.
API version is bumped to 1.12 to indicate this change to users.
Change-Id: I1aad6ddfd03946edc19ae510accd6c8daf5fc268
Closes-Bug: #1654318
Creates new WSGIService class which keeps base API sercice
initialization functionality and serve flask application. Also
it will configure application for wsgi container[1].
Also creates new `cmd` directory for storing console scripts.
[1] https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html
Related-Bug: #1525218
Change-Id: Ia64228c47a79a3008d435e8323a964f2bc45dfa7
When using the Inspector rules API to query existing introspection
rules, inspector does not return 'invert' or 'multiple' attributes of
conditions associated with the rules.
Change-Id: I08606cea676ecf57bbb3b73077c4832240fbe0d2
Closes-Bug: #1670372
This adds the node state when the GET /v1/introspection/<node uuid or
name> API is performed.
Change-Id: I81c6834933f789cb644a854313aacaf49a4856a7
Closes-Bug: #1665664
The i18n team has decided not to translate the logs because it seems
like it not very useful.
Change-Id: I46c1b0c3efa28c3f887b1a29dc77d47fe749be87
Closes-Bug: #1674374
Use the flake8 plugin flake8-import-order to check import ordering. It
can do it automatically and don't need reviewers to check it.
Change-Id: I9ced9c297273db0eec6ab3995b663b1e8dffe87d
Mostly removes old authentication options and support for [discoverd].
Also update example.conf to the latest version.
Change-Id: Ided8705c4345a1170c211d926d916cec2173ccb9
Db column started_at and finished_at are defined as float type, but
float time doesn't fit into default db Float size. This change migrates
columns type to DateTime.
Closes-Bug: 1658975
Change-Id: I13bbb3c9a3e6e8bc744473437a378453ebc81318
This feature is dangerous, barely maintained and not covered by any CI.
As it was hidden behind a configuration option, we can remove it without
breaking our API contract too much. This change deprecates the option,
and create an API version with this feature already de-activated.
Change-Id: I9e05c36b8c1194f4eeeb80c1f811e808854974c4
Partial-Bug: #1654318
This patch introduces an API endpoint to list introspection statuses. The
endpoint supports pagination with an uuid-marker and a limit query string
fields. Due to the pagination, this change introduces a new configuration
option: ``api_max_limit``.
APIImpact
Change-Id: I74d02698801d5290619161b2d8d7181ab51a0a5e
Partial-Bug: #1525238
Add new option "IRONIC_INSPECTOR_CLEAN_UP_PERIOD" to devstack
plugin, which allow to configure clean up time of node cache.
The periodics.periodic_task decorator application on
main.py:periodic_clean_up and main.py:periodic_update is postponed
until after the inspector.conf file is read.
Change-Id: Ic6563990c789b1d1c3b72088860a4d13a3d57f29
Closes-Bug: #1639286
Enhance the introspection status with the fields:
* uuid
* started_at
* finished_at
Change-Id: I36caa7d954a9bfb029d3f849fdf5e73f06f3da74
Partial-Bug: #1525238
cid