Adds a database retry decorator to capture and retry exceptions
rooted in SQLite locking. These locking errors are rooted in
the fact that essentially, we can only have one distinct writer
at a time. This writer becomes transaction oriented as well.
Unfortunately with our green threads and API surface, we run into
cases where we have background operations (mainly, periodic tasks...)
and API surface transacations which need to operate against the DB
as well. Because we can't say one task or another (realistically
speaking) can have exclusive control and access, then we run into
database locking errors.
So when we encounter a lock error, we retry.
Adds two additional configuration parameters to the database
configuration section, to allow this capability to be further
tuned, as file IO performance is *surely* a contributing factor
to our locking issues as we mostly see them with a loaded CI
system where other issues begin to crop up.
The new parameters are as follows:
* sqlite_retries, a boolean value allowing the retry logic
to be disabled. This can largely be ignored, but is available
as it was logical to include.
* sqlite_max_wait_for_retry, a integer value, default 30 seconds
as to how long to wait for retrying SQLite database operations
which are failing due to a "database is locked" error.
The retry logic uses the tenacity library, and performs an
expoential backoff. Setting the amount of time to a very large
number is not advisable, as such the default of 30 seconds was
deemed reasonable.
Change-Id: Ifeb92e9f23a94f2d96bb495fe63a71df9865fef3
FirmwareInterface base
New Config options [default]
- enabled_firmware_interfaces
- default_firmware_interface
New FirmwareInterface base with update method
Implementations of FirmwareInterface
- FakeFirmware (fake)
- NoFirmware (no-firmware)
New entrypoint ironic.hardware.interfaces.firmware
* fake and no-firmware
Api Controllers
- Updated: driver/node/utils/versions
- Created: firmware
Unit tests
api-ref for Node Firmware
Fake and Noop implementation for FirmwareInterface
Change-Id: Ib3b9cb22099819f97d5eab1e3f1b670cb91cbb25
Adds the following methods to DB API:
* create_firmware_component
* update_firmware_component
* get_firmware_component
* get_firmware_component_list
FirmwareComponent
* create | save | get
FirmwareComponentList
* get_by_node id | sync_firmware_components
Adds two exceptions:
* FirmwareComponentAlreadyExists
* FirmwareComponentNotFound
Tests for db and objects
Changes were required in models, the class name should match the
object name we will create
Story: 2010659
Task: 47977
Change-Id: Ie1e2a4150d4ee4521290737612780c02506f4a9e
Unused by Nova and unlike memory_mb/local_gb also by Ironic (actually,
our usage of local_gb is worth double-checking as well, but at the very
least it's referenced by inspection implementations).
Change-Id: Ie8b0d9f58f4dcd102c183c30ae7f5acf68a5e4c3
A node's interface can be temporarily overriden in instance_info.
However, some parts of the Ironic code still used a node's interface
attribute directly. This change adds a node get_interface method
and updates various parts of the Ironic code to use it.
Change-Id: Ifdaa21383f71b501bccb6cf8fe80e5b34661b6ae
* add fields to Node object
* expose them at endpoint `/v1/nodes/{node_ident}/states`
* update states on powersync / entering managed state.
* tests
* update api endpoint info in api-ref
Story: 2008567
Task: 41709
Change-Id: Iddd1421a6fa37d69da56658a2fefa5bc8cfd15e4
This change modifies the nodes _get_nodes_collection method to
consider and pass in an explicit lisst of requested fields into
the node list method, while also including the required fields
for things like ownership/policy checking.
And slightly modifies node_convert_with_links method to simplify
it while enabling field validity to be checked, and specific
requisite field lists provided in based upon that value.
And also optionally builds the traits list as they are nolonger
*always* populated on all objects with fully populated objects
as only partially hydrated objects are provided back when specific
fields are requested.
Story: 2008885
Task: 42572
Change-Id: Ided419263d84184cab902944b6c518f98618c9d2
Prior to this change, ironic would not pass the API consumer request
for a list of nodes down to the underlying layers with the list of
specific fields being requested.
This resulted in full objects being returned from the database where
in essence the entire database would be downloaded to construct the
objects, and in the case of joined views, whole roles would largely
be duplicated.
In order to be more efficent, we need to hand the user desired
fields down to the database, in order to return only that data,
and thus transform it.
In this case, we already have testing that handles the conversion
of objects at the lower layer, and in this case, the db object
conversion handler already understood fields, so we're just kind
of completing the awareness further downward for increased efficency.
This change only sets the stage, the final change to this is aligned
with API change to leverage this as the API is coded such that the
field does not include all of the required fields needed by the API
to render replies, which is fixed in the API patch to leverage this
with the API.
Story: 2008885
Task: 42495
Change-Id: I6283c4cc1b1ff608c4be24a6c41eb7b430a5ad68
A node object has many fields, and a huge opportunity
for performance improvement is to reduce the amount of work
performed at the lower layers when it is not necessary.
In particular, the easiest case to identify and handle is
when a list of fields to be fulfilled is explicitly supplied.
This is particularlly noteworthy when we collecting a list of
nodes for synconization with Nova, where only 9 columns are
really needed to supply Nova with the information required,
and thus numerous fields are discarded.
This is all particularlly compounded when traits are used,
which presently uses a joined load pattern from SQL.
With this patch, we explicitly load and convert only the fields
requested at the lowest level, and then perform a different style
of loading node traits which requires less overhead by SQLAlchemy
to de-duplicate the returned result set to the application.
This turns out to be far more efficent as we're reducing the amount
of data/object conversion work by 360%, even before we consider
populating traits, which is performed as a separate query when
traits are requested.
Story: 2008885
Task: 42434
Change-Id: Iac703d2a9b7b240a47477be8a64c7c33e28f692f
When the configdrive input is JSON (meta_data, etc), delay the rendering
until the ISO image is actually used. It has two benefits:
1) Avoid storing a large ISO image in instance_info,
2) Allow deploy steps to access the original user's input.
Fix configdrive masking to correctly mask dicts.
Story: #2008875
Task: #42419
Change-Id: I86d30bbb505b8c794bfa6412606f4516f8885aa9
The agent needs to use configdrive, and we do send it over the same
channel when running write_image. There is no point in preventing custom
deploy steps from accessing it.
Change-Id: I93d3966b2c6af1f60bfbb39b3a07056308c6866c
Adds the node's driver_internal_info to the
baremetal.node.provision_set.* notifications. The
driver_internal_info includes useful information
such as deploy and clean steps.
Change-Id: I98784f72e6f93cbc602753ef2da0dbef5ad2c9cf
Story: #2008054
It brings more harm than good, e.g. it breaks fast-track. Since
driver-specific fields are name-spaced, there should be no much
harm in keeping them around.
Change-Id: I397d23af64dfd56074cb563eedbe2d1ef8efff53
Python3 have a standard library for mock in the unittest module,
let's drop the mock requirement and switch tests to unittest mock.
Change-Id: I4f1b3e25c8adbc24cdda51c73da3b66967f7ef23
This change adds a `lessee` field to nodes, and exposes it
to policy. It also updates the non-admin node list API
to match for both owner and lessee; and updates the
allocation conductor to match owner allocations with nodes
with the appropriate lessee.
Change-Id: Ib31b49c7143ec8fd6cb486fc24038215b197c418
Story: 2006506
Task: 37930
In order to improve security of the lookup/heartbeat
endpoints, we need to generate and provide temporary tokens
to the initial callers, if supported, to facilitate the
verification of commands.
This is the first patch in an entire series which utimately
enables the endpoint communication to be better secured.
The idea behind this started in private story 2006634 which
is locked as a security related filing covering multiple
aspects of ironic/ironic-python-agent interaction centered
around miss-use and generally exposed endpoints. That story
will remain marked as a private bug because it has several
different items covered, some of which did not prove to be
actually exploitable, but spawned stories 2006777, 2006773,
2007025, and is ultimately similar to Story 1526748.
Operationally this is a minimally invasive security
enhancement to lay the foundation to harden interactions
with the agent. This will take place over a series of
patches to both Ironic and the Ironic-Python-Agent.
Also see "Security of /heartbeat and /lookup endpoints"
in http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010789.html
Story: 2007025
Task: 37818
Change-Id: I0118007cac3d6548e9d41c5e615a819150b6ef1a
This change adds support for node retirement: nodes can
have additional properties 'retired' and 'retired_reason'
which change the way the nodes (can) traverse the FSM
and which operations are allowed. In particular:
- retired nodes cannot move from manageable to available;
- upon instance deletion, retired nodes move to manageable
(rather than available).
Story: #2005425
Task: #38142
Change-Id: I8113a44c28f62bf83f8e213aeb6704f96055d52b
Pushing too long messages in the node last_error and maintenance reason can
cause node.save() failures, leaving the node in a transient state, with no
conductor actually handling it anymore.
Change-Id: Id4db377781f83cf4d97564ced9622d5a8a8c67af
Story: #2005377
Task: #30359
This patch implements the feature of storing informational free-form text
into ironic node, via the "description" field.
Operators can do simple queries on the context of description.
Change-Id: I787fb0df34566aff30dea4c4a3ba0e1ec820d044
Story: 2003089
Task: 23178
This change adds the database models and API, as well as RPC objects
for the allocation API. Also the node database API is extended with
query by power state and list of UUIDs.
There is one discrepancy from the initially approved spec: since we
do not have to separately update traits in an allocation, the planned
allocation_traits table was replaced by a simple field.
Change-Id: I6af132e2bfa6e4f7b93bd20f22a668790a22a30e
Story: #2004341
Task: #28367
Adds "owner" field on the node object and exposes it for updates
via the API.
Additionally, fixed a couple minor items related to the prior
where we missed updating version numbers in rebases.
Change-Id: Iaaf3db97d21de9b11236cf2d18ffcc3f73f6e50c
Story: #2001814
Task: #12550
When handling the "pet" case, some nodes may be critical for the deployment.
For example, in an OpenStack installer like TripleO you may want to make
sure your controllers are not removed by an incorrect operation.
This changes introduces a new field "protected" on nodes. When it is
set to True, the "deleted" and "rebuild" provisioning actions fail with
HTTP 403. Deleting such nodes is also not possible.
Also adds "protected_reason" for the operators to specify the reason
a node is protected.
Story: #2003869
Task: #26706
Change-Id: I1950bf6dd65b6596cae69d431ef288e578a89d6e
Create the object for automated clean and add the logic
in the conductor to be able to enable clean for specific
nodes, when general automated clean is disabled.
Story: #2002161
Task: #24579
Change-Id: If0130082e16d1205fdf65d083854ef9849754f8b
Adds the fields and bumps the objects versions. Excludes the field from
the node API for now.
Also adds the conductor_group config option, and populates the field in
the conductors table.
Also fixes a fundamentally broken test in ironic.tests.unit.db.test_api.
Change-Id: Ice2f90f7739b2927712ed45c969865136a216bd6
Story: 2001795
Task: 22640
Task: 22642
This patch also increments the object versions for all
objects that inherit the NodePayload object.
Change-Id: I9d986b5a40b185abd6fc694dc26395e8b5da7b72
This is a follow up to address comments about tests left at [1].
By default the test node created by db_utils.get_test_node() will
have a None fault value, so it is unnecessary to assign a None
before tests.
https://review.openstack.org/#/c/555708
Change-Id: Ib4786670a06d1257c71ec95d587effcaebc68157
Story: #1596107
Task: #10469
This adds a fault field to the node table of database, and necessary rpc
object change and version bumping.
Story: #1596107
Task: #10469
Change-Id: I5539aa0406dbfbde25bc9aa91d5c1e615875e50e
* Adds 'bios' interface to 'BaseDriver'
* Adds BIOSInterface driver class
* Adds fake & no-bios drivers and entries
* Implements it for 'fake-hardare' hardware type
* Adds configuration parameters:
+ [DEFAULT]/enabled_bios_interfaces
+ [DEFAULT]/default_bios_interface
* Adds 'bios_interface' field to Node object
* Handle 'bios_interface' field in _convert_to_version
* Adds bios in CLEANING_INTERFACE_PRIORITY
Drivers can implement this interface to do BIOS
configuration.
Co-Authored-By: Yolanda Robla Mota <yroblamo@redhat.com>
Co-Authored-By: Luong Anh Tuan <tuanla@vn.fujitsu.com>
Change-Id: I7e57130242b6cab21b54e35dc3c0b7819bdc43c0
Story: #1712032
The value returned by ironic.objects.IronicObject.as_dict() should be a
plain object, in order for it to be serialised to JSON. Currently,
nested object fields and object list fields are not converted to dict
format. This caused problems during cleaning, when the node object's
as_dict representation is JSON encoded and sent to IPA.
This change adds support for calling as_dict() on nested objects and
list objects, to ensure these are also returned in dict form.
We also change the method used in as_dict() for checking whether an
object has an attribute. The hasattr() function used previously has
problems when used with properties in python 2 [1], in that any
exceptions raised in the property getter result in hasattr() returning
False. Instead we use obj_attr_is_set() to determine whether the object
has a particular attribute.
[1] https://hynek.me/articles/hasattr/
Change-Id: Ib2166040508827db28d6f6e2d9a3e655c16f2993
Closes-Bug: #1750027
Currently the driver_info is passed as is to the ramdisk when calling
get_clean_steps or execute_clean_step. This may lead to their exposure,
as ironic<->ramdisk communication is currently not secure.
This change applies the same logic we use in the API to filter
the fields.
Change-Id: I4fd44786fea6c7092d2b0029cea6d680d31babde
Closes-Bug: #1744836
Adds a traits field to node notifications, and triggers notifications
when node traits are added or removed. Node traits are emitted in
notifications as a list of trait name strings.
Bumps the following notification payload versions:
NodePayload: 1.6
NodeSetPowerStatePayload: 1.6
NodeCorrectedPowerStatePayload: 1.6
NodeSetProvisionStatePayload: 1.6
NodeCRUDPayload: 1.4
Change-Id: I4e0333173250a641b317d466e52742cf7728ed90
Partial-Bug: #1722194
Adds a new traits object to expose traits DB operations to the API. It
also adds a new traits field into the node object, with the appropriate
version compatibility logic.
get_node_by_port_addresses is modified to ensure we correctly join in
the tags and traits in that DB call, this avoids a orphaned db object
lazy load style failure.
_set_from_db_object in the base object is modified such that the new
traits object doesn't have to include the dictionary style compatibility
mix-in.
Change-Id: I69403b9875a020fab7a7975810b57bf646417953
Partial-Bug: #1722194
Co-Authored-By: Mark Goddard <mark@stackhpc.com>
This commit adds `rescue` interface to `BaseDriver` and implements
it for `fake-hardware` hardware type. It adds configuration
parameters '[DEFAULT]/enabled_rescue_interfaces' and
'[DEFAULT]/default_rescue_interface'. The default value of
configuration parameter '[DEFAULT]/enabled_rescue_interfaces' is
`no-rescue`.
It adds new rescue states and a new 'rescue' field to the Node
object. It adds objects.node.Node._convert_to_version().
The method handles converting the new rescue_interface field
between different versions of the Node.
Partial-bug: #1526449
Co-Authored-By: Jay Faulkner <jay@jvf.cc>
Co-Authored-By: Josh Gachnang <josh@pcsforeducation.com>
Co-Authored-By: Jesse J. Cook <jesse.j.cook@member.fsf.org>
Co-Authored-By: Mario Villaplana <mario.villaplana@gmail.com>
Co-Authored-By: Aparna <aparnavtce@gmail.com>
Co-Authored-By: Shivanand Tendulker <stendulker@gmail.com>
Change-Id: I1534247bf207a20a7a58534988192aef392eaff2
Trivial rename to improve consistency within the unit tests. Typically
the base and utils modules imported by various tests are prefixed by
db_, obj_, etc. to differentiate them. This change adds a db_ prefix to
the ironic.tests.unit.db.base and ironic.tests.unit.db.utils imports
outside of ironic.tests.unit.db and submodules.
Change-Id: I39803c592e32e58ce9657ae70bcd45f7b5d51809
This extends versioned objects and uses the object's
convert_to_version() to perform conversions between different
versions of the object. This conversion is only done at the
"boundaries" of the api and conductor services. That is, when
reading/writing to the database and when serializing/deserializing
(for RPC). Internally, the services deal with the latest versions
of the objects.
Version column is introduced to make sure reading the DB object
versions happens transparently for the developer. An exception
is raised in case of a version compatibility error.
The version column is null at first and will be filled with the
appropriate versions by a data migration script in a following patch.
Change-Id: I34629a5cbab7ff3f246ea19c0cb766badc7061db
Partial-Bug: #1526283
Co-Authored-By: Ruby Loo <ruby.loo@intel.com>
After a node is saved to the database, we weren't updating the
Node object to reflect what was saved. This caused a problem
where the node's update_at field was incorrect. It was fixed
in 065326c0f5 by explicitly
setting node.update_at. However, that doesn't address other
node fields that may be out of sync.
The more correct fix would be to do a similar thing that (most
of) the other Objects do, which is for the node to update itself
via ._from_db_object().
Doing this revealed several incorrect tests and code in the conductor
and agent where changes to the node's dictionaries were incorrectly
being set and thus, not being saved. Those are fixed in this patch.
Change-Id: Ia84cd60c1a4eabcc1ad0a756124c338fa9f644c8
Closes-Bug: #1679297
Related-Bug: #1281638
Use the flake8 plugin flake8-import-order to check import ordering. It
can do it automatically and don't need reviewers to check it.
Change-Id: I821fd7467f6c5cc1487149297f26e4ad539cf25d
This patch adds notifications for create, update or delete
port groups. Event types are:
baremetal.portgroup.{create, update, delete}.{start,end,error}.
Developer documentation updated. "portgroup_uuid" field added to
port payload.
Closes-Bug: #1660292
Change-Id: I9a8ce6c34e9c704b1aeeb526babcb20a5b1261db
When we try to save() object after its refresh(), it leads to a
TypeError. Let's not keep track of changed fields in refresh(),
all fields are reloaded from database.
Closes-Bug: #1612666
Co-Authored-By: Vladyslav Drok <vdrok@mirantis.com>
Change-Id: I0f47552c424973af3704903a52f98db819b18c3f
This adds tests for node, chassis, and port Payload classes
to ensure that:
1. all the items in their SCHEMAs have corresponding fields within
the payload
2. all the items in their SCHEMAs have matching fields in the
resource object's class
This should help mitigate the potential for either the resource
objects or Payloads to change in a way that breaks notifications.
Change-Id: I32d7dc7e1a2f2929142cff2d00880670121f32f8
node.save() doesn't update object from db because 'updated_at'
field for the node is not up-to-date after a save().
Change-Id: I58076d71e5977dded3f5485f295b3235d467872d
Partial-Bug: #1281638
Co-Authored-By: Galyna Zholtkevych <gzholtkevych@mirantis.com>
This adds a test case for NodePayload to ensure that:
1. All the items in its SCHEMA have corresponding fields within the
payload
2. All the items in its SCHEMA have matching fields in the Node
class
This should help mitigate the potential for either the Node object or
NodePayload to change in a way that breaks node notifications.
Change-Id: I85eb45cbd2b01b539b16b89b4e580cbbda51cbda
This change moves the check on network interface to driver_factory.py
and moves node creation from API to conductor side. A new exception
InterfaceNotFoundInEntrypoint (similar to DriverNotFoundInEntrypoint)
was introduced for the case when invalid interface is requested either
during creation or during updating.
The current approach already duplicates the check in two places.
With the driver composition in place it will not be quite possible:
different conductors may have different interfaces enabled as long as
no hardware type ends up with different interfaces on different
conductors. Also when calculating the defaults, we'll need access
to a real hardware type object, which may not be possible on API side.
Also there is a demand for driver-side validations on creation
(e.g. detecting duplicating IPMI addresses), which also requires
creation to happen on conductor side. Such features, however, are
out of scope for this change.
A side effect of this change is that objects.Node.network_interface
is now defined the same way as other interfaces (i.e. nullable string).
Also added more clean ups to base unit test class, as otherwise newly
added tests randomly break other tests.
Change-Id: Id1da20ccd5bb50e61a82449ef3d2ffce91822d71
Partial-Bug: #1524745
This change adds a new policy setting, "show_instance_secrets", whose
behavior mirrors that of the existing "show_passwords" policy setting.
Whereas "show_passwords" has historically blocked all sensitive
information from the node's driver_info field, the new setting blocks
all sensitive information from the node's instance_info field, including
image_url.
The name of the old setting, "show_passwords", is not being changed at
this time because such a change is not backwards-compatible. Instead,
the documentation string for this setting has been changed to clarify
what it does. Note that the behavior has not actually changed.
Note that this change moves the policy.check("show_password") call from
the Pecan hook into the API's Nodes() class, where the
policy.check("show_instance_secrets") is also added. This makes the code
a little cleaner and more maintainable, especially if we want to add any
more checks like this in the future.
As a result of this cleanup, the ironic-specific
RequestContext.show_password property is removed.
Partial-bug: #1530972
Partial-bug: #1526752
Related-bug: #1613903
Change-Id: I48493c53971cdab3b9122897e51322e19ce2f600
In some part in the code we import objects.
In the Openstack style guidelines they recommend to import only
modules.
http://docs.openstack.org/developer/hacking/#imports
Change-Id: Ib3ded82524071c97f4e0508af7933ead63cc2dc5
Zuul