Based on the Libvirt and QEMU versions, two traits,
COMPUTE_ADDRESS_SPACE_PASSTHROUGH and COMPUTE_ADDRESS_SPACE_EMULATED,
are controlled. Since the two are supported from the same version on
the Libvirt and QEMU, Nova handles them in the same way.
Blueprint: libvirt-maxphysaddr-support
Depends-On: https://review.opendev.org/c/openstack/os-traits/+/871226
Signed-off-by: Nobuhiro MIKI <nmiki@yahoo-corp.jp>
Change-Id: If6c7169b7b8f43ad15a8992831824fb546e85aab
This allows us to drop some logic from tests.
Change-Id: Idc11c1ce6e208774314a5a867789958372f0d591
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
sqlalchemy-migrate does not (and will not) support sqlalchemy 2.0. We
need to drop these migrations to ensure we can upgrade our sqlalchemy
version.
Change-Id: I7756e393b78296fb8dbf3ca69c759d75b816376d
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
As discussed in PTG, we need to test the new RBAC in the
integrated gate and accordingly enable the new defaults
and scope check by default. A new integrated testing job
has been added and results show that the new defaults and
scope checks are working fine. During testing, we found a
few bugs in neutron policies but all are fixed now.
enforce_scope and enforce_new_defaults are oslo policy config
options but they are per service level and the default value
can be overridden. Oslo policy 3.11.0 version allows to override
the default value for these config options[1] so upgrading the
oslo policy version in requirements.txt
Depends-On: https://review.opendev.org/c/openstack/devstack/+/869781
Depends-On: https://review.opendev.org/c/openstack/placement/+/869525
[1] https://github.com/openstack/oslo.policy/blob/3.11.0/oslo_policy/opts.py#L125
Change-Id: I977b2daedf880229c8d364ca011f2ea965b86e3a
Without the latest version, nova fails many unit tests (it failed
with os-traits 2.7.0 at least).
Closes-Bug: #1990121
Change-Id: I6b320ae1f9058aaa5bac91c7c7ca60136e0cee5c
The oslo.concurrency 5.0.1 fixes the fair lock we use heavily in the
ResourceTracker.
Closes-Bug: #1988311
Change-Id: I68914b2e21726138ee9a178fdf6a8bb6389c09be
The PowerVM driver was deprecated in November 2021 as part of change
Icdef0a03c3c6f56b08ec9685c6958d6917bc88cb. As noted there, all
indications suggest that this driver is no longer maintained and may be
abandonware. It's been some time and there's still no activity here so
it's time to abandon this for real.
This isn't as tied into the codebase as the old XenAPI driver was, so
removal is mostly a case of deleting large swathes of code. Lovely.
Change-Id: Ibf4f36136f2c65adad64f75d665c00cf2de4b400
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We no longer support Python 3.6 so there's no need for Python 3.6-only
dependencies.
Change-Id: I139e454a2c6e49a1d5229c90a56d518b0d96ca57
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The current rootwrap.conf file is outdated and doesn't include some
parameters. This change updates the content to make it consistent with
the latest example file in oslo.rootwrap.
Change-Id: I989b7da5ff6afae0c1a476e01c3df07870be5f74
Local API and DB limits are limits on resources that are counted either
as an API request parameter (example: server metadata items) or as
records in the database (example: server key pairs).
Future patches will make use of this logic, and actually enforce the
limits. This patch just adds the infrastructure to allow for the
enforcement of the limits.
We are moving all existing quotas to be managed via Keystone's
unified limits.
To stop confusion between injected_file_path_length and
injected_file_path_bytes, the unified limit in Keystone will use the
latter name to match the name used the API.
These local limits are all about preventing excessive load on the API
and database and have little to do with resource usage. These limits
are represented by keystone registered limits only, accordingly.
Local limits include things that just limit items in an API request:
* metadata_items
* injected_files
* injected_file_content_bytes
* injected_file_path_bytes
Local limits also include things that are stored in the database:
* key_pairs
* server_groups
* server_group_members
Some resource names have been changed to prepend a prefix of "server_"
in order to disambiguate them from other potential unified limits in
keystone:
* metadata_items => server_metadata_items
* injected_files => server_injected_files
* injected_file_content_bytes => server_injected_file_content_bytes
* injected_file_path_bytes => server_injected_file_path_bytes
* key_pairs => server_key_pairs
Note that each of the above are counted via a different scope. This new
code ensures that key_pairs are counted per user, server_groups are
counted per project and server_group_members are counted per
server_group.
Note: Previously server_group_member were counted per user inside each
server_group, which has proved very confusing, as adding more users into
a project increases the maximum size of allowed for a server_group.
blueprint unified-limits-nova
Change-Id: I0b6f4d29aaee1d71541a95cbecfd0708aac325d2
This commit introduces the LightOS driver for nova. LightOS is a
software-defined disaggregated clustered storage solution running on
commodity servers with commodity SSDs. It it developed by Lightbits
Labs (https://www.lightbitslabs.com) and is actively developed and
maintained. LightOS is proprietary but the openstack drivers are
licensed under Apache v2.0.
The Cinder driver for LightOS currently supports the following
functionality:
Create volume
Delete volume
Attach volume
Detach volume
Create image from volume
create volume from image
Live migration
Volume replication
Thin provisioning
Multi-attach
Extend volume
Create snapshot
Delete snapshot
Create volume from snapshot
Create volume from volume (clone)
This driver has been developed and has been in use for a couple of
years by Lightbits and our clients. We have tested it extensively
internally with multiple openstack versions, including Queens, Rocky,
Stein, and Train. We have also tested it with master (19.1 xena) and we
are working to extend testing to cover additional openstack releases.
We are glad to join the openstack community and hope to get your
feedback and comments on this driver, and if it is acceptable, to see
it merged into the tree.
Note: the patch depends on os-brick 5.2.0. That version also increased
the lower constraints of several dependencies, thus needs nova to
increase those as well in requirements.txt, lower-constraints.txt and
setup.cfg.
Depends-On: I2e86fa84049053b7c75421d33ad1a1af459ef4e0
Signed-off-by: Yuval Brave yuval@lightbitslabs.com
Change-Id: Ic314b26695d9681d31a18adcec0794c2ff41fe71
The new version contains changes needed by the multi-architecture
support and off-path SmartNIC DPU support code.
Needed-By: I168d3ccc914f25a3d4255c9b319ee6b91a2f66e2
Needed-By: Ia070a29186c6123cf51e1b17373c2dc69676ae7c
Change-Id: Ic1179f3e5e2c1aeb069972f21edffe5b003eb525
...and start skipping the relevant tests in 'nova.tests.unit.virt' if
these dependencies are not present. Thanks to hacking rules N311 and
N312, which prevent virt drivers using code from other virt drivers,
simply skipping the unit tests for a virt driver module is enough to
ensure we never load those modules.
This means users that want to use the powervm driver can install the
required dependencies using e.g. 'pip install .[powervm]', and packagers
can choose to skip packaging a dependency safe in the knowledge that the
relevant tests will be skipped.
Change-Id: I3787f9afd78cd0a7c7feb4dfe1bcb21437b5a128
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We no longer have a Xen driver. This is an unnecessary dependency.
Change-Id: Ic298fa9ac4a8935ce4e0dc17d8842d399d4eb808
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The commit I168fffac8002f274a905cfd53ac4f6c9abe18803 added a wrapper
around fasteners.ReaderWriterLock to fix up an issue with eventlet. But
the wrapper was added to nova.utils module that is use not only by the
nova tests but also the nova production code. This made the fixture
library a dependency of the nova production code. While the current
ReaderWriterLock usage only limited to the nova test sub tree. The
I712f88fc1b6053fe6d1f13e708f3bd8874452a8f commit fix the issue of not
having fixtures in the nova requirements.txt. However I think a better
fix is to move the wrapper to the test subtree instead. This patch does
that and restores the state of the requirements.txt
Change-Id: I6903ce53b9b91325f7268cf2ebd02e4488579560
Related-Bug: #1958075
The commit 887c445a7a made the nova.utils
module dependent on the fixtures library but the change missed updating
requirements and the fixtures library is not installed automatically.
This change migrates the fixtures library from test-requirements.txt to
requirements.txt so that the library is installed without test codes.
Closes-Bug: #1958075
Change-Id: I712f88fc1b6053fe6d1f13e708f3bd8874452a8f
Some things that were missed in previous patches and are thrown together
here:
- Add alembic as an explicit dependency (we were getting it transitively
from oslo.db). We also bump the sqlalchemy dependency to a 1.4.x
release, which is the minimum supported by our chosen version of
alembic (more on this below)
- Remove tooling related to the old migrations
- Fix the tox whitelisting of the flaky MySQL tests
On the SQLAlchemy front, we opt for 1.4.13. Technically alembic should
support anything from 1.4.0, however, with SQLAlchemy >= 1.4.0, < 1.4.13
we see errors like the following in some tests:
sqlalchemy.exc.InvalidRequestError: Entity namespace for
"count(instance_mappings.id)" has no property "queued_for_delete"
There's nothing specific about this in the release notes for 1.4.13 [1]
but it definitely fixes things.
[1] https://docs.sqlalchemy.org/en/14/changelog/changelog_14.html#change-1.4.13
Change-Id: I4c8eb13f11aa7471c26a5ba326319aef245c9836
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This makes it possible to partially revert
If48dd949ec4d69a09c87178f16d56a2517e21fd8 as one of the warning silenced
there now fixed in oslo.db.
Change-Id: I0051c0f8694699865bc16636953908d60ba456d2
Take advantage of the neutronclient bindings for the port binding APIs
added in neutronclient 7.1.0 to avoid having to vendor this stuff
ourselves.
Change-Id: Icc284203fb53658abe304f24a62705217f90b22b
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Since 3.7.0, oslo policy started the DeprecationWarning[1] if
deprecated_reason and deprecated_since param are not passed
in DeprecatedRule or they are passed in RuleDefault object.
[1] https://github.com/openstack/oslo.policy/blob/3.7.0/oslo_policy/policy.py#L1538
Change-Id: Idbbc203c6ae65aee29f9463a4911bae2bb541f41
Use the new oslo.confg type HostDomainOpt to support underscore in the
name. You can see the bugzilla[1] to have more information.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1868940
Related-Bug: #1892044
Change-Id: Ib6c8fc1e3d90b79b10066c429670fcb957bddd23
os-vif 1.15.0 added the ability to create an OVS port during plugging
by specifying the 'create_port' attribute in the 'port_profile' field.
By delegating port creation to os-vif, we can rely on it's 'isolate_vif'
config option [1] that will temporarily configure the VLAN to 4095
(0xfff), which is reserved for implementation use [2] and is used by
neutron to as a dead VLAN [3]. By doing this, we ensure VIFs are plugged
securely, preventing guests from accessing other tenants' networks
before the neutron OVS agent can wire up the port.
This change requires a little dance as part of the live migration flow.
Since we can't be certain the destination host has a version of os-vif
that supports this feature, we need to use a sentinel to indicate when
it does. Typically we would do so with a field in
'LibvirtLiveMigrateData', such as the 'src_supports_numa_live_migration'
and 'dst_supports_numa_live_migration' fields used to indicate support
for NUMA-aware live migration. However, doing this prevents us
backporting this important fix since o.vo changes are not backportable.
Instead, we (somewhat evilly) rely on the free-form nature of the
'VIFMigrateData.profile_json' string field, which stores JSON blobs and
is included in 'LibvirtLiveMigrateData' via the 'vifs' attribute, to
transport this sentinel. This is a hack but is necessary to work around
the lack of a free-form "capabilities" style dict that would allow us do
backportable fixes to live migration features.
Note that this change has the knock on effect of modifying the XML
generated for OVS ports: when hybrid plug is false will now be of type
'ethernet' rather than 'bridge' as before. This explains the larger than
expected test damage but should not affect users.
[1] https://opendev.org/openstack/os-vif/src/tag/2.4.0/vif_plug_ovs/ovs.py#L90-L93
[2] https://en.wikipedia.org/wiki/IEEE_802.1Q#Frame_format
[3] https://answers.launchpad.net/neutron/+question/231806
Change-Id: I11fb5d3ada7f27b39c183157ea73c8b72b4e672e
Depends-On: Id12486b3127ab4ac8ad9ef2b3641da1b79a25a50
Closes-Bug: #1734320
Closes-Bug: #1815989
The os-brick 4.3.1 release fixes a critical data loss bug that can be
triggered by reconfiguring [libvirt]/volume_use_multipath while volumes
are attached.
The following requirements and lower-requirements are also bumped as
introduced into os-brick by Ie7dcc859e1291bd1d50b1ae516be38d8732de836:
os-brick 4.3.1 depends on eventlet>=0.30.1
os-brick 4.3.1 depends on oslo.concurrency>=4.4.0
os-brick 4.3.1 depends on oslo.service>=2.5.0
os-brick 4.3.1 depends on oslo.serialization>=4.1.0
os-brick 4.3.1 depends on oslo.utils>=4.8.0
os-brick 4.3.1 depends on pbr>=5.5.1
os-brick 4.3.1 depends on requests>=2.25.1
os-brick 4.3.1 depends on tenacity>=6.3.1
Co-Authored-By: Lee Yarwood <lyarwood@redhat.com>
Related-Bug: #1921381
Change-Id: I0d5df271dec1c6035229b02ec45e03abad6f27e7
Ceph 16 (Pacific) will support attaching RBD images to Windows
hosts as well as Hyper-V VMs [1].
This patch updates the Hyper-V driver so that it may be able to
consume RBD volumes.
We're also updating the os-win requirement to 5.4.0. It should've
been bumped by [2], vms won't spawn without it. Note this introduces no
new or unmet direct or in-direct version dependencies and so is included
as part of this change.
[1] https://github.com/ceph/ceph/pull/33750
[2] Ibe6aff4edeb32208bc9865e9216a7432caddab2b
Implements: blueprint hyperv-rbd
Change-Id: I9ad90817648ca12f80a6b53f6ba728df15cbafab
This change simply bumps the version of os-brick required by Nova to
version 4.2.0. This is itself required by
I9ad90817648ca12f80a6b53f6ba728df15cbafab that introduces support for
rbd volumes within the HyperV driver.
After much back and fourth it was decided to bump the required version
of os-brick ahead of that change to allow the following requirements.txt
and lower-constraints.txt changes caused by direct and in-direct
dependency changes to be documented clearly:
os-brick 4.2.0 depends on oslo.log>=4.4.0
\_ oslo-log 4.4.0 depends on python-dateutil>=2.7.0
os-brick 4.2.0 depends on oslo.serialization>=4.0.1
os-brick 4.2.0 depends on pbr>=5.5.0
os-brick 4.2.0 depends on oslo.privsep>=2.4.0
\_ oslo.privsep 2.4.0 depends on msgpack>=0.6.0
os-brick 4.2.0 depends on oslo.service>=2.4.0
os-brick 4.2.0 depends on tenacity>=6.2.0
os-brick 4.2.0 depends on oslo.context>=3.1.1
os-brick 4.2.0 depends on oslo.concurrency>=4.3.0
os-brick 4.2.0 depends on oslo.i18n>=5.0.1
os-brick 4.2.0 depends on six>=1.15.0
os-brick 4.2.0 depends on os-win>=5.1.0
The above changes have been tested with pip 21.0.1 to ensure the new
resolver is happy and that nothing has been missed.
Change-Id: Ic83f3c7c955d0df89d75f700ee4fe2bd7f715794
Implements: blueprint hyperv-rbd
Since only Wallaby compute nodes will support the 'socket' PCI NUMA
affinity policy, this patch adds a ResourceRequest translator that adds
a required trait if the value of '(hw_|hw:)pci_numa_affinity_policy' is
'socket'.
The actual trait reporting by the libvirt driver will be added in a
future patch. Until then the 'socket' value remains a hidden no-op.
Implements: blueprint pci-socket-affinity
Change-Id: I908ff07e1107304ca5926cc04d2fdc8fe0da5ed9
This is now maintained as a Jazzband project [1].
[1] https://github.com/jazzband/prettytable
Change-Id: Ie422a243a97ffbe2f553934004369f01562b95b9
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Replace six.text_type with str.
This patch completes six removal.
Change-Id: I779bd1446dc1f070fa5100ccccda7881fa508d79
Implements: blueprint six-removal
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
The dnspython 2.0.0 incompatibility[1] is worked around in eventlet 0.26.1
by restricting the dnspython version to < 2.0.0. So the issue affects
nova-novncproxy as well we need to bump our lower constraint as well.
[1] https://github.com/eventlet/eventlet/issues/619
Related-Bug: #1888237
Change-Id: Ie9cc718e654cb37a4f31ccb4b052b23878d476a2
md5 is not an approved algorithm in FIPS mode, and trying to
instantiate a hashlib.md5() will fail when the system is running in
FIPS mode.
md5 is allowed when in a non-security context. There is a plan to
add a keyword parameter (usedforsecurity) to hashlib.md5() to annotate
whether or not the instance is being used in a security context.
In the case where it is not, the instantiation of md5 will be allowed.
See https://bugs.python.org/issue9216 for more details.
Some downstream python versions already support this parameter. To
support these versions, a new encapsulation of md5() has been added to
oslo_utils. See https://review.opendev.org/#/c/750031/
This patch is to replace the instances of hashlib.md5() with this new
encapsulation, adding an annotation indicating whether the usage is
a security context or not.
The instances being replaced here appear to be used to provide
representations for paths. There is in fact already a sha256 version
of get_hash_str that is supposed to be used in security sensitive
usages.
With this change (and the related dependent changes), the unit and
functional tests pass when run on a FIPS enabled system.
Change-Id: If0ec11e7b7fcde4dacc57265c4dd77b0f536bfab
Depends-On: https://review.opendev.org/#/c/756432
Depends-On: https://review.opendev.org/#/c/756153
Depends-On: https://review.opendev.org/#/c/760160
Default value of 'CONF.oslo_policy.policy_file' config option
has been changed from 'policy.json' to 'policy.yaml'. If new default
file 'policy.yaml' does not exist but old default 'policy.json' exist
then fallback to use old default file.
An upgrade checks is added to check the policy_file format and
fail upgrade checks if it is JSON formatted.
Added a warning in policy doc about JSON formatted file is deprecated,
also removed all the reference to policy.json file in doc as well as
in tests.
Related Blueprint: policy-json-to-yaml
Closes-Bug: #1875418
Change-Id: Ic4d3b998bb9701cb1e3ef12d9bb6f4d91cc19c18
Several version specified in our lower-constraints files had conflicting
dependencies. This updates a few packages to avoid those conflicts. It
also removes the linters that are tracked in the global requirements
blocklist since we do not need to enforce lower-constraints for linters.
Change-Id: Iae93f0631d77794f093fd330e849d2fd805b238f
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This series implements the referenced blueprint to allow for specifying
custom resource provider traits and inventories via yaml config files.
This first commit includes schema version 1.0 and functions to load and
validate against that schema.
Since this patch, package ddt>=1.2.1 is required to run tests.
Co-Author: Tony Su <tao.su@intel.com>
Author: Dustin Cowles <dustin.cowles@intel.com>
Blueprint: provider-config-file
Change-Id: I58099726f799c427e9174a0dcce9889344d51e7c
To support new AVX512 cpu feature, this patch adds the mapping
between cpu features reported by libvirt [1] and corresponding
os-traits [2]. The traits include 'AVX512VBMI', 'AVX512IFMA',
'AVX512VBMI2', 'AVX512BITALG', 'AVX512VAES', 'AVX512GFNI',
'AVX512VPCLMULQDQ' and 'AVX512VPOPCNTDQ'.
[1] https://github.com/libvirt/libvirt/blob/master/src/cpu_map/x86_features.xml
[2] https://review.opendev.org/#/c/728576/
Depends-On: I58cba02f204d2e19a1a838852c628694edd47d73
Change-Id: I212a84e5cae15bc133aafd538d55f85149ab360f
Zuul