Change method to check supported REST version on the backend.
This will mkae it easier to specify minimum REST versions for
specific feature support.
Closes-Bug: #1929219
Change-Id: Ibab932d3536d44dd7414560768ed5bdeaa9e61e1
Update the minimum os-brick version to include a fix addressing
a potential data loss issue.
Change-Id: If28cf471aaeef8207d530dd914d0f63a19bb5941
Related-Bug: #1921381
Update min brick version to the wallaby brick release, and adjust
requirements files and lower constraints to accommodate the versions
of dependencies requested by os-brick 4.3.0.
Change-Id: I74b897a6ec27ca2ef00811fbbc2e6c43d31e307f
The SMBFS driver stores volumes as VHD/x image files. This change
will set the unique id VHD field using the volume id.
Hyper-V propagates this id to the guest through the SCSI VPD,
which allows it to safely identify the disk. The kubernetes openstack
provider will also benefit from this change.
Here's an example: http://paste.openstack.org/raw/801011/
This operation requires os-win 5.4.0, for which reason we're bumping
the requirement.
Depends-On: Ied73997e6f5f3ded9827703867f059ef3dfca159
Change-Id: I459ff3c8798e218919b6b2f9e1c2200203efeeff
Now that we're starting to set scope_types on default policies, we
should make sure we handle InvalidScope exceptions from oslo.policy in
the event enforce_scope=True. Operators won't use this switch for a
while, but it prepares us for when cinder will be system-scope aware.
This commit also bumps the minimum version of oslo.policy to 3.6.2,
which is safer when running tests in parallel with different
policy configurations.
Change-Id: I680cb8c4be13bcd3ac6785a7afa81ce5d3477f91
Update requirements to use the wallaby intermediate brick release
containing the updated nvmeof connector and other bugfixes.
Depends-on: https://review.opendev.org/c/openstack/requirements/+/775792
Change-Id: Id0128baa36f995e4dd74090bc30dd2dbbd2a21e9
The driver requires the new rbd-iscsi-client package, which is used
to talk to the rbd-target-api on the ceph iscsi gateway node.
The rbd-target-api is a python script meant to keep ceph iscsi gw nodes in
sync with each other, but the API is works for creating iscsi targets.
This is a new driver that makes heavy use of the ceph-iscsi project's
rbd-target-api python REST client here:
https://github.com/ceph/ceph-iscsi
The driver is a derivation of the rbd driver, and the intention is to reuse
as much of the base rbd driver as possible and just do iSCSI specific
code here.
Change-Id: Iff0e4d1137851c8f0b8ec25632d1186c2859b2fc
As per the community goal of migrating the policy file
format from JSON to YAML[1], we need to do two things:
1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.
2. Deprecate the JSON formatted policy file on project side
via warning in doc and format releasenotes.
1st item if already done by cinder in Stein so this commit
only cover the 2nd item.
Since oslo.policy 3.6.0, by default oslo policy will
fallback to existing policy.json file to give operator some
time to migrate it to new default policy.yaml. But cinder
already changed the default value to policy.yaml long back
since Stein so no need to fallback to default JSON file. To
do that it disable this fallback via flag to oslo.policy.
Also convert the tests/unit/policy.json to policy.yaml file.
Additionally, made some corrections to outdated documentation
when removing references to a "policy.json" file.
[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Co-Authored-By: Brian Rosmaita <rosmaita.fossdev@gmail.com>
Change-Id: Iaf8a454e60d1e4b66981b61175f89203cc15e439
Bring the versions specified in requirements.txt and test-req.txt
closer to what's actually being used in the py36 and py38 testenvs,
and update lower-constraints to reflect the updated requirements.
What has made this necessary right now is that the latest pip has
introduced a much stricter resolver, and the cinder lower-constraints
job is failing because cinder is imposing some constraints which are
too low for some dependencies, and we can't get away with that under
the stricter resolver.
Change-Id: I42af21b1c4247d04d479f1fc1ecd6f9baac0cfc9
md5 is not an approved algorithm in FIPS mode, and trying to
instantiate a hashlib.md5() will fail when the system is running in
FIPS mode.
md5 is allowed when in a non-security context. There is a plan to
add a keyword parameter (usedforsecurity) to hashlib.md5() to annotate
whether or not the instance is being used in a security context.
In the case where it is not, the instantiation of md5 will be allowed.
See https://bugs.python.org/issue9216 for more details.
Some downstream python versions already support this parameter. To
support these versions, a new encapsulation of md5() has been added to
oslo_utils. See https://review.opendev.org/#/c/750031/
This patch is to replace the instances of hashlib.md5() with this new
encapsulation, adding an annotation indicating whether the usage is
a security context or not.
Reviewers need to pay particular attention as to whether the keyword
parameter (usedforsecurity) is set correctly. Almost all instances
of md5 usage appear to be to refer to etags, to do checksums, or to
generate uuids for paths.
I had hoped to update the bandit config to enable scanning for instances
of md5 and bad algorithms, so that instances would not creep in in future,
but I couldn't find the bandit config.
With this patch (and the corresponding os-brick and oslo-versioned_object
dependent changes) all the functional tests and alnmost all the unit tests
pass on a FIPS enabled system.
Issues I found were as follows:
- Cinder appears to be using md5 in a security context in
cinder/volume/drivers/synology/synology_common.py. If this is really
the case, then we'll need to consider how to replace md5 in this usage.
This case did not appear to exercised in the unit or functional tests I ran.
- Cinder appears to use md5 in a security context in
cinder/volume/drivers/stx/client.py, which resulted in the failed unit test
cinder.tests.unit.volume.drivers.test_seagate.TestSeagateClient.test_login
This was the only unit test that failed.
Change-Id: I57ec3e7e99c78535fa8051d011d970adb7fb89ab
Depends-On: https://review.opendev.org/#/c/756151
Add a "mypy" tox environment which runs mypy
type checking against Cinder code.
Taken from Stephen Finucane's Nova work at
https://review.opendev.org/#/c/676208/
Added "show_error_codes" and "pretty" options.
Generates an html report in ./mypy-report/
This adds stubs for oslo.i18n, so that _() calls
are annotated as intended. It may be possible to
do this with less .pyi files carried along here.
Change-Id: I2589d22c1f16f2e177d34730a520591743c0c1e3
As per victoria cycle testing runtime and community goal,
we need to migrate upstream CI/CD to Ubuntu Focal(20.04).
Bump lower constraints to make testing work for Focal.
Co-Author: tushargite96 <tushargite96@gmail.com>
Story: #2007865
Task: #40179
Change-Id: I5f37fb5611362e550610e2094d9cb3778548bf47
Require victoria os-brick for the upcoming victoria cinder release.
Contains a fix related to OSSN-0086, among other things.
Change-Id: I81b49aaa5dd0a157b1c29244b9923186c2c2f610
Several version specified in our lower-constraints files had conflicting
dependencies. This updates a few packages to avoid those conflicts. It
also removes the linters that are tracked in the global requirements
blocklist since we do not need to enforce lower-constraints for linters.
Change-Id: Iaa2b3e1518614caf8664af017b2a2e1a7c005b07
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Importing pkg_resources scans all of the installed modules for data
that won't be used. Switch to using importlib.metdata, which more
efficiently loads the metadata for a package.
Since the name of the module where importlib.metadata is found depends
on the version of python, mocking a function in the library is more
complicated. Provide a wrapper in the module that uses
importlib.metadata.version() so its tests can examine behavior using
different versions via mocks.
The distutils package in the standard library is deprecated. Use the
packaging library for parsing version strings into something that can
be compared.
Change-Id: I45d0851cdb5f241ff8dc774dc22123b410502cd9
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
VxFlex OS password is not stored in block_device_mapping table. Instead of this
passwords are stored in separate file and are retrieved during each attach/detach
operation.
Closes-Bug: #1823200
Change-Id: Ia7cc17472677a693c6162f0b8b0529df62eed7cf
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* parallelizing building of documents
Update Sphinx version as well.
Remove docs requirements from lower-constraints, they are not needed
during install or test but only for docs building.
openstackdocstheme renames some variables, so follow the renames. A
couple of variables are also not needed anymore, remove them.
Set openstackdocs_pdf_link to link to PDF file.
Set openstackdocs_auto_name to use project as name.
Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.
See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html
Change-Id: I3544c1f95dd3ce510c00bfeda4f3ced87a3cd60c
Python 3.6 is now our minimum Python runtime version to support.
This raises several lower constraints versions to the minimum that
supports 3.6 or later.
This also raises greenlets and lxml versions to account for updates to
cpython changes that break installation when installing under 3.7.
Depends-on: https://review.opendev.org/725412
Change-Id: I34f7346a183e662f862a79df54a28d608a93c6c8
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
flake8 new release 3.8.0 added new checks and gate pep8
job start failing. hacking 3.0.1 fix the pinning of flake8 to
avoid bringing in a new version with new checks.
Though it is fixed in latest hacking but 2.0 and 3.0 has cap for
flake8 as <4.0.0 which mean flake8 new version 3.9.0 can also
break the pep8 job if new check are added.
To avoid similar gate break in future, we need to bump the hacking min
version.
- http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014828.html
Change-Id: I0a0504f9ab35cf35fb02d4edf2564f23a278c5a2
The retrying library is no longer maintained and users are encouraged to
migrate to tenacity.
This has a small behavior change in that before we were applying an
exponential backoff to the first time a retry was needed. This no longer
happens, but retries will exponentially back off with subsequent each
retry.
Change-Id: I25ca007386a7b8ca6a7d572742334ba3d7dcbf1e
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This repo is now testing only with Python 3, so let's make
a few cleanups:
- Remove python 2.7 stanza from setup.py
- Add requires on python >= 3.6 to setup.cfg so that pypi and pip
know about the requirement
- Remove obsolete sections from setup.cfg
- Update classifiers
- Use newer openstackdocstheme, Sphinx, os-api-ref versions,
remove unused mock requirement
Change-Id: If7db350ea55a5edf7055ce049d27d9412b0e40bc
This makes several types of changes to get rid of all of the deprecation
warnings that are spewed during our unit test runs.
Context changes:
The user_domain and project_domain properties have been deprecated
and user_domain_id and project_domain_id should be used instead. We
have used the newer correct properties for some time now, but for
compatibility we were still setting the old properties as well.
Collections.abc:
Between py2 and py3, some abstract classes were moved from
collections to collections.abc. We had some compatibility code to
try to handle this in most cases, but also had a few cases of using
the classed directly from the old, deprecated location.
assertRaisesRegexp:
This was deprecated in favor of assertRaisesRegex.
DefusedXML:
DefusedXML has deprecated several things now that lxml handles
things correctly. The project now recommends just using lxml.
Change-Id: Ie2e46b45248d6a06edbe6cffc2eb8e2d341cd4c3
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
The suds-jurko requirement was needed by the Disco and Datacore drivers,
but of which have since been removed. This cleans up the requirements to
remove that library.
Change-Id: I141c2c141aeea84272d2b04610e53822bc3b3371
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
New attempt to update Datera Cinder in-tree driver. This review request
builds on https://review.opendev.org/#/c/661359/ with fixed zuul gating
tests.
Full list of changes since last update:
* Added Pypi packaging installation support
* Dropping support for v2 API. No Datera products exist in production
with this API version.
* Added v2.2 API support
* Rewrite of the driver to use the Datera Python-SDK instead of
hand-rolled connections. Usage requires the dfs_sdk python package
* Dropping support for default_storage_name and default_volume_name
volume-type keys
* Added CHAP support
* Implemented fast-path Glance-->Datera image cloning with clone_image RPC
* Implemented fast-path volume retype
* Rewrote unit tests from scratch
* Added iops_per_gb and bandwidth_per_gb volume-type keys
* Implemented update_migrated_volume
* Increased number of stats reported with get_volume_stats
* API fallback now only occurs during driver initialization. This
increases driver performance
* Added config option for customizing volume-type default values
* Implemented template size override
* Implemented LDAP support
* Added support for filter_functions and goodness_functions
* Changed version string to date-based
* Implemented manage_existing_snapshot and related RPCs
* Removed ancient py25 compatibility imports
* Updated Copyright to 2020
* Fixed almost all requests From walter and Sean
* Added comprehensive version history
Change-Id: I56a1a24d60a7bc0dc59bfcfa89da23f43696a31e
Version 2.35.0 contains a fix needed for the "Support multiple stores
of Glance" feature.
Change-Id: I64937ba8abdb0b0c5b6c1aab6ed08d67bac13590
Closes-bug: #1857094
When a volume record is soft-deleted in the database,
dependent records in other tables (for example,
Transfers, VolumeGlanceMetadata, etc.) must be soft
deleted as well. Otherwise, we will get FK dependency
errors when the database is purged.
This patch adds that support for VolumeAttachment table.
(other tables were already covered, just refactored)
Also adds tests.
Co-authored-by: Rajat Dhasmana <rajatdhasmana@gmail.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>
Change-Id: Ibfa6c4ba2f162681756ec3203991351345b65346
Related-Bug: #1542169
Yandong Xuan