Commit Graph

364 Commits

Author SHA1 Message Date
Brian Rosmaita f380a5137a Update README
Even though cinderlib development is stopping, the cinder project is
still around, so add the #openstack-cinder IRC channel as another
source of info for people interested in the status of cinderlib.

Change-Id: Id3cfc4ca100f0968f1898ae9c91776cbf5d20121
2024-01-18 12:41:39 -05:00
Brian Rosmaita f165c6ff5e Stop cinderlib development
Remove files from master, as development will no longer occur there.
The stable branches continue to be supported while they are in
Maintained status.

Updated the README to indicate this change.

Depends-on: Ib186ac5830e5920e264d79be946995e63e960426
Depends-on: I081cd363117671eaab6a3193094d5872f9820354
Depends-on: If2b9a82cddb20543b176ee22765049db257c89b9
Depends-on: I1143e5e5ccf8103e386fe1ce614a554e7f152d9a
Change-Id: I4722b869033ad1bd357e36c4a258b6d3ea61f5d6
2024-01-05 16:03:46 +00:00
OpenStack Release Bot ee1d86c058 Update master for stable/2023.2
Add file to the reno documentation build to show release notes for
stable/2023.2.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2023.2.

Sem-Ver: feature
Change-Id: I90a9f25c43f9dfae97db72e73cb2f751ea32dbaf
2023-11-30 09:13:33 +00:00
Brian Rosmaita ab28330d85 Continue 2023.2 (Bobcat) development
As a cycle-with-intermediary release with trailing deliverable type,
cinderlib's master remains the 2023.2 development branch until early
in the 2024.1 (Caracal) cycle when the cinderlib 2023.2 (Bobcat)
release is made.  So update tox.ini and .zuul.yaml to test against
cinder and os-brick stable/2023.2 instead of their master branches,
and restrict cinder and os-brick in requirements.txt to the 2023.2
series.

See the cinderlib docs for more info about this change:
https://docs.openstack.org/cinderlib/latest/contributor/contributing.html#cinderlib-tox-and-zuul-configuration-maintenance

Change-Id: I830b8e2f9766746a7f81b24ce1b7c1f4d4c7c5a1
2023-10-09 15:33:15 +00:00
Stephen Finucane f1af46dc6a persistence: Handle switch to alembic
Change-Id: I02e3fdd7cb16ee420f47eb08ecce983f50d32697
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-09-29 11:18:49 +00:00
Brian Rosmaita 98fc4b6c68 Add sqlalchemy-migrate dependency
It was removed as a cinder dependency by change I39448af0eb8f
but we haven't caught up yet, and are still using it.  Add it
to our requirements as a temporary measure.

Change-Id: I135f0a91ae00b03942a3bf62fea4789e30c06f84
2023-09-27 09:54:42 -04:00
Brian Rosmaita 6bb58dd596 Use openstack-tox-py311
This addresses a TODO in .zuul.yaml.  The openstack-tox-py311 job
was added by change I725b94bee6601a.

The job runs on debian-bookworm, but we don't need to modify
cinderlib's bindep.txt because we don't include any packages not
recognized by debian.  Cinder does, however, so this patch depends
on the cinder patch that updates cinder's bindep.txt.

Depends-on: https://review.opendev.org/c/openstack/cinder/+/891645
Change-Id: Iff9d4079655c8c4b15ad54d6936a81d024aa2c74
2023-08-31 09:02:02 -04:00
Zuul ae4388ed94 Merge "Open cinderlib for 2023.2 (Bobcat) development" 2023-08-23 16:56:04 +00:00
Brian Rosmaita fb6178197f Add tooz to test-requirements
setUpClass (cinderlib.tests.functional.test_basic.BackendFunctBasic)
is failing with "ModuleNotFoundError: No module named 'etcd3gw'"
from file site-packages/tooz/drivers/etcd3gw.py

Address this by adding tooz with the 'etcd3gw' extra to cinderlib's
test-requirements.txt, using the same minimum currently used by
cinder.

example failures on cinder CI jobs:
cinder-plugin-ceph-tempest on https://review.opendev.org/785099/5
cinder-plugin-ceph-tempest on https://review.opendev.org/868485/9

Change-Id: I341c6be5329487d13a1bdd383b4157105baee368
2023-08-15 16:44:01 -04:00
Brian Rosmaita 496ef8dfdb Open cinderlib for 2023.2 (Bobcat) development
See "cinderlib tox and zuul configuration maintenance" in
https://docs.openstack.org/cinderlib/latest/contributor/contributing.html
for an explanation of these changes.

Notes:
- checked, but can't address the TODO in .zuul.yaml yet
- no changes needed for setup.cfg
- py39 testing wasn't required for Antelope, but it is for Bobcat

Change-Id: I4b68ec8594fca14445333143b2852ee122aa6bd6
2023-07-21 09:46:51 -04:00
OpenStack Release Bot b66c34fb55 Update master for stable/2023.1
Add file to the reno documentation build to show release notes for
stable/2023.1.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2023.1.

Sem-Ver: feature
Change-Id: I4b98ac85b5d5dbb5c4d0eb38909eb2dc53824f65
2023-05-04 09:32:24 +00:00
Brian Rosmaita 18c81f0023 Continue 2023.1 (Antelope) development
As a cycle-with-intermediary release with trailing deliverable type,
cinderlib's master remains the 2023.1 development branch until early
in the 2023.2 (Bobcat) cycle when the cinderlib 2023.1 (Antelope)
release is made.  So update tox.ini and .zuul.yaml to test against
cinder and os-brick stable/2023.1 instead of their master branches,
and restrict cinder and os-brick in requirements.txt to the 2023.1
series.

Change-Id: Id74b4d0e2bad8fa7a3629d85458b089e00a60211
2023-03-30 10:45:16 +00:00
Brian Rosmaita d3c48523cc Don't use deprecated config opt name in unit test
With cinder change Iccfbe62406b6, the 'iscsi_secondary_ip_addresses'
opt was deprecated in favor of 'target_secondary_ip_addresses'.  This
broke a cinderlib unit test that checked to see if a mocked logger
was not called (which it now was, to log a deprecation message).  Use
the current option name in the test so that the test continues to
test what it was designed to test (i.e., cinderlib config loading,
not oslo.config option deprecation).

Change-Id: Ia243368a9d68aca3a717fb2be1da99bceb271e56
2023-03-30 06:34:56 -04:00
Zuul 9c37686f35 Merge "Remove unnecessary #egg= URL fragment" 2023-01-21 01:23:16 +00:00
Zuul 6c18ec3d66 Merge "remove unicode prefix from code" 2023-01-20 22:33:39 +00:00
Gorka Eguileor a123ff5f82 Fix DBMS storage plugin
In previous patches (I469b4b9dd9966be8477f3cefbd6284bdaf17ca17 and
I4981ad558195895341c91fa37803713c8c0ab684) we assumed that unit tests
were failing because they needed to be updated, but that's only half of
the issue.  The other half is that the DBMS plugin needs to be updated
as well.

This misunderstanding lead to a broken DBMS system that unit tests don't
detect.

This patch fixes the DBMS system and reverts changes to unit tests that
those 2 patches introduced and are not necessary as well as removed the
FIXME comments that are now fixed.

Here's a brief description of why some of the issues were happening:

- Locked database issues were happening when a unit test from
  TestBasePersistence run before a DBMS unit test. It was caused by the
  replacement of the main_context_manager when it had not been used.  No
  longer a problem since we no longer replace it and we only replace its
  transaction factory when it has been used.

- "ValueError: Backend named fake_backend already exists with a
  different configuration" error was caused because in TestDBPersistence
  the tearDown method only called the parent's tearDown after the
  cleanup of the DB, so if the cleanup of the DB failed then the global
  existing backend would remain and a following test would have trouble
  with it.  Calling it first now to fix this.

- Splitting the running of unit tests in tox.ini was necessary because
  methods in Cinder DB API were always using the context manager
  transaction factory from the first DBMS Test Case class.  So some DBMS
  plugin methods would end up writing data to the previous database
  while others would write to the new database.  Now that we replace the
  transaction factory everything is going to the same DB.

Change-Id: I74ce945291697bc1fcc63a35d60ef71494fdaf19
2023-01-19 17:31:13 +01:00
Gorka Eguileor 436bc6d74b Fix tox.ini
CI fails to run on Tox v4 complaining about spaces in pass_env and not
being able to run special_install.sh.

Those are easy to fix and this patch fixes them in tox.ini:

- Using multiple lines for the different environmental variable values
  in pass_env.

- Allowing running tools/special_install.sh and tools/generate_uc.sh as
  an external commands.

There is also an issue with the cinderlib entry points not being
available during unit tests.  This issues is not reproducible locally
even using the same tox and python version as the gate, so the patch
limits the tox version to v3 and prevents jobs from using v4 to unblock
the gate.

The patch also updates the values of the constraints branch from
antelope to 2023.1 since antelope doesn't exist and without this change
the patch cannot merge.

Change-Id: I14ea8821aeccc43b5ddcc83862d59b1bbf4094be
2023-01-19 17:29:55 +01:00
Zuul 2d1246d41b Merge "Open cinderlib for 2023.1 (Antelope) development" 2022-12-20 19:02:38 +00:00
Brian Rosmaita 642fbea5e4 Remove unnecessary #egg= URL fragment
There's discussion in the pip project about removing support for
the '#egg=name' in a URL requirement specifier [0].  Since it's
redundant in our current usage in cinderlib (since the URL base
component and the package name are the same for os-brick and
cinder), just remove it and then whatever they decide to do about
this in upstream pip won't affect us.

[0] https://discuss.python.org/t/killing-off-the-egg-fragment-once-and-for-all/21660

Change-Id: If4ce5cbac388eea99ecfc5d4baee15397a61d6e5
2022-12-16 10:09:26 -05:00
Brian Rosmaita 0c0879a681 Open cinderlib for 2023.1 (Antelope) development
See "cinderlib tox and zuul configuration maintenance" in
https://docs.openstack.org/cinderlib/latest/contributor/contributing.html
for an explanation of these changes.

Change-Id: I68d038a3a64a81e6bd21e65423898d9f5b514234
2022-12-16 10:06:08 -05:00
OpenStack Release Bot 09dc1f7acf Update master for stable/zed
Add file to the reno documentation build to show release notes for
stable/zed.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/zed.

Sem-Ver: feature
Change-Id: I5de46c7549117c3912faadfc26848a353e946ef3
2022-12-16 08:50:54 +00:00
Brian Rosmaita 3356e71cb9 Add Zed release note
Change-Id: I9ddf1195d180520267ab7140d24996508f029f56
2022-12-14 08:06:31 -05:00
Gorka Eguileor 5c4349cffe Support driver's setup method
As part of the effort to simplify and properly define the driver
interface, Cinder consolidated methods do_setup and
check_for_setup_error in a single method called setup.

This patch provides support for both mechanisms in cinderlib.

Change-Id: Id037c1b8a570eea704c136f35ac0b262ca5b0dfa
2022-12-14 11:50:44 +00:00
Takashi Kajinami 94bd3fffc0 Remove six
This library no longer supports Python 2, thus usage of six can be
removed. This also removes workarounds for Python 2.

Change-Id: Ia3f56a2a984cc1d82b717041583a727d9cb5642c
2022-12-14 05:40:11 +00:00
Brian Rosmaita c722aefb67 Don't limit use of importlib-metadata to Python < 3.8
Change I5febaed02e95 to global-requirements lifted the cap
on importlib-metadata, so we should too.

The cinderlib restriction to py<3.8 was originally made by
change I1709952cf0d0  (merged 28 July 2020).

Change-Id: Ib0397640e524a3d932e71d4c776eca19234c057a
2022-12-14 04:44:10 +00:00
Brian Rosmaita 6d482ef697 Fix py310 unit tests
Tests are failing because importlib_resources is limited to <3.10
in requirements.txt, which makes sense because the files module is
available in the standard library importlib.resources for python
3.10 and later.  The code, however, currently tries to import
importlib_resources when running in python 3.10.  Fix the code
so that it uses the standard library module for >=3.10.

Change-Id: Ie7dff06a4599f96e1983bd04fc10dbfd37dbe5a0
2022-12-14 04:27:08 +00:00
Brian Rosmaita def0c7faa2 Open cinderlib for zed development
NOTE: don't use this patch as an example of "open cinderlib for
xxx development" because it's happening *after* cinder and os-brick
stable/zed have been cut (normally, it would happen before).

In this patch we:
- Switch tox.ini to using stable/zed instead of stable/yoga for
  requirements, cinder, and os-brick
- update .zuul.yaml to use jobs specified by the
  openstack-python3-zed-template
- add a cinderlib-os-brick-src-tempest-lvm-lio-barbican-zed job to
  .zuul.yaml
- update tox.ini to support the current supported python runtimes
  for unit and functional tests plus python 3.10
- update requirements.txt to use only Zed release cinder and os-brick
- update setup.cfg to reflect the Zed supported python runtimes

Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>
Co-authored-by: Takashi Kajinami <tkajinam@redhat.com>

Change-Id: I6482088acfaf4c6de436d1a12195e96b3334e906
2022-12-14 04:26:23 +00:00
Rajat Dhasmana 728b933bf2 Fix cinderlib tests
The persistence tests were broken when run with zed cinder.

Co-authored-by: Rajat Dhasmana <rajatdhasmana@gmail.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>

Change-Id: I4981ad558195895341c91fa37803713c8c0ab684
Closes-bug: #1996738
2022-12-14 04:25:30 +00:00
Brian Rosmaita b1db48c6ed Fix ValueError when running with zed cinder
When using the stable/zed cinder branch and zed upper-constraints,
we're getting a bunch of "ValueError: Backend named fake_backend
already exists with a different configuration" in unit tests.  The
backend name is set in the test setUp(), but it looks like isolation
isn't being respected, maybe because the Backend instantiation uses
a singleton pattern.  (Which doesn't, however, explain why this was
working previously.)  Address this by giving each test backend a
unique name.

Change-Id: I469b4b9dd9966be8477f3cefbd6284bdaf17ca17
2022-12-09 13:56:49 +00:00
Takashi Natsume e4da0146a5 Fix wrong assertion methods in unit tests
Replace 'called_once_with' with 'assert_called_once_with'.
Replace 'asssert_called_once_with' with 'assert_called_once_with'.

Change-Id: Ie5f07de8c09cf4764223278acf1724c20c8d7694
Partial-Bug: 1989280
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
2022-11-14 13:04:28 +00:00
tushargite96 2715218e2b Update metadata in setup.cfg
maintaining setup.cfg as per other repos[1]

[1] https://review.opendev.org/c/openstack/cinder-tempest-plugin/+/828475

Change-Id: I2f2d0502c26ce91a88237aa4cc4a50df036cf803
2022-09-16 09:52:40 +00:00
niuke af3a22e306 remove unicode prefix from code
Change-Id: Iea2b4f8ce22b7592242d140a722ac234d84a7f7d
2022-08-23 19:06:45 +08:00
Zuul 47914d29c3 Merge "[docs] add zuul.yaml maintenance tasks" 2022-07-22 13:05:15 +00:00
Zuul a8c11021ca Merge "Update deprecated zuul syntax" 2022-07-19 15:28:36 +00:00
Zuul 0f52ade42e Merge "Don't use yum to install ceph" 2022-07-19 15:00:52 +00:00
Brian Rosmaita 9c09e93937 Update deprecated zuul syntax
Replace declaring shared queues at the project-pipeline level
with declaring them at the project level so that cinderlib CI
won't break at opendev's next Zuul major update.

Details:
https://lists.zuul-ci.org/pipermail/zuul-discuss/2022-May/001801.html

Change-Id: Ie0b14c0c64d1b233a37aeae61618efdc2d6e262e
2022-07-18 18:08:40 +00:00
Brian Rosmaita 30af4ccedf Don't use yum to install ceph
Use the ansible builtin 'package' module instead.

Change-Id: I244c1896a3c17b43496bef09ee4c8ae76522f565
2022-07-18 11:31:48 -04:00
Zuul a59051690c Merge "Update doc/requirements.txt" 2022-07-15 18:24:43 +00:00
Brian Rosmaita 7dfef254c0 [docs] add zuul.yaml maintenance tasks
Change-Id: I20f5c64eaa2de3116efb12c4046b6ee63888a0f3
2022-07-11 10:06:49 -04:00
OpenStack Release Bot 6815d80445 Update master for stable/yoga
Add file to the reno documentation build to show release notes for
stable/yoga.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/yoga.

Sem-Ver: feature
Change-Id: I3d2f8c1981aa67881dd308e5043b179525b743f6
2022-06-27 08:52:36 +00:00
Brian Rosmaita 7be24a5578 [docs] add tox, zuul maintenance tasks
Explain what configuration changes need to be made during
the trailing phase of cinderlib development, when cinderlib
master is still the development branch of the previous release
cycle.

Change-Id: I8c56b1ee15dfdb71d82ea856729b3ebe65029bbe
2022-06-25 09:45:38 +00:00
Brian Rosmaita e4dd75a3b4 Add local upper constraints support
The issue is that we want to install os-brick from source, but if we
do this using upper-constraints on the install, it will fail.  This
patch modifies the tox install command used by appropriate testenvs
so it will create and use a local constraints file with os-brick
removed.

The local u-c file is named 'local-upper-constraints.txt'.  The
constraints file used can be overridden via the
CINDERLIB_CONSTRAINTS_FILE environment variable.

Change-Id: I8cb4085dd465043a783c974886f56e49820871a1
2022-06-25 06:25:13 +00:00
Gorka Eguileor 4d784d23a9 Fix privsep issues inside virtual env
When a virtual environment is created with the "--system-site-packages"
option and privsep is installed on the system privsep will only use the
system packages and completely ignore the ones in the virtual
environment.

This results in errors such as the ones we see:

- In the Ussuri gate: ModuleNotFoundError: No module named
  'os_brick.privileged.rootwrap'

- In the Wallaby gate: ModuleNotFoundError: No module named
  'os_brick.privileged.nvmeof'

This happens because os-brick and cinder are starting privsep using the
"privsep-helper" mechanism, and privsep was not installed in the virtual
env because it was already present system wide, so the "privsep-helper"
that is executed is the one from "/usr/local/bin/privsep-helper".

This python script "privsep-helper" ignores the virtual environment and
forces usage of the system's python, for example in a Wallaby
installation this could be "#!/usr/bin/python3.6".

Since it ignores the virtual environment it won't use its packages and
anything that's not present on system wide will not be found, and if
found it may be executing different code.

This patch fixes this issue by replacing the helper used to start
privsep with our own command.

This command is the same as the one usually installed in /usr/local/bin
but using /usr/bin/env to select the python to use.

This new script has been included as data in the cinderlib namespace
instead of making it install as a system script (like the original
privsep command) because we don't want to polute the system wide
binaries directory just for a corner case.

We also need to preserve user site-packages for the running Python when
calling root from the virtual environment, since the packages installed
on the virtual environment with "--system-site-packages" would have
taken those into consideration during the installation and not the ones
present on the root user.

To help debug issues at the gate all functional tests are now running
with debug logs.

Change-Id: I0278b42785d14f92a521e6deff872dcba6505270
Related-Bug: #1958159
Closes-Bug: #1979534
2022-06-24 15:26:48 +02:00
Zuul 31c93f2f07 Merge "Use stable/yoga override for CI jobs (conclusion)" 2022-06-22 08:22:38 +00:00
Brian Rosmaita eec3b54628 Use stable/yoga override for CI jobs (conclusion)
Make the change from I82e2bbf7e0380e2bf for the
os-brick-src-tempest-lvm-lio-barbican job, which is the only
remaining unchanged job (other than docs and releasenotes).

Change-Id: Iac7cb2296712ee230e5d53374d3b2ce71a2bf1a1
2022-06-16 09:29:13 -04:00
Gorka Eguileor 30c9c0f280 Update Ceph version on CI job
Job cinderlib-ceph-functional is using Ceph Luminous, which has been EOL
since March 2020.

This patch updates the job to use an active release, Ceph Pacific.

In the new container they no longer automatically create the rbd pool,
so the patch also updates the playbook to create the pool.

Change-Id: I09a2a9b07ed8b6d994485079febd0bd0190aa6c9
2022-06-15 18:38:14 +02:00
Brian Rosmaita 84c6b102b8 Use stable/yoga override for CI jobs
cinderlib master is still Yoga development, but when zuul runs on a
master patch, zuul.branch is 'master', and this cascades throughout
the various jobs.  It's particularly evident for requirements, but
also affects cinder and os-brick (which are installed from source);
from what I can tell, an initial install is done to gather
dependencies, and this seems to be from master by default.  So
explicitly use override-checkout for requirements, cinder, and
os-brick on all jobs.

Why are we only noticing this now?  I think it hasn't been a problem
in the past because python 3.6 has been supported from Rocky through
Yoga, and is dropped from Zed, and hence running the cinderlib Yoga
development branch against master for other projects is encountering
some serious incompatibilities.  We probably should have been using
the stable branch in cinderlib's pre-release trailing phase all
along; it just hasn't mattered until now.

Change-Id: I82e2bbf7e0380e2bffbb44d394e9751e1578afcf
2022-06-08 19:10:13 -04:00
Brian Rosmaita 94eaa5e39b Update doc/requirements.txt
Requirements check job will fail if we don't have a min for
sphinxcontrib-svg2pdfconverter.  Setting it to >=0.1.0 based
on a codesearch of openstack (that's what everyone else is
using).

Change-Id: Iaff8ad0df30dda8b439dbce832680130b91fbc1a
2022-06-08 08:53:22 -04:00
Brian Rosmaita 82b8c25c97 Continue yoga development
As a cycle-with-intermediary release with trailing deliverable type,
cinderlib's master remains the yoga development branch until early in
the zed cycle when the cinderlib yoga release is made.  So update
tox.ini to test against cinder and os-brick stable/yoga instead of
their master branches, and restrict cinder and os-brick in
requirements.txt to yoga versions.

Change-Id: I8da1ddc38607eca378a503a961bf55a03566c5bc
2022-03-30 11:32:37 -04:00
Zuul eea377e683 Merge "setup.cfg: Replace dashes by underscores" 2022-02-18 17:12:51 +00:00