Commit Graph

1261 Commits

Author SHA1 Message Date
Zuul b8375913bf Merge "Replace remaining usage of [DEFAULT] sql_connection" 2024-04-19 05:27:57 +00:00
Takashi Kajinami b38a567ad4 Replace remaining usage of [DEFAULT] sql_connection
The previous attempt[1] was incomplete and these was still one usage
left. The deprecated alias was removed from oslo.db[2] and no longer
works.

[1] b6b9f043ff
[2] bf90aed0f7b19441644de2be807ec680ad86b1b1

Change-Id: Ib3b0271057578a64ae556a11224e3b366ddf89bf
2024-04-18 16:51:40 +00:00
Takashi Kajinami 835e7f36a5 Remove unused run_sql_cmd
The run_sql_cmd method in functional tests is not used. The method uses
Engine.execute() method which was removed in 2.0.

Change-Id: I6f9ea1dfd852a2d311e17964da0ea56c052b3fda
2024-04-15 11:54:16 +09:00
Abhishek Kekane 9b9111f819 Make `centralized_db` cache driver default
Made `centralized_db` cache driver as default driver so that we
can test it using tempest jobs in gate.

Implements blueprint centralized-cache-db
Depends-On: https://review.opendev.org/c/openstack/devstack/+/907110

Change-Id: Id94e93e3ba3fc207b39c7dbff92495805aa0f6f9
2024-03-05 19:35:20 +00:00
Zuul 8651aefa23 Merge "Introduce centralized database driver for image cache" 2024-02-27 21:57:34 +00:00
Zuul 7b40c33cb6 Merge "Add required database API(s) for cache" 2024-02-27 14:59:26 +00:00
Zuul 02be49c998 Merge "Add new tables for cache operations" 2024-02-19 20:35:01 +00:00
Abhishek Kekane 0bbaec2eeb Introduce centralized database driver for image cache
Added centralized_db driver for image cache which will use
centralized database for storing cache related information.

Related blueprint centralized-cache-db

Change-Id: Iafaaa86666176cc95f77d85a4ab77286f0042bdd
2024-02-16 16:38:06 +00:00
Zuul e31022bf9d Merge "Drop ineffective options from config file" 2024-02-15 15:17:20 +00:00
Abhishek Kekane 10d663d04d Add required database API(s) for cache
Added required database API methods for cache related
operations.

Related blueprint centralized-cache-db

Change-Id: Iae1f1927566f4bda74c5aeccf464df820cfae9d4
2024-02-15 14:35:05 +00:00
Abhishek Kekane b6bd03d4bc Add new tables for cache operations
Added expand script and model to define `cached_images` and
`node_reference` database tables.

Related blueprint centralized-cache-db

Change-Id: I9991a4e945bd4d231e942096445fd5374ff7ff2a
2024-02-09 08:40:46 +00:00
Takashi Kajinami 1e99767d0e Drop ineffective options from config file
The following options set in functional tests do not exist.
 [DEFAULT] send_identity_credentials
 [DEFAULT] send_identity_headers

Change-Id: Ia739af13e0f88d0861e453a4d54ecc626e32b34b
2024-01-31 01:16:12 +00:00
Takashi Kajinami 9f896ab03d Bump hacking
hacking 3.0.x is too old.

Change-Id: I33875c36dbbdb946841f8c583636ccdf88d3331f
2024-01-27 16:55:11 +00:00
Takashi Kajinami b6b9f043ff Replace usage of deprecated [DATABASE] sql_connection
This option was deprecated in oslo.db a long ago and was removed in
master.

Change-Id: Iba6aa00e4f86fc4b82fa354fddf2fc7d6b47c52f
2024-01-28 01:06:30 +09:00
Takashi Kajinami 5378396b84 Fix test failures with oslo.limit 2.3.0
The oslo.limit 2.3.0 release contains the validation to ensure
the [oslo_limit] endpoint_id option is set[1]. This fixes the test
failures caused by that validation by overriding the option in all
test cases where unified quota implementation is used.

[1] https://review.opendev.org/c/openstack/oslo.limit/+/897509

Closes-Bug: #2049064
Change-Id: I704393892799ee0d5edf45fabe07afe01473e38d
2024-01-12 00:20:42 +09:00
Danylo Vodopianov 66566c3286 Add Packed Virtqueue extra spec and image properties
Virtqueue feature was added to the metadata properties for flavor and
images

- Add the hw:virtio_packed_ring flavor extra spec and corresponding
  hw_virtio_packed_ring image property.

Nova spec: https://review.opendev.org/c/openstack/nova-specs/+/868377

Depends-On: https://review.opendev.org/c/openstack/nova/+/876075

Change-Id: Ia8caf39467a91431ca85b2a56ec93a60c41a8e7d
2023-12-05 10:59:18 +02:00
Zuul 88bb1568fc Merge "db: Remove explicit autocommit" 2023-09-21 22:35:10 +00:00
Zuul 7cb1d5e0ae Merge "db: Drop use of implicit auto-commit" 2023-09-14 13:31:46 +00:00
Zuul 269cc2f105 Merge "db: Enable foreign keys for SQLite backend" 2023-09-13 14:09:48 +00:00
Zuul 990621c4f5 Merge "db: Replace dict-style access of Row attributes" 2023-09-13 10:44:06 +00:00
Zuul fc3e21d01b Merge "db: Don't pass strings to Connection.execute()" 2023-09-13 09:34:00 +00:00
Zuul 6f5f24623c Merge "tests: Add tests for loading, unloading metadefs" 2023-08-29 15:20:18 +00:00
Abhishek Kekane fd222f3128 Sort locations based on store weight
Related to blueprint store-weight

Change-Id: I2383a476cb7e79c7efecdf33203cff0b50ef3bbb
2023-08-01 14:16:08 +00:00
Stephen Finucane d1368745c5 db: Remove explicit autocommit
We already got rid of implicit autocommit. Now get rid of explicit
autocommit. This is far more involved than the prior patch but the basic
strategy here is that no public method should take a 'session' argument
and instead should call 'get_session'. In addition, all private methods
should take both a 'context' and 'session' argument and they should be
the first and second arguments, respectively. Finally, public methods
should only call private methods. Combined, this ensures we only ever
have one session in play and the transactions are correctly started and
ended as expected.

Change-Id: I5245c402699e179c219039514b626a52fa0028d8
2023-07-25 10:08:52 +01:00
Stephen Finucane 8c5091cd48 db: Drop use of implicit auto-commit
Resolve the following RemovedIn20Warning warning:

  The current statement is being autocommitted using implicit
  autocommit, which will be removed in SQLAlchemy 2.0. Use the .begin()
  method of Engine or Connection in order to use an explicit transaction
  for DML and DDL statements.

This is the trickiest one hence it's left to last. We really need to get
off of the legacy EngineFacade pattern since it's harder to track
transactions when they're not local (the request context-based binding
of sessions allows us to decorate individual methods) but that's a
larger work item that we're not going to tackle here.

Change-Id: Ie16965a8e3e2f316b807068a868cee6fbb2f33ba
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-25 10:03:11 +01:00
Stephen Finucane 2e716b53f3 db: Enable foreign keys for SQLite backend
We were doing this adhoc in tests, which doesn't seem to be possible in
SQLAlchemy 2.0. Do it everywhere instead.

This highlights a small issue that we have to workaround in the tests.

Change-Id: I9dd70c9976f83a00a8cb270422e545ff2c77f12f
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-25 09:56:29 +01:00
Stephen Finucane ce5056ad3a db: Replace dict-style access of Row attributes
Resolve the following RemovedIn20Warning:

  Using non-integer/slice indices on Row is deprecated and will be
  removed in version 2.0; please use row._mapping[<key>], or the
  mappings() accessor on the Result object.

Change-Id: Ife8bdf766eb6d6fe9748565c9c61aa971d1af6f8
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-24 21:34:11 +00:00
Stephen Finucane 25dcacea7c db: Don't pass strings to Connection.execute()
Resolve the following RemovedIn20Warning warning:

  Passing a string to Connection.execute() is deprecated and will be
  removed in version 2.0.  Use the text() construct, or the
  Connection.exec_driver_sql() method to invoke a driver-level SQL
  string.

We also resolve two additional RemovedIn20Warning warnings along the
way, since we're already touching this code:

  The Engine.execute() method is considered legacy as of the 1.x series
  of SQLAlchemy and will be removed in 2.0. All statement execution in
  SQLAlchemy 2.0 is performed by the Connection.execute() method of
  Connection, or in the ORM by the Session.execute() method of Session.

  The Executable.execute() method is considered legacy as of the 1.x
  series of SQLAlchemy and will be removed in 2.0. All statement
  execution in SQLAlchemy 2.0 is performed by the Connection.execute()
  method of Connection, or in the ORM by the Session.execute() method of
  Session.

Change-Id: I547bd2f441085fcdc5dc4a0e58d27d11e8a8b223
2023-07-24 21:34:03 +00:00
Stephen Finucane 604ffd390c tests: Add tests for loading, unloading metadefs
These methods were not being tested. Add some basic tests to validate
loading and unloading of our stock definitions, preventing regressions
during e.g. the SQLAlchemy 2.0 preparation series.

Implementation of this test highlights an minor issue in the DB API,
whereby the default values for some methods with 'sort_key' and
'sort_dir' parameters are not valid. This means you can't call these
without overriding the parameters. This is corrected.

Change-Id: I6e76dcacc32f05a70757df28ab2f119b25da440b
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-07-24 17:33:43 +01:00
Takashi Kajinami fa9450d5b5 Deploy healthcheck middleware as app instead of filter
Using the healthcheck middleware as a filter is deprecated and
the middleware should be used as an application[1].
 [1] 6feaa13610c450c8486f969703768db5319b4846

This change updates definition and usage of the healthcheck middleware
accordingly to avoid the following deprecation warning.

DeprecationWarning: Using function/method 'Healthcheck.factory()' is
deprecated: The healthcheck middleware must now be configured as
an application, not as a filter.

Closes-Bug: #1937901
Change-Id: Id41e0313a481bea4e2bb14c69f2ad8a2070aa9be
2023-05-02 19:21:42 +02:00
Cyril Roelandt 41da684fe4 Fix functional test failures with PasteDeploy 3.x
PasteDeploy replaced pkg_resources with importlib_metadata[1] in 3.0.
This causes functional test failures because the line breaks in the
example configs from glance/tests/functional/__init__.py are read as a
"prefix" by PasteDeploy. For instance,

paste.filter_factory =
  glance.api.middleware.version_negotiation:VersionNegotiationFilter.factory

is read as "\nglance.api.middleware.version_negotiation:VersionNegotiationFilter.factory".

This might be a bug in PasteDeploy or (quite unlikely) in
configparser.ConfigParser rather than in our code, so this issue should
be revisited at some point. For now, remove the cosmetic line breaks in
order to fix our gates.

[1] ef58c20961

Change-Id: I1e557e731cceb3cfbc4ebc3f0887adf372a91d1d
2023-04-12 20:37:47 +02:00
Zuul ac93d8b289 Merge "Remove deprecated ``enforce_secure_rbac`` option" 2023-02-20 23:11:14 +00:00
Zuul 09fce985f7 Merge "Enabled new defaults and scope checks by default" 2023-02-20 15:13:36 +00:00
Zuul 410b23cfbd Merge "Fix E741 issues" 2023-02-16 11:53:32 +00:00
Pranali Deore b20cc91e6f Remove deprecated ``enforce_secure_rbac`` option
As per the revised SRBAC community goals, glance service is now
switching to new defaults by default hence removing the deprecated
``enforce_secure_rbac`` option which is no longer needed.

The ``enforce_secure_rbac`` option was introduced EXPERIMENTAL in
Wallaby release for operators to opt into enforcing authorization
based on common RBAC personas.

Related blueprint secure-rbac

Change-Id: I273527c85d30c1c09c086c73c892aaa6d127df6b
2023-02-16 11:12:59 +00:00
Pranali Deore 8c04d19e88 Enabled new defaults and scope checks by default
Enabling the enforce scope and new defaults by default in glance

Related blueprint secure-rbac

Change-Id: I0808dc0b1b34b527e38aa137c1dd25e1fc06409f
2023-02-16 11:11:31 +00:00
Zuul 0f8e5ef017 Merge "Update migration constant" 2023-02-08 15:45:16 +00:00
Abhishek Kekane 9786936915 Update migration constant
Update the data migration current release to '2023_1'. Added
empty migration scripts to rule out regression scenarios as well.

Change-Id: Ic97825dd9c38ec1c759e5ca610a19c93fe4ca6a6
2023-01-18 04:57:34 +00:00
Cyril Roelandt 399438cd04 Fix E741 issues
In 2.6.0, pycodestyle is going to find more E741 issues[1], which is
going to break the gate by making "tox -epep8" fail.

[1] https://www.flake8rules.com/rules/E741.html

Change-Id: I94ee03a00dd01726de15a29424183faabdeba844
2023-01-16 12:01:06 +01:00
whoami-rajat 0e70f66165 Fix cinder tests for refactoring effort
We are refactoring the glance cinder store[1] to support the
extend in-use feature[2] due to which cinder store tests on
glance side are failing[3] as they aren't able to locate the
files in their new location.
This patch fixes the issue to import from new path while keeping
backward compatibility to import from old path to pass gate until
the new changes have merged.

[1] https://review.opendev.org/c/openstack/glance_store/+/843103
[2] https://review.opendev.org/c/openstack/glance_store/+/868742
[3] https://zuul.opendev.org/t/openstack/build/41a1abf9c749476bb10a56600ca07f0e

Change-Id: Ib9f7160fdbac74fc419faa6b3ab8acb17400d392
2023-01-03 11:52:04 +05:30
Zuul 5c675bddcf Merge "Extending stores-detail API" 2022-08-24 12:05:46 +00:00
Mridula Joshi b0d727d028 Extending stores-detail API
This patch extends the functionality of the API
``GET /v2/info/stores/detail`` to expose store details
of other stores. Currently the ``stores-detail`` API
exposes store details of RBD backend.

Implements: blueprint expanding-stores-detail
Change-Id: I0c7bbc315b56dc0e40397b4ed8a68d1668203c44
2022-08-23 14:42:53 +00:00
Rajat Dhasmana f40ac40c2c Mock getaddrinfo in cinder store tests
With change Ic79815972e654a8bfe2775f57c68cfa0bf115e2f, we started
using getaddrinfo method from socket lib to fetch the system
IP address (v4 or v6) and pass it to os-brick.
The test works for most systems since a valid IP is returned but
for some deployments, it fails with the following error.

raise socket.gaierror(socket.EAI_NONAME, 'No address found')

In any case, it is a good practice to mock it in tests.
This patch mocks getaddrinfo in the current cinder store tests.

Change-Id: Ib03fefbb6034a7e7d2ff1e16430d28962c2ca355
2022-08-08 07:59:14 +00:00
Abhishek Kekane c1a2b9e221 Immediate caching of an image
Till now glance was dependent on periodic job to get image
cached locally and also has dependency on ``cachemanage``
middleware for the same.

This patch removes the periodic job and dependency of
``cachemanage`` middleware and initiates immediate caching
of an image via ``PUT /cache/image_id`` API call.

Co-Author: Dan Smith <dms@danplanet.com>

Implements: blueprint instant-caching
Change-Id: I9ab3f1b7595e22dbb03af95168314352a44eb930
2022-07-19 05:14:38 +00:00
Zuul c66705069b Merge "Remove unicode literal strings" 2022-07-08 07:55:24 +00:00
Zuul bef3a70aed Merge "Drop support for SQLite < 3.7" 2022-07-07 15:01:32 +00:00
xuanyandong be997b53ab Remove unicode literal strings
Co-Authored-By: Cyril Roelandt <cyril@redhat.com>
Change-Id: Id9e1a5fb9c732c207ee08f0dbf387436a1783174
2022-06-30 19:37:03 +02:00
Abhishek Kekane a71f7ba44e [APIImpact] Correct API response code for DELETE cache APIs
DELETE /v2/cache/{image_id} and /v2/cache returns HTTP 200 response
code to user but as per proposal it should be HTTP 204. This change
returns HTTP 204 response to user.

Closes-Bug: #1980049
Change-Id: I9d1c25638584fe346e3937d0536413d548d46e8e
2022-06-28 17:31:12 +00:00
Cyril Roelandt 1962f47ed2 Delay string interpolations at logging calls
This was already fixed in 4889dc1814 but
we did not enforce this rule and reintroduced "bad" string
interpolations. This patch adds a hacking rule to prevent us from doing
this again in the future.

Change-Id: I96d9a157d3887286542859d67138ffdae5a589f1
2022-05-14 03:17:34 +02:00
Zuul 331ce59a50 Merge "Fix tests for logging connection info" 2022-05-10 20:05:32 +00:00