Commit Graph

275 Commits

Author SHA1 Message Date
Takashi Kajinami 9f896ab03d Bump hacking
hacking 3.0.x is too old.

Change-Id: I33875c36dbbdb946841f8c583636ccdf88d3331f
2024-01-27 16:55:11 +00:00
Takashi Kajinami 08eedc8c45 Drop unused pyOpenSSL
glance no longer imports the library directly.

Change-Id: I9e09fa6396a959d97e690a08dc318d3aa0a78cfd
2023-11-06 22:06:39 +09:00
Cyril Roelandt bdb8b51539 Bump eventlet to a version that fixes #632[1]
Closes-Bug: #1889664

[1] https://github.com/eventlet/eventlet/issues/632

Change-Id: Iec398ead178a60a318e6eb8d1487b74720eac72e
2023-05-10 17:37:06 +02:00
Pranali Deore 8c04d19e88 Enabled new defaults and scope checks by default
Enabling the enforce scope and new defaults by default in glance

Related blueprint secure-rbac

Change-Id: I0808dc0b1b34b527e38aa137c1dd25e1fc06409f
2023-02-16 11:11:31 +00:00
Takashi Kajinami 7e6a118dfd Add missing oslo_limit options
Glance now depends on the oslo.limit library to support the unified
quota[1], but parameters of the library are still missing from
glance-api.conf.

This change ensures the parameter of the oslo.limit library are
included in glance-api.conf generated by oslo-config-generator.

[1] 06e6542f15

Closes-Bug: #1946100
Change-Id: I56a2a9d8184f50624239b90dd9bef671c195fb90
2022-07-05 22:00:11 +09:00
Brian Rosmaita 6af8b6e51c Remove workaround for python_exec cannot be None
oslo.config 4.5.1 contains change I2e1f187feaf4, which makes
the workaround introduced by change Ic40f582f83e0 unnecessary.

Change-Id: Ib7fc2f2082981b1765e901ca5b277fce08221ba8
Related-bug: #1962581
Related-bug: #1962603
2022-05-23 17:35:29 -04:00
Zuul b085fbecc8 Merge "Remove final six usage" 2022-02-14 20:05:38 +00:00
Dan Smith f865b8cac7 [APIImpact] Quota usage API
This adds a /v2/info/usage API endpoint which exposes to the user
their current limits and usage.

The discovery API does not (appear to) have existing tests, so this
adds a module for that, although only usage tests are added currently.

Implements: blueprint quota-api
Change-Id: I50c98bac50f815bdb9baae024e77afd388f74554
2022-02-03 09:55:50 -08:00
Stephen Finucane 33741138d9 Remove final six usage
We also update docs since guidance has necessarily changed here.

Change-Id: I7c24a1aa3545f3499a7a2ce30b73e2656666c764
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2022-01-27 16:37:20 +00:00
Erno Kuvaja 71c1e4b18a Bump SQLalchemy to version that supports PY3.9
Bumping SQLAlchemy to 1.3.14, which is first version supporting
Python 3.9 runtime, this hopefully addresses the issue reported.

Change-Id: I73f0c7c067fcab00a97cd925cad19edd64b6eb45
Closes-Bug: #1957167
2022-01-21 12:59:01 +00:00
Abhishek Kekane 416791f3bc Bump oslo.policy to version 3.8.1
Closes-Bug: #1944445

Change-Id: Iecdbf2555c5ce7ca1b2556bb5a0814db6fdf5e90
2021-09-21 15:50:18 +00:00
Abhishek Kekane 4892988491 Bump oslo.log to version 4.5.0
In this version versionutils were updated for Xena release.

Closes-Bug: #1943985
Change-Id: I2159e74dd9004e6d901dbcecf651de078fb2c193
2021-09-17 15:44:37 +00:00
Stephen Finucane 5cb97f3a40 trivial: Remove references to sqlalchemy-migrate
Change-Id: I9eda5e5ac697c110e6047c6086855e04cbfe6b87
Implements: blueprint remove-sqlalchemy-migrate
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-07-13 17:19:03 +01:00
Zuul 77be0e3e23 Merge "Revert "Remove all usage of keystoneclient"" 2021-07-06 17:11:49 +00:00
Ghanshyam Mann 2dc1084fcd Fix oslo policy DeprecatedRule warnings
Since 3.7.0, oslo policy started the DeprecationWarning[1] if
deprecated_reason and deprecated_since param are not passed
in DeprecatedRule or they are passed in RuleDefault object.

These warnings are logged for every test which increase the
log size and sometime can full the log buffer and fail the
job.
- https://zuul.opendev.org/t/openstack/build/ddb517bc792b49b9a8cf508eb3e361b0/log/job-output.txt#918

[1] https://github.com/openstack/oslo.policy/blob/3.7.0/oslo_policy/policy.py#L1538

Change-Id: Iae5d4c06c736e9cb24037f32cbe369fdfee9c2ab
2021-07-05 11:48:17 -05:00
Dan Smith 06e6542f15 Add unified quotas infrastructure
This adds some infrastructure to be able to query and honor limits
declared in keystone. It adds a single initial quota value for the
total size of all active images for bootstrapping the tests.

Checking these values is controlled by a new configuration option
that globally enables and disables the checking, defaulting to
False.

Related to blueprint glance-unified-quotas
Change-Id: I8d8f4aaed465486e80be85bc9a5d2c2be7f1ecad
2021-06-21 10:58:55 -07:00
Erno Kuvaja 9b683678b2 Revert "Remove all usage of keystoneclient"
This reverts commit 810417df86.

The Swift driver not being able to use Trusts had nothing to do
about the store driver itself nor that keystoneauth1 would have
broken the feature, but rather it not having the functionality
in the first place and us not catching that on reviews.

We should figure out how to test this before we try to replace
this code again.

Change-Id: If12a013404296486dc387b099477d1608b24ba63
Closes-Bug: #1916052
2021-06-10 13:51:49 +01:00
Lance Bragstad 764f29a99d Bump requirements to prepare for secure RBAC
To implement proper scope checking, we need some updated libraries that
properly handle tokens and relay that information to the underlying
service. This commit updates the oslo.policy, oslo.context, oslo.log,
and keystonemiddleware requirements to versions that understand all the
various scopes so that we can update the default policies.

Additionally, this commit updates transitive dependencies in
requirements.txt and lower-constraints.txt to install properly with
lower-constraints defined.

Implements: blueprint secure-rbac

Change-Id: I7dec6b9919e7679aff1a0bb5db1e806384730386
2021-02-23 17:27:03 +00:00
Stephen Finucane 4b884b9e9b Uncap PrettyTable
This is now maintained. We can uncap it.

Change-Id: I7c19e8f24be792ea8672a02c5c02f585a95acd61
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-02-11 12:21:54 +00:00
Ghanshyam Mann c107629f90 [goal] Deprecate the JSON formatted policy file
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:

1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.

2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.

Also convert the ./glance/tests/etc/policy.json to policy.yaml
file. Replace policy.json to policy.yaml ref from doc and tests.

[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Depends-On: https://review.opendev.org/c/openstack/nova/+/773192
Change-Id: I17d0374dd4223688e5f95253802a4ae87377953a
2021-01-29 15:31:47 -08:00
Ade Lee 8027d90710 Replace md5 with oslo version
md5 is not an approved algorithm in FIPS mode, and trying to
instantiate a hashlib.md5() will fail when the system is running in
FIPS mode.

md5 is allowed when in a non-security context.  There is a plan to
add a keyword parameter (usedforsecurity) to hashlib.md5() to annotate
whether or not the instance is being used in a security context.

In the case where it is not, the instantiation of md5 will be allowed.
See https://bugs.python.org/issue9216 for more details.

Some downstream python versions already support this parameter.  To
support these versions, a new encapsulation of md5() has been added to
oslo_utils.  See https://review.opendev.org/#/c/750031/

This patch is to replace the instances of hashlib.md5() with this new
encapsulation, adding an annotation indicating whether the usage is
a security context or not.

Reviewers need to pay particular attention as to whether the keyword
parameter (usedforsecurity) is set correctly.

It looks like the usage of md5() here is solely to determine a checksum
of an image.

With this patch and the dependent patch for glance_store, all the
unit and functional tests pass on a FIPS enabled system.

Depends-On: https://review.opendev.org/#/c/756157
Depends-On: https://review.opendev.org/#/c/760160
Change-Id: I3b6d78d9792d4655bf0f4989cf82aced3f27491b
2020-12-15 10:43:19 -05:00
Erno Kuvaja 795dad7633 Bump lower_constraints and requirements
This is to unblock the stuck gate due to the lower_constraints
job failing.

Change-Id: Ifd55c44fef4e2187052d77084dc9c0fa9c9a0d16
2020-12-14 21:39:58 +00:00
whoami-rajat 98a1e792c6 Support cinder multiple stores
This patch updates the location URL of the legacy images while
upgrading from single cinder store to multiple stores.
It does that with the help of lazy loading logic i.e. while
GET images call, it checks the location URL and metadata
of the image against the configured store ids and updates
images to respective stores on the basis of volume type (comparing
image-volume's type with the configured cinder_volume_type).
Legacy image URL:
cinder://<volume-id>
New image URL:
cinder://<store-id>/<volume-id>

NOTE: bumping lower-constraints/requirements of glance-store to 2.3.0 as
it includes changes[1] that are a hard requirement for cinder multiple
stores to work with glance

[1] https://review.opendev.org/#/c/746556/

Change-Id: I087a89c20813378fea8ff22ddf81d7a10c220db3
Implements: blueprint multiple-cinder-backend-support
2020-09-07 09:07:42 +00:00
Erno Kuvaja 3ce486f94d Don't use Stevedore 3.0.0 which breaks gate
Depends-on: https://review.opendev.org/#/c/740681/
Change-Id: Ic686300fc56a668100d0073170963d100fdff454
2020-07-13 11:52:50 +00:00
Zuul f4f56b457d Merge "Cap jsonschema 3.2.0 as the minimal version" 2020-07-10 19:22:36 +00:00
Sean McGinnis 9eb562ebfa Update lower-constraints versions
This updates lower constraints to versions that will work with py38 so
that when we move to running on focal nodes, which has py38 as its
default py3 runtime, the lower-constraints job will continue to pass.

It also cleans out some secondary requirements that are no longer needed
due to our direct dependencies being updated.

Linters are removed that are kept in the global requirements blacklist
as those are not version tracked and are not relevant for our
lower-constraints unit test runs.

Change-Id: Ie3698c7334b31c2291b41fd3f7e21add0dd6a19b
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-07-03 13:32:23 -05:00
Hervé Beraud 4cd56a49d3 Cap jsonschema 3.2.0 as the minimal version
Previous versions of jsonschema (<3.2.0) doesn't support python 3.8 [1].
Python 3.8 is part of the victoria supported runtimes [2] so we now force
to use jsonschema version 3.2.0 to avoid issues, remove ambiguity and ensure
that everything works with python 3 in general.

[1] https://github.com/Julian/jsonschema/pull/627
[2] https://governance.openstack.org/tc/reference/runtimes/victoria.html#python-runtimes-for-victoria

Change-Id: Ia2435620da6a91a8bd1c08b8ae2e3b231f3c1371
2020-05-26 21:49:00 +02:00
Zane Bitter 8acedfd48a Add deprecation message to policy "default" rule
The default check_str for the "default" policy rule has changed. From
the Queens release until this series it was "role:admin" (prior to
Queens it was "@", which means allow all requests). In order to
accomodate existing policies as best as possible, the default check_str
has been changed back to "".

In the unlikely event that any existing policies are relying on the
default check_str for the "default" rule, they need to start explicitly
specifying it to maintain similar behaviour. This patch adds a
deprecation warning when using the "default" rule without overriding the
value in the policy file.

This will result in users who carry over an unsafe policy file from the
Queens-Train error seeing a warning. Unfortunately it will also result
in a warning for users who no longer supply a policy file and are
relying on the default policies in code.

Change-Id: I3d3ab9b0172521fb42314f2de33ff02985ad4864
Depends-On: https://review.opendev.org/698790
Depends-On: https://review.opendev.org/699299
2020-03-30 05:41:49 +00:00
Erik Olof Gunnar Andersson e5435bb7b6 Add support for oslo.reports
Oslo Reports enables OpenStack projects to dump Guru Meditation
Reports with useful debugging information on live services
to files or stderr.

Change-Id: I33b6e52870b583c70aa8141ab55d8738beaf5c59
2019-09-20 05:17:58 +00:00
manchandavishal 64d14d328b Blacklist eventlet 0.23.0, 0.25.0
Kombu 4.6.4 causes issues with eventlet 0.23.0 and 0.25.0[1].
Due to an upper-constraint version bump, we're seeing failures
on openstack gate.
This patch fixes the issue.

[1] https://review.opendev.org/#/c/678078

Change-Id: I7e89a9710b6be8b58872d4f44168fcf92b15f711
2019-09-10 06:55:00 +00:00
Zuul e475581c72 Merge "Delete secret key on image deletion" 2019-09-06 13:49:25 +00:00
Cyril Roelandt b190a39a28 Delete secret key on image deletion
We add two extra properties for images:
- cinder_encryption_key_id, which stores the encryption key id;
- cinder_encryption_key_deletion_policy, which states whether the secret
  key should be deleted on image deletion.

This feature uses the Castellan key manager, and will therefore work
with all its supported backends.

Implements: blueprint barbican-secret-deletion-support
DocImpact

Change-Id: Iacd0b3785ad4cdd06961e6d11967775806e009ff
2019-09-05 03:16:39 +02:00
Abhishek Kekane 0e55ad71a8 Change location metadata key 'backend' to 'store'
As a part of vocabulary correction, changed the location
metadata key name from 'backend' to 'store'. Modified
corresponding tests as well.

bp:multi-store-vocabulary-correction
Change-Id: Iae3503cba6be362b372e1fc3e75c2ddb1e99b763
2019-08-08 04:52:51 +00:00
pengyuesheng f6a938660b Do not use glance_store 0.29.0
glance_store 0.29.0 introduced backwards incompatible change
breaking multi-store feature that was supposed to be included in
1.0.0 release without any mention in release notes. Revert is in
process and 0.29.1 will be released after. 0.29.0 should not be
used.

Change-Id: I82e0438de5aaed2ef1975a5aa61062637e32a69e
2019-06-27 15:18:19 +08:00
Erik Olof Gunnar Andersson 9834253f2d Uncap jsonschema
The current cap on jsonschema is breaking
the requirements test.

The global cap was removed with this change
https://review.openstack.org/#/c/649669/

Change-Id: I4dc433267b123e5396e378dfda2e05551896d62a
2019-04-06 16:10:56 -07:00
Lucian Petrut 5759ec0b1c glance Windows support
This change will allow glance services to run on Windows, using
eventlet wsgi for API services.

This change will:
* avoid monkey patching the os module on Windows (which causes Popen
  to fail)
* avoiding unavailable signals
* avoid renaming in-use files or leaking handles
* update the check that ensures that just one scrubber process may
  run at a time. We can't rely on process names as there might be
  wrapper processes that have similar names (no she-bangs on Windows,
  so the scripts are called a bit differently). We'll use a global
  named mutex instead.

A subsequent change will leverage Windows job objects as a
replacement for process groups, also avoiding forking when spawning
workers.

At the moment, some Glance tests cannot run on Windows, which is
also covered by subsequent patches.

DocImpact

blueprint windows-support

Change-Id: I3bca69638685ceb11a1a316511ad9a298c630ad5
2019-03-13 16:41:11 +02:00
Lance Bragstad c82ecc7463 Implement scaffolding for upgrade checks
One of the community goals for Stein is to implement a command-line
tool for operators that runs programmable checks that might impact
upgradability.

This commit lays down the basic structure for the upgrade checks and
ties it up to `glance-status` command.

Change-Id: I7fcf5235a76d15dbcb2c49255bc26c2b586cd71c
Story: 2003657
Task: 26135
2018-12-05 16:43:00 -05:00
Matt Riedemann 509d494f0d Drop dependency on monotonic
Since it's no longer used we can stop requiring it.
This also fixes the requirements check job after
change Ib8c1bf08f5fa7463911602b0df19315907c81e04.

Change-Id: I2f165999fbcc208067a265a3bba049fc86943a47
2018-12-05 16:43:00 -05:00
Zuul 02a405a410 Merge "Do not use oslo.messaging 9.0.0" 2018-10-11 22:10:06 +00:00
Brian Rosmaita b9047e87b6 Do not use oslo.messaging 9.0.0
A patch proposed to change the version of olso.messaging in
upper-constraints indicates that 9.0.0 breaks glance unit tests [0].
The periodic job that runs current glance tests against oslo
masters indicates that the problem has been fixed in olso messaging
master by commit 172cfb33f3ee207531a9e82fbc8293d24009a256 [1,2].

[0] https://review.openstack.org/#/c/607521/
[1] http://zuul.openstack.org/builds?pipeline=periodic&project=openstack%2Fglance&job_name=openstack-tox-py35-with-oslo-master
[2] http://zuul.openstack.org/builds?pipeline=periodic&project=openstack%2Fglance&job_name=openstack-tox-py27-with-oslo-master

Depends-On: https://review.openstack.org/608835
Change-Id: I7d3d76cb2aae7914e4a78669b9755b793cf6ee8a
2018-10-09 08:42:03 -04:00
Brian Rosmaita 9b9d0567ef Use WebOb 1.8.1
The requirements/upper-constraints file was modified to allow WebOb
1.8.1 by commit 88bafa11deb9bd7595983d97ffabca338f073ba3.  This
patch simplifies some Glance code that could handle both WebOb
1.7.4 and 1.8.1 so that it now only handles the latter, and updates
the glance/requirements.txt and glance/lower-constraints.txt to
reflect that we support WebOb 1.8.1+ only.

Change-Id: I03e03013927cc5434aa0d97657d5e7efd8223ee5
Closes-bug: #1770410
2018-08-22 10:36:32 -04:00
Brian Rosmaita 0b24dbd620 Multihash implementation for Glance
Partially implements blueprint multihash.

Requires glance_store 0.26.1

Co-authored-by: Scott McClymont <scott.mcclymont@verizonwireless.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>

Change-Id: Ib28ea1f6c431db6434dbab2a234018e82d5a6d1a
2018-07-31 21:28:38 -04:00
Zuul 8a2d154234 Merge "Remove all usage of keystoneclient" 2018-04-25 08:33:28 +00:00
Nguyen Hai 623d914348 Fix incompatible requirement in requirement.txt
Requirement for package eventlet has an exclusion not found in
the global list: set(['<0.21.0', '!=0.20.1', '!=0.18.3'])
vs. set(['!=0.20.1', '!=0.18.3'])

Change-Id: Ieb6f40efde78fc0df42c395f0ed650df5b2d0f77
2018-04-11 15:02:18 +09:00
Gage Hugo 810417df86 Remove all usage of keystoneclient
A lot of keystoneclient has been long deprecated and is slated
for removal. This change removes two usages of keystoneclient
and replaces them with similar functionality in keystoneauth.

Change-Id: I0128a7bb42b0d691600fdd03aac287633c8b2451
2018-04-09 17:45:10 -05:00
OpenStack Proposal Bot ca51cb8465 Updated from global requirements
Change-Id: I69cbca08813bf47892594d7485489a77e465c790
2018-03-26 08:02:30 +00:00
OpenStack Proposal Bot ec81cb97ca Updated from global requirements
Change-Id: I371c316e1dcc2df092eb45af31033d5a8782a894
2018-03-17 08:28:33 +00:00
OpenStack Proposal Bot 3914fefd50 Updated from global requirements
Change-Id: I97c43ba4c7d0525ee11a51adbe75a8e43c6b3958
2018-03-13 01:04:44 +00:00
OpenStack Proposal Bot 0e1eceff91 Updated from global requirements
Change-Id: Ic2ab79ce8bdcd5410ee8aa8b1e2e94816990c7a1
2018-02-17 09:27:52 +00:00
Vladislav Kuzmin 6e82ea023a Replace xml defusedxml
xml was considered as vulnerable to different atacks.
It is recommended to replace this library with defused_xml

Change-Id: I2b146dc34ada37a3ed9ecf49513d024a8ca2fb19
Related-Bug: #1625402
2018-01-31 14:49:13 +00:00