Commit Graph

34 Commits

Author SHA1 Message Date
Takashi Kajinami 0a5b511729 Use LOG.warning instead of deprecated LOG.warn
The LOG.warn method is deprecated[1] and the LOG.warning method should
be used instead.

[1] https://docs.python.org/3/library/logging.html#logging.warning

Change-Id: Ie0cac63dedf69728392f293f4551e495aebf4d40
2022-01-19 11:36:12 +09:00
Dan Smith 76c3011a64 Enforce keystone limits for image upload
This adds enforcement of the image_size_total keystone limit for
image upload and import. We simply check the quota before either of
these operations and refuse to proceed further if the user is over
their quota.

Note that this disables checking of the global size quota if keystone
quotas are enabled.

Note this includes another fix to couple unit tests that do not
properly pass context to the get_flow() method.

Partially-implements: blueprint glance-unified-quotas
Change-Id: Idf5f004b72436df1f9c77bb32d60b9be5ae77a68
2021-06-29 08:53:18 -07:00
Dan Smith 314e93abe4 Exclude os_glance namespace from property quota
Now that glance is using properties in the os_glance namespace for
internal purposes, we should exclude the counting of these from the
enforced image property quota.

Change-Id: I5fbe5eb12fd34e054137732a02c4cc5b687e7c77
Related-Bug: #1912001
2021-01-25 12:30:50 -08:00
Dan Smith 16a5431c66 Make glance-api able to do async tasks in WSGI mode
This teaches glance-api how to do async threading things when it is
running in pure-WSGI mode. In order to do that, a refactoring of things
that currently depend on eventlet is required.

It adds a [wsgi]/task_pool_threads configuration knob, which is used
in the case of pure-WSGI and native threads to constrain the number
of threads in that pool (and thus the task parallelism). This will
allow tuning by the operator, but also lets us default that to just
a single thread in the backport of these fixes so that we can avoid
introducing a new larger footprint in the backport unexpectedly.

Partial-Bug: #1888713
Depends-On: https://review.opendev.org/#/c/742047/
Change-Id: Ie15028b75fb8518ec2b0c0c0386d21782166f759
2020-07-24 11:13:45 -07:00
weiweigu 4889dc1814 Fix string interpolation to delay by logging
String interpolation should be delayed to be handled
by the logging code, rather than being done
at the point of the logging call.

See the oslo i18n guideline.

* http://docs.openstack.org/developer/oslo.i18n/guidelines.html

Change-Id: I400a38a61e1ed6288296e8572218c7642795cbd4
Closes-Bug: #1596829
2020-01-09 08:53:55 -06:00
Thierry Carrez 84c8f9e3ff Quiece 'invalid escape sequence' deprecation msg
Glance tests are currently too verbose for subunit and make
them randomly fail under python 3.

Using raw strings should avoid the DeprecationWarnings around
'invalid escape sequence' being used with the re module.

Change-Id: I14c381ac4b17ca4c5755ed78c55dc44362ab37ef
2019-04-03 17:09:47 +02:00
Matt Riedemann 88a8ad7823 Fix DeprecationWarnings for RequestContext.tenant/user usage
RequestContext.tenant and user fields are deprecated in favor
of project_id and user_id respectively.

This change modifies the glance.context.RequestContext constructor
to transition usage of tenant/user to project_id/user_id until
all tests are moved over to the new attributes. Runtime usage of
the old fiels is updated.

To prevent new code from using the deprecated fields, a warnings
filter is added which will make tests fail if they hit code using
the old fields.

Co-Authored-By: Abhishek Kekane <akekane@redhat.com>
Change-Id: I351380840308a24769ece93abc6d1a9a6d6aa06f
2019-02-06 09:59:36 -05:00
Jenkins 2682dfe200 Merge "Import i18n functions directly" 2015-12-10 03:42:30 +00:00
kairat_kushaev c27c25f779 Import i18n functions directly
Import i18n functions from module directly and do not use
global module variables like _ = i18n._. It makes code shorter
and cleaner. It also to detect cases when these functions are not
used in module.

Change-Id: Iaa593ac1f2dd15cbcad049bef6ba68f1cfa610da
2015-12-07 18:20:43 +03:00
leo.young bcbfa598e8 Change the format of some inconsistent docstring
Replace all the ":return:" with ":returns:"
Replace all the ":returns " wtih ":returns:"
Replace all the ":returns :" with ":returns:"

Add the changes for ":raise" also, change ":raise" to ":raise:"

Change-Id: I5d0def4665d0cd69b73c3c3826ad8f208e1e970e
2015-12-01 10:50:49 -05:00
PranaliDeore 68c2a323bd Remove unused imported marker functions
Removed unused imported marker functions
( _LI = i18n._LI, _LW = i18n._LW) from several
places.

Closes-Bug: #1431243
Change-Id: I71b551bfa72631036fcf9d2c38701d74a0e14611
2015-06-29 05:57:37 +00:00
Louis Taylor 0acff569e0 Move to graduated oslo.log module
This removes the log module from the oslo incubator directory.

Change-Id: Ib017cc2777e27c58b376c3138c9fba614783cc34
2015-03-12 11:48:24 +00:00
abhishekkekane ea2252b4fd Zero downtime configuration reload
Add the ability to dynamically change configuration settings of a running
glance server with no impact to service.

A running glance server consists of a parent/master process and one or
more child/worker processes.

On receipt of a SIGHUP signal the master process will:

* reload the configuration
* send a SIGHUP to the original workers
* start (a potentially different number of) new workers with the new
  configuration
* its listening socket will *not* be closed

On receipt of a SIGHUP signal each original worker process will:

* close the listening socket so as not to accept new requests
* complete any in-flight requests
* complete async requests (V1 create with copy-from option and V2 task api)
* exit

This approach is based on nginx's behaviour and avoids some of the
disadvantages of the current oslo Launcher reload:

* Race conditions: Launcher does not shutdown eventlet cleanly, existing
  requests can fail.
* If all workers are busy (likely if #workers is small) there can be a
  lengthy delay when new requests are not processed.
* Long lived pre-SIGHUP idle client connections can stall request
  processing indefinitely.
* Not all parameters can be changed, eg number of workers.
* The wsgi pipeline cannot be changed, eg to enable caching.

Change-Id: I73875c654410f207e3efdb2932147cf26d75b3f1
Partially implements: bp sighup-conf-reload
2015-03-09 05:12:25 -07:00
Louis Taylor d759a6fd1a Move from oslo.config to oslo_config
oslo_config was moved out of the oslo namespace in oslo.config>=1.6.0.

Related-to: blueprint drop-namespace-packages

Change-Id: I30ecbf8f9de77496fcb643c7ad9738d79ad359f0
2015-01-12 14:34:22 +00:00
Louis Taylor 41d45dfb1c Move from oslo.utils to oslo_utils
oslo_utils has been moved out of the oslo namespace.

bp drop-namespace-packages

Change-Id: Ie818e72b31efd2a6ab182444967bdfaca9366f4a
2015-01-09 20:01:42 +00:00
Rajesh Tailor 42cb4bfb0d Fix typo in common module
Fixed following typo in glance/api/common module:
'exceeed' --> 'exceed'

Change-Id: I5763d74473a43615da9e10f87a4ad048953ca0f5
2014-12-08 01:47:04 -08:00
James Carey 3f2a56ae65 Move from using _ builtin to using glance.i18n _
The builtin _ should not longer be used.  Instead _
should be explicitly assigned from glance.i18n.

This patch does the following:
  (1) adds these explict assignments to any
      glance file that was missing them,
  (2) removes the defintion of _ as a builtin from
      tox.ini so that a pep8 failure will occur
      if _ is not defined,
  (3) removes calls to install().

This removes the last direct use of gettextutils by glance.
The indirect uses, via openstack/common, will be removed
in the next sync with oslo.

Change-Id: Ie54e8a67e747e022cc01022dfad35d89686bdfc4
2014-11-21 18:51:19 +00:00
Erno Kuvaja fa7888d920 Change Glance to use i18n instead of gettextutils
Changing all Glance files to use i18n instead of gettextutils for log
translations.

Finishes bp refactoring-glance-logging

Change-Id: I90d6ab4f7e09f4e7591921f9526de6370ebdb789
2014-11-21 17:12:35 +00:00
Erno Kuvaja 4f46fb67f2 Raising glance logging levels
This change complements glance change #106776 by raising some logging
levels and fixing some of the translations of logging.

Related to bp refactoring-glance-logging
Closes-Bug: #1336955

Change-Id: Ib547f4603025ebe4f6c055779032ab4064784490
2014-11-21 17:11:17 +00:00
Mike Fedosin 6a9e172135 Raise an exception when quota config parameter is broken
Function get_remaining_quota returns None when quota config
param doesn't match a pattern (i.e. it's broken). It messes up 
the logic of its callers and leads to unpredictible behavior of 
the entire system.

This code raises InvalidOptionValue exception if config is broken

Change-Id: Ib08d2114f460d127aacf580fc519a58d6e8eba16
2014-10-30 21:56:36 +00:00
Julien Danjou 874e643549 Switch to oslo.utils
Change-Id: I47dc734c6d6e2ab99c25701ae3492acd5e442212
2014-10-07 14:55:57 +02:00
Julien Danjou fadbef8511 hacking: upgrade to 0.9.x serie
Change-Id: I252758fd633662de9659a402c5e3d7e3ce1fae0f
2014-10-06 14:41:03 +02:00
Erno Kuvaja e694ed04f6 Refactoring Glance logging lowering levels
This change fixes translations to be using correct translation functions.
Contains few log level drops for too high logging levels.

This PS does not address logging changes for modules under glance/store/
due to the new repo request. After stores are moved to their own repo
new PS will be submitted for those.

This PS does not touch the test code.

Separate PS will be submitted for log lever raises withing codebase
as requested during the spec review.

Related to bp refactoring-glance-logging

Change-Id: I683c35face8d5b7056ef7fedcd75deb63dbbf8e2
2014-07-17 10:03:12 +00:00
Vaibhav Bhatkar cff35c1999 user_storage_quota now accepts units with value
user_storage_quota now accept value in B, KB, MB, GB
or TB.  The unit is optional. If no unit is specified
Bytes is used as default.

DocImpact

Change-Id: Icc3f672869a5947cbcae38de92993c88ce0ef4e1
Closes-Bug: #1261747
2014-04-06 12:34:32 +00:00
Sam Stavinoha eb40bb6760 When re-raising exceptions, use save_and_reraise
In some cases the exception context can be cleared,
resulting in None being attempted to be re-raised after
an exception handler is run.

Change-Id: I16811ba8f0bfaa15199a648748bbc7d7a6a500bc
Partial-Bug:#1281453
2014-03-26 12:00:29 -05:00
Matt Fischer 1777e019e0 Log a warning when a create fails due to quota
When a create call fails due to user_storage_quota
being exceeded, log a warning message.

Closes-Bug: #1270832

Change-Id: Ic7ec266aace03cff0a1f72dc3ba6f17b89f5fd55
2014-01-31 13:39:31 -07:00
Luis A. Garcia e99780fe8d Remove use of locals() when creating messages
Per the hacking guidelines locals() should not be used when creating
messages, but additionally this is preparation for upcoming message
internationalization work.

Change-Id: I9ce616b94faf2d7df895c6bffb94baac8c82c7ce
2013-11-12 00:50:40 +00:00
John Bresnahan e3e4f4d927 Add a storage quota
This patch adds a storage quota that is applied against the sum total
of a users storage consumption against all configured storage systems.
A single quota is applied to all users via the configuration option
'total_storage_quota'.  Most of the patch is about enforcement so
when a separate service for quota management emerges in OpenStack the
per user value to enforce can be obtained from that service but the
enforcement code will remain the same.

blueprint glance-basic-quotas
docImpact
Change-Id: I251832f7372c70942be6f0c6aa12285145dd7c18
2013-08-28 17:12:34 -10:00
Stuart McLaren 19b82a022c Standardize on newer except syntax
We currently have a mix of 'except X, y:' and 'except X as y:' syntax.
Standardize on the newer 'except as' form.

This patch was generated automatically using:

find * -type f -name 'glance-*' ! -name '*\.conf' ! -name '*\.ini' -o \
 -name '*\.py' | xargs 2to3 -f except | patch -p0

Addresses bug 1160331.

Change-Id: Ica8d1303973af64d0406457267ee75f608e869d0
2013-03-26 12:14:55 +00:00
annegentle a2b9bed75d Updates OpenStack LLC with OpenStack Foundation
Patchset addresses reviewer's comments, rebase against master,
and follows new guidance from Foundation:
http://wiki.openstack.org/Documentation/Copyright

Change-Id: I94367461505778973528eb8835d991f4fb796dab
2013-02-17 19:19:43 -08:00
Brian Waldon 0ee5e7e848 Remove unused imports
Change-Id: Ie5c4e7258a857d73c852897c7eef99bd1d062182
2012-11-16 13:28:49 -08:00
Zhongyue Luo cb64f58b99 Clean up pep8 E128 violations
Fixed E128 errors
All ignores are to be removed in the next sequence of patches

Change-Id: I3ccff7df8c382f83ae6aa9aa6a7f16324c3aec75
2012-10-09 10:04:19 +08:00
Paul Bourke 95e00c9247 Fix cache not handling backend failures
1) caching_iter doesn't handle backend exceptions:

caching_iter assumes any exception that occurs is the result of being
unable to cache.  Hence the IOError raised from size_checked_iter, which
indicates a problem with the backend, means the caching_iter will
continuing trying to serve non-existent data.  The exception was not
been re-raised in this case, making wsgi keep the connection open and
clients stuck forever waiting for more data.

Raising a GlanceException in size_checked_iter rather than an IOError
allows caching_iter to distinguish between a problem fetching data, and
a problem writing to the cache.

2) Checksum verification happens after cache commit rather than before:

This block was outside the context manager block which meant the
GlanceException was not caught by open_for_write and the rollback didn't
happen.  This resulted in an error been logged, but the bad image still
placed in and subsequently served from the cache.

Also:

* Fix test_gate_caching_iter_bad_checksum - the loop to consume the
  iterator in was in a subroutine that never got called.

* Move test_gate_caching_iter_(good|bad)_checksum into
  ImageCacheTestCase to excercise both the sql and xattr drivers.

* Remove invalid registry_host/registry_port params from
  TestImageCacheXattr/TestImageCacheSqlite setup which caused a failure
  when testing the file on it's own using nosetests.

Fixes bug 1045792

Change-Id: I8aedec347e7f50566c44c5b6c6db424573c5ebaf
2012-09-04 13:39:39 +01:00
Brian Waldon bd018d68bd Move checked_iter from v1 API glance.api.common
The checked_iter image streaming wrapper should be in a common
place so the v2 API code can use it as well. This moves it to
glance/api/common.py and gives it a slightly more descriptive
name 'size_checked_iter'.

Change-Id: I9fbf2c76666bfc0b9227a6a6c4d824367526e56a
2012-07-17 16:30:48 -07:00