We also update docs since guidance has necessarily changed here.
Change-Id: I7c24a1aa3545f3499a7a2ce30b73e2656666c764
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We also remove hacking tests for this, along with those for iterkeys and
itervalues (no usage of these).
Change-Id: If5b46580078eb756651ac6118f502eccdc693646
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
If operator has configured read-only 'http' store in multiple stores and while
importing or copying existing image in multiple stores if user has specified
--all-stores true then depending on `allow-failure`; if it is True then image
is imported/copied to other stores than http or if it is false then image can
not be imported/copied to other stores as well. Aslo, new image property
`os_glance_failed_imports` shows `http` as failed store.
Excluded `http` store from the list of stores for import/copying workflow
if user specifies --all-stores as True.
Change-Id: I2ad41022aa709491881e78129708da0ccc25f4f6
Closes-Bug: #1881958
SSL support was removed by [1] which resulted in removal of some
SSL related config options, but two config options 'key_file' and
'cert_file' are still used in utils.stash_conf_values() which is called
in reload functionality. When SIGHUP is sent to the parent process,
this usage causes it to crash and its children processes become orphaned,
which then causes the reload test to be stale (fail).
This change also includes:
- Remove 2 attributes and the decorator that help to skip this test
(self.diable, self.disabled_message, and @skip_if_disabled)
[1] https://review.opendev.org/#/c/697969/5/glance/common/wsgi.py
Change-Id: I361c9d0b53a67b1cc60f0f3012e22accdcc46055
Closes-Bug: #1855708
Newly added multiple import fails if user passes "all_stores" as "true"
in the request. The reason is if all_stores is specified then we are getting
stores using CONF.enabled_backends and unfortunately we are injecting
reserved stores 'os_glance_staging_store' and 'os_glance_tasks_store'
runtime. As a result import job tries to import the image in the
'os_glance_staging_store' store as well where it fails as that image
is already staged in this staging store.
Made a provision to exclude available reserved stores if "all_stores"
is specified.
Change-Id: If2616c275a969cdad5649e2cb0851275e5d0c7d2
Closes-Bug: #1863879
The import image api now supports a list of stores to import data into.
This list can be specified through a new "stores" field that has been
added to the request body.
During import stage, Glance iterates overs this list and send the data
to each store one by one.
If an invalid backend is requested by the user, an exception is raised.
If an errors occurs during verify, already pushed data is removed and
image state is unchanged.
Change-Id: Id3ac19488c0a693d7042be4a3c83f3b9f12313d0
Implements: blueprint import-multi-stores
As eventlet ssl termination is broken with python 3 and
we won't be supporting python 2.7 anymore we will just
remove ssl termination to glance-api and expect the
termination being handled by something else, like HAProxy.
This patch also removes the broken ssl test job as the
non-existing feature is not broken anymore.
Change-Id: Iaf16dfcfdb3a2c93312dcad1ea1229e6b3c8caaa
Glance tests are currently too verbose for subunit and make
them randomly fail under python 3.
Using raw strings should avoid the DeprecationWarnings around
'invalid escape sequence' being used with the re module.
Change-Id: I14c381ac4b17ca4c5755ed78c55dc44362ab37ef
This should always be bytes; previously there was a path that returned an empty
native string.
Change-Id: I1cb4997cd3e766f9a7aea41ebe41a73971496bc3
Closes-Bug: #1818919
In Python 3.7, "async" is a keyword. To prevent it from
conflicting, rename the async package to async_.
Change-Id: I1eaf87eedb86679d9ca9323aac05f0770c33efea
Closes-Bug: #1781617
ImageSizeLimitExceeded exception block [1] is unreachable in upload because
it is caught at [2] and raised StorageQuotaFull exception from there. The
problem here is that we have nested usage of the limiting reader.
To make it correct changed the limiting reader to accept
exception class as parameter so that we can pass the StorageQuotaFull
in case LimitingReader is used for quota check and ImageSizeExceeded exception
if it is used for image size cap check.
[1] fd16fa4f25/glance/api/v2/image_data.py (L230)
[2] fd16fa4f25/glance/quota/__init__.py (L305)
Closes-Bug: #1734832
Change-Id: I5a419b763bee7f983c2a94c6f3a2245281e86743
Clarify the help text and clean up some log messages. Includes
the regenerated glance-image-import.conf.sample file.
Change-Id: I7f9087aaf9c6969e15f63029cc38fe5a0939ad40
Closes-bug: #1749788
Two parts to this fix:
* add a call to oslo.config.cfg.import_group so that the function
that checks a uri against the configured white/blacklists can
access them
* move the location where these options are defined into the
module's __init__ so that they can be imported without causing a
circular import (which happens if you import them from their
current location)
Change-Id: I6363faba0c4cbe75e6e4d0cbf0209a62c10474ef
Closes-bug: #1750205
Implement URI filtering to prevent port scanning with the web-download
Image import method.
Closes-Bug: #1748512
Change-Id: Ide5ace8979bb12239c99a312747b3151c1e64ce8
Probably the most common format for documenting arguments is reST field
lists [1]. This change updates some docstrings to comply with the field
lists syntax.
[1] http://sphinx-doc.org/domains.html#info-field-lists
Change-Id: I0300dbac6dc02681cec2725b3c44a7dd1bfd7b4e
When listing images there are several optional parameters that
can be used to filter the list of images retrieved by the API. The
following two parameters are not working properly: the created_at and
the updated_at. Before the Mitaka release it was possible to use these
filters just using a datetime in the format ISO 8601, starting on
Mitaka, you need to add an operator along with the datetime stamp or
the API call fails. This commit adds backwards compatibility so it is
possible to filter the images list using only a datetime stamp without
also specifing an operator. If no operator is used an eq operator is
assumed.
Change-Id: Id5d5455e77637e0dc7baec25c8163b21634d72c4
Partial-Bug: 1584415
Currently 'glance-manage db purge' fails if given a
very large number for max_rows.
Moved and renamed validate_mysql_int() from
glance.common.utils to glance.db.sqlalchemy.api as
_validate_db_int() since it is related to database
only so that it can be used to validate max_rows
for maximum limit.
Closes-Bug: #1543937
Change-Id: Id16694807c180632c1785e9b1ebe8d1c79d885ab
validate_key_cert(): encode explicitly data to UTF-8 to avoid a
DeprecationWarning of OpenSSL.
crypto.sign() calls text_to_bytes_and_warn() to encode Unicode to
UTF-8 but it raises a DeprecationWarning. Encode in
validate_key_cert() to avoid the warning.
Example of warning:
glance/common/utils.py:427: DeprecationWarning: str for data
is no longer accepted, use bytes
crypto.verify(cert, out, data, digest)
Change-Id: Iae63dd6f5a208a8835e025618b989153056bc604
Added support filtering images based on lists using the 'in' operator.
Filters:
*id
*name
*container_format
*disk_format
*status
DocImpact
ApiImpact
Implements bp: in-filtering-operator
Change-Id: I9cac81b9d5cbec979e88cf2dd0e3b710ed45630c
Import i18n functions from module directly and do not use
global module variables like _ = i18n._. It makes code shorter
and cleaner. It also to detect cases when these functions are not
used in module.
Change-Id: Iaa593ac1f2dd15cbcad049bef6ba68f1cfa610da
Replace all the ":return:" with ":returns:"
Replace all the ":returns " wtih ":returns:"
Replace all the ":returns :" with ":returns:"
Add the changes for ":raise" also, change ":raise" to ":raise:"
Change-Id: I5d0def4665d0cd69b73c3c3826ad8f208e1e970e
Provide support in Glance API for querying the image list for
created_at and updated_at times using guidance from the API Working
Group recommendations for filtering. Filtering is applied at the DB
layer.
DocImpact
ApiImpact
MitakaPriority
Change-Id: Ie94295bb82779ec17ab773928c71ae4a9ee8fbcc
Implements bp: v2-additional-filtering
The current code builds and compile a closure function each time its
called, which is unneeded. Let's write a simpler version of that.
Change-Id: I1380344a1e23fe00f616eaf43a4eb572d2dcdeb4
* CooperativeReader: buffer is a byte string, use b'' syntax
and use bytes() instead of str().
* Don't compare None to int, it raises a TypeError on Python 3.
* Use str[i:i+1] to get a substring from a byte string. On Python 3,
str[i] now returns an integer for byte strings.
* tox.ini: add glance.tests.unit.common.test_utils to Python 3.4
Change-Id: Ie678555ffd8ac48913f9bf08ac674d1c643bec29
Dictionary creation could be rewritten as a dictionary literal.
for example:
filters = {}
filter['namespace'] = namespace
could be rewritten as
filters = {'namespace': namespace}
Closes-Bug: 1482586
Change-Id: I860483e958296da248a579a6032c14370843d9e4
The Catalog Index Service added in the kilo cycle has been split into a new
project named searchlight. This code now lives in a seperate repository:
https://git.openstack.org/openstack/searchlight
For more information about the split, see the governance change:
I8b44aac03585c651ef8d5e94624f64a0ed2d10b2
DocImpact
UpgradeImpact
APIImpact
Change-Id: I239ac9e32857f6a728f40c169e773ee977cca3ca
* Replace glance.common.utils.exception_to_str() with
oslo_utils.encodeutils.exception_to_unicode().
* Remove exception_to_str() from glance.common.utils
* Add "from oslo_utils import encodeutils" and remove "from
glance.common import utils" (when utils is no more used)
* glance/api/v2/images.py: replace utils.exception_to_str() with
six.text_type() to cast the oslo_i18n message to Unicode
Glance exception_to_str() function was not compatible with Python 3,
whereas exception_to_unicode() is compatible with Python 3, it's in Oslo
common libraries and it's well tested.
This patch was first generated by a tool (modified version of sixer),
and then fixed to respect the PEP8 (especially the constraint of 80
columns).
Change-Id: I86008c8adc0c5664f96573c1015cc15e2d06e3e2
* Replace unicode with six.text_type
* Replace "raise a, b, c" with six.reraise(a, b, c)
* Replace dict.iteritems() with six.iteritems(dict)
* Replace StringIO.StringIO with six.BytesIO for image content
* Get AssertionError from builtins, don't use the exceptions module
which was removed in Python 3
* Open configuration file in text mode (not in binary mode) in
functional tests
* Use absolute imports in artifacts_sample and image_artifact plugins
instead of relative imports.
* Fix usage of __import__() function: the level parameter expects a
positive integer, use level=0, not level=-1
* Get the input function from six.moves: use raw_input() on Python 2 and
input() on Python 3. raw_input() was renamed to input() in Python 3.
* Unpacking parameters in function definition is no more possible:
unpack the tuple manually in test_glance_replicator.py
* On Python 3, socket.fromfd() now returns directly the right socket
type. It's not more needed to cast manually the result of
socket.fromfd().
For more information on Python 3, see:
https://wiki.openstack.org/wiki/Python3
Change-Id: Iaa1a392590154058eab4645fb288d1534f66e528
Image params such as checksum, min_ram and min_disk are validated.
Checksum is expected to be not longer than 32 characters,
min_ram/min_disk can't exceed pow(2, 31) - 1.
A method for mysql_int validation has been moved to glance.common.utils.
Change-Id: I78554ef1ba65fc11da3de56467d6d70bb7991787
Closes-Bug: #1454730
Closes-Bug: #1460060
In case of v1 api if min_disk or min_ram is passed with a negative value
to the image create command, it returns 400 response with error message
formatted to HTML entity.
Eliminated special characters (>=) from error message, so it will not
cause any encoding issues. Also with this change the error message for
v1 api will be identical to v2 api.
Closes-Bug: 1461776
Change-Id: I71a3cc3ed6bf14f3c0273df053667f1cbf99e98f
Glance Search clients currently don't have
anyway to know what "types" are available.
Search clients need to know the "type" to
include in the search request for the
particular resource.
Change-Id: I8baed7c5183752b1f52f8050ce321aacec701f1c
Closes-Bug: #1439061
Pick up changes to socket configuration on reload.
To avoid any interruption to request handling on reload we recycle the
original listening socket rather than closing it (if at all possible).
Closes-bug: 1436272
Change-Id: Ibf0ad60a3a50e60b1bc8b32f9a2e567079a6f821
Abhishek Kekane