These URLs are now in the form docs.openstack.org/$project/latest. Make
sure to use always use https for these URLs.
Change-Id: I4a75bb6097399efca373d2f9a0e8697eb92312a7
It was observed that md-tag-create-multiple
(/v2/metadefs/namespaces/{namespace_name}/tags) API overwrites
existing tags for specified namespace rather than creating new one
in addition to the existing tags.
This patch resolves the issue by introducing a header 'X-Openstack-Append'
which on being True will append the new tags to existing ones and
if False will continue to overwrite the tags.
Implements: blueprint append-tags
Closes-Bug: #1939169
Change-Id: I29448746b14c542e5fbf0283011968ae1516642e
We also remove hacking tests for this, along with those for iterkeys and
itervalues (no usage of these).
Change-Id: If5b46580078eb756651ac6118f502eccdc693646
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The import image api now supports a list of stores to import data into.
This list can be specified through a new "stores" field that has been
added to the request body.
During import stage, Glance iterates overs this list and send the data
to each store one by one.
If an invalid backend is requested by the user, an exception is raised.
If an errors occurs during verify, already pushed data is removed and
image state is unchanged.
Change-Id: Id3ac19488c0a693d7042be4a3c83f3b9f12313d0
Implements: blueprint import-multi-stores
In HTML documents with openstackdocstheme, vertical lines are shown
at the left side for literal blocks. Indented blocks are considered
as literal blocks in ReST text.
Unnecessary indented blocks are found in the glance document and
it leads to blocks with unexpected vertical left lines and
sometimes with unexpected fonts like [1].
Unexpected literal blocks are cleanup.
This commit also converts Definition lists in user/formats.rst and
user/common-image-properties.rst into the proper way in ReST text.
[1] https://docs.openstack.org/glance/latest/user/formats.html#container-format
Change-Id: I1b026f919bb22a59d23e3bb93bb7919d202a62fc
RequestContext.tenant and user fields are deprecated in favor
of project_id and user_id respectively.
This change modifies the glance.context.RequestContext constructor
to transition usage of tenant/user to project_id/user_id until
all tests are moved over to the new attributes. Runtime usage of
the old fiels is updated.
To prevent new code from using the deprecated fields, a warnings
filter is added which will make tests fail if they hit code using
the old fields.
Co-Authored-By: Abhishek Kekane <akekane@redhat.com>
Change-Id: I351380840308a24769ece93abc6d1a9a6d6aa06f
Made provision for multi-store support. Added new config option
'enabled_backends' which will be a comma separated Key:Value pair
of store identifier and store type.
DocImpact
Depends-On: https://review.openstack.org/573648
Implements: blueprint multi-store
Change-Id: I9cfa066bdce51619a78ce86a8b1f1f8d05e5bfb6
This change replaces the existing boolean 'is_public' column for
the 'images' table with enum 'visibility' column featuring the
four explicit visibility values - public, private, shared,
and community.
This change also implements and enables all backend code to
utilize the new values.
Co-Authored-By: Timothy Symanczyk <timothy_symanczyk@symantec.com>
Co-Authored-By: Dharini Chandrasekar <dharini.chandrasekar@intel.com>
Implements: blueprint community-level-v2-image-sharing
Closes-Bug: #1394299
Closes-Bug: #1452443
Depends-On: I6e3268f3712cbc0aadb51d204c694023b92d55a5
Change-Id: I94bc7708b291ce37319539e27b3e88c9a17e1a9f
The usage of oslo_messaging.get_transport is not meant for
notifications; And oslo_messaging.get_notification_transport is
favored for those means. So this change introduces the usage of that
function.
If the settings for the notifications are not set with the
configuration that's under the oslo_messaging_notifications group,
this will fall back to the old settings which are under the DEFAULT
group; just like this used to work.
Closes-Bug: #1567231
Change-Id: I137d0f98533d8121334e8538d321fb6c7495f35f
The virtual_size property was added to the v2 API in Icehouse as a
"core" image property [0], but the field was not added to image
notifications. This patch addresses that oversight.
[0] https://blueprints.launchpad.net/glance/+spec/split-image-size
Change-Id: I423719a88ba1ef17e7475ab5388fb1720a28011e
Closes-bug: #1554206
Image members CRUD doesn't generate notifications which
is impacting searchlight service by not having latest
changes to Image memberships.
If you create an image and later change its members,
the members are not updated via notifications.
You have to run the index sync again to get the updated
member list.
See: https://bugs.launchpad.net/searchlight/+bug/1490697
Membership information is critical for horizon filtering.
Typically, a person is allowed to view an image under the
following conditions:
1) The image is owned by the project I am currently logged into.
2) The image is public
3) The image is owned by another project which has added me
as a member and I have accepted membership to it.
Without current membership information, 3) above is not possible.
See: https://bugs.launchpad.net/searchlight/+bug/1491085
Change-Id: Ia56e42d3d8da36cfa419d5c3c7d69c9ccf8974fd
Closes-Bug: #1441453
This change introduces glance.common.timeutils that provides the
timeutils previously consumed from oslo_utils.
Oslo is deprecating some timeutils functionality which of Glance
depends on. Suggested replacement (isoformat) would break glance APIs
so it's cleaner to carry this functionality in Glance rather than
re-invent the wheel.
Co-Authored-By: Erno Kuvaja <jokke@usr.fi>
Co-Authored-By: Sabari Kumar Murugesan <smurugesan@vmware.com>
Change-Id: I91e1cc9a273249fd88749cecf21200f3f5e2bab1
Import i18n functions from module directly and do not use
global module variables like _ = i18n._. It makes code shorter
and cleaner. It also to detect cases when these functions are not
used in module.
Change-Id: Iaa593ac1f2dd15cbcad049bef6ba68f1cfa610da
As we agreed that we'll change the default notification exchange name
from 'openstack' to 'glance' in M release.
The openstack convention is to send out all the notifications to the
exchange with the same name as the project name, i.e. nova sends all its
notifications to the exchange 'nova', cinder sends all its notifications
to the exchange 'cinder', etc. In such way, other projects interested in
those notifications (e.g. ceilometer) can listen on different exchanges
for different notifications.
Change-Id: I658fb0896aecea4fd5db1a9d3a8bbb64a91a7224
Closes-Bug: #1467776
* Replace glance.common.utils.exception_to_str() with
oslo_utils.encodeutils.exception_to_unicode().
* Remove exception_to_str() from glance.common.utils
* Add "from oslo_utils import encodeutils" and remove "from
glance.common import utils" (when utils is no more used)
* glance/api/v2/images.py: replace utils.exception_to_str() with
six.text_type() to cast the oslo_i18n message to Unicode
Glance exception_to_str() function was not compatible with Python 3,
whereas exception_to_unicode() is compatible with Python 3, it's in Oslo
common libraries and it's well tested.
This patch was first generated by a tool (modified version of sixer),
and then fixed to respect the PEP8 (especially the constraint of 80
columns).
Change-Id: I86008c8adc0c5664f96573c1015cc15e2d06e3e2
Creating a metadef object without any properties associated with it
raises an Internal Server Error. It is easily reproducible by running
following glanceclient command:
glance --os-image-api-version 2 md-object-create --name obj_name
--schema '{"description": "some desc"}' OS::Software::DBMS
The metadef object itself is created successfully, but the notification
is not fired and the response has code 500. Proposed patch fixes it by
making sure that notification system won't iterate over None value.
Change-Id: I591801d80d9cb2738dc3d27d7719fcc53a3d0645
Closes-Bug: 1438154
Metadefinition resources - namespaces, objects, properties, tags and
resource types - don't provide any notification events when certain
operations are performed on them. This patch includes following events
that will be triggered when necessary:
* metadef_namespace.create - namespace has been created
* metadef_namespace.update - namespace has been updated
* metadef_namespace.delete - namespace has been deleted
* metadef_namespace.delete_properties - all properties have been removed
from namespace
* metadef_namespace.delete_objects - all objects have been removed from
namespace
* metadef_namespace.delete_tags - all tags have been removed from
namespace
* metadef_object.create - object has been created
* metadef_object.update - object has been updated
* metadef_object.delete - object has been deleted
* metadef_property.create - property has been created
* metadef_property.update - property has been updated
* metadef_property.delete - property has been deleted
* metadef_tag.create - tag has been created
* metadef_tag.update - tag has been updated
* metadef_tag.delete - tag has been deleted
* metadef_resource_type.create - resource type has been added to
namespace
* metadef_resource_type.delete - resource type has been removed from
namespace
Additionally new configuration option has been added to allow for
disabling either individual or group of notifications.
DocImpact
UpgradeImpact
Depends-On: Iaa771ead0114e3941667b1e07ff32472d2f77afd
Change-Id: Ie1635793d80188f8f7a07aea91b9f0842900ffa6
Implements: blueprint metadefs-notifications
Currently image data cannot be removed synchronously for an image that
is in saving state. And when, the upload operation for such an image is
completed the operator configured quota can be exceeded.
This patch fixes the issue of left over chunks for an image which was
deleted from saving status. However, by the limitation of the design we
cannot enforce a global quota check for the image in saving status.
This change introduces a inconsonance between http response codes of
v1 and v2 APIs. The status codes which we will now see after the upload
process completes on an image which was deleted mid way are:
v1: 412 Precondition Failed
v2: 410 Gone
SecurityImpact
UpgradeImpact
APIImpact
Closes-Bug: 1383973
Closes-Bug: 1398830
Closes-Bug: 1188532
Change-Id: I47229b366c25367ec1bd48aec684e0880f3dfe60
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
oslo_config was moved out of the oslo namespace in oslo.config>=1.6.0.
Related-to: blueprint drop-namespace-packages
Change-Id: I30ecbf8f9de77496fcb643c7ad9738d79ad359f0
The builtin _ should not longer be used. Instead _
should be explicitly assigned from glance.i18n.
This patch does the following:
(1) adds these explict assignments to any
glance file that was missing them,
(2) removes the defintion of _ as a builtin from
tox.ini so that a pep8 failure will occur
if _ is not defined,
(3) removes calls to install().
This removes the last direct use of gettextutils by glance.
The indirect uses, via openstack/common, will be removed
in the next sync with oslo.
Change-Id: Ie54e8a67e747e022cc01022dfad35d89686bdfc4
This patch adds support for random access to images to the API v2. It's
possible to partially access the image data by passing the offset where
the image should be fetched from and the size of the chunk expected.
Note that not all stores support random access to image data. In such
cases, the API will return an error mentioning the selected store
doesn't support such a feature.
DocImpact:
Content-Range header is now parsed to support random access to image
data.
Implements blueprint: restartable-image-download
Change-Id: Iade692a8197a9a1d9532711a5c572c505e5e7d20
This commits removes the old `store` package from glance and adopts the
usage of the new glance.store library. The library was designed to
preserve backwards compatibility as much as possible. In fact, most of
the changes in this patch are related to function args ordering and not
function renames or workflow changes.
Some changes that are worth mentioning:
1. Glance store doesn't rely on a global config object. All config
options must be explicitly registered.
2. All store operations now accepted an optional context. This is a
fallout from the context not being required in the `Store` constructor
anymore.
3. Store drivers are behind a private package called `_drivers` and
they're not suppose to be accessed directly. Instead, functions like
`get_store_from_scheme` should be used.
4. Stores are disabled by default
5. All the store specific options are under the `glance_store` group.
DocImpact:
The old store related configuration options have been moved under the
`glance_store` section. However, the old options will go through a
deprecation path. That is, they'll still be read from the `DEFAULT`
section to give deployers enough time to update their config files.
In k-2, the deprecated options will be completely obsolete.
Closes-bug: #1291848
Implements-blueprint: create-store-package
Change-Id: Iaacc70993ad5da292b93de42bbecda73d53b19fd
*) Add osprofiler wsgi middleware
This middleware is used for 2 things:
1) It checks that person who want to trace is trusted and knows
secret HMAC key.
2) It start tracing in case of proper trace headers
and add first wsgi trace point, with info about HTTP request
*) Add initialization of osprofiler at start of serivce
Set's olso.messaging notifer instance (to send notifications to
Ceilometer)
*) Add tracing support for sqlalchemy
*) Add profiler CONF group that has to options:
1) Enable or disable profiler
2) Enable or disable sqlalchemy tracing
(the reason why we put this in Conf options, is that usually
db requests create a lot of trace info)
*) Glance-registry wasn't setted properly to send notifications.
To test it you need 2 things:
1) Enable ceilometer in devstack:
http://paste.openstack.org/show/85724/
2) Initialize osprofiler and send out API request. You can use
one of follow approaches:
a. Run this script (pls provide proper credentials)
http://paste.openstack.org/show/85722/
b. Use the function https://review.openstack.org/#/c/111184/
added from glanceclient CLI directly.
DocImpact
Change-Id: I45a19f5eb5304c2b78a9e12cbc0744941a807304
Co-author: Zhi Yan Liu <zhiyanl@cn.ibm.com>#
Use utils.exception_to_str to consistently cast the
exception to avoid the unicodeError stack trace.
Closes-Bug: #1340002
Related-Id: I54fce91c73adca846ec99ed50a57b95d16428ec1
Change-Id: I77478a8eeeefd85bf98138156429df53d0a4c079
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
If a user is not allowed to download an image because of a policy,
glance v2 API responds with a HTTP 200 status and no data instead
of HTTP 403 for no cache only.
The problem is that get_data implementation for notification
proxy is a generator, this situation delays the other proxies
get_data calls (including the policy proxy) for the first time
data is retrieved.
Hence, there is a delay in enforcing policy, so 200 is sent before
the API gets the chance to catch the policy exception.
DocImpact
Closes-Bug: #1326781
Change-Id: I1e50a069a6b7f9eed7160cd5908a5fa30274e227
The notifier_strategy option is deprecated since we merged commit
I8cd84772bc5867e06b2a50ed7e15b9e86f0b94ad during the Icehouse cycle.
Assuming the usual single cycle deprecation period, this can now be
removed.
DocImpact: the notifier_strategy has been removed in favor of the
notification_driver and related oslo.messaging options.
Change-Id: I3ab0133b697a820536e5da84e30b078915a29f6c
In order to handle the deprecated notifier_strategy configuration option
we just need to translate the notifier_strategy value as follows:
default -> transport_url=None, driver='noop'
noop -> transport_url=None, driver='noop'
logging -> transport_url=None, driver='log'
rabbit -> transport_url=rabbit:///, driver='messaging'
qpid -> transport_url=qpid:///, driver='messaging'
None -> transport_url=None, driver=None
where the last bit simply means that the user hasn't specified any
notifier_strategy and we should just have oslo.messaging load a notifier
based on oslo.messaging's config options.
It really doesn't need to be any more complicated than that, and we
certainly shouldn't need to poke at what the user may have configured
for transport_url in their config file.
The reason this is important is that CONF.transport_url is a private
configuration option which soon will not be visible to users of the
oslo.messaging API. See I9c3bf74f13a820ce295aa77a2bbcd781778b5926.
Also, simplify the testing of this stuff by mocking the oslo.messaging
APIs used and using the test scenarios outlined above. This also means
we stop poking around in oslo.messaging internals like:
self.assertEqual(str(nfier._transport._driver._url),
'qpid:///')
Naughty, naughty!
Note: it's arguable we should just remove notifier_strategy in Juno
since we deprecated it in Icehouse, but this patch will be useful to
backport to stable/icehouse in any case.
Change-Id: I067248974baee50a2a0afb38bf022b65d1bea57a
To support Python3, unicode() calls has been replaced by
six.text_type.
Added utils.exception_to_str(): is the proper way to convert
an exception to string, since it manages logic related to
encoding.
Change-Id: I27101390e4f95e5c7690b1b445b7e75b8bcb9a08
Closes-Bug: #1284677
Change I0b406e4662cfd3cb496b71be77ad10a90c178baa introduces a new
class TaskStub. This class is designed to be a subset of what a
task is in the domain layer: it doesn't contain input, message and
result. TaskStub should be used when listing tasks in order to avoid
extra-overhead of transporting these text fields.
As a result, the TaskDetails introduced by change
I0b406e4662cfd3cb496b71be77ad10a90c178baa doesn't need to exist.
With this patch, the domain layer contains Task and TaskStub instead
of Task, TaskStub and TaskDetails.
partially implements bp async-glance-workers
Change-Id: I2a20b0c5033c6920749370355c2d672dec707c28
Partial Task patch, Id I4fbadc9a97e3147128c7c733384c7bb50918806f
removed the result and message vars of the Tasks class. However,
the succeed and fail methods were using those to update them
respectively. This patch introduces a new TaskStub class which
enables the PartialTask functionality without looking result
and message attrs of the Task class.
Fixes bug 1284975
Change-Id: I0b406e4662cfd3cb496b71be77ad10a90c178baa
The current implementation always tries to get a transport from
oslo.messaging assuming the transport_url option has been set. This is
done to keep backwards compatibility. However, since the default
`rpc_backend` is rabbit, it'll always try to load such driver. The
problem raises when `kombu` is not installed and the `notifier_strategy`
is set to qpid. This will make glance-api fail because it'll try to load
the rabbit driver *before* loading the qpid one.
This patch puts the first load attempt in a try / except block. The
error caught (if any) will be re-raised *just* if transport_url has been
configured, otherwise it'll move forward and try to load the notifier
driver using the old configuration options.
The patch also removes an obsolete test.
Change-Id: I00e653704463ea8245e2960050e0d5e6839f278a
Closes-bug: #1302661