In 2.6.0, pycodestyle is going to find more E741 issues[1], which is
going to break the gate by making "tox -epep8" fail.
[1] https://www.flake8rules.com/rules/E741.html
Change-Id: I94ee03a00dd01726de15a29424183faabdeba844
It was observed that md-tag-create-multiple
(/v2/metadefs/namespaces/{namespace_name}/tags) API overwrites
existing tags for specified namespace rather than creating new one
in addition to the existing tags.
This patch resolves the issue by introducing a header 'X-Openstack-Append'
which on being True will append the new tags to existing ones and
if False will continue to overwrite the tags.
Implements: blueprint append-tags
Closes-Bug: #1939169
Change-Id: I29448746b14c542e5fbf0283011968ae1516642e
As noted by lbragstad, we need to make ImageTarget contain a member
field so that we can generically apply policies and be able to
properly include images for which we are a member. This was hacked
into place for ImageRepoProxy.get() but in order to apply it generally
to listing and other ops, we need to formalize that.
Partially-Implements: blueprint policy-refactor
Change-Id: I92d3792602a69922078d109095ad8ac9afc89d14
The db.ImageRepo.save() operation covers up a Forbidden error with
NotFound in the case of trying to update an image that the user does
not own. This is actually never hit in reality as the authorization
layer would have caught it before, and returned Forbidden. The API is
the layer that should be deciding to hide images for which the user
cannot see, to avoid things like being able to show an image, but get
back a 404 on update. In order to do that, we need the lower layers to
report the situation faithfully and let the upper layers decide how
to expose that to the user.
Specifically, for the policy refactor, we have tests that assert the
Forbidden behavior, and after disabling the authorization layer, will
break (and rightfully so) because they get NotFound when they hit the
lower-layer check. Because it is hidden so deep, even the API can't
distinguish between the two.
I imagine this check was added long ago to provide the "if you can't
see it, return NotFound instead of Forbidden, which would expose the
fact that it exists" behavior which is desired. However, the
authorization layer means we never get here anymore.
This patch is provided without any test changes to prove that it does
not actually alter the real behavior. The test_permissions functional
test asserts the proper behavior before and after this change, as well
as after subsequent refactor patches.
Change-Id: I0084350ebb09cc1cb3752b45165e49f166bfdf91
DB layer for metadef covers up a Forbidden error with NotFound in case
trying to perform any operation on namespace which user does not own.
This patch has removed this cover up so that end user will get
appropriate error message.
Change-Id: I9f05961c0b56efbede321271259848cfc96a98ee
Made provision to pass image_id, request_id and user_id information
while creating new task.
Partially-Implements: blueprint messages-api
Change-Id: I299a222eeef81431143db3ba7fc08365c924326b
The image_update() code will clobber, revert, or update lock values
in keys that we use as atomic properties. This adds an exclusion
list of properties that we handle specially and plumbs them down
to image_update() so that they will be excluded from the
add/update/delete logic.
Change-Id: Ib910274472346ce0c336cd1ead8370d5799d0b96
This adds a new DB API method to atomically delete a property on an image
in a way that we can be sure that it is deleted one and only once, and
without affecting the rest of the image. This can be used in conjunction
with image_set_property_atomic() to create and delete properties for locking
without the risk of clobbering other image data in the process.
Change-Id: I0b71a7df04cd330749f35b07f96a120b49b412c7
This adds a new DB API method to atomically create a property on an image
in a way that we can be sure it is created once and only once for the
purposes of exclusion of multiple threads.
Change-Id: Ifdb711cb241ef13eccaa5ae29a234f2fe4a52eb8
Related-Bug: #1884596
This patch removes majority of the registry and it's related
endpoints and config options that has been deprecated for
removal in various releases.
Change-Id: I75014bd50bf382efebe56bd89c20ffefbdde25f5
Added new boolean column "os_hidden" in images table. Images where
"os_hidden" = True will be omitted from the image list presented
to the user. This will apply to all image visibilities. However,
the images will continue to be discoverable. User can use
filter "os_hidden=true" in GET v2/images call to see all hidden
images.
Implements: blueprint hidden-images
Change-Id: If8f02ca94fdb8e1ac7a81853cd392988900172d1
Glare became a separate project during Newton. The code was copied
out of the Glance tree, but remained in the Glance repository. It
is no longer being maintained, and that has begun to cause some
problems, for example, blocking a recent stevedore upper constraints
change; see I141b17f9dd2acebe2b23f8fc93206e23bc70b568
This patch removes the Glare code from the Glance repository. It
includes Alembic database migration scripts, in both the all-in-one
and E-M-C format. It also includes release notes.
Implements: blueprint glare-ectomy
Change-Id: I3026ca6287a65ab5287bf3843f2a9d756ce15139
In glance v2, when one opts to use v2_registry, it is required that
'data_api' is set to 'glance.db.registry.api'. This is returned by
method 'get_api()' which currently simply returns whatever is provided
to 'data_api'. This is suitable for v2. But when using v1, this same
method is used to fetch the db api. This returns 'glance.db.registry.api'
which inturn relies on the registry rpc client (v2).
To prevent this, this patch proposes to change what get_api()
will return based on whether it is serving v1 api or v2 api.
Change-Id: Ifef36859b3f7692769a6991364b6063c9f7cc451
Closes-Bug: 1516706
This change replaces the existing boolean 'is_public' column for
the 'images' table with enum 'visibility' column featuring the
four explicit visibility values - public, private, shared,
and community.
This change also implements and enables all backend code to
utilize the new values.
Co-Authored-By: Timothy Symanczyk <timothy_symanczyk@symantec.com>
Co-Authored-By: Dharini Chandrasekar <dharini.chandrasekar@intel.com>
Implements: blueprint community-level-v2-image-sharing
Closes-Bug: #1394299
Closes-Bug: #1452443
Depends-On: I6e3268f3712cbc0aadb51d204c694023b92d55a5
Change-Id: I94bc7708b291ce37319539e27b3e88c9a17e1a9f
Fix update all props when you delete an image from the database.
Extra updating image increases the load on the db.
Added update status to 'deleted' or 'pending_delete'.
Closes-bug: #1539006
Change-Id: If58862139ad8a3a45cdb5d3e84eca6358d40608b
EXPERIMENTAL Glance v3 API has been removed in favor of standalone API
(EXPERIMENTAL Artifacts API of v0.1).
This patch introduces a new process entry point to run on a different
port (9494 by default), with its own configuration file and a paste
config.
A controller stub for old /v3 api remains in the glnace.api package for
the compatibility with existing paste configuration which may reference
it. This stub returns a 301 redirects to glare endpoint if it is present
or 410 errors otherwise.
To reuse the existing version_negotiation middleware some refactoring
has been made.
Implements blueprint: move-v3-to-glare
Change-Id: I5b7bd4cdcc5f2a40fc4a5f74bcc422fd700c4fb0
Image members CRUD doesn't generate notifications which
is impacting searchlight service by not having latest
changes to Image memberships.
If you create an image and later change its members,
the members are not updated via notifications.
You have to run the index sync again to get the updated
member list.
See: https://bugs.launchpad.net/searchlight/+bug/1490697
Membership information is critical for horizon filtering.
Typically, a person is allowed to view an image under the
following conditions:
1) The image is owned by the project I am currently logged into.
2) The image is public
3) The image is owned by another project which has added me
as a member and I have accepted membership to it.
Without current membership information, 3) above is not possible.
See: https://bugs.launchpad.net/searchlight/+bug/1491085
Change-Id: Ia56e42d3d8da36cfa419d5c3c7d69c9ccf8974fd
Closes-Bug: #1441453
Fix all assert problems raised by Bandit. Asserts are potentially
problematic, since Python optimization sometimes removes them, so code
needs to remain safe and functional without the assert.
Two asserts are safe to skip, so they are deleted for improved error
messages. Three asserts are probably necessary, and are converted to
exceptions. Two asserts are probably necessary, and are instead made to
fail safely, and `# nosec` is added to the assert line.
This also enables the assert test in bandit's configuration.
Change-Id: Ic69a204ceb15cac234c6b6bca3d950256a98016d
Partial-bug: 1511862
Import i18n functions from module directly and do not use
global module variables like _ = i18n._. It makes code shorter
and cleaner. It also to detect cases when these functions are not
used in module.
Change-Id: Iaa593ac1f2dd15cbcad049bef6ba68f1cfa610da
If glance backend database is not MySQL or postgreSQL,the unique
constraint of image-member only includes image-id and member.
If then an image-member is deleted, then create it again with the
same parameters, glance initiates a query to check if there is
already an existing one, but the result doesn't include the record
which was marked as deleted, glance will try to create a new one
with the same parameters, it will fail with SQL0803N error. To fix
this,we should check all existing image-member records including
the deleted image-member before create image-member, then update it
if it exists, otherwise create a new one.
APIImpact
Closes-Bug: #1462315
Implements: bp reuse-the-deleted-image-member
Change-Id: I84f88d133bf4ac6daa0ff5d148aed86c0ff2cb2d
* add(), save(): don't compare None to an integer, it raises a TypeError
on Python 3.
* _pop_task_info_values(): copy values items because the values
dictionary is modified in the loop body.
* replace filter() with a list-comprehension: filter() returns a
generator on Python 3 which has no length.
* tox.ini: add test_db and test_quota unit tests to Python 3.4
Change-Id: I1494e81b154fc64df814b5f96c15fe032bb5057b
Introduces a layered domain model for Artifact Repository designed
similar to the domain model of v2 Images: a number of proxies for
Artifact Objects, their Repositories and collections split into layers
by appropriate functional aspect.
The following layers are added:
* Database Repository layer - encapsulates DB APIs;
* Dependencies Layer - encapsulates dependecy management (artifact ids
are mapped to the actual Artifact References and back);
* Location Layer - encapsulates store interaction for Blobs (similar to
location layer of Images API);
* Updater layer - wraps the collection-based properties of Artifacts
for proper updates by JSONPatch calls.
Artifact-specific layers are added into "artifacts" subdirectory of
domain package. A gateway which creates layered proxy is added as well.
Implements-blueprint: artifact-repository
FastTrack
Co-Authored-By: Mike Fedosin <mfedosin@mirantis.com>
Co-Authored-By: Inessa Vasilevskaya <ivasilevskaya@mirantis.com>
Co-Authored-By: Alexander Tivelkov <ativelkov@mirantis.com>
Change-Id: I9b6d0e86c6577929230d58e7403fbefab167f36b
When an image can not be found an ImageNotFound exception
will be raised instead of a generic NotFound
exception. Does not apply to Image Metadata,
Image Members, Image Tags, Image Tasks
Closes-Bug #1247633
Change-Id: I84c754e51fb40ef1f9744bf85dbaa716b55cbb95
Right now if you want to get a single image, two queries need to be
executed:
- Get image + associated locations and properties
- Get associated tags for image
Proposed solution combines these two queries into single one which gets
image and all related data - locations, properties and tags. Similar
pattern has been already implemented when querying for list of images.
Change-Id: Ic4424bb1eb1769d8f621ebe111d95961ecf08479
Closes-Bug: 1434578
Extend rest images api v2 with multiple sort directions support.
Example:
/v2/images/detail?sort_key=name&sort_dir=asc&sort_key=size&sort_dir=desc
Changed database api which now can take sort_dir param as a list.
python-glanceclient support will be added in separate commit.
Implements-blueprint: glance-sorting-enhancements
DocImpact
APIImpact
Change-Id: Ib43b53abfba7cb5789d916a014376cf38fc5245b
Currently image data cannot be removed synchronously for an image that
is in saving state. And when, the upload operation for such an image is
completed the operator configured quota can be exceeded.
This patch fixes the issue of left over chunks for an image which was
deleted from saving status. However, by the limitation of the design we
cannot enforce a global quota check for the image in saving status.
This change introduces a inconsonance between http response codes of
v1 and v2 APIs. The status codes which we will now see after the upload
process completes on an image which was deleted mid way are:
v1: 412 Precondition Failed
v2: 410 Gone
SecurityImpact
UpgradeImpact
APIImpact
Closes-Bug: 1383973
Closes-Bug: 1398830
Closes-Bug: 1188532
Change-Id: I47229b366c25367ec1bd48aec684e0880f3dfe60
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
oslo_config was moved out of the oslo namespace in oslo.config>=1.6.0.
Related-to: blueprint drop-namespace-packages
Change-Id: I30ecbf8f9de77496fcb643c7ad9738d79ad359f0
Extend rest api v2 with multiple sort keys support.
Example:
/v2/images/detail?sort_key=name&sort_key=size
Changed database api which now takes sort_key param as a list instead of string
python-glanceclient support will be added in separate commit
Change-Id: Ib7a6aeb2df3bc5d23fe8e070290b5bfcab00c0f5
DocImpact
Partial-Bug: 1221274
The builtin _ should not longer be used. Instead _
should be explicitly assigned from glance.i18n.
This patch does the following:
(1) adds these explict assignments to any
glance file that was missing them,
(2) removes the defintion of _ as a builtin from
tox.ini so that a pep8 failure will occur
if _ is not defined,
(3) removes calls to install().
This removes the last direct use of gettextutils by glance.
The indirect uses, via openstack/common, will be removed
in the next sync with oslo.
Change-Id: Ie54e8a67e747e022cc01022dfad35d89686bdfc4
Several metadef module routines use:
from wsme.rest.json import fromjson
from wsme.rest.json import tojson
To follow the import rules at http://docs.openstack.org/developer/hacking/
(only import modules), these should be replaced with:
from wsme.rest import json
Change-Id: Idc496c8753d93a3d39f0c9cbbb31205948eeb081
Closes-bug: 1392485
The metadef_properties and metadef_objects tables both have
a column named schema. Unfortunately, schema is a reserved word
in some relational database products, including MySQL and PostgreSQL.
The metadef_properties.schema and metadef_objects.schema
columns should be renamed to a non reserved word.
Change-Id: I9c1b497d2b09b9282a83bd8c19c32edfa4dd159f
Closes-Bug: 1378968
Currently GET call to API to retrieve property details ends with
404 error when property name includes resource type prefix.
This patch extends show method to take filters as a parameter.
If 'resource_type' is included in filters then the prefix of included
resource type is removed from property name. This enables user to look
for property name starting with prefix that comes from associated
resource type.
Change-Id: I3c4d96fbc9ce15016631017bf76089c338ac3cdc
Closes-Bug: #1367564
DocImpact
Co-Authored-By: Bartosz Fic <bartosz.fic@intel.com>
Co-Authored-By: Pawel Koniszewski <pawel.koniszewski@intel.com>
The MetadefProperty and MetadefObject ORM classes currently specify the
JSON schema columns as type Text. It is preferred to use the
JSONEncodedDict Type Decorator instead. This fix also includes necessary
code changes to remove JSON encoding/decoding that was previously done
in other layers. Fixes for unit tests involving the schema columns are
also included.
Change-Id: I2c574210f8d62c77a438afab83ff80f3e5bd2fe7
Closes-Bug: 1368479
Implements: blueprint metadata-schema-catalog
A common API hosted by the Glance service for vendors, admins,
services, and users to meaningfully define available key / value
pair and tag metadata. The intent is to enable better metadata
collaboration across artifacts, services, and projects for
OpenStack users.
This is about the definition of the available metadata that can
be used on different types of resources (images, artifacts,
volumes, flavors, aggregates, etc). A definition includes the
properties type, its key, it's description, and it's constraints.
This catalogue will not store the values for specific instance
properties.
- REST API for CRUD on metadef namespace
- REST API for CRUD on metadef objects
- REST API for CRUD on metadef properites
- REST API for CRUD on metadef resource types
- REST API for JSON schemas on metadef API's
Change-Id: I8e6d88ffee9a9337bf82b1da85648ba638a154ab
DocImpact
Co-Authored-By: Lakshmi N Sampath <lakshmi.sampath@hp.com>
Co-Authored-By: Wayne Okuma <wayne.okuma@hp.com>
Co-Authored-By: Travis Tripp <travis.tripp@hp.com>
Co-Authored-By: Pawel Koniszewski <pawel.koniszewski@intel.com>
Co-Authored-By: Michal Jastrzebski <michal.jastrzebski@intel.com>
Co-Authored-By: Michal Dulko <michal.dulko@intel.com>
Abhishek Kekane