* Functional tests now use alembic instead of sqlalchmey-migrate
to build and destroy test database.
* All tests now use a file-based sqlite db as opposed to an in-memory
database.
Partially-Implements: blueprint alembic-migrations
Change-Id: I77921366a05ba6f9841143af89c1f4059d8454c6
Depends-On: Ie8594ff339a13bf190aefa308f54e97ee20ecfa2
The query added with CI change [0] was using the python 'and'
instead of sqlalchemy's 'and_()'. This patch changes that.
This patch also further refines the query to look specifically
for visility 'private' instead of '<> public' for accuracy during
setting of visibility to 'shared'.
[0] I94bc7708b291ce37319539e27b3e88c9a17e1a9f
Change-Id: I6851aa0e5ca8cecaff518609c14cd528bca95ade
This change replaces the existing boolean 'is_public' column for
the 'images' table with enum 'visibility' column featuring the
four explicit visibility values - public, private, shared,
and community.
This change also implements and enables all backend code to
utilize the new values.
Co-Authored-By: Timothy Symanczyk <timothy_symanczyk@symantec.com>
Co-Authored-By: Dharini Chandrasekar <dharini.chandrasekar@intel.com>
Implements: blueprint community-level-v2-image-sharing
Closes-Bug: #1394299
Closes-Bug: #1452443
Depends-On: I6e3268f3712cbc0aadb51d204c694023b92d55a5
Change-Id: I94bc7708b291ce37319539e27b3e88c9a17e1a9f
As downgrade are not supported after Kilo, we should remove them now.
Roll backs can be performed as mentioned in the below link:
http://docs.openstack.org/ops-guide/ops-upgrades.html#rolling-back-a-failed-upgrade
The DB downgrades were deprecated in Glance Mitaka release by commit
e3366afdfb.
Change-Id: I937d15d93f16a3e44a50e6ff1a469098eab67c79
Implements: blueprint remove-db-downgrade
In many places, there are more specific asserts
which can be used. I replaced the generic assert
with more specific ones, where it was possible.
This change enhances readibility, and on fail, more useful
message is displayed
Change-Id: I76dca80b6bd8886860fb383ee8d35e16cd99762e
There were some notes on the test_migrations that referred to swift
specific tests as S3 URIs. This commit simply corrects those notes.
TrivialFix
Change-Id: Iec8f905b31f80e3e94c3f5f0adc7e53de837ffdb
In https://review.openstack.org/#/c/117837/16 the file test_migrations.conf
was removed. Updating outdated comment.
Change-Id: I0644cb504392c4590f272baf6b03e483d42d6de2
Since we droped python 2.6:
* There is no need to set OrderedDict manually into SafeConfigParser
for compatibility with py26.
* Use py27 assertItemsEqual to assert equality of lengths and sorted
collections.
Change-Id: Ied749af72d29fbb4191142e23b69c4abe3782c99
EXPERIMENTAL Glance v3 API has been removed in favor of standalone API
(EXPERIMENTAL Artifacts API of v0.1).
This patch introduces a new process entry point to run on a different
port (9494 by default), with its own configuration file and a paste
config.
A controller stub for old /v3 api remains in the glnace.api package for
the compatibility with existing paste configuration which may reference
it. This stub returns a 301 redirects to glare endpoint if it is present
or 410 errors otherwise.
To reuse the existing version_negotiation middleware some refactoring
has been made.
Implements blueprint: move-v3-to-glare
Change-Id: I5b7bd4cdcc5f2a40fc4a5f74bcc422fd700c4fb0
This code fixes the situation when several
models are mismatched for SQLite because of
type inconsistencies between Integer and
BigInteger in sqlalchemy.
Change-Id: I52b3a0158db8e3dc48f19509d1f9f80420ee40ea
Closes-bug: #1526804
Closes-bug: #1526675
This change introduces glance.common.timeutils that provides the
timeutils previously consumed from oslo_utils.
Oslo is deprecating some timeutils functionality which of Glance
depends on. Suggested replacement (isoformat) would break glance APIs
so it's cleaner to carry this functionality in Glance rather than
re-invent the wheel.
Co-Authored-By: Erno Kuvaja <jokke@usr.fi>
Co-Authored-By: Sabari Kumar Murugesan <smurugesan@vmware.com>
Change-Id: I91e1cc9a273249fd88749cecf21200f3f5e2bab1
Fix all assert problems raised by Bandit. Asserts are potentially
problematic, since Python optimization sometimes removes them, so code
needs to remain safe and functional without the assert.
Two asserts are safe to skip, so they are deleted for improved error
messages. Three asserts are probably necessary, and are converted to
exceptions. Two asserts are probably necessary, and are instead made to
fail safely, and `# nosec` is added to the assert line.
This also enables the assert test in bandit's configuration.
Change-Id: Ic69a204ceb15cac234c6b6bca3d950256a98016d
Partial-bug: 1511862
Import i18n functions from module directly and do not use
global module variables like _ = i18n._. It makes code shorter
and cleaner. It also to detect cases when these functions are not
used in module.
Change-Id: Iaa593ac1f2dd15cbcad049bef6ba68f1cfa610da
Current Glance uses method CONF.set_override to change config option's
value with designated value in unit test, but never check if the
designated vaule is valid. Each config option has a type like strOpt,
BoolOpt, etc. StrOpt with parameter choices only allows values in set
of choices. In short word, each config option has limitation for type
and value. In production code, oslo.conf can ensure user's input is
valid, but in unit test, test methods can pass if we use method
CONF.set_override without parameter enforce_type=True even we pass wrong
type or wrong value to config option. This commit makes sure calling
method CONF.set_override with enforce_type=True,
Note: We can't set enforce_type=True by default in oslo.config now, it
may break all project's unit test. We can switch enforce_type=True by
default when all project fix violations like this commit.
Change-Id: I2defdfff3b14828133921e97acfea6b08c8fc20a
Related-Bug: #1517839
Some tests used incorrect order assertEqual(observed, expected).
The correct order expected by testtools is
assertEqual(expected, observed).
At some places, corrected argument order for assertNotEqual method as well
and replaced assertEqual with assertTrue/assertFalse wherever required.
Closes-Bug: #1259292
Change-Id: I6fff8a3eedd053d0de8e261245a32f9a9fdab213
Sometime during Kilo, the unique constraints on
metadef_namespaces (namespace)
metadef_objects(namespace_id, name)
metadef_properties(namespace_id, name)
metadef_tags(namespace_id, name)
metadef_resource_types(name)
were replaced with non-unique indices.
I believe this was done erroneously to make the migrate_repo/versions/scripts
match the db/sqlalchemy/models_metadef.py definitions. Unfortunately, the
schema scripts were correct with the unique constraints and what should have
changed was models_metadef.py.
This bug, puts one more migrate script in place which will rename any
duplicate records it finds to make them unique and then re-establishes
the unique constraints while dropping the non-unique indices. It also,
fixes models_metadef.py and adds in tests to attempt to create duplicates
which should result in an HTTPConflict.
Change-Id: Idf3569a27d64abea3ed6ec92fb77b36a4d6d5fd5
Closes-Bug: 1468946
oslo.db mandates all tables should be created with utf-8 charset
when using MySQL DB. We added an option (deprecated) to skip this
sanity check and it was subsequently removed in Liberty.
This patch fixes existing migration scripts to create tables with
utf-8 charset. Since oslo.db does a sanity check before running
the migrations, it's safe to assume that only new tables created
since the previous migration will be affected.
Closes-bug: #1279000
Change-Id: Ia31dc9e5d4494d1b9633f916fa6e2704d33dffb0
Utilizing assertIn and assertNotIn in a consistent way rather than
assertTrue( x <not> in y) and assertFalse(x in y).
Change-Id: Ic43d107c40a5a50b852ee476b1167c21e5ad86e8
* Replace unicode with six.text_type
* Replace "raise a, b, c" with six.reraise(a, b, c)
* Replace dict.iteritems() with six.iteritems(dict)
* Replace StringIO.StringIO with six.BytesIO for image content
* Get AssertionError from builtins, don't use the exceptions module
which was removed in Python 3
* Open configuration file in text mode (not in binary mode) in
functional tests
* Use absolute imports in artifacts_sample and image_artifact plugins
instead of relative imports.
* Fix usage of __import__() function: the level parameter expects a
positive integer, use level=0, not level=-1
* Get the input function from six.moves: use raw_input() on Python 2 and
input() on Python 3. raw_input() was renamed to input() in Python 3.
* Unpacking parameters in function definition is no more possible:
unpack the tuple manually in test_glance_replicator.py
* On Python 3, socket.fromfd() now returns directly the right socket
type. It's not more needed to cast manually the result of
socket.fromfd().
For more information on Python 3, see:
https://wiki.openstack.org/wiki/Python3
Change-Id: Iaa1a392590154058eab4645fb288d1534f66e528
PEP-0274 introduced dict comprehensions to replace dict constructor
with a sequence of key-pairs[1], these are two benefits:
- First, it makes the code look neater.
- Second, it gains a micro-optimization.
Glance dropped python 2.6 support in Kilo, we can leverage this now.
Note: This commit doesn't handle dict constructor with kwargs.
This commit also adds a hacking rule.
[1]http://legacy.python.org/dev/peps/pep-0274/
Co-Authored-By: ChangBo Guo(gcb) <eric.guo@easystack.cn>
Co-Authored-By: Kamil Rykowski <kamil.rykowski@intel.com>
Change-Id: I0ba408f9c616dcdb09618f6256db76b9facc0c1d
The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names.
The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.
Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.
Blueprint: remove-namespace-packages
https://blueprints.launchpad.net/oslo-incubator/+spec/remove-namespace-packages
Change-Id: Ifa8baab33cdb3e606cf175a8c29c3a4ef6c44480
Adds a Data Model (Tables: Artifact, ArtifactDependecy, ArtifactTag,
ArtifactProperty, ArtifactBlob, ArtifactBlobLocation), Migrations and DB
API for SQLAlchemy. Adds wrappers for simple and registry DB drivers.
Adds appropriate unittests to DB API tests and to migration tests.
Implements-blueprint: artifact-repository
Co-Authored-By: Alexander Tivelkov <ativelkov@mirantis.com>
Co-Authored-By: Mike Fedosin <mfedosin@mirantis.com>
Co-Authored-By: Inessa Vasilevskaya <ivasilevskaya@mirantis.com>
Change-Id: Ia491a58956101a1c40f1bca95cd9efe432f13dce
Adds new migration to fix an issue with an index created by
mysql.
Also includes patch I00da67ab06cd0a3c46aa8a6cd815d0559408c2f0,
because without it tests won't pass.
Change-Id: Iffb54d5ee404889d49c40dbe0401d61e55723872
Co-Authored-By: Mike Fedosin <mfedosin@mirantis.com>
Co-Authored-By: Andreas Jaeger <aj@suse.de>
Co-Authored-By: Oleksii Chuprykov <ochuprykov@mirantis.com>
Correct migrations to satisfy models_metadef.
Add nullable=False for json_schema in metadef_object
and metadef_property according to changes in
Change-Id: I2c574210f8d62c77a438afab83ff80f3e5bd2fe7
Set updated_at nullable=True due to problems
with setting default value NOW() for datetime
type for mysql (the same as in models.py).
Closes-Bug: #1365436
Change-Id: I5399923621913724914499aad9148a9410faa4ef
Mainly to improve consistency, use range() from six.moves
renames across glance.
Behaves consistently like py2 xrange() and py3 range().
Removes unnecessary range() from glace/api/v2/images.py
Change-Id: Id21f923d05600b902f2239e25ef01716c07e74a3
oslo_db was moved out of the oslo namespace in
oslo.db>=1.4.0.
Change-Id: I13620d26ff12c55b2294b3b051669e0e98513a10
Related-to: blueprint drop-namespace-packages
oslo_config was moved out of the oslo namespace in oslo.config>=1.6.0.
Related-to: blueprint drop-namespace-packages
Change-Id: I30ecbf8f9de77496fcb643c7ad9738d79ad359f0
Add classes for testing correspondence between migration scripts
and metadata.
We need oslotest in test-requirements due to bug#1356425
Partial-Bug: #1365436
Change-Id: I0db68ea3557dd9d214ea50ff7c96de1a47a58d4b
We must have correct models i.e. models that correspond
a database schema to use sqlalchemy features.
Update models.py and add migration script to correct
database schema and get rid of the difference between
schema and models.
Partial-Bug: #1365436
Change-Id: I9b4571906d39bcdb35048caa58d16ad5e888cce4
The builtin _ should not longer be used. Instead _
should be explicitly assigned from glance.i18n.
This patch does the following:
(1) adds these explict assignments to any
glance file that was missing them,
(2) removes the defintion of _ as a builtin from
tox.ini so that a pep8 failure will occur
if _ is not defined,
(3) removes calls to install().
This removes the last direct use of gettextutils by glance.
The indirect uses, via openstack/common, will be removed
in the next sync with oslo.
Change-Id: Ie54e8a67e747e022cc01022dfad35d89686bdfc4
The metadef_properties and metadef_objects tables both have
a column named schema. Unfortunately, schema is a reserved word
in some relational database products, including MySQL and PostgreSQL.
The metadef_properties.schema and metadef_objects.schema
columns should be renamed to a non reserved word.
Change-Id: I9c1b497d2b09b9282a83bd8c19c32edfa4dd159f
Closes-Bug: 1378968
Refactored migration tests to use OpportunisticTestCase, removed
unused code and ``test_migrations.conf`` file.
The main feature of this approach is to create a new database with
random name for each migration test. This will avoid migration tests of
race conditions and reduce tests intersection. After this change, database
``openstack_citest`` will be used only for initial connection to the database.
``test_migrations.conf`` file not required anymore, because we create test
database for migration test, so we no longer need to keep database credentials.
Partial-Bug: #1368274
Change-Id: Ib1d6dfae62cc60f814c01d07adc53f68e7c234f6
Implements: blueprint metadata-schema-catalog
A common API hosted by the Glance service for vendors, admins,
services, and users to meaningfully define available key / value
pair and tag metadata. The intent is to enable better metadata
collaboration across artifacts, services, and projects for
OpenStack users.
This is about the definition of the available metadata that can
be used on different types of resources (images, artifacts,
volumes, flavors, aggregates, etc). A definition includes the
properties type, its key, it's description, and it's constraints.
This catalogue will not store the values for specific instance
properties.
Change-Id: I01cc63df3d3abe383c94cfb54598868b4bb729bb
DocImpact
Co-Authored-By: Lakshmi N Sampath <lakshmi.sampath@hp.com>
Co-Authored-By: Wayne Okuma <wayne.okuma@hp.com>
Co-Authored-By: Travis Tripp <travis.tripp@hp.com>
Co-Authored-By: Pawel Koniszewski <pawel.koniszewski@intel.com>
Co-Authored-By: Michal Jastrzebski <michal.jastrzebski@intel.com>
Co-Authored-By: Michal Dulko <michal.dulko@intel.com>
According to the OpenStack translation policy available at
https://wiki.openstack.org/wiki/LoggingStandards debug messages
should not be translated. Like mentioned in several changes in
Nova by garyk this is to help prioritize log translation.
This patch adds a new hacking check - N319 - that ensures all
debug log messages don't have translations.
Change-Id: I9dd958b904671a7eb95883026e14684469dc52d5
Closes-Bug: #1317847
The following replacements were done in unit tests to have
clearer messages in case of failure:
- assertTrue(* is None) with assertIsNone
- assertTrue(* is not None) with assertIsNotNone
- assertTrue(* in *) with assertIn
- assertTrue(* not in *) with assertNotIn
- assertFalse(* in *) with assertNotIn
- assertTrue(* == *) with assertEqual
- assertTrue(* != *) with assertNotEqual
Change-Id: I0c47f991c3974e441335e71c9d26fab8a127f2ca
- handle all BadStoreUri exceptions raised by 'legacy_parse_uri'
function in
one location so that it becomes easier to log the error message with
'image_id'
- add tests for 'legacy_parse_uri'
Closes-Bug: #1243704
Change-Id: Ifc5de11832860ed51c1eb359d6f5cf78de8c0ba4
To keep Python 3.x compatibility, use six.moves.urllib.parse to
replace urlparse.
Partial-Bug: #1279611
Change-Id: Ibe5e028acd12e8d753d95cfd9c3f63072278b120
Hemanth Makkapati