hacking 3.1.0 is too old.
Note:
We can't directly bump hacking to 6.x.0 (which is the latest major
version) because of the existing cap by diskimage-builder. The cap is
now being updated by [1].
[1] https://review.opendev.org/c/openstack/diskimage-builder/+/909336
Change-Id: I8778a7decc6669b4d95d6886c971433e7c34c5c8
Fix the following deprecation warnings.
* DeprecationWarning: Using the 'user' argument is deprecated
in version '2.18' and will be removed in version '3.0',
please use the 'user_id' argument instead
* DeprecationWarning: Property 'user' has moved to 'user_id'
in version '2.6' and will be removed in version '3.0'
Change-Id: I18802ae1e2318eae3c62a26459178b259c728ac4
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
The tenant argument of RequestContext in oslo.context had been
deprecated long time ago and it was finally removed in
oslo.context-4.0.0. We should remove the tenant argument of
TroveContext that derives from RequestContext and we should also
update the requirements.txt in the master branch.
Task: 44723
Story: 2009906
Change-Id: I69c7098cc0d61fbbba1dbf2eca87df0dd6fd70ba
So far, when Swift wasn't installed, there was no error handlig
in trove API.
This change adds LogsNotAvailable exception which is raised when
swift endpoints are not available.
Change-Id: I9a8a75b40bde90163560cdd758f17f9a98a41f0a
Story: #2005378
Task: #30360
Signed-off-by: Kasper Hasior <k.hasior@samsung.com>
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I9ee34642c700d1e6ba9c2f3891b7fa1f7f7e1e1d
Depends-On: I8989fd4798e80eae27408017e1543819a68b4ab1
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
New releases of oslo.config support a 'mutable' parameter to Opts.
oslo.service provides an option here Icec3e664f3fe72614e373b2938e8dee53cf8bc5e
allows services to tell oslo.service they want mutate_config_files to be
called by passing a parameter.
This commit is to use the same. This allows trove to benefit from
I1e7a69de169cc85f4c09954b2f46ce2da7106d90, where the 'debug' option
(owned by oslo.log) is made mutable. we should be able to turn debug
logging on and off by changing the config.
tc goal:
https://governance.openstack.org/tc/goals/rocky/enable-mutable-configuration.html
Co-Authored-By: Anusree A <anusree.a04:@gmail.com>
Change-Id: I02da8ce44d52b0f0dc67660c8ec76b5effb5f73c
* add Swift token verification in create instance method in Trove API
when slave_id is given
* catch ConnectionError exception in verify_swift_auth_token
* add Swift token verification to guestagent before backup and restore
* add new exception representing Swift connection error
* set missing fault information when replication snapshot fails
* mock verify auth token method in replication and restore unit tests
Closes-Bug: #1395523
Change-Id: I6a21ba2ba890a82875f9b6dae3c6b93bc9fdb4b0
Signed-off-by: Dariusz Krol <d.krol@samsung.com>
Starting with Python 3.6, invalid escape sequences in string literals
are now deprecated[1]. This influence also automatic style checkers like
pycodestyle which starting with 2.4.0 complains about invalid escape
sequences (W605)[2].
Let's fix all those warnings at once by using raw strings where possible
and adding additional \ where not.
Footnotes:
1 - https://docs.python.org/3/whatsnew/3.6.html#deprecated-python-behavior
2 - https://github.com/PyCQA/pycodestyle/pull/676
Change-Id: I009a366fd8342edfd30890df6fe8e1fca88bf3cc
Signed-off-by: Krzysztof Opasiak <k.opasiak@samsung.com>
When disabling root, there is no need to call guestagent to check
whether the root user is ever enabled. Root hisotry table should
be used for this purpose.
As datastore specific root controller of MySQL/Cassandra/PostgreSQL
were created only for the '_find_root_user' which were calling
guestagent to find root user, these controllers are removed and
'DefaultRootController' is used instead.
RedisRootController is also updated as it didn't do this check
previously.
Unittests directory structure is also slightly changed. It's more
clear to use similar directory hierarchies for testing and source
code, e.g.
trove/extensions/common/service.py
trove/tests/unitests/extensions/common/test_service.py
Change-Id: I9faac61d9650347b51f23e8fcaf5a92aed5fbf93
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
According to discussions on the ML, log messages should not be
translated any more. This patch also:
* removes all usage of _LI, _LW, _LE, _LC;
* updates log translation hacking rule.
ML discussions:
http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.htmlhttp://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html
The original commit messages are kept as following to make a historic
note:
remove unwanted log translations
recent discussions on the ML have led to the decision to eliminate all
translations for messages that are destined for log files but retain
them for messages that are going to be returned as exceptions and
potentially shown to end users via an API. see [1], [2].
This change does that as follows.
1. If a string is being used to generate an exception, it will still
be translated. Still Trove used both _LE and _ for these translations,
there are some cases where _LE has been simply changed to _, and not
removed.
2. If a string is used for a logging message, remove the use of _,
_LE, _LW, _LI.
Also, I have long felt that we have had a creep of checks in the pep8
tests that apparently make the code easier to read. I strongly believe
that these kinds of "easier to read" things make sense if they are
followed across all projects and not just gratuitously added one
project at a time.
I've taken this opportunity to reduce the merge mess caused by this
change, to sync up our ignore flags with a long list of ignores from
Nova. When they made the change for removing log translation, they
could do it in an automated-way like I have because they didn't have
to deal with under and overindented lines for visual edit (E127). Same
for Cinder.
Changes 448443 [3] and 447839 [4] were inadequate because they only
addressed a little part of the problem, namely removing the use of
_LE, _LI, and _LW, and I think this is a change we don't need to
dribble in a few files at a time. The changes are straightforward and
should be taken in a single lump to make it easy to deal with the
merges coming.
[1] http://lists.openstack.org/pipermail/openstack-operators/2017-March/012887.html
[2] http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html
[3] https://review.openstack.org/448443
[4] https://review.openstack.org/447839
Co-Authored-By: Amrith Kumar <amrith@amrith.org>
Co-Authored-By: Valencia Serrao <vserrao@us.ibm.com>
Change-Id: I5f86c982469e625997fc8bd15c6fae0fc77a8c64
The Oslo Policy library provides support for RBAC policy
enforcement across all OpenStack services.
Update the devstack plugin to copy the default policy file
over to /etc/trove in the gate environments.
Note: Not adding a rule for 'reset-password' instance
action as that API was discontinued years ago
and is now just waiting for removal (Bug: 1645866).
DocImpact
Co-Authored-By: Ali Adil <aadil@tesora.com>
Change-Id: Ic443a4c663301840406cad537159eab7b0b5ed1c
Implements: blueprint trove-policy
* HTTP body type must be bytes on Python 3:
* WsgiLimiterProxy, JSONDictSerializer: encode JSON as bytes
* Use byte strings for literal HTTP bodies
* Encode explicitly HTTP body to UTF-8 on Python 3
* Fix the prototype of FakeHttplibSocket.makefile(): only the first
mode parameter is mandatory
* Replace dict.keys().sort() with sorted(dict.keys())
* tox.ini: add the following unit tests to Python 3.4:
- api/common/test_extensions.py
- api/test_versions.py
Partially implements: blueprint trove-python3
Change-Id: I175dc55424e9b5786819f08a584ac967d083db11
Replaced basestring with six.string_types
Replaced (int, long) with six.integer_types
Replaced unicode with six.text_type
This patch is generated by the following tool using
'basestring', 'long' and 'unicode' options.
Manually replaced unicode(ex) with oslo_utils.encodeutils(ex)
https://github.com/haypo/sixer
Command:
python sixer.py -w 'basestring,long,unicode' trove/
Partially implements: blueprint trove-python3
Change-Id: I6fa42674060067663ef819247ea793ef3d8aa0da
This changeset handles the details of applying,
removing, listing and retrieving 'modules'
from Trove instances.
See https://review.openstack.org/#/c/290177 for
the corresponding troveclient changes.
Scenario tests have been extended to cover the
new functionality. These tests can be run by:
./redstack int-tests --group=module
A sample module type 'driver' - ping - is included
that simply parses the module contents for a
message=Text string and returns the 'Text' as the
status message. If no 'message=' tag is found, then
the driver reports an error message.
Due to time constraints, a few unimplemented
parts/tests of the blueprint have been triaged as bugs
and are scheduled to be fixed before mitaka-rc1.
These include:
Vertica license module driver:
https://bugs.launchpad.net/trove/+bug/1554898
Incomplete module-instances command:
https://bugs.launchpad.net/trove/+bug/1554900
Incomplete 'live-update' of modules:
https://bugs.launchpad.net/trove/+bug/1554903
Co-Authored-by: Peter Stachowski <peter@tesora.com>
Co-Authored-by: Simon Chang <schang@tesora.com>
Partially Implements: blueprint module-management
Change-Id: Ia8d3ff2f4560a6d997df99d41012ea61fb0096f7
Depends-On: If62f5e51d4628cc6a8b10303d5c3893b3bd5057e
This changeset handles the details of creating,
updating, listing and deleting Trove 'modules.'
Two new tables have been added to the Trove database:
modules
instance_modules
although the instance_modules table is at present unused.
Scenario tests have been written as well, to exercise the
new functionality. These tests can be run by:
./redstack int-tests --group=module_create
In the follow-up changeset, all module tests can be run
by:
./redstack int-tests --group=module
Since module support is available for all datastores
(controlled by a CONF option) the module test has been
added to the common modules group.
Note: Trying to do admin tasks with none admin
credentials results in an Unauthorized exception
being thrown, instead of Forbidden. This
is due to the fact that Forbidden is in the
HTTPUnauthorized section of wsgi.py instead of
the HTTPForbidden section. Moving the exception
caused too many failures, so I created a 'Module'
Forbidden exception and put it in the right section.
Change-Id: I755b0431b33b870ae02d903527f071fd8e23130d
Depends-On: I54d37025275dee4731ad49ebbd21612c4464e4c4
Depends-On: I779c24472d3d96a7b2fe4ed0284fd5869cdef93b
Partially-Implements: blueprint module-maintenance
Move pastedeploy[1] and local[2] of oslo-incubator modules to
the trove.common package because they are dropped.
Use base_exception module instead of the exception of oslo-incubator module.
Use i18n module instead of gettextutils.
Remove openstack.common package and openstack-common.conf file.
[1] I741f7d6d1327c9ce23ad5203d31667aff17e387f
[2] I271debac1e05a353f9876ced77970e6030912111
Change-Id: Ie289dac7a06704d5c55a5074c0afe50184e4d2f3
Remove the log module of oslo-incubator.
Move the WritableLogger wrapper to the base_wsgi module.
Add oslo.log to the requirements.
Change-Id: I724fa6090cebf40e7d7c78cc6b8458dfba9508a8
oslo.service has graduated, so trove should consume it.
Remove a "ticks_between_runs" parameter from the periodic tasks.
All periodic_tasks will be executed with the same interval with the value of
the "report_interval" option.
According to the old default value of the "report_interval" option and
the value of the ticks_between_runs paramentr (3), increase
the "report_interval" option to the 30 seconds.
Rename the "exists_notification_ticks" option to the "exists_notification_interval"
and increase the default value of it to 3600, according to the old value of
the "report_interval" option.
DocImpact
The option name and default value of exists_notification_ticks was changed to
exists_notification_interval with 3600 sec. And the new default value of
the report_interval option is 30 sec. The deployers need to make sure that
the new values in the seconds, not in the ticks.
Change-Id: Id14d28146f677faf017160ac93289fd119674fc4
Depends-On: Ia5b887e69853f39b387d309831fb7ce51b881149
Closes-Bug: #1466851
The bug specifically states that when trove starts up on 0.0.0.0 it
should be logged, but there is no specific reason that there shouldn't
be a log for admins to quickly reference what interface has the trove
port.
Change-Id: I1f6ba92d58096cff07f6e42391c59ed9fb07e0ac
Closes-Bug: 1350060
This rather useful debug tool in the bowels of common/wsgi.py produces
a misleading message because, most often, the matching_schema has no
property called 'name'. Fixed it to print the schema in its entirety
because in most cases one would take the name and go lookup the schema
anyway.
The other way around would be to go and change all of apischema.py and
add a name property. This seemed to be the less intrusive and
disruptive change.
Change-Id: I4075bc911d2cc07cb6e7d98ffcff96441f7b3416
Closes-Bug: #1460033
This patchset helps reducing the ignored pep8 errors.
Fixed some ignored hacking rules, as listed follows:
- E111 indentation is not a multiple of four
- E122 continuation line missing indentation or outdented
- E123 closing bracket does not match indentation of opening bracket's line
- E128 continuation line under-indented for visual indent
- E251 unexpected spaces around keyword / parameter equals
- E265 block comment should start with '# '
- E713 test for membership should be 'not in'
- H105 Don't use author tags
- H306 imports not in alphabetical order
Change-Id: Iadf2af4f6ec90420153ad63b5a41650392ef2cbd
The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names.
The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.
Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.
Blueprint: remove-namespace-packages
https://blueprints.launchpad.net/oslo-incubator/+spec/remove-namespace-packages
Change-Id: I9d3e80d94795060d375aae30ce249513aae3fd97
When creating a backup with no swift, the error message is very
general. Added an exception handler to provide a more meaningful
error message.
Closes-bug 1395107
Change-Id: I7cdc9ddd29d094493ce652425addfa461138675f
Explicitly specifying DatastoreNotFound exception to return http
status code 404. Previously we weren't specifying the status code
for the exception and were returning 400 by default. The unit test
depending on the previous status code has been updated to reflect
this change.
Closes-Bug: 1415031
Change-Id: I30bd97d53d1d51704133eb63e25f32a20cf0d9a1
PEP-0274 introduced dict comprehensions to replace dict constructor
with a sequence of key-value pair, these are benefits copied
from [1]:
The dictionary constructor approach has two distinct disadvantages
from the proposed syntax though. First, it isn't as legible as a
dict comprehension. Second, it forces the programmer to create an
in-core list object first, which could be expensive.
There is deep dive about PEP-0274[2] and basic tests about
performance[3].
Note: This commit doesn't handle dict constructor with kwagrs.
[1]http://legacy.python.org/dev/peps/pep-0274/
[2]http://doughellmann.com/2012/11/12/the-performance-impact-of-using-dict-instead-of-in-cpython-2-7-2.html
[3]http://paste.openstack.org/show/154798/
Change-Id: I45d0c289ecaf63a343fc9ad935cf2893d67d938a
This change is part of a multi-part change set to handle obsolete and
graduated oslo modules. This commit handles the wsgi module. The way
in which other projects appear to have handled this situation is to
copy the wsgi module into the project. See, for example,
https://review.openstack.org/#/c/129260/
I've done the same thing, and since we already have a wsgi module in
trove/common, I've called the newly imported module base_wsgi.py and
renamed the imports accordingly.
In the fullness of time, we could consider merging wsgi.py and
base_wsgi.py
However since there are internal dependencies between
middleware/context.py and wsgi.py (within oslo-incubator) I can't
delete or modify those files.
This change has been rebased on
https://review.openstack.org/#/c/133051/
Change-Id: I029b7f569342f768ebcdfc2fb7224f4f7f00d48c
Partial-Bug: #1380789
blueprint: retire-unused-trove-modules
This change is part of a multi-part change set to handle obsolete and
graduated oslo modules. This commit handles the gettextutils module
which is now oslo.i18n. The change here is to remove it from
openstack-common.conf and point all people who used to use that at
oslo.i18n. requirements.txt has been updated to point to this
dependency.
We aren't deleting gettextutils.py from openstack/common because it is
used by other oslo-incubator modules.
This change has been rebased on https://review.openstack.org/#/c/129668/
Change-Id: I39fd288fb29a8317f896812a2f64bcd69a6b8e21
Partial-Bug: #1380789
blueprint: retire-unused-oslo-incubator-modules
This change is part of a multi-part change set to handle obsolete and
graduated oslo modules. This commit handles the jsonutils module which
is now oslo.serialization.jsonutils. The change here is to remove it
from openstack-common.conf and point all people who used to use that
at oslo.serialization. requirements.txt has been updated to point to
this dependency.
The file jsonutils.py is not being deleted because of dependencies in
oslo-incubator.
Change has been rebased on https://review.openstack.org/#/c/129294/
Change-Id: I2a7d4c41ce77d6375fa08033514baa815a087016
Partial-Bug: #1380789
blueprint: retire-unused-oslo-incubator-modules
Reasons:
- A user is able to pass a backup that is larger
than the volume/disk of the instance can handle.
Changes:
- Add validation to backup size according to volume
or flavor size of the disk;
- Add tests.
Change-Id: Id549cff2e572a0ada632759bd9fcbbb67edf6ae0
Closes-Bug: #1304603
If the user attempts to delete an instance which has
replicas attached to it, forbid the delete.
Closes bug: #1362299
Change-Id: I644243aef74d9a34b7726139f9fa5ca8bdb1452c
adds clusters api, with mongodb sharding as the first implemenation.
Co-Authored-By: Michael Yu <michayu@ebaysf.com>
Co-Authored-By: Mat Lowery <mlowery@ebaysf.com>
Co-Authored-By: rumale <rumale@ebaysf.com>
Co-Authored-By: Timothy He <the@ebaysf.com>
Partially implements: blueprint clustering
Change-Id: Icab6fc3baab72e97f3231eaa4476b56b8dafb2a3
Adjust log messages to conform to logging standards.
Cleanup some messages that were unclear.
Change-Id: I0bed14a98477d7e2b83e43ce020b6b41e5ac8fa8
Partial-Bug: #1324206
According to the OpenStack translation policy available at
https://wiki.openstack.org/wiki/LoggingStandards debug messages
should not be translated. Like mentioned in several changes in
Nova by garyk this is to help prioritize log translation.
Change-Id: Ia38a426a1a3154302b19f2dc603487175566c863
Closes-Bug: #1318384
* Add catalog to TroveContext
* Populate context with catalog in middleware
* No new keystone calls made
* If *_url from conf available, it overrides catalog
* Following clients now use service_catalog:
* cinder
* heat
* nova
* swift
* Add new conf options (necessary to maintain parity with flexibility
of *_url in conf):
* os_region_name
* cinder_service_type
* heat_service_type
* nova_service_type
* swift_service_type
Closes-Bug: #1222891
Change-Id: I669bfd9fe63e538f9412f53723c6606b04c2e2ae
Added exception for not implemented exceptions
that indicates the operation name.
Added missing un-implemented methods to redis.
Change handling of remote errors from guest agent
api to only pass along the root exception message
rather than the full traceback
Change-Id: I0d4fdc5340bfb40370595636cc57cc5c87889507
Closes-Bug: #1276860
The XML api is no longer a requirement, as dictated by the TC.
implements blueprint destroy-xml-api
Change-Id: Ib4669155c19562ae1dda75cd30ad1a8f92b6be6a
Remove vim setting, Remove line containing:
comment - # vim: tabstop=4 shiftwidth=4 softtabstop=4
at the top of source code files, except for files in
openstack/common.
Change-Id: I5e2aad6dca8407890fc3c2afbda541bd4f124ea8
Closes-Bug: #1229324
The max header is exceeded in the following scenario
- Auth tokens built with a keystone v3 API catalog
- A catalog with approximately 8 or more endpoints defined
An equivalent fix will be required for all projects which use
eventlet.wsgi.
Closes-Bug: #1190149
Change-Id: Ia0882e43395f1025137173b51aab147433b54255
This allows users to make custom configurations groups and apply them to
multiple instances. Configuration parameters that can be set are listed
in the validation_rules.json file.
implements blueprint configuration-management
Change-Id: I99b0bfe51c131ace30774e75e30d620101ed2e0b
Takashi Kajinami