Commit Graph

170 Commits

Author SHA1 Message Date
wu.chunyang 93f35c7f04 Get rid of nose and Proboscis
they are not maintained for a long time, and trove already switch
the tests to templest.
This commit removes the usage of them. more details see story.

Story: 2010882
Task: 48606

Depends-On: https://review.opendev.org/c/openstack/trove/+/914228

Change-Id: Ie021e0bd708cf8ed13853dea632312af38190205
2024-03-28 07:07:15 +00:00
Zuul 50aca3d713 Merge "Use py3 as the default runtime for tox" 2024-01-16 02:15:44 +00:00
Hirotaka Wakabayashi 1126317f87 Disables shell=True in backup and restore
This PR disables shell=True in backup and restore.

Co-Authored-By: wu.chunyang <wchy1001@gmail.com>

Story: 2010004
Task: 45128
Change-Id: I13405e8bc93586441da28898e99dce936fc6d68a
2023-12-28 12:09:10 +09:00
likui 6c5fa1ddc3 Use py3 as the default runtime for tox
Moving on py3 as the default runtime for tox to avoid to update
this at each new cycle.

Change-Id: I54ac6b6c699c7799813653474234d039f775e1c2
2023-08-02 13:56:27 +08:00
Hirotaka Wakabayashi 0d4a24c71d Fixes cluster creation error
This PR adds the import statement to fix cluster creation error
and enables flake8-f821 rule checks.
F821 rule checks "undefined name"

Original problem:
we have currently got `NameError: name 'common_glance' is not defined`
error when creating a cluster because I failed to edit the patch[1].

Co-Authored-By: wu.chunyang <wchy1001@gmail.com>

[1]: 2f52b2fddb

Story: 2010151
Task: 45797
Change-Id: I846f433ff35a9638a070382bf89e02db617dcd3d
2023-06-30 14:32:32 +08:00
wu.chunyang 0fd7bf2ad9 Enable flake8 E129 rule
E129: Visually indented line with same indent
as next logical line (E129)
refer to: https://www.flake8rules.com/rules/E129.html

Change-Id: Ib34fe5f5eb2aa5bc4980ed67beb027d2bf7186e9
2023-06-19 10:38:10 +08:00
wu.chunyang 89514c0d5b Don't ignore E125 check
E125: https://www.flake8rules.com/rules/E125.html
Change-Id: I8156bb87ce3e4d1a19d2df9e2c3f67f789bffa46
2023-06-14 14:35:07 +08:00
wu.chunyang c1761147ba Don't ignore H306 pep8 check
H306: import in alphabeical order

Imports should be grouped in the following order:
standard library imports
related third party imports
local application/library specific imports
put a blank line between each group of imports.
https://docs.openstack.org/hacking/latest/user/hacking.html#imports

Change-Id: I2c188d27f0595ac3b2d71f4612d93829915e389c
2023-05-30 17:51:16 +08:00
wu.chunyang ad1e5fe464 Fix tox cover check
Change-Id: Id6cec4d2f94bcdbdf0954f1fe3be1ce4404c983c
2023-02-28 20:47:52 +08:00
wu.chunyang 7a62e826a4 Fixing tests with tox 4.2.6
1. Remove basepython from testenv
2. this change also fix unittests because stevedore has dropped pkg_resources[0],
after tox upgraded to 4.0.*, tox with usedevelop parameter installs trove in
develop mode, hence stevedore unable to get the entrypoints from sdist[1].
[0]: d5297167e0
[1]: https://tox.wiki/en/2.8.1/config.html#confval-usedevelop=BOOL

Change-Id: Ie65d099509e3cb7a61a09d975c58c1e95aa12b51
2023-01-18 09:57:00 +08:00
wu.chunyang 44c3cdef5c Rename app.wsgi to app_wsgi.py
1. app.wsgi is a python script and should be end with py extension.
2. Generate trove-wsgi script which is needed by kolla to support
https and ipv6.
3. Ignore build directory in git
4. update docs

Change-Id: Ibbd6ecb8db25aa3a443d3f526f4cf6d418eb26f3
2022-08-25 11:10:33 +08:00
wu.chunyang f9f3bca8f7 Add release note(aka. reno) guide
This PR adds reloease note guide for trove project and
renames some notes due to uid collision

Change-Id: I55f49425dc3df6599ff6714efd8f7c1dc656ddf2
2022-07-11 21:56:26 +08:00
wu.chunyang 2725612128 Add python3.9 support
Add python3.9 support and set the default test env
from py38 to py39

Change-Id: Ie1a4790762b80b8f60eda86b879b73b8a6005eba
2022-04-28 21:50:45 +08:00
likui 77bb97272e Changed minversion in tox to 3.18.0
The patch bumps min version of tox to 3.18.0 in order to
replace tox's whitelist_externals by allowlist_externals option:
https://github.com/tox-dev/tox/blob/master/docs/changelog.rst#v3180-2020-07-23

Change-Id: If39d3834724ff3116a2a5ee1ae0974148c339b79
2021-06-03 17:00:18 +08:00
zhangboye 070f82f8ca Dropping lower constraints testing
We facing errors related to the new pip resolver, this
topic was discussed on the ML and QA team proposed to
to test lower-constraints [1].

I propose to drop this test because the complexity and recurring pain needed
to maintain that now exceeds the benefits provided by this mechanismes.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019390.html

Change-Id: I7261114a545c8c016f96ecf20e3fcdc00c3bab99
2021-05-03 09:28:44 +08:00
Dmitriy Rabotyagov dfca9b1943 [doc] Add configuration reference
Config reference has been dropped for some reason several releases before.
We return configuration reference along with policy to the docs.

Change-Id: I0927466d18eca5572932fcf9e2ca6349b11ca847
2021-04-06 06:37:59 +00:00
likui 3753c70225 Update TOX_CONSTRAINTS_FILE
UPPER_CONSTRAINTS_FILE is old name and deprecated
-https://zuul-ci.org/docs/zuul-jobs/python-roles.html#rolevar-tox.tox_constraints_file
This allows to use lower-constraints file as more
readable way instead of UPPER_CONSTRAINTS_FILE=<lower-constraints file>.

[1] https://review.opendev.org/#/c/722814/

Change-Id: Ida407c05ad1026820be71ddfb84a97bcb3418df9
2020-10-30 09:04:53 +08:00
wangzihao e954184693 Remove six usage and basestring check
Remove basestring check.
Remove six Replace the following items with Python 3 style code.

- six.string_types
- six.int2byte
- six.indexbytes
- six.add_metaclass
- six.StringIO
- six.text_type
- six.integer_types
- six.binary_type
- six.BytesIO
- six.reraise

Change-Id: I4fb9033d152963c504ceb4d5c4d08f934ee4accb
2020-10-16 10:40:22 +08:00
likui d729fe9c2e bump py37 to py38 in tox.ini
in 'victoria' cycle, we should test py38 by default.

ref:
  https://governance.openstack.org/tc/reference/runtimes/victoria.html

Change-Id: I7947901ac31b170e334a51880b996d2d3143f310
2020-10-09 13:52:37 +08:00
Lingxian Kong 828e873846 Support backup strategy API
Change-Id: I0ddd7214dae6e29ddfaf045fdb282f4980a8afff
2020-07-17 11:35:53 +12:00
Lingxian Kong aa1d4d2246 Datastore containerization
Significant changes:

* Using docker image to install datastore.
* Datastore image is common to different datastores.
* Using backup docker image to do backup and restore.
* Support MariaDB replication
* Set most of the functional jobs as non-voting as nested
  virtualization is not supported in CI.

Change-Id: Ia9c97a63a961eebc336b70d28dc77638144c1834
2020-05-27 10:31:50 +12:00
Andreas Jaeger f33c213051 Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found by updated hacking version.

Update local hacking checks for new flake8.

Remove hacking and friends from lower-constraints, they are not
needed in installations.

Change-Id: I6e6638c07b5d24c90e9deae277e98149f383eb3d
2020-04-03 21:51:00 +02:00
Lingxian Kong 7d78c74158 Add python-troveclient in requirements.txt
Change-Id: I2bd0d81171df85c05423e19ab2e4afa82fcf880e
2020-03-15 20:57:39 +13:00
Andreas Jaeger ae2709dcd6 Small cleanups
Setup.cfg:
- This repo is python 3 only ,remove universal wheel building.
- Remove pbr.hooks.setup_hook, those are obsolete since a long time

tox.ini:
- remove tox:jenkins, this is not used since ages

Change-Id: Ia6c1bc28b4935859c29924141965f47c4bcbd2c3
2020-03-10 10:08:52 +13:00
Lingxian Kong 52c5646a9d Fix trovestack tox job
Change-Id: I9662c4c23f3d2879d78e26fe4b32a6f55b6cfb99
2019-12-19 13:54:26 +13:00
Lingxian Kong a0a10f0b94 Support HEALTHY status for db instance
- 'HEALTHY' means the db service is responsive, 'ACTIVE' means the db
  service is alive.
- Remove the CI job fakemodetests, but will add similar testing task in
  the future.
- Fix the periodic CI job
- Remove MongoDB and  related jobs

Change-Id: I5abe9091ba203297dc87db5fba139179166321f7
2019-12-14 12:55:56 +13:00
Ghanshyam Mann e6f55c57d5 [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in ussuri cycle.

Trove is ready with python 3 and ok to drop the
python 2.7 support.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: I6b448747c873bfc9c0138c0f7e3ca67ddce02dee
2019-12-05 22:38:19 +13:00
Corey Bryant cc16d3b3bc Add Python 3 Train unit tests
This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.

See the Train python3-updates goal document for details:
https://governance.openstack.org/tc/goals/train/python3-updates.html

Change-Id: I1fe23c6115bd6b9aac0bf2bd4fec9c6eaee88c40
Story: #2005924
Task: #34254
2019-08-15 09:50:49 +12:00
Lingxian Kong 37e8dedac2 Remove the trove-tox-apiexamples CI job
- The job relies on python-troveclient, sometimes python-troveclient
  only changes the URL sending to Trove but neither request nor
  response.
- The job is hard to maintain, especially when there are changes
  relating the API. The recommended way is to manually update the API
  doc and add tests in trove-tox-fakemodetests job.
- Remove that trovestack subcommand as well.

Change-Id: I108203915ef79ef93bbfbb15acdb7480854878e0
2019-06-06 15:57:26 +12:00
ZhongShengping b51f371d97 Update Python 3 test runtimes for Train
This goal is to implement the process set out in the 2018-10-24 Python
Update Process TC resolution[1], for the Train cycle to ensure unit
testing is in place for all of the Tested Runtimes for Train[2].
In practice, this generally means adding unit tests for Python 3.7 and dropping
unit tests for Python 3.5. Using the Zuul template for Train will ensure that
all projects that support Python3 will be tested against the agreed runtime
versions, and make it easier to update them in future.

[1]https://governance.openstack.org/tc/resolutions/20181024-python-update-process.html
[2]https://governance.openstack.org/tc/reference/runtimes/train.html

Change-Id: I018f79c6819ead9ffee777addb7ba6b369304962
Depends-On: https://review.opendev.org/#/c/641878/
2019-05-09 17:34:56 +08:00
ZhongShengping d9a63f7cb0 Update min tox version to 2.0
The commands used by constraints need at least tox 2.0.  Update to
reflect reality, which should help with local running of constraints
targets.

Change-Id: Ifc2c2ad88d77fb2590b5bb598458bdfd4a423a9e
2019-04-29 08:39:54 +08:00
ZhongShengping 4e63540245 Dropping the py35 testing
All the integration testing has been moved to
Bionic now[1] and py3.5 is not tested runtime for
Train or stable/stein[2].

As per below ML thread, we are good to drop the py35
testing now:
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005097.html

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004647.html
[2]
https://governance.openstack.org/tc/reference/runtimes/stein.html
https://governance.openstack.org/tc/reference/runtimes/train.html

Change-Id: I0d0a3f62d652853963dc4fe204a413eec2bc1830
2019-04-28 15:36:48 +08:00
ZhongShengping 6a5afea40a Move to opendev
1.Use opendev.org instead of git.openstack.org.
2.Use review.opendev.org instead of review.openstack.org.

Change-Id: I8e9da8fbb04bdec42a0b2f012f6a38b8ee413b27
2019-04-24 16:08:08 +08:00
Marcin Piwowarczyk e9cc6ca372 Execute functional test jobs running under python3
Functional tests are executed by trovestack script, which uses python
in default OS version which is 2.7.

This change wraps python in tox and executes trovestack int_tests in
virtual enviroment, starting them in python3.

Any future python version change could be managed by tox framework.

Change-Id: I3a849978241d3d0669ef9e1c802ff504ac3c32cb
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
2018-12-14 13:07:13 +01:00
Nguyen Hai Truong 7f867bbf41 Add python 3.6 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.6 as part of the python3-first goal.

See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: I2b04e865fb72296f672ab206fcda15e30450db97
2018-11-06 00:29:44 -08:00
Zuul c03ec08276 Merge "Fix cover job" 2018-10-05 17:19:13 +00:00
Andreas Jaeger 53576ec07a Fix cover job
The job is currently broken, it runs with testr while
we switched to stestr. Use the usual setup for coverage as done by
other projects using stestr in cover.

Previously, we run with concurrency=1 - continue to run in serial mode.

Change-Id: I6e389db5aed8db25b49906b2479af081237fc909
2018-10-01 20:32:12 +02:00
Zuul bfe0536cf9 Merge "make tox -e pylint only run pylint" 2018-09-25 07:19:42 +00:00
Doug Hellmann 2602ebc7e5 switch documentation job to new PTI
This is a mechanically generated patch to switch the documentation
jobs to use the new PTI versions of the jobs as part of the
python3-first goal.

See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html

Also, add docs tox environment so that the job works.

Change-Id: I8dd911d46623c0185d238b293e0218cfa435ed46
Story: #2002586
Task: #24342
2018-09-23 14:00:58 +00:00
Doug Hellmann 205512df1b make tox -e pylint only run pylint
Having the pylint environment run redundant commands just makes the
job take longer.

Change-Id: Ibb86b40e589d1b6f9c3af5901549d7aa5170e8c8
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-09-19 08:30:33 +02:00
Marcin Piwowarczyk ad84829882 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I9ee34642c700d1e6ba9c2f3891b7fa1f7f7e1e1d
Depends-On: I8989fd4798e80eae27408017e1543819a68b4ab1
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
2018-09-18 17:19:20 +00:00
Marcin Piwowarczyk 6c6d43eb1a Use latests version of python-troveclient in tests
This patchset changes version of python-troveclient package version
used in tests.
So far, version from global constrain file was used [1].

In order to achive this, a local constrain file for tests requirements
has been added. This made it possible to override global constrain
versions.

[1] openstack/requirements/upper-constraints.txt

Change-Id: Ic4383f0249e5bd367461f63f66217af959aefba4
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
2018-09-14 10:33:29 +02:00
Zhao Chao d2868a59cb Migrate to Zuul v3 native job definitions
This patch migrates the legacy gate jobs to Zuul v3 native ones(a new
trove-devstack role is created for all those legecy jobs, and the
previous way of running gate jobs is still kept as we still cannot
migrate the genade job to a Zuul v3 native one), following the
documents below:
https://docs.openstack.org/infra/manual/zuulv3.html
https://docs.openstack.org/devstack/latest/zuul_ci_jobs_migration.html

Change-Id: I10a1adae3a79b7322118fce180119f0ee5ff2e1b
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
2018-07-17 10:29:55 +08:00
Zhao Chao df7362702a Cleanup testrepository and os-testr requirements
We have migrated to stestr for a while, however the related requirements
is not updated, this patch does the cleanup, also updates requirements
for stestr.

Change-Id: I9c8cf2f90ced2261293f91faefc3099740540fe7
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
2018-04-18 10:08:09 +08:00
Zhao Chao 0c9a4fd659 Run unittests under the low-constraints job
The low-constraints job should run unittests by the original desgin, we
didn't put the commands about running unittests inside the default
testenv settings, so in fact the low-constraints job did nothing except
creating a virtualenv.

This patch fixes and also move the unittests commands to the default
testenv settings like the other OpenStack projects.

Change-Id: Ia344a14a9baf7574b043fc08332869083da6dc1e
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
2018-04-16 16:59:54 +08:00
Doug Hellmann 6eaa305cf6 add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: I5352398603e5755d46c9c033b05af637e379eea0
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-03-22 19:07:53 -04:00
Zhao Chao 8ce9d735d2 Fix guestagent.test_operating_system for Python3
* Tests for _write_file_as_root failed because tempfile.NamedTemporaryFile
  use binary mode by default, changing to text mode fixes.
* Migrate to olso_serialization.base64 for Base64Codec. Also change the
  return value from bytearray to bytes for Base64Codec.deserialize.
  Base64Codec supports reverse encoding(i.e. binary data will be written
  to the dest file), in this situation, the dest file should be opened
  with binary mode(and when reading from the file, binary mode should
  also be used).
* stream_codecs.StringConverter converts iterable objects with map
  function. The behavior of map function is different under Python 2.x
  and 3.x. However csv.writerows(before Python 3.5) and unpack_singleton in
  trove.common.utils both need a list object. Converting the return
  value of map function to a list explicitly before passing to these
  functions fixes.

guestagent.test_operating_system is the last blacklist regex pattern for
py3 unittests. With the above problems fixed, blacklist-py3.txt is not
needed any more, and tox.ini is also updateted.

Migrating from ostestr to stestr is also done while updating tox.ini.
Here is the ML post for more information about os-testr and stestr:

http://lists.openstack.org/pipermail/openstack-dev/2017-September/122135.html

Partially implements: blueprint trove-python3

Change-Id: I31f1f97901d6ebff8a91c1b70a343e724ab806eb
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
2018-03-01 10:52:26 +08:00
npraveen35 d6bd37f3f3 Remove install-guide tox env
The install-guide tox env is not used any more after docs migration.

Change-Id: I93c53d02afbbb36d4a3e5641e6d11aab9dcd7d8a
2018-02-22 17:45:22 +00:00
Zhao Chao 71ebd353ca Generate policy sample file automatically.
A new entrypoint in setup.cfg and a config file are added for
using olso.policy helper script to generate the sample file.

A new tox target also is added to simplify the environment
setting up. Now policy sample file can be generated
automatically, so the in-repo sample file is no longer needed.

Co-Authored-By: Andrew Laski <andrew@lascii.com>
Partial-Implements: blueprint policy-in-code
Change-Id: Ic336fa154ccc05b5e9db3a8e751a484b1cc5aa9c
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
2018-02-12 18:11:22 +08:00
Zhao Chao 690aa0c23f Add bandit-baseline check job.
As bandit-baseline check just call tox to run tests, adding it to
in-tree checks instead of gate jobs.

Closes-Bug: #1618549
Change-Id: If6848c69e09117789d566ae47977034d5503fa09
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
2018-02-07 18:10:30 +08:00