Commit Graph

58 Commits

Author SHA1 Message Date
Takashi Kajinami a892465f19 pre-commit: Integrate bandit
We also remove these unnecessary linter dependencies from
test-requirements.txt.

Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
Change-Id: Ie797e48fb9d04649fa70e3d5c2f4161d123ed8f5
2024-02-06 15:18:54 +09:00
Hervé Beraud f104bc4411 tox - fix allowlist_external issues
Change-Id: I07d2126176d00acf361349fa6f8de6b34ae64a16
2023-01-27 10:28:13 +01:00
songwenping 92f0e8f29c Drop lower-constraints
Lower-constraints is not a requirement of the OpenStack Python PTI
[0] and there currently is a discussion on the mailing list [1]
about dropping the test, with the oslo team already having done
so [2].

The new dependency resolver in pip fails due to incompatible
dependency versions in our lower-constraints file, meaning that
we were never providing any real guarantees with it.

To unblock the CI, I am disabling lower-constraints job for now,
with the option to reenable it in case we fix the constraints,
and based on the outcome of the mailing list discussions and
consensus.

[0]. https://governance.openstack.org/tc/reference/pti/python.html
[1].
http://lists.openstack.org/pipermail/openstack-discuss/2021-January/019672.html
[2].
http://lists.openstack.org/pipermail/openstack-discuss/2021-January/019659.html

Change-Id: I04e49f005841eb341c2d53cbc469252c089d1ab5
2021-04-16 10:25:26 +00:00
Zuul 17be566d54 Merge "Use py3 as the default runtime for tox" 2021-01-21 17:22:25 +00:00
Hervé Beraud 0bb0d9abeb Use TOX_CONSTRAINTS_FILE
UPPER_CONSTRAINTS_FILE is old name and deprecated
This allows to use upper-constraints file as more
readable way instead of UPPER_CONSTRAINTS_FILE=<lower-constraints file>.

[1] https://review.opendev.org/#/c/722814/
[2] https://zuul-ci.org/docs/zuul-jobs/python-roles.html#rolevar-tox.tox_constraints_file

Change-Id: Ic4b4627cf0bbd401c870f813f2a5884950c7cb6a
2020-11-04 10:48:39 +01:00
Hervé Beraud a3d9c99ded Use py3 as the default runtime for tox
Moving on py3 as the default runtime for tox to avoid to update this at
each new cycle.

Wallaby support officially the following runtimes [1]:
- Python 3.6
- Python 3.8

During Victoria Python 3.7 was used as the default runtime [2] however this
version isn't longer officially supported.

[1] https://governance.openstack.org/tc/reference/runtimes/wallaby.html#python-runtimes-for-wallaby
[2] https://governance.openstack.org/tc/reference/runtimes/victoria.html#python-runtimes-for-victoria

Change-Id: I4bad27f2daab336437ae7a85c79a4928b9fe894c
2020-11-04 10:02:26 +01:00
Hervé Beraud ad42c89f3b Adding pre-commit
Introduced changes:
- pre-commit config and rules.
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.

Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.

pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker
  (check-byte-order-marker);
- Checks that non-binary executables have a proper
  shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings
  (check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
  calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)

For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks

Change-Id: I6663d5e0866e03f8eb8c0543d3e3e0bbfb2399ac
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
2020-09-08 14:21:18 +02:00
Sean McGinnis 3bb0513882
Bump default tox env from py37 to py38
Python 3.8 is now our highest level supported python runtime.
This updates the default tox target environments to swap out
py37 for py38 to make sure local development testing is
covering this version.

This does not impact zuul jobs in any way, nor prevent local
tests against py37. It just changes the default if none is
explicitly provided.

Change-Id: Ida22dd3800f4c18f4fb05040d4d7d6ace3e7dee4
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-24 10:25:49 -05:00
Cyril Roelandt 0a66500081 Switch to hacking 2.x
Change-Id: I2c34c98ba6c927a0b1048e998ff50e063e94b481
2020-02-07 15:53:21 +01:00
Stephen Finucane 2dd526d1ed Drop python 2.7 support and testing
Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: If6d6357501dfdff5a5edb53f24104e216607ea00
Sem-Ver: api-break
2020-01-30 16:15:33 +01:00
Hervé Beraud d22760ad49 tox: Trivial cleanup
Move 'basepython' to the top-level 'testenv'.

Change-Id: I7269b71668b7d37886bdfd02a188cc9d6fd0c958
2020-01-08 19:34:02 +01:00
caoyuan 3d4070e0fe tox: Keeping going with docs
Sphinx 1.8 introduced [1] the '--keep-going' argument which, as its name
suggests, keeps the build running when it encounters non-fatal errors.
This is exceptionally useful in avoiding a continuous edit-build loop
when undertaking large doc reworks where multiple errors may be
introduced.

[1] https://github.com/sphinx-doc/sphinx/commit/e3483e9b045

Change-Id: Icbc670ba49e0cf9e1ce33b33a16231ec64ddf042
2019-10-24 14:15:32 +08:00
pengyuesheng 3ff497d1cf Add Python 3 Train unit tests
See the Train python3-updates goal document for details:
https://governance.openstack.org/tc/goals/train/python3-updates.html

Change-Id: Iae43c960b980ba3c1aa83be8844ae2f873f9c186
2019-08-14 16:11:51 +00:00
Natal Ngétal d693dc2247 Modify the constraints url in tox.
Use the hotness for constraints.

Change-Id: I73e284393ce26c91ea675d5be213ff5e910b4dfe
2019-06-17 12:40:39 +02:00
Stephen Finucane 8c5317152b Avoid tox_install.sh for constraints support
We do not need tox_install.sh, pip can handle constraints itself
and install the project correctly. Thus update tox.ini and remove
the now obsolete tools/tox_install.sh file.

This follows https://review.openstack.org/#/c/508061 to remove
tools/tox_install.sh.

Change-Id: I67c50430ced6efcc2f29ec7d8adc871415922d6f
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-05-14 18:47:48 +02:00
Zuul 29671ef2bf Merge "Replace git.openstack.org URLs with opendev.org URLs" 2019-05-05 19:53:34 +00:00
jacky06 815df26f0d Replace git.openstack.org URLs with opendev.org URLs
Change-Id: I35110a02e8b16f9c1d3b44441409db990eb1a23f
2019-04-30 13:38:11 +08:00
Ghanshyam Mann 6742d1f1c1 Dropping the py35 testing
All the integration testing has been moved to
Bionic now[1] and py3.5 is not tested runtime for
Train or stable/stein[2].

As per below ML thread, we are good to drop the py35
testing now:
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005097.html

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004647.html
[2]
https://governance.openstack.org/tc/reference/runtimes/stein.html
https://governance.openstack.org/tc/reference/runtimes/train.html

Change-Id: Ib110f4a3c9475d2655dbf67d28474f54928b142a
2019-04-15 18:28:07 +00:00
ZhongShengping fa12452bbd Follow the new PTI for document build
For compliance with the Project Testing Interface as described in:

https://governance.openstack.org/tc/reference/project-testing-interface.html

For more detials information, please refer to:

http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html

Change-Id: I10537cf02d83f127b6932fa3f427043fb267d8b1
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
2018-08-10 16:01:51 -04:00
Stephen Finucane f6069e9e42 Migrate to stestr
This should both keep mtreinish happy and get rid of annoying 'db type
could not be determined' messages in one fell swoop.

Change-Id: I0db004a8c8d75359e7d08c73b82f98c62a7e5f7d
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2018-08-10 15:57:52 -04:00
Zuul c01fc8fc5c Merge "tox: Group targets and tool configuration together" 2018-06-15 23:50:00 +00:00
Zuul 1aca8a362a Merge "tox: Don't set basepython in testenv" 2018-06-14 22:17:57 +00:00
Doug Hellmann 2d0d08081b fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I5f2ebf16d2768811f55939a55bedf59b808e0b9e
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-06 14:53:49 -04:00
Stephen Finucane f41f9f77c9 tox: Group targets and tool configuration together
Easy readability win.

Change-Id: Ic93c57cf2ffd73d75a88bc1efc8d67ee163a6c1f
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2018-06-01 13:46:19 +01:00
Stephen Finucane 90e2181d15 tox: Don't set basepython in testenv
Turns out doing this overrides the basepython for every 'py{version}'
implicit target, meaning py35, py36 etc. will use the same Python 3.x
version. We don't want that so set basepython manually.

This will hopefully be a non-issue when the gate bumps to Bionic.

Change-Id: I8d47f9ede4e9d1d221f6643135407ad2d5cea282
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2018-05-30 14:54:29 +01:00
Kenneth Giusti 5a4c7cbcdf Remove stale pip-missing-reqs tox test
pip_missing_reqs tool is no longer maintained and has broken with
release 10 of pip

Refer to:
 http://lists.openstack.org/pipermail/openstack-dev/2018-April/130027.html

Change-Id: Iad38f39390fbdd33fc9c64bba44d72613d4f58d9
2018-05-01 09:57:43 -04:00
Doug Hellmann 4819030b40 set default python to python3
Set the default python to python3 except for the py27 environment. We
have to set that explicitly to override the new default.

Change-Id: Ibaf75b2da94c8ba4e0c1f4b3e60992c1a3f41001
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-04-13 16:01:46 -04:00
Doug Hellmann 28a3dad5dc add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: I8a91da1f0453f943501619b6c9bc56322f732675
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-03-24 21:02:30 -04:00
Eric Brown aed513015c Remove references to Python 3.4
Now that there exists only a gate job for Python 3.5 and not 3.4,
we should remove those references to the 3.4 that is untested.

Change-Id: I5b6a963a9fb0ee7c77ed28210bbf529ee5499334
2017-01-17 10:27:47 -08:00
Tony Breeds bdb5a07627 Add Constraints support
Adding constraints support to libraries is slightly more complex than
services as the libraries themselves are listed in upper-constraints.txt
which leads to errors that you can't install a specific version and a
constrained version.

This change adds constraints support by also adding a helper script to
edit the constraints to remove oslo.log.

Change-Id: Ib6001210350743827eae4f5306464e68e057d4c6
2016-12-20 13:10:56 +11:00
ChangBo Guo(gcb) 3ae0e8732c Add Python 3.5 classifier and venv
Now that there is a passing gate job, we can claim support for
Python 3.5 in the classifier. This patch also adds the convenience
py35 venv.

Change-Id: I4be37bf1a97bf76a2302a4f16999e6464665dc21
2016-07-12 15:46:54 +08:00
Eric Brown c0a1b7c892 Explicitly exclude tests from bandit scan
Now that the bandit linter runs without a bandit.yaml specified,
it should exclude the tests directory similar to how it did before
in the bandit.yaml.

The reason this has not caused a problem yet is because bandit is
still finding a config bandit.yaml in .tox/pep8/etc/bandit/bandit.yaml.
But in a newer version of bandit this will not be the case.

Change-Id: I4fd5b0cc60309dfc207cdf2025caae37836391da
2016-03-12 20:07:51 -08:00
ChangBo Guo(gcb) a010cde94c Enable bandit in gate
Run security linter bandit as part of pep8. Pep8 is the usual
linter target and thus let's use it there instead of starting
another short-running job to enable it.

Change-Id: I8b5dc551b51ea5b92d755927d811f9b60e87e4d2
2016-03-03 11:34:34 +08:00
Jenkins 8f6747911d Merge "Add reno for release notes management" 2016-02-18 20:49:12 +00:00
Jenkins 12ae4b35e8 Merge "remove pypy from default tox environment list" 2016-02-12 18:49:33 +00:00
Jenkins 6d3a2f2f89 Merge "Remove bandit.yaml in favor of defaults" 2016-02-11 18:56:52 +00:00
Ronald Bradford ad9f8fcddc Add reno for release notes management
Release notes now required to report removal of deprecated configuration
options from oslo.log.

Change-Id: I3649049a571490c5ec44a14a0c8640a4b747ad64
2016-02-11 18:26:08 +00:00
Doug Hellmann 17fcaea0c5 remove pypy from default tox environment list
Interest in supporting pypy has waned, and it's not clear that the tests
work there anyway. Remove it from the default list of environments where
we run tests. Note this doesn't change anything about how the check or
gate queues work in CI.

Change-Id: I8adb8b321287773ce7ddac828df82a6e6c0101d2
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2016-02-10 15:26:40 -05:00
Doug Hellmann c0afcf5cd6 always run coverage report
ignore coverage data files

Change-Id: I1a9d02f90a6f3782c64db42fe99ad86e8988a092
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2016-02-08 12:53:48 -05:00
Davanum Srinivas 292952d4e5 Remove bandit.yaml in favor of defaults
With latest bandit version, The bandit.yaml is only necessary
for advanced tweaking. We can just use the defaults.

Change-Id: Ic14f582b34f1477d81099777bcfa834811c6875a
2016-02-04 19:05:11 -05:00
Jenkins b374806beb Merge "Clean up removed hacking rule from [flake8] ignore lists" 2016-01-29 21:48:31 +00:00
Doug Hellmann 83ea58d71e Revert "Pass environment variables of proxy to tox"
It looks like you can work around this problem by setting an environment variable on your dev system. See https://tox.readthedocs.org/en/latest/config.html#confval-passenv=SPACE-SEPARATED-GLOBNAMES

This reverts commit fff86fa546.

Change-Id: Ibe6654f7a30360b49ce8826101c3d3f5addb060a
2016-01-14 17:58:35 +00:00
ChangBo Guo(gcb) cc7400ad55 Clean up removed hacking rule from [flake8] ignore lists
We bump hacking>=0.10.0, and hacking removed some rules, for
the full list of rules please see [1]. So don't need them any more.

Hacking related commits:
Remove H904 in commit b1fe19ebebe47a36b905d709467f5e82521bbd96
Remove H803 in commit f01ce4fd822546cbd52a0aedc49184bddbfe1b10
Remove H307 in commit ec4833b206c23b0b6f9c6b101c70ab925a5e9c67
Remove H305 in commit 8f1fcbdb9aa4fc61349e5e879153c722195b1233

[1]https://github.com/openstack-dev/hacking/blob/master/setup.cfg#L30

Change-Id: Ifcbb505053acfdb3d69caf088f34f391a72cb637
2016-01-14 16:16:26 +08:00
sonu.kumar fff86fa546 Pass environment variables of proxy to tox
Change-Id: I204512373d47672b65ed12ea337fca3e0b6ed46d
Related-Bug: #1465086
2015-12-14 10:29:59 +05:30
Cyril Roelandt b8be549a4a Add a 'bandit' target to tox.ini
All checkers are enabled, and we only need to silence one silence false
positive.

Change-Id: I829f2b2ef7e94b76b348c1e6ec6e4249009724c8
2015-11-20 19:53:21 +01:00
Davanum Srinivas 18abab76c6 Remove python 2.6 and cleanup tox.ini
* Remove support for python 2.6
* Remove skipsdist : Needs to be set only if 
  sdist is expensive
* Remove usedevelop : only needed when skipsdist 
  is set to True
* Remove install_command : We can just use the 
  default, we don't need to override
* Remove setenv : We can just use the default as
  we don't need any extra environment variables
* Remove requirements.txt from deps, as this is
  already added automatically
* sitepackages : default is already false


Change-Id: Ie2f14094bb896227021f3d0bb81f721032a0c531
2015-11-16 00:26:45 +00:00
Ronald Bradford ff6fd44f9c Fix coverage configuration and execution
A number of configuration errors prevent the successful creation of code
coverage. This corrects the .coveragerc source/omit setup and the tox
package name generation.

http://lists.openstack.org/pipermail/openstack-dev/2015-October/076193.html

Change-Id: Ie23fa6233a49e9db318b08d5a93e4dd7b0dae82d
2015-10-05 17:22:34 -04:00
Mark Doffman 362f329bcf Add optional 'fixture' dependencies.
Add extra 'fixture' section so that projects needing to use
oslo_log.fixture can add 'oslo.log [fixture]' to their
requirements to install the fixture dependencies

Closes-Bug: #1493976
Change-Id: I67f3f59c50627a41b672928d037a7bd9997caa11
2015-09-22 13:03:10 -05:00
Davanum Srinivas c2210a20dc Add tox target to find missing requirements
Change-Id: I8072f9f1bda796cc7a26e1617fbfe43d0421d656
2015-06-26 07:22:18 -04:00
Davanum Srinivas cc713ab5b1 Advertise support for Python3.4 / Remove support for Python 3.3
Change-Id: Ieed553003351e754a761361faee40c9efddc35b7
2015-05-10 19:18:06 -04:00