We also remove these unnecessary linter dependencies from
test-requirements.txt.
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
Change-Id: Id17f4865b2da9f4a5c1cd5ea360cd007315d1c0e
The fasteners lib in version 0.15.0 removed the
threading.current_thread workaround for eventlet[1] because eventlet
seemed to fixed the current_thread issues tracked in [2]. However the
fix for [2] only fixed half of the problem. The threading.current_thread
call works if it is called from thread created by eventlet.spawn.
However if the thread is created with eventlet.spawn_n then
threading.current_thread is still broken and returns the ID of the
python native thread.
The fasteners' ReaderWriterLock depends heavily on
threading.current_thread to decide which thread holds a lock and to
allow re-entry of that thread. This leads to the situation that
multiple threads created from spawn_n could take the same
ReaderWriterLock at the same time.
The fair internal lock in oslo.concurrency uses ReaderWriterLock and
as a result such lock is broken for threads created with spawn_n.
Note that this issue was raised with eventlet in [3] when the nova team
detected it via a direct usage of ReaderWriterLock in the nova test
code. As [3] did not lead to a solution in eventlet nova implemented a
nova local fix for the test code in [4].
However now we detected that oslo.concurrency is affected by this issue
as well.
This patch adds tests to show the scope of the problem.
Note that the coverage tox target is changed to explicitly enable native
threading otherwise it runs eventlet specific tests in a native
environment.
Also note that [5] was opened to reintroduce the workaround[1] in fasteners.
[1] 467ed75ee1
[2] https://github.com/eventlet/eventlet/issues/172
[3] https://github.com/eventlet/eventlet/issues/731
[4] https://review.opendev.org/c/openstack/nova/+/813114
[5] https://github.com/harlowja/fasteners/issues/96
Related-Bug: #1988311
Change-Id: Ibc193c855b49b95b46ebd2aac82ea89e33f885f0
Introduced changes:
- pre-commit config and rules
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.
Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.
pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker (check-byte-order-marker);
- Checks that non-binary executables have a proper
shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings (check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)
For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks
Change-Id: I6ae02d4e52fd812360414e7b8c1c9c4f17230812
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
Python 3.8 is now our highest level supported python runtime.
This updates the default tox target environments to swap out
py37 for py38 to make sure local development testing is
covering this version.
This does not impact zuul jobs in any way, nor prevent local
tests against py37. It just changes the default if none is
explicitly provided.
Change-Id: Ia2304240ac09a756249b5689e7fff9c829c7bc9a
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.
Blacklist: W504 line break after binary operator
Change-Id: I8bca364fdc4c09cbd34127de10fa31da47078b4f
Update the minversion parameter to use the python -m pip to install
python packages:
https://tox.readthedocs.io/en/latest/changelog.html#id185
Also Inherit the jobs requirements to simplify maintainance.
Change-Id: Icb5116908148dcbb58a7531bbed498d29ca1d2ea
Currently, we are overriding 'install_command' to use 'pip'. This is
considered poor behavior and 'python -m pip' should be used instead:
https://snarky.ca/why-you-should-use-python-m-pip/
It turns out that this is the the default value provided by tox:
https://tox.readthedocs.io/en/latest/config.html#conf-install_command
So we can remove the line and simply use the default value.
Also these changes introduce a new explicit env var related to eventlet
testing to avoid misunderstand.
Change-Id: I03b9b179c4cb6ea5a9a8d8c74af3502a0243ace6
Sphinx 1.8 introduced [1] the '--keep-going' argument which, as its name
suggests, keeps the build running when it encounters non-fatal errors.
This is exceptionally useful in avoiding a continuous edit-build loop
when undertaking large doc reworks where multiple errors may be
introduced.
[1] https://github.com/sphinx-doc/sphinx/commit/e3483e9b045
Change-Id: Id12730272782a7cfda89cba79d40919e2a5ecd24
- Tests are based on oslotest.base.BaseTestCase, which uses the
NestedTempFile fixture, which uses the TempDir fixture, which adds a
cleanup routine to remove the base temporary directory it creates.
There's therefore no need for tests to clean up this directory, so all
the code that does that is removed.
- The eventlet incarnation of tests was trying to make use of the `env`
external without whitelisting it, resulting in an ugly red deprecation
warning. This commit adds `env` to whitelist_externals in [testenv].
- A handful of typos in the Beowulf quote are corrected.
Change-Id: I91cc52e00e0a918dadd2a3a771bd322b0f165ed2
1. Thorough replacement of git.openstack.org URLs with their
opendev.org counterparts.
2. dict_object.keys() is not required for *in* operator
Change-Id: Ie1d0c828522ae92b01f28e7017fd7fd42c2e91e7
This is deprecated in pbr and the preferred way to do doc builds now
is to use sphinx-build directly.
Change-Id: Id5081ffb9f5b5e6ce608d2f10e9ef89a8b40eeed
Quotes around {posargs} cause the entire string to be combined into one
arg that gets passed to stestr. This prevents passing multiple args
(e.g. '--concurrency=16 some-regex')
Change-Id: I62860fe0f6cfea902c60a251408577bbe0ea894f
Use openstack-tox-cover template, this runs the cover job as
non-voting in the check queue only.
Use openstack-lower-constraints-jobs template
Remove jobs that are part of the templates.
Sort template list alphabetically.
Fix cover tox.ini setting so that this works.
Change-Id: I6e7d197b780e79f22ad936fe63da13ca48098764
According to Openstack summit session [1],
stestr is maintained project to which all Openstack projects should migrate.
Let's switch to stestr as other projects have already moved to it.
[1] https://etherpad.openstack.org/p/YVR-python-pti
Change-Id: I753f32ecf3275cf49d8c93bf648a6a26bc6da8e7
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: Ia0bccf6ed1b28f64622217d88dc6401ba6bd4406
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Set the default python to python3 except for the py27 environment. We
have to set that explicitly to override the new default.
Change-Id: I3fa047b21fa6bc6e5bdf3222287fdce31c5ccef2
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
pep8 under python 3 is more strict than under python 2, so to make
sure our projects meet the more strict standards we want to run the
pep8 jobs using python 3 by default.
Change-Id: Ice3cffa36e9167f2f1f30337372cf9187af5e02e
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.
Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.
Add openstack-tox-lower-constraints job to the zuul configuration.
See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.
Change-Id: I6183925f1acc8bb020caa099c60b6baee61e579a
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
'pip install -U' ugrades specified packages, this is not necessary
since we use constraints, remove the parameter '-U' from the line.
With tools/tox_install.sh - which a previous change of mine removed -
the -U was not harmful, but with the current set up, it might cause
upgrades, so remove it.
Change-Id: I3174ca15402723da2eea20b17d7ce1cc1ed18e41
We do not need tox_install.sh, pip can handle constraints itself
and install the project correctly. Thus update tox.ini and remove
the now obsolete tools/tox_install.sh file.
This follows https://review.openstack.org/#/c/508061 to remove
tools/tox_install.sh.
Change-Id: Ice6d75bcf236713aaad4dfbfa51e6b886555f517
The gating on python 3.4 is restricted to <= Mitaka. This is due
to the change from Ubuntu Trusty to Xenial, where only python3.5
is available. There is no need to continue to keep these settings.
Change-Id: I69fedf01df329ab2f60c4b2583ac5541debeeee4
Adding constraints support to libraries is slightly more complex than
services as the libraries themselves are listed in upper-constraints.txt
which leads to errors that you can't install a specific version and a
constrained version.
This change adds constraints support by also adding a helper script to
edit the constraints to remove oslo.concurrency.
Change-Id: Ic887ce8e220df6b87d823490dfb14894935cac61
Now that there is a passing gate job, we can claim support for
Python 3.5 in the classifier. This patch also adds the convenience
py35 venv. This commit also removes py34 specific venv.
Change-Id: I50ddda7d96c77e7db60aa0b42e8e3b701dadf404
* Remove support for python 2.6
* Remove skipsdist : Needs to be set only if
sdist is expensive
* Remove usedevelop : only needed when skipsdist
is set to True
* Remove install_command : We can just use the
default, we don't need to override
* Remove setenv : We can just use the default as
we don't need any extra environment variables
* Remove requirements.txt from deps, as this is
already added automatically
Change-Id: I5c4addd8be5797fec439b011d99c00a7801704cd
We no longer need to test on Python 3.3, so remove that environment from
the default list in the tox configuration.
Change-Id: Icb3f22224678f9dad5de610a40d43fbef47be7ef