These fixes have to go in together as they are not mergeable
individually.
First up remove the TripleO jobs
TripleO is retired so removing its jobs
- https://review.opendev.org/q/topic:%22retire-tripleo%22
Next the train EOL has largely completed and leaves us in an awkward
position for running tempest-full against python2. While branchless
tempest continues to define a tempest-full job this job is no longer
working after devstack's stable/train branch deletion. There are two
reasons for this. The first is checking out correct branches of
openstack software which we can address in a straightforward manner
through zuul branch checkout overrides from stable/train to train-eol.
The other issue is that we previously relied on zuul running the
stable/train version of devstack's base jobs under tempest-full. Now
that the branch is gone we're getting master devstack base jobs which
don't do things like setup virtualenv (needed for python2) or override
the nodeset to ubuntu-bionic.
Instead of trying to continue to make this job work we accept the
decrease in python2 coverage and will rely on python2 unittesting
instead. An alternative would be to set up functional testing with swift
since swift continues to maintain python2 compatibility in its newer
branches.
Finally we update a test case to check for modern pip's normalized
package names (previously installing pkg_b resulted in a normalized name
of pkg-b, but now pkg_b remains pkg_b and we need to check for that
instead). Check for both variations as we want the tests to pass
regardless of pip and setuptools versions used.
Co-Authored-By: Clark Boylan <clark.boylan@gmail.com>
Change-Id: I9d3417d5803b9ba3e4731fdebbd26778031effcd
In case the method signatures ever change. It also keeps us consistent.
Change-Id: I62c171e9e759b963e80236bcf605df58049dea9a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
As with PEP-517 support, this is simply a shim over the setuptools
implementation added in setuptools v64.0.0 [1].
[1] https://github.com/pypa/setuptools/pull/3488
Change-Id: I32f974db37e364cf634b050b40bf0820dce0a3a3
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The openstack installation test is failing because it checks for a file
that doesn't exist. This occurs because Nova no longer uses
sqlalchemy-migrate [1] which means it won't have a 'migrate.cfg' file.
Update references to use 'alembic.ini' instead.
The tempest-full job fails because stable/train is not a valid branch on
some projects like neutron, nova, and placement. We fix this by having
the job checkout train-eol for those projects. We also reduce the
tempest test set to the smoke tests to reduce the possibility for test
failures. We don't actually need to run all tempest tests to have a good
indiciation of whether or not package installation was successful. Smoke
tests should be sufficient for that.
[1] https://github.com/openstack/nova/commit/fd39e4b4b
Change-Id: Idfd234cf4f968d646346a0e2fc0dd1be2332bb52
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Co-authored-by: Clark Boylan <clark.boylan@gmail.com>
The imp was removed in Python 3.12 [1]. This commit replaces
get_suffixes() method call with _SUFFIXES array from
importlib.machinery.
This was imported from https://github.com/openstack/pbr/pull/22
on behalf of the Gentoo maintainer who proposed it.
Co-Authored-By: Petr Vaněk <arkamar@atlas.cz>
Change-Id: Icf00beee255aba3da2b81d9dc5b1f8130bbcd869
Setuptools deprecated -'s at some point and they result in a bunch of
warnings in failed tests. This is distracting and eventually is likely
to create real errors. Go ahead and fix them now.
Change-Id: If3594269495ae1f3cefffd4c4f38d87af5c7fdfe
Setuptools and sphinx [0] have removed support for building docs through
setup.py. This has broken imports which break our own local doc builds.
Lets just remove it all (including the tests) so that we are compatible
with modern setuptools and sphinx.
An alternative approach would be to do conditional imports and continue
to support this for old setuptools and sphinx. But that feels like a
dead end that will be difficult to test. Probably better to rip this
bandaid off.
[0] https://www.sphinx-doc.org/en/master/changes.html#id7
Change-Id: I65038caceb192f495288414079ca0f07ce6318bb
Python 3.10 on Jammy is not compatible with old pip and setuptools due
to collections.abc moves. It is still desireable to test modern PBR
functions against these old library versions because old distro installs
will often use latest PBR. Update our testing to also run PBR
integration testing on Focal selecting appropriate pip and setuptools
versions based on the running python version.
This should help ensure PBR continues to run on old platforms even as we
modernize pip.
Change-Id: Ib53f637d5826bb51a348e3e629fbdbabec2a95e0
In change I1f2b4d34e587389f7e11b99d000e14477cf5091b, we attempted to
resolve an issue whereby using PBR as a PEP-517 backend could result in
recursion. We did this by setting 'dist.pbr' to 'None' but later
discovered this introduced a regression that prevented us from writing
the 'pbr.json' file into generated sdists. The fix for the regression,
change I407ae88ab8de4b61f94034b3d79a2ca7f7d79d16, stopped setting
'dist.pbr' to 'None' and instead set a global flag to indicate whether
we had already been called or not. Unfortunately it seems this fix is
also insufficent. As discussed in tox#2712 [1], tox v4 has implemented
its own implementation of PEP-517 - 'pyproject-api' [2] - and unlike
'build' this implementation optionally allows re-use of the backend
process for multiple PEP-517 commands. tox's 'Pep517VirtualEnvFrontend'
does just this. This means if we run multiple commands that require
generation of a 'Distribution' object - say,
'prepare_metadata_for_build_wheel' followed by 'build_sdist' - anything
but the first step will not result in proper population of said
'Distribution' objects.
The solution to this issue is simple: per $subject, instead of setting
our recursion-detection canary at the module level, set it at the
Distribution level.
[1] https://github.com/tox-dev/tox/issues/2712
[2] github.com/tox-dev/pyproject-api/
Change-Id: I67909d732a74550fbcd7c06a9e2f4ac88c063444
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Resolve the following warnings:
tox.tox_env.errors.Fail: pass_env values cannot contain whitespace,
use comma to have multiple values in a single line, invalid values
found 'PBR_INTEGRATION PIPFLAGS PIPVERSION PBRVERSION REPODIR
WHEELHOUSE PROJECTS'
Also, reduce down the envlist as tox 4 when run in OpenStack CI is
also attempting to validate interpreters utilizing the
--skip-missing-interpreters=false setting [1].
Finally, fix up the integration job so that it passes on Ubuntu 22.04.
This necessitates removing any Python 2-based dependencies used in the
these jobs (since these are no longer available on Ubuntu 22.04) and
removing tests for pip versions that are not compatible with Python 3.10
(the Python version found on Ubuntu 22.04).
[1] https://github.com/tox-dev/tox/issues/2811
Change-Id: Ib4dbe4e09e269219c08a87f6fb64ad9628b5864a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* Fix an issue where a multiline message with leading spaces would cause
invalid symbol identification.
* Add unit-tests around multiple symbols in a single line
Sem-Ver: bugfix
Change-Id: Id799c2fd368d9df38a5c01fc2071283aecef67c6
Update the logic that checks for sem-ver to strip out leading spaces.
This helps solve an issue during automated squashes appending a tab
character to the start of each line in a commit message.
Closes-Bug: #1977479
Change-Id: I6f56500c366fd9610d98507f01e8af21862e4421
The importlib_metadata package isn't guaranteed to be available, but the
stlib version will always be there (at least on Python 3.8 or later).
Change-Id: I532a365aa0f1a3206edf6b01e821bd83ea04ab3c
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We were using pkg_resources for runtime package version lookups but
pkg_resources incurs a pretty significant cost to use it in thise way
(because it scans all installed packages on disk and sorts them).
importlib-metadata is the way of the future (part of stdlib in
python3.8) and is supposed to be significantly more performant. Replace
our use of pkg_resources for runtime version lookups with
importlib-metadata so that we are both quicker and future proof.
Note that this doesn't handle the problem of no pbr deps yet. We will
probably end up needing to vendor importlib-metadata in pbr or something
similar.
Change-Id: Ife68089d997b266adb37f18e226f49bdd67766fc
Add a check in pbr.core.pbr() to make sure it hasn't already run,
and return immediately if it has. This replaces the earlier attempt
at loop-busting by setting the entrypoint to None in the dist, in
order to support PEP 517 builds without setup_requires.
Also test that pbr.json is now included in the dist tree, in order
to avoid future regressions.
Change-Id: I407ae88ab8de4b61f94034b3d79a2ca7f7d79d16
Two unclosed file descriptors in packaging are raising
ResourceWarning on termination in newer Python interpreters. Make
sure they're closed as soon as we're done with them.
Change-Id: I07f500e1157da18de799eb0ff9bf82d3bcb46510
When using PBR PEP517 support without setup_requires were were
infinitely recursing because Distribution.finalize_options calls pbr()
and pbr() calls Distribution.finalize_options. With a setup_requires
this doesn't seem to happen because something is resetting dist.pbr's
value from True (callable) to None (not callable) after the initial
pbr() call and before further recursion can occur.
Explicitly enforce this behavior in pbr() via an explicit update of
dist.pbr to None to avoid calling pbr() more than once resulting in the
unwanted loop.
Change-Id: I1f2b4d34e587389f7e11b99d000e14477cf5091b
There was a misunderstanding that PBRs PEP517 build hooks meant we no
longer needed a setup.py. But it turns out the setuptools build_meta
hooks that we wrap call out to setup.py. This means PBR will continue to
need to be used in conjunction with a setup.py.
This is better for backward compatibiltiy anyway so not a huge loss.
Document this more properly and update the test to reflect it.
Change-Id: If4320da1e89a5c765e0bfe56b724364f9cba34f3
pep517 defines a new module method of specifying build backends.
To allow pbr to exist in this world, we should define the interface
that's needed. For this to be used, one will put:
[build-system]
requires = ["pbr>=5.7.0", "setuptools>=36.6.0", "wheel"]
build-backend = "pbr.build"
Into pyproject.toml - and the pep517 interface will be used.
This doesn't really change anything else - it just makes us support this.
So by itself this commit isn't SUPER helpful. But maybe let's take baby
steps with something this prone to strife, yeah?
After this we can start teasing some things apart and doing our
own things directly.
Co-Authored-By: Clark Boylan <clark.boylan@gmail.com>
Change-Id: I293f59b5074a38c78adffe580de2f1533bb01ce7
We simplify virtualenv setup for PBR package installation testing by
switching to a simple file path for the PBR installation rather than a
full url with egg specifier. The egg specifer isn't necessary and
neither is the url and simpler is easier to understand.
We also add the verbose flag to the pip installations of tools into that
virtualenv to better understand problems when they occur.
Finally we add the option to skip writing package files if their
contents are empty. This allows us to ensure that we can omit certain
files which will be useful for the next change.
Change-Id: I1f081d29d62c905c1bb1abee5402b0990c83e948
Change Ic7b71f1093671533c9f17fd621e02299bfbd7a6d reversed the order of
this list but didn't update the comment. Do so now.
Change-Id: I36f2dd791297a9e3df4f1e427db5b45017820bbf
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The previous ordering was confusing and placed new values before old
ones. Switch this order, making it more obvious what we're going *from*
and what we're going *to*. With this reordering, the old name no longer
makes sense so we rename the attribute.
Change-Id: Ic7b71f1093671533c9f17fd621e02299bfbd7a6d
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Correct a small logic error that would result in us passing an empty
'long_description' value to 'setup' if 'description_file' pointed to a
file that was empty. Fixing this also makes it a little more obvious
that we are only setting values that are present in 'setup.cfg', since
that wasn't totally clear before.
A small, unrelated typo is addressed.
Change-Id: Ib32b5fb5da84215376c8e372470b2ff08eb47908
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Attempt two at the problem, this time done properly. As discussed in
change I7158e75934535b80eb60d6b4b7dcef08866b5bab, the previous attempt
actually mapped 'python_requires' to 'requires_python' rather than the
other way around. This resolves that.
Change-Id: I415b64d5b51a248d2f45a28234f1635e8b82ffcc
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This fixes a bug that prevented successful test gpg key creation on
some machines.
gpg: invalid item '(setpref)' in preference string
Bug: #1627102
Change-Id: I848e2c0972c3def9e67ff954eb17c560e474e8cf
Dicts can't have duplicate keys. Lists of tuples can. We're going to
need duplicates.
Change-Id: Id43ba91585160448ad9737ba958ace5163d26f3d
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This reverts commit d5da2121f8. This was
done incorrectly. The 'D1_D2_SETUP_ARGS' dictionary modified here is a
mapping of distutils (and by extension, setuptools) supported options to
their legacy distutils2 equivalents, and is used to transform the latter
to the former. For example, consider the following entry:
"long_description": ("metadata", "description"),
This will transform an entry like so in a 'setup.cfg' file:
[metadata]
description = ...
Into the following equivalent call to setuptools' 'setup' function:
setup(
...,
long_description='...',
...
)
By that measure, the entry added in this change, namely:
"requires_python": ("metadata", "python_requires"),
Will result in a 'requires_python' argument being added to the 'setup'
call. That's not what we want. Rather, we likely want to transform a
'[metadata] requires_python' entry to a 'python_requires' argument to
'setup'.
The change is reverted here in expectation of a forthcoming proper fix.
Change-Id: I7158e75934535b80eb60d6b4b7dcef08866b5bab
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We facing errors related to the new pip resolver, this
topic was discussed on the ML and QA team proposed to
to test lower-constraints [1].
I propose to drop this test because the complexity and recurring pain needed
to maintain that now exceeds the benefits provided by this mechanismes.
Replace assertTrue by assertIn to fix with the latest flake8 version.
Add six in the documentation requirements to fix the build.
Fix the conflict status with hacking.
[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019390.html
Change-Id: I580da15fefd6ad352e69749b77cc4664aef84f9c
Odds are this was never used by any PBR-based projects, but was
included from the beginning to support a feature of Setuptools for a
utility which has been long since discouraged. As of Setuptools
46.2.0 and Python 3.9 it's generating deprecation warnings, and is
slated for removal from the Python 3.10 standard library.
https://bugs.python.org/issue40360https://github.com/pypa/setuptools/issues/2086
Change-Id: I6972984f29413d2534d802b142fa9323a7e4a4a3
Setuptools 12.0 deprecated easy_install.get_script_header() when it
was released five years ago, and if called with PYTHONWARNINGS=error
raises a deprecation exception. Switch to the preferred class
method, but keep the original call as a fallback since it won't work
on older Setuptools versions (such as the 3.3 shipped with Ubuntu
Trusty). This continues 9a219f9a4f to
remove a second call for the same deprecated function.
Change-Id: I365075c2cff88283f92298ce2f5e53d263dc7f4b
The git web URL is now: https://opendev.org/openstack/pbr
Also stop linking README.rst to no more existing history/changelog
documentation page (removed by 96b84a9014).
Change-Id: I56b0e2b86a7fc66c5ae25e06175033a885f500f6
Setuptools 12.0 deprecated easy_install.get_script_header() when it
was released five years ago, and if called with PYTHONWARNINGS=error
raises a deprecation exception. Switch to the preferred class
method, but keep the original call as a fallback since it won't work
on older Setuptools versions (such as the 3.3 shipped with Ubuntu
Trusty).
Change-Id: I2ba68a47dd3f309e356a6e88bf890ac2f80d99a0
The metadata field, 'python-requires' was introduced in PEP440 and
appears to have superseded 'requires-python'. [1] The metadata that pbr
uses for itself and it's test package was using the old incorrect field,
this commit updates it to be the proper metadata to show a working
example. To a certain extent it probably doesn't matter because it's very
unlikely someone will try to install pbr with python 2.5, but it's more
about making sure that pbr itself is leading by example and that it's own
config actually works.
[1] https://github.com/mtreinish/stestr/issues/292
Change-Id: I5eb14d490a3731b6b95d7a1137c33ec2d73f193f
Apparently requires-python is the old value and isn't valid
anymore. What people mean is python-requires. Of course, people
are still using the old value. Fix it for them.
Change-Id: Id49cf641f2eddac17f1b21b186aa3a150bd863f9
This allows us to run tox -epep8 on newer python versions. Fix the
issues newer hacking calls out. They were complaints about indentation
and ambigious l value variable names.
Change-Id: I9552fba791bdf54649fbe25fc26bffdf8186ff0a
virtualenv is undergoing a rewrite and has changed how the programmatic
API works [1]. Switch to the "new way".
While we're here, we also need to get Python 2.7 tests passing again.
That requires dropping support for upper-constraints and using our own,
limited local constraints based on supported Python versions. We also
need to migrate integration tests since those run with Python 3 now.
Update the scenarios for pip/setuptools in integration testing
similarly. Finally clean up the installation of all devstack repos as
requirements are managed different now. Instead of worrying about
syncing them we use constraints.
[1] https://github.com/pypa/virtualenv/issues/1585#issuecomment-585228492
Change-Id: I493e88985d2c4d09612fea4d20d8ffa20043a0cb
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Depends-On: https://review.opendev.org/739014
Ghanshyam Mann