There are projects which no longer support python3.8 in our integration
test. This means they cannot be installed on Ubuntu Focal any longer
without extra steps like checking out old eol tags. Rather than try and
keep up with this list (that will only grow over time) we drop the Focal
integration test and simply rely on unittesting for python3.8 and older.
This is in line with previous decisions made for python2.7 testing. It
isn't ideal, but should still provide sufficient coverage for the most
glaring incompatibilities we may try to introduce.
Change-Id: I7c80e93627597101cbed64a0efdc18140dbc854a
These fixes have to go in together as they are not mergeable
individually.
First up remove the TripleO jobs
TripleO is retired so removing its jobs
- https://review.opendev.org/q/topic:%22retire-tripleo%22
Next the train EOL has largely completed and leaves us in an awkward
position for running tempest-full against python2. While branchless
tempest continues to define a tempest-full job this job is no longer
working after devstack's stable/train branch deletion. There are two
reasons for this. The first is checking out correct branches of
openstack software which we can address in a straightforward manner
through zuul branch checkout overrides from stable/train to train-eol.
The other issue is that we previously relied on zuul running the
stable/train version of devstack's base jobs under tempest-full. Now
that the branch is gone we're getting master devstack base jobs which
don't do things like setup virtualenv (needed for python2) or override
the nodeset to ubuntu-bionic.
Instead of trying to continue to make this job work we accept the
decrease in python2 coverage and will rely on python2 unittesting
instead. An alternative would be to set up functional testing with swift
since swift continues to maintain python2 compatibility in its newer
branches.
Finally we update a test case to check for modern pip's normalized
package names (previously installing pkg_b resulted in a normalized name
of pkg-b, but now pkg_b remains pkg_b and we need to check for that
instead). Check for both variations as we want the tests to pass
regardless of pip and setuptools versions used.
Co-Authored-By: Clark Boylan <clark.boylan@gmail.com>
Change-Id: I9d3417d5803b9ba3e4731fdebbd26778031effcd
setuptools is pretty opinionated about using a build/ directory in the
source tree, and concurrent invocations have a habit of stomping on
each other.
Change-Id: I9f7108ea1a77b9333c0fdf51d94b105fcd5a9e06
Note that there are already many openstack projects using pbr on these
platforms, seemingly successfully.
Change-Id: If49434ff0b60b5382f374635ccaf2ac2e6489db9
In case the method signatures ever change. It also keeps us consistent.
Change-Id: I62c171e9e759b963e80236bcf605df58049dea9a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
As with PEP-517 support, this is simply a shim over the setuptools
implementation added in setuptools v64.0.0 [1].
[1] https://github.com/pypa/setuptools/pull/3488
Change-Id: I32f974db37e364cf634b050b40bf0820dce0a3a3
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The openstack installation test is failing because it checks for a file
that doesn't exist. This occurs because Nova no longer uses
sqlalchemy-migrate [1] which means it won't have a 'migrate.cfg' file.
Update references to use 'alembic.ini' instead.
The tempest-full job fails because stable/train is not a valid branch on
some projects like neutron, nova, and placement. We fix this by having
the job checkout train-eol for those projects. We also reduce the
tempest test set to the smoke tests to reduce the possibility for test
failures. We don't actually need to run all tempest tests to have a good
indiciation of whether or not package installation was successful. Smoke
tests should be sufficient for that.
[1] https://github.com/openstack/nova/commit/fd39e4b4b
Change-Id: Idfd234cf4f968d646346a0e2fc0dd1be2332bb52
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Co-authored-by: Clark Boylan <clark.boylan@gmail.com>
The imp was removed in Python 3.12 [1]. This commit replaces
get_suffixes() method call with _SUFFIXES array from
importlib.machinery.
This was imported from https://github.com/openstack/pbr/pull/22
on behalf of the Gentoo maintainer who proposed it.
Co-Authored-By: Petr Vaněk <arkamar@atlas.cz>
Change-Id: Icf00beee255aba3da2b81d9dc5b1f8130bbcd869
Setuptools deprecated -'s at some point and they result in a bunch of
warnings in failed tests. This is distracting and eventually is likely
to create real errors. Go ahead and fix them now.
Change-Id: If3594269495ae1f3cefffd4c4f38d87af5c7fdfe
Setuptools and sphinx [0] have removed support for building docs through
setup.py. This has broken imports which break our own local doc builds.
Lets just remove it all (including the tests) so that we are compatible
with modern setuptools and sphinx.
An alternative approach would be to do conditional imports and continue
to support this for old setuptools and sphinx. But that feels like a
dead end that will be difficult to test. Probably better to rip this
bandaid off.
[0] https://www.sphinx-doc.org/en/master/changes.html#id7
Change-Id: I65038caceb192f495288414079ca0f07ce6318bb
Python 3.10 on Jammy is not compatible with old pip and setuptools due
to collections.abc moves. It is still desireable to test modern PBR
functions against these old library versions because old distro installs
will often use latest PBR. Update our testing to also run PBR
integration testing on Focal selecting appropriate pip and setuptools
versions based on the running python version.
This should help ensure PBR continues to run on old platforms even as we
modernize pip.
Change-Id: Ib53f637d5826bb51a348e3e629fbdbabec2a95e0
There do not appear to be any openstack projects relying on this package
these days. Remove the unnecessary dependencies.
Change-Id: Iddd95afad9e6cf10d2cf0a3d6cae4b6ba75a7a05
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
In change I1f2b4d34e587389f7e11b99d000e14477cf5091b, we attempted to
resolve an issue whereby using PBR as a PEP-517 backend could result in
recursion. We did this by setting 'dist.pbr' to 'None' but later
discovered this introduced a regression that prevented us from writing
the 'pbr.json' file into generated sdists. The fix for the regression,
change I407ae88ab8de4b61f94034b3d79a2ca7f7d79d16, stopped setting
'dist.pbr' to 'None' and instead set a global flag to indicate whether
we had already been called or not. Unfortunately it seems this fix is
also insufficent. As discussed in tox#2712 [1], tox v4 has implemented
its own implementation of PEP-517 - 'pyproject-api' [2] - and unlike
'build' this implementation optionally allows re-use of the backend
process for multiple PEP-517 commands. tox's 'Pep517VirtualEnvFrontend'
does just this. This means if we run multiple commands that require
generation of a 'Distribution' object - say,
'prepare_metadata_for_build_wheel' followed by 'build_sdist' - anything
but the first step will not result in proper population of said
'Distribution' objects.
The solution to this issue is simple: per $subject, instead of setting
our recursion-detection canary at the module level, set it at the
Distribution level.
[1] https://github.com/tox-dev/tox/issues/2712
[2] github.com/tox-dev/pyproject-api/
Change-Id: I67909d732a74550fbcd7c06a9e2f4ac88c063444
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Resolve the following warnings:
tox.tox_env.errors.Fail: pass_env values cannot contain whitespace,
use comma to have multiple values in a single line, invalid values
found 'PBR_INTEGRATION PIPFLAGS PIPVERSION PBRVERSION REPODIR
WHEELHOUSE PROJECTS'
Also, reduce down the envlist as tox 4 when run in OpenStack CI is
also attempting to validate interpreters utilizing the
--skip-missing-interpreters=false setting [1].
Finally, fix up the integration job so that it passes on Ubuntu 22.04.
This necessitates removing any Python 2-based dependencies used in the
these jobs (since these are no longer available on Ubuntu 22.04) and
removing tests for pip versions that are not compatible with Python 3.10
(the Python version found on Ubuntu 22.04).
[1] https://github.com/tox-dev/tox/issues/2811
Change-Id: Ib4dbe4e09e269219c08a87f6fb64ad9628b5864a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
* Fix an issue where a multiline message with leading spaces would cause
invalid symbol identification.
* Add unit-tests around multiple symbols in a single line
Sem-Ver: bugfix
Change-Id: Id799c2fd368d9df38a5c01fc2071283aecef67c6
Update the CONTRIBUTING file to specify how release notes are managed.
This allows contributors to follow best-practices to ensure the
auto-generated Changelog stays up-to-date.
Change-Id: I7902c33a76c4cce8f50307843476e5d9435c9e2c
Update the logic that checks for sem-ver to strip out leading spaces.
This helps solve an issue during automated squashes appending a tab
character to the start of each line in a commit message.
Closes-Bug: #1977479
Change-Id: I6f56500c366fd9610d98507f01e8af21862e4421
The importlib_metadata package isn't guaranteed to be available, but the
stlib version will always be there (at least on Python 3.8 or later).
Change-Id: I532a365aa0f1a3206edf6b01e821bd83ea04ab3c
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We were using pkg_resources for runtime package version lookups but
pkg_resources incurs a pretty significant cost to use it in thise way
(because it scans all installed packages on disk and sorts them).
importlib-metadata is the way of the future (part of stdlib in
python3.8) and is supposed to be significantly more performant. Replace
our use of pkg_resources for runtime version lookups with
importlib-metadata so that we are both quicker and future proof.
Note that this doesn't handle the problem of no pbr deps yet. We will
probably end up needing to vendor importlib-metadata in pbr or something
similar.
Change-Id: Ife68089d997b266adb37f18e226f49bdd67766fc
Per https://github.com/pypa/setuptools/pull/3056 the addition of
wheel in build_system.requires is redundant since SetupTools already
drags it in anyway. Simplify our recommendations accordingly.
Change-Id: I0359ed11a6f0c54dc1af6a7f5f34e49476c58fb0
Include a release note for I407ae88ab8de4b61f94034b3d79a2ca7f7d79d16
which corrected a regression for missing pbr.json metadata files in
packages.
Change-Id: I3f0e48c215e890f9b26bae6600d5fcacf67831c6
Add a check in pbr.core.pbr() to make sure it hasn't already run,
and return immediately if it has. This replaces the earlier attempt
at loop-busting by setting the entrypoint to None in the dist, in
order to support PEP 517 builds without setup_requires.
Also test that pbr.json is now included in the dist tree, in order
to avoid future regressions.
Change-Id: I407ae88ab8de4b61f94034b3d79a2ca7f7d79d16
Setuptools v54.1.0 introduces a warning that the use of
dash-separated options in 'setup.cfg' will not be supported in a
future version [1]. Get ahead of the issue by updating to canonical
configuration[2]. Without this, we see 'UserWarning' messages like
the following on new enough versions of setuptools:
UserWarning: Usage of dash-separated 'description-file' will not
be supported in future versions. Please use the underscore name
'description_file' instead
[1] https://github.com/pypa/setuptools/commit/a2e9ae4cb
[2] https://setuptools.pypa.io/en/stable/userguide/declarative_config.html
Change-Id: I9c51ffdeb548833f2cc303d2e53ba169756690d3
We noticed some aberrations in log level/threshold handling as of
Setuptools 60.0.0, and the only substantive change for that release
toggles the default selection for its internal ("local") distutils
replacement vs the "stdlib" one. For now, toggle the
SETUPTOOLS_USE_DISTUTILS envvar in our testing back to the old
value. This should be revisited once logging is handled better in
Setuptools. https://github.com/pypa/setuptools/issues/3038
Change-Id: If372a33607fff884cd7bf888ee5a23a2fbb351b2
Two unclosed file descriptors in packaging are raising
ResourceWarning on termination in newer Python interpreters. Make
sure they're closed as soon as we're done with them.
Change-Id: I07f500e1157da18de799eb0ff9bf82d3bcb46510
I think this was lost when we expanded the openstack-python job
templates to include more versions of python3. Oops.
Change-Id: I0ebaff67634857216ca8d565311ed7c63f687a16
When using PBR PEP517 support without setup_requires were were
infinitely recursing because Distribution.finalize_options calls pbr()
and pbr() calls Distribution.finalize_options. With a setup_requires
this doesn't seem to happen because something is resetting dist.pbr's
value from True (callable) to None (not callable) after the initial
pbr() call and before further recursion can occur.
Explicitly enforce this behavior in pbr() via an explicit update of
dist.pbr to None to avoid calling pbr() more than once resulting in the
unwanted loop.
Change-Id: I1f2b4d34e587389f7e11b99d000e14477cf5091b
There was a misunderstanding that PBRs PEP517 build hooks meant we no
longer needed a setup.py. But it turns out the setuptools build_meta
hooks that we wrap call out to setup.py. This means PBR will continue to
need to be used in conjunction with a setup.py.
This is better for backward compatibiltiy anyway so not a huge loss.
Document this more properly and update the test to reflect it.
Change-Id: If4320da1e89a5c765e0bfe56b724364f9cba34f3
pep517 defines a new module method of specifying build backends.
To allow pbr to exist in this world, we should define the interface
that's needed. For this to be used, one will put:
[build-system]
requires = ["pbr>=5.7.0", "setuptools>=36.6.0", "wheel"]
build-backend = "pbr.build"
Into pyproject.toml - and the pep517 interface will be used.
This doesn't really change anything else - it just makes us support this.
So by itself this commit isn't SUPER helpful. But maybe let's take baby
steps with something this prone to strife, yeah?
After this we can start teasing some things apart and doing our
own things directly.
Co-Authored-By: Clark Boylan <clark.boylan@gmail.com>
Change-Id: I293f59b5074a38c78adffe580de2f1533bb01ce7