Commit Graph

1250 Commits

Author SHA1 Message Date
Ghanshyam Mann f75e1056f1 Omnibus PBR CI fixups
These fixes have to go in together as they are not mergeable
individually.

First up remove the TripleO jobs

TripleO is retired so removing its jobs

- https://review.opendev.org/q/topic:%22retire-tripleo%22

Next the train EOL has largely completed and leaves us in an awkward
position for running tempest-full against python2. While branchless
tempest continues to define a tempest-full job this job is no longer
working after devstack's stable/train branch deletion. There are two
reasons for this. The first is checking out correct branches of
openstack software which we can address in a straightforward manner
through zuul branch checkout overrides from stable/train to train-eol.
The other issue is that we previously relied on zuul running the
stable/train version of devstack's base jobs under tempest-full. Now
that the branch is gone we're getting master devstack base jobs which
don't do things like setup virtualenv (needed for python2) or override
the nodeset to ubuntu-bionic.

Instead of trying to continue to make this job work we accept the
decrease in python2 coverage and will rely on python2 unittesting
instead. An alternative would be to set up functional testing with swift
since swift continues to maintain python2 compatibility in its newer
branches.

Finally we update a test case to check for modern pip's normalized
package names (previously installing pkg_b resulted in a normalized name
of pkg-b, but now pkg_b remains pkg_b and we need to check for that
instead). Check for both variations as we want the tests to pass
regardless of pip and setuptools versions used.

Co-Authored-By: Clark Boylan <clark.boylan@gmail.com>
Change-Id: I9d3417d5803b9ba3e4731fdebbd26778031effcd
2024-03-04 09:23:36 -08:00
Zuul f55524c08e Merge "Use summary for the package summary" 2024-01-30 12:50:25 +00:00
Zuul e502ba1e94 Merge "Test on py310 and py311" 2024-01-26 12:05:35 +00:00
Tim Burke f401a59db1 Serialize tests
setuptools is pretty opinionated about using a build/ directory in the
source tree, and concurrent invocations have a habit of stomping on
each other.

Change-Id: I9f7108ea1a77b9333c0fdf51d94b105fcd5a9e06
2024-01-22 09:35:43 -08:00
Tim Burke 8252a7d54e Fix tempest-full job
Also, Use an alias to reduce some duplication.

Change-Id: I11c19ab657ef0dfea5d6929ce937791b82e86941
2024-01-18 21:19:27 -08:00
Tim Burke f42ef66d1b Use summary for the package summary
Otherwise, the long_description file isn't used.

Change-Id: Ib494a0bac947cc6b5fb8d3d4315a3d386cbffdc2
2024-01-18 14:53:58 -08:00
Tim Burke 049282adbb Test on py310 and py311
Note that there are already many openstack projects using pbr on these
platforms, seemingly successfully.

Change-Id: If49434ff0b60b5382f374635ccaf2ac2e6489db9
2024-01-18 14:19:42 -08:00
Stephen Finucane 11151154b9 build: Use kwargs
In case the method signatures ever change. It also keeps us consistent.

Change-Id: I62c171e9e759b963e80236bcf605df58049dea9a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-11-01 12:53:18 -07:00
Stephen Finucane 457a89ffda Add support for PEP-660
As with PEP-517 support, this is simply a shim over the setuptools
implementation added in setuptools v64.0.0 [1].

[1] https://github.com/pypa/setuptools/pull/3488

Change-Id: I32f974db37e364cf634b050b40bf0820dce0a3a3
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-11-01 12:53:18 -07:00
Stephen Finucane 6df16432aa Fix PBR integration testing
The openstack installation test is failing because it checks for a file
that doesn't exist. This occurs because Nova no longer uses
sqlalchemy-migrate [1] which means it won't have a 'migrate.cfg' file.
Update references to use 'alembic.ini' instead.

The tempest-full job fails because stable/train is not a valid branch on
some projects like neutron, nova, and placement. We fix this by having
the job checkout train-eol for those projects. We also reduce the
tempest test set to the smoke tests to reduce the possibility for test
failures. We don't actually need to run all tempest tests to have a good
indiciation of whether or not package installation was successful. Smoke
tests should be sufficient for that.

[1] https://github.com/openstack/nova/commit/fd39e4b4b

Change-Id: Idfd234cf4f968d646346a0e2fc0dd1be2332bb52
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Co-authored-by: Clark Boylan <clark.boylan@gmail.com>
2023-11-01 11:29:38 -07:00
Jay Faulkner d03d617c09 Replace imp with importlib.machinery
The imp was removed in Python 3.12 [1]. This commit replaces
get_suffixes() method call with _SUFFIXES array from
importlib.machinery.

This was imported from https://github.com/openstack/pbr/pull/22
on behalf of the Gentoo maintainer who proposed it.

Co-Authored-By: Petr Vaněk <arkamar@atlas.cz>
Change-Id: Icf00beee255aba3da2b81d9dc5b1f8130bbcd869
2023-06-02 11:46:31 -07:00
Zuul 4d6c377028 Merge "Add release note for build_sphinx removal" 2023-06-01 17:38:31 +00:00
Clark Boylan 0e335e6ddb Add release note for build_sphinx removal
This wasn't done in the change that removed the feature but seems like a
good idea.

Change-Id: I97bdda4ce89b26c6e99de6f1d6ad7d734cac9041
2023-06-01 08:24:37 -07:00
Clark Boylan ee04b62de2 Use _ instead of - in setup.cfg directives
Setuptools deprecated -'s at some point and they result in a bunch of
warnings in failed tests. This is distracting and eventually is likely
to create real errors. Go ahead and fix them now.

Change-Id: If3594269495ae1f3cefffd4c4f38d87af5c7fdfe
2023-05-31 13:25:23 -07:00
Clark Boylan 47c5afe79a Remove sphinx doc building integration
Setuptools and sphinx [0] have removed support for building docs through
setup.py. This has broken imports which break our own local doc builds.
Lets just remove it all (including the tests) so that we are compatible
with modern setuptools and sphinx.

An alternative approach would be to do conditional imports and continue
to support this for old setuptools and sphinx. But that feels like a
dead end that will be difficult to test. Probably better to rip this
bandaid off.

[0] https://www.sphinx-doc.org/en/master/changes.html#id7

Change-Id: I65038caceb192f495288414079ca0f07ce6318bb
2023-05-31 13:25:13 -07:00
Zuul 98c84b5f87 Merge "Run PBR integration on Ubuntu Focal too" 2023-01-07 00:32:47 +00:00
Zuul 1ca7781493 Merge "Tie recursion calls to Dist object, not module" 2023-01-07 00:17:13 +00:00
Clark Boylan 571e14581d Run PBR integration on Ubuntu Focal too
Python 3.10 on Jammy is not compatible with old pip and setuptools due
to collections.abc moves. It is still desireable to test modern PBR
functions against these old library versions because old distro installs
will often use latest PBR. Update our testing to also run PBR
integration testing on Focal selecting appropriate pip and setuptools
versions based on the running python version.

This should help ensure PBR continues to run on old platforms even as we
modernize pip.

Change-Id: Ib53f637d5826bb51a348e3e629fbdbabec2a95e0
2023-01-06 09:34:59 -08:00
Stephen Finucane 535254d5c8 Remove numpy dependencies
There do not appear to be any openstack projects relying on this package
these days. Remove the unnecessary dependencies.

Change-Id: Iddd95afad9e6cf10d2cf0a3d6cae4b6ba75a7a05
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-01-06 14:36:51 +00:00
Stephen Finucane 61c6a4bcb3 Tie recursion calls to Dist object, not module
In change I1f2b4d34e587389f7e11b99d000e14477cf5091b, we attempted to
resolve an issue whereby using PBR as a PEP-517 backend could result in
recursion. We did this by setting 'dist.pbr' to 'None' but later
discovered this introduced a regression that prevented us from writing
the 'pbr.json' file into generated sdists. The fix for the regression,
change I407ae88ab8de4b61f94034b3d79a2ca7f7d79d16, stopped setting
'dist.pbr' to 'None' and instead set a global flag to indicate whether
we had already been called or not. Unfortunately it seems this fix is
also insufficent. As discussed in tox#2712 [1], tox v4 has implemented
its own implementation of PEP-517 - 'pyproject-api' [2] - and unlike
'build' this implementation optionally allows re-use of the backend
process for multiple PEP-517 commands. tox's 'Pep517VirtualEnvFrontend'
does just this. This means if we run multiple commands that require
generation of a 'Distribution' object - say,
'prepare_metadata_for_build_wheel' followed by 'build_sdist' - anything
but the first step will not result in proper population of said
'Distribution' objects.

The solution to this issue is simple: per $subject, instead of setting
our recursion-detection canary at the module level, set it at the
Distribution level.

[1] https://github.com/tox-dev/tox/issues/2712
[2] github.com/tox-dev/pyproject-api/

Change-Id: I67909d732a74550fbcd7c06a9e2f4ac88c063444
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-01-06 13:22:51 +00:00
Stephen Finucane 491ce1ab09 Update tox.ini to work with tox 4
Resolve the following warnings:

  tox.tox_env.errors.Fail: pass_env values cannot contain whitespace,
  use comma to have multiple values in a single line, invalid values
  found 'PBR_INTEGRATION PIPFLAGS PIPVERSION PBRVERSION REPODIR
  WHEELHOUSE PROJECTS'

Also, reduce down the envlist as tox 4 when run in OpenStack CI is
also attempting to validate interpreters utilizing the
--skip-missing-interpreters=false setting [1].

Finally, fix up the integration job so that it passes on Ubuntu 22.04.
This necessitates removing any Python 2-based dependencies used in the
these jobs (since these are no longer available on Ubuntu 22.04) and
removing tests for pip versions that are not compatible with Python 3.10
(the Python version found on Ubuntu 22.04).

[1] https://github.com/tox-dev/tox/issues/2811

Change-Id: Ib4dbe4e09e269219c08a87f6fb64ad9628b5864a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2023-01-06 13:22:38 +00:00
Zuul 45d4c9f02f Merge "Fix symbol identification in multiline message" 2022-08-05 17:56:35 +00:00
Chris Dohmen 0aecbea81b Fix symbol identification in multiline message
* Fix an issue where a multiline message with leading spaces would cause
  invalid symbol identification.
* Add unit-tests around multiple symbols in a single line

Sem-Ver: bugfix
Change-Id: Id799c2fd368d9df38a5c01fc2071283aecef67c6
2022-08-05 10:08:09 -04:00
Zuul cb2c15ea2b Merge "Replace deprecated readfp method with read_file" 2022-08-03 14:37:16 +00:00
Zuul 553218d8f5 Merge "Use stdlib importlib.metadata where possible" 2022-08-03 12:45:16 +00:00
ljhuang 2290f58eb9 Replace deprecated readfp method with read_file
The readfp method has been deprecated since version 3.2 [1].

[1] https://docs.python.org/3/library/configparser.html?highlight=deprecated#configparser.ConfigParser.readfp

Change-Id: Ib9549509bfcc749740953e3769030d787b2bf251
2022-07-22 17:41:30 +08:00
Chris Dohmen 1eadf23b94 Specify Changelog procedure
Update the CONTRIBUTING file to specify how release notes are managed.
This allows contributors to follow best-practices to ensure the
auto-generated Changelog stays up-to-date.

Change-Id: I7902c33a76c4cce8f50307843476e5d9435c9e2c
2022-06-13 17:06:43 -04:00
Chris Dohmen 3793d87deb Allow leading spaces when determining symbols
Update the logic that checks for sem-ver to strip out leading spaces.
This helps solve an issue during automated squashes appending a tab
character to the start of each line in a commit message.

Closes-Bug: #1977479
Change-Id: I6f56500c366fd9610d98507f01e8af21862e4421
2022-06-03 09:21:25 -04:00
Stephen Finucane 6a6546a362 Use stdlib importlib.metadata where possible
The importlib_metadata package isn't guaranteed to be available, but the
stlib version will always be there (at least on Python 3.8 or later).

Change-Id: I532a365aa0f1a3206edf6b01e821bd83ea04ab3c
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2022-05-18 17:41:28 +01:00
Hervé Beraud 5ded0ee1ff Adding python classifiers py38 & py39
Change-Id: Ib8e251ae9dd93a5eb356478d633f1fef271911d9
2022-05-05 12:06:18 +02:00
Jiri Podivin b0a1b7209d Future-proofing pyproject.toml
Setting the 'py-modules' key to empty list should prevent
reemergence of setuptools related issues[1] should the toml
file be ever introduced.

[1] https://github.com/pypa/setuptools/issues/3197

Related-Bug: #1966382

Depends-On: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/835425

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: I89985a68cc497fd26a30058778fc1a1a52f6d96a
2022-03-28 11:01:23 +00:00
Zuul e6ee869737 Merge "Changed minversion in tox to 3.18.0." 2022-03-22 14:04:04 +00:00
Zuul 359c73cb81 Merge "Upgrade the pre-commit-hooks version" 2022-03-22 14:03:42 +00:00
Zuul 064ee4523d Merge "Drop wheel from pyproject.toml examples" 2022-03-22 14:03:40 +00:00
Clark Boylan 5c8461013f Use importlib-metadata for runtime package version lookups
We were using pkg_resources for runtime package version lookups but
pkg_resources incurs a pretty significant cost to use it in thise way
(because it scans all installed packages on disk and sorts them).

importlib-metadata is the way of the future (part of stdlib in
python3.8) and is supposed to be significantly more performant. Replace
our use of pkg_resources for runtime version lookups with
importlib-metadata so that we are both quicker and future proof.

Note that this doesn't handle the problem of no pbr deps yet. We will
probably end up needing to vendor importlib-metadata in pbr or something
similar.

Change-Id: Ife68089d997b266adb37f18e226f49bdd67766fc
2022-03-01 11:02:30 +01:00
Jeremy Stanley 571e259969 Drop wheel from pyproject.toml examples
Per https://github.com/pypa/setuptools/pull/3056 the addition of
wheel in build_system.requires is redundant since SetupTools already
drags it in anyway. Simplify our recommendations accordingly.

Change-Id: I0359ed11a6f0c54dc1af6a7f5f34e49476c58fb0
2022-02-28 14:35:37 +00:00
Daniel Bengtsson 479a2954a1 Changed minversion in tox to 3.18.0.
The patch bumps min version of tox to 3.18.0 in order to
replace tox's whitelist_externals[1] by allowlist_externals option.

[1] https://github.com/tox-dev/tox/blob/master/docs/changelog.rst#v3180-2020-07-23

Change-Id: I7ee77638d44ee314cdfa6f6d9320b19d794b718c
2022-02-16 12:05:40 +01:00
Zuul 97e7a60f42 Merge "setup.cfg: Replace dashes with underscores" 2022-02-08 12:12:34 +00:00
Jeremy Stanley 407d921751 Add release note about missing pbr.json fix
Include a release note for I407ae88ab8de4b61f94034b3d79a2ca7f7d79d16
which corrected a regression for missing pbr.json metadata files in
packages.

Change-Id: I3f0e48c215e890f9b26bae6600d5fcacf67831c6
2022-02-05 16:06:59 +00:00
Jeremy Stanley 949c47c979 Avoid recursive calls into SetupTools entrypoint
Add a check in pbr.core.pbr() to make sure it hasn't already run,
and return immediately if it has. This replaces the earlier attempt
at loop-busting by setting the entrypoint to None in the dist, in
order to support PEP 517 builds without setup_requires.

Also test that pbr.json is now included in the dist tree, in order
to avoid future regressions.

Change-Id: I407ae88ab8de4b61f94034b3d79a2ca7f7d79d16
2022-02-04 23:36:13 +00:00
yangyawei 09417bf0a3 setup.cfg: Replace dashes with underscores
Setuptools v54.1.0 introduces a warning that the use of
dash-separated options in 'setup.cfg' will not be supported in a
future version [1]. Get ahead of the issue by updating to canonical
configuration[2]. Without this, we see 'UserWarning' messages like
the following on new enough versions of setuptools:

  UserWarning: Usage of dash-separated 'description-file' will not
  be supported in future versions. Please use the underscore name
  'description_file' instead

[1] https://github.com/pypa/setuptools/commit/a2e9ae4cb
[2] https://setuptools.pypa.io/en/stable/userguide/declarative_config.html

Change-Id: I9c51ffdeb548833f2cc303d2e53ba169756690d3
2022-01-21 10:54:56 +01:00
Ben Greiner 397b6fcdf9 remove explicit mock
The explicit mock package is not required if you can use unittest.mock
from python >= 3.3

Change-Id: I1e3a764b38be66b994d790768bc5eb9be4237444
2022-01-21 10:44:41 +01:00
Jeremy Stanley c718425299 Don't test with setuptools local distutils
We noticed some aberrations in log level/threshold handling as of
Setuptools 60.0.0, and the only substantive change for that release
toggles the default selection for its internal ("local") distutils
replacement vs the "stdlib" one. For now, toggle the
SETUPTOOLS_USE_DISTUTILS envvar in our testing back to the old
value. This should be revisited once logging is handled better in
Setuptools. https://github.com/pypa/setuptools/issues/3038

Change-Id: If372a33607fff884cd7bf888ee5a23a2fbb351b2
2022-01-20 21:01:47 +00:00
Jeremy Stanley b6b204e33e Use context blocks for open() calls in packaging
Two unclosed file descriptors in packaging are raising
ResourceWarning on termination in newer Python interpreters. Make
sure they're closed as soon as we're done with them.

Change-Id: I07f500e1157da18de799eb0ff9bf82d3bcb46510
2021-11-19 18:36:32 +00:00
Zuul 8cd1a06e82 Merge "Allow PEP517 without setup_requires" 2021-11-13 03:06:29 +00:00
Clark Boylan a6006f6085 Add python2 testing back to PBR
I think this was lost when we expanded the openstack-python job
templates to include more versions of python3. Oops.

Change-Id: I0ebaff67634857216ca8d565311ed7c63f687a16
2021-11-12 10:01:14 -08:00
Clark Boylan de0c717b17 Allow PEP517 without setup_requires
When using PBR PEP517 support without setup_requires were were
infinitely recursing because Distribution.finalize_options calls pbr()
and pbr() calls Distribution.finalize_options. With a setup_requires
this doesn't seem to happen because something is resetting dist.pbr's
value from True (callable) to None (not callable) after the initial
pbr() call and before further recursion can occur.

Explicitly enforce this behavior in pbr() via an explicit update of
dist.pbr to None to avoid calling pbr() more than once resulting in the
unwanted loop.

Change-Id: I1f2b4d34e587389f7e11b99d000e14477cf5091b
2021-11-12 09:16:04 -08:00
Clark Boylan d21c2caf1d Clarify the need for setup.py with PEP517
There was a misunderstanding that PBRs PEP517 build hooks meant we no
longer needed a setup.py. But it turns out the setuptools build_meta
hooks that we wrap call out to setup.py. This means PBR will continue to
need to be used in conjunction with a setup.py.

This is better for backward compatibiltiy anyway so not a huge loss.

Document this more properly and update the test to reflect it.

Change-Id: If4320da1e89a5c765e0bfe56b724364f9cba34f3
2021-11-08 08:56:18 -08:00
Monty Taylor 09ee153410 Add a PEP517 interface
pep517 defines a new module method of specifying build backends.
To allow pbr to exist in this world, we should define the interface
that's needed. For this to be used, one will put:

  [build-system]
  requires = ["pbr>=5.7.0", "setuptools>=36.6.0", "wheel"]
  build-backend = "pbr.build"

Into pyproject.toml - and the pep517 interface will be used.

This doesn't really change anything else - it just makes us support this.
So by itself this commit isn't SUPER helpful. But maybe let's take baby
steps with something this prone to strife, yeah?

After this we can start teasing some things apart and doing our
own things directly.

Co-Authored-By: Clark Boylan <clark.boylan@gmail.com>
Change-Id: I293f59b5074a38c78adffe580de2f1533bb01ce7
2021-11-03 10:05:32 -07:00
Clark Boylan 8c0d5c3141 PBR package testing improvements
We simplify virtualenv setup for PBR package installation testing by
switching to a simple file path for the PBR installation rather than a
full url with egg specifier. The egg specifer isn't necessary and
neither is the url and simpler is easier to understand.

We also add the verbose flag to the pip installations of tools into that
virtualenv to better understand problems when they occur.

Finally we add the option to skip writing package files if their
contents are empty. This allows us to ensure that we can omit certain
files which will be useful for the next change.

Change-Id: I1f081d29d62c905c1bb1abee5402b0990c83e948
2021-11-03 10:01:43 -07:00