Commit Graph

54 Commits

Author SHA1 Message Date
Stephen Finucane f3b1c7533e Correct comment for 'D1_D2_SETUP_ARGS'
Change Ic7b71f1093671533c9f17fd621e02299bfbd7a6d reversed the order of
this list but didn't update the comment. Do so now.

Change-Id: I36f2dd791297a9e3df4f1e427db5b45017820bbf
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-04-23 10:05:50 +01:00
Stephen Finucane 607b4e0547 Reverse ordering of 'D1_D2_SETUP_ARGS'
The previous ordering was confusing and placed new values before old
ones. Switch this order, making it more obvious what we're going *from*
and what we're going *to*. With this reordering, the old name no longer
makes sense so we rename the attribute.

Change-Id: Ic7b71f1093671533c9f17fd621e02299bfbd7a6d
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-04-22 10:00:38 +01:00
Stephen Finucane f40c41d28f Add test for cfg -> py transformation
Change-Id: I176dc6a2c6d8f37717e1e17ba41ee72bef9536fb
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-04-22 10:00:38 +01:00
Stephen Finucane 16a8553e9f Don't pass empty 'long_description'
Correct a small logic error that would result in us passing an empty
'long_description' value to 'setup' if 'description_file' pointed to a
file that was empty. Fixing this also makes it a little more obvious
that we are only setting values that are present in 'setup.cfg', since
that wasn't totally clear before.

A small, unrelated typo is addressed.

Change-Id: Ib32b5fb5da84215376c8e372470b2ff08eb47908
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-04-22 10:00:38 +01:00
Stephen Finucane a96a094c39 Map requires-python to python-requires (attempt 2)
Attempt two at the problem, this time done properly. As discussed in
change I7158e75934535b80eb60d6b4b7dcef08866b5bab, the previous attempt
actually mapped 'python_requires' to 'requires_python' rather than the
other way around. This resolves that.

Change-Id: I415b64d5b51a248d2f45a28234f1635e8b82ffcc
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-04-08 15:54:07 +00:00
Stephen Finucane 1d6ae9c59a util: Convert 'D1_D2_SETUP_ARGS' to a list of tuples
Dicts can't have duplicate keys. Lists of tuples can. We're going to
need duplicates.

Change-Id: Id43ba91585160448ad9737ba958ace5163d26f3d
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-03-15 17:25:01 +00:00
Stephen Finucane 5195593256 Revert "Map requires-python to python-requires"
This reverts commit d5da2121f8. This was
done incorrectly. The 'D1_D2_SETUP_ARGS' dictionary modified here is a
mapping of distutils (and by extension, setuptools) supported options to
their legacy distutils2 equivalents, and is used to transform the latter
to the former. For example, consider the following entry:

  "long_description": ("metadata", "description"),

This will transform an entry like so in a 'setup.cfg' file:

  [metadata]
  description = ...

Into the following equivalent call to setuptools' 'setup' function:

  setup(
      ...,
      long_description='...',
      ...
  )

By that measure, the entry added in this change, namely:

  "requires_python": ("metadata", "python_requires"),

Will result in a 'requires_python' argument being added to the 'setup'
call. That's not what we want. Rather, we likely want to transform a
'[metadata] requires_python' entry to a 'python_requires' argument to
'setup'.

The change is reverted here in expectation of a forthcoming proper fix.

Change-Id: I7158e75934535b80eb60d6b4b7dcef08866b5bab
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2021-03-15 17:18:07 +00:00
Jeremy Stanley 242f7bd1b6 Remove use_2to3 backward compat for Setuptools
Odds are this was never used by any PBR-based projects, but was
included from the beginning to support a feature of Setuptools for a
utility which has been long since discouraged. As of Setuptools
46.2.0 and Python 3.9 it's generating deprecation warnings, and is
slated for removal from the Python 3.10 standard library.

https://bugs.python.org/issue40360
https://github.com/pypa/setuptools/issues/2086

Change-Id: I6972984f29413d2534d802b142fa9323a7e4a4a3
2020-09-19 14:07:01 +00:00
Monty Taylor d5da2121f8 Map requires-python to python-requires
Apparently requires-python is the old value and isn't valid
anymore. What people mean is python-requires. Of course, people
are still using the old value. Fix it for them.

Change-Id: Id49cf641f2eddac17f1b21b186aa3a150bd863f9
2020-08-06 08:26:54 +00:00
Clark Boylan 206b795517 Update hacking
This allows us to run tox -epep8 on newer python versions. Fix the
issues newer hacking calls out. They were complaints about indentation
and ambigious l value variable names.

Change-Id: I9552fba791bdf54649fbe25fc26bffdf8186ff0a
2020-08-05 15:01:21 -07:00
Stephen Finucane 57d7c53739 Update to latest hacking
Change-Id: If73238a39d0934c6f577fc52132fcbc63e8eb25f
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2019-08-27 16:57:23 +01:00
Zuul e4c7dba0bd Merge "Fix Windows support" 2019-06-17 18:19:29 +00:00
Lucian Petrut a7e5c0202e Fix Windows support
A recent commit [1] broke Windows support by using shlex to
unquote paths.

The issue with shlex.split is that it doesn't work with Windows
paths, treating backslashes as escape characters.

We'll just replace backslashes with slashes before using shlex.split,
converting them back afterwards.

Closes-Bug: #1831242

[1] Id2cc32e4e40c1f834b19756e922118d8526358d3

Change-Id: Icb3abca004a35ab9760db8116fedfa96d012d0d0
2019-05-31 15:35:45 +03:00
Ben Nemec 3b102a551b Read description file as utf-8
Currently pbr fails if the description file contains unicode
characters.  To fix this we need to open the description file as
utf-8 explicitly.  Since open() in Python 2 doesn't support an
encoding parameter, use io.open() which works on both 2 and 3.

Co-Authored-By: Hervé Beraud<hberaud@redhat.com>

Change-Id: I1bee502ac84b474cc9db5523d2437a8c0a861c00
Closes-Bug: 1704472
2019-05-24 10:59:17 +02:00
Will Szumski 50356da839 Fix white space handling in file names
Previously, when using data_files with a glob that matched a file with
whitespace in the name, pip would error with a message that the file
does not exist, e.g:

    error: can't copy 'ansible/roles/yatesr.timezone/templates/timezone-Arch': doesn't exist or not a regular file

The problem was that ansible/roles/yatesr.timezone/templates/timezone-Arch
was a truncated form of the actual filename:

    ansible/roles/yatesr.timezone/templates/timezone-Arch Linux.j2

Note the space in the filename and that it has been split on this space.

This change allows you to use a glob that matches files with whitespace
in the name. It does this by quoting the path.

Change-Id: Id2cc32e4e40c1f834b19756e922118d8526358d3
Fixes-Bug: 1810934
2019-04-25 11:25:56 +01:00
Ben Nemec 7f9d653c29 Support provides_extra metadata
This was added to setuptools in [1] so we should support passing it
through from pbr.

Change-Id: I46493c18756bcb01c53575ab51ec5e5e97670fc3
1: b2ea3c4a20
Closes-Bug: 1760938
2019-03-25 16:39:59 +00:00
Hervé Beraud 2e3aa0d851 Fix error when keywords are defined as a list in cfg
When keywords are defined as a list in cfg file the generated
output have errors and breaks metadata generation.

Change-Id: Ie8a5f30d6af1e81ecf3ca40bc94bc460cca38179
Closes-Bug: #1811475
2019-03-04 18:44:35 +01:00
Brandon LeBlanc e28fc7e870 Resolve ``ValueError`` when mapping value contains a literal ``=``.
Example ``setup.cfg``::

    project_urls =
      Documentation = https://format-pipfile.readthedocs.io/en/latest/?badge=latest

Current result::

    $ ./setup.py
    ERROR:root:Error parsing
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/site-packages/pbr/core.py", line 96, in pbr
        attrs = util.cfg_to_args(path, dist.script_args)
      File "/usr/local/lib/python3.7/site-packages/pbr/util.py", line 258, in cfg_to_args
        kwargs = setup_cfg_to_setup_kwargs(config, script_args)
      File "/usr/local/lib/python3.7/site-packages/pbr/util.py", line 336, in setup_cfg_to_setup_kwargs
        k, v = i.split(=)
    ValueError: too many values to unpack (expected 2)
    error in setup command: Error parsing /Users/brandon/src/format-pipfile/setup.cfg: ValueError: too many values to unpack (expected 2)

After changes::

    $ ./setup.py egg_info
    [...]
    $ grep -i project-url *.egg-info/PKG-INFO
    [...]
    Project-URL: Documentation, https://format-pipfile.readthedocs.io/en/latest/?badge=latest
    [...]

add unit tests

    {1} pbr.tests.test_util.TestMapFieldsParsingScenarios.test_project_url_parsing(simple_project_urls) [0.034230s] ... ok
    {1} pbr.tests.test_util.TestMapFieldsParsingScenarios.test_project_url_parsing(query_parameters) [0.029791s] ... ok

Closes-Bug: #1817592
Change-Id: Ifd4c46111528d99dadee77d6aabed201d9e84bdb
Signed-off-by: Brandon LeBlanc <brandon@leblanc.codes>
2019-02-25 17:09:55 +01:00
Zuul 6a88aadaa8 Merge "Correct documentation hyperlink for environment-markers" 2018-10-30 00:24:34 +00:00
Sean McGinnis 8080c62f81 Fix incorrect use of flake8:noqa
Adding the comment flake8:noqa in a file will skip linting the entire
file. Most of the time, the intent was just to skip individual lines to
handle exception cases.

This gets rid of the "flake8:" prefix where it was used incorrectly and
fixes a few legitimate errors that were being hidden by the entire file
being skipped.

The behavior is change in flake8 to handle this better, which will
result in pep8 job failures if these are not fixes first. See more
information in the 3.6.0 release notes:

http://flake8.pycqa.org/en/latest/release-notes/3.6.0.html#features

Change-Id: I1af4eb8bef9d0d0e3aa3bbbfd66a2fcb8ab336cb
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2018-10-26 13:43:14 -05:00
Sorin Sbarnea 656aac93c9 Correct documentation hyperlink for environment-markers
Previous hyperlink was redirected to root of the new documentation
location which did not contain the right information.

New link points directly to the right location.

Change-Id: I3c83553a50036eadec74e7a00f3b312a7e5afa11
2018-10-26 15:16:55 +00:00
Ben Nemec 74680aa5f1 Support python-requires metadata
Expose the python_require setuptools option via the python-requires
metadata item in setup.cfg.

Closes-Bug: 1735668

Change-Id: Idf2bb90882121e118f5b7531e027d350c0e4a441
Co-authored-by: Dennis Verspuij <dennisverspuij@users.noreply.github.com>
2018-06-01 18:02:55 +00:00
Ben Nemec 10fce39a7b Explicitly read setup.cfg as utf-8 on Python 3
Per the referenced bug, relying on the terminal encoding to read
setup.cfg is not safe.  Unfortunately, Python 2 doesn't accept an
encoding when reading config files so we need a fallback path for
that version.

Change-Id: If49344db2f9139c0557f6acd17671163e02468a5
Closes-Bug: 1745396
2018-03-27 10:54:57 +00:00
Stephen Finucane 1fe0ceab39 builddoc: Treat '[pbr] autodoc_tree_excludes' as a multi-line opt
We were treating this a single line, comma- or space-delimited option.
This was incorrect and was causing issues for projects like neutron-lib
when Sphinx 1.7, which switches from optparse to the stricter argparse
was used. In addition, this project was including comments in the
multi-line opt. These were being passed through but Sphinx < 1.7 was
simply ignoring them. These are now filtered out.

Change-Id: I177edf0f44714175da220cf3a960b8f23aa4ab09
Closes-Bug: #1753082
2018-03-26 12:40:20 +00:00
Zuul 8a9a6d80bf Merge "Support PEP 345 Project-URL metadata" 2018-01-08 13:54:33 +00:00
Monty Taylor 225e882322 Put test-requirements into an extra named 'test'
In bindep files we use a 'test' environment to indicate dependencies
that are needed for testing. Make the same thing available for our
python dependencies, allowing things like "pip install .[test]" or "pip
install shade[test]" to work.

Change-Id: If3ad8b6a79a8cab2f7434b73207f35384e8516ba
2017-12-06 11:01:52 +00:00
Jeremy Stanley 538d2c21a1 Support Description-Content-Type metadata
As of setuptools 36.4.0, support for Description-Content-Type
metadata was added via the long_description_content_type variable.
Add a metadata.description-content-type option for setup.cfg so it
can be provided through PBR. This makes it possible to instruct PyPI
and potentially other frontends to know an explicit RFC 1341 content
type, along with character set and variant, for the accompanying
long description.

Change-Id: I852f603d9ca4e4cd469eab4cae9933ef2581e3af
2017-12-04 19:46:41 +00:00
Jeremy Stanley 839ef3428d Support PEP 345 Project-URL metadata
By including one or more Project-URL entries in PKG-INFO metadata,
PyPI can display helpful hyperlinks in a generic manner. Add support
here to be able to pass it through setup.cfg with a project_urls
dict. See the corresponding section of the Core Metadata
Specifications from the Python Packaging User Guide for details:

https://packaging.python.org/specifications/core-metadata/#project-url-multiple-use

Setuptools implementation is underway here:

    https://github.com/pypa/setuptools/pull/1210

Change-Id: I14e580c654b619cab7eb24e31f736056d6cf9bd4
2017-11-22 23:55:17 +00:00
Stephen Finucane 32c90ba598 Remove support for command hooks
distutils2 is long dead and many of its best ideas have been
incorporated into setuptools. One of the ideas that *has not* been
incorporated is the idea of command hooks, of either the pre or post
kind. pbr is still carrying the code for this, and there are several
issues with this:

- No one is using this functionality in OpenStack and, given the
  complete lack of documentation on the matter, it's very doubtful that
  it's being used anywhere else [1]
- It's causing issues for projects attempting to hook into the
  'distutils.commands' entry point on Python 2.7, which it seems no else
  must have attempted yet [2].
- distutils2 is dead and advanced features like this that we don't
  explicitly need should not be retained

We could attempt to fix this but why bother? Good riddance, I say.

[1] http://codesearch.openstack.org/?q=pre_hook%5C.&i=nope&files=&repos=
[2] http://codesearch.openstack.org/?q=distutils.commands&i=nope&files=&repos=

Change-Id: I01f657034cffbf55ce830b7e8dbb6b3d94c1fd18
2017-10-09 10:48:33 +01:00
Stephen Finucane 87637bdf1d Remove dead code
The 'monkeypatch_method' function no longer has any callers and can be
removed.

Change-Id: I402eb482d2c1db7331bd872c5169db8e59c9a34f
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2017-10-06 16:12:52 +01:00
Rajath Agasthya 87514e3e10 Trivial: Fix docstring
Why not?

Change-Id: Ibe98c901b788a2629e1ba794484a02096f54d0b1
2017-06-25 11:36:55 -07:00
Jenkins 4c8bd90846 Merge "Clean imports in code" 2017-01-03 21:04:20 +00:00
Dirk Mueller 9fd7aa2cc7 Don't raise exception on missing man pages
The revert in Ia6cfbfe5b10a5b714fbb9f21ca61380aaf231638 actually
broke Sphinx 1.3.x support again. Try to fix it for real this
time by avoiding an exception on missing man_pages.

NOTE(dmllr): don't change dict while iterating over it, hopefully
this fixes the gating failure with python 3.5.x

Change-Id: I52d45fa0a0d42de690d3a492068f7bb03483a224
Related-Bug: 1379998
2017-01-02 23:08:30 +01:00
Cao Xuan Hoang 8f1e6dc73f Clean imports in code
This patch set modifies lines which are importing objects
instead of modules. As per openstack import guide lines, user should
import modules in a file not objects.

http://docs.openstack.org/developer/hacking/#imports

Change-Id: Iab667c04f4aa57d0c8c882144a550443dacc8f05
2016-11-17 09:45:29 +07:00
Cao Xuan Hoang 1454461714 Docstrings should not start with a space
As per OpenStack Docstrings guide lines [1]:
[H401] Docstrings should not start with a space.
[H403] Multi line docstrings should end on a new line.
[H404] Multi line docstrings should start without a leading new line.
[H405] Multi line docstrings should start with a one line summary
followed by an empty line.

[1] http://docs.openstack.org/developer/hacking/#docstrings

Change-Id: Ic943e837b8b4f55c2957643ad29dfa004b3c2eb3
2016-11-16 08:44:54 +07:00
Jenkins 2ccb2d21c0 Merge "Don't ignore data-files" 2016-10-02 01:47:53 +00:00
jiansong 77d9ab7d07 Deprecated warning for SafeConfigParser
tox -e py34 is reporting a deprecation warning for SafeConfigParser
/octavia/.tox/py34/lib/python3.4/site-packages/pbr/util.py:207:
DeprecationWarning: The SafeConfigParser class has been renamed to
ConfigParser in Python 3.2. This alias will be removed in future
versions. Use ConfigParser directly instead.
  parser = configparser.SafeConfigParser()

Closes-Bug: #1618666
Change-Id: Ib280b778938b64717ee1cf94efae2f7b553c8f5e
2016-09-11 23:15:54 -07:00
Sachi King a432bc2930 Don't ignore data-files
We're currently ignoring data-files, and it looks like the problem
would sometimes be present starting with 2796f9, 0.5.7, and always be present
from 04984a, 0.5.15.

This normalises all config keys from - to _ as soon as we read the
config, which means future access and modification does not need to
concern itself with the possibility of the key being a '-' instead '_'.

This should make it more difficult for code accessing/modifying values
in the config to clobber user set values or be unaware of them, like
in the case of the files hook.

As well, support download-url, but properly expose it as download_url.

Co-Authored-By: Julien Danjou <julien@danjou.info>
Change-Id: I062774c706b8f7339dda46689a226b80ae6ac277
2016-08-26 14:28:38 +10:00
Sachi King 11ffee94fa Do not silently drop markers that fail to evaluate
We are silently dropping requirements that do not parse, do not do this
and bubble the error up.

Change-Id: I5bfce7c7d0be2ccd880e78509856074a09c57166
Closes-Bug: 1554331
2016-03-09 11:26:43 +11:00
Sachi King d19459daa8 Support entry point patching on setuptools < 12
In patch 789888 we began relying on EntryPoint.resolve(), which only
exists in setuptools >= 12, breaking support for setuptools < 12.  This
includes the default virtualenv installations from may LTS distributions.
As ep.load with arguments is deprecated we will attempt to use
ep.resolve if it exists and fall back to ep.load(false) if it does not.
setuptools 11 is the odd release out, as this was when load with
arguments was deprecated, suggesting the use of ep._load.  We will
continue to use ep.load(False) here.

Change-Id: I272f76fef7e447378697a3980372191f4ffe1a2f
2016-01-21 13:22:06 +11:00
Sachi King 7898882fa5 Handle markers to support sdist on pip < 6
Old versions of pip do not properly evaluate blank envmarkers such as
':python_version>=2.7', so as long as we are not creating a bdist_wheel
we evaluate these immediately to support these old installations

Change-Id: I94a5f9bccd658a2529a727d99f5a2b79e6b00aa8
Closes-Bug: 1502692
2015-12-17 10:59:00 +11:00
Sachi King 379e42abde Add patch to properly get all commands from dist
Distribution().get_command_list() is not hooked by setuptools like
print_commands() is, as such we do not get a complete command list.

Here we perform the hook ourselves to prevent the need to depend on
a super new version of setuptools.

Change-Id: Ie6088aedb92713be4b8b9af4a4ce3d300a114d7b
2015-12-07 19:08:36 +13:00
Sachi King daa905cad4 Strip comments present in setup.cfg
When processing setup.cfg strip trailing comments from lines as this is
unsupported by setuptools.

Change-Id: I7712c07552b50830549644a6ff4167b90d897125
Closes-bug: 1494330
2015-09-14 18:26:11 +10:00
Monty Taylor 61a3a1ecdf Wrap env markers in parens
If there are boolean conditions in the env marker, the existing join
causes the extra tag to only be applied to the first of them, causing
carnage.

Change-Id: I7b735befae76b330f923276e1c48ef23816774d4
2015-07-10 21:24:35 +00:00
James Polley 2b29c4fc2b Teach pbr to read extras and env markers
This adds support for reading extras from setup.cfg.

It also adds support for handling environment markers, both in the
extras section and in install_requires and in requirements.txt.

Change-Id: I6fd8276012e65f82934df9c374613b1ce6856b5a
2015-05-15 21:31:21 +10:00
Jeremy Stanley 52fcd502bf Be safe, don't let friends use RawConfigParser
Replace uses of ConfigParser() and RawConfigParser() with
SafeConfigParser().

Change-Id: I78395eae381e1b911ab6b48c572665f4601ec315
2015-04-23 18:42:16 +00:00
Masaki Matsushita 07aee3241f Add truncated lincense text
Change-Id: I2da74726cd56f9d29123e8ee3ed79b97d312a27e
Closes-Bug: #1428211
2015-03-18 16:09:57 +09:00
Monty Taylor c01b8dae1e Port in git sha changes from 0.10 line
Stop including git sha in version strings

We include it in pbr.json now. Including it is contentious in the world
of python, and it's up for debate as to whether or not it provides value.

Write and read more complex git sha info

Instead of encoding the git sha into the version string, add it to
a metadata file. This will allow us to get out of the business of
arguing with pip and setuptools about version info. In order to make
this really nice, provide a command line utility called "pbr" that has
subcommands to print out the metadata that we're now including in the
egg-info dir.

Only import sphinx during hook processing

When pbr is imported to handle writing the egg_info file because of
the entry point, it's causing sphinx to get imported. This has a
cascading effect once docutils is trying to be installed on a system
with pbr installed. If some of the imports fail along the way, allow
pbr to continue usefully but without the Sphinx extensions
available. Eventually, when everything is installed, those
extensions will work again when the commands for build_sphinx, etc.
are run separately.

Also slip in a change to reorder the default list of environments
run by tox so the testr database is created using a dbm format
available to all python versions.

Integration test PBR commits

Make sure that if a PBR commit is being tested then we install and
use that source rather than the latest PBR release.

Change-Id: Ie121e795be2eef30822daaa5fe8ab1c2315577ae
(cherry picked from commit 65f4fafd90)
(cherry picked from commit cd7da23937)
Closes-Bug: #1403510
Co-Authored-By: Clark Boylan <clark.boylan@gmail.com>
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
Co-Authored-By: Jeremy Stanley <fungi@yuggoth.org>
2014-12-21 22:20:49 +00:00
Doug Hellmann 3eb741afcd Do not force log verbosity level to info
Forcing the log level to info causes pbr to emit status log messages
in a way that breaks commands like "python setup.py --maintainer" and
"python setup.py --name", which are useful when scripting operations
that work on packages.

Change-Id: I6607c96e171f2f8efa89bd701018f4af6ec9c15a
2014-02-05 12:28:01 -08:00
Monty Taylor d26d43014b Replace setuptools_git with a smarter approach
Implement a local egg_info command that only re-generates the
SOURCES.txt file when we need to. That is:

 - If there is no SOURCES.txt, make one
 - If we have run the sdist command, make one

Otherwise, leave well enough alone.

Also, skip doing any git processing if SKIP_GIT_SDIST is specified.

This should mean that consumers of our tarballs should not get screwed
by the need to inject git processing into the sdist.

Change-Id: I163b1c153d030e79b120600a2890edeb49e1fa90
2013-08-04 11:30:32 -04:00