Commit Graph

192 Commits

Author SHA1 Message Date
Stephen Finucane 69e21be288 Remove six
Change-Id: I1d6e583f41f95be7c2af232a2f22ee20fa83c4be
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2024-01-24 15:49:09 +01:00
Hervé Beraud fac394f79c Stop to use the __future__ module.
The __future__ module [1] was used in this context to ensure compatibility
between python 2 and python 3.

We previously dropped the support of python 2.7 [2] and now we only support
python 3 so we don't need to continue to use this module and the imports
listed below.

Imports commonly used and their related PEPs:
- `division` is related to PEP 238 [3]
- `print_function` is related to PEP 3105 [4]
- `unicode_literals` is related to PEP 3112 [5]
- `with_statement` is related to PEP 343 [6]
- `absolute_import` is related to PEP 328 [7]

[1] https://docs.python.org/3/library/__future__.html
[2] https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html
[3] https://www.python.org/dev/peps/pep-0238
[4] https://www.python.org/dev/peps/pep-3105
[5] https://www.python.org/dev/peps/pep-3112
[6] https://www.python.org/dev/peps/pep-0343
[7] https://www.python.org/dev/peps/pep-0328

Change-Id: I15d451f6a832b2e698b28e7351a36ef7aea92abe
2020-06-02 20:56:04 +02:00
Sean McGinnis b57da6e888
Add tool to generate a health report for all reqs
This adds a tool script that loops through all of our requirements and
output basic information for each one along with any things we might be
concerned about with each one. For now it just checks a few things, but
we could extend this issue checking as we find other things to check on.

Also refactors the unused package tool to put some common things into a
shared function library used by both scripts.

Change-Id: I140c3a5fb71a8fb82ccb696bade5be13bb9b0ba0
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-05-08 11:14:54 -05:00
Sean McGinnis 1bba56441d
Add script to help check for unused packages
This adds a script that will check through all the requirements in
global-requirements for any that are no longer being used.

Change-Id: I9c8fb2c07c794739bcab07a55e68e67d16736acc
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-05-01 15:47:18 -05:00
Matthew Thode 698b571749
mark regex as regex for pycodestyle
Change-Id: Id66a3e4d574fa4da5fbffe6a1aa24c5ea23ab8cc
2019-12-13 23:17:32 -06:00
Sean McGinnis 58d622019d
Remove requirements-integration job
This job was a legacy job that attempted to verify each project could be
independently installed after all global-requirements had been
installed.

This job is no longer necessary since we have much better coverage with
requirements-check and other jobs that make sure the requirements are
compatible. It is also a legacy zuulv2 job, uses the long deprecated
devstack-gate, and runs with Python 2.7.

For historical reference, this was added originally to PBR back in 2013
[0] and moved to requirements in 2014 [1].

[0] 378261a8cb
[1] 8e073738db

Change-Id: I29abcba929995f9307c392110160d889a29b47c3
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-11-12 11:02:50 -06:00
Ian Wienand ce33bf27e5 Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I53162497c2f53e45139e6a3aa9ff20538ec1b35e
2019-03-24 20:36:01 +00:00
Zuul f0d276428f Merge "Revert "Remove networking-odl from g-r in the requirements-integration job"" 2018-09-21 12:01:36 +00:00
Tony Breeds 8bf1b46bd8 [tools/grep-all] Hide eol releases by default
As we collect more history displaying all the EOL releases becomes noise
to filter out.  Hide eol tags by default but provide a quick way to
display them again if wanted.

Change-Id: I89d0c9ae2fa2e94505d44bfa8d2e0b0f00e1d842
2018-09-12 13:26:19 +10:00
Tony Breeds 3314a6dbca [tools/grep-all] Sort the open branches by the series name
Currently we don't sort the open branches which leads to them printing
in an order up to git.  Lets sort them by the series name so we have:

 1.2.0-1052-ga00761ff  : requests  # Apache-2.0
 origin/master         : requests  # Apache-2.0
 origin/stable/rocky   : requests  # Apache-2.0
 origin/stable/queens  : requests>=2.14.2  # Apache-2.0
 origin/stable/pike    : requests>=2.14.2  # Apache-2.0
 origin/stable/ocata   : requests>=2.10.0,!=2.12.2  # Apache-2.0
 origin/stable/newton  : requests>=2.10.0  # Apache-2.0
 mitaka-eol            : requests>=2.8.1,!=2.9.0  # Apache-2.0
 liberty-eol           : requests>=2.5.2,!=2.8.0,!=2.9.0
 kilo-eol              : requests>=2.2.0,!=2.4.0,<2.8.0
 juno-eol              : requests>=2.1.0,!=2.4.0,<=2.2.1
 icehouse-eol          : requests>=1.1,<=2.6.0
 havana-eol            : requests>=1.1
 grizzly-eol           :
 folsom-eol            :

Which I think is easier to parse as the lower on the list the older.

Change-Id: Ib436bc5cc1bd3f13176485909003cc00cf348c1a
2018-09-12 13:26:19 +10:00
Matthew Thode e57eee29be Revert "Remove networking-odl from g-r in the requirements-integration job"
This reverts commit 3fe4d09d8f.

Depends-On: https://review.openstack.org/601488
Change-Id: Ie680485d06e4e83052693f777218793001680079
2018-09-11 04:19:47 +00:00
Tony Breeds 3fe4d09d8f Remove networking-odl from g-r in the requirements-integration job
This is a terrible idea!

Since networking-odl 13.0.0 hit pypi which includes I955c9d737864b8a2557366be518922e192f4be91
the integration gate fails because ceilometer isn't on pypi[1].  There
aren't any good fixes but this *should* at least unblock the,
requirements, gate and buy us time to fix it correctly.

Note:
 * We can't just install ceilometer from git as we'll just fail when we
   try to generate a constraints list anyway.
 * We can't blacklist networking-odl as the backlist isn't used for
   generating a freeze only for filtering the output constraints
 * We can't stop building the constratints as that introduces a new
   failure path in the gate.

So with those options ruled out lets just pretend that networking-odl
isn't in global-requirements.  This will still fail if something
(directly or not) installs it from pypi.

[1] http://logs.openstack.org/96/594496/2/check/requirements-integration/8378cd8/job-output.txt.gz#_2018-08-31_22_54_49_357505

Change-Id: I1f117f9f4d8f49b5c4ef0cb74d98560c9a551999
2018-09-05 06:19:36 +00:00
Piotr Bielak de2adc5377 Fixed "pip-install" test
The "pip-install" tox environment was failing, because the name
of the "ConfigParser" package was changed into "configparser".
This commit fixes that by using the six.moves module to do the right
thing on py2 and py3

Change-Id: I6b16c8f0e182850cda041ca294edc5ad04c1a3c3
2018-08-01 12:56:57 +10:00
Tony Breeds 438312aea5 Fix grep-all after we removed minimums from global-requirements
Change-Id: I18d22247be4d6859615fb95d96287400889ea2f8
2018-06-29 09:25:37 +10:00
Brian Haley 08f1ef0141 Fix typo in fix-lower-constraints.py
The argument when creating the virtualenv is a directory,
so use it when activating.

Change-Id: I74b608e88407354ec9889805d8881d718ade998c
2018-06-07 12:30:41 -04:00
Zuul 3176586927 Merge "Trivial: Update pypi url to new url" 2018-04-21 05:46:48 +00:00
melissaml 62728e1927 Trivial: Update pypi url to new url
Pypi url changed from [1] to [2]

[1] https://pypi.python.org/pypi/<package>
[2] https://pypi.org/project/<package>

Change-Id: I69093744a6a96c8fc0b9cd361db6ce1f64f19b0b
2018-04-21 02:32:36 +00:00
Doug Hellmann 8d4eb15136 script to fix lower constraints entries that do not match local requirements
We have a few projects that have lower constraints lists that do not
match their own local requirements settings. This script tries to fix
that by changing the constraints to match.

Change-Id: I5f9bfc30e5dab855c9cef77cd9c4063658aab4c1
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-04-07 21:27:40 -04:00
Tony Breeds ac724d07bd [tools] include newton-eol in tools/publish_constraints.sh
Change-Id: I0716a1c5bd28dc4ceac9c7f570f4aa28a39673cd
2018-02-14 18:42:23 +11:00
Tony Breeds 351b878093 tools: Simplify grep_all.sh
Branches and tags are really just refs in git so rather than
open-codeing that and having 2 for loops just call everything a ref.

While we're there sort tags in reverse order this has the cosmetic
advantage that series are chronological order ie:

    1.1.0-1215-g8dc48120  : openstackdocstheme===1.18.1
    origin/master         : openstackdocstheme===1.18.1
    origin/stable/newton  : openstackdocstheme===1.5.0
    origin/stable/ocata   : openstackdocstheme===1.6.1
    origin/stable/pike    : openstackdocstheme===1.16.1
    folsom-eol            :
    grizzly-eol           :
    havana-eol            :
    icehouse-eol          :
    juno-eol              :
    kilo-eol              : openstackdocstheme===1.1.0
    liberty-eol           : openstackdocstheme===1.2.6
    mitaka-eol            : openstackdocstheme===1.3.0
becomes:
    1.1.0-1215-g8dc48120  : openstackdocstheme===1.18.1
    origin/master         : openstackdocstheme===1.18.1
    origin/stable/newton  : openstackdocstheme===1.5.0
    origin/stable/ocata   : openstackdocstheme===1.6.1
    origin/stable/pike    : openstackdocstheme===1.16.1
    mitaka-eol            : openstackdocstheme===1.3.0
    liberty-eol           : openstackdocstheme===1.2.6
    kilo-eol              : openstackdocstheme===1.1.0
    juno-eol              :
    icehouse-eol          :
    havana-eol            :
    grizzly-eol           :
    folsom-eol            :

Change-Id: If32cb11f1bcd3aefdc3717d3e943b0fa15a42002
2018-01-24 16:07:20 +11:00
Doug Hellmann 260d100eb6 show values at -eol tags as well as on branches
Sometimes we want to go further back in history than the current open
branches, so include the -eol tags as well.

Change-Id: Idb7ca9d0f4dd2f7dc6701577cc495a8010ee1340
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-01-22 16:55:31 -05:00
Doug Hellmann 5251018b60 add argument error handling to grep-all.sh
Change-Id: I172ca7902af16069ab3da080ab5a1b7a325cc50e
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-01-22 16:55:07 -05:00
Tony Breeds 0592ee179b [tools] Add a tool to grep local repos
Sean wanted to know which repos still use mox/mox 3 which can be done
with this tool like:

tools/code-search.sh --prefix ~/projects/openstack/ \
                     --projects ~/projects/openstack/*/* \
                     -- mox origin/master -- *requirements.txt setup.cfg

The prefix is essentially a sting to remove from the project name.  So
the example above would change:

/home/tony/projects/openstack/openstack/astara => openstack/astara

Change-Id: I927bc96b6eafd2d95dc008b7f1a4374eb9f725b4
2018-01-10 11:59:08 +11:00
Andreas Jaeger f4f57e0024 Handle doc/requirements.txt
Change Ief487e1d20ea20facf398367ca03a6385182fb38 updated the PTI for doc
jobs and introduces doc/requirements.txt. Support that for requirements
syncing.

Change-Id: I147a129438ae370eb4c90ee93a634fa61cc179f4
2017-11-23 20:09:01 +01:00
Tony Breeds 1627c8ad63 [tools] Add a tool to make a no-op change
I used something like this when testing reno 2.5.0.  It's clearly a work
in progress but I figure we could all fix/test it rather than me having
another half implemented tool

Change-Id: I2fa844f4cf8dd7e5c06461b7f7926a7cf84515c3
2017-10-12 13:42:59 +11:00
Tony Breeds bf3ce4dc03 [tools/grep-all.sh] Get branches from git
Rather than hard code the branches to look at just do all the stable
branches that exist.

Change-Id: I8beb6ed5e537f37e707d1e04ad9c1174fd608544
2017-09-28 12:42:27 +10:00
Tony Breeds d827e71b22 Add a bashate tox environment and address errors
Add a bashate environment for checking shell code quality.
Exclude bashate warnings:
 E006: Line too long:
 E010: The "do" should be on same line as for

In my opinion the places that trip these do not impact code readability

Change-Id: Iab0a34542c3938f127515291eca8f027f3accaa0
2017-09-28 12:09:04 +10:00
Tony Breeds 5a16177116 Add a tool to generate the upper-constraints.txt file for publishing
Change-Id: I25a381e55362df0e7b31f18ff45c95ce3a28dc6a
2017-09-27 09:14:04 +10:00
Tony Breeds 6c75ce543e [tools] Add a README.txt
Change-Id: I1daba8a6852255daca62c0c772a44b822360a84e
2017-09-20 16:11:20 -04:00
Tony Breeds 5b32f5237b [tools] Add pike and remove mitaka from grep-all
We now have a stable/pike branch and mitaka is EOLd (though we can't yet
remove the branch because there are still projects that are using it :(

Update the grep-all script to look in releases we care about.

Change-Id: Ia3eeda9db6b5715a3bfff5fe4f0ccd3d50ebf653
2017-08-15 23:49:00 +00:00
Tony Breeds 2b1d8f9152 Remove bogus comment
When I added the license to grep-all I copied it from another shell file
clearly no one read it as it describes tools/cruft.sh

Change-Id: Ia48d5000a360519be596c09273ba653d10e3daa6
2017-08-03 15:18:27 +10:00
Dirk Mueller 958b700317 Use git grep -h for skipping the filenames
That seems to be more portable accross different environments
(aka make it work for dirk)

Change-Id: I4eee201ed0114cf7aa9f9be39d1335326691bf99
2017-02-21 16:19:54 +01:00
Tony Breeds a4edd9d11a Add Ocata branch remove Liberty
Change the branches looked at to match the current tracked releases.

Change-Id: Ie406b3743c296905c8db196797b5c632d4d91dca
2017-02-16 11:52:03 +11:00
Tony Breeds b4e3ac5714 Narrow down the results/matches
If you run grep-all.sh requests, you get all the item that match that
prefix.  It is a little messy.  Make the match patten more specific so
you only get the library you pass on the command line.

Change-Id: I16544372f7efff28933da728d11f85dc5e5f9aea
2017-02-16 11:51:56 +11:00
zhangyanxian 7148c8eeb2 Fix typo in what-broke.py
"programatically" should be "programmatically"

Change-Id: I4ab60ddc688e48a76862d739fddfcfe711ad19eb
2016-12-19 07:13:35 +00:00
Jenkins 5a4f776506 Merge "Fix "wrap functions with 2 blank lines" pep8 check" 2016-11-16 15:28:57 +00:00
Cady_Chen 6898f5d20d Fix "wrap functions with 2 blank lines" pep8 check
Now gates are requires this pep8 rule as mandatory so until it'll be
fixed merge is blocked.

Change-Id: I5b1ad7ec820911b5e765e184223f786c733836a5
2016-11-16 18:52:03 +08:00
Tony Breeds e699ece94c Add a tool do dump requirements and constraints for all branches
Often it's helpful to quickly compare the requirements and constraints
for a given library on all branches.  This tool does that.

Change-Id: I2458c5929d8812cd74f657050eea5f4f3fa11df0
2016-11-16 11:59:00 +11:00
Ihar Hrachyshka b97f008059 Use python3 to calculate freezes
We need the integration gate to work with both trusty (python3.4) and
xenial (python3.5).  Rather than hard code the full python binary just
use python 3 and let the OS mange this for us. This is less than ideal
but it's the best we can do right now to unblock the gate.

We also address a short term issue with out upper-constraints.txt files
and manually copy the constraints from 3.4 to 3.5

Related-Bug: 1620436
Change-Id: I6acabf86933b7a7bce7baf44e6b512e7d0d2f6db
2016-09-06 03:34:23 +00:00
Andreas Jaeger 1f3975247a Move other-requirements.txt to bindep.txt
The default filename for documenting binary dependencies has been
changed from "other-requirements.txt" to "bindep.txt" with the release
of bindep 2.1.0. While the previous name is still supported, it will
be deprecated.

Move the file around to follow this change.

Note that this change is self-testing, the OpenStack CI infrastructure
will use a "bindep.txt" file to setup nodes for testing.

For more information about bindep, see also:
http://docs.openstack.org/infra/manual/drivers.html#package-requirements
http://docs.openstack.org/infra/bindep/

As well as this announcement:
http://lists.openstack.org/pipermail/openstack-dev/2016-August/101590.html

Change-Id: If4db21748c4cf2b457e02efdd92b33eee44e2319
2016-08-13 16:45:13 +10:00
Tony Breeds 9a428f3ddb Check for missing run-time requirements
This change creates a new tox environment that *only* installs
openstack_requirements and then verifies that each of the console
scripts has all of the modules that it imports.

This will need to be added to our gate RSN

Change-Id: Ibc37593afcc4d9f820cb88168e1aa15e773b2087
2016-07-26 19:46:49 -05:00
Davanum Srinivas 836a390b91 Add semi-colon as separator for parsing package names
If we don't do this then the following line:
weakrefmethod;python_version=='2.7' or python_version=='2.6' # PSF

ends up as:
weakrefmethod;python_version

in the cruft output. So we need to add ';' to get the output
as 'weakrefmethod'

Change-Id: I7b32bfe70df3bb24ca301db9952d76451674a0db
2016-05-06 16:05:17 -04:00
Andreas Jaeger e3305bf513 Babel Integration tests
With Babel 2.2 and then with Babel 2.3.2 and 2.3.3 releases, the
OpenStack infra scripts broke and uncovered bugs in the upstream
Babel release. The infra scripts run in post and periodic queues, so
failures here are not directly visible. With Babel 2.3.x invalid strings
were send to the translation server, and we had to recover old content
to fix this.

Let's add some Babel integration tests that can be run whenever the
upper-constraints file changes so that any new Babel release will be at
least minimally tested.

The shell script babel-test.sh runs a simple extraction of strings to
translate from a simple input file. It uses the same commands that the
infra scripts run and handles the two different cases of extraction of
the normal strings and extraction of the non-standard log-level strings.

The script then compares the new content with previously created good
content.

Once this change is in, we will run this test whenever
upper-constraints.txt file changes.

Needed-By: I15fa6a706323615d6072d8acaa859fea9bf35883
Change-Id: Iab7844ba748ecd84ec07ad4280b2317bd626fe9a
2016-04-24 17:21:18 -05:00
Swapnil Kulkarni (coolsvap) 533afe3671 Keep py3.X compatibility for urllib
Change-Id: Ib243de3c6dfef9fdc3e8d73629dc94cf45ee648b
Partial-Bug:#1280105
2016-01-05 08:39:43 +05:30
Jenkins eeb5a1d827 Merge "Stop special casing jeepyb and pypi-mirror in test" 2015-10-23 18:29:02 +00:00
Doug Hellmann cd88f6c3f6 do not commit requirements syncs
We need to test without committing the requirements changes because pbr
throws an error if the pre and post versioning do not match. The first
change on a stable branch must be the one to remove the pre-versioning
configuration.

Change-Id: I812af614b7a6c7f499c67310f078ac4835b9e91a
2015-10-16 14:27:08 +00:00
Clark Boylan 7fe60b6f33 Stop special casing jeepyb and pypi-mirror in test
We have removed these projects from the PROJECTS list so no longer need
to special case them here.

Change-Id: Ic3116a646389438ded03f285819529477bf849d8
Depends-On: I61c2f3f8a724dc789479278e7ba493e095c52305
2015-10-15 10:44:43 -07:00
Jenkins 96c547c1ed Merge "Move integration test dependencies into bindep" 2015-07-29 04:07:22 +00:00
Robert Collins 4c028dba0a Move integration test dependencies into bindep
bindep makes this a lot easier to work with as well as making it
possible to support Fedora, Suse etc as needed.

Change-Id: I4d19df8bbcd08e06edcf71ee51c30bb9c3d57fd0
2015-07-19 08:37:11 +12:00
Jenkins b2788ceb19 Merge "Revert "Move integration test dependencies into bindep"" 2015-07-18 16:03:11 +00:00