We add the script used to do this. This script will grow in complexity
in future changes as we add logic to e.g. remove old version constraints
that are no longer necessary. There's probably more sorting to be done
here, but that can always be tacked on with a follow-up.
Psst, reviewers: you can ensure we haven't lost/gained any dependencies
with this little script:
#!/usr/bin/env python3
import pathlib
import subprocess
import sys
previous = subprocess.check_output(
['git', 'show', 'HEAD~:global-requirements.txt']
).decode(sys.stdout.encoding)
current = pathlib.Path('global-requirements.txt').read_text()
assert sorted(
x for x in previous.split('\n') if x and not x.startswith('#')
) == sorted(
x for x in current.split('\n') if x and not x.startswith('#')
)
We haven't checked _this_ in since it's trivial and not broadly useful
here.
Change-Id: If836960167fe66534034a8a512ea63553649c3ff
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This adds a tool script that loops through all of our requirements and
output basic information for each one along with any things we might be
concerned about with each one. For now it just checks a few things, but
we could extend this issue checking as we find other things to check on.
Also refactors the unused package tool to put some common things into a
shared function library used by both scripts.
Change-Id: I140c3a5fb71a8fb82ccb696bade5be13bb9b0ba0
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This adds a script that will check through all the requirements in
global-requirements for any that are no longer being used.
Change-Id: I9c8fb2c07c794739bcab07a55e68e67d16736acc
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This job was a legacy job that attempted to verify each project could be
independently installed after all global-requirements had been
installed.
This job is no longer necessary since we have much better coverage with
requirements-check and other jobs that make sure the requirements are
compatible. It is also a legacy zuulv2 job, uses the long deprecated
devstack-gate, and runs with Python 2.7.
For historical reference, this was added originally to PBR back in 2013
[0] and moved to requirements in 2014 [1].
[0] 378261a8cb
[1] 8e073738db
Change-Id: I29abcba929995f9307c392110160d889a29b47c3
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.
This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.
This update should result in no functional change.
For more information see the thread at
http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html
Change-Id: I53162497c2f53e45139e6a3aa9ff20538ec1b35e
As we collect more history displaying all the EOL releases becomes noise
to filter out. Hide eol tags by default but provide a quick way to
display them again if wanted.
Change-Id: I89d0c9ae2fa2e94505d44bfa8d2e0b0f00e1d842
Currently we don't sort the open branches which leads to them printing
in an order up to git. Lets sort them by the series name so we have:
1.2.0-1052-ga00761ff : requests # Apache-2.0
origin/master : requests # Apache-2.0
origin/stable/rocky : requests # Apache-2.0
origin/stable/queens : requests>=2.14.2 # Apache-2.0
origin/stable/pike : requests>=2.14.2 # Apache-2.0
origin/stable/ocata : requests>=2.10.0,!=2.12.2 # Apache-2.0
origin/stable/newton : requests>=2.10.0 # Apache-2.0
mitaka-eol : requests>=2.8.1,!=2.9.0 # Apache-2.0
liberty-eol : requests>=2.5.2,!=2.8.0,!=2.9.0
kilo-eol : requests>=2.2.0,!=2.4.0,<2.8.0
juno-eol : requests>=2.1.0,!=2.4.0,<=2.2.1
icehouse-eol : requests>=1.1,<=2.6.0
havana-eol : requests>=1.1
grizzly-eol :
folsom-eol :
Which I think is easier to parse as the lower on the list the older.
Change-Id: Ib436bc5cc1bd3f13176485909003cc00cf348c1a
This is a terrible idea!
Since networking-odl 13.0.0 hit pypi which includes I955c9d737864b8a2557366be518922e192f4be91
the integration gate fails because ceilometer isn't on pypi[1]. There
aren't any good fixes but this *should* at least unblock the,
requirements, gate and buy us time to fix it correctly.
Note:
* We can't just install ceilometer from git as we'll just fail when we
try to generate a constraints list anyway.
* We can't blacklist networking-odl as the backlist isn't used for
generating a freeze only for filtering the output constraints
* We can't stop building the constratints as that introduces a new
failure path in the gate.
So with those options ruled out lets just pretend that networking-odl
isn't in global-requirements. This will still fail if something
(directly or not) installs it from pypi.
[1] http://logs.openstack.org/96/594496/2/check/requirements-integration/8378cd8/job-output.txt.gz#_2018-08-31_22_54_49_357505
Change-Id: I1f117f9f4d8f49b5c4ef0cb74d98560c9a551999
The "pip-install" tox environment was failing, because the name
of the "ConfigParser" package was changed into "configparser".
This commit fixes that by using the six.moves module to do the right
thing on py2 and py3
Change-Id: I6b16c8f0e182850cda041ca294edc5ad04c1a3c3
We have a few projects that have lower constraints lists that do not
match their own local requirements settings. This script tries to fix
that by changing the constraints to match.
Change-Id: I5f9bfc30e5dab855c9cef77cd9c4063658aab4c1
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Branches and tags are really just refs in git so rather than
open-codeing that and having 2 for loops just call everything a ref.
While we're there sort tags in reverse order this has the cosmetic
advantage that series are chronological order ie:
1.1.0-1215-g8dc48120 : openstackdocstheme===1.18.1
origin/master : openstackdocstheme===1.18.1
origin/stable/newton : openstackdocstheme===1.5.0
origin/stable/ocata : openstackdocstheme===1.6.1
origin/stable/pike : openstackdocstheme===1.16.1
folsom-eol :
grizzly-eol :
havana-eol :
icehouse-eol :
juno-eol :
kilo-eol : openstackdocstheme===1.1.0
liberty-eol : openstackdocstheme===1.2.6
mitaka-eol : openstackdocstheme===1.3.0
becomes:
1.1.0-1215-g8dc48120 : openstackdocstheme===1.18.1
origin/master : openstackdocstheme===1.18.1
origin/stable/newton : openstackdocstheme===1.5.0
origin/stable/ocata : openstackdocstheme===1.6.1
origin/stable/pike : openstackdocstheme===1.16.1
mitaka-eol : openstackdocstheme===1.3.0
liberty-eol : openstackdocstheme===1.2.6
kilo-eol : openstackdocstheme===1.1.0
juno-eol :
icehouse-eol :
havana-eol :
grizzly-eol :
folsom-eol :
Change-Id: If32cb11f1bcd3aefdc3717d3e943b0fa15a42002
Sometimes we want to go further back in history than the current open
branches, so include the -eol tags as well.
Change-Id: Idb7ca9d0f4dd2f7dc6701577cc495a8010ee1340
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Sean wanted to know which repos still use mox/mox 3 which can be done
with this tool like:
tools/code-search.sh --prefix ~/projects/openstack/ \
--projects ~/projects/openstack/*/* \
-- mox origin/master -- *requirements.txt setup.cfg
The prefix is essentially a sting to remove from the project name. So
the example above would change:
/home/tony/projects/openstack/openstack/astara => openstack/astara
Change-Id: I927bc96b6eafd2d95dc008b7f1a4374eb9f725b4
Change Ief487e1d20ea20facf398367ca03a6385182fb38 updated the PTI for doc
jobs and introduces doc/requirements.txt. Support that for requirements
syncing.
Change-Id: I147a129438ae370eb4c90ee93a634fa61cc179f4
I used something like this when testing reno 2.5.0. It's clearly a work
in progress but I figure we could all fix/test it rather than me having
another half implemented tool
Change-Id: I2fa844f4cf8dd7e5c06461b7f7926a7cf84515c3
Add a bashate environment for checking shell code quality.
Exclude bashate warnings:
E006: Line too long:
E010: The "do" should be on same line as for
In my opinion the places that trip these do not impact code readability
Change-Id: Iab0a34542c3938f127515291eca8f027f3accaa0
We now have a stable/pike branch and mitaka is EOLd (though we can't yet
remove the branch because there are still projects that are using it :(
Update the grep-all script to look in releases we care about.
Change-Id: Ia3eeda9db6b5715a3bfff5fe4f0ccd3d50ebf653
When I added the license to grep-all I copied it from another shell file
clearly no one read it as it describes tools/cruft.sh
Change-Id: Ia48d5000a360519be596c09273ba653d10e3daa6
If you run grep-all.sh requests, you get all the item that match that
prefix. It is a little messy. Make the match patten more specific so
you only get the library you pass on the command line.
Change-Id: I16544372f7efff28933da728d11f85dc5e5f9aea
Often it's helpful to quickly compare the requirements and constraints
for a given library on all branches. This tool does that.
Change-Id: I2458c5929d8812cd74f657050eea5f4f3fa11df0
We need the integration gate to work with both trusty (python3.4) and
xenial (python3.5). Rather than hard code the full python binary just
use python 3 and let the OS mange this for us. This is less than ideal
but it's the best we can do right now to unblock the gate.
We also address a short term issue with out upper-constraints.txt files
and manually copy the constraints from 3.4 to 3.5
Related-Bug: 1620436
Change-Id: I6acabf86933b7a7bce7baf44e6b512e7d0d2f6db
This change creates a new tox environment that *only* installs
openstack_requirements and then verifies that each of the console
scripts has all of the modules that it imports.
This will need to be added to our gate RSN
Change-Id: Ibc37593afcc4d9f820cb88168e1aa15e773b2087
If we don't do this then the following line:
weakrefmethod;python_version=='2.7' or python_version=='2.6' # PSF
ends up as:
weakrefmethod;python_version
in the cruft output. So we need to add ';' to get the output
as 'weakrefmethod'
Change-Id: I7b32bfe70df3bb24ca301db9952d76451674a0db
With Babel 2.2 and then with Babel 2.3.2 and 2.3.3 releases, the
OpenStack infra scripts broke and uncovered bugs in the upstream
Babel release. The infra scripts run in post and periodic queues, so
failures here are not directly visible. With Babel 2.3.x invalid strings
were send to the translation server, and we had to recover old content
to fix this.
Let's add some Babel integration tests that can be run whenever the
upper-constraints file changes so that any new Babel release will be at
least minimally tested.
The shell script babel-test.sh runs a simple extraction of strings to
translate from a simple input file. It uses the same commands that the
infra scripts run and handles the two different cases of extraction of
the normal strings and extraction of the non-standard log-level strings.
The script then compares the new content with previously created good
content.
Once this change is in, we will run this test whenever
upper-constraints.txt file changes.
Needed-By: I15fa6a706323615d6072d8acaa859fea9bf35883
Change-Id: Iab7844ba748ecd84ec07ad4280b2317bd626fe9a
We need to test without committing the requirements changes because pbr
throws an error if the pre and post versioning do not match. The first
change on a stable branch must be the one to remove the pre-versioning
configuration.
Change-Id: I812af614b7a6c7f499c67310f078ac4835b9e91a
We have removed these projects from the PROJECTS list so no longer need
to special case them here.
Change-Id: Ic3116a646389438ded03f285819529477bf849d8
Depends-On: I61c2f3f8a724dc789479278e7ba493e095c52305
bindep makes this a lot easier to work with as well as making it
possible to support Fedora, Suse etc as needed.
Change-Id: I4d19df8bbcd08e06edcf71ee51c30bb9c3d57fd0