There do not appear to be any openstack projects relying on this package
these days. Remove the unnecessary dependencies.
Change-Id: Iddd95afad9e6cf10d2cf0a3d6cae4b6ba75a7a05
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Resolve the following warnings:
tox.tox_env.errors.Fail: pass_env values cannot contain whitespace,
use comma to have multiple values in a single line, invalid values
found 'PBR_INTEGRATION PIPFLAGS PIPVERSION PBRVERSION REPODIR
WHEELHOUSE PROJECTS'
Also, reduce down the envlist as tox 4 when run in OpenStack CI is
also attempting to validate interpreters utilizing the
--skip-missing-interpreters=false setting [1].
Finally, fix up the integration job so that it passes on Ubuntu 22.04.
This necessitates removing any Python 2-based dependencies used in the
these jobs (since these are no longer available on Ubuntu 22.04) and
removing tests for pip versions that are not compatible with Python 3.10
(the Python version found on Ubuntu 22.04).
[1] https://github.com/tox-dev/tox/issues/2811
Change-Id: Ib4dbe4e09e269219c08a87f6fb64ad9628b5864a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We're timing out installing some packages and it seems we're right up
against the timeout based on logs. Increase the timeout by 50% to give
us a better chance of running these jobs successfully.
Change-Id: I33ce50f57af2a1d366148e24b49cae5eda2d3dfb
PBR does integration testing by installing all of openstack's python
projects to ensure we can install all of them in a variety of different
ways with different pip versions. Unfortunately, these jobs are still
using devstack-gate. We can simplify them significantly because zuul is
now doing what devstack-gate does and provides repos configured the way
we want them to be.
This change updates the jobs to drop devstack-gate and simply consume
zuul's repo prep.
A follow up will remove the old legacy jobs as they need to be removed
from project-config first.
Needed-By: https://review.opendev.org/745189
Change-Id: I77a38c4611dc28db79d6f25d96ad0de36b224c98
virtualenv is undergoing a rewrite and has changed how the programmatic
API works [1]. Switch to the "new way".
While we're here, we also need to get Python 2.7 tests passing again.
That requires dropping support for upper-constraints and using our own,
limited local constraints based on supported Python versions. We also
need to migrate integration tests since those run with Python 3 now.
Update the scenarios for pip/setuptools in integration testing
similarly. Finally clean up the installation of all devstack repos as
requirements are managed different now. Instead of worrying about
syncing them we use constraints.
[1] https://github.com/pypa/virtualenv/issues/1585#issuecomment-585228492
Change-Id: I493e88985d2c4d09612fea4d20d8ffa20043a0cb
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Depends-On: https://review.opendev.org/739014
When running this locally I needed to install a few more distro
packages to provide build dependencies for pip packages. This change
adds them to the list of things installed in tools/integration.sh.
Change-Id: I2ae04b893c7dfddcb6d94b724f53ee08436bf577
It appears this no longer exists on bionic, and we shouldn't need it
for pbr tests. I suspect this may have been some copy-pasta from
another project's integration tests.
Change-Id: Ife631f77a92ee0b34c19e77cad782d94d18f2e74
Closes-Bug: 1820855
This is not needed since reno 2.1.0 (commit f8fc8f97) [1].
[1] https://github.com/openstack/reno/commit/f8fc8f97
Change-Id: I335bba8c8bf4f2d73aba32ae762607e67ee3814c
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Pip10 has removed the ability to poke inside it for things like
requests. Explicitly add it to the venv.
Change-Id: Ie12168091dc4c737f3913720c1743bea666b5d9c
We do not need tox_install.sh, pip can handle constraints itself
and install the project correctly. Thus update tox.ini and remove
the now obsolete tools/tox_install.sh file.
This follows https://review.openstack.org/#/c/508061 to remove
tools/tox_install.sh.
Change-Id: I0a80865a8a18c49b4d883850488cc0c73fe1bfcd
Adding constraints support to clients is slightly more complex than
services as the clients themselves are listed in
upper-constraints.txt which leads to errors that you can't install
a specific version and a constrained version.
This change adds constraints support by also adding a helper script
to edit the constraints to remove pbr.
Change-Id: I80a10f80e2c5efafb099a6c1dcebe8f0935ace09
Apt config does not have sources anymore, so we copy apt-cache and run
apt-get install ourselves to provide our dependencies
Change-Id: Ie70cb124120d1012d38e5189044c21b83be20819
Pilow needs libjpeg and liberasurecode is needed by PyECLib. In the
future we should be able to get this done with bindep, but we're not
quite there yet.
Change-Id: Ib56d9fdd99731e3eefc9f69f07d0f002d5415868
We were generating versions lower than the last tag when preversioning
was in use and pre-release versions had been tagged.
Also attempt to run the script without devstack.
Change-Id: If984939003ae7c513ee76b52c6808db0c1766050
Closes-Bug: #1450210
Instead test pip install more thoroughly.
Also document that we don't support setup.py install's easy-install
behaviour.
Change-Id: I87872a69425f9d4f17047e584c12516508d57b16
pip tries to build all wheels, and can avoid re-processing all the
requirements over the network N times - basically quadratic down to
linear scaling.
Change-Id: I69441b2378bf4f165d0b475cd16a9e56797df575
Add global flags to pip install to see more about what its doing (when
set). pip is too verbose to enable -v mode by default as yet.
Document the environment variable parameters the script takes.
Change the way we enable the wheelhouse: rather than magic environment
variables which require action-at-a-distance knowledge, use actual
parameters. This means that the easy_install invocation which recurses
back into pip won't use the wheel house... but we're removing that
reentry anyway.
Change-Id: Ie7ed8d60a6dd6b5766a2c7116b860b8e98a1f1f1
This allows testing with unreleased pips, which is important to know
if something is going to cause us or our users trouble.
Change-Id: Icb3a9c1486c76b673d25a3d18792baad3d26c763
setuptools > 12 has a new flow for writing out generated script text.
It's nicer, actually, because it means we can just subclass and extend
one method instead of monekypatching.
Change-Id: I56e7bea60df8a59d859575d426ce93c45ffee314
When "pip wheel" hits a package where building the egg-info metadata
fails, it completely falls over and stops trying to build more wheels.
Loop over our requirements one by one, so we can ignore the packages
that fail to build wheels, and build any that come in the list after
those.
Change-Id: Ia23672b851d50af38c61823936f19b8572a78c4f
The integration test can take >2 hours running into a timeout. Much of
the time spent in this test is consumed by recompiling dependencies like
lxml. Reduce this overhead by compiling these items once in a
wheelhouse then installing all dependencies from that wheelhouse
preventing recompilation.
Change-Id: I40cd7587ff32f1a4cf881939069dde015dc9589d
Stop including git sha in version strings
We include it in pbr.json now. Including it is contentious in the world
of python, and it's up for debate as to whether or not it provides value.
Write and read more complex git sha info
Instead of encoding the git sha into the version string, add it to
a metadata file. This will allow us to get out of the business of
arguing with pip and setuptools about version info. In order to make
this really nice, provide a command line utility called "pbr" that has
subcommands to print out the metadata that we're now including in the
egg-info dir.
Only import sphinx during hook processing
When pbr is imported to handle writing the egg_info file because of
the entry point, it's causing sphinx to get imported. This has a
cascading effect once docutils is trying to be installed on a system
with pbr installed. If some of the imports fail along the way, allow
pbr to continue usefully but without the Sphinx extensions
available. Eventually, when everything is installed, those
extensions will work again when the commands for build_sphinx, etc.
are run separately.
Also slip in a change to reorder the default list of environments
run by tox so the testr database is created using a dbm format
available to all python versions.
Integration test PBR commits
Make sure that if a PBR commit is being tested then we install and
use that source rather than the latest PBR release.
Change-Id: Ie121e795be2eef30822daaa5fe8ab1c2315577ae
(cherry picked from commit 65f4fafd90)
(cherry picked from commit cd7da23937)
Closes-Bug: #1403510
Co-Authored-By: Clark Boylan <clark.boylan@gmail.com>
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
Co-Authored-By: Jeremy Stanley <fungi@yuggoth.org>
Sometimes there are network errors communicating with pypi
during a long running (1h+) integration gate job. This
causes the job to fail, and wastes a lot of time re-running
the job to get it to pass. This change makes it less likely
to fail, as if there is a timeout or socket error, the
integration setup will retry.
Change-Id: Ic4c888626699eb710e85e3bdc48dd6ee74bdf024
We don't need to test the mirroring tool in this integration script any
more, because we're using a different tool to build the mirror now
anyway.
Change-Id: I1a3f739a9530f4f5cce18f64b0705ae9ae77704d
Apache 2.4 requires site configuration files to have a ".conf"
extension, and Apache 2.2 does not want the extension. Add logic to
figure out the right name for the file so we can run the tests against
both versions of Apache.
Fixes bug: #1334326
install required libraries for mixed c things
use build-dep to install the required libraries needed for
the python build deps. Without this we can get cryptic library
compilation issues.
And use ccache explicitly
Change-Id: Ia750e4221b119097521cf373752aae364759913b squashed
into this one because we can't move forward without both.
Co-Authored-By: Sean Dague <sean@dague.net>
Change-Id: I5ab8898bd3cc2de18681fe3262cb784f7d9519ab
Instead of always using master branch support the option to use stable
branches. Read the branch via the $OVERRIDE_ZUUL_BRANCH environmental
variable that devstack-gate sets
Change-Id: Iab4ae1efddc4dca38dcd3779dae1a79ddc2b0852
* tools/integration.sh: Use the --no-update flag to run-mirror to make
sure that the repo checked out by devstack-gate is the repo used to
build the mirror. Otherwise `git reset --hard remote/origin/master` is
used which we do not want. Also, we can't update the requirements repo
until integration.sh is working and for that we need to install d2to1
into the mirror. Usually this would require a requirements repo change,
but chicken and egg. Force it in along with pip and setuptools instead.
This can be removed when d2to1 is added to the requirements repo.
Change-Id: Id5c2c902832e475f28310fdd547672464bf42c0c
Add an option to pbr to use wheels when installing things from pip. The
speed boost this gives is quite sexy, as it can avoid having to
re-build C things all the time.
Change-Id: Id9b9b21ef949ff44c7c962e81576266661264a13
This change will display the difference between which python packages pip
installed and the packages from the requirements lists. It will print
the list of dependencies to the console (not including the directly declared
dependencies from the requirements list).
Closes-Bug: #1172417
Change-Id: I9363b50d6d7f5ef19c42bb2e30de45e352060ccd
There are several things in the script that are no longer necessary.
Remove them to clean things up so that we can reason about this
script with less stress.
Change-Id: I02863c8ab98d7e403fada2973a0b5807e1b75595
There are enough behavioral differences between http and file urls
for pypi mirrors that we should do the local mirror via apache. That
way we're testing what we care about - consumption of changes in the
mirror context - and not what we don't - how well pip deals with file
urls.
Change-Id: Icb16e8bbe77c03f224d06010eeb6bd0877b0d6ca
We are going to start having wheels in the gate. Having them there
should not break folks who are not using wheels. Don't start actually
consuming wheels yet, but add awareness about the wheel structure to the
pypi config.
Change-Id: Id29bb1bb31fc1647f8e1ea00bab487e447ba2838
We no longer care about or support old pip, because it turns out that
evil happens. Remove it from the mkvenv function.
Change-Id: I27dbef7816eaac856e97ec5a036f6d5ca9b183af
* tools/integration.sh: Overriding the PyPI URL is logic best left
to devstack-gate since it does those things already. Remove the
similar dance found here.
Change-Id: Ie2fcabb11b30dc3ab6564bdedd55b2b173c1bf2a
So that we don't test combinations of things that we won't see in the
gate, run the same requirements sync job that devstack does.
Change-Id: I0ee375b4e41aa214acf4cd3bfeb1b3a8ccb08dff
ksh -n, weirdly enough, can be used as a rudimentary syntax and style
checker for bash. tools/integration.sh was mostly clean, but it
suggested that `` should be replaced with $(). Sure, why not.
Change-Id: I76beff1194133444284c1e8fbbd5d87db369e47f
PyPI upstream obviously breaks all the time. Like, literally, I've spent
my whole life dealing with it this past week. In this test, we were
avoiding the OpenStack mirror so that we could be sure to be testing
that we could build the mirror when new requirements are added. However,
the only time that's a concern is on a requirements repo change.
Add an option --no-mirror that will trigger the "don't use OpenStack's
mirrors as an upstream" behavior, and we'll add that option to the
invocation on requirements changes.
Change-Id: I8d01c1fbb8a5df5aa83ffba3b6489e374abef1af
integration.sh already expects there to be a pbr source dir in
$REPODIR/pbr. If we pull jeepyb and requirements from the same place,
then we can gate them as wel as pbr.
Change-Id: Ie0eb513c2deade957d480a25dfba2c5486d14876
Stephen Finucane