We are seeing failures when using an updated setuptools version
installed together with distro pip on Ubuntu 22.04. Install the version
from u-c only when we are also installing pip from upstream.
Change-Id: Ibb6e9424e5794ccbf9a937d2eecfa3bf60ed312e
Since pip v22, python3.6 is not supported (the minimum version is
python3.7). This patch adds the reference for the pip3.6 URL to be
used instead of the default one.
Closes-Bug: #1959600
Change-Id: Iab2c391d5388461fe9e9037cee81884ce8032e72
Running get-pip.py fails on Ubuntu when running twice, e.g. after a
unstack/stack cycle. Just use distro pip instead.
Closes-Bug: #1957048
Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: I87a8d53ed8860dd017a6c826dee6b6f4baef3c96
This patch includes changes required to run devstack on CentOS Stream 9
which has been already published in official repos by CentOS team [1]:
- Add RDO deps repository for CS9.
- remove xinetd package from installation for swift. Note that
rsync-daemon is installed which should work fine.
- Replace genisoimage by xorriso in CS9.
- Use /etc/os-release to identify the distro in CS9 as it doesn't
provide lsb_release command.
- Use pip from rpm package instead of from get-pip.py as done in Fedora.
- Add non-voting job devstack-platform-centos-9-stream to the check
pipeline.
Change-Id: Ic67cddabd5069211dc0611994b8b8360bcd61bef
continue is not used in a proper context here (outside of loop). Use
null cmd instead to simply fall through the pip installation.
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Change-Id: Iaea2e5c0177b475edf19d08d71933a74debbb5d9
The uninstall here has been around since
Ibb4b42119dc2e51577c77bbbbffb110863e5324d. At the time, there might
have been conflicts between packaged and installed pip. We don't need
it today; get-pip.py keeps itself separate enough in /usr/local on all
platforms. Thus we can also remove the suse/centos special-casing.
python3-pip is in the RPM list so we don't need to
re-install for Fedora.
Add a note on why we are over-installing pip.
Remove some old setuptools workarounds that are commented out.
Change-Id: Ie3cb81a8ff71cf4b81e23831c380f83b0381de71
This uses the python3-pip package for Fedora but maintains the status
quo for existing distributions (i.e. for Suse we run get-pip.py but
don't uninstall, and for everything else we uninstall python3-pip and
run get-pip.py to be running the latest pip).
As noted inline, installing get-pip.py over Fedora 34's package no
longer works, and likely won't ever work again. Unlike the LTS
distributions, the Fedora pip should be more up-to-date, so I think
it's best we just avoid any package overwrites.
Change-Id: I84129aadfcf585bb150a3daa39616246d3d84bbd
On some platforms, "python -m pip" isn't available. Currently this is
run undconditionally from the "get_versions" function; remove the call.
Change-Id: I91d6c66d055f02fa7b4368593b629933f82d8117
This reverts commit 7a3a7ce876 and
bcd0acf6c0 and part of
f1ed7c77c5 which all cap our pip
installs.
Given the pip ecosystem can often incorporate major changes, tracking
upstream at least generally gives us one problem at a time to solve
rather than trying to handle version jumps when LTS distros update.
The new dependency resolver included some changes that disallow
setting URL's like "file:///path/to/project#egg=project" in
constraints. Apparently the fact it used to work was an accident of
the requires/constraints mechanism; it does make some sense as the URL
doesn't really have a version-number that the resolver can put in an
ordering graph.
The _setup_package_with_constraints_edit function comment highlights
what this is trying to do
# Updates the constraints from REQUIREMENTS_DIR to reflect the
# future installed state of this package. This ensures when we
# install this package we get the from source version.
In other words; if constraints has "foo==1.2.3" and Zuul has checked
out "foo" for testing, we have to make sure pip doesn't choose version
1.2.3 from pypi.
It seems like removing the entry from upper-requirements.txt is the
important part; adding the URL path to the on-disk version was just
something that seemed to work at the time, but isn't really necessary.
We will install the package in question which will be the latest
version (from Zuul checkout) and without the package in
upper-requirements.txt nothing will try and downgrade it.
Therefore the solution proposed here is to remove the adding of the
URL parts.
This allows us to uncap pip and restore testing with the new
dependency resolver.
Closes-Bug: #1906322
Change-Id: Ib9ba52147199a9d6d0293182d5db50c4a567d677
This patch caps pip version during bootstrap to avoid the issue:
"ERROR: Links are not allowed as constraints"
A proper fix would be to adapt to new pip behavior.
Depends-On: https://review.opendev.org/764811
Change-Id: I1feed4573820436f91f8f654cc189fa3a21956fd
Focal only provides a pip3 binary, no pip3.8. Instead of working around
that with a symlink, use the module instead.
Add version information output for this variant.
Change-Id: I7af194ecc40e4d43c10ce067a661bb6ab4ca37d4
Ubuntu Focal doesn't have python-pip, only python3-pip. Trying to
uninstall a package that apt doesn't know about (installed or
uninstalled) results in a nonzero exit code so devstack fails. This
patch makes the package removal safer for both python2 and python3 cases
by checking first if the package exists.
Change-Id: I3b1118888cb0617ffb99b72c7e9a32308033783e
This reverts commit f99d1771ba.
Added workarounds that might want to get split into their own patch
before merging:
- Don't install python-psutil
- Don't run peakmem_tracker
Change-Id: If4fb16555e15082a4d97cffdf3cfa608a682997d
This updates links going to git.openstack.org and review.openstack.org
to go to their respective opendev locations to avoid redirects.
Change-Id: I78e3bb5303718962f591117f9c0ee11f2314b128
Closes-Bug: #1833256
This adds the necessary fixes to pass a devstack run
on openSUSE Tumbleweed. Also removes opensuse 42.2 as it
is EOL for some time already and no longer actively tested
in the OpenStack infra.
Depends-On: I1b68c08c07cf6653ea58506f738cbe0054b38f3a
Change-Id: I2894482deef063fd02b0818c695a2ddbf6767039
Use pip_install_gr so we get the version pinned by requirements. The
depends-on is an example of where we're trying to pin to workaround
issues.
Depends-On: I9c57c08a150571c5bb62235d502839394d53a4c1
Change-Id: I780cca681b12a3e9d228dbf2fd9fa6e8ab1a82e1
The default get_pip url regulary times out when starting devstack
from behind company firewalls. Making this a configureable variable,
user can make use of internal git-pip.py mirrors without modifying
any code.
Change-Id: I66a5534d51ab23a4d8586c27d37b4b6b8a6892c9
Looks like f24 does not have any special change compared to the
previous release, we just need to add f24 where f23 present.
Change-Id: Ia4a58de4973ef228735c48b33453a0562dc65258
PS4 can include functions, so when running in the LOG_COLORS=True mode
provide a grey function line so that it's easier to visually
distinguish the content from the location.
Also make it so the main prompt chunks off all the common path, which
means we can printf to 40 characters and have a pretty reasonable and
readable PS4.
Change-Id: I4a10a49db97d413349bcfceeb8c4164936fbcc40
Pip 8 just release which made uninstalling distutils installed packages
fatal. This was previously a deprecation warning and is now causing all
dsvm jobs to fail.
Depends-On: I511d216d9d8619c7cf919c482aaead4e833cdaac
Change-Id: I22f5c3af1adf96cfbd5747122f915a82e947843a
Add USE_PYTHON3 and PYTHON3_VERSION variables to allow services to use
python 3 if they indicate support in their python package metadata.
Tested in Heat here -> I837c2fba682ab430d50e9f43913f2fed20325a7a.
Project config change to add a dedicated job to Heat is here -> I0837e62d6ccc66397a5e409f0961edd4be31f467
Change-Id: I079e18b58b214bf8362945c253d6d894ca8b1a6b
Python in f23 and f22 depends on the python-pip package so removing it
results in a nonfunctional system. pip on fedora installs to /usr so pip
can safely override the system pip for all versions of Fedora.
Change-Id: I336c7ffdf00784ca8deba7d6612a08b96a0ad098
Closes-Bug: #1467569
This will make curl fail on pypi errors, and should prevent corrupt
images from pypi going offline for a few hours randomly, which it does
from time to time.
Closes-Bug: #1503909
Change-Id: Ib4a740b7d1772e1e36aa701e42d3ac0f0ee12883
If get-pip servers fall over and return 503 for a few hours (which
they do medium regularly) we'll cache crud html, and everything will
suck. We know this script should be python, so if it isn't, delete it.
Change-Id: Ia9f6f7c7217939bc1ab5745f4a9d568acfbf04c8
we were unconditionally adding -z to the curl command even if the file
doesn't exist that we are referencing. That produces a scary warning
for users. Lets not do that.
Change-Id: Id2860c1c702510b0f8fd496abce579d0fa3ff867
Having behavior on your laptop diverge from behavior in the gate is
confusing. Just use constraints on every devstack run to be consistent.
Users of devstack can edit the requirements repo in order to change
these constraints locally if necessary.
Change-Id: I843208e2e982eb04931b76f5cb4bd219fbcd70de
People can leave their devstack installs around for a long time, and
in the mean time new versions of pip can be released.
The current check does not download a new version if an old one
exists. We want to check for new versions, but we also don't want the
gate jobs trying this sometimes unreliable fetch.
So add a flag-file that tells devstack if it downloaded get-pip.py
originally. If so, on each run check for a new version using curl's
"-z" flag to request only files modified since the file's timestamp.
Change-Id: I91734528f02deafabf3d18d968c3abd749751199
Closes-Bug: #1429943
We updated other usage of sudo to pass -H when installing pip things,
to avoid creating a .cache directory in $STACK_USER's $HOME that is
owned by root. get-pip.py also ends up creating a ~/.cache, so we
need to update sudo usage there as well.
Closes-bug: #1405626
Related-bug: #1405732
Change-Id: If791b9b25d6a4280dab19117004184e57e78d038
This reverts commit 3b782d304e.
The blockers for setuptools 8 compatibility should all be resolved
now.
Change-Id: I6d2d63746f98f0f885816395f36022a2706fb9c5
Latest release of setuptool 8.0 made several versions used in
requirements.txt of OpenStack projects invalid. Instances:
* SQLAlchemy>=0.8.4,<=0.8.99,>=0.9.7,<=0.9.99 in oslo.db 1.2.0
* python-neutronclient 2.3.9.40.g9ed73c0 in openstackclient
Cap '<8.0' is set as a temporary fix until a better solution
comes up.
Change-Id: I4cfe2e4c86474ec9bf69a3c2007c0277288ea2b6
Currently pip will get the package from the https://pypi.python.org server.
For CI, it's a problem as Internet connection can be down,
the pypi server down, etc...
The usecase is for a company/user that maintain a local pypi mirror
and give the option to use this server instead of the official one
Change-Id: I83aac4646cb78827a92c9636d78238f8a6118642
Implements: blueprint support-local-pypi-server
Support for .dist-info directories was added in setuptools 0.6.28.
At this moment, Ubuntu Precise 12.04 provides setuptools 0.6.24
which is too old for our needs.
Six is installed from wheel which uses the .dist-info directory.
For six to be found, we need to install setuptools >= 0.6.28.
Updating setuptools to the latest version using pip will provide use
the needed version to make six discoverable.
Closes-bug: #1326811
Change-Id: I761d0aeb2b8b593cee38d512afc8fed6a2d1fe37
curl dying ends up being a really unclear failure condition, and
hard to fingerprint in the gate. We should make this much more
explicit when we die.
Also, don't trust the upstream filename, because all the rest of
our logic would break if it changes anyway.
Change-Id: Ibc2a96b33471d24c597af0d7af896fb10523156f
get-pip.py is now on a CDN, and is the prefered way to get pip.
Remove the default path of using pip tarballs from pypi and use
get-pip.py on from here on.
Closes-Bug: #1326539
Change-Id: I0661f7c6913ba6b3e1d00b30e22740d150bfd060
Check that function calls look like ^function foo {$ in bash8, and fix
all existing failures of that check. Add a note to HACKING.rst
Change-Id: Ic19eecb39e0b20273d1bcd551a42fe400d54e938
devstack-gate wants to pre-cache and then use get-pip, but we can't
throw the flag currently. Make the flag default settable via env vars.
Change-Id: I661b52670b6ce494666cbdd611e4eee6b96c8321
Partial-Bug: #1254275
Run ./stack.sh will dump ~400 lines of information, because of
tar xvfz pip-*.tar.gz, and python setup.py install.
We'd better mute stdout for the two steps, to make console cleaner
Change-Id: Icf87947e020acb48d8cbe4cdcc1641f060e50f6d
Martin Kopec