Commit Graph

807 Commits

Author SHA1 Message Date
Clark Boylan a40f9cb91f Add option to install everything in global venvs
Since we are python3 only for openstack we create a single python3
virtualenv to install all the packages into. This gives us the benefits
of installing into a virtualenv while still ensuring coinstallability.
This is a major change and will likely break many things.

There are several reasons for this. The change that started this effort
was pip stopped uninstalling packages which used distutils to generate
their package installation. Many distro packages do this which meant
that pip installed packages and distro packages could not coexist in the
global install space. More recently git has made pip installing repos as
root more difficult due to file ownership concerns.

Currently the switch to the global venv is optional, but if we go down
this path we should very quickly remove the old global installation
method as it has only caused us problems.

Major hurdles we have to get over are convincing rootwrap to trust
binaries in the virtualenvs (so you'll notice we update rootwrap
configs).

Some distros still have issues, keep them using the old setup for now.

Depends-On: https://review.opendev.org/c/openstack/grenade/+/880266
Co-Authored-By: Dr. Jens Harbott <frickler@offenerstapel.de>
Change-Id: If9bc7ba45522189d03f19b86cb681bb150ee2f25
2023-08-02 07:07:25 +02:00
Zuul 61cd302439 Merge "Remove unused file" 2023-07-25 14:45:38 +00:00
Zuul b52dceee7b Merge "Switch TLS tests to TLSv1.2+ only" 2023-07-21 16:37:18 +00:00
Zuul 9845128969 Merge "Support RHEL 9" 2023-07-21 12:35:17 +00:00
Dr. Jens Harbott 4a1b2808af Remove unused file
This was forgotten in [0]

[0] I20501fec140998b91c9ddfd84b7b10168624430a

Change-Id: Iacd86e3953f573a0fc38dc4898aafefccb3a9a79
2023-06-26 12:23:58 +02:00
Yamato Tanaka 03c3fd763e Support RHEL 9
This patch includes changes required to run devstack on RHEL 9.

- en_US.utf8 is provided by glibc-langpack-en
- iptables command is provided by iptables-nft
- Use /etc/os-release to identify the distro in RHEL 9 as it doesn't
  provide lsb_release command.
- CRB repository name is different from CentOS 9

Change-Id: I8f6d9263b24f9c2cf82e09258e2d14d7766ad337
2023-03-03 14:47:10 +09:00
Martin Kopec ec07b343d2 Remove support for opensuse
We haven't been testing the distro for a while in CI, e.g. in
Tempest, the jobs on opensuse15 haven't been executed for a year
now.
Therefore the patch removes opensuse support from devstack.

Closes-Bug: #2002900
Change-Id: I0f5e4c644e2d14d1b8bb5bc0096d1469febe5fcc
2023-02-16 12:01:39 +01:00
wangxiyuan 6440c6d7e6 Add openEuler 22.03 LTS support
openEuler 20.03 LTS SP2 support was removed from devstack in last
few months due to its python version is too old and the CI job
always fail. And openEuler 20.03 LTS SP2 was out of maintainer in May
2022 by openEuler community.

The newest LTS version was released in March 2022 called 22.03 LTS.
This release will be maintained for at least 2 years. And the python
version is 3.9 which works well for devstack.

This Patch add the openEuler distro support back. And add the related
CI job to make sure its works well.

Change-Id: I99c99d08b4a44d3dc644bd2e56b5ae7f7ee44210
2022-12-07 10:03:27 +08:00
Ian Wienand 9ece457b7b Update to Fedora 36
Update the Fedora job to the latest release nodes

Depends-On: https://review.opendev.org/c/openstack/devstack/+/860634
Change-Id: If2d7f99e3665a2e3df4cf763efc64dd381f02350
2022-10-07 06:14:57 +00:00
Dr. Jens Harbott ca5f919561 Clean up n-net remnants
In I90316208d1af42c1659d3bee386f95e38aaf2c56 support for nova-network
was removed, but some bits remained, fix this up.

Change-Id: Iba7e1785fd0bdf0a6e94e5e03438fc7634621e49
2022-08-21 10:52:41 +02:00
Dr. Jens Harbott 560ee16a85 Drop openEuler support
The job is broken since it is running with python3.7 and most services
now require at least python3.8.

Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: Ie21f71acffabd78c79e2b141951ccf30a5c06445
2022-05-19 14:06:11 +02:00
Sean Mooney af75f689fa install mod_ssl on centos 9 stream by default
This change adds mod_ssl to the default set of rpms installed
on rpm based distros.

this is required if the tls-proxy service is enabled
for multi node centos based jobs.

Change-Id: I52652de88352094c824da68e5baf7db4c17cb027
2022-04-21 20:40:44 +01:00
Zuul 9bf989eb1a Merge "Made LVM backing disk persistent" 2022-03-02 15:04:47 +00:00
Victor Morales c0882aeaae Add rsync deb package for swift
The rsync debian package is required for swift service. This
requirement has been covered by rpms but not for deb packages.

Change-Id: Iefd1302be9c7fd80e037bbae3638602d6d823580
2022-02-16 18:15:12 -08:00
Jakob Meng accd99e7cd Made LVM backing disk persistent
Previously, loop devices for LVM volume groups backing files were not
created after reboots, causing e.g. Cinder to fail with messages such
as

  ERROR cinder.service [-] Manager for service cinder-volume
  devstack@lvmdriver-1 is reporting problems, not sending
  heartbeat. Service will appear "down".

Now, we use systemd services to manage loop devices for backing files.

Change-Id: I27ec027834966e44aa9a99999358f5b4debc43e0
2022-02-08 11:05:14 +01:00
Ian Wienand 343e351627 Bump fedora-latest to F35
Generally this is straight forward, but Horizon has a dependency issue
with pyScss (described in [1]) so it is disabled, for now.

[1] https://bugs.launchpad.net/horizon/+bug/1960204

Co-Authored-By: Dr. Jens Harbott <harbott@osism.tech>
Depends-On: https://review.opendev.org/c/openstack/devstack/+/827578
Change-Id: I7c4bf0945f9ac5bd563fe0a698c09b8571c97c5e
2022-02-08 08:08:19 +11:00
Zuul 3155217fb6 Merge "openEuler 20.03 LTS SP2 support" 2021-12-30 12:04:44 +00:00
Zuul 559f8cc150 Merge "Use MDB backend in Ubuntu" 2021-12-30 10:28:54 +00:00
Kevin Zhao 7880ba665e openEuler 20.03 LTS SP2 support
openEuler is an open-source Linux based operating system. The current
openEuler kernel is based on Linux and supports multi arch, such as X86_64
and aarch64. It fully unleashes the potential of computing chips. As an
efficient, stable, and secure open-source OS built by global open-source
contributors, openEuler applies to database, big data, cloud computing,
and AI scenarios. openEuler is using RPM for package management.

Note:
Currently there is no available package for uwsgi-plugin-python3 and ovn, so that
openEuler needs manually install them from source.

Website: https://www.openeuler.org/en/

Change-Id: I169a0017998054604a63ac6c177d0f43f8a32ba6
Co-Authored-By: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org>
2021-12-22 14:47:27 +08:00
Alfredo Moralejo 5ea4c3c18c Support CentOS Stream 9
This patch includes changes required to run devstack on CentOS Stream 9
which has been already published in official repos by CentOS team [1]:

- Add RDO deps repository for CS9.
- remove xinetd package from installation for swift. Note that
  rsync-daemon is installed which should work fine.
- Replace genisoimage by xorriso in CS9.
- Use /etc/os-release to identify the distro in CS9 as it doesn't
  provide lsb_release command.
- Use pip from rpm package instead of from get-pip.py as done in Fedora.
- Add non-voting job devstack-platform-centos-9-stream to the check
  pipeline.

Change-Id: Ic67cddabd5069211dc0611994b8b8360bcd61bef
2021-12-02 09:10:48 +01:00
Jens Harbott c2491bac9d Stop creating a keystone admin site
Keystone no longer has any special functionality hidden behind the admin
site. KEYSTONE_AUTH_URI which used to point to the admin site has long
ago been changed to be a copy of KEYSTONE_SERVICE_URI, which points to
the public site.

Drop all KEYSTONE_AUTH_* variables except KEYSTONE_AUTH_URI which may
still be in use in some plugins.

This also allows to finally drop the fixup_keystone() function.

Change-Id: I549f3cadc27d137e014241cdd47e90267859c848
2021-10-17 17:11:03 +02:00
Radosław Piliszek a38d41ed92 Drop dep on libmysqlclient-dev
It was required to build MySQL-python bindings but, for some time,
we test and rely solely on PyMySQL which is pure Python and hence
does not require this dep.

This package is going away as distros move towards MariaDB.

Change-Id: I6004ccf28a23009a0fc07bfc9458b59a927b969a
2021-09-08 07:51:47 +00:00
Grzegorz Grasza 26f8149218 Use MDB backend in Ubuntu
The MDB backend is the default in Ubuntu and specifying
HDB in debconf doesn't change it to HDB.

Closes-Bug: #1939700
Change-Id: If98f7fc8395678365fb73f0c5cd926cef083e470
2021-08-17 09:41:33 +02:00
Ian Wienand 156ccbad85 Add Fedora 34 support
* update the support distro filter

* don't install xinetd which doesn't exist in F34 any more.  I think
  there is probably a bit more to do with swift ring-server but that
  can be a problem for another time.

* remove old F31 workaround

Change-Id:If2f74f146a166b9721540aaf3f1f9fce3030525c
2021-08-11 14:07:21 +10:00
Zuul 6a4e7b0413 Merge "[TrivialFix] Delete symlink apts-debs" 2021-05-25 15:26:48 +00:00
Lucas Alvares Gomes c19c1262c8 Replace dstat with pcp-dstat
The dstat project is no longer maintained.

The pcp-dstat package installs a dstat command so no further updates to
the scripts should be needed.

Change-Id: Ied8c9d29bed4f887c364db7080a0f2a0c02328af
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
2021-05-17 16:11:34 +01:00
Radosław Piliszek d04e795b31 [TrivialFix] Delete symlink apts-debs
Follow up on old I0416180db5b6add996ce5b48c6966c1b68adbcb0

Change-Id: If2f6166cf7c585bf303d0f6c28a2745d85eabbed
2021-04-19 06:52:30 +00:00
Toshiaki Takahashi d207ba9015 Move gawk into general for post-config
Devstack script for setting post-config needs gawk.
So this patch moves gawk from files/*/nova into files/*/general.

Closes-Bug: #1909041
Change-Id: I06a1a5524f146a8d7337963e846b5a6b7561be13
2021-03-30 09:44:06 +00:00
Federico Ressi 7e3428b987 Install systemd-python from distribution package
CentOS 8 support start to fail because of an issue [1]
compiling systemd python binding modules.
Let install it from distribution packages as documented by
python-systemd project Web page [2]

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1862714
[2] https://github.com/systemd/python-systemd

Closes-Bug: #1908386
Change-Id: Ic7cfd72ce1b875e75b1cdbdd44a902b25d51abb8
2020-12-23 14:22:27 +00:00
Luigi Toscano 52e52d8937 Add a dependency file for os-brick (and add lsscsi)
There are a few dependencies which are really os-brick-specific.
They are listed in its bindep.txt file, but os-brick is usually
installed from pip, so its bindep.txt file is not available.

As those dependencies are needed by the various services
which use os-brick, move them to their own dependency file
(with the addition of the new lsscsi, required by the next
os-brick stable release) and make sure that file is parsed
when installing the services which require os-brick.

Side note: there should be a way to avoid this duplication
also for pip-installed libraries (normal services can use
files/ or even bindep, but in this case the source is not always
available), (temporarily?) duplicate them, as it has been
the case for the other os-brick dependencies already listed here.

Change-Id: I9ab6e215dbef9ebdb1946da2f9a40ce020ecc95b
2020-08-13 09:55:08 +02:00
Abhishek Kekane f8dbfd394a Revert "Drop keystone dedicated ports"
This reverts commit f6286cb586.

This patch is blocking glance as it needs mod_wsgi to perform new import workflow.

Change-Id: I4475247dfe986114d37678b3d3d552c0c7d02ddc
2020-07-06 14:01:08 -05:00
Zuul 48c0d38502 Merge "Add "OVN" debs and rpms package files" 2020-06-30 11:37:29 +00:00
Jens Harbott f6286cb586 Drop keystone dedicated ports
Those historic references to port 5000 and 35357 aren't being used
anymore for some time, so let us drop them.

Clean up some python2/3 wording along the way.

No longer mention Identity API v2, which is also a thing of the past.

Change-Id: Iafff097eee082f24ea2ae27ad038ad115aa36c61
2020-06-26 15:26:22 +02:00
Jens Harbott cd57449c9f Stop claiming to support Debian distros
We haven't had a working job for stretch or jessie in years, attempts
to get things fixed have been dropped, set let's be honest and drop
those from our support list.

Change-Id: Ia6152be79f8044f7ff039ec0911ad4938d6271f4
2020-06-23 08:08:13 +02:00
Jens Harbott f1ed7c77c5 Use python3 pip module instead of pip binary
Focal only provides a pip3 binary, no pip3.8. Instead of working around
that with a symlink, use the module instead.

Add version information output for this variant.

Change-Id: I7af194ecc40e4d43c10ce067a661bb6ab4ca37d4
2020-06-23 08:02:52 +02:00
Ian Wienand 1147300b9c Remove bridge-utils/brctl usage
This package isn't available on some distributions hasn't been
required in Neutron for several years
If679e79fa3242ee1cd8610b5525deca35b41c87e.  Remove it.

Change-Id: I7308a885c1d084efe2f0b9f542443d35966140ed
2020-04-30 10:09:47 +10:00
Ian Wienand 39bc71cedd libs/rpms : update for CentOS 7 removal
With the removal of CentOS 7, we can re-evaluate the rpm
installations.

We should not need virtualenv after https://review.opendev.org/712609

There should be no need for python-devel as we're python3

pyOpenSSL was added to workaround memory issues in
9e98f9435e (2015) ... I think we've
moved on.

pyxattr is not a package; remove it.

I don't see we need packaged m2crypto, which isn't a package on CentOS
8.  nor libxml2-python; these days it has wheels which should work
with the normal installation process.

centos8 has:

 * targetcli
 * pcp-system-tools (and no dstat)
 * iptables-services
 * java-1.8.0-openjdk-headless
 * kernel-modules
 * rsync-daemon

just as all supported fedora's do, so we can remove any matching here.

Change-Id: I542c426a67a98f331d2a29bacd220af81fab8cc4
2020-04-22 14:02:10 +10:00
Ian Wienand 812e7846c9 Cleanup rpm-distro mariadb install
We have lib/databases/mysql which is installing databases, remove it
from the bulk package lists.

Split is_fedora (fedora & centos8 -- soon) to install mariadb-server
and mariadb-devel to retain status-quo.

On suse this seems to be a meta-package

 'mariadb-server' not found in package names. Trying capabilities.

so split that out.  It seems it has never been installing the -devel
package, and things work (presumably clients are coming from wheels so
don't need to build against it).

Change-Id: I86433318e8f76c40c5c792b795411a5c9d8351d3
2020-04-22 14:02:10 +10:00
Zuul fb40a23713 Merge "Fedora: Update required RPMs for Fedora 31" 2020-04-16 17:11:37 +00:00
Lee Yarwood f9e81d3438 Fedora: Update required RPMs for Fedora 31
This includes the addition of the python3-virtualenv package required to
provide the virtualenv binary that is no longer present in the image.

Change-Id: Ie8e66d8b9f93063b97f88f41a626daddf235339b
2020-04-14 10:11:33 +01:00
Ian Wienand e6852eadb1 Use venv for VIRTUALENV_CMD
Using venv, which is part of python3, we avoid an extra dependency on
the virtualenv package.  For Debuntu, which splits this out into a
separate package, add this to debs/general.

This is part of the infra efforts to ship "plain" nodes without any
dependencies installed.  While devstack can re-install virtualenv, we
don't need any features it provides and it means one less dependency.

Change-Id: I3c323640f288e57581a4eb8adba2a08d0b0cbd8f
2020-04-14 15:32:42 +10:00
Witek Bedyk 7a2c1c2b1b Add python3-dev package for Debian based distros
When stacking outside of Zuul CI the wheels have to be built locally and
python3-dev package is required.

Story: 2007491
Task: 39213

Change-Id: I0960269d5cf193c9ececc5490485522c74646382
2020-04-03 10:42:21 +00:00
Zuul d89eb74437 Merge "Amend python pkgs for debian-based distros" 2020-03-19 21:09:51 +00:00
Lucas Alvares Gomes e87428908a Add "OVN" debs and rpms package files
The networking-ovn code is being moved into the neutron repository as
part of the effort [0].

This change is needed so we are able to install OVN from packages
when running the networking-ovn functional tests along with
the Neutron ones (see [1]). In the old networking-ovn repository we did
compile OVN from source instead of installing it from packages but
that took time. We want to do better in the Neutron repository.

[0] https://blueprints.launchpad.net/neutron/+spec/neutron-ovn-merge
[1] https://review.opendev.org/#/c/697440/

Change-Id: I92ab727d9954eb729c41b9a67ecb60b56883097b
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
2020-03-13 16:42:54 +00:00
Lee Yarwood 347abd41ce Drop Fedora 29 support
With fedora-latest now using Fedora 30 images and Fedora 29 itself EOL
we can now remove support for it from devstack. This change also cleans
up references to earlier Fedora releases under files/rpms/.

Change-Id: I24332f7016ebb549ea678acf677c477b55ec4d4b
2020-02-18 11:38:34 +00:00
Lee Yarwood f1966d7763 Add Fedora 30 as a supported distro
With Fedora 29 now officially EOL [1] we need to start the move to 30. A
later change will remove f29 support from devstack entirely once
fedora-latest moves to the based f30 image.

[1] https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org/thread/VUK3CJ5LO4ROUH3JTCDVHYAVVYAOCU62/

Change-Id: If87fe93757129f931a3417fc6275ffad280cdf46
2020-02-18 11:38:33 +00:00
Lee Yarwood 79ffa1f04f Fedora: Always install python3-devel
With the recent switch to py3 we need to ensure python3-devel is
installed on Fedora >= 29 before pip installing any dependencies.

Change-Id: I21417687db5b3827c5272ed22f6dc95db13f2870
2020-02-18 11:38:33 +00:00
Dr. Jens Harbott 64ae08383a Amend python pkgs for debian-based distros
Drop python2 packages that are no longer needed. Replace other py2 pkgs
with their py3 variant.

Change-Id: Id5fc11d1ae0a7daf439c7d72f831f298f5d0ce31
2020-02-17 17:14:50 +00:00
Zuul 5ae6f5ec37 Merge "Revert "Do not use pip 10 or higher"" 2020-01-21 18:36:06 +00:00
Slawek Kaplonski f7a7076fac Include dnsmasq-utils package on all Ubuntu versions
This package provides dhcp_release tool but in files/debs/neutron-common
it was listed to be installed only on Ubuntu Precise.
The same file is also in Nova's packages but there is no restriction to
Ubuntu Precise only there.
So on all Neutron jobs it was fine but on Ironic's job where Nova
wasn't enabled, this package was not installed and caused problems
in Neutron DHCP agent.

Change-Id: Idd0711cfe6d43f21754a2f0c230cd094ea33cb27
Closes-Bug: #1855910
2019-12-11 09:51:58 +01:00