* Update ansible-role-python_venv_build from branch 'master'
to 366222725e53de3a293275ecd44f0c0ede9cc3e9
- Include REQUESTS_CA_BUNDLE in pip task environment
If the SSH or login session is not restarted, it is possible that
recently written contents of /etc/environment are not accounted for
when installing the built wheels when TLS is enabled on the internal
VIP.
REQUESTS_CA_BUNDLE must be set to allow pip to trust the certificate
presented on the internal VIP.
Change-Id: I7ed37d76c4c3854dc4939c884062bd03a76ece37
* Update ansible-role-python_venv_build from branch 'master'
to d80a61f81089338e5dc9a2cfd98ee3d322014c3a
- Workaround ansible unsafe text templating bug
See https://github.com/ansible/ansible/issues/82598
Setting the 'use' parameter to something other than 'auto'
should cause a different code path to be use in the package
action plugin which could avoid the templating unsafe text
inside the plugin.
Change-Id: I0c14510eb625d74a654a6398387ab74c4355a4e8
* Update ansible-role-python_venv_build from branch 'master'
to c2c961317a5ca9ee36530f0d7e16ae68c867fb58
- Update master for stable/2023.2
Add file to the reno documentation build to show release notes for
stable/2023.2.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2023.2.
Sem-Ver: feature
Change-Id: I79b16ce2f6b0045f8c9239ed86d37389af14c1db
* Update ansible-role-python_venv_build from branch 'master'
to 158ef2dbf1f38843d501e1ecbbaf2fc7f01fa70e
- Use distribution_major_version for all distros except Ubuntu
That is a follow-up patch for Rocky, that as EL derivative also follows
naming convention, where distribution_version contains also minor
version, while we need to build only against major versions of distros.
With that the only distro we need to use distro version for is Ubuntu.
Change-Id: I62f69bc31ed04ab65a167d07de44067fcaa74a66
* Update ansible-role-python_venv_build from branch 'master'
to 3d1740bde20b63cfe95f3a3934d36c2cf229c3ac
- Drop unneeded become overrides
We add become overrides on the task level which neglect provided by
user options in favor of quite narrow usecase. As a result, role can not
be run as a non-root user in cases where venv_build_host is not localhost
Removing become override potenially unblocks running OSA with non-root
user on hosts.
Change-Id: I663bdb1bcc60fec0d9efc55cbb1b2a0c566ab2ab
* Update ansible-role-python_venv_build from branch 'master'
to 965cad09fa4d3d1d5f45b25039181fb51b453b85
- Use distribution_major_version for Debian and CentOS
Debian does include minor versioning into distribution_version facts
causing versions to look like "12.2" or "11.4", which is causing issues
once debian is minorly updated since there might be no repo container
to satisfy such specific request across the board (ie having repo 11.3
while compute reports 11.4). Thus we're taking into account only major
versions for these distros. Same approach has been taken for building
wheels in project-config [1]
[1] https://review.opendev.org/c/openstack/project-config/+/897545
Change-Id: Iaf6ed4dd5e01b25b5935b959c73ab657cfefef47
* Update ansible-role-python_venv_build from branch 'master'
to 824639308e86bf67eb44911ff5f7b8b2b37623e5
- Rename _package_list variable
This is renamed to be more specific to the wheel build process
and not to collied with similarly named variables in other ansible
roles.
Change-Id: I470507ed904bca1845fdccc9494cc62bc10229b7
* Update ansible-role-python_venv_build from branch 'master'
to 43032d25cf73c8c26e9b96195bef15f08ea70dd3
- Fix linters and metadata
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.
With that we also update metdata to reflect current state.
Change-Id: I6b4b83ec472d4a3de9139eb4e7c7b2dc8a9fc260
* Update ansible-role-python_venv_build from branch 'master'
to 0e1abb285872c2f677e3e382f9e7fd13abc76ed9
- Remove warn argument for command/shell
Since ansible-core 2.14 you can't use warn as module argument.
Instead, a tag is used to instruct ansible-lint to
supress alerts.
Change-Id: I4ceeee4bf6c1020851824450ec7b30f6d14573f3
* Update ansible-role-python_venv_build from branch 'master'
to 01a08f39bbb2632500ba318f83882970bdb8accb
- Build wheels depending on presence of venv_build_group
In case venv_build_group is not present in inventory or does not contain
a single host, it does not make sense to attept building wheels.
With that we're changing default behaviour to avoid building wheels
if there're no potential targets exists.
Change-Id: Ifd0e80dd1d1f002a1b80f57b53b81ca9110e719c
* Update ansible-role-python_venv_build from branch 'master'
to e24c1a4f715afcc4b8da56eae21f6806c0300c03
- Add venv_build_group variable
In order to be flexible and get rid of hardcoded repo_all group, new
variable named `venv_build_group` was added. It's set to repo_all by
default, so it is not changing current behaviour remains
Change-Id: I30c2c26abeb103de63aff0946ec7783f902886b8
* Update ansible-role-python_venv_build from branch 'master'
to a3ea0c74abcf694e3142306a2d6377b351c75810
- Add a failure message when no build targets are present
At the moment if no appropriate build target is present role fails with
quite unclear message. This patch aims to provide human-readable
failure message when it's not possible to select build target, but
wheels build is enabled.
Change-Id: Ice35e3e4a14ee74b6ff40bd2385550f2da162878
* Update ansible-role-python_venv_build from branch 'master'
to e43dcdbf6a90057b6f4c8328303d817760b99624
- Align distro_arch naming in the role
At the moment we have 2 slightly different variations of merging
distro_arch - one through `-` and another through `_`. This patch
aligns usage and re-use _venv_build_dist_arch.
Change-Id: If478ab0f0c0a5acc8974a69a11453377c98c28be
* Update ansible-role-python_venv_build from branch 'master'
to 70b25cb6bab60a09438439c0230a3b4e9e41c865
- Merge "Fix venv_build_targets when used with multi-arch"
- Fix venv_build_targets when used with multi-arch
Change venv_build_targets data structure to a single-level dict
with keys of the form <distro_ver>_<arch> instead of nested dicts.
When adding new architectures to the old structure, previous
entries for other architectures were overwritten, leaving only
the last seen architecture for each distro version. This could
result in a "Dict object has no attribute ..." error when trying
to build a venv for any other architure.
Closes-Bug: #2018012
Change-Id: I8ddabf996559b5300b52cad1649d8657889337cd
* Update ansible-role-python_venv_build from branch 'master'
to 95405d21bb197802d1de11a8df5ab3b600613e43
- Merge "setup.cfg: Replace dashes with underscores"
- setup.cfg: Replace dashes with underscores
Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:
UserWarning: Usage of dash-separated 'description-file' will not be
supported in future versions. Please use the underscore name
'description_file' instead
[1] https://github.com/pypa/setuptools/commit/a2e9ae4cb
Change-Id: I3699a7b8ae858463cdd84b11072557049e798309
* Update ansible-role-python_venv_build from branch 'master'
to 26ef6bd7b8913328f688ed58d9fd6344386895a2
- Reduce amount of task that are executed
With this patch we're trying to optimize role runtime by replacing
separate tasks with loops. Hoepfully, this will slighlty improve
permorfance as we need less connections now.
We also create a venv with pip module rather then command.
Change-Id: I219c0336079ca89bde586c8076bd964fada747c6
* Update ansible-role-python_venv_build from branch 'master'
to 185713df0e5126378b8bcbe99ab9c56327b979fb
- Remove TripleO jobs
We are tearing down TripleO master because of project removal.
Change-Id: I2eadaafa33b829e5f8445c947644791166cc78f5
* Update ansible-role-python_venv_build from branch 'master'
to 84f0a74e778e7788839d32c6240ce1178bf5eb94
- Merge "Always build wheels by default"
- Always build wheels by default
Due to some corner cases that are possible with currnet logic,
it was decided to simply it and always build wheel regardless of host
count. While runtime might take a bit longer, it's always better for
scaling up to already have wheels prepared.
Closes-Bug: #2004252
Change-Id: I5f53db8476eb394516fb35d593932d2552b95a57
* Update ansible-role-python_venv_build from branch 'master'
to 2a25337b9844b4b8a90420e864ee38e8168ce586
- Drop empty elements from constraint/requirement files
We should filter out empty elements from lists that are provided as
input to the role, to ensure idempotence of build constraints and
requirements file as a result.
Additionally to that we should ensure to control jinja whitespaces
not to add extra new lines to the end of files.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/872896
Change-Id: Ide47712f775c91d90b749bc8e4d8f87c5bd658f2
Closes-Bug: #2004546
* Update ansible-role-python_venv_build from branch 'master'
to 7fd75008cf483027be72e3536da5cb61463d6eab
- Allow to set ENV vars for wheels build and venv install
It's not always enough to provide extra arguments for pip installation.
For usescases, like isolated installations, and some specific packages
that still utilize deprecated setuptools.installer instead of
PEP 517 installer,
you might find impossible to provide some easyisntall options without
having a configuration file that should be created by deployer as
pre-step. However, these options might be covered with ENV variables.
Change-Id: I9a060cbcdf9f5c54efd423a4b4fe32b418377f86
* Update ansible-role-python_venv_build from branch 'master'
to a6c6dfe358d5644b3056a34efb406ad2820914cf
- Update tox.ini to work with 4.0
With tox release of 4.0, some parameters were deprecated and are ignored now
which causes tox failures. One of the most spread issues we have is using
`whitelist_externals` isntead of `allowlist_externals`
Change-Id: Ic77fb02afbf08e071d7ba0f0eb3a273f03a3a960
* Update ansible-role-python_venv_build from branch 'master'
to 61141126d0a8b05942d6e447ef9ccc8efa402f3f
- Update master for stable/zed
Add file to the reno documentation build to show release notes for
stable/zed.
Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/zed.
Sem-Ver: feature
Change-Id: I720d64d26b796db633fe9cae25c0b0497637ad00
* Update ansible-role-python_venv_build from branch 'master'
to 83998be6b81e756828edc723059e6a5405dd2da6
- Merge "Change default value for venv_wheel_build_enable"
- Change default value for venv_wheel_build_enable
With fixing multi-distro support logic, venv_wheel_build_enable logic
stopped working for metal deployments, that have more then 1 host.
Currently, we were checking, that we are not building wheels if build_host
is the same as inventory_hostname. Which is now true for all metal
deployments. However, other hosts do expect wheels to present, which
results in installation failure.
According to the original idea we need to build wheels only when we have
more then 1 host with same arch/distro combination otherwise it does
not make sense to build wheels and faster to install tarball.
To achieve that we build mapping of all distro/arch combinations and
enable wheels building only when there distro-arch combination
is not unique for the host we're playing against.
Despite it might be overkill, as scenario of having only single host
with uniqe distro-arch combination, and overhead of building wheels for
it would be quite small, alternative would be to hardcode `True` for
venv_wheel_build_enable.
Closes-Bug: #1989506
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/858258
Change-Id: I762e36acf76729fd61f28ca1b03bc9f562b5db0a
* Update ansible-role-python_venv_build from branch 'master'
to 4d766a1f9d9993e2bb3647fdcf19da23fffbae61
- Remove redundant vars line
This line was introduced by I3c2164ca8f610fbef88744d9acfca4e926059c81
but should already be covered by the distribution_major_version line
above.
Change-Id: I33a9a85649ff955adfdc8164f42610400381f5bb
* Update ansible-role-python_venv_build from branch 'master'
to a3b3b7d7aa4751877d7227dec265f1010ce8743e
- Merge "Gather build hosts facts once."
- Gather build hosts facts once.
In [1] the build host facts gathering was moved from a place in the
code which was covered by run_once to one that is not. The task is
now missing a run_once so it iterates over the repo_all group for
each host in the play which could be a very large number of tasks.
[1] I79a10a3f774246ab9c57313f19f3804f9422e08c
Change-Id: Ic86c19ca48fb55ad2c464c22f31ac41c5cc4c2e5
* Update ansible-role-python_venv_build from branch 'master'
to 57a2f226ebca7a2ec21920d4fd2b7ccf45490684
- Run wheels build for each unique distro/arch
There may be multiple architecures and OS versions in
ansible_play_hosts. However, with run_once we build wheels only
for one selected distro and do not repsect multi-arch/multi-distro
setups.
Instead of run_once we need to select single (and first in play)
host of each architecture and distro and delegate wheels building
from it. That is needed, because venv_build_host is selected based
on the facts gathered for current inventory_hostname and will
depend on it's arch/distro.
Change-Id: I492d17169538ad2768e28f7c48314bdec407ab36
Closes-Bug: #1964535
* Update ansible-role-python_venv_build from branch 'master'
to 04f58473495eb3127ffd2486ae993355269c9905
- Switch sphinx language to en
With sphinx release of 5.0.0, they changed default for language variable
to 'en' from None. With that current None valuable is not valid and should
not be used.
Change-Id: I693a7be59720ab2e00088e8f0dfbfd646316afb7
* Update ansible-role-python_venv_build from branch 'master'
to 5051b52275271f4b2c2d2a6488f7e00c938687c3
- Split venv_rebuild functionality
This patch aims to split functionality of ``venv_rebuild`` into 2
separate parts - rebuilding venv and rebuilding wheels. That will give
more control over what needs to be done.
Change-Id: Ie14f12c6756cd1f866b660acc8fd6aa5695f6c33
Related-Bug: #1914301
* Update ansible-role-python_venv_build from branch 'master'
to 43d785eff201eeb2c0e0dcfce1e4fef536912932
- Improve python version detection when symlinking libraries
The previous code assumed that the major version ('python3') could be
obtained by removing the last two characters from 'python3.X. This
does not work for pyhton releases 3.10 and onward, so this patch
changes the code to split the string on '.' and take the first portion.
Change-Id: I771cac2fc9badb61c22cfde65775aee97c31f89c
* Update ansible-role-python_venv_build from branch 'master'
to 88cb4d60fb5524efa0e2e4a11439076fdaca870d
- Revert "Cleanup setup.py config"
This reverts commit 7166e396d1f9fa3168472a0a2240b762cc6bf32b.
Reason for revert: unbreak tripleo-quickstart
Change-Id: I91a21138a20331e2123f878ac8fb51a6552e2cef
Disable setuptools auto discovery
Jobs are broken due to a release of setuptools 61.0 introducing
non-backwards compatible breaking changes [1]
Setting py_modules to an empty string here will disable autodiscovery
and should permit jobs to run as before.
[1] https://github.com/pypa/setuptools/issues/3197
Signed-off-by: Neil Hanlon <neil@shrug.pw>
Change-Id: Ie0215582fe630f1897a85c1e88fd578d100bc3cf
* Update ansible-role-python_venv_build from branch 'master'
to 3dddb491a999e8a8a1409ef98a2aed84f419c7df
- Merge "Slurp constraints.txt separately for each host in a batch"
- Slurp constraints.txt separately for each host in a batch
Currently playbook slurps constraints.txt valid for a first host in a
batch and then uses it for each other host.
It saves time but batch may consists of hosts with different operating
systems or even OSA versions and current mechanism doesn't take this
into account.
In my opinion we should slurp constraints.txt separately for each host.
Change-Id: I9ba1d342af13e9c923151f735940bda29bf1b262
* Update ansible-role-python_venv_build from branch 'master'
to 0220a8405e09811e3fa05f60858ad5e38e1d09f8
- Move TripleO zuul jobs layout to centos9 only for master branch
See topic branch [1] for related reviews.
[1] https://review.opendev.org/q/topic:c8_teardown_master
Depends-On: https://review.opendev.org/c/openstack/tripleo-ci/+/829236
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Change-Id: Iddb33f1b842b0844ce2cdaaf6fa0fb24d9553beb
* Update ansible-role-python_venv_build from branch 'master'
to a57336728f8cb72d369792c28b38ab08d939825a
- Merge "Remove legacy comment"
- Remove legacy comment
The repo-build process is removed long ago so this comment is no
longer required.
Change-Id: I8ac4d9e45708d17bf4e3af99249b4cb204da29ea
* Update ansible-role-python_venv_build from branch 'master'
to 4a1490377ccde785d4f066f94608d18978894e0c
- Merge "Add per-distro vars files"
- Add per-distro vars files
The conditionals for packge installation are getting more complex
than can be handled with the ternary operator, so create per distro
vars files to describe these differences.
Change-Id: I3c2164ca8f610fbef88744d9acfca4e926059c81
* Update ansible-role-python_venv_build from branch 'master'
to c46feec5c604879db5979cd2ee1710dec895f4e5
- Fix conditional for installing required wheel build packages
Previously the build base package list was ignored, and the
condition included the install package list which was not relevant.
Change-Id: Iadedc5b35f1919058c9971b3ef03d403d153ed62
* Update ansible-role-python_venv_build from branch 'master'
to 4d826ad133fcd5c6ad924dac565672f5267c33d8
- Fix wheel build alert condition
Currently alert is raised when wheels are built while it should be triggered only
when they are not.
Change-Id: I3c3547c3fa6df2bef6834e0cec4d43ef03b82f5f
Closes-Bug: #1956026
* Update ansible-role-python_venv_build from branch 'master'
to 8701591120144766442f4e915f37465f0506a9e2
- Replace virtualenv with exacutable for pip
We create virtualenv with a separate command since we need to create
constraints before running pip. In the meanwhile when using pyenv
it becomes more messy to use virtualenv, as it would require
setting virtualenv_command and you can't define virtualenv_python then.
Using exacutable requires setuptools to be present
for ansible_python_interpreter
Change-Id: I5bf617ad0eca3dd5e58e25af7b44d536dc4579d3
* Update ansible-role-python_venv_build from branch 'master'
to a70aba465e9f7ae71bbbf01944016743baa51d73
- Merge "Switch to LXC jobs"
- Switch to LXC jobs
This role might fail with LXC scenario and metl jobs may not catch these
scenarios as we need to have separate build and install hosts.
Change-Id: I089a42fd06dc3874abd050e7556d2ce60ca0ce00
* Update ansible-role-python_venv_build from branch 'master'
to 775091546db094dfdb1cd84b2851dbee83d2e77f
- Merge "Install python3-venv on destination hosts"
- Install python3-venv on destination hosts
python3-venv is a requirement for destination hosts, not only build host.
Thus, we introduce new variable `venv_install_base_distro_package_list`
which acts simmilar way to `venv_build_base_distro_package_list`
except it's installs packages for install hosts as well.
Change-Id: I64843518b39f930f6983ab51bb3d053c04e6b8ed
* Update ansible-role-python_venv_build from branch 'master'
to aa39e0952991ef8dd8116b7630a32c944701207c
- Merge "Verify that wheels are built while running against multiple hosts"
- Verify that wheels are built while running against multiple hosts
Infra team has reported that OSA deployments use to DDoS gitea.
During some investigation we discovered that the potential workflow that
lead to DDoS was incorrect order during upgrade, when
compute nodes were upgraded before controllers and thus
each compute node was independently pulling same repos as no repo
server was available to this specific OS version.
Based on that we're implementing check that allows such behaviour only
if it's performed in a serial manner.
Undocumented variable `venv_wheel_build_skip_check` is added for cases
when local repos are used for service deployment and limitation should
not be applicable.
Change-Id: I54b2d4bdf6d101cc72c92be196962902a6138757
* Update ansible-role-python_venv_build from branch 'master'
to 4ec289281becca2f34fdf53a5657cfaf76807a57
- Use the native python module to create virtual environments
Do this instead of using the 'virtualenv' command
Change-Id: I9fa8c3379d7f01486abdfb8c8264575328d2661e
* Update ansible-role-python_venv_build from branch 'master'
to b59fd648c2953f37df8b5c4a04733e9314f406f1
- Improve speed of permissions changes
The previous patch was very slow given the number of files
which required permissions changes. This targets the entire
temporary directory recursively.
Change-Id: I7418d285ea052f67febee31038d92c772b72593b
* Update ansible-role-python_venv_build from branch 'master'
to 680a72887656515c928c3d767ff68a01ebb6df8c
- Add distro/arch to requirements file path
This addresses an issue when upgrading between different operating
systems. In this case, the requirements files already exist for
the old OS, which prevents new wheels from being created for the
new OS. By using a different requirements path for each OS this is
avoided.
As part of this, repeated variable construction for the
requirements path is factored our into a vars file.
Change-Id: I881a40fee31df78bf96e451509671543a49520d9
* Update ansible-role-python_venv_build from branch 'master'
to 1f69673fe2f3be270418da8e131987088eff31c4
- Add variables to set permissions for created files
When this role is used alongside the OSA repo server it is
important to be able to control file and directory permissions.
This patch adds variables to optionally set these when creating
requirements files and wheels.
Change-Id: I1131a7e9aa2345d4c07a88a2ad2c8a36e7a35d2b
* Update ansible-role-python_venv_build from branch 'master'
to 82dcd020fa950d1df3a2a71e263cb0cce3386874
- Replace linters test with integarted one
We've created integrated linters check job a while back and it's successfully
working for several releases. At the moment we experience difficulties
with future maintenance of the linters check from the openstack-ansible-tests
repo. So instead of fixing current one, we replace it with modern version of
the test.
Change-Id: I7e1109dfe14ff11cc3a36d881ab2477a53a272d2
* Update ansible-role-python_venv_build from branch 'master'
to 53db970168dd3847e244173c740470c493f2760e
- Fix facts gathering when run with tags
Currently facts are not gathered when role is run with tags
This patch fixes that behaviour and prevents role failure.
Change-Id: If27a6e57c56a0b294442b71aa777d759bc401868
Jonathan Rosser
OpenStack Release Bot