Commit Graph

68 Commits

Author SHA1 Message Date
Satish Patel 2a14242aa1 Re-adding PrettyTable dependency
Re-adding prettytable module which has been removed earlier [1].
"ModuleNotFoundError: No module named 'prettytable'" when
running inventory-manage.py command.

http://paste.openstack.org/show/804143/

[1] https://review.opendev.org/c/openstack/openstack-ansible/+/775126

Change-Id: I6d259e3697ac72262be53b22fcd052d97cd1120a
2021-04-03 06:08:03 +00:00
Zuul 1b967ce57e Merge "Stop installing client packages for ansible venv" 2021-02-20 10:20:06 +00:00
Jonathan Rosser a59d81516f Remove version cap on PrettyTable
This causes the requirements-check job to fail as a result of
uncapping PrettyTable in [1].

[1] https://review.opendev.org/c/openstack/requirements/+/774917

Change-Id: I19c6ad150cca56a6022f9b9b0c9b5d0b7a2cb15d
2021-02-11 11:52:13 +00:00
Dmitriy Rabotyagov 08a3620e05 Stop installing client packages for ansible venv
We delegate openstack client operations to the utility host nowadays,
so no reason to install them into ansible venv.

Change-Id: I4a183598150186f54ef52bef9e10ee915e9b85de
2021-02-08 16:31:51 +02:00
Duncan Martin Walker bf10b55c9e Use parallel git clone
The git clones of required roles contained in ansible-role-requirements
are currently single-threaded and can take a significant time when
running scripts/bootstrap-ansible.sh. A new module, git_requirements has
been implemented in order to multithreaded this, which takes
input as provided in clone_roles in get-ansible-requirements.yml.

The result of this implementation is a sizeable speed increase (~factor
of five across the full bootstrap_ansible script) for the git clone
repos task when tested on a two core VM.

The new module requires all repo information ahead of time in order to
distribute the clones between threads. This is unlike the previous
implementation, which performed the clones in sequence and generated
the appropriate data (destination URL, default refspec etc) per loop
iteration. This information is now generated within the ansible module,
with defaults supplied at the task level.

The retry and delay parameters used for the loop with the built in ansible
git module have been re-implemented in the new module to allow force
cloning, alongside retries and time delays for failed clones.
Change-Id: I3cdb248204c5e03ef98803d366e3a19ed7f8bdd8
2020-11-09 15:22:06 +00:00
Ghanshyam Mann 7720597343 [ussuri][goal] Drop python 2.7
OpenStack is dropping the py2.7 support in ussuri cycle.

openstack ansible also drops python 2.7 support
and cleanup tox and requirements for python2.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: I909ef41d8a8d857c56ed93a142d2ff951a9bf1ba
2020-02-29 18:42:55 -06:00
Dmitriy Rabotyagov 3800754c1f Drop virtualenv pip package for CI
As CI images have pre-installed latest virtualenv package wich breaks
deployments. This will ensure that this package will be dropped and
appropriate one will be installed later.

This also makes ubuntu use appropriate python3 venv tooling and
drops py2 packages installation

Additionally virtualenv is dropped from requirements and ubuntu
upgrade jobs is marked as non-voting until backporting change
to previous release

Change-Id: I50931e0f68176bbaebfb68eec9f84241816c7fe3
2020-02-13 20:40:51 +00:00
Kevin Carter 6e00f579c4 Update ansible to latest stable 2.6.x
With this, the requirement for the os_* modules
changes from shade to openstacksdk. The 'ansible-next'
job is also updated to test stable-2.7.

Depends-On: https://review.openstack.org/#/c/611765
Change-Id: If32f88592caa849c3c7c9701d4803530d9e549ed
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-10-20 07:55:03 -07:00
Zuul 393ec9c425 Merge "Add openstack client to Ansible venv" 2018-08-03 20:55:21 +00:00
Jesse Pretorius a75af3e6fd Add jmespath to requirements.txt
Adding this python library enables us to use the
json_query filter, which is amazing, and deserves
to be president.

Change-Id: I196a2a65268a00a6077b39de6309b5111e98ce97
2018-07-26 20:25:00 +01:00
Jesse Pretorius c8d1f024d5 Add openstack client to Ansible venv
In order for the octavia role to upload the amphora image using
tags, we need to have access to the openstack CLI.

This will be necessary until the Ansible os_image module can
replace this shell task.

Change-Id: Ic75483dbab8a5dd43672b3234dbd19bad3d6e735
2018-07-14 16:23:21 +00:00
Jesse Pretorius 1325449304 Ensure python-keystoneclient has a lower bound
In order to pass the requirements test, every lib in
the *requirements.txt files must have a lower bound.

Change-Id: Ib20057b9cc2860167c6eeed064c303519c4c9b5c
2018-07-02 07:36:21 +01:00
Jesse Pretorius 8530dcc653 Add python-keystoneclient to the ansible-runtime venv
In order to reduce the packages required to pip install on to the hosts,
we install the extra library into the ansible-runtime venv so that tasks
interacting with keystone using our keystone module can do so by using
localhost delegation. Using this strategy, the library does not need to
be installed on the target host.

Change-Id: I7213252ac7fe59550379db3fe755996c1349a959
2018-05-13 20:36:34 +01:00
Jonathan Rosser 3999020199 Add shade library to openstack-ansible venv
This allows playbooks to be run locally on the deploy host using the
os_* ansible modules.

Change-Id: I8941dbb436039076e89dba6fbb3f367f610861d8
2018-03-28 15:20:38 +00:00
OpenStack Proposal Bot 41433a9815 Updated from global requirements
Change-Id: I772b0e7a9b91921f5a0d0ae1f93e84f9615efdba
2018-03-15 07:28:38 +00:00
Major Hayden c834eb9a81
Remove pycrypto from requirements.txt
We stopped using pycrypto for secrets generation a while back
but the requirement remained in requirements.txt. This patch
removes it.

Change-Id: I7cfce70bbf630d1cf15fcc358bdba3371c03ceee
2018-02-14 10:05:30 -06:00
OpenStack Proposal Bot cbd552eee7 Updated from global requirements
Change-Id: I7f083ecf8e19030c621324fa2b162f22cb831bd0
2017-11-16 11:11:46 +00:00
OpenStack Proposal Bot 7dcc04866a Updated from global requirements
Change-Id: I05381aba9b37477adf0c28ec3069cdc4f194a0c8
2017-09-16 23:18:59 +00:00
OpenStack Proposal Bot 9ba94a05e7 Updated from global requirements
Change-Id: I854ea0f1d778d4fed0c4c387222f395b3b127c89
2017-09-13 12:57:41 +00:00
Major Hayden 353694a9ae
Remove urllib3/requests from requirements
This patch removes urrlib3 and requests from the requirements.txt
file so that the CentOS Ceph gate will pass.

At the moment, yum tries to install python-urllib3 and throws errors
when it finds that urrlib3 files already exist (since they were
installed via pip).

This also breaks our periodic builds which, in turn, makes infra
quite displeased.

Closes-Bug: 1715888
Change-Id: I2a8b0ebcca9839d033552130688356c055f71553
2017-09-12 11:08:00 -06:00
OpenStack Proposal Bot e1a997a835 Updated from global requirements
Change-Id: I846f67840c9d478f64595a501ed423f79451c01f
2017-08-18 15:53:50 +00:00
Jesse Pretorius 5f7000e185 Improve ansible host & venv bootstrap
The current ansible bootstrap process tried to
pin the versions of pip, setuptools and wheel
on the host and also uses inherited python
packages from the host. This causes problems
when the host has a version of setuptools which
is cannot be changed (perhaps due to some bug)
or when the host has otherwise undesirable
python packages.

The ansible bootstrap process only needs to
be concerned with whether pip is installed
and understands how to use constraints.

From there we can bootstrap the venv using
get-pip and completely avoid package conflicts
with the host. Once Ansible is bootstrapped,
the pip_install role will ensure that pip,
setuptools and wheel are correctly bootstrapped
and pinned across all hosts.

The pip_install role now uses constraints for
the initial get-pip [1], so we can also remove
the list of pip pins from group_vars.

We remove pip, setuptools and wheel from the
requirements.txt to allow the packages on the
host to be installed in the bootstrap without
forcing a change of pip/setuptools/wheel at the
same time (which causes failures in some
circumstances).

[1] https://review.openstack.org/483905

Change-Id: Ida84fb6bb726e1332f0e29ade51b67a5721f0785
2017-07-20 09:23:30 +00:00
OpenStack Proposal Bot 90016a2350 Updated from global requirements
Change-Id: Ide58f68cf5c9a1d1e97bda1600784a026c4f2e61
2017-06-12 12:58:14 +00:00
OpenStack Proposal Bot 4e074e89fd Updated from global requirements
Change-Id: Ic799162bb9869958966502aea74813a4ed5c4c5e
2017-06-10 21:43:45 +00:00
OpenStack Proposal Bot 2feef333ca Updated from global requirements
Change-Id: I00c76139f2d6292dcfcc27027a9f25994fced337
2017-03-16 22:57:03 +00:00
OpenStack Proposal Bot fe84f80533 Updated from global requirements
Change-Id: Id971e74812642ff3085daa50e66527fc08ffe486
2017-03-08 23:28:42 +00:00
OpenStack Proposal Bot cf6600d303 Updated from global requirements
Change-Id: I2f9decee7ca7c3ea374c4f0a907fe9d9710f5d20
2017-03-01 04:12:48 +00:00
Jenkins fb76f7741a Merge "Use an explicit version of urrlib3" 2017-02-28 16:06:00 +00:00
Jean-Philippe Evrard be08a064b6 Use an explicit version of urrlib3
If not forcing to install urllib3, ansible on the deploy node
will reuse the urrlib3 from site-packages, which could be older
than the one in global-requirements, because ansible doesn't
list it in its dependencies, so we don't explicit force a
certain version to be installed.

On my machine, I had an urllib3 installed with a version of
1.7.1, and all the lookups with https had SSL issues. Moving to
urllib3 fixed the thing.

This only cares about the ansible side, to ensure ansible venv
has the proper version. Hosts targetted by ansible (for example
when using get_url), also need a fix. This will be done in
different patches.

Change-Id: Ia07969ac9aee90724eecc6657affd1dbe16045a7
2017-02-28 14:26:14 +00:00
OpenStack Proposal Bot 9aeea15940 Updated from global requirements
Change-Id: I0d4c10bdfa676a57ef75e5a5cba62a4064d664fd
2017-02-27 01:16:06 +00:00
OpenStack Proposal Bot 41cf2f51d7 Updated from global requirements
Change-Id: I898728f2bb12bb97d6035d165b8b142c6996a903
2017-02-24 00:13:13 +00:00
OpenStack Proposal Bot 2ce635636c Updated from global requirements
Change-Id: Ic019245df1d48639dd447d4393823c988498ff2b
2017-02-11 17:48:07 +00:00
OpenStack Proposal Bot 8d0aa62109 Updated from global requirements
Change-Id: Idcbfb2cfb413b22131c06d01e9648cd10c911bb9
2016-12-02 05:10:09 +00:00
OpenStack Proposal Bot 1b341c8bea Updated from global requirements
Change-Id: Ibb088f3c393aee7d7633f5964b1bc9beec1d3e22
2016-11-24 22:56:07 +00:00
OpenStack Proposal Bot 7b98e41d88 Updated from global requirements
Change-Id: I8c400d62ed237219c8b10aa0872b47bd09737682
2016-09-30 20:03:37 +00:00
OpenStack Proposal Bot 2c2fe6c593 Updated from global requirements
Change-Id: I50ebe2e63f57bb4380355f9e02c83eacda0047aa
2016-09-15 20:23:46 +00:00
OpenStack Proposal Bot a889885848 Updated from global requirements
Change-Id: I36a014d6dfde13a97430f1dc4d1d6c1fd98edd07
2016-08-25 11:09:31 +00:00
Kevin Carter 423c409a67 Remove the ansible.cfg file
The ansible.cfg file has been removed to allow a deployer to specify their
own configurations without having to modify the ansible.cfg file within the
playbooks directory.

Change-Id: I94a8647edd04c21c34f7892f3179a8aa66548ed3
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-08-25 02:06:14 +00:00
OpenStack Proposal Bot 268308289f Updated from global requirements
Change-Id: I1692e7a3ee37c48b81f317cb3637cfa2de54a09e
2016-08-21 03:53:38 +00:00
Jesse Pretorius ab887ee13d Include python requirements to resolve SNI issue for Ansible venv
This patch adds the packages needed to resolve SNI issues that may
be seen when executing tasks on the deployment host.

Change-Id: I2d7c58ffe39faf204485dffb39d982f3ff26c34a
2016-08-18 11:28:17 +00:00
OpenStack Proposal Bot 6cb3b1e216 Updated from global requirements
Change-Id: I8719df1c641a3304c6a74637fa6bde7466ef94bc
2016-08-05 20:25:54 +00:00
Jesse Pretorius 17db05965f Add SNI support via OS/python packages
To ensure that the Ansible venv has the right packages installed,
requirements.txt is updated with the appropriate packages.

To ensure that the OS for the deployment host is appropriately
prepared for the Ansible venv creation, to install pip, and
python packages into the venv, OS packages are installed.

Fedora install support is removed from the script as it is an
unsupported platform and unmaintained section.

Change-Id: Ib4e219294f5fd93c6a50814955ab97424a79dafd
2016-08-01 12:29:44 +00:00
Jesse Pretorius 60bad86d54 Change requirements pin method
This implements the following changes:

- The sources-branch-updater script will only update SHA's/tags
  for OpenStack roles. This is primarily for usage in stable
  branches where external roles must not be changed after release.

- The script will no longer update the *requirements.txt files.
  These files are managed by the OpenStack Global Requirements
  management process.

- The pins for pip, setuptools and wheels are now set in the
  global-requirement-pins.txt file in order to ensure that it
  takes precedence in the repo build process above all other
  requirements.

- The order of the requirements.txt file has been changed to ensure
  that pip, setuptools and wheels is installed first whenever the
  requirements.txt file is used.

- The pin comment has been removed from the requirements.txt file
  as it no longer applies..

Change-Id: Ib98e63153ace7f02b9bbce878aa67fbdddd784b6
2016-07-16 07:27:12 +00:00
OpenStack Proposal Bot 47bd970672 Updated from global requirements
Change-Id: I2476f413fd182a8203604c099d074d06eb1a5656
2016-07-15 12:58:23 +00:00
Jesse Pretorius bb69b667f0 Update all SHAs for Newton-1 2016-06-02
This patch updates all the roles to the latest available SHA's,
updates all the OpenStack Service SHA's and also updates the
appropriate python requirements pins.

Change-Id: Ifc77c02d456500651e8adcaf9338f81601e2c148
2016-06-02 21:05:45 +00:00
Kevin Carter c904de2db5 Isolate Ansible from the deployment host
This change allows Ansible to be installed and executed from within
a virtual environment.

Closes-Bug: 1431324
Implements: blueprint ansible-2-1-support
Change-Id: I2fb19dd8218753f75bd7550ed7ba9b79f777083d
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-05-26 10:41:27 +00:00
Jesse Pretorius 30c59b297d Updates all SHAs for Newton 2016-05-19
This patch updates all the roles to the latest available stable
SHA's, updates all the OpenStack Service SHA's and also updates
the appropriate python requirements pins.

Note that 'ceilometer_api' has been removed from the standard
tempest testing in the integrated build as the tests have been
removed from the tempest repository with
I0775bcc15dc9cbae6e075fe92f44b5f6c9b9d5d2

The ceilometer testing will have to be re-introduced using the
tempest plugin framework.

Change-Id: Iea7e4213893fb331c410c2d09e3258493f4643f3
2016-05-23 14:52:36 +01:00
Kevin Carter b15363c5fa Remove paramiko restriction
Upstream has removed the paramiko restriction of <2 and now has a
dependency on paramiko >2. This is the commit where the requirement
changed: [0]

[0] - e379813e9c
Related-Change-Id: I2369638282b4fefccd8484a5039fcfa9795069a7

Change-Id: I06ba8a9da100a3b02e8eb7f15042f9573b71ed59
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-05-20 12:03:39 -05:00
Kevin Carter e971e15997
Integrated updates after the multi-distro changes
* Corrected the repo server log directory. This change updates the repos
  server log directory which was being incorrectly indexed. The old container
  build process would create a service log directory based on "properties".
  The "properties" log directory is unused or is re-defined by the various
  service roles. This change simply corrects the log paths which the
  `rsyslog_client` role uses to ensure all logs are shipped to the right
  places.
* Add a log directory creation task to the utility container. This play
  incorrectly assumed that the log directory based on "properties" would
  be automatically created. This update simply makes that more explicit.
* paramiko has been pinned to match global requirements and use a version
  <2. This will ressolve runtime issues in Newton with an unbound
  requirements.

Closes-Bug: #1576755
Change-Id: I15d841106ec9a13555b9737c9388f40557f5bec5
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-05-03 22:51:43 -05:00
Jesse Pretorius 2fc728daa7 Update Newton SHA's 2016-04-22
Change-Id: Icab701a3ee2594f9f126582dcfe8eb99ebc79d0c
2016-04-22 17:26:03 +01:00