- Ceiling of pbr version can create installation conflicts with pip.
- Fixes tox default env list to be python version agnostic
- Updated template to use newer jobs (include testing with newer python)
Bug: #1921679
Change-Id: Ifd0d2f810799992faa1db41b80bf93bc97d5d19d
Looks like 'dib-run-parts' is not available in $PATH for 'os-refresh-config'
to use. This seems to be installed in the venv for os-refresh-config. Create
a symlink in /usr/local/bin like 'os-refresh-config'.
Change-Id: Ib6ad331292720ec418a176dd3f2e0df520e6b2bf
Nothing in the os-refresh-config element actually depends on the
os-collect-config element.
Also the os-collect-config element is being replaced by the
tripleo-puppet-element overcloud-agent element.
Currently the os-collect-config (or overcloud-agent) elements are
specified explicitly by image builders in the root elements list, so
removing it here has no negative effect.
Change-Id: Iaf70003fdd8d1782dbcb9e0f01e6468887c6de27
They contained json data, so they need application/json as the
content types. Else, some wsgi implementations will interpret the
contents in an undesired way, such as apache's mod_wsgi setting the
whole POST body as a POST parameter with no value.
Change-Id: Id988e8d286761550da4849c0695f5f5a37116a11
Closes-Bug: #1641589
Currently we send signals for every deployment, even those already
completed, so instead keep a record of already-signalled deployments
similar to how heat-config does, which will remove the annoying
SIGNAL_COMPLETE: Unknown events (and possibly be slightly faster too).
Partial-Bug: #1564627
Change-Id: I0ec7ed4cc4e58a43a91e8323cb9a1bcaaafc9752
The version of pbr we were using was having trouble parsing a new version
identifier for python-qpid, We have the pbr version hardcoded in many
places so update it to match whats currently in global-requirements.
Fixes bug #1470871
Change-Id: Ia5aec472555ad1368684b224a55c29242ae40b58
Currently signalling via 99-refresh-completed only uses the legacy
curl data payload (e.g that for CFN compatible WaitConditions),
but now we're using it exclusively for signalling SoftwareDeployment
resources, which support a different payload, including deploy_stdout
which is accessible via a resource attribute.
Returning a payload for deploy_stdout containing the derived config ID
is potentially useful as it changes whenever the SoftwareConfig or any
inputs change, thus can be used as a unique identifier for dependent
resources which need to be re-triggered when some config is reapplied,
for example puppet manifests with a dependency on hieradata applied
via a SoftwareDeployment.
get_attr: [ControllerDeployment, deploy_stdout]
Partial-Bug: #1463092
Change-Id: Ic62b6303795f70f3b9ace46731a21d1c0e7fe6c7
Currently we have a TripleO specific template pattern, where all
deployment resources are configured NO_SIGNAL, regardless of what
DefaultSignalTransport is set to, and only one signal for all
deployments is sent via the *AllNodesDeployment resources, by adding
the heat-generated deploy_signal_id to the structured config data
consumed by os-apply-config (as "completion-signal").
This is inconsistent with all signalling done via heat-config (e.g
everything other than o-a-c, which is triggered via a hook via
55-heat-config, this transparently consumes the heat-generated
deploy_signal_id and signals heat after each deployment hook is
run.
To allow per-deployment signalling for os-apply-config configs,
this adds logic which looks in the deployment data processed by
os-apply-config and signals all deployments deploying a config with
group "os-apply-config" (everything else should be handled by 55-heat-config)
Note that if the deployment is configured NO_SIGNAL, no deploy_signal_id
will be set, thus this will do nothing, and currently this won't work with
HEAT_SIGNAL, only CFN_SIGNAL (which is the default for deployments with no
signal_transport specified). In future it would be good to add support for
HEAT_SIGNAL to break the dependency on heat-api-cfn.
This is backwards compatible, but after it's merged we can remove all the
NO_SIGNAL's in the templates, and the completion-signal key from the
allNodesConfig, which should in future allow more flexible control of
the ordering of metadata update for configs applied via o-a-c (as well
as better visibility of progres during deployment).
Co-Authored-By: Dan Prince <dprince@redhat.com>
Change-Id: I72ea524effd07deeb432fb38ee7da5f3dc7990a7
Closes-Bug: #1389178
The default os-refresh-config base dir changed over 6 months
ago in I98e93d91685ae25ae74a5470483a1cc8b97da4e5.
Using the deprecated default may cause problems in the future,
instead query os-refresh-config for the default directory.
Change-Id: Ia5b8fceb83ebd32d9a537d793a80cd2b8341c8ec
Migrating install-packages commands to package-installs-* scripts where
possible.
This patch is broken up into multiple parts to minimize impact and
review overhead.
Change-Id: Idd1be32149e7238d53d04a34170c5029dda83f1f
We were installing some packages with support for prerelease versions in
expectation of needing it in our CI. This turns out to not be needed so
we should remove it.
Change-Id: I7ae440d3534e33144942c2230a88cf54f8a153d3
This reverts commit d088fde2ea.
Since the underlying problem with argparse has been resolved
in python 2.7 and filtering out argparse for python 2.6 has been removed,
there is no longer a need to explicitly install argparse for python 2.6.
Closes-bug: #1346364
Change-Id: If9534bf5a4400f5f72475c67a8e6e14da08f9562
Since RHEL 6.5 uses Python 2.6, which lacks argparse,
they need to be installed to ensure that os-apply-config
and os-refresh-config work properly.
Change-Id: I80dcce56d00fc6f1240a5fcccd425cc2b4db29d5
Specify pip install --pre so that local git built packages will be
installed if available. It should be possible to always specify
--pre since public repos do not have snapshot release packages for
these tools
Change-Id: Id2e539ba72ac2ef8d39a3ab694991e9b8bf724d7
As advised in I072cf8bf6748d0c910fecffdf2282bcc4656d038, code should
use 4 spaces for indentation.
This commit enforces the use of 4 spaces indentation.
In order to simplify the review process, this patch only cover the
following elements:
- os-apply-config
- os-collect-config
- os-refresh-config
- os-svc-install
- pacemaker
- pypi-mirror
- qpidd
- rabbitmq-server
- snmpd
- stackuser
- tripleo-cd
Change-Id: I3f365f6a1cd6fd9e56402ad3bd6572192b85d798
When we curl to the complete waitcondition url we don't consider heat
returning an error as reason to exit with error. This prevents us from
attempting to re-run the configuration and thus we never signal heat
again.
Change-Id: I915825d76ec889bc09b81ad93f70b34d909262b0
Closes-bug: #1306294
Since I43fd90647acba400cea11c665fb587856514b0ee dib-lint
ensure the elment-deps file is properly sorted.
Change-Id: Ie0347951d99a227fe60530645eeda0f4aa55c325
This updates the source install scripts for os-apply collect and
refresh config elements to activate their virtualenvs before
performing pip installs, as without this specifing a manifest for
e.g. os-apply-config breaks devtest build
Closes-bug: #1305151
Change-Id: I3c42de180f76fb122637c801541f14e50169de89
The new software config / deployments resources use signals.
A subtle difference between wait condition handles and signals in Heat
is that signals use POST, but wait conditions use PUT. This has
historical reasons that mostly are due to Amazon compatibility. We need
to continue to support both until we can eliminate all use of wait
conditions.
Change-Id: I69d9103b07100f700fb71a2cbbf267042625e7a7
Create the pip-manifest element to encapsulate the saving and use
of pip manifest files.
Ensure that it installs prior to any elements that should be able
to use it.
Write pip manifest files for installs in:
* os-svc-install
* os-apply-config
* os-collect-config
* os-refresh-config
* openstack-clients
Enable using a manifest file to reinstall exact versions of
packages in a subsequent run through the DIB_PIP_MANIFEST_*
environment variables.
Change-Id: I4d4ab5921c534727b48cb9969ec8ecfd2c26e6ed
When a pypi mirror is used with certain conditions installing pbr as a
dependency fails. This is a temporary fix until the underlying cause
is resolved.
This is also required for a wheels based pip mirror to work.
Change-Id: Ia08eb16b96d9897fa2f86239ea264b145ef0b9b5
Closes-Bug: #1293812
os-refresh-config relies on being able to run virtualenv, which is only
guaranteed to be possible after running 01-install-pip from the
pip-and-virtualenv element.
To guarantee this ordering, move
install.d/os-refresh-config-source-install/01-os-refresh-config to
install.d/os-refresh-config-source-install/10-os-refresh-config
And for consistency in script execution timing, move
install.d/os-refresh-config-package-install/01-os-refresh-config to
install.d/os-refresh-config-package-install/10-os-refresh-config
Change-Id: I3b0a127c0c6eca16854f84a3c733cbd54c8a4880
Closes-Bug: #1292514
Package install support for:
* os-collect-config
* os-refresh-config
* os-apply-config
This change requires a corresponding change to dib to move the install
type enablement out of source-repositories since the os-*-config
packages do not use source-repositories:
cccdcb904d485fce92443edbb088740e074c3046
Change-Id: I22672a4312152ab594de2a149e16866c7564e3d4
We install pip with get-pip.py and use this to install virtualenv. This
is done as part of the new pip-and-virtualenv element. All elements that
need pip or virtualenv should use this element to install them.
Our motivation is that we need to talk to a pypi mirror generated with
a recent http://github.com/openstack-infra/pypi-mirror This mirror
caches some 'wheels' that the previous version of pip & virtualenv
can't install.
virtualenv contains its own version of pip that is used for all
virtualenv environments.
Change-Id: I282fd8fffc8d5707a078f78f2b4571138d7266f3
This script should be set -e. Also added -ux for good measure. On RHEL,
os-refresh-config was failing to run due to a missing requirements.txt line on
argparse (separate patch), however the failure was not failing the image build
b/c this script was not -e.
Change-Id: Id36d61442d8104cada8346f47e63858cf7f08096
Need to use setuptools>=1.0 in order to get fixes which
allow fetches of dependencies from pypi to work correctly
when operating behind a proxy.
Ref: https://bitbucket.org/pypa/setuptools/issue/52/ssl-errors-with-https-proxy
and also some prior fixes to bug #1201253 in change id
I7cca000857f7691d4cb723d0a0c22a202da703f8
Change-Id: I3f11b4b9b8d2bd59f32c5d4de5ee58f1423287f5
Fixes: bug #1201253
This was used required back when the pip install referenced a
git+https://... url and is no longer required.
Change-Id: Id4243712ece3c847ad26d585448bff20b2aae8fe
We had been installing this from a git repository into the global site-packages.
Which is inconsistent with how we install os-apply-config.
Change-Id: I1a2167d3e71cc97bd387191b44eb4d59b62ce00c
Replace `run-parts' with `dib-run-parts' since os-refresh-config is
using it. Correct markdown code block format.
Change-Id: If0184b80d0f35bc30f9799135bd4f659889bcbbd
Most of the time we will find it useful to pass a wait condition in for
os-refresh-config to signal when it has completed successfully.
This supersedes the notcompute element's completion-handle, so we can
remove that element. It should be removed from tripleo-incubator and
tripleo-ci as well.
Change-Id: Iff65da3dd0a1db496cfd33b4a55abb13c68c9558
We now prefer git.openstack.org to github.com. Use
openstack git repo URIs in source-repository-*
files of projects we host.
Change-Id: I81c393461efee80f9aa8f540369fdfe32f0d159b
Os-collect-config is meant as a replacement for cfn-hup. The @reboot job
can go because os-collect-config will be run when the system is booted.
The periodic job is handled by os-collect-config's default behaviour of
running its command (by default, os-refresh-config) whenever any of its
metadata sources change.
Change-Id: Id768927ddfc4e7b4873fac347905e44458fd341a
cfn-hup needs to be the only thing running os-refresh-config. Otherwise
the other orc scripts will race with cfn-hup writing out new metadata.
The exception introduced here is to also let crond (which runs cfn-hup)
run os-refresh-config @reboot to seed the configuration.
Without a need to start up automatically, there doesn't seem to be any
good reason to use an upstart job / systemd service unit. Instead we
can just pipe os-refresh-config output to logger.
Also because we have several places to run os-refresh-config, we
need to move the locking into os-refresh-config itself. Please see
https://review.openstack.org/37319 for that change.
Change-Id: Ic35eca943ba76d421b61710b9fff018e17fb40bf
cloud-init is populating the file /var/lib/cloud/data/cfn-init-data
with metadata and needs to happen before os-refresh-config is run
which in turn needs to happen before the cfn-hup cronjob.
Change-Id: I07fc60c4a165a42eeae1a8e5a20609a6e17fd790
Systemd services should be created at /lib/systemd/system instead of
/etc/systemd/system, once the service is enabled systemd will create a
link from /lib/systemd... to /etc/systemd...
Also systemd services should contain the absolute path to the executables.
Change-Id: I741fe249de8ecc7b2af100ca6cf55c51f86f84b0
Ghanshyam Mann