- Ceiling of pbr version can create installation conflicts with pip.
- Fixes tox default env list to be python version agnostic
- Updated template to use newer jobs (include testing with newer python)
Bug: #1921679
Change-Id: Ifd0d2f810799992faa1db41b80bf93bc97d5d19d
map-services will be removed on dib v2 branch, but
tripleo still relies heavily on it. To prevent breakage,
copy the script to the internal element using it.
In the future, all elements will need to be migrated
to new svc-map, but it is a massive change.
Change-Id: I9f450de66044123ab07fa498b68f744d1029089a
Looking through the image build logs, I noticed we were pulling in
the mysql-devel packages for some reason. It turns out this is a
historical artifact of the old TripleO source installs using
os-svc-install. It was universally installing mysql-python on
every image where it was used because that package wasn't in the
requirements.txt for the OpenStack projects.
This was wrong in the first place, and now that we aren't supporting
that installation method anymore it doesn't make sense to keep
installing nine development packages that we'll never use on our
images.
Note that the only thing we use from os-svc-install now appears to
be os-svc-enable, so this should have no effect on the current
TripleO flow, except slightly smaller/faster image builds.
Change-Id: Iae3e6ae3f57de57639c4259361e82c8150fe2f0e
The version of pbr we were using was having trouble parsing a new version
identifier for python-qpid, We have the pbr version hardcoded in many
places so update it to match whats currently in global-requirements.
Fixes bug #1470871
Change-Id: Ia5aec472555ad1368684b224a55c29242ae40b58
We are no longer piping our service output to logger, so we should be
logging the stdout via upstart. This is useful for detecting tracebacks
and other info which does not go through python logging.
Change-Id: I5d4abb94ff15f74fd2478e70d0fa4d3bb9c09f2d
This was added in If3300a6b1ef1c633fc339f884e9dab0d82bc0720 and should
be removed now that the new setuptools version fixes our issues.
Change-Id: I3ded4f83918d74ca91a8c7a4aede9ef4d1d7a489
We currently have a horrible hack for upstart where we run our services
with stdout and stderr piped into logger. There are lots of issues with
this, such as upstart has a hard time knowing which PID to track.
Conveniently, python logging knows how to syslog, so why not just use
that!
This changes the default behaviour of os-svc-daemon to create a python
logging.conf so services log directly to syslog.
This also changes the default upstart respawn limiting to be 2 fails in
10 secs. This is becauce making upstart monitor the actual service
daemon was causing some services to respawn loop due to time it took
them to fail.
Closes-Bug: #1385346
Change-Id: Ibd491881d79160ad1457526d51512f5fc1cf9ea5
Setuptools 8.0 changed the way it parses version strings and many of our
openstack packages break when using it. Pinning this for now until our
packages are fixed.
Closes-Bug: #1402305
Change-Id: If3300a6b1ef1c633fc339f884e9dab0d82bc0720
We advertise supporting passthrough arguments in os-svc-daemon by using
'--' followed by extra params. This does not work if using the
undocumented old-style positional arguments and fixing this is a
non-trivial parsing problem. Lets just deprecate this old behaviour so
we can remove it.
Change-Id: I0e75f7f975325b5f199daf697004628461a20075
When installing openstack clients, which have been pulled in as
requirements for openstack services, allow for the installation
from source repos, if the source install type has been selected.
Change-Id: I72037c5679b17102de5df02d08194fdb56f856d4
Closes-Bug: 1390051
This reverts commit 615c6bb95b.
Using "expect fork" results in hitting a bug causing some service
restarts to hang which in turn is preventing os-refresh-config
from completing.
https://bugs.launchpad.net/upstart/+bug/406397
Change-Id: Ib893b1428e3cac3084d07b4d98c9cb69d60bb3cf
Otherwise upstart doesn't get the correct process id for the service
and in some causes can fail to kill the process during a servcie stop
Change-Id: I138dd2206ed1e511fbabaf43bd3cb1d163c76db4
Closes-bug: #1385346
Debian and Ubuntu use /lib/systemd whereas Fedora usr /usr/lib/systemd as the
default system unit directory. This already leads to conflict when the
debian-systemd is used.
Fedora provides a symbolic link between /lib/systemd and /usr/lib/systemd
to also support /lib/systemd directory. With this patch, we use /lib/systemd
everywhere.
This patch supersedes I413a92284e6a79e7fcdde99c5138bc3ce8e85e80.
Change-Id: I459f7514ab35082d31607968252a9005fa25de2a
See: https://bugs.debian.org/719634
The 'a' option was missing from the getops set of available options
in os-svc-restart. This is causing CI failures in patces with an
"illegal operation -- a" error.[1] This patch adds the 'a' option
to os-svc-restart.
[1] Ib3bc5c47de1b8bb6006530d56e9d47c3763af7b5
Change-Id: I527c5b0cda0cda7877d23422702517760c1bf340
This patch incorporates the svc-map patch[1] into other
elements by integrating svc-map into the os-svc-install
element. This patch adds the -a option to select svc-map
instead of the default of map-services. The -a option
enables a phased transition from map-services to svc-map
usage throughout the rest of the elements.
[1] Id11433ea342aace71a358936a7ca3151ec11d506
Change-Id: I382631746e458f2f8603a7dcf5221a21f4b53255
Since we already have a wrapper script for enabling services called
os-svc-enable, drop all direct usage of systemctl enable, replacing
it with os-svc-enable.
Also drive-by replacements of == with =. == is a pattern match.
Change-Id: I707bf6f02207308e8f61caff17a27b0697096359
This reverts commit 0011e19c15.
Eventlet version 0.15.1 has been released with the fix to the
"Second Simultaneous error"
Change-Id: I1e6e27f32f1b24e14a4ecb91f3433647fbb69a3e
/var/run/<service> needs to be relabeled with the correct file
contexts. With systemd, these directories are created anew each
boot by /lib/systemd/system/<service>-create-dir.service.
os-svc-daemon was updated to correct the file contexts.
Change-Id: I6487d0726db08912eed3062dfee2552940eadcfb
Closes-Bug: 1346559
We have patched eventlet to resolve the issues around
Second simultanous read errors. Pulling in the eventlet
pull request here.
This is a temporary workaround until the changes get merged
into eventlet and released at which point this will be dropped.
Closes-bug: #1308079
Closed-bug: #1229475
Closed-bug: #1308045
Change-Id: I0495f8c443429d9bdbbd97d46fe9a8e180b867ee
os-svc-enable-upstart currently only works for Upstart scripts,
and will print a warning that the job doesn't exist, and then
create an .enable file anyway. If the action is enable, and the
Upstart config file doesn't exist, but the initscript does, run
update-rc.d.
Change-Id: Ib1e3d4e2963dd2ce4d96078781a929205c25d886
Logging to an upstart logfile results in log entries that are not
associated with a timestamp(e.g. os-collect-config). This isn't
consistent with the log files produced in other openstack CI tests
and means we can't index them in logstash (at least consistently
with how the other projects are indexed).
As a result of this we will need to remove the double time stamps that
get produced (by taking them out in each of the services configuration)
and extract the log at the end of CI into per-service files so that we
can selectively index the files we want.
In particular the service that doesn't contain timestamps in all of the
log entries is os-collect-config which is arguably one of the most
important service logs we want indexed. We could simply just divert this
service alone to syslog but doing them all is more consistent with other
distros and how this will presumably work when ubuntu moves to systemd.
Change-Id: Ib8c1fc39d56b2b9c6d8e9b64a868def619aa2f1f
As advised in I072cf8bf6748d0c910fecffdf2282bcc4656d038, code should
use 4 spaces for indentation.
This commit enforces the use of 4 spaces indentation.
In order to simplify the review process, this patch only cover the
following elements:
- os-apply-config
- os-collect-config
- os-refresh-config
- os-svc-install
- pacemaker
- pypi-mirror
- qpidd
- rabbitmq-server
- snmpd
- stackuser
- tripleo-cd
Change-Id: I3f365f6a1cd6fd9e56402ad3bd6572192b85d798
In some circumstances we need to be able to create a /var/run
directory without actually installing a service. This behavior
was supported before, but these changes allow the user to specify
the directory name and streamline the process by making -c an
optional parameter instead of requiring a dummy value. It also
makes this functionality available on upstart-based systems.
The existing uses of -d had to be updated as well to reflect this
new behavior.
Change-Id: I34f33f2290eeefb5be82c0cf5198ae53d34332ce
Create the pip-manifest element to encapsulate the saving and use
of pip manifest files.
Ensure that it installs prior to any elements that should be able
to use it.
Write pip manifest files for installs in:
* os-svc-install
* os-apply-config
* os-collect-config
* os-refresh-config
* openstack-clients
Enable using a manifest file to reinstall exact versions of
packages in a subsequent run through the DIB_PIP_MANIFEST_*
environment variables.
Change-Id: I4d4ab5921c534727b48cb9969ec8ecfd2c26e6ed
When a pypi mirror is used with certain conditions installing pbr as a
dependency fails. This is a temporary fix until the underlying cause
is resolved.
This is also required for a wheels based pip mirror to work.
Change-Id: Ia08eb16b96d9897fa2f86239ea264b145ef0b9b5
Closes-Bug: #1293812
Update os-svc-install with a new -i option which can be used
to specify the virtualenv directory.
Update os-svc-install with a new -s option which can be used
to enable system-site-packages within the virtualenv. By
default this is disabled (matches virtualenv's default...)
In doing this we remove a hard coded check for the 'nova' service
by making nova use the -s option instead.
Update os-svc-daemon with a new -i option which can be used
to specify the virtualenv directory.
NOTE:
I used -i for both os-svc-daemon and os-svc-install since it wasn't
already used and I wanted to make the option the same in both places.
Change-Id: Ief53567711b0153de890d50a28c471b1e5acd56c
Updates os-svc-install so that we disallow the -n and -c options.
Also, corrects the optargs for -h so that it doesn't take a parameter.
And fix some variable bound issues that could occur if the script
was executed with invalid args.
Change-Id: I4eb26fe6793aa1ad19b1dddc63a5f399e17a8325
Closes-bug: #1285283
We install pip with get-pip.py and use this to install virtualenv. This
is done as part of the new pip-and-virtualenv element. All elements that
need pip or virtualenv should use this element to install them.
Our motivation is that we need to talk to a pypi mirror generated with
a recent http://github.com/openstack-infra/pypi-mirror This mirror
caches some 'wheels' that the previous version of pip & virtualenv
can't install.
virtualenv contains its own version of pip that is used for all
virtualenv environments.
Change-Id: I282fd8fffc8d5707a078f78f2b4571138d7266f3
For systemd services, we create 2 services: one for the daemon and one
that creates the run directory under /var/run for the daemon.
For the packaged installs, the daemon service is already provided by the
package, so we need the ability to add just the *-create-dir service.
Some might say that the package should create it's run directory...and
it in fact does. However, we are reconfiguring the services to use
/var/run and the package uses /var/lib. Since /var/run is tmpfs mounted,
we need the *-create-dir services for the packages as well.
Change-Id: If26d9d7da04d3d1d97809db0e21e54dd9f8042ca
Install systemd service files into a location that will work on Fedora as well
as on openSUSE. Note, that on Fedora /lib is just a symlink to /usr/lib anyway.
Change-Id: If1dcd0d428d36d587fab501d42b0c8689a972e31
We don't always use os-db-create on an empty database server. This script
should just create, as its name implies. Dropping whole databases is an
extremely important operation and should not be done "just in case".
Change-Id: I1dbdab0fd81ba6abbd87b5bd6fcf213f0fe51455
Add os-svc-restart, a helper script to restart a service. It uses
map-services to figure out the actual service name, as the service name
can differ based on the distro being used and also if installing from
source or packages. Currently, only if the init system is systemd will
a call out to map-services be done. This matches the pattern from
install-packages where map-packages is not used on dpkg systems since
those package names are used as the default in the source install
scripts.
Having a helper script will avoid the proliferation of the following
pattern everywhere:
service foo restart || service foo-by-another-name restart
This also significantly reduces errors in the log due to the above,
which can be misleading.
Depends on Change-Id: Ie7b2dcfa7cabd887d5c212df110d90f9d00a7f65
Change-Id: I15eb2a5db4b0a08e1fb40bda640cd8f224939a92
Without this export os-svc-enable.conf never sees the starting event and
thus never does anything to prevent starting the job.
Change-Id: Ib94c817ce3f2e6c3aada1515beea6e7fb41605af
Recent changes landed to set the environment variable
OS_SVC_ENABLE_CONTROL=1 but the env statement was left out, rendering
the upstart job completely broken as the syntax is now invalid.
Change-Id: I88a7a63304d1fe759067e47e2fab88ee27f91914
Now that we call os-svc-enable in the individual elements we
can drop the call here.
With this commit *most* services will no longer autostart on first boot
and will rely on post-configuration scripts in os-refresh-config
to enable services. This should clean up our first boot logs
quite a bit...
Change-Id: Ied3622a5e7fce4c3ac2fbd9a0e513ae877cafc5d
Split out the code which enables services to start on boot into
a new os-svc-enable script. This script is currently called by
os-svc-deamon so that services automatically start on first boot
(as is the case with os-svc-install)
The motivation here is to decouple service installation and enabling
so that we can eventually avoid starting services on first boot
until after they are configured. Auto starting services on boot like
we currently do is causing lots of log ERROR messages, etc.
Change-Id: I66d4660c169f5918e924eab1876134891c3a24a1
Upstart doesn't have its own service enable/disable mechanism. We are in
need of a facility to enable/disable services without writing to /etc at
runtime to enable a readonly root for image based updates. This command
will be used by later changes to have a generic service enable/disable
mechanism for all init systems.
Change-Id: I931281988aa746f93de190794f68f4c9e628db19
Ghanshyam Mann