Commit Graph

56 Commits

Author SHA1 Message Date
Pete Vander Giessen 322514211b snap-config-keys is now a map
It maps the name of the config values that we use in our templates to
the name of the key in the snap config. This allows us to stick a
bunch of stuff in the questions namespace in the snap config, and to
use dashes, with minimal changes to our templates.

Drop Python 2 support, to fix tests.

Change-Id: I48b86b5e557e30f81e9cc415e7fa3a9133aa9f39
2019-10-07 12:58:12 -04:00
Pete Vander Giessen f612dd2e61 Added check for database readiness before starting daemons
Fixes the issue where our logs get spammed with errors before we have
things setup.

Change-Id: I7e6b90832b7d6fb601c64b81c6f4e6afc5e956b6
2019-09-30 17:47:37 -04:00
Zuul 38686c176c Merge "fix error url" 2019-07-15 09:20:06 +00:00
Pete Vander Giessen e26e49af35 Broke up execution into "setup" and "launch" commands.
We present these as seperate invocations of the script, rather than
automatically running them one after the other. This allows us to run
the setup script once for multiple daemons, then run launch steps
individually for the daemons.

Change-Id: Ia223f6bd6c1d3b544831652d4a076c4bee13ce43
2019-07-12 20:27:11 +00:00
Pete Vander Giessen ab7e4ce124 Added return from snapctl config to snap_env.
This allows us to access values from the snap config in our templates,
just like we access the snap environment variables.

Change-Id: Id9d499482ec7c2f1b63cd854b4a5cee8e519a712
2019-07-12 18:58:17 +00:00
OpenDev Sysadmins 1aeb0dbc6c OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:50:19 +00:00
lvxianguo c472632a22 fix error url
Change-Id: Id75e3dd648951279be3e85f4654a1e77c1744745
2018-08-16 09:31:58 +08:00
James Page b2b84c96a9 Add automatic setup of REQUESTS_CA_BUNDLE
Add automatic addition of REQUESTS_CA_BUNDLE to the execution
environment if $SNAP_COMMON/etc/ssl/certs/ca-certificates.crt
is detected, allowing the certs from the core snap to be
overridden on a per snap basis.

This supports use of privately signed certificates in snap
based deployments.

Change-Id: Iac789d25b9a5d0e71ba0afdc8462ff0c2344ef7b
Related-Bug: 1724012
2017-10-26 09:39:42 +01:00
Corey Bryant f9ae8f4813 Add unit tests for SnapUtils
Change-Id: I0e0bcf1edf37aa0214bc28ad5602cb830477a34d
2017-07-15 16:18:51 -04:00
Corey Bryant 995054e370 Add unit tests for OpenStackSnap.setup()
Change-Id: I35856ae3dfc2b5e8dd9e81daa96b749ed08c54cb
2017-07-15 15:17:32 -04:00
Corey Bryant 1b6970a551 Update unit tests for strict mode and path changes
This mostly just updates test paths to align with new locations based
on the changes that enabled overriding default configs. The test
snap-openstack.yaml is also updated with the the current one from
the keystone snap, dropping run-as which is no longer used since the
move to strict confinement.

Change-Id: I9ffc15e561b36ba87585b3615f34fccf8ef2d753
2017-07-14 10:43:00 -04:00
Corey Bryant 4544448c83 Drop unreachable/unnecessary code
Drop code that is no longer used or no longer required since the move
back to strictly confined snaps.

* Everything is run as root in strict snaps so there's no need
  to drop privileges, no need to chown dirs/templates to anything
  but root, and the default dir/file modes should suffice.
* copyfiles: we wanted to move away from this and finally did, so
  so it's no longer used.
* rchown and rchmod: these are no longer used.

chmod is still used by nova-hypervisor, so let's keep it. I'm not
sure that we still need chown, but I've left that in case we happen
to.

Change-Id: I45be76121b73f0c354a9bcc4014af534455cb533
2017-07-14 13:47:28 +00:00
Corey Bryant 85ecf808ce Allow multiple OS configs and flexible overrides
The config override patch was a bit too aggressive in not allowing
multiple OpenStack config files/dirs to be specified. Some projects
need to specify more than one, so this re-enables the ability to
specify multiple OpenStack config files/dirs.

Additionally this makes the override of OpenStack config files more
flexible in that individual files can be overridden while others are
not.

Change-Id: Id512a96e3a7b408b519f29f860d522a584f1e5d9
2017-06-03 06:59:14 -04:00
Corey Bryant c1b51aa5e0 Enable override of config files
Enables the ability for default config files to be overriden for
each entry point type (simple, uwsgi, and nginx).

Additionally, align unit tests with keystone's snap-openstack.yaml,
as features tend to land in the keystone snap first.

Change-Id: I9fe54f1dae2b94d2f78783f38ced48ebe7c66f0f
2017-05-25 10:16:10 -04:00
James Page 8ae4426363 Refactor template rendering, use entry point templates
Refactor the template rending code into a static function.

Ensure that templates for the entry point are rendered during
execute, not the global templates.

Change-Id: Idddd6361deabb8a2c21caa77a42a55cd8b8d2cbc
2017-05-19 09:13:53 +01:00
Corey Bryant 28ee5bce72 Fixup template rendering in execute()
Change-Id: Icd072c99123f8956819302ddaefadb08766ad41a
2017-05-19 01:31:33 +00:00
James Page acdfc97bda Make lock path snap specific
As snaps only write out to snap specific locations, create locks
scoped to the individual snap, not globally for all snaps.

Change-Id: Ife03fb66eb97b89040495ffa51e867adca10a3a1
2017-05-18 14:31:16 +01:00
James Page ba1ecfee7b Always execute copyfiles and template directives
Targets for copyfiles and template directives should be considered
as owned by the snap, and as a result should always be executed,
irrespective of whether the target file exists or not.

This ensures that any use of {{ snap }} is current, and that changes
to templates in the snap are reflected on refresh.

Execute copyfiles prior to template rendering to ensure templated
files overwrite any files copied as a copyfiles target.

Change-Id: I97bf668cc86099227670a00defd4ee3c36b58ed7
2017-05-18 13:52:56 +01:00
Corey Bryant 44b532041f Enable dynamic creation of uwsgi pyargv option
Specifying hard-coded config files via the uwsgi pyargv option
caused failures when a file didn't exist.

This patch enables dynamic creation of the pyargv option. This
value is pushed onto the context for templated file generation
which is now supported as a subelement of an entry point as
well as a top level global option.

Change-Id: I1d0fae42b2e43fe8808fda3de83e122502233a4c
2017-05-18 13:52:24 +01:00
Corey Bryant ed986bac6d Add support for dropping privileges
Add support for running commands, defined by a snap, as a specific
user/group.

Additionally, file permissions and ownership of setup files can
be adjusted to limit access from other users.

Change-Id: I8563abce55b2b20936eb4e1d55a9016b97e8f6e0
2017-05-18 13:51:43 +01:00
Corey Bryant 920715bd4f Default logging level to WARNING
Change-Id: I8849ff981566fa483d10a03ddb12017cf190f971
2017-05-04 10:27:14 -04:00
Corey Bryant d4dad15941 Don't render template files if they already exist
This enables overriding of snap template files by users or
deployment tools.

Change-Id: Ia130f6e7af947a77e415be5847df54142b5f6781
2017-05-01 14:54:59 +00:00
Corey Bryant 8b1f6581fa Drop support for install setup key
By dropping the install setup key, snaps will decide for themselves
the full path where setup dirs, templates, and copyfiles are created.

Change-Id: I5347186cc586b54dfb11bf1884a0d804d5c1303f
2017-04-21 14:49:03 +00:00
Corey Bryant 30ae5eb8ba Add home to uwsgi default args
Change-Id: I34d65ab795f94f48f97de17b948c110ed0cb425e
2017-04-21 01:07:07 +00:00
Corey Bryant 034efe40ef Allow specifying snap env vars on binary entry points
Change-Id: I6aa6d80a81729277c99df8470e7acf04f5c906c6
2017-04-21 00:49:16 +00:00
Corey Bryant 39210c7412 Use full paths for nginx and uwsgi binaries
Change-Id: I2db1ef030d45d4f86520f8527eb0e4086f46b534
2017-04-21 00:47:32 +00:00
Corey Bryant 110b773d98 Move common utility functions to Utils class
Common utility functions that were defined in base.py are moved to their
own Utils class. Additionally this patch adds some checks to ensure keys
exist in setup() before attempting to access them.

Change-Id: Ib940eefce140e3552f41ff0e32123ae90fe81fe4
2017-04-06 17:34:48 +00:00
Corey Bryant 6898cb6c95 Add nginx entrypoint support
Change-Id: Id1f11ea3ba3984407a32b1fee83d83dc59d1ad04
2017-04-05 18:58:26 +00:00
Corey Bryant 933f404664 Add install setup support and drop symlinks
Add support for an install setup key that is set in snap-openstack.yaml
and defines whether the snap should be installed as a 'classic' or
'strict' snap. This determines the root directory where setup dirs,
templates, and copyfiles are created. If install is set to classic
the root directory will be / and if install is set to strict the root
directory will be $SNAP_COMMON.

Additionally the symlinks setup key is dropped. Classic snaps will
install files into / as defined by the install key.

Change-Id: I6750957654e75169108e6e866ab0d43e588cbd12
2017-04-04 12:08:52 +00:00
Corey Bryant 3d8525f459 Add locking to enable single process code execution
The code in OpenStackSnap.setup() creates users, files,
directories, symlinks, etc. Locking support is added to ensure
that setup() code is only executed by a single process at a time.

Change-Id: Ia7ea6251d68284f5ea6e726e474f65d393b296ea
2017-03-22 16:40:21 +00:00
Corey Bryant 144b820f53 Add support for copyfiles
Change-Id: If270801252d95d0a7fa1ece301516ebec216702c
2017-03-15 22:15:39 -04:00
Corey Bryant 7cd5ca9cbd Add support for symlinks
Change-Id: Ic37f8e00a53642501aa9c7ae108e819d7d8cf3c0
2017-03-15 21:58:05 -04:00
Corey Bryant ab4d973e47 Add support for uwsgi log file
Change-Id: I6fcd9773837e75987b6df2cb5d5a0c093085a774
2017-01-03 13:18:45 -05:00
James Page 29972e5066 Drop missing configuration file log to debug
Missing configuration files are a 'normal' thing to expect, so
don't need to be logged as warning messages.

Change-Id: I2413969d48c3785db497549465564a4d493a6b26
2017-01-03 10:42:25 +00:00
James Page 1c552167ee Add support for uwsgi based applications 2016-11-21 14:50:46 +00:00
James Page 8e9d2c5aa2 Tidy test yaml file 2016-11-21 12:33:36 +00:00
James Page 9ac2e821d4 Give test class a better name 2016-11-21 12:31:04 +00:00
James Page 04fe473a3a Add some basic unit tests 2016-11-21 12:29:29 +00:00
James Page cd2ba26af6 Update HACKING format 2016-11-10 12:43:45 +00:00
James Page c734727bf4 Tidy lint 2016-11-10 12:41:41 +00:00
James Page b21aeccf90 Tweak README 2016-11-10 12:18:22 +00:00
James Page 3bd58c94c7 bash->yaml 2016-11-10 12:17:06 +00:00
James Page ae671e29ce Update code blocks in README 2016-11-10 12:16:03 +00:00
James Page ea489e99da Switch to execing wrapper binary 2016-11-10 11:06:30 +00:00
James Page 5e330b9b63 Tune down default log level 2016-11-10 10:52:19 +00:00
James Page e6b43e1f07 Add log for cmd 2016-11-10 10:34:12 +00:00
James Page 668149fa45 Pass correct part of argv to wrapped command 2016-11-10 10:33:14 +00:00
James Page 4bd46300a3 Fixup directory creation process 2016-11-10 10:14:16 +00:00
James Page 8f090fa8ad Add more logging 2016-11-10 10:07:06 +00:00
James Page 85328c6aa1 Fixup jinja2 loader configuration 2016-11-10 09:58:20 +00:00