Commit Graph

31 Commits

Author SHA1 Message Date
Yoshi Kadokawa 536540781a Add support of image format conversion on importing
Optinally enable conversion of images to raw format when uploading
images to Glance. Only supported when "image_conversion" is enabled in
Glance.

This is an equivalent change to the one in simplestreams:
https://git.launchpad.net/simplestreams/commit/?id=cbc0ac9

Closes-Bug: #1889729
Change-Id: Id15fab364c27242347c520b959065c490e4da6b6
2023-08-30 13:52:21 +09:00
Guilherme Maluf Balzana 4754aca420 Add set_latest_property config to new image
When creating Openstack VMs the user has to specify the image it wants
to use. sstream-mirror-glance adds a date to the image name, so they
always have to recheck which is the current latest image.

This commit adds the `set_latest_property` configuration to include
--set-latest-property to sstream-mirror-glance command line argument.

When --set-latest-property is given sstream-mirror-glance will set the
recently synced image with the `latest=true` property and then remove
the `latest` property from all the os_version/architecture matching
images.

Configure bundles to fetch simplestreams snap from edge channel

Closes-bug: #1933130
Change-Id: Idf78294db7abb8c81d637086e8142782bf1dd36f
Func-Test-PR: https://github.com/openstack-charmers/zaza-openstack-tests/pull/853
2022-09-23 10:31:43 +02:00
Nobuto Murata f8165cebb8 Support --visibility option for simplestreams
With the new option, Glance images can be uploaded with
"visibility=shared" for example instead of "public" so those can be
hidden from usual projects by default.

Closes-Bug: #1975663
Change-Id: I1955f3d2a56654c9a683a2b9d36b33c0f0fd63d4
2022-05-29 13:32:36 +09:00
Yoshi Kadokawa bf23ef75b2 Add support for juju resources
Simplestreams package is now installed via snap. By having the juju
resources for the snap package, it can benefit in offline environment.

Closes-Bug: 1921832
Change-Id: I5f21ca9faff35427281076e142942c3458e71091
2021-10-05 06:04:44 +00:00
Billy Olsen 6733e22006 Add use-internal-endpoints as a new config option
Enable the glance-simplestreams-sync charm to be configured to only
use internal endpoints, removing the requirement that the charm
access APIs via the public network.

This introduces a new option use-internal-endpoints, which is in-line
with other openstack charms which can be configured to use the internal
endpoints rather than public endpoints.

This option will only be valid for Keystone v3 API versions, which
should not be problematic as the v2 API has been deprecated for quite
some time now.

Closes-Bug: #1896438
Change-Id: I6d83a5469c34c2add3c1bc4ac5052be6d51ed343
2021-09-02 11:03:32 -07:00
Dmitrii Shcherbakov d1f8c4ba4c Remove local apache2 install, rework use_swift
The functionality of use_swift where a local apache2 instance was set up
for metadata was deprecated for removal and this change actually removes
this. However, the option itself can still be used as an indicator
whether Swift is intended to be used for simplestreams metadata or not.
If the usage is explicitly requested, the charm but Swift endpoints are
not yet present, the charm will now will set the unit state to
maintenance in addition to local logging that was done previously.

Swift presence checking is now endpoint-based instead of being
service-based (no use for the case where endpoints are not set up while
the service is in the catalog).

The lack of swift presence is also ignored for the purposes of
generating proxy setting context - if the endpoint is not there there is
not point in trying to generate NO_PROXY rules for it.

This change also makes test bundles use ceph as a storage medium for
glance since out of space errors were encountered during testing as the
glance unit was running out of space for image storage.

Closes-Bug: #1942047
Closes-Bug: #1934563
Closes-Bug: #1938069

Change-Id: I6519b1449806ad19ee4545501bc4631c9f8e535f
2021-09-02 17:34:25 +03:00
Dmitrii Shcherbakov 009c8a7b92 Dynamically generate proxy settings for image syncs
sstream-mirror-glance has several endpoints it needs to talk to:

* Image mirrors - typically, public Internet endpoints;
* Keystone - typically, a directly reachable endpoint;
* Glance - typically, a directly reachable endpoint;
* Object store (Swift) - typically, a directly reachable endpoint but
  sometimes it may be deployed externally and added to the region
  catalog in Keystone (in which case it might be accessible via a proxy
  only).

While sstream-mirror-glance does not support specifying proxy settings
for individual directions, since we know all of them based on the
Keystone catalog, a list of endpoints to add to NO_PROXY environment
variable can be generated dynamically.

The complication is that image syncs are periodically done via a cron
job so a juju-run invocation is needed to retrieve relevant proxy
settings from model-config at each invocation of the synchronization
script.

Additionally, the charm is long-lived so there may be some environments
that rely on legacy proxy settings. This change accounts for that and
acts both on juju-prefixed (new) and unprefixed (legacy) proxy settings.

Whether to use proxy settings for connections to the object store API
is controlled by a charm option which the script is made to react to.
Proxy settings are ignored for object store connections by default.

Closes-Bug: #1843486
Change-Id: Ib1fc5d2eebf43d5f98bb8ee405a3799802c8b8dc
2021-07-22 21:14:03 +03:00
Alex Kavanagh 787a9c5ae9 Ensure py3 pkgs installed on series-upgrade
On series-upgrade the g-s-s stopped syncing due python packages required
by the sync file (that is run in the context of the unit, rather than
the charm) not being present.  I believe they may have been removed
during the series-upgrade when unused packages were removed.

This patch ensures that on both upgrade and series upgrade, all the
packages that are needed are installed.

Change-Id: I8dfa9b0111fa3f204364e2c0833595bf8b219a7c
Closes-Bug: #1934306
2021-07-01 15:44:40 +01:00
Liam Young 9c7c4c4736 Update location for CA cert
Look for ca cert using new name `vault_juju_ca_cert.crt` and
fallback to the old name `keystone_juju_ca_cert.crt` if needed.

Change-Id: Ifa4f404a0f8283659448c2b8a3d622768d2c4c8f
Closes-Bug: #1912234
2021-01-22 13:50:49 +00:00
James Page bfddde02ac Ensure all config files written on config-changed
Write all configuration files when charm configuration options
change to ensure that the ssl_ca is written into the identity.yaml
file.

This commit also drops some parts of the unit tests for hooks
which to be honest need reworking as they have a number of
boundary violations.

Change-Id: I5e84ebd23fd487f64b2902327ce3f33739ed9840
Closes-Bug: 1843484
2020-07-23 07:10:18 +01:00
James Page 2dfcecf8d2 Ensure ssl_ca is always installed to system certs
If an encoded SSL CA cert is provided via configuration ensure
it is installed to the system certificate store.

Change-Id: I58f4b89996dc734c4253c2bc2237616434c8143c
Closes-Bug: 1851474
2020-06-18 16:19:57 +01:00
James Page 9698b5ca2a Drop RabbitMQ support
Drop support for sending sync status messages to a RMQ exchange.

This was used by conjure-up which is no longer maintained so
drop the supporting code, metadata and options and remove
RabbitMQ from all test bundles.

Change-Id: I504508a64e50a9d331f6c5d69b292d9ccc20da3d
Func-Test-PR: https://github.com/openstack-charmers/zaza-openstack-tests/pull/321
2020-06-18 08:50:34 +01:00
James Page fd3d2b7610 Switch to using snap for simplestreams tools
Switch over to using the new simplestreams snap instead of using
the largely unmaintained packaging in distro (and various PPA's).

This drops direct integration with the simplestreams codebase
in preference to just calling the sstream-mirror-glance command
with the correct parameters.

This commit includes refactoring of 'custom_properties' handling
which was actually broken - there was no loading of the yaml
formatted list, which was probably overkill, so the option now
takes a space separated list of key=value pairs, for example:

 custom_properties="hw_firmware_type=uefi hw_vif_multiqueue_enabled=true"

Fix version comparison in script wrapper.

Drop Trusty support - snaps on Trusty are awkward requiring new
kernel versions and trusty support was only retained for upgrade
purposes anyway.

Drop unsupported Xenial OpenStack versions.

Fixup CA cert handling to use any charm installed CA cert (including
that provided via the certificates relation) and install cert
to snap compatible location for simplestreams to use.

Add basic action to perform image sync on demand an refactor
the glance simplestreams sync wrapper to work within a hook
context to support the action.

Disable automatic scheduling of image syncs by default as this
tends to be racey during deployment resulting in images being
synced to glance unit local storage.

Add bionic-ussuri bundle and make it the default smoke test.

Deprecate source and key options - no longer required for
deployment with snap.

Change-Id: I730df6b7f5955ddfeea5b8de15490ac083823f5a
Func-Test-PR: https://github.com/openstack-charmers/zaza-openstack-tests/pull/321
2020-06-18 08:50:02 +01:00
Sérgio Manso b9f7f7402c Focal enablement
Added support for charm to install py3 packages

Fixed gss script to run on both py2 and py3

Added the ability for the script wrapper to get the distro version
so the gss script can be launched with the right python binary.

focal-ussuri test bundle added and smoke/dev tests changed to focal-ussuri

charmhelpers sync

-----
LP related bug: #1853456

Change-Id: I161d8a0fa9787f0d5409f7bf386c22d98aae0a64
2020-04-24 22:26:15 +01:00
Jacek Nykis 0bf193dc85 Add custom_properties config option
This change add "custom_properties" configuration option which
allows custom image properties to be added to glance.

Change-Id: I32d97c9f8026ac1aa16a2bc6cd134f26860c59b2
Closes-Bug: #1783125
2019-12-02 10:34:02 -08:00
Ryan Beisner e69a562494
Standardize auxiliary file location across os-charms
Change-Id: Ic06c4fcdd87d350a0e41cb9de62bd7887aae7168
Closes-Bug: #1843826
2019-09-12 15:27:36 -05:00
Joe Guo 660ce5b021 Migrate from Amulet to Zaza
This patch migrate tests from Amulet to Zaza, and Python3 only.

SSL is enabled for all bundles with charm-vault, except trusty, which is
not supported by vault.

func-test-pr: https://github.com/openstack-charmers/zaza-openstack-tests/pull/55

Partial-Bug: 1828424

Change-Id: I4dacf33d3ab4dedbbe62a36b1b2ce5606e51d234
Signed-off-by: Joe Guo <guoqiao@gmail.com>
2019-09-12 22:41:32 +12:00
David Ames 3482fb7707 Allow local hosting of product-streams
When use_swift=False in addition to syncing images into glance, also
host the product streams metadata using apache.

Change-Id: I6aad2fee3670f575014d0845f3bd7aae6fa06785
Closes-Bug: #1840830
2019-08-20 15:38:39 -07:00
Zuul 87c159c473 Merge "Migrate charm to Python3" 2019-02-12 12:45:37 +00:00
Ryan Beisner d597eaedbb
Update pre-install hooks to fail on error
The pre-install operations may fail, yet that failure is not
elevated to the user. This masks the failure and makes early
package install issues difficult to troubleshoot.

If the basic pre-install script fails, the charm should not
proceed to later hooks as the requirements may not be met.

Hashbangs for bash should specify -e (errexit) on all of the
pre-install bash scripts.

Change-Id: I61301f3ed953f685bca3c4060907ccda8cec6f49
Closes-bug: #1815243
Partial-bug: #1815231
2019-02-08 16:07:37 -06:00
Alex Kavanagh 6aa32f2d67 Migrate charm to Python3
Change-Id: Ibb46dfa766b12aedea90f823d3964c569bdfcb01
2019-02-04 19:18:02 +00:00
Zuul ac851be0f1 Merge "Add ssl_ca option to enable to gss" 2019-01-11 11:01:35 +00:00
Alex Kavanagh ac1d2b5dda Add ssl_ca option to enable to gss
This patch enables SSL to be used with glance-simplestreams-sync.
The ssl_ca option allows a base64 encoded PEM CA certificate to be
used with g-s-s such that the keystone and glance HTTPS sessions are
verified using that certificate.

A new basic_deployment_ssl.py is introduced that just verifies that the
gss charm can get gss to perform a sync; this verifies that gss can
communicate with https versions of keystone and glance.

Note that the simplestreams package also requires a change for SSL to
function properly.  As simplestreams doesn't seem to use PyPi, the
version from the git master will need to be used.

Change-Id: Idcdcb2c933a92a558e729aeb718b58d4077621a7
Closes-Bug: #1802407
2019-01-11 09:37:56 +00:00
David Ames e9d7e291b7 Series Upgrade
Implement the series-upgrade feature allowing to move between Ubuntu
series.

Change-Id: I8a9a3e61613322a692389b8718f29a91aba65267
2018-10-23 22:24:27 +00:00
Ryan Beisner e942583cc3 Update copyright date 2018-05-08 12:37:36 -05:00
Ryan Beisner 3098be44c7 Relicense to Apache2 2018-05-08 12:30:29 -05:00
Ryan Beisner 9952463c67 Update hooks hashbang to be consistent 2018-04-10 19:08:35 +00:00
Ryan Beisner 74e4cd4ae4 Clean up pre-existing lint 2018-04-10 13:49:47 +00:00
Ryan Beisner 79c2a7c3c6 Ensure gss config file permissions only on existing files 2018-04-09 17:15:54 +00:00
Ryan Beisner 013d008005 Ensure gss config file permissions 2018-04-09 14:50:34 +00:00
James Page 6cc0f9dc88 Initial baseline from bzr 2017-04-10 09:00:27 +01:00