Commit Graph

169 Commits

Author SHA1 Message Date
Zuul cf202a64f5 Merge "Stop hard-coding config file for db sync" 2024-03-08 13:37:14 +00:00
Takashi Kajinami ea6c158a75 Stop hard-coding config file for db sync
The cloudkitty-storage-init command by default loads
/etc/cloudkitty/cloudkitty.conf. Removing the override allows us to use
additional paths such as cloudkitty.conf.d in the future.

Change-Id: I3109005c76a1ffd8f23b3aebd92861a0a4fb419a
2024-03-05 12:27:20 +09:00
Takashi Kajinami f624709cfc healthcheck: Expose ignore_proxied_requests parameter
Depends-on: https://review.opendev.org/909807
Change-Id: I8c754cd823a501924567c68457be6e16b884ce85
2024-02-26 01:29:42 +09:00
Takashi Kajinami 4bf6dd0f4b healthcheck: Expose allowed_source_ranges
... which was added to puppet-oslo recently.

Depends-on: https://review.opendev.org/905557
Change-Id: Ia9b0534f4e4a6b0d2325ccef318f35ef3789eb08
2024-01-17 02:52:55 +09:00
Takashi Kajinami 6394817cb1 Remove usage of deleted manifest_dir
Recent update in rspec-puppet removed some of the config interfaces for
old puppet versions[1]. This drops usage of these interfaces to resolve
the following error in unit tests.

```
An error occurred while loading ./spec/unit/provider/manila_spec.rb.
Failure/Error: c.manifest_dir = File.join(fixture_path, 'manifests')

NoMethodError:
  undefined method `manifest_dir='...
```

[1] 316d95923c

Change-Id: I891b85fdba76186a2b97683526ac05d483dc6ff9
2023-10-11 11:57:46 +09:00
Takashi Kajinami e17c077aa3 RabbitMQ: Add support for quorum queue options
Depends-on: https://review.opendev.org/894866
Change-Id: Ia11d14110a0a49056b4e582c20d360cf2953c28f
2023-09-15 08:42:36 +09:00
Takashi Kajinami dc2ab9e5a2 Add missing fetcher_keystone options
We've seen workers of cloudkitty-processor continuously crash with
the following traceback logged in syslog

Traceback (most recent call last):
  File "/usr/lib64/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib64/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.9/site-packages/cotyledon/_utils.py", line 63, in _bootstrap_process
    target(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/cotyledon/_service.py", line 161, in create_and_wait
    sw = cls(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/cotyledon/_service.py", line 175, in __init__
    self.service = config.service(worker_id, *args, **kwargs)
  File "/usr/lib/python3.9/site-packages/cloudkitty/orchestrator.py", line 552, in __init__
    self.fetcher = driver.DriverManager(
  File "/usr/lib/python3.9/site-packages/stevedore/driver.py", line 54, in __init__
    super(DriverManager, self).__init__(
  File "/usr/lib/python3.9/site-packages/stevedore/named.py", line 78, in __init__
    extensions = self._load_plugins(invoke_on_load,
  File "/usr/lib/python3.9/site-packages/stevedore/extension.py", line 218, in _load_plugins
    self._on_load_failure_callback(self, ep, err)
  File "/usr/lib/python3.9/site-packages/stevedore/extension.py", line 206, in _load_plugins
    ext = self._load_one_plugin(ep,
  File "/usr/lib/python3.9/site-packages/stevedore/named.py", line 156, in _load_one_plugin
    return super(NamedExtensionManager, self)._load_one_plugin(
  File "/usr/lib/python3.9/site-packages/stevedore/extension.py", line 242, in _load_one_plugin
    obj = plugin(*invoke_args, **invoke_kwds)
  File "/usr/lib/python3.9/site-packages/cloudkitty/fetcher/keystone.py", line 71, in __init__
    auth_url=self.auth.auth_url)
AttributeError: 'NoneType' object has no attribute 'auth_url'

This adds the fetcher_keystone options to avoid that error.

Change-Id: I27530c279afd21f8b32ee6aa44ec194ac019887c
2023-08-25 15:02:46 +09:00
Takashi Kajinami 4ae2f42a7a fetcher_keystone: Fix ignored auth_type
The class parameter has been ignored and the given value was not put
in the config file properly. This fixes the problem.

Change-Id: I764a98553b766e64fad71d418f4158a47e413282
2023-08-25 15:02:46 +09:00
Takashi Kajinami 8d52c71e5e Configure [orchestration] option in acceptance tests
... so that we can test the behavior of coordination backend
configuration.

Change-Id: I45b62e7329dd610dd9e6ab84628b247657d88bf9
2023-08-25 13:15:09 +09:00
Takashi Kajinami 7cadccb30e Modernize manifest in acceptance tests
This updates the manifest used in acceptance tests according to
the recent implementations in p-o-i manifests.

Change-Id: I9fa8ecf4bc799a086388372a5edd0419053e81ff
2023-08-23 00:41:41 +09:00
Takashi Kajinami 439373e1c0 Enable memcached in acceptance tests
Now memcached is used as cache backend for some services and authtoken
middleware.

Change-Id: I0195d54a4801926699652a2b1c2c453c854bfdb9
2023-08-22 23:40:02 +09:00
Takashi Kajinami 9869a67a9d Clean up deprecated parameters
This removes a few parameters deprecated in favor of the new
cloudkitty::fetcher::keystone class.

[1] 4e6806f54f

Change-Id: I0e363d7269bfea31464d8d74b07828bebd792c3a
2023-07-28 16:39:59 +09:00
Zuul 9174927541 Merge "Remove deprecated cloudkitty::processor::max_workers" 2023-06-26 14:51:26 +00:00
Takashi Kajinami 6c21dd1dc2 Remove deprecated cloudkitty::processor::max_workers
Change-Id: I412a366755a4aa7933be7feeb040209efc735b44
2023-06-26 04:17:52 +00:00
Tobias Urdin 2abda4f574 Add per module policy service refresh
Updating the policies for this project should only
refresh the services that reads it.

Change-Id: I7eda28c684af2def0d4acacfa5ba453ef02b42a3
2023-06-26 00:03:50 +02:00
Takashi Kajinami 70d66f8f55 [orchestrator] coordination_url should be hidden
The coordination_url option can sometimes contain secrets.

For example when redis coordination backend is used and authentication
is enabled in redis, the plain redis password is put as an URL element.

[orchestrator]
coordination_url=redis://:password@127.0.0.1:6379

Closes-Bug: #2012246
Change-Id: Iba1e7715b290ee4c104f11221e250b23936b12dc
2023-03-20 21:11:32 +09:00
Takashi Kajinami 25e2da3b42 Replace legacy facts and use fact hash
... because the latest lint no longer allows usage of legacy facts and
top scope fact.

Change-Id: I8986bcb5fdd5d1c28aa0e706267f689b894f7167
2023-03-01 16:17:18 +09:00
Takashi Kajinami b5af91b7ff Expose policy_default_rule
The option has been managed by the underlying puppet-oslo module but
has not been configurable. This introduces the parameter to customize
the option.

Change-Id: Ia4e917d879a175c1919131bfa0bee9d679fe1a76
2023-01-23 14:28:45 +09:00
Takashi Kajinami d89093fa76 Add Apache WSGI logging parameters for pipe/syslog
Add parameters for advanced logging configurations in Apache to
support piped logging and support for syslog (via mod_syslog
available in Apache >= 2.5.0)

Co-Authored-By: Andy Botting <andy@andybotting.com>
Change-Id: Iee4cf4b3e20d16624b89f6b004cc857ed5288510
2022-08-26 17:03:11 +09:00
Zuul cdcc255735 Merge "Add support for Elasticsearch storage backend" 2022-08-18 22:51:33 +00:00
Takashi Kajinami 2072b01aac Add support for Elasticsearch storage backend
Change-Id: I9b3b74f3c32a43ae982f0c429c95819dc387bf80
2022-08-17 13:59:17 +09:00
Takashi Kajinami 161894b0d5 Ubuntu/Debian: Enable deployment in acceptance tests
... because cloudkitty packages are already available in these
operating systems.

Change-Id: Ieae898cd718377d7ad0494cd5051651ed90dcdd2
2022-08-17 13:48:16 +09:00
Zuul 021f77ca19 Merge "Adapt to new type validation in puppetlabs-apache" 2022-08-08 17:51:35 +00:00
Takashi Kajinami 355805aba1 Adapt to new type validation in puppetlabs-apache
The puppetlabs-apache module is enforcing more strict data type
validation[1].

This change updates the default values to adapt to that change.

[1] f41251e336

Closes-Bug: #1983300
Depends-on: https://review.opendev.org/851652
Change-Id: Idd4918618831ffab98db91fa90ae493775373bc9
2022-08-02 07:48:35 +00:00
Zuul 92e60c353a Merge "Expose headers option of apache::vhost" 2022-07-30 14:13:04 +00:00
Takashi Kajinami 1aab15afc9 Revert "CentOS 9: Disable unit tests dependent on puppet-postgresql"
This reverts commit 62e136b037.

Reason for revert:
puppet-postgresql 8.1.0 was released and now the module supports RHEL 9
(and CentOS 9 effectively).

Note:
This change adds the service_provider fact in test fact data because
it is required by puppet-postgresql.

Depends-on: https://review.opendev.org/850705
Change-Id: Ibe9ea9a9442ec69be226e150b424401ff733128b
2022-07-24 00:37:09 +09:00
Zuul 140e0f5d1f Merge "Add acceptance tests for config management resources" 2022-07-15 19:44:39 +00:00
Zuul a20aca6eca Merge "Fix wrong test description" 2022-07-15 03:02:14 +00:00
Takashi Kajinami 5aaacb2caf Fix wrong test description
The value with a white space should be accepted instead of being
rejected. This change fixes the wrong test case description and makes
sure the description explains what is actually tested.

Change-Id: I07e423693b88b71d98f596dea18bcc0884092955
2022-07-11 23:28:19 +09:00
Takashi Kajinami be052f825d Add acceptance tests for config management resources
Change-Id: Ia9a5f6927866b4bd60b347429bed206a53f8aa38
2022-07-10 12:43:28 +09:00
Takashi Kajinami d4db053e91 Expose headers option of apache::vhost
The headers option in apache::vhost is required in some case, for
example when adding the X-XSS-Protection header. This change allows
customizing the option for the api vhost.

This change also adds support for request_headers so that both request
headers and response headers can customized.

Change-Id: I4ff7ea57f9c1b29b2209506969497b773cef02ec
2022-06-30 02:02:31 +00:00
Takashi Kajinami 71ad6471e9 Remove deprecated cloudkitty::ui
This class was deprecated during Yoga cycle[1].

[1] 0f89c9ab8f

Change-Id: I703934b1b5dcecb01cef100a512a4c30fbf63522
2022-06-22 00:03:37 +09:00
Zuul 52f40f4a2e Merge "Fix typos in parameter descriptions" 2022-06-21 13:16:47 +00:00
Takashi Kajinami 15a5bfb27d Fix unit test failure in Ruby 3
This fixes the following error raised in unit tests when the tests
run on Ruby 3 (in CentOS Stream 9).

NameError:
  uninitialized constant TRUE

Change-Id: Ib62b8e921b350971732f85307c5bdcb1e8fe12f5
2022-06-20 20:19:05 +09:00
Rajesh Tailor 17f8ee1b31 Fix typos in parameter descriptions
Change-Id: I865f8778b4efbb84f295ad22b9f9a16828774e44
2022-06-20 16:04:06 +05:30
Takashi Kajinami 0ddc83eb3e apache+mod_wsgi: Disable SSL by default
During the previous cycle, a warning message was added to inform users
of this change.

Now the default value is updated so that SSL is disabled by default.

Change-Id: I705dabab19974c2d9323e49051c8952e6d537820
2022-05-06 21:09:45 +09:00
Takashi Kajinami 0b08499e20 Globally support system scope credentials
After spending huge effort to understand the exact requirements to
enforce SRBAC, we learned it's very difficult to find the required
scope in each credential. This requires understanding implementation of
client-side as well as server-side, and requirement might be different
according to the deployment architecture or features used.

Instead of implementing support based on the actual implementation,
this introduces support for system scope credentials to all places
where keystone user credential is defined, and make all credential
configurations consistent.

Change-Id: I50c029b07a30c201b4d9bd4821265d4bf465ba9f
2022-03-04 02:11:11 +09:00
Zuul 38fa7028bd Merge "Avoid testing details of oslo::messaging(::*)" 2022-02-18 08:56:14 +00:00
Takashi Kajinami 62e136b037 CentOS 9: Disable unit tests dependent on puppet-postgresql
The puppt-postgresql module does not support CentOS 9 yet and requires
some version parameters to be run on CentOS 9. This change disables
unit tests requiring that module, until the module supports CentOS 9.

Change-Id: If26b9d1a31028bb56106a2cce0faadcc7d38757f
2022-02-16 00:18:48 +09:00
Zuul af68446b62 Merge "Do not define service resource when service management is disabled" 2022-02-15 09:36:59 +00:00
Zuul 61e2f535ad Merge "Add support for MultiStrOpt" 2022-02-15 09:16:59 +00:00
Takashi Kajinami bd9098ec87 Support more [orchestrator] parameters
This change introduces a separate new class to manage [orchestrator]
parameters. The max_workers parameter in the processor class is
deprecated in favor of that new class.

Change-Id: Icd30923802064aec6a3ae9716b2ae30665998aae
2022-02-13 21:20:15 +09:00
Takashi Kajinami b13a35dd3a Add support for MultiStrOpt
This replaces the provider implementation of cloudkitty_config type so
that MultiStrOpt, which is used by several options like
 - oslo_messaging_notifications/driver
 - oslo_policy/policy_dirs
are handled correctly.

Change-Id: Iefdfa5b83cb1f5ff691cc38e0f691babac9bdfb0
2022-02-13 00:22:44 +09:00
Takashi Kajinami e9f6080596 Do not define service resource when service management is disabled
Change-Id: Ib43b45c7232ef5ccbb91949a7b6d1a1aa3da1d28
2022-02-10 13:44:38 +09:00
Takashi Kajinami cb76d9c50a Avoid testing details of oslo::messaging(::*)
... so that any change in puppet-oslo would not directly break unit
tests.

Change-Id: I8bfd797ac6f9f1a930533112182587b53bfe13f9
2022-02-04 22:58:07 +09:00
Takashi Kajinami d9d0ab9923 Support more [fetcher_keystone] parameters
Change-Id: I7f60d26ab9554867a30f53df7520e6cbfc18fdd1
2022-01-25 21:35:57 +09:00
Takashi Kajinami 6c3336b5d9 Accept system scope credentials for Keystone API request
This change is the first step to support secure RBAC and allows usage
of system scope credentials for Keystone API request.

This change covers the following two items.
 - assignment of system scope roles to system user
 - credential parameters for authtoken middleware

Depends-on: https://review.opendev.org/804325
Change-Id: Ia12867af22002b6a6d1e40e9ded60c92d493023b
2022-01-24 15:01:26 +09:00
Takashi Kajinami 847e4f7e49 Do not test authtoken parameters directly
The authtoken parameters are not managed directly but managed by
the keystone::resource::authtoken class. Thus we should avoid testing
parameters directly otherwise any change in the resource type can
cause test failures.

Change-Id: Ic6864a4713ad5e9c2bc865d201e5e133866d9e60
2022-01-24 11:36:14 +09:00
Takashi Kajinami 1b7715cafe Load libraries in a single place
This change refactors how the dependent libraries are loaded during
unit tests, and load the libraries in the base spec_helper to avoid
duplicate and redundant implementations.

Change-Id: I4a33c13c50a4686909b082ea0093e5e848fd731f
2021-12-27 10:32:02 +09:00
Takashi Kajinami 3976a25fbb Clean up direct dependencies on puppetlabs-apache
This change removes direct reference to some classes in
puppetlabs-apache. Details are explained below.

- The api class doesn't need access to anything defined in
  apache::params

- The following classes are included by the openstacklib::wsgi::apache
  resource type, and current inclusions are just redundant.

Change-Id: I110615c7cd4759e17a96c45d58402aba9216c05f
2021-12-08 22:54:02 +09:00