Commit Graph

1150 Commits

Author SHA1 Message Date
Zuul 42a0e28288 Merge "Allow more options to limit number of resources" 2024-03-08 05:33:48 +00:00
Zuul b674962603 Merge "common: Replace deprecated importlib API (2/2)" 2024-01-29 07:32:45 +00:00
Stephen Finucane d48a6a2170 common: Replace deprecated importlib API (2/2)
Resolve the following deprecation warning:

  DeprecationWarning: the load_module() method is deprecated and slated
  for removal in Python 3.12; use exec_module() instead

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I403289d5df2143872503fc246a993cb19ebba68e
2024-01-29 01:06:14 +09:00
lujiefsi 7d86fc6d84 Allow more options to limit number of resources
This commit adds the configuration options related to resource limits
in the Heat project. The `max_software_configs_per_tenant`,
`max_software_deployments_per_tenant`, and `max_snapshots_per_stack`
options have been added to control the maximum limits for software
configs, software deployments, stack snapshots.

Story: 2011006
Task: 49401
Change-Id: If33a1c6f3eb9e93f586931bc5c05104439c92bf9
2024-01-26 22:22:11 +00:00
David Vallee Delisle 0f7ce2cfa6 Adding oslo.config entry points for yaql and cache opts
To assist with automated configuration validation, we need entry points
for oslo.config.opts for yaql and cache options.

Change-Id: I228282d2c05f6583fe972470b326e7182f635b39
2024-01-09 10:39:46 +09:00
Zuul fc94b3e49a Merge "common: Replace deprecated importlib APIs (1/2)" 2023-12-15 05:56:38 +00:00
Zuul 0967d76321 Merge "Drop redundant code to initialize RequestContext" 2023-12-15 05:38:21 +00:00
Zuul b207d54612 Merge "Enable new defaults and scope checks by default" 2023-12-04 09:13:22 +00:00
Takashi Kajinami d131a1ada2 Remove deprecated [DEFAULT] onready
This option was deprecated a long ago[1] and has had no effect for
multiple releases.

[1] 46d6b9306c

Change-Id: I0971f67b43292f8794aa3329b2e7dccc67ec3f6f
2023-11-21 12:27:25 +09:00
Takashi Kajinami dabf3a6d38 Drop redundant code to initialize RequestContext
The underlying oslo.context implements the logic to look up request id
from request environment since 2.6.0 [1].

Also, some of the init arguments of RequestContext class are already
implemented by the base class in oslo.context. So we don't have to
implement these additionally.

[1] 740b81784be72fff95957e8491863ef9e7f7db1a

Change-Id: Ic1ab055a8780fd64b67cf92e86d5b8ef10047bbf
2023-10-26 18:55:22 +09:00
Takashi Kajinami 6d13e93a37 Enable new defaults and scope checks by default
Based on the agreed steps to implement the SRBAC community goal, this
enables the new policy defaults and scope checks by default.

Change-Id: I315893150549d1174c3270c37c031e6a519f9a28
2023-10-24 13:18:40 +09:00
Stephen Finucane 559e840053 common: Replace deprecated importlib APIs (1/2)
Resolve the following deprecation warning:

   DeprecationWarning: find_module() is deprecated and slated for
   removal in Python 3.12; use find_spec() instead

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I88005c694b94d249bce53f46df67b51c7e01adae
2023-10-16 17:24:23 +09:00
Stephen Finucane bb6b5ed5c7 db: Migrate to enginefacade pattern
This also gets us away from autocommit, i.e. the last outstanding item
in our preparation for SQLAlchemy 2.x.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I95c5e5d6b33e73a1df2bf036df44aa2e06cc9640
2023-10-12 15:46:21 +00:00
Zuul 346f1d7611 Merge "Isolate project scope and system scope" 2023-09-15 09:35:43 +00:00
Zuul 17ed569181 Merge "Skip loading auth plugin in case the property is set" 2023-09-05 06:43:14 +00:00
Takashi Kajinami 7197bd028b Skip loading auth plugin in case the property is set
Currently StoredContext object tries to load auth plugin even when
a specific property(eg. user_domain_id) is already set. However this
is redundant and we can use the value loaded from stack instead.

This also disables GLOBAL_VENV which was added to devstack recently[1].
The feature now breaks some functional tests which requires access
to the heat-manage command.

[1] a40f9cb91fbedddec89f0ffd6c7dd4b3828a232e

Story: 2010905
Task: 48723
Change-Id: I09805aa4e2d1b43528f70fb29c8981c01deff955
2023-09-05 12:27:20 +09:00
Takashi Kajinami 185f28a3b4 Isolate project scope and system scope
This change updates the default policies implemented in Heat, to follow
the updated guideline[1] to implement SRBAC.

The main change is that system users are no longer allowed to perform
any operations about project-level resources like stacks, while project
admin(*1) is still allowed to perform operations about project-level
resources BEYOND project (like getting stacks for all projects by list
stacks API).

[1] https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rbac.html#direction-change

This also adds the test cases to validate reader role which was almost
implemented in heat.

(*1)
If Keystone has an admin project defined, Heat checks an additional
requirement that request context is scoped by that admin project.

Change-Id: I943b3c1ce021cc05445b73fbc342b8386cf5bf6a
2023-06-28 18:38:59 +09:00
Stephen Finucane 46f301c964 context: Replace final use of user_domain
Resolves the following warning:

  DeprecationWarning: Property 'user_domain' has moved to
  'user_domain_id' in version '2.6' and will be removed in version '3.0'

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Iff7857e45942babc1f91a75365a00c5eb7ac59b3
2023-06-19 10:45:05 +01:00
Stephen Finucane 43a5f3984e db: Remove layer of indirection
We don't have another ORM to content with here. Simplify
'heat.db.sqlalchemy' to 'heat.db'.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Id1db6c0ff126859f436c6c9b1187c250f38ebb62
2023-03-25 12:02:27 +09:00
Tobias Urdin c253b38e21 Move CircularDependencyException to common
Moves the exception from inside the engine code
to the common code so that we can use it in the
API fault middleware.

Change-Id: I017b95153c358829501f6a5740918cdb005fb32f
2023-02-27 12:22:38 +00:00
Tobias Urdin 29eaf4ef71 Use new get_rpc_client API from oslo.messaging
Use the new API that is consistent with
the existing API instead of instantiating the client
class directly.

This was introduced in release 14.1.0 here [1] and
added into oslo.messaging here [2]

[1] https://review.opendev.org/c/openstack/requirements/+/869340
[2] https://review.opendev.org/c/openstack/oslo.messaging/+/862419

Change-Id: Ia48de36e3cb97378785bb4946b1a387c0da1ff81
2023-01-19 20:25:47 +00:00
ramishra ceef7869e0 Don't load all resources for policy check
We only need the resource types for the check. Let's
avoid loading all resources.

Change-Id: Iaf5650b2ae972064292cfe0d17f98b5635c51bad
2022-11-15 09:35:38 +05:30
Zuul 3d228d0449 Merge "Remove remaining implementation for Heat CloudWatch API" 2022-08-17 05:49:39 +00:00
Rabi Mishra d9fe7301b3 Fix parameter merging with merge strategies
Earlier users had to specify merge strategy for
a parameter/parameter_default in every environment
file where they are used(or the defaults for that
environment was used). However, the strategy
for a parameter in the first environment was
ignored as there is nothing to merge.

With Iec3367667248d0b46196a6d569f3cbb3b200d842, we
changed to use existing merge strategies, but the
strategies in the first environment were still
ignored. This patch fixes it.

Task: 45578
Change-Id: Ic6125c6af60c1007243523cc8510a17f49c7b5af
2022-07-01 08:38:49 +05:30
Zuul 1080490d94 Merge "Fix AttributeError in Python 3.8" 2022-06-21 08:13:41 +00:00
Takashi Kajinami eec1f92bc0 Fix AttributeError in Python 3.8
The importlib-metadata package was imported to the core Python since
Python 3.8, and stevedore uses the standard library implementation
instead of the 3rd party package in Python 3.8 and later. However
the implementation available in Python 3.8 is quite old and especially
it does not include the change which added the module attribute[1].

This change makes sure we avoid referring to that attribute if it does
not exist, to avoid AttributeError in Python 3.8.

[1] dfc96137ac

Story: 2010095
Task: 45650
Change-Id: Ieba5f353b0fe8a9b187bac1f6ff1b3a8f3b896b3
2022-06-20 21:32:08 +09:00
rabi c0be1f01fd Log parameters merged
This can help in troubleshooting.

Task: 45579
Change-Id: I6aee0719b0cdef2b17666a43074c5c18645fc180
2022-06-09 11:11:27 +05:30
rabi 94f32ee39f Use previous parameter merge strategy
We don't allow different parameter merge strategies
in different environment files for a parameter. If
a merge strategy has already been provided in one
environment file, let's use that for subsequent ones
unless a different merge strategy has been provided.

Task: 45578
Change-Id: Iec3367667248d0b46196a6d569f3cbb3b200d842
2022-06-09 11:06:24 +05:30
Takashi Kajinami ba15d70e19 Remove remaining implementation for Heat CloudWatch API
The CloudWatch API was removed during Queens cycle[1]. This change
removes the remaining parameters because these have been kept for
enough cycles to let users aware of the removal.

Change-Id: I4f57f02332cb1b61a0681794e519dd5ecfbec6c3
2022-05-06 19:43:56 +09:00
Takashi Kajinami 968969b99c Fix compatibility with oslo.context >= 4.0.0
The tenant argument of RequestContext is longer available since
oslo.context 4.0.0. This change removes usage of the deprecated
argument in unit tests to prepare for version bump.

Change-Id: I0493b5fbe09d183e57a207c1dda0b2ee7a4e1b34
2022-03-01 18:40:34 +09:00
Takashi Kajinami 271156446f Drop support for Block Storage API v2
... because it was already removed from cinder[1].

[1] e05b261af7dcd24096b229860df65dff1d385910

Change-Id: I8baf1701483ce2addd47dd4cb472f92ad4a192cd
2021-08-02 05:41:26 +00:00
Takashi Kajinami 77bde0120b Add volumev3 to shared_services_types by default
The shared_services_types parameter defines the shared services located
in the other region. This parameter by default include volume and
volumev2 but doesn't include volumev3 which represents Bloack Storage
API v3.
This change adds volumev3 to the default items so that all of volume
services are included by default.

Change-Id: Idde186b2d88321e7902e4330780aabc369754b14
2021-08-02 05:41:11 +00:00
ramishra 892f365a1e Suppress policy deprecation warnings
Logs are now filled with these deprecated policy warnings.
We'll remove it once we remove the deprecated rules.

Closes-Bug: #1933241
Change-Id: I5b52b0bdaa41510c0d962876241bf9b7ad121d96
2021-06-23 14:51:19 +05:30
ramishra 3cf6c575db Switch to use <user/project>_domain_id in context
Unreadable logs from oslo.context warnings.

"Using the 'project_domain' argument is deprecated in version
'2.18' and will be removed in version '3.0'"

Change-Id: Icfe18a5d299da4b6b4024374791be53f2b3e41f2
2021-06-23 14:22:55 +05:30
Ghanshyam Mann 157f358057 [goal] Deprecate the JSON formatted policy file
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:

1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.

2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.

Also replace policy.json to policy.yaml ref from doc and code.

[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: I1aa12bcd2638390f25d57ce8abeeec248121dc02
2021-06-17 01:35:45 +08:00
Zuul c7babfd3c2 Merge "Allow turning off max_stacks limit" 2021-03-15 14:05:32 +00:00
Zuul 98dc789fbc Merge "Switch to collections.abc.*" 2021-03-12 08:07:07 +00:00
Zuul 4d07bb7914 Merge "Fix duplicated words issue like "can be be used"" 2021-03-11 21:19:08 +00:00
Rico Lin 8daa7e9389 Allow using database configs on db retry
Allow following db configs when calling wrap_db_retry:
    * database.db_max_retries
    * database.db_retry_interval
    * database.db_inc_retry_interval
    * database.db_max_retry_interval
So database cofig can now control db retries.
Please reference [1] for what each config options can do.

[1] https://opendev.org/openstack/oslo.db/src/branch/master/oslo_db/options.py

Change-Id: I034625733c2d22f0f5635f58e9df3d5785e58cf5
2021-02-27 15:42:07 +08:00
Stephen Finucane 57e9754093 Switch to collections.abc.*
The abstract base classes previously defined in 'collections' were moved
to 'collections.abc' in 3.3. The aliases will be removed in 3.10.
Preempt this change now with a simple find-replace:

  $ ag -l 'collections.($TYPES)' | \
      xargs sed -i 's/\(collections\)\.\($TYPES\)/\1.abc.\2/g'

Where $TYPES is the list of moved ABCs from [1].

[1] https://docs.python.org/3/library/collections.abc.html

Change-Id: Ia282479bb1d466bd2189ebb21b51d91e89b9581e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-02-01 11:15:59 +00:00
Tim Burke a78ddafdec Use err.errno to access errno for socket errors
Otherwise, you may see errors like

    TypeError: 'OSError' object is not subscriptable

Change-Id: I2aa952d6e323f2f30ab915c3fbf609c651ed1fe9
2020-09-26 21:33:03 -07:00
Zuul 8e1e3d6011 Merge "Set context username if session client is used" 2020-09-20 10:02:46 +00:00
Rabi Mishra 2aea5c9b32 Use correct attribute from entrypoint
The type of the entry point objects returned in stevedore has
been changed from`pkg_resources.EntryPoint` to
`importlib.metadata.EntryPoint`[1] and it does not have module_name
attribute.

It also bumps requirements/lower-constraints as stevedore==3.1.0 is
not backward compatible.

[1] https://review.opendev.org/#/c/740515/
Task: 40355
Change-Id: Icd68f6eab73e44406f2ff5073636dcdd111e00ea
2020-07-15 17:51:57 +05:30
Andrii Ostapenko 402750b099 Set context username if session client is used
Now context username is set from X-Auth-User header, that is not set
with session client. This results in stack_owner attribute None for
newly created stacks.
This commit sets context username from auth token in case of absent
X-Auth-User header.

Affects all releases at least since Ocata.

Story: 2007856
Change-Id: I9cba3495fd472769498486d52d3909bff03010d4
2020-07-06 19:34:52 +00:00
Zuul 0e7d99ac3f Merge "Logging Consistency" 2020-05-04 10:12:13 +00:00
Andreas Jaeger 7f770b95b8 Update pep8 rules
Remove E402 from flake8 list, issues are fixed or addressed with noqa.
Remove E305 as comment, the warning is enabled.
Enable E241 and fix all problems

Change-Id: Ifd2bbce98be4206908da769df686a370328c7ee1
2020-04-30 16:33:43 +02:00
Marc Methot 156f276665 Logging Consistency
A simple matter of having log lines fit
with all the others. It will now match the
ambiguous standard.

Also change comments inline to alias with log fixes.

Change-Id: I4a2ed6134f70c2965811f75ccb6ab8221fa0e960
Story: #2007319
Task: 38830
2020-04-27 06:34:31 +00:00
Hervé Beraud 5877da06a1 Remove six and python 2.7 full support
Six is in use to help us to keep support for python 2.7.
Since the ussuri cycle we decide to remove the python 2.7
support so we can go ahead and also remove six usage from
the python code.

Review process and help
-----------------------
Removing six introduce a lot of changes and an huge amount of modified files
To simplify reviews we decided to split changes into several patches to avoid
painful reviews and avoid mistakes.

To review this patch you can use the six documentation [1] to obtain help and
understand choices.

Additional informations
-----------------------
Changes related to 'six.b(data)' [2]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

six.b [2] encode the given datas in latin-1 in python3 so I did the same
things in this patch.

Latin-1 is equal to iso-8859-1 [3].

This encoding is the default encoding [4] of certain descriptive HTTP
headers.

I suggest to keep latin-1 for the moment and to move to another encoding
in a follow-up patch if needed to move to most powerful encoding (utf8).

HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5].

Note that this commit message is autogenerated and not necesserly contains
changes related to 'six.b'

[1] https://six.readthedocs.io/
[2] https://six.readthedocs.io/#six.b
[3] https://docs.python.org/3/library/codecs.html#standard-encodings
[4] https://www.w3schools.com/charsets/ref_html_8859.asp
[5] https://www.w3schools.com/html/html_charset.asp

Patch 4 of a serie of 28 patches

Change-Id: I871c2dad10abc35790e730c7c4c5272f499b7623
2020-04-22 12:23:44 +02:00
Hervé Beraud fccd312871 Remove six and python 2.7 full support
Six is in use to help us to keep support for python 2.7.
Since the ussuri cycle we decide to remove the python 2.7
support so we can go ahead and also remove six usage from
the python code.

Review process and help
-----------------------
Removing six introduce a lot of changes and an huge amount of modified files
To simplify reviews we decided to split changes into several patches to avoid
painful reviews and avoid mistakes.

To review this patch you can use the six documentation [1] to obtain help and
understand choices.

Additional informations
-----------------------
Changes related to 'six.b(data)' [2]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

six.b [2] encode the given datas in latin-1 in python3 so I did the same
things in this patch.

Latin-1 is equal to iso-8859-1 [3].

This encoding is the default encoding [4] of certain descriptive HTTP
headers.

I suggest to keep latin-1 for the moment and to move to another encoding
in a follow-up patch if needed to move to most powerful encoding (utf8).

HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5].

Note that this commit message is autogenerated and not necesserly contains
changes related to 'six.b'

[1] https://six.readthedocs.io/
[2] https://six.readthedocs.io/#six.b
[3] https://docs.python.org/3/library/codecs.html#standard-encodings
[4] https://www.w3schools.com/charsets/ref_html_8859.asp
[5] https://www.w3schools.com/html/html_charset.asp

Patch 3 of a serie of 28 patches

Change-Id: If6e66839c128dde5fb80d90155dedb598da8d53b
2020-04-22 12:23:44 +02:00
Hervé Beraud bb02b2b5f1 Remove six and python 2.7 full support
Six is in use to help us to keep support for python 2.7.
Since the ussuri cycle we decide to remove the python 2.7
support so we can go ahead and also remove six usage from
the python code.

Review process and help
-----------------------
Removing six introduce a lot of changes and an huge amount of modified files
To simplify reviews we decided to split changes into several patches to avoid
painful reviews and avoid mistakes.

To review this patch you can use the six documentation [1] to obtain help and
understand choices.

Additional informations
-----------------------
Changes related to 'six.b(data)' [2]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

six.b [2] encode the given datas in latin-1 in python3 so I did the same
things in this patch.

Latin-1 is equal to iso-8859-1 [3].

This encoding is the default encoding [4] of certain descriptive HTTP
headers.

I suggest to keep latin-1 for the moment and to move to another encoding
in a follow-up patch if needed to move to most powerful encoding (utf8).

HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5].

Note that this commit message is autogenerated and not necesserly contains
changes related to 'six.b'

[1] https://six.readthedocs.io/
[2] https://six.readthedocs.io/#six.b
[3] https://docs.python.org/3/library/codecs.html#standard-encodings
[4] https://www.w3schools.com/charsets/ref_html_8859.asp
[5] https://www.w3schools.com/html/html_charset.asp

Patch 2 of a serie of 28 patches

Change-Id: I2795dee87f0e27b64820686acfc614ac2ba19a4f
2020-04-22 12:23:44 +02:00