Commit Graph

520 Commits

Author SHA1 Message Date
Thomas Goirand b82269621d Python 3.12: do not use utcnow()
The datetime.datetime.utcnow() function is deprecated in Python 3.12,

Change-Id: I52e947e017ad1455c0310fe20fdc63354a254757
2023-12-19 00:45:48 +09:00
Zuul 9ade074c9b Merge "External OAuth2.0 Authorization Server Support" 2023-11-07 16:46:40 +00:00
Zuul e49893e598 Merge "External OAuth2.0 Authorization Server Support" 2023-09-13 18:32:57 +00:00
Yusuke Niimi 53b4cb21ad External OAuth2.0 Authorization Server Support
Added the ability to authenticate using a system-scoped token and the
ability to authenticate using a cached token to the
external_oauth2_token filter.

Implements: blueprint enhance-oauth2-interoperability
Change-Id: I1fb4921faaafd5288d5909762ff5553e5e2475dc
2023-09-07 09:43:14 +00:00
sunyonggen de15a610e1 External OAuth2.0 Authorization Server Support
The external_oauth2_token filter has been added for accepting or denying
incoming requests containing OAuth 2.0 access tokens that are obtained
from an External Authentication Server.

Depends-On: https://review.opendev.org/c/openstack/keystoneauth/+/860614
Implements: blueprint enhance-oauth2-interoperability
Change-Id: I529c5b0c89933395b126e86651ef09368dd7e6b4
2023-08-30 13:30:32 +00:00
Sahid Orentino Ferdjaoui 70337682d9 auth_token: fix issue when data in cache gets corrupted
Previously token cache was not correctly handling the case when data
in memcached is un-decryptable.
The cache process was returning a null value that was not considered
resulting a python exception raised

The commit fixes the issue by adding a condition to validate the value
returned.

Closes-bug: #2023015
Change-Id: Ic48d20569980781febc194083651736bed446953
Signed-off-by: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@industrialdiscipline.com>
2023-08-14 14:42:50 +00:00
Stephen Finucane 22408f8da0 Remove six
Change-Id: Ib3edfdd087ed1d954f1ecf72a191138f8f1c46a1
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-06-27 18:03:31 +01:00
Stephen Finucane 0f48ff3e8d Bump hacking to 6.0.x
A note about dependency ordering is removed from the requirements file:
this is no longer true with the dependency resolver introduced with pip
20.3.

Change-Id: I615be3453db37588edf98a46ce484efc5e051f11
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-06-27 18:03:30 +01:00
Stephen Finucane 626df3a5e9 Make tox.ini tox 4.0.0 compatible/fix gate
* Removed skipsdist=True to make sure placement available in the virtual
  env. Without this, our entrypoints are not available.

* Removed basepython = python3 as we assume all developer switched to
  python3 in their env already

* Removed ignore_basepython_conflict = True as without the basepython
  definition generative targets now work without conflict

See [1] for a similar change made to placement.

It is also necessary to fix issues with the gate. For reasons that I
have yet to grok, a mock of 'requests.request' used in some test is no
longer functioning as expected. My guess is that something is now
importing requests before us and interfering with the mock but never
mind - we can easily bypass the issue by mocking 'requests.post'
instead.

[1] https://review.opendev.org/c/openstack/placement/+/868418/

Change-Id: I3b8263afbf0ccee88ceaac2040d5ad274f22d74a
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2023-06-27 18:03:28 +01:00
Hiromu Asahina 92cdf8a0a5 Add timeout for requests
Bandit emits errors for request methods without the timeout parameter.
It's better to follow the instruction to avoid hanging.

Added timeout parameters and config options to set timeout.

[1] https://bandit.readthedocs.io/en/1.7.5/plugins/b113_request_without_timeout.html

Change-Id: I0c022c3cc57f30530ebdef6e434753ece2bdf912
2023-03-18 01:11:53 +09:00
sunyonggen a59020fdab OAuth 2.0 Mutual-TLS Support
The oauth2_mtls_token filter has been added for accepting or denying
incoming requests containing OAuth 2.0 certificate-bound access
tokens that are obtained from keystone identity server by users
through their OAuth 2.0 credentials and Mutual-TLS certificates.

Co-Authored-By: Hiromu Asahina <hiromu.asahina.az@hco.ntt.co.jp>
Depends-On: https://review.opendev.org/c/openstack/keystoneauth/+/860614
Change-Id: I49127d845954ad6eab39e6e6305948ef0e4ed7b5
Implements: blueprint support-oauth2-mtls
2023-03-03 11:28:01 +09:00
Jorge Merlino e05466c5f4 Remove cache invalidation when using expired token
This can create a race condition for long running services that reuse
their token (eg. Kubernetes Cinder CSI plugin) in this case for
example:

1 [user] Asks nova to attach a volume to a server
2 ...the user's token expires
3 [user] Asks cinder if the volume has been attached
4 [nova] Asks cinder to attach the volume

In step 3 the token is marked as invalid in the cache and step 4 fails
even if allow_expired is true

Closes-Bug: #1987355
Change-Id: Ice8e34440a5fe1baa370646ed70b5e085c4af70e
2022-12-12 20:16:14 +00:00
Dave Wilde dc0b796495 Fix pep8 gate
This updates the test-requirements to more recent flake8-docstrings
and pep8 versions.  I also added some more ignores that would take
significant work to update and fixed some minor linting issues.

Change-Id: Ia416658b8a4bfb8f43f8df170053abb3ae958d44
2022-12-09 10:28:06 -06:00
Yi Feng f7ac6a1b24 OAuth2.0 Client Credentials Grant Flow Support
The oauth2_token filter has been added for accepting or denying
incoming requests containing OAuth2.0 client credentials access tokens
that are obtained from keystone identity server by users through their
application credentials.

Change-Id: I15e438681749ed2c2666804a9efd8d4712a7b01c
2022-08-23 09:39:18 +00:00
Ayumu Ueha 9c49da6887 Fix logging notifier unit test
For unknown reasons, the `create_notifier()` in
`test_api_request_no_messaging()` used `oslo_messaging.Notifier`
instead of the `_LogNotifier` that should be originally used,
causing unit test to fail.

This patch fixes this issue by changing `use_oslo_messaging` config to
False for this test.

Change-Id: I32f9dc596525e912e37984764f68564e26ecfd3b
2022-08-09 14:33:32 +00:00
Zuul fef9959705 Merge "Remove references to 'sys.version_info'" 2022-05-30 06:20:23 +00:00
Takashi Kajinami 7b8a72d2c5 Add oslo.config.opts entrypoint for audit middleware options
... so that each service using the audit middleware can include these
parameters in .conf file generated by oslo-config-generator by adding
that entrypoint to the command.

Closes-Bug: #1939632
Change-Id: Ied954c633570c51af9504514ffed18e12de8caac
2021-08-12 11:17:13 +09:00
dengzhaosen 6a0974117e Remove references to 'sys.version_info'
We support Python 3.6 as a minimum now, making these checks no-ops.
Change-Id: Iff6abdc56c5627505e774f40af339e1b5790c4d5
2021-04-27 10:27:47 +08:00
Michal Arbet 788d3c4969 Switch to eventlet-safe oslo.cache's MemcacheClientPool
In past days there were discussions about various issues
with memcached connections [1][2][3].

After investigation it looks like common root cause for above
problems is keystonemiddleware. More precisely said the way
how keystonemiddleware is caching tokens.

Currently it's using some home-made CachePool with direct
usage of memcached library, moreover it looks like its
approach is not eventlet-safe.
Discussion can be mainly found in [4].

Fortunately keystonemiddleware can use "advanced cache pool",
which is oslo.cache's implementation and was added long time ago [5],
but it is turned on only if memcache_use_advanced_pool=True.

This patch is switching to more elaborated oslo.cache CachePool
and adding deprecation warning about eventlet-unsafe variant
of keystonemiddleware's memcache pool.

How to reproduce ?

with memcache_use_advanced_pool=False

1. Build clean ENV of openstack
2. Deploy core projects (keystone,glance,nova,placement...)
3. Run while true; do COMMAND FOR SERVICE; done
   - several bashes, in parallel (5-7)

COMMAND FOR SERVICE:
- openstack network list
- openstack volume list
- openstack server list
- openstack image list

4. Check memcached connections (which will grow up):
    - ss | grep 11211 | wc -l   every second

How to fix and test it ?

Repeat above, to fix:
 - with memcache_use_advanced_pool=True
   OR
 - apply this patch

Compare measurements in graph.

[1] https://bugs.launchpad.net/keystonemiddleware/+bug/1892852
[2] https://bugs.launchpad.net/oslo.cache/+bug/1888394
[3] https://bugs.launchpad.net/keystonemiddleware/+bug/1883659

[4] https://review.opendev.org/c/openstack/oslo.cache/+/742193

[5] https://review.opendev.org/c/openstack/keystonemiddleware/+/268664

Closes-Bug: #1883659
Closes-Bug: #1892852
Closes-Bug: #1888394

Change-Id: I0e96334b65a0bf369ebf1d88651d13feb8d2ecac
2021-02-11 14:36:25 +00:00
Zuul 11ea4a8bd2 Merge "Use unittest.mock instead of third party mock" 2020-08-24 19:38:38 +00:00
Zuul 654d31a1bd Merge "Change the default Identity endpoint to internal" 2020-07-06 10:43:56 +00:00
Jens Harbott 8f9a596fff Change the default Identity endpoint to internal
In [0] the ``interface``option was added in order to allow the Identity
endpoint that is being used when validating tokens to be
configured by the deployer. Change the default to using the internal
endpoint, as that should be what most deployments will end up using.

[0] https://review.opendev.org/651790

Depends-On: https://review.opendev.org/651492
Closes-Bug: 1830002
Change-Id: I0ce8b6d8cd408c7fac8107972e7be70839e337fb
2020-06-14 16:20:05 +00:00
Sean McGinnis 215217ea8b
Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.

Change-Id: Idc319f3f8a3ddd57cba91e4cefc66dbb18d5cc22
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-18 11:53:42 -05:00
Andreas Jaeger f32fcc6623 Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Blacklist:
W504 line break after binary operator
W503 line break before binary operator

Fix other problems found

Change-Id: I2fb257a4f42b499df3702f3e8f3c99ecb28557d6
2020-03-30 10:32:07 +00:00
Zuul 2a8e6fb053 Merge "Remove keystoneclient exception usage in tests" 2020-01-02 18:58:56 +00:00
Zuul 620f768000 Merge "Fix DeprecationWarning: invalid escape sequence issues" 2020-01-02 18:52:32 +00:00
OpenStack Proposal Bot 62c3eaf093 Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I4a5f9f48ae099291cf47f4d08c40535223761b1b
2019-12-22 07:08:56 +00:00
Gage Hugo 7c33d8ebb6 Rename _v3_to_v2_catalog to _normalize_catalog
This change renames the _v3_to_v2_catalog to _normalize_catalog as
part of the v2 removal effort. Several services still rely on the
converted catalog format, so this change maintains the conversion
but removes the v2 association.

Change-Id: Ic7bca16d8c6211d006fc2ba09dc2ecd83f8955db
Partial-Bug: #1845539
Partial-Bug: #1777177
2019-12-20 14:37:43 -06:00
Zuul 97350d16b1 Merge "Change ec2 URLs to v3" 2019-12-03 07:54:19 +00:00
Gage Hugo 09a33cce89 Change ec2 URLs to v3
This change modifies any URLs specifying v2.0 to v3. This is part
of the effort to remove v2.0 functionality from keystonemiddleware.

Change-Id: I9cde8963333ea95b4ab05d9aea4d196ab4357763
Partial-Bug: #1829453
Partial-Bug: #1845539
2019-11-29 04:20:14 +00:00
Gage Hugo a6a3edb80e Remove v2.0 functionality
This change removes v2.0 functionality from
keystonemiddleware, as well as associated tests.

Partial-Bug: #1845539
Partial-Bug: #1777177

Change-Id: If47e90085d8a59c52fb23876dc329cd4f0b05ef0
2019-11-27 10:52:23 -06:00
Gage Hugo d3090bfbc0 Remove keystoneclient exception usage in tests
This change replaces the usage of keystoneclient exceptions in the
auth_token_middleware unit tests to use the ConnectFailure exception
from keystoneauth.

This is part of the process of removing keystoneclient from
keystonemiddleware.

Change-Id: Ie00fc310728c319faf2cfdfb651f0c7a8f48d757
2019-11-24 00:12:24 -06:00
Sean McGinnis 4d6e9cb162
Fix DeprecationWarning: invalid escape sequence issues
Some regex strings contain invalid escape sequences for normal strings,
causing newer version of Python to emit DeprecationWarning messages.
This updates those instances to raw strings so they are not interpreted
as invalid.

Change-Id: I28ac26516bacab36578a5a7f6ec7f9dcf7d7eeb1
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-11-13 07:16:33 -06:00
Bernhard M. Wiedemann 4a4c96ce9b Make tests pass in 2022
Background:
As part of my work on reproducible builds for openSUSE, I check that software still gives identical build results in the future.
The usual offset is +15 years, because that is how long I expect some software will be used in some places.
This showed up failing tests in our package build.
See https://reproducible-builds.org/ for why this matters.

This makes it expire 1 year in the future to model realistic tokens.

Change-Id: I73bde68be53afff4e8dff12d756b8381f34b2adb
2019-09-17 13:19:24 +02:00
Q.hongtao d1bd783a77 Fix misspell word
Change-Id: I2b727f28733b1f7e2bb0e2dfa94a39bfdd7449fc
2019-09-09 09:02:14 +08:00
Zuul 3183b3d2fc Merge "Add validation of app cred access rules" 2019-07-18 19:47:31 +00:00
Colleen Murphy 5f093bf5ee Add validation of app cred access rules
This commit adds a validation step in the auth_token middleware to check
for the presence of an access_rules attribute in an application
credential token and to validate the request against the permissions
granted for that token. During token validation it sends a header to
keystone to indicate that it is capable of validating these access
rules, and not providing this header for a token like this would result
in the token failing validation. This disregards access rules for a
service request made by a service on behalf of a user, such as nova
making a request to glance, because such a request is not under the
control of the user and is not expected to be explicitly allowed in the
access rules.

bp whitelist-extension-for-app-creds

Depends-On: https://review.opendev.org/670377

Change-Id: I185e0541d5df538d74edadf9976b3034a2470c88
2019-07-15 16:05:59 -07:00
Zuul 06fc3e42d7 Merge "print auth version for request strategy in debug" 2019-07-12 18:27:43 +00:00
Zuul 2ed915f4fe Merge "Remove PKI/PKIZ support" 2019-06-21 21:31:56 +00:00
Colleen Murphy d040cf67fb Remove Diablo compatibility tests
We really don't care about Diablo compatibility any more. Clean up the
old cruft.

Change-Id: Ib1f628eb40ba0cb6334300cb6dca7dcdfcddba1b
2019-06-20 12:40:19 -07:00
Lance Bragstad 0c3b3f5ee2 Fix bandit warning
Bandit is throwing warnings because we use 'token' in a couple of
variables.

Change-Id: I9fd21974027bc2bda6036c34fa587a044faaacae
2019-06-20 15:10:35 +00:00
Morgan Fainberg b3e84aafc0 Remove PKI/PKIZ support
Keystone server no longer supports PKI/PKIZ. This change removes
keystonemiddleware's support of PKI/PKIZ and associated code.

Change-Id: I9a6639a2aa3774be61972d57f38220f66fd5c0e8
closes-bug: #1649735
partial-bug: #1736985
2019-06-19 12:16:47 -07:00
Zuul 3e62d25dac Merge "Add a new option to choose the Identity endpoint" 2019-06-12 15:47:50 +00:00
Jens Harbott f6037a3d50 Add a new option to choose the Identity endpoint
Previously the admin Identity endpoint was hardcoded to be used. Now
that keystone has dropped v2 support, deploying an admin Identity
endpoint is no longer useful, so allow this to be changed by the
deployer. Keep the default as using the `admin` endpoint, but create
a deprecation message so that we can change the default in the future.

Partial-Bug: 1830002
Change-Id: I993a45ccb1109d67e65bf32d1e134cc9bec2d88e
2019-06-03 10:34:25 +00:00
ushen fe36fa6b7c print auth version for request strategy in debug
previously it will print auth version of _requested_auth_version
which will be none all the time. Change it to klass makes more sense.

Change-Id: I1cec8f163e808f03f15ef053e5768cf711238f0d
2019-05-20 16:48:03 -07:00
Sean McGinnis 27b916ee1b
Fix string format error
Attempt to escape quotes actually forgot one of the quotes, causing
"invalid escape sequence" warnings in the logs.

Change-Id: I843257ba5c26f7ba6c5cd3b1b7c53ed26cb9ee8d
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-04-09 17:16:42 -05:00
ZhongShengping caa899b93d Fix service_token_role_required option
The service_token_roles_required should be correct.

Change-Id: I009e3a495953d61fb0c29a8b629efa3322cb0ddd
2019-02-15 08:35:24 +08:00
Yang Youseok 4e51cb8e6b Add auth invalidation in auth_token for identity endpoint update
Currently auth_token middleware does not concern identity endpoint
update since service catalog is not updated after service having
auth_token middleware started.

Add invalidation logic when EndpointNotfound exception occurs so
that auth_token middleware can be notified of sevice catalog update
without restart.

Change-Id: I631ee1538883d732fe3987b172d987f703dad5c0
Closes-Bug: #1813739
2019-02-07 12:14:51 +09:00
Leehom Li (feli5) 82707e15a5 Make sure audit middleware use own context
Keystone audit middleware requires to iterate req.context as dict,
but Glance requires to access req.context.read_only.
When glance enabled audit, they are conflict with each other.
This patch fix this issue by store audit context in
req.environ['audit.context']

Change-Id: Ib9a62a4cd0b7b9ffb9fa2d6440e8072d45ee0fee
Closes-Bug: #1809101
Signed-off-by: Leehom Li <feli5@cisco.com>
2018-12-24 02:02:17 +00:00
Artem Vasilyev f2f5820c5f Added request_id and global_request_id to CADF notifications
Change-Id: I8d571d3414071c68b4fa565dec46cc2d2941331c
Closes-Bug: #1803940
2018-11-19 11:49:24 +03:00