Commit Graph

10510 Commits

Author SHA1 Message Date
Zuul 0e5aeb5045 Merge "s3api: Fix handling of non-ascii access keys" 2024-03-25 21:40:58 +00:00
Zuul cdc4f264d2 Merge "recon-cron: Tolerate missing directories" 2024-03-25 06:45:56 +00:00
Tim Burke 8424b02290 s3api: Fix handling of non-ascii access keys
We stuff the access key into the request path until we get back a
more-authoritative account name from auth. But it needs to be a WSGI
string when we do!

Closes-Bug: #2058748
Change-Id: I34adb8141cc9e62d17a27f01c63f40d1dd25991c
2024-03-22 10:02:39 -07:00
Tim Burke f31b6f7353 recon-cron: Tolerate missing directories
Any of these directories may get unlinked between when we saw them in
their parent's directory listing and when we go to descend.

Change-Id: I1dfc0ee1d9e70cb0600557cde980bd5880bd40b3
2024-03-21 14:10:14 -07:00
Zuul 537b6885e3 Merge "Update master for stable/2024.1" 2024-03-21 16:21:07 +00:00
Zuul d3938c3547 Merge "CI: Move probe tests to centos 9 stream" 2024-03-21 01:46:32 +00:00
Zuul fd3997f027 Merge "tests: Update CORS geckodriver" 2024-03-20 03:48:32 +00:00
OpenStack Release Bot 5d25108648 Update master for stable/2024.1
Add file to the reno documentation build to show release notes for
stable/2024.1.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2024.1.

Sem-Ver: feature
Change-Id: Ic940ff424aef9cc402bf54ebe5e5fc16330fc25c
2024-03-15 08:27:17 -07:00
Tim Burke 1c03d58643 CI: Move probe tests to centos 9 stream
Pin selenium to 3.x for now, until we can run down the issues with 4.x

Change-Id: I596415d17f77f48a6e8a63a61b734a8ca0865847
2024-03-15 08:09:25 -07:00
Tim Burke af15ad53fb tests: Update CORS geckodriver
Change-Id: I5ab762dfe0f85e346c4868ec4540884ba5f0a7f4
2024-03-14 20:45:47 -07:00
Jianjian Huo 27ef11ea14 test: implement cache expiration time in MockMemcached
Change-Id: I16ec414f87ac1a5e1e87e7560290c5ef0ca4f7cf
2024-03-15 13:50:32 +11:00
Zuul b6dc24dbc0 Merge "s3api test for zero byte mpu" 2024-03-13 17:21:42 +00:00
Zuul 891d06345e Merge "s3api: Support GET/HEAD request with ?partNumber" 2024-03-13 08:28:03 +00:00
Zuul 60db1f847c Merge "slo: part-number=N query parameter support" 2024-03-13 00:13:45 +00:00
Tim Burke 5f95b6b26d AUTHORS/CHANGELOG for 2.33.0
Change-Id: I6b0a074fca4d6bd3b8b95888ae8721a62f3a6df0
2024-03-12 11:59:59 -07:00
Zuul 7941cf0efc Merge "CI: Bring unit test jobs in line with 2024.1 tested runtimes" 2024-03-12 18:51:46 +00:00
Zuul 2c4ff7bf2c Merge "CI: Update rolling-upgrade jobs to point to unmaintained branches" 2024-03-12 18:39:52 +00:00
Zuul b6f5971471 Merge "reno: Update master for unmaintained/wallaby" 2024-03-12 17:23:07 +00:00
Zuul e951a42788 Merge "reno: Update master for unmaintained/victoria" 2024-03-12 17:23:04 +00:00
Tim Burke 9279e3d2c0 CI: Bring unit test jobs in line with 2024.1 tested runtimes
The last time I really looked at this was probably Yoga, when we were
targetting 3.6 through 3.9 (and left 3.7 and 3.8 as experimental jobs).

Now, though, OpenStack is targetting 3.8 through 3.11; as before, we
can assume that if tests pass on those two versions, they should pass
on the versions in-between, too. (But still have them as experimental,
on-demand jobs).

See https://governance.openstack.org/tc/reference/runtimes/2024.1.html

Keep 2.7 and 3.6 testing as our own self-imposed minimums.

Change-Id: I7700aa3c93df311644655e7ebaf0b67aa692ee80
2024-03-12 09:19:53 -07:00
Tim Burke b06ffea941 CI: Update rolling-upgrade jobs to point to unmaintained branches
Change-Id: I936d9074ab60e34b379fb207d63f10bd5c3a4312
2024-03-12 09:06:23 -07:00
Clay Gerrard d10351db30 s3api test for zero byte mpu
Change-Id: I89050cead3ef2d5f8ebfc9cb58f736f33b1c44fe
2024-03-12 13:48:02 +00:00
indianwhocodes 46e7da97c6 s3api: Support GET/HEAD request with ?partNumber
Co-Authored-By: Alistair Coles <alistairncoles@gmail.com>
Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Closes-Bug: #1735284
Change-Id: Ib396309c706fbc6bc419377fe23fcf5603a89f45
2024-03-12 13:47:55 +00:00
indianwhocodes 6adbeb4036 slo: part-number=N query parameter support
This change allows individual SLO segments to be downloaded by adding
an extra 'part-number' query parameter to the GET request.  You can
also retrieve the Content-Length of an individual segment with a HEAD
request.

Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Co-Authored-By: Alistair Coles <alistairncoles@gmail.com>
Change-Id: I7af0dc9898ca35f042b52dd5db000072f2c7512e
2024-03-12 06:47:02 -07:00
OpenStack Release Bot dc379af893 reno: Update master for unmaintained/xena
Update the xena release notes configuration to build from
unmaintained/xena.

Change-Id: I72576af46cc13fac0a7a69461b673931c8b8496e
2024-03-12 10:49:32 +00:00
OpenStack Release Bot 3ad1565f0a reno: Update master for unmaintained/wallaby
Update the wallaby release notes configuration to build from
unmaintained/wallaby.

Change-Id: Ieb2d4542d395bb1f4498beab0d9ec146fad7ba84
2024-03-12 10:49:01 +00:00
OpenStack Release Bot 0d8c89b123 reno: Update master for unmaintained/victoria
Update the victoria release notes configuration to build from
unmaintained/victoria.

Change-Id: I5a420de74a2ef5096135511dfea2489419414856
2024-03-12 10:48:24 +00:00
Tim Burke 6a426f7fa0 sharder: Add periodic_warnings_interval to example config
Change-Id: Ie3c64646373580b70557f2720a13a5a0c5ef7097
2024-03-11 10:35:13 -07:00
Zuul 0f6ecb641b Merge "docs: add discussion of content-type metadata" 2024-03-08 11:15:52 +00:00
Alistair Coles cc27780042 docs: add discussion of content-type metadata
Change-Id: I2aa13e2b23bda86c51ef6aaa69ea3fd0075bb9ad
2024-03-06 11:02:15 +00:00
Matthew Oliver 4135133a63 memcachering: change failed to yield log message
Currently when the memcachering `_get_conns` method runs out of memcached
servers to try and so fails to yield anything we log a:

  All memcached servers error-limited

However, this error message isn't entirely accurate. It can also fail
because it failed to connect all it's memcached servers not just because
they're error limited.
You can disable error-limiting of memcached servers. So in this case
this error message is a red-herring.

Downstream we use a mcrouter client on each node which itself talks to a
bunch of memcache servers. Therefore in swift's memcachering client we
only configure the 1 mcrouter client as a single server in the ring.
Because of this we disable memcached error-limiting.
If the node gets too overloaded we've had timeouts talking to the local
mcrouter client. This fires off error-limitted log messages which can
confuse things.

Because it's possible to turn off error-limiting, the log line isn't
quite adequate anymore. So this patch changes it to:

  No more memcached servers to try

Change-Id: I97fb4f3ee2ac45831aae14a782b2c6dc73e82d85
2024-03-05 14:44:37 +11:00
Zuul 627448362a Merge "CI: Remove centos-7 jobs" 2024-03-04 19:14:07 +00:00
Zuul dd066946a1 Merge "CI: fix rolling-upgrade jobs" 2024-03-02 03:57:36 +00:00
Zuul 6cc5262bb7 Merge "CI: pin python-dateutil for py2" 2024-03-01 20:38:54 +00:00
Tim Burke bd3b2256a9 CI: Remove centos-7 jobs
CentOS 7 will go EOL later this year, and infra wants to drop the nodes
soon-ish -- don't make them wait on our account.

The only major loss is py2 probe tests, but officially, yoga was the
last release we pledged to support py2.

Change-Id: I8f6c247c21f16aa4717569cc69308f846c6a0245
2024-03-01 11:54:50 -08:00
Tim Burke 275af9e008 CI: fix rolling-upgrade jobs
train and ussuri are now EOL; drop them.

yoga has moved from stable to unmaintained.

Change-Id: I3516823fdacbe8fd3c2434c0de9dedd1d82980fe
2024-03-01 11:54:50 -08:00
Tim Burke f32f2dd023 CI: pin python-dateutil for py2
Their 2.9.0 release is known-broken for py27-py35.

Change-Id: I40c1724fa673ac252f5052ac85006788ba69d5c7
2024-03-01 09:12:28 -08:00
Zuul 3478803a95 Merge "zero bytes manifests are not legacy" 2024-02-28 17:34:04 +00:00
Zuul e9cf2a31aa Merge "tests: Clear txn id on init for all debug loggers" 2024-02-28 17:05:34 +00:00
Zuul 0947e94f66 Merge "staticweb: Work with prefix-based tempurls" 2024-02-28 07:42:52 +00:00
Clay Gerrard 130188b6c0 zero bytes manifests are not legacy
Change-Id: I7c8adb129b8770eee501748a378f3adc42c8cd39
2024-02-27 17:21:00 -06:00
Zuul 4c5f41cc1f Merge "Fix diskfile test failing on macOS" 2024-02-27 21:32:05 +00:00
Tim Burke 1ee9b1e3ba tests: Clear txn id on init for all debug loggers
Since we fake out all the greenthread stuff to run in the main thread,
we can (sometimes?) find that a transaction ID has already been set,
leading to failures in test_bad_request_app_logging like

    AssertionError: b'X-Trans-Id: test-trans-id' not found
    in b'X-Trans-Id: tx...'

By resetting the logger's txn_id, we're assured that our mock will be
run and the expected transaction ID will be used.

Change-Id: I465eed5372a2a5e591f80a09676f4b7f091cd444
2024-02-27 09:49:50 -08:00
Zuul 07c8e8bcdc Merge "Object-server: add periodic greenthread yielding during file read." 2024-02-27 04:03:00 +00:00
Jianjian Huo d5877179a5 Object-server: add periodic greenthread yielding during file read.
Currently, when object-server serves GET request and DiskFile
reader iterate over disk file chunks, there is no explicit
eventlet sleep called. When network outpace the slow disk IO,
it's possible one large and slow GET request could cause
eventlet hub not to schedule any other green threads for a
long period of time. To improve this, this patch add a
configurable sleep parameter into DiskFile reader, which
is 'cooperative_period' with a default value of 0 (disabled).

Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: I80b04bad0601b6cd6caef35498f89d4ba70a4fd4
2024-02-27 11:24:41 +11:00
Alistair Coles 2da150b890 Fix diskfile test failing on macOS
The existing test fails on macOS because the value of errno.ENODATA is
platform dependent. On macOS ENODATA is 96:

% man 2 intro|grep ENODATA
     96 ENODATA No message available.

Change-Id: Ibc760e641d4351ed771f2321dba27dc4e5b367c1
2024-02-26 11:12:47 +00:00
Alistair Coles 2500fbeea9 proxy: don't use recoverable_node_timeout with x-newest
Object GET requests with a truthy X-Newest header are not resumed if a
backend request times out. The GetOrHeadHandler therefore uses the
regular node_timeout when waiting for a backend connection response,
rather than the possibly shorter recoverable_node_timeout. However,
previously while reading data from a backend response the
recoverable_node_timeout would still be used with X-Newest requests.

This patch simplifies GetOrHeadHandler to never use
recoverable_node_timeout when X-Newest is truthy.

Change-Id: I326278ecb21465f519b281c9f6c2dedbcbb5ff14
2024-02-26 09:54:36 +00:00
Alistair Coles 8061dfb1c3 proxy-server: de-duplicate _get_next_response_part method
Both GetOrHeadHandler (used for replicated policy GETs) and
ECFragGetter (used for EC policy GETs) have _get_next_response_part
methods that are very similar. This patch replaces them with a single
method in the common GetterBase superclass.

Both classes are modified to use *only* the Request instance passed to
their constructors. Previously their entry methods
(GetOrHeadHandler.get_working_response and
ECFragGetter.response_parts_iter) accepted a Request instance as an
arg and the class then variably referred to that or the Request
instance passed to the constructor. Both instances must be the same
and it is therefore safer to only allow the Request to be passed to
the constructor.

The 'newest' keyword arg is dropped from the GetOrHeadHandler
constructor because it is never used.

This refactoring patch makes no intentional behavioral changes, apart
from the text of some error log messages which have been changed to
differentiate replicated object GETs from EC fragment GETs.

Change-Id: I148e158ab046929d188289796abfbbce97dc8d90
2024-02-26 09:50:22 +00:00
Zuul 50336c5098 Merge "test: all primary error limit is error" 2024-02-21 19:52:20 +00:00
Zuul fe8227e56c Merge "reno: Update master for unmaintained/yoga" 2024-02-21 18:45:09 +00:00