Commit Graph

8935 Commits

Author SHA1 Message Date
Tim Burke 481f126e6b Merge remote-tracking branch 'gerrit/master' into feature/losf
Change-Id: If9d7c63f3c4c15fbccff31e2b77a6911bb95972a
2020-06-18 09:41:46 -07:00
Tim Burke 51a587ed8d Use ensure-pip role
Hopefully this will fix the currently-broken probe test gate?

Depends-On: https://review.opendev.org/#/c/736070/
Change-Id: Ib652534b35236fdb6bcab131c7dc08a079bf72f6
2020-06-16 21:29:01 -07:00
Zuul 0bed59e3fa Merge "Use ini_file to update timeout instead of crudini" 2020-06-11 21:03:01 +00:00
Zuul 3c0a448f06 Merge "memcached: Plumb logger into MemcacheRing" 2020-06-11 18:54:58 +00:00
Tim Burke 79811df34c Use ini_file to update timeout instead of crudini
crudini seems to have trouble on py3 -- still not sure *why* it's using
py3 for the losf job, though...

Change-Id: Id98055994c8d59e561372417c9eb4aec969afc6a
2020-06-09 18:27:54 -07:00
Tim Burke e4586fdcde memcached: Plumb logger into MemcacheRing
This way proxies log memcached errors in the normal way instead of
to the root logger (which eventually gets them out on STDERR).

If no logger is provided, fall back to the root logger behavior.

Change-Id: I2f7b3e7d5b976fab07c9a2d0a9b8c0bd9a840dfd
2020-06-09 14:32:04 -05:00
Zuul 0b86f681f5 Merge "swift-get-nodes: Allow users to specify either quoted or unquoted paths" 2020-06-09 17:54:26 +00:00
Zuul 3b651d945c Merge "tests: Skip s3api copy-version test when OV not enabled" 2020-06-08 21:53:06 +00:00
Zuul f58791f376 Merge "Remove etag-quoter from 2.25.0 release notes" 2020-06-08 21:15:09 +00:00
Tim Burke 1dfa41dada swift-get-nodes: Allow users to specify either quoted or unquoted paths
Now that we can have null bytes in Swift paths, we need a way for
operators to be able to locate such containers and objects. Our usual
trick of making sure the name is properly quoted for the shell won't
suffice; running something like

   swift-get-nodes /etc/swift/container.ring.gz $'AUTH_test/\0versions\0container'

has the path get cut off after "AUTH_test/" because of how argv works.

So, add a new option, --quoted, to let operators indicate that they
already quoted the path.

Drive-bys:

  * If account, container, or object are explicitly blank, treat them
    as though they were not provided. This provides better errors when
    account is explicitly blank, for example.
  * If account, container, or object are not provided or explicitly
    blank, skip printing them. This resolves abiguities about things
    like objects whose name is actually "None".
  * When displaying account, container, and object, quote them (since
    they may contain newlines or other control characters).

Change-Id: I3d10e121b403de7533cc3671604bcbdecb02c795
Related-Change: If912f71d8b0d03369680374e8233da85d8d38f85
Closes-Bug: #1875734
Closes-Bug: #1875735
Closes-Bug: #1875736
Related-Bug: #1791302
2020-06-08 12:03:56 -07:00
Zuul b51dfcb45c Merge "dsvm: Use devstack's s3api "service"" 2020-06-07 18:58:39 +00:00
Zuul 119fd25ca5 Merge "s3api: Check whether versioning is enabled more" 2020-06-06 04:18:26 +00:00
Zuul c2b03f6779 Merge "Latch shard-stat reporting" 2020-06-06 04:01:26 +00:00
Zuul 713f056493 Merge "Quiet eventlet exceptions in test" 2020-06-06 03:31:35 +00:00
Zuul 00b9ee2538 Merge "Don't auto-create shard containers" 2020-06-06 02:21:10 +00:00
Tim Burke 1b6c8f7fdf Remove etag-quoter from 2.25.0 release notes
This was released in 2.24.0, which already has a release note for it.

Change-Id: I9837df281ec8baa19e8e4a7976f415e8add4a2da
2020-06-05 16:36:32 -07:00
Tim Burke 11dd0da29a tests: Bump up timeout for unit and in-process func tests
We've been seeing more TIMEOUT failures lately where the jobs still seem
to be making steady (if slow) progress.

Change-Id: I19c1f48bace551c78ad0c6c8b6ccad75e44e8904
2020-06-05 14:20:01 -07:00
Zuul ba01f93ad9 Merge "func tests: work with etag-quoter on by default" 2020-06-05 20:52:30 +00:00
Clay Gerrard fc731198ac Quiet eventlet exceptions in test
This traceback would probably show up in the wild, but it's not relevant
to this test.

Change-Id: I9e6e7679f674bddddcc4440b38d5741aaece3393
2020-06-05 09:49:01 -05:00
Zuul 56278100a1 Merge "relinker: Improve performance by limiting I/O" 2020-06-05 06:09:24 +00:00
Tim Burke ce4c0fb14b Don't auto-create shard containers
...unless the client requests it specifically using a new flag:

   X-Backend-Auto-Create: true

Previously, you could get real jittery listings during a rebalance:

 * Partition with a shard DB get reassigned, so one primary has no DB.
 * Proxy makes a listing, gets a 404, tries another node. Likely, one of
   the other shard replicas responds. Things are fine.
 * Update comes in. Since we use the auto_create_account_prefix
   namespace for shards, container DB gets created and we write the row.
 * Proxy makes another listing. There's a one-in-three chance that we
   claim there's only one object in that whole range.

Note that unsharded databases would respond to the update with a 404 and
wait for one of the other primaries (or the old primary that's now a
hand-off) to rsync a whole DB over, keeping us in the happy state.

Now, if the account is in the shards namespace, 404 the object update if
we have no DB. Wait for replication like in the unsharded case.

Continue to be willing to create the DB when the sharder is seeding all
the CREATED databases before it starts cleaving, though.

Change-Id: I15052f3f17999e6f432951ba7c0731dcdc9475bb
Closes-Bug: #1881210
2020-06-03 13:26:31 -07:00
Zuul 9aced4db27 Merge "Switch to newer openstackdocstheme and reno versions" 2020-06-03 18:32:59 +00:00
Zuul 99e6a49208 Merge "Remove <py3.5 dependencies from requirements.txt" 2020-06-03 18:32:57 +00:00
Andreas Jaeger 3d105b623d Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems

Update Sphinx version as well.

Disable openstackdocs_auto_name to use 'project' variable as name.

Set openstackdocs_pdf_link to link to PDF file. Note that
the link to the published document only works on docs.openstack.org
where the PDF file is placed in the top-level html directory. The
site-preview places the PDF in a pdf directory.

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.

Remove docs requirements from lower-constraints, they are not needed
during install or test but only for docs building.

openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: I131850d2a5c6164dfd48c9c95885d4754b5236c6
2020-06-03 08:31:04 +02:00
Andreas Jaeger 019bade19c Remove <py3.5 dependencies from requirements.txt
The requirements repo is support python 3.5 as oldest python version
while swift still supports py27.

thus, requirements-check will fail on a couple of lines in swift.  The
check is only run when these files are touched.

The py2.7 packagers we know about aren't depending on upstream
requirements.txt for correctness and aside from all the production
deployments running on py2.7 we only realistically support >=py3.7

There's no good reason for our requirements.txt to be "unspported" by
the openstack requirements check job.  Since they only support >=py3.5
we can change our requirements.txt inline with that.  This should be
fine for everything we could hope to get out of both our
requirements.txt and the check!

Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com>
Change-Id: Ibf8000498528c401707be8b0b91b8355cd993786
2020-06-03 08:30:43 +02:00
Tim Burke 738514c164 docs: s3api doesn't support tagging
Change-Id: I227e8c2defe96fb5f33054167677ae551830d9e8
2020-06-02 21:51:00 -07:00
Zuul 45a9ca39a4 Merge "Enable s3api and staticweb tests across all func tests" 2020-06-02 22:25:15 +00:00
Zuul b26d208b61 Merge "Simplify wsgify()" 2020-06-02 19:53:58 +00:00
Thiago da Silva 20c6bdb71c Enable s3api and staticweb tests across all func tests
This patch removed the separate s3api, staticweb functional tests
gate jobs and added them across all other functional test jobs.

Change-Id: Ie1c606132a054defc2b3cc14a66031090e7b8449
2020-06-01 19:16:48 -07:00
Tim Burke 99947150dd func tests: work with etag-quoter on by default
Also, run the in-process encryption func tests like that.

Change-Id: I984ab8d1304d23b89589973950b10dda4aea0db3
2020-06-01 18:38:23 -05:00
Zuul 6c1bc3949d Merge "ratelimit: Allow multiple placements" 2020-06-01 21:25:41 +00:00
Clay Gerrard ede9dad9f6 Better functest quarantine cleanup
Change-Id: I9218aaeb5fcd21f1bc2a5d655e3216059a209aeb
2020-06-01 14:43:05 +00:00
Zuul 4851c11600 Merge "Ussuri contrib docs community goal" 2020-05-30 06:55:23 +00:00
Tim Burke cedec8c5ef Latch shard-stat reporting
The idea is, if none of

  - timestamp,
  - object_count,
  - bytes_used,
  - state, or
  - epoch

has changed, we shouldn't need to send an update back to the root
container.

This is more-or-less comparable to what the container-updater does to
avoid unnecessary writes to the account.

Closes-Bug: #1834097
Change-Id: I1ee7ba5eae3c508064714c4deb4f7c6bbbfa32af
2020-05-29 22:33:10 -07:00
Zuul 9b1ca0eb67 Merge "dsvm: Run service-user tests under keystoneauth" 2020-05-30 01:07:34 +00:00
Zuul 4f440ad44d Merge "dsvm: Run service-user tests under tempauth" 2020-05-28 00:27:40 +00:00
Zuul e545a9291f Merge "dsvm: Run domain_remap tests with tempauth" 2020-05-28 00:27:38 +00:00
Zuul 8f1a40eaae Merge "swift-dsvm: Enable s3api" 2020-05-28 00:27:36 +00:00
Tim Burke 60f052f69a dsvm: Run service-user tests under keystoneauth
Change-Id: I5b0ae0b78d37a31928e0b34a1b6118802a3a8236
2020-05-27 15:07:52 -07:00
Alexandre Lécuyer b3fd0bd9d8 obj: _finalize_durable may succeed even when data file is missing (LOSF)
Bring 69b8165cd8 to LOSF

Change-Id: I91c72a7a98468497ab323a920e9573fcec765b66
2020-05-27 23:52:09 +02:00
Tim Burke 984b57a873 tests: Skip s3api copy-version test when OV not enabled
Change-Id: I175b9e1464494454b1193b9b9d5311cb6fd6503f
2020-05-27 13:29:08 -07:00
Tim Burke 3aa4e3ec85 Remove swift-dsvm-functional-py3 job
Following https://github.com/openstack/devstack/commit/6b6bdc711,
the swift-dsvm-functional job has been running everything under
Python 3 anyway.

Change-Id: Ie285f513e1ed71dbaf4e12fe747ea6087664f843
2020-05-27 13:16:55 -07:00
OpenStack Proposal Bot 650272eacb Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I1bb2aa262730d45a5852eaa3b3351f455a6ab337
2020-05-27 10:03:59 +00:00
Zuul ee59909b65 Merge "Skip tests when only changing README.rst, CONTRIBUTING.rst, etc." 2020-05-27 04:30:54 +00:00
Tim Burke fa768b4342 Simplify wsgify()
Change-Id: Iec399aa8b58e72152a17265f2af1131f02667131
2020-05-27 03:19:13 +00:00
Tim Burke a2feefb045 dsvm: Use devstack's s3api "service"
...instead of hacking up the pipeline ourselves.

Depends-On: https://review.opendev.org/731003
Depends-On: https://review.opendev.org/731065
Change-Id: Iea8a42ef54e1a2fd9c1d6132c840a20015cc5d7e
2020-05-27 03:12:50 +00:00
Zuul 5d9c373618 Merge "versioning: Have versioning symlinks make pre-auth requests to reserved container" 2020-05-27 00:15:58 +00:00
Tim Burke 5034916c18 Skip tests when only changing README.rst, CONTRIBUTING.rst, etc.
Change-Id: Ib78398111ee3d150b6f56aa4e59a65750aa115dd
2020-05-26 16:42:52 -07:00
Matthew Oliver 78cce72f8a Ussuri contrib docs community goal
This patch standardizes the CONTRIBUTING.rst file and adds the
required doc/source/contributor/contributing.rst

Swift already had a detailed CONTIRBUTING.rst and an informative
REVIEW_GUIDELINES.rst in the root of the repo. So we are also pulling
them into the contributor documentation so they can not only be easily
found in the checked repo but in the online documentation.

Change-Id: I4c84efbe50eb25ab922c9d6b69198dae341af48b
2020-05-26 15:06:02 -07:00
Zuul 7b6546b076 Merge "fix s3api functional tests" 2020-05-26 20:36:06 +00:00