This commit is part of a series to retire the Packaging Deb
project. Step 2 is to remove all content from the project
repos, replacing it with a README notification where to find
ongoing work, and how to recover the repo if needed at some
future point (as in
https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project).
Change-Id: I4de52b8b818225083f85fc144fc2963992bdd5e2
Sphinx wants babel!=2.0,>=1.3 which drags in 2.4.0, but oslo.i18n
(required by keystone) wants Babel!=2.4.0,>=2.3.4 leading to an
error when starting keystone.
Also, bump up the sleep before verifying that keystone is running
-- at just one second, keystone hadn't had a chance to raise the
ContextualVersionConflict yet. Further, verify that curl can actually
reach keystone on the expected port.
Change-Id: I6cbbfd7260571f42ea65c6622aa6b410a0e43b28
This allows (some) auth middlewares to stop needing to know the details
of S3 request signing. The function takes the user's secret and returns
True if the signatures match, False otherwise.
This offers an easy way for auth middlewares that have user secrets
in-memory (such as swauth and tempauth) to add support for v4 signatures.
Change-Id: Iafb6114c12deb9a40d0f8324611de27b48ed95f6
TrivilalFix
Following OpenStack Style Guidelines:
[1] http://docs.openstack.org/developer/hacking/#unit-tests-and-assertraises
[H203] Unit test assertions tend to give better messages for more specific
assertions. As a result, assertIsNone(...) is preferred over
assertEqual(None, ...) and assertIs(..., None)
Change-Id: Ia4e28fbcb836df9f0bebe2734bceb0e2ca98a6b8
Keystone master currently responds with v3 tokens. Preserve v2 support
for stable/newton and earlier Keystones. Note that stable/ocata simply
doesn't work.
Related-Change: I5d0c18121ba4bf8e33209daa48b9d87864951362
Change-Id: I747de516ab69a47622eecbf8ab3faa34444b3ad5
This prevents version conflicts when Keystone drags in a too-new
oslo.policy, which blackballs latest requests (but Sphinx would try to
pull it in anyway).
Change-Id: I3b2196fdef9936f1c7d468f9c7c9b9246d3d26fd
Previously, we would use client-accessible headers to pass the S3 access
key, signature, and normalized request to authentication middleware.
Specifically, we would send the following headers:
Authorization: AWS <access key>:<signature>
X-Auth-Token: <base64-encoded normalized request>
However, few authentication middleware would validate that the
Authorization header actually started with "AWS ", the only prefix that
Swift3 would actually handle. As a result, the authentication
middlewares had no way to validate that the normalized request came from
swift3 rather than the client itself. This leads to a security hole
wherein an attacker who has captured a single valid request through the
S3 API or who has obtained a valid pre-signed URL may impersonate the
user that issued the request or pre-signed URL indefinitely through the
Swift API.
Now, the S3 authentication information will be placed in a separate
namespace in the WSGI environment, completely inaccessible to the
client. Specifically,
environ['swift3.auth_details'] = {
'access_key': <access key>,
'signature': <signature>,
'string_to_sign': <normalized request>,
}
(Note that the normalized request is no longer base64-encoded.)
UpgradeImpact
This is a breaking API change. No currently-deployed authentication
middlewares will work with this. This patch includes a fix for s3_token
(used to authenticate against Keystone); any deployers still using
keystonemiddleware to provide s3_token should switch to using swift3.
Similar changes are being proposed for Swauth and tempauth. Proprietary
authentication middlewares will need to be updated to use the new
environment keys as well. When upgrading Swift3, operators will need to
upgrade their Swift3-capable authentication middleware at the same time.
Closes-Bug: 1561199
Change-Id: Ia3fbb4938f0daa8845cba4137a01cc43bc1a713c
Depends-On: Ib90adcc2f059adaf203fba1c95b2154561ea7487
Use:
* v3 auth
* keystone-manage bootstrap
* uwsgi
... which all brings us loads closer to Keystone's current documented
developer setup.
Change-Id: Id7a31545e6bfb0da900b1456d7baa449636f12d7
Previously, s3token would make one request to Keystone to validate the
request signature provided by the user, then use the response to add an
X-Auth-Token header to the request environment. This would get picked up
by the authtoken middleware which would make *another* request to
validate the token *we just got*.
Now, we'll populate the request environment with the following headers:
* X-Identity-Status
* X-Roles
* X-User-Id
* X-User-Name
* X-Tenant-Id
* X-Tenant-Name
* X-Project-Id
* X-Project-Name
This allows Swift's keystoneauth middleware to function without needing
the authtoken middleware at all.
UpgradeImpact
-------------
The recommended pipeline ordering has changed. Whereas authoken previously
had to be between s3token and keystoneauth like
... swift3 s3token authtoken keystoneauth ...
it should now be placed before swift3, as in
... authtoken swift3 s3token keystoneauth ...
Alternatively, if Keystone users should only ever access Swift through
the S3 API, the authtoken middleware may be removed entirely. Note that
the old pipeline ordering will continue to work, but still requires two
Keystone requests per client request as before.
To upgrade an existing cluster to take advantage of this change
operators should, for each proxy server:
1. Upgrade swift3
2. Optionally, restart proxy-server
3. Update proxy-server.conf with the new pipeline
4. Restart proxy-server
Updating proxy-server.conf *before* upgrading swift3 will prevent the
proxy from starting if there is an unexpected reboot before the upgrade.
Closes-Bug: #1653017
Change-Id: I21e38884a2aefbb94b76c76deccd815f01db7362
Previously, if a user-provided timestamp was after the largest-possible
Swift timestamp, we would raise AccessDenied. However, AWS continues to
complain about time skew.
Note that we may regret this come 2286, but by then I'll be dead.
Change-Id: I88952a28a7e7c42540c61514f82582815fabf611
We are already passing arguments to nosetests for non ceph functional
tests, this change does the same for run_ceph_tests.py.
Change-Id: I7c54fddb98871ed3a6437a457dcf775d91f0ed45
Swift has removed the minimum segment size setting for multipart upload.
To make it compatible with S3, we are re-implementing it in swift3.
Each upload part except the last should be more than the minimum segment
size (default 5MB, same as the S3 multipart upload chunk size). When a
"complete multipart upload" request comes, check all the parts and
return a EntityTooSmall error if they are smaller than the minimum
segment size.
Change-Id: I883b25ab3d43d330ffc60fa2c3ade7a6b5802cee
After running "tox -e s3tests_tempauth", there are python files from
ceph-tests in swift3/test/functional/scratch directory that fail flake8
so we should just ignore the scratch directory.
Change-Id: I7f0e89c4da7590d5929b2f65a81fc6ddeb0ffe37
Previously, when clients sent non-printable characters in headers, we
might reply with InvalidDigest or some other flavor of 400 while AWS
would have sent a 403 SignatureDoesNotMatch in a similar situation.
See, for example, the removed known failures for ceph/s3-tests.
Additionally, factor out canonical request from string-to-sign for
SigV4Mixin. This simplifies the diagnosing of certain unit test failures.
Change-Id: I703d3db134b8e2202d271eca018b920fbedd08e7
run_test.sh doesn't use any arguments at the moment. Any arguments
passed to run_test.sh can be passed to nosetests so it is possible to
run single tests or use nosetests options.
Change-Id: I22ae0aaf6266ecbd8e80025c21479f01a54df23e
Apparently long names cause issues when the job is running in the gate?
Change-Id: I4b9def2ce867580fb0f5b6b64818eef6d65b7a43
Related-Change: I80a7a32a415c27ac9de9d72591ed293ac8546cba
...because the alternative is to add pins for one or more of
* osc-lib
* os-client-config
* cliff
* whatever else...
Also drop the unused OS_* env vars, as they make more-recent
python-openstackclient barf with
TypeError: __init__() got an unexpected keyword argument 'project_name'
Change-Id: Ibfda43cfc28b5aa6530d4ab3a87a7bc696b9ca7c
Otherwise, requests may wait forever for a response.
Now, we will wait at most 10 seconds by default, and allow operators to
adjust that to between 0 and 60 seconds.
This option closely mirrors the http_connect_timeout option in
Keystone's authtoken middleware.
Change-Id: I43fe784551abe6de790c781d0addfa25519a1f55
pbr won't let development versions install otherwise.
Also set up some build_sphinx configuration to mimic what Swift does.
This seems to be what's causing the lingering docs failure, though I'm
not sure why it wasn't a problem before.
Change-Id: I2e4c416a21d98d10377b7d2424bac93a139220fa
Prior to Python 2.7.4 [1], this would convert the input to a local
timestamp, then adjust for both the local timezone and the timezone of
the input. Ordinarily, this would be fine (excluding, apparently, some
"argument out of range" issues on Windows).
However, Swift (since v1.9) manually adjusts the TZ environment
variable, apparently with the intention of making the timezone static
and avoiding extra overhead from checking /etc/timezone for changes.
In practice this sets TZ to "+0000" which has the effect of setting the
timezone to UTC (at least for some functions, such as time.localtime and
time.mktime) while *not* changing the offset stored in time.timezone.
This, in turn, causes email.utils.mktime_tz to produce bad timestamps
and swift3 to reject requests with RequestTimeTooSkewed errors if the
server was not in UTC.
Now, we'll avoid local timestamps by using calendar.timegm ourselves,
essentially inlining the upstream Python fix.
Note that Ubuntu Precise provides Python 2.7.3 (and thus *is* affected);
neither Ubuntu Trusty (which provides 2.7.6) nor CentOS 7 (which
provides 2.7.5) is affected.
[1] https://hg.python.org/cpython/rev/a283563c8cc4
Change-Id: Iee7488d03ab404072d3d0c1a262f004bb0f2da26
Related-Change: I007425301914144e228b9cfece5533443e851b6e
Related-Change: Ifc78236a99ed193a42389e383d062b38f57a5a31
Related-Change: I8ec80202789707f723abfe93ccc9cf1e677e4dc6
Closes-Bug: 1593863
Tony Breeds