Commit Graph

5851 Commits

Author SHA1 Message Date
OpenDev Sysadmins 187b5cb14f OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:28:26 +00:00
Jenkins 76ea6e4b53 Merge "crypto - Test with non-zero offset" into feature/crypto 2016-06-16 21:15:43 +00:00
Mahati Chamarthy 215d106c05 crypto - Test with non-zero offset
Change-Id: Iab8545db0ba09de6d9d5e9d28b4cb999da5f4c94
2016-06-16 17:18:21 +01:00
Tim Burke c944e73d43 Tighten up some sysmeta purging
Drop all container-update overrides on COPY, as that seems more
consistent than selecting just a couple overrides to drop.

Check that all `actual` headers are also `retained_headers` in
test_purge_crypto_sysmeta_headers.

Check that we really did purge the crypto meta in test_GET_success and
test_HEAD_success. GETs were already getting verified over in
test_encrypter_decrypter.py, but they would fail in a less-obvious way.

Change-Id: I7c17fdc02a9895495a1737d4040e868482bb9e98
2016-06-08 10:55:45 -07:00
Jenkins d420d03273 Merge "crypto - refactor footers callback" into feature/crypto 2016-06-08 14:40:24 +00:00
Jenkins be9b57f413 Merge "crypto - purge crypto sysmeta from responses" into feature/crypto 2016-06-08 14:40:10 +00:00
Jenkins f60beaa349 Merge "crypto - remove stale TODOs" into feature/crypto 2016-06-08 10:13:08 +00:00
Alistair Coles 73b91e4ac8 crypto - remove stale TODOs
We do not need to test the contaiher listing etag length before
decrypting - if it has crypto-meta it will be decrypted.  The Etag
value in a response to a ranged GET should be the Etag of the whole
object.

Change-Id: Ib713a899b5d65d979d51db36cbca809998e87847
2016-06-07 18:00:21 +01:00
Alistair Coles 73ca61afbe crypto - remove unnecessary changes
Removes some changes that are no longer needed but
were left over when other code got removed.

Change-Id: I17bd5e84b750ef2a7318a3fae098a2fabe585329
2016-06-07 15:11:22 +01:00
Alistair Coles ce0e16bcc5 Merge branch 'master' into feature/crypto
Conflicts:
	swift/obj/server.py
	test/probe/test_object_async_update.py
	test/unit/obj/test_server.py

Change-Id: Ifeba1cfe8c4244eff069ab10b2c77f8ea5e2a39b
2016-06-06 15:51:52 +01:00
Jenkins 0d15c41060 Merge "crypto - stop using key id to derive GET path keys" into feature/crypto 2016-06-06 11:39:03 +00:00
Jenkins e80a5a0d01 Merge "Add Expires header for successful GETs using tempurls" 2016-06-04 06:16:41 +00:00
Jenkins 0a5c9af5c5 Merge "crypto - add overview doc" into feature/crypto 2016-06-03 19:44:18 +00:00
Jenkins 34da482ca0 Merge "crypto - make some probe tests compatible with crypto" into feature/crypto 2016-06-03 19:43:22 +00:00
Jenkins 93dca0eca3 Merge "Imported Translations from Zanata" 2016-06-03 17:35:03 +00:00
Alistair Coles c689acd2f1 crypto - add overview doc
Change-Id: Ia249583f545d7c47eb1c7236817c673e94b092ed
2016-06-03 17:40:17 +01:00
Alistair Coles 1d1859d577 crypto - purge crypto sysmeta from responses
Remove all crypto related (transient-) sysmeta from responses so that
they are not inadvertantly copied to another object. If a COPY is to
an unencrypted destination then the source crypto meta would not apply
and could result in corrupted data when reading the destination.

Also remove container update override headers when copying an object
and the source etag is not trusted (e.g. Range copy or large object
copy).

Drive-by change to stop FakeSwift updating request headers with etag
and content-length that were not in the actual PUT request. That
change was made earlier on this branch to support tests that have now
been deleted.

Change-Id: Ib7cf7676a2f62f79b067f93aa1d2cae64c393fe9
2016-06-03 17:15:01 +01:00
Alistair Coles 343d81673c crypto - refactor footers callback
Move some lines in footers_callback. Re-use the
already calculated encrypted-plaintext-etag for container
update override for efficiency rather than re-calculating,
since this is probably the common case.

Drive-by fix for some formatting in test_encrypter.

Change-Id: I2c881e98577a9f2c826b977f766398c29f63e565
2016-06-03 11:34:03 +01:00
Alistair Coles e6898c2a8d crypto - stop using key id to derive GET path keys
Currently the key id saved in X-Object-Sysmeta-Crypto-Id is only set
during a PUT request. The sysmeta cannot be set by a POST request. The
key id value is the path used to derive keys and is currently used by
the keymaster to determine what keys should be for decrypter to use in
the GET path. So if we have an object that was PUT without crypto and
then POSTed to with crypto, we have no X-Object-Sysmeta-Crypto-Id
value to derive keys from and cannot therefore decrypt the POSTed
metadata.

We do not actually need to use X-Object-Sysmeta-Crypto-Id to derive
keys - objects do not change location. So in this patch we stop using
it for key derivation in the GET and HEAD paths and instead just use
the path of the GET/HEAD request.

The key id may prove useful in the future if for example an object
were ever relocated, or a keymaster chose a different key with
different id for a POST vs the original PUT. If that were to happen
then we could use the persisted key id to inform the keymaster as to
which key is required for decryption.

For such future-proofing this patch changes the location for
persisting the key id to the crypto_meta dicts for each encrypted
item, and no longer uses the X-Object-Sysmeta-Crypto-Id header. That
way, if ever a different key id were used by a keymaster for a POST vs a
PUT then we'd have the right value stored with the metadata
crypto-meta vs the data crypto-meta.

To enable that, the keymaster now includes the key id in the dict of
results returned from fetch_crypto_keys, so that the encrypter can
embed the key id in crypto meta. The key id is also changed to be a
dict that contains the key derivation path and also a version number
for the keymaster. This is purely for future-proofing - we now persist
sufficient information in crypto-meta to determine which keymaster
version was used and how it derived the keys.

This changes also allows some simplification of the keymaster, and now
the keymaster only calculates keys when they are asked for by a call
to fetch_keys.

In addition, the keymaster will never set the 'swift.crypto.override'
flag anymore. There are still encrypter and decrypter unit tests to
handle the case where somebody uses such a keymaster, but this one
will not make use of that flag.

Change-Id: Icb368305a15e1bbe32483f2e2bbb98a1441a4dad
2016-06-03 11:20:23 +01:00
Jenkins fa94d19ffe Merge "Remove unused (but defaulted) args" 2016-06-03 06:49:24 +00:00
OpenStack Proposal Bot e9c9e1870b Imported Translations from Zanata
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: I0e061ecba6dc215d0d3b12aac14b8d4b08fbc682
2016-06-03 06:49:09 +00:00
Thiago da Silva 886fa0822a update pyeclib and liberasurecode links
Change-Id: Ic6d04083618362778363fea1570caaa865e44557
Signed-off-by: Thiago da Silva <thiago@redhat.com>
2016-06-02 23:03:26 -04:00
Jenkins 9fc32661d4 Merge "Add content-disposition header to tempurl request" 2016-06-02 21:06:21 +00:00
Jenkins 76de2dd0af Merge "pids in probe is no longer used" 2016-06-02 19:22:20 +00:00
Sivasathurappan Radhakrishnan 6f78646ea2 Add content-disposition header to tempurl request
Added content-disposition header to HEAD tempurl request.
As per HTTP docs[1] HEAD response must be identical to GET
except return message-body response.

[1]https://tools.ietf.org/html/rfc2616#section-9.4

Change-Id: Ie60a6fb632613055da5279db5b128ce5ee5172ae
Closes-Bug:#1539805
2016-06-02 18:03:51 +00:00
Tim Burke 6b0e9a3e24 Remove unused (but defaulted) args
Every time we call start_server, check is True.
Every time we call check_server, we use the default timeout.

Change-Id: Id38182f15bcbfbb145b57cee179a8fd47ec8e2b7
2016-06-02 16:49:32 +00:00
Alistair Coles 6557792683 crypto - remove default root secret
Make it a requirement that root_encryption_secret is
a minimum of 44 base64 encoded characters i.e. a minimum
of 32 bytes encoded to base 64.

This patch still has a default root_secret in the proxy
config.  This is just as temporary as the proxy config
having encryption middleware in the pipeline.  Without this,
functests would fail since devstack has no root secret defined.

Change-Id: I82d183f0b89bfd730578bb64623928bcbfaf657c
2016-06-02 13:10:48 +01:00
Alistair Coles 766f4dc2ae crypto - add tests for override headers
adds a test for other middlewares setting override headers,
verifying that container listing is correctly updated.

Drive-by fix to a doc string, and adding etag to test PUT requests.

Change-Id: Id096bd5bece339e2bcd32f4c545fb3aa7aa2b659
2016-06-02 09:51:09 +01:00
Alistair Coles 79d401033d crypto combine crypto.py with crypto_utils.py
...and move crypto_utils.py to swift/common/middleware

Also delete unused method and remove some unnecessary
mocking from test_decrypter.py

Change-Id: Ia4a2699db53eb4753c7f73db18fc86c84535b344
2016-06-02 08:55:57 +01:00
Kota Tsuyuzaki e56a1a550a pids in probe is no longer used
Change-Id: I1fd76004257a8c05ce8bb1f3ca0e45000509f833
2016-06-01 23:53:35 -07:00
OpenStack Proposal Bot 99186aded9 Imported Translations from Zanata
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: Iaf727e51ab444e8cb6199fda45595fc69497ea2d
2016-06-02 06:41:44 +00:00
Jenkins 3f00170825 Merge "Fix swift-dispersion in multi-region setups" 2016-06-02 04:57:41 +00:00
Jenkins f46a251b0d Merge "Adding sorting_method to admin_guide" 2016-06-02 04:01:52 +00:00
Jenkins 2a0935e9e3 Merge "Send correct size in POST async update for EC object" 2016-06-01 22:15:31 +00:00
Jenkins e24838afc7 Merge "Merge branch 'master' into feature/crypto" into feature/crypto 2016-06-01 21:11:37 +00:00
Jenkins 37d9750965 Merge "crypto - remove version from derived iv_base" into feature/crypto 2016-06-01 20:36:16 +00:00
Nelson Almeida daae74ca65 Adding sorting_method to admin_guide
Change-Id: I1162f154e3a577a95f9f5ea0e0f723b7df5a4baf
2016-06-01 17:29:10 -03:00
Janie Richling 83add8b84b crypto - remove version from derived iv_base
The version part is NOT included in the hash path for an object,
and should similarly not be included in the path used for a
derived IV, so that if/when the API version changes,
the derived IV would not change.

Change-Id: Idc527f9f056adb5b3c8c01135bb993b05b2c242b
2016-06-01 11:10:42 -05:00
Jenkins bd5adc9320 Merge "crypto - add test to verify ondisk container listing" into feature/crypto 2016-06-01 14:03:36 +00:00
Alistair Coles 03b8b4bfa9 Merge branch 'master' into feature/crypto
Change-Id: I5a09de094fca043a41d5080513d582390291d1c4
2016-06-01 14:40:58 +01:00
Félix Cantournet 4f053c61d6 Fix swift-dispersion in multi-region setups
If you have 2 swift regions served by the same keystone,
then the client cannot get the correct URL for the swift endpoint
without specifying a region_name.

Closes-Bug: 1587088
Change-Id: Iaab883386e125c3ca6b9554389e63df17267a135
2016-06-01 15:35:47 +02:00
Alistair Coles ff8100402a crypto - add test to verify ondisk container listing
...also make versions of the test for replication and EC policies.

Change-Id: Icb7c52b68001aec088609c4be49c3c28b0ecc4a0
2016-06-01 12:37:03 +01:00
Alistair Coles 2cec70530b crypto - use random iv when wrapping body key
Change-Id: Ia32a7b1cbafd5f593d0609310e4a38de6c52f220
2016-06-01 11:45:40 +01:00
Jenkins ae0149811d Merge "Don't include holes when reporting how many devices a ring has" 2016-06-01 09:41:12 +00:00
Jenkins bbb837de48 Merge "add explicit HA info to the deployment guide" 2016-06-01 09:38:22 +00:00
John Dickinson 86e9e827ba add explicit HA info to the deployment guide
Change-Id: I7614952c523080fe50eaf839b54a8064439817ce
2016-05-31 11:27:43 -07:00
Ondřej Nový 5fe392b562 Fixed typo
Change-Id: I7a35c0076360c7a23cf405189828d3c252ec6708
2016-05-31 16:25:50 +02:00
Jenkins 4728e3e8d3 Merge "crypto: Verify on-disk data" into feature/crypto 2016-05-31 12:46:55 +00:00
Jenkins 6b8305b949 Merge "crypto: use derived iv for container listing etag" into feature/crypto 2016-05-31 10:42:00 +00:00
Mahati Chamarthy 1c0276e87d crypto: Verify on-disk data
Tests that on-disk data, metadata and etag values are encrypted

Change-Id: I66ed5522f38f4219671be42f0607fda5cec0b8fd
2016-05-31 11:36:28 +01:00