Update git submodules
* Update python-swiftclient from branch 'master'
- Merge "Add python 3.6 unit test job"
- Use template for lower-constraints
Small cleanups:
* Use openstack-lower-constraints-jobs template, remove individual
jobs.
* Sort list of templates
Change-Id: Idb31ca14478641cba6f896af35fa766d7bdb9e1e
Needed-By: https://review.openstack.org/623229
- Merge "Add delimiter to get_account()."
- Change openstack-dev to openstack-discuss
Mailinglists have been updated. Openstack-discuss replaces openstack-dev.
Change-Id: I3193f2d12f75c36b59881a51b605d25274b335e0
- Add delimiter to get_account().
Exposes the delimiter parameter, which the Swift API supports for
container listings.
Change-Id: Id8dfce01a9b64de9d1222aab9a4a682ce9e0f2b7
- Add Python 3.6 classifier to setup.cfg
Change-Id: If3b2cdcd009136286d68fe07b14e06261b3069a0
- update .functests to run stestr
Updated .functests script to run similar to how it's defined
in tox.ini
Change-Id: I17df28d8cbe0e10e48b26c2f9737308552ea88db
- Merge "Stop leaking quite so many connections"
- Merge "Update reno for stable/rocky"
- Update reno for stable/rocky
Change-Id: I840f4363dfdb3b485dbaf768c71fbcc5227c330f
- Stop leaking quite so many connections
While investigating the failures when you move func tests to py3, I
noticed a whole bunch of
ResourceWarning: unclosed <socket.socket ...>
noise. This should fix it.
While we're at it, make get_capabilities less stupid.
Change-Id: I3913e9334090b04a78143e0b70f621aad30fc642
Related-Change: I86d24104033b490a35178fc504d88c1e4a566628
- Add python 3.6 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.6 as part of the python3-first goal.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: Iae4acab507e45a379c8af129912e13621a2a553b
- Switch to stestr
According to Openstack summit session [1],
stestr is maintained project to which all Openstack projects should migrate.
Let's switch to stestr as other projects have already moved to it.
[1] https://etherpad.openstack.org/p/YVR-python-pti
Change-Id: Ic098f8560599554e0b6bb16ae326d4d30a8a5504
- Merge "Stop lazy importing keystoneclient"
- py2 functional testing
Change-Id: I24ff8fb28969a0b074313bc9491b299afac3b49c
- Merge "fix tox python3 overrides"
- Stop lazy importing keystoneclient
There were two basic problems:
- We'd try to import on every attempt at getting auth, even when we
already know keystoneclient isn't available.
- Sometimes devs would hit some crazy import race involving (some
combination of?) greenthreads and OS threads.
So let's just try the imports *once*, at import time, and have None
sentinels if it fails. Try both versions separately to decouple
failures; this should let us support a wider range of keystoneclient
versions.
Change-Id: I2367310aac74f1b7c5ea0cb1a822a491e4ba8e68
- Use Swift's in-tree DSVM test
While we're at it, make a new job that inherits from it to bring the
legacy-swiftclient-dsvm-functional testing in-tree, too.
For naming, follow naming policy and remove "dsvm" from names.
Remove legacy jobs, they are not needed anymore.
Change-Id: I919c0b77ac4888350194f55a9c12e0742845024f
Depends-On: https://review.openstack.org/589270
- fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: I86d24104033b490a35178fc504d88c1e4a566628
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
- Merge "Make py36 unit test job voting"
- Merge "Add release note link in README"
- Make py36 unit test job voting
Change-Id: I42cd4e19bba89c9dd4d7d20c75ee59217b9ea75d
- add python 3.6 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.6 as part of the python3-first goal.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: I6fd051fd0b01a308d16734c5b12e11a12a38c3be
Story: #2002586
Task: #24337
- switch documentation job to new PTI
This is a mechanically generated patch to switch the documentation
jobs to use the new PTI versions of the jobs as part of the
python3-first goal.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: I510e70f2222006df661c6a3d9e26af57b68be835
Story: #2002586
Task: #24337
- import zuul job settings from project-config
This is a mechanically generated patch to complete step 1 of moving
the zuul job settings out of project-config and into each project
repository.
Because there will be a separate patch on each branch, the branch
specifiers for branch-specific jobs have been removed.
Because this patch is generated by a script, there may be some
cosmetic changes to the layout of the YAML file(s) as the contents are
normalized.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: I59f4cbc0a21b8be3a1cae28a64f90d5adcf6be24
Story: #2002586
Task: #24337
- Add .idea into .gitignore
Generated by IDE PyCharm
Change-Id: Ifc99b34aae581221ae4b8d2533adfc21e91cd291
- Merge "Remove unnecessary calls to parse_header_string()."
- Merge "authors/changelog update for 3.6.0"
- Merge "Back out some version bumps"
- Remove unnecessary calls to parse_header_string().
Since we define the getheader() method on the response from
HTTPConnection, we don't have to call parse_header_string, as the values
will already be converted properly.
Change-Id: Ia81e8674b828b3ff1f014454126b469e41adfc23
- authors/changelog update for 3.6.0
Change-Id: I471d3d56d98915804aaf848f7ff98d91f586d572
- Merge "Properly handle unicode headers."
- Properly handle unicode headers.
Fix unicode handling in Python 3 and Python 2. There are currently two
failure modes. In python 2, swiftclient fails to log in debug mode if
the account name has a non-ASCII character. This is because the account
name will appear in the storage URL, which we attempt to pass to the
logger as a byte string (whereas it should be a unicode string). This
patch changes the behavior to convert the path strings into unicode by
calling the parse_header_string() function.
The second failure mode is with Python 3, where http_lib returns headers
that are latin-1 encoded, but swiftclient expects UTF-8. The patch
automatically converts headers from latin-1 (iso-8859-1) to UTF-8, so
that we can properly handle non-ASCII headers in responses.
Change-Id: Ifa7f3d5af71bde8127129f1f8603772d80d063c1
- Add close() to _RetryBody.
Allows clients to give up on reading the rest of the server response,
if they so choose.
Change-Id: Iccc95b1b5e7d066470966ee0c62a3beb260846e5
- Merge "Stop mutating header dicts"
- Merge "Drop py34 target in tox.ini"
- Add bash_completion to swiftclient
This patch basically follows the bash completion
model that other OpenStack clients use. It creates
a new command to swiftclient called `bash_completion`.
The `bash_completion` command by default will print
all base flags and exsiting commands. If you pass
it a command, it'll print out all base flags and
any flags that command accepts. So as you type out
your swift command and auto-complete, only the current
available flags are offered to you.
This is used by the swift.bash_completion script to
allow swift commands to be bash completed.
To make it work, place the swift.bash_completion file
into /etc/bash_completion.d and source it:
cp tools/swift.bash_completion /etc/bash_completion.d/swift
source /etc/bash_completion.d/swift
Because swiftclient itself is creating this flag/command output
it should automatically add anything we add to the swiftclient
CLI.
Change-Id: I5609a19018269762b4640403daae5827bb9ad724
- Merge "Treat 404 as success when deleting segments"
- Back out some version bumps
I'm giving up on trying to back out all of the test-requirements
up-revs, but let's try to stay compatibile with old requests/six.
As part of that, only disable some requests warnings on new-enough requests.
Note that we should now be compatible with distro packages back to
Ubuntu 16.04 and CentOS 6. Our six is still too new for Trusty, but
hey, there's less than a year left on that anyway, right?
Change-Id: Iccb23638393616f9ec3da660dd5e39ea4ea94220
Related-Change: I2a8f465c8b08370517cbec857933b08fca94ca38
- Add ability to generate a temporary URL with an
IP range restriction
Change-Id: I4734599886e4f4a563162390d0ff3bb1ef639db4
- Merge "Add option for user to enter password"
- Merge "Make OS_AUTH_URL work in DevStack by default"
- Add release note link in README
Change-Id: Ie282f4a731e333b5d21490a8fb59e1c7e8640821
- Merge "Remove some pointless code"
- Remove some pointless code
Change-Id: I3163834c330c5ea44c1096e83127588c88f0d761
- Make OS_AUTH_URL work in DevStack by default
An earlier change added support for versionless authurls, but the
huristic to detect them didn't work for some configurations I've
encountered.
Now we use a little bit tighter pattern matching and support auth_url
values with more than one path component.
Change-Id: I5a99c7b4e957ee7c8a5b5470477db49ab2ddba4b
Related-Change-Id: If7ecb67776cb77828f93ad8278cc5040015216b7
- Add option for user to enter password
Add the --prompt option for the CLI which will cause the user to be
prompted to enter a password. Any password otherwise specified by
--key, --os-password or an environment variable will be ignored.
The swift client will exit with a warning if the password cannot be
entered without its value being echoed.
Closes-Bug: #1357562
Change-Id: I513647eed460007617f129691069c6fb1bfe62d7
- Remove PyPI downloads
According to official site,
https://packaging.python.org/guides/analyzing-pypi-package-downloads/
PyPI package download statistics is no longer maintained and thus
should be removed.
Change-Id: I05e7d48d191bdaaf029f0ad1373a9c7c8b22e81e
- Merge "Use a valid default for auth_version"
- Merge "show option per line"
- Merge "Make swiftclient respect region_name when using sessions"
- Switch from oslosphinx to openstackdocstheme
openstackdocstheme is a theme and extension support for
Sphinx documentation that is published to docs.openstack.org
and developer.openstack.org.
Change-Id: I37d1d50fb88b35e72b017d5dfbf148c35ac7e323
- Make swiftclient respect region_name when using sessions
Change-Id: I94aca6f1120c34616562be7345f0e5aa51a69499
- Use a valid default for auth_version
The valid set of values for auth_version does not include
values starting with the 'v'.
In this particular function, the auth_version variable is
only used for comparisons with v3. So, the code worked
correctly. However, let's clean this up in order to reduce
review confusion and defuse possible future landmine in case
of code changes.
Change-Id: I671016d7992a1922b786b4eb8876b3fbb2532e15
- add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.
Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.
Add openstack-tox-lower-constraints job to the zuul configuration.
Update the dependencies needed to make the unit tests pass while
constrained to the lower bounds.
See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.
Co-Authored-By: Nguyen Hai <nguyentrihai93@gmail.com>
Change-Id: I2a8f465c8b08370517cbec857933b08fca94ca38
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
- Trivial: Update pypi url to new url
Pypi url changed from [1] to [2]
[1] https://pypi.python.org/pypi/<package>
[2] https://pypi.org/project/<package>
Change-Id: Ia406e9c8be6ba672b96e8584ef26f92348c8328b
- Merge "Remove trailing white space in tox.ini"
- Remove trailing white space in tox.ini
Change-Id: I706b69b7230390234ab255682478e8f69261cafe
- Remove py34 from envlist in tox.ini
py35 is enough.
Change-Id: Iebd7a6741dd60ed2fb11d1758bfec8e03e30a086
- show option per line
ading multiple options on the same line makes
it easy to miss when quickly scanning the options.
Change-Id: I8e324fca48cd05d9e381d5106135542274c2ff7f
Signed-off-by: Thiago da Silva <thiago@redhat.com>
- Add force auth retry mode in swiftclient
This patch attemps to add an option to force get_auth call while retrying
an operation even if it gets errors other than 401 Unauthorized.
Why we need this:
The main reason why we need this is current python-swiftclient requests could
never get succeeded under certion situation using third party proxies/load balancers
between the client and swift-proxy server. I think, it would be general situation
of the use case.
Specifically describing nginx case, the nginx can close the socket from the client
when the response code from swift is not 2xx series. In default, nginx can wait the
buffers from the client for a while (default 30s)[1] but after the time past, nginx
will close the socket immediately. Unfortunately, if python-swiftclient has still been
sending the data into the socket, python-swiftclient will get socket error (EPIPE,
BrokenPipe). From the swiftclient perspective, this is absolutely not an auth error,
so current python-swiftclient will continue to retry without re-auth.
However, if the root cause is sort of 401 (i.e. nginx got 401 unauthorized from the
swift-proxy because of token expiration), swiftclient will loop 401 -> EPIPE -> 401...
until it consume the max retry times.
In particlar, less time to live of the token and multipart object upload with large
segments could not get succeeded as below:
Connection Model:
python-swiftclient -> nginx -> swift-proxy -> swift-backend
Case: Try to create slo with large segments and the auth token expired with 1 hour
1. client create a connection to nginx with successful response from swift-proxy and its auth
2. client continue to put large segment objects
(e.g. 1~5GB for each and the total would 20~30GB, i.e. 20~30 segments)
3. after some of segments uploaded, 1 hour past but client is still trying to
send remaining segment objects.
4. nginx got 401 from swift-proxy for a request and wait that the connection is closed
from the client but timeout past because the python-swiftclient is still sending much data
into the socket before reading the 401 response.
5. client got socket error because nginx closed the connection during sending the buffer.
6. client retries a new connection to nginx without re-auth...
<loop 4-6>
7. finally python-swiftclient failed with socket error (Broken Pipe)
In operational perspective, setting longer timeout for lingering close would be an option but
it's not complete solution because any other proxy/LB may not support the options.
If we actually do THE RIGHT THING in python-swiftclient, we should send expects: 100-continue
header and handle the first response to re-auth correctly.
HOWEVER, the current python's httplib and requests module used by python-swiftclient doesn't
support expects: 100-continue header [2] and the thread proposed a fix [3] is not super active.
And we know the reason we depends on the library is to fix a security issue that existed
in older python-swiftclient [4] so that we should touch around it super carefully.
In the reality, as the hot fix, this patch try to mitigate the unfortunate situation
described above WITHOUT 100-continue fix, just users can force to re-auth when any errors
occurred during the retries that can be accepted in the upstream.
1: http://nginx.org/en/docs/http/ngx_http_core_module.html#lingering_close
2: https://github.com/requests/requests/issues/713
3: https://bugs.python.org/issue1346874
4: https://review.openstack.org/#/c/69187/
Change-Id: I3470b56e3f9cf9cdb8c2fc2a94b2c551927a3440
- Merge "Add a query_string option to head_object()."
- Add a query_string option to head_object().
Submitting a path parameter with a HEAD request on an object can be
useful if one is trying to find out information about an SLO/DLO without
retrieving the manifest.
Change-Id: I39efd098e72bd31de271ac51d4d75381929c9638
- Update links in README
Change the outdated links to the latest links in README
Change-Id: Ic0adc686592265f2be2acb55f0520c35d1717f76
- Merge "Update the old http doc links"
- Update the old http doc links
Update the old http doc links to the https ones according to the
official OpenStack website.
Change-Id: Ibf9ecbccb743d2b9a678a1ca69f0b3adc9106a12
- Update reno for stable/queens
Change-Id: I7f7b21e3dd0d1ef71c159e2b72f9ad9f963b0773
- Treat 404 as success when deleting segments
Change-Id: I76be70ddb289bd4f1054a684a247279ab16ca34a
- authors/changelog updates for 3.5.0 release
Change-Id: I70b79c0fd6e9adbfdcc799459dc52063c7402be2
- Allow for object uploads > 5GB from stdin.
When uploading from standard input, swiftclient should turn the upload
into an SLO in the case of large objects. This patch picks the
threshold as 10MB (and uses that as the default segment size). The
consumers can also supply the --segment-size option to alter that
threshold and the SLO segment size. The patch does buffer one segment
in memory (which is why 10MB default was chosen).
(test is updated)
Change-Id: Ib13e0b687bc85930c29fe9f151cf96bc53b2e594
- Revert "Add Constraints support"
Per http://lists.openstack.org/pipermail/openstack-dev/2017-December/125348.html
> For many projects, tox_install.sh is not needed at all
Let's see if that holds for python-swiftclient!
This reverts commit f2f278fcbec3ad52a1726bb5a3f775d13bcc99dc.
Change-Id: I0462c50ec71d87bac226f83a0d0942871ef5a0e7
- Merge "Allow --meta on upload"
- Make tox runnable in a directory with spaces
I noticed a disturbing lack of quote-wrapping in change
I7cb4b44952713752435e1faf0f63bf0d37e7dda6 but as I poked at it, I
realized that trouble runs rampant.
This seems to clean it all up, though I haven't tested *every*
environment we define.
Change-Id: I1454eb113e5bd9125d39f2e57e2ed96f6ddc42fc
- Merge "Update tox_install.sh to align for sphinx jobs"
- Update tox_install.sh to align for sphinx jobs
The updates to the sphinx docs jobs in support of the updates to
the PTI wound up exposing an unintended interface. There are two flavors
of the tox_install.sh file out there, and we basically need to collapse
them into one flavor.
Update the tox_install.sh script to match the
constraints-as-first-argument form.
Change-Id: I7cb4b44952713752435e1faf0f63bf0d37e7dda6
- Remove setting of version/release from releasenotes
Release notes are version independent, so remove version/release
values. We've found that projects now require the service package
to be installed in order to build release notes, and this is entirely
due to the current convention of pulling in the version information.
Release notes should not need installation in order to build, so this
unnecessary version setting needs to be removed.
This is needed for new release notes publishing, see
I56909152975f731a9d2c21b2825b972195e48ee8 and the discussion starting
at
http://lists.openstack.org/pipermail/openstack-dev/2017-November/124480.html
.
Change-Id: I623fe918c1e4ddafa93efc91ed550a365cec1cf0
- Stop mutating header dicts
Change-Id: Ia1638c216eff9db6fbe416bc0570c27cfdcfe730
- Allow --meta on upload
Previously, the --meta option was only allowed on post or copy subcommands.
Change-Id: I87bf0338c34b5e89aa946505bee68dbeb37d784c
Closes-Bug: #1616238
- Drop py34 target in tox.ini
We support py35 now.so it is no need to keep
the supoort for py34.
Change-Id: Ie76e897bea3c184410e2b151fbe978d93bc21624
This commit is contained in:
Zuul
Gerrit Code Review
parent
8200c6aff4
commit
20e049e6a7
|
|
@ -1 +1 @@
|
|||
Subproject commit 3be8eca70b53148f2dbcbab98dafd04b07e99da6
|
||||
Subproject commit 4b99492dc521886f949bff757580497160b3228d
|
||||
Loading…
Reference in New Issue