To make bin/swift importable for testing it needs to be renamed/moved.
The final idea is to move it to swiftclient/shell.py and add only a stub
in bin/swift to import swiftclient/shell.py.
However, doing this in a single step makes reviewing very hard, because
now there are > 1400 lines deleted from bin/swift and added to
swiftclient/shell.py - Gerrit doesn't detect the moved file.
For example: https://review.openstack.org/#/c/73710/2/
This patch first moves the code to swiftclient/shell.py and uses
setup.py/cfg to create the stub file in bin/swift. A follow up
(https://review.openstack.org/#/c/76489/) will then add the stub itself
in bin/swift (and remove the entry in setup.py).
The change to tox.ini is related to bug 1290562 and can be removed in
the future.
Change-Id: Id86392ccaf09e1d80a5d808fb49827d6d18c2588
Make help strings more consistent:
* Fix wording and grammar, remove duplicated words
* Add "." consistently everywhere.
Change-Id: I37f94600df2a58532633f6bab2ab82fc5e91a75b
Currently, httplib implementation does not support SSL certificate
verification. This patch fixes this. Note that ssl compression parameter
and 100-continue thing is still missing from requests, though those are
lower priority.
Requests now takes care of:
* proxy configuration (get_environ_proxies),
* chunked encoding (with data generator),
* bulk uploading (with files dictionary),
* SSL certificate verification (with 'insecure' and 'cacert' parameter).
This patch have been tested with requests 1.1.0 (CentOS 6) and requests
2.2.1 (current version).
Change-Id: Ib5de962f4102d57c71ad85fd81a615362ef175dc
Closes-Bug: #1199783
DocImpact
SecurityImpact
swift has already the upload option "--changed" to upload only changed
files. This patch adds a similar feature by comparing the md5 of the
local file to the remote object etag.
When used in combination with download the MD5 hexdigest of each file
is sent with an "If-None-Match" header to skip downloading identical files.
When used in combination with upload the MD5 is compared to the remote
etag by using the already existing HEAD request.
Change-Id: I727b0456558c6a7742b2428c6d1c45c4bfaf66e9
This patch adds a capabilities option on swiftclient.
This option uses the new /info endpoint to request the
remote capabilities and nicely display it.
Change-Id: Ie34b454511d5527e402e66e1fdb72120f427f2fd
Add parameter --object-name, which:
1) Sets target object name when upload single file
2) Sets object prefix when upload a directory
Change-Id: Idc4357c3355e66d31c100540b901e70db20b03c3
Closes-Bug: 1012979
Some positional arguments are optional but the help indicate
they are mendatory and can be confuse for an user.
swift delete <container> [object]
swift post <container> [object]
Change-Id: I7be525905b51d3ef160b0f47194e817b5cba0f9b
We noticed a difference in download bandwidth while using
authentication details and pre-auth information for download.
Download using authentication details included authentication
time into the download bandwidth at each call of the function.
This time should have been excluded from download bandwidth.
This patch set is adding a timer variable to the client library.
That allows to accurately measure and store the time of each action.
Then while printing the download bandwith in the swift command
authentication time is excluded from bandwidth.
Change-Id: I63df9023e169f637f120ae9e25dac9c90a4e75a0
Fixes: bug #1197443
This fixes bug #1051046
A repeatable option, --header or -H, is added so a user can specify
custom headers such as Range or If-Modified-Since when downloading
an object with the swift CLI.
Change-Id: I1f7dcf64cf625f2e5a4488c210894cfe6e0d5974
* log to debug for successes and info for errors
* remove dead code, neither body or raw_body are sent in kwargs
* include resp.reason and response headers in logging
* remove trailing \n in logging. users who want them can set logging.basicConfig(format=FORMAT)
* add --info option to swift cli
Change-Id: If07af46cb377f3f3d70f6c4284037241d360a8b7
This lets you build python-swiftclient packages that don't require pbr
to be installed at all. You would need pbr on the machine running
rpmbuild / debuild, but not on the machines that install the packages.
Unfortunately, this does not make python-swiftclient able to be
installed via pip 0.3.1 on Lucid; you'll need to uninstall the system
python-pip package and install a new pip some other way. Given that
pip < 1.3 doesn't perform SSL certificate validation for pypi (trivial
MITM attack, anyone?), you'd probably want to get a new pip anyway.
Change-Id: I85d4d77aacf094e48d39e48e750594b95dbc7af0
When you stat a container or object with the verbose flag the full path of the
reousrce will be displayed with the token similarlly to how an account stat
displays the auth url and token.
* move some logic out of bin/swift.st_stat to test it
* new module swiftclient.commnad_helpers for code you want to test
* moved prt_bytes into swiftclient.utils to test it
* fixed IndexError with prt_bytes on sizes >= 1024Y
Change-Id: Iaaa96e0308b08c554205b0055b8a04de581fefa4
This commit fixes a problem where headers passed through the option
--header were not taken into account when doing "post" queries.
Change-Id: Ia34808c53a1816d9956a98e0803d8c4d20d2ad44
Closes-Bug: #1226256
Make it clear that you do not have to provide either the container or
object positional parameters, and what swiftclient will do if you omit
them.
Change-Id: I3766fc1eabd51f197e2175771af579577a96c192
Reformatted the help outputs in the swift client to bring them more
in line with the rest of the openstack clients. Changes were added to
all of the st_command_help variables and a new variable
st_command_options was added to help with correct formatting of
help outputs.
Change-Id: I0c549cf96292384ddb36b43ecec41a883d8b2e13
Implements: changes to help outputs
This patch extracts the multi-threading code from bin/swift into
swiftclient/multithreading and adds tests. In particular, this new way
of doing it (with context managers) will prevent non-daemonic threads
from wedging the process when unexpected exceptions happen.
I enabled reporting of which lines, specifically, are not covered by
unit tests (added -m option to "coverage report" in .unittests).
This patch includes a drive-by fix for uploading a segmented file with
--use-slo when that object already exists. A key of "name" was used
instead of "path", raising KeyError.
There's also another drive-by fix for uploading segmented objects with
--use-slo. Commit 874e0e4427 regressed
this by removing the capturing of thread-worker results in
QueueFunctionThread.run(). This patch restores that functionality and
the feature (uploading SLO objects).
Change-Id: I0b4f677e4a734e83d1a25088d9a74f7d46384e53
When --os-storage-url is specified on the command-line to bin/swift, it
will override the used storage URL regardless of authentication for both
authentication version 1 and version 2. This can be used to bypass a
load-balancer to hit a specific proxy server for testing/debugging
purposes.
Within the client library, this feature is accessed by passing the
desired storage URL into swiftclient.client.Conection.__init__() via the
os_options keyword argument. For example:
conn = Connection(auth_url, user, key, os_options={
'object_storage_url': 'http://overridden.storage.url/AUTH_foo'})
This patch also adds a dependency on mock>=0.8.0, which is the same as
openstack/swift.
Change-Id: Id2a36ed6abffd65e7762b6beea5bbfc6c036e848
Allow the ability to download a subset of containers (--all with -p) or
a subset of objects within a container (container name with -p).
This patch also includes a drive-by fix for "download --all" which would
not actually download any objects (for me, at least) because the object
queue got filled with "stop" messages before the container workers had
run long enough to put work in the object queue. Doh!
I also closed up a few holes where an (unexpected, obviously) Exception
could cause the process to hang because non-daemon threads still
existed.
Change-Id: I71c6935c60282b5353badc2dfce8a935d47e3bb7
Allows optionally disabling SSL compression. This can significantly
improve HTTPS upload/download performance in some cases -- in particular
when the object is not compressible and you have very high network
bandwidth.
Implements blueprint ssl-compression.
Change-Id: I1260055f9c2e83cdabfeb51aed11b3899bed4d55
The current implementation uses busy-wait and checking flags to
communicate between threads. It wastes a lot of CPU powers. With
python 2.6 is required for Swift, the communication between
threads should now be using queue and signal mechanisms. This
patch removed the busy-wait loops and use queue and queue join
for threads coordination which will not consume CPU cycles if
a thread is blocked.
Change-Id: I648cd637a92a159d5c13baa83f357cee2dfe7937
Christian Schwede