The test infrastructure for swiftbench is pretty bad. In order to start
cleaning it up this patch starts with some initial scaffolding for
test_cli.py. Sd such it refactors the code out of bin/swiftbench and
puts it into swiftbench/cli/__init__.py. Leaving the cli namespace free
for future expansions.
A very basic tests/test_cli.py has been added which initialises some test
scaffolding. Additionally, fix some bugs this uncovered:
- The --delete-concurrency option now actually has an effect.
- Stop mutating global state on every run.
Partial-Bug: #1263290
Change-Id: Ibb2c3bb17522b6302697e2d2b01df3a6aa62800e
provide ability for user to select which storage policy
to run benchmark against
Change-Id: I9973f964ec3989281f47f853be053498e7a6e51e
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Fixes a bug when using the --log-level/-o option. These are currently not
usable, because a condition always exits if there are not exactly two
command-line arguments used (the ip and port).
This patch also ensures that the used log-level settings are valid choices.
Change-Id: I5d64e57b59947c059924fce3e458cfe9fa284318
Remove the dependence on swift, import the only needed functions from
swift.common.utils to swiftbench.utils
Add tests for utils using mock instead.
Change-Id: I1b69dce750b55f3ee0e999fb5a7100cf811f7ebe
- Update .gitignore to ignore all testrunners.
- Move swift-bench to swiftbench.
- Move tests to tests/
- Add some simple {test-,}requirements (more works needs to be done to get proper versioning)
Change-Id: Iab4d65f02cbce8c99ecafa30d15c4cb11f0b4293
With enable write affinity, it's necessary to wait until
replication has moved things to their proper homes before
running delete request. With write affinity turned on, only
nodes in local region will get the object right after PUT request.
Fix bug #1198926
Change-Id: I3aa8933d45c47a010ae05561e12176479e7c9bcc
When swift-bench is run in direct mode, don't try to delete the
containers which weren't created.
Fixes bug 1177960.
Change-Id: Ice07e8729bb776e2b215894cf95fb80b64167a8d
Using root logger will make public module log failed by unknow keyword.
Just change the logger name to module itself could dismiss this impact.
And disable logger.propagate could prevent double outputing.
Change-Id: I18696d124ebac9ca970d502558972e51de759097
Fixes: bug #1105133
These settings:
1. are already exposed via config file
2. were not exposed as command-line options, and
3. are things I wanted on the command line while benchmarking recently.
I didn't include short options for --(get|put|delete)-concurrency
since I couldn't think of good single-letter ones, and optparse won't
take "-gc", "-pc", or "-dc".
Change-Id: I0bd7c7ae2892244eed37b8c10c9357dc7e1190d3
This patch ensures that the command-line arg format (boolean)
doesn't conflict with the conf file format (string) and the
proper action is taken.
Change-Id: I3284096e1a9478897e1c3246ab190b46d2590243
You run one or more swift-bench-client processes like this:
$ swift-bench-client 127.0.0.1 20001
$ swift-bench-client 127.0.0.1 20002
Then you run swift-bench with a new option, --bench-clients (-b), which is
specified once for each swift-bench-client:
$ swift-bench -b 127.0.0.1:20001 -b 127.0.0.1:20002
You get log lines from each client (interleaved) along with a final report
for all clients:
127.0.0.1:20002 swift-bench-server 2012-08-25 22:44:06,148 INFO Auth version: 1.0
127.0.0.1:20001 swift-bench-server 2012-08-25 22:44:06,148 INFO Auth version: 1.0
127.0.0.1:20001 swift-bench-server 2012-08-25 22:44:12,249 INFO 83 PUTS [0 failures], 41.5/s
127.0.0.1:20002 swift-bench-server 2012-08-25 22:44:14,430 INFO 74 PUTS [0 failures], 34.3/s
...
127.0.0.1:20002 swift-bench-server 2012-08-25 22:45:18,942 INFO Auth version: 1.0
127.0.0.1:20002 swift-bench-server 2012-08-25 22:45:20,946 INFO 238 DEL [2 failures], 118.9/s
swift-bench 2012-08-25 22:45:27,549 INFO 2000 PUTS **FINAL** [0 failures], 56.8/s
swift-bench 2012-08-25 22:45:27,550 INFO 30000 GETS **FINAL** [50 failures], 974.6/s
swift-bench 2012-08-25 22:45:27,550 INFO 2000 DEL **FINAL** [20 failures], 237.1/s
The concurrency, PUT count, and GET count config settings are divided by
the number of bench_clients. In other words, the same volume of work is
attempted (vs. not specifying --bench-clients), but it can now span
servers and CPU cores.
Benchmark containers are created (if use_proxy = yes) and deleted (if
delete = yes), with appropriate concurrency, in the initiating
swift-bench process, not any of the swift-bench-client processes.
Change-Id: Idbf31a23093244ab357a9bf77e6031257774f24a
swift-bench now honors the environment variables, ST_AUTH, ST_USER, and
ST_KEY like python-swiftclient does.
Added --lower-object-size (or -l) command-line option which, if
specified, will turn a specified --object-size into --upper-object-size.
It will raise a ValueError if --object-size is not specified or is <=
--lower-object-size.
BenchController how handles SIGINT (KeyboardInterrupt) similarly to the
swift command-line client: the first Ctrl-C will make it fast-track to
completion (no new PUT or GET operations are started, but everything PUT
is DELETE'ed). A second Ctrl-C will immediately exit. The behavior for
SIGTERM is unchanged (a single SIGTERM will immediately terminate the
process).
Added a sample configuration file for swift-bench, with documenting
comments.
Change-Id: I6f394ad995300fc8af3d565d95c3b45559ee510a
We're still using saio:11000 in a few spots so a few things
don't work out of the box on the saio. Fixes bug #1024561
Change-Id: I226de54c2785b0d0b681c8d0cc24260adbd3d663
New log level "notice" set to python log level 25 maps to syslog priority
LOG_NOTICE. Used for some messages in the proxy server, but will be available
to all apps using the LogAdapter returned from get_logger. Cleaned up some
code in get_logger so that console logging works with log_routes and removed
some unneeded bits. NamedFormatter functionality was split between LogAdapter
(which now inherits from logging.LoggerAdapter) and TxnFormatter (which now is
only responsible for adding the log records txn_id).
The proxy server app now configures a separate logger for access line logging.
By default it will use the same settings as the regular proxy logger.
LogAdapter.tnx_id so that it works with multiple calls to get_logger, fixed
common.middleware.catch_errors to only call get_logger if it needs too, renamed
x-cf-trans-id to x-swift-tnx-id
Thomas Goirand