Instead of syntribos taking in ``--custom_install_root` during init and
``--syntribos-custom_root`` during run, which sometimes led to the custom root
option not being set correctly, the redundant config options have been
deprecated.
Change-Id: I95f4d345fe8a8608ff6d1daaf06ff49912917a3f
Fixes:
1) Crashes in runner and file_utils
2) Binary strings being read in as payloads
Updates:
1) Clarified error messages in parser
2) Confusing variable names in test cases vs issues
Adds:
1) A `syntribos root` CLI sub command to display the current syntribos root dir
Change-Id: I22edf7a1f3d39724522aee88d08b00d299b67248
1) Allows for YAML body in request templates. If a content-type
is specified for a request template, Syntribos will validate the
body against the header. This is to prevent templates from silently
failing to parse and sending unintended data to the target.
2) Added extension to support basicauth
3) Lowered confidence ratings for various tests
Change-Id: I672b0e0aa3da1aa6dd7e9a8456da73f0a15759b7
This change:
1) rewrites the runner to spawn a thread pool for each template
and assigns a worker for each test case
2) makes the output colorized by default
3) makes minor changes to the output
Change-Id: I49906f5daaa339ca9429913680203c762a0ad9fe
The i18n team has decided not to translate the logs because it
seems like it not very useful; operators prefer to have them in
English so that they can search for those strings on the internet.
Change-Id: I84b1a2397f424430ed332aa7d4d874379abd0357
Running syntribos throws an error due to nits in the i18n
patch. This patch fixes the errors in runner.py and result.py.
Change-Id: Ic1c041f586e4f556fd1c1e30b6dc3f3dcd0284fe
Syntribos now allows the user to specify variables in their request
templates by reading from a meta.json file. This is part 1 of 3 of
the full effort, dealing primarily with the template parser itself.
Change-Id: Id41d331f595cd3bc32f085ef49cb5d1b16779a5c
Changing six.iteritems(kwargs) to kwargs.items() to improve
readability of code and reduce the use of python package six
Change-Id: I7460c0274b790efcd53ee9d1c0bc538effb950dd
Messages displayed for some exceptions are too vague and have
stack traces, cleaning up a few messages and the way they are
handled.
Change-Id: I7b134844e8a9cfa21a119cd5b5b6c37871d0381a
KeyboardInterrupt now pauses the runner. From there, the user can
Ctrl-C again or type "quit" to stop execution, or press ENTER to continue.
Change-Id: Ib0384bbd06c248ba0f5dd61cfa16c8c3aaea975e
Syntribos now downloads payloads as part of the initialization process, and
also can download payloads and templates as part of the 'syntribos download'
command.
Change-Id: I17501535e5fd341c2705e07e3797643dc2d4a7df
Adds several tests to exercise more of runner and the env utils.
Makes several modifications to config.py, runner.py, and utils/env.py,
with the aim of making them more testable.
Change-Id: I708df9e5315f3dadd028c8107ac6416c707e90a0
Closes-Bug: #1634296
This command will set up the basics needed for a syntribos environment
(i.e. template folder, payload folder, log folder, default config file)
Some modifications have also been made to Runner to facilitate these
changes. These changes will (hopefully) allow us to better unit test
components of the runner.
Change-Id: I001e798b89539468cfbd92d5df301f7e59f1f33d
When syntribos attempts to download templates, it fails with
unicode error. This patch fixes it.
Closes-Bug: #1636609
Change-Id: Id62e9ed70d1b4501fe531587947c1f6d88bf034a
POC on loading payloads using remote URI. This is part of a larger
effort in packaging syntribos to ensure that the project would
work without much configuration post install from pypi.
Change-Id: Id61e840d4f49d5b6deb72bce2e8bcc0e1096fa52
Wrote empty debug test as a replacement for dry_run. dry_run no longer executes
any test code, or send any requests. Rather, it serves only to check that the
template file and all external calls inside the template are parsed and run
correctly.
Change-Id: I1cd817476c9ca4bd27b013dcdbd0cbce334847c4
As we are moving to the next iteration of the project for Ocata,
I think it would be helpul to makesure that the codebase is
fully compatible in py27 and in py35. Also, some minor styling
changes, removing uncessary spaces etc.
Change-Id: I9aac07dc180cd3a7c7885661bdad985c183ae0a9
Doc strings for 2 methods modified to reflect accurate info.
Fixes a bug in dry_run; dry_run should not send requests.
Also, making a few styling changes.
Change-Id: I6fc64adae8c38e3ab834068696ca6693a653992e
Some terminals does not display unicode correctly thus changing
progress character to a `#` and also fixing some minor nits in
progress display.
Change-Id: Id604ab9f646b1e846faabf9003ecb1a702a17313
Results are now formatted in the schema as defined here:
https://gist.github.com/cneill/a511451284a0c5f33295477150bd94d4
Furthermore, the json formatter is no longer responsible for the aggregation
of issues. Instead, this logic has been moved to the IssueTestResult class
Change-Id: Id39e122b2b4c1c9cafab09fdbc5d172dec012d22
1) results are now printed at the end of the run or on interrupt, not after
every template
2) made test_id's globally unique
3) print number of errors next to number of failures
4) number of failures printed after next to each progress bar and reported
in total at end of run now match up
Change-Id: I28617c64218ab632c1a35083ef9351874054a26d
1) Runner now only considers .template files
1b) renamed all template files
1c) fixed broken keystone template files
2) moved print_log() out of print_result()
3) fixed "ran x tests in x seconds" stats
Change-Id: I4600e5e154b5d3c951a08375a15904ea42621075
This change adds some commands to syntribos
sub commands:
- list_tests
- run
- dry_run
Also, refactoring runner.py and added utils/cli.py
to add all utility methods used for cli output.
Change-Id: Ieed2e06e0fb6eec34be640ae1db86785403546df
When specifying multiple test types to exclude, the "get_tests" method
in runner would only honor the first exclusion even if more than one
was specified. This patch allows multiple exclusion flags to be honored,
and adds unit tests to ensure it functions as expected.
Closes-Bug: #1612338
Change-Id: Ibbbc0ae27a7c5f2ce8e0f3e68f9b04aef72b8cdf
Adding a simple counter based id to tests to track them in debug
log and results log. Also, this patch improves the result output
like adding a progress bar and uniform test names..
Change-Id: Ib83181b25a0c18c7993f491cde98d73555b01404
This adds a *very* simple change to start capturing the values of
config options from the Runner in the debug log.
Change-Id: Id5872ed9f31a55beb820328ce945e136a9853d7a
Andreas Jaeger