Fixes:
1) Crashes in runner and file_utils
2) Binary strings being read in as payloads
Updates:
1) Clarified error messages in parser
2) Confusing variable names in test cases vs issues
Adds:
1) A `syntribos root` CLI sub command to display the current syntribos root dir
Change-Id: I22edf7a1f3d39724522aee88d08b00d299b67248
1) Allows for YAML body in request templates. If a content-type
is specified for a request template, Syntribos will validate the
body against the header. This is to prevent templates from silently
failing to parse and sending unintended data to the target.
2) Added extension to support basicauth
3) Lowered confidence ratings for various tests
Change-Id: I672b0e0aa3da1aa6dd7e9a8456da73f0a15759b7
This change:
1) rewrites the runner to spawn a thread pool for each template
and assigns a worker for each test case
2) makes the output colorized by default
3) makes minor changes to the output
Change-Id: I49906f5daaa339ca9429913680203c762a0ad9fe
Running syntribos throws an error due to nits in the i18n
patch. This patch fixes the errors in runner.py and result.py.
Change-Id: Ic1c041f586e4f556fd1c1e30b6dc3f3dcd0284fe
POC on loading payloads using remote URI. This is part of a larger
effort in packaging syntribos to ensure that the project would
work without much configuration post install from pypi.
Change-Id: Id61e840d4f49d5b6deb72bce2e8bcc0e1096fa52
Results are now formatted in the schema as defined here:
https://gist.github.com/cneill/a511451284a0c5f33295477150bd94d4
Furthermore, the json formatter is no longer responsible for the aggregation
of issues. Instead, this logic has been moved to the IssueTestResult class
Change-Id: Id39e122b2b4c1c9cafab09fdbc5d172dec012d22
1) renamed "user" section for clarity
2) Added option to exclude defect type from output
3) modified default time_diff_percentage and length_diff_percentage values
4) added sample default values to config for oslo-config-generator
Change-Id: I4f88c255f8d4e6edfe3df3a4e463aee0cdd9c26b
1) results are now printed at the end of the run or on interrupt, not after
every template
2) made test_id's globally unique
3) print number of errors next to number of failures
4) number of failures printed after next to each progress bar and reported
in total at end of run now match up
Change-Id: I28617c64218ab632c1a35083ef9351874054a26d
1) Runner now only considers .template files
1b) renamed all template files
1c) fixed broken keystone template files
2) moved print_log() out of print_result()
3) fixed "ran x tests in x seconds" stats
Change-Id: I4600e5e154b5d3c951a08375a15904ea42621075
This change adds some commands to syntribos
sub commands:
- list_tests
- run
- dry_run
Also, refactoring runner.py and added utils/cli.py
to add all utility methods used for cli output.
Change-Id: Ieed2e06e0fb6eec34be640ae1db86785403546df
Adding a simple counter based id to tests to track them in debug
log and results log. Also, this patch improves the result output
like adding a progress bar and uniform test names..
Change-Id: Ib83181b25a0c18c7993f491cde98d73555b01404
We are currently using block quotes for license comments, which means
that Sphinx pulls this into every module, which is bad. Hash-prefixed
comments for licenses seem to be the standard for OpenStack as well.
Change-Id: Iabc60228c1201a960f672e1b0c7ab3c5312c9d8e
Implements: blueprint docstring-add-to-framework
This PR adds docstrings to a number of important components of
Syntribos, and adds this documentation to our Sphinx doc structure. It
also removes copyrights from __init__.py files with no other content,
in line with OpenStack style guidelines.
Set 2: Fixed PEP8 failure.
Change-Id: Ic57b31f451ec3ecf7f5b308da4544f808c9c9a5d
Implements: blueprint docstring-add-to-framework
Modified the unittest result class to store Issue objects instead
of stacktraces. Modified the test suite runner to use the new result
class
Change-Id: Ib7c80bde83a149a723221e288c04b89c9c1229f4
Andreas Jaeger