The subunit.test_results module is no longer imported by the top-level
subunit module and we have to explicitly import it.
This also fixes the broken doc build caused by the history.rst file.
The change log is now maintained differently so we don't need it
any more.
Closes-Bug: #2045766
Change-Id: Iae1ae0c55b8e1660a2bb41379a1f8f8506679cca
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: I8c269117dba546ee2c75685dbd29756407488aaf
The patch encodes a data string which contains
non-ASCII characters to unicode type.
In python2 there are two text types str and unicode.
When we add two different text types, for example:
>>> "Hello " + u"World"
u'Hello World'
The string on the left is decoded by using the default
system encoding into a Unicode string.
If it would contain non-ASCII characters(as in our case)
this normally blow up with an UnicodeDecodeError because
the default encoding is set to ASCII.
Change-Id: I17e5aa6f45133d9d12aa38cec9cc42694913d4bd
Closes-Bug: #1857901
This commit adds deprecation comments to README, the user doc and
warning message to in the command. We already switched to use
`stestr` command instead of `ostestr`. And we haven't maintained
this command recently. So, it would be good to make it clearer for
users.
Change-Id: I3a6084db9f86627e3e94abaa4fb4aec52a01126a
The traceback would look like
Traceback (most recent call last):
File ".../bin/ostestr", line 10, in <module>
sys.exit(main())
File ".../os_testr/ostestr.py", line 263, in main
exit(ostestr(sys.argv[1:]))
File ".../os_testr/ostestr.py", line 259, in ostestr
return _select_and_call_runner(opts, regex, others)
File ".../os_testr/ostestr.py", line 212, in _select_and_call_runner
black_regex=opts.black_regex)
File ".../os_testr/ostestr.py", line 167, in call_testr
black_regex=black_regex)
File ".../stestr/commands/run.py", line 334, in run_command
concurrency = _to_int(concurrency)
File ".../stestr/commands/run.py", line 39, in _to_int
i = int(possible)
TypeError: int() argument must be a string or a number, not 'NoneType'
New enough stestr will tolerate it, but still write a warning to stderr.
See https://github.com/mtreinish/stestr/commit/ac8dac0
Change-Id: I70a3ed6c3edaafc8bb6269b1ae07bd46b06f4db4
This commit fixes a small oversight in the ostestr --pdb/--no-discover
path. When we go to load the results from the run the command used to
load the results was still using testrepository, which is not used
anywhere else in ostestr at this point.(and is not in requirements.txt
anymore) This commit fixes this and switches it to use stestr instead.
That also exposed another bug in that code path where it was assuming
a repository was created before ostestr was run which is fixed as part
of this commit.
Change-Id: Icb3492ab0f08248a33f711807b150b4a9748fda6
Currently when the blacklist_file and regex string is provided,
the constructed regex looks like '^((?!black1|black2|...).)*$regex'.
This is incorrect, as it will match nothing - some string
is expected after end of the line [denoted as $ in the regex].
The proper construction is like ^(?!black1|black2|...).*(regex).*$
This solves the issue with Tempest, where using blacklist for smoke
tests is not working now, as it leads into the issue described above.
Change-Id: Icdeb3c311f7eb414158aedb4c030494b419211c0
Closes-Bug: #1506215
Closes-Bug: #1595119
Closes-Bug: #1622722
Closes-Bug: #1669455
Apparently if -t is not specified, the path to the unit test directory
follows immediately the "discover" keyword.
This fixes the discovery for the .testr.conf in the sahara repository
(even if is being replaced by .stestr.conf, the fix may still be
useful).
Also, split the code which discover the legacy values in its own function
to simplify the testing.
Finally, disable hacking rule H405 which kept flagging incorrectly a long
multiline string as docstring.
Change-Id: Ide155a8e6b2b746c81388bacc0822c68d853b5a1
This commit switches the ostestr command to use stestr under the covers.
This simplifies the majority of the logic, eventually ostestr will
be deprecated as a result of this migration. (since almost all the logic
here is contained in stestr already)
Depends-On: I2c7618a742439fd2ed26879f3114f0f66fd6337f
Change-Id: Id7cb2a39a8308f1413608dcf19273a1d7f33592e
Previous code worked fine in python 2.7 but generated
"ResourceWarning: Unclosed file" in python 3.4+
Does not change the funtionality of the code at all.
Also updated the unit tests to work with the new code.
Change-Id: If98df5c4905f4b5a22602ba19224f9368fc8129f
When the list based filtering logic was used
and no test was selected at the end.
testr was invoked to run all test.
For example:
ostestr --regex a --black-regex a
Change-Id: I7b580c3954f71dc5fe347b82d6d14004bfdb8f22
This commit fixes some typos and changes 'regexp' to 'regex'. I think
'regexp' is also correct. However, in ostestr, they are mixed. So users
may be confused with it. So let's fix it.
Change-Id: I6c9042527707315f5dbb56f4a406327668c57d04
The current logic only considers the first occurrence of a test class
and doesn't update a class's pass/fail status after that. So a test
class, for example:
TestClass
test_1 pass
test_2 fail
test_3 pass
is counted as a passing test class even though it has a failure.
This adds failure tracking to the test result sorting so that any
failure in a test class will make it considered a failing test class
and be sorted at the top of the HTML result page.
Closes-Bug: #1640889
Change-Id: I06919336a09c4afda8ec3a9e7d64d305fbd169c3
It's very common for blacklist files and/or a black regex to have
overlapping matches. In this case ostestr was incorrectly trying to
remove tests from the set of test cases more than once. This commit
fixes this case by simply adding a check if the test is in the set
before trying to delete it. If it's not there, then something else
already removed it from the set of tests we'll be running so we can
just move on.
Change-Id: I1dabf9fb9c182af3dd6e124e79e54e1eb99bed82
The new entry point is created to allow external python programs,
to call ostestr without using an execve(2), and get the same
functionalty as using it form command line.
ostestr expected to work if it called multiple times in a
sequence. Parallel usage is not considered, but is similary safe as
the underlaying test(r) commands.
The explicit exit calls get replaced by returns.
Change-Id: I8efa495347a6956ef0a188063e6f474a4bbe8101
pdb and no_discover cannot be combined with black or white lists,
it was documented, but not checked.
Change-Id: I7168aef00505cb94051e2e587381efba9fd3e445
--whitelist_file, --blacklist_file is an unusual argument,
because of the underscore usage instead of
--blacklist-file/--blacklist-file .
Adding support for the '-' variant, and documenting only the
the '-' variant in the rst.
Change-Id: I8cc8197719f45bcfc05dc9412a03b9b1fe69e722
Adding option for reject regexp,
it works like any entry in the black list file,
so after the basic test selection regexp (including the white list),
the test cases satisfying the black-regexp search condition
will be removed from the final test list.
Change-Id: Id9fecf4aaa799828ae38b31ff3a4798763edfdd1
The listbuilder is able to support very long white list.
construct_regex is considered as deprecated function,
but external project like tempest still using it at the moment.
os-testr internally does not uses it anymore.
Change-Id: I0748605207eaad3065e18b84eecc9ddc4e47404e
The way how we handled the regular expressions had a lot of
limitation.
- We are not able to pass huge arguments to testr, it makes
difficult to have long list if accepted and/or rejected test cases,
but we can pass a path to a file of test cases,
which can be arbitrary big.
- How we wanted to handle the backlists before was not worked together
with the regular selecting regex because it consumed the pattern.
Now the blacklisting happens in a separated phase after the selecting
regex search.
This change just allows the new code path to run when both
a blacklist_file and a selecting regexp specified.
The new way depends on the usual test discovery and just
filters the output of the discovery command,
this strategy can be the default in the future, now I just
wanted to preserve the old behavior as much as possible in
all the other cases.
Change-Id: Ie8e5928e286d0c9076c4eee23319149c9869a6fa
Closes-Bug: #1506215
When using ostestr --no-discover flag. The output subunit
results file is not saved under .testrepository as it is
when using testr run command.
Change-Id: Ied6c5a46661b3b36bf15fd538bc6cac764fddcc8
Closes-Bug: #1597111
This commit makes subprocess outputs to str with universal_newlines
option. In Python 3, we need to do it explicitly. Otherwise an error
occurs. This commit also adds a unit test to prevent regression.
Change-Id: I2e428a3c57357e47c9057e8dbbbb51ae4b65e4e6
This commit makes unexpected success (uxsuccess) as fail. We already
consider expected fail as success. So this is a consistent change.
Change-Id: Ic8f9f07354dc16bc0daac71f59aa9799c04c27a6
Currently tests are sorted alphabetically by test class name regardless
of the pass/fail status. When troubleshooting failed unit tests in
jenkins jobs with many unit tests (> 11242 in nova), one must scroll
potentially a lot or wait as the page loads to search for the word
'fail' to get to the details of the failed test.
This sorts failed (and error) tests at the top of the HTML page for
easier troubleshooting (sorted failed tests + sorted passed tests).
Change-Id: I0b575e77c4a3e1cc73e60ea48ca5b9bf2e84d76f
The regex building logic is independently useful and will likely start
to be used in other places. This commits splits it out from the ostestr
cli and makes it an independent module that just contains the pieces
necessary for building a selection regex.
Change-Id: Ic8494d0f54357fdafd650b40219e6ad7fd5a65ad
Takashi Kajinami