Commit Graph

308 Commits

Author SHA1 Message Date
Jenkins c3f341eb29 Merge "Fixed format_exception vs format_exec bug" 2017-02-15 16:54:45 +00:00
Daryl Walleck 525e44c925 Resolves issues discovered using flake8 with Python 3.5
These are issues in Python 2.7 but were not caught by
its version of flake8.

Change-Id: I90349a6b4345b0787671a249a9d3b6a634372fc7
2017-01-26 13:22:11 -06:00
Jenkins 65e7480dd0 Merge "Added workaround for issue affecting namespace plugins introduced by setuptools 31.0.0" 2017-01-04 21:15:02 +00:00
Jose Idar b53ad3f0b1 Added workaround for issue affecting namespace plugins introduced by setuptools 31.0.0
Change-Id: I510ce520e362ad10de93e2defb83429520acd140
2017-01-04 14:21:41 -06:00
Stephen Lowrie a47d0d1df0 Fix pep8 errors under new flake8
Change-Id: Iab6df427ddb91d1659e3206f6ef6edc6e6389fbe
2016-12-13 15:12:01 -06:00
Nathan Buckner df8e63de08 Fixed format_exception vs format_exec bug
Change-Id: I32c5b6f5754bc62f54f5b042e00b54c4ccf99239
2016-09-23 12:34:13 -05:00
Jenkins dcd29f3253 Merge "Added mechanism to allow retries around Requests Lib exceptions" 2016-06-08 20:27:19 +00:00
Anna Eilering e2723c9f5f Added mechanism to allow retries around Requests Lib exceptions
Often exceptions caused by the Request Library are either:
* Impacting, in which case they are consistent
* Non-Impacting, in which case they are sporadic and cause false
  failures

In tests that do API testing using the HTTP plugin, there is a use
case for retrying Request Library exceptions as some of these
failures can be caused by issues unrelated to tests and the system
under test. This PR provides a mechanism to configure how many of
these errors a test runner is willing to ignore.

Change-Id: I2d15095ad435b151df56f0a23f068f0fae662c37
2016-06-08 16:06:25 +00:00
Eli Uriegas 7956e09b97 Adding a better plugin not found message
Updates the plugin not found error message to actually say plugin not
found

Change-Id: Icceb7616a9222ae2aa013a2e0902cf0ca9acea41
2016-06-02 14:16:36 -05:00
Jason Meridth 8ffd9baf67 Fix import of StringIO to work with Python3+
Change-Id: Iddb7836a88100eddb415e0860d564c1b01b254e5
2016-06-01 10:28:27 -05:00
Jenkins fd5391ce74 Merge "Add skipped tests to output of parallel runner" 2016-05-16 15:25:57 +00:00
Jose Idar 60c014bd7c Minor fixes for cafe-brew runfile runner
* Cleans up some unittests/makes Brew more testable
  * Cleans up __repr__ output on some classes
  * Removes unneeded imports

Change-Id: Ib33bdcc21c2f554f2312f9ed92de8e500e8356f2
2016-05-11 20:28:03 +00:00
Jason Meridth 9655d5274e Add skipped tests to output of parallel runner
- unit test captures stdout of print_results
- mock out init to get past all the argparse requirements

Change-Id: Ifec5dff1a96288289b73ea270964165cc67b6b1a
2016-05-11 11:45:03 -05:00
Jenkins 2b0ec92892 Merge "Adds new Brew file runner." 2016-05-10 21:03:45 +00:00
Jose Idar 75039a8cc4 Adds new Brew file runner.
* Generates suites from test-containing classes,
    unittest.TestSuite-derived classes (Fixtures), and
    opencafe DatasetList classes as defined in a flat text file
    called a 'brew' (runfile).
  * Adds brew package in cafe.drivers.unittest
  * Adds arguments, parser and runner modules in brew
    package.
  * Adds examples directory in repository root directory
  * Adds example brewfile in examples directory
  * Adds new 'cafe-brew' entry point to setup.py
  * Adds unittests for brew package

Change-Id: I379c5b2c4bdf17bc49ce055ef4600b38a72a1140
2016-05-10 11:55:20 -05:00
Jose Idar 5b8be5e231 Locking paramiko version to <2 for the sshv2 plugin
* Added README explaining the paramiko transition.
  * NOTE: The sshv2 plugin will eventually transition
    to utilizing paramkio>2 once plugins are external
    to opencafe.

Change-Id: Iab3249ce31503367ed862114b71c460d5eb1b208
2016-05-09 16:23:16 -05:00
Jenkins 4519791020 Merge "Added support for the multiprocess library in all unittest runners" 2016-05-06 15:30:42 +00:00
Jenkins 2f49b720da Merge "Adding new datagen metrics to reporting" 2016-05-06 15:26:25 +00:00
Eli Uriegas 18005e276d Catch ImportError for users of paramiko<2.0.0
This PR prevents users of paramiko<2.0.0 from running into an
ImportError while trying to import the cryptography library. This comes
with no functional changes and should remediate any ImportError's users
of paramiko<2.0.0 would be getting.

Change-Id: I6c45bdbf288c4cef5d5e5fb055276d9d3978e8b8
2016-05-04 12:02:47 -05:00
Jenkins 837db9e29c Merge "Updating SSHV2 client for paramiko 2.0.0 issue #104" 2016-05-04 16:09:54 +00:00
Jenkins 7008e8f157 Merge "Removes long-deprecated files from cafe/plugins" 2016-05-04 02:23:50 +00:00
Anna Eilering 43ffa1656d Update paramiko dependancy to be fixed at 1.17.0 due to paramiko updates.
Paramiko 2.0 changed the paramiko dependancy to require cryptography
which can cause issues with OpenCafe installations of the SSH plugin.

Change-Id: Ied6793e0a42e1d7de6a15db19e9883399d3301d6
2016-05-03 15:39:05 -05:00
Eli Uriegas 4be20ff3db Updating SSHV2 client for paramiko 2.0.0 issue #104
This PR updates the SSHV2 client to have the fix the issue
outlined in
https://github.com/paramiko/paramiko/issues/735#issuecomment-216083986
where a new library added by paramiko 2.0.0 (cryptography) causes a
threadlocking issue where the SSHV2 client is unable to function when
connecting on import or through the unittest runner.

This PR should not affect any functionality users may have but may
affect users who have not yet installed the cryptography library. To
install the correct cryptography library it is recommended to upgrade
paramiko using `pip install --upgrade paramiko`

Change-Id: Iabcce8a66b5958b117c45977708cab777dc90a39
2016-05-03 14:45:18 -05:00
Marcus Medley 96bbe1d0db Adding new datagen metrics to reporting
Modified json_report.py to include new metrics
Modified xml_report.py to include new metrics
Modified SummarizeResults to handle new metrics
Added metatests for compile_results method

Change-Id: Iecd23d519e7cbaf4887f64b3d362942d39f50006
2016-05-03 14:28:05 -05:00
Jose Idar 315a90e790 Removes long-deprecated files from cafe/plugins
* Affects the http, ssh, and mongo plugins.

Change-Id: I669f60c1900ac7c52cd7232e9d965cdbf63a8ab1
2016-05-03 11:20:29 -05:00
Jenkins 8823a48b7e Merge "Added Safety and clarity around tests setups missing Super()" 2016-05-02 02:06:16 +00:00
Jenkins 3949c7fc76 Merge "Moved code that ran on Import into a method that is called." 2016-05-02 01:59:14 +00:00
Anna Eilering ee4d848e98 Added Safety and clarity around tests setups missing Super()
Test setUp methods that do not have a 'super()' call will
fail and cause the entire suite to fail despite tests
passing. Runner failures should, whenever possible, not
interfere with test results.

This is to add a try/except around getting the '_duration'
value for tests. If a test uses a setUp() method without super
it will recieve 'NaN' as a 'duration' and the user will
recieve a warning in console output but will not recieve
failures.

This is considered a safety mechanism intended to provide
test writers time to update their tests using 'setUp()' to
correctly call super().

Additionally added some newlines around the warning so that it is
very clear that there is an issue.

Change-Id: I32d2352ff7081b84383835bcc8812e37735c68d2
2016-04-27 16:20:03 -05:00
Anna Eilering 2e89949dae Moved code that ran on Import into a method that is called.
Change-Id: I38f366023c6f55469fa166a2c8c49220af49ec43
2016-04-27 15:17:26 -05:00
Marcus Medley 2285e60c3c Fixing error is datagen time calculation
Modified call to compile_results method to correct timing calculation

Change-Id: I688eb073344bf59e7296b77f1833f38e77d56c4e
2016-04-27 11:36:56 -05:00
Jose Idar af88e5129b Added support for the multiprocess library in all unittest runners
* Adds the pathos_multiprocess plugin which serves only to
    install the multiprocess package.  If installed, it will
    be used by all unittest runners instead of the default
    multiprocessing library.

Change-Id: Id18ee56d3fb1466f038aaa8c6fd9e2d890847f9e
2016-04-27 11:10:48 -05:00
Nathan Buckner 67353ed226 Simplifies __ne__ in BaseModel
This change uses 'not __eq__' to implement __ne__ preventing
execeptions from being raised when obj passed in does not have
__dict__. Example are instances of tuple, string, int, float,
bool...

Change-Id: Idf82eeeb865a121d5c81c5654e7c21e71e555238
2016-04-13 15:23:24 -05:00
Jenkins 57ad10c550 Merge "Updates to the parallel runner's test run_time calculation" 2016-04-13 16:05:58 +00:00
Marcus Medley a0b35d8c84 Updates to the parallel runner's test run_time calculation
Added datagen runtime and total runtime to results output

Change-Id: Icfa92bc8951eb692b1ab0e1700e38fef18ec3d2b
2016-04-12 16:02:29 -05:00
Anna Eilering 799656da85 Change to DataDrivenClass generator to fail when DSL does not generate data.
If a DataSetList is passed into a test that does not generate or
contain data the current behavior is that the test suites passes.
This proposal is to change that behavior. If a DSL cannot generate
data the test suite as a whole should still run but it should
fail. Otherwise teams using tooling to track and manage test results
will have false positives with mysteriously missing tests.

Added Metatests to check basic functionality.
Added Engine config section for unittest driver.

Change-Id: I3430e9573c3e99b80f25e7577709144d49c8eae2
2016-04-12 13:57:24 -05:00
Jose Idar d14fc7dd0d Reworked module check in get_modules for accuracy.
Change-Id: I77b954eb6cb84c23753ce8efb2c53aaf38efc64b
2016-04-07 10:39:01 -05:00
Nathan Buckner 374e35d0b8 Added dirty hack to fix paramiko
Change-Id: I007efb59ddf0913868af1c3151a37761e40bda2d
2016-04-05 11:10:51 -05:00
Jenkins 1a2d3fa057 Merge "Add docstrings for sshv2 behaviors" 2016-03-31 14:14:46 +00:00
Jose Idar f6dee31d0a Fixed split() bug in http plugin logging
Change-Id: I01432b9eaba65fcbba8cf547b932b1e81aefe1d5
2016-03-28 18:05:56 -05:00
seemethere 139d9370bc Add docstrings for sshv2 behaviors
Adds docstrings for the sshv2 behaviors to include parameters and types
for parameters. Some documentation taken directly from underlying
modules.

This should affect no users as it touches no functional code.

Change-Id: I67ef58f99455a64d1942a4b4c2d4768a7437048a
2016-03-19 13:27:22 -05:00
Jenkins 8e3b738b4b Merge "Remove get_pty, use stdin_str and stdin_file" 2016-03-14 17:21:42 +00:00
Jenkins 84d4af0ce6 Merge "Add individual test times to xml report." 2016-03-10 16:06:35 +00:00
seemethere 8ebef82982 Remove get_pty, use stdin_str and stdin_file
Change-Id: Ia4237bb4b756952ba5abaa668a78d263d8888113
2016-03-09 18:40:23 -06:00
Jenkins 2ca1441aad Merge "Basic Ping Client Rework" 2016-03-01 16:12:39 +00:00
Lindsay Kusmierek 96f747133f Add individual test times to xml report.
Currently, the xml report only reports the total test execution time, and do not provide data on the time
taken for individual test cases. This change add a _duration attribute to the base test case that is then
populated during tearDown and added to a 'test_time' attribute on the result object. The duration is
determined using a the same mechanism as the test_metrics reporting. The time is reported in seconds.

Change-Id: I41931e12e13e073debd37403fcc0030c26041fb0
2016-02-26 14:04:44 -08:00
Nathan Buckner d1025bfa08 Args in wrong order on getfo call
Change-Id: I3f697485b7e49c31cb9df9fc1def7f2af43d40e1
2016-02-24 15:44:10 -06:00
Christopher Hunt 333812211a Basic Ping Client Rework
* Added ability to specify number of pings to be transmitted
* Abstracted ping logic to return "percent of pings requests not responded". The logic was not reworked,
  and stayed the same for now... (could be reworked, but currently out of scope for current need).
* Added wrapper functions to return different information:
   ping() - returns True/False --> Same as original ping() command (for backwards compatability)
   ping_percent_loss() --> Returns percent of (missing replies)/requests
   ping_percent_success() --> Returns percent of (received replies)/requests

Change-Id: If242fd02dde3642372c6c5e1f3e117a38e51c1d7
2016-02-23 15:35:20 -06:00
Nathan Buckner af6c42498b Create sshv2 client for compatibility with parallel runner
Change-Id: I5e80177859be74a869dd3d908f516d751258ae7c
2016-01-18 13:49:17 -06:00
Jenkins a56f13e51c Merge "Fully Qualified Class Names in Reports - Changes the unittest parser to provide fully qualified class names to the result objects provided to reports." 2015-11-20 20:34:36 +00:00
step6829 94332f028e Add SubunitReport
Adds 'subunit' option to --result commandline argument.

SubunitReport outputs to a file subunit_results in the specified directory

Change-Id: I33f418c5d150551c0bde28c5e31aaa30b2324110
2015-11-17 23:17:40 +00:00