Check and close all the popup messages in one step
and return all the levels of present messages.
There is no longer necessary to use a separate step for checking
presence/absence of Success message, Error message, etc.
Change-Id: I15a0dbe99282f00c5970d9d7d5c9b480989e9e14
The way we were handling attaching video in integration tests, it would
not attach it if the error happened in a cleanup, not in the main body
of the test. This should fix that.
Change-Id: I356e6849892ed87b2827740e265fcf15fc076cb5
Firefox will throw an "HTTP method not allowed" error randomly when
trying to retrieve the browser log. This change will prevent the
selenium test from failing when that happens, and instead logs
the exact exception that happened as the content of that log.
Change-Id: I3384c3cfdf87f475bf62b2df6b895569723eb187
In python3, super() does not always require a class and self reference.
In other words, super() is enough for most cases.
This is much simpler and it is time to switch it to the newer style.
pylint provides a check for this.
Let's enable 'super-with-arguments' check.
NOTE: _prepare_mappings() method of FormRegion in
openstack_dashboard/test/integration_tests/regions/forms.py is refactored.
super() (without explicit class and self referece) does not work when
a subclass method calls a same method in a parent class multiple times.
It looks better to prepare a separate method to provide a common logic.
Change-Id: Id9512a14be9f20dbd5ebd63d446570c7b7c825ff
* Add fetch-video-output role based on fetch-tox-output
* Fix test log collection to work with Python3
* Fix page HTML source log
Change-Id: I2e865cc5544d6e77bc3af5e413cbaedceb6df1c1
The aim of this PS is to remove hardcoded test_reports dir path
so test run can get screenshot dir from the specified configuration.
Change-Id: I454ccdccbed93b831c41199481caf8b5eb302337
Switching the test runner to pytest in horizon assumes pytest is always
installed, but horizon test helpers are used in horizon plugin tests and
pytest is not used in horizon plugin tests. As a result, all horizon plugin
tests are now broken.
This commit considers a case where pytest does not exist.
A wrapper decorator for pytest.mark.xxxx is introduced and
it acts as a null decorator if pytest is not installed.
Change-Id: I80736b108f5ae9a36c0e756bf386468879be3293
Closes-Bug: #1873532
Changes test invocation from `manage.py test` to `pytest`. Adds addtitional
test requirements like pytest, pytest-django, pytest-html. Adds
`pytest.mark` alongside django's test `tag`. Adds posibility to export test
results into xml and html formats.
Depends-On: https://review.opendev.org/#/c/712315/
Related-Bug: #1866666
Co-Authored-By: Ivan Kolodyazhny <e0ne@e0ne.info>
Change-Id: Idb6e63cd23ca2ba8ca56f36eb8b63069bd211944
We don't support Python 2 anymore so we don't need this
compatibility library.
six.reraise usages are left as is until it'll be moved to some
base lib like oslo.utils to not re-implenent this method in
Horizon.
This patch also removes Python2-specific base test case methods
assertItemsEqual and assertNotRegexpMatches in flavor of new
Python 3 analogues.
Change-Id: I26a59176be9e9f213128e4945a58b9459334b626
tox env for the integration tests is now renamed to 'integration'
as it is no longer run under python 2.7.
__hash__() method is now defined in integration_tests.helpers.BaseTestCase
to avoid unhashable error during loading django test runner. This is
originally caused by the fact that the base testcase class for integration
tests is implemented on top of testtools and testtools does not define
__hash__() method. In Python 3, __hash__() method must be defined
if a class (re)defines __eq__() method to make the class hashable [1].
Ideally integration base TestCase class can be implemented on top of
Django test case, but the current implementation depends on features
from testtools a lot, so it seems better to add __hash__() method
as a workaround.
Unbound methods don't exist in Python 3[2], so six.create_unbound_method
doesn' work as expected. To dynamic method generation we have to use new
descriptors interface [3] or just generate new
functools.partialmethod [4] function introduced in Python 3.4 to
generate class methods with pre-defined 'path' argument and pass 'self'
as a first function argument for class instance.
[1] https://docs.python.org/3.5/reference/datamodel.html#object.__hash__
[2] https://six.readthedocs.io/#six.create_unbound_method
[3] https://docs.python.org/3/howto/descriptor.html
[4] https://docs.python.org/3/library/functools.html#functools.partialmethod
Change-Id: I5c3078bdc66e33fe676d431bb28d92b35faaf479
* updated admin user case navigation to admin - overview page on login
* updated test_update_compute_defaults making it specific to compute
* defined test_update_volume_defaults & updated defaults page module
* removed skip decorator for test_create_delete_project as bug 1777359
is resolved
* cleaned up test_flavors and updated flavors page module
Change-Id: If529bfd7bc7fd0e29b42b12e601824ec6d684fca
Nose has been in maintenance mode for the past several years. It has
issue with exit code [1] which leads to false positive results for our
seleniun-headless job.
This patch changes test runner for Horizon tests and does the following
things:
* Django test runner executes test in a different order than Nose does.
That's why we've got an issue with side-effect in
horizon.tests.unit.tables.test_tables.MyToggleAction class. This patch
adds workaround to it.
* Rename filename of test files to names starting with 'test_'
so that the django test runner can find tests expectedly.
* '--with-html-output' option is temporary dropped and will be added in
a following patch.
* Integraion tests is marked via django.test.tag mechanism which is
introduced in Django 1.10
* 'selenium-headless' is broken now because we don't have geckodriver on
gates, this patch makes it non-voting.
* 'tox -e cover' is fixed
* Remove @memorized decorator from
dashboards.project.images.images.tables.filter_tenant_ids function.
[1] https://github.com/nose-devs/nose/issues/984
Depends-On: https://review.openstack.org/572095
Depends-On: https://review.openstack.org/572124
Depends-On: https://review.openstack.org/572390
Depends-On: https://review.openstack.org/572391
Related blueprint: improve-horizon-testing
Change-Id: I7fb2fd7dd40f301ea822154b9809a9a07610c507
In reviews we usually check import grouping but it is boring.
By using flake8-import-order plugin, we can avoid this.
This flake8 plugin is already used in tempest.
It enforces loose checking so it sounds good to use.
Note that flake8-import-order version is pinned to avoid unexpected
breakage of pep8 job.
Change-Id: I8ccd05eb70350a2441cc2a4d1eafc09ee690b83b
* Use module-specific logger in integration helper.
Previously the root logger (returned by getLogger without parameter)
was used for module-specifc log messages.
It is better to use module-specific logger for such messages
and we can easily identify where log messages come from.
* In case of requiring access to the root logger,
get the root logger explicitly.
* Use 'LOG' for logger name. Almost all modules in horizon use
'LOG' for logger name, but some do not.
It is better to follow the convention.
* Use LOG.warning instead of LOG.warn.
LOG.warning is recommended way in Python3.
Change-Id: Ib9de7ee08a235de9820b95910d8f54724a1f2b91
The uuid format of uuidutils.generate_uuid(dashed=False) equal to
uuidutils.generate_uuid().replace("-", "").
Change-Id: Iea46b7ce146e5e7e48b80c0936d1c66d78d047dc
Updated forms.py with MetadataFormRegion.
Updated tables.py with method to bind anchor row column.
Added test for update image metadata.
Implements blueprint: horizon-integration-tests-coverage
Change-Id: Idd3651955b8f0e1a0c08dd43abd657aafa5a3bc2
Recently introduced manager handled only 'bad' failure scenarios when
Selenium became non responsive and caused again cryptic failures for
failures that were perfectly reported before exceptions_captured
introduction. Now both kind of (responsive and non responsive
Selenium) failures are addressed inside it.
Change-Id: I564b5e8b35fb2e84c27374bcac8a49cb6262b058
Closes-Bug: #1545042
Having the window maximise during a test run makes it difficult
to work with other windows, especially on a single-monitor
computer. This patch removes the maximisation call.
This patch also adds a "local" configuration mechanism to allow
developers to have local configuration of the integration test
environment without it affecting the git repository.
Change-Id: I8a7acbe40deaec5cca904526e3f3c8bc3357744c
Closes-Bug: 1540719
Admin users run tests in an admin project, regular users run them in a
demo project. That should prevent situations when tests don't have an
access to a resources created in a project different from the current
one.
Closes-Bug: #1542211
Change-Id: I497648ce5126e187744b5795bd524b7aba22c7a6
Achieve this by muffling exceptions (raised due to Selenium became not
resposive) while taking integration test failure screenshot.
Also extract the pattern of muffling and capturing exceptions common
for dump_browser_log, dump_html_page and save_screenshot in a common
@exceptions_captured context manager.
Closes-Bug: #1543541
Change-Id: I37fa18a302c553b43529df056d2beacff70f6189
Since Selenium 2.50.1 release it's no longer possible to click
elements which are outside of visible viewport. To keep both tests
working and error screenshots still visible, standard xvfb screensize
is set to 1920x1080 and page scale is set to 67%.
Also temporarily disable create_instance test since it fails on some
nodes due to missing network.
Related-Bug: #1542211
Closes-Bug: #1541876
Change-Id: Ie96606cf52860dd8bb3286971962a16cb3415daf
xvfbwrapper 0.2.8 renamed the xvfb_cmd attribute of xvfbwrapper.Xvfb
to extra_xvfb_args. This change checks if the new attribute name is
available to support new and old xvfbwrapper version.
xvfbwrapper commit changing the API:
9a8c0e80f6
Closes-Bug: #1540495
Change-Id: I45704450ff396787453937362029945b94887189
Horizon currently has two webdrivers for selenium-based
testing; this patch merges them into one so all tests
(selenium functional tests and integration tests) benefit
from improvements we make to the webdriver.
Change-Id: Iac471c2c7e40c799711746ef9e3e4ef87aecf098
This prevents screenshots in plugins integration tests being placed
into places like .tox/py27integration/horizon/openstack_dashboard...
Change-Id: Idf831a2fee4eb9b0c34fdf03c50317e253a082b9
This change increses the timeout for the communications between the
test process and the selenium web driver process.
Change-Id: I79e46713338a59d45c117dc86bf59b9992dbdd94
Fixes-Bug: 1526611
Having all messages from browser console inside a test log sometimes
may be very valuable in understanding the cause of test failure.
Change-Id: I9ef39e412e0b9f1faacf0a3e42418ca138c52d66
Related-Bug: #1505320
Having screenshot of failed test might increase speed of
understanding the failure reason.
partially-implements blueprint: selenium-integration-testing
Depends-On: I0fd89ede7a3d04ce9b4bd4c8fba0789770048d40
Change-Id: Ibc4a6235423736e5938fb348bbe83211917b08d1
Sometimes it is not possible to reproduce failure in integration test
locally. Adding html source code of page where failure happened should
increase our ability to debug properly errors on the gates. For this
purpose the addDetail method from testtools package is used.
Following changes were done:
* TestCase and AdminTestCase classes were modified in order to share
common functionality and remove code repetition.
* Testcases have been modified in order to use newly introduce class
variables(TEST_USER_NAME, TEST_PASSWORD, CONFIG).
* Methods _get_page_html_source and _dump_page_html_source have been created
in the BaseTestCase class. _get_page_html_source uses javascript innerHTML
attribute in order to make javascript modification to the page visible
in the output. (javascript modificaton would not have been visible if
webdriver's property page_source was used)
Partially implements blueprint: selenium-integration-testing
Change-Id: I064405337fb68c966da007f2a5bf91d5c0ec2b3d
Adding function gen_random_resource_name for generation random resources
that should be used in the integration tests. Mentioned method
automatically adds horizon string into the resource name in order to clearly
distinguish resource origin.
Also timestamp is added for better debugging of the tests in more
complex testcases.
uuid4 function is used to minimize resource name collisions.
Closes-Bug: #1399219
Partially implements blueprint: selenium-integration-testing
Change-Id: Id5cb26e50b7b442af69798a2928be514adf7d8fe
Verified by grep, the method wasn't used anywhere, so there is no
purpose to have it there.
Partially implements blueprint: selenium-integration-testing
Closes-Bug: #1414296
Change-Id: I2646ae988d45af81a4dd75e918806465748eeea3
There is outstanding memory leak issue with Xvfb, that should be fixed
by -noreset argument.
-ac disables X access control - which i believe is not necessary for
testing
Partially implements blueprint: selenium-integration-testing
Closes-Bug: #1412469
Change-Id: I5f603266f891728bb8e83331f5a4e57799f07f79
In TestCase and AdminTestCase
* Added condition to check login status inside teardown method.
* Added try and finally, in case logout fails.
In BaseTestCase->teardown:
* replaced driver.close() with driver.quit()
Partially implements blueprint: selenium-integration-testing
Closes-Bug: #1405553
Change-Id: Id1b520ec0a747ace89864bf7cc6b5d9b670fdc60
A lambda function is passed to selenium wait call until() but the docstrings
of that call suggest a better way of passing the lambda function.
Its redundant to return a specific driver instance when until() function
already takes care of that.
Change-Id: Ie0916587bf6a7488b3828d91ad553a3944619dfc
Partially implements blueprint: selenium-integration-testing
Closes-Bug: #1404416
Jan Jasek