Until now neutron fullstack tests which were run by the same worker were
using same DB but after test content of the DB was cleaned.
This could cause problems e.g. for default security group rules which
weren't created properly in second test run by the same worker.
To fix that issue patch [1] was proposed and merged some time ago. But
this didn't solve the problem so this patch is effectively reverting [1]
and proposing another solution which will make each fullstack test to
use own DB and run db migration script.
As running DB migration before every test makes this jobs to run a bit
longer than it took before, this patch also increases timeout for the
fullstack job(s) to 3h (10800 seconds).
[1] https://review.opendev.org/c/openstack/neutron/+/891040
Related-bug: #1983053
Change-Id: Ia261b4c62db9a99ef6eb161acb4609520e45d101
Db migration is when OpportunisticSqlFixture is setup during the test.
But it may happen, when 2 tests are run by the same worker that db
migration will be run only for first test and next one will have
just empty db schema (especially when tests are run serially, like
security groups related tests in fullstack CI job).
To avoid that issue in serially run fullstack tests and to have
always proper data in the db before test will start, new flag
FORCE_DB_MIGRATION is added to the OpportunisticDBTestMixin class. It is
set to False by default so behavior for tests which inherits from this
class is not changed.
It's set to True only in the
BaseSecurityGroupsSameNetworkTest so those fullstack tests will have
always data in the db at the beginning of the test.
Related-bug: #1983053
Change-Id: I1c93f80d6bc19084d30340be5c4b57dbe756a808
With python 3.x, six.text_type and six.string_type
are just str.
Also removed a six.integer_type since it was the only
one left in a file.
Another step in removing all of six usage from neutron.
Change-Id: I5208dc41bff1983ecd323286f427296b722da62a
This patch uses neutron-lib's SqlFixture and StaticSqlFixture by
removing those fixtures from neutron and using them from neutron-lib
instead.
NeutronLibImpact
Depends-On: https://review.opendev.org/#/c/668000/
Change-Id: I0c28f7436a7a8797b6b226748f8b5de8f7636b7e
The neutron.db.api.context_manager already references neutron-lib's
context manager; so consumers of it are already using neutron-lib. This
patch switches neutron's references to the context_manager over to
use neutron-lib's directly rather than that in neutron.db.api.
NeutronLibImpact
Change-Id: I97120faeec73690592ed21a5ec3c6202f61e1429
This incorporates flake8 2.6.x and pycodestyle will be used
instead of older pep8. This ensures future python3 compatibility
and a bit better code styling.
Change-Id: Ia7c7c5a44727f615a151e1e68dd94c7ed42f974f
It is deprecated since version 1.9.0 of testtools.
Instead of this deprecated method skipTest() is now used.
Change-Id: I7b6bbdbf43e7f428d5b3873a07fba7e6d629fa82
Allows posting revision number matching in the If-Match header
so updates/deletes will only be satisfied if the current revision
number of the object matches.
DocImpact: The Neutron API now supports conditional updates to resources
that contain the standard 'revision_number' attribute by
setting the revision_number in an HTTP If-Match header.
APIImpact
Partial-Bug: #1493714
Partially-Implements: blueprint push-notifications
Change-Id: I7d97d6044378eb59cb2c7bdc788dc6c174783299
This adjusts the SqlFixture class to preserve the settings
we configure on the enginefacade transaction factory and
to actually pass our enginefacade into to the
test DatabaseResource we create.
This is necessary for unit tests to be able to use configuration
options we set on the enginefacade like the flush_on_subtransaction
flag added in I71e274d0f0199896f28daae6db71241d9b43a2f3.
Related-Bug: #1664643
Partially-Implements: blueprint enginefacade-switch
Change-Id: Idabc7b6b4174575c89c8010886c2b440de6a8e05
While running the following was observed in the captured stderr:
neutron/tests/unit/testlib_api.py:104: DeprecationWarning: Using the
'synchronous_reader' argument is deprecated: argument value is
propagated from the parent _TransactionFactory
synchronous_reader=True)
I simply removed the argument.
The parent ref is here and defaults to True:
https://github.com/openstack/oslo.db/blob/master/oslo_db/sqlalchemy/enginefacade.py#L160
TrivialFix
Change-Id: Ic1744d97edb69b0f806aea7f2cde0b89b6bfbce1
Introduce get_reader_session() and get_writer_session()
and replace get_session() with them.
Mark get_session as depricated.
Stop using get_engine from legacy facade. Use writer engine for
places where it is required.
Partially-Implements blueprint: enginefacade-switch
Change-Id: I28b741bfa27bf04cbe273586e6e3e00e14fbe683
This patch add database migration for Neutron subprojects, needed for
fullstack testing
Signed-off-by: Thomas Morin <thomas.morin@orange.com>
Submitted on behalf of a third-party: Orange
Change-Id: I946b23b5a9446d7263c64cf29bf3fd295ed6cf00
Needed-By: I967ba442ef433b7ada5a1a381f1571ec3cf23517
The oslo.db opportunistic test fixtures were not being
used effectively and the MySQL / PG databases were not
being used. This patch restores working patterns against
oslo.db. Additionally, the migration level tests have also
been updated to make use of oslo.db provisioning functionality
and unused methods have been removed.
The current approach makes use of oslo.db provisioning constructs
directly, as well as some semi-private attribute access within
oslo.db enginefacade, in order to work around some issues
that have arisen in oslo.db's test_base.
A new release of oslo.db will provide
public API points to resolve pending issues, and to
allow neutron's use cases here which will
also be of general applicability to openstack projects.
Closes-bug: #1594898
Change-Id: Ie27cf174fa24c2f479af47335d9ae139fb7d159a
There were two broad issues with fixtures.
Firstly, the 'SafeFixture' workaround for resource leaks in fixtures
<1.3 is not needed if we depend on fixtures>=1.3.1. While testtools
may raise a TypeError when trying to query a fixture that failed to
setup, this is only ever a cascading failure - it will not cause
tests to fail, cause leaks, or cause tests to incorrectly pass. That
will be fixed in testtools soon to stop it happening (but as it cannot
affect whether a test passes or fails or leaks happen there is no
reason to wait for that). Leaks are seen with fixtures 1.3.0 still
because eventlet raises a BaseException subclass rather than an
Exception subclass, and fixtures 1.3.0 didn't handle that - 1.3.1 does.
Secondly, some of the fixtures had race conditions where things were
started and then cleanups scheduled. Where possible I've fixed those,
but some of them require more significant work to fully address.
Change-Id: I3290712f7274970defda19263f4955e3c78e5ed6
Depends-On: I8c01506894ec0a92b53bc0e4ad14767f2dd6a6b3
Closes-bug: #1453888
Currenty useFixture(myfixture)[1] ensures to call myfixture.cleanUp only
if myfixture.setUp succeed.
This change defines a workaround to ensure cleanUp call even if setUp
fails until testtools/fixtures support it: SafeFixture[2] which ensures
cleanUp call if setUp fails and replaces fixtures.Fixture use by
SafeFixture. This workaround will be removed when the bug will fixed in
testtools and fixtures[3].
[1] testtools.TestCase.useFixture, fixtures.Fixture.useFixture
[2] neutron.tests.tools
[3] see related bugs
Change-Id: I875934e8dde321a450c83fb95d175affd1f3bb83
Closes-Bug: #1464410
Partial-Bug: #1453888
Related-Bug: #1456353
Related-Bug: #1456370
This newly added class simply extends DietTestCase
with support for SQL operations.
Related-Blueprint: better-quotas
Change-Id: I2debc6a144ee93155c8e36f06aecb9735dbf55d4
The retargetable testing prototype previously relied on each test case
defining the 'scenarios' attribute used to parametize testing with
testscenarios. Anticipating the requirement to retrofit the imported
tempest api test cases, this change moves scenario definition to a
base class since scenarios are common across all api tests.
This change also sets the retargetable test to skip when invoked
against rest. Tempest uses class-level setup for auth and this needs
to be broken out into fixtures before the retargetable testing will
work again.
Change-Id: I70eb21db9b983d45e9bcc7ea90e36f202d3e3e45
XML support in Neutron has always been a second class feature to the
JSON API and broken for many extensions and outputs. The XML API been marked as
deprecated for the Icehouse and Juno release and is ready for removal in
Kilo.
Change-Id: I611aa9382ba3bfb08b2970c63e83b0fdd3d2e8a4
Closes-Bug: #1380787
Now that the schema is fixed for all of the plugins,
there isn't a need to delete and recreate the entire
schema for every unit test.
This patch clears the tables at the end of each test
instead of deleting them. This eliminated overhead seems
to save 10%+ execution time of the entire set of unit
tests.
Example of performance gain from tox -epy27 tests.unit.ml2:
Before: Ran 2495 tests in 284.186s
After: Ran 2495 tests in 223.299s
Change-Id: Ic5bcbb0cf941e0745890abc776d719e58bb42e35
This patch removes the Neutron capability of creating database tables
from sqlalchemy models for all those model classes for which
a table is not found in the database schema.
Migrations should be the official and only solution for creating and
managing the Neutron db schema.
This patch also adapts unit tests in order to ensure test schemas
are still correctly created.
DocImpact
Update deployment documentation accordingly.
Closes-Bug: #1207402
Change-Id: Ie4ee5507888ecad5f6dc32ce7a029c43014687a2
Co-Authored-By: Henry Gessau <gessau@cisco.com>
This change renames everything to Neutron while providing backwards
compatible adjustments for Grizzly configuration files.
implements blueprint: remove-use-of-quantum
Change-Id: Ie7d07ba7c89857e13d4ddc8f0e9b68de020a3d19
Slawek Kaplonski