Change the functional tests to use the PlacementFixture instead
of devstack as the source of a placement API. This speeds up
the tests considerably and lowers the number of dependencies.
There are four primary changes:
* For each test a PlacementFixture is instantiated, using the
usual in-RAM db and in-process placement.
* Because of some exceedingly confusing optimizations in
osc_lib and python-openstackclient, done to improve start
up time, a session to placement was caching the service
url. This meant that after a first test succeeded, every
subsequent one would not because it was trying to talk
to a fake hostname that was no longer being intercepted.
The workaround for this was to monkeypatch the method
in the ClientCache class which provides access to a client (per
service-type). The replacement method makes a new client
every time.
* The previous tests would subprocess out to a real call of
the openstack command and then interpret the results.
Now, a run() method on OpenStackShell is called instead.
This accepts arguments in the same way, but we need to
a) capture stderr and stdout, b) make a try/except for
SystemExit to get some error responses (mostly from
the argparse lib which has a tendency to exit for you
instead of politely telling you it wants to), c) deal
with errors from commands ourself rather than using
exceptions from the subprocess module.
Switching to this form means that logging becomes in-process
and more visible. To accomodate this the Capture fixture
from placement is used. This was chosen because we are already
pulling in the PlacementFixture. If this seems icky, I can fix
it with a local one. This was the shorter path.
* The legacy dsvm jobs have been removed in favor of "standard"
functional jobs for 2.7 and 3.6 that require openstack/placement.
The playbooks associated with the legacy jobs are removed.
tox.ini is adjusted to reflect this new setup. Because tox-siblings
functional is being used, we don't want to share tox envs with the unit
tests. The 3.5 functional job is removed because we no longer target
that.
After running these for a while it became clear that there were
intermittent failures being caused by subunit attachments being too
large. This was eventually traced back to logging from all packages
being set to DEBUG even when something else was requested. That was
traced back to a poor interaction between the way that osc does
logging and the way oslo_logging does logging (used by placement and
thus the placement fixture). The workaround, embodied in the
RESET_LOGGING list in osc_placement/tests/functional/base.py, is to
get and reset the log level for a subset of the packages that are
used.
Change-Id: I7deda200b372ff6a7ba67b0c4fa0e53c4fa16ffc
Story: 2005411
Task: 30428
The functional test jobs require a devstack environment which
makes the job longer, but we should really only need keystone
and placement (and a database) so this change explicitly sets
the ENABLED_SERVICES list that will be used in devstack.
As a result, the task to disable swift from the py3 job can
be removed.
Change-Id: Ifbd9c1777b2d565f95e42f6264e2e10fd1679f33
Story: #2005411
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.
This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.
This update should result in no functional change.
For more information see the thread at
http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html
Change-Id: I06d8cb7be0963f0377f56fd3dcdb3a97c9b6cc4e
While trying to make some changes I discovered that the tox
configuration for osc-placement was rather out of date and
functional tests were not working for python3. With this
change we bring tox.ini into a style that is more in keeping
with modern standards, use stestr, and update some functional
tests so they work with python3.
The functional tests changes are either:
* to fix the decoding of response
* to adapt argparse error response checking between python
version.
These changes cascade some required change into how the gate-side
functional testing is performed. We make it explicit that in the
python2 job, the 'functional' tox job is run. When the python3
job is run, 'functional-py3' is run. Also stestr replaces
testr in the post-test hook.
When the functional-py3 test had been run in the past it
was actually running a python2 osc-placement against a
python3 devstack. We change that here to be python3 and
python3. Once that was happening, additional failures
were revealed, now fixed.
One particular issue was that while the json module
for python 3.6 and greater will decode strings or bytes,
the version of 3.5 will only do strings. We switch
to using simplejson throughout which smooths things over.
This is added as a new requirement but it isn't really:
simplejson is required by osc-lib.
Finally, some requirements need to be tuned to pass the
gate requirements job.
Change-Id: I999a3103dd85c0a437785766eef533875fca31fc
This adds a py3 variant of the functional dsvm job.
This still uses devstack-gate (not native zuulv3) because
I'm not yet cool enough to know how to do native zuulv3 things.
Swift has to be disabled since it doesn't support py3 (this is
something all other py3 jobs are doing right now too).
Change-Id: I9a0074d9658eab76b7c73b820f498d61b89545e7
This migrates the legacy-osc-placement-dsvm-functional job
from openstack-zuul-jobs and renames it to drop the legacy part
and branch override stuff.
Change-Id: I6dfcd4e327ea535fdee3ce740c9b817ed790de7e
Chris Dent