The imp module is deprecated[1] since version 3.4, use importlib to
instead
[1]: https://docs.python.org/3/library/imp.html
Change-Id: Iba4d686f9452728dd6c90a5c2e37e59dc78c6e2b
This patch fixes multiple issues to unblock gate:
1. With new isort releases (5.X.X) pylint started to fail, with error:
AttributeError: module 'isort' has no attribute 'SortImports'
This patch pin isort to 4.3.21, the last version before 5.0.0.
2. lower-constraints of appdirs and pytz need to be updated as
currently lower-constraints job fails.
Since patch Ia012a8e116a276a6674f86366c803e0e2d8ff704 modified the
NEUTRON_PATH variable the functional test fails with:
[..]neutron/tools/configure_for_func_testing.sh: line 113: /home/zuul/src/opendev.org/openstack/neutron-vpnaas/devstack/lib/ovs: No such file or directory
3. Functional job fails, because during tox run the latest released
neutron is installed and it requires mock to be installed. This
patch adds mock to functional test requirements. Note: this needs
to be deleted as soon as new neutron will be released as on
development branch neutron does not depend on mock anymore.
Change-Id: Ibc4cd7f8a055457b288ef101893c252cf0464a28
neutron CLI is deprecated and will be removed in the future. Use
openstack CLI instead.
Highlighted texts are now emphasized by "emphasize-lines" option
of the code-block directive.
Co-Authored-By: Akihiro Motoki <amotoki@gmail.com>
Change-Id: I552171fe3e22290580e2cdff20adccf655573d03
According to Openstack summit session [1] stestr is
maintained project to which all Openstack projects
should migrate.
Let's switch it then.
[1] https://etherpad.openstack.org/p/YVR-python-pti
Change-Id: Ia4c71631e4c51ec814bcefc115d4f5690c11e37b
Instead of using tox-install.sh, we can install horizon and neutron
now directly from pypi - or use the current branch in CI system using
tox-siblings.
Update tox.ini for this change and add neutron to requirements file.
Change-Id: I435297ada6cdeb53a9bd8da8b88714f2c6a838a2
This patch adds doc for how to test VPNaaS with devstack
in order to help developers, operators and users easy to
understand VPNaaS as fast as possible.
This patch addresses the gap and improves docs as
discussed in Boston summit.
The content of this patch is basic inheritted from wiki page [1]
[1] https://wiki.openstack.org/wiki/Neutron/VPNaaS/HowToInstall
Change-Id: I8465cb90645af0fc3ea8cda554b0477b15f01fbe
Related-bug: 1692131
This is the iteration of the VPNaaS Agent with some basic
functionality to enable integration of Plugin - Agent - Driver.
Co-Authored-By: Van Hung Pham <hungpv@vn.fujitsu.com>
Change-Id: I0b86c432e4b2210e5f2a73a7e3ba16d10467f0f2
Closes-Bug: 1692128
This makes tox use the latest version of the tool from os-testr repo,
which contains some valuable changes, like the one making the gate fail
if for some reason no tests are executed because all of them are
skipped.
Change-Id: If88be4b310d746ccc885e3d0c1db8a4dfabb2a28
As part of Id56b9b70fb9c4113f4641e5b3f35ab63cfff6d4e, the special
handling of NEUTRON_DIR was removed. Readd it since it's needed to
install the proper version of neutron when testing neutron changes with
neutron-fwaas's functional tests. Otherwise those tests install neutron
HEAD.
Add a big comment to explain the reason for this.
Change-Id: Ibe1a057958f7c5b791788e2b44a1f13993a620bf
Use -x to get proper debug output of the tool.
Paul pointed out that my previous change broke neutron and
neutron-vpnaas testing together in a very specific case. Looking at this
in detail, we need some more debugging output to fix the problem. This
script is not logged right now, so, enable logging with -x and then
uplaod logs with an infra change.
Change-Id: I6546d71c51eec74c7997d0da44db79624774c073
Do not install in /tmp/openstack/neutron and leave the git repository
there after the script is run - if zuul-cloner is used.
We run jobs on long lived workers and also on developers machines, let's
cleanup afterwards.
Install into a temporary directory that can be removed with "git clean"
afterwards. Add this directory to ignore list of pep8.
Simplify with using pushd/popd.
Also, remove setup of ZUUL_BRANCH, this is not needed with current zuul
anymore.
Sync file with neutron-lbaas.
Change-Id: Id56b9b70fb9c4113f4641e5b3f35ab63cfff6d4e
A VPNaaS test case for 'tox -e api' was failing as
the test_vpnaas.py code catching a tempest_lib.NotFound
but the tempest code is throwing a tempest.lib.NotFound
causing a Mismatch.
==============================
Failed 1 tests - output below:
==============================
neutron_vpnaas.tests.api.test_vpnaas.VPNaaSTestJSON.
test_create_update_delete_ipsec_policy[smoke]
Captured traceback:
~~~~~~~~~~~~~~~~~~~
Traceback (most recent call last):
File "neutron_vpnaas/tests/api/test_vpnaas.py", line 312,
in test_create_update_delete_ipsec_policy
Fix was to change to tempest.lib, as recommended in:
/usr/local/lib/python2.7/dist-packages/tempest_lib/__init__.py:28:
DeprecationWarning: tempest-lib is deprecated for future bug-fixes
and code changes in favor of tempest. Please change your imports
from tempest_lib to tempest.lib
DeprecationWarning)
Fix base.py for same issue, plus clean up some obvious code
issues like a 'self' for a class method and indentation while
in there.
And since bash not my default shell, fix tox_install.sh to
use bash as otherwise get error.
opt/stack/neutron-vpnaas/tools/tox_install.sh: 19:
/opt/stack/neutron-vpnaas/tools/tox_install.sh: Bad substitution
Change-Id: I2ced6fd5cf9cc8a6972e6c937e1da4a0fc0236f4
The script was pinning oslo.config to 3.0.0, which does not work
now (deprecation error). Changed to not pin to a version and
elaborated on note as to why this needs to be there.
Change-Id: I970903983229e37dfc93dba396914a01fa5bc057
Was seeing py34 (and DSVM) tests failing with an Invocation error, even
though the test passed. Switched to ostestr, as is done in Neutron under
commit 240facf9e67aa5b205679f397107abfe059a869a, and the test passes
(not sure what the root cause is here).
Change-Id: If85987865e0393892c52f319d7cef526c021e0df
tools/tox_install.sh exists to deal with installing neutron without
neutron existing as a pypi package.
The way this operates causes problems in the periodic jobs on the stable
branches. Make it explicit which branch to install in the gate
(zuul-cloner) and via pip.
This is done with a BRANCH_NAME variable to make it trivial to change
once "master" is tagged as stable.
Change-Id: I484637d2502f276313edc2279078a200ebc2689a
Added support for running the constraints version of DSVM
functional targets, made constraints targets the default for
gate invocation, and tested each target variation.
Removed the functional test coverage related targets, because to
run them, requires that DSVM is configured (like the functional
tests).
This will need I3a2c3c81533ce6145cd16c2a08a4a2af0dfb0d07
upstreamed, so that the constraints jobs can all be tested
(as experimental).
Change-Id: Idb9902418ea2e6f35b00ffb78a8973e49b6e140d
Partial-Bug: 1522503
Since the script is called multiple times during install, we loose
valuable information. Also, tox is capable of capturing and hiding logs
on its own. Finally, they say the exec call is not safe for concurrent
use.
No reasons to keep it. Die witch die.
Change-Id: I1dfa3a0c4044ac8fc772b29ecf0bda78f58b94fc
This adds a new tox environment, genconfig, which generates sample
neutron VPNaaS configuration file using oslo-config-generator.
Updates to some configuration option help messages to reflect useful
details that were missing in the code but were present in config files.
DocImpact: Update the docs that VPNaaS no longer includes static example
configuration files. Instead, use tools/generate_config_file_samples.sh
to generate them and the files generated now end with .sample extension.
Partially-Implements: blueprint autogen-neutron-conf-file
Change-Id: I4a6094b8218dfd320d05bfb1e3bc121e8930c551
Partial-bug: #1199963
These are eventually going to be used in gate instead of unconstrained
jobs. There is some code duplication in commands definitions, but tox
does not allow to inherit definitions with {posargs} substitution.
Made tox_install.sh use constraints file to install both the project and
neutron dependency, if the first argument is 'constrained'.
Note: To test the docs targets, needed to fix a formatting issue in
multiple-local-subnets.rst document.
Change-Id: I699022b285594edf3c5db10fbdac2f904dc42928
Partial-Bug: #1522503
Depends-On: Icfc50a0a9330dbc2a59f9bb27f09944bdf0e00d1
In most cases, ZUUL_BRANCH is set in gate, but not all the time. One
exception is e.g. periodic stable jobs, where there is no actual patch
to validate, hence no ZUUL_REF or ZUUL_BRANCH is set. In that case, we
need to help zuul-cloner to determine the proper branch to fetch
neutron.
It could be achieved by e.g. hardcoding the branch using --zuul_branch
CLI argument passed into zuul-cloner. In this patch, I instead set
ZUUL_BRANCH to BRANCH that seems to be always present in all kinds of
jobs. It is then used by zuul-cloner to choose the correct branch.
This fixes the issue for stable/liberty periodic jobs for neutron-vpnaas
when we tried to validate it against neutron/master.
Change-Id: I360239a1443f8d4e16210a1ef743912e1bf85f41
Currently, when the VPN functional tests run, it uses tox_install.sh to install
neutron code (as well as other packages, as passed in).
For functional tests invoked via the post_test_hook.sh script, tox_install.sh
would try to "import neutron", and if successful, then it is assumed that
Neutron is already installed. Otherwise a check is done to see if it can pip
install using the cache from Zuul cloner. For a local test run, tox_install.sh
would pip install Neutron from git.openstack.org.
If, however, one invokes the new Neutron experimental job that runs the VPN
functional tests on a Neutron patch set, tox_install.sh was determining that
Neutron was already installed. This was using the upstream version of Neutron
in /tmp/openstack, instead of the changeset code in /opt/stack/new.
This change attempts to fix this case, but first checking to see if the
Neutron directory exists at /opt/stack/new ($HOME/neutron actually). If it
does exist, that is pip installed for the VPN functional tests. Otherwise,
it does the same checks as explained above.
The end goal here is to be able to detect cross project breakages, by using
the (currently experimental) gate-neutron-vpnaas-dsvm-functional-sswan.
Change-Id: Ifa8ef994a571c72236a4a22a4eeb27496202db2b
With the new DevStack plugin for neutron-vpnaas, the method names for
various stacking actions were renamed. One is used by the functional
test setup script tools/configure_for_vpn_func_testing.sh.
The change wasn't noticed, because the neutron-vpnaas repo still was
using DevStack's existing VPN setup, where the old method name is still
in use. But, when removing the VPN setup in DevStack, then the function
tests will fail.
This commit renames the method to match the name of the DevStack
plugin method, instead of the older DevStack repo method. It'll be
needed, before upstreaming Iffa9901e24adbacb581425c4b38c7e8cff0da9e8.
In addition, it makes sure that VPN environment variables are available
for use by the configuration script, and reuses more plugin.sh code,
to remove duplication.
Change-Id: Ib22a4b9abdc82c2b14d933b02329c6483c075ea8
As part of the work to allow VPN functionals job to run when Neutron
commits are reviewed, we need to be more intelligent about how
dependencies are resolved in the tox.ini for this repo.
This commit does the same as LBaaS, using a script to determine whether
an existing Neutron repo exists (and use that for a Neutron patch-set
running the VPN tests) or whether the Neutron repo needs to be cloned
(for a VPN patch set).
With the VPN repo, the rootwrap config file and filters need to be set
up. The deploy_rootwrap script is modified to do all actions related to
setup, instead of doing part of that in tox.ini, as before. In addition,
rootwrap config and needed filters for testing from Neutron are added to
this repo, so there is no dependency on Neutron (whose location cannot
be assumed in the gate).
NOTE: To run functional tests locally, one needs to set VENV for the
desired functional target and invoke configure_vpn_for_func_testing.sh.
Next, run tox for the functional target with --notest to setup the
virtual environment. Then, install Neutron into the virtual env. Finally,
the functional test can then be invoked.
This must be upstreamed, before allowing the VPN functional job to run
for Neutron jobs.
Change-Id: Idb3c4dff0dbb40e30b2b49591df0b014b769f167
Functional tests need to have access to configuration files.
Since the tests are started under 'stack' user
(sudo -H -u stack tox -e dsvm-functional) it would be better
to change owner of the Q_VPN_CONF_FILE file to STACK_USER.
Change-Id: I1ca677a805872726667df641255201cdd4cf4037
This commit does several things to allow functional tests to run, using a
DevStack configuration, but without stacking.
First, the gate_hook.sh is modified to specified environment variables
needed for processing, and will call configure_vpn_for_func_testing.sh.
The new configure_for_vpn_func_testing.sh script will first call the Neutron
configure_for_func_testing.sh script (as modified in [1]) to setup DevStack.
The script then installs the IPSec package, based on the functional job, and
will create the vpn_agent.ini file, in case it is needed for tests.
Minor changes are made to the post-test_hook.sh to indicate the location for
the Neutron and VPN repos.
Lastly, tox.ini is modified to place the rootwrap related files into the
virtual environment, copy in the VPN rootwrap filter, set up the rootwrap
commands, and copy the (customized) rootwrap.conf to /etc/neutron.
This last step is needed, because the neutron-vpn-netns-wrapper script that
is (currently) used for StrongSwan, uses /etc/neutron/rootwrap.conf and
doesn't allow overriding. This copying is a simple approach that will work
when multiple functional jobs are running, as the same config is used.
As a future follow-up commit, we could look into modifying the VPN device
drivers to pass the full path to the rootwrap.conf as a argument to the
neutron-vpn-netns-wrapper script (--rootwrap_config). The setting could
be added to the vpnagent.ini config file, available via config, to allow
this to be customized to something other than /etc/neutron (for production
use), and the functional tests could override it to point to the right
virtual environment.
To configure the environment for local run of the functional test, one
can do:
tools/configure_for_vpn_func_testing.sh <devstack-repo> [-i]
where:
devstack-repo... Path to devstack repo (/opt/stack/new/devstack
when run by gate)
-i.............. Indicates to install Neutron package dependencies
For StrongSwan, prefix the command with VENV=dsvm-functional-sswan
Ref: [1] https://review.openstack.org/#/c/176064/
Change-Id: I2243b06b3cdbb352dac7bc47d03c830efb87be3c
Closes-Bug: #1445052
Depends-On: Ie490e89c1a65e126ad0a57d062076e472762837e
Like Neutron, removing the check for bash script usage and
changed bash scripts to use '#!/usr/bin/env bash'.
Change-Id: Iab8ce0382695be8d8c7cd644b036dbf41dfd90b3
Closes-Bug: #1441244
Similar to what is happening in Neutron, this commit renames and moves unit
test files so that they match the directory structure of the code tree.
Since the VPN repo's test files have imports from Neutron, there are import
changes as well, to the imports to match the Neutron test file organization.
A check script is added to monitor the test file naming and locations (to
ensure consistency).
Change-Id: I875285230127ed660137d271f7e98d82eb57ffab
Partial-Bug: #1440834
Namespaces are not used anymore, so imports are for oslo_concurrency,
not oslo.concurrency.
lockutils direct call is not supported anymore, so using shipped
lockutils-wrapper instead.
Closes-Bug: #1387092
Change-Id: I2ba2295f8f5d55f272a9d23555940586b25b5a1c
While bash is not available on every environments, there seems to be
a tendency to introduce more bash dependency very casually. This check
is intended to be a reminder to give people a chance to consider
alternatives.
Related-Bug: #1398266
Change-Id: I5103e0d7c111880d9a02dd93de56c567066ed14c
A recent change (commit 0d5a11d9c722870f9c5e31a993219c7e240b4e19)
introduced bash dependency. It broke my environment, where bash
is not available. This change aims to restore it.
As far as I undestand, the change in question requires bash for
the "pipefail" feature. Fortunately there's a more portable
well-known trick for the purpose.
Closes-Bug: #1398266
Change-Id: Id3561d81462923d20ad39584ba4cf8e1a294a8a8
This patch imports some magic to enable showing test output as the
tests run. Seeing output showing tests as they execute is reassuring
that the test run is continuing to make progress. This originated in
tempest and was then brought over to Nova in this commit:
https://review.openstack.org/#/c/121914/
Note the justification in that commit to just copy it instead of move
it to a shared repo.
To get the old behavior where you only get output when something
fails, you can set the TRACE_FAILONLY env var. That originated in
this commit in Nova:
https://review.openstack.org/#/c/123801/
Change-Id: I6a079ad5f630383650c0f9dee09e04250d0e2c5a
In tools/check_i18n.py, the error messages printed out
had format strings that included 5 interpolations, but only
four variables were supplied. This patch simply corrects that
mismatch by including the name of the predicate or checker
that failed.
Closes-Bug: #1263813
Change-Id: I5cacee9ea000f56a9ef45f07367e85f20fa333c7
See I62ce43a330d7ae94eda4c7498782a655e63747fa for the gorey details on
why this exists.
As of this fix:
https://github.com/eventlet/eventlet/pull/34
which was released in eventlet 0.13, we no longer need the patch.
This has now been removed from oslo-incubator, so this is really just
syncing that removal.
Change-Id: I84267f3c6726cb2e750f615e107c48b12c6ed353
In python 3 print statement is not supported, so we should use
only print() functions.
This patch also removes print in unit test, left by accident
Fixes bug 1226943
Change-Id: I5ace50cb9e149682344b4c986ef9318f8dc50f72
Partial-Bug: #1217100
Using tools/check_i18n.py to scan source directory, and fix most of
the errors.
- Message internationalization
- First letter must be capital
- Using comma instead of percent in LOG.xxx
Note: all extension's description are not touched in this patch,
can be fixed after discussing.
Note: all nicira/check_nvp_config.py print messages are not fixed.
Change-Id: I79ef06fd42f6780beb5019c592662536c2a51864
wu.shiming