Fix API Tests

Currently, when we run api tests with command: "tox -e api",
tests are failing with errors.

This change first fixes errors, make api tests pass. Then refactor
the neutron-vpnaas api tests tree to fit the tempest plugin.

As a tempest plugin, the steps to run tests by hands are:

1. Setup a local working environment for running tempest
    tempest init ${your_tempest_dir}

2. Enter ${your_tempest_dir}
    cd ${your_tempest_dir}

3. Check neutron_vpnaas_tests exist in tempest plugins:
    tempest list-plugins
    +----------------------+------------------------------------------------------+
    |         Name         |                      EntryPoint                      |
    +----------------------+------------------------------------------------------+
    |    neutron_tests     |  neutron.tests.tempest.plugin:NeutronTempestPlugin   |
    | neutron_vpnaas_tests | neutron_vpnaas.tests.tempest.plugin:VPNTempestPlugin |
    +----------------------+------------------------------------------------------+

4. Run neutron_vpnaas tests:
    tempest run --regex "^neutron_vpnaas.tests.tempest.api\."

In the jenkins gate,  devstack-gate/devstack-vm-gate-wrap.sh will invoke tempest
with proper configurations, such as:
    DEVSTACK_GATE_TEMPEST=1
    DEVSTACK_GATE_TEMPEST_ALL_PLUGINS=1
    DEVSTACK_GATE_TEMPEST_REGEX="^neutron_vpnaas.tests.tempest.api\."
The actual raw command in gate running under the tempest code directory is:
    tox -eall-plugin -- "^neutron_vpnaas.tests.tempest.api\."

This is a part of activate the API job:
https://review.openstack.org/#/c/337011/

Change-Id: Icdc946bccb77b4c78d161a590de1aa15e0a57139

doc/source/devref/vpnaas-tempest-test.rst

@@ -0,0 +1,68 @@
+====================
+VPNaaS Tempest Tests
+====================
+
+This contains the tempest test codes for the Neutron VPN as a Service (VPNaaS) service. The tests
+currently require tempest to be installed via devstack or standalone. It is assumed that you
+also have Neutron with the Neutron VPNaaS service installed. These tests could also be run against
+a multinode openstack.
+
+Please see /neutron-vpnaas/devstack/README.md for the required devstack configuration settings
+for Neutron-VPNaaS.
+
+How to test:
+============
+
+As a tempest plugin, the steps to run tests by hands are:
+
+1. Setup a local working environment for running tempest
+    ::
+
+        tempest init ${your_tempest_dir}
+
+2. Enter ${your_tempest_dir}
+    ::
+
+        cd ${your_tempest_dir}
+
+3. Check neutron_vpnaas_tests exist in tempest plugins:
+    ::
+
+        tempest list-plugins
+
+    +----------------------+--------------------------------------------------------+
+    |         Name         |                       EntryPoint                       |
+    +----------------------+--------------------------------------------------------+
+    | neutron_tests        |   neutron.tests.tempest.plugin:NeutronTempestPlugin    |
+    | neutron_vpnaas_tests |   neutron_vpnaas.tests.tempest.plugin:VPNTempestPlugin |
+    +----------------------+--------------------------------------------------------+
+
+
+4. Run neutron_vpnaas tests:
+    ::
+
+        tempest run --regex "^neutron_vpnaas.tests.tempest.api\."
+
+Usage in gate:
+==============
+
+In the jenkins gate, devstack-gate/devstack-vm-gate-wrap.sh will invoke tempest with proper
+configurations, such as:
+
+    ::
+
+        DEVSTACK_GATE_TEMPEST=1
+        DEVSTACK_GATE_TEMPEST_ALL_PLUGINS=1
+        DEVSTACK_GATE_TEMPEST_REGEX="^neutron_vpnaas.tests.tempest.api\."
+
+The actual raw command in gate running under the tempest code directory is:
+
+    ::
+
+        tox -eall-plugin -- "^neutron_vpnaas.tests.tempest.api\."
+
+
+External Resources:
+===================
+
+For more information on the tempest, see: <http://docs.openstack.org/developer/tempest/>

neutron_vpnaas/tests/contrib/post_test_hook.sh

@@ -25,24 +25,19 @@ case $VENV in
     dsvm-functional | dsvm-functional-sswan)
         owner=stack
         sudo_env=
-        ;;
-    api)
-        owner=tempest
-        # Configure the api tests to use the tempest.conf set by devstack.
-        sudo_env="TEMPEST_CONFIG_DIR=$TEMPEST_DIR/etc"
+
+        # Set owner permissions according to job's requirements.
+        cd $NEUTRON_VPNAAS_DIR
+        sudo chown -R $owner:stack $NEUTRON_VPNAAS_DIR
+
+        echo "Running neutron $VENV test suite"
+        set +e
+        sudo -H -u $owner $sudo_env tox -e $VENV
+        testr_exit_code=$?
+        set -e
+
+        # Collect and parse results
+        generate_testr_results
+        exit $testr_exit_code
         ;;
 esac
-
-# Set owner permissions according to job's requirements.
-cd $NEUTRON_VPNAAS_DIR
-sudo chown -R $owner:stack $NEUTRON_VPNAAS_DIR
-
-echo "Running neutron $VENV test suite"
-set +e
-sudo -H -u $owner $sudo_env tox -e $VENV
-testr_exit_code=$?
-set -e
-
-# Collect and parse results
-generate_testr_results
-exit $testr_exit_code

neutron_vpnaas/tests/tempest/README.rst

@@ -0,0 +1,6 @@
+===============================================
+Tempest Integration of neutron-vpnaas
+===============================================
+
+This directory contains Tempest tests to cover the neutron-vpnaas project.
+

neutron_vpnaas/tests/tempest/api/base.py

@@ -21,7 +21,7 @@ from tempest import test
 
 from neutron.tests.tempest import config
 
-from neutron_vpnaas.tests.api import clients
+from neutron_vpnaas.tests.tempest.api import clients
 
 
 CONF = config.CONF
@@ -167,8 +167,8 @@ class BaseNetworkTest(test.BaseTestCase):
         ip_version = ip_version if ip_version is not None else cls._ip_version
         gateway_not_set = gateway == ''
         if ip_version == 4:
-            cidr = cidr or netaddr.IPNetwork(CONF.network.tenant_network_cidr)
-            mask_bits = mask_bits or CONF.network.tenant_network_mask_bits
+            cidr = cidr or netaddr.IPNetwork(CONF.network.project_network_cidr)
+            mask_bits = mask_bits or CONF.network.project_network_mask_bits
         elif ip_version == 6:
             cidr = (
                 cidr or netaddr.IPNetwork(CONF.network.tenant_network_v6_cidr))

neutron_vpnaas/tests/tempest/api/clients.py

@@ -25,6 +25,41 @@ from neutron.tests.tempest.services.network.json.network_client import \
 CONF = config.CONF
 
 
+class NetworkClient(NetworkClientJSON):
+
+    def pluralize(self, resource_name):
+
+        resource_plural_map = {
+            'ikepolicy': 'ikepolicies',
+            'ipsecpolicy': 'ipsecpolicies'
+        }
+
+        if resource_name in resource_plural_map:
+            return resource_plural_map.get(resource_name)
+
+        return super(NetworkClient, self).pluralize(resource_name)
+
+    def get_uri(self, plural_name):
+        # get service prefix from resource name
+
+        service_resource_prefix_list = [
+            'vpnservices',
+            'ikepolicies',
+            'ipsecpolicies',
+            'ipsec_site_connections',
+            'endpoint_groups',
+        ]
+
+        if plural_name in service_resource_prefix_list:
+            plural_name = plural_name.replace("_", "-")
+            service_prefix = 'vpn'
+            uri = '%s/%s/%s' % (self.uri_prefix, service_prefix,
+                                plural_name)
+            return uri
+
+        return super(NetworkClient, self).get_uri(plural_name)
+
+
 class Manager(manager.Manager):
 
     """
@@ -51,7 +86,7 @@ class Manager(manager.Manager):
 
         self._set_identity_clients()
 
-        self.network_client = NetworkClientJSON(
+        self.network_client = NetworkClient(
             self.auth_provider,
             CONF.network.catalog_type,
             CONF.network.region or CONF.identity.region,

neutron_vpnaas/tests/tempest/api/test_vpnaas.py

@@ -21,7 +21,7 @@ from tempest import test
 from neutron.api.v2 import attributes as attr
 from neutron.tests.tempest import config
 
-from neutron_vpnaas.tests.api import base
+from neutron_vpnaas.tests.tempest.api import base
 
 CONF = config.CONF
 

neutron_vpnaas/tests/tempest/plugin.py

@@ -0,0 +1,36 @@
+# Copyright 2015
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+
+import os
+
+from tempest.test_discover import plugins
+
+import neutron_vpnaas
+
+
+class VPNTempestPlugin(plugins.TempestPlugin):
+    def load_tests(self):
+        base_path = os.path.split(os.path.dirname(
+            os.path.abspath(neutron_vpnaas.__file__)))[0]
+        test_dir = "neutron_vpnaas/tests/tempest"
+        full_test_dir = os.path.join(base_path, test_dir)
+        return full_test_dir, base_path
+
+    def register_opts(self, conf):
+        pass
+
+    def get_opt_lists(self):
+        pass

setup.cfg

@@ -41,6 +41,8 @@ neutron.db.alembic_migrations =
 oslo.config.opts =
     neutron.vpnaas = neutron_vpnaas.opts:list_opts
     neutron.vpnaas.agent = neutron_vpnaas.opts:list_agent_opts
+tempest.test_plugins =
+    neutron_vpnaas_tests = neutron_vpnaas.tests.tempest.plugin:VPNTempestPlugin
 
 [build_sphinx]
 all_files = 1

tox.ini

@@ -40,13 +40,6 @@ commands =
   {toxinidir}/tools/deploy_rootwrap.sh {toxinidir} {envdir}
   sh tools/pretty_tox.sh '{posargs}'
 
-[testenv:api]
-sitepackages=True
-setenv =
- OS_TEST_PATH=./neutron_vpnaas/tests/api
- OS_TESTR_CONCURRENCY=1
- TEMPEST_CONFIG_DIR={env:TEMPEST_CONFIG_DIR:/opt/stack/tempest/etc}
-
 [testenv:dsvm-functional-sswan]
 setenv =
   OS_TEST_PATH=./neutron_vpnaas/tests/functional/strongswan