Commit Graph

162 Commits

Author SHA1 Message Date
Salvatore Orlando 624f312208 Various fixes for gate jobs
- Update BGP plugin for changes in neutron-dynamyic-routing
- Skip test for duplicate port MAC for NSX-V plugin
- Fix test for network creation in default AZ for NSX-V plugin
- Adapt .zuul.yaml for changes in tap-as-a-service
- Fix issues with requirements check job
- Minor changes to tox.ini

Change-Id: Idbf051e98d7c961273f46c6b6a1aaa9b358ab3d8
2021-08-19 14:35:44 -07:00
Adit Sarfaty 2a1991a3a6 Revert "Temporarily disable admin utils tests"
This reverts commit 09fbad47d8.

Reason for revert: Re-Enable the tests

Change-Id: I48e9e7f7e41084c8def0d46804a05dc17bdb5ade
2020-12-10 08:19:56 +00:00
asarfaty eeac28dd80 Update master branch requirements
Bump neutron-lib, and osc-lib, and update some more requirements

Depends-on: Ie74ea517a403e6e2a7a4e0a245dd20e5281339e8
Change-Id: If34a9889fb0f137856f7c241788cf593e722d665
2020-12-09 21:32:11 +02:00
asarfaty 50afa71853 Fix broken Victoria branch
1.Upgrade pylint to 2.4.4, add exclusions to the tests, and
  fix some lint errors in the code

2. Fix user creation with GRANT in MySQL 8.0(Ubuntu Focal)
In Ubuntu Bionic (18.04) mysql 5.7 version used to create
the user implicitly when using using the GRANT.
Ubuntu Focal (20.04) has mysql 8.0 and with mysql 8.0 there
is no implicit user creation with GRANT. We need to
create the user first before using GRANT command.
See also commit I97b0dcbb88c6ef7c22e3c55970211bed792bbd0d

3. Remove fwaas from the zuul.yaml
4. Remove DB migration test which is failing ue to FWaaS migration
with py38
5. Fix cover tests python version in .tox
6. fix requirememnts

Change-Id: I22654a5d5ccaad3185ae3365a90afba1ce870695
2020-09-21 15:31:18 +02:00
Ghanshyam Mann a3b61c2d27 Use the networking-l2gw project new location
networking-l2gw and networking-l2gw-tempest-plugin are
imported to new location now and supposed to be retired from
openstack/namespace[1].

-https://review.opendev.org/#/c/744110/

Change-Id: I902ea79cdd3a8e56540ab44a7bd2f986f5d18bca
2020-09-08 07:34:08 +02:00
asarfaty 1c9e123b80 Stop getting upstream master neutron-fwaas
As neutron-fwaas is no longer maintained, get it from pypi instead of Git.
Also update neutron projects in requirements to latested released versions

Change-Id: I866d0b6f57f7b34ccc8bc998b8df729103497f6c
2020-06-21 10:10:43 +02:00
asarfaty 98b2832e4d Add coverage tests
As a first step add coverage test as non-voting with a low threshold,
and exclude older plugins.
Also removing some unused code, and relocating tests-only code, and adding
some unit tests to improve coverage.

Change-Id: Ib7af0b5de49e1a0ee2927b01f2a5f71acf633fb5
2020-04-13 06:13:32 +02:00
asarfaty a3af5dc7e0 Support zuul & tox with py38
Commit Idbfefe85dc0ddb04bd2d88fafcf3f6e751a1c3d7 added py38 to
the gate jobs.
This patch adds the necessary configuration for the vmware-nsx

Change-Id: I47a8b8b553f4a8b852508ccc4e8df48a1cb87832
2020-01-27 09:18:47 +02:00
Adit Sarfaty 4a3e8191cd Stop testing python 2
Since it's no longer supported past Train, lets stop
running the tests.
Please also see the related neutron patch:
I1c171ab906a3b4c66558163ad26947ebf710a276

Change-Id: Ic5361ac8495493d45603a57fcaac5f8385c681b7
2019-11-02 08:21:10 +00:00
Boden R c3bf87fd05 update for python3 train jobs
Change-Id: Ic104e1a67e69a2e06fb47db23896a14cacf9ca24
2019-07-10 12:07:07 -06:00
Boden R 17d031ca8d update test-requirements.txt to only what's needed
This patch slims down the test-requirements.txt and
lower-constraints.txt so that we only require what we really use/need.
It also updates the flake8 section of tox.ini to match [1] except H904
is not included herein since vmware-nsx doesn't translate messages.

[1] https://review.opendev.org/#/c/667723/1/tox.ini

Change-Id: I4e0b1564640cf38b90e084abb410bd4de6d76733
2019-06-27 11:11:19 -06:00
Boden R 5362c65416 update bandit, hacking and flake8 requirements
This patch bumps the hacking, bandit and flake8 requirements to match
suit with similar work (ex [1]). It also updates the code to fix a few
new pep8 errors as well as adds a local tox target for
requirements-check-dev.

[1] https://review.opendev.org/#/c/658245/

Change-Id: I6caeb52dc1a5842338ec989a742ae5989608e0da
2019-05-31 08:42:37 -06:00
Adit Sarfaty d753ec6945 Remove neutron-lbaas support & dependencies
Commit Ia4f4b335295c0e6add79fe0db5dd31b4327fdb54 removed all the
neutron-lbaas code from the master (Train) branch

Change-Id: I9035f6238773aad0591436c856550b7a5e01e687
2019-05-19 11:16:45 +03:00
Boden R 4dfb2ecd50 cleanup openstack git refs and lower constraints
This patch:
- Updates git.openstack based URLs to use opendev.
- Cleans up the lower-constraints.txt file to only include what we
really need.

Change-Id: I3eecd97c313c33c820ca2be8f01f6848244cd52a
2019-04-30 07:16:41 -06:00
Boden R 841fa1a720 Dropping the py35 testing
All the integration testing has been moved to
Bionic now[1] and py3.5 is not tested runtime for
Train or stable/stein [2][3].

As per below ML thread, we are good to drop the py35
testing now:
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005097.html

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004647.html
[2] https://governance.openstack.org/tc/reference/runtimes/stein.html
[3] https://governance.openstack.org/tc/reference/runtimes/train.html

Depends-On: I1b0496d3048f832cd60be70703a23cc712f8693d

Change-Id: I071bd473617691ee72e29547e320dd8d9781ee75
2019-04-22 06:33:18 -06:00
Akihiro Motoki b7930ae821 Convert policy.json into policy-in-code
vmware-nsx specific policies are defined as policy-in-code.

- vmware_nsx/policies/lsn.py, qos_queue.py and maclearning.py
  are moved from the neutron repo.
- vmware_nsx/policies/providersecuritygroup.py is based on the difference
  between etc/policy.json and the old neutron policy.json
- vmware_nsx/policies/security_group.py is based on
  etc/policy.d/security-groups.json
- vmware_nsx/policies/network_gateway.py is based on
  etc/policy.d/network-gateways.json

etc/policy.d/dynamic-routing.json and etc/policy.d/neutron-fwaas.json
have no policies specific to vmware-nsx, so they can be dropped and
we can use policy-in-code definitions in neutron-fwaas and
neutron-dynamic-routing.

etc/policy.d/routers.json and flow-classifier.json cannot be
converted into policy-in-code because the default policies are
different from those defined in neutron and networking-sfc.
Note that etc/policy.d/routers.json now has policies which are
different from the default policies defined in the neutron repo.
(Others are clean up by this commit.)

This commit depends on the following patches under review:

(neutron-fwaas policy-in-code support)
Depends-On: https://review.openstack.org/527282
(neutron-dynamic-routing policy-in-code support)
Depends-On: https://review.openstack.org/625429
(networking-sfc policy-in-code support)
Depends-On: https://review.openstack.org/625431
(Drop 3rd-party plugin specific policies)
Depends-On: https://review.openstack.org/625394

Partially Implements: blueprint neutron-policy-in-code

Co-Authored-By: Michal Kelner Mishali <mkelnermishal@vmware.com>
Co-Authored-By: Adit Sarfaty <asarfaty@vmware.com>
Change-Id: I96a9dbd759d54308abbc12ce65c97b06a76453cd
2019-01-24 20:00:49 +00:00
Nguyen Hai Truong 5bb4f882df Update min tox version to 2.0
The commands used by constraints need at least tox 2.0.
Update to reflect reality, which should help with local running of
constraints targets.

Change-Id: I301278dd742037fd4a1ca498ace8e3a4e299bf56
2018-11-05 19:18:48 -08:00
Zuul 2dea64def5 Merge "NSX|V+V3: Octavia driver" 2018-10-07 09:25:57 +00:00
Boden R 822f7c4cf7 add py36 jobs and tox target
As part of the python 3 first effort [1], this patch adds UT support for
python 3.6 by:
- Updating the zuul.yaml to include the py36 template and jobs.
- Adding a py36-dev target to tox.ini for local testing.
- Updating the upper constraints to use master rather than stable/rocky.
- Updating the version of eventlet used as previous 2x.x versions had
a python 3 bug for monkey patching.
- Consolidating the eventlet monkey patching in the code.

[1] https://governance.openstack.org/tc/goals/stein/python3-first.html

Change-Id: I3c11ac9dff9a2e25b33e8310929c961ffd0bcc52
2018-10-02 08:23:08 -06:00
Kobi Samoray 83d9b3abdd NSX|V+V3: Octavia driver
Implementing the Octavia support for NSX-V & NSX-T.
Follow up patches will handle the TVD plugin, Status updates,
and migration.

Since Octavia is not (yet?) in the requirements, using a hack to allow unittests
to be skipped.

Co-Authored-by: Adit Sarfaty <asarfaty@vmware.com>
Change-Id: Iadb24e7eadcab658faf3e646cc528c2a8a6976e5
2018-10-02 11:19:55 +03:00
Adit Sarfaty 772f1c78ea Update UPPER_CONSTRAINTS_FILE for stable/rocky
The new stable upper-constraints file is only available
after the openstack/requirements repository is branched.
This will happen around the RC1 timeframe.

Recheck and merge this change once the requirements
repository has been branched.

The CI system will work with this patch before the requirements
repository is branched because zuul configues the job to run
with a local copy of the file and defaults to the master branch.
However, accepting the patch will break the test configuration
on developers' local systems, so please wait until after the
requirements repository is branched to merge the patch.

Change-Id: I7ae697a8023c76ec352616b269150633acdea119
2018-08-22 09:39:59 +03:00
Boden R 954e678202 add venv target to tox for release job
The release job will fail without a venv target in tox, so this patch
adds it to allow our release jobs to run.

Change-Id: I5cdeaed4455db352255906211d0092ab15e9a092
2018-07-19 22:16:13 +00:00
Nguyen Van Duc bd6835f37f Add py36 testenv
Python 3.6 is installed by default in Ubuntu 18.04 LTS.
Therefore, according to Transition Plan [1],
it'll be handy to have py36 testenv.

For more details, please check Python2 Deprecation Timeline [2]
and Python3-first Goal - Completion Criteria [3].

[1] https://wiki.ubuntu.com/Python/Python36Transition
[2] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html
[3] https://review.openstack.org/#/c/575933/8/goals/stein/python3-first.rst@42

Change-Id: I8eb2f5f4de5d9831bf539f2588389482c6f88d25
2018-07-17 12:29:09 +00:00
Boden R 2633ec0607 make python 3 the default for tox.ini
This patch is part of the community goals to enable python 3 first, and
only use python 2 when explict.

To do so, this patch:
- Makes python 3 the default env for non py27 tox targets.
- Adds a py3-dev target for running py3 locally.
- Refactors the pip install commands for stable dependency install into
their own target and refs them where needed.
- Updates the code to pass pep8 in python 3.
- Bumps the version of pylint to 1.7.1 to address some py3 issues in
earlier versions.

As part of this effort we should also look into making python 3 the
default for our VMware NSX 3rd party CI jobs.

Change-Id: Ibaa3e9d717f32ffb6479346163c14d4be7df50cf
2018-06-21 07:43:22 +00:00
Boden R 3cce888421 fix lower constraints targets in tox ini
This patch updates tox ini to support both a dev and build target;
lower-constraints-dev and lower-constraints respectively.
The dev target is for running locally so it manually installs the
required projects whereas the build target doesn't as per zuul's
required project definition in project.yaml.

Also see https://review.openstack.org/#/c/573386/

Change-Id: I0e6cf1ca0f4b07c080ff5a7b221dabedfd8bf6cf
2018-06-14 06:45:46 -06:00
wangqi d753897c6b tox: Remove unnecessary configuration
These were either unnecessary or overrode default configuration.

Change-Id: Id55b41ef1f16acb4eb763ed207df0f8aa6508520
2018-04-03 07:10:46 +03:00
Adit Sarfaty 794ae68734 Add lower-constraints job
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.

Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.

Add openstack-tox-lower-constraints job to the zuul configuration.

See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.

Change-Id: Ia65cd41c6d0f4a4847458188cc5afddb06035a23
2018-04-02 13:38:00 +03:00
Boden R d48a790378 add py27-dev and pep8-dev tox targets
This patch adds the py27-dev and pep8-dev tox targets that enable local
(outside the gate) running of py27 and pep8 using master branches from
our sub-project dependencies as discussed in [1] in editable mode.
If additional local/dev targets are needed (ex functional) they can be
added in the same fashion as shown herein.

With this patch to run pep8 locally use:
   tox -e pep8-dev

And to run py27 locally:
   tox -e py27-dev

The editable projects will be under .tox/{env}/src/

This patch also adds to our devstack/lib scripts by cleaning out any
existing (dependency git) projects and installing them in editable mode
from git.

Depends-On: https://review.openstack.org/552865
Depends-On: https://review.openstack.org/554297

[1] http://lists.openstack.org/pipermail/openstack-dev/2018-March/128328.html

Change-Id: I4d0dcba15e5b30e4cfa7335edf9c2dd961530184
2018-03-22 14:39:37 -06:00
Andreas Jaeger 4c0802d796 Remove tox_install.sh
Remove tox_install.sh, we have now all dependencies in
global-requirements and don't need this setup anymore.

Update tox.ini for this, move constraints into deps for proper
working.

This enables proper installation of dependencies from git.

Depends-On: https://review.openstack.org/552865
Depends-On: https://review.openstack.org/554297
Change-Id: I49d432bdf01221d0244449c4e2be807d0f646afa
2018-03-21 09:17:08 -06:00
Boden R 7223bbf963 migrate to stestr
ostestr is no longer installed by default by infra and will be totally
removed longer term; stestr [1] is its replacement.

This is the reason out gate is failing with the following today:

nsx/tools/ostestr_compat_shim.sh: ostestr: not found

This patch moves us to stestr by:
- Updating tox.ini to use stestr rather than call ostestr_compat_shim.sh
and in turn ostestr.
- Removing ostestr_compat_shim.sh as its no longer used.

For more details on stestr see [1].

Also note that this is a quick migration to stestr to get the gate
working; additional tweaks may come up.

[1] http://stestr.readthedocs.io/en/latest/MANUAL.html

Change-Id: I9a871d3542c2f2615d757c0009ea3d6695f0e3f7
2018-03-05 14:54:00 -07:00
Sean McGinnis 7dd27bfc7c Use the new PTI for document build
For compliance with the Project Testing Interface as described in:

https://governance.openstack.org/tc/reference/project-testing-interface.html

For more detials information, please refer to:

http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html

Change-Id: I9cf0bac7edc5f4d45af2729428f92d8a6fd7860d
2018-02-02 07:58:29 +01:00
Gary Kotton 508c6262a2 Remove vmware_nsx_tempest
This now existsin the vmware_nsx_tempest_plugin

Change-Id: I76e5856deeeb06b87675314635d06aa0291143eb
Depends-On: I804c3fd1e0c9cbeb454677e7951072ad74391fec
2017-08-15 17:02:55 +00:00
Gary Kotton 50cd210d59 Use flake8-import-order plugin
Automate the process

Change-Id: I54451e78c24eef2016293be87760a349df07a384
2017-07-05 00:39:10 -07:00
Gary Kotton 6228a06399 Drop log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.

See:
http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html
http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html

Change-Id: I9d37ae28a3fecbe910e60dc7f22e229a7b65940c
2017-03-26 05:09:02 -07:00
Rajiv Kumar ade538424d H402 hacking have been deprecated
H402 hacking have been removed since hacking 0.10.
https://github.com/openstack-dev/hacking/blob/master/setup.cfg.
So removed H402 ignore statement

Closes-Bug: #1654117
Change-Id: I31b160cc4267b9f3f77a6a75d671b695e509ee3f
2017-01-18 10:33:37 +05:30
Eric Brown 53b8498fc6 Remove references to Python 3.4
Now that there exists only a gate job for Python 3.5 and not 3.4,
we should remove those references to the 3.4 that is untested.

Change-Id: Idb66d124611de879b33c0f8bd20f37f24da443b4
2017-01-05 15:42:39 -08:00
melissaml 16affb58af Don't include openstack/common in flake8 exclude list
There is no this directory openstack/common which was used to keep codes from
oslo-incubator, we have retired oslo-incubator. Removing openstack/common from
all OpenStack code in favor of the Oslo libraries is a project wide goal
for the Ocata release. So don't use this directory any more. We should drop it
for improving searching efficiency.

Change-Id: I23f918ee6cd9939bfd46c4bf77cf56c0995c6e35
2016-11-24 18:09:43 +08:00
Armando Migliaccio a951f5f929 Make vmware-nsx capable of handling depends-on
Credits to networking-midonet tox_install* files.

Change-Id: I3ddf84317501ef516a33717b71f66399b9b26875
2016-11-21 18:10:14 -08:00
Abhishek Raut e009eed875 Fix test_migration import
Model migration tests were not run because of incorrect import.
Run them in unit test mode instead of functional jobs.
Also fix existing migrations and models which are not in sync.

Change-Id: I4f7362972929308201776214fe01c007f14a6580
2016-09-14 00:01:08 -07:00
AvnishPal d1b4a4b8b5 Update tox.ini for upper constraints
Openstack infra now supports upper constraints for
all jobs. Updated tox.ini to use upper constraints
for all jobs.

Change-Id: Ie446293f52a87ba89e090d566e3d62141a5c4f51
Closes-Bug: #1614361
2016-08-23 10:34:37 +05:30
Henry Gessau eef8943f79 Enable DeprecationWarning in test environments
Many deprecations are triggered early (on imports, for example)
before the warnings are enabled by the WarningsFixture in the
base test class.

To make sure all DeprecationWarning messages are emitted we enable
them via the PYTHONWARNINGS environment variable.

Change-Id: I02725cb71f5a55ccc257e3897b4141f99f4ab8d3
2016-08-12 07:15:55 +00:00
Jenkins 21001d4174 Merge "NSX-V Service insertion support" 2016-07-19 20:05:45 +00:00
Adit Sarfaty ce9003f498 NSX-V Service insertion support
The service insertion feature allows us to redirect some of the NSX traffic to an external
security vendor like Palo-Alto or checkpoint for advanced inspection.

The implementation contains:
Enable the flow classifier plugin, and use it to create redirect rules on NSX
When the flow classifier plugin is initialized a new security group is created
and added to the configured service profile

When a vm port with port security is created/updated, it is added to this security group
When the admin user create a flow classifier entry, a backed redirect rule will be created.

DocImpact: new NSXV Configuration parameters:
service_insertion_profile_id = <service profile id, i.e. serviceprofile-1>

DocImpact: The flow classifier methods should be added to the policy.json as admin only

Change-Id: I67a132d4b35764c6940516a8365a2749d574aad2
2016-07-13 11:29:05 +03:00
Boden R 28c14f567e Update gate logic
As bug 1568706 uncovered, we were using zuul-cloner
in our gate jobs; this was preventing our translation from
syncing.

After digging into this issue a number of changes in this
associated logic were found to not be in sync with neutron.
This patch updates out tox/tools logic to follow that of neutron.
In addition this patch fixes any pylint checks that were failing to
make pep8 pass.

IMPORTANT:
Please review closely, not only to the tools/tox updates but also
to the ignored pylint checks in the code. We only want to disable
checks where appropriate.

Change-Id: I6c5fee3ca3073ad079eac1636cc3b9ec45926a68
Closes-Bug: #1568706
2016-07-12 09:05:22 -06:00
Eric Brown b152dda04b Add Python 3.5 venv and classifier
Now that there is a passing gate job, we can claim support for
Python 3.5 in the classifier. This patch also adds the convenience
py35 venv.

Change-Id: I14f4f90ca0f0c863f9175934f63c95b9115f9b8c
2016-07-09 22:31:01 -07:00
Abhishek Raut 00610df088 [NSXv3]: Tap-as-a-Service NSXv3 driver
This patch adds support for Port Mirroring feature for
NSXv3 plugin. This driver implements the Tap-as-a-Service
APIs and maps the tap-flow and tap-service resources
to backend PortMirroring objects.

Closes-Bug: #1598318
Change-Id: I5719fbbacfd3ec39250c59ec73cac2dc84af596d
2016-07-09 13:41:39 -07:00
Aaron Rosen c7ac488a5f python3: make unit tests pass
now all tests pass with tox -epy34

Depends-On: I5894485e55c04a8ca69825128798227714550c9d

Change-Id: I719a6cddcbe7f2b7a15bcd35375075affc2513b8
2016-07-07 14:42:35 -07:00
Aaron Rosen 2bee0a331f Show statistics after running coverage
TrivialFix

Similar to what was done here: I98a350d5eb9c217c99adf151e207523601712316

Change-Id: I78d577b2dc0e2378ec88bb4e938a41b0b0333985
2016-07-05 13:51:57 -07:00
Doug Wiegley 0de44da638 Switch to using hacking checks from neutron-lib
Temporarily comment out the 'no neutron' check.

Change-Id: I34d5204c90aed34571dbdf6d501dbcb682d57d81
Partially-Implements: blueprint neutron-lib
2016-04-24 23:21:55 -05:00
Abhishek Raut 6c1d841ff3 Automatically generate vmware-nsx configuration files
This patch adds support to automatically generate config files
for vmware-nsx repo using oslo config generator[1] for all
VMware plugins.
Tox can be used to generate a sample config file using the
following command:
    tox -e genconfig
This will generate a config file "nsx.ini.sample" under
vmware-nsx/etc/ folder.

This patch also modifies devstack scripts to use the
auto-generated config files and adds more information to help
texts in config modules.

[1] http://docs.openstack.org/developer/oslo.config/generator.html

Change-Id: Iff4ea37b52616295b262ead53947acb5b0cd9cd7
Partial-Bug: #1568215
2016-04-13 07:24:06 +00:00