Commit Graph

45 Commits

Author SHA1 Message Date
harsha.dhake d05c20bb60 Adding credsmgr support in devstack
Change-Id: Ied7da6e314d81426e60063f7347dbeb1dc05216b
2019-06-17 14:36:03 +05:30
lijunjie dda3df3ffc Fix the misspelling of "except"
Change-Id: I0f7246eb5382e94cbb3f761b76e0cd50c03132ff
2019-02-25 10:16:57 +00:00
Harsha Dhake 3fc8b3f97d Added credentials manager and updated omni drivers.
This change:
1. Adds credmanager service which handles credentials for AWS drivers.
2. Adds support for managing multiple AWS accounts through use of credmanager. Each account is mapped to a single project in keystone.
3. Adds support for multiple AZs by running one nova-compute and cinder-volume process per AZ.
4. Improves support for AWS networking in neutron.
5. Also, made few stability fixes in GCP and Azure drivers.

Change-Id: I0f87005a924423397db659ab754caaa6cde90274
2019-02-25 00:38:42 -05:00
Pratik Shah 14f465f99d [AWS] Fixed error while deleting VPC
- Issue: When trying to delete network immediately after deleting VM,
throws "DependencyViolation" error. This is due to security group was
not getting deleted before deleting VPC

Change-Id: I8f1e642819188c56fddcfbb1337b2459c939e7f4
2017-11-08 10:45:47 +05:30
Pratik Shah fc2b12b79b [AZURE] Added unit test cases for Neutron
Description:
- Added unit test cases
- Fixed a bug which was throwing exception while creating resource group
in mech_azure.py

Change-Id: I4c96d96f96a0ac640407df5ff0d49937f2690fa3
2017-10-16 13:43:05 +05:30
Pratik Shah d407b5f482 [AZURE] Added a fix to create resource group in Azure if not present
Change-Id: I8ad06bd32d381573f794c37a577bb6ff2e056efb
Closes-Bug: #1720982
2017-10-11 09:37:20 +05:30
Sanket 38f171bdea Unit tests fixes in nova and neutron
Patch 1:
[General] Removed unused parameters from InstanceInfo

Many of unused parameters were removed from nova.virt.hardware.InstanceInfo in Pike.
Same was notified to OOT(Out of tree) drivers.

References:
[1] https://review.openstack.org/#/c/471146/6/nova/virt/hardware.py
[2] http://lists.openstack.org/pipermail/openstack-dev/2017-June/117962.html

Closes-Bug: #1717907

Patch 2:
[General] Remove l3_db.subscribe calls from router plugins

l3_db.subscribe has been deprecated from newton. Removing it
was recently disabled from pike release.

Refer bug description for more details.
Closes-Bug: #1718180

Change-Id: Ied1868736c4edb9da839d6d4b49aa1d98cbef251
2017-09-21 14:31:09 +05:30
Sanket 92ee8cde01 Cleanup unused requirements from Omni
Consolidating all python requirements required to run unit tests
in test-requirements.txt. Removed rest of all files.

Change-Id: Ifed7288e2b1d92fb180cb8c722578e9887f6f844
2017-09-11 09:39:39 +05:30
Jenkins b4369a7e5f Merge "Add Azure support for Neutron" 2017-09-08 23:22:01 +00:00
Sanket 370e7eeb72 Add Azure support for Neutron
Operations Supported:
1. Operations of network, subnetworks, router
2. Fixed IP allocation to VM from tenant network
3. Assign, delete, remove floating ip
4. Security groups and security group rules

Change-Id: Ia38f37356852c50da5b65cb5c55d1e73ef4a8041
Implements: blueprint azure-support
2017-09-08 23:34:48 +05:30
Pratik Shah d9fa689b8d Added a fix to run Neutron tests on master and newton branches.
Files modified:
- gce_mock.py: Changed line endings to 'unix'
- test_gce_router.py: Changed line endings to 'unix'
- test_ec2.py: Updated mocking for deleting and updating router
- aws_router_plugin.py: Added a fix when trying to remove router interface

Closes-Bug: #1715814

Change-Id: I8981e2e2207370d40bf786c7077ffc512755cc69
2017-09-08 22:09:10 +05:30
Jenkins 963d736599 Merge "[Trivialfix]Fix typos in omni" 2017-08-31 06:32:40 +00:00
Jenkins 4fdcdd95a8 Merge "[AWS] Added Neutron router plugin unit test cases" 2017-08-30 17:48:42 +00:00
Jenkins 93b9f1187f Merge "[AWS] Added Neutron mechanism test cases" 2017-08-30 17:38:03 +00:00
shangxiaobj 1aeb62b0ef [Trivialfix]Fix typos in omni
Fix the typos in omni.

Change-Id: I6e7c1e53752cdf064b12f9de7642af5d138f1463
2017-08-30 02:22:55 -07:00
Pratik Shah ffa3e4c212 [AWS] Added Neutron mechanism test cases
Files modified:
- omni-requirements.txt: Added boto3 package

- run_tests.sh: Added support for running AWS test cases for Neutron

Closes-Bug: #1708586

Change-Id: I91e46a5422a044eeccc171c8f820235e1dc583ef
2017-08-30 11:54:53 +05:30
Pratik Shah 0ac440c7d6 [AWS] Added Neutron router plugin unit test cases
Files modified:
- aws_router_plugin.py: Fixed bugs encountered while running test cases

- run_tests.sh: Added support for running AWS test cases for Neutron

- omni-requirements.txt: Changed version of moto and added boto3 package

- test_ec2.py in Cinder and Nova: With latest moto version, '@mock_ec2'
is deprecated. Hence Cinder and Nova test cases were unable to mock
EC2 credentials. Hence used '@mock_ec2_deprecated' instead of '@mock_ec2'

Closes-Bug: #1708585

Change-Id: I51d2bb99ef7d13b75940df6bbb33475de4a4630d
2017-08-30 11:50:51 +05:30
Pushkar Acharya df2a8218f7 Fix floating IP associate in AWS
The router plugin failed to create a valid nova
client and failed the floating IP associate API.

Change-Id: I8333deb06fc8d0358c0ddc41e768f390ea6be656
Closes-Bug: #1713820
2017-08-29 18:28:43 -07:00
Jenkins 6538ef949e Merge "Adding unit test cases for GCE Router plugin" 2017-08-25 05:34:29 +00:00
Yusuf Kanchwala 8e28863a4d Adding unit test cases for GCE Router plugin
Change-Id: I37df2e0cf1c16aba4d45ee9692c062f88ebc5d1c
Closes-Bug: #1707862
2017-08-18 07:58:47 +05:30
Jenkins 4aa67e3eea Merge "[GCE] Skip Security group rule verification for egress rules" 2017-08-11 02:36:44 +00:00
Jenkins 57727d3e19 Merge "Added a fix to delete a network." 2017-08-10 18:05:08 +00:00
Sanket 60e596283c [GCE] Skip Security group rule verification for egress rules
Neutron by default adds two egress rules to security group creation
API calls. If we block egress rules, any type of security
group creation fails. So we just log warning in case of neutron egress
security group rules being not supported on GCE.
Switched to gce beta APIs as firewall group creation calls are deprecated
on v1 APIs. Firewall related call report error "feature not supported yet"

Change-Id: I7baded2df5b34239e2cf99ca49c9d6c8eba46294
Closes-bug: #1709002
2017-08-10 16:09:54 +05:30
Pratik Shah d444a9511b [AWS] Fixed Security Group Creation Issues
Issues:
1. Security Group created using Openstack API, creates 2 outbound rules
(1 for IPv4 and 1 for IPv6) and no inbound rules. Hence the rule_dict
that was getting created contained same data for both outbound rules.
Hence while authorization of egress direction, it was throwing error
"ClientError: The same permission must not appear multiple times"

2. Since there are no inbound rules in security group, empty list was
passed to AuthorizeSecurityGroupIngress operation. Hence it was raising
"ClientError: Missing source specification: include source security group
or CIDR information".

3. While updating the security group which was created using API, I was
seeing error: "ClientError: Missing source specification: include source
security group or CIDR information" as there were no inbound rules.

Fixes:
1. Added a check for IPv4 ether type in "_convert_openstack_rules_to_vpc()"

2. Moved authorization for ingress inside if block

3. Added checks in "_update_sec_group()"

Closes-Bug: #1709545

Change-Id: I4e4acad4da3b095e7a8951abbfb94f1b0d9c7e41
2017-08-10 11:59:17 +05:30
Sanket f7085c1883 [GCE] Validate security group at the time of creation
Openstack Security Group should be validated if they are compatible
with GCE firewall rules. If not we should raise approriate error.
This fix processes security group info in BEFORE_RESPONSE event
of security group and rollbacks earlier created security group if
not compatible.

We can not use BEFORE_CREATE/PRECOMMIT_CREATE as they do not contain
required security group rules info.

Change-Id: I5f1fc67208085ef399f3dcfe5fdec63d4f2ffc51
Closes-bug: #1709002
2017-08-08 17:23:48 +05:30
Pratik Shah 3c2185aac3 Added a fix to delete a network.
Description:
Deleting network was raising an Exception
"AWS Error: 'NeutronError' - 'Required parameter id not set'"

Issue:
When VPC and Subnet are deleted, neutron core code notifies the change.
But since VPC is already deleted, "create_tags_for_vpc()" doesn't get
VPC ID. Hence it raises AWS Exception when it tries to create tags.

Neutron core code calls "update_network_precommit()" when notifying the
change which eventually calls "create_tags_for_vpc()"

Fix:
Added a check for VPC ID in "create_tags_for_vpc()"

Closes-Bug: #1709046

Change-Id: I6a94911e41bc194b98ddaf51cf9585ac3f4ed07e
2017-08-08 12:16:49 +05:30
Yusuf Kanchwala 7613ef6103 [AWS] Fix AWS Router Plugin Crashes
1. Handle deprecated imports for Router Plugin
2. Define AwsException at one place

Change-Id: I2986934ef4a89fcc914ca3aebd33428565632810
Closes-Bug: #1708573
2017-08-04 09:47:54 +05:30
Sanket 8d58fd2f16 Create GCE Service outside Neutron service start path
Creating gce client inside Neutron service start causes neutron-server
crash if service_key is not there at mentioned path. Neutron service
should not crash at start if service key is not there.

Change-Id: I8e0f8b150c60d55d71846face217d5213b5df9b7
Closes-Bug: #1707872
2017-08-01 17:57:58 +05:30
Pratik Shah 264939cfb5 Applied pep8 fixes
Closes-Bug: #1703514

Change-Id: I89c25bd9e678dc2f7cc27f952aa4833e26c90f4a
2017-07-24 16:25:25 +05:30
Pratik Shah c030c3d645 Removed neutron/neutron/common/exceptions.py as it is duplicate file from
neutron component

Added AWSException class in
neutron/neutron/plugins/ml2/drivers/aws/mechanism_aws.py

Fixed run_tests.sh bug to handle failed tests

Closes-Bug: #1704150

Change-Id: If52a55fdde8067a509ce48a3718699256d1eb132
2017-07-21 23:29:17 +05:30
Pratik Shah ed23220a8d Fixed a bug in Neutron GCE router to run on master and stable/newton
Issue:
- Classes used in gce_router_plugin.py are different on master and
stable/newton branch.
Example:
- In newton branch, we are importing "from neutron.db import l3_db"
and we are using this import as l3_db.Router. But on master branch,
l3_db doesn't have attribute "Router". Hence to solve this, we need
to import "from neutron.db.models import l3".

File Modified:
- neutron/neutron/services/l3_router/gce_router_plugin.py
Closes-Bug: #1702881

Change-Id: I4d872ee0034417cdf38e774d69d1acec7924cc17
2017-07-15 14:17:45 +00:00
Pratik Shah 84b9c632f5 Modified tox.ini to clone repos using zuul-cloner
Replaced "," with "%" in logs statement for files:
- cinder/volume/drivers/aws/ebs.py
- cinder/volume/drivers/gce/gceutils.py
- glance/glance_store/_drivers/aws.py
- neutron/neutron/common/gceutils.py
- neutron/neutron/plugins/ml2/drivers/gce/mech_gce.py
- neutron/neutron/plugins/ml2/managers.py
- neutron/neutron/services/l3_router/gce_router_plugin.py
- nova/virt/ec2/ec2driver.py
- nova/virt/gce/driver.py
- nova/virt/gce/gceutils.py

Added import fix in nova/tests/unit/virt/ec2/test_ec2.py

Modified run_tests.sh to get total number of passed and failed tests

Closes-Bug: #1704147

Change-Id: I4b144b8095cf9ff4807c4b9f4ca21577a43de27f
2017-07-14 13:19:22 +05:30
kavithahr 9e791e79ac Replace six.iteritems() with .items()
1.As mentioned in [1], we should avoid using six.iteritems to
achieve iterators. We can use dict.items instead, as it will
return iterators in PY3 as well. And dict.items/keys will more
readable.
2.In py2, the performance about list should be negligible, see
the link [2].
[1] https://wiki.openstack.org/wiki/Python3
[2] http://lists.openstack.org/pipermail/openstack-dev/2015-June
/066391.html

Change-Id: Ie875158b01666ac0b307492519146a8f005090bc
2017-07-10 18:00:22 +05:30
Jenkins 14f8258865 Merge "Remove translation of log messages" 2017-06-30 18:32:52 +00:00
Pratik Shah 874e3058de Added a fix for Neutron GCE to run on stable/newton and master branch
Files modified:
- neutron/neutron/plugins/ml2/drivers/gce/mech_gce.py
- neutron/tests/plugins/ml2/drivers/gce/test_gce.py

Change-Id: Ia91c77ede8acf9d90e98384cd5af5c027be4d685
2017-06-29 15:55:27 +05:30
Ngo Quoc Cuong 7823d3ddb4 Remove translation of log messages
The i18n team has decided not to translate the logs because it
seems like it not very useful; operators prefer to have them in
English so that they can search for those strings on the internet.

cinder, nova, neutron and glance are removing i18n log translation
functions, once it release, it will cause omni build failed.

Related-Bug: #1701139

Change-Id: I090fde16602945035cbda66f0633371d3dfebd3c
2017-06-28 21:25:52 -04:00
Pratik Shah 699caafaa0 Added tox integration to run unit tests for Omni project.
Description:

1. Fixed file paths in Nova, Neutron according to actual Openstack repository.
2. tox.ini: init file to run to tox command
3. run_tests.sh: actual script to run unit tests
4. omni-requirements.txt: Contains Python packages from Nova, Glance, Neutron and Cinder

Usage:
To run unit tests, execute 'tox -epy27' command in Omni project directory.

Change-Id: Ife3a5d4c009198fb46faff7a21525dd739a87e9c
2017-06-19 16:17:03 +05:30
Jenkins 0cf63a5004 Merge "Added Nova Instace Snapshot Operation and security group rules support" 2017-06-02 14:04:10 +00:00
Pratik Shah 59f3a47e61 [WIP] Unit Testcases for GCE Neutron Driver
Additional data files in gce/data:

1. Fake service key file(omni.json)
2. Data required to generate Gce Service(service/service_data.json)
3. Fake network data(network/get_network.json)
4. Fake firewall data(network/get_firewall.json)

Change-Id: I975af4d9ef712df1cc5aefc0835357f6681f29c1
2017-06-01 16:13:27 +05:30
Sanket 40e6f7b0ee Added Nova Instace Snapshot Operation and security group rules support
1. Added instance snapshot feature
2. Support for security group rules using GCE firewall rules
2. Added disk, snapshot related operations nova gceutils library
3. Cleanup unwanted operations from glance gceutils library

Change-Id: I1777e99cf4e1f7c230c4a9b286e4c01e8deb3d2f
Signed-off-by: Sanket <sanket@infracloud.io>
2017-06-01 14:42:58 +05:30
Sanket e4fd7a04f5 Add Neutron Support for GCE
1. ML2 Mechanism Driver for GCE
2. Router Service plugin for GCE

Change-Id: Id8d009f1b57cb37701be1957588ef5edd0acaf1c
Implements: blueprint gce-support
Signed-off-by: Sanket <sanket@infracloud.io>
2017-05-02 13:15:16 +05:30
Arun Sriraman db0e05ad14 Adding security group functionality and various stability fixes
+ FIP error handling.
+++ Openstack will not error when deleting a non-present elastic ip.
+++ Error in fip creation on openstack side cleares elastic ip
+++ Remove FIP even when in use if elastic IP not found

+ Waiters for EC2 network object creation
+++ Waiter for VPC and subnets

+ Adding security group functionality
+++ Can create/modify and delete security groups for VPCs

+ Detach VPC from IG stability fixes
+++ Clear VPC routing table entires on VPC interface removal
2017-01-04 18:14:50 -05:00
Arun Sriraman da5390230e Fix neutron credentials for boto3 connection 2016-11-02 17:28:34 -04:00
Arun Sriraman 8a214d927b Added security groups instructions to README.md 2016-10-26 08:29:47 -07:00
Sachin fa89beafab Initial commit: Introducing AWS Drivers for OpenStack 2016-10-26 00:12:14 -07:00