A recent change upstream has broken the python39 job.
Remove voting rights for this gate temporarily, as
python39 currently isn't being deployed.
Change-Id: Ib664e576f306d16afc20a1a4d62c8105cece2877
Add support for wallaby.
Below were the extra changes needed to support
the wallaby branch:
1. Add new attribute 'remote_address_group_id'
for the security group resource.
2. Handle new standard_attr_id argument for resources.
3. Fix kwargs passed to the alembic migrations
create_foreign_key and create_primary_key.
4. Change CONTEXT_WRITER to CONTEXT_READER in the
get_subnets function.
Change-Id: I3835df151cad2f7ca52afcb701de2bc508c90014
Add support for stable victoria. Changes include:
* https://review.opendev.org/#/c/716049/ switched to unittest for mock
Change-Id: I053657f535d985205ae9d3548291ec1d1409cb74
Some repos get their stable branch from the upstream requirements.
This patch reverts to using that for neutron.
Change-Id: I6401e83e85a0e0256bf31960c7fa7013b8909be5
Add support for the ussuri stable branch.
* Removed use of services in devstack (e.g. FWaaS and LBaas), which
were only used by the deprecated legacy plugin.
* https://review.opendev.org/#/c/572767/ changed the return
value of _get_security_groups_on_port from a list of security
group IDs to a list of security group OVOs. The monkey patch
of this method has been updated to be consistent with this
upstream change.
* https://review.opendev.org/#/c/703143/ removed the upstream
get_binding_levels, which is replaced by the corresponding
OVO call, get_binding_level_objs.
* https://review.opendev.org/#/c/709122/ broke the __repr__
method in the AddressScope model class. This patch works
around this by using the dictionary representation instead.
* https://review.opendev.org/#/c/679399/ made the MTU field
of networks non-nullable, and sets it to a constant if not
set explicitly. This broke GBP APIs which create networks
as part of their implementation. This patch adds a monkey
patch to pass in a value of 0, if one wasn't specified.
* Fixed alias uncovered by PEP8 checks.
Change-Id: I219bc9a5c2034499e59788ab11ef0ae310e97e1e
Import stable/stein rather than stable/rocky branches of upstream
and ACI-specific repositories.
Changes include:
* https://review.opendev.org/#/c/634790/ removed the rpc module
from neutron.common, which was rehomed to neutron-lib.
* https://review.opendev.org/#/c/634497/ removed the exceptions
module from neutron.common, which was rehomed to neutron-lib.
* https://review.opendev.org/#/c/581377/ removed exercises from the
devstack gate. The shell scripts that ran the tests from the
devstack exercises are now called directly.
* https://review.opendev.org/#/c/619087/ removed the common_db_mixin
from the FlowClassifierDbPlugin, replacing it with the use of a
method in neutron-lib.
* https://review.opendev.org/#/c/595369/ removed _setUpExtension,
replacing it with the setup_extension method.
* https://review.opendev.org/#/c/623415/ added validation to host
route CIDRs. The metadata CIDRs have been corrected to pass
this new validation.
* https://review.opendev.org/#/c/615486/ added a call to get a
nova client, and https://review.opendev.org/#/c/368631/ was
added to ensure it was a singleton. These are now used to get
a notifier for nova.
* https://review.opendev.org/#/c/628033/ removed the use of the
_resource_extend module, which has been moved to neutron-lib.
* https://review.opendev.org/#/c/585037/ converted policy.json
to policy in code. This resulted in better policy enforcement,
and flagged problems with existing UTs, mainly in the use of
shared resources (requires admin privileges). These UTs have
been fixed.
Change-Id: Ia7bd0799a814e38ff37b7ff062fa1eae7928991c
Import stable/rocky rather than stable/queens branches of upstream
and ACI-specific repositories. Changes needed for compatability
with stable/rocky that were also compatible with stable/queens were
made in previous patches, so only rocky-specific changes are included
here.
Change-Id: If533a955fb4bc23d6e4081a43df7018b1b36a0ba
Enhance compatabilty with newer Neutron branches while maintaining
compatability with stable/queens Neutron, and improve the build/test
process. Highlights include:
* Eliminate unneeded requirements and test-requirements, and update
remaining ones to match upstream stable/queens Neutron.
* Use pip directly instead of the tox_install.sh script to install
dependencies, as is done on newer upstream branches.
* Use stestr directly instead of ostestr to run UTs, as is done in newer
upstream branches.
* Specify basepython as python2.7 for pep8, cover, functional and
dsvm-functional jobs, in case a python3 version of tox is used.
* Fix pep8 issues that result in failures with the versions of hacking
and flake8 used by Neutron's stable/rocky through stable/train
branches. These changes are not necessary with the hacking and
flake8 versions used in stable/queens, but we want to minimize code
differences across our currently supported stable branches.
* Enable flake8-import-order and fix all the pep8 issues that it
uncovered, particularly with order and grouping of import
statements.
* Update pep8 configuration in tox.ini to more closely match upstream
Neutron, and fix resulting issues. Remaining ignored checks that
should be fixed but haven't been are marked with REVISIT in tox.ini.
* Update devstack scripts with proper branches and repository URLs.
Change-Id: I538b8c95c61a09d834be4b7c28a3becf2f3e6a50
1) Revert "Remove tests for master branch", commit
d149f30a4b.
2) Use test-requirements.txt from stable/queens to select the
stable/queens branches of python-opflex-agent and
python-group-based-policy-client, and the noiro-lite branch of
acitoolkit.
3) Use pushd/popd in tox_install.sh to restore initial CWD after
switching requirements branch.
Change-Id: I39895732aac0bdfaee95274cbcb262d6744faeb1
This is an ugly temporary fix to the upstream master gate,
until we can work out the correct way to fix it with the
openstack-infra folks.
Change-Id: Ibc76c899074890564b4579ee6e6e66e9a4af12e7
Previously, if an external network's AIM L3Outside referenced a VRF
that did not exist, validation would fail with a message that an
exception occurred, but with no indication of what caused the
problem. Now, the backtrace is logged when an exception is caught, a
specific error is reported if any of the resources needed for the
NetworkMapping DB record are missing, and a very specific error is
reported if the external VRF is missing.
Also, a requirement on acitoolkit is added to test-requirements.txt to
ensure that acitoolkit's noiro-lite branch is used. This is intended
to avoid acitoolkit's master branch's requirement on deepdiff, whose
latest version no longer supports python 2.
Similarly, the configuration files for the
legacy-group-based-policy-dsvm-aim CI job are modified to explicitly
install the noiro-lite branch of acitoolkit.
Change-Id: I7955f8e77633d9662a629c8c0628b128be3ae546
Based on the openstack networking-sfc project API:
- Introducing an SFC driver that maps the SFC model (port pairs,
port pair groups, port chains) to the AIM Service Graph model;
- Introducing a FlowClassifier driver that maps the FLOWC model (flow
classifier) to the AIM Service Graph model;
- Adding some registry notifications to the AIM MD and the FLOWC driver
for business logic validation.
Current divergence/limitations from the upstream SFC API:
- Added 2 l7_parameters to the flow classifier API,
source_logica_network and destination_logical_network. Representing
the networks involved in the traffic redirection mechanism;
- Every valid flow classifier must include the l7_parameters as
mentioned above. Internal networks and SVI networks are valid values,
but --external networks are excluded;
When SVI networks are specified, the corresponding source/destination
IP prefix must be specified in the API;
- Any other FlowClassifier parameter other than the ones mentioned
above will be ignored;
- On port binding, the chain will fix itself;
- Trunk ports are supported on port-pairs;
- On PPGs, all the Port Pairs must be in the same network pair;
- Ports in Port Pairs must have a univocally retrievable APIC Domain;
- Ports in Port Pairs can't be in the same network;
- Flowc src/dst networks must be distinct;
- Flowc can't be updated if in use by a chain;
- Networks partecipating a port chain must be in the same VRF;
- Src and Dst networks in a chain must be in the same tenant
(temporarily);
- Port Pair's ports' network can't be external or SVI;
- Port Pair's ports' networks can't be re-used in the same PPG.
Change-Id: If40595584ef46f1ac2aa0cf7525e16447f491f48
The following changes have been made to coordinate with the changes
made in Neutron for Pike:
* Partial use of Neutron context has been completely moved to neutron_lib's
context.
* The patching of neutron.db.api.get_session() has been replaced with
patching of sqlalchemy.orm.session to add the notification_queue attribute.
This significantly reduces the earlier complexity of patching.
* Use of top-level start of transaction in GBP plugins:
with context.session.begin(subtransactions=True):
has been migrated to use of:
with db_api.context_manager.writer.using(context):
or
with db_api.context_manager.reader.using(context)
as relevant.
* Calls to _make_resource_xxx_dict() in GBP plugins have been moved
to inside the transaction.
* The use of:
neutron.callbacks.events
neutron.callbacks.exceptions
neutron.callbacks.registry
to
neutron_lib.callbacks.events
neutron_lib.callbacks.exceptions
neutron_lib.callbacks.registry
* The use of:
neutron.api.v2.attributes.resource_xxx
neutron.extensions.extension_xxx
to:
from neutron_lib.api.definitions.resource_xxx
from neutron_lib.api.definitions.extension_xxx
resp.
* The use of:
neutron.db.db_base_plugin_v2.NeutronDbPluginV2.register_dict_extend_funcs
to:
neutron.db._resource_extend.resource_extend
(the latter is a decorator)
* The use of:
neutron.db.db_base_plugin_v2.NeutronDbPluginV2.register_model_query_hook()
to:
from neutron.db import _model_query as model_query.register_hook()
* The use of:
neutron.db.segments_db.NetworkSegment
to:
neutron.db.models.segment.NetworkSegment
* In the case of Neutron ml2plus plugin (used by APIC/AIM solution),
the use of get_admin_context() has been patched to return elevated
version of the current context in use. This helps to preserve the session
and transaction semantics. Ideally, context.elevated() would have been
directly used in all these places, however the current context is not
available in these places, and hence getting the current context and elevating
it is wrapped in the get_admin_context() patched method.
* In the case of the components used by the APIC/AIM solution (including
the ml2plus and l3_plugin) the use of:
with context.session.begin(subtransactions=True):
to
with db_api.context_manager.writer.using(context):
or
with db_api.context_manager.reader.using(context):
as relevant.
* Patching of methods from Neutron which is no longer relevant have been
removed from gbpservice.neutron.extensions.patch module.
* Setting up of UTs has been fixed to load and reset configurations
appropriately. This helps to eleminate some failures when tests are
run in non-deterministic orders.
* In tree devstack plugin has been updated (aim repo commit pin needs
to be reverted).
* Gate jobs have been updated as relevant (including fixes to the exercise
scripts and job configurations).
The associated repos, namely, client, UI and automation have also been
updated (the reference to the client's gerrit patch needs to be updated
once the patch has been merged).
Change-Id: I11dd089effbf40cf104afd720dc40a9911dcf28d
The legacy plugin was deprecated in stable/newton, and
is removed in ocata. This patch removes the references to
the apic-ml2-driver library, which should only be required
by the legacy driver.
Change-Id: I027edc9b74137cd242fab6243536c8331b42ccda
The apicapi and python-opflex-agent dependencies are using
temporary branches to allow GBP to build for ocata. The master
branches on these dependencies have been updated, allowing GBP
to build using their master branch.
Change-Id: I4855c24fbf570e17aa4b13a44aee822cf1aebb41
This introduces driver for Vmware NSX Policy.
The driver assumes nsx_v3 core plugin.
It implements direct configuration of NSX Policy endpoint for security
and inherits connectivity functionality from resource mapping driver.
On startup, the driver will configure NSX Policy enforcement point to be
the NSX manager core plugin is running against.
The driver implements the following resource mapping:
Openstack project => NSX Policy domain
GBP group = > NSX Policy group + communication maps
GBP classifier => NSX Policy service
GBP rule set => NSX Policy communication profile
Change-Id: I0d5593b458f7e51c21fc2b34d1ab4d898abb6c51
* use neutron_lib.directory for plugin retrieval
* switch to neutron_lib for neutron constants, exceptions,
extensions
* add neutron.plugins.ml2.ovo_rpc to OUT_OF_PROCESS_NOTIFICATIONS:
neutron added ovo rpc callback mechanism for ovo objects, and aim
notification manager needs to recognize those as out of process.
Since neutron moved away from get_session API to get_reader_session
and get_writer_session, override for these was added.
Few bugs were fixed in the delayed notification area as well.
* new engine facade: make use of reader and writer to grab db engine
* remove _update_fip_assoc override (didn't find a reason for the
override)
* aim driver: a fix in update_subnetpool_precommit - not to assume
address_scope_id field is returned from neutron update call if it
was not updated.
* extend_XXX_dict call was switched to receive ovo instead of db
object. As a result, foreign keys are not part of the object
anymore, and need to be retrieved from db.
* remove_router_interface - receive port dictinary rather than port
object
* fix patched neutron functions to receive correct parameter types
(like patched_get_locked_port_and_binding)
* use add_agent_status_check_worker instead of add_agent_status_check
* advertise_mtu configuration parameter was removed from neutron. It
is used in aim driver, hence added to aim driver config.
* use of project_id instead of tenant_id where required
* use segments_db module for network segments
* test_aim_mapping_driver: the test used to override uuid generation
in order to get predictable uuid results. New neutron code makes
use of python uuid module where overrides are complicated. It was
easire to remove all uuid-based values from dictionaries under test
* add filters parameter to get_address_scopes calls, otherwise the
call fails (probably should be fixed in neutron)
* in routing tests, remove the assumption that routes are returned in
specific order
Change-Id: I1943fd4196ea6199d825ae53f0e9f5b54d54a260
Pass create_if_absent=False to AIM's get_status() to hopefully reduce
transaction retries due to DBDuplicateEntry exceptions. This required
unpinning the version of AIM used, as well as a couple of fixes in the
AIM repo.
Change the RPC handlers to use Neutron's retry_db_errors decorator
rather than its own, so that DBDuplicateEntry exceptions are retried.
Avoid logging at error level when processing retriable exceptions.
Change-Id: I53740eea3cb7cacafceae589deec3b573ef6a68a
* The IP addresses handed by Neutron’s ipam are no longer sequential per the
following commit:
dcb2a931b5
Several UTs were making sequential assignment assumptions and were expecting
specific IP address allocations. These had to be refactored appropriately by
checking if the assigned IP address belongs to the expected CIDR.
* There was a bug in Neutron until stable/mitaka which prevented duplicate SG
rules being added. Since that is fixed in stable/newton overlapping SG rules
can be added within the same tenant, see:
3c1a068c7a
We actually dont want to add overlapping rules in the resource_mapping driver,
hence a check was added to prevent adding of duplicate rules.
* The unit test discovery path is being set to "gbpservice/neutron" in
.testr.conf to avoid running the tests in gbpservice/contrib path which
currently have some NFP related tests. The path can be reverted back to
“gbpservice” once the contrib code has been updated.
* There is a bug in the neutron code which always requires passing the filters
argument (even if empty) to get_sg_rules() call.
* The flavors service plugin needs to be explicitly configured in the UTs:
0e3f4b8335
* The use of unittest has been migrated to using unittest2.
* The default tenant in the Neutron UTs is no longer ‘test-tenant’. Instead the
following constant should be used:
neutron.tests.unit.db.test_db_base_plugin_v2.TEST_TENANT_ID
* The project_id is now being added to the resource by the API layer. The
extension test cases had to be updated to accommodate for this extra
argument.
* Neutron now sends DHCP and Nova notifications for operations on resources
from the ML2 plugin. See the following relevant commits in Neutron:
181bdb374fa5cd3b65d1877778ee4ca4df99ff5d
With the above changes, it is no longer needed for GBP to send DHCP and Nova
notifications (previously being sent from local_api.py). The neutron_resource
and the aim_mapping drivers, which attempt to provide transactional semantics,
still need the queueing functionality on the notification framework, so this is
being preserved. The send_or_queue_notification method from this framework is
also being preserved since the aim_mapping driver makes use of this method to
send specific notifications which are outside the scope of the notifications
that Neutron sends. When the ML2Plus plugin is used for the aim_mapping driver,
Neutron’s registry notification is patched to allow the notification to be
queued. It should be noted that at this point, some notifications cannot be
queued since for some resources the existence checks fail if they are queued
and sent at a later time. In such cases, the notifications are sent
immediately. This logic needs to be revisited.
* The _get_tenant_id_for_create() method was removed in Neutron since the
context object provides the project_id:
5d53dfb8d6
GBP should also follow this approach, however its a big change, mostly in the
UTs. So this patch temporarily adds the _get_tenant_id_for_create() method to
the GBP service plugins.
* The patch for create_floatingip in gbpservice/neutron/extensions/patch.py is
no longer needed and is being removed. Other such methods in the module
cannot be blindly removed, so for now, they have been updated to sync with
their newton version, but should be revisited to explore their removal.
* The ml2_network_segments table was renamed to network segments, and the
allowed_address_pair definition was moved in neutron:
c8fca1c96f7c0f189309
* DB objects are being detached from the session when the extension attribute
processing happens. However, ml2plus needs the session context in the
extend_dict functions. Hence, a utility function was added in
gbpservice/neutron/plugins/ml2plus/patch_neutron.py to get the currently
active session.
* The following change adds a transaction guard to some operations which
prevent then from being called from within a transaction:
https://review.openstack.org/gitweb?p=openstack/neutron.git;a=commitdiff;h=afe1a834000d33900b8646d308fa26fa807a2ca0
ml2plus however needs to support calling these operations from within a
transaction. Hence the transaction guard is disabled by use of a decorator on
the ml2plus functions.
* Neutron defines a new dns-integration extension and all the DB related DNS
handling was moved out of the DB core plugin:
64f5fc8259
* Retry decorator has been added to ml2plus methods to align with the
following:
acbabaa3db09c87425fa
* A bug in the aim_mapping log statements was fixed by using vars() to displace
aim resource dictionary attributes.
* A bug was fixed in the test_apic_aim code, where the tests set the expected
value of dns_name to None, but the implementation sets it to ‘’.
* The following changes were made to move things to neutron_lib and have been
refactored in this patch:
** neutron.db.model_base was moved to neutron_lib:
61cc14fd67
** neutron.common.exceptions was moved to neutron_lib, and has been refactored
here.
** The converter and validator functions in neutron.api.v2.attributes were
moved to neutron_lib and has been refactored here.
** Constants like ATTR_NOT_SPECIFIED have been moved from
neutron.api.v2.attributes to neutron_lib and has been refactored here.
Note that the integration tests fail in this patch since the DB schema needs to
be updated to rename the tenant_id column to project_id. This is being done in
the dependent patch, and the integration tests should be validated on that
patch.
Follow up items:
* The following test fails sporadically:
gbpservice.neutron.tests.unit.services.grouppolicy.test_aim_mapping_driver.TestNeutronPortOperation.test_gbp_details_for_allowed_address_pair
* Some hacking directives have been disabled and need to enabled but will
* require significant code refactoring.
* HasId and HasTenant are deprecated, move to HasProject and HasId in
* model_base, see commit:
61cc14fd67 (diff-b923b82d6a7b3c5cd77c32354ffc9f13)
* A couple of UTs are being skipped in:
* gbpservice/neutron/tests/unit/plugins/ml2plus/test_extension_driver_api.py
* and need to updated per the comments in the code.
Change-Id: I887ee6cfca8199710cf5c653b5f57dff86bb035a
Implements an L3 service plugin, apic_aim_l3, that, in conjunction
with the apic_aim mechanism driver, maps each Neutron router to an AIM
Contract and ContractSubject whose DNs and status are exposed via
extended attributes similar to those on the core Neutron resources. An
"any" Filter and FilterEntry are created per-tenant, and referenced in
this contract, allowing all traffic from EPGs providing and consuming
this contract to be routed.
The add_router_interface and remove_router_interface methods are stubs
that will be implemented in the next patch set. They will manage the
mapping of router interfaces to AIM Subnets, along with having the
default EPGs associated with those interfaces provide and consume the
router's Contract.
The corresponding GBP policy driver's extension is renamed
apic_aim_gbp for consistency with the apic_aim and apic_aim_l3
extensions at the Neutron level, and all extensions are now in the
gbpservice.neutron.extensions module.
The GBP policy driver's unit tests are updated to account for the
Filter and FilterEntry resources created by the mechanism driver.
The apic_aim unit tests wipe the AIM DB in tearDown, and use the
aci_integration_manager branch of the apicapi repo.
The GBP devstack plugin, when ENABLE_APIC_AIM=True, configures neutron
to use the apic_aim_l3 service plugin, and installs the
aci_integration_manager branch of the apicapi repo.
Change-Id: I1b7f0c80e66d55d58c27fe9e4cb461f62aec3c42
This changeset implements under the cloud receiver for neutron resources.
It registers agents for firewall, loadbalancer and VPN. It receives
notifications from over the cloud and forwards it to neutron plugins.
Change-Id: I1f884aed57b6f0aa156dbbe918e8079401bb12a2
Implements: blueprint gbp-network-services-framework
Co-Authored-By: Deepak S <in.live.in@live.in>
Set bd_name to reference the network's BridgeDomain when creating the
default AIM EndpointGroup for a neutron network.
Closes-Bug: 1604899
Change-Id: I13299f73680486ee7e3e22577a017887ca572c73
Fix the order of pkgs to avoid install of an older version of
opflexagent. Sources for the current version and an older version
were both being installed.
Closes-Bug: 1607011
Change-Id: I1768a192390ed14ae76de7f4b1497ded57f6c32c
This is a very preliminary version of a new APIC mechanism driver
utilizing the ACI Integration Module (AIM) library concurrently being
developed. A corresponding extension driver exposes details regarding
the mapping of the Neutron resources to APIC. These drivers require
the Ml2Plus extended driver APIs.
See the apic-aim-ml2-driver devref for implementation details and for
devstack configuration instructions.
Change-Id: I82df32f0880d6a0d53b305f6c6391fcbea049d1b
This changeset include mainly two utility drivers
(1) openstack client driver
- provides client utilities for Keystone, Nova, Neutron, and GBP clients.
(2) COAL(Commom Openstack Abstration Layer)
- The idea of this driver is to abstract the Network(Neutron/GBP), Compute(VM/Docker), and Fabric(APIC) APIs
- At present, only network APIs are abstracted.
This changeset also include the patch to the test requirements, which has the package names that needs to be installed during integration by the gate.
Change-Id: I5f94290c4fc5955b6797ca9789b7f7d8325d8add
Implements: blueprint gbp-network-services-framework
Co-Authored-By: Ashutosh Mishra <mca.ashu4@gmail.com>
Co-Authored-By: Akash Deep <akash.deep@oneconvergence.com>
git.openstack.org is the authoritative source for openstack repos.
github is a mirror is not gauranteed to be immediately in sync.
Change-Id: I63415d96a4a89b34964e7a3eaa957c269425b56a