The Query.get() method is considered legacy as of the 1.x series of
SQLAlchemy and becomes a legacy construct in 2.0. This should be
replaced with Session.get() method.
Closes-Bug: #2027604
Change-Id: Ib81f2c36a93f08844a629f165d2323bf1198d465
MechanismDriverContext has an attribute _plugin_context, which carries
the current context with it. This is used by many ml2 drivers, as it is
the only way for them to get the current context. We now make this a
public API by adding a property to MechanismDriverContext that returns
_plugin_context as a read-only attribute.
Change-Id: If9b05655286f42081cf26c90c563429ca2e63244
Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/plugins.
Trivialfix
Change-Id: Id9138652f5f07ef12fa682e182fe210019e8f975
Drive the choice of mechanism driver during binding as inferred from
the resource provider allocated by nova and as told to neutron via the
port's binding:profile.
As discussed on a neutron qos irc meeting some time ago
this patch introduces a new assumption on bind_port() implementations.
That is an implementation of bind_port() in any mech driver supporting
Guaranteed Minimum Bandwidth bind_port() must not have a non-idempotent
side effect. Because the last binding level will be redone for a 2nd
time with a narrowed down list of mechanism drivers. And if the 2nd call
does not give the same result as the first all kind of weird things can
happen.
Change-Id: I2b7573ec6795170ce45a13d5d0ad7844fb85182d
Depends-On: https://review.openstack.org/574781
Depends-On: https://review.openstack.org/635160
Partial-Bug: #1578989
See-Also: https://review.openstack.org/502306 (nova spec)
See-Also: https://review.openstack.org/508149 (neutron spec)
This patch integrates Port Binding Level OVO in /plugin/ml2/db.py
and introduces context instead of session for usage in object
operations.
Change-Id: Ifa779f5f70a7502bd96b34d64a84d272af2a6886
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Co-Authored-By: Anindita Das <anindita.das@intel.com>
Co-Authored-By: Slawek Kaplonski <slawek@kaplonski.pl>
Functionality is added to the ML2 plugin to handle multiple port
bindings
Co-Authored-By: Anindita Das <anindita.das@intel.com>
Co-Authored-By: Miguel Lavalle <miguel.lavalle@huawei.com>
Partial-Bug: #1580880
Change-Id: Ie31d4e27e3f55edfe334c4029ca9ed685e684c39
This reverts commit febeaf5d40.
This patch broke postgres tempest jobs, as well as introduced potential
race conditions in database layer because of mixed usage of old and new
engine facades.
Related-Bug: #1744829
Change-Id: Ic142ae7faf4e5f10cbdf761d7e6f3d442e94a3eb
This patch integrates Port Binding OVO in /plugins/ml2/db.py
and /plugins/ml2/plugin.py.
Co-Authored-By: Artur Korzeniewski <artur.korzeniewski@intel.com>
Change-Id: Idb76c0cb2a4d66690c9aca5ba338d5df814cd21e
Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
The ml2 plugin driver API was rehomed into neutron-lib with commit
Ifc829953ef4d5859c3475903965dc08aba42fd9c and the API was shimmed in
neutron with I86a10091b55d1123e8d16f16155e0312bb10e54c. This patch
consumes the ML2 driver api from neutron lib thereby removing the
driver_api module from neutron.
NeutronLibImpact
Change-Id: Ice49572e217eeaf820e48d40f2251d08766490b5
Since Pike log messages should not be translated.
This patch removes calls to i18n _LC, _LI, _LE, _LW from
logging logic throughout the code. Translators definition
from neutron._i18n is removed as well.
This patch also removes log translation verification from
ignore directive in tox.ini.
Change-Id: If9aa76fcf121c0e61a7c08088006c5873faee56e
The workaround of using deepcopy calls on the PortBinding
and PortBindingLevel objects prevents the port relationship
from being loaded to bump its revision because it then fails
to merge.
So in order to allow port bindings to bump the revision we
need to stop using sqlalchemy objects in the PortContext. This
patch adds a new snapshot object that just copies the column
values and provides a method to reconcile them back into the
session.
This workaround can go away after we switch to using OVOs, but
this needs to be backportable so we can't just wait for OVO
adoption.
Partial-Bug: #1699034
Change-Id: Ib85ec8182117fa3c4844dabfffe881e38e68b556
The ml2 MechanismDriver is now in neutron-lib along with its associated
constants. This patch switches over to the lib versions of those, but
leaves a shim of the MechanismDriver that just ref's the driver from
lib. This shim allows our broad consumer base of the driver to switch
over at their leisure.
NeutronLibImpact
Change-Id: I99e3de6d933a1bb341394f85415fb07306a82a01
This allows segments looked up ahead of time to be passed
into NetworkContext objects and NetworkContext objects to
be passed into PortContext objects. This allows us to avoid
doing segments lookups for every PortContext construction
when handling a bunch of ports (e.g. in RPC handler).
Change-Id: Ib4c43a7894fe1285ecf4bdf9af5e5f1b93b0b39b
Partial-Bug: #1665215
Perform a deepcopy on the sqla objects passed into the PortContext
so we get detached versions of them safe to reference forever.
This is necessary because the PortContexts outlive the
transaction context managers they are creating in which means an
object can be expired and result in a query after a commit
(e.g. in bind_port_if_needed) that will fail and result in an
exception.
This required a few additional explicit session.merge calls to deal
with cases where touching the mech context was implicitly expected
to modify the DB state on the next commit.
Closes-Bug: #1669528
Change-Id: Ib5ba2daa80acba53c082bade1f61a3ee44ca41fc
Neutron-lib 1.1.0 is now out and contains the portbindings
API definition (as per commit [1]). This patch moves neutron
references over to the neutron-lib version.
NeutronLibImpact
- Consumers using the public constants within neutron's
portbindings API extension must now use the values
from neutron-lib.
[1] 87e42f993c07ae320159d5123662ee9f3bd4d903
Change-Id: I669af9b4c712877772d91a03857ab108714001d4
This allows a PortContext to be constructed without a network
passed in for cases like update_port_status that don't examine
bindings, segments, or any other network properties.
In the event that a mechanism driver is loaded that does reference
the 'network' property for these occasions, the context will look
up the network then in a late binding fashion.
This will improve the performance of the update_port_status call
from the provisioning blocks callback, which doesn't provide a
cached network to update_port_status.
Partial-Bug: #1665215
Change-Id: I498791614fd456ab67c453cad691f7658d107123
The port context already has a network attached to it that contains
all of the segments for that network. In all cases the segments the
port wants to lookup should be present here so we can just iterate
over them without a separate DB query.
Partial-Bug: #1665215
Change-Id: I9014c4e401134f4db908ed989e0095d0ca95b7c9
Similar to change I498791614fd456ab67c453cad691f7658d107123,
we can avoid an unnecessary network retrieval by making the network
part of the subnet context lazily loaded by allowing None to be
passed into the subnet context.
If a driver does need items off of the network, the context will
trigger a lookup at the time of reference.
Partial-Bug: #1665967
Change-Id: Ic6651f1d44fcdbb749c99a6cc2c7300b1036388c
We have a number of functions that expect to get session as one
of the arguments. Passing session is not correct and this prevents
using new enginefacade which expects context to be passed and
session will be injected in current context.
NeutronLibImpact
Partially-Implements blueprint: enginefacade-switch
Change-Id: Ie1c54138ceaf9ac6f0674ad2786d4aaea9c80f73
We have a number of functions that expect to get session as one
of the arguments. Passing session is not correct and this prevents
of using new enginefacade which expects context to be passed and
session will be injected in current context.
NeutronLibImpact
Partially-Implements blueprint: enginefacade-switch
Change-Id: I68ac442f11a6698bc43210fae877ba3809b1c02d
This adds logic to add segment to host mappings for agents after a
network with one or more segments is created. The previous logic was
doing this only after an agent initially reported after the server
started. This meant that any segmented networks created after the
agent was running would not get mappings to the agent's host.
Change-Id: Ie265500639b1119f0448ed272d235fb709db36f6
Partially-Implements: blueprint routed-networks
This is a clean up for patch [1], the functions map should be removed
to make code easy to read. Start a deprecation cycle for these functions
in case external projects will use them.
[1] https://review.openstack.org/#/c/242393
Change-Id: I77c83bd7ee0c8ef92d8aaaa8e968479b848532fe
Partially-Implements: blueprint routed-networks
- This does NOT break other projects that rely on neutron.i18n,
as this change includes a debtcollector shim to maintain those
older entry points, until they can migrate.
- Also updates _i18n.py to the latest pattern defined by oslo_i18n
- Guidance and template are from the reference:
http://docs.openstack.org/developer/oslo.i18n/usage.html
Partially-Closes-Bug: #1519493
Change-Id: I1aa3a5fd837d9156da4643a367013c869ed8bf9d
The network context is useful in the subnet context so that mechanism
drivers can access information such as the provider network type.
Closes-Bug: #1481882
Signed-off-by: Joe Mills<joe@midokura.com>
Change-Id: I718c80512af0f2a43855efb16c2c0da69ef6b741
This reverts commit e61865807c.
This patch likely violated the idea of a NetworkContext
being a snapshot of the network at the time it was created.
This needs a different approach.
Change-Id: I20b132a0181d35b0517330fb7fbf293c3e979d0e
Avoid call to get network segments for network context objects until
a caller actually tries to lookup the segments. This optimizes cases
where the user of a port context never looks at the segments of the
associated network context (e.g. update_port_status).
Closes-Bug: #1463254
Change-Id: I7e95f81d9a3ef26ccdb18c6bfdf9adc29523aa79
Extends the existing ML2 port binding unit tests to cover the
distributed port bindings used for DVR. Within the test mechanism
driver, bindings are tracked per-host, and additional assertions are
added.
Fixes issues with PortContext attributes that were exposed by these
new tests. Adds new vif_type, original_vif_type, vif_details, and
original_vif_details PortContext attributes, similar to the exising
host, original_host, status, and original_status attributes, to
reflect host-specific details of distributed (or normal) port
bindings. Also fixes original_host and original_status to return None
when in the context of an operation other than an update, and fixes
original_host to reflect the specific host being bound for a
distributed port.
Closes-bug: 1453943
Closes-bug: 1453955
Change-Id: I467db0d48e4b82fdaad8d851e294e639a84a8160
It's mostly a matter of changing imports to a new location.
Non-obvious changes needed:
* pass overwrite= argument to oslo_context since oslo.log reads context
from its thread local store and not local.store from incubator
* don't store context at local.store now that there is no code that
would consume it
* LOG.deprecated() -> versionutils.report_deprecated_feature()
* dropped LOG.audit check from hacking rule since now the method does
not exist
* WritableLogger is now located in oslo_log.loggers
Dropped log module from the tree. Also dropped local module that is now
of no use (and obsolete, as per oslo team).
Added versionutils back to openstack-common.conf since now we use the
module directly from neutron code and not just as a dependency of some
other oslo-incubator module.
Note: tempest tests are expected to be broken now, so instead of fixing
all the oslo.log related issues for the subtree in this patch, I only
added TODOs with directions for later fix.
Closes-Bug: #1425013
Change-Id: I310e059a815377579de6bb2aa204de168e72571e
The ML2 port binding logic is modified to support hierarchical
bindings. If a mechanism driver's bind_port() calls continue_binding()
rather than set_binding(), the supplied dynamic segments are used for
another level of binding.
Gerrit Spec: https://review.openstack.org/#/c/139886/
Implements: blueprint ml2-hierarchical-port-binding
Change-Id: Id741c2d6c443fa0eac5ecf8d964b3fc14b5d3abe
To support hierarchical port binding, the driver and segment columns
are moved from the ml2_port_bindings and ml2_dvr_port_bindings tables
to a new ml2_port_binding_levels table. This new table can store
multiple levels of binding information for each port. It has the host
as part of its primary key so that it can be used for both normal and
DVR port bindings.
The cap_port_filter column is also removed from the
ml2_dvr_port_bindings table, since the adjacent driver and segment
columns are being moved, and this can trivially be done via the same
DB migration. It was included in the table by mistake and was never
used.
The logic required for hierarchical port binding will be implemented
in a dependent patch.
Gerrit Spec: https://review.openstack.org/#/c/139886/
Partially-implements: blueprint ml2-hierarchical-port-binding
Change-Id: I08ddc384763087fbac0fa3da3ed6e99b897df031
Oslo project decided to move away from using oslo.* namespace for all their
libraries [1], so we should migrate to new import path.
This patch applies new paths for:
- oslo.config
- oslo.db
- oslo.i18n
- oslo.messaging
- oslo.middleware
- oslo.rootwrap
- oslo.serialization
- oslo.utils
Added hacking check to enforce new import paths for all oslo libraries.
Updated setup.cfg entry points.
We'll cleanup old imports from oslo-incubator modules on demand or
if/when oslo officially deprecates old namespace in one of the next
cycles.
[1]: https://blueprints.launchpad.net/oslo-incubator/+spec/drop-namespace-packages
Depends-On: https://review.openstack.org/#/c/147248/
Depends-On: https://review.openstack.org/#/c/152292/
Depends-On: https://review.openstack.org/#/c/147240/
Closes-Bug: #1409733
Change-Id: If0dce29a0980206ace9866112be529436194d47e
DVR ports are now bound using the same function,
Ml2Plugin._bind_port_if_needed(), that is used to bind non-DVR ports,
so that concurrent binding attempts are properly handled and mechanism
driver update_port_precommit() and update_port_postcommit() methods
are called. The Ml2Plugin._commit_dvr_port_binding() function is
eliminated, and the DvrPortContext class has been folded into the
PortContext class, reducing duplicated logic.
A followup patch will address the duplication of ML2 DB schema for DVR
and further reduce the duplicated and special-case port binding logic
supporting DVR.
Closes-Bug: 1415526
Closes-Bug: 1416783
Partial-Bug: 1367391
Change-Id: Ic32241297c5f8c67dc77d0af836b1cc0a5df988a
The following ML2 driver API changes are required to support
hierarchical port binding:
* Add segments_to_bind PortContext property containing the list of
network segments with which a mechanism driver should try to bind
the port. All mechanism drivers that bind ports should now use this
property in place of network.network_segments, which contains only
the network's static segments.
* Replace several PortContext properties (bound_segment, bound_driver,
original_bound_segment, original_bound_driver) with new properties
(binding_levels, top_bound_segment, bottom_bound_segment,
original_binding_levels, original_top_bound_segment,
original_bottom_bound_segment) in order to represent hierarchical
bindings.
* Add stubbed-out continue_binding() method to PortContext, allowing
mechanism drivers to partially bind the port.
All existing drivers and unit tests are updated accordingly.
The DB schema changes and logic required for hierarchical port binding
will be implemented in dependent patches.
Gerrit Spec: https://review.openstack.org/#/c/139886/
Partially-implements: blueprint ml2-hierarchical-port-binding
Change-Id: Icb1a016f4661e427cb6cfa3452802ba5e64b7124
jsonutils has been moved to the oslo.serialization library. in this
patch we bring in oslo.serialization and switch all references to
the new library.
Closes-Bug: #1385353
Change-Id: I2898c4040abb2f208959049708e7bc093cfbaba7
This commit builds on top of part 1 to introduce support for creating
dynamic network segments in ML2.
Change-Id: I399e1569baae6f24054aac15c4c51a2e44a20e5b
Partially implements: Blueprint ml2-type-driver-refactor
This change fixes the information used by the L2pop
driver to populate l2pop rules that enables DVR to
route packets across compute servers that have
tenant VMs that belong to different networks.
It also fixes the case where VMs were not able to
obtain IP Addresses when such VMs are on DVR
hosted subnets.
Change-Id: Ib630e57c186da60eb15f9ffa6b1b0bfa74f48caa
Closes-Bug: #1350485
Closes-Bug: #1352857
This patch introduces the changes necessary to
support DVR at Layer 2 with ML2, and L2pop.
Partially-implements: blueprint neutron-ovs-dvr
Change-Id: I709d877e7df83728868e68e023e6a37bb2e11704
Authored-by: Vivekanandan Narasimhan <vivekanandan.narasimhan@hp.com>
Bindings to host or status may need further encapsulation
to avoid exposing mechanism drivers to underlying DB model
details. This was particularly true in the case of the
l2pop mech driver.
As a result, some docstrings were reworded, and the newly
introduced properties used directly in the mech drivers.
Partially-implements: blueprint neutron-ovs-dvr
Supports blueprint: ml2-hierarchical-port-binding
Change-Id: If2a373ef04d19b164585fb4bde4fe6e0cfaf43be
The ML2 plugin now calls the bind_port() operation on the registered
mechanism drivers outside of any enclosing DB transaction. Ports are
created or updated in one transaction, then a binding is established
if possible, and finally a second transaction commits the binding
result.
With [re]binding moved outside the DB transaction that triggered it,
it is now possible that multiple threads or processes will
concurrently try to bind the same port, or that the port will be
updated between transactions. Concurrent attempts to bind the same
port are allowed to proceed, which results are used is resolved in the
second transaction, and binding is retried if necessary.
Improvements to the Cisco Nexus driver and unit tests from Rich Curran
needed due to the binding changes are also included.
Closes-Bug: 1276391
Closes-Bug: 1335226
Change-Id: I65dafc330d6e812dad0667d2383858504d0ba299
The OpenDaylight mechanism driver does not depend on an L2 agent to
plug the port. Now that nova waits for notification that the port
status is ACTIVE, the ML2 driver API is extended so that the mechanism
driver that binds a port can optionally set the port status, and the
OpenDaylight mechanism driver uses this to set the port status to
ACTIVE.
Closes-Bug: 1301449
Change-Id: I171c405f36b4f2354d9585e8ae3dfa50ddaa9a7e
The following properties are added to the PortContext object passed to
ML2 mechanism drivers for port operations:
* bound_driver - name of current bound driver
* original_bound_driver - name previously bound driver in an update
* original_bound_segment - network segment used in previous binding
Some issues with the existing ML2 port binding unit tests were also
fixed.
The remainder of the fix for bug 1276395, making these binding details
available to mechanism drivers when a port is deleted, will be
addressed as part of the fix for bug 1276391.
Partial-Bug: #1276395
Change-Id: I9ecff4a4e044920ed2dde709c89aeb9bc773220d
In addition to binding:vif_type, the neutron core plugin needs to
supply various information to nova's VIF driver, such as VIF security
details and PCI details when SR-IOV is being used. This information is
read-only, requires admin privileges, and is not intended for normal
users. Rather than add separate mechanisms throughout the stack for
each such requirement, the binding:capabilities port attibute, which
is a dictionary and is not currently not used by nova, is renamed to
binding:vif_details to serve as a general-purpose mechanism for
supplying binding-specific details to the VIF driver.
This patch does not remove or replace the CAP_PORT_FILTER boolean
previously used in binding:capabilities. A separate patch should
implement the specific key/value pairs carried by binding:vif_details
to implement VIF security. Another patch will implement the key/value
pairs needed for SR-IOV.
The ML2 plugin now allows the bound mechanism driver to supply the
binding:vif_details dictionary content, instead of just the
CAP_PORT_FILTER boolean previously carried by the binding:capabilities
attribute.
DocImpact: Need to update portbinding extension API, but no impact on
user or administrator documentation.
Implements: blueprint vif-details
Related-Bug: 1112912
Change-Id: I34be746fcfa73c70f72b4f9add8eff3ac88c723f
The ml2 plugin uses mechanism drivers to determine which network
segment and what VIF driver to use for a port. Mechanism drivers
supporting the openvswitch, linuxbridge, and hyperv agents are
added. The binding:host attribute is set on ports belonging to the
dhcp and l3 agents so that they can be bound.
To use with devstack until it is updated, set
"Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch,linuxbridge" in localrc.
The hyperv L2 agent does not currently implement the agents_db RPC,
and will therefore not work with its ml2 mechanism driver. This issue
will be tracked as a bug to be fixed in a separate merge.
implements blueprint: ml2-portbinding
Change-Id: Icb9c70d8b0d7fcb34b57adc760bb713b740e5dad
Define the Mechanism Driver interface for create/update/delete
operations on networks and ports. For each of these event, the
Mechanism Driver provides one method that is called within the
database transaction of the ml2 plugin method, and one that is called
after the transaction is completed.
Support for mechanism drivers is still a work-in-progress, and the
interface is subject to change in future versions before the release
of Havana. However this initial version should be sufficient to enable
others to start defining their own mechanism drivers.
Change-Id: Ife30215589792ee27df9897d3b2bc04392638266
Implements: blueprint ml2-mechanism-drivers
Fixes: bug #1199977
Fixes: bug #1199978
DocImpact
Rodolfo Alonso Hernandez