6.1 KiB
Switching oslo.vmware's backing SOAP library
We want to switch the SOAP library used in olso.vmware to another library, zeep, because the currently-used library (suds-jurko) is unmaintained, not compatible with Python 3.10 and lacks in performance.
Problem description
With Python 2 being removed from all of OpenStack, we need the
library backing oslo.vmware
's SOAP calls to be compatible
with Python 3. We see suds-jurko
as being unmaintained upstream, so we
cannot rely on any changes happening. Since we currently already see
deprecation warnings generated by nova's tests for Python 3.6, we need a
fix before the release of Python 3.10 or oslo.vmware
will
stop working alltogether.
Additionally, the vast number of VMs handled by a single compute-node
with nova
's VMware driver makes suds-jurko
a
performance bottleneck. This can be overcome with switching to a library
more optimized for parsing XML.
Proposed change
We propose to change the backing library of oslo.vmware
to zeep. The
library is
- Still maintained
- Written for Python 2 and 3
- Uses
lxml
for faster XML parsing
With the interface of zeep
being similar but not the
same as suds
, we have to introduce some compatibility
functions. This is also necessary, because some library-specific
representation of objects, e.g. attribute access to
ManagedObjectReference
objects, leaked through to consuming
projects. We therefore propose to change the library and consuming
projects in multiple phases.
Phase 1
Add compatibility functions to oslo.vmware
for value and
type access of ManagedObjectReference
and add additional
functions/helpers for attribute access to move all code using
suds
-specificas into oslo.vmware
.
Phase 2
Introduce patches to cinder
and nova
for
using the functions introduced in phase 1.
Phase 3
Switch the code in oslo.vmware
to use zeep
instead of suds-jurko
while keeping the interface for
consuming projects the same. The core of these changes should be
confined to oslo_vmware.service
.
Alternatives
There's another fork of suds
based on
suds-jurko
called suds-community, which
is still maintained, but while using it may solve the Python
compatibility problem it doesn't improve performance.
While there are other libraries around, see e.g. this list in the Python wiki, a quick glance at most of them shows them as unmaintained, not supporting Python 3 or having an interface that requires too many compatibility shims for keeping changes in consuming projects minimal.
Another alternative would be to get rid of oslo.vmware
alltogether, which is mainly hindered by nova and cinder still including
drivers using this library.
An alternatives to adding helper functions for hiding the differences
between suds
and zeep
would be changing the
depending project's drivers to use the new interface instead. But this
would still leak implementation details of the SOAP library backing
oslo.vmare
into depending projects, which can also hinder
future library switches.
Impact on Existing APIs
Consuming code will be required to use the newly introduced helper-functions for compatibility in attribute access.
Security impact
None
Performance Impact
With the XML parser of the library switching to the C-based
lxml
, we expect a performance increase. In our (very
simple) tests, we achieved up to 50 % reduction in request times and CPU
load.
Configuration Impact
None
Developer Impact
Consuming code will be required to use the newly introduced helper-functions for compatibility in attribute access.
Testing Impact
Since there's a new way to access
ManagedObjectReference
's attributes, we need to update all
tests - including depending project's - to use the helper functions we
are going to introduce. Additionally, we will have to update fake
ManagedObjectReference
objects defined in the tests of
depending projects to include both old and new attributes for the
transition phase. This is necessary, because the project's tests have to
work indepedently of the backing libraries helper-function
implementation, so that introduces changes doesn't break them. An
alternative to this would be providing fake objects importabable via
oslo.vmware
. Even then there can still be cases where
oslo.vmware
's backing SOAP library's object representation
might leak into depending object's tests, because there are other
objects, that are not ManagedObjectReferences
, but behave
exactly the same.
Implementation
Assignee(s)
- Primary assignee:
-
jkulik
Milestones
Target Milestone for completion: unclear
Work Items
- Implement helper functions in
oslo.vmware
for compatibility layer - Patch
nova
's VMware driver to use helper functions - Patch
cinder
's VMware driver to use helper functions - Add
zeep
to global requirements as documented - Implement
Service
object usingzeep
client inoslo.vmware
Documentation Impact
None
Dependencies
This adds zeep
into OpenStack's requirements, while
removing suds-jurko
.
References
- Initial mailing list post: http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013449.html
- Bug regarding
suds
packaging problems for debian: https://bugs.launchpad.net/oslo.vmware/+bug/1465016
Note
This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode