openstack
/
os-vif
Code Issues Proposed changes

Extend port profiles with datapath offload type 

TC Offload capabilities have been present in Linux Kernel 4.13 for
hardware offloads. Currently OVS supports offloads (for OVS 2.8 and later)
with similar efforts planned for vrouter (OpenContrail and Tungsten
Fabric).

The existing method in os-vif is to pass the metadata required for
plugging via a VIFPortProfileOVSRepresentor port profile object. This is
used by the 'ovs' reference plugin and the external 'agilio_ovs' plugin.

With 'vrouter' being the third user of such metadata, it would be more
elegant to clean up this interface by using a composition pattern rather
than an inheritance based pattern.

Change-Id: I401ee6370dad68e62bc2d089e786a840d91d0267
Needed-By: I327894839a892a976cf314d4292b22ce247b0afa
Signed-off-by: Jan Gutter <jan.gutter@netronome.com>
blueprint: generic-os-vif-offloads
This commit is contained in:
Jan Gutter 2018-06-04 11:10:39 +02:00
parent 8df626b671
commit e8f541fa0c
6 changed files with 345 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
doc/source/user/vif-types.rst
View File

@ -128,6 +128,12 @@ This profile provides the metadata required to associate a VIF with a
specified, it indicates a desire to rename the representor to the given name
on plugging.
.. note:: This port profile is provided for backwards compatibility only.
This interface has been superceded by the one provided by the
`DatapathOffloadRepresentor` class, which is now a field element of the
`VIFPortProfileBase` class.
VIFPortProfileFPBridge
----------------------
@ -146,6 +152,23 @@ VIFPortProfileK8sDPDK
This profile provides the metadata required to associate nested DPDK VIF with
a Kubernetes pod.
Datapath Offload type object
============================
Port profiles can be associated with a `datapath_offload` object. This
provides a set of metadata attributes that serve to identify the datapath
offload parameters of a VIF. Each different type of datapath offload is
associated with a versioned object, subclassing `DatapathOffloadBase`.
DatapathOffloadRepresentor
--------------------------
This object provides the metadata required to associate a VIF with a
:term:`VF` representor conforming to the
`switchdev <https://netdevconf.org/1.2/session.html?or-gerlitz>`_ kernel model.
If `representor_name` is specified, it indicates a desire to rename the
representor to the given name on plugging.
VIF network objects
===================

121
os_vif/objects/vif.py
View File

@ -10,6 +10,8 @@
# License for the specific language governing permissions and limitations
# under the License.
from debtcollector import removals
from oslo_utils import versionutils
from oslo_versionedobjects import base
from oslo_versionedobjects import fields
@ -177,11 +179,45 @@ class VIFNestedDPDK(VIFBase):
}
@base.VersionedObjectRegistry.register
class DatapathOffloadBase(osv_base.VersionedObject,
base.ComparableVersionedObject):
# Base class for all types of datapath offload
VERSION = '1.0'
@base.VersionedObjectRegistry.register
class DatapathOffloadRepresentor(DatapathOffloadBase):
# Offload type for VF Representors conforming to the switchdev model
VERSION = '1.0'
fields = {
# Name to set on the representor (if set)
'representor_name': fields.StringField(nullable=True),
# The PCI address of the Virtual Function
'representor_address': fields.StringField(nullable=True),
}
@base.VersionedObjectRegistry.register
class VIFPortProfileBase(osv_base.VersionedObject,
base.ComparableVersionedObject):
# Base class for all types of port profile
VERSION = '1.0'
# Version 1.0: Initial release
# Version 1.1: Added 'datapath_offload'
VERSION = '1.1'
fields = {
# Datapath offload type of the port
'datapath_offload': fields.ObjectField('DatapathOffloadBase',
nullable=True,
subclasses=True),
}
obj_relationships = {
'datapath_offload': (('1.1', '1.0'),),
}
@base.VersionedObjectRegistry.register
@ -189,7 +225,8 @@ class VIFPortProfileOpenVSwitch(VIFPortProfileBase):
# Port profile info for OpenVSwitch networks
# Version 1.0: Initial release
# Version 1.1: Added 'datapath_type'
VERSION = '1.1'
# Version 1.2: VIFPortProfileBase updated to 1.1
VERSION = '1.2'
fields = {
'interface_id': fields.UUIDField(),
@ -205,6 +242,9 @@ class VIFPortProfileOpenVSwitch(VIFPortProfileBase):
target_version = versionutils.convert_version_to_tuple(target_version)
if target_version < (1, 1) and 'datapath_type' in primitive:
del primitive['datapath_type']
if target_version < (1, 2):
super(VIFPortProfileOpenVSwitch, self).obj_make_compatible(
primitive, "1.0")
@base.VersionedObjectRegistry.register
@ -212,7 +252,8 @@ class VIFPortProfileFPOpenVSwitch(VIFPortProfileOpenVSwitch):
# Port profile info for OpenVSwitch networks using fastpath
# Version 1.0: Initial release
# Version 1.1: VIFPortProfileOpenVSwitch updated to 1.1
VERSION = '1.1'
# Version 1.2: VIFPortProfileOpenVSwitch updated to 1.2
VERSION = '1.2'
fields = {
# Name of the bridge (managed by fast path) to connect to
@ -227,14 +268,23 @@ class VIFPortProfileFPOpenVSwitch(VIFPortProfileOpenVSwitch):
if target_version < (1, 1):
super(VIFPortProfileFPOpenVSwitch, self).obj_make_compatible(
primitive, "1.0")
if target_version < (1, 2):
super(VIFPortProfileFPOpenVSwitch, self).obj_make_compatible(
primitive, "1.1")
@removals.removed_class("VIFPortProfileOVSRepresentor",
category=PendingDeprecationWarning)
@base.VersionedObjectRegistry.register
class VIFPortProfileOVSRepresentor(VIFPortProfileOpenVSwitch):
# Port profile info for OpenVSwitch networks using a representor
# This class is now frozen and retained for backwards compatibility. The
# 'datapath_offload' field in port profiles should be used instead.
#
# Version 1.0: Initial release
# Version 1.1: VIFPortProfileOpenVSwitch updated to 1.1
VERSION = '1.1'
# Version 1.2: VIFPortProfileOpenVSwitch updated to 1.2
VERSION = '1.2'
fields = {
# Name to set on the representor (if set)
@ -249,37 +299,58 @@ class VIFPortProfileOVSRepresentor(VIFPortProfileOpenVSwitch):
if target_version < (1, 1):
super(VIFPortProfileOVSRepresentor, self).obj_make_compatible(
primitive, "1.0")
if target_version < (1, 2):
super(VIFPortProfileOVSRepresentor, self).obj_make_compatible(
primitive, "1.1")
@base.VersionedObjectRegistry.register
class VIFPortProfileFPBridge(VIFPortProfileBase):
# Port profile info for LinuxBridge networks using fastpath
VERSION = '1.0'
#
# Version 1.0: Initial release
# Version 1.1: VIFPortProfileBase updated to 1.1
VERSION = '1.1'
fields = {
# Name of the bridge (managed by fast path) to connect to
'bridge_name': fields.StringField(),
}
def obj_make_compatible(self, primitive, target_version):
target_version = versionutils.convert_version_to_tuple(target_version)
if target_version < (1, 1):
super(VIFPortProfileFPBridge, self).obj_make_compatible(
primitive, "1.0")
@base.VersionedObjectRegistry.register
class VIFPortProfileFPTap(VIFPortProfileBase):
# Port profile info for Calico networks using fastpath
VERSION = '1.0'
#
# Version 1.0: Initial release
# Version 1.1: VIFPortProfileBase updated to 1.1
VERSION = '1.1'
fields = {
# The mac address of the host vhostuser port
'mac_address': fields.MACAddressField(nullable=True),
}
def obj_make_compatible(self, primitive, target_version):
target_version = versionutils.convert_version_to_tuple(target_version)
if target_version < (1, 1):
super(VIFPortProfileFPTap, self).obj_make_compatible(
primitive, "1.0")
@base.VersionedObjectRegistry.register
class VIFPortProfile8021Qbg(VIFPortProfileBase):
# Port profile info for VEPA 802.1qbg networks
VERSION = '1.0'
#
# Version 1.0: Initial release
# Version 1.1: VIFPortProfileBase updated to 1.1
VERSION = '1.1'
fields = {
'manager_id': fields.IntegerField(),
@ -288,23 +359,39 @@ class VIFPortProfile8021Qbg(VIFPortProfileBase):
'instance_id': fields.UUIDField(),
}
def obj_make_compatible(self, primitive, target_version):
target_version = versionutils.convert_version_to_tuple(target_version)
if target_version < (1, 1):
super(VIFPortProfile8021Qbg, self).obj_make_compatible(
primitive, "1.0")
@base.VersionedObjectRegistry.register
class VIFPortProfile8021Qbh(VIFPortProfileBase):
# Port profile info for VEPA 802.1qbh networks
VERSION = '1.0'
#
# Version 1.0: Initial release
# Version 1.1: VIFPortProfileBase updated to 1.1
VERSION = '1.1'
fields = {
'profile_id': fields.StringField()
}
def obj_make_compatible(self, primitive, target_version):
target_version = versionutils.convert_version_to_tuple(target_version)
if target_version < (1, 1):
super(VIFPortProfile8021Qbh, self).obj_make_compatible(
primitive, "1.0")
@base.VersionedObjectRegistry.register
class VIFPortProfileK8sDPDK(VIFPortProfileBase):
# Port profile info for Kuryr-Kubernetes DPDK ports
VERSION = '1.0'
#
# Version 1.0: Initial release
# Version 1.1: VIFPortProfileBase updated to 1.1
VERSION = '1.1'
fields = {
# Specify whether this vif requires L3 setup.
@ -317,3 +404,9 @@ class VIFPortProfileK8sDPDK(VIFPortProfileBase):
# the server's internal version of this object.
'resourceversion': fields.StringField()
}
def obj_make_compatible(self, primitive, target_version):
target_version = versionutils.convert_version_to_tuple(target_version)
if target_version < (1, 1):
super(VIFPortProfileK8sDPDK, self).obj_make_compatible(
primitive, "1.0")

20
os_vif/tests/unit/test_objects.py
View File

@ -37,17 +37,19 @@ object_data = {
'VIFGeneric': '1.0-c72e637ed620f0135ea50a9409a3f389',
'VIFHostDevice': '1.0-bb090f1869c3b4df36efda216ab97a61',
'VIFOpenVSwitch': '1.0-e78d355f3505361fafbf0797ffad484a',
'VIFPortProfile8021Qbg': '1.0-167f305f6e982b9368cc38763815d429',
'VIFPortProfile8021Qbh': '1.0-4b945f07d2666ab00a48d1dc225669b1',
'VIFPortProfileBase': '1.0-77509ea1ea0dd750d5864b9bd87d3f9d',
'VIFPortProfileOpenVSwitch': '1.1-70d36e09c8d800345ce71177265212df',
'VIFPortProfileFPOpenVSwitch': '1.1-74e77f46aa5806930df6f37a0b76ff8b',
'VIFPortProfileFPBridge': '1.0-d50872b3cddd245ffebef6053dfbe27a',
'VIFPortProfileFPTap': '1.0-11670d8dbabd772ff0da26961adadc5a',
'VIFPortProfile8021Qbg': '1.1-b3011621809dca9216b50579ce9d6b19',
'VIFPortProfile8021Qbh': '1.1-226b61b2e76ba452f7b31530cff80ac9',
'VIFPortProfileBase': '1.1-4982d1621df12ebd1f3b07948f3d0e5f',
'VIFPortProfileOpenVSwitch': '1.2-25aec86b7ec9fcb3434f896f694818de',
'VIFPortProfileFPOpenVSwitch': '1.2-4a4f230d89a5ea0e43011f678b626dd9',
'VIFPortProfileFPBridge': '1.1-49f1952bf50bab7a95112c908534751f',
'VIFPortProfileFPTap': '1.1-fd178229477604dfb65de5ce929488e5',
'VIFVHostUser': '1.1-1f95b43be1f884f090ca1f4d79adfd35',
'VIFPortProfileOVSRepresentor': '1.1-30e555981003a109b133da5b43ded5df',
'VIFPortProfileOVSRepresentor': '1.2-d0609e93ea884ef7b4949177e9fcdc39',
'VIFNestedDPDK': '1.0-fdbaf6b20afd116529929b21aa7158dc',
'VIFPortProfileK8sDPDK': '1.0-f1e0daa66b041ded4e6dbc053b4a66d5',
'VIFPortProfileK8sDPDK': '1.1-e2a2abd112b14e0239e76b99d9b252ae',
'DatapathOffloadBase': '1.0-77509ea1ea0dd750d5864b9bd87d3f9d',
'DatapathOffloadRepresentor': '1.0-802a5dff22f73046df3742c815c51421',
}

195
os_vif/tests/unit/test_vif.py
View File

@ -10,6 +10,8 @@
# License for the specific language governing permissions and limitations
# under the License.
import warnings
import os_vif
from os_vif import objects
from os_vif.tests.unit import base
@ -52,6 +54,17 @@ class TestVIFS(base.TestCase):
vif_name="vif123",
bridge_name="br0")
def test_port_profile_base_backport_1_0(self):
datapath_offload = objects.vif.DatapathOffloadRepresentor(
representor_name="felix",
representor_address="0002:24:12.3")
obj = objects.vif.VIFPortProfileBase(
datapath_offload=datapath_offload)
primitive = obj.obj_to_primitive(target_version='1.0')
self.assertEqual('1.0', primitive['versioned_object.version'])
data = primitive['versioned_object.data']
self.assertNotIn('datapath_type', data)
def test_vif_bridge_ovs(self):
prof = objects.vif.VIFPortProfileOpenVSwitch(
interface_id="07bd6cea-fb37-4594-b769-90fc51854ee9",
@ -62,7 +75,7 @@ class TestVIFS(base.TestCase):
bridge_name="br0",
port_profile=prof)
def test_vif_bridge_ovs_backport_1_0(self):
def test_port_profile_ovs_backport_1_0(self):
obj = objects.vif.VIFPortProfileOpenVSwitch(
interface_id="07bd6cea-fb37-4594-b769-90fc51854ee9",
profile_id="fishfood",
@ -75,6 +88,24 @@ class TestVIFS(base.TestCase):
self.assertEqual('fishfood', data['profile_id'])
self.assertNotIn('datapath_type', data)
def test_port_profile_ovs_backport_1_1(self):
datapath_offload = objects.vif.DatapathOffloadRepresentor(
representor_name="felix",
representor_address="0002:24:12.3")
obj = objects.vif.VIFPortProfileOpenVSwitch(
interface_id="07bd6cea-fb37-4594-b769-90fc51854ee9",
profile_id="fishfood",
datapath_type='netdev',
datapath_offload=datapath_offload)
primitive = obj.obj_to_primitive(target_version='1.1')
self.assertEqual('1.1', primitive['versioned_object.version'])
data = primitive['versioned_object.data']
self.assertEqual('07bd6cea-fb37-4594-b769-90fc51854ee9',
data['interface_id'])
self.assertEqual('fishfood', data['profile_id'])
self.assertEqual('netdev', data['datapath_type'])
self.assertNotIn('datapath_offload', data)
def test_vif_direct_plain(self):
self._test_vif(objects.vif.VIFDirect,
vif_name="vif123",
@ -118,7 +149,7 @@ class TestVIFS(base.TestCase):
vif_name="tap123",
port_profile=prof)
def test_vif_vhost_user_fp_ovs_backport_1_0(self):
def test_port_profile_fp_ovs_backport_1_0(self):
obj = objects.vif.VIFPortProfileFPOpenVSwitch(
interface_id="07bd6cea-fb37-4594-b769-90fc51854ee9",
profile_id="fishfood",
@ -135,6 +166,28 @@ class TestVIFS(base.TestCase):
self.assertEqual(False, data['hybrid_plug'])
self.assertNotIn('datapath_type', data)
def test_port_profile_fp_ovs_backport_1_1(self):
datapath_offload = objects.vif.DatapathOffloadRepresentor(
representor_name="felix",
representor_address="0002:24:12.3")
obj = objects.vif.VIFPortProfileFPOpenVSwitch(
interface_id="07bd6cea-fb37-4594-b769-90fc51854ee9",
profile_id="fishfood",
datapath_type='netdev',
bridge_name="br-int",
hybrid_plug=False,
datapath_offload=datapath_offload)
primitive = obj.obj_to_primitive(target_version='1.1')
self.assertEqual('1.1', primitive['versioned_object.version'])
data = primitive['versioned_object.data']
self.assertEqual('07bd6cea-fb37-4594-b769-90fc51854ee9',
data['interface_id'])
self.assertEqual('fishfood', data['profile_id'])
self.assertEqual('br-int', data['bridge_name'])
self.assertEqual(False, data['hybrid_plug'])
self.assertEqual('netdev', data['datapath_type'])
self.assertNotIn('datapath_offload', data)
def test_vif_vhost_user_ovs_representor(self):
prof = objects.vif.VIFPortProfileOVSRepresentor(
interface_id="07bd6cea-fb37-4594-b769-90fc51854ee8",
@ -148,7 +201,7 @@ class TestVIFS(base.TestCase):
vif_name="tap123",
port_profile=prof)
def test_vif_vhost_user_ovs_representor_backport_1_0(self):
def test_port_profile_ovs_representor_backport_1_0(self):
obj = objects.vif.VIFPortProfileOVSRepresentor(
interface_id="07bd6cea-fb37-4594-b769-90fc51854ee9",
profile_id="fishfood",
@ -165,6 +218,41 @@ class TestVIFS(base.TestCase):
self.assertEqual("0002:24:12.3", data['representor_address'])
self.assertNotIn('datapath_type', data)
def test_port_profile_ovs_representor_backport_1_1(self):
datapath_offload = objects.vif.DatapathOffloadRepresentor(
representor_name="felix",
representor_address="0002:24:12.3")
obj = objects.vif.VIFPortProfileOVSRepresentor(
interface_id="07bd6cea-fb37-4594-b769-90fc51854ee9",
profile_id="fishfood",
datapath_type='netdev',
representor_name="tap123",
representor_address="0002:24:12.3",
datapath_offload=datapath_offload)
primitive = obj.obj_to_primitive(target_version='1.1')
self.assertEqual('1.1', primitive['versioned_object.version'])
data = primitive['versioned_object.data']
self.assertEqual('07bd6cea-fb37-4594-b769-90fc51854ee9',
data['interface_id'])
self.assertEqual('fishfood', data['profile_id'])
self.assertEqual('tap123', data['representor_name'])
self.assertEqual("0002:24:12.3", data['representor_address'])
self.assertEqual('netdev', data['datapath_type'])
self.assertNotIn('datapath_offload', data)
def test_vif_vhost_user_generic_representor(self):
datapath_offload = objects.vif.DatapathOffloadRepresentor(
representor_name="felix",
representor_address="0002:24:12.3")
prof = objects.vif.VIFPortProfileBase(
datapath_offload=datapath_offload,
)
self._test_vif(objects.vif.VIFVHostUser,
path="/some/socket.path",
mode=objects.fields.VIFVHostUserMode.SERVER,
vif_name="felix",
port_profile=prof)
def test_vif_vhost_user_fp_lb(self):
prof = objects.vif.VIFPortProfileFPBridge(bridge_name="brq456")
self._test_vif(objects.vif.VIFVHostUser,
@ -204,3 +292,104 @@ class TestVIFS(base.TestCase):
pci_adress="0002:24:12.3",
dev_driver="virtio_pci",
port_profile=prof)
def test_port_profile_fp_bridge_backport_1_0(self):
datapath_offload = objects.vif.DatapathOffloadRepresentor(
representor_name="felix",
representor_address="0002:24:12.3")
obj = objects.vif.VIFPortProfileFPBridge(
bridge_name='joe',
datapath_offload=datapath_offload)
primitive = obj.obj_to_primitive(target_version='1.0')
self.assertEqual('1.0', primitive['versioned_object.version'])
data = primitive['versioned_object.data']
self.assertEqual('joe', data['bridge_name'])
self.assertNotIn('datapath_type', data)
def test_port_profile_fp_tap_backport_1_0(self):
datapath_offload = objects.vif.DatapathOffloadRepresentor(
representor_name="felix",
representor_address="0002:24:12.3")
obj = objects.vif.VIFPortProfileFPTap(
mac_address='00:de:ad:be:ef:01',
datapath_offload=datapath_offload)
primitive = obj.obj_to_primitive(target_version='1.0')
self.assertEqual('1.0', primitive['versioned_object.version'])
data = primitive['versioned_object.data']
self.assertEqual('00:de:ad:be:ef:01', data['mac_address'])
self.assertNotIn('datapath_type', data)
def test_port_profile_8021qbg_backport_1_0(self):
datapath_offload = objects.vif.DatapathOffloadRepresentor(
representor_name="felix",
representor_address="0002:24:12.3")
obj = objects.vif.VIFPortProfile8021Qbg(
manager_id=42,
type_id=43,
type_id_version=44,
instance_id='07bd6cea-fb37-4594-b769-90fc51854ee9',
datapath_offload=datapath_offload)
primitive = obj.obj_to_primitive(target_version='1.0')
self.assertEqual('1.0', primitive['versioned_object.version'])
data = primitive['versioned_object.data']
self.assertEqual(42, data['manager_id'])
self.assertEqual(43, data['type_id'])
self.assertEqual(44, data['type_id_version'])
self.assertEqual('07bd6cea-fb37-4594-b769-90fc51854ee9',
data['instance_id'])
self.assertNotIn('datapath_type', data)
def test_port_profile_8021qbh_backport_1_0(self):
datapath_offload = objects.vif.DatapathOffloadRepresentor(
representor_name="felix",
representor_address="0002:24:12.3")
obj = objects.vif.VIFPortProfile8021Qbh(
profile_id='catfood',
datapath_offload=datapath_offload)
primitive = obj.obj_to_primitive(target_version='1.0')
self.assertEqual('1.0', primitive['versioned_object.version'])
data = primitive['versioned_object.data']
self.assertEqual('catfood', data['profile_id'])
self.assertNotIn('datapath_type', data)
def test_port_profile_dpdk_k8s_backport_1_0(self):
datapath_offload = objects.vif.DatapathOffloadRepresentor(
representor_name="felix",
representor_address="0002:24:12.3")
obj = objects.vif.VIFPortProfileK8sDPDK(
l3_setup=False,
selflink="/some/url",
resourceversion="1",
datapath_offload=datapath_offload)
primitive = obj.obj_to_primitive(target_version='1.0')
self.assertEqual('1.0', primitive['versioned_object.version'])
data = primitive['versioned_object.data']
self.assertEqual(False, data['l3_setup'])
self.assertEqual("/some/url", data['selflink'])
self.assertEqual("1", data['resourceversion'])
self.assertNotIn('datapath_type', data)
def test_vif_host_dev_ovs_offload(self):
datapath_offload = objects.vif.DatapathOffloadRepresentor(
representor_name="felix",
representor_address="0002:24:12.3")
prof = objects.vif.VIFPortProfileOpenVSwitch(
interface_id="07bd6cea-fb37-4594-b769-90fc51854ee8",
profile_id="fishfood",
datapath_type='netdev',
datapath_offload=datapath_offload)
self._test_vif(
objects.vif.VIFHostDevice,
dev_type=objects.fields.VIFHostDeviceDevType.ETHERNET,
dev_address="0002:24:12.3",
port_profile=prof)
def test_pending_warnings_emitted_class_direct(self):
with warnings.catch_warnings(record=True) as capture:
warnings.simplefilter("always")
pp = objects.vif.VIFPortProfileOVSRepresentor()
self.assertEqual(1, len(capture))
w = capture[0]
self.assertEqual(PendingDeprecationWarning, w.category)
self.assertEqual(pp.VERSION,
objects.vif.VIFPortProfileOVSRepresentor.VERSION)

11
releasenotes/notes/generic-datapath-offloads-41cabb6842b41533.yaml Normal file
View File

@ -0,0 +1,11 @@
---
features:
- A new set of attributes to port profiles has been introduced, namely
``Datapath Offload Types``, with ``DatapathOffloadRepresentor`` allowing
os-vif to pass the required metadata for representors conforming to the
kernel switchdev representor model.
deprecations:
- The API for ``VIFPortProfileOVSRepresentor`` has been frozen pending
deprecation of the class. Users should transition to setting the
``datapath_offload`` of ``VIFPortProfileOpenVSwitch`` to a
``DatapathOffloadRepresentor`` object to pass representor information.

1
requirements.txt
View File

@ -14,3 +14,4 @@ ovsdbapp>=0.12.1 # Apache-2.0
pyroute2>=0.5.2;sys_platform!='win32' # Apache-2.0 (+ dual licensed GPL2)
six>=1.10.0 # MIT
stevedore>=1.20.0 # Apache-2.0
debtcollector>=1.19.0 # Apache-2.0