Add security groups driver for NP

This commit creates a new security group driver for handling network
policies sg and sg rules.

Partially Implements: blueprint k8s-network-policies
Change-Id: Ie4dfac39704f4bbfb31eb329cd43ab8a06addf0d
This commit is contained in:
Daniel Mellado 2018-10-08 05:16:17 -04:00 committed by Luis Tomas Bolivar
parent 76db817fb5
commit 6dfd4067f5
5 changed files with 323 additions and 32 deletions

View File

@ -9,23 +9,27 @@ handlers at kuryr.conf (further info on how to do this can be found at
[kubernetes]
enabled_handlers=vif,lb,lbaasspec,policy
After that, enable also the security group drivers for policies::
[kubernetes]
service_security_groups_driver = policy
pod_security_groups_driver = policy
Note you need to restart the kuryr controller after applying the above step.
For devstack non-containerized deployments::
$ sudo systemctl restart devstack@kuryr-kubernetes.service
Same for containerized deployments::
$ kubectl -n kube-system get pod | grep kuryr-controller
$ kubectl -n kube-system delete pod KURYR_CONTROLLER_POD_NAME
For directly enabling the driver when deploying with devstack, you just need
to add the policy handler with::
to add the policy handler and drivers with::
KURYR_ENABLED_HANDLERS=vif,lb,lbaasspec,policy
KURYR_SG_DRIVER=policy
Testing the network policy support functionality
------------------------------------------------
@ -35,25 +39,25 @@ Testing the network policy support functionality
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: test-network-policy
namespace: default
name: test-network-policy
namespace: default
spec:
podSelector:
podSelector:
matchLabels:
role: db
policyTypes:
- Ingress
- Egress
ingress:
- from:
role: db
policyTypes:
- Ingress
- Egress
ingress:
- from:
ports:
- protocol: TCP
port: 6379
egress:
- to:
port: 6379
egress:
- to:
ports:
- protocol: TCP
port: 5978
port: 5978
2. Apply the network policy::
@ -116,10 +120,10 @@ Testing the network policy support functionality
securityGroupName: sg-test-network-policy
networkpolicy_spec:
egress:
- ports:
- to:
ports:
- port: 5978
protocol: TCP
to:
ingress:
- from:
ports:
@ -140,9 +144,39 @@ Testing the network policy support functionality
| tcp | 5978:5978 | egress |
+-------------+------------+-----------+
5. Network policies can also be updated in the following way::
5. Create a pod::
$ kubectl patch networkpolicy test-network-policy -p '{"spec":{"ingress":[{"ports":[{"port": 8081,"protocol": "UDP"}]}]}}'
$ kubectl create deployment --image kuryr/demo demo
deployment "demo" created
$ kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP
demo-5558c7865d-fdkdv 1/1 Running 0 44s 10.0.0.68
6. Get the pod port and check its security group rules::
$ openstack port list --fixed-ip ip-address=10.0.0.68 -f value -c ID
5d29b83c-714c-4579-8987-d0c0558420b3
$ openstack port show 5d29b83c-714c-4579-8987-d0c0558420b3 | grep security_group_ids
| security_group_ids | bb2ac605-56ff-4688-b4f1-1d045ad251d0
$ openstack security group rule list bb2ac605-56ff-4688-b4f1-1d045ad251d0
--protocol tcp -c "IP Protocol" -c "Port Range"
+-------------+------------+-----------+
| IP Protocol | Port Range | Direction |
+-------------+------------+-----------+
| tcp | 6379:6379 | ingress |
| tcp | 5978:5978 | egress |
+-------------+------------+-----------+
7. Try to curl the pod on port 8080 (hint: it won't work!)::
$ curl 10.0.0.68:8080
8. Update network policy to allow ingress 8080 port::
$ kubectl patch networkpolicy test-network-policy -p '{"spec":{"ingress":[{"ports":[{"port": 8080,"protocol": "TCP"}]}]}}'
networkpolicy "test-network-policy" patched
$ kubectl get knp np-test-network-policy -o yaml
@ -178,9 +212,9 @@ Testing the network policy support functionality
direction: ingress
ethertype: IPv4
id: 6598aa1f-4f94-4fb2-81ce-d3649ba28f33
port_range_max: 8081
port_range_min: 8081
protocol: udp
port_range_max: 8080
port_range_min: 8080
protocol: tcp
security_group_id: cdee7815-3b49-4a3e-abc8-31e384ab75c5
securityGroupId: cdee7815-3b49-4a3e-abc8-31e384ab75c5
networkpolicy_spec:
@ -191,8 +225,8 @@ Testing the network policy support functionality
to:
ingress:
- ports:
- port: 8081
protocol: UDP
- port: 8080
protocol: TCP
policyTypes:
- Ingress
- Egress
@ -201,16 +235,18 @@ Testing the network policy support functionality
+-------------+------------+-----------+
| IP Protocol | Port Range | Direction |
+-------------+------------+-----------+
| tcp | 6379:6379 | ingress |
| udp | 8081:8081 | egress |
| tcp | 8080:8080 | ingress |
| tcp | 5978:5978 | egress |
+-------------+------------+-----------+
6. Confirm the teardown of the resources once the network policy is removed::
9. Try to curl the pod ip after patching the network policy::
$ curl 10.0.0.68:8080
demo-5558c7865d-fdkdv: HELLO! I AM ALIVE!!!
10. Confirm the teardown of the resources once the network policy is removed::
$ kubectl delete -f network_policy.yml
$ kubectl get kuryrnetpolicies
$ kubectl get networkpolicies
$ openstack security group list | grep sg-test-network-policy

View File

@ -299,6 +299,7 @@ class NetworkPolicyDriver(base.NetworkPolicyDriver):
networkpolicy_name = policy['metadata']['name']
netpolicy_crd_name = "np-" + networkpolicy_name
namespace = policy['metadata']['namespace']
pod_selector = policy['spec'].get('podSelector')
netpolicy_crd = {
'apiVersion': 'openstack.org/v1',
@ -320,6 +321,14 @@ class NetworkPolicyDriver(base.NetworkPolicyDriver):
'networkpolicy_spec': policy['spec']
},
}
if pod_selector:
try:
netpolicy_crd['metadata']['labels'] = pod_selector[
'matchLabels']
except KeyError:
# NOTE(ltomasbo): Only supporting matchLabels for now
LOG.info("Pod Selector only allowed with matchLabels")
try:
LOG.debug("Creating KuryrNetPolicy CRD %s" % netpolicy_crd)
kubernetes_post = '{}/{}/kuryrnetpolicies'.format(

View File

@ -0,0 +1,125 @@
# Copyright 2018 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from six.moves.urllib.parse import urlencode
from kuryr_kubernetes import clients
from kuryr_kubernetes import config
from kuryr_kubernetes import constants
from kuryr_kubernetes.controller.drivers import base
from kuryr_kubernetes import exceptions
from oslo_config import cfg
from oslo_log import log as logging
LOG = logging.getLogger(__name__)
def _get_kuryrnetpolicy_crds(labels=None, namespace='default'):
kubernetes = clients.get_kubernetes_client()
try:
if labels:
LOG.debug("Using labels %s", labels)
labels.pop('pod-template-hash', None)
# removing pod-template-hash is necessary to fetch the proper list
labels = urlencode(labels)
knp_path = '{}/{}/kuryrnetpolicies?labelSelector={}'.format(
constants.K8S_API_CRD_NAMESPACES, namespace, labels)
LOG.debug("K8s API Query %s", knp_path)
knps = kubernetes.get(knp_path)
LOG.debug("Return Kuryr Network Policies with label %s", knps)
else:
knps = kubernetes.get('{}/{}/kuryrnetpolicies'.format(
constants.K8S_API_CRD_NAMESPACES, namespace))
except exceptions.K8sResourceNotFound:
LOG.exception("KuryrNetPolicy CRD not found")
raise
except exceptions.K8sClientException:
LOG.exception("Kubernetes Client Exception")
raise
return knps
class NetworkPolicySecurityGroupsDriver(base.PodSecurityGroupsDriver):
"""Provides security groups for pods based on network policies"""
def get_security_groups(self, pod, project_id):
sg_list = []
pod_namespace = pod['metadata']['namespace']
pod_labels = pod['metadata'].get('labels')
LOG.debug("Using labels %s", pod_labels)
knp_crds = _get_kuryrnetpolicy_crds(pod_labels,
namespace=pod_namespace)
for crd in knp_crds.get('items'):
LOG.debug("Appending %s", str(crd['spec']['securityGroupId']))
sg_list.append(str(crd['spec']['securityGroupId']))
knp_namespace_crds = _get_kuryrnetpolicy_crds(namespace=pod_namespace)
for crd in knp_namespace_crds.get('items'):
if not crd['metadata'].get('labels'):
LOG.debug("Appending %s", str(crd['spec']['securityGroupId']))
sg_list.append(str(crd['spec']['securityGroupId']))
if not sg_list:
sg_list = config.CONF.neutron_defaults.pod_security_groups
if not sg_list:
raise cfg.RequiredOptError('pod_security_groups',
cfg.OptGroup('neutron_defaults'))
return sg_list[:]
def create_namespace_sg(self, namespace, project_id, crd_spec):
LOG.debug("Security group driver does not create SGs for the "
"namespaces.")
return {}
def delete_sg(self, sg_id):
LOG.debug("Security group driver does not implement deleting "
"SGs.")
class NetworkPolicyServiceSecurityGroupsDriver(
base.ServiceSecurityGroupsDriver):
"""Provides security groups for services based on network policies"""
def get_security_groups(self, service, project_id):
sg_list = []
svc_namespace = service['metadata']['namespace']
svc_labels = service['metadata'].get('labels')
LOG.debug("Using labels %s", svc_labels)
knp_crds = _get_kuryrnetpolicy_crds(svc_labels,
namespace=svc_namespace)
for crd in knp_crds.get('items'):
LOG.debug("Appending %s" % str(crd['spec']['securityGroupId']))
sg_list.append(str(crd['spec']['securityGroupId']))
knp_namespace_crds = _get_kuryrnetpolicy_crds(namespace=svc_namespace)
for crd in knp_namespace_crds.get('items'):
if not crd['metadata'].get('labels'):
LOG.debug("Appending %s", str(crd['spec']['securityGroupId']))
sg_list.append(str(crd['spec']['securityGroupId']))
if not sg_list:
sg_list = config.CONF.neutron_defaults.pod_security_groups
if not sg_list:
raise cfg.RequiredOptError('pod_security_groups',
cfg.OptGroup('neutron_defaults'))
return sg_list[:]

View File

@ -0,0 +1,119 @@
# Copyright 2018 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import mock
from kuryr_kubernetes.controller.drivers import network_policy_security_groups
from kuryr_kubernetes.tests import base as test_base
from kuryr_kubernetes.tests.unit import kuryr_fixtures as k_fix
from oslo_config import cfg
class TestNetworkPolicySecurityGroupsDriver(test_base.TestCase):
def setUp(self):
super(TestNetworkPolicySecurityGroupsDriver, self).setUp()
self._labels = mock.sentinel.labels
self._project_id = mock.sentinel.project_id
self._sg_id = mock.sentinel.sg_id
self._namespace = 'default'
self._crd = {
'metadata': {'name': mock.sentinel.name,
'selfLink': mock.sentinel.selfLink},
'spec': {
'egressSgRules': [
{'security_group_rule':
{'description': 'Kuryr-Kubernetes NetPolicy SG rule',
'direction': 'egress',
'ethertype': 'IPv4',
'port_range_max': 5978,
'port_range_min': 5978,
'protocol': 'tcp',
'security_group_id': self._sg_id,
'id': mock.sentinel.id
}}],
'ingressSgRules': [
{'security_group_rule':
{'description': 'Kuryr-Kubernetes NetPolicy SG rule',
'direction': 'ingress',
'ethertype': 'IPv4',
'port_range_max': 6379,
'port_range_min': 6379,
'protocol': 'tcp',
'security_group_id': self._sg_id,
'id': mock.sentinel.id
}}],
'securityGroupId': self._sg_id,
'securityGroupName': mock.sentinel.sg_name}}
self._crds = {
"apiVersion": "v1",
"items": [self._crd],
"kind": "List",
"metadata": {
"resourceVersion": "",
"selfLink": mock.sentinel.selfLink}}
self._empty_crds = {
"apiVersion": "v1",
"items": [],
"kind": "List",
"metadata": {
"resourceVersion": "",
"selfLink": mock.sentinel.selfLink}}
self._pod = {
'apiVersion': 'v1',
'kind': 'Pod',
'metadata': {
'name': mock.sentinel.pod_name,
'namespace': self._namespace,
'labels': {
'run': 'demo'}},
'spec': {
'containers': [{
'image': 'kuryr/demo',
'imagePullPolicy': 'Always',
'name': mock.sentinel.pod_name
}]
}}
self.kubernetes = self.useFixture(k_fix.MockK8sClient()).client
self._driver = (
network_policy_security_groups.NetworkPolicySecurityGroupsDriver())
@mock.patch.object(network_policy_security_groups,
'_get_kuryrnetpolicy_crds')
def test_get_security_groups(self, m_get_crds):
m_get_crds.return_value = self._crds
self._driver.get_security_groups(self._pod, self._project_id)
m_get_crds.assert_called_with(namespace=self._namespace)
@mock.patch.object(network_policy_security_groups,
'_get_kuryrnetpolicy_crds')
def test_get_security_groups_with_label(self, m_get_crds):
labels = {'run': 'demo'}
self._crds['metadata']['labels'] = labels
m_get_crds.return_value = self._crds
self._driver.get_security_groups(self._pod, self._project_id)
m_get_crds.assert_called()
@mock.patch.object(network_policy_security_groups,
'_get_kuryrnetpolicy_crds')
def test_get_security_groups_no_crds(self, m_get_crds):
m_get_crds.return_value = self._empty_crds
self.assertRaises(cfg.RequiredOptError,
self._driver.get_security_groups, self._pod,
self._project_id)

View File

@ -66,10 +66,12 @@ kuryr_kubernetes.controller.drivers.service_subnets =
kuryr_kubernetes.controller.drivers.pod_security_groups =
default = kuryr_kubernetes.controller.drivers.default_security_groups:DefaultPodSecurityGroupsDriver
namespace = kuryr_kubernetes.controller.drivers.namespace_security_groups:NamespacePodSecurityGroupsDriver
policy = kuryr_kubernetes.controller.drivers.network_policy_security_groups:NetworkPolicySecurityGroupsDriver
kuryr_kubernetes.controller.drivers.service_security_groups =
default = kuryr_kubernetes.controller.drivers.default_security_groups:DefaultServiceSecurityGroupsDriver
namespace = kuryr_kubernetes.controller.drivers.namespace_security_groups:NamespaceServiceSecurityGroupsDriver
policy = kuryr_kubernetes.controller.drivers.network_policy_security_groups:NetworkPolicyServiceSecurityGroupsDriver
kuryr_kubernetes.controller.drivers.network_policy =
default = kuryr_kubernetes.controller.drivers.network_policy:NetworkPolicyDriver