Merge "Add security groups driver for NP"
This commit is contained in:
commit
a36437ef86
|
@ -9,23 +9,27 @@ handlers at kuryr.conf (further info on how to do this can be found at
|
|||
[kubernetes]
|
||||
enabled_handlers=vif,lb,lbaasspec,policy
|
||||
|
||||
After that, enable also the security group drivers for policies::
|
||||
|
||||
[kubernetes]
|
||||
service_security_groups_driver = policy
|
||||
pod_security_groups_driver = policy
|
||||
|
||||
Note you need to restart the kuryr controller after applying the above step.
|
||||
For devstack non-containerized deployments::
|
||||
|
||||
$ sudo systemctl restart devstack@kuryr-kubernetes.service
|
||||
|
||||
|
||||
Same for containerized deployments::
|
||||
|
||||
$ kubectl -n kube-system get pod | grep kuryr-controller
|
||||
$ kubectl -n kube-system delete pod KURYR_CONTROLLER_POD_NAME
|
||||
|
||||
|
||||
For directly enabling the driver when deploying with devstack, you just need
|
||||
to add the policy handler with::
|
||||
to add the policy handler and drivers with::
|
||||
|
||||
KURYR_ENABLED_HANDLERS=vif,lb,lbaasspec,policy
|
||||
|
||||
KURYR_SG_DRIVER=policy
|
||||
|
||||
Testing the network policy support functionality
|
||||
------------------------------------------------
|
||||
|
@ -35,25 +39,25 @@ Testing the network policy support functionality
|
|||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: test-network-policy
|
||||
namespace: default
|
||||
name: test-network-policy
|
||||
namespace: default
|
||||
spec:
|
||||
podSelector:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
role: db
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
ingress:
|
||||
- from:
|
||||
role: db
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
ingress:
|
||||
- from:
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 6379
|
||||
egress:
|
||||
- to:
|
||||
port: 6379
|
||||
egress:
|
||||
- to:
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 5978
|
||||
port: 5978
|
||||
|
||||
2. Apply the network policy::
|
||||
|
||||
|
@ -116,10 +120,10 @@ Testing the network policy support functionality
|
|||
securityGroupName: sg-test-network-policy
|
||||
networkpolicy_spec:
|
||||
egress:
|
||||
- ports:
|
||||
- to:
|
||||
ports:
|
||||
- port: 5978
|
||||
protocol: TCP
|
||||
to:
|
||||
ingress:
|
||||
- from:
|
||||
ports:
|
||||
|
@ -140,9 +144,39 @@ Testing the network policy support functionality
|
|||
| tcp | 5978:5978 | egress |
|
||||
+-------------+------------+-----------+
|
||||
|
||||
5. Network policies can also be updated in the following way::
|
||||
5. Create a pod::
|
||||
|
||||
$ kubectl patch networkpolicy test-network-policy -p '{"spec":{"ingress":[{"ports":[{"port": 8081,"protocol": "UDP"}]}]}}'
|
||||
$ kubectl create deployment --image kuryr/demo demo
|
||||
deployment "demo" created
|
||||
|
||||
$ kubectl get pod -o wide
|
||||
NAME READY STATUS RESTARTS AGE IP
|
||||
demo-5558c7865d-fdkdv 1/1 Running 0 44s 10.0.0.68
|
||||
|
||||
6. Get the pod port and check its security group rules::
|
||||
|
||||
$ openstack port list --fixed-ip ip-address=10.0.0.68 -f value -c ID
|
||||
5d29b83c-714c-4579-8987-d0c0558420b3
|
||||
|
||||
$ openstack port show 5d29b83c-714c-4579-8987-d0c0558420b3 | grep security_group_ids
|
||||
| security_group_ids | bb2ac605-56ff-4688-b4f1-1d045ad251d0
|
||||
|
||||
$ openstack security group rule list bb2ac605-56ff-4688-b4f1-1d045ad251d0
|
||||
--protocol tcp -c "IP Protocol" -c "Port Range"
|
||||
+-------------+------------+-----------+
|
||||
| IP Protocol | Port Range | Direction |
|
||||
+-------------+------------+-----------+
|
||||
| tcp | 6379:6379 | ingress |
|
||||
| tcp | 5978:5978 | egress |
|
||||
+-------------+------------+-----------+
|
||||
|
||||
7. Try to curl the pod on port 8080 (hint: it won't work!)::
|
||||
|
||||
$ curl 10.0.0.68:8080
|
||||
|
||||
8. Update network policy to allow ingress 8080 port::
|
||||
|
||||
$ kubectl patch networkpolicy test-network-policy -p '{"spec":{"ingress":[{"ports":[{"port": 8080,"protocol": "TCP"}]}]}}'
|
||||
networkpolicy "test-network-policy" patched
|
||||
|
||||
$ kubectl get knp np-test-network-policy -o yaml
|
||||
|
@ -178,9 +212,9 @@ Testing the network policy support functionality
|
|||
direction: ingress
|
||||
ethertype: IPv4
|
||||
id: 6598aa1f-4f94-4fb2-81ce-d3649ba28f33
|
||||
port_range_max: 8081
|
||||
port_range_min: 8081
|
||||
protocol: udp
|
||||
port_range_max: 8080
|
||||
port_range_min: 8080
|
||||
protocol: tcp
|
||||
security_group_id: cdee7815-3b49-4a3e-abc8-31e384ab75c5
|
||||
securityGroupId: cdee7815-3b49-4a3e-abc8-31e384ab75c5
|
||||
networkpolicy_spec:
|
||||
|
@ -191,8 +225,8 @@ Testing the network policy support functionality
|
|||
to:
|
||||
ingress:
|
||||
- ports:
|
||||
- port: 8081
|
||||
protocol: UDP
|
||||
- port: 8080
|
||||
protocol: TCP
|
||||
policyTypes:
|
||||
- Ingress
|
||||
- Egress
|
||||
|
@ -201,16 +235,18 @@ Testing the network policy support functionality
|
|||
+-------------+------------+-----------+
|
||||
| IP Protocol | Port Range | Direction |
|
||||
+-------------+------------+-----------+
|
||||
| tcp | 6379:6379 | ingress |
|
||||
| udp | 8081:8081 | egress |
|
||||
| tcp | 8080:8080 | ingress |
|
||||
| tcp | 5978:5978 | egress |
|
||||
+-------------+------------+-----------+
|
||||
|
||||
6. Confirm the teardown of the resources once the network policy is removed::
|
||||
9. Try to curl the pod ip after patching the network policy::
|
||||
|
||||
$ curl 10.0.0.68:8080
|
||||
demo-5558c7865d-fdkdv: HELLO! I AM ALIVE!!!
|
||||
|
||||
10. Confirm the teardown of the resources once the network policy is removed::
|
||||
|
||||
$ kubectl delete -f network_policy.yml
|
||||
|
||||
$ kubectl get kuryrnetpolicies
|
||||
|
||||
$ kubectl get networkpolicies
|
||||
|
||||
$ openstack security group list | grep sg-test-network-policy
|
||||
|
|
|
@ -299,6 +299,7 @@ class NetworkPolicyDriver(base.NetworkPolicyDriver):
|
|||
networkpolicy_name = policy['metadata']['name']
|
||||
netpolicy_crd_name = "np-" + networkpolicy_name
|
||||
namespace = policy['metadata']['namespace']
|
||||
pod_selector = policy['spec'].get('podSelector')
|
||||
|
||||
netpolicy_crd = {
|
||||
'apiVersion': 'openstack.org/v1',
|
||||
|
@ -320,6 +321,14 @@ class NetworkPolicyDriver(base.NetworkPolicyDriver):
|
|||
'networkpolicy_spec': policy['spec']
|
||||
},
|
||||
}
|
||||
if pod_selector:
|
||||
try:
|
||||
netpolicy_crd['metadata']['labels'] = pod_selector[
|
||||
'matchLabels']
|
||||
except KeyError:
|
||||
# NOTE(ltomasbo): Only supporting matchLabels for now
|
||||
LOG.info("Pod Selector only allowed with matchLabels")
|
||||
|
||||
try:
|
||||
LOG.debug("Creating KuryrNetPolicy CRD %s" % netpolicy_crd)
|
||||
kubernetes_post = '{}/{}/kuryrnetpolicies'.format(
|
||||
|
|
|
@ -0,0 +1,125 @@
|
|||
# Copyright 2018 Red Hat, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from six.moves.urllib.parse import urlencode
|
||||
|
||||
from kuryr_kubernetes import clients
|
||||
from kuryr_kubernetes import config
|
||||
from kuryr_kubernetes import constants
|
||||
from kuryr_kubernetes.controller.drivers import base
|
||||
from kuryr_kubernetes import exceptions
|
||||
|
||||
from oslo_config import cfg
|
||||
from oslo_log import log as logging
|
||||
|
||||
LOG = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _get_kuryrnetpolicy_crds(labels=None, namespace='default'):
|
||||
kubernetes = clients.get_kubernetes_client()
|
||||
try:
|
||||
if labels:
|
||||
LOG.debug("Using labels %s", labels)
|
||||
labels.pop('pod-template-hash', None)
|
||||
# removing pod-template-hash is necessary to fetch the proper list
|
||||
labels = urlencode(labels)
|
||||
knp_path = '{}/{}/kuryrnetpolicies?labelSelector={}'.format(
|
||||
constants.K8S_API_CRD_NAMESPACES, namespace, labels)
|
||||
LOG.debug("K8s API Query %s", knp_path)
|
||||
knps = kubernetes.get(knp_path)
|
||||
LOG.debug("Return Kuryr Network Policies with label %s", knps)
|
||||
else:
|
||||
knps = kubernetes.get('{}/{}/kuryrnetpolicies'.format(
|
||||
constants.K8S_API_CRD_NAMESPACES, namespace))
|
||||
except exceptions.K8sResourceNotFound:
|
||||
LOG.exception("KuryrNetPolicy CRD not found")
|
||||
raise
|
||||
except exceptions.K8sClientException:
|
||||
LOG.exception("Kubernetes Client Exception")
|
||||
raise
|
||||
return knps
|
||||
|
||||
|
||||
class NetworkPolicySecurityGroupsDriver(base.PodSecurityGroupsDriver):
|
||||
"""Provides security groups for pods based on network policies"""
|
||||
|
||||
def get_security_groups(self, pod, project_id):
|
||||
sg_list = []
|
||||
pod_namespace = pod['metadata']['namespace']
|
||||
pod_labels = pod['metadata'].get('labels')
|
||||
LOG.debug("Using labels %s", pod_labels)
|
||||
|
||||
knp_crds = _get_kuryrnetpolicy_crds(pod_labels,
|
||||
namespace=pod_namespace)
|
||||
|
||||
for crd in knp_crds.get('items'):
|
||||
LOG.debug("Appending %s", str(crd['spec']['securityGroupId']))
|
||||
sg_list.append(str(crd['spec']['securityGroupId']))
|
||||
|
||||
knp_namespace_crds = _get_kuryrnetpolicy_crds(namespace=pod_namespace)
|
||||
|
||||
for crd in knp_namespace_crds.get('items'):
|
||||
if not crd['metadata'].get('labels'):
|
||||
LOG.debug("Appending %s", str(crd['spec']['securityGroupId']))
|
||||
sg_list.append(str(crd['spec']['securityGroupId']))
|
||||
|
||||
if not sg_list:
|
||||
sg_list = config.CONF.neutron_defaults.pod_security_groups
|
||||
if not sg_list:
|
||||
raise cfg.RequiredOptError('pod_security_groups',
|
||||
cfg.OptGroup('neutron_defaults'))
|
||||
|
||||
return sg_list[:]
|
||||
|
||||
def create_namespace_sg(self, namespace, project_id, crd_spec):
|
||||
LOG.debug("Security group driver does not create SGs for the "
|
||||
"namespaces.")
|
||||
return {}
|
||||
|
||||
def delete_sg(self, sg_id):
|
||||
LOG.debug("Security group driver does not implement deleting "
|
||||
"SGs.")
|
||||
|
||||
|
||||
class NetworkPolicyServiceSecurityGroupsDriver(
|
||||
base.ServiceSecurityGroupsDriver):
|
||||
"""Provides security groups for services based on network policies"""
|
||||
|
||||
def get_security_groups(self, service, project_id):
|
||||
sg_list = []
|
||||
svc_namespace = service['metadata']['namespace']
|
||||
svc_labels = service['metadata'].get('labels')
|
||||
LOG.debug("Using labels %s", svc_labels)
|
||||
|
||||
knp_crds = _get_kuryrnetpolicy_crds(svc_labels,
|
||||
namespace=svc_namespace)
|
||||
|
||||
for crd in knp_crds.get('items'):
|
||||
LOG.debug("Appending %s" % str(crd['spec']['securityGroupId']))
|
||||
sg_list.append(str(crd['spec']['securityGroupId']))
|
||||
|
||||
knp_namespace_crds = _get_kuryrnetpolicy_crds(namespace=svc_namespace)
|
||||
|
||||
for crd in knp_namespace_crds.get('items'):
|
||||
if not crd['metadata'].get('labels'):
|
||||
LOG.debug("Appending %s", str(crd['spec']['securityGroupId']))
|
||||
sg_list.append(str(crd['spec']['securityGroupId']))
|
||||
|
||||
if not sg_list:
|
||||
sg_list = config.CONF.neutron_defaults.pod_security_groups
|
||||
if not sg_list:
|
||||
raise cfg.RequiredOptError('pod_security_groups',
|
||||
cfg.OptGroup('neutron_defaults'))
|
||||
|
||||
return sg_list[:]
|
|
@ -0,0 +1,119 @@
|
|||
# Copyright 2018 Red Hat, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import mock
|
||||
|
||||
from kuryr_kubernetes.controller.drivers import network_policy_security_groups
|
||||
from kuryr_kubernetes.tests import base as test_base
|
||||
from kuryr_kubernetes.tests.unit import kuryr_fixtures as k_fix
|
||||
|
||||
from oslo_config import cfg
|
||||
|
||||
|
||||
class TestNetworkPolicySecurityGroupsDriver(test_base.TestCase):
|
||||
|
||||
def setUp(self):
|
||||
super(TestNetworkPolicySecurityGroupsDriver, self).setUp()
|
||||
self._labels = mock.sentinel.labels
|
||||
self._project_id = mock.sentinel.project_id
|
||||
self._sg_id = mock.sentinel.sg_id
|
||||
self._namespace = 'default'
|
||||
self._crd = {
|
||||
'metadata': {'name': mock.sentinel.name,
|
||||
'selfLink': mock.sentinel.selfLink},
|
||||
'spec': {
|
||||
'egressSgRules': [
|
||||
{'security_group_rule':
|
||||
{'description': 'Kuryr-Kubernetes NetPolicy SG rule',
|
||||
'direction': 'egress',
|
||||
'ethertype': 'IPv4',
|
||||
'port_range_max': 5978,
|
||||
'port_range_min': 5978,
|
||||
'protocol': 'tcp',
|
||||
'security_group_id': self._sg_id,
|
||||
'id': mock.sentinel.id
|
||||
}}],
|
||||
'ingressSgRules': [
|
||||
{'security_group_rule':
|
||||
{'description': 'Kuryr-Kubernetes NetPolicy SG rule',
|
||||
'direction': 'ingress',
|
||||
'ethertype': 'IPv4',
|
||||
'port_range_max': 6379,
|
||||
'port_range_min': 6379,
|
||||
'protocol': 'tcp',
|
||||
'security_group_id': self._sg_id,
|
||||
'id': mock.sentinel.id
|
||||
}}],
|
||||
'securityGroupId': self._sg_id,
|
||||
'securityGroupName': mock.sentinel.sg_name}}
|
||||
|
||||
self._crds = {
|
||||
"apiVersion": "v1",
|
||||
"items": [self._crd],
|
||||
"kind": "List",
|
||||
"metadata": {
|
||||
"resourceVersion": "",
|
||||
"selfLink": mock.sentinel.selfLink}}
|
||||
|
||||
self._empty_crds = {
|
||||
"apiVersion": "v1",
|
||||
"items": [],
|
||||
"kind": "List",
|
||||
"metadata": {
|
||||
"resourceVersion": "",
|
||||
"selfLink": mock.sentinel.selfLink}}
|
||||
|
||||
self._pod = {
|
||||
'apiVersion': 'v1',
|
||||
'kind': 'Pod',
|
||||
'metadata': {
|
||||
'name': mock.sentinel.pod_name,
|
||||
'namespace': self._namespace,
|
||||
'labels': {
|
||||
'run': 'demo'}},
|
||||
'spec': {
|
||||
'containers': [{
|
||||
'image': 'kuryr/demo',
|
||||
'imagePullPolicy': 'Always',
|
||||
'name': mock.sentinel.pod_name
|
||||
}]
|
||||
}}
|
||||
|
||||
self.kubernetes = self.useFixture(k_fix.MockK8sClient()).client
|
||||
self._driver = (
|
||||
network_policy_security_groups.NetworkPolicySecurityGroupsDriver())
|
||||
|
||||
@mock.patch.object(network_policy_security_groups,
|
||||
'_get_kuryrnetpolicy_crds')
|
||||
def test_get_security_groups(self, m_get_crds):
|
||||
m_get_crds.return_value = self._crds
|
||||
self._driver.get_security_groups(self._pod, self._project_id)
|
||||
m_get_crds.assert_called_with(namespace=self._namespace)
|
||||
|
||||
@mock.patch.object(network_policy_security_groups,
|
||||
'_get_kuryrnetpolicy_crds')
|
||||
def test_get_security_groups_with_label(self, m_get_crds):
|
||||
labels = {'run': 'demo'}
|
||||
self._crds['metadata']['labels'] = labels
|
||||
m_get_crds.return_value = self._crds
|
||||
self._driver.get_security_groups(self._pod, self._project_id)
|
||||
m_get_crds.assert_called()
|
||||
|
||||
@mock.patch.object(network_policy_security_groups,
|
||||
'_get_kuryrnetpolicy_crds')
|
||||
def test_get_security_groups_no_crds(self, m_get_crds):
|
||||
m_get_crds.return_value = self._empty_crds
|
||||
self.assertRaises(cfg.RequiredOptError,
|
||||
self._driver.get_security_groups, self._pod,
|
||||
self._project_id)
|
|
@ -66,10 +66,12 @@ kuryr_kubernetes.controller.drivers.service_subnets =
|
|||
kuryr_kubernetes.controller.drivers.pod_security_groups =
|
||||
default = kuryr_kubernetes.controller.drivers.default_security_groups:DefaultPodSecurityGroupsDriver
|
||||
namespace = kuryr_kubernetes.controller.drivers.namespace_security_groups:NamespacePodSecurityGroupsDriver
|
||||
policy = kuryr_kubernetes.controller.drivers.network_policy_security_groups:NetworkPolicySecurityGroupsDriver
|
||||
|
||||
kuryr_kubernetes.controller.drivers.service_security_groups =
|
||||
default = kuryr_kubernetes.controller.drivers.default_security_groups:DefaultServiceSecurityGroupsDriver
|
||||
namespace = kuryr_kubernetes.controller.drivers.namespace_security_groups:NamespaceServiceSecurityGroupsDriver
|
||||
policy = kuryr_kubernetes.controller.drivers.network_policy_security_groups:NetworkPolicyServiceSecurityGroupsDriver
|
||||
|
||||
kuryr_kubernetes.controller.drivers.network_policy =
|
||||
default = kuryr_kubernetes.controller.drivers.network_policy:NetworkPolicyDriver
|
||||
|
|
Loading…
Reference in New Issue