Spec: Support OCI image registry with authentication turned on

This specification proposes support for deploying openstack services
using OSH with OCI image registry which has authentication turned on.

Change-Id: I26e34a5a39c06e9d481af58c15fb930d3fe9b1ef
Implements: blueprint support-oci-image-registry-with-authentication-turned-on
Signed-off-by: Angie Wang <angie.wang@windriver.com>
This commit is contained in:
Angie Wang 2019-02-11 14:08:33 -05:00 committed by Pete Birley
parent f191c62d13
commit 75f3083d24
1 changed files with 204 additions and 0 deletions

View File

@ -0,0 +1,204 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
..
========================================================
Support OCI image registry with authentication turned on
========================================================
Blueprint:
support-oci-image-registry-with-authentication-turned-on_
.. _support-oci-image-registry-with-authentication-turned-on: https://blueprints.launchpad.net/openstack-helm/+spec/support-oci-image-registry-with-authentication-turned-on
Problem Description
===================
In the current openstack-helm, all charts provide an ``images:`` section in
their ``values.yaml`` that have the container images references. By default,
the container images are all downloaded from a registry hosted by Docker or Quay.
However, the image references can be overridden by operators to download images
from any OCI image registry. In the case that the OCI image registry has
authentication turned on, kubelet would fail to download the images because the
current Openstack-Helm does not provide a way to pass the OCI image registry
credentials to kubernetes when pulling images.
Use case
========
Operators should be able to use Openstack-Helm to deploy containerized openstack
services with a docker registry has authentication turned on.
Proposed Change
===============
To be able to pull images from an OCI image registry which has the authentication
turned on, kubernetes needs credentials. For each chart, a new ``endpoints:``
entry could be added in ``values.yaml`` to provide image credentials, a secret
needs to be generated to hold the credentials and the ``imagePullSecrets:`` field
should be added in each service account to specify which secret should be used
to get the credentials from when pulling images by kubelet.
The detailed proposes change are described as following:
1. For each chart, add a new entry ``oci_image_registry:`` under ``endpoints:`` in
``values.yaml``. The entry ``oci_image_registry:`` has the ``auth:`` section which
provides the credentials for accessing registry images and an option ``enabled:``
to determine whether images authentication is required or not. The registry basic
information would also be included for generating the registry URL by the endpoint
lookup functions. Also add a new entry ``oci_image_registry:`` under ``secrets:``
to indicate the secret name. In order to create the secret that holds the provided
credentials, add a new component ``secret_registry`` in ``manifests:`` section.
For example:
.. code-block:: yaml
secrets:
oci_image_registry:
nova: nova-oci-image-registry-key
endpoints:
...
oci_image_registry:
name: oci-image-registry
namespace: oci-image-registry
auth:
enabled: false
nova:
username: nova
password: password
hosts:
default: localhost
host_fqdn_override:
default: null
port:
registry:
default: 5000
manifests:
secret_registry: true
The option ``enabled:`` under ``auth:`` and the manifest ``secret_registry:``
provide the ability for operator to determine whether they would like to have
secrets generated and passed to kubernetes for pulling images.
The secret would not be created with the default option ``enabled: false`` and
``secret_registry: true``. To enable secret creation, operator should override
``enabled:`` to true. The above example shows the default credentials, operator
should override the ``username:`` and ``password:`` under ``auth:`` section to
provide their own credentials.
Then, add manifest ``secret-registry.yaml`` in ``templates/`` to leverage
the function that will be added in helm-toolkit to create the secret. For example:
.. code-block:: yaml
{{- if and .Values.manifests.secret_registry .Values.endpoints.oci_image_registry.auth.enabled }}
{{ include "helm-toolkit.manifests.secret_registry" ( dict "envAll" . "registryUser" .Chart.Name ) }}
{{- end }}
2. Add a helm-toolkit function ``helm-toolkit.manifests.secret_registry`` to create a
manifest for secret generation. For example:
.. code-block:: rst
{{- define "helm-toolkit.manifests.secret_registry" -}}
{{- $envAll := index . "envAll" }}
{{- $registryUser := index . "registryUser" }}
{{- $secretName := index $envAll.Values.secrets.oci_image_registry $registryUser }}
{{- $registryHost := tuple "oci_image_registry" "internal" $envAll | include "helm-toolkit.endpoints.endpoint_host_lookup" }}
{{- $registryPort := tuple "oci_image_registry" "internal" "registry" $envAll | include "helm-toolkit.endpoints.endpoint_port_lookup" }}
{{- $imageCredentials := index $envAll.Values.endpoints.oci_image_registry.auth $registryUser }}
{{- $dockerAuthToken := printf "%s:%s" $imageCredentials.username $imageCredentials.password | b64enc }}
{{- $dockerAuth := printf "{\"auths\": {\"%s:%s\": {\"auth\": \"%s\"}}}" $registryHost $registryPort $dockerAuthToken | b64enc }}
---
apiVersion: v1
kind: Secret
metadata:
name: {{ $secretName }}
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: {{ $dockerAuth }}
{{- end }}
3. Reference the created secret by adding the ``imagePullSecrets:`` field to ServiceAccount
resource template [2]_ in ``helm-toolkit/snippets/_kubernetes_pod_rbac_serviceaccount.tpl``.
To handle it as optional, the field is wrapped in a conditional. For example,
.. code-block:: yaml
---
apiVersion: v1
kind: ServiceAccount
...
{{- if $envAll.Values.endpoints.oci_image_registry.auth.enabled }}
imagePullSecrets:
- name: {{ index $envAll.Values.secrets.oci_image_registry $envAll.Chart.Name }}
{{- end }}
If .Values.endpoints.oci_image_registry.auth.enabled will be set to true, then any
containers created with the current service account will have the ``imagePullSecrets``
automatically added to their spec and the secret will be passed to kubelet to be
used for pulling images.
Security Impact
---------------
The credentials for the registry could be exposed by running the kubectl command:
kubectl get secret <secret-name> --output="jsonpath={.data.\.dockerconfigjson}" | base64 --decode
Authentication should be enabled for normal users to access Kube API server via
either kubectl command or kube REST API call.
Performance Impact
------------------
No performance impact
Alternatives
------------
Before using Openstack-Helm to deploy openstack services,
1. Put .docker/config.json in docker/kubelet root directory on all nodes
2. Pre-pulling images on all nodes
But above alternatives have limitations and security impact. i.e...require root access
to configure on all nodes, all pods can read any configured private registries, all pods
can use any images cached on a node [1]_
Implementation
==============
Assignee(s)
-----------
Primary assignees:
* Angie Wang (angiewang)
Work Items
----------
#. Provide the credentials and add the manifest across all charts in OSH and OSH-infra
#. Update helm-toolkit to provide manifest to create secret for registry authentication
#. Update helm-toolkit serviceaccount template to pass the secret in a conditional
Testing
=======
None
Documentation Impact
====================
Documentation of how to enable the registry secret generation
References
==========
.. [1] https://kubernetes.io/docs/concepts/containers/images
.. [2] https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#add-imagepullsecrets-to-a-service-account