Merge "[docs] Add documentation on document encryption"
This commit is contained in:
commit
9ae805f988
|
@ -127,7 +127,8 @@ This type of metadata allows the following metadata hierarchy:
|
|||
* ``storagePolicy`` - string, required - Either ``cleartext`` or ``encrypted``.
|
||||
If ``encyrpted`` is specified, then the ``data`` section of the document will
|
||||
be stored in a secure backend (likely via OpenStack Barbican). ``metadata``
|
||||
and ``schema`` fields are always stored in cleartext.
|
||||
and ``schema`` fields are always stored in cleartext. More information
|
||||
on document encryption is available :ref:`here <encryption>`.
|
||||
* ``layeringDefinition`` - dict, required - Specifies layering details. See the
|
||||
Layering section below for details.
|
||||
|
||||
|
|
|
@ -0,0 +1,58 @@
|
|||
..
|
||||
Copyright 2018 AT&T Intellectual Property.
|
||||
All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
.. _encryption:
|
||||
|
||||
Data Encryption
|
||||
===============
|
||||
|
||||
Deckhand supports encrypting the ``data`` section of documents at-rest to
|
||||
secure sensitive data. This encryption behavior is triggered by setting
|
||||
``metadata.storagePolicy: encrypted``. It is solely the document author's
|
||||
responsibility to decide the appropriate storagePolicy for the data contained
|
||||
in the document.
|
||||
|
||||
.. note::
|
||||
|
||||
Note that encryption of document data incurs **runtime overhead** as the
|
||||
price of encryption is performance. As a general rule, the more documents
|
||||
with ``storagePolicy: encrypted``, the longer it will take to render the
|
||||
documents, particularly because Barbican has a built-in `restriction`_
|
||||
around retrieving only one encrypted payload a time. This means that
|
||||
if 50 documents have ``storagePolicy: encrypted`` within a revision, then
|
||||
Deckhand must perform 50 API calls to Barbican when rendering the documents
|
||||
for that revision.
|
||||
|
||||
Encrypted documents, like cleartext documents, are stored in Deckhand's
|
||||
database, except the ``data`` section of each encrypted document is replaced
|
||||
with a reference to Barbican.
|
||||
|
||||
.. _Barbican: https://docs.openstack.org/barbican/latest/api/
|
||||
.. _restriction: https://docs.openstack.org/barbican/latest/api/reference/secrets.html#get-v1-secrets
|
||||
|
||||
Supported Data Types
|
||||
--------------------
|
||||
|
||||
Barbican supports encrypting `any`_ data type via its "opaque" secret type.
|
||||
Thus, Deckhand supports encryption of any data type by utilizing this
|
||||
secret type.
|
||||
|
||||
However, Deckhand will attempt to use Barbican's `other`_ secret types where
|
||||
possible. For example, Deckhand will use "public" for document types with kind
|
||||
``PublicKey``.
|
||||
|
||||
.. _any: https://github.com/openstack/barbican/blob/7991f8b4850d76d97c3482428638f788f5798a56/barbican/plugin/interface/secret_store.py#L272
|
||||
.. _other: https://docs.openstack.org/barbican/latest/api/reference/secret_types.html
|
|
@ -43,7 +43,9 @@ User's Guide
|
|||
revision-history
|
||||
documents
|
||||
document-types
|
||||
encryption
|
||||
validation
|
||||
rendering
|
||||
substitution
|
||||
layering
|
||||
replacement
|
||||
|
|
|
@ -0,0 +1,45 @@
|
|||
..
|
||||
Copyright 2018 AT&T Intellectual Property.
|
||||
All Rights Reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
not use this file except in compliance with the License. You may obtain
|
||||
a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
License for the specific language governing permissions and limitations
|
||||
under the License.
|
||||
|
||||
.. _rendering:
|
||||
|
||||
Document Rendering
|
||||
==================
|
||||
|
||||
Document rendering involves extracting all raw revision documents from
|
||||
Deckhand's database, retrieving encrypted information from Barbican,
|
||||
and applying substitution, layering and replacement algorithms on the
|
||||
data.
|
||||
|
||||
The following algorithms are involved during the rendering process:
|
||||
|
||||
:ref:`substitution`
|
||||
-------------------
|
||||
|
||||
Substitution provides an "open" data sharing model in which any source
|
||||
document can be used to substitute data into any destination document.
|
||||
|
||||
:ref:`layering`
|
||||
---------------
|
||||
|
||||
Layering provides a "restricted" data inheritance model intended to help
|
||||
reduce duplication in configuration.
|
||||
|
||||
:ref:`replacement`
|
||||
------------------
|
||||
|
||||
Replacement builds on top of layering to provide yet another mechanism
|
||||
for reducing data duplication.
|
|
@ -19,14 +19,52 @@
|
|||
Document Substitution
|
||||
=====================
|
||||
|
||||
Introduction
|
||||
------------
|
||||
|
||||
Document substitution, simply put, allows one document to overwrite *parts* of
|
||||
its own data with that of another document. Substitution involves a source
|
||||
document sharing data with a destination document, which replaces its own data
|
||||
with the shared data.
|
||||
|
||||
Substitution is primarily designed as a mechanism for inserting secrets into
|
||||
configuration documents, but works for unencrypted source documents as well.
|
||||
Substitution is applied at each layer after all merge actions occur.
|
||||
Substitution may be leveraged as a mechanism for:
|
||||
|
||||
* inserting secrets into configuration documents
|
||||
* reducing data duplication by declaring common data within one document and
|
||||
having multiple other documents substitute data from the common location as
|
||||
needed
|
||||
|
||||
During document rendering, substitution is applied at each layer after all
|
||||
merge actions occur. For more information on the interaction between
|
||||
document layering and substitution, see: :ref:`rendering`.
|
||||
|
||||
Requirements
|
||||
------------
|
||||
|
||||
Substitutions between documents are not restricted by ``schema``, ``name``,
|
||||
nor ``layer``. Source and destination documents do not need to share the same
|
||||
``schema``.
|
||||
|
||||
No substitution dependency cycle may exist between a series of substitutions.
|
||||
For example, if A substitutes from B, B from C, and C from A, then Deckhand
|
||||
will raise an exception as it is impossible to determine the source data to use
|
||||
for substitution in the presence of a dependency cycle.
|
||||
|
||||
Substitution works like this:
|
||||
|
||||
The source document is resolved via the ``src.schema`` and ``src.name``
|
||||
keys and the ``src.path`` key is used relative to the source document's
|
||||
``data`` section to retrieve the substitution data, which is then injected
|
||||
into the ``data`` section of the destination document using the ``dest.path``
|
||||
key.
|
||||
|
||||
If all the constraints above are correct, then the substitution source data
|
||||
is injected into the destination document's ``data`` section, at the path
|
||||
specified by ``dest.path``.
|
||||
|
||||
The injection of data into the destination document can be more fine-tuned
|
||||
using a regular expression; see the :ref:`substitution-pattern` section
|
||||
below for more information.
|
||||
|
||||
.. note::
|
||||
|
||||
|
@ -34,12 +72,18 @@ Substitution is applied at each layer after all merge actions occur.
|
|||
because a document's ``metadata`` and ``schema`` sections should be
|
||||
immutable within the scope of a revision, for obvious reasons.
|
||||
|
||||
Rendering Documents with Substitution
|
||||
-------------------------------------
|
||||
|
||||
Concrete (non-abstract) documents can be used as a source of substitution
|
||||
into other documents. This substitution is layer-independent, so given the 3
|
||||
layer example above, which includes ``global``, ``region`` and ``site`` layers,
|
||||
a document in the ``region`` layer could insert data from a document in the
|
||||
``site`` layer.
|
||||
|
||||
Example
|
||||
^^^^^^^
|
||||
|
||||
Here is a sample set of documents demonstrating substitution:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
@ -149,5 +193,96 @@ The rendered document will look like:
|
|||
KEY DATA
|
||||
...
|
||||
|
||||
This substitution is also ``schema`` agnostic, meaning that source and
|
||||
destination documents can have a different ``schema``.
|
||||
.. _substitution-pattern:
|
||||
|
||||
Substitution with Patterns
|
||||
--------------------------
|
||||
|
||||
Substitution can be controlled in a more fine-tuned fashion using
|
||||
``dest.pattern`` (optional) which functions as a regular expression underneath
|
||||
the hood. The ``dest.pattern`` has the following constraints:
|
||||
|
||||
* ``dest.path`` key must already exist in the ``data`` section of the
|
||||
destination document and must have an associated value.
|
||||
* The ``dest.pattern`` must be a valid regular expression string.
|
||||
* The ``dest.pattern`` must be resolvable in the value of ``dest.path``.
|
||||
|
||||
If the above constraints are met, then more precise substitution via a pattern
|
||||
can be carried out.
|
||||
|
||||
Example
|
||||
^^^^^^^
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
---
|
||||
# Source document.
|
||||
schema: deckhand/Passphrase/v1
|
||||
metadata:
|
||||
name: example-password
|
||||
schema: metadata/Document/v1
|
||||
layeringDefinition:
|
||||
layer: site
|
||||
storagePolicy: cleartext
|
||||
data: my-secret-password
|
||||
---
|
||||
# Destination document.
|
||||
schema: armada/Chart/v1
|
||||
metadata:
|
||||
name: example-chart-01
|
||||
schema: metadata/Document/v1
|
||||
layeringDefinition:
|
||||
layer: region
|
||||
substitutions:
|
||||
- dest:
|
||||
path: .chart.values.some_url
|
||||
pattern: INSERT_[A-Z]+_HERE
|
||||
src:
|
||||
schema: deckhand/Passphrase/v1
|
||||
name: example-password
|
||||
path: .
|
||||
data:
|
||||
chart:
|
||||
details:
|
||||
data: here
|
||||
values:
|
||||
some_url: http://admin:INSERT_PASSWORD_HERE@service-name:8080/v1
|
||||
|
||||
After document rendering, the output for ``example-chart-01`` (the destination
|
||||
document) will be:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
---
|
||||
schema: armada/Chart/v1
|
||||
metadata:
|
||||
name: example-chart-01
|
||||
schema: metadata/Document/v1
|
||||
layeringDefinition:
|
||||
layer: region
|
||||
substitutions:
|
||||
- dest:
|
||||
path: .chart.values.some_url
|
||||
pattern: INSERT_[A-Z]+_HERE
|
||||
src:
|
||||
schema: deckhand/Passphrase/v1
|
||||
name: example-password
|
||||
path: .
|
||||
data:
|
||||
chart:
|
||||
details:
|
||||
data: here
|
||||
values:
|
||||
# Notice how the data from the source document is injected into the
|
||||
# exact location specified by ``dest.pattern``.
|
||||
some_url: http://admin:my-secret-password@service-name:8080/v1
|
||||
|
||||
Substitution of Encrypted Data
|
||||
------------------------------
|
||||
|
||||
Deckhand allows :ref:`data to be encrypted using Barbican <encryption>`.
|
||||
Substitution of encrypted data works the same as substitution of cleartext
|
||||
data.
|
||||
|
||||
Note that during the rendering process, source and destination documents
|
||||
receive the secrets stored in Barbican.
|
||||
|
|
Loading…
Reference in New Issue