From 64efc26da94ceaa74cb84d4300f7591fa3df4d6a Mon Sep 17 00:00:00 2001 From: Liam Young Date: Thu, 7 Jun 2018 13:39:08 +0100 Subject: [PATCH] Add Certificate Lifecycle Management Appendix Add Certificate Lifecycle Management Appendix. Since setting up Vault is common between Encryption at Rest and Certificate Lifecycle management, it has been moved to its own appendix which can be referenced by EaR and CLM. Change-Id: I867f938fc9e836334469d4fc2eb102fd32bb69e6 --- .../source/app-certificate-management.rst | 146 ++++++++++++++ .../source/app-encryption-at-rest.rst | 175 +---------------- deploy-guide/source/app-vault.rst | 185 ++++++++++++++++++ deploy-guide/source/app.rst | 2 + 4 files changed, 335 insertions(+), 173 deletions(-) create mode 100644 deploy-guide/source/app-certificate-management.rst create mode 100644 deploy-guide/source/app-vault.rst diff --git a/deploy-guide/source/app-certificate-management.rst b/deploy-guide/source/app-certificate-management.rst new file mode 100644 index 0000000..956049f --- /dev/null +++ b/deploy-guide/source/app-certificate-management.rst @@ -0,0 +1,146 @@ +Appendix E: Certificate Lifecycle Management +============================================ + +Overview +++++++++ + +As of the 18.05 release, the OpenStack charms preview using Vault for the +provisioning of TLS certificates. Currently, the only supported workflow is for +Vault to generate a certificate signing request for an intermediate +certificate authority. This csr then needs to be signed by an external ca, the +signed certificate is then uploaded to Vault along with the root certificate. + +Vault ++++++ + +See `Appendix C Vault <./app-vault.html>`__ + + +Enabling Vault Certificate Management ++++++++++++++++++++++++++++++++++++++ + +OpenStack charms providing an API service have a new 'certificates' relation. +Adding this relation will trigger the OpenStack charm to request +certificates and keys from vault. Once vault has provided these the charm +will install them and switch to listening on https, the catalog will also be +updated. + +.. code:: bash + + juju add-relation keystone:certificates vault:certificates + juju add-relation glance:certificates vault:certificates + juju add-relation cinder:certificates vault:certificates + juju add-relation nova-cloud-controller:certificates vault:certificates + juju add-relation neutron-api:certificates vault:certificates + ... + + +Retrieve csr from vault +~~~~~~~~~~~~~~~~~~~~~~~ + +Run the *get-csr* action against the lead unit of the vault application: + +.. code:: bash + + $ juju run-action vault/0 get-csr + Action queued with id: 0495b6ce-09d8-4e57-8d21-efa13794034a + $ juju show-action-output 0495b6ce-09d8-4e57-8d21-efa13794034a + results: + output: |- + -----BEGIN CERTIFICATE REQUEST----- + MIICijCCAXICAQAwRTFDMEEGA1UEAxM6VmF1bHQgSW50ZXJtZWRpYXRlIENlcnRp + ZmljYXRlIEF1dGhvcml0eSAoY2hhcm0tcGtpLWxvY2FsKTCCASIwDQYJKoZIhvcN + AQEBBQADggEPADCCAQoCggEBAKEkTFyX2SgzDDUMvJNnoptdAb9AAiDZzc3U/aaX + g55TmAQ5jIfYbXb/39O+81iWD8esWGzTkg9YyzfPUBwcF3FrsyyEPFjiFRhTEATl + 6W2U5tA981hKiEScF2BvMm4PJZM/1ND12yoIsg45n1AGUfY8GShqtKKNXvAR3TEj + mYWQ35QhVFnsvh9hc5TUORRwlKy74FnCBDfuWuIGph/Ge4GRctAv0MUUsfqCv9DZ + +xMHeE8IdyqMakTJ2iUXktbl0khSwbb5KUT9NOzC1mDPZPBICqSmhXQ1In9FR5ic + VTsVOdHShaN4T4qjrvWI4+S5CJynDnSvDeUWPYu1lOM24H0CAwEAAaAAMA0GCSqG + SIb3DQEBCwUAA4IBAQCHZoVqKJ0TCaYimHT2VGqklomGuQymxqkKIrBHQD4oe/Ts + 6nXXZaUWVPQF96o+Au4oz2R/l/UF4a89Z2xyCN95tys/QJv0Hw93MeN7cIGUZ8p+ + nOpb7IX5ZqWEKu1g+2cTQ0mxa1hGb+rbO0LtArbhWVQjrkqKdY26O78Ct76eutfO + 1uKYXMbbUlXH8Qb/MyfRt30nPqyLKz4bIqH0fm670s/MiK8zoJ8DwI3NpkzB0cfG + qzYxRu078uWcenueDBa8HqvBQNT6meZ68y+eesXrIM4EY3dk4YSb5A5QSeqUAWjK + Fp36v30jKNJ29rs8vduytRu6bz7bI+qZ1hCJbzQQ + -----END CERTIFICATE REQUEST----- + status: completed + timing: + completed: 2018-06-07 10:21:17 +0000 UTC + enqueued: 2018-06-07 10:21:13 +0000 UTC + started: 2018-06-07 10:21:13 +0000 UTC + + +Retrieve the csr from the action output and place it in a file, removing any +leading whitespace. + +Sign csr +~~~~~~~~ + +The exact command from signing the csr will depend on the setup of the +external ca. Below is an example: + +.. code:: bash + + openssl ca -config openssl.cnf -extensions v3_intermediate_ca -days 3650 \ + -notext -md sha256 -in csr_file -out /tmp/vault-charm-int.pem -batch \ + -passin pass:secretpassword + +*If the singing is rejected due to mismatched O or OU or C etc then rerun the +get-csr actions and specify the mismatched items* + +Upload signed csr and root ca cert to vault +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +(Where /tmp/root-ca.pem is the root ca cert) + +.. code:: bash + + juju run-action vault/0 upload-signed-csr \ + pem="$(cat /tmp/vault-charm-int.pem | base64)" \ + root-ca="$(cat /tmp/root-ca.pem | base64)" \ + allowed-domains='openstack.local' + +Vault issues certificates +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Vault will now issue certificates to all clients that have requested them. +This process will trigger the api charms to request endpoint updates from +keystone to reflect that they are now using https. This can be a lengthy +process, so monitor keystone units and wait for them to become idle. + +.. code:: bash + + + watch -d juju status keystone + +Test +~~~~ + +Where /tmp/root-ca.pem is the root ca cert: + +.. code:: bash + + source novarc # make sure you have https in OS_AUTH_URL + + echo "Testing: keystone" + openstack --os-cacert /tmp/root-ca.pem catalog list + echo "Testing: nova-cloud-controller" + openstack --os-cacert /tmp/root-ca.pem server list + echo "Testing: cinder" + openstack --os-cacert /tmp/root-ca.pem volume list + echo "Testing: neutron" + openstack --os-cacert /tmp/root-ca.pem network list + echo "Testing: image" + openstack --os-cacert /tmp/root-ca.pem image list + deactivate + +Reissuing certificates +~~~~~~~~~~~~~~~~~~~~~~ + +The vault charm has an *reissue-certificates* action. Running the action +will cause vault to issue new certificates for all charm clients. The action +must be run on the lead unit. + +.. code:: bash + + juju run-action vault/0 reissue-certificates diff --git a/deploy-guide/source/app-encryption-at-rest.rst b/deploy-guide/source/app-encryption-at-rest.rst index 9b2bdb9..6a93322 100644 --- a/deploy-guide/source/app-encryption-at-rest.rst +++ b/deploy-guide/source/app-encryption-at-rest.rst @@ -1,4 +1,4 @@ -Appendix C: Encryption at Rest +Appendix D: Encryption at Rest ============================== Overview @@ -22,178 +22,7 @@ event that disks or full servers are removed from data center deployments. Vault +++++ -Deploy vault -~~~~~~~~~~~~ - -First deploy the vault charm along with supporting services: - -.. code:: bash - - juju deploy --to lxd:0 vault - juju add-relation vault percona-cluster - -.. note:: - - When running on hardware or in KVM, the vault charm will configure - vault to enable mlock (memory locking) which ensures that vault - won't get swapped out to disk, potentially compromising secrets. - Using mlock is not supported in LXD containers and will be - automatically disabled. - -Vault can make use of the existing Percona XtraDB Cluster deployed to -support the rest of the OpenStack applications or could be deployed -with a separate instance if required. All data stored is encrypted by -vault using its master encryption key. - -Vault will deploy and startup in an un-initialized state; For production -deployments the unseal keys used to manage the Vault master key used for -encryption should be managed from outside of the Juju model hosting vault and -the OpenStack Charms. - -.. note:: - - Production deployments you should also secure vault using the ssl-* - configuration options provided by the charm; this ensures that communication - with the Vault REST API is always secure as data on the network is also - encrypted using TLS encryption. - -Initialize and unseal vault -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Install the vault snap and configure the VAULT_ADDR environment variable: - -.. code:: bash - - sudo snap install vault - export VAULT_ADDR="https://:8200" - -and then initialize the vault deployment: - -.. code:: bash - - vault operator init -key-shares=5 -key-threshold=3 - -you should get a response like: - -.. code:: bash - - Unseal Key 1: XqeOza3SY6f4L6xfuk6f8JumrEF7cak9mUXCCPRXzs4B - Unseal Key 2: djvVAAste0F5iSe43nmBs2ZX5r+wUqHe4UfUrcprWkyM - Unseal Key 3: iSXHBdTNIKrbd3JIEI+n+q7j04Q4HPsQOHgk7apupttT - Unseal Key 4: J8jzdUi0HOWgx8Ig75Mu8uVQTXkw6yNv2kMhjeNz2/5B - Unseal Key 5: xTJkk3guA9Mq3CYfDhIBevSWrD1CIer6q70HVACMbduc - - Initial Root Token: ebded15e-c908-5d3a-1df0-1e7e7218c162 - - Vault initialized with 5 key shares and a key threshold of 3. Please securely - distribute the key shares printed above. When the Vault is re-sealed, - restarted, or stopped, you must supply at least 3 of these keys to unseal it - before it can start servicing requests. - - Vault does not store the generated master key. Without at least 3 key to - reconstruct the master key, Vault will remain permanently sealed! - - It is possible to generate new unseal keys, provided you have a quorum of - existing unseal keys shares. See "vault rekey" for more information. - -This response details the 5 keys that can be used to unseal the vault deployment -and an initial root token for accessing the Vault API. - -.. warning:: - - Do not lose the unseal keys! It's impossible to unseal - vault without them which must be completed after any restart - of the vault daemon as part of ongoing maintenance. - -.. warning:: - - Do not lose the root token! Without it the vault deployment will - be inaccessible. - -you can then unseal the vault deployment for operation: - -.. code:: bash - - vault operator unseal XqeOza3SY6f4L6xfuk6f8JumrEF7cak9mUXCCPRXzs4B - vault operator unseal djvVAAste0F5iSe43nmBs2ZX5r+wUqHe4UfUrcprWkyM - vault operator unseal iSXHBdTNIKrbd3JIEI+n+q7j04Q4HPsQOHgk7apupttT - - -Authorize vault charm -~~~~~~~~~~~~~~~~~~~~~ - -vault is now ready for use - however the charm needs to be authorized -using a root token to be able to create secrets storage back-ends and -roles to allow other applications to access vault for encryption key -storage. - -First generate a one-shot root token with a limited TTL using the -initial root token for this purpose: - -.. code:: bash - - export VAULT_TOKEN=ebded15e-c908-5d3a-1df0-1e7e7218c162 - vault token create -use-limit=1 -ttl=10m - -you should get a response like: - -.. code:: bash - - Key Value - --- ----- - token 03ceadf5-529d-6a64-0cfd-1e341b1dacb1 - token_accessor 17390537-2012-51dc-93d0-9cc26ba953eb - token_duration 10m - token_renewable true - token_policies [root] - -This token can then be used to setup access for the charm to -Vault: - -.. code:: bash - - juju run-action vault/0 authorize-charm token=03ceadf5-529d-6a64-0cfd-1e341b1dacb1 - -After the action completes execution, the vault unit will go active -and any pending requests for secrets storage will be processed for -consuming applications. - -Enabling HA -~~~~~~~~~~~ - -The vault charm supports deployment in HA configurations; this requires -the use of etcd to provide HA storage to vault units, with access to -vault being provided a virtual IP or DNS-HA hostname: - -.. code:: bash - - juju deploy --to lxd:0 vault - juju add-unit --to lxd:1 vault - juju add-unit --to lxd:2 vault - juju config vault vip=10.20.30.1 - juju deploy hacluster vault-hacluster - juju add-relation vault vault-hacluster - - juju deploy --to lxd:0 etcd - juju add-unit --to lxd:1 etcd - juju add-unit --to lxd:2 etcd - - juju deploy --to lxd:0 easyrsa # required for TLS certs for etcd - - juju add-relation etcd easyrsa - juju add-relation etcd vault - juju add-relation vault percona-cluster - -Only a single vault unit is 'active' at any point in time (reflected in juju -status output). Other vault units will proxy incoming API requests to the -active vault unit over a secure cluster connection between units. - -.. note:: - - When deploying vault in HA configurations, all vault units must be - unsealed using the unseal keys generated during initialization - in order to unlock the master key. This is performed externally - to the charm using the Vault API. +See `Appendix C Vault <./app-vault.html>`__ Enabling Encryption +++++++++++++++++++ diff --git a/deploy-guide/source/app-vault.rst b/deploy-guide/source/app-vault.rst new file mode 100644 index 0000000..d655f16 --- /dev/null +++ b/deploy-guide/source/app-vault.rst @@ -0,0 +1,185 @@ +Appendix C: Vault +============================== + +Overview +++++++++ + +Vault provides a secure storage and certificate management service. +Integrating vault into an OpenStack deployment involves a few +post-deployment steps which have been encapsulated in charm actions. + +Vault ++++++ + +Deploy vault +~~~~~~~~~~~~ + +First deploy the vault charm along with supporting services: + +.. code:: bash + + juju deploy --to lxd:0 vault + juju add-relation vault percona-cluster + +.. note:: + + When running on hardware or in KVM, the vault charm will configure + vault to enable mlock (memory locking) which ensures that vault + won't get swapped out to disk, potentially compromising secrets. + Using mlock is not supported in LXD containers and will be + automatically disabled. + +Vault can make use of the existing Percona XtraDB Cluster deployed to +support the rest of the OpenStack applications or could be deployed +with a separate instance if required. All data stored is encrypted by +vault using its master encryption key. + +Vault will deploy and startup in an un-initialized state; For production +deployments the unseal keys used to manage the Vault master key used for +encryption should be managed from outside of the Juju model hosting vault and +the OpenStack Charms. + +.. note:: + + Production deployments you should also secure vault using the ssl-* + configuration options provided by the charm; this ensures that communication + with the Vault REST API is always secure as data on the network is also + encrypted using TLS encryption. + +Initialize and unseal vault +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Install the vault snap and configure the VAULT_ADDR environment variable: + +.. code:: bash + + sudo snap install vault + export VAULT_ADDR="https://:8200" + +and then initialize the vault deployment: + +.. code:: bash + + vault operator init -key-shares=5 -key-threshold=3 + +you should get a response like: + +.. code:: bash + + Unseal Key 1: XqeOza3SY6f4L6xfuk6f8JumrEF7cak9mUXCCPRXzs4B + Unseal Key 2: djvVAAste0F5iSe43nmBs2ZX5r+wUqHe4UfUrcprWkyM + Unseal Key 3: iSXHBdTNIKrbd3JIEI+n+q7j04Q4HPsQOHgk7apupttT + Unseal Key 4: J8jzdUi0HOWgx8Ig75Mu8uVQTXkw6yNv2kMhjeNz2/5B + Unseal Key 5: xTJkk3guA9Mq3CYfDhIBevSWrD1CIer6q70HVACMbduc + + Initial Root Token: ebded15e-c908-5d3a-1df0-1e7e7218c162 + + Vault initialized with 5 key shares and a key threshold of 3. Please securely + distribute the key shares printed above. When the Vault is re-sealed, + restarted, or stopped, you must supply at least 3 of these keys to unseal it + before it can start servicing requests. + + Vault does not store the generated master key. Without at least 3 key to + reconstruct the master key, Vault will remain permanently sealed! + + It is possible to generate new unseal keys, provided you have a quorum of + existing unseal keys shares. See "vault rekey" for more information. + +This response details the 5 keys that can be used to unseal the vault deployment +and an initial root token for accessing the Vault API. + +.. warning:: + + Do not lose the unseal keys! It's impossible to unseal + vault without them which must be completed after any restart + of the vault daemon as part of ongoing maintenance. + +.. warning:: + + Do not lose the root token! Without it the vault deployment will + be inaccessible. + +you can then unseal the vault deployment for operation: + +.. code:: bash + + vault operator unseal XqeOza3SY6f4L6xfuk6f8JumrEF7cak9mUXCCPRXzs4B + vault operator unseal djvVAAste0F5iSe43nmBs2ZX5r+wUqHe4UfUrcprWkyM + vault operator unseal iSXHBdTNIKrbd3JIEI+n+q7j04Q4HPsQOHgk7apupttT + + +Authorize vault charm +~~~~~~~~~~~~~~~~~~~~~ + +vault is now ready for use - however the charm needs to be authorized +using a root token to be able to create secrets storage back-ends and +roles to allow other applications to access vault for encryption key +storage. + +First generate a one-shot root token with a limited TTL using the +initial root token for this purpose: + +.. code:: bash + + export VAULT_TOKEN=ebded15e-c908-5d3a-1df0-1e7e7218c162 + vault token create -use-limit=1 -ttl=10m + +you should get a response like: + +.. code:: bash + + Key Value + --- ----- + token 03ceadf5-529d-6a64-0cfd-1e341b1dacb1 + token_accessor 17390537-2012-51dc-93d0-9cc26ba953eb + token_duration 10m + token_renewable true + token_policies [root] + +This token can then be used to setup access for the charm to +Vault: + +.. code:: bash + + juju run-action vault/0 authorize-charm token=03ceadf5-529d-6a64-0cfd-1e341b1dacb1 + +After the action completes execution, the vault unit will go active +and any pending requests for secrets storage will be processed for +consuming applications. + +Enabling HA +~~~~~~~~~~~ + +The vault charm supports deployment in HA configurations; this requires +the use of etcd to provide HA storage to vault units, with access to +vault being provided a virtual IP or DNS-HA hostname: + +.. code:: bash + + juju deploy --to lxd:0 vault + juju add-unit --to lxd:1 vault + juju add-unit --to lxd:2 vault + juju config vault vip=10.20.30.1 + juju deploy hacluster vault-hacluster + juju add-relation vault vault-hacluster + + juju deploy --to lxd:0 etcd + juju add-unit --to lxd:1 etcd + juju add-unit --to lxd:2 etcd + + juju deploy --to lxd:0 easyrsa # required for TLS certs for etcd + + juju add-relation etcd easyrsa + juju add-relation etcd vault + juju add-relation vault percona-cluster + +Only a single vault unit is 'active' at any point in time (reflected in juju +status output). Other vault units will proxy incoming API requests to the +active vault unit over a secure cluster connection between units. + +.. note:: + + When deploying vault in HA configurations, all vault units must be + unsealed using the unseal keys generated during initialization + in order to unlock the master key. This is performed externally + to the charm using the Vault API. diff --git a/deploy-guide/source/app.rst b/deploy-guide/source/app.rst index c2cd079..fa82371 100644 --- a/deploy-guide/source/app.rst +++ b/deploy-guide/source/app.rst @@ -7,4 +7,6 @@ Appendices app-ceph-migration.rst app-upgrade-openstack.rst + app-vault.rst app-encryption-at-rest.rst + app-certificate-management.rst