Move UCP to Airship documentation

Updates the Shipyard documentation to reflect changes related
to moving to OpenStack Airship. Adds some additional information.

Change-Id: I32a3aa24ce268cbc925a512aa2ae54eaf95acd2a
This commit is contained in:
Bryan Strassner 2018-05-30 12:40:29 -05:00
parent b87569abe8
commit c1539ae95d
5 changed files with 79 additions and 145 deletions

View File

@ -28,8 +28,7 @@ functionality in Shipyard.
Standards used by the API
-------------------------
See `UCP API
conventions <https://github.com/att-comdev/ucp-integration/blob/master/docs>`__
See `API Conventions`_
Notes on examples
-----------------
@ -258,20 +257,19 @@ Entity Structure
^^^^^^^^^^^^^^^^
The response will be the list of validations from all downstream systems that
perform validation during the commit process. The structure will match the
error response object described in the `UCP API
conventions <https://github.com/att-comdev/ucp-integration/blob/master/docs>`__
and will be an aggregation of each UCP components responses.
error response object described in the `API Conventions`_ and will be an
aggregation of each validating components responses.
POST /v1.0/commitconfigdocs
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Synchronous. Performs the commit of the Shipyard Buffer to the Committed
Documents. This invokes each of the UCP components to examine the Shipyard
Buffer version of the configuration documents and aggregate the responses.
While performing this commit, further POSTing of configdocs, or other commits
may not be invoked (Shipyard will block those requests with a 409 response). If
there are any failures to validate, the Shipyard Buffer and Committed Documents
will remain unchanged. If successful, the Shipyard Buffer will be cleared, and
the Committed documents will be updated.
Documents. This invokes each of the validating components to examine the
Shipyard Buffer version of the configuration documents and aggregate the
responses. While performing this commit, further POSTing of configdocs, or
other commits may not be invoked (Shipyard will block those requests with a 409
response). If there are any failures to validate, the Shipyard Buffer and
Committed Documents will remain unchanged. If successful, the Shipyard Buffer
will be cleared, and the Committed documents will be updated.
.. note::
@ -1033,3 +1031,6 @@ Example
[2018-04-11 07:30:43,892] {{cli.py:374}} INFO - Running on host airflow-worker-0.airflow-worker-discovery.ucp.svc.cluster.local
[2018-04-11 07:30:43,945] {{base_task_runner.py:98}} INFO - Subtask: [2018-04-11 07:30:43,944] {{python_operator.py:90}} INFO - Done. Returned value was: None
[2018-04-11 07:30:43,992] {{base_task_runner.py:98}} INFO - Subtask: """)
.. _API Conventions: https://airshipit.readthedocs.io/en/latest/api-conventions.html

View File

@ -34,12 +34,9 @@ configuration documents. Steps, conceptually:
Prevents concurrent site modifications by conflicting
actions/workflows.
#. Preflight checks
Ensures all UCP components are in a responsive state.
#. Get design version
Uses Deckhand to discover the latest committed version of design for
the site.
Ensures all Airship components are in a responsive state.
#. Validate design
Asks each involved UCP component to validate the design. This ensures
Asks each involved Airship component to validate the design. This ensures
that the previously committed design is valid at the present time.
#. Drydock build
Orchestrates the Drydock component to configure hardware and the
@ -92,12 +89,11 @@ Schemas
DeploymentConfiguration_ schema - Provides for validation of the
deployment-configuration documents
Documents
~~~~~~~~~
deployment-configuration - Allows for specification of configurable options
used by the site deployment related workflows, including the timeouts used for
various steps, and the name of the armada manifest that will be used during the
deployment/update.
Deployment Configuration
~~~~~~~~~~~~~~~~~~~~~~~~
Allows for specification of configurable options used by the site deployment
related workflows, including the timeouts used for various steps, and the name
of the armada manifest that will be used during the deployment/update.
A `sample deployment-configuration`_ shows a completely specified example.
@ -114,8 +110,8 @@ Supported values:
- deployment_strategy:
Not currently used, placeholder for specifying how server deployments
should be grouped. (e.g. all-at-once, by-rack, by-label)
The name of the deployment strategy document to be used. There is a default
deployment strategy that is used if this field is not present.
- deploy_interval:
@ -201,6 +197,20 @@ Supported values:
The name of the Armada manifest document that the workflow will use during
site deployment activities. e.g.:'full-site'
.. _DeploymentConfiguration: https://github.com/att-comdev/shipyard/blob/master/shipyard_airflow/schemas/deploymentConfiguration.yaml
.. _`sample deployment-configuration`: https://github.com/att-comdev/shipyard/blob/master/tests/unit/yaml_samples/deploymentConfiguration_full_valid.yaml
.. _`Default configuration values`: https://github.com/att-comdev/shipyard/blob/a88a5cf15a81de32707fee49d7a626908a8c8f38/shipyard_airflow/plugins/deployment_configuration_operator.py#L47
Deployment Strategy
~~~~~~~~~~~~~~~~~~~
The deployment strategy document is optionally specified in the Deployment
Configuration and provides a way to group, sequence, and test the deployments
of groups of hosts deployed using `Drydock`_. The `deployment strategy design`_
provides details for the structures and usage of the deployment strategy.
A `sample deployment-strategy`_ shows one possible strategy, in the context of
the Shipyard unit testing.
The `DeploymentStrategy`_ schema is a more formal definition of this document.
.. _`Default configuration values`: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/shipyard_airflow/plugins/deployment_configuration_operator.py
.. _DeploymentConfiguration: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/shipyard_airflow/schemas/deploymentConfiguration.yaml
.. _DeploymentStrategy: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/shipyard_airflow/schemas/deploymentStrategy.yaml
.. _`deployment strategy design`: https://airshipit.readthedocs.io/en/latest/blueprints/deployment-grouping-baremetal.html
.. _Drydock: https://git.airshipit.org/cgit/airship-drydock
.. _`sample deployment-configuration`: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/tests/unit/yaml_samples/deploymentConfiguration_full_valid.yaml
.. _`sample deployment-strategy`: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/tests/unit/yaml_samples/deploymentStrategy_full_valid.yaml

View File

@ -97,8 +97,7 @@ Setup
Server Components
~~~~~~~~~~~~~~~~~
Use of the Shipyard client requires that a working installation of the Shipyard
API is available. This may be accomplished by using the basic-ucp_ setup.
Detailed instructions are not part of this guide.
API is available. See :ref:`shipyard_deployment_guide`
Local Environment
~~~~~~~~~~~~~~~~~
@ -153,6 +152,8 @@ response::
the target Shipyard API/Keystone environment.
- The ``shipyard`` and ``password`` values are the insecure values used by
default if not overridden by the installation of Shipyard.
- The dev_minimal manifest deployment from Airship-in-a-bottle referenced in
the deployment guide provides a set of credentials that can be used.
``Configure hosts file, if necessary``::
@ -196,9 +197,9 @@ The output will resemble the following::
Use Case: Ingest Site Design
----------------------------
Shipyard serves as the entrypoint for a deployment of the Undercloud Platform
(UCP). One can imagine the following activities representing part of the
lifecycle of a group of servers that the UCP would serve as the control plane:
Shipyard serves as the entrypoint for a deployment of Airship. One can imagine
the following activities representing part of the lifecycle of a group of
servers for which Airship would serve as the control plane:
Definition
A group of servers making up a ``site`` has been identified. Designs covering
@ -207,10 +208,10 @@ Definition
Preparation
The site is assembled, racking, and wiring is completed, and the hardware is
readied for operation. The ``Genesis Node`` is preinstalled with an
(Ubuntu 16.04) image. The Promenade_ genesis process similar to the
basic-ucp_ process is configured on and then run on the ``Genesis Node``.
(Ubuntu 16.04) image. Airship is deployed; See
:ref:`shipyard_deployment_guide`
At this point, the UCP is ready for use. This is the when the Shipyard API
At this point, Airship is ready for use. This is the when the Shipyard API
is available for use.
Load Configuration Documents
@ -218,7 +219,7 @@ Load Configuration Documents
with Shipyard, perhaps by using the CLI. The operator loads ``configdocs``
which are a product of the definition step. These ``configdocs`` are
declarative set of YAML documents using a format compatible with
Deckhand_ and containing information usable by the other UCP components.
Deckhand_ and containing information usable by the other Airship components.
The interaction with Shipyard could happen as follows::
@ -252,7 +253,7 @@ of an action::
$ shipyard commit configdocs
During this command, the other UCP components are contacted to validate the
During this command, the other Airship components are contacted to validate the
designs in Deckhand. If the validations are not successful, Shipyard will not
mark the revision as committed.
@ -271,7 +272,6 @@ mark the revision as committed.
site-specific collection that combines with the other two collections to
fully define the site.
Use Case: Deploy Site
---------------------
Continuing the lifecycle steps from the Ingest Site Design use case, the
@ -314,14 +314,14 @@ action, the operator can query the status and results of the action::
...
More information is returned than shown here - for sake of abbreviation. The
process of maintenance is very similar to the process of deploying a site.
process of maintenance (update_site) is very similar to the process of
deploying a site.
.. _api-client: https://github.com/att-comdev/shipyard/tree/master/shipyard_client/api_client
.. _Armada: https://github.com/att-comdev/armada
.. _basic-ucp: https://github.com/att-comdev/ucp-integration/tree/master/manifests/basic_ucp
.. _Deckhand: http://deckhand.readthedocs.io/en/latest/
.. _Divingbell: https://github.com/att-comdev/divingbell
.. _Drydock: https://github.com/att-comdev/drydock
.. _api-client: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_client
.. _Armada: https://git.airshipit.org/cgit/airship-armada
.. _Deckhand: https://git.airshipit.org/cgit/airship-deckhand
.. _Divingbell: https://git.airshipit.org/cgit/airship-divingbell
.. _Drydock: https://git.airshipit.org/cgit/airship-drydock
.. _Keystone: https://developer.openstack.org/api-ref/identity/index.html
.. _Promenade: https://github.com/att-comdev/promenade
.. _Promenade: https://git.airshipit.org/cgit/airship-promenade

View File

@ -23,109 +23,29 @@ Deployment Guide
Shipyard is still under active development and this guide will evolve along
the way
The current deployment makes use of the `ucp-integration`_ repository to set up
the underlaying Kubernetes infrastructure, Ceph and UCP components. This approach
sets up an 'All-In-One' UCP environment that allows developers to bring up Shipyard
and the rest of the UCP components on a single Ubuntu 16.04 Virtual Machine.
Deployment
----------
.. note::
The minimum recommended size of the VM is 4 vCPUs, 16GB of RAM with 64GB disk space
The current deployment makes use of the `airship-in-a-bottle`_ project to
set up the underlaying Kubernetes infrastructure, container networking
(Calico), disk provisioner (Ceph or NFS), and Airship components that are used
by Shipyard.
The `dev_minimal`_ manifest is the recommended manifest. Please see the
README.txt that exists in that manifest's directory.
Pre-Deployment Preparations
---------------------------
#. Set up ``etc/hosts`` on a freshly installed Ubuntu 16.04 Virtual Machine with the
IP and hostname of the VM::
HOST_IFACE=$(ip route | grep "^default" | head -1 | awk '{ print $5 }')
LOCAL_IP=$(ip addr | awk "/inet/ && /${HOST_IFACE}/{sub(/\/.*$/,\"\",\$2); print \$2}")
cat << EOF | sudo tee -a /etc/hosts
${LOCAL_IP} $(hostname)
EOF
#. Clone the `ucp-integration`_ repository
Production Deployment
---------------------
We will use this approach if we are not making any changes to Shipyard and would like to bring up
the UCP environment as it is
#. Switch to root user after performing the steps in the *Pre-Deployment Preparations* section::
sudo -i
cd /home/ubuntu/ucp-integration/manifests/basic_ucp/
#. Export the variables that are unique to the environment. For instance, we can do the following
to update the environment variables for a VM with hostname **node1** that is assigned an IP of
**30.30.30.4** on interface **ens3** (network **30.30.30.0/24**)::
export GENESIS_NODE_NAME=node1
export CEPH_CLUSTER_NET=30.30.30.0/24
export CEPH_PUBLIC_NET=30.30.30.0/24
export GENESIS_NODE_IP=30.30.30.4
export NODE_NET_IFACE=ens3
#. Start the UCP deployment::
./deploy_ucp.sh
Dev Environment Deployment
--------------------------
We will use this approach if we want to bring up a dev environment that allows us to test our own
dags and operators in Airflow. Changes to Shipyard API/CLI will require a rebuild of the Shipyard
images with the updates. We will need to reference to the custom image in our environment variables
so that it does not point to the image in the Master branch.
#. Create the following directories on the target host machine::
mkdir -p /var/tmp/airflow/dags
mkdir -p /var/tmp/airflow/plugins
mkdir -p /var/tmp/airflow/logs
#. Copy the `rest_api_plugin`_ into the newly created plugins directory, i.e. ``/var/tmp/airflow/plugins``
so that it can be loaded by Airflow during startup. **Note** that other custom operators
should be added to the directory as required.
**Note** that custom dags should be added into the newly created dags directory, i.e. ``/var/tmp/airflow/dags``
#. Switch to root user after performing Step 1 and 2::
sudo -i
cd /home/ubuntu/ucp-integration/manifests/basic_ucp/
#. Export the variables that are unique to the environment. For instance, we can do the below
to update the environment variables for a VM with hostname **node1** that is assigned an IP
of **30.30.30.4** on interface **ens3** (network **30.30.30.0/24**).
Update image references in the environment variables if we want to test a new Shipyard and
Airflow image, e.g. image v0.1.0 as a result of code changes::
export SHIPYARD_PROD_DEPLOY=false
export GENESIS_NODE_NAME=node1
export CEPH_CLUSTER_NET=30.30.30.0/24
export CEPH_PUBLIC_NET=30.30.30.0/24
export GENESIS_NODE_IP=30.30.30.4
export NODE_NET_IFACE=ens3
export SHIPYARD_IMAGE="attcomdev/shipyard:v0.1.0"
export AIRFLOW_IMAGE="attcomdev/airflow:v0.1.0"
#. Start the UCP deployment::
./deploy_ucp.sh
This approach sets up an 'All-In-One' Airship environment that allows
developers to bring up Shipyard and the rest of the Airship components on a
single Ubuntu Virtual Machine.
The deployment is fully automated and can take a while to complete (it can take
30 minutes to an hour for a full deployment to complete)
Post Deployment
---------------
#. The deployment is fully automated and can take a while to complete (it can take 30 minutes
to an hour for a full deployment to complete)
#. The environment should resemble the following after executing the required steps::
#. The environment should resemble the following after executing the required
steps::
# sudo kubectl get pods -n ucp
NAME READY STATUS RESTARTS AGE
@ -182,5 +102,6 @@ Post Deployment
ucp-ucp-memcached 1 Fri Dec 1 10:02:44 2017 DEPLOYED memcached-0.1.0 ucp
.. _ucp-integration: https://github.com/att-comdev/ucp-integration
.. _rest_api_plugin: https://github.com/att-comdev/shipyard/blob/master/shipyard_airflow/plugins/rest_api_plugin.py
.. _airship-in-a-bottle: https://git.airshipit.org/cgit/airship-in-a-bottle
.. _dev_minimal: https://git.airshipit.org/cgit/airship-in-a-bottle/tree/manifests/dev_minimal
.. _rest_api_plugin: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/shipyard_airflow/plugins/rest_api_plugin.py

View File

@ -19,7 +19,7 @@ Welcome to Shipyard's documentation!
====================================
Shipyard is a directed acyclic graph controller for Kubernetes and OpenStack
control plane life-cycle management.
control plane life-cycle management, and is part of the `Airship`_ platform.
User's Guide
============
@ -37,3 +37,5 @@ Shipyard Configuration Guide
CLI
client_user_guide
deployment_guide
.. _Airship: https://airshipit.org