Merge branch 'master' into docs_handler_ansible
This commit is contained in:
Jedrzej Nowak
commit
cc98cc4213
|
|
@ -14,6 +14,7 @@ Contents:
|
|||
source/glossary
|
||||
source/resource
|
||||
source/orchestration
|
||||
source/transports
|
||||
source/handler_ansible
|
||||
source/examples
|
||||
source/deployment_plan
|
||||
|
|
|
|||
|
|
@ -3,8 +3,9 @@
|
|||
Preparing deployment plan
|
||||
=========================
|
||||
|
||||
Solar allows to make transitions between different versions of infrastructure
|
||||
based on changes found by solar control plane and configured events between this changes.
|
||||
Solar allows you to make transitions between different versions of
|
||||
infrastructure based on changes found by solar control plane and events
|
||||
configured between this changes.
|
||||
|
||||
Required information
|
||||
----------------------
|
||||
|
|
@ -27,11 +28,11 @@ user (human/program). Examples of changes are:
|
|||
Staged changes
|
||||
---------------
|
||||
|
||||
On demand solar runs procedure that will find all changed from last deployment
|
||||
resources, and using transitions from solar state machine will determine
|
||||
list of actions.
|
||||
On demand solar runs procedure that will find all resources changed from last
|
||||
deployment and will determine list of actions using transitions from solar
|
||||
state machine .
|
||||
|
||||
This procedure performed by ::
|
||||
This procedure is performed by ::
|
||||
|
||||
solar changes stage -d
|
||||
|
||||
|
|
@ -58,7 +59,7 @@ It prints information like ::
|
|||
++ collation: utf8_general_ci
|
||||
++ location_id: 694b35afa622da857f95e14a21599d81
|
||||
|
||||
At this point information is stored as a list, and doensnt know anything
|
||||
At this point information is stored as a list, and user doesn't know anything
|
||||
about dependencies between found changes.
|
||||
|
||||
Events usage
|
||||
|
|
@ -66,21 +67,22 @@ Events usage
|
|||
|
||||
For events definition see :ref:`res-event-term`.
|
||||
|
||||
Events used during this procedure to insert dependencies between found changes,
|
||||
and add new actions that are reactions for changes.
|
||||
Events are used during this procedure to insert dependencies between found
|
||||
changes, and to add new actions that are reactions for changes.
|
||||
|
||||
Example of dependency between changes would be *nova service* that depends
|
||||
on successful creation of *database*.
|
||||
|
||||
And for removal we might add dependencies that will allow reverse
|
||||
order, e.g when removing *nova service* and *database*, *database* will be
|
||||
removed only after successfull *nova service*.
|
||||
Can be specified as ::
|
||||
For removal we might add dependencies that will allow reverse order, e.g. when
|
||||
removing *nova service* and *database*, *database* will be removed only after
|
||||
successful *nova service* removal.
|
||||
|
||||
This can be specified as ::
|
||||
|
||||
Dependency database1.run -> nova1.run
|
||||
Dependency nova1.remove -> database1.remove
|
||||
|
||||
Reaction allows to construct more advacned worklows that will take into
|
||||
Reaction allows to construct more advanced workflows that will take into
|
||||
account not only changes, but also arbitrary actions for resources in solar.
|
||||
|
||||
Good example of usage is provisioning procedure, where reboot must be
|
||||
|
|
@ -100,9 +102,9 @@ of deployment plan for current version of infrastructure ::
|
|||
|
||||
solar changes process
|
||||
|
||||
After this command plan can be viewed by ::
|
||||
After this deployment command plan can be viewed by ::
|
||||
|
||||
# graphviz representataion
|
||||
# graphviz representation
|
||||
solar orch dg last
|
||||
|
||||
# text report
|
||||
|
|
|
|||
|
|
@ -6,29 +6,30 @@ Examples
|
|||
Create resource for the puppet handler
|
||||
--------------------------------------
|
||||
|
||||
Let's create an example :ref:`resource-term` for the puppet :ref:`res-handler-term`
|
||||
version 1 [#]_. The resource should install and configure OpenStack Nova API
|
||||
service.
|
||||
Let's create an example :ref:`resource-term` for the puppet
|
||||
:ref:`res-handler-term` version 1 [#]_. The resource should install and
|
||||
configure OpenStack Nova API service.
|
||||
|
||||
.. [#] There is also puppet handler version 2 but it is out of the scope
|
||||
of this example.
|
||||
|
||||
Step 1: Find an apropriate puppet module
|
||||
Step 1: Find an appropriate puppet module
|
||||
++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
The `Puppet OpenStack <https://wiki.openstack.org/wiki/Puppet>`_
|
||||
module for `Nova <https://github.com/openstack/puppet-nova>`_
|
||||
provides all of the required functionality.
|
||||
|
||||
Step 2: Define granularity scope for a resource
|
||||
Step 2: Define granularity level for a resource
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
One may want to implement resources as atomic entities doing each its only
|
||||
task, like running one and only puppet manifest [#]_. Or as a single entity doing
|
||||
all required tasks instead. In order to configure and run the Nova API service
|
||||
at least two manifests should be executed: the
|
||||
One may want to implement resources as atomic entities doing their only single
|
||||
task, like running one and only puppet manifest [#]_. Other option might be
|
||||
single entity doing all required tasks instead. In order to configure and run
|
||||
the Nova API service at least two manifests should be executed:
|
||||
`init.pp <https://github.com/openstack/puppet-nova/blob/master/manifests/init.pp>`_
|
||||
and the `api.pp <https://github.com/openstack/puppet-nova/blob/master/manifests/api.pp>`_ [#]_.
|
||||
and
|
||||
`api.pp <https://github.com/openstack/puppet-nova/blob/master/manifests/api.pp>`_ [#]_.
|
||||
|
||||
.. [#] Puppet manifests may contain references to externally defined classess
|
||||
or services in the catalog. Keep that in mind then designing the resource.
|
||||
|
|
@ -40,26 +41,26 @@ Assuming the atomic tasks approach, the example resource for Nova API service
|
|||
should only use the `api.pp` manifest. Note that the puppet handler is normally
|
||||
executed in its own isolated puppet catalog with its specific hiera data only.
|
||||
This assumes every puppet manifest called by every action to be executed as a
|
||||
separate puppet run and shares nothing.
|
||||
separate puppet run and shares nothing with other tasks.
|
||||
|
||||
Step 3: Define resource inputs
|
||||
++++++++++++++++++++++++++++++
|
||||
|
||||
Once the granularity scope of the resource was clearly defined, one should
|
||||
Once the granularity level of the resource is clearly defined, one should
|
||||
define the resource's :ref:`res-input-term` data. The puppet class `nova::api`
|
||||
contains a lots of parameters. It looks reasonable to use them as the resource
|
||||
contains lots of parameters. It looks reasonable to use them as the resource
|
||||
inputs as is.
|
||||
|
||||
.. note ::
|
||||
There is a `helper script <https://github.com/bogdando/convert_puppet_parameters>`_
|
||||
to convert a puppet class parameters into the format expeted by the
|
||||
to convert a puppet class parameters into the format expected by the
|
||||
`meta.yaml` inputs file.
|
||||
|
||||
Step 4: Implement the basic action run
|
||||
Step 4: Implement basic action run
|
||||
++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
Each resource should have all of the mandatory actions defined. In this example
|
||||
we define only the :ref:`ref-action-term` `run`. For the example Nova API
|
||||
we define only the :ref:`ref-action-term` `run`. With the example of Nova API
|
||||
resource, the action run should:
|
||||
|
||||
- fetch the resource inputs from the hiera [#]_ ::
|
||||
|
|
@ -86,7 +87,7 @@ resource, the action run should:
|
|||
}
|
||||
|
||||
.. note ::
|
||||
Otherwise, the called class would assume the package and exec are
|
||||
Otherwise, called class would assume the package and exec are
|
||||
already included in the catalog by the `init.pp`. And would fail as
|
||||
there is no `class { 'nova': }` call expected for the Nova API resource
|
||||
action run.
|
||||
|
|
@ -101,7 +102,7 @@ Step 5: Think of the rest of the resource actions
|
|||
One should also design other actions for the resource. Mandatory are only
|
||||
`run`, `update` and `remove`. There might be additional ones like `on-fail`,
|
||||
`on-retry` or whichever are actually required to implement expected behavior.
|
||||
For the given API resource there is none specific actions expected and there
|
||||
For the given API resource there are no specific actions expected and there
|
||||
is nothing to do for the action remove. For the action update, it is likely
|
||||
the same steps should be done as for the action run.
|
||||
|
||||
|
|
@ -116,7 +117,7 @@ Step 7: Think of the deployment composition
|
|||
|
||||
The deployment composition is which resources should be used and in which order
|
||||
it should be executed to achive the expected result, which is a successfull
|
||||
:ref:`deploy-plan-term`. For the given example, the composition may be as the
|
||||
:ref:`deploy-plan-term`. For the given example, the composition may be as
|
||||
following:
|
||||
|
||||
- Install and configure MySQL DB [#]_
|
||||
|
|
|
|||
|
|
@ -9,28 +9,33 @@ Solar Glossary
|
|||
Resource
|
||||
========
|
||||
|
||||
You can learn more about it in :ref:`Resource Details <resource_details>`
|
||||
Resource is an abstraction of item in system managed by Solar. It's a basic
|
||||
building block used to assemble your system. Almost every entity in Solar
|
||||
is a resource.
|
||||
|
||||
You can learn more about it in :ref:`resource details <resource_details>`
|
||||
|
||||
.. _res-input-term:
|
||||
|
||||
Input
|
||||
-----
|
||||
Resource configuration that will be used in actions, handlers and orchestration.
|
||||
All known inputs should be provided in meta.yaml
|
||||
Resource configuration that will be used in actions, handlers and
|
||||
orchestration. All known inputs for a resource should be defined in meta.yaml
|
||||
|
||||
.. _res-connection-term:
|
||||
|
||||
Connection
|
||||
----------
|
||||
Allows to build hierarchy between inputs of several resources,
|
||||
parent value will be always used in child while connection is created.
|
||||
If connection will be removed - original value of child will be preserved.
|
||||
Allows to build hierarchy between inputs of several resources, parent value
|
||||
will be always used in child while connection is created. If connection is
|
||||
removed - original value of child will be preserved.
|
||||
|
||||
.. _res-action-term:
|
||||
|
||||
Action
|
||||
------
|
||||
Solar wraps deployment code into actions with specific names.
|
||||
Solar wraps deployment code into actions with specific names. Actions are
|
||||
executed from the resource.
|
||||
|
||||
.. _res-tag-term:
|
||||
|
||||
|
|
@ -44,20 +49,20 @@ used for different user operations.
|
|||
Handler
|
||||
=======
|
||||
|
||||
Layer that responsible for action execution and tracking result.
|
||||
Layer responsible for action execution and tracking results.
|
||||
|
||||
.. _res-transports-term:
|
||||
|
||||
Transport
|
||||
=========
|
||||
|
||||
Used in handlers to communicate with managed by solar hosts. List of transports
|
||||
should be added to a node. Transports will be added to a resource by means
|
||||
of transports id.
|
||||
Used in handlers to communicate with hosts managed by Solar.
|
||||
|
||||
Two different types of transports are used: run and sync.
|
||||
Run transport - reponsible for running command on remote host.
|
||||
Sync transport - uploads required information.
|
||||
.. seealso::
|
||||
|
||||
:ref:`More details about transports <transports_details>`
|
||||
|
||||
.. _location-id-term:
|
||||
|
||||
location_id
|
||||
-----------
|
||||
|
|
@ -65,12 +70,15 @@ Used in transport layer to find ip address of a node. ::
|
|||
|
||||
'location_id': '96bc779540d832284680785ecd948a2d'
|
||||
|
||||
.. _transports-id-term:
|
||||
|
||||
transports_id
|
||||
-------------
|
||||
Used to find transports array that will be used for transport selection. ::
|
||||
|
||||
'transports_id': '3889e1790e68b80b4f255cf0e13494b1'
|
||||
|
||||
|
||||
BAT transport
|
||||
-------------
|
||||
According to preferences solar will choose best available transport for
|
||||
|
|
@ -85,12 +93,11 @@ Used in solar to describe all possible transitions between resources changes.
|
|||
Each event allows to specify two points of transitions, condition of this
|
||||
transition and type of event.
|
||||
|
||||
Right now we are supporting 2 types of events.
|
||||
Right now we are supporting 2 types of events:
|
||||
|
||||
1. Dependency
|
||||
Inserts edge between 2 changes into the deployment plan.
|
||||
2. Reaction
|
||||
Inserts change specified in reaction and makes edge between parent and child.
|
||||
1. Dependency - inserts edge between 2 changes into the deployment plan.
|
||||
2. Reaction - inserts change specified in reaction and makes edge between
|
||||
parent and child.
|
||||
|
||||
Example ::
|
||||
|
||||
|
|
@ -106,7 +113,7 @@ Example ::
|
|||
Virtual resource/template
|
||||
=========================
|
||||
|
||||
Composition layer that allows to:
|
||||
Composition layer that allows user to:
|
||||
|
||||
- group resources
|
||||
- specify connections between inputs
|
||||
|
|
@ -130,9 +137,9 @@ History
|
|||
After action that is related to change will be executed - it will be moved to
|
||||
history with same uuid.
|
||||
|
||||
Commited resource data
|
||||
Committed resource data
|
||||
----------------------
|
||||
After each succesfull change commited copy of resource data will be updated
|
||||
After each successful change committed copy of resource data will be updated
|
||||
with diff of that change.
|
||||
|
||||
.. _orch-term:
|
||||
|
|
|
|||
|
|
@ -5,7 +5,6 @@ Deployment operations
|
|||
|
||||
Stage changes
|
||||
-------------
|
||||
|
||||
After user created all required resource - it is possible to automatically
|
||||
detect which resource requires changes with ::
|
||||
|
||||
|
|
@ -14,30 +13,30 @@ detect which resource requires changes with ::
|
|||
|
||||
History
|
||||
-------
|
||||
After changes are staged - they will be used to populate history which can be seen
|
||||
with command (*n* option used to limit number of items, -1 will return all changes) ::
|
||||
After changes are staged - they will be used to populate history which can be
|
||||
previewed with command (*n* option used to limit number of items, -1 will
|
||||
return all changes) ::
|
||||
|
||||
solar changes history -n 5
|
||||
|
||||
Prepare deployment plan
|
||||
-----------------------
|
||||
|
||||
User is able to generate deployment scenario based on changes found by system log. ::
|
||||
User is able to generate deployment scenario based on changes found by system
|
||||
log. ::
|
||||
|
||||
solar changes process
|
||||
|
||||
|
||||
This command will prepare deployment graph, and return uid of deployment graph to
|
||||
work with.
|
||||
This command will prepare deployment graph, and return uid of deployment graph
|
||||
to work with.
|
||||
|
||||
All commands that are able to manipulate deployment graph located in
|
||||
*orch* namespace.
|
||||
|
||||
|
||||
.. tip::
|
||||
Solar writes returned deployment graph uid into special file (`.solar_cli_uids`), it
|
||||
allows you to use `last` instead of full returned uid:
|
||||
`solar orch report <uid>` becomes `solar orch report last`
|
||||
Solar writes returned deployment graph uid into special file
|
||||
(`.solar_cli_uids`), it allows you to use `last` instead of full returned
|
||||
uid: `solar orch report <uid>` becomes `solar orch report last`
|
||||
|
||||
|
||||
Report
|
||||
|
|
@ -84,6 +83,7 @@ All tasks will be returned to PENDING state, and deployment will be restarted ::
|
|||
|
||||
Retry deployment
|
||||
----------------
|
||||
Orchestrator will reset all ERROR tasks to PENDING state and restart deployment ::
|
||||
Orchestrator will reset all ERROR tasks to PENDING state and restart
|
||||
deployment ::
|
||||
|
||||
solar orch retry <uid>
|
||||
|
|
|
|||
|
|
@ -3,7 +3,15 @@
|
|||
Resource
|
||||
========
|
||||
|
||||
Resource is one of the key Solar components. Resoruce definition takes place in ``meta.yaml`` file.
|
||||
Resource is one of the key Solar components, almost every entity in Solar is a
|
||||
resource. Examples are:
|
||||
|
||||
* packages
|
||||
* services
|
||||
|
||||
Resources are defined in ``meta.yaml`` file. This file is responsible for basic
|
||||
configuration of given resource. Below is an explanation what constitutes
|
||||
typical resource.
|
||||
|
||||
|
||||
Basic resource structure
|
||||
|
|
@ -23,14 +31,34 @@ Handler
|
|||
|
||||
.. TODO: add link to handlers doc there
|
||||
|
||||
Layer that is responsible for action execution. You need to specify handler per resource, definition in ``meta.yaml`` looks like ::
|
||||
Pluggable layer that is responsible for executing an action on resource. You
|
||||
need to specify handler per every resource. Handler is defined in ``meta.yaml``
|
||||
as below ::
|
||||
|
||||
handler: puppet
|
||||
|
||||
Solar currently supports following handlers:
|
||||
|
||||
* puppet - first version of puppet handler (legacy, will be deprecated soon)
|
||||
* puppetv2 - second, improved version of puppet, supporting hiera integration
|
||||
* ansible_playbook - first version of ansible handler (legacy, will be
|
||||
deprecated soon)
|
||||
* ansible_template - second generation of ansible implementation, includes
|
||||
transport support
|
||||
|
||||
Handlers are pluggable, so you can write your own easily to extend
|
||||
functionality of Solar. Interesting examples might be Chef, SaltStack,
|
||||
CFEngine etc. Using handlers allows Solar to be quickly implemented in various
|
||||
environments and integrate with already used configuration management tools.
|
||||
|
||||
Input
|
||||
-----
|
||||
Treat them as values that your resouce have. All needed inputs should be provided in ``meta.yaml`` for example ::
|
||||
Inputs are essentially values that given resource can accept. Exact usage
|
||||
depends on handler and actions implementation. If your handler is puppet,
|
||||
inputs are basically parameters that can be accepted by puppet manifest
|
||||
underneath.
|
||||
|
||||
All needed inputs should be defined in ``meta.yaml`` for example: ::
|
||||
|
||||
input:
|
||||
keystone_password:
|
||||
|
|
@ -48,7 +76,9 @@ Treat them as values that your resouce have. All needed inputs should be provide
|
|||
|
||||
Input schema
|
||||
~~~~~~~~~~~~
|
||||
It allows to validate if all values are correct. ``!`` at the end of a type means that this is required (``null`` value is not valid).
|
||||
Input definition contains basic schema validation that allows to validate if
|
||||
all values are correct. ``!`` at the end of a type means that it is required
|
||||
(``null`` value is not valid).
|
||||
|
||||
* string type ``str``, ``str!``
|
||||
* integer type ``int``, ``int!``
|
||||
|
|
@ -63,14 +93,18 @@ It allows to validate if all values are correct. ``!`` at the end of a type mean
|
|||
|
||||
Action
|
||||
------
|
||||
Solar wraps deployment code into actions with specific names. Actions are executed by :ref:`res-handler-term`
|
||||
Solar wraps deployment code into actions with specific names. Actions are
|
||||
executed by :ref:`res-handler-term`
|
||||
|
||||
Several actions of resource are mandatory:
|
||||
|
||||
- run
|
||||
- remove
|
||||
- update
|
||||
|
||||
You can just put files into ``actions`` subdir in your resource or solar will detect them automaticaly, you can also provide actions in ``meta.yaml`` ::
|
||||
You can just put files into ``actions`` subdir in your resource and solar will
|
||||
detect them automatically based on their names, or you can also customize
|
||||
action file names in ``meta.yaml`` ::
|
||||
|
||||
actions:
|
||||
run: run.pp
|
||||
|
|
@ -79,6 +113,7 @@ You can just put files into ``actions`` subdir in your resource or solar will de
|
|||
Tag
|
||||
---
|
||||
|
||||
You can attach as many tags to resource as you want, later you can use those tags for grouping etc ::
|
||||
Tags are used for flexible grouping of resources. You can attach as many tags
|
||||
to resource as you want, later you can use those tags for grouping etc ::
|
||||
|
||||
tags: [resource=hosts_file, tag_name=tag_value, just_some_label]
|
||||
|
|
|
|||
|
|
@ -0,0 +1,50 @@
|
|||
.. _transports_details:
|
||||
|
||||
|
||||
Transports
|
||||
==========
|
||||
|
||||
Transports are used by Solar to communicate with managed nodes.
|
||||
Transports are also resources, so they have all resources features and
|
||||
flexibility.
|
||||
Transports should be added to a node, but if you need you can add different
|
||||
transports for different resources.
|
||||
|
||||
How it works
|
||||
------------
|
||||
|
||||
Each resource in solar has a random :ref:`transports-id-term` generated,
|
||||
when resources are connected to each other. Solar will ensure that correct
|
||||
`transport_id` is used. Then using this `transport_id` a correct real value is
|
||||
fetched. Changing transports contents will not cause `resource.update` action
|
||||
for related resources.
|
||||
|
||||
Sync transport
|
||||
--------------
|
||||
|
||||
This transport uploads required information to target node.
|
||||
|
||||
Currently there are following sync transports available:
|
||||
|
||||
* ssh
|
||||
* rsync
|
||||
* solar_agent
|
||||
* torrent
|
||||
|
||||
Run transport
|
||||
-------------
|
||||
|
||||
This transport is responsible for running commands on remote host.
|
||||
|
||||
Currently there are following run transports available:
|
||||
|
||||
* ssh
|
||||
* solar_agent
|
||||
|
||||
BAT transport
|
||||
-------------
|
||||
|
||||
A transport that will automatically select best available transport (BAT) that
|
||||
is available for a given resource. Currently it's default transport in the
|
||||
system, so when you add more transports, everything should configure
|
||||
automatically.
|
||||
6
tox.ini
6
tox.ini
|
|
@ -21,6 +21,12 @@ deps = -r{toxinidir}/requirements.txt
|
|||
sphinx
|
||||
commands = {posargs:}
|
||||
|
||||
[testenv:cover]
|
||||
commands =
|
||||
coverage erase
|
||||
python setup.py testr --coverage \
|
||||
--testr-args='--concurrency=1 {posargs}'
|
||||
|
||||
[testenv:devenv]
|
||||
envdir = devenv
|
||||
usedevelop = True
|
||||
|
|
|
|||
|
|
@ -0,0 +1,14 @@
|
|||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
# should be executed from directory with required Dockerfile
|
||||
name_w_tags=$1
|
||||
|
||||
if [[ -z "$name_w_tags" ]]; then
|
||||
name_w_tags='solarproject/solar-celery:latest'
|
||||
fi
|
||||
|
||||
echo "Building image with name $name_w_tags"
|
||||
docker build -t "$name_w_tags" .
|
||||
docker push "$name_w_tags"
|
||||
|
||||
Loading…
Reference in New Issue