openstack
/
freezer
Code Issues Proposed changes

Updating install-guide docs 

Change-Id: I7586763cf25be7ca115c7325129a7c9aa7022603
This commit is contained in:
Saad Zaher 2017-07-03 19:19:02 +01:00
parent fe5395cd7e
commit 95b49ffa24
14 changed files with 300 additions and 230 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
devstack/gate_hook.sh
View File

@ -22,8 +22,8 @@ fi
echo "Start Gate Hook"
export DEVSTACK_LOCAL_CONFIG="enable_plugin freezer https://git.openstack.org/openstack/freezer"
export DEVSTACK_LOCAL_CONFIG+=$'\n'"enable_plugin freezer-api https://git.openstack.org/openstack/freezer-api"
#export DEVSTACK_LOCAL_CONFIG="enable_plugin freezer https://git.openstack.org/openstack/freezer"
# export DEVSTACK_LOCAL_CONFIG+=$'\n'"enable_plugin freezer-api https://git.openstack.org/openstack/freezer-api"
# Swift is needed for some of the integration tests
export DEVSTACK_LOCAL_CONFIG+=$'\n'"enable_service s-proxy s-object s-container s-account"

184
install-guide/source/common_configure.rst
View File

@ -1,10 +1,186 @@
2. Edit the ``/etc/freezer-api/freezer-api.conf`` file and complete the following
2. Copy the configuration files to ``/etc/freezer/``:
.. code-block:: bash
sudo cp etc/freezer/freezer-api.conf.sample /etc/freezer/freezer-api.conf
sudo cp etc/freezer/freezer-paste.ini /etc/freezer/freezer-paste.ini
3. Edit the ``/etc/freezer/freezer-api.conf`` file and complete the following
actions:
* In the ``[database]`` section, configure database access:
* In the ``[storage]`` section, configure database access:
.. code-block:: ini
[database]
[storage]
...
connection = mysql+pymysql://freezer-api:FREEZER-API_DBPASS@controller/freezer-api
backend = elasticv2
driver = freezer_api.storage.elasticv2.ElasticSearchEngineV2
* In the ``[elasticv2]`` section, configure elasticsearch access:
You might need to create the elasticv2 section first.
.. code-block:: ini
[elasticv2]
...
hosts=http://localhost:9200
index=freezer
use_ssl=False
ca_certs=''
use_ssl=False
timeout=60
retries=20
number_of_replicas = 1
Start elasticsearch
-------------------
The currently supported db is Elasticsearch. In case you are using a dedicated
instance of the server, you'll need to start it. Depending on the OS flavor
it might be a:
.. code-block:: bash
service elasticsearch start
or, on systemd
.. code-block:: bash
systemctl start elasticsearch
Using freezer-manage
--------------------
Elasticsearch needs to know what type of data each document's field contains.
This information is contained in the `mapping`, or schema definition.
Elasticsearch will use dynamic mapping to try to guess the field type from
the basic datatypes available in JSON, but some field's properties have to be
explicitly declared to tune the indexing engine.
To do that, use the freezer-manage command:
::
freezer-manage db sync
You should have updated your configuration files before doing this step.
freezer-manage has the following options:
* To create the db mappings use the following command
.. code-block:: bash
freezer-manage db sync
* To update the db mappings using the following command. Update means that you
might have some mappings and you want to update it with a more recent ones
.. code-block:: bash
freezer-manage db update
* To remove the db mappings using the following command
.. code-block:: bash
freezer-manage db remove
* To print the db mappings using the following command
.. code-block:: bash
freezer-manage db show
* To update your settings (number of replicas) all what you need to do is to
change its value in the configuration file and then run the following command
.. code-block:: bash
freezer-manage db update-settings
If you provided an invalid number of replicas that will cause problems later on,
so it's highly recommended to make sure that you are using the correct number
of replicas. For more info click here `Elasticsearch_Replicas_instructions <https://www.elastic.co/guide/en/elasticsearch/guide/current/replica-shards.html>`_
* To get information about optional additional parameters
.. code-block:: bash
freezer-manage -h
* If you want to add any additional parameter like --yes or --erase, they should
be before the db option. Check the following examples
Wrong Example
.. code-block:: bash
freezer-manage db sync -y -e
Correct Example
.. code-block:: bash
freezer-manage -y -e db sync
create the mappings
-------------------
.. code-block:: bash
freezer-manage -y -e db sync
run simple instance
-------------------
.. code-block:: console
$ freezer-api --config-file /etc/freezer/freezer-api.conf
examples running using uwsgi
----------------------------
.. code-block:: console
$ uwsgi --http :9090 --need-app --master --module freezer_api.cmd.wsgi:application
$ uwsgi --https :9090,foobar.crt,foobar.key --need-app --master --module freezer_api.cmd.wsgi:application
example running freezer-api with apache2
----------------------------------------
.. code-block:: none
sudo vi /etc/apache2/sites-enabled/freezer-api.conf
<VirtualHost ...>
WSGIDaemonProcess freezer-api processes=2 threads=2 user=freezer display-name=%{GROUP}
WSGIProcessGroup freezer-api
WSGIApplicationGroup freezer-api
WSGIScriptAlias / /opt/stack/freezer_api/cmd/wsgi.py
<IfVersion >= 2.4>
ErrorLogFormat "%M"
</IfVersion>
ErrorLog /var/log/%APACHE_NAME%/freezer-api.log
LogLevel warn
CustomLog /var/log/freezer-api/freezer-api_access.log combined
<Directory /opt/stack/freezer_api>
Options Indexes FollowSymLinks MultiViews
Require all granted
AllowOverride None
Order allow,deny
allow from all
LimitRequestBody 102400
</Directory>
</VirtualHost>
Install and configure freezer-scheduler/agent
---------------------------------------------
.. include:: install_agent.rst

41
install-guide/source/common_prerequisites.rst
View File

@ -1,41 +1,6 @@
Prerequisites
-------------
Before you install and configure the Backup service,
you must create a database, service credentials, and API endpoints.
#. To create the database, complete these steps:
* Use the database access client to connect to the database
server as the ``root`` user:
.. code-block:: console
$ mysql -u root -p
* Create the ``freezer`` database:
.. code-block:: console
CREATE DATABASE freezer;
* Grant proper access to the ``freezer`` database:
.. code-block:: console
GRANT ALL PRIVILEGES ON freezer-api.* TO 'freezer'@'localhost' \
IDENTIFIED BY 'FREEZER_DBPASS';
GRANT ALL PRIVILEGES ON freezer.* TO 'freezer'@'%' \
IDENTIFIED BY 'FREEZER_DBPASS';
Replace ``FREEZER_DBPASS`` with a suitable password.
* Exit the database access client.
.. code-block:: console
exit;
#. Source the ``admin`` credentials to gain access to
admin-only CLI commands:
@ -68,8 +33,8 @@ you must create a database, service credentials, and API endpoints.
.. code-block:: console
$ openstack endpoint create --region RegionOne \
backup public http://controller:9090/vY/%\(tenant_id\)s
backup public http://controller:9090/
$ openstack endpoint create --region RegionOne \
backup internal http://controller:9090/vY/%\(tenant_id\)s
backup internal http://controller:9090/
$ openstack endpoint create --region RegionOne \
backup admin http://controller:9090/vY/%\(tenant_id\)s
backup admin http://controller:9090/

3
install-guide/source/db-install.rst
View File

@ -48,6 +48,3 @@ you must install the database.
.. code-block:: console
$ sudo yum install -y elasticsearch-2.3.0.rpm

8
install-guide/source/devstack_plugin.rst
View File

@ -13,12 +13,14 @@ of your local.conf file:
.. code-block:: ini
enable_plugin freezer-api <GITURL> [GITREF]
enable_plugin freezer <GITURL> [GITREF]
enable_plugin freezer-web-ui <GITURL> [GITREF]
where
.. code-block:: none
<GITURL> is the URL of a freezer-api repository
<GITURL> is the URL of a freezer-api, freezer, freezer-web-ui repository
[GITREF] is an optional git ref (branch/ref/tag). The default is master.
For example
@ -26,6 +28,8 @@ For example
.. code-block:: ini
enable_plugin freezer-api https://git.openstack.org/openstack/freezer-api.git master
enable_plugin freezer https://git.openstack.org/openstack/freezer.git master
enable_plugin freezer-web-ui https://git.openstack.org/openstack/freezer-web-ui.git master
Plugin Options
--------------
@ -39,7 +43,7 @@ To use the *uwsgi* server set the following environment variable
The default port is *9090*. To configure the api to listen on a different port
set the variable `FREEZER_API_PORT`.
For example to make use of port 19090 use
For example to make use of port 19090 instead of 9090 use
.. code-block:: bash

25
install-guide/source/get_started.rst
View File

@ -22,11 +22,26 @@ Installation Guide.
Concepts and definitions
========================
*hostname* is _probably_ going to be the host fqdn.
*backup_id*
defined as UUID of a backup
``freezer-api`` service
Accepts and responds to end user API calls...
``freezer-scheduler`` service
Does API calls to ``freezer-api`` to schedule, fetch, update or Delete backup
jobs.
``freezer-agent`` service
Python application run on the same node like ``freezer-scheduler`` and it
gets called by ``freezer-scheduler`` to execute backups/restore operations.
``freezer-dr`` service
Independent service from all other freezer services. It runs on the control
plane to do disaster recovery in case of any compute node failed (more to
follow)
*hostname* is _probably_ going to be the host fqdn.
*backup_id* defined as UUID of a backup.

18
install-guide/source/index.rst
View File

@ -5,15 +5,15 @@ Backup/Restore and DR service
.. toctree::
:maxdepth: 2
get_started.rst
install.rst
verify.rst
next-steps.rst
api_routes.rst
metadata_structure.rst
client_structure.rst
api_documents.rst
known_issues.rst
get_started
install
verify
next-steps
api_routes
metadata_structure
client_structure
api_documents
known_issues
This chapter assumes a working setup of OpenStack following the

2
install-guide/source/install-obs.rst
View File

@ -16,7 +16,7 @@ Install and configure components
.. code-block:: console
# zypper --quiet --non-interactive install
zypper --quiet --non-interactive install python-dev python-pip
.. include:: common_configure.rst

6
install-guide/source/install-rdo.rst
View File

@ -16,7 +16,7 @@ Install and configure components
.. code-block:: console
# yum install
$ sudo yum install python-dev python-pip
.. include:: common_configure.rst
@ -28,6 +28,6 @@ the system boots:
.. code-block:: console
# systemctl enable openstack-freezer-api.service
$ sudo systemctl enable openstack-freezer-api.service
# systemctl start openstack-freezer-api.service
$ sudo systemctl start openstack-freezer-api.service

8
install-guide/source/install-ubuntu.rst
View File

@ -4,7 +4,7 @@ Install and configure for Ubuntu
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This section describes how to install and configure the Backup
service for Ubuntu 14.04 (LTS).
service for Ubuntu 16.04 (LTS).
.. include:: common_prerequisites.rst
@ -15,9 +15,9 @@ Install and configure components
.. code-block:: console
# apt-get update
$ sudo apt-get update
# apt-get install
$ sudo apt-get install python-dev python-pip
.. include:: common_configure.rst
@ -28,4 +28,4 @@ Restart the Backup services:
.. code-block:: console
# service openstack-freezer-api restart
$ sudo service openstack-freezer-api restart

159
install-guide/source/install.rst
View File

@ -15,159 +15,16 @@ Note that installation and configuration vary by distribution.
.. toctree::
:maxdepth: 2
db-install.rst
install-obs.rst
install-rdo.rst
install-ubuntu.rst
db-install
install-obs
install-rdo
install-ubuntu
.. code-block:: bash
.. code-block:: console
# git clone https://git.openstack.org/openstack/freezer-api.git
# cd freezer-api
# pip install ./
edit config file
----------------
.. code-block:: bash
# sudo cp etc/freezer/freezer-api.conf /etc/freezer/freezer-api.conf
# sudo cp etc/freezer/freezer-paste.ini /etc/freezer/freezer-paste.ini
# sudo vi /etc/freezer/freezer-api.conf
# sudo vi /etc/freezer/freezer-paste.ini
setup/configure the db
----------------------
The currently supported db is Elasticsearch. In case you are using a dedicated instance
of the server, you'll need to start it. Depending on the OS flavor it might be a:
.. code-block:: bash
# service elasticsearch start
or, on systemd
.. code-block:: bash
# systemctl start elasticsearch
Elasticsearch needs to know what type of data each document's field contains.
This information is contained in the `mapping`, or schema definition.
Elasticsearch will use dynamic mapping to try to guess the field type from
the basic datatypes available in JSON, but some field's properties have to be
explicitly declared to tune the indexing engine.
To do that, use the freezer-manage command:
::
# freezer-manage db sync
You should have updated your configuration files before doing this step.
freezer-manage has the following options:
* To create the db mappings use the following command::
# freezer-manage db sync
* To update the db mappings using the following command. Update means that you
might have some mappings and you want to update it with a more recent ones
::
# freezer-manage db update
* To remove the db mappings using the following command ::
# freezer-manage db remove
* To print the db mappings using the following command ::
# freezer-manage db show
* To update your settings (number of replicas) all what you need to do is to
change its value in the configuration file and then run the following command ::
# freezer-manage db update-settings
If you provided an invalid number of replicas that will cause problems later on,
so it's highly recommended to make sure that you are using the correct number
of replicas. For more info click here `Elasticsearch_Replicas_instructions <https://www.elastic.co/guide/en/elasticsearch/guide/current/replica-shards.html>`_
* To get information about optional additional parameters::
# freezer-manage -h
* If you want to add any additional parameter like --yes or --erase, they should
be before the db option. Check the following examples:
Wrong Example::
# freezer-manage db sync -y -e
Correct Example::
# freezer-manage -y -e db sync
run simple instance
-------------------
.. code-block:: bash
# freezer-api
examples running using uwsgi
----------------------------
.. code-block:: bash
# uwsgi --http :9090 --need-app --master --module freezer_api.cmd.wsgi:application
# uwsgi --https :9090,foobar.crt,foobar.key --need-app --master --module freezer_api.cmd.wsgi:application
example running freezer-api with apache2
----------------------------------------
.. code-block:: none
# sudo vi /etc/apache2/sites-enabled/freezer-api.conf
<VirtualHost ...>
WSGIDaemonProcess freezer-api processes=2 threads=2 user=freezer display-name=%{GROUP}
WSGIProcessGroup freezer-api
WSGIApplicationGroup freezer-api
WSGIScriptAlias / /opt/stack/freezer_api/cmd/wsgi.py
<IfVersion >= 2.4>
ErrorLogFormat "%M"
</IfVersion>
ErrorLog /var/log/%APACHE_NAME%/freezer-api.log
LogLevel warn
CustomLog /var/log/freezer-api/freezer-api_access.log combined
<Directory /opt/stack/freezer_api>
Options Indexes FollowSymLinks MultiViews
Require all granted
AllowOverride None
Order allow,deny
allow from all
LimitRequestBody 102400
</Directory>
</VirtualHost>
API registration
================
.. code-block:: bash
# openstack user create --domain default --password-prompt freezer
# openstack role add --project service --user freezer admin
# openstack service create --name freezer --description "Freezer Backup Service" backup
# openstack endpoint create --region RegionOne backup public http://freezer_api_publicurl:port
# openstack endpoint create --region RegionOne backup internal http://freezer_api_internalurl:port
# openstack endpoint create --region RegionOne backup admin http://freezer_api_adminurl:port
$ git clone https://git.openstack.org/openstack/freezer-api.git
$ cd freezer-api
$ pip install ./
.. toctree::

50
install-guide/source/install_agent.rst Normal file
View File

@ -0,0 +1,50 @@
.. _install_agent:
This section describes how to install and configure freezer-scheduler and
freezer-agent, on any node in the cloud or any vm inside the cloud.
This section assumes that you already have a working OpenStack
environment with at least the following components installed:
.. Keystone
.. Swift
.. code-block:: bash
git clone https://git.openstack.org/openstack/freezer.git
cd freezer
pip install ./
Configure the scheduler
-----------------------
1. Copy the configuration files to ``/etc/freezer/``:
.. code-block:: console
$ sudo cp etc/scheduler.conf.sample /etc/freezer/scheduler.conf
2. Edit the ``/etc/freezer/scheduler.conf`` file and complete the following
actions:
* In the ``[DEFAULT]`` section, configure database access:
The ``client_id`` has to be set to the hostname of the machine. It will be
used as an identifier for this node to fetch its scheduled backups
.. code-block:: ini
[DEFAULT]
...
client_id = hostname_of_machine
jobs_dir = /etc/freezer/scheduler/conf.d
3. Start ``freezer-scheduler``
.. code-block:: console
$ sudo freezer-scheduler --config-file /etc/freezer/scheduler.conf

16
install-guide/source/jobs.rst
View File

@ -29,7 +29,7 @@ Scheduling information enables future/recurrent execution of jobs
job document structure
.. code-block:: none
.. code-block:: json
"job": {
"job_action": { parameters for freezer to execute a specific action }
@ -130,7 +130,7 @@ Job examples
example backup freezer_action
.. code-block:: none
.. code-block:: json
"freezer_action": {
"action" : "backup"
@ -150,7 +150,7 @@ example backup freezer_action
example restore freezer_action
.. code-block:: none
.. code-block:: json
"freezer_action": {
"action": "restore"
@ -165,7 +165,7 @@ example restore freezer_action
example scheduled backup job.
job will be executed once at the provided datetime
.. code-block:: none
.. code-block:: json
"job": {
"job_actions":
@ -231,7 +231,7 @@ job will be executed once at the provided datetime
multiple scheduling choices allowed
.. code-block:: none
.. code-block:: json
"job": {
"job_actions":
@ -256,7 +256,7 @@ multiple scheduling choices allowed
Finished job with result
.. code-block:: none
.. code-block:: json
"job": {
"job_actions": [ ... ],
@ -286,7 +286,7 @@ If properties are specifically set in one action, then the specified value is th
Example
.. code-block:: none
.. code-block:: json
"job": {
"action_defaults": {
@ -322,7 +322,7 @@ Example
Is Equivalent to
.. code-block:: none
.. code-block:: json
"job": {
"job_actions": [{

6
install-guide/source/verify.rst
View File

@ -22,3 +22,9 @@ Verify operation of the Backup service.
.. code-block:: console
$ openstack endpoint list
#. List available backup jobs for current node:
.. code-block:: console
$ freezer job-list