Commit Graph

30 Commits

Author SHA1 Message Date
gengchc2 857c239e97 Remove redundant code and improve test coverage
storage/driver.py is redundant file
get_opts in elasticsearchv2.py is redundant

Change-Id: I2c6d892016508c21fbc782d1628136618342c21f
2019-01-08 05:29:23 +00:00
gengchc2 df5cca26bb Delete invalid codes and Add a unit test for elastic
1.In _validate_opts:
The following code is True forever, We will delete it.
2. Add unit test for ca_certs in _validate_opts.

Change-Id: I25a9a4aaae6fd5677d07d376f66188dc0b92bc3b
2018-12-28 22:07:28 -08:00
Mohammed Naser e0976af072
Allow bootstrapping against clusters and unify host options
The current settings inside freezer-manage reflect on using host and
port which are no longer used.  In addition, there seems to be no
splitting of the values in freezer-manage which means that it would
create a problem when trying to install against a list of hosts as
the ElasticSearch driver will refuse to accept the list.

This patch unifies all references of to 'hosts' and changes the option
to use ListOpt which is comma seperated values in the configuration
file which are then split into an array by oslo.config, which avoids
having to split things inside freezer.

Closes-Bug: #1681931

Change-Id: I613d0672ec2b186cd0d6365e795b35d9cff26dcd
2017-04-13 15:13:59 -04:00
yfzhao c91ef0b365 Remove log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.

See:
http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html
http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html

Change-Id: I300dc59286f0f6c363b2db06f98bb02c37cae80b
Closes-bug: #1674599
2017-03-21 17:34:02 +08:00
Saad Zaher 528f3234e9 Allow using different database backends
Abstract storage configuration section to load different database
backend drivers. so we can keep supporting elasticsearch for
a while after moving to oslo.db also this will help in adding
API v2 as the elasticsearch driver will be changed.

Each db driver will register it's configuration on the run time.

Change-Id: Ie861e569f4add783c6f580cd0c08ed0fb3440151
2017-03-15 19:20:56 +00:00
Memo Garcia 7e069dba53 Fix for preventing the client to register itself again and again
This fix prevents the scheduler to register itself more than once.

Change-Id: I3d402fe6c1901abc9ed0d11d0f84e9d081ee4c48
Closes-bug: #1668671
2017-02-28 15:57:47 +00:00
Partha Bera 2847a4c97d No schema validation for client doc in Freezer API
This patch has added the necessary helper class
in the existing mechanism to validate client doc.

Change-Id: I9d2ade12b9306936fe32cbe7fba3adfcf4e367b6
Closes-Bug: #1650163
Partial-Bug: #1592308
2016-12-28 08:55:11 +00:00
Ruslan Aliev cf95fa1e86 Add full pep8 check
This patch adds full pep8 check support. All code style problems
is fixed.

Closes-Bug: #1646103
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I1fce70a1b4e4c70379fc3876f85170b06a044059
2016-12-05 19:57:32 +04:00
Ruslan Aliev 8002c1ff8f Fix get backup object
Method on_get returns list of backups, not a single backup.
This patch fixes this issue.

Change-Id: I50ae24b11b58bcea3d2c9f117957300a926cd3ab
Closes-Bug: #1646573
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
2016-12-02 01:11:46 +03:00
Saad Zaher d44c089262 Fixed elasticsearch scroll problem in scan search
Change-Id: I587c366d86462f4e18cbb3ba05f1178ae83f11d4
Closes-Bug: #1616426
2016-08-24 16:28:09 +00:00
Memo Garcia 3345c5337b Fixed mapper_parser_error in elasticsearch when inserting documents with _version
Closes-bug: 1595144
Change-Id: I80d6ba968b0053ccba3d1efebb18959df51c2253
2016-06-22 12:18:15 +01:00
Saad Zaher a94ea3fe73 Remove _version from the document passed to elk
fix freezer-scheduler job-start action by remove _version from elk
document being passed to elasticsearch.

Change-Id: I655ed59a5687ef1ea14c2b75d973f01666b3bb24
Closes-Bug: #1590413
2016-06-08 12:33:29 +00:00
Saad Zaher fe2a5cf8a8 Remove version string from elk documents
Remove the version string from job, sessions and action documents

Change-Id: I36ad9375e528746effd8f7b4f5a5d3d67b8853e2
Closes-Bug: #1587140
2016-05-30 16:47:17 +00:00
Jonas Pfannschmidt 67a8bdb9e4 Remove version when creating records in elasticsearch
Closes-Bug: 1557091

Change-Id: Ieecde522f228d01d3054855ea8096b6352ac3667
2016-05-06 17:43:35 +01:00
Memo Garcia 3e95bc6448 Fix refresh problem when a document gets created in elasticsearch
Closes-bug: 1569939
Change-Id: I5cdb6416bb6916b70bc0d6854bfc3534d6968fd5
2016-04-13 17:17:04 +01:00
Fausto Marzi 8d220b56a7 Switch to testr from pytest
Align to the other Openstack project and
use testr instead of pytest

Includes:
    - pylint and pep8 fixes.
    - py34 compliance
    - requirements updated
    - updated path to dsvm gate job

Change-Id: I10a5ea8d581029eba8f3ab5e016a1dd1919fa117
Depends-On: I4f9050cd551e87d2e398b6f94dc904154d0b1b13
2016-02-28 10:37:21 +00:00
Fabrizio Vanni 0dcacd908e remove elasticsearch method delete_by_query
The elasticsearch method delete_by_query has been removed
from the api because deprecated since elasticsearch 1.7

see:
https://www.elastic.co/guide/en/elasticsearch/reference/1.7/docs-delete-by-query.html

Change-Id: Ib21708c7147f4202eb7e4dba7a64adbc192d94af
2015-11-23 14:56:23 +00:00
memo dfa9598545 Updated LICENSE file and headers
removed unnecesary header in license
added missing header in files

Change-Id: Iea79505307479541de01231dd25c3821e9e8999c
2015-11-09 16:13:30 +00:00
Fabrizio Vanni 4deb7ead04 config options for elasticsearch connection
Add support for options to be passed the the elasticsearch connection

The following options are supported in the freezer-api.conf file:

  hosts='http://localhost:9200'
  use_ssl=False
  ca_certs=''
  use_ssl=False
  timeout=60
  retries=20

The provided values are the respective defaults.

Implements blueprint: api-conf-db-options

Change-Id: I373503bf845d0ac8091c83280ac21611d30d1a07
2015-09-09 18:58:02 +01:00
Fausto Marzi 4eda863997 Fixed pep8 failure
Few pep8 failure are now fixed.
2015-08-02 19:07:50 +01:00
Fabrizio Vanni 60b6f60d89 fix i18n related to the api
some imports were incorrect

Introduce an integration module for i18n, as suggested in:

http://docs.openstack.org/developer/oslo.i18n/usage.html

Change-Id: Iefbe56cda42120d0c58b4b331f58108ad4f9515a
2015-07-31 13:28:08 +00:00
Erno Kuvaja 7a571cb13d Refactoring logging and translations on freezer_api
Refactoring logging to use oslo.log to have standard OpenStack logging.
Refactoring log messages to enable i18n translations.
Refactoring Exception messages to enable i18n translations.

New requirement oslo.log.

Change-Id: I68850f9236f228c2ba99db2c3f3fb3a412f5ee4c
2015-07-30 19:45:42 +01:00
Erno Kuvaja 1e22e01a79 Refactor api imports to OpenStack Hacking compliant
http://docs.openstack.org/developer/hacking/

Change-Id: I533c7d075c55d019d55990ea9e9053db920fda45
2015-07-30 12:03:50 +00:00
Memo Garcia 249f287af9 Job creation for multiple clients
Implements: blueprint freezer-api-web-ui

Change-Id: I6d2b29b18909cef627646b437454d6075b725df5
2015-07-16 13:31:36 +01:00
Fabrizio Vanni a389bf79fb freezer scheduler
The freezer scheduler is to be executed
as daemon process on the client machines

It has the following responsibilities:

  * when using the api:
    - register -if necessary- as a client in the api
    - download the list of jobs from the api
    - schedule the jobs for execution
    - launch the freezer client at the scheduled time
    - collect metadata and exit codes and upload them to the api
    - periodically poll the api for new/updated jobs
    - if a job is part of a session (a coordinated group of jobs)
      it updates the session status when job starts/stops

  * when not using the api
    - load jobs configurations from files
    - schedule the jobs for execution
    - launch the freezer client at the scheduled time

The freezer scheduler can also be used to manage jobs
and sessions using the following positional parameters:

  job-list
  job-get
  job-create
  job-delete
  job-start
  job-stop
  session-list
  session-get
  session-create
  session-delete
  session-list-job
  session-add-job
  session-remove-job

or to register the client in the api using the positional parameter:

  register

Implements blueprint: freezer-scheduler-start

Change-Id: I06ae202a0f464f7240c137744a5b54d1177cabd9
2015-07-10 18:51:17 +01:00
Fabrizio Vanni f4a9d19a12 Add api support for jobs
removes distinction between actions and configs in the api

The v1/jobs endpoint is the single point of access to the api
to store, search and retrieve operating instruction about
a single freezer run.

optional scheduling informations can be added to a job to
allow for future/recurrent job execution.

removes api endpoints
  - v1/actions
  - v1/configs

adds api endpoints:
  - v1/jobs

Implements blueprint: freezer-api-jobs

Change-Id: Ideeef14dfccd21ddd10b4faa438124c04d2e1ff8
2015-06-21 18:13:54 +00:00
Jonas Pfannschmidt 0b12bf7274 This version contains the following pages:
- Overview displays charts/reports (Currently only placeholders)
 - Configurations allows to define new backup configurations and link them to
   instances
 - Backups shows a list of all succesful backups and allows to restore them
 - Restores shows a history of all restored backups

It communicates with the api server via the client apis (freeezer.client.client).
The "glue code" between client apis and the UI lives in freezer.api.api.

Implements: Blueprint freezer-api-web-ui
Change-Id: I48cd8cba2b0169c6e64f650233c1a31b91ced34f
2015-05-31 17:01:57 +01:00
Fabrizio Vanni 5aa02c14ed freezer api support for action
Adds an endpoint to the api for managing actions

Change-Id: I8a84ea4aa46a7acaecb103aa20f3dc8acd128d0c
2015-05-13 19:05:25 +01:00
Fabrizio Vanni c86f04978e freezer api support for client registration
Adds an endpoint to the api for the registration of freezer clients

Change-Id: I1ca2a5e0021d383df70dfd001ab12967714c35bc
Implements: blueprint freezerclient-registration
2015-04-30 14:07:19 +01:00
Fabrizio Vanni acbc7aefae Freezer API
First implementation of the freezer API.
Slightly more than a skeleton with basic functionality

Change-Id: Iae04affea3aa0f4a943599b528df49d9d4a5b845
Implements: blueprint freezer-api-first-rel
2015-04-08 18:25:45 +01:00