watcher/doc/source/architecture.rst

System Architecture

This page presents the current technical Architecture of the Watcher system.

Overview

Below you will find a diagram, showing the main components of Watcher:

Components

AMQP Bus

The AMQP message bus handles asynchronous communications between the different Watcher components.

Cluster History Database

This component stores the data related to the Cluster History <cluster_history_definition>.

It can potentially rely on any appropriate storage system (InfluxDB, OpenTSDB, MongoDB,...) but will probably be more performant when using Time Series Databases which are optimized for handling time series data, which are arrays of numbers indexed by time (a datetime or a datetime range).

Watcher API

This component implements the REST API provided by the Watcher system to the external world.

It enables the Administrator <administrator_definition> of a Cluster <cluster_definition> to control and monitor the Watcher system via any interaction mechanism connected to this API:

• CLI <archi_watcher_cli_definition>
• Horizon plugin
• Python SDK

You can also read the detailed description of Watcher API.

Watcher Applier

This component is in charge of executing the Action Plan <action_plan_definition> built by the Watcher Decision Engine <watcher_decision_engine_definition>.

It connects to the message bus <amqp_bus_definition> and launches the Action Plan <action_plan_definition> whenever a triggering message is received on a dedicated AMQP queue.

The triggering message contains the Action Plan UUID.

It then gets the detailed information about the Action Plan <action_plan_definition> from the Watcher Database <watcher_database_definition> which contains the list of Actions <action_definition> to launch.

It then loops on each Action <action_definition>, gets the associated class and calls the execute() method of this class. Most of the time, this method will first request a token to the Keystone API and if it is allowed, sends a request to the REST API of the OpenStack service which handles this kind of atomic Action <action_definition>.

Note that as soon as Watcher Applier <watcher_applier_definition> starts handling a given Action <action_definition> from the list, a notification message is sent on the message bus <amqp_bus_definition> indicating that the state of the action has changed to ONGOING.

If the Action <action_definition> is successful, the Watcher Applier <watcher_applier_definition> sends a notification message on the bus <amqp_bus_definition> informing the other components of this.

If the Action <action_definition> fails, the Watcher Applier <watcher_applier_definition> tries to rollback to the previous state of the Managed resource <managed_resource_definition> (i.e. before the command was sent to the underlying OpenStack service).

Watcher CLI

The watcher command-line interface (CLI) can be used to interact with the Watcher system in order to control it or to know its current status.

Please, read the detailed documentation about Watcher CLI

Watcher Database

This database stores all the Watcher domain objects which can be requested by the Watcher API <archi_watcher_api_definition> or the Watcher CLI <archi_watcher_cli_definition>:

• Audit templates <audit_template_definition>
• Audits <audit_definition>
• Action plans <action_plan_definition>
• Actions <action_definition>
• Goals <goal_definition>

The Watcher domain being here "optimization of some resources provided by an OpenStack system".

Watcher Decision Engine

This component is responsible for computing a set of potential optimization Actions <action_definition> in order to fulfill the Goal <goal_definition> of an Audit <audit_definition>.

It first reads the parameters of the Audit <audit_definition> from the associated Audit Template <audit_template_definition> and knows the Goal <goal_definition> to achieve.

It then selects the most appropriate Strategy <strategy_definition> depending on how Watcher was configured for this Goal <goal_definition>.

The Strategy <strategy_definition> is then dynamically loaded (via stevedore). The Watcher Decision Engine <watcher_decision_engine_definition> calls the execute() method of the Strategy <strategy_definition> class which generates a set of Actions <action_definition>.

These Actions <action_definition> are scheduled in time by the Watcher Planner <watcher_planner_definition> (i.e., it generates an Action Plan <action_plan_definition>).

In order to compute the potential Solution <solution_definition> for the Audit, the Strategy <strategy_definition> relies on two sets of data:

• the current state of the Managed resources <managed_resource_definition> (e.g., the data stored in the Nova database)
• the data stored in the Cluster History Database <cluster_history_db_definition> which provides information about the past of the Cluster <cluster_definition>

So far, only one Strategy <strategy_definition> can be associated to a given Goal <goal_definition> via the main Watcher configuration file.