openstack
/
watcher
Code Issues Proposed changes

Merge "Several fixes of strategies docs"

This commit is contained in:
Zuul 2018-04-11 09:12:22 +00:00 committed by Gerrit Code Review
parent 831e58df10 92dad3be2d
commit b537979e45
18 changed files with 94 additions and 174 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
doc/source/strategies/actuation.rst
View File

@ -9,7 +9,7 @@ Synopsis
**goal**: ``unclassified``
.. watcher-term:: watcher.decision_engine.strategy.strategies.actuation
.. watcher-term:: watcher.decision_engine.strategy.strategies.actuation.Actuator
Requirements
------------

2
doc/source/strategies/basic-server-consolidation.rst
View File

@ -9,7 +9,7 @@ Synopsis
**goal**: ``server_consolidation``
.. watcher-term:: watcher.decision_engine.strategy.strategies.basic_consolidation
.. watcher-term:: watcher.decision_engine.strategy.strategies.basic_consolidation.BasicConsolidation
Requirements
------------

6
doc/source/strategies/outlet_temp_control.rst
View File

@ -9,11 +9,7 @@ Synopsis
**goal**: ``thermal_optimization``
Outlet (Exhaust Air) temperature is a new thermal telemetry which can be
used to measure the host's thermal/workload status. This strategy makes
decisions to migrate workloads to the hosts with good thermal condition
(lowest outlet temperature) when the outlet temperature of source hosts
reach a configurable threshold.
.. watcher-term:: watcher.decision_engine.strategy.strategies.outlet_temp_control
Requirements
------------

2
doc/source/strategies/saving_energy.rst
View File

@ -9,7 +9,7 @@ Synopsis
**goal**: ``saving_energy``
.. watcher-term:: watcher.decision_engine.strategy.strategies.saving_energy
.. watcher-term:: watcher.decision_engine.strategy.strategies.saving_energy.SavingEnergy
Requirements
------------

2
doc/source/strategies/uniform_airflow.rst
View File

@ -9,7 +9,7 @@ Synopsis
**goal**: ``airflow_optimization``
.. watcher-term:: watcher.decision_engine.strategy.strategies.uniform_airflow
.. watcher-term:: watcher.decision_engine.strategy.strategies.uniform_airflow.UniformAirflow
Requirements
------------

2
doc/source/strategies/vm_workload_consolidation.rst
View File

@ -9,7 +9,7 @@ Synopsis
**goal**: ``vm_consolidation``
.. watcher-term:: watcher.decision_engine.strategy.strategies.vm_workload_consolidation
.. watcher-term:: watcher.decision_engine.strategy.strategies.vm_workload_consolidation.VMWorkloadConsolidation
Requirements
------------

2
doc/source/strategies/workload-stabilization.rst
View File

@ -9,7 +9,7 @@ Synopsis
**goal**: ``workload_balancing``
.. watcher-term:: watcher.decision_engine.strategy.strategies.workload_stabilization
.. watcher-term:: watcher.decision_engine.strategy.strategies.workload_stabilization.WorkloadStabilization
Requirements
------------

2
doc/source/strategies/workload_balance.rst
View File

@ -9,7 +9,7 @@ Synopsis
**goal**: ``workload_balancing``
.. watcher-term:: watcher.decision_engine.strategy.strategies.workload_balance
.. watcher-term:: watcher.decision_engine.strategy.strategies.workload_balance.WorkloadBalance
Requirements
------------

2
doc/source/strategies/zone_migration.rst
View File

@ -9,7 +9,7 @@ Synopsis
**goal**: ``hardware_maintenance``
.. watcher-term:: watcher.decision_engine.strategy.strategies.zone_migration
.. watcher-term:: watcher.decision_engine.strategy.strategies.zone_migration.ZoneMigration
Requirements
------------

22
watcher/decision_engine/strategy/strategies/actuation.py
View File

@ -14,16 +14,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
*Actuator*
This strategy allows anyone to create an action plan with a predefined set of
actions. This strategy can be used for 2 different purposes:
- Test actions
- Use this strategy based on an event trigger to perform some explicit task
"""
from oslo_log import log
@ -34,7 +24,17 @@ LOG = log.getLogger(__name__)
class Actuator(base.UnclassifiedStrategy):
"""Actuator that simply executes the actions given as parameter"""
"""Actuator
Actuator that simply executes the actions given as parameter
This strategy allows anyone to create an action plan with a predefined
set of actions. This strategy can be used for 2 different purposes:
- Test actions
- Use this strategy based on an event trigger to perform some explicit task
"""
@classmethod
def get_name(cls):

38
watcher/decision_engine/strategy/strategies/basic_consolidation.py
View File

@ -16,24 +16,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
*Good server consolidation strategy*
Consolidation of VMs is essential to achieve energy optimization in cloud
environments such as OpenStack. As VMs are spinned up and/or moved over time,
it becomes necessary to migrate VMs among servers to lower the costs. However,
migration of VMs introduces runtime overheads and consumes extra energy, thus
a good server consolidation strategy should carefully plan for migration in
order to both minimize energy consumption and comply to the various SLAs.
This algorithm not only minimizes the overall number of used servers, but also
minimizes the number of migrations.
It has been developed only for tests. You must have at least 2 physical compute
nodes to run it, so you can easily run it on DevStack. It assumes that live
migration is possible on your OpenStack cluster.
"""
from oslo_config import cfg
from oslo_log import log
@ -47,7 +29,25 @@ LOG = log.getLogger(__name__)
class BasicConsolidation(base.ServerConsolidationBaseStrategy):
"""Basic offline consolidation using live migration"""
"""Good server consolidation strategy
Basic offline consolidation using live migration
Consolidation of VMs is essential to achieve energy optimization in cloud
environments such as OpenStack. As VMs are spinned up and/or moved over
time, it becomes necessary to migrate VMs among servers to lower the
costs. However, migration of VMs introduces runtime overheads and
consumes extra energy, thus a good server consolidation strategy should
carefully plan for migration in order to both minimize energy consumption
and comply to the various SLAs.
This algorithm not only minimizes the overall number of used servers,
but also minimizes the number of migrations.
It has been developed only for tests. You must have at least 2 physical
compute nodes to run it, so you can easily run it on DevStack. It assumes
that live migration is possible on your OpenStack cluster.
"""
HOST_CPU_USAGE_METRIC_NAME = 'compute.node.cpu.percent'
INSTANCE_CPU_USAGE_METRIC_NAME = 'cpu_util'

6
watcher/decision_engine/strategy/strategies/outlet_temp_control.py
View File

@ -18,7 +18,7 @@
#
"""
*Good Thermal Strategy*:
*Good Thermal Strategy*
Towards to software defined infrastructure, the power and thermal
intelligences is being adopted to optimize workload, which can help
@ -26,6 +26,10 @@ improve efficiency, reduce power, as well as to improve datacenter PUE
and lower down operation cost in data center.
Outlet (Exhaust Air) Temperature is one of the important thermal
telemetries to measure thermal/workload status of server.
This strategy makes decisions to migrate workloads to the hosts with good
thermal condition (lowest outlet temperature) when the outlet temperature
of source hosts reach a configurable threshold.
"""
from oslo_config import cfg

18
watcher/decision_engine/strategy/strategies/storage_capacity_balance.py
View File

@ -14,24 +14,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
*Workload balance using cinder volume migration*
*Description*
This strategy migrates volumes based on the workload of the
cinder pools.
It makes decision to migrate a volume whenever a pool's used
utilization % is higher than the specified threshold. The volume
to be moved should make the pool close to average workload of all
cinder pools.
*Requirements*
* You must have at least 2 cinder volume pools to run
this strategy.
"""
from oslo_config import cfg
from oslo_log import log

25
watcher/decision_engine/strategy/strategies/uniform_airflow.py
View File

@ -16,31 +16,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
[PoC]Uniform Airflow using live migration
*Description*
It is a migration strategy based on the airflow of physical
servers. It generates solutions to move VM whenever a server's
airflow is higher than the specified threshold.
*Requirements*
* Hardware: compute node with NodeManager 3.0 support
* Software: Ceilometer component ceilometer-agent-compute running
in each compute node, and Ceilometer API can report such telemetry
"airflow, system power, inlet temperature" successfully.
* You must have at least 2 physical compute nodes to run this strategy
*Limitations*
- This is a proof of concept that is not meant to be used in production.
- We cannot forecast how many servers should be migrated. This is the
reason why we only plan a single virtual machine migration at a time.
So it's better to use this algorithm with `CONTINUOUS` audits.
- It assumes that live migrations are possible.
"""
from oslo_config import cfg
from oslo_log import log

70
watcher/decision_engine/strategy/strategies/vm_workload_consolidation.py
View File

@ -17,41 +17,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
*VM Workload Consolidation Strategy*
A load consolidation strategy based on heuristic first-fit
algorithm which focuses on measured CPU utilization and tries to
minimize hosts which have too much or too little load respecting
resource capacity constraints.
This strategy produces a solution resulting in more efficient
utilization of cluster resources using following four phases:
* Offload phase - handling over-utilized resources
* Consolidation phase - handling under-utilized resources
* Solution optimization - reducing number of migrations
* Disability of unused compute nodes
A capacity coefficients (cc) might be used to adjust optimization
thresholds. Different resources may require different coefficient
values as well as setting up different coefficient values in both
phases may lead to to more efficient consolidation in the end.
If the cc equals 1 the full resource capacity may be used, cc
values lower than 1 will lead to resource under utilization and
values higher than 1 will lead to resource overbooking.
e.g. If targeted utilization is 80 percent of a compute node capacity,
the coefficient in the consolidation phase will be 0.8, but
may any lower value in the offloading phase. The lower it gets
the cluster will appear more released (distributed) for the
following consolidation phase.
As this strategy leverages VM live migration to move the load
from one compute node to another, this feature needs to be set up
correctly on all compute nodes within the cluster.
This strategy assumes it is possible to live migrate any VM from
an active compute node to any other active compute node.
"""
from oslo_config import cfg
from oslo_log import log
@ -66,7 +31,40 @@ LOG = log.getLogger(__name__)
class VMWorkloadConsolidation(base.ServerConsolidationBaseStrategy):
"""VM Workload Consolidation Strategy"""
"""VM Workload Consolidation Strategy
A load consolidation strategy based on heuristic first-fit
algorithm which focuses on measured CPU utilization and tries to
minimize hosts which have too much or too little load respecting
resource capacity constraints.
This strategy produces a solution resulting in more efficient
utilization of cluster resources using following four phases:
* Offload phase - handling over-utilized resources
* Consolidation phase - handling under-utilized resources
* Solution optimization - reducing number of migrations
* Disability of unused compute nodes
A capacity coefficients (cc) might be used to adjust optimization
thresholds. Different resources may require different coefficient
values as well as setting up different coefficient values in both
phases may lead to to more efficient consolidation in the end.
If the cc equals 1 the full resource capacity may be used, cc
values lower than 1 will lead to resource under utilization and
values higher than 1 will lead to resource overbooking.
e.g. If targeted utilization is 80 percent of a compute node capacity,
the coefficient in the consolidation phase will be 0.8, but
may any lower value in the offloading phase. The lower it gets
the cluster will appear more released (distributed) for the
following consolidation phase.
As this strategy leverages VM live migration to move the load
from one compute node to another, this feature needs to be set up
correctly on all compute nodes within the cluster.
This strategy assumes it is possible to live migrate any VM from
an active compute node to any other active compute node.
"""
HOST_CPU_USAGE_METRIC_NAME = 'compute.node.cpu.percent'
INSTANCE_CPU_USAGE_METRIC_NAME = 'cpu_util'

31
watcher/decision_engine/strategy/strategies/workload_balance.py
View File

@ -16,35 +16,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
*[PoC]Workload balance using live migration*
*Description*
This strategy migrates a VM based on the VM workload of the hosts.
It makes decision to migrate a workload whenever a host's CPU or RAM
utilization % is higher than the specified threshold. The VM to
be moved should make the host close to average workload of all
hosts nodes.
*Requirements*
* Hardware: compute node should use the same physical CPUs
* Software: Ceilometer component ceilometer-agent-compute
running in each compute node, and Ceilometer API can
report such telemetry "cpu_util" and "memory.resident" successfully.
* You must have at least 2 physical compute nodes to run
this strategy.
*Limitations*
- This is a proof of concept that is not meant to be used in
production.
- We cannot forecast how many servers should be migrated.
This is the reason why we only plan a single virtual
machine migration at a time. So it's better to use this
algorithm with `CONTINUOUS` audits.
"""
from __future__ import division
@ -76,7 +47,7 @@ class WorkloadBalance(base.WorkloadStabilizationBaseStrategy):
* Software: Ceilometer component ceilometer-agent-compute running
in each compute node, and Ceilometer API can report such telemetry
"cpu_util" and "memory.resident" successfully.
* You must have at least 2 physical compute nodes to run this strategy
* You must have at least 2 physical compute nodes to run this strategy.
*Limitations*

23
watcher/decision_engine/strategy/strategies/workload_stabilization.py
View File

@ -16,18 +16,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
*Workload Stabilization control using live migration*
This is workload stabilization strategy based on standard deviation
algorithm. The goal is to determine if there is an overload in a cluster
and respond to it by migrating VMs to stabilize the cluster.
This strategy has been tested in a small (32 nodes) cluster.
It assumes that live migrations are possible in your cluster.
"""
import copy
import itertools
@ -59,7 +47,16 @@ def _set_memoize(conf):
class WorkloadStabilization(base.WorkloadStabilizationBaseStrategy):
"""Workload Stabilization control using live migration"""
"""Workload Stabilization control using live migration
This is workload stabilization strategy based on standard deviation
algorithm. The goal is to determine if there is an overload in a cluster
and respond to it by migrating VMs to stabilize the cluster.
This strategy has been tested in a small (32 nodes) cluster.
It assumes that live migrations are possible in your cluster.
"""
MIGRATION = "migrate"
MEMOIZE = _set_memoize(CONF)

13
watcher/decision_engine/strategy/strategies/zone_migration.py
View File

@ -11,13 +11,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
*Zone migration using instance and volume migration*
This is zone migration strategy to migrate many instances and volumes
efficiently with minimum downtime for hardware maintenance.
"""
from dateutil.parser import parse
import six
@ -48,7 +41,11 @@ IN_USE = "in-use"
class ZoneMigration(base.ZoneMigrationBaseStrategy):
"""Zone migration using instance and volume migration"""
"""Zone migration using instance and volume migration
This is zone migration strategy to migrate many instances and volumes
efficiently with minimum downtime for hardware maintenance.
"""
def __init__(self, config, osc=None):