Commit Graph

410 Commits

Author SHA1 Message Date
Lucian Petrut 424e9a76af vm workload consolidation: use actual host metrics
The "vm workload consolidation" strategy is summing up instance
usage in order to estimate host usage.

The problem is that some infrastructure services (e.g. OVS or Ceph
clients) may also use a significant amount of resources, which
would be ignored. This can impact Watcher's ability to detect
overloaded nodes and correctly rebalance the workload.

This commit will use the host metrics, if available. The proposed
implementation uses the maximum value between the host metric
and the sum of the instance metrics.

Note that we're holding a dict of host metric deltas in order to
account for planned migrations.

Change-Id: I82f474ee613f6c9a7c0a9d24a05cba41d2f68edb
2023-10-27 21:54:42 +03:00
Lucian Petrut 00fea975e2 Handle deprecated "cpu_util" metric
The "cpu_util" metric has been deprecated a few years ago.
We'll obtain the same result by converting the cumulative cpu
time to a percentage, leveraging the rate of change aggregation.

Change-Id: I18fe0de6f74c785e674faceea0c48f44055818fe
2023-10-24 10:47:23 +00:00
chenker c7be34fbaa update saving_energy docs
Change-Id: I3b0c86911a8d32912c2de2e2392af9539b8d9be0
2023-02-07 10:27:54 +00:00
wangjiaqi07 c55143bc21 remove unicode from code
Change-Id: I747445d482a2fb40c2f39139c5fd2a0cb26c27bc
2022-08-19 14:17:10 +08:00
Dantali0n 2414f66e38 Add watcher dashboard to devstack documentation
Since installing watcher dashboard is fixed in devstack deployments
we can update documentation so it recommends to install dashboard
plugin.

Change-Id: I284a1ec31536ea258cc1979ffd46b22d3e1ac18b
2021-07-09 10:37:28 +02:00
Ghanshyam Mann 863815153e [goal] Deprecate the JSON formatted policy file
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:

1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.

2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.

Also replace policy.json to policy.yaml ref from doc and tests.

[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: I207c02ba71fe60635fd3406c9c9364c11f259bae
2021-02-12 19:59:27 +00:00
xuanyandong 16a0486655 Remove six
Replace the following items with Python 3 style code.

- six.string_types
- six.integer_types
- six.moves
- six.PY2

Implements: blueprint six-removal

Change-Id: I2a0624bd4b455c7e5a0617f1253efa05485dc673
2020-09-30 16:25:13 +08:00
Andreas Jaeger 1ff940598f Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems

Update Sphinx version as well.

Set openstackdocs_pdf_link to link to PDF file. Note that
the link to the published document only works on docs.openstack.org
where the PDF file is placed in the top-level html directory. The
site-preview places the PDF in a pdf directory.

Set openstackdocs_auto_name to False to use 'project' variable as name.

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.

Remove docs requirements from lower-constraints, they are not needed
during install or test but only for docs building.

openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: Ia9a3fb804fb59bb70edc150a3eb20c07a279170b
2020-05-21 15:15:16 +00:00
zhangbailin f0f15f89c6 Remove future imports
These particular imports are no longer needed in a Python 3-only world.

Change-Id: I5e9e15556c04871c451f6363380f2a7ac026c968
2020-05-02 00:33:39 +00:00
licanwei de9d250537 update description about audit argument interval
Change-Id: I5ae8ab672edac1637c2bef4201fec30e896cd8ed
2020-04-13 08:31:11 +00:00
licanwei 3a3a487c71 remove wsmeext.sphinxext
Error when importing wsmeext.sphinxext
Could not import extension wsmeext.sphinxext
(exception: cannot import name 'l_')

Change-Id: Id23c9c1fd35153d67d4ffb50dc1cd40f30b7ab41
2020-04-13 11:27:45 +08:00
zhangbailin 6a0fe94e5c Block Sphinx 3.0.0
Sphinx 3.0.0 breaks the building here, block it for now.

Depends-On: https://review.opendev.org/#/c/717949/

Change-Id: Ibf0c93ea79fec647fbf749257835f1fa99d5f59d
2020-04-08 06:20:24 +00:00
licanwei 60a3f1f072 Removed py27 in testing doc
Change-Id: Ib7e45aec73c4d3b11eaf5288d739edad3b12c4ee
2020-03-23 10:22:31 +08:00
licanwei e71aaa66db simplify doc directory
Change-Id: I11bfa4d3bcb7c01ef638c0fa97cb872e96698e29
2020-02-17 17:19:13 +08:00
Zuul 11d55bc9fc Merge "Doc: Add EVENT audit description" 2020-02-13 07:04:44 +00:00
Zuul 3e4eda2a80 Merge "Community Goal: Project PTL & Contrib Docs Update" 2020-02-10 14:44:40 +00:00
licanwei 9b6629054a Doc: Add EVENT audit description
Change-Id: Ia6db6f8c21282a4755997cf47fd618670148c23f
Implements: blueprint event-driven-optimization-based
2020-02-10 11:13:06 +08:00
licanwei 56b2f113ed Community Goal: Project PTL & Contrib Docs Update
Change-Id: I07de7b94ed51eebc31886793aa5a1e87353dfbc6
Story: #2007236
Task: #38570
2020-02-10 10:09:23 +08:00
licanwei 05e81c3d88 doc: move Concurrency doc to admin guide
Change-Id: Ia1b034a5f79a5c7eeffdba2df727fd26cf13d1cc
2020-02-03 09:55:23 +00:00
licanwei ae83ef02e7 doc for event type audit
Partially Implements: blueprint event-driven-optimization-based

Change-Id: I11211b606afd55dfa46a0942132be58dc30e28a4
2020-01-14 17:07:24 +08:00
licanwei 91b58a6775 Move install doc to user guide
Change-Id: I7b4b4ddffbe66a00fdcec4d497c6efa2e9e7729e
2020-01-11 10:16:27 +08:00
licanwei 693d214166 Update user guide doc
Change-Id: I881b429552e15f13ddcd0ccf1663fb0e2f4123aa
2020-01-09 19:09:15 +08:00
Zuul 42fea1c568 Merge "Documentation on concurrency for contributors" 2019-11-30 02:59:19 +00:00
Dantali0n b5f8e9a910 Documentation on concurrency for contributors
Documentation with details on general concurrency as well as OpenStack
specific libraries.

Describes how different libraries are effectively used across different
Watcher services. This includes describing how futurist is used in the
Decision Engine and how taskflow is used in the Applier.

Finally, this documentation describes how contributors can use the
new DecisionEngineThreadpool effectively and includes examples.

https://docs.openstack.org/futurist/latest/
https://docs.openstack.org/taskflow/latest/

Change-Id: Ic1cd1f3733a0e9a239c9b8d49951e1e4ece49f3a
Partially Implements: blueprint general-purpose-decision-engine-threadpool
2019-11-27 08:48:16 +01:00
Ghanshyam Mann 17f5a65a62 [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in ussuri cycle.

Watcher is ready with python 3 and ok to drop the
python 2.7 support.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Depends-On: https://review.opendev.org/#/c/693631/

Change-Id: I603c6d2c22779e8ef2e70eb6369fc521a77c9c3a
2019-11-16 14:55:01 +00:00
jacky06 7d2191d4e6 tox: Keeping going with docs
Sphinx 1.8 introduced [1] the '--keep-going' argument which, as its name
suggests, keeps the build running when it encounters non-fatal errors.
This is exceptionally useful in avoiding a continuous edit-build loop
when undertaking large doc reworks where multiple errors may be
introduced.

[1] https://github.com/sphinx-doc/sphinx/commit/e3483e9b045

Change-Id: If2bbfd8ae6d1fc75cbc494578310c1dc03c367e6
2019-10-24 00:45:42 +00:00
Zuul 0b21ae333f Merge "Build pdf docs" 2019-09-18 02:00:24 +00:00
licanwei ffd8e27bd1 Build pdf docs
Add a new pdf-docs environment to enable PDF build.
 sphinxcontrib-svg2pdfconverter is used to handle SVG properly.

Change-Id: I1563579486da8912ba8a220bb08a5331e7df910b
2019-09-16 23:09:25 -07:00
licanwei 10d8a5fbc2 Add node_resource_consolidation doc
Partially Implements: blueprint node-resource-consolidation
Depends-on: Ia979781b32202c1821aa1cb91d24253fe6d7bd2d
Depends-on: I104c864d532c2092f5dc6f0c8f756ebeae12f09e

Change-Id: I8fa33a6559c9821e730f8e0babd8438ef45d7338
2019-09-05 00:58:58 -07:00
Andreas Jaeger 54344208fc Update api-ref location
The api documentation is now published on docs.openstack.org instead
of developer.openstack.org. Update all links that are changed to the
new location.

Note that redirects will be set up as well but let's point now to the
new location.

For details, see:
http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007828.html

Change-Id: I4101eced9c4bd26741f760e5651204f5d2dfea0f
2019-07-22 19:00:11 +02:00
chenke 8cb7158790 Replace human_id with name in grafana doc
This patch does two things:

1. replace instance's human_id with name.
2. remove ComputeNode human_id.

Now name field in Watcher Compute Data Model is availible.
Use name is better than human_id. For the reason, please see[1].

[1]. https://bugs.launchpad.net/watcher/+bug/1833665

Change-Id: I04f40e7d2a2bda48e9a362f9d0b23f449c40324e
2019-07-10 15:10:59 +08:00
pengyuesheng dd1800fbc1 Blacklist sphinx 2.1.0 (autodoc bug)
See https://github.com/sphinx-doc/sphinx/issues/6440 for upstream details
Depend-On: https://review.opendev.org/#/c/663060/

Change-Id: I1a5b87b5771b9fd8f7ccdd6fd83f7ed0d7bfef64
2019-07-01 14:59:18 +08:00
Zuul 550b10b586 Merge "Documentation configuring grafana datasource" 2019-06-27 06:30:32 +00:00
Dantali0n fa1642e323 Documentation configuring grafana datasource
Documentation for administrators on how to configure the Grafana
datasource.

Change-Id: I5d1d3129b5d225f0f2fc86d149c046f9aab94d47
2019-06-26 10:58:14 +02:00
licanwei 90291923a1 Update strategy doc
Ceilometer removed cpu_util metric in [1].
Another metric compute.node.cpu.percent need to set
compute_monitors option to cpu.virt_driver in the
nova.conf, we should remind user about these.
[1]: https://review.opendev.org/#/c/580709/

Change-Id: I89306ef7c26fa2927945bd4f3ee88b670511d147
2019-06-20 10:57:05 +08:00
Zuul 2a2f7902bd Merge "Improve DevStack documentation to support metrics" 2019-06-14 02:28:48 +00:00
licanwei 7f37f7b92a Remove apidoc
Now there are some errors when running apidoc,
actually we don't need apidoc, so remove it.
Closes-Bug: #1831515

Change-Id: I3b91a2c05ed62ae7bbd30a29e9db51d0e021410f
2019-06-04 11:34:07 +08:00
Matt Riedemann 7489126d83 Require nova_client.api_version >= 2.56
The [nova_client]/api_version defaults to 2.56 since
change Idd6ebc94f81ad5d65256c80885f2addc1aaeaae1. There
is compatibility code for that change but if 2.56 is
not available watcher_non_live_migrate_instance will
still fail if a destination host is used.

Since 2.56 has been available since the Queens version of
nova it should be reasonable to require at least that
version of nova is running for using Watcher.

This adds code which enforces the minimum version along
with a release note and "watcher-status upgrade check"
check method.

Note that it's kind of weird for watcher to have a config
option like nova_client.api_version since compute API
microversions are per API request even though novaclient
is constructed with the single configured version. It should
really be something the client (watcher in this case) determines
using version discovery and gracefully enables features if
the required nova API version is available, but that's a bigger
change.

Change-Id: Id34938c7bb8a5ca934d997e52cac3b365414c006
2019-05-23 15:49:19 -04:00
Dantali0n 5a35b30763 Improve DevStack documentation to support metrics
Support DevStack setups with datasources configured starting with
Gnocchi.

Change-Id: Ibcf0909ccce2dbb646c23a179ca763b6c3e62633
2019-05-21 15:36:46 +02:00
Zuul f731aa1748 Merge "Fix Stein version in watcher-status docs" 2019-05-21 07:17:33 +00:00
Zuul 2834610fa6 Merge "docs: fix link to install guide from user guide" 2019-05-15 15:48:00 +00:00
chenke 15c842fba2 Update Sphinx requirement
Sphinx 2.0 no longer works on python 2.7, so we need to start capping
it there as well.

The errors are as follow:

Requirement(package='sphinx', location='', specifiers='!=1.6.6,!=1.6.7,>=1.6.5'
does not match "python_version>='3.4'"
Requirement(package='sphinx', location='', specifiers='!=1.6.6,!=1.6.7,>=1.6.5'
does not match "python_version=='2.7'"
Could not find a global requirements entry to match package sphinx. If the package
is already included in the global list, the name or platform markers there may not
match the local settings.

Change-Id: I6dad56ffbb9e85e36cacea1a89565c2fc8248fbf
2019-05-15 11:03:48 +08:00
Matt Riedemann 3edafc9ba9 Fix Stein version in watcher-status docs
The final Stein version of Watcher was 2.0.0
so this fixes the version mentioned in the
watcher-status man page docs.

Change-Id: I7fce35471cf31222f9cdafc35e5a7b287bc4598e
2019-05-14 20:51:28 -04:00
Matt Riedemann 14d4cb54fa docs: fix link to install guide from user guide
This fixes the wrong installation guide link from the
user guide which was pointing to the watcherclient docs
for some reason, maybe it was just a copy/paste error.

Change-Id: I38f536e187245523ac37d70054a2df8cdfcbe4b2
Closes-Bug: #1828584
2019-05-10 11:37:28 -04:00
Zuul 583c9d8f6d Merge "Allow for global datasources preference from config" 2019-05-10 06:04:00 +00:00
Dantali0n bd8636f3f0 Allow for global datasources preference from config
Allows to define a global preference for metric datasources with the
ability for strategy specific overrides. In addition, strategies which
do not require datasources have the config options removed this is
done to prevent confusion.

Some documentation that details the inner workings of selecting
datasources is updated.

Imports for some files in watcher/common have been changed to resolve
circular dependencies and now match the overall method to import
configuration.

Addtional datasources will be retrieved by the manager if the
datasource throws an error.

Implements: blueprint global-datasource-preference
Change-Id: I6fc455b288e338c20d2c4cfec5a0c95350bebc36
2019-05-09 11:02:15 +02:00
chenke d2e1d69d37 Replace git.openstack.org with opendev.org
Change-Id: Ibccf32b71d307d9c80c91035907dc8292722ab31
2019-04-29 09:49:24 +02:00
zhulingjie 647b5e9483 Drop use of git.openstack.org
Our cgit instance will be going away and opendev.org is the new
preferred URL for browsing our git repos. Redirects will exist for the
foreseeable future, but it's more efficient to just go directly to the
new locations.

Change-Id: I7dd9d454da63167832bab02c89be98a2ce03b72a
2019-04-23 13:20:10 +02:00
Zuul 187b0e6a4b Merge "Update meeting schedule to new bi-weekly format" 2019-04-01 01:28:16 +00:00
Dantali0n 3b80b35270 Update meeting schedule to new bi-weekly format
Change-Id: I24bca9cd07262387282acc9ba458e9ef50587146
2019-03-26 16:36:34 +01:00