Since Ussuri, nova can be configured with new policy
defaults. These defaults enforce the need to use system-scoped
tokens for admin operations like listing hypervisors.
The new os_system_scope configuration allows masakari to
request a system-scoped token using keystoneauth1 to query nova.
Implements: blueprint support-nova-system-scope-policies
Change-Id: I5f5b3f05358c28d60cfd05bc62a388f9087d75ac
Nova services and hypervisor naming can differ, as they retireve node
names in different way.
In the meanwhile we operate with nova.services while enabling/disabling
nodes duringh the incident. So we're supposed to have in database record
matching to what we have in service list, but not in hypervisor list.
Closes-Bug: #1839715
Change-Id: I9c591d33f17a8d5950bdb1fc2d686e2301fc6d95
Convert all code to not require six library and instead
use python 3.x logic.
We don't need this in a Python 3-only world.
Depends-On: Ibfee28132e102c40d0eb5b8dc73228ec8fb9ce98
Depends-On: Ibe08ffa9072f1f0c1090f2b9e2ab9af59474d6b1
Depends-On: Ic4d00d081699015ccc2d529b872a17565c881c4e
Depends-On: I2995305abf7cb704f0fe4db9634c87ea2ed378b4
Depends-On: Id9192849a78d57ecfefe1dc20f931406884e055e
Depends-On: I0147dc00943e13531ffb5e2701a91c2d2838958b
Change-Id: I764e61b86cdcaa9e74b1281344599f2d3a13d68f
Co-Authored-By: zhoulinhui <df.some@foxmail.com>
If a custom CA file is configured via nova_ca_certificates_file,
currently communication with Keystone will fail, since the session is
not created using this CA file. The same is true for nova_api_insecure.
This change fixes the issue by using a keystoneauth session loader.
Closes-Bug: #1873736
Change-Id: I54a4f398a06c61e0f6f8f3efd3e4b6214bca756b
Presently, you can update host name with any non-existing host name.
It doesn't check whether this host exists or not in nova. This patch
fixes this issue by checking whether the host name exists in nova or
not before updating it. If it doesn't exists, it will raise 400 error.
APIImpact
BadRequest(400) is returned if host doesn't exists in nova instead of
200 during host update.
Closes-Bug: #1814656
Change-Id: Ibd113f2328deae0f1114544436631bdc434eff92
nova_api_version has been bump to 2.53 in bug/1800073,
with this version, nova-client only take service_uuid to disable service,
args host_name and binary are no longer supported.
Change-Id: I6ab942f657f8983a22c9e16747090399c01fc3f8
Closes-bug: 1811742
Presently, you can add any hostname to the failover segment. It
doesn't check whether this host exists or not in nova. This patch
fixes this issue by checking whether the hostname exists in nova or
not before adding it to the failover segment. If it doesn't exists,
it will raise 400 error.
APIImpact
BadRequest(400) is returned if host doesn't exists in nova instead
of 200 during host create.
Closes-Bug: #1800073
Change-Id: I0ccc9f3a61e3f17f2cb7c1ad1888947c6fe724c8
Masakari has never used service_type and service_name of
nova_catalog_admin_info. However, these can be specified
when masakari creates a novalient object.
This patch modified so that masakari uses these.
Change-Id: I7fdd3b47230bea004a9129234c88d20d611814bf
Closes-Bug: #1724764
Domain name is needed when using keystone v3 to create keystoneauth
session[1], otherwise the following error will be raised:
InvalidInput: Invalid input received: Expecting to find domain in
project - the server could not comply with the request since it is
either malformed or otherwise incorrect
[1]: https://docs.openstack.org/keystoneauth/latest/authentication-plugins.html#v3-identity-plugins
Change-Id: I557a7107b51ae4ffab15d045a4be6e3ed1940bd8
Closes-bug: #1710570
This patch adds the global_request_id to the constructor for nova
client, which will pass the global_request_id into nova services
on all API calls. Supporting global_request_id makes debugging [1]
easier when request touches many services in cloud. The masakari
request-id will be sent to nova in the request header like below and
it will be available with context.global_id:
-H "X-OpenStack-Request-ID: req-1a9b7b24-02ed-4400-bcc3-cc1bcbb59147"
Masakari is already using newer python-novacliant which supports
global_request_id [2].
This patch also fixes the below debug log message which gets emitted
when any argument gets dropped while creating request context:
"Arguments dropped when creating context: {u'global_request_id': None}"
As decided earlier while fixing the issue [3] this patch adds the
'global_request_id' to the base RequestContext hence removes the
earlier debug log message.
[1] I65de8261746b25d45e105394f4eeb95b9cb3bd42
[2] I5b247f75edeea9da50fe524eadf5f9a2c626d665
[3] d4dd11d7bd
Change-Id: I2139976f6774b10518c7455a9af1b32b1e7b3e7d
Starting since version 2.14, Nova automatically detects whether the
server data is on shared storage or not.
Removed 'on_shared_storage' parameter from nova evacuate call and
bumped nova api version from 2.9 to 2.14 so that shared storage
deployment can be detected by nova. Also added a related note in
README.rst to point out. Operators should configure shared storage
to use maskari otherwise instance data will be lost after evacuation.
Change-Id: I0b0581a5c84143fc91c9fc6e2c440096013c7438
Masakari can face a race condition where after evacuation of an
instance to other host user might perform some actions on that
instance which gives wrong instance vm_state to ConfirmEvacuationTask
that results into notification failure.
To fix this issue this patch proposes to lock the instance before
evacuation till its confirmation so that any normal user will not
be able to perform any actions on it. To achieve this the
ConfirmEvacuationTask is completly removed and the confirmation is
done in the EvacuateInstancesTask itself by per instance.
Evacuating an instance and confirming it's evacuation immediately
can reduce the performance so this patch uses the
eventlet.greenpool.GreenPool which executes the complete evacuation
and confirmation of an instance in a separate thread.
To check if the server is already locked or not upgraded the
novaclient's NOVA_API_VERSION from 2.1 to 2.9 as the 'locked'
property is available in nova api_version 2.9 and above.
This patch introduces a new config option
'host_failure_recovery_threads' which will be the number of threads
to be used for evacuating and confirming the instances evacuation.
The default value for this config option is 3.
Closes-Bug: #1693728
Change-Id: Ib5145878633fd424bca5bcbd5cfed13d20362f94
When host-failure occurs, masakari-engine adds reserve_host
to aggregate.
However, when masakari-engine adds reserved_host,
masakari-engine passes an aggregate_name to novaclient.
This patch is modified so that masakari-engine passes
aggregate_id instead of aggregate_name to novaclient.
Change-Id: I669b19dea04c8ebb3a27a8ae746ae4c3f88d66f0
Closes-Bug: #1667246
Reserved hosts can be shared between multiple host_aggregates. So
before evacuating the instances from failed_host to reserved_host,
the target resered_host should be added to the same aggregate in
which the failed_host is.
This patch adds the reserved_host to failed_host's aggregate.
Adding reserved_host to aggregate is optional and can be configured
by operators with the help of new configuration parameter
'add_reserved_host_to_aggregate' which is added under the 'host_failure'
section. This config option defaults to 'False'.
Change-Id: I7478e0f24ecd6fd6385dd67e7f0cad5ca3460526
Added _process_unfinished_notifications to process notifications
which are in error or new state. This periodic task will execute at
regular interval defined by new config option
'process_unfinished_notifications_interval' defaults to 120 seconds.
The notifications which are in ‘new’ status will be picked up based
on a new config option ‘retry_notification_new_status_interval’ defaults
to 60 seconds.
Implements: bp add-periodic-tasks
Change-Id: I6e607d83f04618ad695a9614f84ad690b8804848
Nova community has removed support of HTTPClient [1] thus they have
removed service_catalog as well. Services which are going to call nova
has to use SessionClient only. It is recommended to use
keystoneauth1.service_catalog and not nova_client.service_catalog as it
is removed in commit [1].
We are already using SessionClient. This patch uses the service_catalog
from keystoneauth1.
NOTE:
novaclient will be called only with admin_endpoint and privileged_user.
So there is no need to form url for normal user. So removed unused code
from masakari.compute.nova.py
[1] 2e5576dafc21906a95196054f85c80b57c48663a
Closes-Bug: #1653052
Change-Id: I2a28eb94e190cd0416d0d7ea56af8f7ca76dbae5
Implement taskflow for instance failure workflow which will
stop and start the failed instances.
For process failure notification it will set host on_maintenance
to True and disables compute service running on that host.
Change-Id: I45d772fa4502ca914141f650c85c192989b61439
sue