Attached volumes already had support for explicit volume type,
availability zone, and instance locality. Extend this capability to
boot volumes.
(For posterity's sake, a summary of why other options related to
attached volumes were not in fact appropriate for boot volumes:
- volumes_per_node: there can only be one boot volume
- volumes_size: boot volume size given by flavor
- volume_mount_prefix: boot volume handled differently)
Change-Id: I009c0da4179c880fd87a7c1903c93825e8997a38
Story: 2004507
Task: 28229
Also switch to pycodestyle from pep8 which requires a bump
of the the version of flake8 too.
Skip the following checks for now:
- E123 Closing bracket does not match indentation of opening bracket's line
- E226 Missing whitespace around arithmetic operator
- E402 Module level import not at top of file
- E731 Do not assign a lambda expression, use a def
- W503 Line break occurred before a binary operator
They should be probably revisited and fixed, if possible,
or skipped individually using # noqa
The following checks reported a limited number of errors and
they were fixed directly in this review:
- E241 Multiple spaces after ','
- E501 Line too long (82 > 79 characters)
Change-Id: I1f185d2efd1adf27a26e3ac93f2e1011c0b63124
Sahara anti-affinity was broken since some problematic changes in
cd15698526.
This should be able to fix it.
Change-Id: I374c0340cb0f85c00b9a04cd1b23e3912737994c
Co-Authored-By: Joe Topjian <joe@topjian.net>
Story: #2002656
Task: #22466
Adding the option to decommission specific node from cluster.
Partially implements bp: decommission-specific-instance
Change-Id: I1a858fecc1b32f91c76aea6db14e0d5a419211d1
Nova network has been fully removed, remove use_neutron config
and related codes.
Change-Id: Ib9d87dd339d637b69fb27315d92228cbc523c8eb
Closes-Bug: #1693891
Implements: bp sahara-remove-nova-network
Got an unbound variable error without this fix:
UnboundLocalError: local variable 'sec_groups'
referenced before assignment
This appears to have been caused by the move to use the neutron client
in the following commit:
b4fb2c404c
Co-Authored-By: Jeremy Freudberg <jeremyfreudberg@gmail.com>
Closes-Bug: #1718940
Change-Id: I4f0b5a6b27ee2fb7821ec42faba3a69ce5a4bb78
Latest novaclient 8.0.0 removed nova-network
related functionality which Sahara depended on for
network operations, even for Neutron.
Now, use neutronclient (and Neutron resources in
Heat) for all network operations.
Additionally, fix devstack coordination backend.
Partially-Implements: bp sahara-remove-nova-network
Closes-Bug: #1692551
Co-Authored-By: Vitaly Gridnev <vgridnev@mirantis.com>
Change-Id: I6b8e12f0d33752e7790c5f40ebe6a3940ff1b0c1
Introduces a new field called anti_affinity_ratio.
Creates more server groups in case the
user defines the anti_affinity_ratio more than 1.
Default value of anti_affinity_ratio is kept 1.
Instances are placed in the server groups in round-robin
fashion.
Partially-Implements: blueprint improving-anti-affinity
Change-Id: I86bda256ea661fa0ba9d6d3819477edff080bcee
This workaroud adds appropriate entries for orchestration service
in /etc/hosts file
Change-Id: Iaf1b60d4d891e90ee6259fb62654476819f93ec2
co-authored-by: Vitaly Gridnev <vgridnev@mirantis.com>
closes-bug: 1583617
This patch implements designate integration. Added:
* 'use_designate' and 'nameservers' configs
* 'domain_name' column to Cluster and Cluster Templates and
'dns_hostname' to Instance in DB
* get_management_ip() method for Instance for properly setup
hostnames in cluster info and use it in ambari, cdh, mapr, spark,
storm and vanilla plugins
* method for change /etc/resolv.conf on all cluster instances
according to 'nameservers' config
* method for generate heat template for designate A and PTR records
* validation check for cluster template creation because now it
contains new field - 'domain_name'
* fix unit tests appropriatly and new tests added
* updated heat_template_version to '2016-04-08' (because we use
'str_split' method)
NOTE: in spec we proposed to use two domains: one for internal
resolution and another for external. But we decided to use only one
domain for both sides so an instance will have one hostname in the
domain and two appropriate A records because it's more convenient.
bp: designate-integration
Change-Id: I66525c0305450d8ba08840be1c95bfe79f21fff7
Sahara used to write Heat template to log output in format
which cannot be parsed with JSON or YAML validation tools.
As a result it caused difficultis for a user to read the
initial template.
This issue is solved by making Sahara write logs in json format.
Closes-bug: 1552239
Change-Id: Ie7a9f6fcad29899dd53fa43884e783c58ea304ec
To have ability of creating clusters on deployments
with Ironic we should avoid using precreated ports. Now
to create floating ip we will use Nova floaiting ip
serializations.
Closes-bug: 1557066
Change-Id: I64eef2d645edf12022f686fac14cf04227eedb86
rewriting wait condition script in the following manner:
* trying to send signal until successed, because sometimes
heat can be unavailable;
* insecure flag passed to allow wait condition work when
SSL enabled. Since this signal don't have any important
info, we can use this flag until secured way to pass certificate
will be invented.
Change-Id: I6a8003af8ae655ed49c7f93dacc8420d7d7a5054
Closes-bug: 1532766
Closes-bug: 1535274
actually heat template version is an important constant.
to make that same everywhere moving that to common configs.
Change-Id: I1af5147e4113eda11328a7b521c07d2b798eaaec
Server group is not defined inside of ResourceGroup
for node group and because of that heat can't validate
heat template. To fix that we introduced new parameter
in ResourceGroup for node group for correct id of Server
Group.
Closes-bug: 1537156
Change-Id: I133ba075fe6a989f786b1cc51ebfd126a1c2a6a9
Set heat_enable_wait_condition to True by the default. Also
disable that in gate-sahara-dsvm-scenario-nova-heat job.
Related-bug: 1516784
Change-Id: If2e8a4ad010d516f65e2f9ae4449d70cc30d8e44
It looks like wait condition feature works nice
but there are some issues with that on several deployments.
Added option heat_enable_wait_condition to allow disabling of that.
Closes-bug: 1516784
Change-Id: I082ce0bb4ab8c3c939dcafc2bc821e38c5d760a7
It looks like this serialization introduced
because of some rebase, anyway it's clear that
it's unneeded.
Closes-bug: 1494650
Change-Id: If0140ca9f22fb3073fafccada512c877322efc2d
If auto_security_group is disabled, but one or more
security groups is provided, cluster can't be created
and fails with yaml RepresenterError exception.
Change-Id: I294237c788c2d14d77f12e1a8d7e0dd60a6638bb
Closes-bug: #1504093
Sometimes, status of heat stack is not changed in DB
right after executing updating of heat stack. Because of
that sahara thinks that heat stack was successfully updated,
but it's wrong, because heat stack updating still in progress.
Because of that it's proposed to use field updated_time
for verification of successful update of heat stack.
We will store that field in variable, and will wait until that
will be changed.
Closes-bug: 1472536
Change-Id: Ifeee9a81ff1b37ff061de0ded371ae6f6bda4c95
Heat supports wait condition for reporting signals.
We use wait condition for getting status of VM.
If VM send signal to Heat this is means that VM booted and accessible
via SSH.
implements bp: sahara-heat-wait-conditions
Change-Id: I6f0b52840359bb46144b31fee245f0c104f1686e
Since sahara uses HOT since liberty, we should switch on
get_resource usage in heat templates.
Change-Id: I2ba29d82654779406915311e391181926e9f59a5
Closes-bug: 1494673
If volumes_per_node or volumes_size is 0, there is no sense
in creating empty ResourceGroup.
Also remove volumes from one node group in gate to be sure
that sahara correctly handle such case.
Closes-bug: 1494650
Change-Id: Ic592710773b9338f5857e431b0567bc5c87af2bd
Heat infra-engine uses AWS::EC2::SecurityGroup.
But AWS::EC2::SecurityGroup for Neutron doesn't support IPv6.
This patch migrate from AWS::EC2::SecurityGroup to
OS::Neutron::SecurityGroup, for Neutron.
Change-Id: I04e4aacdd843d3efa388f9527efec2e141eabbbb
Closes-Bug: #1488452
Having files as member field reduces number of arguments across all
functions.
Change-Id: I7e2652796ac0f4a5abc9c74b36cb4ec55a82c55f
Partial-Bug: #1459788
Sahara now uses two layers of ResourceGroups:
1. Each node group is represented as ResourceGroup with specified
count.
2. Volumes for the instance are also represented as ResourceGroup
with a specified count.
Note: Heat creates two nested stacks for ResourceGroup (one for
nested group, other to support multiple copies). This means that
'max_nested_stack_depth' config in Heat must be at least 4.
Also, there is no sense in checking exact structure of resulting
Heat templates. But there is a sense in checking specific methods.
So, deleted all template tests and added test for security_groups.
Implements blueprint: heat-template-decomposition
Change-Id: I1bc1ea31344a2e4d624bbf2836aa51897453efe2
All heatclient calls wrapped in execute_with_retry
method to avoid occasional errors
partially implements bp clients-calls-retry
Change-Id: I835fd79b5783f4a5271cec67b013ba8aeba3091a
Swiftched all JSON templates to YAML templates. Since Sahara works
with objects now there are no much changes.
Implements blueprint: heat-hot
Change-Id: I2ac13e08eb4af0848f42b20b032af425d30818c7
* added support to heat engine
* added support to direct engine
* added unit tests for heat templates
partially implements bp: volume-instance-locality
Change-Id: I9fde3967c46942c8adcbca31300e948b4d0e228e
Jeremy Freudberg