* Drop use of stack abandon: just use a regular delete instead
* Return stack name from force delete API call
Change-Id: I33ee7323fade1b237957abb8f7c79b87eb20148f
* Add force delete operation to engine using stack abandon
* Basic unit tests for force delete
* Necessary removal of "direct engine" code
* Change schema of APIv2 cluster delete to allow force delete
* Unit tests for new cluster delete schema
* Basic docs about force delete
* Release note
bp sahara-force-delete
Partial-Bug: #1647411
Change-Id: Ida72677c0a4110cb78edf9d62d8330cd4608ff76
Adding the option to decommission specific node from cluster.
Partially implements bp: decommission-specific-instance
Change-Id: I1a858fecc1b32f91c76aea6db14e0d5a419211d1
Create a cluster more than 10 slave nodes, or create a cluster
named against the first half of the alphabet and scale up it
after renaming the cluster named against the second half, and
then scale down the cluster. Cluster will go to Error
"No router to host".
The root cause is that the record in instances schema doesn't
sync with nova instances. Heat deletes instances according to
the index of instances, so sahara should keep with heat to delete
the same on.
Change-Id: I05182a7f7b37a66caf45d56716f7b84a93c5c7f2
Closes-Bug: #1713012
Log messages are no longer being translated. This removes all use of
the _LE, _LI, _LW and _LC translation markers to simplify logging and to
avoid confusion with new contributions.
See:
http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.htmlhttp://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html
It will be a giant patch if all these _LE, _LI, _LW and _LC being deleted
within one patch, so this patch only delete _LW, _LC and _LE has been
handled in previous patch; deletion of _LI will be in the following patches
related with this one.
Change-Id: Ia696ee76f22e488b9a219305f4f220eca2161cee
Introduces a new field called anti_affinity_ratio.
Creates more server groups in case the
user defines the anti_affinity_ratio more than 1.
Default value of anti_affinity_ratio is kept 1.
Instances are placed in the server groups in round-robin
fashion.
Partially-Implements: blueprint improving-anti-affinity
Change-Id: I86bda256ea661fa0ba9d6d3819477edff080bcee
This patch implements designate integration. Added:
* 'use_designate' and 'nameservers' configs
* 'domain_name' column to Cluster and Cluster Templates and
'dns_hostname' to Instance in DB
* get_management_ip() method for Instance for properly setup
hostnames in cluster info and use it in ambari, cdh, mapr, spark,
storm and vanilla plugins
* method for change /etc/resolv.conf on all cluster instances
according to 'nameservers' config
* method for generate heat template for designate A and PTR records
* validation check for cluster template creation because now it
contains new field - 'domain_name'
* fix unit tests appropriatly and new tests added
* updated heat_template_version to '2016-04-08' (because we use
'str_split' method)
NOTE: in spec we proposed to use two domains: one for internal
resolution and another for external. But we decided to use only one
domain for both sides so an instance will have one hostname in the
domain and two appropriate A records because it's more convenient.
bp: designate-integration
Change-Id: I66525c0305450d8ba08840be1c95bfe79f21fff7
Sometimes, status of heat stack is not changed in DB
right after executing updating of heat stack. Because of
that sahara thinks that heat stack was successfully updated,
but it's wrong, because heat stack updating still in progress.
Because of that it's proposed to use field updated_time
for verification of successful update of heat stack.
We will store that field in variable, and will wait until that
will be changed.
Closes-bug: 1472536
Change-Id: Ifeee9a81ff1b37ff061de0ded371ae6f6bda4c95
We should define a set of CLUSTER_STATUS in stead of using direct string
in code.
1. Add cluster.py in utils/
2. Add cluster status.
3. move cluster operation related methods from general.py to cluster.py
Change-Id: Id95d982a911ab5d0f789265e03bff2256cf75856
There is no sense in separate Launcher class that inherited from
Engine to have access to protected methods. This is just method
call. So, moved all class vars to arguments. Code became much
more clear.
Change-Id: I436f38cd8424158b10783c70f9ad43d49c8c2f28
Partial-Bug: #1459788
Suppose we have following scenario. I created cluster with
direct engine, after that I switched to Heat engine and removed
cluster. After that, cluster will be removed from Sahara DB,
but instances will be alive. This patch propose to add following:
if we try to delete direct-engine-created cluster with heat engine,
we will try to delete cluster in same way, like direct engine do it.
Related blueprint: deprecate-direct-engine
Change-Id: I65e22a8090560763547960213859b54b30f37f7e
Sahara now uses two layers of ResourceGroups:
1. Each node group is represented as ResourceGroup with specified
count.
2. Volumes for the instance are also represented as ResourceGroup
with a specified count.
Note: Heat creates two nested stacks for ResourceGroup (one for
nested group, other to support multiple copies). This means that
'max_nested_stack_depth' config in Heat must be at least 4.
Also, there is no sense in checking exact structure of resulting
Heat templates. But there is a sense in checking specific methods.
So, deleted all template tests and added test for security_groups.
Implements blueprint: heat-template-decomposition
Change-Id: I1bc1ea31344a2e4d624bbf2836aa51897453efe2
Attached volumes is a technical information and should not be in
cluster output. Moreover this informatio is not used by Sahara in
most cases. Currently it is used by direct engine only to detach
volumes it attached before. After removing direct provisioning
engine "volumes" table should be removed too.
Change-Id: I10d99aed74be1126065be68082673a2e8aa1027c
Closes-Bug: #1459447
All heatclient calls wrapped in execute_with_retry
method to avoid occasional errors
partially implements bp clients-calls-retry
Change-Id: I835fd79b5783f4a5271cec67b013ba8aeba3091a
Currently some ids in Sahara log messages are noticed twice.
All this messages should be checked and fixed to avoid information
duplication.
partially implements bp: logs-improvement
Change-Id: I1fd3f8b76ed6f56d9e8116ea453d05246a68f7dc
Swiftched all JSON templates to YAML templates. Since Sahara works
with objects now there are no much changes.
Implements blueprint: heat-hot
Change-Id: I2ac13e08eb4af0848f42b20b032af425d30818c7
During log rewriting process we mention some places where we should add
'Warning' and 'Info' level messages within messages we have at 'Debug'
level.
Closes-Bug: 1435208
Change-Id: Ib8bfc29c6f8c9ad5392ff09fa29a14e1209b3cc7
This is an initial commit to make sahara logs relevant to new
logging spec.
We need to change some log levels to become closer to the spec, also
we need to change some log messages (e.g. for INFO level).
partially-implement bp new-style-logging
Change-Id: I8e7b7c4c3f375648d2999b1d0e19b68390cc22a8
Now heat engine works in a strange way. There are .heat files that
contain parts of JSON with placeholders. Placeholders are embedded
without logic. Sometimes it is just variable, sometimes JSON
key-value representation, sometimes the whole line including
trailing comma.
These .heat templates loaded by heat engine and after
placeholders replacement they are placed as strings to a large
template. These three step string manipulations are completely
non-transporent and lead to errors.
The proposed change removes .heat templates and switches heat
engine to work with objects. Resulting template is dict now.
This change removes a lot of dances with types and makes code
more straightforward.
Change was done to leave heat engine resulting template without
any changes. That's why there are unnessary type castings to
string sometimes.
The only change in output is a "userdata" field. There is no sense
in using "Fn::Join" there.
Also swapped expected and actual arguments in corresponding tests.
Related blueprint: heat-template-decomposition
Change-Id: Ib3cc152c9c3e146403eb9b52947af4c676044365
* Extracted heat template generation from
'utils/openstack/heat.py' to 'service/heat/templates.py'.
Now heat.py is for heat client only.
* Moved heat template resources from 'resources' to
'service/heat/resources'
* Separated tests for heat templates and heat client.
Change-Id: I4c7a561f1648f34e71574a556cbc07ed2cf9b173
Closes-Bug: #1373075
Jeremy Freudberg