On the effort to make Sahara more user and operators friendly
we are splitting the plugins from Sahara core.
The main goal of this change is to facilitate installation,
maintainance and upgrade of plugins. With the plugins outside
the main Sahara code, operators will be able to install a subset
of plugins, as well as upgrade to newer versions of the plugins
without having to wait for a new version of OpenStack to be
released. As well, it aims to facilitate new contributors to
develop and maintain their own plugins.
Sahara Spec: https://specs.openstack.org/openstack/sahara-specs/specs/rocky/plugins-outside-sahara-core.html
Change-Id: I7ed0945fd82e37daaf6b29f947d3dba06db9d158
Log messages are no longer being translated. This removes all use of
the _LE, _LI, _LW and _LC translation markers to simplify logging and to
avoid confusion with new contributions.
See:
http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.htmlhttp://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html
It will be a giant patch if all these _LE, _LI, _LW and _LC being deleted
within one patch, so this patch only delete _LI; _LW, _LC and _LE has been
handled in previous patchies. This is the last patch for sahara to delete
log translations.
Change-Id: I73e99ca7fd5bc0cd8df7cce30b5f12e00e70149c
Adding support to labels to the Storm plugin as well as marking storm
version 0.9.2 as deprecated.
Change-Id: If58ddce1324341da48fb1d1d3da3d456316c1d10
Right now, storm cluster can exist exactly 1 nimbus node.
If user creates a cluster with 2 nimbus nodes,
"Missing nimbus service" will prompt.
The error message is ambiguous.
Change-Id: I674a23d9932659d56292ceb4340eb10b0bced727
This patch implements designate integration. Added:
* 'use_designate' and 'nameservers' configs
* 'domain_name' column to Cluster and Cluster Templates and
'dns_hostname' to Instance in DB
* get_management_ip() method for Instance for properly setup
hostnames in cluster info and use it in ambari, cdh, mapr, spark,
storm and vanilla plugins
* method for change /etc/resolv.conf on all cluster instances
according to 'nameservers' config
* method for generate heat template for designate A and PTR records
* validation check for cluster template creation because now it
contains new field - 'domain_name'
* fix unit tests appropriatly and new tests added
* updated heat_template_version to '2016-04-08' (because we use
'str_split' method)
NOTE: in spec we proposed to use two domains: one for internal
resolution and another for external. But we decided to use only one
domain for both sides so an instance will have one hostname in the
domain and two appropriate A records because it's more convenient.
bp: designate-integration
Change-Id: I66525c0305450d8ba08840be1c95bfe79f21fff7
Zookeeper path is hardcoded but we recently changed the path to avoid
having to update the path every time zookeeper changed versions. We
forgot to update this path in one place which causes the cluster to fail
to scale. The patch fix that
Change-Id: Idde36e39ac4c2e2e006b4a92d92e77440666c314
Closes-bug: #1607386
In order to use Pyleus we need to have it installed on image during image
generation process and have some configurations done.
This patch configures pylues on the master node.
Change-Id: Id45c4878e3a115284192a8a83196df74cc05dbe7
In order to allow user to develop storm topologies that we can also can
call storm jobs in a pure python form we are adding support to Pyleus in
Sahara.
Pyleus is a framework that allows the creation of Storm topologies in
python and uses yaml to wire how the flow is going to work.
Change-Id: I3a657164c91f1c7705d47882a1334925adf8db39
Partially-implements: blueprint python-storm-jobs
This is the last part of upgrading Strom to version 1.0.1.
Change-Id: I74660f5734100c04066e1a96f46cddf3f9de0f10
Partially-implements: blueprint upgrade-storm-1-0-1
* add hacking check: no_mutable_default_args()
* remove the mutable default arguments "[]" or "{}" when the function
is defined.
ref: http://docs.python-guide.org/en/latest/writing/gotchas/
Closes-Bug: #1530282
Change-Id: Ice6f7654211b73d7f8bc3ca0e4dfae3dca354397
This patch implements the get_open_ports method to allow storm to be used with
auto_security_groups.
Closes-bug: #1503709
Change-Id: I4939ee9c7d53f760ee654792e017c294a288be0f
get_instances will return a list, so we use zk_instances instead
of zk_instance. We remove the "if" check for method
_start_zookeeper_processes already checks zk_instances.
Change-Id: Iac48dcad3439871ab3bc8479bed715fe14625c1d
Implementation of Storm scaling. This will allow users to quickly scale
a storm cluster.
Change-Id: Id2f00e0aadc8aac5c79ebec5e55e13c2964c7fec
Implements: bp storm-scaling
Implementation of Storm EDP. This will allow users to submit Storm Jobs
via UI.
This patch implements the EDP engine, Storm Job Type.
Implements: bp storm-edp
Change-Id: I7d5937f26df715ef66826396a6387ab512af1f47
This is an initial commit to make sahara logs relevant to new
logging spec.
We need to change some log levels to become closer to the spec, also
we need to change some log messages (e.g. for INFO level).
partially-implement bp new-style-logging
Change-Id: I8e7b7c4c3f375648d2999b1d0e19b68390cc22a8
Sahara-image-elements removed zookeeper's version number from the folder
name. Storm needs to adapt to that.
Depends on the patch from sahara-image-elements #154889
Change-Id: Ibb39a7b10ea590d817e0ca4db8f6cce812c50687
Closes-bug: #1420838
Changes:
* using oslo_config instead of oslo.config
* using oslo_concurrency instead of oslo.concurrency
* using oslo_db instead of oslo.db
* using oslo_i18n instead of oslo.i18n
* using oslo_messaging instead of oslo.messaging
* using oslo_middleware instead of oslo.middleware
* using oslo_serialization instead of oslo.serialization
* using oslo_utils instead of oslo.utils
Change-Id: Ib0f18603ca5b0885256a39a96a3620d05260a272
Closes-bug: #1414587
oslo.log was added to global requirements, so we can migrate to this module.
Also we need this migration to be ensure that it works correctly with
log-improvements.
Since openstack.common.log is not dropped in oslo-incubator,
we shouldn't remove it.
Change-Id: I90468e4db812ae0b5d8a43a37206b236f8904661
Closes-bug: #1412673
The check for the port is not necessary in this context.
This patch removes it
Change-Id: Ie2b8e69ba7d017d66f214727dc06ded64dcb9215
Closes-bug: #1400822
Telles Nobrega