Excising Sahara content from Horizon

This plugin moves the current content from the horizon repo to this
plugin repo. The code has been tested in a devstack install using the
following steps:

    1. packaging the plugin: "python setup.cfg sdist"
    2. pip installing the tar.gz in the resulting dist directory
    3.  a. (temporary step) remove existing sahara enabled files from
            horizon
            "rm openstack_dashboard/enabled/_18*.py"
        b. finding the install location and changing to it
            "cp sahara_dashboard/enabled/* /opt/stack/horizon/local/enabled"
    4. in /opt/stack/horizon
        a. python manage.py collectstatic
        b. python manage.py compress
    5. restarting the horizon server

Additionally, you can run the unit tests by:
    ./run_tests.sh

    Note: added script to programmatically remove the old configuration
    files from the targeted horizon install, either in venv or system
    install.

Known issues:
  1. running tests locally emits missing neutron service messages.
  2. plugin code for devstack needs to be added
  3. README is inadequate
  4. integration tests are still in horizon repo
  5. local copy of run_tests is heavy weight, but a better solution is
     not available currently.
  6. localization tooling and strings

Change-Id: Icdce2d3e945e612d368556dd5cea1930194c7b67
This commit is contained in:
David Lyle 2015-11-20 19:05:25 -07:00
parent 305cf7afa6
commit 6c5898813c
204 changed files with 13666 additions and 19 deletions

View File

@ -173,3 +173,4 @@
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

4
MANIFEST.in Normal file
View File

@ -0,0 +1,4 @@
recursive-include sahara_dashboard *.html *.scss *.js
include AUTHORS
include ChangeLog

View File

@ -1,6 +1,19 @@
OpenStack Dashboard plugin for Sahara project
=============================================
How to use:
-----------
Use pip to install the package on the server running Horizon. Then either copy
or link the files in sahara_dashboard/enabled to
openstack_dashboard/local/enabled. This step will cause the Horizon service to
pick up the Sahara plugin when it starts.
To run unit tests:
------------------
./run_tests.sh
NOTE:
=====

View File

@ -19,5 +19,5 @@ from django.core.management import execute_from_command_line
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE",
"sahara-dashboard.test.settings")
"sahara_dashboard.test.settings")
execute_from_command_line(sys.argv)

20
run_tests.sh Normal file → Executable file
View File

@ -55,7 +55,7 @@ root=`pwd -P`
venv=$root/.venv
venv_env_version=$venv/environments
with_venv=tools/with_venv.sh
included_dirs="sahara-dashboard"
included_dirs="sahara_dashboard"
always_venv=0
backup_env=0
@ -165,7 +165,7 @@ function warn_on_flake8_without_venv {
function run_pep8 {
echo "Running flake8 ..."
warn_on_flake8_without_venv
DJANGO_SETTINGS_MODULE=sahara-dashboard.test.settings ${command_wrapper} flake8
DJANGO_SETTINGS_MODULE=sahara_dashboard.test.settings ${command_wrapper} flake8
}
function run_pep8_changed {
@ -178,13 +178,13 @@ function run_pep8_changed {
files=$(git diff --name-only $base_commit | tr '\n' ' ')
echo "Running flake8 on ${files}"
warn_on_flake8_without_venv
diff -u --from-file /dev/null ${files} | DJANGO_SETTINGS_MODULE=sahara-dashboard.test.settings ${command_wrapper} flake8 --diff
diff -u --from-file /dev/null ${files} | DJANGO_SETTINGS_MODULE=sahara_dashboard.test.settings ${command_wrapper} flake8 --diff
exit
}
function run_sphinx {
echo "Building sphinx..."
DJANGO_SETTINGS_MODULE=sahara-dashboard.test.settings ${command_wrapper} python setup.py build_sphinx
DJANGO_SETTINGS_MODULE=sahara_dashboard.test.settings ${command_wrapper} python setup.py build_sphinx
echo "Build complete."
}
@ -322,6 +322,10 @@ function run_tests {
export SELENIUM_HEADLESS=1
fi
# TODO(david-lyle) remove when configuration files for Sahara are not loaded
# by default in Horizon
${command_wrapper} python tools/clean_enabled_files.py
if [ -z "$testargs" ]; then
run_tests_all
else
@ -335,8 +339,8 @@ function run_tests_subset {
}
function run_tests_all {
echo "Running Sahara-Dashboard application tests"
export NOSE_XUNIT_FILE=sahara-dashboard/nosetests.xml
echo "Running sahara_dashboard application tests"
export NOSE_XUNIT_FILE=sahara_dashboard/nosetests.xml
if [ "$NOSE_WITH_HTML_OUTPUT" = '1' ]; then
export NOSE_HTML_OUT_FILE='sahara_dashboard_nose_results.html'
fi
@ -344,7 +348,7 @@ function run_tests_all {
${command_wrapper} python -m coverage.__main__ erase
coverage_run="python -m coverage.__main__ run -p"
fi
${command_wrapper} ${coverage_run} $root/manage.py test sahara-dashboard --settings=sahara-dashboard.test.settings $testopts
${command_wrapper} ${coverage_run} $root/manage.py test sahara_dashboard --settings=sahara_dashboard.test.settings $testopts
# get results of the Horizon tests
SAHARA_DASHBOARD_RESULT=$?
@ -545,4 +549,4 @@ if [ $runserver -eq 1 ]; then
fi
# Full test suite
run_tests || exit
run_tests || exit

View File

@ -0,0 +1,5 @@
from sahara_dashboard.api import sahara
__all__ = [
"sahara"
]

View File

@ -0,0 +1,465 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.conf import settings
from horizon import exceptions
from horizon.utils.memoized import memoized # noqa
from openstack_dashboard.api import base
from saharaclient.api.base import APIException
from saharaclient import client as api_client
LOG = logging.getLogger(__name__)
# "type" of Sahara service registered in keystone
SAHARA_SERVICE = 'data-processing'
# Sahara service_type registered in Juno
SAHARA_SERVICE_FALLBACK = 'data_processing'
SAHARA_AUTO_IP_ALLOCATION_ENABLED = getattr(
settings,
'SAHARA_AUTO_IP_ALLOCATION_ENABLED',
False)
VERSIONS = base.APIVersionManager(
SAHARA_SERVICE,
preferred_version=getattr(settings,
'OPENSTACK_API_VERSIONS',
{}).get(SAHARA_SERVICE, 1.1))
VERSIONS.load_supported_version(1.1, {"client": api_client,
"version": 1.1})
def safe_call(func, *args, **kwargs):
"""Call a function ignoring Not Found error
This method is supposed to be used only for safe retrieving Sahara
objects. If the object is no longer available the None should be
returned.
"""
try:
return func(*args, **kwargs)
except APIException as e:
if e.error_code == 404:
return None # Not found. Exiting with None
raise # Other errors are not expected here
@memoized
def client(request):
try:
service_type = SAHARA_SERVICE
sahara_url = base.url_for(request, service_type)
except exceptions.ServiceCatalogException:
# if no endpoint found, fallback to the old service_type
service_type = SAHARA_SERVICE_FALLBACK
sahara_url = base.url_for(request, service_type)
insecure = getattr(settings, 'OPENSTACK_SSL_NO_VERIFY', False)
cacert = getattr(settings, 'OPENSTACK_SSL_CACERT', None)
return api_client.Client(VERSIONS.get_active_version()["version"],
sahara_url=sahara_url,
service_type=service_type,
project_id=request.user.project_id,
input_auth_token=request.user.token.id,
insecure=insecure,
cacert=cacert)
def image_list(request, search_opts=None):
return client(request).images.list(search_opts=search_opts)
def image_get(request, image_id):
return client(request).images.get(id=image_id)
def image_unregister(request, image_id):
client(request).images.unregister_image(image_id=image_id)
def image_update(request, image_id, user_name, desc):
client(request).images.update_image(image_id=image_id,
user_name=user_name,
desc=desc)
def image_tags_update(request, image_id, image_tags):
client(request).images.update_tags(image_id=image_id,
new_tags=image_tags)
def plugin_list(request, search_opts=None):
return client(request).plugins.list(search_opts=search_opts)
def plugin_get(request, plugin_name):
return client(request).plugins.get(plugin_name=plugin_name)
def plugin_get_version_details(request, plugin_name, hadoop_version):
return client(request).plugins.get_version_details(
plugin_name=plugin_name,
hadoop_version=hadoop_version)
def plugin_convert_to_template(request, plugin_name, hadoop_version,
template_name, file_content):
return client(request).plugins.convert_to_cluster_template(
plugin_name=plugin_name,
hadoop_version=hadoop_version,
template_name=template_name,
filecontent=file_content)
def nodegroup_template_create(request, name, plugin_name, hadoop_version,
flavor_id, description=None,
volumes_per_node=None, volumes_size=None,
node_processes=None, node_configs=None,
floating_ip_pool=None, security_groups=None,
auto_security_group=False,
availability_zone=False,
volumes_availability_zone=False,
volume_type=None,
is_proxy_gateway=False,
volume_local_to_instance=False,
use_autoconfig=None):
return client(request).node_group_templates.create(
name=name,
plugin_name=plugin_name,
hadoop_version=hadoop_version,
flavor_id=flavor_id,
description=description,
volumes_per_node=volumes_per_node,
volumes_size=volumes_size,
node_processes=node_processes,
node_configs=node_configs,
floating_ip_pool=floating_ip_pool,
security_groups=security_groups,
auto_security_group=auto_security_group,
availability_zone=availability_zone,
volumes_availability_zone=volumes_availability_zone,
volume_type=volume_type,
is_proxy_gateway=is_proxy_gateway,
volume_local_to_instance=volume_local_to_instance,
use_autoconfig=use_autoconfig)
def nodegroup_template_list(request, search_opts=None):
return client(request).node_group_templates.list(search_opts=search_opts)
def nodegroup_template_get(request, ngt_id):
return client(request).node_group_templates.get(ng_template_id=ngt_id)
def nodegroup_template_find(request, **kwargs):
return client(request).node_group_templates.find(**kwargs)
def nodegroup_template_delete(request, ngt_id):
client(request).node_group_templates.delete(ng_template_id=ngt_id)
def nodegroup_template_update(request, ngt_id, name, plugin_name,
hadoop_version, flavor_id,
description=None, volumes_per_node=None,
volumes_size=None, node_processes=None,
node_configs=None, floating_ip_pool=None,
security_groups=None, auto_security_group=False,
availability_zone=False,
volumes_availability_zone=False,
volume_type=None,
is_proxy_gateway=False,
volume_local_to_instance=False,
use_autoconfig=None):
return client(request).node_group_templates.update(
ng_template_id=ngt_id,
name=name,
plugin_name=plugin_name,
hadoop_version=hadoop_version,
flavor_id=flavor_id,
description=description,
volumes_per_node=volumes_per_node,
volumes_size=volumes_size,
node_processes=node_processes,
node_configs=node_configs,
floating_ip_pool=floating_ip_pool,
security_groups=security_groups,
auto_security_group=auto_security_group,
availability_zone=availability_zone,
volumes_availability_zone=volumes_availability_zone,
volume_type=volume_type,
is_proxy_gateway=is_proxy_gateway,
volume_local_to_instance=volume_local_to_instance,
use_autoconfig=use_autoconfig)
def cluster_template_create(request, name, plugin_name, hadoop_version,
description=None, cluster_configs=None,
node_groups=None, anti_affinity=None,
net_id=None, use_autoconfig=None):
return client(request).cluster_templates.create(
name=name,
plugin_name=plugin_name,
hadoop_version=hadoop_version,
description=description,
cluster_configs=cluster_configs,
node_groups=node_groups,
anti_affinity=anti_affinity,
net_id=net_id,
use_autoconfig=use_autoconfig)
def cluster_template_list(request, search_opts=None):
return client(request).cluster_templates.list(search_opts=search_opts)
def cluster_template_get(request, ct_id):
return client(request).cluster_templates.get(cluster_template_id=ct_id)
def cluster_template_delete(request, ct_id):
client(request).cluster_templates.delete(cluster_template_id=ct_id)
def cluster_template_update(request, ct_id, name, plugin_name,
hadoop_version, description=None,
cluster_configs=None, node_groups=None,
anti_affinity=None, net_id=None,
use_autoconfig=None):
try:
template = client(request).cluster_templates.update(
cluster_template_id=ct_id,
name=name,
plugin_name=plugin_name,
hadoop_version=hadoop_version,
description=description,
cluster_configs=cluster_configs,
node_groups=node_groups,
anti_affinity=anti_affinity,
net_id=net_id,
use_autoconfig=use_autoconfig)
except APIException as e:
raise exceptions.Conflict(e)
return template
def cluster_create(request, name, plugin_name, hadoop_version,
cluster_template_id=None, default_image_id=None,
is_transient=None, description=None, cluster_configs=None,
node_groups=None, user_keypair_id=None, anti_affinity=None,
net_id=None, count=None, use_autoconfig=None):
return client(request).clusters.create(
name=name,
plugin_name=plugin_name,
hadoop_version=hadoop_version,
cluster_template_id=cluster_template_id,
default_image_id=default_image_id,
is_transient=is_transient,
description=description,
cluster_configs=cluster_configs,
node_groups=node_groups,
user_keypair_id=user_keypair_id,
anti_affinity=anti_affinity,
net_id=net_id,
count=count,
use_autoconfig=use_autoconfig)
def cluster_scale(request, cluster_id, scale_object):
return client(request).clusters.scale(
cluster_id=cluster_id,
scale_object=scale_object)
def cluster_list(request, search_opts=None):
return client(request).clusters.list(search_opts=search_opts)
def cluster_get(request, cluster_id, show_progress=False):
return client(request).clusters.get(
cluster_id=cluster_id,
show_progress=show_progress)
def cluster_delete(request, cluster_id):
client(request).clusters.delete(cluster_id=cluster_id)
def data_source_create(request, name, description, ds_type, url,
credential_user=None, credential_pass=None):
return client(request).data_sources.create(
name=name,
description=description,
data_source_type=ds_type,
url=url,
credential_user=credential_user,
credential_pass=credential_pass)
def data_source_list(request, search_opts=None):
return client(request).data_sources.list(search_opts=search_opts)
def data_source_get(request, ds_id):
return client(request).data_sources.get(data_source_id=ds_id)
def data_source_delete(request, ds_id):
client(request).data_sources.delete(data_source_id=ds_id)
def data_source_update(request, ds_id, data):
return client(request).data_sources.update(ds_id, data)
def job_binary_create(request, name, url, description, extra):
return client(request).job_binaries.create(
name=name,
url=url,
description=description,
extra=extra)
def job_binary_list(request, search_opts=None):
return client(request).job_binaries.list(search_opts=search_opts)
def job_binary_get(request, jb_id):
return client(request).job_binaries.get(job_binary_id=jb_id)
def job_binary_delete(request, jb_id):
client(request).job_binaries.delete(job_binary_id=jb_id)
def job_binary_get_file(request, jb_id):
return client(request).job_binaries.get_file(job_binary_id=jb_id)
def job_binary_update(request, jb_id, data):
return client(request).job_binaries.update(jb_id, data)
def job_binary_internal_create(request, name, data):
return client(request).job_binary_internals.create(
name=name,
data=data)
def job_binary_internal_list(request, search_opts=None):
return client(request).job_binary_internals.list(search_opts=search_opts)
def job_binary_internal_get(request, jbi_id):
# The argument name looks wrong. This should be changed in the sahara
# client first and then updated here
return client(request).job_binary_internals.get(job_binary_id=jbi_id)
def job_binary_internal_delete(request, jbi_id):
# The argument name looks wrong. This should be changed in the sahara
# client first and then updated here
client(request).job_binary_internals.delete(job_binary_id=jbi_id)
def job_create(request, name, j_type, mains, libs, description, interface):
return client(request).jobs.create(
name=name,
type=j_type,
mains=mains,
libs=libs,
description=description,
interface=interface)
def job_list(request, search_opts=None):
return client(request).jobs.list(search_opts=search_opts)
def job_get(request, job_id):
return client(request).jobs.get(job_id=job_id)
def job_delete(request, job_id):
client(request).jobs.delete(job_id=job_id)
def job_get_configs(request, job_type):
return client(request).jobs.get_configs(job_type=job_type)
def job_execution_create(request, job_id, cluster_id,
input_id, output_id, configs,
interface):
return client(request).job_executions.create(
job_id=job_id,
cluster_id=cluster_id,
input_id=input_id,
output_id=output_id,
configs=configs,
interface=interface)
def _resolve_job_execution_names(job_execution, cluster=None,
job=None):
job_execution.cluster_name = None
if cluster:
job_execution.cluster_name = cluster.name
job_execution.job_name = None
if job:
job_execution.job_name = job.name
return job_execution
def job_execution_list(request, search_opts=None):
job_execution_list = client(request).job_executions.list(
search_opts=search_opts)
job_dict = dict((j.id, j) for j in job_list(request))
cluster_dict = dict((c.id, c) for c in cluster_list(request))
resolved_job_execution_list = [
_resolve_job_execution_names(
job_execution,
cluster_dict.get(job_execution.cluster_id),
job_dict.get(job_execution.job_id))
for job_execution in job_execution_list
]
return resolved_job_execution_list
def job_execution_get(request, jex_id):
jex = client(request).job_executions.get(obj_id=jex_id)
cluster = safe_call(client(request).clusters.get, jex.cluster_id)
job = safe_call(client(request).jobs.get, jex.job_id)
return _resolve_job_execution_names(jex, cluster, job)
def job_execution_delete(request, jex_id):
client(request).job_executions.delete(obj_id=jex_id)
def job_types_list(request):
return client(request).job_types.list()

View File

@ -0,0 +1,58 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from horizon import forms
from sahara_dashboard.api import sahara as saharaclient
from sahara_dashboard.content.data_processing. \
utils import workflow_helpers
LOG = logging.getLogger(__name__)
class UploadFileForm(forms.SelfHandlingForm,
workflow_helpers.PluginAndVersionMixin):
template_name = forms.CharField(max_length=80,
label=_("Cluster Template Name"))
def __init__(self, request, *args, **kwargs):
super(UploadFileForm, self).__init__(request, *args, **kwargs)
sahara = saharaclient.client(request)
self._generate_plugin_version_fields(sahara)
self.fields['template_file'] = forms.FileField(label=_("Template"))
def handle(self, request, data):
try:
# we can set a limit on file size, but should we?
filecontent = self.files['template_file'].read()
plugin_name = data['plugin_name']
hadoop_version = data.get(plugin_name + "_version")
saharaclient.plugin_convert_to_template(request,
plugin_name,
hadoop_version,
data['template_name'],
filecontent)
return True
except Exception:
exceptions.handle(request,
_("Unable to upload cluster template file"))
return False

View File

@ -0,0 +1,28 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from django.utils.translation import ugettext_lazy as _
import horizon
from openstack_dashboard.dashboards.project import dashboard
class ClusterTemplatesPanel(horizon.Panel):
name = _("Cluster Templates")
slug = 'data_processing.cluster_templates'
permissions = (('openstack.services.data-processing',
'openstack.services.data_processing'),)
dashboard.Project.register(ClusterTemplatesPanel)

View File

@ -0,0 +1,149 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.core import urlresolvers
from django.template import defaultfilters as filters
from django.utils import http
from django.utils.translation import ugettext_lazy as _
from django.utils.translation import ungettext_lazy
from horizon import tables
from sahara_dashboard.api import sahara as saharaclient
LOG = logging.getLogger(__name__)
class ClusterTemplatesFilterAction(tables.FilterAction):
filter_type = "server"
filter_choices = (('name', _("Name"), True),
('plugin_name', _("Plugin"), True),
('hadoop_version', _("Version"), True),
('description', _("Description")))
class UploadFile(tables.LinkAction):
name = 'upload_file'
verbose_name = _("Upload Template")
url = 'horizon:project:data_processing.cluster_templates:upload_file'
classes = ("btn-launch", "ajax-modal")
icon = "upload"
class CreateCluster(tables.LinkAction):
name = "create cluster"
verbose_name = _("Launch Cluster")
url = "horizon:project:data_processing.clusters:configure-cluster"
classes = ("ajax-modal",)
icon = "plus"
def get_link_url(self, datum):
base_url = urlresolvers.reverse(self.url)
params = http.urlencode({"hadoop_version": datum.hadoop_version,
"plugin_name": datum.plugin_name,
"cluster_template_id": datum.id})
return "?".join([base_url, params])
class CopyTemplate(tables.LinkAction):
name = "copy"
verbose_name = _("Copy Template")
url = "horizon:project:data_processing.cluster_templates:copy"
classes = ("ajax-modal", )
class EditTemplate(tables.LinkAction):
name = "edit"
verbose_name = _("Edit Template")
url = "horizon:project:data_processing.cluster_templates:edit"
classes = ("ajax-modal", )
class DeleteTemplate(tables.DeleteAction):
@staticmethod
def action_present(count):
return ungettext_lazy(
u"Delete Template",
u"Delete Templates",
count
)
@staticmethod
def action_past(count):
return ungettext_lazy(
u"Deleted Template",
u"Deleted Templates",
count
)
def delete(self, request, template_id):
saharaclient.cluster_template_delete(request, template_id)
class CreateClusterTemplate(tables.LinkAction):
name = "create"
verbose_name = _("Create Template")
url = ("horizon:project:data_processing.cluster_templates:"
"create-cluster-template")
classes = ("ajax-modal", "create-clustertemplate-btn")
icon = "plus"
class ConfigureClusterTemplate(tables.LinkAction):
name = "configure"
verbose_name = _("Configure Cluster Template")
url = ("horizon:project:data_processing.cluster_templates:"
"configure-cluster-template")
classes = ("ajax-modal", "configure-clustertemplate-btn")
icon = "plus"
attrs = {"style": "display: none"}
def render_node_groups(cluster_template):
node_groups = [node_group['name'] + ': ' + str(node_group['count'])
for node_group in cluster_template.node_groups]
return node_groups
class ClusterTemplatesTable(tables.DataTable):
name = tables.Column("name",
verbose_name=_("Name"),
link=("horizon:project:data_processing."
"cluster_templates:details"))
plugin_name = tables.Column("plugin_name",
verbose_name=_("Plugin"))
hadoop_version = tables.Column("hadoop_version",
verbose_name=_("Version"))
node_groups = tables.Column(render_node_groups,
verbose_name=_("Node Groups"),
wrap_list=True,
filters=(filters.unordered_list,))
description = tables.Column("description",
verbose_name=_("Description"))
class Meta(object):
name = "cluster_templates"
verbose_name = _("Cluster Templates")
table_actions = (UploadFile,
CreateClusterTemplate,
ConfigureClusterTemplate,
DeleteTemplate,
ClusterTemplatesFilterAction,)
row_actions = (CreateCluster,
EditTemplate,
CopyTemplate,
DeleteTemplate,)

View File

@ -0,0 +1,76 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from horizon import tabs
from openstack_dashboard.api import nova
from sahara_dashboard.api import sahara as saharaclient
from sahara_dashboard.content. \
data_processing.utils import workflow_helpers as helpers
LOG = logging.getLogger(__name__)
class GeneralTab(tabs.Tab):
name = _("General Info")
slug = "cluster_template_details_tab"
template_name = (
"project/data_processing.cluster_templates/_details.html")
def get_context_data(self, request):
template_id = self.tab_group.kwargs['template_id']
try:
template = saharaclient.cluster_template_get(request, template_id)
except Exception as e:
template = {}
LOG.error("Unable to fetch cluster template details: %s" % str(e))
return {"template": template}
class NodeGroupsTab(tabs.Tab):
name = _("Node Groups")
slug = "cluster_template_nodegroups_tab"
template_name = (
"project/data_processing.cluster_templates/_nodegroups_details.html")
def get_context_data(self, request):
template_id = self.tab_group.kwargs['template_id']
try:
template = saharaclient.cluster_template_get(request, template_id)
for ng in template.node_groups:
if not ng["flavor_id"]:
continue
ng["flavor_name"] = (
nova.flavor_get(request, ng["flavor_id"]).name)
ng["node_group_template"] = saharaclient.safe_call(
saharaclient.nodegroup_template_get,
request, ng.get("node_group_template_id", None))
ng["security_groups_full"] = helpers.get_security_groups(
request, ng.get("security_groups"))
except Exception:
template = {}
exceptions.handle(request,
_("Unable to fetch node group details."))
return {"template": template}
class ClusterTemplateDetailsTabs(tabs.TabGroup):
slug = "cluster_template_details"
tabs = (GeneralTab, NodeGroupsTab, )
sticky = True

View File

@ -0,0 +1,22 @@
{% load i18n horizon %}
<div class="well">
<p>
{% blocktrans %}This Cluster Template will be created for:{% endblocktrans %}
<br >
<b>{% blocktrans %}Plugin{% endblocktrans %}</b>: {{ plugin_name }}
<br />
<b>{% blocktrans %}Version{% endblocktrans %}</b>: {{ hadoop_version }}
<br />
</p>
<p>
{% blocktrans %}The Cluster Template object should specify Node Group Templates that will be used to build a Cluster.
You can add Node Groups using Node Group Templates on a &quot;Node Groups&quot; tab.{% endblocktrans %}
</p>
<p>
{% blocktrans %}You may set <b>cluster</b> scoped configurations on corresponding tabs.{% endblocktrans %}
</p>
<p>
{% blocktrans %}The Cluster Template object may specify a list of processes in anti-affinity group.
That means these processes may not be launched more than once on a single host.{% endblocktrans %}
</p>
</div>

View File

@ -0,0 +1,4 @@
{% load i18n horizon %}
<p class="well">
{% blocktrans %}Select a plugin and version for a new Cluster template.{% endblocktrans %}
</p>

View File

@ -0,0 +1,55 @@
{% load i18n sizeformat %}
<div class="detail">
<dl class="dl-horizontal">
<dt>{% trans "Name" %}</dt>
<dd>{{ template.name }}</dd>
<dt>{% trans "ID" %}</dt>
<dd>{{ template.id }}</dd>
<dt>{% trans "Description" %}</dt>
<dd>{{ template.description|default:_("None") }}</dd>
</dl>
<dl class="dl-horizontal">
<dt>{% trans "Plugin" %}</dt>
<dd><a href="{% url 'horizon:project:data_processing.data_plugins:details' template.plugin_name %}">{{ template.plugin_name }}</a></dd>
<dt>{% trans "Version" %}</dt>
<dd>{{ template.hadoop_version }}</dd>
<dt>{% trans "Use auto-configuration" %}</dt>
<dd>{{ template.use_autoconfig }}</dd>
</dl>
<dl class="dl-horizontal">
<dt>{% trans "Anti-affinity enabled for" %}</dt>
{% if template.anti_affinity %}
<dd>
<ul class="list-bullet">
{% for process in template.anti_affinity %}
<li>{{ process }}</li>
{% endfor %}
</ul>
</dd>
{% else %}
<h6>{% trans "no processes" %}</h6>
{% endif %}
</dl>
<dl class="dl-horizontal">
<dt>{% trans "Node Configurations" %}</dt>
{% if template.cluster_configs %}
<dd>
{% for service, service_conf in template.cluster_configs.items %}
<h4>{{ service }}</h4>
{% if service_conf %}
<ul>
{% for conf_name, conf_value in service_conf.items %}
<li>{% blocktrans %}{{ conf_name }}: {{ conf_value }}{% endblocktrans %}</li>
{% endfor %}
</ul>
{% else %}
<h6>{% trans "No configurations" %}</h6>
{% endif %}
{% endfor %}
</dd>
{% else %}
<dd>{% trans "Cluster configurations are not specified" %}</dd>
{% endif %}
</dl>
</div>

View File

@ -0,0 +1,81 @@
{% load i18n sizeformat %}
<div class="detail">
{% for node_group in template.node_groups %}
<dl class="well">
<h4>{% blocktrans with node_group_name=node_group.name %}Node Group: {{ node_group_name }}{% endblocktrans %}</h4>
<dt>{% trans "Nodes Count" %}</dt>
<dd>{{ node_group.count }}</dd>
<dt>{% trans "Flavor" %}</dt>
<dd>{{ node_group.flavor_id|default:_("Flavor is not specified") }}</dd>
<dt>{% trans "Template" %}</dt>
{% if node_group.node_group_template_id %}
<dd><a href="{% url 'horizon:project:data_processing.nodegroup_templates:details' node_group.node_group_template_id %}">{{ node_group.node_group_template.name }} </a></dd>
{% else %}
<dd>{% trans "Template not specified" %}</dd>
{% endif %}
{% if node_group.availability_zone %}
<dt>{% trans "Availability Zone" %}</dt>
<dd>{{ node_group.availability_zone }}</dd>
{% endif %}
<dt>{% trans "Use auto-configuration" %}</dt>
<dd>{{ node_group.use_autoconfig }}</dd>
<dt>{% trans "Proxy Gateway" %}</dt>
<dd>{{ node_group.is_proxy_gateway|yesno }}</dd>
<dt>{% trans "Auto Security Group" %}</dt>
<dd>{{ node_group.auto_security_group|yesno }}</dd>
<dt>{% trans "Security Groups" %}</dt>
<dd>
<ul class="list-bullet">
{% for group in node_group.security_groups_full %}
{% if group.id %}
<li><a href="{% url 'horizon:project:access_and_security:security_groups:detail' group.id %}">{{ group.name }}</a></li>
{% else %}
<li>{{ group.name }}</li>
{% endif %}
{% endfor %}
</ul>
</dd>
<dt>{% trans "Node Processes" %}</dt>
{% if node_group.node_processes %}
<dd>
<ul class="list-bullet">
{% for process in node_group.node_processes %}
<li>{{ process }}</li>
{% endfor %}
</ul>
</dd>
{% else %}
<dd>{% trans "Node processes are not specified" %}</dd>
{% endif %}
<dt>{% trans "Node Configurations" %}</dt>
{% if node_group.node_configs %}
<dd>
{% for service, service_conf in node_group.node_configs.items %}
<h6>{{ service }}</h6>
{% if service_conf %}
<ul>
{% for conf_name, conf_value in service_conf.items %}
<li>{% blocktrans %}{{ conf_name }}: {{ conf_value }}{% endblocktrans %}</li>
{% endfor %}
</ul>
{% else %}
<h6>{% trans "No configurations" %}</h6>
{% endif %}
{% endfor %}
</dd>
{% else %}
<dd>{% trans "Node configurations are not specified" %}</dd>
{% endif %}
</dl>
{% endfor %}
</div>

View File

@ -0,0 +1,23 @@
{% extends "horizon/common/_modal_form.html" %}
{% load i18n %}
{% block form_id %}upload_file{% endblock %}
{% block form_action %}{% url 'horizon:project:data_processing.cluster_templates:upload_file' %}{% endblock %}
{% block form_attrs %}enctype="multipart/form-data"{% endblock %}
{% block modal-header %}{% trans "Upload Template" %}{% endblock %}
{% block modal-body %}
<div class="left">
<fieldset>
{% include "horizon/common/_form_fields.html" %}
</fieldset>
</div>
{% endblock %}
{% block modal-footer %}
<input class="btn btn-primary pull-right" id="upload_file_btn" type="submit" value="{% trans "Upload" %}"/>
<a href="{% url 'horizon:project:data_processing.cluster_templates:index' %}" class="btn btn-default secondary cancel close">{% trans "Cancel" %}</a>
{% endblock %}

View File

@ -0,0 +1,159 @@
{% load i18n %}
<script>
var template =
'<tr class="row data-template-row" id_attr="$id" style="padding:5px">' +
'<td class="col-sm-4 small-padding">' +
'<input id="template_id_$id" value="$template_id" type="hidden" name="template_id_$id">' +
'<input id="group_name_$id" value="$group_name" type="text" name="group_name_$id" class="form-control">' +
'</td>' +
'<td class="col-sm-3 small-padding">' +
'<input disabled value="$template_name" class="form-control" />' +
'</td>' +
'<td class="col-sm-3 small-padding">' +
'<span class="input-group input-append">' +
'<input id="count_$id" class="count-field form-control" value="$node_count" type="text" max="4" maxlength="4" name="count_$id" size="4">' +
'<span class="input-group-btn">' +
'<div class="btn btn-default dec-btn" data-count-id="count_$id"><i class="fa fa-minus"></i></div>' +
'<div class="btn btn-default inc-btn" data-count-id="count_$id"><i class="fa fa-plus"></i></div>' +
'</span>' +
'</span>' +
'</td>' +
'<td class="col-sm-2 small-padding">' +
'<input type="button" class="btn btn-danger" id="delete_btn_$id" data-toggle="dropdown" onclick="delete_node_group(this)" value="Remove" />' +
'<input type="text" id="serialized_$id" name="serialized_$id" value="$serialized" style="display:None;">' +
'</td>' +
'</tr>';
function mark_element_as_wrong(id){
$("#"+id).parent("div").addClass("error");
}
function get_next_id() {
var max = -1;
$(".data-template-row").each(function () {
max = Math.max(max, parseInt($(this).attr("id_attr")));
});
return max + 1;
}
function set_nodes_ids() {
var ids = [];
$(".data-template-row").each(function () {
var id = parseInt($(this).attr("id_attr"));
if (!isNaN(id)) {
ids.push(id);
}
});
$("#forms_ids").val(JSON.stringify(ids));
}
function add_node(node_count, group_name, template_id, id, deletable, serialized) {
var template_name = $("select option[value='" + template_id + "']").html();
var tmp = template.
replace(/\$id/g, id).
replace(/\$group_name/g, group_name).
replace(/\$template_id/g, template_id).
replace(/\$node_count/g, node_count).
replace(/\$serialized/g, serialized).
replace(/\$template_name/g, template_name);
$("#groups_table").find("tr:last").after(tmp);
if (!deletable) {
$("#delete_btn_" + id).remove();
$("#group_name_" + id).prop('readonly', true);
}
$("#node-templates").show();
set_nodes_ids();
}
function add_node_group_template(node_count) {
if ($("select option:selected").html() == "Select") {
return;
}
var template_id = $("#template_id option:selected").val();
var template_name = $("#template_id option:selected").html();
add_node(node_count, template_name, template_id, get_next_id(), true, null);
$(".count-field").change();
}
function delete_node_group(el) {
$(el).closest("tr").remove();
var id = get_next_id();
if (id == 0) {
$("#node-templates").hide();
}
set_nodes_ids();
}
$("#template_id").change(function () {
if ($("select option:selected").html() == "Select") {
$("#add_group_button").addClass("disabled");
} else {
$("#add_group_button").removeClass("disabled");
}
});
$("#node-templates").hide();
</script>
<input type="hidden" value="[]" name="forms_ids" id="forms_ids">
<label for="template_id">{% trans "Select a Node Group Template to add:" %}</label>
<span class="row">
<span class="input-group col-sm-4">
<select id="template_id" name="template" class="form-control">
<option>Select</option>
{% for template in form.templates %}
<option value="{{ template.id }}">{{ template.name }}</option>
{% endfor %}
</select>
<span class="input-group-btn">
<button type="button" id="add_group_button" class="btn btn-default disabled" onclick="add_node_group_template(1);">
<span class="fa fa-plus"></span>
</button>
</span>
</span>
</span>
<br/>
<div id="node-templates">
<table id="groups_table">
<tr id="header_row" class="row">
<th class="col-sm-4 small-padding"><label>Group Name</label></th>
<th class="col-sm-3 small-padding"><label>Template</label></th>
<th class="col-sm-3 small-padding"><label>Count</label></th>
</tr>
</table>
</div>
<script>
{% for group in form.groups %}
add_node("{{ group.count }}", "{{ group.name }}", "{{ group.template_id }}", "{{ group.id }}", {{ group.deletable }}, "{{ group.serialized }}");
{% endfor %}
{% for field_id in form.errors_fields %}
mark_element_as_wrong("{{ field_id }}");
{% endfor %}
var handlers_registred;
var lower_limit = 1;
$(function() {
if (!handlers_registred) {
handlers_registred = true;
$(".inc-btn").live("click", function(e) {
var id = $(this).attr("data-count-id");
$("#" + id).val(parseInt($("#" + id).val()) + 1);
$(".count-field").change();
});
$(".dec-btn").live("click", function(e) {
var id = $(this).attr("data-count-id");
var val = parseInt($("#" + id).val());
if (val > lower_limit) {
$("#" + id).val(val - 1);
}
$(".count-field").change();
});
}
$(".count-field").live("change", function() {
var val = $(this).val();
if (val > lower_limit) {
$(this).parent("div").find(".dec-btn").removeClass("disabled");
} else {
$(this).parent("div").find(".dec-btn").addClass("disabled");
}
}).change();
});
</script>

View File

@ -0,0 +1,63 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Data Processing" %}{% endblock %}
{% block main %}
<div class="cluster_templates">
{{ cluster_templates_table.render }}
</div>
<script type="text/javascript">
addHorizonLoadEvent(function () {
horizon.modals.addModalInitFunction(function (modal) {
var $navbar = $(modal).find(".nav-tabs");
if ($navbar.find("li").size() == 1) {
// hide tab bar for plugin/version modal wizard
$navbar.hide();
}
var add_ng_btn_label = '{% trans "Add Node Group" %}';
$(".hidden_nodegroups_field").after("<input type='button' id='add_nodegroup' value='" + add_ng_btn_label + "'/><br/>");
$("#add_nodegroup").click(function() {
$(".hidden_nodegroups_field").val("create_nodegroup");
$(".hidden_configure_field").val("create_nodegroup");
var form = $(".hidden_nodegroups_field").closest("form");
form.submit();
});
$(".hidden_nodegroups_field").val("");
$(".hidden_configure_field").val("");
if ($(modal).find(".hidden_create_field").length > 0) {
var form = $(".hidden_create_field").closest("form");
var successful = false;
form.submit(function (e) {
var oldHref = $(".configure-clustertemplate-btn")[0].href;
var plugin = $("#id_plugin_name option:selected").val();
var version = $("#id_" + plugin + "_version option:selected").val();
form.find(".close").click();
$(".configure-clustertemplate-btn")[0].href = oldHref +
"?plugin_name=" + encodeURIComponent(plugin) +
"&hadoop_version=" + encodeURIComponent(version);
$(".configure-clustertemplate-btn").click();
$(".configure-clustertemplate-btn")[0].href = oldHref;
return false;
});
$(".plugin_version_choice").closest(".form-group").hide();
}
//display version for selected plugin
$(document).on('change', '.plugin_name_choice', switch_versions);
function switch_versions() {
$(".plugin_version_choice").closest(".form-group").hide();
var plugin = $(this);
$("." + plugin.val() + "_version_choice").closest(".form-group").show();
}
$(".plugin_name_choice").change();
});
});
</script>
{% endblock %}

View File

@ -0,0 +1,7 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Create Cluster Template" %}{% endblock %}
{% block main %}
{% include 'horizon/common/_workflow.html' %}
{% endblock %}

View File

@ -0,0 +1,11 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{{ name }}{% endblock %}
{% block page_header %}
{% include "horizon/common/_page_header.html" with title={{ name }} %}
{% endblock page_header %}
{% block main %}
{% include 'horizon/common/_workflow.html' %}
{% endblock %}

View File

@ -0,0 +1,7 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Upload Template" %}{% endblock %}
{% block main %}
{% include 'project/data_processing.cluster_templates/_upload_file.html' %}
{% endblock %}

View File

@ -0,0 +1,167 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import base64
import copy
from django.core.urlresolvers import reverse
from django import http
from mox3.mox import IsA # noqa
from oslo_serialization import jsonutils
import six
from openstack_dashboard import api as dash_api
from openstack_dashboard.test import helpers as test
from sahara_dashboard import api
INDEX_URL = reverse('horizon:project:data_processing.cluster_templates:index')
DETAILS_URL = reverse(
'horizon:project:data_processing.cluster_templates:details', args=['id'])
class DataProcessingClusterTemplateTests(test.TestCase):
@test.create_stubs({api.sahara: ('cluster_template_list',)})
def test_index(self):
api.sahara.cluster_template_list(IsA(http.HttpRequest), {}) \
.AndReturn(self.cluster_templates.list())
self.mox.ReplayAll()
res = self.client.get(INDEX_URL)
self.assertTemplateUsed(res,
'project/data_processing.cluster_templates/'
'cluster_templates.html')
self.assertContains(res, 'Cluster Templates')
self.assertContains(res, 'Name')
@test.create_stubs({api.sahara: ('cluster_template_get',),
dash_api.nova: ('flavor_get',)})
def test_details(self):
flavor = self.flavors.first()
ct = self.cluster_templates.first()
dash_api.nova.flavor_get(IsA(http.HttpRequest), flavor.id) \
.MultipleTimes().AndReturn(flavor)
api.sahara.cluster_template_get(IsA(http.HttpRequest),
IsA(six.text_type)) \
.MultipleTimes().AndReturn(ct)
self.mox.ReplayAll()
res = self.client.get(DETAILS_URL)
self.assertTemplateUsed(res, 'horizon/common/_detail.html')
@test.create_stubs({api.sahara: ('cluster_template_get',
'plugin_get_version_details',
'nodegroup_template_find')})
def test_copy(self):
ct = self.cluster_templates.first()
ngts = self.nodegroup_templates.list()
configs = self.plugins_configs.first()
api.sahara.cluster_template_get(IsA(http.HttpRequest),
ct.id) \
.AndReturn(ct)
api.sahara.plugin_get_version_details(IsA(http.HttpRequest),
ct.plugin_name,
ct.hadoop_version) \
.MultipleTimes().AndReturn(configs)
api.sahara.nodegroup_template_find(IsA(http.HttpRequest),
plugin_name=ct.plugin_name,
hadoop_version=ct.hadoop_version) \
.MultipleTimes().AndReturn(ngts)
self.mox.ReplayAll()
url = reverse('horizon:project:data_processing.cluster_templates:copy',
args=[ct.id])
res = self.client.get(url)
workflow = res.context['workflow']
step = workflow.get_step("generalconfigaction")
self.assertEqual(step.action['cluster_template_name'].field.initial,
ct.name + "-copy")
@test.create_stubs({api.sahara: ('cluster_template_list',
'cluster_template_delete')})
def test_delete(self):
ct = self.cluster_templates.first()
api.sahara.cluster_template_list(IsA(http.HttpRequest), {}) \
.AndReturn(self.cluster_templates.list())
api.sahara.cluster_template_delete(IsA(http.HttpRequest), ct.id)
self.mox.ReplayAll()
form_data = {'action': 'cluster_templates__delete__%s' % ct.id}
res = self.client.post(INDEX_URL, form_data)
self.assertNoFormErrors(res)
self.assertRedirectsNoFollow(res, INDEX_URL)
self.assertMessageCount(success=1)
@test.create_stubs({api.sahara: ('cluster_template_get',
'cluster_template_update',
'plugin_get_version_details',
'nodegroup_template_find')})
def test_update(self):
ct = self.cluster_templates.first()
ngts = self.nodegroup_templates.list()
configs = self.plugins_configs.first()
new_name = "UpdatedName"
new_ct = copy.copy(ct)
new_ct.name = new_name
api.sahara.cluster_template_get(IsA(http.HttpRequest), ct.id) \
.AndReturn(ct)
api.sahara.plugin_get_version_details(IsA(http.HttpRequest),
ct.plugin_name,
ct.hadoop_version) \
.MultipleTimes().AndReturn(configs)
api.sahara.nodegroup_template_find(IsA(http.HttpRequest),
plugin_name=ct.plugin_name,
hadoop_version=ct.hadoop_version) \
.MultipleTimes().AndReturn(ngts)
api.sahara.cluster_template_update(request=IsA(http.HttpRequest),
ct_id=ct.id,
name=new_name,
plugin_name=ct.plugin_name,
hadoop_version=ct.hadoop_version,
description=ct.description,
cluster_configs=ct.cluster_configs,
node_groups=ct.node_groups,
anti_affinity=ct.anti_affinity,
use_autoconfig=False)\
.AndReturn(new_ct)
self.mox.ReplayAll()
url = reverse('horizon:project:data_processing.cluster_templates:edit',
args=[ct.id])
def serialize(obj):
return base64.urlsafe_b64encode(jsonutils.dump_as_bytes(obj))
res = self.client.post(
url,
{'ct_id': ct.id,
'cluster_template_name': new_name,
'plugin_name': ct.plugin_name,
'hadoop_version': ct.hadoop_version,
'description': ct.description,
'hidden_configure_field': "",
'template_id_0': ct.node_groups[0]['node_group_template_id'],
'group_name_0': ct.node_groups[0]['name'],
'count_0': 1,
'serialized_0': serialize(ct.node_groups[0]),
'template_id_1': ct.node_groups[1]['node_group_template_id'],
'group_name_1': ct.node_groups[1]['name'],
'count_1': 2,
'serialized_1': serialize(ct.node_groups[1]),
'forms_ids': "[0,1]",
'anti-affinity': ct.anti_affinity,
})
self.assertNoFormErrors(res)
self.assertRedirectsNoFollow(res, INDEX_URL)
self.assertMessageCount(success=1)

View File

@ -0,0 +1,43 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from django.conf.urls import patterns
from django.conf.urls import url
import sahara_dashboard.content. \
data_processing.cluster_templates.views as views
urlpatterns = patterns('',
url(r'^$', views.ClusterTemplatesView.as_view(),
name='index'),
url(r'^$', views.ClusterTemplatesView.as_view(),
name='cluster-templates'),
url(r'^upload_file$',
views.UploadFileView.as_view(),
name='upload_file'),
url(r'^create-cluster-template$',
views.CreateClusterTemplateView.as_view(),
name='create-cluster-template'),
url(r'^configure-cluster-template$',
views.ConfigureClusterTemplateView.as_view(),
name='configure-cluster-template'),
url(r'^(?P<template_id>[^/]+)$',
views.ClusterTemplateDetailsView.as_view(),
name='details'),
url(r'^(?P<template_id>[^/]+)/copy$',
views.CopyClusterTemplateView.as_view(),
name='copy'),
url(r'^(?P<template_id>[^/]+)/edit$',
views.EditClusterTemplateView.as_view(),
name='edit'))

View File

@ -0,0 +1,149 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.core.urlresolvers import reverse_lazy
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from horizon import forms
from horizon import tables
from horizon import tabs
from horizon.utils import memoized
from horizon.utils.urlresolvers import reverse # noqa
from horizon import workflows
from sahara_dashboard.api import sahara as saharaclient
from sahara_dashboard.content.data_processing. \
cluster_templates import forms as cluster_forms
import sahara_dashboard.content.data_processing. \
cluster_templates.tables as ct_tables
import sahara_dashboard.content.data_processing. \
cluster_templates.tabs as _tabs
import sahara_dashboard.content.data_processing. \
cluster_templates.workflows.copy as copy_flow
import sahara_dashboard.content.data_processing. \
cluster_templates.workflows.create as create_flow
import sahara_dashboard.content.data_processing. \
cluster_templates.workflows.edit as edit_flow
LOG = logging.getLogger(__name__)
class ClusterTemplatesView(tables.DataTableView):
table_class = ct_tables.ClusterTemplatesTable
template_name = (
'project/data_processing.cluster_templates/cluster_templates.html')
page_title = _("Cluster Templates")
def get_data(self):
try:
search_opts = {}
filter = self.get_server_filter_info(self.request)
if filter['value'] and filter['field']:
search_opts = {filter['field']: filter['value']}
cluster_templates = saharaclient.cluster_template_list(
self.request, search_opts)
except Exception:
cluster_templates = []
exceptions.handle(self.request,
_("Unable to fetch cluster template list"))
return cluster_templates
class ClusterTemplateDetailsView(tabs.TabView):
tab_group_class = _tabs.ClusterTemplateDetailsTabs
template_name = 'horizon/common/_detail.html'
page_title = "{{ template.name|default:template.id }}"
@memoized.memoized_method
def get_object(self):
ct_id = self.kwargs["template_id"]
try:
return saharaclient.cluster_template_get(self.request, ct_id)
except Exception:
msg = _('Unable to retrieve details for '
'cluster template "%s".') % ct_id
redirect = reverse("horizon:project:data_processing."
"cluster_templates:cluster-templates")
exceptions.handle(self.request, msg, redirect=redirect)
def get_context_data(self, **kwargs):
context = super(ClusterTemplateDetailsView, self)\
.get_context_data(**kwargs)
context['template'] = self.get_object()
return context
class UploadFileView(forms.ModalFormView):
form_class = cluster_forms.UploadFileForm
template_name = (
'project/data_processing.cluster_templates/upload_file.html')
success_url = reverse_lazy(
'horizon:project:data_processing.cluster_templates:index')
page_title = _("Upload Template")
class CreateClusterTemplateView(workflows.WorkflowView):
workflow_class = create_flow.CreateClusterTemplate
success_url = ("horizon:project:data_processing.cluster_templates"
":create-cluster-template")
classes = ("ajax-modal",)
template_name = "project/data_processing.cluster_templates/create.html"
page_title = _("Create Cluster Template")
class ConfigureClusterTemplateView(workflows.WorkflowView):
workflow_class = create_flow.ConfigureClusterTemplate
success_url = "horizon:project:data_processing.cluster_templates"
template_name = "project/data_processing.cluster_templates/configure.html"
page_title = _("Configure Cluster Template")
class CopyClusterTemplateView(workflows.WorkflowView):
workflow_class = copy_flow.CopyClusterTemplate
success_url = "horizon:project:data_processing.cluster_templates"
template_name = "project/data_processing.cluster_templates/configure.html"
page_title = _("Copy Cluster Template")
def get_context_data(self, **kwargs):
context = super(CopyClusterTemplateView, self)\
.get_context_data(**kwargs)
context["template_id"] = kwargs["template_id"]
return context
def get_object(self, *args, **kwargs):
if not hasattr(self, "_object"):
template_id = self.kwargs['template_id']
try:
template = saharaclient.cluster_template_get(self.request,
template_id)
except Exception:
template = {}
exceptions.handle(self.request,
_("Unable to fetch cluster template."))
self._object = template
return self._object
def get_initial(self):
initial = super(CopyClusterTemplateView, self).get_initial()
initial['template_id'] = self.kwargs['template_id']
return initial
class EditClusterTemplateView(CopyClusterTemplateView):
workflow_class = edit_flow.EditClusterTemplate
success_url = "horizon:project:data_processing.cluster_templates"
template_name = "project/data_processing.cluster_templates/configure.html"

View File

@ -0,0 +1,99 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import base64
import json
import logging
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from sahara_dashboard.api import sahara as saharaclient
import sahara_dashboard.content.data_processing. \
cluster_templates.workflows.create as create_flow
import sahara_dashboard.content.data_processing.utils. \
workflow_helpers as wf_helpers
LOG = logging.getLogger(__name__)
class CopyClusterTemplate(create_flow.ConfigureClusterTemplate):
success_message = _("Cluster Template copy %s created")
entry_point = "generalconfigaction"
def __init__(self, request, context_seed, entry_point, *args, **kwargs):
self.cluster_template_id = context_seed["template_id"]
try:
self.template = saharaclient.cluster_template_get(
request,
self.cluster_template_id)
self._set_configs_to_copy(self.template.cluster_configs)
request.GET = request.GET.copy()
request.GET.update({"plugin_name": self.template.plugin_name,
"hadoop_version": self.template.hadoop_version,
"aa_groups": self.template.anti_affinity})
super(CopyClusterTemplate, self).__init__(request, context_seed,
entry_point, *args,
**kwargs)
# Initialize node groups.
# TODO(rdopieralski) The same (or very similar) code appears
# multiple times in this dashboard. It should be refactored to
# a function.
for step in self.steps:
if isinstance(step, create_flow.ConfigureNodegroups):
ng_action = step.action
template_ngs = self.template.node_groups
if 'forms_ids' in request.POST:
continue
ng_action.groups = []
for i, templ_ng in enumerate(template_ngs):
group_name = "group_name_%d" % i
template_id = "template_id_%d" % i
count = "count_%d" % i
serialized = "serialized_%d" % i
# save the original node group with all its fields in
# case the template id is missing
serialized_val = base64.urlsafe_b64encode(
json.dumps(wf_helpers.clean_node_group(templ_ng)))
ng = {
"name": templ_ng["name"],
"count": templ_ng["count"],
"id": i,
"deletable": "true",
"serialized": serialized_val
}
if "node_group_template_id" in templ_ng:
ng["template_id"] = templ_ng[
"node_group_template_id"]
ng_action.groups.append(ng)
wf_helpers.build_node_group_fields(
ng_action, group_name, template_id, count,
serialized)
elif isinstance(step, create_flow.GeneralConfig):
fields = step.action.fields
fields["cluster_template_name"].initial = (
self.template.name + "-copy")
fields['use_autoconfig'].initial = (
self.template.use_autoconfig)
fields["description"].initial = self.template.description
except Exception:
exceptions.handle(request,
_("Unable to fetch template to copy."))

View File

@ -0,0 +1,337 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import base64
import json
import logging
from django.utils.translation import ugettext_lazy as _
from saharaclient.api import base as api_base
from horizon import exceptions
from horizon import forms
from horizon import workflows
from sahara_dashboard.api import sahara as saharaclient
from sahara_dashboard.content.data_processing. \
utils import helpers as helpers
from sahara_dashboard.content.data_processing. \
utils import anti_affinity as aa
import sahara_dashboard.content.data_processing. \
utils.workflow_helpers as whelpers
LOG = logging.getLogger(__name__)
class SelectPluginAction(workflows.Action):
hidden_create_field = forms.CharField(
required=False,
widget=forms.HiddenInput(attrs={"class": "hidden_create_field"}))
def __init__(self, request, *args, **kwargs):
super(SelectPluginAction, self).__init__(request, *args, **kwargs)
try:
plugins = saharaclient.plugin_list(request)
except Exception:
plugins = []
exceptions.handle(request,
_("Unable to fetch plugin list."))
plugin_choices = [(plugin.name, plugin.title) for plugin in plugins]
self.fields["plugin_name"] = forms.ChoiceField(
label=_("Plugin name"),
choices=plugin_choices,
widget=forms.Select(attrs={"class": "plugin_name_choice"}))
for plugin in plugins:
field_name = plugin.name + "_version"
choice_field = forms.ChoiceField(
label=_("Version"),
choices=[(version, version) for version in plugin.versions],
widget=forms.Select(
attrs={"class": "plugin_version_choice "
+ field_name + "_choice"})
)
self.fields[field_name] = choice_field
class Meta(object):
name = _("Select plugin and hadoop version for cluster template")
help_text_template = ("project/data_processing.cluster_templates/"
"_create_general_help.html")
class SelectPlugin(workflows.Step):
action_class = SelectPluginAction
class CreateClusterTemplate(workflows.Workflow):
slug = "create_cluster_template"
name = _("Create Cluster Template")
finalize_button_name = _("Next")
success_message = _("Created")
failure_message = _("Could not create")
success_url = "horizon:project:data_processing.cluster_templates:index"
default_steps = (SelectPlugin,)
class GeneralConfigAction(workflows.Action):
hidden_configure_field = forms.CharField(
required=False,
widget=forms.HiddenInput(attrs={"class": "hidden_configure_field"}))
hidden_to_delete_field = forms.CharField(
required=False,
widget=forms.HiddenInput(attrs={"class": "hidden_to_delete_field"}))
cluster_template_name = forms.CharField(label=_("Template Name"))
description = forms.CharField(label=_("Description"),
required=False,
widget=forms.Textarea(attrs={'rows': 4}))
use_autoconfig = forms.BooleanField(
label=_("Auto-configure"),
help_text=_("If selected, instances of a cluster will be "
"automatically configured during creation. Otherwise you "
"should manually specify configuration values"),
required=False,
widget=forms.CheckboxInput(),
initial=True,
)
anti_affinity = aa.anti_affinity_field()
def __init__(self, request, *args, **kwargs):
super(GeneralConfigAction, self).__init__(request, *args, **kwargs)
plugin, hadoop_version = whelpers.\
get_plugin_and_hadoop_version(request)
self.fields["plugin_name"] = forms.CharField(
widget=forms.HiddenInput(),
initial=plugin
)
self.fields["hadoop_version"] = forms.CharField(
widget=forms.HiddenInput(),
initial=hadoop_version
)
populate_anti_affinity_choices = aa.populate_anti_affinity_choices
def get_help_text(self):
extra = dict()
plugin, hadoop_version = whelpers\
.get_plugin_and_hadoop_version(self.request)
extra["plugin_name"] = plugin
extra["hadoop_version"] = hadoop_version
return super(GeneralConfigAction, self).get_help_text(extra)
def clean(self):
cleaned_data = super(GeneralConfigAction, self).clean()
if cleaned_data.get("hidden_configure_field", None) \
== "create_nodegroup":
self._errors = dict()
return cleaned_data
class Meta(object):
name = _("Details")
help_text_template = ("project/data_processing.cluster_templates/"
"_configure_general_help.html")
class GeneralConfig(workflows.Step):
action_class = GeneralConfigAction
contributes = ("hidden_configure_field", )
def contribute(self, data, context):
for k, v in data.items():
context["general_" + k] = v
post = self.workflow.request.POST
context['anti_affinity_info'] = post.getlist("anti_affinity")
return context
class ConfigureNodegroupsAction(workflows.Action):
hidden_nodegroups_field = forms.CharField(
required=False,
widget=forms.HiddenInput(attrs={"class": "hidden_nodegroups_field"}))
forms_ids = forms.CharField(
required=False,
widget=forms.HiddenInput())
def __init__(self, request, *args, **kwargs):
super(ConfigureNodegroupsAction, self). \
__init__(request, *args, **kwargs)
plugin = request.REQUEST.get("plugin_name")
version = request.REQUEST.get("hadoop_version")
if plugin and not version:
version_name = plugin + "_version"
version = request.REQUEST.get(version_name)
if not plugin or not version:
self.templates = saharaclient.nodegroup_template_find(request)
else:
self.templates = saharaclient.nodegroup_template_find(
request, plugin_name=plugin, hadoop_version=version)
deletable = request.REQUEST.get("deletable", dict())
request_source = None
if 'forms_ids' in request.POST:
request_source = request.POST
elif 'forms_ids' in request.REQUEST:
request_source = request.REQUEST
if request_source:
self.groups = []
for id in json.loads(request_source['forms_ids']):
group_name = "group_name_" + str(id)
template_id = "template_id_" + str(id)
count = "count_" + str(id)
serialized = "serialized_" + str(id)
self.groups.append({"name": request_source[group_name],
"template_id": request_source[template_id],
"count": request_source[count],
"id": id,
"deletable": deletable.get(
request_source[group_name], "true"),
"serialized": request_source[serialized]})
whelpers.build_node_group_fields(self,
group_name,
template_id,
count,
serialized)
def clean(self):
cleaned_data = super(ConfigureNodegroupsAction, self).clean()
if cleaned_data.get("hidden_nodegroups_field", None) \
== "create_nodegroup":
self._errors = dict()
return cleaned_data
class Meta(object):
name = _("Node Groups")
class ConfigureNodegroups(workflows.Step):
action_class = ConfigureNodegroupsAction
contributes = ("hidden_nodegroups_field", )
template_name = ("project/data_processing.cluster_templates/"
"cluster_node_groups_template.html")
def contribute(self, data, context):
for k, v in data.items():
context["ng_" + k] = v
return context
class ConfigureClusterTemplate(whelpers.ServiceParametersWorkflow,
whelpers.StatusFormatMixin):
slug = "configure_cluster_template"
name = _("Create Cluster Template")
finalize_button_name = _("Create")
success_message = _("Created Cluster Template %s")
name_property = "general_cluster_template_name"
success_url = "horizon:project:data_processing.cluster_templates:index"
default_steps = (GeneralConfig,
ConfigureNodegroups)
def __init__(self, request, context_seed, entry_point, *args, **kwargs):
ConfigureClusterTemplate._cls_registry = set([])
hlps = helpers.Helpers(request)
plugin, hadoop_version = whelpers.\
get_plugin_and_hadoop_version(request)
general_parameters = hlps.get_cluster_general_configs(
plugin,
hadoop_version)
service_parameters = hlps.get_targeted_cluster_configs(
plugin,
hadoop_version)
self._populate_tabs(general_parameters, service_parameters)
super(ConfigureClusterTemplate, self).__init__(request,
context_seed,
entry_point,
*args, **kwargs)
def is_valid(self):
steps_valid = True
for step in self.steps:
if not step.action.is_valid():
steps_valid = False
step.has_errors = True
errors_fields = list(step.action.errors.keys())
step.action.errors_fields = errors_fields
if not steps_valid:
return steps_valid
return self.validate(self.context)
def handle(self, request, context):
try:
node_groups = []
configs_dict = whelpers.parse_configs_from_context(context,
self.defaults)
ids = json.loads(context['ng_forms_ids'])
for id in ids:
name = context['ng_group_name_' + str(id)]
template_id = context['ng_template_id_' + str(id)]
count = context['ng_count_' + str(id)]
raw_ng = context.get("ng_serialized_" + str(id))
if raw_ng and raw_ng != 'null':
ng = json.loads(base64.urlsafe_b64decode(str(raw_ng)))
else:
ng = dict()
ng["name"] = name
ng["count"] = count
if template_id and template_id != u'None':
ng["node_group_template_id"] = template_id
node_groups.append(ng)
plugin, hadoop_version = whelpers.\
get_plugin_and_hadoop_version(request)
# TODO(nkonovalov): Fix client to support default_image_id
saharaclient.cluster_template_create(
request,
context["general_cluster_template_name"],
plugin,
hadoop_version,
context["general_description"],
configs_dict,
node_groups,
context["anti_affinity_info"],
use_autoconfig=context['general_use_autoconfig']
)
hlps = helpers.Helpers(request)
if hlps.is_from_guide():
request.session["guide_cluster_template_name"] = (
context["general_cluster_template_name"])
self.success_url = (
"horizon:project:data_processing.wizard:cluster_guide")
return True
except api_base.APIException as e:
self.error_description = str(e)
return False
except Exception:
exceptions.handle(request,
_("Cluster template creation failed"))
return False

View File

@ -0,0 +1,103 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import base64
import json
import logging
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from horizon import forms
from sahara_dashboard.api import sahara as saharaclient
import sahara_dashboard.content.data_processing. \
cluster_templates.workflows.create as create_flow
import sahara_dashboard.content.data_processing. \
cluster_templates.workflows.copy as copy_flow
import sahara_dashboard.content.data_processing. \
utils.workflow_helpers as whelpers
LOG = logging.getLogger(__name__)
class EditClusterTemplate(copy_flow.CopyClusterTemplate):
success_message = _("Cluster Template %s updated")
entry_point = "generalconfigaction"
finalize_button_name = _("Update")
name = _("Edit Cluster Template")
def __init__(self, request, context_seed, entry_point, *args, **kwargs):
try:
super(EditClusterTemplate, self).__init__(request, context_seed,
entry_point, *args,
**kwargs)
for step in self.steps:
if isinstance(step, create_flow.GeneralConfig):
fields = step.action.fields
fields["cluster_template_name"].initial = (
self.template.name)
fields["cluster_template_id"] = forms.CharField(
widget=forms.HiddenInput(),
initial=self.cluster_template_id)
except Exception:
exceptions.handle(request,
_("Unable to fetch template to edit."))
def handle(self, request, context):
try:
node_groups = []
configs_dict = whelpers.parse_configs_from_context(context,
self.defaults)
ids = json.loads(context['ng_forms_ids'])
for id in ids:
name = context['ng_group_name_' + str(id)]
template_id = context['ng_template_id_' + str(id)]
count = context['ng_count_' + str(id)]
raw_ng = context.get("ng_serialized_" + str(id))
if raw_ng and raw_ng != 'null':
ng = json.loads(base64.urlsafe_b64decode(str(raw_ng)))
else:
ng = dict()
ng["name"] = name
ng["count"] = count
if template_id and template_id != u'None':
ng["node_group_template_id"] = template_id
node_groups.append(ng)
plugin, hadoop_version = whelpers. \
get_plugin_and_hadoop_version(request)
saharaclient.cluster_template_update(
request=request,
ct_id=self.cluster_template_id,
name=context["general_cluster_template_name"],
plugin_name=plugin,
hadoop_version=hadoop_version,
description=context["general_description"],
cluster_configs=configs_dict,
node_groups=node_groups,
anti_affinity=context["anti_affinity_info"],
use_autoconfig=context['general_use_autoconfig']
)
return True
except exceptions.Conflict as e:
self.error_description = str(e)
return False
except Exception:
exceptions.handle(request,
_("Cluster template update failed"))
return False

View File

@ -0,0 +1,28 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from django.utils.translation import ugettext_lazy as _
import horizon
from openstack_dashboard.dashboards.project import dashboard
class ClustersPanel(horizon.Panel):
name = _("Clusters")
slug = 'data_processing.clusters'
permissions = (('openstack.services.data-processing',
'openstack.services.data_processing'),)
dashboard.Project.register(ClustersPanel)

View File

@ -0,0 +1,177 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.http import Http404 # noqa
from django.template.loader import render_to_string
from django.utils.translation import ugettext_lazy as _
from django.utils.translation import ungettext_lazy
from horizon import messages
from horizon import tables
from horizon.tables import base as tables_base
from sahara_dashboard.api import sahara as saharaclient
from saharaclient.api import base as api_base
LOG = logging.getLogger(__name__)
class ClustersFilterAction(tables.FilterAction):
filter_type = "server"
filter_choices = (('name', _("Name"), True),
('status', _("Status"), True))
class ClusterGuide(tables.LinkAction):
name = "cluster_guide"
verbose_name = _("Cluster Creation Guide")
url = "horizon:project:data_processing.wizard:cluster_guide"
class CreateCluster(tables.LinkAction):
name = "create"
verbose_name = _("Launch Cluster")
url = "horizon:project:data_processing.clusters:create-cluster"
classes = ("ajax-modal",)
icon = "plus"
class ScaleCluster(tables.LinkAction):
name = "scale"
verbose_name = _("Scale Cluster")
url = "horizon:project:data_processing.clusters:scale"
classes = ("ajax-modal", "btn-edit")
def allowed(self, request, cluster=None):
return cluster.status == "Active"
class DeleteCluster(tables.DeleteAction):
@staticmethod
def action_present(count):
return ungettext_lazy(
u"Delete Cluster",
u"Delete Clusters",
count
)
@staticmethod
def action_past(count):
return ungettext_lazy(
u"Deleted Cluster",
u"Deleted Clusters",
count
)
def delete(self, request, obj_id):
saharaclient.cluster_delete(request, obj_id)
class UpdateRow(tables.Row):
ajax = True
def get_data(self, request, instance_id):
try:
return saharaclient.cluster_get(request, instance_id)
except api_base.APIException as e:
if e.error_code == 404:
raise Http404
else:
messages.error(request,
_("Unable to update row"))
def get_instances_count(cluster):
return sum([len(ng["instances"])
for ng in cluster.node_groups])
class RichErrorCell(tables_base.Cell):
@property
def status(self):
# The error cell values becomes quite complex and cannot be handled
# correctly with STATUS_CHOICES. Handling that explicitly.
status = self.datum.status.lower()
if status == "error":
return False
elif status == "active":
return True
return None
def get_rich_status_info(cluster):
return {
"status": cluster.status,
"status_description": cluster.status_description
}
def rich_status_filter(status_dict):
# Render the status "as is" if no description is provided.
if not status_dict["status_description"]:
return status_dict["status"]
# Error is rendered with a template containing an error description.
return render_to_string(
"project/data_processing.clusters/_rich_status.html", status_dict)
class ConfigureCluster(tables.LinkAction):
name = "configure"
verbose_name = _("Configure Cluster")
url = "horizon:project:data_processing.clusters:configure-cluster"
classes = ("ajax-modal", "configure-cluster-btn")
icon = "plus"
attrs = {"style": "display: none"}
class ClustersTable(tables.DataTable):
name = tables.Column("name",
verbose_name=_("Name"),
link=("horizon:project:data_processing."
"clusters:details"))
plugin = tables.Column("plugin_name",
verbose_name=_("Plugin"))
version = tables.Column("hadoop_version",
verbose_name=_("Version"))
# Status field need the whole cluster object to build the rich status.
status = tables.Column(get_rich_status_info,
verbose_name=_("Status"),
status=True,
filters=(rich_status_filter,))
instances_count = tables.Column(get_instances_count,
verbose_name=_("Instances Count"))
class Meta(object):
name = "clusters"
verbose_name = _("Clusters")
row_class = UpdateRow
cell_class = RichErrorCell
status_columns = ["status"]
table_actions = (ClusterGuide,
CreateCluster,
ConfigureCluster,
DeleteCluster,
ClustersFilterAction)
row_actions = (ScaleCluster,
DeleteCluster,)

View File

@ -0,0 +1,195 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from horizon import tables
from horizon import tabs
from sahara_dashboard.content.data_processing.utils \
import workflow_helpers as helpers
from openstack_dashboard.api import glance
from openstack_dashboard.api import network
from openstack_dashboard.api import neutron
from openstack_dashboard.api import nova
from sahara_dashboard.api import sahara as saharaclient
LOG = logging.getLogger(__name__)
class GeneralTab(tabs.Tab):
name = _("General Info")
slug = "cluster_details_tab"
template_name = "project/data_processing.clusters/_details.html"
def get_context_data(self, request):
cluster_id = self.tab_group.kwargs['cluster_id']
cluster_info = {}
try:
sahara = saharaclient.client(request)
cluster = sahara.clusters.get(cluster_id)
for info_key, info_val in cluster.info.items():
for key, val in info_val.items():
if str(val).startswith(('http://', 'https://')):
cluster.info[info_key][key] = build_link(val)
base_image = glance.image_get(request,
cluster.default_image_id)
if getattr(cluster, 'cluster_template_id', None):
cluster_template = saharaclient.safe_call(
sahara.cluster_templates.get,
cluster.cluster_template_id)
else:
cluster_template = None
if getattr(cluster, 'neutron_management_network', None):
net_id = cluster.neutron_management_network
network = neutron.network_get(request, net_id)
net_name = network.name_or_id
else:
net_name = None
cluster_info.update({"cluster": cluster,
"base_image": base_image,
"cluster_template": cluster_template,
"network": net_name})
except Exception as e:
LOG.error("Unable to fetch cluster details: %s" % str(e))
return cluster_info
def build_link(url):
return "<a href='" + url + "' target=\"_blank\">" + url + "</a>"
class NodeGroupsTab(tabs.Tab):
name = _("Node Groups")
slug = "cluster_nodegroups_tab"
template_name = (
"project/data_processing.clusters/_nodegroups_details.html")
def get_context_data(self, request):
cluster_id = self.tab_group.kwargs['cluster_id']
try:
sahara = saharaclient.client(request)
cluster = sahara.clusters.get(cluster_id)
for ng in cluster.node_groups:
if ng["flavor_id"]:
ng["flavor_name"] = (
nova.flavor_get(request, ng["flavor_id"]).name)
if ng["floating_ip_pool"]:
ng["floating_ip_pool_name"] = (
self._get_floating_ip_pool_name(
request, ng["floating_ip_pool"]))
if ng.get("node_group_template_id", None):
ng["node_group_template"] = saharaclient.safe_call(
sahara.node_group_templates.get,
ng["node_group_template_id"])
ng["security_groups_full"] = helpers.get_security_groups(
request, ng["security_groups"])
except Exception:
cluster = {}
exceptions.handle(request,
_("Unable to get node group details."))
return {"cluster": cluster}
def _get_floating_ip_pool_name(self, request, pool_id):
pools = [pool for pool in network.floating_ip_pools_list(
request) if pool.id == pool_id]
return pools[0].name if pools else pool_id
class Instance(object):
def __init__(self, name=None, id=None, internal_ip=None,
management_ip=None):
self.name = name
self.id = id
self.internal_ip = internal_ip
self.management_ip = management_ip
class InstancesTable(tables.DataTable):
name = tables.Column("name",
link="horizon:project:instances:detail",
verbose_name=_("Name"))
internal_ip = tables.Column("internal_ip",
verbose_name=_("Internal IP"))
management_ip = tables.Column("management_ip",
verbose_name=_("Management IP"))
class Meta(object):
name = "cluster_instances"
verbose_name = _("Cluster Instances")
class InstancesTab(tabs.TableTab):
name = _("Instances")
slug = "cluster_instances_tab"
template_name = "project/data_processing.clusters/_instances_details.html"
table_classes = (InstancesTable, )
def get_cluster_instances_data(self):
cluster_id = self.tab_group.kwargs['cluster_id']
try:
sahara = saharaclient.client(self.request)
cluster = sahara.clusters.get(cluster_id)
instances = []
for ng in cluster.node_groups:
for instance in ng["instances"]:
instances.append(Instance(
name=instance["instance_name"],
id=instance["instance_id"],
internal_ip=instance.get("internal_ip",
"Not assigned"),
management_ip=instance.get("management_ip",
"Not assigned")))
except Exception:
instances = []
exceptions.handle(self.request,
_("Unable to fetch instance details."))
return instances
class EventLogTab(tabs.Tab):
name = _("Cluster Events")
slug = "cluster_event_log"
template_name = "project/data_processing.clusters/_event_log.html"
def get_context_data(self, request, **kwargs):
cluster_id = self.tab_group.kwargs['cluster_id']
kwargs["cluster_id"] = cluster_id
kwargs['data_update_url'] = request.get_full_path()
return kwargs
class ClusterDetailsTabs(tabs.TabGroup):
slug = "cluster_details"
tabs = (GeneralTab, NodeGroupsTab, InstancesTab, EventLogTab)
sticky = True

View File

@ -0,0 +1,20 @@
{% load i18n horizon %}
<div class="well">
<p>
{% blocktrans %}This Cluster will be started with:{% endblocktrans %}
<br >
<b>{% blocktrans %}Plugin{% endblocktrans %}</b>: {{ plugin_name }}
<br />
<b>{% blocktrans %}Version{% endblocktrans %}</b>: {{ hadoop_version }}
<br />
</p>
<p>
{% blocktrans %}Cluster can be launched using existing Cluster Templates.{% endblocktrans %}
</p>
<p>
{% blocktrans %}The Cluster object should specify OpenStack Image to boot instances for Cluster.{% endblocktrans %}
</p>
<p>
{% blocktrans %}User has to choose a keypair to have access to clusters instances.{% endblocktrans %}
</p>
</div>

View File

@ -0,0 +1,22 @@
{% extends "horizon/common/_modal_form.html" %}
{% load i18n %}
{% block form_id %}create_cluster_form{% endblock %}
{% block form_action %}{% url 'horizon:project:data_processing.clusters:create' %}{% endblock %}
{% block modal-header %}{% trans "Launch Cluster" %}{% endblock %}
{% block modal-body %}
<div class="left">
<fieldset>
{% include "horizon/common/_form_fields.html" %}
</fieldset>
</div>
{% endblock %}
{% block modal-footer %}
<input class="btn btn-primary pull-right" id="create_cluster_btn" type="submit" value="{% trans " Done" %}"/>
<a href="{% url 'horizon:project:data_processing.clusters:index' %}" class="btn btn-default secondary cancel close">{% trans "Cancel" %}</a>
{% endblock %}

View File

@ -0,0 +1,4 @@
{% load i18n %}
<p class="well">
{% trans "Select a plugin and version for a new Cluster." %}
</p>

View File

@ -0,0 +1,92 @@
{% load i18n sizeformat %}
<div class="detail">
<dl class="dl-horizontal">
<dt>{% trans "Name" %}</dt>
<dd>{{ cluster.name }}</dd>
<dt>{% trans "ID" %}</dt>
<dd>{{ cluster.id }}</dd>
<dt>{% trans "Description" %}</dt>
<dd>{{ cluster.description|default:_("None") }}</dd>
<dt>{% trans "Status" %}</dt>
<dd>{{ cluster.status }}</dd>
</dl>
{% if cluster.error_description %}
<h4>{% trans "Error Details" %}</h4>
<p class="well">
{{ cluster.error_description }}
</p>
{% endif %}
<dl class="dl-horizontal">
<dt>{% trans "Plugin" %}</dt>
<dd><a href="{% url 'horizon:project:data_processing.data_plugins:details' cluster.plugin_name %}">{{ cluster.plugin_name }}</a></dd>
<dt>{% trans "Version" %}</dt>
<dd>{{ cluster.hadoop_version }}</dd>
</dl>
<dl class="dl-horizontal">
<dt>{% trans "Template" %}</dt>
{% if cluster_template %}
<dd><a href="{% url 'horizon:project:data_processing.cluster_templates:details' cluster_template.id %}">{{ cluster_template.name }} </a></dd>
{% else %}
<dd>{% trans "Template not specified" %}</dd>
{% endif %}
<dt>{% trans "Base Image" %}</dt>
<dd><a href="{% url 'horizon:project:images:images:detail' base_image.id %}">{{ base_image.name }}</a></dd>
{% if network %}
<dt>{% trans "Neutron Management Network" %}</dt>
<dd>{{ network }}</dd>
{% endif %}
<dt>{% trans "Keypair" %}</dt>
<dd>{{ cluster.user_keypair_id }}</dd>
<dt>{% trans "Use auto-configuration" %}</dt>
<dd>{{ cluster.use_autoconfig }}</dd>
</dl>
<dl class="dl-horizontal">
<dt>{% trans "Anti-affinity enabled for" %}</dt>
{% if cluster.anti_affinity %}
<dd>
<ul class="list-bullet">
{% for process in cluster.anti_affinity %}
<li>{{ process }}</li>
{% endfor %}
</ul>
</dd>
{% else %}
<h6>{% trans "no processes" %}</h6>
{% endif %}
</dl>
<dl class="dl-horizontal">
<dt>{% trans "Node Configurations" %}</dt>
{% if cluster.cluster_configs %}
<dd>
{% for service, service_conf in cluster.cluster_configs.items %}
<h4>{{ service }}</h4>
{% if service_conf %}
<ul>
{% for conf_name, conf_value in service_conf.items %}
<li>{% blocktrans %}{{ conf_name }}: {{ conf_value }}{% endblocktrans %}</li>
{% endfor %}
</ul>
{% else %}
<h6>{% trans "No configurations" %}</h6>
{% endif %}
{% endfor %}
</dd>
{% else %}
<dd>{% trans "Cluster configurations are not specified" %}</dd>
{% endif %}
</dl>
<dl class="dl-horizontal">
{% for info_key, info_val in cluster.info.items %}
<dt>{{ info_key }}</dt>
{% for key, val in info_val.items %}
<dd>
{% autoescape off %}{% blocktrans %}{{ key }}: {{ val }}{% endblocktrans %}{% endautoescape %}
</dd>
{% endfor %}
{% endfor %}
</dl>
</div>

View File

@ -0,0 +1,62 @@
{% load i18n %}
<h4>{% trans "Cluster provision steps" %}</h4>
<table id="steps_table" class="table table-bordered datatable">
<thead>
<tr>
<th>{% trans "Step Description" %}</th>
<th>{% trans "Started at" %}</th>
<th>{% trans "Duration" %}</th>
<th>{% trans "Progress" %}</th>
<th>{% trans "Status" %}</th>
</tr>
</thead>
<tbody id="steps_body">
</tbody>
</table>
<div id="events_modal" class="modal fade">
<div class="modal-dialog" style="width: 85%">
<div class="modal-content">
<div class="modal-header">
<button type="button" class="close" data-dismiss="modal" aria-hidden="true">×</button>
<h4 id="events_modal_header"></h4>
<span id="modal_status_marker"></span>
</div>
<div class="modal-body">
<table id="events_table" class="table table-bordered datatable">
<thead>
<tr>
<th>{% trans "Node Group" %}</th>
<th>{% trans "Instance" %}</th>
<th>{% trans "Event time" %}</th>
<th>{% trans "Info" %}</th>
<th>{% trans "Status" %}</th>
</tr>
</thead>
<tbody id="events_body">
</tbody>
</table>
</div>
</div>
</div>
</div>
<script type="text/javascript">
$(function () {
// Initialize everything.
horizon.event_log.cluster_id = "{{ cluster_id }}";
horizon.event_log.data_update_url = "{{ data_update_url }}";
horizon.event_log.fetch_update_events();
});
$(".show_events_btn").live("click", function () {
// Bind "show events" buttons to modals.
horizon.event_log.modal_step_id = $(this).data("step-id");
horizon.event_log.clear_events();
horizon.event_log.clear_modal_status();
horizon.event_log.update_events_rows(horizon.event_log.cached_data);
});
</script>

View File

@ -0,0 +1,4 @@
{% load i18n sizeformat %}
<div class="detail">
{{ cluster_instances_table.render }}
</div>

View File

@ -0,0 +1,82 @@
{% load i18n sizeformat %}
<div class="detail">
{% for node_group in cluster.node_groups %}
<dl class="well">
<h4>{% blocktrans with node_group_name=node_group.name %}Name: {{ node_group_name }}{% endblocktrans %}</h4>
<dt>{% trans "Number of Nodes" %}</dt>
<dd>{{ node_group.count }}</dd>
<dt>{% trans "Flavor" %}</dt>
<dd>{{ node_group.flavor_name|default:_("Flavor is not specified") }}</dd>
{% if node_group.floating_ip_pool %}
<dt>{% trans "Floating IP Pool" %}</dt>
<dd><a href="{% url 'horizon:project:networks:detail' node_group.floating_ip_pool %}">{{ node_group.floating_ip_pool_name }}</a></dd>
{% endif %}
<dt>{% trans "Template" %}</dt>
{% if node_group.node_group_template_id %}
<dd><a href="{% url 'horizon:project:data_processing.nodegroup_templates:details' node_group.node_group_template_id %}">{{ node_group.node_group_template.name }} </a></dd>
{% else %}
<dd>{% trans "Template not specified" %}</dd>
{% endif %}
<dt>{% trans "Use auto-configuration" %}</dt>
<dd>{{ node_group.use_autoconfig }}</dd>
<dt>{% trans "Proxy Gateway" %}</dt>
<dd>{{ node_group.is_proxy_gateway|yesno }}</dd>
<dt>{% trans "Auto Security Group" %}</dt>
<dd>{{ node_group.auto_security_group|yesno }}</dd>
<dt>{% trans "Security Groups" %}</dt>
<dd>
<ul class="list-bullet">
{% for group in node_group.security_groups_full %}
{% if group.id %}
<li><a href="{% url 'horizon:project:access_and_security:security_groups:detail' group.id %}">{{ group.name }}</a></li>
{% else %}
<li>{{ group.name }}</li>
{% endif %}
{% endfor %}
</ul>
</dd>
<dt>{% trans "Node Processes" %}</dt>
{% if node_group.node_processes %}
<dd>
<ul class="list-bullet">
{% for process in node_group.node_processes %}
<li>{{ process }}</li>
{% endfor %}
</ul>
</dd>
{% else %}
<dd>{% trans "Node processes are not specified" %}</dd>
{% endif %}
<dt>{% trans "Node Configurations" %}</dt>
{% if node_group.node_configs %}
<dd>
{% for service, service_conf in node_group.node_configs.items %}
<h4>{{ service }}</h4>
{% if service_conf %}
<ul>
{% for conf_name, conf_value in service_conf.items %}
<li>{% blocktrans %}{{ conf_name }}: {{ conf_value }}{% endblocktrans %}</li>
{% endfor %}
</ul>
{% else %}
<h6>{% trans "No configurations" %}</h6>
{% endif %}
{% endfor %}
</dd>
{% else %}
<dd>{% trans "Node configurations are not specified" %}</dd>
{% endif %}
</dl>
{% endfor %}
</div>

View File

@ -0,0 +1,6 @@
{{ status }}&nbsp;<span
class="fa fa-question-circle"
data-toggle="tooltip"
data-placement="right"
title="{{ status_description }}">
</span>

View File

@ -0,0 +1,58 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Data Processing" %}{% endblock %}
{% block main %}
<div class="cluster_templates">
{{ clusters_table.render }}
</div>
<script type="text/javascript">
addHorizonLoadEvent(function () {
horizon.modals.addModalInitFunction(function (modal) {
var $navbar = $(modal).find(".nav-tabs");
if ($navbar.find("li").size() == 1) {
// hide tab bar for plugin/version modal wizard
$navbar.hide();
}
$(".hidden_nodegroups_field").val("");
$(".hidden_configure_field").val("");
lower_limit = 0;
$(".count-field").change();
if ($(modal).find(".hidden_create_field").length > 0) {
var form = $(".hidden_create_field").closest("form");
var successful = false;
form.submit(function (e) {
var oldHref = $(".configure-cluster-btn")[0].href;
var plugin = $("#id_plugin_name option:selected").val();
var version = $("#id_" + plugin + "_version option:selected").val();
form.find(".close").click();
$(".configure-cluster-btn")[0].href = oldHref +
"?plugin_name=" + encodeURIComponent(plugin) +
"&hadoop_version=" + encodeURIComponent(version);
$(".configure-cluster-btn").click();
$(".configure-cluster-btn")[0].href = oldHref;
return false;
});
$(".plugin_version_choice").closest(".form-group").hide();
}
//display version for selected plugin
$(document).on('change', '.plugin_name_choice', switch_versions);
function switch_versions() {
$(".plugin_version_choice").closest(".form-group").hide();
var plugin = $(this);
$("." + plugin.val() + "_version_choice").closest(".form-group").show();
}
$(".plugin_name_choice").change();
});
});
</script>
{% endblock %}

View File

@ -0,0 +1,7 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Launch Cluster" %}{% endblock %}
{% block main %}
{% include 'horizon/common/_workflow.html' %}
{% endblock %}

View File

@ -0,0 +1,7 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Launch Cluster" %}{% endblock %}
{% block main %}
{% include 'horizon/common/_workflow.html' %}
{% endblock %}

View File

@ -0,0 +1,7 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Launch Cluster" %}{% endblock %}
{% block main %}
{% include 'templates/data_processing.clusters/_create_cluster.html' %}
{% endblock %}

View File

@ -0,0 +1,7 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Scale Cluster" %}{% endblock %}
{% block main %}
{% include 'horizon/common/_workflow.html' %}
{% endblock %}

View File

@ -0,0 +1,93 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from django.core.urlresolvers import reverse
from django import http
from mox3.mox import IsA # noqa
from oslo_serialization import jsonutils
from openstack_dashboard.test import helpers as test
from sahara_dashboard import api
INDEX_URL = reverse('horizon:project:data_processing.clusters:index')
DETAILS_URL = reverse(
'horizon:project:data_processing.clusters:details', args=['id'])
class DataProcessingClusterTests(test.TestCase):
@test.create_stubs({api.sahara: ('cluster_list',)})
def test_index(self):
api.sahara.cluster_list(IsA(http.HttpRequest), {}) \
.AndReturn(self.clusters.list())
self.mox.ReplayAll()
res = self.client.get(INDEX_URL)
self.assertTemplateUsed(
res, 'project/data_processing.clusters/clusters.html')
self.assertContains(res, 'Clusters')
self.assertContains(res, 'Name')
@test.create_stubs({api.sahara: ('cluster_template_list', 'image_list')})
def test_launch_cluster_get_nodata(self):
api.sahara.cluster_template_list(IsA(http.HttpRequest)) \
.AndReturn([])
api.sahara.image_list(IsA(http.HttpRequest)) \
.AndReturn([])
self.mox.ReplayAll()
url = reverse(
'horizon:project:data_processing.clusters:configure-cluster')
res = self.client.get("%s?plugin_name=shoes&hadoop_version=1.1" % url)
self.assertContains(res, "No Images Available")
self.assertContains(res, "No Templates Available")
@test.create_stubs({api.sahara: ('cluster_get',)})
def test_event_log_tab(self):
cluster = self.clusters.list()[-1]
api.sahara.cluster_get(IsA(http.HttpRequest),
"cl2", show_progress=True).AndReturn(cluster)
self.mox.ReplayAll()
url = reverse(
'horizon:project:data_processing.clusters:events', args=["cl2"])
res = self.client.get(url)
data = jsonutils.loads(res.content)
self.assertIn("provision_steps", data)
self.assertEqual(data["need_update"], False)
step_0 = data["provision_steps"][0]
self.assertEqual(2, step_0["completed"])
self.assertEqual(2, len(step_0["events"]))
for evt in step_0["events"]:
self.assertEqual(True, evt["successful"])
step_1 = data["provision_steps"][1]
self.assertEqual(3, step_1["completed"])
self.assertEqual(0, len(step_1["events"]))
@test.create_stubs({api.sahara: ('cluster_list',
'cluster_delete')})
def test_delete(self):
cluster = self.clusters.first()
api.sahara.cluster_list(IsA(http.HttpRequest), {}) \
.AndReturn(self.clusters.list())
api.sahara.cluster_delete(IsA(http.HttpRequest), cluster.id)
self.mox.ReplayAll()
form_data = {'action': 'clusters__delete__%s' % cluster.id}
res = self.client.post(INDEX_URL, form_data)
self.assertNoFormErrors(res)
self.assertRedirectsNoFollow(res, INDEX_URL)
self.assertMessageCount(success=1)

View File

@ -0,0 +1,40 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from django.conf.urls import patterns
from django.conf.urls import url
import sahara_dashboard.content.data_processing.clusters.views as views
urlpatterns = patterns('',
url(r'^$', views.ClustersView.as_view(),
name='index'),
url(r'^$', views.ClustersView.as_view(),
name='clusters'),
url(r'^create-cluster$',
views.CreateClusterView.as_view(),
name='create-cluster'),
url(r'^configure-cluster$',
views.ConfigureClusterView.as_view(),
name='configure-cluster'),
url(r'^(?P<cluster_id>[^/]+)$',
views.ClusterDetailsView.as_view(),
name='details'),
url(r'^(?P<cluster_id>[^/]+)/events$',
views.ClusterEventsView.as_view(),
name='events'),
url(r'^(?P<cluster_id>[^/]+)/scale$',
views.ScaleClusterView.as_view(),
name='scale'))

View File

@ -0,0 +1,225 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from datetime import datetime
import json
import logging
from django.http import HttpResponse
from django.utils.translation import ugettext as _
from django.views.generic import base as django_base
import six
from horizon import exceptions
from horizon import tables
from horizon import tabs
from horizon.utils import memoized
from horizon.utils.urlresolvers import reverse # noqa
from horizon import workflows
from sahara_dashboard.api import sahara as saharaclient
import sahara_dashboard.content.data_processing.clusters. \
tables as c_tables
import sahara_dashboard.content.data_processing.clusters. \
tabs as _tabs
import sahara_dashboard.content.data_processing.clusters. \
workflows.create as create_flow
import sahara_dashboard.content.data_processing.clusters. \
workflows.scale as scale_flow
from saharaclient.api.base import APIException
LOG = logging.getLogger(__name__)
class ClustersView(tables.DataTableView):
table_class = c_tables.ClustersTable
template_name = 'project/data_processing.clusters/clusters.html'
page_title = _("Clusters")
def get_data(self):
try:
search_opts = {}
filter = self.get_server_filter_info(self.request)
if filter['value'] and filter['field']:
search_opts = {filter['field']: filter['value']}
clusters = saharaclient.cluster_list(self.request, search_opts)
except Exception:
clusters = []
exceptions.handle(self.request,
_("Unable to fetch cluster list"))
return clusters
class ClusterDetailsView(tabs.TabView):
tab_group_class = _tabs.ClusterDetailsTabs
template_name = 'horizon/common/_detail.html'
page_title = "{{ cluster.name|default:cluster.id }}"
@memoized.memoized_method
def get_object(self):
cl_id = self.kwargs["cluster_id"]
try:
return saharaclient.cluster_get(self.request, cl_id)
except Exception:
msg = _('Unable to retrieve details for cluster "%s".') % cl_id
redirect = reverse(
"horizon:project:data_processing.clusters:clusters")
exceptions.handle(self.request, msg, redirect=redirect)
def get_context_data(self, **kwargs):
context = super(ClusterDetailsView, self).get_context_data(**kwargs)
context['cluster'] = self.get_object()
return context
class ClusterEventsView(django_base.View):
_date_format = "%Y-%m-%dT%H:%M:%S"
@staticmethod
def _created_at_key(obj):
return datetime.strptime(obj["created_at"],
ClusterEventsView._date_format)
def get(self, request, *args, **kwargs):
cluster_id = kwargs.get("cluster_id")
try:
cluster = saharaclient.cluster_get(request, cluster_id,
show_progress=True)
node_group_mapping = {}
for node_group in cluster.node_groups:
node_group_mapping[node_group["id"]] = node_group["name"]
provision_steps = cluster.provision_progress
# Sort by create time
provision_steps = sorted(provision_steps,
key=ClusterEventsView._created_at_key,
reverse=True)
for step in provision_steps:
# Sort events of the steps also
step["events"] = sorted(step["events"],
key=ClusterEventsView._created_at_key,
reverse=True)
successful_events_count = 0
for event in step["events"]:
if event["node_group_id"]:
event["node_group_name"] = node_group_mapping[
event["node_group_id"]]
event_result = _("Unknown")
if event["successful"] is True:
successful_events_count += 1
event_result = _("Completed Successfully")
elif event["successful"] is False:
event_result = _("Failed")
event["result"] = event_result
if not event["event_info"]:
event["event_info"] = _("No info available")
start_time = datetime.strptime(step["created_at"],
self._date_format)
end_time = datetime.now()
# Clear out microseconds. There is no need for that precision.
end_time = end_time.replace(microsecond=0)
if step["successful"] is not None:
updated_at = step["updated_at"]
end_time = datetime.strptime(updated_at,
self._date_format)
step["duration"] = six.text_type(end_time - start_time)
result = _("In progress")
step["completed"] = successful_events_count
if step["successful"] is True:
step["completed"] = step["total"]
result = _("Completed Successfully")
elif step["successful"] is False:
result = _("Failed")
step["result"] = result
status = cluster.status.lower()
need_update = status not in ("active", "error")
except APIException:
# Cluster is not available. Returning empty event log.
need_update = False
provision_steps = []
context = {"provision_steps": provision_steps,
"need_update": need_update}
return HttpResponse(json.dumps(context),
content_type='application/json')
class CreateClusterView(workflows.WorkflowView):
workflow_class = create_flow.CreateCluster
success_url = \
"horizon:project:data_processing.clusters:create-cluster"
classes = ("ajax-modal",)
template_name = "project/data_processing.clusters/create.html"
page_title = _("Launch Cluster")
class ConfigureClusterView(workflows.WorkflowView):
workflow_class = create_flow.ConfigureCluster
success_url = "horizon:project:data_processing.clusters"
template_name = "project/data_processing.clusters/configure.html"
page_title = _("Configure Cluster")
def get_initial(self):
initial = super(ConfigureClusterView, self).get_initial()
initial.update(self.kwargs)
return initial
class ScaleClusterView(workflows.WorkflowView):
workflow_class = scale_flow.ScaleCluster
success_url = "horizon:project:data_processing.clusters"
classes = ("ajax-modal",)
template_name = "project/data_processing.clusters/scale.html"
page_title = _("Scale Cluster")
def get_context_data(self, **kwargs):
context = super(ScaleClusterView, self)\
.get_context_data(**kwargs)
context["cluster_id"] = kwargs["cluster_id"]
return context
def get_object(self, *args, **kwargs):
if not hasattr(self, "_object"):
template_id = self.kwargs['cluster_id']
try:
template = saharaclient.cluster_template_get(self.request,
template_id)
except Exception:
template = None
exceptions.handle(self.request,
_("Unable to fetch cluster template."))
self._object = template
return self._object
def get_initial(self):
initial = super(ScaleClusterView, self).get_initial()
initial.update({'cluster_id': self.kwargs['cluster_id']})
return initial

View File

@ -0,0 +1,258 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from horizon import exceptions
from horizon import forms
from horizon import workflows
from openstack_dashboard.api import nova
from sahara_dashboard.content.data_processing.utils import neutron_support
import sahara_dashboard.content.data_processing.utils. \
workflow_helpers as whelpers
from django.utils.translation import ugettext_lazy as _
from sahara_dashboard.api import sahara as saharaclient
import sahara_dashboard.content.data_processing. \
cluster_templates.workflows.create as t_flows
from saharaclient.api import base as api_base
import logging
LOG = logging.getLogger(__name__)
KEYPAIR_IMPORT_URL = "horizon:project:access_and_security:keypairs:import"
BASE_IMAGE_URL = "horizon:project:data_processing.data_image_registry:register"
TEMPLATE_UPLOAD_URL = (
"horizon:project:data_processing.cluster_templates:upload_file")
class SelectPluginAction(t_flows.SelectPluginAction):
class Meta(object):
name = _("Select plugin and hadoop version for cluster")
help_text_template = (
"project/data_processing.clusters/_create_general_help.html")
class SelectPlugin(t_flows.SelectPlugin):
action_class = SelectPluginAction
class CreateCluster(t_flows.CreateClusterTemplate):
slug = "create_cluster"
name = _("Launch Cluster")
success_url = "horizon:project:data_processing.cluster_templates:index"
default_steps = (SelectPlugin,)
class GeneralConfigAction(workflows.Action):
populate_neutron_management_network_choices = \
neutron_support.populate_neutron_management_network_choices
hidden_configure_field = forms.CharField(
required=False,
widget=forms.HiddenInput(attrs={"class": "hidden_configure_field"}))
hidden_to_delete_field = forms.CharField(
required=False,
widget=forms.HiddenInput(attrs={"class": "hidden_to_delete_field"}))
cluster_name = forms.CharField(label=_("Cluster Name"))
description = forms.CharField(label=_("Description"),
required=False,
widget=forms.Textarea(attrs={'rows': 4}))
cluster_template = forms.DynamicChoiceField(label=_("Cluster Template"),
initial=(None, "None"),
add_item_link=
TEMPLATE_UPLOAD_URL)
cluster_count = forms.IntegerField(min_value=1,
label=_("Cluster Count"),
initial=1,
help_text=(
_("Number of clusters to launch.")))
image = forms.DynamicChoiceField(label=_("Base Image"),
add_item_link=BASE_IMAGE_URL)
keypair = forms.DynamicChoiceField(
label=_("Keypair"),
required=False,
help_text=_("Which keypair to use for authentication."),
add_item_link=KEYPAIR_IMPORT_URL)
def __init__(self, request, *args, **kwargs):
super(GeneralConfigAction, self).__init__(request, *args, **kwargs)
plugin, hadoop_version = whelpers.\
get_plugin_and_hadoop_version(request)
if saharaclient.base.is_service_enabled(request, 'network'):
self.fields["neutron_management_network"] = forms.ChoiceField(
label=_("Neutron Management Network"),
choices=self.populate_neutron_management_network_choices(
request, {})
)
self.fields["plugin_name"] = forms.CharField(
widget=forms.HiddenInput(),
initial=plugin
)
self.fields["hadoop_version"] = forms.CharField(
widget=forms.HiddenInput(),
initial=hadoop_version
)
def populate_image_choices(self, request, context):
try:
all_images = saharaclient.image_list(request)
plugin, hadoop_version = whelpers.\
get_plugin_and_hadoop_version(request)
details = saharaclient.plugin_get_version_details(request,
plugin,
hadoop_version)
choices = [(image.id, image.name) for image in all_images
if (set(details.required_image_tags).
issubset(set(image.tags)))]
except Exception:
exceptions.handle(request,
_("Unable to fetch image choices."))
choices = []
if not choices:
choices.append(("", _("No Images Available")))
return choices
def populate_keypair_choices(self, request, context):
try:
keypairs = nova.keypair_list(request)
except Exception:
keypairs = []
exceptions.handle(request,
_("Unable to fetch keypair choices."))
keypair_list = [(kp.name, kp.name) for kp in keypairs]
keypair_list.insert(0, ("", _("No keypair")))
return keypair_list
def populate_cluster_template_choices(self, request, context):
templates = saharaclient.cluster_template_list(request)
plugin, hadoop_version = whelpers.\
get_plugin_and_hadoop_version(request)
choices = [(template.id, template.name)
for template in templates
if (template.hadoop_version == hadoop_version and
template.plugin_name == plugin)]
if not choices:
choices.append(("", _("No Templates Available")))
# cluster_template_id comes from cluster templates table, when
# Create Cluster from template is clicked there
selected_template_name = None
if request.REQUEST.get("cluster_template_name"):
selected_template_name = (
request.REQUEST.get("cluster_template_name"))
if selected_template_name:
for template in templates:
if template.name == selected_template_name:
selected_template_id = template.id
break
else:
selected_template_id = (
request.REQUEST.get("cluster_template_id", None))
for template in templates:
if template.id == selected_template_id:
self.fields['cluster_template'].initial = template.id
return choices
def get_help_text(self):
extra = dict()
plugin, hadoop_version = whelpers.\
get_plugin_and_hadoop_version(self.request)
extra["plugin_name"] = plugin
extra["hadoop_version"] = hadoop_version
return super(GeneralConfigAction, self).get_help_text(extra)
def clean(self):
cleaned_data = super(GeneralConfigAction, self).clean()
if cleaned_data.get("hidden_configure_field", None) \
== "create_nodegroup":
self._errors = dict()
return cleaned_data
class Meta(object):
name = _("Configure Cluster")
help_text_template = \
("project/data_processing.clusters/_configure_general_help.html")
class GeneralConfig(workflows.Step):
action_class = GeneralConfigAction
contributes = ("hidden_configure_field", )
def contribute(self, data, context):
for k, v in data.items():
context["general_" + k] = v
return context
class ConfigureCluster(whelpers.StatusFormatMixin, workflows.Workflow):
slug = "configure_cluster"
name = _("Launch Cluster")
finalize_button_name = _("Launch")
success_message = _("Launched Cluster %s")
name_property = "general_cluster_name"
success_url = "horizon:project:data_processing.clusters:index"
default_steps = (GeneralConfig, )
def handle(self, request, context):
try:
# TODO(nkonovalov) Implement AJAX Node Groups.
node_groups = None
plugin, hadoop_version = whelpers.\
get_plugin_and_hadoop_version(request)
cluster_template_id = context["general_cluster_template"] or None
user_keypair = context["general_keypair"] or None
saharaclient.cluster_create(
request,
context["general_cluster_name"],
plugin, hadoop_version,
cluster_template_id=cluster_template_id,
default_image_id=context["general_image"],
description=context["general_description"],
node_groups=node_groups,
user_keypair_id=user_keypair,
count=context['general_cluster_count'],
net_id=context.get("general_neutron_management_network", None))
return True
except api_base.APIException as e:
self.error_description = str(e)
return False
except Exception:
exceptions.handle(request,
_('Unable to create the cluster'))
return False

View File

@ -0,0 +1,172 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import base64
import json
import logging
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from sahara_dashboard.api import sahara as saharaclient
import sahara_dashboard.content.data_processing. \
cluster_templates.workflows.create as clt_create_flow
import sahara_dashboard.content.data_processing. \
clusters.workflows.create as cl_create_flow
from sahara_dashboard.content.data_processing.utils import workflow_helpers
from saharaclient.api import base as api_base
LOG = logging.getLogger(__name__)
class NodeGroupsStep(clt_create_flow.ConfigureNodegroups):
pass
class ScaleCluster(cl_create_flow.ConfigureCluster,
workflow_helpers.StatusFormatMixin):
slug = "scale_cluster"
name = _("Scale Cluster")
finalize_button_name = _("Scale")
success_url = "horizon:project:data_processing.clusters:index"
default_steps = (NodeGroupsStep, )
def __init__(self, request, context_seed, entry_point, *args, **kwargs):
ScaleCluster._cls_registry = set([])
self.success_message = _("Scaled cluster successfully started.")
cluster_id = context_seed["cluster_id"]
try:
cluster = saharaclient.cluster_get(request, cluster_id)
plugin = cluster.plugin_name
hadoop_version = cluster.hadoop_version
# Initialize deletable node groups.
deletable = dict()
for group in cluster.node_groups:
deletable[group["name"]] = "false"
request.GET = request.GET.copy()
request.GET.update({
"cluster_id": cluster_id,
"plugin_name": plugin,
"hadoop_version": hadoop_version,
"deletable": deletable
})
super(ScaleCluster, self).__init__(request, context_seed,
entry_point, *args,
**kwargs)
# Initialize node groups.
for step in self.steps:
if not isinstance(step, clt_create_flow.ConfigureNodegroups):
continue
ng_action = step.action
template_ngs = cluster.node_groups
if 'forms_ids' in request.POST:
continue
ng_action.groups = []
for i, templ_ng in enumerate(template_ngs):
group_name = "group_name_%d" % i
template_id = "template_id_%d" % i
count = "count_%d" % i
serialized = "serialized_%d" % i
serialized_val = base64.urlsafe_b64encode(json.dumps(
workflow_helpers.clean_node_group(templ_ng)))
ng_action.groups.append({
"name": templ_ng["name"],
"template_id": templ_ng["node_group_template_id"],
"count": templ_ng["count"],
"id": i,
"deletable": "false",
"serialized": serialized_val
})
workflow_helpers.build_node_group_fields(ng_action,
group_name,
template_id,
count,
serialized)
except Exception:
exceptions.handle(request,
_("Unable to fetch cluster to scale"))
def format_status_message(self, message):
# Scaling form requires special handling because it has no Cluster name
# in it's context
error_description = getattr(self, 'error_description', None)
if error_description:
return error_description
else:
return self.success_message
def handle(self, request, context):
cluster_id = request.GET["cluster_id"]
try:
cluster = saharaclient.cluster_get(request, cluster_id)
existing_node_groups = set([])
for ng in cluster.node_groups:
existing_node_groups.add(ng["name"])
scale_object = dict()
ids = json.loads(context["ng_forms_ids"])
for _id in ids:
name = context["ng_group_name_%s" % _id]
template_id = context["ng_template_id_%s" % _id]
count = context["ng_count_%s" % _id]
if name not in existing_node_groups:
if "add_node_groups" not in scale_object:
scale_object["add_node_groups"] = []
scale_object["add_node_groups"].append(
{"name": name,
"node_group_template_id": template_id,
"count": int(count)})
else:
old_count = None
for ng in cluster.node_groups:
if name == ng["name"]:
old_count = ng["count"]
break
if old_count != count:
if "resize_node_groups" not in scale_object:
scale_object["resize_node_groups"] = []
scale_object["resize_node_groups"].append(
{"name": name,
"count": int(count)}
)
except Exception:
scale_object = {}
exceptions.handle(request,
_("Unable to fetch cluster to scale."))
try:
saharaclient.cluster_scale(request, cluster_id, scale_object)
return True
except api_base.APIException as e:
self.error_description = str(e)
return False
except Exception:
exceptions.handle(request,
_("Scale cluster operation failed"))
return False

View File

@ -0,0 +1,116 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from horizon import forms
from horizon import messages
from openstack_dashboard.api import glance
from sahara_dashboard.api import sahara as saharaclient
class ImageForm(forms.SelfHandlingForm):
image_id = forms.CharField(widget=forms.HiddenInput())
tags_list = forms.CharField(widget=forms.HiddenInput())
user_name = forms.CharField(max_length=80, label=_("User Name"))
description = forms.CharField(max_length=80,
label=_("Description"),
required=False,
widget=forms.Textarea(attrs={'rows': 4}))
def handle(self, request, data):
try:
image_id = data['image_id']
user_name = data['user_name']
desc = data['description']
saharaclient.image_update(request, image_id, user_name, desc)
image_tags = json.loads(data["tags_list"])
saharaclient.image_tags_update(request, image_id, image_tags)
updated_image = saharaclient.image_get(request, image_id)
messages.success(request,
_("Successfully updated image."))
return updated_image
except Exception:
exceptions.handle(request,
_("Failed to update image."))
return False
class EditTagsForm(ImageForm):
image_id = forms.CharField(widget=forms.HiddenInput())
class RegisterImageForm(ImageForm):
image_id = forms.ChoiceField(label=_("Image"))
def __init__(self, request, *args, **kwargs):
super(RegisterImageForm, self).__init__(request, *args, **kwargs)
self._populate_image_id_choices()
def _populate_image_id_choices(self):
images = self._get_available_images(self.request)
choices = [(image.id, image.name)
for image in images
if image.properties.get("image_type", '') != "snapshot"]
if choices:
choices.insert(0, ("", _("Select Image")))
else:
choices.insert(0, ("", _("No images available.")))
self.fields['image_id'].choices = choices
def _get_images(self, request, filter):
try:
images, _more, _prev = (
glance.image_list_detailed(request, filters=filter))
except Exception:
images = []
exceptions.handle(request,
_("Unable to retrieve images with filter %s.") %
filter)
return images
def _get_public_images(self, request):
filter = {"is_public": True,
"status": "active"}
return self._get_images(request, filter)
def _get_tenant_images(self, request):
filter = {"owner": request.user.tenant_id,
"status": "active"}
return self._get_images(request, filter)
def _get_available_images(self, request):
images = self._get_tenant_images(request)
if request.user.is_superuser:
images += self._get_public_images(request)
final_images = []
try:
image_ids = set(img.id for img in saharaclient.image_list(request))
except Exception:
image_ids = set()
exceptions.handle(request,
_("Unable to fetch available images."))
for image in images:
if (image not in final_images and
image.id not in image_ids and
image.container_format not in ('aki', 'ari')):
final_images.append(image)
return final_images

View File

@ -0,0 +1,28 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from django.utils.translation import ugettext_lazy as _
import horizon
from openstack_dashboard.dashboards.project import dashboard
class ImageRegistryPanel(horizon.Panel):
name = _("Image Registry")
slug = 'data_processing.data_image_registry'
permissions = (('openstack.services.data-processing',
'openstack.services.data_processing'),)
dashboard.Project.register(ImageRegistryPanel)

View File

@ -0,0 +1,83 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django import template
from django.utils.translation import ugettext_lazy as _
from django.utils.translation import ungettext_lazy
from horizon import tables
from sahara_dashboard.api import sahara as saharaclient
LOG = logging.getLogger(__name__)
class EditTagsAction(tables.LinkAction):
name = "edit_tags"
verbose_name = _("Edit Tags")
url = "horizon:project:data_processing.data_image_registry:edit_tags"
classes = ("ajax-modal",)
def tags_to_string(image):
template_name = (
'project/data_processing.data_image_registry/_list_tags.html')
context = {"image": image}
return template.loader.render_to_string(template_name, context)
class RegisterImage(tables.LinkAction):
name = "register"
verbose_name = _("Register Image")
url = "horizon:project:data_processing.data_image_registry:register"
classes = ("ajax-modal",)
icon = "plus"
class UnregisterImages(tables.DeleteAction):
@staticmethod
def action_present(count):
return ungettext_lazy(
u"Unregister Image",
u"Unregister Images",
count
)
@staticmethod
def action_past(count):
return ungettext_lazy(
u"Unregistered Image",
u"Unregistered Images",
count
)
def delete(self, request, obj_id):
saharaclient.image_unregister(request, obj_id)
class ImageRegistryTable(tables.DataTable):
name = tables.Column("name",
verbose_name=_("Image"),
link=("horizon:project:"
"images:images:detail"))
tags = tables.Column(tags_to_string,
verbose_name=_("Tags"))
class Meta(object):
name = "image_registry"
verbose_name = _("Image Registry")
table_actions = (RegisterImage, UnregisterImages,)
row_actions = (EditTagsAction, UnregisterImages,)

View File

@ -0,0 +1,28 @@
{% extends "horizon/common/_modal_form.html" %}
{% load i18n %}
{% block form_id %}edit_tags_form{% endblock %}
{% block form_action %}{% url 'horizon:project:data_processing.data_image_registry:edit_tags' image.id %}{% endblock %}
{% block modal-header %}{% trans "Edit Image Tags" %}{% endblock %}
{% block modal-body %}
<div class="left">
<fieldset>
{% include "horizon/common/_form_fields.html" %}
</fieldset>
{% include 'project/data_processing.data_image_registry/_tag_form.html' %}
</div>
<div class="right">
{% include 'project/data_processing.data_image_registry/_help.html' %}
</div>
{% endblock %}
{% block modal-footer %}
<input class="btn btn-primary pull-right" id="edit_image_tags_btn" type="submit" value="{% trans "Done" %}"/>
<a href="{% url 'horizon:project:data_processing.data_image_registry:index' %}" class="btn btn-default secondary cancel close">{% trans "Cancel" %}</a>
{% endblock %}

View File

@ -0,0 +1,21 @@
{% load i18n %}
<div class="well well-small">
<h3>{% blocktrans %}Image Registry tool:{% endblocktrans %}</h3>
<br />
<p>
{% blocktrans %}Image Registry is used to provide additional information about images for Data Processing.{% endblocktrans %}
</p>
<p>
{% blocktrans %}Specified User Name will be used by Data Processing to apply configs and manage processes on instances.{% endblocktrans %}
</p>
<p>
{% blocktrans %}Tags are used for filtering images suitable for each plugin and each Data Processing version.
To add required tags, select a plugin and a Data Processing version and click &quot;Add plugin tags&quot; button.{% endblocktrans %}
</p>
<p>
{% blocktrans %}You may also add any custom tag.{% endblocktrans %}
</p>
<p>
{% blocktrans %}Unnecessary tags may be removed by clicking a cross near tag's name.{% endblocktrans %}
</p>
</div>

View File

@ -0,0 +1,5 @@
<ul>
{% for tag in image.tags %}
<li><span class="label label-info" style="float: left;display: block; margin: 2px;">{{ tag }}</span></li>
{% endfor %}
</ul>

View File

@ -0,0 +1,26 @@
{% extends "horizon/common/_modal_form.html" %}
{% load i18n %}
{% block form_id %}register_image_form{% endblock %}
{% block form_action %}{% url 'horizon:project:data_processing.data_image_registry:register' %}{% endblock %}
{% block modal-header %}{% trans "Register Image" %}{% endblock %}
{% block modal-body %}
<div class="left">
<fieldset>
{% include "horizon/common/_form_fields.html" %}
{% include 'project/data_processing.data_image_registry/_tag_form.html' %}
</fieldset>
</div>
<div class="right">
{% include 'project/data_processing.data_image_registry/_help.html' %}
</div>
{% endblock %}
{% block modal-footer %}
<input class="btn btn-primary pull-right" id="edit_image_tags_btn" type="submit" value="{% trans "Done" %}"/>
<a href="{% url 'horizon:project:data_processing.data_image_registry:index' %}" class="btn btn-default secondary cancel close">{% trans "Cancel" %}</a>
{% endblock %}

View File

@ -0,0 +1,123 @@
{% load i18n %}
<div id="image_tags_list" class="well"></div>
<div style="clear: both;"></div>
<div id="plugin_populate_section">
<h5>{% trans "Register tags required for the Plugin with specified Data Processing Version" %}</h5>
<span class="row">
<span class="col-sm-4 small-padding"><b>{% trans "Plugin" %}</b></span>
<span class="col-sm-4 small-padding"><b>{% trans "Version" %}</b></span>
</span>
<span class="row">
<span class="col-sm-4 small-padding">
<select id="plugin_select" class="plugin-choice form-control">
{% for plugin, version_dict in plugins.items %}
<option value="{{ plugin }}">{{ plugin }}</option>
{% endfor %}
</select>
</span>
<span class="col-sm-4 small-padding">
{% for plugin, version_dict in plugins.items %}
<div id="version_group_{{ plugin }}" class="data_processing-version-choice" >
<select id="data_processing_version_{{ plugin }}" class="form-control">
{% for version, tags in version_dict.items %}
<option value="{{ version }}">{{ version }}</option>
{% endfor %}
</select>
</div>
{% endfor %}
</span>
<span class="col-sm-4 small-padding">
<input type="button" id="add_all_btn" class="btn btn-default" value="{% trans "Add plugin tags" %}" />
</span>
</span>
<span class="row">
<span class="col-sm-8 small-padding">
<input type="text" class="tag-input form-control" id="_sahara_image_tag"/>
</span>
<span class="col-sm-3 small-padding">
<button type="button" id="add_tag_btn" class="btn btn-default btn-small btn-create btn-inline" onclick="add_tag_to_image()">{% trans "Add custom tag" %}</button>
</span>
</span>
</div>
<script type="text/javascript">
$(function() {
$(".plugin-choice").change(function(e) {
$(".data_processing-version-choice").hide();
var val = $(this).val();
$("#version_group_" + val).show();
}).change();
$("#add_all_btn").click(function(e) {
var plugin = $("#plugin_select").val();
var version = $("#data_processing_version_" + plugin).val();
var tags = plugin_tags_map[plugin][version];
$(tags).each(function(idx, tag) {
add_tag_to_image(tag);
});
})
});
$("#_sahara_image_tag").keypress(function (event) {
if (event.keyCode == 13) {
add_tag_to_image();
return false;
}
return true;
});
function add_tag_to_image(tag) {
if (!tag) {
tag = $.trim($("#_sahara_image_tag").val());
}
if (tag.length == 0) {
return;
}
$("#image_tags_list span").each(function (el) {
if ($.trim($(this).text()) == tag) {
return;
}
});
var tags = get_current_tags();
if ($.inArray(tag, tags) == -1) {
var span = ' <span class="label label-warning" style="float: left;display: block; margin: 2px;">$tag <i class="fa fa-close" onclick="remove_tag(this);"></i></span>'.replace("$tag", tag)
$("#image_tags_list").append(span);
update_image_tags();
}
$("#_sahara_image_tag").val("");
}
function get_current_tags() {
var tags = [];
$("#image_tags_list span").each(function (el) {
tags.push($.trim($(this).text()));
});
return tags;
}
function update_image_tags() {
var tags = get_current_tags();
$("#id_tags_list").val(JSON.stringify(tags));
}
function remove_tag(icon) {
span = icon.parentNode;
span.parentNode.removeChild(span);
update_image_tags()
}
// {"plugin": {"v1": [...tags...], "v2": [...tags...]},
// "other_plugin": ... }
var plugin_tags_map = {};
{% for plugin, version_dict in plugins.items %}
plugin_tags_map["{{ plugin }}"] = {};
{% for version, tags in version_dict.items %}
plugin_tags_map["{{ plugin }}"]["{{ version }}"] = [];
{% for tag in tags %}
plugin_tags_map["{{ plugin }}"]["{{ version }}"].push("{{ tag }}");
{% endfor %}
{% endfor %}
{% endfor %}
</script>

View File

@ -0,0 +1,7 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Edit Image Tags" %}{% endblock %}
{% block main %}
{% include 'project/data_processing.data_image_registry/_edit_tags.html' %}
{% endblock %}

View File

@ -0,0 +1,24 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Data Processing" %}{% endblock %}
{% block main %}
<div class="image_registry">
{{ image_registry_table.render }}
</div>
<script type="text/javascript">
addHorizonLoadEvent(function () {
horizon.modals.addModalInitFunction(function (modal) {
var tags = JSON.parse($("#id_tags_list").val());
$.each(tags, function(i, tag) {
var tagspan = '<span class="label label-info" style="float: left;display: block; margin: 2px;">' +
tag +
'<i class="fa fa-close" onclick="remove_tag(this);"></i></span>';
$("#image_tags_list").append(tagspan);
});
});
});
</script>
{% endblock %}

View File

@ -0,0 +1,7 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Register Image" %}{% endblock %}
{% block main %}
{% include 'project/data_processing.data_image_registry/_register_image.html' %}
{% endblock %}

View File

@ -0,0 +1,131 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from django.core.urlresolvers import reverse
from django import http
from mox3.mox import IsA # noqa
from openstack_dashboard import api as dash_api
from openstack_dashboard.test import helpers as test
from sahara_dashboard import api
INDEX_URL = reverse(
'horizon:project:data_processing.data_image_registry:index')
REGISTER_URL = reverse(
'horizon:project:data_processing.data_image_registry:register')
class DataProcessingImageRegistryTests(test.TestCase):
@test.create_stubs({api.sahara: ('image_list',)})
def test_index(self):
api.sahara.image_list(IsA(http.HttpRequest)) \
.AndReturn(self.images.list())
self.mox.ReplayAll()
res = self.client.get(INDEX_URL)
self.assertTemplateUsed(
res,
'project/data_processing.data_image_registry/image_registry.html')
self.assertContains(res, 'Image Registry')
self.assertContains(res, 'Image')
self.assertContains(res, 'Tags')
@test.create_stubs({api.sahara: ('image_get',
'image_update',
'image_tags_update',
'image_list'),
dash_api.glance: ('image_list_detailed',)})
def test_register(self):
image = self.images.first()
image_id = image.id
test_username = 'myusername'
test_description = 'mydescription'
api.sahara.image_get(IsA(http.HttpRequest),
image_id).MultipleTimes().AndReturn(image)
dash_api.glance.image_list_detailed(IsA(http.HttpRequest),
filters={'owner': self.user.id,
'status': 'active'}) \
.AndReturn((self.images.list(), False, False))
api.sahara.image_update(IsA(http.HttpRequest),
image_id,
test_username,
test_description) \
.AndReturn(True)
api.sahara.image_tags_update(IsA(http.HttpRequest),
image_id,
{}) \
.AndReturn(True)
api.sahara.image_list(IsA(http.HttpRequest)) \
.AndReturn([])
self.mox.ReplayAll()
res = self.client.post(
REGISTER_URL,
{'image_id': image_id,
'user_name': test_username,
'description': test_description,
'tags_list': '{}'})
self.assertNoFormErrors(res)
self.assertRedirectsNoFollow(res, INDEX_URL)
self.assertMessageCount(success=1)
@test.create_stubs({api.sahara: ('image_list',
'image_unregister')})
def test_unregister(self):
image = self.images.first()
api.sahara.image_list(IsA(http.HttpRequest)) \
.AndReturn(self.images.list())
api.sahara.image_unregister(IsA(http.HttpRequest), image.id)
self.mox.ReplayAll()
form_data = {'action': 'image_registry__delete__%s' % image.id}
res = self.client.post(INDEX_URL, form_data)
self.assertNoFormErrors(res)
self.assertRedirectsNoFollow(res, INDEX_URL)
self.assertMessageCount(success=1)
@test.create_stubs({api.sahara: ('image_get',
'image_update',
'image_tags_update')})
def test_edit_tags(self):
image = self.registered_images.first()
api.sahara.image_get(IsA(http.HttpRequest),
image.id).MultipleTimes().AndReturn(image)
api.sahara.image_update(IsA(http.HttpRequest),
image.id,
image.username,
image.description) \
.AndReturn(True)
api.sahara.image_tags_update(IsA(http.HttpRequest),
image.id,
{"0": "mytag"}) \
.AndReturn(True)
self.mox.ReplayAll()
edit_tags_url = reverse(
'horizon:project:data_processing.data_image_registry:edit_tags',
args=[image.id])
res = self.client.post(
edit_tags_url,
{'image_id': image.id,
'user_name': image.username,
'description': image.description,
'tags_list': '{"0": "mytag"}'})
self.assertNoFormErrors(res)
self.assertRedirectsNoFollow(res, INDEX_URL)
self.assertMessageCount(success=1)

View File

@ -0,0 +1,33 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from django.conf.urls import patterns
from django.conf.urls import url
import sahara_dashboard.content. \
data_processing.data_image_registry.views as views
urlpatterns = patterns('',
url(r'^$', views.ImageRegistryView.as_view(),
name='index'),
url(r'^$', views.ImageRegistryView.as_view(),
name='image_registry'),
url(r'^edit_tags/(?P<image_id>[^/]+)/$',
views.EditTagsView.as_view(),
name='edit_tags'),
url(r'^register/$',
views.RegisterImageView.as_view(),
name='register'),
)

View File

@ -0,0 +1,129 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
import logging
from django.core.urlresolvers import reverse_lazy
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from horizon import forms
from horizon import tables
from horizon.utils import memoized
from sahara_dashboard.api import sahara as saharaclient
from sahara_dashboard.content. \
data_processing.data_image_registry.forms import EditTagsForm
from sahara_dashboard.content. \
data_processing.data_image_registry.forms import RegisterImageForm
from sahara_dashboard.content. \
data_processing.data_image_registry.tables import ImageRegistryTable
LOG = logging.getLogger(__name__)
class ImageRegistryView(tables.DataTableView):
table_class = ImageRegistryTable
template_name = (
'project/data_processing.data_image_registry/image_registry.html')
page_title = _("Image Registry")
def get_data(self):
try:
images = saharaclient.image_list(self.request)
except Exception:
images = []
msg = _('Unable to retrieve image list')
exceptions.handle(self.request, msg)
return images
def update_context_with_plugin_tags(request, context):
try:
plugins = saharaclient.plugin_list(request)
except Exception:
plugins = []
msg = _("Unable to process plugin tags")
exceptions.handle(request, msg)
plugins_object = dict()
for plugin in plugins:
plugins_object[plugin.name] = dict()
for version in plugin.versions:
try:
details = saharaclient. \
plugin_get_version_details(request,
plugin.name,
version)
plugins_object[plugin.name][version] = (
details.required_image_tags)
except Exception:
msg = _("Unable to process plugin tags")
exceptions.handle(request, msg)
context["plugins"] = plugins_object
class EditTagsView(forms.ModalFormView):
form_class = EditTagsForm
template_name = (
'project/data_processing.data_image_registry/edit_tags.html')
success_url = reverse_lazy(
'horizon:project:data_processing.data_image_registry:index')
page_title = _("Edit Image Tags")
def get_context_data(self, **kwargs):
context = super(EditTagsView, self).get_context_data(**kwargs)
context['image'] = self.get_object()
update_context_with_plugin_tags(self.request, context)
return context
@memoized.memoized_method
def get_object(self):
try:
image = saharaclient.image_get(self.request,
self.kwargs["image_id"])
except Exception:
image = None
msg = _("Unable to fetch the image details")
exceptions.handle(self.request, msg)
return image
def get_initial(self):
image = self.get_object()
return {"image_id": image.id,
"tags_list": json.dumps(image.tags),
"user_name": image.username,
"description": image.description}
class RegisterImageView(forms.ModalFormView):
form_class = RegisterImageForm
template_name = (
'project/data_processing.data_image_registry/register_image.html')
success_url = reverse_lazy(
'horizon:project:data_processing.data_image_registry:index')
page_title = _("Register Image")
def get_context_data(self, **kwargs):
context = super(RegisterImageView, self).get_context_data(**kwargs)
update_context_with_plugin_tags(self.request, context)
return context
def get_initial(self):
# need this initialization to allow registration
# of images without tags
return {"tags_list": json.dumps([])}

View File

@ -0,0 +1,28 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from django.utils.translation import ugettext_lazy as _
import horizon
from openstack_dashboard.dashboards.project import dashboard
class PluginsPanel(horizon.Panel):
name = _("Plugins")
slug = 'data_processing.data_plugins'
permissions = (('openstack.services.data-processing',
'openstack.services.data_processing'),)
dashboard.Project.register(PluginsPanel)

View File

@ -0,0 +1,40 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.template import defaultfilters as filters
from django.utils.translation import ugettext_lazy as _
from horizon import tables
LOG = logging.getLogger(__name__)
class PluginsTable(tables.DataTable):
title = tables.Column("title",
verbose_name=_("Title"),
link=("horizon:project:data_processing."
"data_plugins:details"))
versions = tables.Column("versions",
verbose_name=_("Supported Versions"),
wrap_list=True,
filters=(filters.unordered_list,))
description = tables.Column("description",
verbose_name=_("Description"))
class Meta(object):
name = "plugins"
verbose_name = _("Plugins")

View File

@ -0,0 +1,46 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from horizon import tabs
from sahara_dashboard.api import sahara as saharaclient
LOG = logging.getLogger(__name__)
class DetailsTab(tabs.Tab):
name = _("Details")
slug = "plugin_details_tab"
template_name = ("project/data_processing.data_plugins/_details.html")
def get_context_data(self, request):
plugin_id = self.tab_group.kwargs['plugin_id']
plugin = None
try:
plugin = saharaclient.plugin_get(request, plugin_id)
except Exception as e:
LOG.error("Unable to get plugin with plugin_id %s (%s)" %
(plugin_id, str(e)))
exceptions.handle(self.tab_group.request,
_('Unable to retrieve plugin.'))
return {"plugin": plugin}
class PluginDetailsTabs(tabs.TabGroup):
slug = "cluster_details"
tabs = (DetailsTab,)
sticky = True

View File

@ -0,0 +1,20 @@
{% load i18n %}
<div class="detail">
<dl class="dl-horizontal">
<dt>{% trans "Name" %}</dt>
<dd>{{ plugin.name }}</dd>
<dt>{% trans "Title" %}</dt>
<dd>{{ plugin.title }}</dd>
<dt>{% trans "Description" %}</dt>
<dd>{{ plugin.description }}</dd>
<dt>{% trans "Supported Versions" %}</dt>
<dd>
<ul class="list-bullet">
{% for version in plugin.versions %}
<li>{{ version }}</li>
{% endfor %}
</ul>
</dd>
</dl>
</div>

View File

@ -0,0 +1,11 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Data Processing" %}{% endblock %}
{% block main %}
<div class="plugins">
{{ plugins_table.render }}
</div>
{% endblock %}

View File

@ -0,0 +1,49 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from django.core.urlresolvers import reverse
from django import http
from mox3.mox import IsA # noqa
from openstack_dashboard.test import helpers as test
import six
from sahara_dashboard import api
INDEX_URL = reverse(
'horizon:project:data_processing.data_plugins:index')
DETAILS_URL = reverse(
'horizon:project:data_processing.data_plugins:details', args=['id'])
class DataProcessingPluginsTests(test.TestCase):
@test.create_stubs({api.sahara: ('plugin_list',)})
def test_index(self):
api.sahara.plugin_list(IsA(http.HttpRequest)) \
.AndReturn(self.plugins.list())
self.mox.ReplayAll()
res = self.client.get(INDEX_URL)
self.assertTemplateUsed(
res, 'project/data_processing.data_plugins/plugins.html')
self.assertContains(res, 'vanilla')
self.assertContains(res, 'plugin')
@test.create_stubs({api.sahara: ('plugin_get',)})
def test_details(self):
api.sahara.plugin_get(IsA(http.HttpRequest), IsA(six.text_type)) \
.AndReturn(self.plugins.list()[0])
self.mox.ReplayAll()
res = self.client.get(DETAILS_URL)
self.assertTemplateUsed(res, 'horizon/common/_detail.html')
self.assertContains(res, 'vanilla')
self.assertContains(res, 'plugin')

View File

@ -0,0 +1,25 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from django.conf.urls import patterns
from django.conf.urls import url
from sahara_dashboard.content.\
data_processing.data_plugins import views
urlpatterns = patterns('',
url(r'^$', views.PluginsView.as_view(), name='index'),
url(r'^(?P<plugin_id>[^/]+)$',
views.PluginDetailsView.as_view(), name='details'),
)

View File

@ -0,0 +1,49 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from horizon import tables
from horizon import tabs
from sahara_dashboard.api import sahara as saharaclient
import sahara_dashboard.content.data_processing. \
data_plugins.tables as p_tables
import sahara_dashboard.content.data_processing. \
data_plugins.tabs as p_tabs
LOG = logging.getLogger(__name__)
class PluginsView(tables.DataTableView):
table_class = p_tables.PluginsTable
template_name = 'project/data_processing.data_plugins/plugins.html'
page_title = _("Data Processing Plugins")
def get_data(self):
try:
plugins = saharaclient.plugin_list(self.request)
except Exception:
plugins = []
msg = _('Unable to retrieve data processing plugins.')
exceptions.handle(self.request, msg)
return plugins
class PluginDetailsView(tabs.TabView):
tab_group_class = p_tabs.PluginDetailsTabs
template_name = 'horizon/common/_detail.html'
page_title = _("Data Processing Plugin Details")

View File

@ -0,0 +1,28 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from django.utils.translation import ugettext_lazy as _
import horizon
from openstack_dashboard.dashboards.project import dashboard
class DataSourcesPanel(horizon.Panel):
name = _("Data Sources")
slug = 'data_processing.data_sources'
permissions = (('openstack.services.data-processing',
'openstack.services.data_processing'),)
dashboard.Project.register(DataSourcesPanel)

View File

@ -0,0 +1,78 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.utils.translation import ugettext_lazy as _
from django.utils.translation import ungettext_lazy
from horizon import tables
from sahara_dashboard.api import sahara as saharaclient
LOG = logging.getLogger(__name__)
class CreateDataSource(tables.LinkAction):
name = "create data source"
verbose_name = _("Create Data Source")
url = "horizon:project:data_processing.data_sources:create-data-source"
classes = ("ajax-modal",)
icon = "plus"
class DeleteDataSource(tables.DeleteAction):
@staticmethod
def action_present(count):
return ungettext_lazy(
u"Delete Data Source",
u"Delete Data Sources",
count
)
@staticmethod
def action_past(count):
return ungettext_lazy(
u"Deleted Data Source",
u"Deleted Data Sources",
count
)
def delete(self, request, obj_id):
saharaclient.data_source_delete(request, obj_id)
class EditDataSource(tables.LinkAction):
name = "edit data source"
verbose_name = _("Edit Data Source")
url = "horizon:project:data_processing.data_sources:edit-data-source"
classes = ("ajax-modal",)
class DataSourcesTable(tables.DataTable):
name = tables.Column("name",
verbose_name=_("Name"),
link=("horizon:project:data_processing."
"data_sources:details"))
type = tables.Column("type",
verbose_name=_("Type"))
description = tables.Column("description",
verbose_name=_("Description"))
class Meta(object):
name = "data_sources"
verbose_name = _("Data Sources")
table_actions = (CreateDataSource,
DeleteDataSource)
row_actions = (DeleteDataSource,
EditDataSource,)

View File

@ -0,0 +1,44 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.utils.translation import ugettext_lazy as _
from horizon import tabs
from sahara_dashboard.api import sahara as saharaclient
LOG = logging.getLogger(__name__)
class GeneralTab(tabs.Tab):
name = _("General Info")
slug = "data_source_details_tab"
template_name = ("project/data_processing.data_sources/_details.html")
def get_context_data(self, request):
data_source_id = self.tab_group.kwargs['data_source_id']
try:
data_source = saharaclient.data_source_get(request, data_source_id)
except Exception as e:
data_source = {}
LOG.error("Unable to fetch data source details: %s" % str(e))
return {"data_source": data_source}
class DataSourceDetailsTabs(tabs.TabGroup):
slug = "data_source_details"
tabs = (GeneralTab,)
sticky = True

View File

@ -0,0 +1,15 @@
{% load i18n horizon %}
<div class="well">
<p>
{% blocktrans %}Create a Data Source with a specified name.{% endblocktrans %}
</p>
<p>
{% blocktrans %}Select the type of your Data Source.{% endblocktrans %}
</p>
<p>
{% blocktrans %}You may need to enter the username and password for your Data Source.{% endblocktrans %}
</p>
<p>
{% blocktrans %}You may also enter an optional description for your Data Source.{% endblocktrans %}
</p>
</div>

View File

@ -0,0 +1,18 @@
{% load i18n sizeformat %}
<div class="detail">
<dl class="dl-horizontal">
<dt>{% trans "Name" %}</dt>
<dd>{{ data_source.name }}</dd>
<dt>{% trans "ID" %}</dt>
<dd>{{ data_source.id }}</dd>
<dt>{% trans "Type" %}</dt>
<dd>{{ data_source.type }}</dd>
<dt>{% trans "URL" %}</dt>
<dd>{{ data_source.url }}</dd>
<dt>{% trans "Description" %}</dt>
<dd>{{ data_source.description|default:_("None") }}</dd>
<dt>{% trans "Create time" %}</dt>
<dd>{{ data_source.created_at }}</dd>
</dl>
</div>

View File

@ -0,0 +1,7 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Create Data Source" %}{% endblock %}
{% block main %}
{% include 'horizon/common/_workflow.html' %}
{% endblock %}

View File

@ -0,0 +1,11 @@
{% extends 'base.html' %}
{% load i18n %}
{% block title %}{% trans "Data Processing" %}{% endblock %}
{% block main %}
<div class="data_sources">
{{ data_sources_table.render }}
</div>
{% endblock %}

View File

@ -0,0 +1,124 @@
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from django.core.urlresolvers import reverse
from django import http
from mox3.mox import IsA # noqa
from openstack_dashboard.test import helpers as test
import six
from sahara_dashboard import api
INDEX_URL = reverse('horizon:project:data_processing.data_sources:index')
DETAILS_URL = reverse(
'horizon:project:data_processing.data_sources:details', args=['id'])
CREATE_URL = reverse(
'horizon:project:data_processing.data_sources:create-data-source')
EDIT_URL = reverse(
'horizon:project:data_processing.data_sources:edit-data-source',
args=['id'])
class DataProcessingDataSourceTests(test.TestCase):
@test.create_stubs({api.sahara: ('data_source_list',)})
def test_index(self):
api.sahara.data_source_list(IsA(http.HttpRequest)) \
.AndReturn(self.data_sources.list())
self.mox.ReplayAll()
res = self.client.get(INDEX_URL)
self.assertTemplateUsed(
res, 'project/data_processing.data_sources/data_sources.html')
self.assertContains(res, 'Data Sources')
self.assertContains(res, 'Name')
self.assertContains(res, 'sampleOutput')
self.assertContains(res, 'sampleOutput2')
@test.create_stubs({api.sahara: ('data_source_get',)})
def test_details(self):
api.sahara.data_source_get(IsA(http.HttpRequest), IsA(six.text_type)) \
.MultipleTimes().AndReturn(self.data_sources.first())
self.mox.ReplayAll()
res = self.client.get(DETAILS_URL)
self.assertTemplateUsed(res, 'horizon/common/_detail.html')
self.assertContains(res, 'sampleOutput')
@test.create_stubs({api.sahara: ('data_source_list',
'data_source_delete')})
def test_delete(self):
data_source = self.data_sources.first()
api.sahara.data_source_list(IsA(http.HttpRequest)) \
.AndReturn(self.data_sources.list())
api.sahara.data_source_delete(IsA(http.HttpRequest), data_source.id)
self.mox.ReplayAll()
form_data = {'action': 'data_sources__delete__%s' % data_source.id}
res = self.client.post(INDEX_URL, form_data)
self.assertNoFormErrors(res)
self.assertRedirectsNoFollow(res, INDEX_URL)
self.assertMessageCount(success=1)
@test.create_stubs({api.sahara: ('data_source_create',)})
def test_create(self):
data_source = self.data_sources.first()
api.sahara.data_source_create(IsA(http.HttpRequest),
data_source.name,
data_source.description,
data_source.type,
data_source.url,
"",
"") \
.AndReturn(self.data_sources.first())
self.mox.ReplayAll()
form_data = {
'data_source_url': data_source.url,
'data_source_name': data_source.name,
'data_source_description': data_source.description,
'data_source_type': data_source.type
}
res = self.client.post(CREATE_URL, form_data)
self.assertNoFormErrors(res)
self.assertRedirectsNoFollow(res, INDEX_URL)
self.assertMessageCount(success=1)
@test.create_stubs({api.sahara: ('data_source_update',
'data_source_get',)})
def test_edit(self):
data_source = self.data_sources.first()
api_data = {
'url': data_source.url,
'credentials': {'user': '', 'pass': ''},
'type': data_source.type,
'name': data_source.name,
'description': data_source.description
}
api.sahara.data_source_get(IsA(http.HttpRequest),
IsA(six.text_type)) \
.AndReturn(self.data_sources.first())
api.sahara.data_source_update(IsA(http.HttpRequest),
IsA(six.text_type),
api_data) \
.AndReturn(self.data_sources.first())
self.mox.ReplayAll()
form_data = {
'data_source_url': data_source.url,
'data_source_name': data_source.name,
'data_source_description': data_source.description,
'data_source_type': data_source.type
}
res = self.client.post(EDIT_URL, form_data)
self.assertNoFormErrors(res)
self.assertRedirectsNoFollow(res, INDEX_URL)
self.assertMessageCount(success=1)

View File

@ -0,0 +1,35 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from django.conf.urls import patterns
from django.conf.urls import url
import sahara_dashboard.content.data_processing. \
data_sources.views as views
urlpatterns = patterns('',
url(r'^$', views.DataSourcesView.as_view(),
name='index'),
url(r'^$', views.DataSourcesView.as_view(),
name='data-sources'),
url(r'^create-data-source$',
views.CreateDataSourceView.as_view(),
name='create-data-source'),
url(r'^(?P<data_source_id>[^/]+)/edit$',
views.EditDataSourceView.as_view(),
name='edit-data-source'),
url(r'^(?P<data_source_id>[^/]+)$',
views.DataSourceDetailsView.as_view(),
name='details'))

View File

@ -0,0 +1,99 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from horizon import tables
from horizon import tabs
from horizon.utils import memoized
from horizon.utils.urlresolvers import reverse # noqa
from horizon import workflows
from sahara_dashboard.api import sahara as saharaclient
import sahara_dashboard.content.data_processing. \
data_sources.tables as ds_tables
import sahara_dashboard.content.data_processing. \
data_sources.tabs as _tabs
import sahara_dashboard.content.data_processing. \
data_sources.workflows.create as create_flow
import sahara_dashboard.content.data_processing. \
data_sources.workflows.edit as edit_flow
LOG = logging.getLogger(__name__)
class DataSourcesView(tables.DataTableView):
table_class = ds_tables.DataSourcesTable
template_name = 'project/data_processing.data_sources/data_sources.html'
page_title = _("Data Sources")
def get_data(self):
try:
data_sources = saharaclient.data_source_list(self.request)
except Exception:
data_sources = []
exceptions.handle(self.request,
_("Unable to fetch data sources."))
return data_sources
class CreateDataSourceView(workflows.WorkflowView):
workflow_class = create_flow.CreateDataSource
success_url = \
"horizon:project:data_processing.data-sources:create-data-source"
classes = ("ajax-modal",)
template_name = "project/data_processing.data_sources/create.html"
page_title = _("Create Data Source")
class EditDataSourceView(CreateDataSourceView):
workflow_class = edit_flow.EditDataSource
page_title = _("Edit Data Source")
def get_context_data(self, **kwargs):
context = super(EditDataSourceView, self) \
.get_context_data(**kwargs)
context["data_source_id"] = kwargs["data_source_id"]
return context
def get_initial(self):
initial = super(EditDataSourceView, self).get_initial()
initial['data_source_id'] = self.kwargs['data_source_id']
return initial
class DataSourceDetailsView(tabs.TabView):
tab_group_class = _tabs.DataSourceDetailsTabs
template_name = 'horizon/common/_detail.html'
page_title = "{{ data_source.name|default:data_source.id }}"
@memoized.memoized_method
def get_object(self):
ds_id = self.kwargs["data_source_id"]
try:
return saharaclient.data_source_get(self.request, ds_id)
except Exception:
msg = _('Unable to retrieve details for data source "%s".') % ds_id
redirect = reverse(
"horizon:project:data_processing.data_sources:data-sources")
exceptions.handle(self.request, msg, redirect=redirect)
def get_context_data(self, **kwargs):
context = super(DataSourceDetailsView, self).get_context_data(**kwargs)
context['data_source'] = self.get_object()
return context

View File

@ -0,0 +1,121 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from horizon import forms
from horizon import workflows
from sahara_dashboard.api import sahara as saharaclient
from sahara_dashboard.content.data_processing \
.utils import helpers
LOG = logging.getLogger(__name__)
class GeneralConfigAction(workflows.Action):
data_source_name = forms.CharField(label=_("Name"))
data_source_type = forms.ChoiceField(
label=_("Data Source Type"),
choices=[("swift", "Swift"), ("hdfs", "HDFS"), ("maprfs", "MapR FS")],
widget=forms.Select(attrs={
"class": "switchable",
"data-slug": "ds_type"
}))
data_source_url = forms.CharField(label=_("URL"))
data_source_credential_user = forms.CharField(
label=_("Source username"),
required=False,
widget=forms.TextInput(attrs={
"class": "switched",
"data-switch-on": "ds_type",
"data-ds_type-swift": _("Source username")
}))
data_source_credential_pass = forms.CharField(
widget=forms.PasswordInput(attrs={
'class': 'switched',
'data-switch-on': 'ds_type',
'data-ds_type-swift': _("Source password"),
'autocomplete': 'off'
}),
label=_("Source password"),
required=False)
data_source_description = forms.CharField(
label=_("Description"),
required=False,
widget=forms.Textarea(attrs={'rows': 4}))
def __init__(self, request, *args, **kwargs):
super(GeneralConfigAction, self).__init__(request, *args, **kwargs)
class Meta(object):
name = _("Create Data Source")
help_text_template = ("project/data_processing.data_sources/"
"_create_data_source_help.html")
class GeneralConfig(workflows.Step):
action_class = GeneralConfigAction
def contribute(self, data, context):
for k, v in data.items():
context["general_" + k] = v
context["source_url"] = context["general_data_source_url"]
if context["general_data_source_type"] == "swift":
if not context["general_data_source_url"].startswith("swift://"):
context["source_url"] = "swift://{0}".format(
context["general_data_source_url"])
return context
class CreateDataSource(workflows.Workflow):
slug = "create_data_source"
name = _("Create Data Source")
finalize_button_name = _("Create")
success_message = _("Data source created")
failure_message = _("Could not create data source")
success_url = "horizon:project:data_processing.data_sources:index"
default_steps = (GeneralConfig, )
def handle(self, request, context):
try:
self.object = saharaclient.data_source_create(
request,
context["general_data_source_name"],
context["general_data_source_description"],
context["general_data_source_type"],
context["source_url"],
context.get("general_data_source_credential_user", None),
context.get("general_data_source_credential_pass", None))
hlps = helpers.Helpers(request)
if hlps.is_from_guide():
request.session["guide_datasource_id"] = self.object.id
request.session["guide_datasource_name"] = self.object.name
self.success_url = (
"horizon:project:data_processing.wizard:jobex_guide")
return True
except Exception:
exceptions.handle(request)
return False

View File

@ -0,0 +1,79 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from sahara_dashboard.api import sahara as saharaclient
from sahara_dashboard.content.data_processing \
.data_sources.workflows import create
LOG = logging.getLogger(__name__)
class EditDataSource(create.CreateDataSource):
slug = "edit_data_source"
name = _("Edit Data Source")
finalize_button_name = _("Update")
success_message = _("Data source updated")
failure_message = _("Could not update data source")
success_url = "horizon:project:data_processing.data_sources:index"
default_steps = (create.GeneralConfig,)
FIELD_MAP = {
"data_source_name": "name",
"data_source_type": "type",
"data_source_description": "description",
"data_source_url": "url",
"data_source_credential_user": None,
"data_source_credential_pass": None,
}
def __init__(self, request, context_seed, entry_point, *args, **kwargs):
self.data_source_id = context_seed["data_source_id"]
data_source = saharaclient.data_source_get(request,
self.data_source_id)
super(EditDataSource, self).__init__(request, context_seed,
entry_point, *args, **kwargs)
for step in self.steps:
if isinstance(step, create.GeneralConfig):
fields = step.action.fields
for field in fields:
if self.FIELD_MAP[field]:
fields[field].initial = getattr(data_source,
self.FIELD_MAP[field],
None)
def handle(self, request, context):
try:
update_data = {
"name": context["general_data_source_name"],
"description": context["general_data_source_description"],
"type": context["general_data_source_type"],
"url": context["source_url"],
"credentials": {
"user": context.get("general_data_source_credential_user",
None),
"pass": context.get("general_data_source_credential_pass",
None)
}
}
return saharaclient.data_source_update(request,
self.data_source_id,
update_data)
except Exception:
exceptions.handle(request)
return False

View File

@ -0,0 +1,311 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
import uuid
from django.forms import widgets
from django import template
from django.template import defaultfilters
from django.utils.encoding import force_text
from django.utils.safestring import mark_safe
from django.utils.translation import ugettext_lazy as _
from horizon import exceptions
from horizon import forms
from horizon import messages
from sahara_dashboard.api import sahara as saharaclient
LOG = logging.getLogger(__name__)
class LabeledInput(widgets.TextInput):
def render(self, name, values, attrs=None):
input = super(LabeledInput, self).render(name, values, attrs)
label = "<span id='%s'>%s</span>" %\
("id_%s_label" % name,
"swift://")
result = "%s%s" % (label, input)
return mark_safe(result)
class JobBinaryCreateForm(forms.SelfHandlingForm):
NEW_SCRIPT = "newscript"
UPLOAD_BIN = "uploadfile"
action_url = ('horizon:project:data_processing.'
'job_binaries:create-job-binary')
def __init__(self, request, *args, **kwargs):
super(JobBinaryCreateForm, self).__init__(request, *args, **kwargs)
self.help_text_template = ("project/data_processing.job_binaries/"
"_create_job_binary_help.html")
self.fields["job_binary_name"] = forms.CharField(label=_("Name"))
self.fields["job_binary_type"] = forms.ChoiceField(
label=_("Storage type"),
widget=forms.Select(
attrs={
'class': 'switchable',
'data-slug': 'jb_type'
}))
self.fields["job_binary_url"] = forms.CharField(
label=_("URL"),
required=False,
widget=LabeledInput(
attrs={
'class': 'switched',
'data-switch-on': 'jb_type',
'data-jb_type-swift': _('URL')
}))
self.fields["job_binary_internal"] = forms.ChoiceField(
label=_("Internal binary"),
required=False,
widget=forms.Select(
attrs={
'class': 'switched switchable',
'data-slug': 'jb_internal',
'data-switch-on': 'jb_type',
'data-jb_type-internal-db': _('Internal Binary')
}))
self.fields["job_binary_file"] = forms.FileField(
label=_("Upload File"),
required=False,
widget=forms.ClearableFileInput(
attrs={
'class': 'switched',
'data-switch-on': 'jb_internal',
'data-jb_internal-uploadfile': _("Upload File")
}))
self.fields["job_binary_script_name"] = forms.CharField(
label=_("Script name"),
required=False,
widget=forms.TextInput(
attrs={
'class': 'switched',
'data-switch-on': 'jb_internal',
'data-jb_internal-newscript': _("Script name")
}))
self.fields["job_binary_script"] = forms.CharField(
label=_("Script text"),
required=False,
widget=forms.Textarea(
attrs={
'rows': 4,
'class': 'switched',
'data-switch-on': 'jb_internal',
'data-jb_internal-newscript': _("Script text")
}))
self.fields["job_binary_username"] = forms.CharField(
label=_("Username"),
required=False,
widget=forms.TextInput(
attrs={
'class': 'switched',
'data-switch-on': 'jb_type',
'data-jb_type-swift': _('Username')
}))
self.fields["job_binary_password"] = forms.CharField(
label=_("Password"),
required=False,
widget=forms.PasswordInput(
attrs={
'autocomplete': 'off',
'class': 'switched',
'data-switch-on': 'jb_type',
'data-jb_type-swift': _('Password')
}))
self.fields["job_binary_description"] = (
forms.CharField(label=_("Description"),
required=False,
widget=forms.Textarea()))
self.fields["job_binary_type"].choices =\
[("internal-db", "Internal database"),
("swift", "Swift")]
self.fields["job_binary_internal"].choices =\
self.populate_job_binary_internal_choices(request)
self.load_form_values()
def load_form_values(self):
if "job_binary" in self.initial:
jb = self.initial["job_binary"]
for field in self.fields:
if self.FIELD_MAP[field]:
if field == "job_binary_url":
url = getattr(jb, self.FIELD_MAP[field], None)
(type, loc) = url.split("://")
self.fields['job_binary_type'].initial = type
self.fields[field].initial = loc
else:
self.fields[field].initial = (
getattr(jb, self.FIELD_MAP[field], None))
def populate_job_binary_internal_choices(self, request):
try:
job_binaries = saharaclient.job_binary_internal_list(request)
except Exception:
exceptions.handle(request,
_("Failed to get list of internal binaries."))
job_binaries = []
choices = [(job_binary.id, job_binary.name)
for job_binary in job_binaries]
choices.insert(0, (self.NEW_SCRIPT, '*Create a script'))
choices.insert(0, (self.UPLOAD_BIN, '*Upload a new file'))
return choices
def handle(self, request, context):
try:
extra = {}
bin_url = "%s://%s" % (context["job_binary_type"],
context["job_binary_url"])
if(context["job_binary_type"] == "internal-db"):
bin_url = self.handle_internal(request, context)
elif(context["job_binary_type"] == "swift"):
extra = self.handle_swift(request, context)
bin_object = saharaclient.job_binary_create(
request,
context["job_binary_name"],
bin_url,
context["job_binary_description"],
extra)
messages.success(request, "Successfully created job binary")
return bin_object
except Exception:
exceptions.handle(request,
_("Unable to create job binary"))
return False
def get_help_text(self, extra_context=None):
text = ""
extra_context = extra_context or {}
if self.help_text_template:
tmpl = template.loader.get_template(self.help_text_template)
context = template.RequestContext(self.request, extra_context)
text += tmpl.render(context)
else:
text += defaultfilters.linebreaks(force_text(self.help_text))
return defaultfilters.safe(text)
class Meta(object):
name = _("Create Job Binary")
help_text_template = ("project/data_processing.job_binaries/"
"_create_job_binary_help.html")
def handle_internal(self, request, context):
result = ""
bin_id = context["job_binary_internal"]
if(bin_id == self.UPLOAD_BIN):
try:
result = saharaclient.job_binary_internal_create(
request,
self.get_unique_binary_name(
request, request.FILES["job_binary_file"].name),
request.FILES["job_binary_file"].read())
bin_id = result.id
except Exception:
exceptions.handle(request,
_("Unable to upload job binary"))
return None
elif(bin_id == self.NEW_SCRIPT):
try:
result = saharaclient.job_binary_internal_create(
request,
self.get_unique_binary_name(
request, context["job_binary_script_name"]),
context["job_binary_script"])
bin_id = result.id
except Exception:
exceptions.handle(request,
_("Unable to create job binary"))
return None
return "internal-db://%s" % bin_id
def handle_swift(self, request, context):
username = context["job_binary_username"]
password = context["job_binary_password"]
extra = {
"user": username,
"password": password
}
return extra
def get_unique_binary_name(self, request, base_name):
try:
internals = saharaclient.job_binary_internal_list(request)
except Exception:
internals = []
exceptions.handle(request,
_("Failed to fetch internal binary list"))
names = [internal.name for internal in internals]
if base_name in names:
return "%s_%s" % (base_name, uuid.uuid1())
return base_name
class JobBinaryEditForm(JobBinaryCreateForm):
FIELD_MAP = {
'job_binary_description': 'description',
'job_binary_file': None,
'job_binary_internal': None,
'job_binary_name': 'name',
'job_binary_password': None,
'job_binary_script': None,
'job_binary_script_name': None,
'job_binary_type': None,
'job_binary_url': 'url',
'job_binary_username': None,
}
def handle(self, request, context):
try:
extra = {}
bin_url = "%s://%s" % (context["job_binary_type"],
context["job_binary_url"])
if (context["job_binary_type"] == "swift"):
extra = self.handle_swift(request, context)
update_data = {
"name": context["job_binary_name"],
"description": context["job_binary_description"],
"extra": extra,
"url": bin_url,
}
bin_object = saharaclient.job_binary_update(
request, self.initial["job_binary"].id, update_data)
messages.success(request, "Successfully updated job binary")
return bin_object
except Exception:
exceptions.handle(request,
_("Unable to update job binary"))
return False

View File

@ -0,0 +1,28 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from django.utils.translation import ugettext_lazy as _
import horizon
from openstack_dashboard.dashboards.project import dashboard
class JobBinariesPanel(horizon.Panel):
name = _("Job Binaries")
slug = 'data_processing.job_binaries'
permissions = (('openstack.services.data-processing',
'openstack.services.data_processing'),)
dashboard.Project.register(JobBinariesPanel)

View File

@ -0,0 +1,98 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.utils.translation import ugettext_lazy as _
from django.utils.translation import ungettext_lazy
from horizon import tables
from sahara_dashboard.api import sahara as saharaclient
from saharaclient.api import base as api_base
LOG = logging.getLogger(__name__)
class CreateJobBinary(tables.LinkAction):
name = "create job binary"
verbose_name = _("Create Job Binary")
url = "horizon:project:data_processing.job_binaries:create-job-binary"
classes = ("ajax-modal",)
icon = "plus"
class DeleteJobBinary(tables.DeleteAction):
@staticmethod
def action_present(count):
return ungettext_lazy(
u"Delete Job Binary",
u"Delete Job Binaries",
count
)
@staticmethod
def action_past(count):
return ungettext_lazy(
u"Deleted Job Binary",
u"Deleted Job Binaries",
count
)
def delete(self, request, obj_id):
jb = saharaclient.job_binary_get(request, obj_id)
(jb_type, jb_internal_id) = jb.url.split("://")
if jb_type == "internal-db":
try:
saharaclient.job_binary_internal_delete(request,
jb_internal_id)
except api_base.APIException:
# nothing to do for job-binary-internal if
# it does not exist.
pass
saharaclient.job_binary_delete(request, obj_id)
class DownloadJobBinary(tables.LinkAction):
name = "download job binary"
verbose_name = _("Download Job Binary")
url = "horizon:project:data_processing.job_binaries:download"
classes = ("btn-edit",)
class EditJobBinary(tables.LinkAction):
name = "edit job binary"
verbose_name = _("Edit Job Binary")
url = "horizon:project:data_processing.job_binaries:edit-job-binary"
classes = ("btn-edit", "ajax-modal",)
class JobBinariesTable(tables.DataTable):
name = tables.Column(
"name",
verbose_name=_("Name"),
link="horizon:project:data_processing.job_binaries:details")
type = tables.Column("url",
verbose_name=_("Url"))
description = tables.Column("description",
verbose_name=_("Description"))
class Meta(object):
name = "job_binaries"
verbose_name = _("Job Binaries")
table_actions = (CreateJobBinary,
DeleteJobBinary)
row_actions = (DeleteJobBinary, DownloadJobBinary, EditJobBinary)

View File

@ -0,0 +1,43 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
from django.utils.translation import ugettext_lazy as _
from horizon import tabs
from sahara_dashboard.api import sahara as saharaclient
LOG = logging.getLogger(__name__)
class JobBinaryDetailsTab(tabs.Tab):
name = _("General Info")
slug = "job_binaries_details_tab"
template_name = ("project/data_processing.job_binaries/_details.html")
def get_context_data(self, request):
job_binary_id = self.tab_group.kwargs['job_binary_id']
try:
job_binary = saharaclient.job_binary_get(request, job_binary_id)
except Exception as e:
job_binary = {}
LOG.error("Unable to fetch job binary details: %s" % str(e))
return {"job_binary": job_binary}
class JobBinaryDetailsTabs(tabs.TabGroup):
slug = "job_binary_details"
tabs = (JobBinaryDetailsTab,)
sticky = True

View File

@ -0,0 +1,26 @@
{% extends "horizon/common/_modal_form.html" %}
{% load i18n %}
{% block form_id %}create-job-binary{% endblock %}
{% block form_action %}{{ submit_url }}{% endblock %}
{% block form_attrs %}enctype="multipart/form-data"{% endblock %}
{% block modal-header %}{{ page_title }}{% endblock %}
{% block modal-body %}
<div class="left">
<fieldset>
{% include "horizon/common/_form_fields.html" %}
</fieldset>
</div>
<div class="form-help-block right">
{{ form.get_help_text }}
</div>
{% endblock %}
{% block modal-footer %}
<input class="btn btn-primary pull-right" id="upload_file_btn" type="submit" value="{{ submit_label }}"/>
<a href="{% url 'horizon:project:data_processing.job_binaries:index' %}" class="btn btn-default secondary cancel close">{% trans "Cancel" %}</a>
{% endblock %}

View File

@ -0,0 +1,32 @@
{% load i18n horizon %}
<div class="well">
<p>
{% blocktrans %}<b>Important</b>: The name that you give your job binary will be the name used in your job execution.
If your binary requires a particular name or extension (ie: ".jar"), be sure to include it here.{% endblocktrans %}
</p>
<p>
{% blocktrans %}Select the storage type for your job binary.{% endblocktrans %}
<ul class="list-bullet">
<li>{% blocktrans %}Data Processing internal database{% endblocktrans %}</li>
<li>{% blocktrans %}Swift{% endblocktrans %}</li>
</ul>
</p>
<p>
{% blocktrans %}For Data Processing internal job binaries, you may choose from the following:{% endblocktrans %}
<ul class="list-bullet">
<li>{% blocktrans %}Choose an existing file{% endblocktrans %}</li>
<li>{% blocktrans %}Upload a new file{% endblocktrans %}</li>
<li>{% blocktrans %}Create a script to be uploaded dynamically{% endblocktrans %}</ul>
</ul>
</p>
<p>
{% blocktrans %}For Object Store job binaries, you must:{% endblocktrans %}
<ul class="list-bullet">
<li>{% blocktrans %}Enter the URL for the file{% endblocktrans %}</li>
<li>{% blocktrans %}Enter the username and password required to access that file{% endblocktrans %}</li>
</ul>
</p>
<p>
{% blocktrans %}You may also enter an optional description for your job binary.{% endblocktrans %}
</p>
</div>

Some files were not shown because too many files have changed in this diff Show More