diff --git a/monasca_log_api_tempest/README.md b/monasca_log_api_tempest/README.md new file mode 100644 index 00000000..75dcf17d --- /dev/null +++ b/monasca_log_api_tempest/README.md @@ -0,0 +1,176 @@ +# Introduction + +**Monasca-Log-Api** requires following components set up in the environment: + +* monasca-log-transformer - component receiving data from monasca-log-api +* monasca-log-persister - component saves data to ElasticSearch +* ElasticSearch - database that stores logs + +Those three components are all part of [monasca-elkstack](https://github.com/FujitsuEnablingSoftwareTechnologyGmbH/ansible-monasca-elkstack) +Ansible role. + +**Monasca-Log-Api** can be installed with following [role](https://github.com/FujitsuEnablingSoftwareTechnologyGmbH/ansible-monasca-log-api). +In order to setup schema (kafka topics) please see [this](https://github.com/FujitsuEnablingSoftwareTechnologyGmbH/ansible-monasca-log-schema) role. + +## Installation next to monasca-api + +**Monasca-Api** and **Monasca-Log-Api** can be installed next to each other. +Each one has been designed to work with different aspects of monitoring. +Therefore it is possible to proceed with installation as described +[here](https://github.com/openstack/monasca-vagrant). + +# Configuration +1. Clone the OpenStack Tempest repo, and cd to it. + + ``` + git clone https://git.openstack.org/openstack/tempest.git + cd tempest + ``` + +2. Create a virtualenv for running the Tempest tests and activate it. +For example in the Tempest root dir + + ``` + virtualenv .venv + source .venv/bin/activate + ``` + +3. Install the Tempest requirements in the virtualenv. + + ``` + pip install -r requirements.txt -r test-requirements.txt + pip install nose + ``` + +4. Create ```etc/tempest.conf``` in the Tempest root dir by +running the following command: + + ``` + oslo-config-generator --config-file etc/config-generator.tempest.conf --output-file etc/tempest.conf + ``` + + Add the following sections to ```tempest.conf``` for testing + using the monasca-vagrant environment. + + ``` + [identity] + auth_version = v2 + admin_domain_name = Default + admin_tenant_name = admin + admin_password = admin + admin_username = admin + alt_tenant_name = demo + alt_password = admin + alt_username = alt_demo + tenant_name = mini-mon + password = password + username = mini-mon + uri_v3 = http://192.168.10.5:35357/v3/ + uri = http://192.168.10.5:35357/v2.0/ + force_tenant_isolation = False + allow_tenant_isolation = False + disable_ssl_certificate_validation = True + + [auth] + allow_tenant_isolation = true + ``` + + Edit the the variable values in the identity section to match your particular + monasca-vagrant environment. + +5. Create ```etc/logging.conf``` in the Tempest root dir by making a copying +```logging.conf.sample```. + +6. Clone the monasca-log-api repo in a directory somewhere outside of the +Tempest root dir. + +7. Install the monasca-log-api in your venv, which will also register + the Monasca Log Api Tempest Plugin as, monasca_log_api_tempest. + + cd into the monasa-log-api root directory. Making sure that the tempest + virtual env is still active, run the following command. + + ``` + python setup.py install + ``` + +See the [OpenStack Tempest Plugin +Interface](http://docs.openstack.org/developer/tempest/plugin.html), for more +details on Tempest Plugins and the plugin registration process. + +# Running the Monasca Log Api Tempest +The Monasca Tempest Tests can be run using a variety of methods including: +1. [Testr](https://wiki.openstack.org/wiki/Testr) +2. [Os-testr](http://docs.openstack.org/developer/os-testr/) +3. [PyCharm](https://www.jetbrains.com/pycharm/) +4. Tempest Scripts in Devstack + +## Run the tests from the CLI using testr + +[Testr](https://wiki.openstack.org/wiki/Testr) is a test runner that can be used to run the Tempest tests. + +1. In the Tempest root dir, create a list of the Monasca Tempest Tests in a file. + + ```sh + testr list-tests monasca_log_api_tempest > monasca_log_api_tempest + ``` + +2. Run the tests using testr + + ```sh + testr run --load-list=monasca_log_api_tempest + ``` + +You can also use testr to create a list of specific tests for your needs. + +## Run the tests from the CLI using os-testr (no file necessary) +[Os-testr](http://docs.openstack.org/developer/os-testr/) is a test wrapper +that can be used to run the Monasca Tempest tests. + +1. In the Tempest root dir: + + ``` + ostestr --serial --regex monasca_log_api_tempest + ``` + + ```--serial``` option is necessary here. Monasca Log Api tempest tests can't + be run in parallel (default option in ostestr) because some tests depend on the + same data and will randomly fail. + +## Running/Debugging the Monasca Tempest Tests in PyCharm + +Assuming that you have already created a PyCharm project for the +```monasca-log-api``` do the following: + +1. In PyCharm, Edit Configurations and add a new Python tests configuration by selecting Python tests->Nosetests. +2. Name the test. For example TestSingleLog. +3. Set the path to the script with the tests to run. For example, ~/repos/monasca-log-api/monasca_log_api_tempest/tests/test_single.py +4. Set the name of the Class to test. For example TestVersions. +5. Set the working directory to your local root Tempest repo. For example, ~/repos/tempest. +6. Select the Python interpreter for your project to be the same as the one virtualenv created above. For example, ~/repos/tempest/.venv +7. Run the tests. You should also be able to debug them. +8. Step and repeat for other tests. + +## Run the tests from the CLI using tempest scripts in devstack + +1. In /opt/stack/tempest, run ```./run_tempest.sh monasca_log_api_tempest``` +2. If asked to create a new virtual environment, select yes +3. Activate the virtual environment ```source .venv/bin/activate``` +4. In your monasca-log-api directory, run ```python setup.py install``` +5. In /opt/stack/tempest, run ```./run_tempest.sh monasca_log_api_tempest``` + +# References +This section provides a few additional references that might be useful: +* [Tempest - The OpenStack Integration Test Suite](http://docs.openstack.org/developer/tempest/overview.html#quickstart) +* [Tempest Configuration Guide](https://github.com/openstack/tempest/blob/master/doc/source/configuration.rst#id1) +* [OpenStack Tempest Plugin Interface](http://docs.openstack.org/developer/tempest/plugin.html) + +In addition to the above references, another source of information is the following OpenStack projects: +* [Manila Tempest Tests](https://github.com/openstack/manila/tree/master/manila_tempest_tests) +* [Congress Tempest Tests](https://github.com/openstack/congress/tree/master/congress_tempest_tests). +In particular, the Manila Tempest Tests were used as a reference implementation to develop the Monasca Tempest Tests. +There is also a wiki [HOWTO use tempest with manila](https://wiki.openstack.org/wiki/Manila/docs/HOWTO_use_tempest_with_manila) that might be useful for Monasca too. + +# Issues +* Update documentation for testing using Devstack when available. +* Consider changing from monasca_tempest_tests to monasca_api_tempest_tests. diff --git a/monasca_log_api_tempest/__init__.py b/monasca_log_api_tempest/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/monasca_log_api_tempest/clients.py b/monasca_log_api_tempest/clients.py new file mode 100644 index 00000000..72f9e6a1 --- /dev/null +++ b/monasca_log_api_tempest/clients.py @@ -0,0 +1,33 @@ +# Copyright 2015 FUJITSU LIMITED +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +from tempest import clients + +from monasca_log_api_tempest.services import log_api_client +from monasca_log_api_tempest.services import log_search_client + + +class Manager(clients.Manager): + def __init__(self, credentials=None, service=None): + super(Manager, self).__init__(credentials, service) + self.log_api_client = log_api_client.LogApiClient( + self.auth_provider, + 'logs', + None + ) + self.log_search_client = log_search_client.LogsSearchClient( + self.auth_provider, + 'logs-search', + None + ) diff --git a/monasca_log_api_tempest/config.py b/monasca_log_api_tempest/config.py new file mode 100644 index 00000000..04350915 --- /dev/null +++ b/monasca_log_api_tempest/config.py @@ -0,0 +1,36 @@ +# Copyright 2015 FUJITSU LIMITED +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +from oslo_config import cfg + +service_available_group = cfg.OptGroup(name='service_available', + title='Available OpenStack Services') +ServiceAvailableGroup = [ + cfg.BoolOpt('logs', + default=True, + help=('Whether or not Monasca-Log-Api ' + 'is expected to be available')), + cfg.BoolOpt('logs-search', + default=True, + help=('Whether or not Monasca-Log-Api search engine ' + '(ElasticSearch) is expected to be available')), +] + +monitoring_group = cfg.OptGroup(name='monitoring', + title='Monitoring Service Options') +MonitoringGroup = [ + cfg.StrOpt('api_version', + default='v2.0', + help='monasca-log-api API version') +] diff --git a/monasca_log_api_tempest/plugin.py b/monasca_log_api_tempest/plugin.py new file mode 100644 index 00000000..1e77e270 --- /dev/null +++ b/monasca_log_api_tempest/plugin.py @@ -0,0 +1,45 @@ +# Copyright 2015 FUJITSU LIMITED +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +import os + +from tempest import config +from tempest.test_discover import plugins + +from monasca_log_api_tempest import config as config_log_api + +_ROOT_PCKG_NAME = "monasca_log_api_tempest" + + +class MonascaLogApiTempestPlugin(plugins.TempestPlugin): + def load_tests(self): + base_path = os.path.split(os.path.dirname( + os.path.abspath(__file__)))[0] + test_dir = "%s/tests" % _ROOT_PCKG_NAME + full_test_dir = os.path.join(base_path, test_dir) + return full_test_dir, base_path + + def register_opts(self, conf): + config.register_opt_group( + conf, + config_log_api.service_available_group, + config_log_api.ServiceAvailableGroup + ) + config.register_opt_group(conf, + config_log_api.monitoring_group, + config_log_api.MonitoringGroup) + + def get_opt_lists(self): + return [(config_log_api.monitoring_group.name, + config_log_api.MonitoringGroup)] diff --git a/monasca_log_api_tempest/services/__init__.py b/monasca_log_api_tempest/services/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/monasca_log_api_tempest/services/log_api_client.py b/monasca_log_api_tempest/services/log_api_client.py new file mode 100644 index 00000000..ee465f57 --- /dev/null +++ b/monasca_log_api_tempest/services/log_api_client.py @@ -0,0 +1,46 @@ +# Copyright 2015 FUJITSU LIMITED +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +from oslo_serialization import jsonutils as json +from tempest import config +from tempest.common import service_client + +CONF = config.CONF + + +def _uri(url): + return CONF.monitoring.api_version + url + + +class LogApiClient(service_client.ServiceClient): + def get_version(self): + resp, response_body = self.send_request('GET', '/') + return resp, response_body + + def send_single_log(self, + log, + headers=None): + default_headers = { + 'X-Tenant-Id': 'b4265b0a48ae4fd3bdcee0ad8c2b6012', + 'X-Roles': 'admin', + 'X-Dimensions': 'dev:tempest' + } + default_headers.update(headers) + + uri = "/log/single" + msg = json.dumps(log) + + resp, body = self.post(_uri(uri), msg, default_headers) + + return resp, body diff --git a/monasca_log_api_tempest/services/log_search_client.py b/monasca_log_api_tempest/services/log_search_client.py new file mode 100644 index 00000000..fa8d11b4 --- /dev/null +++ b/monasca_log_api_tempest/services/log_search_client.py @@ -0,0 +1,56 @@ +# Copyright 2015 FUJITSU LIMITED +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +from oslo_serialization import jsonutils as json +from tempest.common import service_client + + +class LogsSearchClient(service_client.ServiceClient): + uri_prefix = "/elasticsearch" + + @staticmethod + def deserialize(body): + return json.loads(body.replace("\n", "")) + + @staticmethod + def serialize(body): + return json.dumps(body) + + def get_metadata(self): + uri = "/" + + response, body = self.get(self._uri(uri)) + self.expected_success(200, response.status) + + if body: + body = self.deserialize(body) + return response, body + + def count_search_messages(self, message): + return len(self.search_messages(message)) + + def search_messages(self, message): + uri = '_search' + body = self.serialize(dict( + query=dict( + term=dict(message=message) + ) + )) + response, body = self.post(self._uri(uri), body) + self.expected_success(200, response.status) + body = self.deserialize(body) + return body.get('hits', {}).get('hits', []) + + def _uri(self, url): + return '{}/{}'.format(self.uri_prefix, url) diff --git a/monasca_log_api_tempest/tests/__init__.py b/monasca_log_api_tempest/tests/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/monasca_log_api_tempest/tests/base.py b/monasca_log_api_tempest/tests/base.py new file mode 100644 index 00000000..8e534a89 --- /dev/null +++ b/monasca_log_api_tempest/tests/base.py @@ -0,0 +1,115 @@ +# Copyright 2015 FUJITSU LIMITED +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +import random +import string + +from oslo_config import cfg +from tempest.common import credentials_factory as cred_factory +from tempest import test +from tempest import exceptions + +from monasca_log_api_tempest import clients + +CONF = cfg.CONF +_ONE_MB = 1024 * 1024 # MB + + +def _get_message_size(size_base): + """Returns message size in number of characters. + + Method relies on UTF-8 where 1 character = 1 byte. + + """ + return int(round(size_base * _ONE_MB, 1)) + + +_SMALL_MESSAGE_SIZE = _get_message_size(0.001) +_MEDIUM_MESSAGE_SIZE = _get_message_size(0.01) +_LARGE_MESSAGE_SIZE = _get_message_size(0.1) +_REJECTABLE_MESSAGE_SIZE = _get_message_size(10) + + +def generate_unique_message(message=None, size=50): + letters = string.ascii_lowercase + + def rand(amount, space=True): + space = ' ' if space else '' + return ''.join((random.choice(letters + space) for _ in range(amount))) + + sid = rand(10, space=False) + + if not message: + message = rand(size) + return sid, sid + ' ' + message + + +def generate_small_message(message=None): + return generate_unique_message(message, _SMALL_MESSAGE_SIZE) + + +def generate_medium_message(message=None): + return generate_unique_message(message, _MEDIUM_MESSAGE_SIZE) + + +def generate_large_message(message=None): + return generate_unique_message(message, _LARGE_MESSAGE_SIZE) + + +def generate_rejectable_message(message=None): + return generate_unique_message(message, _REJECTABLE_MESSAGE_SIZE) + + +def _get_headers(headers=None, content_type="application/json"): + if not headers: + headers = {} + headers.update({ + 'Content-Type': content_type + }) + return headers + + +def _get_data(data, content_type="application/json"): + if 'application/json' == content_type: + data = { + 'message': data + } + return data + + +class BaseLogsTestCase(test.BaseTestCase): + """Base test case class for all Monitoring API tests.""" + + @classmethod + def skip_checks(cls): + super(BaseLogsTestCase, cls).skip_checks() + + @classmethod + def resource_setup(cls): + super(BaseLogsTestCase, cls).resource_setup() + auth_version = CONF.identity.auth_version + cred_provider = cred_factory.LegacyCredentialProvider(auth_version) + credentials = cred_provider.get_primary_creds() + cls.os = clients.Manager(credentials=credentials) + + cls.logs_client = cls.os.log_api_client + cls.logs_search_client = cls.os.log_search_client + + @staticmethod + def cleanup_resources(method, list_of_ids): + for resource_id in list_of_ids: + try: + method(resource_id) + except exceptions.EndpointNotFound: + pass diff --git a/monasca_log_api_tempest/tests/log_api/__init__.py b/monasca_log_api_tempest/tests/log_api/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/monasca_log_api_tempest/tests/log_api/test_single.py b/monasca_log_api_tempest/tests/log_api/test_single.py new file mode 100644 index 00000000..cccd060e --- /dev/null +++ b/monasca_log_api_tempest/tests/log_api/test_single.py @@ -0,0 +1,85 @@ +# Copyright 2015 FUJITSU LIMITED +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +from tempest import test + +from monasca_log_api_tempest.tests import base + +_RETRY_COUNT = 15 +_RETRY_WAIT = 2 + + +class TestSingleLog(base.BaseLogsTestCase): + def _run_and_wait(self, key, data, content_type='application/json', + headers=None): + def wait(): + return self.logs_search_client.count_search_messages(key) > 0 + + self.assertEqual(0, self.logs_search_client.count_search_messages(key), + 'Find log message in elasticsearch: {0}'.format(key)) + + headers = base._get_headers(headers, content_type) + data = base._get_data(data, content_type) + + response, _ = self.logs_client.send_single_log(data, headers) + self.assertEqual(204, response.status) + + test.call_until_true(wait, _RETRY_COUNT, _RETRY_WAIT) + response = self.logs_search_client.search_messages(key) + self.assertEqual(1, len(response)) + + return response + + @test.attr(type="gate") + def test_small_message(self): + self._run_and_wait(*base.generate_small_message()) + + @test.attr(type="gate") + def test_medium_message(self): + self._run_and_wait(*base.generate_medium_message()) + + @test.attr(type="gate") + def test_big_message(self): + self._run_and_wait(*base.generate_large_message()) + + @test.attr(type="gate") + def test_small_message_multiline(self): + sid, message = base.generate_small_message() + self._run_and_wait(sid, message.replace(' ', '\n')) + + @test.attr(type="gate") + def test_medium_message_multiline(self): + sid, message = base.generate_medium_message() + self._run_and_wait(sid, message.replace(' ', '\n')) + + @test.attr(type="gate") + def test_big_message_multiline(self): + sid, message = base.generate_large_message() + self._run_and_wait(sid, message.replace(' ', '\n')) + + @test.attr(type="gate") + def test_send_header_application_type(self): + sid, message = base.generate_unique_message() + headers = {'X-Application-Type': 'application-type-test'} + response = self._run_and_wait(sid, message, headers=headers) + self.assertEqual('application-type-test', + response[0]['_source']['application_type']) + + @test.attr(type="gate") + def test_send_header_dimensions(self): + sid, message = base.generate_unique_message() + headers = {'X-Dimensions': 'server:WebServer01,environment:production'} + response = self._run_and_wait(sid, message, headers=headers) + self.assertEqual('production', response[0]['_source']['environment']) + self.assertEqual('WebServer01', response[0]['_source']['server']) diff --git a/setup.cfg b/setup.cfg index 75f694f3..3bd8c17f 100644 --- a/setup.cfg +++ b/setup.cfg @@ -20,9 +20,10 @@ classifier = [files] packages = monasca_log_api + monasca_log_api_tempest data_files = - /etc/monasca = + etc/monasca = etc/monasca/log-api-config.conf etc/monasca/log-api-config.ini @@ -30,5 +31,8 @@ data_files = console_scripts = monasca-log-api = monasca_log_api.server:launch +tempest.test_plugins = + monasca_log_api_tests = monasca_log_api_tempest.plugin:MonascaLogApiTempestPlugin + [pbr] warnerrors = True diff --git a/test-requirements.txt b/test-requirements.txt index e52aa543..3ce68383 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -4,4 +4,3 @@ coverage>=4.0 hacking>=0.9.6,<0.10 mock nose - diff --git a/tox.ini b/tox.ini index ec8c7a05..bb0f0f7c 100644 --- a/tox.ini +++ b/tox.ini @@ -10,7 +10,7 @@ deps = -r{toxinidir}/requirements.txt -r{toxinidir}/test-requirements.txt commands = find ./ -type f -name '*.pyc' -delete - nosetests + nosetests -w monasca_log_api [testenv:pep8] commands = flake8 monasca_log_api