diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..36f0400 --- /dev/null +++ b/.gitignore @@ -0,0 +1,73 @@ +# Compiled source # +################### +*.com +*.class +*.dll +*.exe +*.o +*.so +*.pyc +build/ +dist/ +doc/build/ + +# Packages # +############ +# it's better to unpack these files and commit the raw source +# git has its own built in compression methods +*.7z +*.dmg +*.gz +*.iso +*.jar +*.rar +*.tar +*.zip + +# Logs and databases # +###################### +*.log +*.sql +*.sqlite +logs/* + +# OS generated files # +###################### +.DS_Store +.DS_Store? +._* +.Spotlight-V100 +.Trashes +.idea +.tox +*.sublime* +*.egg-info +Icon? +ehthumbs.db +Thumbs.db +.eggs + +# User driven backup files # +############################ +*.bak +*.swp + +# Generated by pbr while building docs +###################################### +AUTHORS +ChangeLog + +# Files created by releasenotes build +releasenotes/build + +# Test temp files +tests/plugins +tests/playbooks +tests/common +tests/*.retry + +# Vagrant artifacts +.vagrant + +# IDE created files +.idea/ diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst new file mode 100644 index 0000000..9204325 --- /dev/null +++ b/CONTRIBUTING.rst @@ -0,0 +1,79 @@ +os_searchlight +############## +:tags: openstack, cloud, ansible +:category: \*nix + +Contributor Guidelines +^^^^^^^^^^^^^^^^^^^^^^ + +Filing Bugs +----------- + +Bugs should be filed on Launchpad, not GitHub: "https://bugs.launchpad.net/openstack-ansible" + + +When submitting a bug, or working on a bug, please ensure the following criteria are met: + * The description clearly states or describes the original problem or root cause of the problem. + * Include historical information on how the problem was identified. + * Any relevant logs and configuration details are included. + * If the issue is a bug that needs fixing in a branch other than Master, add the ‘backport potential’ tag TO THE ISSUE (not the PR). + * The provided information should be totally self-contained. External access to web services/sites should not be needed. + * If the issue is needed for a hotfix release, add the 'expedite' label. + * Steps to reproduce the problem if possible. + + +Submitting Code +--------------- + +Changes to the project should be submitted for review via the Gerrit tool, following +the workflow documented at: "http://docs.openstack.org/infra/manual/developers.html#development-workflow" + +Pull requests submitted through GitHub will be ignored and closed without regard. + + +Extra +----- + +Tags: + If it's a bug that needs fixing in a branch in addition to Master, add a '\-backport-potential' tag (eg ``juno-backport-potential``). There are predefined tags that will autocomplete. + +Status: + Please leave this alone, it should be New till someone triages the issue. + +Importance: + Should only be touched if it is a Blocker/Gating issue. If it is, please set to High, and only use Critical if you have found a bug that can take down whole infrastructures. + + +Style guide +----------- + +For the definitive style guide for Openstack Ansible please see the `Openstack Ansible Contributor guidelines `_ +some relevant information is duplicated below. + +When creating tasks and other roles for use in Ansible please create then using the YAML dictionary format. + +Example YAML dictionary format: + .. code-block:: yaml + + - name: The name of the tasks + module_name: + thing1: "some-stuff" + thing2: "some-other-stuff" + tags: + - some-tag + - some-other-tag + + +Example **NOT** in YAML dictionary format: + .. code-block:: yaml + + - name: The name of the tasks + module_name: thing1="some-stuff" thing2="some-other-stuff" + tags: + - some-tag + - some-other-tag + + +Usage of the ">" and "|" operators should be limited to Ansible conditionals and command modules such as the ansible ``shell`` module. + + diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..8f71f43 --- /dev/null +++ b/LICENSE @@ -0,0 +1,202 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "{}" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright {yyyy} {name of copyright owner} + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + diff --git a/README.rst b/README.rst new file mode 100644 index 0000000..d47a394 --- /dev/null +++ b/README.rst @@ -0,0 +1,61 @@ +OpenStack-Ansible Searchlight +############################# + +Ansible role which installs and configures OpenStack Searchlight. + + +Default Variables +================= + +.. literalinclude:: ../../defaults/main.yml + :language: yaml + :start-after: under the License. + +Required Variables +================== + +This list is not considered exhaustive at present. See role internals for +further details. + +.. code-block:: yaml + + elasticsearch_apt_java_package: "openjdk-8-jre" + searchlight_rabbitmq_userid: searchlight + searchlight_rabbitmq_vhost: /searchlight + searchlight_rabbitmq_servers: "{{ rabbitmq_servers }}" + searchlight_rabbitmq_port: "{{ rabbitmq_port }}" + searchlight_rabbitmq_use_ssl: "{{ rabbitmq_use_ssl }}" + +Example Playbook +================ + +.. code-block:: yaml + + - name: Installation and setup of Searchlight + hosts: keystone_all + user: root + roles: + - role: elasticsearch + - role: "os_searchlight" + searchlight_venv_tag: "{{ openstack_release }}" + searchlight_venv_download_url: "{{ openstack_repo_url }}/venvs/{{ openstack_release }}/{{ ansible_distribution | lower }}/searchlight-{{ openstack_release }}-{{ ansible_architecture | lower }}.tgz" + pip_lock_to_internal_repo: "{{ (pip_links | length) >= 1 }}" + tags: + - "os-searchlight" + vars: + elasticsearch_apt_java_package: "openjdk-8-jre" + searchlight_rabbitmq_userid: searchlight + searchlight_rabbitmq_vhost: /searchlight + searchlight_rabbitmq_servers: "{{ rabbitmq_servers }}" + searchlight_rabbitmq_port: "{{ rabbitmq_port }}" + searchlight_rabbitmq_use_ssl: "{{ rabbitmq_use_ssl }}" + +Tags +==== + +This role supports two tags: ``searchlight-install`` and ``searchlight-config`` + +The ``searchlight-install`` tag can be used to install and upgrade. + +The ``searchlight-config`` tag can be used to maintain configuration of the +service. diff --git a/Vagrantfile b/Vagrantfile new file mode 100644 index 0000000..ec1f163 --- /dev/null +++ b/Vagrantfile @@ -0,0 +1,9 @@ +Vagrant.configure(2) do |config| + config.vm.box = "ubuntu/xenial64" + config.vm.provision "shell", inline: <<-SHELL + sudo su - + cd /vagrant + apt-get update + ./run_tests.sh + SHELL +end diff --git a/ansible-role-requirements.yml b/ansible-role-requirements.yml new file mode 100644 index 0000000..a6e613c --- /dev/null +++ b/ansible-role-requirements.yml @@ -0,0 +1,5 @@ +- name: plugins + src: https://github.com/openstack/openstack-ansible-plugins + path: /etc/ansible + scm: git + version: master diff --git a/defaults/main.yml b/defaults/main.yml new file mode 100755 index 0000000..e2365af --- /dev/null +++ b/defaults/main.yml @@ -0,0 +1,146 @@ +--- +# Copyright 2016, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +## Verbosity Options +debug: False + +# Set the package install state for distribution and pip packages +# Options are 'present' and 'latest' +searchlight_package_state: "latest" +searchlight_pip_package_state: "latest" + +# These variables are used in 'developer mode' in order to allow the role +# to build an environment directly from a git source without the presence +# of an OpenStack-Ansible repo_server. +searchlight_git_repo: https://git.openstack.org/openstack/searchlight +# searchlight_git_install_branch: 43e734917378f7eb990cc85533982abbb51d6d63 # Pin this for ocata - when back to pike remove +searchlight_git_install_branch: master +searchlight_requirements_git_repo: https://git.openstack.org/openstack/requirements +searchlight_requirements_git_install_branch: master +searchlight_developer_mode: false +searchlight_developer_constraints: + - "git+{{ searchlight_git_repo }}@{{ searchlight_git_install_branch }}#egg=searchlight" + +#: Name of the virtual env to deploy into +searchlight_venv_tag: untagged +searchlight_bin: "/openstack/venvs/searchlight-{{ searchlight_venv_tag }}/bin" +# searchlight_venv_pkgs: "/openstack/venvs/searchlight-{{ searchlight_venv_tag }}/lib/python2.7/site-packages" + +# venv_download, even when true, will use the fallback method of building the +# venv from scratch if the venv download fails. +searchlight_venv_download_url: http://127.0.0.1/venvs/untagged/ubuntu/searchlight.tgz +searchlight_venv_download: "{{ not searchlight_developer_mode | bool }}" + +#: Use of deprecated config options will cause a fatal application error +searchlight_fatal_deprecations: false + +#: External SSL forwarding proto, assumes TLS termination at load balancer +searchlight_ssl_external: true +searchlight_secure_proxy_ssl_header: HTTP_X_FORWARDED_PROTO + +#: Set this to false to disable API service through Apache + mod_wsgi +searchlight_use_mod_wsgi: true + +#: System info +searchlight_system_user_name: searchlight +searchlight_system_group_name: searchlight + +searchlight_system_shell: /bin/false +searchlight_system_comment: searchlight system user +searchlight_system_user_home: "/var/lib/{{ searchlight_system_user_name }}" + +#: searchlight packages that must be installed before anything else +searchlight_required_pip_packages: + - virtualenv + - virtualenv-tools + - python-keystoneclient # Keystoneclient needed for OSA keystone lib + - httplib2 # so we can use the uri module + - python-searchlightclient + +#: Common pip packages +searchlight_pip_packages: + - searchlight + - keystonemiddleware + - python-memcached + - pycrypto + +# This variable is used by the repo_build process to determine +# which host group to check for members of before building the +# pip packages required by this role. The value is picked up +# by the py_pkgs lookup. +searchlight_role_project_group: searchlight_all + +#: Tunable overrides +searchlight_conf_overrides: {} +searchlight_api_paste_ini_overrides: {} +searchlight_policy_overrides: {} + +elasticsearch_apt_java_package: openjdk-8-jre +searchlight_keystone_auth_plugin: password +searchlight_service_project_domain_id: default +searchlight_service_project_description: "OpenStack Services" +searchlight_service_user_domain_id: default +searchlight_service_name: "searchlight" +searchlight_service_type: "search" +searchlight_service_project_name: "service" +searchlight_service_user_name: "searchlight" +searchlight_service_description: "Searchlight service" +searchlight_api_program_name: "searchlight-api" +searchlight_manage_program_name: "searchlight-manage" +searchlight_manage_index_command: "index sync --force" +searchlight_listener_program_name: "searchlight-listener" +searchlight_role_name: "admin" + +searchlight_service_address: 0.0.0.0 +searchlight_service_port: 9393 +searchlight_service_proto: http +searchlight_service_registry_proto: "{{ searchlight_service_proto }}" +searchlight_service_publicuri_proto: "{{ openstack_service_publicuri_proto | default(searchlight_service_proto) }}" +searchlight_service_adminuri_proto: "{{ openstack_service_adminuri_proto | default(searchlight_service_proto) }}" +searchlight_service_internaluri_proto: "{{ openstack_service_internaluri_proto | default(searchlight_service_proto) }}" +searchlight_service_publicurl: "{{ searchlight_service_publicuri_proto }}://{{ external_lb_vip_address }}:{{ searchlight_service_port }}" +searchlight_service_internalurl: "{{ searchlight_service_internaluri_proto }}://{{ internal_lb_vip_address }}:{{ searchlight_service_port }}" +searchlight_service_adminurl: "{{ searchlight_service_adminuri_proto }}://{{ internal_lb_vip_address }}:{{ searchlight_service_port }}" + +# It would be better to make this discoverable - discuss with searchlight team +searchlight_nova_compute_api_version: "2.26" + +# Resource Plugins +# TODO: base these on group length where possible +# +searchlight_include_region_name: False +# If resource_plugin.include_region_name is set to True, then this value +# will be the default value for the 'region_name' field on all documents +searchlight_default_os_region_name: "" +searchlight_resource_plugin_os_nova_server_enabled: True +searchlight_resource_plugin_os_nova_server_admin_only_fields: "OS-EXT-SRV*,OS-EXT-STS:vm_state" +searchlight_resource_plugin_os_nova_hypervisor: True +searchlight_resource_plugin_os_nova_flavor_enabled: True +searchlight_resource_plugin_os_nova_servergroup_enabled: False +searchlight_resource_plugin_os_glance_image_enabled: True +searchlight_resource_plugin_os_glance_metadef_enabled: True +searchlight_resource_plugin_os_cinder_volume_enabled: True +searchlight_resource_plugin_os_cinder_snapshot_enabled: True +searchlight_resource_plugin_os_neutron_net_enabled: True +searchlight_resource_plugin_os_neutron_net_admin_only_fields: "admin_state_up,status" +searchlight_resource_plugin_os_neutron_port_enabled: True +searchlight_resource_plugin_os_neutron_security_group_enabled: True +searchlight_resource_plugin_os_designate_zone_enabled: False +searchlight_resource_plugin_os_designate_recordset_enabled: False +searchlight_resource_plugin_os_swift_account_enabled: False +searchlight_resource_plugin_os_swift_container_enabled: False +searchlight_resource_plugin_os_swift_object_enabled: False + +horizon_enable_searchlight_ui: True diff --git a/doc/Makefile b/doc/Makefile new file mode 100644 index 0000000..66267ec --- /dev/null +++ b/doc/Makefile @@ -0,0 +1,195 @@ +# Makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXOPTS = +SPHINXBUILD = sphinx-build +PAPER = +BUILDDIR = build + +# User-friendly check for sphinx-build +ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) +$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) +endif + +# Internal variables. +PAPEROPT_a4 = -D latex_paper_size=a4 +PAPEROPT_letter = -D latex_paper_size=letter +ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source +# the i18n builder cannot share the environment and doctrees with the others +I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source + +.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest coverage gettext + +help: + @echo "Please use \`make ' where is one of" + @echo " html to make standalone HTML files" + @echo " dirhtml to make HTML files named index.html in directories" + @echo " singlehtml to make a single large HTML file" + @echo " pickle to make pickle files" + @echo " json to make JSON files" + @echo " htmlhelp to make HTML files and a HTML help project" + @echo " qthelp to make HTML files and a qthelp project" + @echo " applehelp to make an Apple Help Book" + @echo " devhelp to make HTML files and a Devhelp project" + @echo " epub to make an epub" + @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" + @echo " latexpdf to make LaTeX files and run them through pdflatex" + @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" + @echo " text to make text files" + @echo " man to make manual pages" + @echo " texinfo to make Texinfo files" + @echo " info to make Texinfo files and run them through makeinfo" + @echo " gettext to make PO message catalogs" + @echo " changes to make an overview of all changed/added/deprecated items" + @echo " xml to make Docutils-native XML files" + @echo " pseudoxml to make pseudoxml-XML files for display purposes" + @echo " linkcheck to check all external links for integrity" + @echo " doctest to run all doctests embedded in the documentation (if enabled)" + @echo " coverage to run coverage check of the documentation (if enabled)" + +clean: + rm -rf $(BUILDDIR)/* + +html: + $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html + @echo + @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." + +dirhtml: + $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml + @echo + @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." + +singlehtml: + $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml + @echo + @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." + +pickle: + $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle + @echo + @echo "Build finished; now you can process the pickle files." + +json: + $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json + @echo + @echo "Build finished; now you can process the JSON files." + +htmlhelp: + $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp + @echo + @echo "Build finished; now you can run HTML Help Workshop with the" \ + ".hhp project file in $(BUILDDIR)/htmlhelp." + +qthelp: + $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp + @echo + @echo "Build finished; now you can run "qcollectiongenerator" with the" \ + ".qhcp project file in $(BUILDDIR)/qthelp, like this:" + @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/openstack-ansible-lxc_hosts.qhcp" + @echo "To view the help file:" + @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/openstack-ansible-lxc_hosts.qhc" + +applehelp: + $(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp + @echo + @echo "Build finished. The help book is in $(BUILDDIR)/applehelp." + @echo "N.B. You won't be able to view it unless you put it in" \ + "~/Library/Documentation/Help or install it in your application" \ + "bundle." + +devhelp: + $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp + @echo + @echo "Build finished." + @echo "To view the help file:" + @echo "# mkdir -p $$HOME/.local/share/devhelp/openstack-ansible-lxc_hosts" + @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/openstack-ansible-lxc_hosts" + @echo "# devhelp" + +epub: + $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub + @echo + @echo "Build finished. The epub file is in $(BUILDDIR)/epub." + +latex: + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex + @echo + @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." + @echo "Run \`make' in that directory to run these through (pdf)latex" \ + "(use \`make latexpdf' here to do that automatically)." + +latexpdf: + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex + @echo "Running LaTeX files through pdflatex..." + $(MAKE) -C $(BUILDDIR)/latex all-pdf + @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." + +latexpdfja: + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex + @echo "Running LaTeX files through platex and dvipdfmx..." + $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja + @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." + +text: + $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text + @echo + @echo "Build finished. The text files are in $(BUILDDIR)/text." + +man: + $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man + @echo + @echo "Build finished. The manual pages are in $(BUILDDIR)/man." + +texinfo: + $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo + @echo + @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." + @echo "Run \`make' in that directory to run these through makeinfo" \ + "(use \`make info' here to do that automatically)." + +info: + $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo + @echo "Running Texinfo files through makeinfo..." + make -C $(BUILDDIR)/texinfo info + @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." + +gettext: + $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale + @echo + @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." + +changes: + $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes + @echo + @echo "The overview file is in $(BUILDDIR)/changes." + +linkcheck: + $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck + @echo + @echo "Link check complete; look for any errors in the above output " \ + "or in $(BUILDDIR)/linkcheck/output.txt." + +doctest: + $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest + @echo "Testing of doctests in the sources finished, look at the " \ + "results in $(BUILDDIR)/doctest/output.txt." + +coverage: + $(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage + @echo "Testing of coverage in the sources finished, look at the " \ + "results in $(BUILDDIR)/coverage/python.txt." + +xml: + $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml + @echo + @echo "Build finished. The XML files are in $(BUILDDIR)/xml." + +pseudoxml: + $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml + @echo + @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." + +livehtml: html + sphinx-autobuild -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html diff --git a/doc/source/conf.py b/doc/source/conf.py new file mode 100644 index 0000000..797f6f4 --- /dev/null +++ b/doc/source/conf.py @@ -0,0 +1,337 @@ +#!/usr/bin/env python3 + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or +# implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# This file is execfile()d with the current directory set to its +# containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +import openstackdocstheme +import pbr.version +import os + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# sys.path.insert(0, os.path.abspath('.')) + +# -- General configuration ------------------------------------------------ + +# If your documentation needs a minimal Sphinx version, state it here. +# needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + 'sphinx.ext.autodoc', + 'oslosphinx', + 'sphinxmark' +] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix(es) of source filenames. +# You can specify multiple suffix as a list of string: +# source_suffix = ['.rst', '.md'] +source_suffix = '.rst' + +# The encoding of source files. +# source_encoding = 'utf-8-sig' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +author = 'OpenStack-Ansible Contributors' +category = 'Miscellaneous' +copyright = '2014-2016, OpenStack-Ansible Contributors' +description = 'OpenStack-Ansible deploys OpenStack environments using Ansible.' +project = 'OpenStack-Ansible' +role_name = 'os_searchlight' +target_name = 'openstack-ansible-' + role_name +title = 'OpenStack-Ansible Documentation: ' + role_name + 'role' + +# The link to the browsable source code (for the left hand menu) +oslosphinx_cgit_link = 'https://git.openstack.org/cgit/openstack/' + target_name + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. +# +# The short X.Y version. +version_info = pbr.version.VersionInfo(target_name) +# The full version, including alpha/beta/rc tags. +release = version_info.version_string_with_vcs() +# The short X.Y version. +version = version_info.canonical_version_string() + +# A few variables have to be set for the log-a-bug feature. +# giturl: The location of conf.py on Git. Must be set manually. +# gitsha: The SHA checksum of the bug description. Automatically extracted from git log. +# bug_tag: Tag for categorizing the bug. Must be set manually. +# These variables are passed to the logabug code via html_context. +giturl = ("https://git.openstack.org/cgit/openstack/{0}" + "/tree/doc/source").format(target_name) +git_cmd = "/usr/bin/git log | head -n1 | cut -f2 -d' '" +gitsha = os.popen(git_cmd).read().strip('\n') +bug_project = project.lower() +bug_title = "Documentation bug" +html_context = {"gitsha": gitsha, "giturl": giturl, + "bug_tag": "docs", "bug_title": bug_title, + "bug_project": bug_project } + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +# +# This is also used if you do content translation via gettext catalogs. +# Usually you set "language" from the command line for these cases. +language = None + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +# today = '' +# Else, today_fmt is used as the format for a strftime call. +# today_fmt = '%B %d, %Y' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +exclude_patterns = [] + +# The reST default role (used for this markup: `text`) to use for all +# documents. +# default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +# add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +# add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +# show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# A list of ignored prefixes for module index sorting. +# modindex_common_prefix = [] + +# If true, keep warnings as "system message" paragraphs in the built documents. +# keep_warnings = False + +# If true, `todo` and `todoList` produce output, else they produce nothing. +todo_include_todos = False + + +# -- Options for HTML output ---------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +html_theme = 'openstackdocs' + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +# html_theme_options = {} + +# Add any paths that contain custom themes here, relative to this directory. +html_theme_path = [openstackdocstheme.get_html_theme_path()] + +# The name for this set of Sphinx documents. If None, it defaults to +# " v documentation". +# html_title = None + +# A shorter title for the navigation bar. Default is the same as html_title. +# html_short_title = None + +# The name of an image file (relative to this directory) to place at the top +# of the sidebar. +# html_logo = None + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +# html_favicon = None + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + +# Add any extra paths that contain custom files (such as robots.txt or +# .htaccess) here, relative to this directory. These files are copied +# directly to the root of the documentation. +# html_extra_path = [] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +html_last_updated_fmt = '%Y-%m-%d %H:%M' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +# html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +# html_sidebars = {} + +# Additional templates that should be rendered to pages, maps page names to +# template names. +# html_additional_pages = {} + +# If false, no module index is generated. +# html_domain_indices = True + +# If false, no index is generated. +# html_use_index = True + +# If true, the index is split into individual pages for each letter. +# html_split_index = False + +# If true, links to the reST sources are added to the pages. +# html_show_sourcelink = True + +# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. +# html_show_sphinx = True + +# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. +# html_show_copyright = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +# html_use_opensearch = '' + +# This is the file name suffix for HTML files (e.g. ".xhtml"). +# html_file_suffix = None + +# Language to be used for generating the HTML full-text search index. +# Sphinx supports the following languages: +# 'da', 'de', 'en', 'es', 'fi', 'fr', 'h', 'it', 'ja' +# 'nl', 'no', 'pt', 'ro', 'r', 'sv', 'tr' +# html_search_language = 'en' + +# A dictionary with options for the search language support, empty by default. +# Now only 'ja' uses this config value +# html_search_options = {'type': 'default'} + +# The name of a javascript file (relative to the configuration directory) that +# implements a search results scorer. If empty, the default will be used. +# html_search_scorer = 'scorer.js' + +# Output file base name for HTML help builder. +htmlhelp_basename = target_name + '-docs' + +# -- Options for LaTeX output --------------------------------------------- + +latex_elements = { + # The paper size ('letterpaper' or 'a4paper'). + # 'papersize': 'letterpaper', + + # The font size ('10pt', '11pt' or '12pt'). + # 'pointsize': '10pt', + + # Additional stuff for the LaTeX preamble. + # 'preamble': '', + + # Latex figure (float) alignment + # 'figure_align': 'htbp', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, +# author, documentclass [howto, manual, or own class]). +latex_documents = [ + (master_doc, target_name + '.tex', + title, author, 'manual'), +] + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +# latex_logo = None + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +# latex_use_parts = False + +# If true, show page references after internal links. +# latex_show_pagerefs = False + +# If true, show URL addresses after external links. +# latex_show_urls = False + +# Documents to append as an appendix to all manuals. +# latex_appendices = [] + +# If false, no module index is generated. +# latex_domain_indices = True + + +# -- Options for manual page output --------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + (master_doc, target_name, + title, [author], 1) +] + +# If true, show URL addresses after external links. +# man_show_urls = False + + +# -- Options for Texinfo output ------------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + (master_doc, target_name, + title, author, project, + description, category), +] + +# Documents to append as an appendix to all manuals. +# texinfo_appendices = [] + +# If false, no module index is generated. +# texinfo_domain_indices = True + +# How to display URL addresses: 'footnote', 'no', or 'inline'. +# texinfo_show_urls = 'footnote' + +# If true, do not generate a @detailmenu in the "Top" node's menu. +# texinfo_no_detailmenu = False + + +watermark = os.popen("git branch --contains $(git rev-parse HEAD)\ +| awk -F/ '/stable/ {print $2}'").read().strip(' \n\t').capitalize() +if watermark == "": + watermark = "Pre-release" + +# -- Options for sphinxmark ----------------------------------------------- +sphinxmark_enable = True +sphinxmark_div = 'docs-body' +sphinxmark_image = 'text' +sphinxmark_text = watermark +sphinxmark_text_color = (128, 128, 128) +sphinxmark_text_size = 70 diff --git a/doc/source/index.rst b/doc/source/index.rst new file mode 100644 index 0000000..a6210d3 --- /dev/null +++ b/doc/source/index.rst @@ -0,0 +1 @@ +.. include:: ../../README.rst diff --git a/handlers/main.yml b/handlers/main.yml new file mode 100755 index 0000000..d04dbd3 --- /dev/null +++ b/handlers/main.yml @@ -0,0 +1,25 @@ +--- +# Copyright 2015, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# searchlight_service_names should exclude the API if {{ searchlight_use_mod_wsgi }} +- name: Restart Searchlight services + systemd: + name: "{{ item }}" + state: "restarted" + daemon_reload: yes +# when: not searchlight_use_mod_wsgi | bool + with_items: + - "{{ searchlight_listener_program_name }}" + - "{{ searchlight_api_program_name }}" diff --git a/library/searchlight b/library/searchlight new file mode 100644 index 0000000..3ca5f26 --- /dev/null +++ b/library/searchlight @@ -0,0 +1,313 @@ +#!/usr/bin/env python +# Copyright 2014, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +import json +import re + +from ansible.module_utils.basic import * + +DOCUMENTATION = """ +--- +module: searchlight +short_description: + - Module for working with OpenStack Searchlight +description: + - Module for working with OpenStack Searchlight +options: + command: + description: + - Operation to perform. Options include + choices: + - ensure_archive_policy + - ensure_archive_policy_rule + required: True + openrc_path: + decription: + - Path to openrc file containing credentials which will be used to + authorize with Keystone and work with Searchlight. + required: True + region_name: + decription: + - OpenStack Region Name for the Searchlight service to work with. + This value is only used when OS_REGION_NAME is not specified in + the provided openrc file + required: False + default: None + name: + description: + - Name assigned to the policy or policy-rule + required: True + definition: + description: + - Array of dictionaries containing parameters for each entry. + Required when specifying an archive-policy. + required: False + default: None + back_window: + description: + - Back window of the archive policy + required: False + default: 0 + aggregation_methods: + description: + - Aggregation method of the archive policy. Defaults to Searchlight + default aggregation methods from configuration when omitted. + required: False + default: None + archive_policy_name: + description: + - Name of the archive-policy to use. Required when specifying an + archive-policy-rule. + required: False + default: None + metric_pattern: + description: + - Wildcard string of metric name(s) to match. Required when + specifying an archive-policy-rule. + required: False + default: None + +requirements: [ searchlightclient ] +author: Steve Lewis -- https://github.com/stevelle +""" + +EXAMPLES = """ +# Ensure a given policy exists + searchlight: + command: 'ensure_archive_policy' + openrc_path: '/root/openrc' + name: 'simple-policy' + definition: + - { granularity: '1m', points: 60 } + - { granularity: '1h', points: 24 } + +# Ensure the default policy is set + searchlight: + command: 'ensure_archive_policy_rule' + openrc_path: '/root/openrc' + name: 'default' + archive_policy_name: 'low' + metric_pattern: '*' +""" + +COMMAND_MAP = { + 'ensure_archive_policy': { + 'vars': ['name', 'definition', 'back_window', 'aggregation_methods'] + }, + 'ensure_archive_policy_rule': { + 'vars': ['name', 'archive_policy_name', 'metric_pattern'] + } +} + +# Options accepted by keystoneauth1.identity.generic.password.Password +AUTH_OPTS = ['auth_url', 'tenant_id', 'tenant_name', 'project_id', + 'project_name', 'project_domain_id', 'project_domain_name', + 'domain_id', 'domain_name', 'trust_id', 'username', 'user_id', + 'password', 'user_domain_id', 'user_domain_name'] + + +class ManageSearchlight(object): + def __init__(self, module): + self.state_change = False + self.searchlight = None + self.module = module + try: + self._init_searchlight() + except Exception as e: + self.module.fail_json( + err='Initialisation Error: %s' % e, + rc=1, msg=str(e)) + + def _parse_openrc(self): + """Get credentials from an openrc file.""" + openrc_path = self.module.params['openrc_path'] + line_re = re.compile('^export (?POS_\w*)=(?P[^\n]*)') + with open(openrc_path) as openrc: + matches = [line_re.match(line) for line in openrc] + return dict( + (g.groupdict()['key'].lower()[3:], g.groupdict()['value']) + for g in matches if g + ) + + def _auth_session(self): + """Create a keystone auth session.""" + from keystoneauth1.identity.generic import password + from keystoneauth1 import session + + opts = {k: v for k, v in self._parse_openrc().iteritems() + if k in AUTH_OPTS} + + auth = password.Password(**opts) + return session.Session(auth=auth) + + def _init_searchlight(self): + """Create searchlight client object using token and url from keystone. + """ + opts = self._parse_openrc() + params = self.module.params + interface = opts.get('endpoint_type', 'internal') + region = opts.get('region_name', params.get('region_name', None)) + if not region: + self.module.fail_json(msg='Missing region_name in openrc or as ' + 'option to module') + + try: + import searchlightclient.v1.client as gclient + except ImportError: + self.module.fail_json('Missing searchlightclient') + else: + self.searchlight = gclient.Client( + session=self._auth_session(), + interface=interface, + region_name=region) + + def _get_vars(self, variables, required=None): + """Return a dict of all variables as found within the module. + + :param variables: ``list`` List of all variables that are available to + use within the Keystone Command. + :param required: ``list`` Name of variables that are required. + """ + return_dict = {} + for variable in variables: + return_dict[variable] = self.module.params.get(variable) + else: + if isinstance(required, list): + for var_name in required: + check = return_dict.get(var_name) + if check is None: + self.failure( + error='Missing [ %s ] from Task or found a None' + ' value' % var_name, + rc=3, + msg='variables %s - available params [ %s ]' + % (variables, self.module.params) + ) + return return_dict + + def route(self): + """Run the command as its provided to the module.""" + command_name = self.module.params['command'] + if command_name not in COMMAND_MAP: + self.module.fail_json( + error='No Command Found', + rc=2, + msg='Command [ %s ] was not found.' % command_name + ) + + action_command = COMMAND_MAP[command_name] + if hasattr(self, '%s' % command_name): + action = getattr(self, '%s' % command_name) + resource = action(vars=action_command['vars' + '' + '']) + if resource is None: + self.module.exit_json(changed=self.state_change) + else: + self.module.exit_json( + changed=self.state_change, + resource=json.dumps(resource) + ) + else: + self.module.fail_json( + error='Command not in ManageKeystone class', + rc=2, + msg='Method [ %s ] was not found.' % command_name + ) + + def ensure_archive_policy(self, vars): + """Create a new archive policy within Searchlight if it does not exist. + + Returns the policy name on a successful run. + + :param vars: ``list`` List of all variables that are available + to use within the archive policy create + command. + """ + import searchlightclient.exceptions + + variables = self._get_vars(vars) + policy_mgr = self.searchlight.archive_policy + try: + ap = policy_mgr.get(name=variables.get('name', None)) + except searchlightclient.exceptions.ArchivePolicyNotFound: + if 'definition' not in vars: + msg = 'Missing archive policy %s' + self.module.fail_json(msg=msg % 'definition') + + # remove a blank value, as default + if ('aggregation_methods' in variables and + variables['aggregation_methods'] == ''): + variables.pop('aggregation_methods') + + self.state_change = True + try: + ap = policy_mgr.create(archive_policy=variables) + except searchlightclient.exceptions.ClientException as e: + self.module.fail_json(msg='Policy Create Error', + rc=e.code, details=e.message) + + return ap + + def ensure_archive_policy_rule(self, vars): + """Create a new archive policy rule within Searchlight if it does not + exist. + + Returns the policy rule name on a successful run. + + :param vars: ``list`` List of all variables that are available to + use within the archive policy rule create + command. + """ + import searchlightclient.exceptions as exceptions + + variables = self._get_vars(vars) + named = variables.get('name', None) + rule_mgr = self.searchlight.archive_policy_rule + try: + apr = rule_mgr.get(name=named) + # NOTE(stevelle) client lib raises wrong exception type. + # see https://bugs.launchpad.net/python-searchlightclient/+bug/1527402 + except (exceptions.ArchivePolicyRuleNotFound, + exceptions.ArchivePolicyNotFound): + self.state_change = True + try: + apr = rule_mgr.create(archive_policy_rule=variables) + except exceptions.ClientException as e: + self.module.fail_json(msg='Policy Rule Create Error', + rc=e.code, details=e.message) + + return apr + + +def main(): + module = AnsibleModule( + argument_spec=dict( + command=dict(required=True, choices=COMMAND_MAP.keys()), + openrc_path=dict(required=True), + region_name=dict(required=False), + name=dict(required=True), + definition=dict(required=False), + back_window=dict(required=False, default='0'), + aggregation_methods=dict(required=False), + archive_policy_name=dict(required=False), + metric_pattern=dict(required=False) + ), + supports_check_mode=False + ) + mg = ManageSearchlight(module) + mg.route() + +if __name__ == '__main__': + main() diff --git a/meta/main.yml b/meta/main.yml new file mode 100755 index 0000000..1a16e6e --- /dev/null +++ b/meta/main.yml @@ -0,0 +1,22 @@ +--- +galaxy_info: + author: Neill Cox + description: An Ansible Galaxy Role for Deploying OpenStack Searchlight + company: Rackspace + license: Apache + min_ansible_version: 2.1 + platforms: + - name: Ubuntu + versions: + - xenial + categories: + - cloud + - openstack +dependencies: + - pip_install + - role: elasticsearch + tags: skip_ansible_lint + - role: apt_package_pinning + when: + - ansible_pkg_mgr == 'apt' + - openstack_openrc diff --git a/other-requirements.txt b/other-requirements.txt new file mode 100644 index 0000000..3307388 --- /dev/null +++ b/other-requirements.txt @@ -0,0 +1,21 @@ +# This file facilitates OpenStack-CI package installation +# before the execution of any tests. +# +# See the following for details: +# - http://docs.openstack.org/infra/bindep/ +# - https://github.com/openstack-infra/bindep +# +# Even if the role does not make use of this facility, it +# is better to have this file empty, otherwise OpenStack-CI +# will fall back to installing its default packages which +# will potentially be detrimental to the tests executed. + +# OpenStack-CI's Jenkins needs curl +# TODO(odyssey4me) remove this once https://review.openstack.org/288634 has merged +# and the disk images are rebuilt and redeployed. +curl + + +# Requirements for Paramiko 2.0 +libssl-dev +libffi-dev diff --git a/releasenotes/notes/.placeholder b/releasenotes/notes/.placeholder new file mode 100644 index 0000000..e69de29 diff --git a/releasenotes/source/_static/.placeholder b/releasenotes/source/_static/.placeholder new file mode 100644 index 0000000..e69de29 diff --git a/releasenotes/source/_templates/.placeholder b/releasenotes/source/_templates/.placeholder new file mode 100644 index 0000000..e69de29 diff --git a/releasenotes/source/conf.py b/releasenotes/source/conf.py new file mode 100644 index 0000000..22c485d --- /dev/null +++ b/releasenotes/source/conf.py @@ -0,0 +1,284 @@ +#!/usr/bin/env python3 + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or +# implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# This file is execfile()d with the current directory set to its +# containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +import pbr.version + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# sys.path.insert(0, os.path.abspath('.')) + +# -- General configuration ------------------------------------------------ + +# If your documentation needs a minimal Sphinx version, state it here. +# needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + 'oslosphinx', + 'reno.sphinxext', +] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix of source filenames. +source_suffix = '.rst' + +# The encoding of source files. +# source_encoding = 'utf-8-sig' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +author = 'OpenStack-Ansible Contributors' +category = 'Miscellaneous' +copyright = '2014-2016, OpenStack-Ansible Contributors' +description = 'OpenStack-Ansible deploys OpenStack environments using Ansible.' +project = 'OpenStack-Ansible' +role_name = 'os_searchlight' +target_name = 'openstack-ansible-' + role_name +title = 'OpenStack-Ansible Release Notes: ' + role_name + 'role' + +# The link to the browsable source code (for the left hand menu) +oslosphinx_cgit_link = 'https://git.openstack.org/cgit/openstack/' + target_name + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. +# +# The short X.Y version. +version_info = pbr.version.VersionInfo(target_name) +# The full version, including alpha/beta/rc tags. +release = version_info.version_string_with_vcs() +# The short X.Y version. +version = version_info.canonical_version_string() + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +# language = None + +# There are two options for replacing |today|: either, you set today to some +# non-false value, then it is used: +# today = '' +# Else, today_fmt is used as the format for a strftime call. +# today_fmt = '%B %d, %Y' + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +exclude_patterns = [] + +# The reST default role (used for this markup: `text`) to use for all +# documents. +# default_role = None + +# If true, '()' will be appended to :func: etc. cross-reference text. +# add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +# add_module_names = True + +# If true, sectionauthor and moduleauthor directives will be shown in the +# output. They are ignored by default. +# show_authors = False + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# A list of ignored prefixes for module index sorting. +# modindex_common_prefix = [] + +# If true, keep warnings as "system message" paragraphs in the built documents. +# keep_warnings = False + + +# -- Options for HTML output ---------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +html_theme = 'default' + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +# html_theme_options = {} + +# Add any paths that contain custom themes here, relative to this directory. +# html_theme_path = [] + +# The name for this set of Sphinx documents. If None, it defaults to +# " v documentation". +# html_title = None + +# A shorter title for the navigation bar. Default is the same as html_title. +# html_short_title = None + +# The name of an image file (relative to this directory) to place at the top +# of the sidebar. +# html_logo = None + +# The name of an image file (within the static path) to use as favicon of the +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 +# pixels large. +# html_favicon = None + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + +# Add any extra paths that contain custom files (such as robots.txt or +# .htaccess) here, relative to this directory. These files are copied +# directly to the root of the documentation. +# html_extra_path = [] + +# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, +# using the given strftime format. +html_last_updated_fmt = '%Y-%m-%d %H:%M' + +# If true, SmartyPants will be used to convert quotes and dashes to +# typographically correct entities. +# html_use_smartypants = True + +# Custom sidebar templates, maps document names to template names. +# html_sidebars = {} + +# Additional templates that should be rendered to pages, maps page names to +# template names. +# html_additional_pages = {} + +# If false, no module index is generated. +# html_domain_indices = True + +# If false, no index is generated. +# html_use_index = True + +# If true, the index is split into individual pages for each letter. +# html_split_index = False + +# If true, links to the reST sources are added to the pages. +# html_show_sourcelink = True + +# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. +# html_show_sphinx = True + +# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. +# html_show_copyright = True + +# If true, an OpenSearch description file will be output, and all pages will +# contain a tag referring to it. The value of this option must be the +# base URL from which the finished HTML is served. +# html_use_opensearch = '' + +# This is the file name suffix for HTML files (e.g. ".xhtml"). +# html_file_suffix = None + +# Output file base name for HTML help builder. +htmlhelp_basename = target_name + '-docs' + + +# -- Options for LaTeX output --------------------------------------------- + +latex_elements = { + # The paper size ('letterpaper' or 'a4paper'). + # 'papersize': 'letterpaper', + + # The font size ('10pt', '11pt' or '12pt'). + # 'pointsize': '10pt', + + # Additional stuff for the LaTeX preamble. + # 'preamble': '', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, +# author, documentclass [howto, manual, or own class]). +latex_documents = [ + (master_doc, target_name + '.tex', + title, author, 'manual'), +] + +# The name of an image file (relative to this directory) to place at the top of +# the title page. +# latex_logo = None + +# For "manual" documents, if this is true, then toplevel headings are parts, +# not chapters. +# latex_use_parts = False + +# If true, show page references after internal links. +# latex_show_pagerefs = False + +# If true, show URL addresses after external links. +# latex_show_urls = False + +# Documents to append as an appendix to all manuals. +# latex_appendices = [] + +# If false, no module index is generated. +# latex_domain_indices = True + + +# -- Options for manual page output --------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + (master_doc, target_name, + title, [author], 1) +] + +# If true, show URL addresses after external links. +# man_show_urls = False + + +# -- Options for Texinfo output ------------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + (master_doc, target_name, + title, author, project, + description, category), +] + +# Documents to append as an appendix to all manuals. +# texinfo_appendices = [] + +# If false, no module index is generated. +# texinfo_domain_indices = True + +# How to display URL addresses: 'footnote', 'no', or 'inline'. +# texinfo_show_urls = 'footnote' + +# If true, do not generate a @detailmenu in the "Top" node's menu. +# texinfo_no_detailmenu = False + +# -- Options for Internationalization output ------------------------------ +locale_dirs = ['locale/'] diff --git a/releasenotes/source/index.rst b/releasenotes/source/index.rst new file mode 100644 index 0000000..71f7357 --- /dev/null +++ b/releasenotes/source/index.rst @@ -0,0 +1,8 @@ +================================ + OpenStack-Ansible Release Notes +================================ + +.. toctree:: + :maxdepth: 1 + + unreleased diff --git a/releasenotes/source/unreleased.rst b/releasenotes/source/unreleased.rst new file mode 100644 index 0000000..cd22aab --- /dev/null +++ b/releasenotes/source/unreleased.rst @@ -0,0 +1,5 @@ +============================== + Current Series Release Notes +============================== + +.. release-notes:: diff --git a/run_tests.sh b/run_tests.sh new file mode 100755 index 0000000..5bd6a64 --- /dev/null +++ b/run_tests.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash +# Copyright 2016, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -euov + +FUNCTIONAL_TEST=${FUNCTIONAL_TEST:-true} + +# prep the host +if [ "$(which apt-get)" ]; then + apt-get install -y build-essential python2.7 python-dev git-core libssl-dev libffi-dev +fi + +# get pip, if necessary +if [ ! "$(which pip)" ]; then + curl --silent --show-error --retry 5 \ + https://bootstrap.pypa.io/get-pip.py | sudo python2.7 +fi + +# install tox +pip install tox + +# run through each tox env and execute the test +for tox_env in $(awk -F= '/envlist/ {print $2}' tox.ini | sed 's/,/ /g'); do + if [ "${tox_env}" != "functional" ]; then + tox -e ${tox_env} + elif [ "${tox_env}" == "functional" ]; then + if ${FUNCTIONAL_TEST}; then + tox -e ${tox_env} + fi + fi +done diff --git a/setup.cfg b/setup.cfg new file mode 100644 index 0000000..755e384 --- /dev/null +++ b/setup.cfg @@ -0,0 +1,25 @@ +[metadata] +name = openstack-ansible-os_searchlight +summary = os_searchlight for OpenStack Ansible +description-file = + README.rst +author = OpenStack +author-email = openstack-dev@lists.openstack.org +home-page = http://www.openstack.org/ +classifier = + Intended Audience :: Developers + Intended Audience :: System Administrators + License :: OSI Approved :: Apache Software License + Operating System :: POSIX :: Linux + +[build_sphinx] +all_files = 1 +build-dir = doc/build +source-dir = doc/source + +[pbr] +warnerrors = True + +[wheel] +universal = 1 + diff --git a/setup.py b/setup.py new file mode 100644 index 0000000..782bb21 --- /dev/null +++ b/setup.py @@ -0,0 +1,29 @@ +# Copyright (c) 2013 Hewlett-Packard Development Company, L.P. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or +# implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT +import setuptools + +# In python < 2.7.4, a lazy loading of package `pbr` will break +# setuptools if some other modules registered functions in `atexit`. +# solution from: http://bugs.python.org/issue15881#msg170215 +try: + import multiprocessing # noqa +except ImportError: + pass + +setuptools.setup( + setup_requires=['pbr>=1.8'], + pbr=True) diff --git a/tasks/main.yml b/tasks/main.yml new file mode 100755 index 0000000..af89aef --- /dev/null +++ b/tasks/main.yml @@ -0,0 +1,53 @@ +--- +# Copyright 2015, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: Gather variables for each operating system + include_vars: "{{ item }}" + with_first_found: + - "{{ ansible_distribution | lower }}-{{ ansible_distribution_version | lower }}.yml" + - "{{ ansible_distribution | lower }}-{{ ansible_distribution_major_version | lower }}.yml" + - "{{ ansible_os_family | lower }}-{{ ansible_distribution_major_version | lower }}.yml" + - "{{ ansible_distribution | lower }}.yml" + - "{{ ansible_os_family | lower }}.yml" + tags: + - always + +- include: searchlight_pre_install.yml + tags: + - searchlight-install + +- include: searchlight_install.yml + tags: + - searchlight-install + +- include: searchlight_post_install.yml + tags: + - searchlight-install + - searchlight-config + +- include: searchlight_init.yml + tags: + - searchlight-install + +- include: searchlight_index_sync.yml + when: inventory_hostname == groups['searchlight_all'][0] + tags: + - searchlight-install + +- include: searchlight_service_setup.yml + when: inventory_hostname == groups['searchlight_all'][0] + tags: + - searchlight-install + diff --git a/tasks/searchlight_index_sync.yml b/tasks/searchlight_index_sync.yml new file mode 100644 index 0000000..b20d92c --- /dev/null +++ b/tasks/searchlight_index_sync.yml @@ -0,0 +1,22 @@ +--- +# Copyright 2014, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# TODO: See if this can be made idempotent +- name: Perform a Searchlight Index sync + command: "{{ searchlight_bin }}/{{ searchlight_manage_program_name }} {{ searchlight_manage_index_command }}" + become: yes + become_user: "{{ searchlight_system_user_name }}" + changed_when: false + tags: skip_ansible_lint diff --git a/tasks/searchlight_init.yml b/tasks/searchlight_init.yml new file mode 100644 index 0000000..59a224a --- /dev/null +++ b/tasks/searchlight_init.yml @@ -0,0 +1,24 @@ +--- +# Copyright 2015, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- include: searchlight_init_common.yml + with_items: + - program_name: "{{ searchlight_api_program_name }}" + - program_name: "{{ searchlight_listener_program_name }}" + vars: + service_name: "{{ searchlight_service_name }}" + system_user: "{{ searchlight_system_user_name }}" + system_group: "{{ searchlight_system_group_name }}" + service_home: "{{ searchlight_system_user_home }}" diff --git a/tasks/searchlight_init_common.yml b/tasks/searchlight_init_common.yml new file mode 100644 index 0000000..9459033 --- /dev/null +++ b/tasks/searchlight_init_common.yml @@ -0,0 +1,23 @@ +--- +# Copyright 2016, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- include: searchlight_init_systemd.yml + +- name: Load service + systemd: + name: "{{ item.program_name }}" + enabled: "yes" + notify: + - Restart Searchlight services diff --git a/tasks/searchlight_init_systemd.yml b/tasks/searchlight_init_systemd.yml new file mode 100755 index 0000000..6cb3479 --- /dev/null +++ b/tasks/searchlight_init_systemd.yml @@ -0,0 +1,55 @@ +--- +# Copyright 2016, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: Create TEMP run dir + file: + path: "/var/run/{{ item.program_name }}" + state: directory + owner: "{{ system_user }}" + group: "{{ system_group }}" + mode: "02755" + +- name: Create TEMP lock dir + file: + path: "/var/lock/{{ item.program_name }}" + state: directory + owner: "{{ system_user }}" + group: "{{ system_group }}" + mode: "02755" + +- name: Create tempfile.d entry + template: + src: "searchlight-systemd-tempfiles.j2" + dest: "/etc/tmpfiles.d/searchlight.conf" + mode: "0644" + owner: "root" + group: "root" + +- name: Place the systemd init script + template: + src: "searchlight-systemd-init.j2" + dest: "/etc/systemd/system/{{ item.program_name }}.service" + mode: "0744" + owner: "root" + group: "root" + register: systemd_init + +- name: Reload the systemd daemon + command: "systemctl daemon-reload" + when: systemd_init | changed + notify: + - Restart Searchlight services + + diff --git a/tasks/searchlight_install.yml b/tasks/searchlight_install.yml new file mode 100644 index 0000000..631f2b9 --- /dev/null +++ b/tasks/searchlight_install.yml @@ -0,0 +1,109 @@ +--- +# Copyright 2015, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- include: searchlight_install_apt.yml + when: + - ansible_pkg_mgr == 'apt' + +- name: Create developer mode constraint file + copy: + dest: "/opt/developer-pip-constraints.txt" + content: | + {% for item in searchlight_developer_constraints %} + {{ item }} + {% endfor %} + when: + - searchlight_developer_mode | bool + +- name: Clone requirements git repository + git: + repo: "{{ searchlight_requirements_git_repo }}" + dest: "/opt/requirements" + clone: yes + update: yes + version: "{{ searchlight_requirements_git_install_branch }}" + when: + - searchlight_developer_mode | bool + +- name: Install required pip packages + pip: + name: "{{ searchlight_required_pip_packages | join(' ') }}" + state: "{{ searchlight_pip_package_state }}" + extra_args: >- + {{ searchlight_developer_mode | ternary(pip_install_developer_constraints | default('--constraint /opt/developer-pip-constraints.txt'), '') }} + {{ (pip_install_upper_constraints is defined) | ternary('--constraint ' + pip_install_upper_constraints | default(''),'') }} + {{ pip_install_options | default('') }} + register: install_packages + until: install_packages|success + retries: 5 + delay: 2 + +- name: Attempt venv download + get_url: + url: "{{ searchlight_venv_download_url }}" + dest: "/var/cache/{{ searchlight_venv_download_url | basename }}" + checksum: "sha1:{{ lookup('url', searchlight_venv_download_url | replace('tgz', 'checksum')) }}" + register: searchlight_get_venv + when: + - searchlight_venv_download | bool + +- name: Remove existing venv + file: + path: "{{ searchlight_bin | dirname }}" + state: absent + when: searchlight_get_venv | changed + +- name: Create searchlight venv dir + file: + path: "{{ searchlight_bin | dirname }}" + state: directory + register: searchlight_venv_dir + when: searchlight_get_venv | changed + +- name: Unarchive pre-built venv + unarchive: + src: "/var/cache/{{ searchlight_venv_download_url | basename }}" + dest: "{{ searchlight_bin | dirname }}" + copy: "no" + when: searchlight_get_venv | changed + notify: + - Restart Searchlight services + +- name: Install pip packages + pip: + name: "{{ searchlight_pip_packages }}" + state: "{{ searchlight_pip_package_state }}" + virtualenv: "{{ searchlight_bin | dirname }}" + virtualenv_site_packages: "no" + extra_args: >- + {{ searchlight_developer_mode | ternary(pip_install_developer_constraints | default('--constraint /opt/developer-pip-constraints.txt'), '') }} + {{ (pip_install_upper_constraints is defined) | ternary('--constraint ' + pip_install_upper_constraints | default(''),'') }} + {{ pip_install_options | default('') }} + register: install_packages + until: install_packages|success + retries: 5 + delay: 2 + when: searchlight_get_venv | failed or searchlight_get_venv | skipped + notify: + - Restart Searchlight services + + +- name: Update virtualenv path + command: > + virtualenv-tools --update-path=auto {{ searchlight_bin | dirname }} + when: + - not searchlight_developer_mode | bool + - searchlight_get_venv | success + diff --git a/tasks/searchlight_install_apt.yml b/tasks/searchlight_install_apt.yml new file mode 100644 index 0000000..fa1c726 --- /dev/null +++ b/tasks/searchlight_install_apt.yml @@ -0,0 +1,38 @@ +--- +# Copyright 2015, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +#TODO(evrardjp): Replace the next 2 tasks by a standard apt with cache +#when https://github.com/ansible/ansible-modules-core/pull/1517 is merged +#in 1.9.x or we move to 2.0 (if tested working) + +- name: Check apt last update file + stat: + path: /var/cache/apt + register: apt_cache_stat + +- name: Update apt if needed + apt: + update_cache: yes + when: "ansible_date_time.epoch|float - apt_cache_stat.stat.mtime > {{cache_timeout}}" + +- name: Install apt packages + apt: + pkg: "{{ item }}" + state: "{{ searchlight_package_state }}" + register: install_packages + until: install_packages|success + retries: 5 + delay: 2 + with_items: "{{ searchlight_apt_packages }}" diff --git a/tasks/searchlight_policy_setup.yml b/tasks/searchlight_policy_setup.yml new file mode 100755 index 0000000..b4bc98c --- /dev/null +++ b/tasks/searchlight_policy_setup.yml @@ -0,0 +1,24 @@ +--- +# Copyright 2015, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: Ensure Searchlight API service is (re)started + meta: flush_handlers + +- name: Wait for Searchlight API to come up + wait_for: + host: "{{ container_address }}" + port: "{{ searchlight_service_port }}" + timeout: 25 + delay: 10 diff --git a/tasks/searchlight_post_install.yml b/tasks/searchlight_post_install.yml new file mode 100644 index 0000000..123bf52 --- /dev/null +++ b/tasks/searchlight_post_install.yml @@ -0,0 +1,40 @@ +--- +# Copyright 2015, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: Drop Searchlight Config(s) + config_template: + src: "{{ item.src }}" + dest: "{{ item.dest }}" + owner: "{{ searchlight_system_user_name }}" + group: "{{ searchlight_system_group_name }}" + mode: "0644" + config_overrides: "{{ item.config_overrides }}" + config_type: "{{ item.config_type }}" + with_items: + - src: "searchlight-api-paste.ini.j2" + dest: "/etc/searchlight/api-paste.ini" + config_overrides: "{{ searchlight_api_paste_ini_overrides }}" + config_type: "ini" + - src: "searchlight.conf.j2" + dest: "/etc/searchlight/searchlight.conf" + config_overrides: "{{ searchlight_conf_overrides }}" + config_type: "ini" + - src: "policy.json.j2" + dest: "/etc/searchlight/policy.json" + config_overrides: "{{ searchlight_policy_overrides }}" + config_type: "json" + notify: + - Restart Searchlight services + diff --git a/tasks/searchlight_pre_install.yml b/tasks/searchlight_pre_install.yml new file mode 100644 index 0000000..201eaa7 --- /dev/null +++ b/tasks/searchlight_pre_install.yml @@ -0,0 +1,65 @@ +--- +# Copyright 2015, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +- name: Create the system group + group: + name: "{{ searchlight_system_group_name }}" + state: "present" + system: "yes" + +- name: Create the searchlight system user + user: + name: "{{ searchlight_system_user_name }}" + group: "{{ searchlight_system_group_name }}" + comment: "{{ searchlight_system_comment }}" + shell: "{{ searchlight_system_shell }}" + system: "yes" + createhome: "yes" + home: "{{ searchlight_system_user_home }}" + +- name: Create searchlight dirs + file: + path: "{{ item.path }}" + state: directory + owner: "{{ item.owner|default(searchlight_system_user_name) }}" + group: "{{ item.group|default(searchlight_system_group_name) }}" + mode: "{{ item.mode|default('0755') }}" + with_items: + - { path: "/openstack", owner: "root", group: "root" } + - { path: "/etc/searchlight" } + - { path: "{{ searchlight_system_user_home }}" } + - { path: "/var/lib/searchlight/tmp" } + - { path: "/var/lib/searchlight/locks" } + - { path: "/var/cache/searchlight", mode: "0700" } + +- name: Test for log directory or link + shell: | + if [ -h "/var/log/searchlight" ]; then + chown -h {{ searchlight_system_user_name }}:{{ searchlight_system_group_name }} "/var/log/searchlight" + chown -R {{ searchlight_system_user_name }}:{{ searchlight_system_group_name }} "$(readlink /var/log/searchlight)" + else + exit 1 + fi + register: log_dir + failed_when: false + changed_when: log_dir.rc != 0 + +- name: Create searchlight log dir + file: + state: directory + owner: "{{ searchlight_system_user_name }}" + group: "{{ searchlight_system_group_name }}" + mode: "0755" + path: "/var/log/searchlight" + when: log_dir.rc != 0 diff --git a/tasks/searchlight_service_setup.yml b/tasks/searchlight_service_setup.yml new file mode 100644 index 0000000..75f99d8 --- /dev/null +++ b/tasks/searchlight_service_setup.yml @@ -0,0 +1,107 @@ +--- +# Copyright 2015, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Create a service +- name: Ensure Searchlight service + keystone: + command: "ensure_service" + endpoint: "{{ keystone_service_adminurl }}" + login_user: "{{ keystone_admin_user_name }}" + login_password: "{{ keystone_auth_admin_password }}" + login_project_name: "{{ keystone_admin_tenant_name }}" + service_name: "{{ searchlight_service_name }}" + service_type: "{{ searchlight_service_type }}" + description: "{{ searchlight_service_description }}" + insecure: "{{ keystone_service_adminuri_insecure }}" + register: add_service + until: add_service|success + retries: 5 + delay: 2 + +# Create the project if needed, assumed to be in default domain. +# In many cases this will be present but under some circumstances the project +# may be unique to Searchlight, esp. when Swift is used for storage. +- name: Ensure Searchlight project + keystone: + command: ensure_project + project_name: "{{ searchlight_service_project_name }}" + endpoint: "{{ keystone_service_adminurl }}" + login_user: "{{ keystone_admin_user_name }}" + login_password: "{{ keystone_auth_admin_password }}" + login_project_name: "{{ keystone_admin_tenant_name }}" + description: "{{ searchlight_service_project_description }}" + insecure: "{{ keystone_service_adminuri_insecure }}" + register: add_project + until: add_project|success + retries: 5 + delay: 10 + +# Create an admin user +- name: Ensure Searchlight user + keystone: + command: "ensure_user" + endpoint: "{{ keystone_service_adminurl }}" + login_user: "{{ keystone_admin_user_name }}" + login_password: "{{ keystone_auth_admin_password }}" + login_project_name: "{{ keystone_admin_tenant_name }}" + user_name: "{{ searchlight_service_user_name }}" + tenant_name: "{{ searchlight_service_project_name }}" + password: "{{ searchlight_service_password }}" + insecure: "{{ keystone_service_adminuri_insecure }}" + register: add_user + until: add_user|success + retries: 5 + delay: 10 + +# Add a role to the user +- name: Ensure Searchlight user maps to admin role + keystone: + command: "ensure_user_role" + endpoint: "{{ keystone_service_adminurl }}" + login_user: "{{ keystone_admin_user_name }}" + login_password: "{{ keystone_auth_admin_password }}" + login_project_name: "{{ keystone_admin_tenant_name }}" + user_name: "{{ searchlight_service_user_name }}" + tenant_name: "{{ searchlight_service_project_name }}" + role_name: "{{ searchlight_role_name }}" + insecure: "{{ keystone_service_adminuri_insecure }}" + register: add_admin_role + until: add_admin_role|success + retries: 5 + delay: 10 + +# Create an endpoint +- name: Ensure Searchlight endpoint + keystone: + command: "ensure_endpoint" + endpoint: "{{ keystone_service_adminurl }}" + login_user: "{{ keystone_admin_user_name }}" + login_password: "{{ keystone_auth_admin_password }}" + login_project_name: "{{ keystone_admin_tenant_name }}" + region_name: "{{ searchlight_service_region }}" + service_name: "{{ searchlight_service_name }}" + service_type: "{{ searchlight_service_type }}" + insecure: "{{ keystone_service_adminuri_insecure }}" + endpoint_list: + - url: "{{ searchlight_service_publicurl }}" + interface: "public" + - url: "{{ searchlight_service_internalurl }}" + interface: "internal" + - url: "{{ searchlight_service_adminurl }}" + interface: "admin" + register: add_endpoint + until: add_endpoint|success + retries: 5 + delay: 10 diff --git a/templates/policy.json.j2 b/templates/policy.json.j2 new file mode 100644 index 0000000..a07563c --- /dev/null +++ b/templates/policy.json.j2 @@ -0,0 +1,26 @@ +{ + "context_is_admin": "role:admin and is_admin_project:True", + "admin_or_owner": "rule:context_is_admin or project_id:%(project_id)s", + "default": "", + + "search:query": "rule:admin_or_owner", + "search:query:aggregations": "rule:admin_or_owner", + "search:plugins_info": "rule:admin_or_owner", + "search:facets": "rule:admin_or_owner", + + "resource:OS::Glance::Image": "rule:admin_or_owner", + "resource:OS::Glance::Metadef": "rule:admin_or_owner", + "resource:OS::Nova::Server": "rule:admin_or_owner", + "resource:OS::Nova::Hypervisor": "rule:context_is_admin", + "resource:OS::Nova::ServerGroup": "rule:admin_or_owner", + "resource:OS::Nova::Flavor": "rule:admin_or_owner", + "resource:OS::Cinder::Volume": "rule:admin_or_owner", + "resource:OS::Cinder::Snapshot": "rule:admin_or_owner", + "resource:OS::Designate::Zone": "rule:admin_or_owner", + "resource:OS::Designate::RecordSet": "rule:admin_or_owner", + "resource:OS::Neutron::Net": "rule:admin_or_owner", + "resource:OS::Neutron::Port": "rule:admin_or_owner", + "resource:OS::Neutron::Subnet": "rule:admin_or_owner", + "resource:OS::Neutron::Router": "rule:admin_or_owner", + "resource:OS::Neutron::SecurityGroup": "rule:admin_or_owner" +} diff --git a/templates/searchlight-api-paste.ini.j2 b/templates/searchlight-api-paste.ini.j2 new file mode 100644 index 0000000..7f68373 --- /dev/null +++ b/templates/searchlight-api-paste.ini.j2 @@ -0,0 +1,37 @@ +# {{ ansible_managed }} +# Use this pipeline for no auth - DEFAULT +# TODO: Change to use keystone + +[pipeline:searchlight] +pipeline = cors versionnegotiation unauthenticated-context rootapp + +[pipeline:searchlight-keystone] +pipeline = cors versionnegotiation authtoken context rootapp + +[composite:rootapp] +paste.composite_factory = searchlight.api:root_app_factory +/: apiversions +/v1: apiv1app + +[app:apiversions] +paste.app_factory = searchlight.api.versions:create_resource + +[app:apiv1app] +paste.app_factory = searchlight.api.v1.router:API.factory + +[filter:versionnegotiation] +paste.filter_factory = searchlight.api.middleware.version_negotiation:VersionNegotiationFilter.factory + +[filter:unauthenticated-context] +paste.filter_factory = searchlight.api.middleware.context:UnauthenticatedContextMiddleware.factory + +[filter:authtoken] +paste.filter_factory = keystonemiddleware.auth_token:filter_factory +delay_auth_decision = true + +[filter:context] +paste.filter_factory = searchlight.api.middleware.context:ContextMiddleware.factory + +[filter:cors] +paste.filter_factory = oslo_middleware.cors:filter_factory +oslo_config_project = searchlight diff --git a/templates/searchlight-ports.conf.j2 b/templates/searchlight-ports.conf.j2 new file mode 100644 index 0000000..a0e4e43 --- /dev/null +++ b/templates/searchlight-ports.conf.j2 @@ -0,0 +1,3 @@ +# {{ ansible_managed }} + +Listen {{ searchlight_service_port }} diff --git a/templates/searchlight-systemd-init.j2 b/templates/searchlight-systemd-init.j2 new file mode 100644 index 0000000..ea8f646 --- /dev/null +++ b/templates/searchlight-systemd-init.j2 @@ -0,0 +1,25 @@ +# {{ ansible_managed }} + +[Unit] +Description=searchlight openstack service +After=syslog.target +After=network.target + +[Service] +Type=simple +User={{ system_user }} +Group={{ system_group }} + +{% if program_override is defined %} +ExecStart={{ program_override }} {{ program_config_options|default('') }} --log-file=/var/log/searchlight/{{ item.program_name }}.log +{% else %} +ExecStart={{ searchlight_bin }}/{{ item.program_name }} {{ program_config_options|default('') }} --log-file=/var/log/searchlight/{{ item.program_name }}.log +{% endif %} + +# Give a reasonable amount of time for the server to start up/shut down +TimeoutSec=300 +Restart=on-failure +RestartSec=150 + +[Install] +WantedBy=multi-user.target diff --git a/templates/searchlight-systemd-tempfiles.j2 b/templates/searchlight-systemd-tempfiles.j2 new file mode 100644 index 0000000..223f41f --- /dev/null +++ b/templates/searchlight-systemd-tempfiles.j2 @@ -0,0 +1,4 @@ +# {{ ansible_managed }} + +D /var/lock/{{ item.program_name }} 2755 {{ system_user }} {{ system_group }} +D /var/run/{{ item.program_name }} 2755 {{ system_user }} {{ system_group }} diff --git a/templates/searchlight.conf.j2 b/templates/searchlight.conf.j2 new file mode 100755 index 0000000..9a9184d --- /dev/null +++ b/templates/searchlight.conf.j2 @@ -0,0 +1,128 @@ +# {{ ansible_managed }} + +[DEFAULT] +# Disable stderr logging +use_stderr = False +debug = {{ debug }} +fatal_deprecations = {{ searchlight_fatal_deprecations }} +# searchlight_rabbitmq_userid: searchlight +# searchlight_rabbitmq_vhost: /searchlight +# searchlight_rabbitmq_port: {{ searchlight_rabbitmq_port }} +# searchlight_rabbitmq_server: {{ searchlight_rabbitmq_servers }} + +# This seems to be ignored? +searchlight_rabbitmq_use_ssl: {{ searchlight_rabbitmq_use_ssl }} + +transport_url = rabbit://{% for host in searchlight_rabbitmq_servers.split(',') %}{{ searchlight_rabbitmq_userid }}:{{ searchlight_rabbitmq_password }}@{{ host }}:{{ searchlight_rabbitmq_port }}{% if not loop.last %},{% else %}/{{ searchlight_rabbitmq_vhost }}{% endif %}{% endfor %} + +{% if searchlight_ssl_external | bool %} +[oslo_middleware] +secure_proxy_ssl_header = {{ searchlight_secure_proxy_ssl_header }} +{% endif %} + +# Messaging +[oslo_messaging_rabbit] +rabbit_use_ssl = {{ searchlight_rabbitmq_use_ssl }} +rpc_conn_pool_size = 30 + +[indexer] + +[keystone_authtoken] +insecure = {{ keystone_service_adminuri_insecure | bool }} +auth_plugin = {{ searchlight_keystone_auth_plugin }} +signing_dir = /var/cache/searchlight +auth_url = {{ keystone_service_adminuri }} +auth_uri = {{ keystone_service_internaluri }} +region_name = {{ searchlight_service_region }} +project_domain_id = {{ searchlight_service_project_domain_id }} +user_domain_id = {{ searchlight_service_user_domain_id }} +project_name = {{ searchlight_service_project_name }} +username = {{ searchlight_service_user_name }} +password = {{ searchlight_service_password }} + +memcached_servers = {{ memcached_servers }} +token_cache_time = 300 +revocation_cache_time = 60 + +# if your memcached server is shared, use these settings to avoid cache poisoning +memcache_security_strategy = ENCRYPT +memcache_secret_key = {{ memcached_encryption_key }} + +# if your keystone deployment uses PKI, and you value security over performance: +check_revocations_for_cached = False + +[service_credentials] +# These are needed to make API calls to other services when indexing +auth_plugin = password +# TODO (neillc) Ensure this user has admin privileges on nova, cinder etc +username = {{ searchlight_service_user_name }} +password = {{ searchlight_service_password }} +tenant_name = {{ searchlight_service_project_name }} +auth_url = {{ keystone_service_adminuri }} + +# If resource_plugin.include_region_name is set, this value will be +# the default value for the 'region_name' field on all documents +os_region_name = {{ searchlight_default_os_region_name }} + +[listener] +notifications_pool = searchlight + +[resource_plugin] +resource_group_name = searchlight +include_region_name = {{ searchlight_include_region_name }} + +[service_credentials:nova] +compute_api_version = {{ searchlight_nova_compute_api_version }} + +[resource_plugin:os_nova_server] +enabled = {{ searchlight_resource_plugin_os_nova_server_enabled }} +admin_only_fields = {{ searchlight_resource_plugin_os_nova_server_admin_only_fields }} + +[resource_plugin:os_nova_hypervisor] +enabled = {{ searchlight_resource_plugin_os_nova_hypervisor }} + +[resource_plugin:os_nova_flavor] +enabled = {{ searchlight_resource_plugin_os_nova_flavor_enabled }} + +[resource_plugin:os_nova_servergroup] +enabled = {{ searchlight_resource_plugin_os_nova_servergroup_enabled }} + +[resource_plugin:os_glance_image] +enabled = {{ searchlight_resource_plugin_os_glance_image_enabled }} + +[resource_plugin:os_glance_metadef] +enabled = {{ searchlight_resource_plugin_os_glance_metadef_enabled }} + +[resource_plugin:os_cinder_volume] +enabled = {{ searchlight_resource_plugin_os_cinder_volume_enabled }} + +[resource_plugin:os_cinder_snapshot] +enabled = {{ searchlight_resource_plugin_os_cinder_snapshot_enabled }} + +[resource_plugin:os_neutron_net] +enabled = {{ searchlight_resource_plugin_os_neutron_net_enabled }} +admin_only_fields={{ searchlight_resource_plugin_os_neutron_net_admin_only_fields }} + +[resource_plugin:os_neutron_port] +enabled = {{ searchlight_resource_plugin_os_neutron_port_enabled }} + +[resource_plugin:os_neutron_security_group] +enabled = {{ searchlight_resource_plugin_os_neutron_security_group_enabled }} + +[resource_plugin:os_designate_zone] +enabled = {{ searchlight_resource_plugin_os_designate_zone_enabled }} + +[resource_plugin:os_designate_recordset] +enabled = {{ searchlight_resource_plugin_os_designate_recordset_enabled }} + +[resource_plugin:os_swift_account] +enabled = {{ searchlight_resource_plugin_os_swift_account_enabled }} + +[resource_plugin:os_swift_container] +enabled = {{ searchlight_resource_plugin_os_swift_container_enabled }} + +[resource_plugin:os_swift_object] +enabled = {{ searchlight_resource_plugin_os_swift_object_enabled }} + +[paste_deploy] +flavor = keystone diff --git a/test-requirements.txt b/test-requirements.txt new file mode 100644 index 0000000..b85bf7b --- /dev/null +++ b/test-requirements.txt @@ -0,0 +1,17 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +bashate>=0.2 # Apache-2.0 +flake8<2.6.0,>=2.5.4 # MIT +pyasn1 # BSD +pyOpenSSL>=0.14 # Apache-2.0 +requests!=2.12.2,!=2.13.0,>=2.10.0 # Apache-2.0 +ndg-httpsclient>=0.4.2;python_version<'3.0' # BSD + +# this is required for the docs build jobs +sphinx!=1.6.1,>=1.5.1 # BSD +oslosphinx>=4.7.0 # Apache-2.0 +openstackdocstheme>=1.5.0 # Apache-2.0 +doc8 # Apache-2.0 +reno!=2.3.1,>=1.8.0 # Apache-2.0 +sphinxmark>=0.1.14 # Apache-2.0 diff --git a/tests/ansible-role-requirements.yml b/tests/ansible-role-requirements.yml new file mode 100755 index 0000000..de6a5f9 --- /dev/null +++ b/tests/ansible-role-requirements.yml @@ -0,0 +1,49 @@ +- name: apt_package_pinning + src: https://git.openstack.org/openstack/openstack-ansible-apt_package_pinning + scm: git + version: master +- name: pip_install + src: https://git.openstack.org/openstack/openstack-ansible-pip_install + scm: git + version: master +- name: openstack_hosts + src: https://git.openstack.org/openstack/openstack-ansible-openstack_hosts + scm: git + version: master +- name: lxc_hosts + src: https://git.openstack.org/openstack/openstack-ansible-lxc_hosts + scm: git + version: master +- name: lxc_container_create + src: https://git.openstack.org/openstack/openstack-ansible-lxc_container_create + scm: git + version: master +- name: memcached_server + src: https://git.openstack.org/openstack/openstack-ansible-memcached_server + scm: git + version: master +- name: rabbitmq_server + src: https://git.openstack.org/openstack/openstack-ansible-rabbitmq_server + scm: git + version: master +- name: openstack_openrc + src: https://git.openstack.org/openstack/openstack-ansible-openstack_openrc + scm: git + version: master +- name: os_keystone + src: https://git.openstack.org/openstack/openstack-ansible-os_keystone + scm: git + version: master +- name: elasticsearch + src: https://github.com/Stouts/Stouts.elasticsearch.git + scm: git + version: master +# Required for tests, but not needed by this role +- name: galera_client + src: https://git.openstack.org/openstack/openstack-ansible-galera_client + scm: git + version: master +- name: galera_server + src: https://git.openstack.org/openstack/openstack-ansible-galera_server + scm: git + version: master diff --git a/tests/group_vars/all_containers.yml b/tests/group_vars/all_containers.yml new file mode 100644 index 0000000..f37df47 --- /dev/null +++ b/tests/group_vars/all_containers.yml @@ -0,0 +1,26 @@ +--- +# Copyright 2016, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +container_name: "{{ inventory_hostname }}" +container_networks: + management_address: + address: "{{ ansible_host }}" + bridge: "br-mgmt" + interface: "eth1" + netmask: "255.255.255.0" + type: "veth" +physical_host: localhost +properties: + service_name: "{{ inventory_hostname }}" diff --git a/tests/host_vars/infra1.yml b/tests/host_vars/infra1.yml new file mode 100644 index 0000000..37954c2 --- /dev/null +++ b/tests/host_vars/infra1.yml @@ -0,0 +1,18 @@ +--- +# Copyright 2017, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +ansible_host: 10.1.0.2 +ansible_become: True +ansible_user: root diff --git a/tests/host_vars/localhost.yml b/tests/host_vars/localhost.yml new file mode 100644 index 0000000..cfb18fc --- /dev/null +++ b/tests/host_vars/localhost.yml @@ -0,0 +1,19 @@ +--- +# Copyright 2016, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +bridges: + - name: "br-mgmt" + +ansible_python_interpreter: "/usr/bin/python2" diff --git a/tests/host_vars/openstack1.yml b/tests/host_vars/openstack1.yml new file mode 100644 index 0000000..10225ab --- /dev/null +++ b/tests/host_vars/openstack1.yml @@ -0,0 +1,18 @@ +--- +# Copyright 2017, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +ansible_host: 10.1.0.3 +ansible_become: True +ansible_user: root diff --git a/tests/inventory b/tests/inventory new file mode 100644 index 0000000..7482621 --- /dev/null +++ b/tests/inventory @@ -0,0 +1,28 @@ +[all] +localhost +infra1 +openstack1 + +[all_containers] +infra1 +openstack1 + +[rabbitmq_all] +infra1 + +[galera_all] +infra1 + +[memcached_all] +infra1 + +[service_all:children] +rabbitmq_all +galera_all +memcached_all + +[keystone_all] +infra1 + +[searchlight_all] +openstack1 diff --git a/tests/os_searchlight-overrides.yml b/tests/os_searchlight-overrides.yml new file mode 100644 index 0000000..67f3e0e --- /dev/null +++ b/tests/os_searchlight-overrides.yml @@ -0,0 +1,29 @@ +--- +searchlight_developer_mode: true +searchlight_container_mysql_password: "secrete" +searchlight_service_password: "secrete" +searchlight_service_publicurl: "http://{{ hostvars[groups['searchlight_all'][0]]['ansible_host'] }}:9393" +searchlight_service_internalurl: "http://{{ hostvars[groups['searchlight_all'][0]]['ansible_host'] }}:9393" +searchlight_service_adminurl: "http://{{ hostvars[groups['searchlight_all'][0]]['ansible_host'] }}:9393" +searchlight_rabbitmq_userid: searchlight +searchlight_rabbitmq_vhost: /searchlight +searchlight_rabbitmq_password: "{{ rabbitmq_password }}" +searchlight_rabbitmq_use_ssl: "{{ rabbitmq_use_ssl }}" +searchlight_rabbitmq_host_group: "{{ rabbitmq_host_group }}" +searchlight_rabbitmq_userid: "{{ searchlight_service_user_name }}" +searchlight_rabbitmq_servers: "{{ rabbitmq_servers }}" +searchlight_rabbitmq_port: "{{ rabbitmq_port }}" + +searchlight_service_port: 9393 +searchlight_service_proto: http +searchlight_service_internaluri_proto: "{{ openstack_service_internaluri_proto | default(searchlight_service_proto) }}" +searchlight_service_internalurl: "{{ searchlight_service_internaluri_proto }}://{{ internal_lb_vip_address }}:{{ searchlight_service_port }}" + +searchlight_ssl_external: False +searchlight_service_region: "RegionOne" + +# Ensure that the package state matches the global setting +searchlight_package_state: "latest" + +# Ensure that keystone authentication is enabled for gnocchi +searchlight_keystone_auth: "{{ (groups['keystone_all'] is defined) and (groups['keystone_all'] | length > 0) }}" diff --git a/tests/test-install-searchlight.yml b/tests/test-install-searchlight.yml new file mode 100644 index 0000000..b1cfbde --- /dev/null +++ b/tests/test-install-searchlight.yml @@ -0,0 +1,31 @@ +--- +# Copyright 2017, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +- name: Deploy searchlight + hosts: searchlight_all + user: root + gather_facts: true + pre_tasks: + - include: common/ensure-rabbitmq.yml + vhost_name: "{{ searchlight_rabbitmq_vhost }}" + user_name: "{{ searchlight_rabbitmq_userid }}" + user_password: "{{ searchlight_rabbitmq_password }}" + when: "{{ groups['rabbitmq_all'] is defined }}" + vars: + elasticsearch_apt_java_package: openjdk-8-jre + roles: + - role: "os_searchlight" + vars_files: + - common/test-vars.yml diff --git a/tests/test-searchlight-functional.yml b/tests/test-searchlight-functional.yml new file mode 100755 index 0000000..01f4b15 --- /dev/null +++ b/tests/test-searchlight-functional.yml @@ -0,0 +1,36 @@ +--- +# Copyright 2015, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# TODO: (neillc) These tests are copied from the gnocchi role. replace with tests for searchlight + +- name: Playbook for functional testing of searchlight + hosts: searchlight_all + user: root + gather_facts: false + tasks: + - name: Install curl + package: + name: "curl" + state: "{{ searchlight_package_state }}" + register: install_packages + until: install_packages | success + retries: 5 + delay: 2 + - name: Check the searchlight-api + uri: + url: "http://localhost:{{ searchlight_service_port }}" + status_code: 200,300 + vars_files: + - common/test-vars.yml diff --git a/tests/test.yml b/tests/test.yml new file mode 100644 index 0000000..af10ecd --- /dev/null +++ b/tests/test.yml @@ -0,0 +1,29 @@ +--- +# Copyright 2017, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Setup the host +- include: common/test-setup-host.yml + +# Install RabbitMQ/MariaDB +- include: common/test-install-infra.yml + +# Install Keystone +- include: common/test-install-keystone.yml + +# Install Searchlight +- include: test-install-searchlight.yml + +# Test Searchlight +- include: test-searchlight-functional.yml diff --git a/tox.ini b/tox.ini new file mode 100644 index 0000000..d41dfb4 --- /dev/null +++ b/tox.ini @@ -0,0 +1,120 @@ +[tox] +minversion = 2.0 +skipsdist = True +envlist = docs,linters,functional + + +[testenv] +usedevelop = True +install_command = + pip install -c{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt} {opts} {packages} +deps = + -r{toxinidir}/test-requirements.txt +commands = + /usr/bin/find . -type f -name "*.pyc" -delete +passenv = + HOME + http_proxy + HTTP_PROXY + https_proxy + HTTPS_PROXY + no_proxy + NO_PROXY +whitelist_externals = + bash +setenv = + PYTHONUNBUFFERED=1 + ROLE_NAME=os_searchlight + TEST_IDEMPOTENCE=false + VIRTUAL_ENV={envdir} + WORKING_DIR={toxinidir} + + +[testenv:docs] +commands= + bash -c "rm -rf doc/build" + doc8 doc + python setup.py build_sphinx + + +[doc8] +# Settings for doc8: +extensions = .rst + + +[testenv:releasenotes] +commands = + sphinx-build -a -E -W -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html + + +# environment used by the -infra templated docs job +[testenv:venv] +commands = + {posargs} + + +[testenv:tests_clone] +commands = + bash -c "if [ ! -d "{toxinidir}/tests/common" ]; then \ + git clone https://git.openstack.org/openstack/openstack-ansible-tests {toxinidir}/tests/common; \ + fi" + + +[testenv:pep8] +commands = + {[testenv:tests_clone]commands} + bash -c "{toxinidir}/tests/common/test-pep8.sh" + + +[flake8] +# Ignores the following rules due to how ansible modules work in general +# F403 'from ansible.module_utils.basic import *' used; +# unable to detect undefined names +ignore=F403 + + +[testenv:bashate] +commands = + {[testenv:tests_clone]commands} + bash -c "{toxinidir}/tests/common/test-bashate.sh" + + +[testenv:ansible] +deps = + {[testenv]deps} + -rhttps://git.openstack.org/cgit/openstack/openstack-ansible-tests/plain/test-ansible-deps.txt + + +[testenv:ansible-syntax] +deps = + {[testenv:ansible]deps} +commands = + {[testenv:tests_clone]commands} + bash -c "{toxinidir}/tests/common/test-ansible-syntax.sh" + + +[testenv:ansible-lint] +deps = + {[testenv:ansible]deps} +commands = + {[testenv:tests_clone]commands} + bash -c "{toxinidir}/tests/common/test-ansible-lint.sh" + + +[testenv:functional] +deps = + {[testenv:ansible]deps} +commands = + {[testenv:tests_clone]commands} + bash -c "{toxinidir}/tests/common/test-ansible-functional.sh" + + +[testenv:linters] +deps = + {[testenv:ansible]deps} +commands = + {[testenv:pep8]commands} + {[testenv:bashate]commands} + {[testenv:ansible-lint]commands} + {[testenv:ansible-syntax]commands} + {[testenv:docs]commands} diff --git a/vars/ubuntu-16.04.yml b/vars/ubuntu-16.04.yml new file mode 100755 index 0000000..4483075 --- /dev/null +++ b/vars/ubuntu-16.04.yml @@ -0,0 +1,25 @@ +--- +# Copyright 2016, Rackspace US, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +#: Necessary packages +# TODO (neillc) Confirm that all of these packages are needed. +searchlight_apt_packages: + - apache2 + - apache2-utils + - libapache2-mod-wsgi + - git + - build-essential + - python-dev + - libpq-dev \ No newline at end of file