Migrate log integration test to ZuulV3

- define .zuul.yaml to contain new job definitions
- move gate hooks to appropriate location

Story: 2001238
Task: 5763
Task: 5766

Depends-On: I263b1a92a9ec7b5e38d9c1dc8b02e214102a92c2
Change-Id: Icfb32df7e083c2c58c89fa3d2fe9a3eacfda6beb
This commit is contained in:
Tomasz Trębski 2017-10-04 07:39:04 +02:00
parent 1c6e5db971
commit 9c163622e2
13 changed files with 244 additions and 209 deletions

1
.gitignore vendored
View File

@ -13,6 +13,7 @@ cover/
.tox/
MANIFEST
AUTHORS
ChangeLog
monasca-log-api.log
etc/monasca/log-api.conf.sample

34
.zuul.yaml Normal file
View File

@ -0,0 +1,34 @@
- job:
name: monascalog-tempest-base
parent: legacy-dsvm-base
run: playbooks/legacy/monascalog-tempest-base/run
post-run: playbooks/legacy/monascalog-tempest-base/post
timeout: 7800
required-projects:
- openstack-infra/devstack-gate
- openstack/monasca-agent
- openstack/monasca-api
- openstack/monasca-common
- openstack/monasca-grafana-datasource
- openstack/monasca-notification
- openstack/monasca-persister
- openstack/monasca-statsd
- openstack/monasca-thresh
- openstack/monasca-ui
- openstack/python-monascaclient
- openstack/tempest
- openstack/monasca-log-api
- openstack/monasca-kibana-plugin
- job:
name: monascalog-tempest
parent: monascalog-tempest-base
- project:
name: openstack/monasca-log-api
check:
jobs:
- monascalog-tempest
gate:
jobs:
- monascalog-tempest

204
ChangeLog
View File

@ -1,204 +0,0 @@
CHANGES
=======
* Provide installation documentation
* Updated from global requirements
* Refactor project tree
* Minor language change
* Allow to specify CLI arguments
* Add CLI documentation part
* Configuration documentation
* Updated from global requirements
* Fix deprecation detail
* Update the documentation link for doc migration
* Tweaks in tox.ini to speed it up
2.2.1
-----
* Fix tarball building error missing log-api.conf
2.2.0
-----
* Provide references to new docs
* Use oslo-config-generator
* Updated from global requirements
* Enable codebase documenation
* Add Py35 to setup.cfg
* Migrate documentation to Sphinx
* Fix WSGI startup script
* Fix tempest test for empty\_body
* Updated from global requirements
* Ensure and verify utf-8 support
* Mock monascastatsd in tests
* Enable some off-by-default checks
* Updated from global requirements
* Updated from global requirements
* Fix html\_last\_updated\_fmt for Python3
* Fix wrong configuration file paths
* Fix cross-tenant logs submission
* Composite paste for monasca-log-api
2.1.0
-----
* Updated from global requirements
* Updated from global requirements
* Replace test.attr with decorators.attr
* Updated from global requirements
* Updated from global requirements
* Enable tempest tests with uwsgi keystone
2.0.0
-----
* Update Readme references
* Updated from global requirements
* Optimize the link address
* Replace six.iteritems() with .items()
* Remove monasca\_log\_api/.gitignore
* Updated from global requirements
* Internal interfaces for log listing API implementation
* Updated from global requirements
* Specification for log listing API
* Updated from global requirements
* Updated from global requirements
* Updated from global requirements
* Adding WSGI support
* Updated from global requirements
1.4.0
-----
* Switch to use test\_utils.call\_until\_true
* Use openstack/monasca-kibana-plugin repo in devstack
* Lower log level
* Fixing filename to start server
* Updated from global requirements
* Typo fix: adress => address
* Use bento/ubuntu-16.04 box
* Updated from global requirements
* Make monasca-log-api work with Python 3.5
* Migrate testing to ostestr and adding docs
* Updated from global requirements
* Fix the Monasca Log API tempest tests
* Integration with oslo.context
* Drop Java Implemention codebase
* Enable checking with bandit
* Add test for empty log message
* Fix tempest tests
* Upgrade monasca-common dependency version
1.3.0
-----
* Add bashate check to tox file
* Uprade ELK tarballs
* Adjust monasca-log-api devstack to Xenial
* Add Constraints support
* Show team and repo badges on README
* Replace LOG.warn with LOG.warning
* Fixing problems with tox
1.2.0
-----
* Changed home page link
* Updated from global requirements
* Updated from global requirements
* Updated from global requirements
* Change assertTrue(isinstance()) by optimal assert
* Remove xrange for run both Python 2 and Python 3
* Fix config-generator.tempest.conf path in README
* Updated from global requirements
* Fix typo: remove redundant 'the'
* Add prefix "$" for command examples
* TrivialFix: Removed redundant 'the'
1.1.0
-----
* TrivialFix: Remove config import unused
* Enable size-based log rotation for monasca-log-api
* Removing gunicorn from requirements
* TrivialFix: Remove logging import unused
* TrivialFix: Remove cfg import unused
* Update logstash-output-monasca\_log\_api from 0.5 to 0.5.2
* Add tempest tests for Log API v3
* Fix E126 test
* Manual sync with requirements
* Set up tempest tests for gate job
* Install Kibana plugin in 'extra' phase
* Optimize bulk processing
* Install Kibana plugin for Keystone authentication
* Monitoring for monasca-log-api
* Change Monasca Port
* Add monasca-log-metrics to devstack plugin
* Allow to reuse ELK tarballs
* Update monasca-log-agent in devstack plugin
1.0.0
-----
* Import simport from monasca-common
* Truncating too big message
* Remove unnecessary tests
* Remove kafka key building
* Add Vagrantfile for deploying DevStack in a VM
* Documentation fix
* Remove six useless requirement
* Https Support
* Kafka message max size
* Raise original exception
* Tempest tests modification
* Update specification
* Add devstack plugin installer
* Unification for v2 and v3 output
* Adds [testenv:venv] section to tox.ini
* Common code base for v2 and v3
0.0.2
-----
* Mark /v2.0/log/single as deprecated
* Add support for global dimensions
* Use same abstract class for v2 and v3
* Versions endpoint is working
* Moving unversioned endpoints
* Specification proposal for batch support
* Add validation of log property
* Fix redundant dependency
* Change default config file
* Fix run of java monasca-log-api with version option
* Move dimensions into log object
* Add support for Apache mod\_wsgi
* Fix exception thrown when dimensions missing
* Healthcheck endpoint docs
* Add implementation of /v3.0/logs API
* Adding healthcheck
* Sync to never monasca-common
* Updating requirements.txt
0.0.1
-----
* Security update for monasca-log-api
* Validating message size
* Migrating tempest to project
* Documentation update
* Syncing API version
* Cleaning python bytecode before tests run
* Adding generating coverage
* Remove not used config variable
* Fixing issues spotted in dev environment
* Fixing gunicorn dependency and README
* Update information for PyPI registration
* Convert creation\_time to string
* Read region from config and handle payload
* Change repositories from stackforge to openstack
* Update .gitreview for new namespace
* Adjust to specification
* monasca-log-api python
* Add log api functionality
* Added .gitreview

2
devstack/post_test_hook.sh Normal file → Executable file
View File

@ -88,7 +88,7 @@ if ! function_exists echo_summary; then
}
fi
echo_summary "monasca's post_test_hook.sh was called..."
echo_summary "monasca-log's post_test_hook.sh was called..."
(set -o posix; set)
# save ref to monasca-api dir

0
devstack/pre_test_hook.sh Normal file → Executable file
View File

View File

@ -31,7 +31,7 @@ KIBANA_VERSION=${KIBANA_VERSION:-4.6.3-linux-x86_64}
LOGSTASH_VERSION=${LOGSTASH_VERSION:-2.4.1}
ELASTICSEARCH_VERSION=${ELASTICSEARCH_VERSION:-2.4.2}
NODE_JS_VERSION=${NODE_JS_VERSION:-"4.0.0"}
NVM_VERSION=${NVM_VERSION:-"0.32.1"}
NVM_VERSION=${NVM_VERSION:-"0.33.5"}
# repository settings
MONASCA_LOG_API_REPO=${MONASCA_LOG_API_REPO:-${GIT_BASE}/openstack/monasca-log-api.git}

View File

@ -8,7 +8,7 @@ keys = console, file
keys = context
[logger_root]
level = DEBUG
level = INFO
handlers = console, file
[logger_kafka]

View File

@ -0,0 +1,111 @@
#
# (C) Copyright 2015 Hewlett Packard Enterprise Development Company LP
# (C) Copyright 2016-2017 FUJITSU LIMITED
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Sleep some time until all services are starting
sleep 6
function load_devstack_utilities {
source $BASE/new/devstack/stackrc
source $BASE/new/devstack/functions
source $BASE/new/devstack/openrc admin admin
# print OS_ variables
env | grep OS_
}
function setup_monasca_log {
local constraints="-c $REQUIREMENTS_DIR/upper-constraints.txt"
pushd $TEMPEST_DIR
sudo -EH pip install $constraints -r requirements.txt -r test-requirements.txt
popd;
pushd $MONASCA_LOG_API_DIR
sudo -EH pip install $constraints -r requirements.txt -r test-requirements.txt
sudo -EH python setup.py install
popd;
}
function set_tempest_conf {
local conf_file=$TEMPEST_DIR/etc/tempest.conf
pushd $TEMPEST_DIR
oslo-config-generator \
--config-file tempest/cmd/config-generator.tempest.conf \
--output-file $conf_file
popd
cp -f $DEST/tempest/etc/logging.conf.sample $DEST/tempest/etc/logging.conf
# set identity section
iniset $conf_file identity admin_domain_scope True
iniset $conf_file identity user_unique_last_password_count 2
iniset $conf_file identity user_locakout_duration 5
iniset $conf_file identity user_lockout_failure_attempts 2
iniset $conf_file identity uri $OS_AUTH_URL/v2.0
iniset $conf_file identity uri_v3 $OS_AUTH_URL/v3
iniset $conf_file identity auth_version v$OS_IDENTITY_API_VERSION
iniset $conf_file identity region $OS_REGION_NAME
# NOTE(trebskit) we're processing a lot here, increase http timeout
iniset $conf_file identity-feature-enabled http_timeout 120
# set auth section
iniset $conf_file auth use_dynamic_credentials True
iniset $conf_file auth admin_username $OS_USERNAME
iniset $conf_file auth admin_password $OS_PASSWORD
iniset $conf_file auth admin_domain_name $OS_PROJECT_DOMAIN_ID
iniset $conf_file auth admin_project_name $OS_PROJECT_NAME
iniset $conf_file auth tempest_roles monasca-user
# set monitoring section
iniset $conf_file monitoring kibana_version 4.6.3
}
function function_exists {
declare -f -F $1 > /dev/null
}
if ! function_exists echo_summary; then
function echo_summary {
echo $@
}
fi
echo_summary "monasca-log's post_test_hook.sh was called..."
(set -o posix; set)
# save ref to monasca-api dir
export MONASCA_LOG_API_DIR="$BASE/new/monasca-log-api"
export TEMPEST_DIR="$BASE/new/tempest"
sudo chown -R "${USER}":stack $MONASCA_LOG_API_DIR
sudo chown -R "${USER}":stack $TEMPEST_DIR
load_devstack_utilities
setup_monasca_log
set_tempest_conf
# Run functional tests
echo "Running monasca-log tempest test suite"
(cd $TEMPEST_DIR; testr init)
(cd $TEMPEST_DIR; testr list-tests monasca_log_api_tempest > monasca_log_api_tempest)
(cd $TEMPEST_DIR; cat monasca_log_api_tempest | grep gate > monasca_log_api_tempest_gate)
(cd $TEMPEST_DIR; testr run --subunit --load-list=$TEMPEST_DIR/monasca_log_api_tempest_gate | subunit-trace --fails)

View File

@ -0,0 +1,16 @@
- hosts: primary
tasks:
- name: Copy files from {{ ansible_user_dir }}/workspace/ on node
synchronize:
src: '{{ ansible_user_dir }}/workspace/'
dest: '{{ zuul.executor.log_root }}'
mode: pull
copy_links: true
verify_host: true
rsync_opts:
- --include=/logs/**
- --include=*/
- --exclude=*
- --prune-empty-dirs

View File

@ -0,0 +1,76 @@
- hosts: all
tasks:
- name: Ensure legacy workspace directory
file:
path: '{{ ansible_user_dir }}/workspace'
state: directory
- shell:
cmd: |
set -e
set -x
cat > clonemap.yaml << EOF
clonemap:
- name: openstack-infra/devstack-gate
dest: devstack-gate
EOF
/usr/zuul-env/bin/zuul-cloner -m clonemap.yaml --cache-dir /opt/git \
git://git.openstack.org \
openstack-infra/devstack-gate
executable: /bin/bash
chdir: '{{ ansible_user_dir }}/workspace'
environment: '{{ zuul | zuul_legacy_vars }}'
- shell:
cmd: |
set -e
set -x
cat << 'EOF' >>"/tmp/dg-local.conf"
[[local|localrc]]
enable_plugin monasca-api git://git.openstack.org/openstack/monasca-api
enable_plugin monasca-log-api git://git.openstack.org/openstack/monasca-log-api
EOF
executable: /bin/bash
chdir: '{{ ansible_user_dir }}/workspace'
environment: '{{ zuul | zuul_legacy_vars }}'
- shell:
cmd: |
set -e
set -x
export PYTHONUNBUFFERED=true
export ENABLED_SERVICES=tempest
export DEVSTACK_GATE_NEUTRON=1
export DEVSTACK_GATE_EXERCISES=0
export PROJECTS="openstack/monasca-api $PROJECTS"
export PROJECTS="openstack/monasca-persister $PROJECTS"
export PROJECTS="openstack/monasca-common $PROJECTS"
export PROJECTS="openstack/monasca-agent $PROJECTS"
export PROJECTS="openstack/monasca-notification $PROJECTS"
export PROJECTS="openstack/monasca-thresh $PROJECTS"
export PROJECTS="openstack/monasca-statsd $PROJECTS"
export PROJECTS="openstack/python-monascaclient $PROJECTS"
export PROJECTS="openstack/monasca-grafana-datasource $PROJECTS"
export PROJECTS="openstack/monasca-ui $PROJECTS"
export PROJECTS="openstack/monasca-log-api $PROJECTS"
export PROJECTS="openstack/monasca-kibana-plugin $PROJECTS"
function pre_test_hook {
source $BASE/new/monasca-log-api/monasca_log_api_tempest/contrib/pre_test_hook.sh
}
export -f pre_test_hook
function post_test_hook {
source $BASE/new/monasca-log-api/monasca_log_api_tempest/contrib/post_test_hook.sh
}
export -f post_test_hook
cp devstack-gate/devstack-vm-gate-wrap.sh ./safe-devstack-vm-gate-wrap.sh
./safe-devstack-vm-gate-wrap.sh
executable: /bin/bash
chdir: '{{ ansible_user_dir }}/workspace'
environment: '{{ zuul | zuul_legacy_vars }}'

View File

@ -0,0 +1,3 @@
---
upgrade:
- Upgrade integrations tests setup for Zuul V3.

View File

@ -73,8 +73,6 @@ commands =
[testenv:flake8]
description = Validates codebase with flake
skip_install = True
usedevelop = False
commands =
{[testenv]commands}
flake8 monasca_log_api