Commit Graph

40 Commits

Author SHA1 Message Date
Hasan Acar a24fd834d0 fix: migrate CI to jammy
fix: change tox.ini

fix: change queries for list_dimension_names and list_dimension_values because of influxdb time filter problem

fix: remove build_sphinx group from setup.cfg

fix: handle hashlib security problem

Change-Id: I0d31a8db5ed71c70e7b878ce5e7940e041d0fa43

Change-Id: I6f7066da10e834550cbf0c053c7bf425ac0ead93

Change-Id: If9575aee73d600bbc84fcdf58deb1c57b508d9c2

Change-Id: If515eaeee7539da3ca49997e88785dc65572b334
2024-02-22 14:35:03 +00:00
Martin Chacon Piza 43922f8223 [goal] Migrate testing to ubuntu focal
As per victoria cycle testing runtime and community goal[1]
we need to migrate upstream CI/CD to Ubuntu Focal(20.04).

-Bump the lower constraints for required deps which added python3.8 support
in their later version.

-Changing the way to install and configure Zookeeper.
Installing Zookeeper from official Apache's tarball.
Adding the possiblity to set the specific Zookeeper version.
Minor change in zookeeper logger.

-Use mariadb JDBC for monasca-thresh in devstack, since Drizzle isn't
compatible with MySql Server v8.0.x which is default in Focal

-Python 3.8 doesn't seem to like dictionary keys changing during
iteration.

Fixing RuntimeError: dictionary keys changed during iteration.

Tech. details:
It runs well in py27: 5 iterations
It runs risky in py37: 7 iterations
It is forbbiden in py38: raised RuntimeError
Fixed with list(dic.items()) or tuple(dic.items())

dic = {'1': 'a', '2': 'b', '3': 'c', '4': 'd', '5': 'e'}
for key, value in dic.items():
    print("Key: {0} Value: {1}".format(key,value))
    del dic[key]
    print(dic)
    dic[key] = value
print(dic)

Story: #2007865
Task: #40197
Depends-On: https://review.opendev.org/756859
Change-Id: Ieb4cf38038ffb4d1a152f8ab3b64a14098c7cbb3
2020-10-12 15:38:05 +02:00
Witek Bedyk 3014c840d6 Set legacy_kafka_client_enabled = False on default
We change the default value of kafka.legacy_kafka_client_enabled from
True to False. The use of new Confluent Kafka client is recommended.

DevStack plugin does not set this option anymore.

Depends-On: https://review.opendev.org/740959
Depends-On: https://review.opendev.org/740966
Change-Id: I4d57b8893a6a131769009dc3299789d3fc89bab6
Story: 2007924
Task: 40338
2020-07-14 16:15:23 +02:00
Zuul a88d34ac5c Merge "Remove redundant `install_nodejs` function" 2020-05-31 13:52:01 +00:00
Witek Bedyk d6104a693b [DevStack] Allow testing different Kibana versions
Kibana API has changed between the versions. To allow testing
different Kibana versions `kibana_version` option should be set in
tempest.conf.

This is a partial port of https://review.opendev.org/727195

Change-Id: I0a21bd99c942ab05ce1eae51226132691518bd91
2020-05-26 13:39:46 +02:00
Witek Bedyk 8d31013da5 Remove redundant `install_nodejs` function
The change removes the redundant `install_nodejs` function. After
merging the DevStack plugins function is available in plugin.sh already.

Change-Id: I5e4d6e5b05f9506ba87bdecb601e411b8496453c
2020-05-25 18:10:20 +02:00
arseni.lipinski@ts.fujitsu.com aeed1c63dc Add build and installation for monasca-kibana-plugin in new API
Story: 2006376
Task: 38335

Depends-On: https://review.opendev.org/#/c/680075/

Change-Id: I5887c75dbf499027b21b86c2850370f6729fb451
2020-04-27 10:43:39 +02:00
Witek Bedyk 9be8348896 Set DEBUG log level for Elastisearch
Change-Id: I3eb37af59f5095b1cc090409b5427be7fe8ebcd8
2020-01-28 09:16:00 +01:00
Zuul f482afc0d8 Merge "Upgrade Elkstack in new API" 2020-01-20 10:51:00 +00:00
arseni.lipinski ce0e62584f Upgrade Elkstack in new API
The commit contains upgrade of Elk components, default index pattern
creation in new API.

Story: 2006376
Task: 38125

Depends-On: https://review.opendev.org/#/c/679781

Change-Id: Ib0b966c0d7db993802b5372156c41b3ebdf1a77f
2020-01-15 12:32:34 +01:00
Witek Bedyk bd26facccf Update DevStack plugin Readme
* remove deprecated content
* add Apache Cassandra deployment option
* add monasca-tempest-plugin deployment description

Change-Id: Id28de21c89bbcde50a0eb947b833a98dde3cd691
2019-12-17 13:17:28 +01:00
Adrian Czarnecki 1e3e1173ad Update devstack plugin to support new api
Merge log-api and api

Story: 2003881
Task: 36195

Depends-On: https://review.opendev.org/681419/
Change-Id: Idb2361e5ab701cd3dee4eabb4b43116f6e1205c4
2019-12-04 13:52:57 +01:00
Witek Bedyk 3f31ba407b Configure monasca-ui settings in Horizon dir
The default and preferred way of modifying the settings of Horizon
plugin is to add a configuration file to
openstack_dashboard/local/local_settings.d/ directory. For that to work
we should use the settings from django.conf.settings and not from
monitoring.config.local_settings.

[1] https://docs.openstack.org/horizon/latest/configuration/settings.html

Story: 2006556
Task: 36645

Depends-On: https://review.opendev.org/682321
Change-Id: I718b51ce94b09bb547f03686831c3b1ac1721399
2019-09-30 14:40:56 +00:00
Zuul 42cd1b6c1b Merge "Disable legacy Kafka client in notification engine" 2019-09-26 14:12:42 +00:00
Witek Bedyk 7f6dc5f2c2 Disable legacy Kafka client in DevStack
Story: 2003705
Task: 35844

Depends-On: https://review.opendev.org/671451
Depends-On: https://review.opendev.org/644844
Change-Id: Ic7e984df9f7a0aa97108cddc55b95834cabe3938
2019-09-17 13:26:55 +00:00
Witek Bedyk 29fa4e5e6b Disable legacy Kafka client in notification engine
The change disables legacy Kafka consumer in producer in monasca
notification engine in DevStack plugin.

Story: 2003705
Task: 36094

Depends-On: https://review.opendev.org/674812
Change-Id: Ic5d3523caa7e16f1fbebc73839d19fe0a07f162e
2019-09-01 13:39:41 +02:00
Witek Bedyk dfd9423526 Upgrade Apache Kafka to version 2.0.1
Upgrade Apache Kafka to current stable version 2.0.1.

Issues have been observed with legacy Kafka producer being used in
notification engine together with newer Kafka broker versions.

Following changes are included:

* Deprecated configuration option `advertised.host.name` is replaced
with `listeners`.
* Default `log4j.properties` is used.

Story: 2005624
Task: 30933
Change-Id: I898b511b2ab8f68e4850faab2098044cd3f94ee7
2019-08-14 14:51:44 +02:00
Thomas Bechtold 2bf341c6dd Use monasca-notification.conf during devstack runs
Since [1], monasca-notification supports the standard oslo.config
based configuration style (ini file instead of yaml). The old
notification.yaml file still works but is deprecated.
So switching to use the new oslo.config based file is the way to go
forward.
Also adjust the zuul job to copy /var/log/monasca/notification and
/etc/monasca/notification.conf so debugging will be a bit easier.

[1] https://review.openstack.org/#/c/464768/

Story: 2004862
Task: 35646

Depends-On: https://review.opendev.org/668508
Depends-On: https://review.opendev.org/663960
Change-Id: I489d66d623c3e27ca28ba87ebde32264b13ac0f3
2019-07-02 12:21:52 +02:00
Zuul 4681262d82 Merge "Set up DB schema in DevStack plugin using Alembic" 2019-02-21 12:27:59 +00:00
Witold Bedyk 6409d06439 Set up DB schema in DevStack plugin using Alembic
SQL schema scripts are replaced with Alembic command line tool.
Notification plugins are enabled in notification engine configuration
file.

The change also fixes the `sub_alarm` table schema. The foreign key must
refer to the field with the same data type, charset and collation.
Otherwise the schema migration fails occasionally with Python 3 with the
IntegrityError[1]:

  oslo_db.exception.DBError: (pymysql.err.IntegrityError) (1215, 'Cannot add foreign key constraint')

[1] http://logs.openstack.org/61/622361/8/gate/monasca-tempest-python3-influxdb/26104a3/controller/logs/devstacklog.txt.gz#_2019-01-30_15_52_45_455

Change-Id: I95d7e91e62ff13e84f963fdbb3bca6b7dfd7c310
Story: 2001654
Task: 14421
2019-02-20 14:53:10 +01:00
Witek Bedyk 036b9cb519 Use new default persister configuration file
The default configuration file name for monasca-persister has been
changed to monasca-persister.conf. This change updates DevStack plugin
to use the default.

Change-Id: I95039356c768173fd902e7e253e1fad77c8a0b87
Story: 2004867
Task: 29112
2019-02-18 18:17:05 +01:00
Adrian Czarnecki ed71d7681d Switch tempest test to zuul3 native
Change temepst-java-influxdb test to no voting.
Java-persister fully support only casanda db.

Story: 2004175
Task: 27657
Change-Id: Iedb3844fa455fd186d6c36f0bbb69a3aa313984a
2018-11-20 07:45:28 +00:00
Witold Bedyk a23fd14716 Set up logging for monasca-persister in devstack
Instead of `log_config_append` option use devstack's `setup_logging`
function to set up logging in the way consistent with other OpenStack
components. This should allow correct indexing of monasca-persister logs
with project-infra grok patterns.

Also, set default log level for kafka_lib.client module to INFO to
reduce log message size and avoid long log messages.

Story: 2003911
Task: 26839

Change-Id: I19c13427b52f906b2e9e028cf6272a5e7261238e
2018-10-05 11:59:14 +02:00
Zuul 01740e584b Merge "Restart persister on failure in devstack plugin" 2018-09-21 21:25:31 +00:00
Adrian Czarnecki 3a5b8775e2 Enable python3 in devstack
Change-Id: I44f09c9068ecd682d7c91945a5e32842ab335b80
2018-08-24 12:59:16 +00:00
akhiljain23 c11fbd6c70 Add keystone parameters to authenticate webhook notifications
This commit will update devstack plugin to add required parameters to
notification.yaml for keystone authentication.

Change-Id: I61b3b437a06f5e62b41c567d361e1d73a4e55d0e
Partially-Implements: blueprint add-monasca-push-driver
Story: 2003105
Task: 23221
2018-08-07 12:28:37 +00:00
Witold Bedyk 6a2bae2db2 Restart persister on failure in devstack plugin
Change-Id: Iff771e6ff824da8bfe46c361cc1286675bb01686
2018-07-27 11:45:04 +02:00
Tomasz Trębski 4a652c654d Enable ui & logviewer locally
Commit enables storm-{logviewer,ui} for the purposes
of the local development but keeps them disabled
in the gate runs.

Depends-On: Iccc30d411a49b9e47d6625909427f071397aedfd
Change-Id: I8cc2d6c616eff9ab058b24f837c655228b84f184
2018-05-11 14:27:04 +00:00
bandorf ce8cdce46f devstack: download storm from archive.apache.org
Change-Id: I14102e3652e5f2214cf3115cfad378a01a827ad3
Story: 2001559
Task: 6503
2018-02-20 09:32:12 +00:00
Adrian Czarnecki 3c1b716e92 Remove installation of mysql-python
This library is no longer used in code.

Change-Id: I4278b09414c0213c571abcb4d64f9eb50a5b9851
2018-01-02 12:38:05 +00:00
James Gu ba43f07726 Add cassandra support
Support Cassandra db installation and Cassandra related
configurations for Monasca api and persister services
in devstack. Add Monasca rest API Cassandra plugin for
retrieving metric, dimension, measurement, statistics
and alarms.

Change-Id: Ie60d668692e1f25f555dda2355f4e513d582736c
story: 2001231
task: 5759
2017-12-15 19:11:00 +00:00
Tomasz Trębski 58d927d083 Better handling of storm processes
Use devstack utilities to launch storm processes
and be able to see their logs.

Change-Id: Iccc30d411a49b9e47d6625909427f071397aedfd
2017-10-06 08:19:11 +02:00
Georgia-Anna Farmaki 62cf4e26e2 Added a field grafana_url in the installation script
Included a field 'grafana_url' in notification.yaml
for email notifications and replaced it with the
respective grafana host in plugin.sh.

Change-Id: I581b2e9d12632cc7509d24525cc225b60514a8c3
Story: 2001052
Task: 4653
2017-09-01 12:26:24 +00:00
Adrian Czarnecki de5b35dde8 Update devstack to use new config file for persister.
Depends-On: I490de4110843f7dfd618a6a8cc8d6a4e35db65ce
Change-Id: If5464f837fa92cb021bc73251de566e5184be018
2017-07-28 13:26:11 +02:00
Tomasz Trębski c767948175 Use devstack utilities for monasca-persister
Adjust monasca-persister installation with changes:
- start/stop of monasca-persister process handles through run_process
  and stop_process and no service files
- setting up environment with devstack function
- use USE_VENV for monasca-persister
- installing monasca-persister dependencies from git
- remove service file from devstack/files/monasca-persister
- reorganized also java part

Change-Id: I08195ad281637e6567a8acbace9862df2d5f9502
2017-07-19 06:40:54 +02:00
Tomasz Trębski 18f1dad2d8 Integrate fully with new python-monascaclient
* shell commands completition
* usage of openstack openrc utility to source
environmental variables to use client

Extra:

* removed cassandra env and replaced it with bash code
* reworked creating bash profile
* removed devstack/files/env.sh as it is no longer needed and
everything is available through profile

Depends-On: I579f6bcd5975a32af2a255be41c9b6c4043fa1dc
Depends-On: Ifee5b88ccb632222310aafb1081ecb9c9d085150
Depends-On: I01d0f3d9814376f9358feb23494e1ca5231aedac
Depends-On: I1712a24739438e2d8331a495f18f357749a633c5
Change-Id: I35f5943ae1eadb10dc416179c575cdad62753b69
2017-07-14 12:55:09 +02:00
Tomasz Trębski f9b41eefbe Use devstack utilities for monasca-notification
Adjust monasca-notification installation with changes:
- start/stop of monasca-notification proces handles through run_process
  and stop_process and no service files
- setting up environment with devstack function
- use USE_VENV for monasca-notification
- installing monasca-notification dependencies from git
- remove service file from devstack/files/monasca-notification

Also added code from Idbe5bdf56fb280412c34f04de8f2e2f301a7a432

Depends-On: Idbe5bdf56fb280412c34f04de8f2e2f301a7a432
Change-Id: Ic581fbdd2db57c9a2f13f1646eb9485c5dd0f62b
2017-07-12 13:11:27 +02:00
Tomasz Trębski 175577d35b Validate all bash files inside devstack
Previously only plugin.sh was validated.
At the monent some parts of the plugin
were moved under devstack/lib to increase plugin.sh
readability. Those files were not validated.

Note:
    Tweaked a tox.ini by removing installing the monasca-api
    package in environments that do not actually need it.
    Also removed redundant references to main environment
    dependencies.

Change-Id: I43381647228fad99603b5717c37496e80f409d4f
2017-07-12 06:28:07 +00:00
Tomasz Trębski b37d34f872 Add policies to monasca-ui
Adds copying policies of monasca dashboard into horizon
policies directory.

Extra:
* reworked installation process (now using setup_develop)
* installing python-monascaclient using setup_lib and cloning it
directly from referenced MONASCA_CLIENT_BRANCH

Depends-On: I273f06332fa11a81ea8de2c13059dce9d160e90d
Change-Id: I2cf1ae62e8b80aefe14d093a499f7103a5c288a5
2017-06-19 07:50:23 +02:00
Tomasz Trębski bfcd3eeece Restore zookeeper to devstack
openstack/devstack-dev has removed zookeeper support.
For details see [1].
Following commits brings back zookeper as it was
previously installed from devstack.

Extra:
* added concepts of devstack dependencies. That speeds
up installation a bit as common packages are installed
prior to the plugin code

Ref:
[1] c0d16c279a

Change-Id: I0323eacb5cbba8418550e3322189104c35cf4c36
2017-05-27 18:08:53 +00:00