Commit Graph

200 Commits

Author SHA1 Message Date
Hasan Acar a24fd834d0 fix: migrate CI to jammy
fix: change tox.ini

fix: change queries for list_dimension_names and list_dimension_values because of influxdb time filter problem

fix: remove build_sphinx group from setup.cfg

fix: handle hashlib security problem

Change-Id: I0d31a8db5ed71c70e7b878ce5e7940e041d0fa43

Change-Id: I6f7066da10e834550cbf0c053c7bf425ac0ead93

Change-Id: If9575aee73d600bbc84fcdf58deb1c57b508d9c2

Change-Id: If515eaeee7539da3ca49997e88785dc65572b334
2024-02-22 14:35:03 +00:00
jayonlau 95747eec73 Clean up extra spaces
Although these errors are not important, they affect the code specification.

Change-Id: Iab8e8ce9e73722c78283323a2e350149e8713e78
2021-10-01 18:31:14 +00:00
Martin Chacon Piza a93e1cc979 Workaround for new pip 20.3 behavior
This WA set specific pip version to avoid the issue:
"ERROR: Links are not allowed as constraints" [1]
A proper fix would be to adapt to new pip behavior.
Similar issue on Devstack [2][3]

[1] https://github.com/pypa/pip/issues/8210
[2] https://bugs.launchpad.net/devstack/+bug/1906322
[3] https://review.opendev.org/c/openstack/devstack/+/764803

Change-Id: I49d436b0cf1503b148a04f82e1ee41941ebd9d0a
2021-03-31 16:43:28 +00:00
Martin Chacon Piza 43922f8223 [goal] Migrate testing to ubuntu focal
As per victoria cycle testing runtime and community goal[1]
we need to migrate upstream CI/CD to Ubuntu Focal(20.04).

-Bump the lower constraints for required deps which added python3.8 support
in their later version.

-Changing the way to install and configure Zookeeper.
Installing Zookeeper from official Apache's tarball.
Adding the possiblity to set the specific Zookeeper version.
Minor change in zookeeper logger.

-Use mariadb JDBC for monasca-thresh in devstack, since Drizzle isn't
compatible with MySql Server v8.0.x which is default in Focal

-Python 3.8 doesn't seem to like dictionary keys changing during
iteration.

Fixing RuntimeError: dictionary keys changed during iteration.

Tech. details:
It runs well in py27: 5 iterations
It runs risky in py37: 7 iterations
It is forbbiden in py38: raised RuntimeError
Fixed with list(dic.items()) or tuple(dic.items())

dic = {'1': 'a', '2': 'b', '3': 'c', '4': 'd', '5': 'e'}
for key, value in dic.items():
    print("Key: {0} Value: {1}".format(key,value))
    del dic[key]
    print(dic)
    dic[key] = value
print(dic)

Story: #2007865
Task: #40197
Depends-On: https://review.opendev.org/756859
Change-Id: Ieb4cf38038ffb4d1a152f8ab3b64a14098c7cbb3
2020-10-12 15:38:05 +02:00
Witek Bedyk 3014c840d6 Set legacy_kafka_client_enabled = False on default
We change the default value of kafka.legacy_kafka_client_enabled from
True to False. The use of new Confluent Kafka client is recommended.

DevStack plugin does not set this option anymore.

Depends-On: https://review.opendev.org/740959
Depends-On: https://review.opendev.org/740966
Change-Id: I4d57b8893a6a131769009dc3299789d3fc89bab6
Story: 2007924
Task: 40338
2020-07-14 16:15:23 +02:00
Witek Bedyk d6104a693b [DevStack] Allow testing different Kibana versions
Kibana API has changed between the versions. To allow testing
different Kibana versions `kibana_version` option should be set in
tempest.conf.

This is a partial port of https://review.opendev.org/727195

Change-Id: I0a21bd99c942ab05ce1eae51226132691518bd91
2020-05-26 13:39:46 +02:00
Zuul 7672d09424 Merge "Remove handling legacy monasca-agent service file" 2020-02-06 10:07:45 +00:00
Zuul 16ce482e66 Merge "Reduce number of partitions created by kafka" 2020-02-03 10:14:40 +00:00
Adrian Czarnecki 5918a87fb8 Reduce number of partitions created by kafka
This should help to reduce resource consumed by devstack plugin.

Change-Id: Id0bc58a21d11dda518710896e0fa0ec14e8e4042
2020-01-30 14:32:36 +00:00
Witek Bedyk 6d02001337 Remove handling legacy monasca-agent service file
monasca-agent service file is not used anymore. monasca-collector,
monasca-forwarder and monasca-statsd are new services defined in
monasca-agent target.

This change removes the legacy monasca-agent service file handling from
DevStack plugin.

Story: 2000975
Task: 36886

Change-Id: Iac7d608a202440381f211d04b0978218ac36c54d
2020-01-27 10:40:24 +00:00
Witek Bedyk c3993bf48a Disable not needed services in CI job
To improve reliability of logs tempests tests we deploy only necessary
services in CI environment. This should save a lot of resources.

Change-Id: Id5b8ddc0700b2b7bc100f1eefbe891d9016e918c
2020-01-24 10:01:30 +00:00
Adrian Czarnecki 7c5615617f Update nodejs and npm packages
This updated allow to use python3 to build grafana

Update nodejs from 8.10.0 to 10.17.0
Update npm from 3.5.4 to 6.11.3

Change-Id: I709d1e6134150f1648a8f409ce438544e15ce403
2019-12-11 14:14:51 +00:00
Adrian Czarnecki 1e3e1173ad Update devstack plugin to support new api
Merge log-api and api

Story: 2003881
Task: 36195

Depends-On: https://review.opendev.org/681419/
Change-Id: Idb2361e5ab701cd3dee4eabb4b43116f6e1205c4
2019-12-04 13:52:57 +01:00
Adrian Czarnecki 1941530472 Fix python3 compatibility issue
Make grafana-init.py and influxdb_setup.py scripts compatible with  python3

Change-Id: Id2909094473d82844072a299b9e0dab2d4893557
2019-11-07 14:02:08 +01:00
Ghanshyam Mann 159bc64f41 Correct the IPv6 address and port parsing
There can be lot of possibility for IPv6 address with port,
for example [::1]:80 or [2001:db8:85a3::8a2e:370]:7334.

Parsing that in more standard way is provided by oslo_uilts.netutils
parse_host_port() method[1].

Also unquote '[]' the SERVICE_HOST in case of IPv6 case so that
DB can listen on correct address.

Story: #2006309
Task: #36028

[1] 1b8bafb391/oslo_utils/netutils.py (L37)

Change-Id: I2d0ef40ab71f60564549d031185f99bc7eec40a7
2019-10-22 10:19:07 +00:00
Witek Bedyk 91792de858 Enable monasca-agent libvirt plugin
The change configures and enables monasca-agent libvirt plugin in
Monasca DevStack plugin. It will allow easier integration of Monasca
with other projects for use cases based on VM measurements.

Story: 2005752
Task: 36842
Change-Id: I061e905bed06cc10c4d62201dcd945f715eb3a6e
2019-10-02 13:27:49 +02:00
Witek Bedyk 47c5ad37d5 Use Confluent Kafka client
The change introduces the possibility to run the API with the new
confluent-kafka client. It has to be enabled in the configuration file.

Story: 2003705
Task: 35859

Depends-On: https://review.opendev.org/680653
Change-Id: Id513e01c60ea584548c954a8d2e61b9510eee8de
2019-09-24 09:41:17 +00:00
Witek Bedyk dfd9423526 Upgrade Apache Kafka to version 2.0.1
Upgrade Apache Kafka to current stable version 2.0.1.

Issues have been observed with legacy Kafka producer being used in
notification engine together with newer Kafka broker versions.

Following changes are included:

* Deprecated configuration option `advertised.host.name` is replaced
with `listeners`.
* Default `log4j.properties` is used.

Story: 2005624
Task: 30933
Change-Id: I898b511b2ab8f68e4850faab2098044cd3f94ee7
2019-08-14 14:51:44 +02:00
Doug Szumski 07efdae819 Upgrade InfluxDB to latest release (v1.7.6)
Updated default config to use InfluxDB v1.7.6.
This appears to work, at least for viewing
metrics, persisting them, and some CLI commands.
Updated influxdb.conf to bring in line with sample
config.
Drops support for configs of older versions.

Story: 2005624
Task: 30877
Co-Authored-By: Isaac Prior <isaac@stackhpc.com>
Change-Id: Ib73151e6e7d0934303c9e801bb018bc22f1716ae
2019-06-17 08:36:24 +00:00
Zuul 4681262d82 Merge "Set up DB schema in DevStack plugin using Alembic" 2019-02-21 12:27:59 +00:00
Witold Bedyk 6409d06439 Set up DB schema in DevStack plugin using Alembic
SQL schema scripts are replaced with Alembic command line tool.
Notification plugins are enabled in notification engine configuration
file.

The change also fixes the `sub_alarm` table schema. The foreign key must
refer to the field with the same data type, charset and collation.
Otherwise the schema migration fails occasionally with Python 3 with the
IntegrityError[1]:

  oslo_db.exception.DBError: (pymysql.err.IntegrityError) (1215, 'Cannot add foreign key constraint')

[1] http://logs.openstack.org/61/622361/8/gate/monasca-tempest-python3-influxdb/26104a3/controller/logs/devstacklog.txt.gz#_2019-01-30_15_52_45_455

Change-Id: I95d7e91e62ff13e84f963fdbb3bca6b7dfd7c310
Story: 2001654
Task: 14421
2019-02-20 14:53:10 +01:00
Thomas Bechtold 96e13dad75 Replace api-config.conf with monasca-api.conf
api-config.conf is the deprecated path for the configuration file. The
new path is /etc/monasca/monasca-api.conf .

Story: 2004708
Task: 28987

Change-Id: If06d9114529c3ef4520247dfc7c32d3769f07084
2019-01-19 12:15:22 +01:00
Adrian Czarnecki d033c1cfbe Use PYTHON2.7 for npm
Change-Id: Ic69821bbae6a01bf6c68ca3e4f8c4b40ec5bf950
2019-01-09 10:04:21 +01:00
Adrian Czarnecki cd94a9ca03 Configure system encoding format
This is needed to properly build monasca-common from repository

Change-Id: Iacec60495d02b0a2b882044192058da24f484f97
2019-01-08 09:57:00 +01:00
Witold Bedyk 35fc6cf70a Migrate DevStack plugin to Ubuntu Bionic
We want to migrate to the latest Ubuntu LTS release 18.04 aka Bionic.
See [0] for the big picture.

This change:
* installs Node.js from Ubuntu packages,
* removes apt-get `-y` option which is included in devstack's wrapper
function.

[0] https://etherpad.openstack.org/p/devstack-bionic

Story: 2004474
Task: 28174
Change-Id: I7ed847ee5f4f203f112dd051f0fb157a11907a75
Depends-On: https://review.openstack.org/618169
2018-12-03 13:59:42 +01:00
Lukasz Zajaczkowski 12f1ae68b8 Add grafana init
Change-Id: I6c625272396c6b893f297265ae2389207e60b364
Story: 2003623
Task: 25752
2018-09-25 07:40:30 +02:00
Lukasz Zajaczkowski f8c030db65 Skip agnet init process when service disabled
Change-Id: Id9b2f8c41c90a44b2edeef76f8748e1527cbc0db
Story: 2003797
Task: 26525
2018-09-18 12:35:17 +02:00
Lukasz Zajaczkowski b12f239f29 Change monasca-collector service user and group to root
Starting monasca-collector service as root user will allow
to gather metrics from services which were started with root
privileges.

Change-Id: Ida453cc9e9f3f5a6bf4c69e007b7572665b6efbd
Story: 2003623
Task: 26467
2018-09-17 10:54:12 +02:00
Zuul 1731018f06 Merge "Remove invalid options when deleting Kafka topics" 2018-09-04 09:49:40 +00:00
Adrian Czarnecki 3a5b8775e2 Enable python3 in devstack
Change-Id: I44f09c9068ecd682d7c91945a5e32842ab335b80
2018-08-24 12:59:16 +00:00
Witold Bedyk 9f2e147308 Increase timeouts when installing Cassandra schema
This change tries to avoid devstack plugin failures like this one [1].

[1] http://logs.openstack.org/86/583586/1/gate/monasca-tempest-python-cassandra/ac3bb9b/logs/devstacklog.txt.gz#_2018-07-19_08_35_08_757

Change-Id: I5eae2d759f2bd1b81904825f2417ed03a651292b
Story: 2003030
Task: 23057
2018-07-19 11:15:20 +02:00
Witold Bedyk c440b1a5c1 Remove invalid options when deleting Kafka topics
With Apache Kafka 1.0.1 deleting topics fails if called with not
relevant options like `--partitions` or `--replication-factor`.

Change-Id: I614962f45e8487b34507ef1d695f3f76d0186d55
Story: 2002850
Task: 22799
2018-07-05 15:30:25 +02:00
Zuul 5ffd0a72ca Merge "Changes to manage the agent target" 2018-06-29 12:45:02 +00:00
Adrian Czarnecki 5359e4e633 Enable uWSGI support in devstack
Story: 2001464
Task: 6181
Change-Id: If4329e51feebeb68538917d7a68aad53ae469b07
2018-06-12 14:36:46 +02:00
Vu Cong Tuan ddc765970a Remove deprecated "auth_uri"
According to [1], "auth_uri" has been deprecated.
This patch remove it completely.

[1] https://review.openstack.org/#/c/508522

Change-Id: I094ee869a821fefbbf5de64defda408315e7d594
2018-05-11 14:40:02 +07:00
Stefano Canepa 5effcebd92 Changes to manage the agent target
Changed way to start and stop monasca-agent to take into
account that now it's a systemd target and not a service.
Improved cleanup to take care of all the installed files.

Task: 19584
Story: 2000975

Change-Id: I9420171623f288873fa263f5bc87560629391ea5
2018-05-07 15:56:42 +01:00
Adrian Czarnecki 2e078cdd87 Change default authorized role
monasca-user role is more sufficient then admin role.

Story: 2001895
Task: 14380
Change-Id: I16091dbd631a5e94d08598a23eeb3bdf97cf0a92
2018-04-24 12:26:59 +02:00
Witold Bedyk b918dccad9 Fix installing Cassandra schema in devstack
Change extends `connection-timeout` and `request-timeout` when
installing Cassandra schema.

Change-Id: I83a51ff92ed5ef2fa123d4bce518071c4d12aa2a
Story: 2001858
Task: 14310
2018-04-16 14:34:22 +02:00
Witold Bedyk 745f3fb5b9 Use Drizzle JDBC for monasca-thresh in devstack
MySQL Connector is released under GPLv2 license which restricts the
distribution of the consuming project [1]. This change configures
monasca-thresh to use Drizzle JDBC (BSD license) in devstack.

[1] https://governance.openstack.org/tc/reference/licensing.html

Story: 2001522
Task: 6323
Change-Id: I1154e166e1aa6fe6693bdd169d2e5544719fbe6c
2018-02-06 17:12:13 +01:00
James Gu ba43f07726 Add cassandra support
Support Cassandra db installation and Cassandra related
configurations for Monasca api and persister services
in devstack. Add Monasca rest API Cassandra plugin for
retrieving metric, dimension, measurement, statistics
and alarms.

Change-Id: Ie60d668692e1f25f555dda2355f4e513d582736c
story: 2001231
task: 5759
2017-12-15 19:11:00 +00:00
Zuul 2d84109db1 Merge "1724543-fixed kafka partition creation error in devstack installation" 2017-12-06 22:31:08 +00:00
Witold Bedyk 3d2a3988e3 Download Kafka from Apache Archives
Apache Kafka 0.9.0.1 is not available on mirrors any more.

Change-Id: I47e0a9d1718bd5268610b2183397dec22911947a
2017-11-20 18:37:38 +01:00
anilkumarthovi 71d40c7131 1724543-fixed kafka partition creation error in devstack installation
Change-Id: I29db498a44d527dbbb917c90fae09d67c2ebbc76
2017-10-31 21:08:15 +05:30
Scott Grasley 7651d247a1 Set Kafka Broker advertisement hostname
Default settings only allow localhost advertisement by kafka broker.
with this update kafka broker will advertise devstack configured ip address.

Change-Id: Ie3a146598789d6f63bbf55c4847cc569231c9dec
2017-10-31 05:25:41 +00:00
James Gu 8d87c463bb Remove keystone cache dir
Devstack lib/keystone deprecated the cache dir argument in the
configure_auth_token_middleware method. It was used for PKI.
Monasca devstack deployment fails due to this api change.

Change-Id: Ic88dc5ad38d3b5c136e092fc0b66d78eb19dd6a5
2017-10-19 04:14:15 +00:00
Tomasz Trębski 58d927d083 Better handling of storm processes
Use devstack utilities to launch storm processes
and be able to see their logs.

Change-Id: Iccc30d411a49b9e47d6625909427f071397aedfd
2017-10-06 08:19:11 +02:00
Vu Cong Tuan d38b1de1d7 Remove SCREEN_LOGDIR from devstack
SCREEN_LOGDIR has already been deprecated [1]
[1] https://review.openstack.org/#/c/499186

Change-Id: I16d11b03819ef490524a6feed33dc990c259c291
2017-10-06 08:49:39 +07:00
Tomasz Trębski df9970530f Drop download_file for get_extra_file of devstack
devstack has introduced get_extra_file function
that aims at downloading the files from the URL.
The method is better alternative as it is integrated
into devstack caching process.

Change-Id: I8a804d8842e0921b0f2acfcf1cfbdffab21e5e9c
2017-09-26 07:30:20 +02:00
Tomasz Trębski 5d27af4079 Config-generator for monasca-api
Change migrates the monasca-api to use oslo-config-generator in
order to always build newest configuration file. That removes
the need of maintaning the file along with changes to oslo configuration.
The example of the file is also included in the documentation.

Also:
* ported case for launching api under WSGI and allowing the argument
  parsing of oslo to take place

Story: 2000970
Task: 4865

Story: 2000964
Task: 4106

Change-Id: I57547b0e2122e40f58db5f949773900b76214526
2017-08-28 06:58:57 +00:00
Artur Basiak a23a4f1518 Add Keystone configuration for Grafana
Add Keystone auth_url for Grafana configuration.

Depends-On: I08195ad281637e6567a8acbace9862df2d5f9502

Change-Id: If65c94104c9951e4f4508247bd92449de949def1
2017-07-21 08:23:55 +02:00