Commit Graph

158 Commits

Author SHA1 Message Date
Hasan Acar a24fd834d0 fix: migrate CI to jammy
fix: change tox.ini

fix: change queries for list_dimension_names and list_dimension_values because of influxdb time filter problem

fix: remove build_sphinx group from setup.cfg

fix: handle hashlib security problem

Change-Id: I0d31a8db5ed71c70e7b878ce5e7940e041d0fa43

Change-Id: I6f7066da10e834550cbf0c053c7bf425ac0ead93

Change-Id: If9575aee73d600bbc84fcdf58deb1c57b508d9c2

Change-Id: If515eaeee7539da3ca49997e88785dc65572b334
2024-02-22 14:35:03 +00:00
Martin Chacon Piza 18ba158e5a Rename the deprecated argument tenant with project
Since oslo.context=4.0.0 Feb 3, 2022  the deprecated argument
tenant was removed [1]

- Remove temporarily monasca-tempest-log-python3-influxdb check
- Remove permanently monasca-tempest-python3-cassandra check

[1] f4cbe4c296

Change-Id: I38dc75df3e7a7575be37dbfa3714e694a19f35c4
2022-03-18 12:19:29 +01:00
Adrian Czarnecki 0b63819ff1 Fix problem with pyparsing.operatorPrecedence method
Pyparsing library was recently updated in golab requiremnts [1].
Since version 3.0.0 operatorPrecedence method was renamed to infixNotation [2].

[1]62f92c0187
[2]16b766b97c/CHANGES (L598)

Change-Id: I3bfefe5b9bc601f383e0b9d80046de387e420fd8
2021-12-15 15:28:26 -08:00
Martin Chacon Piza 73e942cc2e Fix create_notification unittest and bump lower-constraints
- The method create_notification returns an uuid string, if it isn't
mocked the 'notification's id' will be:

<MagicMock name='NotificationsRepository().create_notification()'

which can't be dump to json later.

- Bump Mako to 1.0.7 in lower-constraints.txt
- Bump decorator to 4.1.0 in lower-constraints.txt

Change-Id: I1ba563fd3144241127efe1cedf8853603dcca008
2021-09-30 00:09:22 +02:00
Martin Chacon Piza a6e56ab63c Add Support for Falcon 3.0.0, 3.0.1
- Support to new Falcon 3.0.0 and 3.0.1 keeping compatibility
for version 2.0.0

- Remove Falcon's class OptionalRepresentation
Starting from Falcon 3.0.0 version the class OptionalRepresentation
was removed. [1]

- Remove unnecessary URL slashes which are not compatible
with Falcon >= 3.0.0

- Keep facon.API instead of new falcon.App to keep support for
version 2.0.0

- Disable temporary docker-build and docker-publish Zuul jobs.

[1] https://falcon.readthedocs.io/en/stable/changes/3.0.0.html#breaking-changes

Change-Id: Ifb067429dd66fd350110187ac3a8b6a9977bad90
2021-06-06 21:29:22 +02:00
Ghanshyam Mann dea6f95c15 [goal] Deprecate the JSON formatted policy file
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:

1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.

2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.

Also replace policy.json to policy.yaml ref from doc and tests.

[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: Ibfb162f88cb04c0b2af3fbf41cfcd96bc7e351be
2021-02-02 14:36:06 +00:00
Martin Chacon Piza 041f295b6a Add Influxdb Python client v5.3.1 compatibility
- Add a checker of version using the method ping()
if it failes, maybe because of an old version,
the flow will try with SHOW DIAGNOSTICS.

- Keep the timestamp output to 3 decimal digits as
it is working currently, independently of the version
of Influx Python Client (v5.2.3 or v5.3.1).

- Remove the support for Influxdb (the database)
older than v0.11.0

- Unittests: Adding data to handle more than 3 decimal
digits in timestamp, updating the tests to use mocks
for Influxdb from_0.11.0, creating the mocks with
from_0.11.0 explicitly.

- This change fixes monasca-tempest-python3-influxdb
Zuul job.

Change-Id: I5f8e6d2f0b56813f54fe025f91996b9d6863eadc
Story: 2007624
Task: 39658
2020-12-02 22:03:16 +01:00
Martin Chacon Piza 43922f8223 [goal] Migrate testing to ubuntu focal
As per victoria cycle testing runtime and community goal[1]
we need to migrate upstream CI/CD to Ubuntu Focal(20.04).

-Bump the lower constraints for required deps which added python3.8 support
in their later version.

-Changing the way to install and configure Zookeeper.
Installing Zookeeper from official Apache's tarball.
Adding the possiblity to set the specific Zookeeper version.
Minor change in zookeeper logger.

-Use mariadb JDBC for monasca-thresh in devstack, since Drizzle isn't
compatible with MySql Server v8.0.x which is default in Focal

-Python 3.8 doesn't seem to like dictionary keys changing during
iteration.

Fixing RuntimeError: dictionary keys changed during iteration.

Tech. details:
It runs well in py27: 5 iterations
It runs risky in py37: 7 iterations
It is forbbiden in py38: raised RuntimeError
Fixed with list(dic.items()) or tuple(dic.items())

dic = {'1': 'a', '2': 'b', '3': 'c', '4': 'd', '5': 'e'}
for key, value in dic.items():
    print("Key: {0} Value: {1}".format(key,value))
    del dic[key]
    print(dic)
    dic[key] = value
print(dic)

Story: #2007865
Task: #40197
Depends-On: https://review.opendev.org/756859
Change-Id: Ieb4cf38038ffb4d1a152f8ab3b64a14098c7cbb3
2020-10-12 15:38:05 +02:00
Hervé Beraud 9b79bf633b Use unittest.mock instead of mock
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.

Change-Id: I4c7cb63e0a816b361c2544b1be34d8a6dadeb5c0
2020-06-09 01:23:56 +02:00
Zuul 2d8ae05899 Merge "Use unittest.mock instead of third party mock" 2020-04-23 12:30:09 +00:00
Zuul c7e0ecdfc5 Merge "Update hacking for Python3" 2020-04-23 11:08:18 +00:00
Sean McGinnis 3182d90bbf Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.

Change-Id: Ib2843c62ff29b269139981f067ae6afcab624799
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-23 12:21:44 +02:00
Zuul 176985da7c Merge "Allow users to set periodic notifications on all notification types" 2020-04-22 10:09:43 +00:00
Andreas Jaeger abad342eb2 Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found by updated hacking version.

Remove hacking and friends from lower-constraints, they are not needed
there at all.

Change-Id: I35d848e9af297d3561ea2838a4808166d1c36601
2020-04-22 07:37:55 +00:00
Martin Chacon Piza 0b7dc58296 Migrate from ujson to simplejson
The change updates the imports to use simplejson library and
monasca_api.common.rest instead of monasca_common.rest, since
it was moved to this project during the API's merge.

Temporarily set following jobs as non-voting:

* monasca-tempest-python3-influxdb
* build-monasca-docker-image
* publish-monasca-api-docker-image

Change-Id: Ife3d2c9795a9dc406c2927cc9a077dda01c183c6
Story: 2007549
Task: 39389
2020-04-18 10:28:24 +02:00
Witek Bedyk 394be3f330 Use Confluent Kafka producer for logs
As already implemented for metrics, also logs should be published to
Apache Kafka using the new Kafka client.

Change-Id: Ie909b13c7692267e787d481f4de658db3b07a1c4
Story: 2003705
Task: 36866
2020-01-27 08:20:40 +00:00
Doug Szumski 92fbb93091 Allow users to set periodic notifications on all notification types
In some cases, users may want to send periodic notifications for
notification types other than webhooks.

Story: 2006837
Task: 37417
Depends-On: https://review.opendev.org/#/c/694596
Change-Id: Ia2c50e623aa79e06d2d35df4735fb2805fbf40ed
2020-01-21 10:49:22 +00:00
Zuul 38bd4ce6ea Merge "Fix for editing an Alarm Definition" 2019-11-12 10:32:45 +00:00
Martin Chacon Piza 12dee6996c Fix for editing an Alarm Definition
This change solves the issue when: after removing the deterministic option
of an alarm definition by editing it, the alarm(s) asociated continue(s)
to behave as deterministic. Please see the story for more info.

Added Unittest.

Change-Id: I7743f2d2b8cd7c83541f77c7821f9512fb8abc36
story: 2006750
task: 37233
2019-11-05 13:56:06 +01:00
Ghanshyam Mann 159bc64f41 Correct the IPv6 address and port parsing
There can be lot of possibility for IPv6 address with port,
for example [::1]:80 or [2001:db8:85a3::8a2e:370]:7334.

Parsing that in more standard way is provided by oslo_uilts.netutils
parse_host_port() method[1].

Also unquote '[]' the SERVICE_HOST in case of IPv6 case so that
DB can listen on correct address.

Story: #2006309
Task: #36028

[1] 1b8bafb391/oslo_utils/netutils.py (L37)

Change-Id: I2d0ef40ab71f60564549d031185f99bc7eec40a7
2019-10-22 10:19:07 +00:00
Zuul 313a3ad7af Merge "Merge log-api and api" 2019-09-27 17:53:02 +00:00
Bharat Kunwar 967b918bae Implement separate db per tenancy
At present, all time series are accumulated in the same database in
InfluxDB. This makes queries slow for tenants that have less data. This
patch enables the option to use separate database per tenancy.

This changeset implements the changes on monasca-api which handles read
requests to InfluxDB database.

It also updates the relevant docs providing link to the migration tool
which enables users to migrate their existing data to a database per
tenant model.

Change-Id: I7de6e0faf069b889d3953583b29a876c3d82c62c
Story: 2006331
Task: 36073
2019-09-27 14:24:37 +00:00
Zuul b2c9e1551f Merge "Support time range to query dimension names/values" 2019-09-27 10:48:51 +00:00
Bharat Kunwar 233ea9c51b Support time range to query dimension names/values
At present, dimensions are not scoped by time window, which makes
dimension related queries to large databases timeout because it searches
all of time instead of a time window specified on the grafana app.

This commit implements the server side changes required to scope the
search query by the time window specified on the app.

Change-Id: Ia760c6789ac0063b8a25e52c9e0c3cc3b790ad2d
Story: 2005204
Task: 35790
2019-09-26 15:11:19 +00:00
Adrian Czarnecki d2379a018e Merge log-api and api
*Merge monasca-log-api into merge-api
*Enable logs endpoints
*Add configuration that allow to enable/disable metrics and logs part
*Remove redundant log-api source code

Story: 2003881
Task: 30533
Change-Id: Iaa5689694e7081f3375f1a2235cad31d6a7b5f76
2019-09-26 12:02:20 +02:00
Martin Chacon Piza 2485e39b53 Merge monasca-common code into the monasca-api
This change copies the code from monasca-common used by the 3
monasca APIs into monasca-api for the Merge-APIs target.

After mergin the APIs the duplicated code can be removed from
monasca-common.

Change-Id: I52d36fad846637baf10516f5cbbedc541d4c2064
Story: 2003881
Task: 30427
2019-07-26 14:08:31 +02:00
Zuul fb23e914b1 Merge "Add unit test" 2019-07-08 17:49:11 +00:00
Adrian Czarnecki 1ef1aa1acd Add unit test
*Add unit test for helpers module
*Add unit test for metrics and notification endpoints

Story: 2006177
Task: 35701
Change-Id: Ie424f032588e4ad6afea3ea599b6febf9dd7f479
2019-07-08 13:30:18 +02:00
akhiljain23 0fae94fac2 Update hacking version to 1.1.x
This commit updates hacking version in test-requirements and
fixes some related pep8 issues

Change-Id: I67d85eb5bef72c38cc5360b5625d6b1c37adb40f
Story: 2004930
Task: 29315
2019-06-20 04:52:24 +00:00
Doug Szumski a9cc4bb482 Add support for using Falcon 2.0.0
Falcon 2.0.0 introduces some breaking changes. The relevant ones here are:

- falcon.testing.TestCase.api property was removed
- falcon.testing.TestBase class was removed

Additionally, the default behaviour for handling trailing slashes on
URIs also changed:

https://falcon.readthedocs.io/en/latest/user/faq.html#how-does-falcon-
handle-a-trailing-slash-in-the-request-path

This commit adds support for using the new release. It currently makes
no effort to be backwards compatible with older releases.

The change also updates the requirements for influxdb and sphinx
libraries to match global requirements.

Until monasca-log-api implementation is not updated to support the new
version of Falcon, `monascalog-python3-tempest` is marked to be
non-voting as agreed in the team meeting.

Story: 2005695
Task: 31015
Change-Id: I03bc8d502a333a7a71d9c12b8ddc7c5dc0a4f588
2019-05-29 16:10:53 +02:00
Joseph Davis d6852ee1b7 Detect missing metric_id before passing through bytearray()
It is possible for a row in Cassandra to have a missing metric_id
(shows as 'null' in cqlsh).  This causes an ugly NoneType error
to be passed up to the user on the command line for
'monaca metric-list'.

Fix is to detect the missing value, log an error, and return the
row with None for the metric_id.

Change-Id: Ie617932c6b12a6cfe441510e120bb77a3470b9cf
Story: 2005305
Task: 30194
2019-03-26 14:17:18 -07:00
Adrian Czarnecki e18ed3f02d Improve tests coverage
* Add unit tests
* Fix python3 and python2 compatibility

Story: 2003881
Task: 29254

Change-Id: Ib54b94737dace3104976321e38ad1a9076a46877
2019-03-11 14:49:33 +00:00
Doug Szumski ff0c1cc67d Remove built-in plugins from database schema
Add an alembic migration to remove the built-in plugins. Monasca-notification
becomes responsible for creating them, if they are enabled in the
monasca-notification config file. This also adds some very basic tests and
fixes an issue with Python 3 support (highlighted by the tests) where the
schema was stored as unicode which prevented hashing.

Story: 2003801
Task: 26533
Change-Id: I74848bac2137e446c8825f23778f6a16a96a2048
2018-12-11 09:56:04 +00:00
Zuul 42584c12b4 Merge "Add "monasca-status upgrade check" for pre-upgrade" 2018-11-22 14:24:45 +00:00
Joseph Davis 22be5aa57f Add "monasca-status upgrade check" for pre-upgrade
Add a tool to run before upgrading a Monasca installation
that will check for potential issues and report on readiness to upgrade.

For Monasca, this initial version effectively does nothing, but gives a
framework for future checks to be included.

This is a community-wide story for Stein [1].

Story: 2003657
Task: 26142

[1] https://governance.openstack.org/tc/goals/stein/upgrade-checkers.html

Change-Id: I799602caa4029d67f59307bf1ca0d1e8f254d415
2018-11-12 11:24:21 -08:00
Dobroslaw Zybort 856a969da3 Set proper default agent authorized role
Story: 2003930
Task: 26838

Change-Id: I27f191d1baf850d5b9e56928fd6b45818f430216
2018-10-17 14:01:05 +02:00
Adrian Czarnecki c8c5217af0 Py3: Fix encoding in _verify_topics method
Change-Id: I18c649b1880b52f5961e3dee18c3c9aef3afdde9
2018-09-06 13:53:52 +02:00
Adrian Czarnecki 3f530c85e8 Fix incorrect error when topic not found
Story: 2003668
Task: 26186
Change-Id: I4df9e4373544a26c41ff4b0e8f6a0d68eccd0d78
2018-09-06 09:58:49 +02:00
Adrian Czarnecki 7ba3a8b7ab Python3: Fix failing tempest tests
Fix sorting and string encoding

Story: 2000975
Task: 24394

Change-Id: Iaf4575b47fdc299c4e9eb77e19c36bebb0642ec8
2018-08-24 08:33:18 +00:00
Adrian Czarnecki b3d6392140 Python3: Fix NotificationMethodRepoDB tests
Change-Id: Id52fc45d2b276a121c5f4fdaab2196863beda3af
2018-07-05 10:51:59 +02:00
Zuul a0381710f2 Merge "Python 3.5: Fix alarms repository" 2018-07-03 13:05:59 +00:00
Zuul cb7f226a71 Merge "Add py3 support for repositories packages" 2018-06-28 10:17:57 +00:00
Adrian Czarnecki 6af07fb0fc Add py3 support for repositories packages
Story: 2000975
Task: 12579

Change-Id: Iaf01fcf9e600d1f89eb202f6a908d6de50c9578f
2018-06-27 13:29:36 +02:00
Adrian Czarnecki dd99ad56a2 Python 3.5: Fix alarms repository
Fix python3 unit test for alarms reposiotry.

Change-Id: Ibb8cba96f62ac5580cd7b74bf6f2751efa66cd90
2018-06-22 09:57:28 +00:00
Zuul 378408a490 Merge "Using oslo.policy for monasca-api" 2018-06-07 10:52:26 +00:00
Amir Mofakhar 3ca4b057dd Using oslo.policy for monasca-api
Added policies and used policy enforcement engine
from monasca-common.

- Replaced security with oslo.policy
- Updated unit tests and implemented some new tests
- Added a new entry point for generating sample policy file by tox

story: 2001233
task: 6355

Change-Id: I4aa444fe6ec883160c03c201145c77994b6615f9
Signed-off-by: Amir Mofakhar <amofakhar@op5.com>
2018-06-05 12:25:07 +00:00
Zuul 5bd1b5e212 Merge "Add py3.5 support for alarms endpoint/unit test" 2018-05-24 11:31:31 +00:00
Adrian Czarnecki 2e078cdd87 Change default authorized role
monasca-user role is more sufficient then admin role.

Story: 2001895
Task: 14380
Change-Id: I16091dbd631a5e94d08598a23eeb3bdf97cf0a92
2018-04-24 12:26:59 +02:00
Zuul c4fed020db Merge "Add py3 support for versions endpoint/unit test" 2018-04-18 09:02:29 +00:00
Adrian Czarnecki 8efd03aab8 Add py3.5 support for alarms endpoint/unit test
* Python 3 dict.values() returns iterator, must be converted to list
* use `u` prefix for unicode literals
* use builtins.str for handling unicode
* use oslo_serialisation for loading JSON objects

Can be tested with:
  tox -e py35 -- -r monasca_api.tests.test_alarms.*

Story: 2001400
Task: 12580

Change-Id: I0cc7018f7d5efedb4937922b6daa2fb59b3d645c
2018-04-12 17:34:17 +02:00