The contents of this repository are still available in the Git source
code management system. To see the contents of this repository before it
reached its end of life, please check out the previous commit with
"git checkout HEAD^1".
For any further questions, please email
openstack-discuss@lists.openstack.org or join #openstack-telemetry on OFTC.
Change-Id: Ic8a7001e708d8da7cf8951eefd0a96762ea5fa46
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:
1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.
2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.
Also replace policy.json to policy.yaml ref from doc and tests.
[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Change-Id: Ie6e22b0b47c5148290f7b76c95967cd9a343343c
This commit updates the policies for all policies in panko to support
enhanced token scope checking and default roles, but implementing
support for a reader role for read-only APIs.
This is part of a broader change to provide a more consistent and secure
authorization experience for operators and users across OpenStack.
Change-Id: Ia0daee7d909f31e7f8f9e75fa52dca9254441559
This commit adds a new constant to base.py that models a common persona
being implemented across OpenStack called system-reader. We can use this
persona in future patches to update the default policies for the
appropriate APIs.
Change-Id: I12074fe328db71895bbdfda3c9e7c56cc6b6f40d
When thare're a lot of events in the database panko-expire
consumes a lot of memory to delete all records with a single
call. To avoid this behaviour a new config option
`events_delete_batch_size` was introduced.
Change-Id: Icf83ffe089301b3782273923f18efd4d209131c2
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: I92f240b2b2fbb8a0a7ac188ff9bed8b0d11209b3
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.
Fix problems found.
Update local hacking checks to work with new flake8 version.
Change-Id: Id5d224358b44fd7ab4bd3d08f1db6b6faf54042f
Configuration options must be registered before using them.
In impl_elasticsearch.py, using conf.database.es_index_name
or using conf.database.es_index_name are all wrong that will
lead the api server to be invalid.
Change-Id: I6b1364954c231a477d493d4a518c080f87058123
Signed-off-by: yuanrunsen <yuanrunsen@unitedstack.com>
oslo project provide jsonutils, the others project use it
this PS to update the json moudule to oslo jsonutils.
Change-Id: I311d4f9dec7b685a5bccb9725de6010926375c7d
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
1) It checks that person who wants to trace is trusted and knows
secret HMAC key.
2) It starts tracing in case of proper trace headers
and adds first wsgi trace point, with info about HTTP request
* Traces HTTP API calls
* Traces DB (SQLAlchemy) calls
Demo: https://tovin07.github.io/panko/openstack-event-list.html
HOW TO TEST?
1. Install devstack with panko as usual
2. Install osprofiler
pip install osprofiler
3. Add these line to /etc/panko/panko.conf
[profiler]
enabled = true
hmac_keys = SECRET_KEY
connection_string = redis://localhost:6379 # example when using redis
4. Pass HMAC_KEYS to client commands
- panko client with `--profile <HMAC_KEYS>`
- openstack client with `--os-profiler <HMAC_KEYS>`
Output will look like this:
To display trace use the command:
osprofiler trace show --html <TRACE_ID>
5. Use osprofiler to get the trace
osprofiler trace show \
--connection-string redis://localhost:6379 \
--out out.html \
--html <TRACE_ID>
6. Open that html file with browser and view the result
Change-Id: Ic934acbe1340a3e00361f2709f34725e0e4aa3ba
currently when there is a problem with configuration, the logging is
insufficient, especially when running panko-dbsync command.
Change-Id: I22ef74eb45f324fc6e374c4a3e03fbe9d7268eed
Logging the sqlalchemy DuplicateEntry error message on each event
is useless. And people may though it's an error.
This change moves the log level to debug.
Change-Id: I86b64e5cf370386dab6f962b09029feeb35a9dbf
This reverts commit b540674a84.
This changes is wrong in the sense as it removes the ability to filter via ge/le the start and end timestamps.
Change-Id: I0c82917a6fd6795266e34b4d151faf13f43ab5bb
* Intree panko tempest plugin does not contains any tempest tests
and has just dummy tempest plugin skeletion, so it is better to
remove from the project. If in future new panko tempest tests
will be written, will go to a new repo where all the telemetry
tests will resides.
Change-Id: I4f56bdac3ddd7b55ae791c9e6aafb78bc5f6e14b
Panko does not depends on oslo.serialization while it is using it. However,
since it does not really need it, let's use the regular json attribute anyway.
Change-Id: I63e5927c625d0a080bd6a9ea77ff939253039d5e
This is part of a community effort to provide better user experience
for those having to maintain RBAC policy. More information on this
effort can be found below:
https://governance.openstack.org/tc/goals/queens/policy-in-code.html
bp policy-and-docs-in-code
Co-authored-By: Hieu LE <hieulq@vn.fujitsu.com>
Change-Id: I9bad70abcf5543c3e5e5da25c56c408ee3ff0346
Currently admin can only get project_id consistent events, while
sometimes admin need to get all the events, so add a parameter
"all_tenants" for the api to achieve this purpose.
Change-Id: I7c479a63c41d4b0fbf645bdb1b70f0259b0cb859
start/end_timestamp originally are associated with op 'ge' and 'le'.
Changing them to 'eq' fixes the bug and the usage will make more sense.
Related review: https://review.openstack.org/#/c/495763
Change-Id: Ia94a499f9340102553863db48dccb96bea92ea78
Closes-Bug: #1711997
Closes-Bug: #1710812
Original problem:
Event storage backend is sqlalchemy(mariaDB).
When listing events by:
openstack event list [--limit n]
or ceilometer event-list [--limit n]
using non-admin user, the number of events returned does not match the
limit 'n'. It has much fewer events than it should have. Even when
limit is not given, the default limit (100 if not changed) will apply
and that the returned result is still wrong.
Problems in code:
In the file 'panko/storage/impl_sqlalchemy.py
The subquery for trait filtering(project_id, user_id mostly) which
generated in function 'get_events' into variable 'trait_subq' (start
around line 269) does not seem behave correctly.
In the original code, the subquery is being wrapped into a deeper
layered subquery when a new trait is applied. The resulting
complicated subquery makes results have duplicate 'event_id's. And
duplicate ids still count for the amount of the result, resulting in
much fewer number of events than the one 'limit' flag sets.
Fixing:
This part(line 269-299) which generates the subquery is modified.
For each trait, the table the trait is in is adding to the 'from' part
of the query. As for the 'where' section, the trait is filtered, and
the event_id should be the same as other ones.
The function at the beginning is changed from '_build_trait_query' to
'_get_model_and_conditions' which used by the part mentioned above.
'sqlalchemy.orm.aliased' is imported to aliase tables since the same
table may be selected multiple times, and it is required to apply
'aliased' to the table model before adding them to the query.
Test:
$ tox -e py27-mysql
The difference is at function 'test_get_event_multiple_trait_filter'
in file '/tests/functional/storage/test_storage_scenarios.py'.
The order of the traits to filter can affect how the original code
produce the subquery, which is the cause of duplicate event_id.
Patch:
1. Fix coding style according to Julien Danjou's suggestion.
2. Add test code. Actually, modify existing one.
Change-Id: Ic303e4ed7af43c1d31559895c56a29b2c5ea85fc
It was not documented how to filter based on `generated` field.
Added a field and a description for it so that it gets rendered in the
docs.
Change-Id: I3f4929ce4876015817e230d8855dbccbd5d34e27
Matthias Runge