The method assertEquals has been deprecated since python 2.7.
http://docs.python.org/2/library/unittest.html#deprecated-aliases
Also in Python 3, a deprecated warning is raised when using assertEquals
therefore we should use assertEqual instead.
Change-Id: I4e99cd5d58b676f300eddcff673c26d4769e5af7
Closes-Bug: #1218185
Some of tests use different method of
assertTrue(isinstance(A, B)) or assertEqual(type(A), B).
The correct way is to use assertIsInstance(A, B) provided
by testtools
Change-Id: I6bb51ecec188847f67b579694bd47bf7599bfbd6
Following OpenStack Style Guidelines[1]: http://docs.openstack.org/developer/hacking/#unit-tests-and-assertraises
[H234] assertEquals() logs a DeprecationWarning in Python 3.x,
use assertEqual() instead. The same goes for assertNotEquals().
Change-Id: Iaa4b2fd04d2aa661bb196c4b9f2494a281a9294a
Instead fetching of all clusters data we are selecting only
required fields for report.
Change-Id: Id5555d0d0c29a20a3d03e640398a85b961fe37ec
Closes-Bug: #1608889
We don't use Elasticsearch for flexible reports generation on the
fuel-stats web UI, only for five fixed reports. Thus using of
Elasticsearch is overhead and it can be removed from the servers
Instead of Elasticsearch we use fuel-stats json api calls and
PostgreSQL + Memcached.
Changes list:
- api call added to fuel-stats json api for data required on the web UI page,
- column release added to DB installation_structures table schema,
- memcached is used for caching data for the web UI page,
- elasticsearch client removed from js requirement,
- web UI page rewritten to use fuel-stats json api instead Elaticsearch.
Co-Authored-By: Kate Pimenova <kpimenova@mirantis.com>
Change-Id: Ie752e0d0a3c80933888f986e2497b45adce730c9
Closes-Bug: #1595548
Some files were committed or created with executable permissions 755.
Theese permissions changed to 644.
Change-Id: Ic7a99eda0518de89ff72f13936f0d131fd1eb121
Closes-Bug: #1598746
In py3 ,
Xrange is renamed to range,
so it needs to be modified to make it compatible.
the code:
print (xrange(5))
Traceback (most recent call last):
File "code", line 4, in <module>
print (xrange(5))
NameError: name 'xrange' is not defined.
Change-Id: I3a0e22876c35ced5c961c092fcda25db33d1c599
Closes-Bug: #1596127
We had data duplication in the nodes CSV report:
- platform_name is identical to meta.system.product
- manufacturer is identical to meta.system.manufacturer
Product Management team uses data from meta in their reports, thus
we are removig platform_name and manufacturer fields from the nodes
CSV report.
Change-Id: I890d6e651de056b3ca3a1f2ab977652ca8d68ef9
Closes-Bug: #1585564
Interfaces info changes:
- interfaces details added
- offloading modes added
- interface properties added: SR-IOV, mtu, dpdk, numa_node
Meta info about numa topology added: numa nodes, hugepages, distances
Node error_type field removed. This field doesn't reflect actual
node state.
Check that all node attributes were exported added into test.
Change-Id: I9b5f45dad43fefb0e60a563123dab833293e4332
Closes-Bug: #1575669
Closes-Bug: #1575666
Changes list:
- CSV exporter for nodes added
- link on nodes report added to the UI
- export of lists and tuples to CSV implemented. As result
enumeration functions removed from export_utils
- aggregate functions handled in the skeleton
- lists and tuples handled in the skeleton generation util
- workaround for counting of volume attachments removed
- new fields added into installation info and OSWL skeletons
- unnecessary fields removed from installation info and OSWL skeletons
- lists definitions changed in skeletons
- common logic for plugins and nodes reports extracted to helper functions
- node related data removed from cluster report (manufacturer, platform name)
Change-Id: Iacf5421895f6803acf071111ef04a13c53eba6ac
Closes-Bug: #1572490
On huge reports data we can have 500 HTTP error by timeout limitation.
In case of streaming data from the DB cursor we had hanged
DB operation inside the open transaction. For fix the issue releasing of
SQLAlchemy session was added to collector and analytics apps.
Additional changes:
- We stop to migrate action logs data to Elasticsearch due to
no one UI report uses this data.
- Parameters fixed in OSWLs 'export' call in 'all_reports'.
Change-Id: I009081a2304b36fb4ab3705b6fa09a8a3df5710b
Closes-Bug: #1564427
We had huge memory consumption on fetching OSWLs due to fetching
clusters data in each OSWL row.
Joined clusters data removed from each OSWL row. Explicit
fetching of version info from clusters is introduced to CSV
exporter.
We are using SQLAlchemy yield_per for fetching large query results.
Yeld_per page parameter reduced from 1000 to 100.
We still have issues with memory consumption in all
reports generation. Thus we are temporary removing link
on 'All reports' from web UI.
Example uWSGI configuration changed to start http service.
Change-Id: I66c97058c0f15f8a15b626ab1fa222106754a455
Partial-Bug: #1564427
Proper name for fuel_version info is used in the CSV reports
generation process.
If version_info is not provided by OSWL, than OpenStack version,
OS name and release are fetched from cluster data.
Change-Id: If3e7442a5b336af5c1a4f5c257f280e332ab81cf
Closes-Bug: #1547565
Cluster attributes list synchronized with actual Nailgun attributes
list.
PostgreSQL setup provided by the OpenStack infra is used.
Change-Id: Id317f4db80459e55e62ab7885e0514d024bb7296
Closes-Bug: #1551249
Wrong source for Fuel version was used in the OSWL report.
We used the version of Fuel installed on master node, instead of
cluster version. Thus in case of upgrade master node we had
different fuel versions in the clusters and OSWL reports for
the same master node uid.
Now we are using Fuel version from OSWL.version_info or from
the cluster data. Fuel version from cluster is used also in the
plugins report.
Change-Id: I114beae261686fb6e4d77504dd102b722a43e611
Closes-Bug: #1547565
On the 1M number of rows in the table JSON index begins to
work too slow. We are copy action_name and action_type values
from action_logs.body to separate indexed columns in the
action_logs table.
Redundant JSON serialization/deserealization on the python
level is removed by changing column 'structure' type from
TEXT to JSON in the InstallationStrurcure model. We already
had JSON data in 'structure' in the DB, thus only SQLAlchemy
model is changed.
Change-Id: I415f4fe607f44ac670f63f961af7fb4998f75dd6
Closes-Bug: #1540289
If resource removed and added we show only one record
in the report with status True in is_added, is_removed.
Change-Id: I783414ee4c23d07e02d13e76eb19fe3a2b3db80e
Closes-Bug: #1526320
Version info data handled for OpenStackWorkloadStats. If master node
updated and the version info in InstallationStructure changed we have
correct version info in OSWLs stats report.
Fixes in collector:
- version_info added to DB model
- version_info added to API protocol
- tests for version_info added
Fixes in analytics:
- version_info added to DB model
- version_info fields added to CSV
- field installation structure release info removed from CSV
- version_info building on the fly for old OSWLs added
Partial-Bug: #1525902
Change-Id: I27d9d65517b4d5a7c6125b889fb0d1ba1ea213cf
We can have duplicated OSWLs. They have the same checksum but
different external_ids. Whe should ignore such 'new' OSWLs in the
report generation process.
Change-Id: I71b5ef91eb66a9ce475a78da2485f1ec439acd4e
Closes-Bug: #1524820
gulp-lintspaces of version 0.2.2 is not available any more.
0.3.2 version is used instead the old one.
Change-Id: I504d4973a5e9de3a46c27d58be4f47fab3c35822
Closes-Bug: #1516581
Mocking of werkzeug request args raises error on exiting from
'with' block and was removed from analytics tests.
Issues in tests caused by unbound to session SQLAlchemy objects
inside test request were fixed.
Extracting of dict params from HTTP GET request fixed.
Change-Id: I7215247909a8fcd2e4c74b915670ff373538b186
Closes-Bug: #1514337
Old tooltip format {label}: {value}, {percentage}% confuses
when you see, for instance: kvm: 441, 84%. This can be
recognized as 441,84%
Change-Id: I4ccffc2d08a4d35fa2fc257da014403ff1f09480
Closes-Bug: #1507929
We have 3 services: analytics, collector, migration.
Migration is frozen and going to be removed. Also
migration requires Elasticsearch server for tests,
thus it can't be tested on OpenStack CI.
Pep8 checked on all services.
Project works only on python2.7, thus
gate-fuel-stats-python26 should be removed.
Change-Id: Ic5b6f15903918b4b0850ce0cdc12543925de0cfa
Closes-Bug: #1500073
For providing all reports at once we generate, archive them
and transmit as single file.
Generation and archiving of all reports can take long time
and cause http server connection timeout error.
For preventing such errors used lazy reports generation
and on-the-fly data streming without saving they into files.
- tar archive used instead zip
- streaming of tar archive implemented
Change-Id: I8a71c462cf6ad61ef5b5798770d140866d36edb1
Closes-Bug: #1487356
Andreas Jaeger