This commit updates hacking version to 1.1.x and fixes related
pep8 issues.
Also added pycodestyle in test-requirements
Story: 2004930
Task: 29318
Co-Authored-By: Akhil Jain <akhil.jain@india.nec.com>
Change-Id: Id3ad30d23b902ee6f7277f7ec20d7d523df232f6
Quotes around {posargs} cause the entire string to be combined into one
arg that gets passed to stestr. This prevents passing multiple args
(e.g. '--concurrency=16 some-regex')
Change-Id: Ib09247cde72c4c2270a8a1360d9dc10fcfd6d4fe
Closes-Bug:#1801817
Using "python setup.py check -r -s" method of checking the package has
been deprecated with the new recommendation to build the sdist and
wheel, then running "twine check" against the output.
Luckily, there is already a job that covers this that only runs when the
README, setup.py, or setup.cfg files change, making running this in the
pep8 job redundant. This covered by the test-release-openstack-python3
that is defined in the publish-to-pypi-python3 template.
More details can be found in this mailing list post:
http://lists.openstack.org/pipermail/openstack-dev/2018-October/136136.html
Change-Id: Iea823472cc45ce65e11eb9b096aa5d1d943f5520
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Import build-openstack-sphinx-docs job and convert to new
openstack-tox-docs job - using the build-openstack-docs-pti template.
Add tox docs environment for building docs.
Use templates for lower constraints jobs, sort template list
alphabetically.
Change-Id: I4888010e1b473d93f3cc48ed00b31da301068db8
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.
We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.
We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.
Change-Id: Iaa7c9b9fca48dd2e091049da711e70395f6ec28e
This commit is a follow-up of "switch to using stestr"
which was merged already [1].
After switch to using stestr, testrepository and .testr.conf
are unnecessary and should be removed.
This patch also switch from "testr" to "stestr" for coverage jobs.
[1] https://review.openstack.org/575430
Change-Id: Ide998f902197ff24e28e998daa8defda090fab3f
According to Openstack summit session [1] stestr is
maintained project to which all Openstack projects
should migrate.
Let's switch it then.
[1] https://etherpad.openstack.org/p/YVR-python-pti
Change-Id: Ic0ddaae3bd0a4ad4e34264429f717a292e17a1c9
Signed-off-by: Charles Short <zulcss@gmail.com>
* set the maximum line length to 100
* cleaned up the codes for pep8
Change-Id: Iab260a4e77584aae31c0596f39146dd5092b807a
Signed-off-by: Amir Mofakhar <amofakhar@op5.com>
Create a tox environment for running the unit tests against the lower
bounds of the dependencies.
Create a lower-constraints.txt to be used to enforce the lower bounds
in those tests.
Add openstack-tox-lower-constraints job to the zuul configuration.
See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
for more details.
Change-Id: Ib536056a9b1076895f232e84823e853afd4d77a1
Depends-On: https://review.openstack.org/555034
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Following changes were required:
1.)
By default the pre-built distribution
for Spark 2.2.0 is compiled with Scala 2.11.
monasca-transform requires Spark compiled with
Scala 2.10 since we use spark streaming to
pull data from Kafka and the version of Kafka
is compatible with Scala 2.10.
The recommended way is to compile Spark
with Scala 2.10, but for purposes of devstack
plugin made changes to pull the required jars
from mvn directly.
(see SPARK_JARS and SPARK_JAVA_LIB variables in
settings)
All jars get moved to
<SPARK_HOME>/assembly/target/assembly/
target/scala_2.10/jars/
Note: <SPARK_HOME>/jars gets renamed
to <SPARK_HOME>/jars_original.
spark-submit defaults to assembly location
if <SPARK_HOME>/jars directory is missing.
2.) Updated start up scripts for spark
worker and spark master with a new env variable
SPARK_SCALA_VERSIOn=2.10. Also updated
PYTHONPATH variable to add new
py4j-0.10.4-src.zip file
3.) Some changes to adhere to deprecated pyspark
function calls which were removed in Spark 2.0
Change-Id: I8f8393bb91307d55f156b2ebf45225a16ae9d8f4
Some of the available checks are disabled by default, like:
[H106] Don’t put vim configuration in source files
[H203] Use assertIs(Not)None to check for None
Change-Id: I3d9eba508155f7ef8eba5185b27bd2673fe95d0f
To be co-installable with other OpenStack projects and have consistent
testing it's a good idea (tm) to use constraints when creating tox
environments.
Change-Id: Ic53cd023d9007321441bafd6c8e4c9b4f39e58b3
Changing devstack environment vagrant box and
also rename the devstack VM to 'devstack'
from 'pg-tips'
Also fixing all the tests that were broken when
they were moved from tests/unit to tests/functional
with this review
https://review.openstack.org/#/c/400237/
Update devstack README with a section called
Development workflow for monasca-transform with
steps developers can take to develop and run
tests.
Change-Id: I11678148ba2bcb96eb3e2a522176683dc8bca30a
Moved what were called unit tests to the functional directory as there is
more work to be done to make them self supporting.
Added sphinx to the test-requirements along with testr deps.
Adjusted the tox.ini to suit.
Added doc source tree with the minimal set of files to allow sphinx
success.
Change-Id: Ic64c2ccd01778bb0d1142d12ca43b2346bdc7aa7
Breaking down the aggregation into two stages.
The first stage aggregates raw metrics frequently and is
implemented as a Spark Streaming job which
aggregates metrics at a configurable time interval
(defaults to 10 minutes) and writes the intermediate
aggregated data, or instance usage data
to new "metrics_pre_hourly" kafka topic.
The second stage is implemented
as a batch job using Spark Streaming createRDD
direct stream batch API, which is triggered by the
first stage only when first stage runs at the
top of the hour.
Also enhanced kafka offsets table to keep track
of offsets from two stages along with streaming
batch time, last time version row got updated
and revision number. By default it should keep
last 10 revisions to the offsets for each
application.
Change-Id: Ib2bf7df6b32ca27c89442a23283a89fea802d146
The monasca-transform is a new component in Monasca that
aggregates and transforms metrics.
monasca-transform is a Spark based data driven aggregation
engine which collects, groups and aggregates existing individual
Monasca metrics according to business requirements and publishes
new transformed (derived) metrics to the Monasca Kafka queue.
Since the new transformed metrics are published as any other
metric in Monasca, alarms can be set and triggered on the
transformed metric, just like any other metric.
Co-Authored-By: Flint Calvin <flint.calvin@hp.com>
Co-Authored-By: David Charles Kennedy <david.c.kennedy@hpe.com>
Co-Authored-By: Ashwin Agate <ashwin.agate@hp.com>
Implements: blueprint monasca-transform
Change-Id: I0e67ac7a4c9a5627ddaf698855df086d55a52d26
Witek Bedyk