This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.
This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.
This update should result in no functional change.
For more information see the thread at
http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html
Change-Id: Iefd089227b889e88b6053bd752f0e9f0434b6b72
refresh_monasca_script.sh is useful for
development in a devstack environment.
Enhanced the script to
* start and stop monasca-transform process
running in a screen session
* added more hardening to catch for errors
and exit if any command fails.
* added debugging statements help track
down any errors when the script is run.
Change-Id: Idab02d555eed192d8242c870017955b935532c3d
Changing devstack environment vagrant box and
also rename the devstack VM to 'devstack'
from 'pg-tips'
Also fixing all the tests that were broken when
they were moved from tests/unit to tests/functional
with this review
https://review.openstack.org/#/c/400237/
Update devstack README with a section called
Development workflow for monasca-transform with
steps developers can take to develop and run
tests.
Change-Id: I11678148ba2bcb96eb3e2a522176683dc8bca30a
There is a desire to use Monasca Transform to aggregate
kubernetes metrics. This change is a start in that
direction. The transformation specs in the tests
folder now include some representative aggregations
of some kubernetes metrics.
This commit also includes some changes to get
first_attempt_at_spark_test.py working again
after being moved from the unit test folder to the
functional test folder.
Change-Id: I038ecaf42e67d5c994980991232a2a8428a4f4e3
A simple robustness tweak to remove any trailing character,
like a / in "http://web-proxy.example.com:8080/", from
the parsed proxy port.
Change-Id: Iac75f8ae431b923b86dbd5835b1c48ca090e14e7
The devstack plugin carries usage of `sudo -u` which doesn't seem to
work in the ci environment. Replace it with sudo followed by
appropriate permissions changes.
Use ${DEST} instead of literal /opt/stack to fit with gate usage.
Enabled monasca-api plugin in the settings and the required monasca
services along with zookeeper.
Change-Id: I6effede4ac9a2faf1c44eff9cd96bbf9c924d703
Add sudo to command removing monasca-transform directory for renewal.
Without this it fails as the .git directories are permissioned elsehow
from the default user.
Change-Id: I4c9d3ff7ed1c2b841be82a66ec594033eec2af15
This needs to be the admin project id so for devstack this needs to be written
to the configuration file once the users/projects etc are created and
identifiable.
Add a similar process to the refresh script.
Correct the configuration property name to 'project' rather than using the old
nomencature 'tenant'.
Change-Id: Ib9970ffacf5ee0f7f006722038a1db8024c1385e
The refresh script, usable in development to update the code executed in
monasca-transform from the developers local copy was broken. When the refresh
was carried out the script copied a fresh configuration file into place,
wiping the changes made by the devstack plugin during deployment. This
resulted in an incorrect ip address being used for the kafka broker.
Change-Id: I606dad9f6c8a59006e37560f5bb5fdceb51266d4
The metric filter allows metrics to be filtered before aggregation,
for example to exclude metrics from certain environments or nodes
from being included where they are outside the scope of data
aggregation. This is a powerful feature but not appropriate for
all scenarios (e.g. devstack). So default the filter to empty.
Change-Id: Icb790a0ec41133bfac54244aae8782a5cc665186
There is a variable that tells what is the database (mysql)
password. However plugin.sh is using a hardcoded password.
Commits provides using DATABASE_PASSWORD variable (the same
one as devstack is using) + defines a variable for m-transform
user - MONASCA_TRANSFORM_DB_PASSWORD
Change-Id: I9fc8296ef31b22564f2cf1536e51ab3abc8c9dc9
Breaking down the aggregation into two stages.
The first stage aggregates raw metrics frequently and is
implemented as a Spark Streaming job which
aggregates metrics at a configurable time interval
(defaults to 10 minutes) and writes the intermediate
aggregated data, or instance usage data
to new "metrics_pre_hourly" kafka topic.
The second stage is implemented
as a batch job using Spark Streaming createRDD
direct stream batch API, which is triggered by the
first stage only when first stage runs at the
top of the hour.
Also enhanced kafka offsets table to keep track
of offsets from two stages along with streaming
batch time, last time version row got updated
and revision number. By default it should keep
last 10 revisions to the offsets for each
application.
Change-Id: Ib2bf7df6b32ca27c89442a23283a89fea802d146
By default devstack was cloning master when setting up devstack.
This change has the devstack plugin clone the local working branch so
that uncommitted changes can be tested.
Change-Id: If963ea1d703cd9d4d96665e75db08834cf473b33
The patch allowing us to disable other monasca components is no
longer necessary so remove it from our vagrant process.
Change-Id: I490b557de05ce6696d35783d7cc55db9d0044fbd
apache download source to use the archive site to ensure that
the dependency package does not disappear. Also brought the
vagrant environment inline with monasca-api (i.e., use the
same values for private network, add substitution for kafka
brokers ip address to the conf). Also parameterised
dependency sources (i.e., added settings to parameterise the
maven and apache repositories for the devstack plugin).
Change-Id: If9f0e2ed16bbfcd62152d29e5c7c86f5d555f9aa
The monasca-transform is a new component in Monasca that
aggregates and transforms metrics.
monasca-transform is a Spark based data driven aggregation
engine which collects, groups and aggregates existing individual
Monasca metrics according to business requirements and publishes
new transformed (derived) metrics to the Monasca Kafka queue.
Since the new transformed metrics are published as any other
metric in Monasca, alarms can be set and triggered on the
transformed metric, just like any other metric.
Co-Authored-By: Flint Calvin <flint.calvin@hp.com>
Co-Authored-By: David Charles Kennedy <david.c.kennedy@hpe.com>
Co-Authored-By: Ashwin Agate <ashwin.agate@hp.com>
Implements: blueprint monasca-transform
Change-Id: I0e67ac7a4c9a5627ddaf698855df086d55a52d26