Commit Graph

71 Commits

Author SHA1 Message Date
Witek Bedyk f7f3d2e9f8 Deprecate Java implementation
The change adds deprecation notice and updates Readme file.
Also include links to Storyboard, and notes on configuration.

Story: 2005628
Task: 30881
Change-Id: I0d7ae536d02db4bbcb155ad3a5d0b84761a31594
2019-09-17 10:29:34 +02:00
Joseph Davis 4b7e7bf905 Update all columns in metrics on an update to refresh TTL
Cassandra Time to Live (TTL) is applied per-column in a
row, and with a default TTL for the table.  The Cassandra
driver appears to have been written with a misunderstanding
that the TTL was per-row, and as a result the update
statements were written to avoid writing all cells.
But that results in some cells in the row not being updated,
and after their TTL expires they are removed, so a metric
query may result in partial data (updated_at set but
metric_id and created_at missing).

This fix changes the statement used when a new measurement
is received to use the same insert statement for the metric
row and thus refresh the TTL for all the columns.

A follow on patch should be created to remove the Update
statements for Measurements, and consider refactoring/removing
the caching model.

Change-Id: I6b7636a52e8bdb2ce8ad97f839acb6184cd58a8b
Story: 2005832
Task: 35872
2019-07-19 13:04:23 +02:00
Witold Bedyk f48c40f124 Disable ClassPathURLCheck for Java build
This is a workaround for Maven build failures due to bugs in openjdk-8
and surefire plugin until these packages are fixed.

Change-Id: I9115a3567cc1ca7fc59d05e3b74171c031460065
Story: 2004256
Task: 27809
2018-11-06 14:43:01 +01:00
Johannes Grassler 406f521ef1 Java persister config: defaults and robustness
This commit adds defaults for configuration settings where
that makes sense so they can be omitted. Likewise, unknown
settings will now be ignored everywhere except for the
database sections (the relevant configuration classes for that
are in monasca-common).

Change-Id: I35ae02e44a11e088c36943d8c053aac07dbe7524
Story: 2001415
Task: 6115
2018-01-10 17:31:19 +01:00
Zuul 7d852c9612 Merge "Upper pom version to 1.3.0" 2017-12-20 14:34:17 +00:00
James Gu 2cd175995a Upper pom version to 1.3.0
Increases pom version to 1.3.0 after adding cassandra integration.

Depends-On: I0c6c1d027362a0babb5d7a5473c93afc3a72bb09
Change-Id: I1f8ec63e33b8949e477afbd6a8f70f4f26b18ed7
2017-12-19 13:11:41 -08:00
James Gu 20337572e3 Add Cassandra db support
Added cassandra db plugins in persister, inlcuding Java and
python version.

Change-Id: I4bbe48f8fe550385de6ed3e14120850a8542c7c9
story: 2001231
task: 5758
2017-12-15 17:51:26 +00:00
Witold Bedyk ba381b409b Release version 1.2.1 for Java
Depends-On: Ib3da5c9e1f6e5e2d6f77269129bd769179bfd3be
Depends-On: I6e4b73525b3c32c2bdd04c63ae73bcb4c50b5447
Change-Id: I72c9fac6943ae5b2c3b367ed9fe5c884c492b48a
2017-02-15 13:49:25 +00:00
Witold Bedyk 16fb6bb207 Change Java version to 1.2.0
Depends-On: Icce74fd6d7e674f408277e328468bedbc3d0aa6b
Change-Id: I5c2e2f40b61c598528ccf604ff0fd50fbf9865ae
2016-12-15 16:59:28 +00:00
Craig Bryant 0529f0ca3a Check value meta length before writing to Vertica
If value meta length is larger than the size of the Vertica column,
the write will fail and cause the persister to die. Check the length
and drop the value meta if too large. This should not occur as the
value meta is validated by the API before being written to Kafka,
but this is a cheap check and ensures the persister will stay up if
some application starts writing directly to the Kafka queue

Change-Id: I893d02751217beb6ba5a88625f20448c45a1b376
2016-08-19 12:27:20 -06:00
Ryan Brandt 7b4b41ff28 Pass link and lifecycle state in state transitions
Requires changes to monasca-thresh

Change-Id: I93113bc854629d45f34cdff9205ee97f353f999a
2016-01-27 14:27:08 -07:00
Deklan Dieterly 852b42f3c7 Upgrade to Kafka 0.8.2.2
Upgrade Kafka to current stable release - 0.8.2.2.
Upgrade Scala version to 2.11.

Change-Id: Ib05b5d44ba06d88ed2147f9b24d6566578c0f87a
2016-01-04 10:13:25 -07:00
Deklan Dieterly 0d8f1b6328 Make persister shutdown on error
Refactor code to handle errors better. Persister shuts down faster
and more reliably.

On shutdown, messages in buffer will be dropped. On restart,
messages will be read from Kafka and processed again.

Fixes problem with persister not shutting down and restarting 
when a vertica node goes down.

Change-Id: I60b4ba6e06e69e68878f8400e360250e3608e5a5
2015-12-14 15:48:24 +00:00
Jenkins 37832f8477 Merge "Validate metricsString does not exceed max length" 2015-10-20 17:38:08 +00:00
venkatamahesh e947a28c01 Change repositories from stackforge to openstack
Change-Id: I9b0afa3c48fcc29335f5c7d4305b768dc2084614
2015-10-19 09:25:20 +05:30
Roland Hochmuth e6375f6ccc Validate metricsString does not exceed max length
Change-Id: I3d4bf5f2ec8386681d06d26582d9d7267f50e208
2015-10-15 23:56:18 -06:00
Jenkins aeae3262bf Merge "Reconcile example and mini-mon configs" 2015-10-16 02:32:50 +00:00
Deklan Dieterly f88f71c51f Store {} when value_meta is empty
Always store {} when value_meta is empty for Influxdb.

Needed because the Java Influxdb driver is
capable of storing an empty string, but the
Python Influxdb driver is not.

Change-Id: Id6b7b795454afc1a4ab2e6230c6a9d3bd94790f7
2015-10-13 17:06:11 -06:00
Ben Motz cb99a66ee1 Reconcile example and mini-mon configs
Bring contents of the example config and the mini-mon config into
line, add a note on log formatting, and a note in the README about
the sample configurations.

Change-Id: I47794b600d44a87f58825d1e11099c272f06d7aa
2015-10-01 17:23:20 +01:00
Deklan Dieterly 452c020f63 Enhance fail fast on error functionality
Make persister fail faster on errors.
Use one executor service for all perister threads.
Shutdown all threads in thread executor service on any error.
Catch java.lang.Throwable to catch java.lang.Error as well as java.lang.Exception.

Change-Id: I0dc421cf6bb4ab3f52c47e97b7f396483283b561
2015-08-19 08:37:34 -06:00
Jenkins 53fd650296 Merge "Fix java file licenses" 2015-07-28 19:18:36 +00:00
Craig Bryant cc2e5ccdb0 Fix java file licenses
Add it to the one file without a license

Use a consistent license for the other files

Change-Id: I68910584684158bbc400cbbfd843493f0607cce5
2015-07-26 22:14:47 -06:00
Deklan Dieterly e4b5c69e7c Trivial change to get Zuul to push a new jar
2 changes happened too close in time and one resulting jar overwrote the other.

Change-Id: Ie317e29ae3c990dc231c778321ddd79e26820a2c
2015-07-06 15:36:53 -06:00
Jenkins d155a68aaa Merge "Reenable changes from https://review.openstack.org/#/c/193590/" 2015-07-06 14:28:07 +00:00
Deklan Dieterly 509039787e Convert nulls to empty strings or empty JSON
Check all fields that could be null and convert them to empty strings or empty JSON.
Drop events that have null tenant ids or null regions.

Change-Id: I1a0113d9dc1cd9a2662c0dd3e11ce9187c59ab4e
2015-06-25 10:52:56 -06:00
Tim Kuhlman 1921bdd7b7 Reenable changes from https://review.openstack.org/#/c/193590/
Change-Id: Iab74d7906db4a51f4423606641c52af02f312677
2015-06-22 14:47:09 -06:00
Tim Kuhlman 1c2c8bcb7a Reverted influxdb 0.9.0 changes as the Ansible is not yet ready
The original changes are at https://review.openstack.org/#/c/193590/

Change-Id: I032423e7ca65b8bb280b53e4b254e6660b5dd69b
2015-06-22 09:26:06 -06:00
Deklan Dieterly e759423572 Fixes for Influxdb 0.9 latest stable
change 'name' to 'measurement'.
Accept 200 or 204 return on writes.
change 'timestamp' to 'time'.
add precision.

Change-Id: I4f485b6b2b88db75a86b3822fd46eeea04e5e028
2015-06-19 13:18:07 -06:00
Deklan Dieterly 31bb8ba21d Always store value_meta
If value_meta is null or blank, store an empty string.

Change-Id: I6286d9b2f30b6ee6f8839c80c690721af09c5e75
2015-06-02 07:24:18 -06:00
Deklan Dieterly 38c00a3abd Fix jah-1654
Write sub_alarms and metrics JSON in snake case instead of camel case.

Change-Id: I57985fd76052c34cdde8f779aca887ccafa7ae47
2015-05-29 13:53:26 -06:00
Deklan Dieterly 5c3b5f100b Use vertica's merge function
Speed up inserts using vertica's merge function

Change-Id: I46f3703782265df106526d349d81da8ee21be4ea
2015-05-07 16:01:07 -06:00
Deklan Dieterly b829a972ce Fail fast
Change-Id: I9e3fb53de7e28fa083d91afd2bf62ed4e39238b4
2015-04-30 07:57:55 -06:00
Jenkins 3f5597c9d3 Merge "Bump the version to 1.1.0" 2015-04-29 03:40:25 +00:00
Craig Bryant fe9e26869c Bump the version to 1.1.0
Change-Id: I4740f5b95b38aa1bcd81677b3fe8095a35a16808
2015-04-28 21:29:40 -06:00
Deklan Dieterly 5f090f9238 Update vertica code
Change-Id: I308f12d3cc1f73c48d08d424e8ed5dada5ae9965
2015-04-28 15:22:49 -06:00
Deklan Dieterly cc67023c13 Improve exception handling
Make exception handling more precise.

Change-Id: Ib7fb3eed5945c12c92a5d871d7d3f6eb72275a76
2015-04-27 09:35:40 -06:00
Deklan Dieterly 1969fb74be Fix flush time calculation
Capture system time in local var.
Add blank lines for consistency.

Change-Id: I703843c9d45ac9d6be6a9e981eeaf9877b036109
2015-04-24 15:38:15 -06:00
Deklan Dieterly 4c4c1d3464 Fix mark read functionality
perculate exception all the way up so that mark read is not called.
flush on shutdown.

Change-Id: Ie8c1b14765a61c0ed68fc83b23e52ef7bd60f4ab
2015-04-24 11:29:18 -06:00
Deklan Dieterly 515e4f65d8 Minor cleanup
Make methods protected.
Extract methods.
Clean up mixed-case in logging.
Remove unnecessary try-catch block.
Remove extra classes.
Consolidate methods in abstract class.

Change-Id: I6a8064998a2a474c9c51e4ea641387c69142fdd7
2015-04-22 20:04:44 -06:00
Jenkins c819c4153c Merge "Cleanup" 2015-04-21 05:22:50 +00:00
Deklan Dieterly 7f634a5dc8 Cleanup
Remove 4 classes.
Standardize logging.
Add threadId to logging.
Remove need for casting.

Change-Id: I7aeff8a0ed9d7ec7afdf0a09863e3b514e5f9d1e
2015-04-20 23:12:35 -06:00
Jenkins 671a2700f1 Merge "Only the jar and sample config in the deb" 2015-04-20 17:33:24 +00:00
Craig Bryant d898cf874a Only the jar and sample config in the deb
Remove control scripts from deb

Update sample config file to be more current

Change-Id: If97d25c0b8e01eb002aadf642dfc5c7912b92c99
2015-04-19 20:51:48 -06:00
Deklan Dieterly 213fc06707 Remove Influxdb 8
Change-Id: I9d2e4460ff1858335a1857969b18f2303e9470b6
2015-04-17 14:06:47 -06:00
Deklan Dieterly 4f138c3d44 Cleanup dependency injection
Change-Id: I2fef7d2059ed1e86782016a4e11d7273e29cb3a0
2015-04-17 09:44:44 -06:00
Deklan Dieterly 1fc3589d8a Allow unicode
Change-Id: Id2fd0e92047389f1b75bc03c2fb910ae07894562
2015-04-17 08:23:51 -06:00
Craig Bryant 26225d6247 AlarmStateTransitionedEvent timestamp now in ms
This will ensure a unique timestamp. Influx V9 will only keep one
entry with the same timestamp

Change-Id: I9c59e3bf15e731c1f636418c0ffad90356010441
2015-04-14 11:17:55 -06:00
Deklan Dieterly 468dd8b031 Add packages
Move classes to appropriate packages

metric
alarmstate

Remove unnecessary continue statement.

Change-Id: I690f05d5426bf2cae8978b0662c1813453a387bb
2015-04-09 09:43:12 -06:00
Deklan Dieterly 16daf95a1d Refactor influxdb and vertica classes and code
Clean up code for cleaner separation.

Change-Id: I30b7d404713eba9ae2ee64be1e4dfbecd8f6c1d2
2015-04-08 14:52:24 -06:00
Deklan Dieterly 0f233cd76b Send request and response data to Influxdb gzip encoded
Depends on https://review.openstack.org/#/c/171285/

Change-Id: Ic85591ac6dfa10992e96ba974eb9b6fd4911a0d9
2015-04-07 17:37:03 +00:00