Commit Graph

22 Commits

Author SHA1 Message Date
Monsyne Dragon ec95dd9790 Add JSON format for AtomPub Handler
Add JSON content format to AtomPub pipeline handler.
Also add optional args to specify title and categories
of atom post.

Change-Id: I67c133fb61b2a5248090ff562aad07a436285a8f
2015-08-26 21:45:33 +00:00
Monsyne Dragon b5191bb105 Add AtomPub handler
Adds AtomPub handler to send events to an atom server as a feed.

Also, fixed Usage driver to save new events directly from pipeline.
Fixed minor (but annoyingly hard to diagnose) bug in db layer
    where it didn't recognize 'long' values.

Change-Id: I792e9f77accfea4583fd75805a9ff0d946827df8
2015-08-19 06:52:04 +00:00
Monsyne Dragon 665c9ad328 Add optional database trimming for old events.
Alows you to trim events older than a configurable time from
the events database.

(Remerging to fix timex parse error)

Change-Id: Iaa290705815d1c3ac23c2ca7370a5d705f1f834c
2015-08-17 17:51:05 +00:00
cindy oneill 0420542cfd Add or Delete Trigger Definitions to an existing TriggerManager and PipelineManager
Change-Id: I0dd97b140d1e175492de507eb1ebaf99d18c5eae
2015-05-04 18:32:55 -06:00
Levi Blackstone 8892801205 Add PEP8 check and fix related issues
- Add PEP8 section to tox.ini
- Add hacking to requirements to enforce OpenStack style requirements
- Fix large number of formatting issues flagged by flake8 check
- Add copyright notices to all remaining files
- Fix bug in trigger_manager related to logging calls
- Add .gitignore file

Change-Id: I755ab9c8bcc436836f9006fcd671408cc77214c4
2015-05-04 10:33:25 -05:00
Sandy Walsh a2710a98d9 Only consider the last interesting event when checking traits.
Was previously looking at the last .start event when
comparing important traits against the .exists event.
Now it will look at the last important .end event it found (if any).

For delete operations, it will be the last event in the chain.
This may need to change to a specific event as well, like
compute.instance.delete.end or something.

Change-Id: Ie85e69716cd730c3546d06bae28e52e0c301940f
2015-04-17 07:39:44 -07:00
Sandy Walsh 4cb16327c7 NotabeneHandler no longer formats notifications.
It just passes it through verbatim. Only required key is 'event_type'.

Change-Id: Idcd911c9f09d405d57ecaf5b14b72bfebe2a179a
2015-03-25 12:45:04 -07:00
Eddie Sheffield e08924f1f8 Add functionality for counting events
Similar to the existing streams functionality, this patch adds
the ability to get just a count of filtered events rather than
the full list of events.

There are corresponding changes in quince, quincy, and klugman.

Change-Id: Ic14845db48170a8481c764f5f2468f137758bd19
2015-03-20 12:42:00 -04:00
Sandy Walsh 7846f6a20f Add support for find_events()
For use with API. You can find events based on traits, event_type or dates.

Doesn't support wildcarding or anything just yet.

Change-Id: Ide6e13086d50bacf356d8e36b9de257db9d543ca
2015-03-04 09:22:11 -08:00
Sandy Walsh 4875e419a6 Fixes the .verified notification format.
Now it matches the notification format expected by the yagi
CUF handler. Requires expanded event_definition.yaml file
to include extra traits from .exists

Change-Id: I83ba05d73890a578699566648b37697d46b58040
2015-02-27 06:05:41 -08:00
Jenkins 516c2b3d75 Merge "Notabene pipeline handler" 2015-02-10 22:29:26 +00:00
Monsyne Dragon 8044f3e7e2 Add mark/limit paging to find call.
Adds Mark/limit to find_streams for api usage.

The 'mark' is an opaque parameter (called '_mark') added into
each item in the list of returned items (currently a mildly obscured
ID for streams, that could be changed.)

Returned items are sorted in descending order (by ID for streams),
allowing paging from the 'top' to the bottom.
By prefacing the 'mark' param with a '+', you can page in ascending
order as well.

This same paging scheme can be used for a future events api, as well.

Change-Id: Ie358c0d1d8644dac24cad3992188ec22d66c5328
2015-02-10 19:29:07 +00:00
Sandy Walsh ce91d56d51 Notabene pipeline handler
For publishing new events into an exchange.

PipelineHandlers can return new Events for subsequent processing.
However, sometimes we need to publish Notifications. Remember that
Notifications are less restrictive than Events. They can be larger.
They can be nested. They can contain lists.

We store these pending notifications in the handler env variable
so downstream handlers can access them.

The NotabeneHandler takes a configuration variable 'env_keys' which
specifies the env keys to look for. Lists of notifications in these
variables are published to the queue using the connection parameters
supplied.

Errors in transmission are logged and ignored since exceptions during
the commit() phase do not flag the steam as in error.

A sample pipeline definition might look like this:

test_expire_pipeline:
    - logger
    - usage
    - name: notabene
      params:⋅
        host: localhost
        user: guest
        password: guest
        port: 5672
        vhost: /
        library: librabbitmq
        exchange: nova
        exchange_type: topic
        queue_name: monitor.info
        env_keys:
            - usage_notifications

Change-Id: If1958135ad6fbed88e2c18b9fac7efde51ee3113
2015-02-10 07:29:35 -08:00
Sandy Walsh 460f8231ee Add count() support to find_streams
Returns [{'count': num_streams}]

Change-Id: I37e7fc462a14dda793ac5c231439ed7d5e5ee523
2015-01-27 12:02:44 -08:00
Sandy Walsh c5f0eca828 Generate .verified per .exists
Slight change on the old way, which was a single .verified on the
one EOD .exists. Now it generates a .verified for every .exists
seen.

Change-Id: I7bdced936f203cbec4bf8b7ee2b956c388e66ff1
2015-01-19 20:01:42 -08:00
Sandy Walsh 10b21a4e74 Non-EOD .exists usage checks.
Also adds support for warnings. Will generate a
compute.instance.exists.warnings event when there
is something suspicious but not an actual error.

Change-Id: I00f135b20917231a8cf3f603f0954f061de15410
2014-12-12 11:19:50 -08:00
Sandy Walsh e0256a3fdb Support for playback of old events.
Introduces the time_sync object which can sync
time with an external time service (like the time_sync
service in notigen).

This is used for playback of old events. It prevents
premature expiry triggers.

Also some tweaks to the UsageHandler, work in progress
and used for testing.

Change-Id: I45033fbd9c12d98f96816a4f90cf7dc8c915ef51
2014-10-17 12:40:44 -07:00
Sandy Walsh ac49955386 Adding debugger from oahu
Allows for per-trigger-definition debugging. Simply add
debug_level=1 to your trigger definition.

debug_level=2 gives detailed explainations of why your
stream didn't fire or trigger.

This way, you can leave the overall debug level at INFO
and still get details on the stream you're working on.

Yagi worker now uses idle() callback for debug updates.

and minor cleanups for pep8/HACKING

Change-Id: Id0af7a0adbcc47335ad250130958932c708b5c18
2014-10-02 20:50:31 +00:00
Monsyne Dragon db434c24c4 Add ability to purge streams on completion.
default to deleting streams once finished with them (if no error)
2014-09-10 21:43:45 +00:00
Monsyne Dragon 2d47fa6f6e Add reset stream method. 2014-09-08 23:02:52 +00:00
Monsyne Dragon ca0d09f7bc Save newly generated events from pipeline
Save newly created events from pipeline run if pipeline commits.
Refactor trigger manager api wart, move save_event call into add_event
    to make add_event and add_notification symmetric.
2014-09-08 19:57:24 +00:00
Monsyne Dragon a8f373e4bf Added full stream processing, pipeline workers, etc.
Full trigger logic now works.
Added pipeline workers, and test handler.
Added example configs
Lots of unittests.
2014-09-04 01:49:19 +00:00