Deprecate collector

To optimise performance, We have to update ceilometer architecture.
The most important step is to deprecate collector. From now on, we
can configure multiple publishers in pipeline for pushing data to
internal or external system.

Highlight using multiple dispatchers.

Change pipeline publisher and disable ceilometer-collector by default.

Co-Authored-By: gordon chung <gord@live.ca>
Change-Id: I25a6e0b9221844adb4412f1829d9e290b6e198a3

ceilometer/collector.py

@@ -70,6 +70,11 @@ class CollectorService(cotyledon.Service):
         self.event_listener = None
         self.udp_thread = None
 
+        import debtcollector
+        debtcollector.deprecate("Ceilometer collector service is deprecated."
+                                "Use publishers to push data instead",
+                                version="9.0", removal_version="10.0")
+
     def run(self):
         if self.conf.collector.udp_address:
             self.udp_thread = utils.spawn_thread(self.start_udp)

devstack/plugin.sh

@@ -267,9 +267,16 @@ function _ceilometer_configure_storage_backend {
         die $LINENO "Unable to configure unknown CEILOMETER_BACKEND $CEILOMETER_BACKEND"
     fi
 
+    if [ "$CEILOMETER_BACKEND" = 'mysql' ] || [ "$CEILOMETER_BACKEND" = 'postgresql' ] || [ "$CEILOMETER_BACKEND" = 'mongodb' ]; then
+        sed -i 's/gnocchi:\/\//database:\/\//g' $CEILOMETER_CONF_DIR/event_pipeline.yaml $CEILOMETER_CONF_DIR/pipeline.yaml
+    fi
+
     # configure panko
     if is_service_enabled panko-api; then
         iniadd $CEILOMETER_CONF DEFAULT event_dispatchers panko
+        if ! grep -q 'panko' $CEILOMETER_CONF_DIR/event_pipeline.yaml ; then
+            echo '          - direct://?dispatcher=panko' >> $CEILOMETER_CONF_DIR/event_pipeline.yaml
+        fi
     fi
 
     _ceilometer_drop_database

devstack/settings

@@ -3,8 +3,6 @@
 enable_service ceilometer-acompute ceilometer-acentral
 # Notification Agent
 enable_service ceilometer-anotification
-# Data Collector
-enable_service ceilometer-collector
 
 # Default directories
 CEILOMETER_DIR=$DEST/ceilometer

doc/source/configuration.rst

@@ -188,7 +188,8 @@ data should be sent after the possible transformations. The names of the
 publishers should be the same as the related names of the plugins in
 :file:`setup.cfg`.
 
-The default configuration can be found in `pipeline.yaml`_.
+The default configuration can be found in `pipeline.yaml`_. For more details about
+how to configure dispatchers under publisher in the :ref:`dispatcher-configuration`.
 
 .. _pipeline.yaml: https://git.openstack.org/cgit/openstack/ceilometer/tree/etc/ceilometer/pipeline.yaml
 

doc/source/install/custom.rst

@@ -64,6 +64,7 @@ For polling agent using ceilometer-polling.conf, settings like::
 
 Doing this, it's easy to listen/receive data from multiple internal and external services.
 
+..  _dispatcher-configuration:
 
 Using multiple dispatchers
 ==========================

doc/source/install/manual.rst

@@ -20,6 +20,11 @@
  Installing Manually
 =====================
 
+.. note::
+
+   Ceilometer collector service is deprecated. Configure dispatchers under publisher
+   in pipeline to push data instead. For more details about how to configure
+   dispatchers in the :ref:`dispatcher-configuration`.
 
 Storage Backend Installation
 ============================

etc/ceilometer/event_pipeline.yaml

@@ -9,4 +9,4 @@ sinks:
     - name: event_sink
       transformers:
       publishers:
-          - notifier://
+          - gnocchi://

etc/ceilometer/pipeline.yaml

@@ -35,7 +35,7 @@ sinks:
     - name: meter_sink
       transformers:
       publishers:
-          - notifier://
+          - gnocchi://
     - name: cpu_sink
       transformers:
           - name: "rate_of_change"
@@ -46,7 +46,7 @@ sinks:
                     type: "gauge"
                     scale: "100.0 / (10**9 * (resource_metadata.cpu_number or 1))"
       publishers:
-          - notifier://
+          - gnocchi://
     - name: cpu_delta_sink
       transformers:
           - name: "delta"
@@ -55,7 +55,7 @@ sinks:
                     name: "cpu.delta"
                 growth_only: True
       publishers:
-          - notifier://
+          - gnocchi://
     - name: disk_sink
       transformers:
           - name: "rate_of_change"
@@ -70,7 +70,7 @@ sinks:
                         unit: "\\1/s"
                     type: "gauge"
       publishers:
-          - notifier://
+          - gnocchi://
     - name: network_sink
       transformers:
           - name: "rate_of_change"
@@ -85,4 +85,4 @@ sinks:
                         unit: "\\1/s"
                     type: "gauge"
       publishers:
-          - notifier://
+          - gnocchi://

releasenotes/notes/deprecate-ceilometer-collector-b793b91cd28b9e7f.yaml

@@ -0,0 +1,12 @@
+---
+features:
+  - |
+    Because of deprecating the collector, the default publishers in
+    pipeline.yaml and event_pipeline.yaml are now changed using database
+    instead of notifier.
+deprecations:
+  - |
+    Collector is no longer supported in this release. The collector
+    introduces lags in pushing data to backend. To optimize the
+    architecture, Ceilometer push data through dispatchers using
+    publishers in notification agent directly.