From d14e2abc17fff2b920b2065843e44751664bb3d9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tomasz=20Tr=C4=99bski?= Date: Mon, 10 Jul 2017 13:24:10 +0200 Subject: [PATCH] Configuration documentation Commits fills in the documentation part that touches configuring monasca-log-api. With this documentation one can learn about possible ways and options needed to have own monasca-log-api properly configured. Story: 2001058 Task: 4783 Change-Id: I8ae3531fff6d70450b187a4945ce68c472db63b6 --- api-ref/source/http_codes.yaml | 34 ++--- doc/source/configuration/configuring.rst | 139 +++++++++++++++++- doc/source/configuration/files.rst | 85 +++++++++++ doc/source/configuration/index.rst | 18 +-- .../{monasca_log_api.rst => options.rst} | 6 +- doc/source/configuration/sample.rst | 23 +-- 6 files changed, 263 insertions(+), 42 deletions(-) create mode 100644 doc/source/configuration/files.rst rename doc/source/configuration/{monasca_log_api.rst => options.rst} (61%) diff --git a/api-ref/source/http_codes.yaml b/api-ref/source/http_codes.yaml index 756e95b8..9e12b04b 100644 --- a/api-ref/source/http_codes.yaml +++ b/api-ref/source/http_codes.yaml @@ -2,12 +2,12 @@ 200: la_up: | - API is up and running + API is up and running. 204: default: | - Normal response error, everything went as expected (or even better). + Normal response code, everything went as expected (or even better). la_up: | - API is up and running + API is up and running. 400: default: | Sent data was malformed. @@ -25,32 +25,32 @@ Sent body is too large to be processed. 422: default: | - Send data could not be processed properly + Send data could not be processed properly. no_dims: | - Dimension are required + Dimensions are required. dim_name_too_long: | - Dimension name {name} must be 255 characters or less + Dimension name {name} must be 255 characters or less. dim_name_underscore: | - Dimension name {name} cannot start with underscore (_) + Dimension name {name} cannot start with underscore (_). dim_name_forbidden_chars: | - Dimension name {name} may not contain: "> < = { } ( ) \' " , ; &" + Dimension name {name} may not contain: "> < = { } ( ) \' " , ; &". dim_name_empty: | - Dimension name cannot be empty + Dimension name cannot be empty. dim_value_too_long: | - Dimension value {value} must be 255 characters or less + Dimension value {value} must be 255 characters or less. dim_value_empty: | - Dimension value cannot be empty + Dimension value cannot be empty. app_type_too_long: | - Application type {type} must be {length} characters or less + Application type {type} must be {length} characters or less. log_no_msg: | - Log property should have message + Log property must have message. bad_envelope: | - Failed to create Envelope + Failed to create an envelope. 503: default: | The server is currently unable to handle the request due to a - temporary overload or scheduled maintenance, which will likely be - alleviated after some delay. + temporary overload or scheduled maintenance. + This will likely be alleviated after some delay. no_health: - API is running but problems with peripheral components have been spotted + API is running but there are problems with peripheral components. diff --git a/doc/source/configuration/configuring.rst b/doc/source/configuration/configuring.rst index 3992ff59..bf8b453f 100644 --- a/doc/source/configuration/configuring.rst +++ b/doc/source/configuration/configuring.rst @@ -1,4 +1,139 @@ .. _basic-configuration: -Basic Configuration -=================== +----------- +Configuring +----------- + +monasca-log-api has several configuration options. Some of them +are inherited from oslo libraries, others can be found in the monasca-log-api +codebase. + +The entire configuration of monasca-log-api is defined in +configuration files. + +.. note:: This is intended behaviour. One of possible ways to deploy + monasca-log-api is to use **gunicorn**. Unfortunately gunicorn's + argument parsing clashes with oslo's argument parsing. + This means that gunicorn reports the CLI options of + oslo as unknown, and vice versa. + +There are 3 configuration files. For more details on the configuration +options, see :ref:`here `. + +Configuring Keystone Authorization +---------------------------------- + +Keystone authorization (i.e. verification of the token associated +with a request) is a critical part of monasca-log-api. +It prevents from unauthorized access and provides the isolation +needed for multi-tenancy. + +The configuration for ``keystonemiddleware`` should either be provided in +``log-api.conf`` or in a file in one of the configuration directories. +For more details about configuration options, check +`here `_. + +Configuring Log Publishing +-------------------------- + +monasca-log-api sends all logs to the Kafka Message Queue. +Proper configuration should include: + +* ``kafka_url`` - comma-delimited list of Kafka brokers +* ``topics`` - names of the topics to which the logs will be pushed to +* ``max_message_size`` - maximum message size that can be posted a topic + +The configuration for ``log_publisher`` should either be provided in +``log-api.conf`` or in a file in one of the configuration directories. + +Configuring Healthcheck +----------------------- + +Healthcheck is an essential part of monasca-log-api. +It allows sending HTTP requests and getting knowledge about the +availability of the API. Configuration of healthcheck includes: + +* ``kafka_url`` - comma-delimited list of Kafka brokers +* ``kafka_topics`` - list of topics that existence is verified by healthcheck + +The configuration for ``kafka_healthcheck`` should either be provided in +``log-api.conf`` or in a file in one of the configuration directories. + +Configuring Monitoring +---------------------- + +monasca-log-api is capable of self-monitoring. This is achieved +through `monasca-statsd `_. +It assumes that there is monasca-agent available on the system node and +that statsd-server has been launched. + +There are several options you may want to tweak if necessary: + +* ``statsd_host``- the host statsd-server is bound to +* ``statsd_port``- the port statsd-server is bound to +* ``statsd_buffer`` - the amount of metrics to buffer in memory before sending + any +* ``dimensions`` - additional dimensions to be sent with all + metrics for this monasca-log-api instance + +The configuration for ``monitoring`` should either be provided in +``log-api.conf`` or in a file in one of the configuration directories. + +Configuring RBAC +---------------- + +At the moment monasca-log-api does not feature RBAC with ``oslo.policies``. +It provides a custom mechanism, however, that can be configured as follows: + +* ``path`` - list of URIs that RBAC applies to +* ``default_roles`` - list of roles that are permitted to access the API +* ``agent_roles`` - list of roles, that if present, means that requests come + from log-agent +* ``delegate_roles`` - list of roles required by log-agent for sending logs + on behalf of another project (tenant) + +The configuration for ``roles_middleware`` should either be provided in +``log-api.conf`` or in a file in one of the configuration directories. + +Configuring Logging +------------------- + +Logging in monasca-log-api is controlled from the single +``log-api-logging.conf`` configuration file. +Here is a short list of several modifications you may want to apply, +based on your deployment: + +* to log INFO to console:: + + [handler_console] + level = INFO + +* to log DEBUG to file:: + + [handler_file] + level = DEBUG + +* to change the log file location:: + + [handler_file] + args = ('/var/log/log-api.log', 'a') + +* if you have an external script for log rotation:: + + [handler_file] + handler = logging.handlers.WatchedFileHandler + args = ('/var/log/log-api.log', 'a') + + That will store up to 5 rotations (each having maximum size + of 100MBs) + + The configuration of ``logging`` should be presented inside + ``log-api-logging.conf`` file and referenced from ``log-api.conf`` using + ``log_config_append`` option. + + If you want to know more about possible ways to save monasca-log-api logs, + feel free to visit: + + * `oslo.log `_ + * `Python HowTo `_ + * `Logging handlers `_ diff --git a/doc/source/configuration/files.rst b/doc/source/configuration/files.rst new file mode 100644 index 00000000..7408daac --- /dev/null +++ b/doc/source/configuration/files.rst @@ -0,0 +1,85 @@ +.. _configuration-files: + +------------------- +Configuration files +------------------- + +Overview of monasca-log-api's configuration files. + +log-api.conf +------------ + +This is the main configuration file of monasca-log-api. +It can be located in several places. During startup, +monasca-log-api searches for it in the following directories: + +* ``~/.monasca`` +* ``~/`` +* ``/etc/monasca`` +* ``/etc`` + +Alternatively, you can roll with a multi-file-based configuration model. +In this case, monasca-log-api searches the configuration files +in the following directories: + +* ``~/.monasca/monasca.conf.d/`` +* ``~/.monasca/log-api.conf.d/`` +* ``~/monasca.conf.d/`` +* ``~/log-api.conf.d/`` +* ``/etc/monasca/monasca.conf.d/`` +* ``/etc/monasca/log-api.conf.d/`` +* ``/etc/monasca.conf.d/`` +* ``/etc/log-api.conf.d/`` + +Regardless of the location, the name of the main configuration file +should always be ``log-api.conf``. For files located +in ``.conf.d`` directories, the name is irrelevant, but it should +indicate the file content. + +For example, when guring keystone communication. The +`keystonemiddleware `_ +configuration would be, therefore, located in, for example, +``/etc/log-api.conf.d/keystonemiddleware.conf`` + +A sample of this configuration file is also available +:ref:`here ` + +log-api-logging.conf +-------------------- + +This file contains the logging setup for monasca-log-api. It should be +referenced from ``log-api.conf`` using, for example, +the following code snippet:: + + [DEFAULT] + log_config_append = /etc/monasca/log-api-logging.conf + +A sample of this configuration file is also available +:ref:`here ` + +log-api-paste.ini +----------------- + +This file contains the `PasteDeploy `_ +configuration. It describes all pipelines that are running within a single +instance of monasca-log-api. + +There is nothing you should try and modify in this file, +apart from enabling/disabling ``oslo_middleware.debug:Debug``. + +To enable ``oslo_middleware.debug:Debug`` for ``Log v3`` pipeline, +``log-api-paste.ini`` should contain code similar to this one:: + + [composite:main] + use = egg:Paste#urlmap + /v3.0: la_api_v3 + + [pipeline:la_api_v3] + pipeline = debug {{ other pipeline members }} + + [filter:debug] + paste.filter_factory = oslo_middleware.debug:Debug.factory + +This particular filter might be useful for examining the +WSGI environment during troubleshooting or local development. + diff --git a/doc/source/configuration/index.rst b/doc/source/configuration/index.rst index 9f814c1c..180f2512 100644 --- a/doc/source/configuration/index.rst +++ b/doc/source/configuration/index.rst @@ -1,19 +1,17 @@ .. _configuring: -============ -Cofiguration -============ +============= +Configuration +============= -This section provides a list of all possible options for each -configuration file. Refer to :ref:`basic-configuration` for a -detailed guide in getting started with various option settings. - -monasca-log-api uses the following configuration files -for its various services. +This section describes the configuration settings that can be specified. +Refer to :ref:`basic-configuration` for more details on the +available settings. .. toctree:: :maxdepth: 1 configuring - monasca_log_api + files + options sample diff --git a/doc/source/configuration/monasca_log_api.rst b/doc/source/configuration/options.rst similarity index 61% rename from doc/source/configuration/monasca_log_api.rst rename to doc/source/configuration/options.rst index a138f8e7..f7617956 100644 --- a/doc/source/configuration/monasca_log_api.rst +++ b/doc/source/configuration/options.rst @@ -1,8 +1,8 @@ .. _monasca-log-api.conf: --------------------- -monasca-log-api.conf --------------------- +------- +Options +------- .. show-options:: :config-file: config-generator/monasca-log-api.conf diff --git a/doc/source/configuration/sample.rst b/doc/source/configuration/sample.rst index 02807e38..144a19b0 100644 --- a/doc/source/configuration/sample.rst +++ b/doc/source/configuration/sample.rst @@ -1,25 +1,28 @@ .. _sample-configuration: -==================================== -monasca-log-api sample configuration -==================================== +------- +Samples +------- -The following are sample configuration files for monasca-log-api and -utilities. These are generated from code -(apart from logging conf and paste conf) and reflect the current state +The following sections show sample configuration files for monasca-log-api and +related utilities. These are generated from the code +(apart from the samples for logging and paster) and reflect the current state of code in the monasca-log-api repository. -Sample configuration for monasca-log-api ----------------------------------------- +.. _sample-configuration-api: + +Sample Configuration For Application +------------------------------------ This sample configuration can also be viewed in `log-api.conf.sample <../_static/log-api.conf.sample>`_. .. literalinclude:: ../_static/log-api.conf.sample +.. _sample-configuration-logging: -Sample configuration for logging +Sample Configuration For Logging -------------------------------- This sample configuration can also be viewed in `log-api-logging.conf @@ -28,7 +31,7 @@ This sample configuration can also be viewed in `log-api-logging.conf .. literalinclude:: ../../../etc/monasca/log-api-logging.conf -Sample configuration for paste +Sample Configuration For Paste ------------------------------ This sample configuration can also be viewed in `log-api-paste.ini