diff --git a/doc/source/install/mod_wsgi.rst b/doc/source/install/mod_wsgi.rst index 66ebb20e..be08dc0d 100644 --- a/doc/source/install/mod_wsgi.rst +++ b/doc/source/install/mod_wsgi.rst @@ -1,56 +1,56 @@ -.. - Except where otherwise noted, this document is licensed under Creative - Commons Attribution 3.0 License. You can view the license at: - - https://creativecommons.org/licenses/by/3.0/ - - -Installing Karbor API with mod_wsgi -=================================== - -#. Install the Apache Service - - Fedora 21/RHEL7/CentOS7:: - - sudo yum install httpd - - Fedora 22 (or higher):: - - sudo dnf install httpd - - Debian/Ubuntu:: - - apt-get install apache2 - -#. Copy ``etc/apache2/apache-karbor-api.conf`` under the apache sites - - Fedora/RHEL7/CentOS7:: - - sudo cp etc/apache2/apache-karbor-api.conf /etc/httpd/conf.d/apache-karbor-api.conf - - Debian/Ubuntu:: - - sudo cp etc/apache2/apache-karbor-api.conf /etc/apache2/sites-available/apache-karbor-api.conf - -#. Edit ``apache-karbor-api.conf`` according to installation - and environment. - - * Modify the ``WSGIDaemonProcess`` directive to set the ``user`` and - ``group`` values to appropriate user on your server. - * Modify the ``WSGIScriptAlias`` directive to point to the - %KARBOR_BIN_DIR%/karbor-wsgi script. - * Modify the ``Directory`` directive to set the path to the Karbor API - code. - * Modify the ``ErrorLog and CustomLog`` to redirect the logs to the right - directory. - -#. Enable the apache watcher site and reload - - Fedora/RHEL7/CentOS7:: - - sudo systemctl reload httpd - - Debian/Ubuntu:: - - sudo a2ensite apache-karbor-api - sudo apache2ctl -k restart +.. + Except where otherwise noted, this document is licensed under Creative + Commons Attribution 3.0 License. You can view the license at: + + https://creativecommons.org/licenses/by/3.0/ + + +Installing Karbor API with mod_wsgi +=================================== + +#. Install the Apache Service + + Fedora 21/RHEL7/CentOS7:: + + sudo yum install httpd + + Fedora 22 (or higher):: + + sudo dnf install httpd + + Debian/Ubuntu:: + + apt-get install apache2 + +#. Copy ``etc/apache2/apache-karbor-api.conf`` under the apache sites + + Fedora/RHEL7/CentOS7:: + + sudo cp etc/apache2/apache-karbor-api.conf /etc/httpd/conf.d/apache-karbor-api.conf + + Debian/Ubuntu:: + + sudo cp etc/apache2/apache-karbor-api.conf /etc/apache2/sites-available/apache-karbor-api.conf + +#. Edit ``apache-karbor-api.conf`` according to installation + and environment. + + * Modify the ``WSGIDaemonProcess`` directive to set the ``user`` and + ``group`` values to appropriate user on your server. + * Modify the ``WSGIScriptAlias`` directive to point to the + %KARBOR_BIN_DIR%/karbor-wsgi script. + * Modify the ``Directory`` directive to set the path to the Karbor API + code. + * Modify the ``ErrorLog and CustomLog`` to redirect the logs to the right + directory. + +#. Enable the apache watcher site and reload + + Fedora/RHEL7/CentOS7:: + + sudo systemctl reload httpd + + Debian/Ubuntu:: + + sudo a2ensite apache-karbor-api + sudo apache2ctl -k restart diff --git a/doc/source/specs/add-db-manage-purge.rst b/doc/source/specs/add-db-manage-purge.rst index ee5ebf9a..57264ffa 100644 --- a/doc/source/specs/add-db-manage-purge.rst +++ b/doc/source/specs/add-db-manage-purge.rst @@ -1,153 +1,153 @@ -.. - This work is licensed under a Creative Commons Attribution 3.0 Unported - License. - - http://creativecommons.org/licenses/by/3.0/legalcode - -======================= -Karbor db purge utility -======================= - -https://blueprints.launchpad.net/karbor/+spec/clean-deleted-data-in-db - -This spec adds the ability to sanely and safely purge deleted rows from -the karbor database for all relevant tables. Presently, we keep all deleted -rows. I believe this is unmaintainable as we move towards more upgradable -releases. Today, the operators depend on manual DB queries to delete this -data, but this exposes the DB to human errors. - -The goal is to have this be an extension to the `karbor-manage db` command. -Similar specs are being submitted to all the various projects(Cinder, Glance) -that touch a database. - -Problem description -=================== - -Very long lived OpenStack installations will carry around database rows -for years and years. This brings following problems: - -* If deleted data is kept in the DB, the number of rows can grow very large - taking up the disk space of nodes. Larger disk space means more worry - for disaster recovery, long running non-differential backups, etc. - -* Large number of deleted rows also means, an admin or authorized owner - querying for the corresponding rows will get 5xx responses timing out - on the DB, eventually slowing down other queries and API performance. - -* DB upgradeability is a big challenge if the older data style are less - or inconsistent with the latest formats. - -To date, there is no "mechanism" to programmatically purge the deleted -data. - -Proposed change -=============== - -The proposal is to add a "purge" method to DbCommands in -karbor/karbor/cmd/manage.py. This will take a number of days argument, -and use that for a data_sub match Like. -Like:: - - DELETE FROM plans - WHERE deleted != 0 AND deleted_at > data_sub(NOW()...) - -Alternatives ------------- - -Today, this can be accomplished manually with SQL commands, or via script. - -Data model impact ------------------ - -None, all tables presently include a "deleted_at" column. - -REST API impact ---------------- - -None, this would be run from karbor-manage - -CLI impact ----------- - -A new karbor-manage command will be added:: - - karbor-manage db purge - -Security impact ---------------- - -None, This only touches already deleted rows. - -Notifications impact --------------------- - -None - -Other end user impact ---------------------- - -None - -Performance Impact ------------------- - -This has the potential to improve performance for very large databases. -Very long-lived installations can suffer from inefficient operations -on large tables. -This would have negative DB performance impact while the purge is running. - -Other deployer impact ---------------------- - -None - -Developer impact ----------------- - -None - -Implementation -============== - -Assignee(s) ------------ - -Primary assignee: - chenying - -Work Items ----------- - -Implement 'db purge' command. -Add tests to confirm functionality -Add documentation of feature - -Dependencies -============ - -None - -Testing -======= - -The test will be written as such. Three rows will be inserted into a test db. -Two will be "deleted=1", one will be "deleted=0" -One of the deleted rows will have "deleted_at" be NOW(), the other will be -"deleted_at" a few days ago, lets say 10. The test will call the new -function with the argument of "7", to verify that only the row that was -deleted at 10 days ago will be purged. The two other rows should remain. - -Documentation Impact -==================== - -Documentation of this feature will be added to the admin guide and -developer reference. - -References -========== - -This is already discussed and accepted in other OpenStack components, -such as Glance [1] and Cinder [2]. - -[1] https://specs.openstack.org/openstack/glance-specs/specs/mitaka/database-purge.html -[2] https://specs.openstack.org/openstack/cinder-specs/specs/kilo/database-purge.html +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +======================= +Karbor db purge utility +======================= + +https://blueprints.launchpad.net/karbor/+spec/clean-deleted-data-in-db + +This spec adds the ability to sanely and safely purge deleted rows from +the karbor database for all relevant tables. Presently, we keep all deleted +rows. I believe this is unmaintainable as we move towards more upgradable +releases. Today, the operators depend on manual DB queries to delete this +data, but this exposes the DB to human errors. + +The goal is to have this be an extension to the `karbor-manage db` command. +Similar specs are being submitted to all the various projects(Cinder, Glance) +that touch a database. + +Problem description +=================== + +Very long lived OpenStack installations will carry around database rows +for years and years. This brings following problems: + +* If deleted data is kept in the DB, the number of rows can grow very large + taking up the disk space of nodes. Larger disk space means more worry + for disaster recovery, long running non-differential backups, etc. + +* Large number of deleted rows also means, an admin or authorized owner + querying for the corresponding rows will get 5xx responses timing out + on the DB, eventually slowing down other queries and API performance. + +* DB upgradeability is a big challenge if the older data style are less + or inconsistent with the latest formats. + +To date, there is no "mechanism" to programmatically purge the deleted +data. + +Proposed change +=============== + +The proposal is to add a "purge" method to DbCommands in +karbor/karbor/cmd/manage.py. This will take a number of days argument, +and use that for a data_sub match Like. +Like:: + + DELETE FROM plans + WHERE deleted != 0 AND deleted_at > data_sub(NOW()...) + +Alternatives +------------ + +Today, this can be accomplished manually with SQL commands, or via script. + +Data model impact +----------------- + +None, all tables presently include a "deleted_at" column. + +REST API impact +--------------- + +None, this would be run from karbor-manage + +CLI impact +---------- + +A new karbor-manage command will be added:: + + karbor-manage db purge + +Security impact +--------------- + +None, This only touches already deleted rows. + +Notifications impact +-------------------- + +None + +Other end user impact +--------------------- + +None + +Performance Impact +------------------ + +This has the potential to improve performance for very large databases. +Very long-lived installations can suffer from inefficient operations +on large tables. +This would have negative DB performance impact while the purge is running. + +Other deployer impact +--------------------- + +None + +Developer impact +---------------- + +None + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + chenying + +Work Items +---------- + +Implement 'db purge' command. +Add tests to confirm functionality +Add documentation of feature + +Dependencies +============ + +None + +Testing +======= + +The test will be written as such. Three rows will be inserted into a test db. +Two will be "deleted=1", one will be "deleted=0" +One of the deleted rows will have "deleted_at" be NOW(), the other will be +"deleted_at" a few days ago, lets say 10. The test will call the new +function with the argument of "7", to verify that only the row that was +deleted at 10 days ago will be purged. The two other rows should remain. + +Documentation Impact +==================== + +Documentation of this feature will be added to the admin guide and +developer reference. + +References +========== + +This is already discussed and accepted in other OpenStack components, +such as Glance [1] and Cinder [2]. + +[1] https://specs.openstack.org/openstack/glance-specs/specs/mitaka/database-purge.html +[2] https://specs.openstack.org/openstack/cinder-specs/specs/kilo/database-purge.html