From 83011b4e7618921925309b1b41cd15895af3a514 Mon Sep 17 00:00:00 2001 From: gecong1973 Date: Thu, 18 Oct 2018 18:28:57 -0700 Subject: [PATCH] Added Stein directory for stein specs Created Stein directory and required files for Stein release. Change-Id: Ie4568225dbb85c63c71f5447d5484a975564d0d7 --- priorities/stein-priorities.rst | 9 + specs/stein-template.rst | 365 ++++++++++++++++++ .../add-mysql-support-for-freezer.rst | 229 +++++++++++ 3 files changed, 603 insertions(+) create mode 100644 priorities/stein-priorities.rst create mode 100644 specs/stein-template.rst create mode 100644 specs/stein/approved/add-mysql-support-for-freezer.rst diff --git a/priorities/stein-priorities.rst b/priorities/stein-priorities.rst new file mode 100644 index 0000000..720c44a --- /dev/null +++ b/priorities/stein-priorities.rst @@ -0,0 +1,9 @@ +.. _stein-priorities: + +======================== +Stein Project Priorities +======================== + +List of themes (in the form of use cases) the freezer development team will +prioritize in Stein. + diff --git a/specs/stein-template.rst b/specs/stein-template.rst new file mode 100644 index 0000000..3d7cdf9 --- /dev/null +++ b/specs/stein-template.rst @@ -0,0 +1,365 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +========================================== +Example Spec - The title of your blueprint +========================================== + +Include the URL of your launchpad blueprint: + +https://blueprints.launchpad.net/freezer/+spec/example + +Introduction paragraph -- why are we doing anything? A single paragraph of +prose that operators can understand. The title and this first paragraph +should be used as the subject line and body of the commit message +respectively. + +Some notes about the freezer-spec and blueprint process: + +* Not all blueprints need a spec. For more information see + https://docs.openstack.org/nova/latest/contributor/blueprints.html?highlight=when%20blueprint%20needed + +* The aim of this document is first to define the problem we need to solve, + and second agree the overall approach to solve that problem. + +* This is not intended to be extensive documentation for a new feature. + For example, there is no need to specify the exact configuration changes, + + nor the exact details of any DB model changes. But you should still define + that such changes are required, and be clear on how that will affect + upgrades. + +* You should aim to get your spec approved before writing your code. + While you are free to write prototypes and code before getting your spec + approved, its possible that the outcome of the spec review process leads + you towards a fundamentally different solution than you first envisaged. + +* But, API changes are held to a much higher level of scrutiny. + As soon as an API change merges, we must assume it could be in production + somewhere, and as such, we then need to support that API change forever. + To avoid getting that wrong, we do want lots of details about API changes + upfront. + +Some notes about using this template: + +* Your spec should be in ReSTructured text, like this template. + +* Please wrap text at 79 columns. + +* The filename in the git repository should match the launchpad URL, for + example a URL of: https://blueprints.launchpad.net/freezer/+spec/awesome-thing + should be named awesome-thing.rst + +* Please do not delete any of the sections in this template. If you have + nothing to say for a whole section, just write: None + +* For help with syntax, see http://sphinx-doc.org/rest.html + +* To test out your formatting, build the docs using tox and see the generated + HTML file in doc/build/html/specs/ + +* If you would like to provide a diagram with your spec, ascii diagrams are + required. http://asciiflow.com/ is a very nice tool to assist with making + ascii diagrams. The reason for this is that the tool used to review specs is + based purely on plain text. Plain text will allow review to proceed without + having to look at additional files which can not be viewed in gerrit. It + will also allow inline feedback on the diagram itself. + + +Problem description +=================== + +A detailed description of the problem. What problem is this blueprint +addressing? + +Use Cases +--------- + +What use cases does this address? What impact on actors does this change have? +Ensure you are clear about the actors in each use case: Developer, End User, +Deployer etc. + +Proposed change +=============== + +Here is where you cover the change you propose to make in detail. How do you +propose to solve this problem? + +If this is one part of a larger effort make it clear where this piece ends. In +other words, what's the scope of this effort? + +At this point, if you would like to just get feedback on if the problem and +proposed change fit in freezer, you can stop here and post this for review to get +preliminary feedback. If so please say: +Posting to get preliminary feedback on the scope of this spec. + +Alternatives +------------ + +What other ways could we do this thing? Why aren't we using those? This doesn't +have to be a full literature review, but it should demonstrate that thought has +been put into why the proposed solution is an appropriate one. + +Data model impact +----------------- + +Changes which require modifications to the data model often have a wider impact +on the system. The community often has strong opinions on how the data model +should be evolved, from both a functional and performance perspective. It is +therefore important to capture and gain agreement as early as possible on any +proposed changes to the data model. + +Questions which need to be addressed by this section include: + +* What new data objects and/or database schema changes is this going to + require? + +* What database migrations will accompany this change. + +* How will the initial set of new data objects be generated, for example if you + need to take into account existing backups/jobs/... , or modify other + existing data describe how that will work. + +REST API impact +--------------- + +Each API method which is either added or changed should have the following + +* Specification for the method + + * A description of what the method does suitable for use in + user documentation + + * Method type (POST/PUT/GET/DELETE) + + * Normal http response code(s) + + * Expected error http response code(s) + + * A description for each possible error code should be included + describing semantic errors which can cause it such as + inconsistent parameters supplied to the method, or when an + instance is not in an appropriate state for the request to + succeed. Errors caused by syntactic problems covered by the JSON + schema definition do not need to be included. + + * URL for the resource + + * URL should not include underscores, and use hyphens instead. + + * Parameters which can be passed via the url + + * JSON schema definition for the request body data if allowed + + * Field names should use snake_case style, not CamelCase or MixedCase + style. + + * JSON schema definition for the response body data if any + + * Field names should use snake_case style, not CamelCase or MixedCase + style. + +* Example use case including typical API samples for both data supplied + by the caller and the response + +* Discuss any policy changes, and discuss what things a deployer needs to + think about when defining their policy. + +Reuse of existing predefined parameter types such as regexps for +passwords and user defined names is highly encouraged. + +Security impact +--------------- + +Describe any potential security impact on the system. Some of the items to +consider include: + +* Does this change touch sensitive data such as tokens, keys, or user data? + +* Does this change alter the API in a way that may impact security, such as + a new way to access sensitive information or a new way to login? + +* Does this change involve cryptography or hashing? + +* Does this change require the use of sudo or any elevated privileges? + +* Does this change involve using or parsing user-provided data? This could + be directly at the API level or indirectly such as changes to a cache layer. + +* Can this change enable a resource exhaustion attack, such as allowing a + single API interaction to consume significant server resources? Some examples + of this include launching subprocesses for each connection, or entity + expansion attacks in XML. + +For more detailed guidance, please see the OpenStack Security Guidelines as +a reference (https://wiki.openstack.org/wiki/Security/Guidelines). These +guidelines are a work in progress and are designed to help you identify +security best practices. For further information, feel free to reach out +to the OpenStack Security Group at openstack-security@lists.openstack.org. + +Notifications impact +-------------------- + +Please specify any changes to notifications. Be that an extra notification, +changes to an existing notification, or removing a notification. + +Other end user impact +--------------------- + +Aside from the API, are there other ways a user will interact with this +feature? + +* Does this change have an impact on python-freezerclient? What does the user + interface there look like? + +* Does this change have an impact on freezer-web-ui? What does the user + interface there look like? + +* Does this change have an impact on freezeri-dr? + +Performance Impact +------------------ + +Describe any potential performance impact on the system, for example +how often will new code be called, and is there a major change to the calling +pattern of existing code. + +Examples of things to consider here include: + +* A small change in a utility function or a commonly used decorator can have a + large impacts on performance. + +* Calls which result in a database queries can have a profound impact on + performance when called in critical sections of the code. + +* Will the change include any locking, and if so what considerations are there + on holding the lock? + +Other deployer impact +--------------------- + +Discuss things that will affect how you deploy and configure OpenStack +that have not already been mentioned, such as: + +* What config options are being added? Should they be more generic than + proposed? Are the default values ones which will work well in real + deployments? + +* Is this a change that takes immediate effect after its merged, or is it + something that has to be explicitly enabled? + +* If this change is a new binary, how would it be deployed? + +* Please state anything that those doing continuous deployment, or those + upgrading from the previous release, need to be aware of. Also describe + any plans to deprecate configuration values or features. For example, if we + change the directory name that instances are stored in, how do we handle + instance directories created before the change landed? Do we move them? Do + we have a special case in the code? Do we assume that the operator will + recreate all the instances in their cloud? + +Developer impact +---------------- + +Discuss things that will affect other developers working on OpenStack. + +Implementation +============== + +Assignee(s) +----------- + +Who is leading the writing of the code? Or is this a blueprint where you're +throwing it out there to see who picks it up? + +If more than one person is working on the implementation, please designate the +primary author and contact. + +Primary assignee: + + +Other contributors: + + +Work Items +---------- + +Work items or tasks -- break the feature up into the things that need to be +done to implement it. Those parts might end up being done by different people, +but we're mostly trying to understand the timeline for implementation. + + +Dependencies +============ + +* Include specific references to specs and/or blueprints in freezer, or in + other projects, that this one either depends on or is related to. + +* If this requires functionality of another project that is not currently used + by Freezer (such as the glance v2 API when we previously only required v1), + document that fact. + +* Does this feature require any new library dependencies or code otherwise not + included in OpenStack? Or does it depend on a specific version of library? + + +Testing +======= + +Please discuss the important scenarios needed to test here, as well as +specific edge cases we should be ensuring work correctly. + +Please discuss how the change will be tested. We especially want to know what +tempest tests will be added. It is assumed that unit test coverage will be +added so that doesn't need to be mentioned explicitly, but discussion of why +you think unit tests are sufficient and we don't need to add more tempest +tests would need to be included. + +Is this untestable in gate given current limitations (specific hardware / +software configurations available)? If so, are there mitigation plans (3rd +party testing, gate enhancements, etc). + + +Documentation Impact +==================== + +Please describe how and where this change will be documented. + + +References +========== + +Please add any useful references here. You are not required to have any +reference. Moreover, this specification should still make sense when your +references are unavailable. Examples of what you could include are: + +* Links to mailing list or IRC discussions + +* Links to notes from a summit session + +* Links to relevant research, if appropriate + +* Related specifications as appropriate (e.g. if it's an EC2 thing, link the + EC2 docs) + +* Anything else you feel it is worthwhile to refer to + + +History +======= + +Optional section for Pike intended to be used each time the spec +is updated to describe new design, API or any database schema +updated. Useful to let reader understand what's happened along the +time. + +.. list-table:: Revisions + :header-rows: 1 + + * - Release Name + - Description + * - Pike + - Introduced diff --git a/specs/stein/approved/add-mysql-support-for-freezer.rst b/specs/stein/approved/add-mysql-support-for-freezer.rst new file mode 100644 index 0000000..7cf9fe0 --- /dev/null +++ b/specs/stein/approved/add-mysql-support-for-freezer.rst @@ -0,0 +1,229 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. + + http://creativecommons.org/licenses/by/3.0/legalcode + +============================= +Add mysql support for freezer +============================= + +* https://storyboard.openstack.org/#!/story/2004132 + +Provide more db options for users to store the management data. + +Problem description +=================== + +Currently, freezer can only support elasticsearch as db management engine, +so we should increase support for mysql . + +Use Cases +--------- + +* Users that want to backup management data to mysql database. + +* OpenStack distributions deployed without elasticsearch. + +Proposed change +=============== + +Implement in the freezer-api a new db engine mysql. + +With specifying the backend = sqlalchemy and driver = sqlalchemy in storage +section of conf file , the management data will be stored on mysql db server + +Alternatives +------------ + +None + +Technical details +----------------- + +Related docs: +----------------- + +None + +Data model impact +----------------- + +* Add clients table:: + + clients = Table( + 'clients', meta, + Column('created_at', DateTime(timezone=False)), + Column('updated_at', DateTime(timezone=False)), + Column('deleted_at', DateTime), + Column('deleted', Boolean), + Column('user_id', String(36), nullable=False), + Column('id', String(255), primary_key=True, nullable=False), + Column('project_id', String(36), nullable=False), + Column('config_id', String(255), nullable=False), + Column('hostname', String(255), nullable=False), + Column('description', String(255)), + Column('uuid', String(36), nullable=False), + mysql_engine='InnoDB', + mysql_charset='utf8' + ) + +* Add sessions table:: + + sessions = Table( + 'sessions', meta, + Column('created_at', DateTime(timezone=False)), + Column('updated_at', DateTime(timezone=False)), + Column('deleted_at', DateTime), + Column('deleted', Boolean), + Column('id', String(36), primary_key=True, nullable=False), + Column('session_tag', Integer, default=0), + Column('description', String(255)), + Column('hold_off', Integer, default=30), + Column('scheduling', Text), + Column('job', Text), + Column('project_id', String(36), nullable=False), + Column('user_id', String(36), nullable=False), + Column('time_start', Integer), + Column('time_end', Integer), + Column('time_started', Integer), + Column('time_ended', Integer), + Column('status', String(255)), + Column('result', String(255)), + mysql_engine='InnoDB', + mysql_charset='utf8' + ) + +* Add jobs table:: + + jobs = Table( + 'jobs', meta, + Column('created_at', DateTime(timezone=False)), + Column('updated_at', DateTime(timezone=False)), + Column('deleted_at', DateTime), + Column('deleted', Boolean), + Column('id', String(36), primary_key=True, nullable=False), + Column('project_id', String(36), nullable=False), + Column('user_id', String(36), nullable=False), + Column('scheduling', Text), + Column('client_id', String(255), nullable=False), + Column('session_id', String(36), default=''), + Column('session_tag', Integer, default=0), + Column('description', String(255)), + Column('job_actions', Text), + mysql_engine='InnoDB', + mysql_charset='utf8' + ) + +* Add actions table:: + + actions = Table( + 'actions', meta, + Column('created_at', DateTime(timezone=False)), + Column('updated_at', DateTime(timezone=False)), + Column('deleted_at', DateTime), + Column('deleted', Boolean), + Column('id', String(36), primary_key=True, nullable=False), + Column('action', String(255), nullable=False), + Column('project_id', String(36), nullable=False), + Column('user_id', String(36), nullable=False), + Column('mode', String(255)), + Column('src_file', String(255)), + Column('backup_name', String(255)), + Column('container', String(255)), + Column('timeout', Integer), + Column('priority', Integer), + Column('max_retries_interval', Integer, default=6), + Column('max_retries', Integer, default=5), + Column('mandatory', Boolean, default=False), + Column('log_file', String(255)), + Column('backup_metadata', Text), + mysql_engine='InnoDB', + mysql_charset='utf8' + ) + + +* Add backups table:: + + backups = Table( + 'backups', meta, + Column('created_at', DateTime(timezone=False)), + Column('updated_at', DateTime(timezone=False)), + Column('deleted_at', DateTime), + Column('deleted', Boolean), + Column('id', String(36), primary_key=True, nullable=False), + Column('client_id', String(255), nullable=False), + Column('job_id', String(36), nullable=False), + Column('project_id', String(36), nullable=False), + Column('user_id', String(64), nullable=False), + Column('user_name', String(64)), + Column('backup_metadata', Text), + mysql_engine='InnoDB', + mysql_charset='utf8' + ) + + +REST API impact +--------------- +None + +Security impact +--------------- +None + +Notifications impact +-------------------- +Some message log will be added. + +Other end user impact +--------------------- +None + +Performance Impact +------------------ +None + +Other deployer impact +--------------------- + +Developer impact +---------------- +None + +Implementation +============== +Assignee(s) +----------- + +Primary assignee: + gecong + gengchc2 + +Work Items +---------- + +* Implementing the db engine driver (mysql) + +* Implementing the mysql api + +* Implementing the mysql tables creation and upgrade + +Dependencies +============ +None + +Testing +======= +add unit test + +Documentation Impact +==================== +Some instructions should be added . + +References +========== +None + +History +======= +None