diff --git a/doc/source/code_documentation.rst b/doc/source/code_documentation.rst new file mode 100644 index 0000000..4d8925e --- /dev/null +++ b/doc/source/code_documentation.rst @@ -0,0 +1,22 @@ +.. + Gluon documentation master file. + Copyright 2017, AT&T + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Code Documentation +================== +.. toctree:: + :maxdepth: 1 + + api/modules diff --git a/doc/source/conf.py b/doc/source/conf.py index ca0ac0f..3d0c0e5 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -15,7 +15,9 @@ import os import sys -sys.path.insert(0, os.path.abspath('../..')) +sys.path.insert(0, os.path.abspath('../../')) +sys.path.insert(0, os.path.abspath('../')) +sys.path.insert(0, os.path.abspath('./')) # -- General configuration ---------------------------------------------------- # Add any Sphinx extension module names here, as strings. They can be @@ -50,6 +52,8 @@ add_module_names = True # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' +autodoc_mock_imports = ['neutron', 'gluon'] + # -- Options for HTML output -------------------------------------------------- # The theme to use for HTML and HTML Help pages. Major themes that come with diff --git a/doc/source/devref/database_migration.rst b/doc/source/devref/database_migration.inc similarity index 100% rename from doc/source/devref/database_migration.rst rename to doc/source/devref/database_migration.inc diff --git a/doc/source/devref/gluon-auth.rst b/doc/source/devref/gluon-auth.inc similarity index 99% rename from doc/source/devref/gluon-auth.rst rename to doc/source/devref/gluon-auth.inc index c9252c4..000f0bc 100644 --- a/doc/source/devref/gluon-auth.rst +++ b/doc/source/devref/gluon-auth.inc @@ -186,10 +186,11 @@ The pecan-wsgi service in the Neutron will be used as a reference code for Gluon Configuration ~~~~~~~~~~~~~ + The /etc/proton/proton.conf file can be used to configure the authentication details. A sample configuration is shown below. Change 127.0.0.1 to your keystone endpoint. -.. code-block:: conf +.. code-block:: ini [api] auth_strategy = keystone @@ -209,11 +210,12 @@ configuration is shown below. Change 127.0.0.1 to your keystone endpoint. Set Environment Variables ~~~~~~~~~~~~~~~~~~~~~~~~~ + If Keystone is enabled for authentication, some environment variables will need to be set up in order for protonclient commands to work properly. Modify the openrc file in gluon home directory with the appropriate value for you Keystone endpoint. A sample is shown below. -Then run ``source openrc ``to set these variables. +Then run ``source openrc `` to set these variables. .. code-block:: bash @@ -224,6 +226,7 @@ Then run ``source openrc ``to set thes Appendix -------- + Configuring identity details for Keystone, change 10.0.2.15 to your gluon endpoint: 1. Create gluon user: @@ -246,4 +249,5 @@ Configuring identity details for Keystone, change 10.0.2.15 to your gluon endpoi Reference --------- + 1) Port and service binding model - https://review.openstack.org/#/c/392250 diff --git a/doc/source/devref/gluon_api_spec.rst b/doc/source/devref/gluon_api_spec.inc similarity index 98% rename from doc/source/devref/gluon_api_spec.rst rename to doc/source/devref/gluon_api_spec.inc index c84a0c5..b7e392d 100644 --- a/doc/source/devref/gluon_api_spec.rst +++ b/doc/source/devref/gluon_api_spec.inc @@ -1,3 +1,4 @@ +======================= Gluon API Specification ======================= @@ -127,11 +128,12 @@ File Structure -------------- The API is defined by a single file. The Root Object is defined by the -ProtonDef object. See the complete Example_ at the end of this page for an +ProtonDef object. See the complete Example_Specs_ at the end of this page for an example of the ProtonDef syntax. ProtonDef -+++++++++ +~~~~~~~~~ + .. csv-table:: :header: "Fixed Field", "Type", "Required", "Description" :widths: 5, 5, 3, 20 @@ -174,7 +176,8 @@ contains the metadata about the API. The *objects* field contains the base and API object definitions for the API. InfoDef -+++++++ +~~~~~~~ + .. csv-table:: :header: "Fixed Field", "Type", "Required", "Description" :widths: 5, 5, 3, 20 @@ -200,7 +203,8 @@ The InfoDef is where metadata about the API can be specified. At a minimum the email: bh526r@att.com AuthorDef -+++++++++ +~~~~~~~~~ + .. csv-table:: :header: "Fixed Field", "Type", "Required", "Description" :widths: 5, 5, 3, 20 @@ -222,7 +226,8 @@ This information is optional. email: bh526r@att.com ObjectsDef -++++++++++ +~~~~~~~~~~ + .. csv-table:: :header: "Pattern Field", "Type", "Required", "Description" :widths: 10, 5, 3, 20 @@ -256,7 +261,8 @@ field name should follow the lexical definitions for a Python identifier. description: "Route distinguisher for this VPN" ObjectDef -+++++++++ +~~~~~~~~~ + .. csv-table:: :header: "Fixed Field", "Type", "Required", "Description" :widths: 5, 5, 3, 20 @@ -295,7 +301,8 @@ the object. ApiDef -++++++ +~~~~~~ + .. csv-table:: :header: "Fixed Field", "Type", "Required", "Description" :widths: 5, 5, 3, 20 @@ -321,7 +328,8 @@ for the API path during code generation. parent: Port PolicyDef -+++++++++ +~~~~~~~~~ + .. csv-table:: :header: "Fixed Field", "Type", "Required", "Description" :widths: 5, 5, 3, 20 @@ -348,7 +356,8 @@ of the rule specifier string is defined in the Openstack Policy update: "rule:admin_or_network_owner" AttributesDef -+++++++++++++ +~~~~~~~~~~~~~ + .. csv-table:: :header: "Pattern Field", "Type", "Required", "Description" :widths: 10, 5, 3, 20 @@ -371,7 +380,7 @@ identifier. AttributeSchemaDef -++++++++++++++++++ +~~~~~~~~~~~~~~~~~~ .. csv-table:: :header: "Fixed Field", "Type", "Required", "Description" @@ -446,7 +455,7 @@ subnet_prefix, status, and profile. format: json References -++++++++++ +~~~~~~~~~~ `Date_Time `_ `IPV4 `_ @@ -454,10 +463,10 @@ References `URI `_ `EMAIL `_ -.. _Example: +.. _Example_Specs: Complete Example Specification -++++++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This section shows the L3VPN API defined using this specification. The base objects that would be defined in base/base.yaml are in the Base Objects @@ -701,4 +710,3 @@ API Specification type: string length: 32 description: "Route target export policy" - diff --git a/doc/source/devref/gluon_proton_versioning.rst b/doc/source/devref/gluon_proton_versioning.inc similarity index 100% rename from doc/source/devref/gluon_proton_versioning.rst rename to doc/source/devref/gluon_proton_versioning.inc diff --git a/doc/source/devref/high_level_design.rst b/doc/source/devref/high_level_design.inc similarity index 100% rename from doc/source/devref/high_level_design.rst rename to doc/source/devref/high_level_design.inc diff --git a/doc/source/devref/index.rst b/doc/source/devref/index.rst index 417f0b6..1860940 100644 --- a/doc/source/devref/index.rst +++ b/doc/source/devref/index.rst @@ -19,19 +19,18 @@ ''''''' Heading 4 (Avoid deeper levels because they do not render well.) -:tocdepth: 2 - #################### Gluon Developer Docs #################### -.. rest_expand_all:: +.. toctree:: + :maxdepth: 2 -.. include:: high_level_design.rst -.. include:: plugin_wrapper.rst -.. include:: service_binding_model.rst -.. include:: database_migration.rst -.. include:: gluon-auth.rst -.. include:: gluon_api_spec.rst -.. include:: gluon_proton_versioning.rst -.. include:: repo_structure.rst +.. include:: high_level_design.inc +.. include:: plugin_wrapper.inc +.. include:: service_binding_model.inc +.. include:: database_migration.inc +.. include:: gluon-auth.inc +.. include:: gluon_api_spec.inc +.. include:: gluon_proton_versioning.inc +.. include:: repo_structure.inc diff --git a/doc/source/devref/plugin_wrapper.rst b/doc/source/devref/plugin_wrapper.inc similarity index 100% rename from doc/source/devref/plugin_wrapper.rst rename to doc/source/devref/plugin_wrapper.inc diff --git a/doc/source/devref/repo_structure.rst b/doc/source/devref/repo_structure.inc similarity index 97% rename from doc/source/devref/repo_structure.rst rename to doc/source/devref/repo_structure.inc index 316b5fd..6e45b46 100644 --- a/doc/source/devref/repo_structure.rst +++ b/doc/source/devref/repo_structure.inc @@ -32,8 +32,8 @@ Current Repository Structure - **doc** - samples # Sample policy.json and proton.conf files - source # All documentation - - devref # Developer Guides - - testcase # Test Case proposals + - devref # Developer Guides + - testcase # Test Case proposals - **etc** # Config options for model handlers and backends - proton # Config options for Protons diff --git a/doc/source/devref/service_binding_model.rst b/doc/source/devref/service_binding_model.inc similarity index 100% rename from doc/source/devref/service_binding_model.rst rename to doc/source/devref/service_binding_model.inc diff --git a/doc/source/devref/task_list.inc b/doc/source/devref/task_list.inc new file mode 100644 index 0000000..22211b6 --- /dev/null +++ b/doc/source/devref/task_list.inc @@ -0,0 +1,111 @@ +:orphan: + +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +=============== +Gluon Task List +=============== + +--------------------------------- +For Infrastructure and Repository +--------------------------------- + +* Set up cloud infrastructure with 5 blade servers in Ericsson's new data center +* Create repos based on new architecture + * For now, we need to consolidate everything back int one repo +* The requirements.txt files needs to be updated and aligned with the Global Requirements + +--------------------------------- +For Proof-of-Concept in Barcelona +--------------------------------- + +* Setup system with base linux bridge networking +* Use separate SDN-C for each host to support gluon networking +* Implement Gluon Neutron ML2 Plugin + +---------------------------- +For Neutron Gluon ML2 Plugin +---------------------------- + +* Investigate using subclass of Neutron Core plugin to direct bind/unbind operations to Gluon/Proton + +---------------------------------------- +For Enhancements of Gluon Implementation +---------------------------------------- + +* Investigate how different VIF drivers can be used for different port types on the same hypervisor + * Need to understand VIF plugging and how it affects Gluon + * Generic VIF driver may be okay for most use cases + +* Investigate and resolve synchronization issues in the gluon/proton architecture + * Need to re-sync proton with gluon upon restart + * This is partially done + * Synchronization of mysql and etcd databases needs to be investigated + * Are we covered if Proton restarts or etcd is unavailable for a period of time? +* Synchronize bind operation with SDN controller + * There is no feedback if the SDN bind fails + * Need to make sure bind is done before VM is spawned. Synchronize bind with port_update response + * Binding information needs to be pushed back to Proton + * Should Proton just read this from etcd? +* Rebuild Gluon to not use particle generator + +------------------------------------ +For Enhancements of Proton API Model +------------------------------------ + +* Fix VPNAFConfig in L3VPN Model (only one reference currently) +* The API modeling approach needs to be cleaned up + * Need a baseport that is common to all APIs + * Right now each new API should need to copy/paste baseport object + * Binding information needs be be modeled consistently + * Separate table or part of baseport? + * Or, just resides in etcd? + * IP Addressing information should be modeled with a separate table + * IPAM is broken with current model. + * It is possible to assign IP address that cannot be supported because of SDN-C/Host mapping +* Proton needs to be reworked to support multiple APIs +* Create a cookiecutter (or script) to create a new API within the Proton source tree +* Sync thread needs to be updated to use log table to support Proton HA +* The Particle Generator only supports flat APIs at this time. + * Support for hierarchical APIs was started but not completed. +* Need consistent way of handling empty/unset fields in the API + +----------------------------- +For Proton Database Migration +----------------------------- + +* Modify Proton code to use MySQL (currently only using sqlite) +* Need a strategy to handle database migrations if the API model is changed + +----------- +For Testing +----------- + +* We need to add unit, functional, and tempest testing + +----------------------------------------- +For Installer Support (Devstack and Fuel) +----------------------------------------- + +* Need to update devstack code to install Gluon + * Install etcd. + * Setup database +* Create Fuel plugin to install Gluon + +------------------------ +For Keystone Integration +------------------------ + +* Need to hook in keystone authorization to access API (wide open now) + diff --git a/doc/source/devref/task_list.rst b/doc/source/devref/task_list.rst deleted file mode 100644 index 511fb4d..0000000 --- a/doc/source/devref/task_list.rst +++ /dev/null @@ -1,118 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -=============== -Gluon Task List -=============== - ---------------------------------- -For Infrastructure and Repository ---------------------------------- - -#. Set up cloud infrastructure with 5 blade servers in Ericsson's new data center -#. Create repos based on new architecture - #. For now, we need to consolidate everything back int one repo -#. The requirements.txt files needs to be updated and aligned with the Global Requirements - ---------------------------------- -For Proof-of-Concept in Barcelona ---------------------------------- - -#. Setup system with base linux bridge networking -#. Use separate SDN-C for each host to support gluon networking -#. Implement Gluon Neutron ML2 Plugin - ----------------------------- -For Neutron Gluon ML2 Plugin ----------------------------- - -#. Investigate using subclass of Neutron Core plugin to direct bind/unbind operations to Gluon/Proton - ----------------------------------------- -For Enhancements of Gluon Implementation ----------------------------------------- - -#. Investigate how different VIF drivers can be used for different port types on the same hypervisor - - #. Need to understand VIF plugging and how it affects Gluon - #. Generic VIF driver may be okay for most use cases - -#. Investigate and resolve synchronization issues in the gluon/proton architecture - - #. Need to re-sync proton with gluon upon restart - - #. This is partially done - - #. Synchronization of mysql and etcd databases needs to be investigated - - #. Are we covered if Proton restarts or etcd is unavailable for a period of time? - -#. Synchronize bind operation with SDN controller - - #. There is no feedback if the SDN bind fails - #. Need to make sure bind is done before VM is spawned. Synchronize bind with port_update response - #. Binding information needs to be pushed back to Proton - - #. Should Proton just read this from etcd? - -#. Rebuild Gluon to not use particle generator - ------------------------------------- -For Enhancements of Proton API Model ------------------------------------- - -#. Fix VPNAFConfig in L3VPN Model (only one reference currently) -#. The API modeling approach needs to be cleaned up - #. Need a baseport that is common to all APIs - #. Right now each new API should need to copy/paste baseport object - #. Binding information needs be be modeled consistently - #. Separate table or part of baseport? - #. Or, just resides in etcd? - #. IP Addressing information should be modeled with a separate table - #. IPAM is broken with current model. - #. It is possible to assign IP address that cannot be supported because of SDN-C/Host mapping -#. Proton needs to be reworked to support multiple APIs -#. Create a cookiecutter (or script) to create a new API within the Proton source tree -#. Sync thread needs to be updated to use log table to support Proton HA -#. The Particle Generator only supports flat APIs at this time. - #. Support for hierarchical APIs was started but not completed. -#. Need consistent way of handling empty/unset fields in the API - ------------------------------ -For Proton Database Migration ------------------------------ - -#. Modify Proton code to use MySQL (currently only using sqlite) -#. Need a strategy to handle database migrations if the API model is changed - ------------ -For Testing ------------ - -#. We need to add unit, functional, and tempest testing - ------------------------------------------ -For Installer Support (Devstack and Fuel) ------------------------------------------ - -#. Need to update devstack code to install Gluon - #. Install etcd. - #. Setup database -#. Create Fuel plugin to install Gluon - ------------------------- -For Keystone Integration ------------------------- - -#. Need to hook in keystone authorization to access API (wide open now) - diff --git a/doc/source/index.rst b/doc/source/index.rst index 4cdff0d..d5a5e01 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -26,18 +26,17 @@ Welcome to Gluon's documentation! ================================= -Contents: +Contents .. toctree:: - :maxdepth:2 + :maxdepth: 2 readme.rst - devref/high_level_design.rst - devref/repo_structure.rst installation.rst usage.rst contributing.rst devref/index.rst + code_documentation.rst ================== Indices and tables @@ -46,4 +45,3 @@ Indices and tables * :ref:`genindex` * :ref:`modindex` * :ref:`search` - diff --git a/doc/source/installation.rst b/doc/source/installation.rst index d9e60cc..c13b2c4 100644 --- a/doc/source/installation.rst +++ b/doc/source/installation.rst @@ -38,7 +38,7 @@ still needs to be manually set up. Contents: .. toctree:: - :maxdepth:2 + :maxdepth: 2 installation/install_etcd.rst installation/install_gluon.rst diff --git a/doc/source/testcase/database.rst b/doc/source/testcase/database.rst index 352546a..5ac45c2 100644 --- a/doc/source/testcase/database.rst +++ b/doc/source/testcase/database.rst @@ -1,3 +1,5 @@ +:orphan: + ..Database @@ -81,7 +83,7 @@ |Output Specifications |The responses after executing gluon.db.sqlalchemy.api| | | | +-----------------------+-----------------------------------------------------+ -|Pass/Fail Criteria |If testcase fails, tox should output errors with | +|Pass/Fail Criteria |If testcase fails, tox should output errors with | | |trace. | | | | | | | diff --git a/gluon/api/hooks/context.py b/gluon/api/hooks/context.py index 81aaf86..0524b3f 100644 --- a/gluon/api/hooks/context.py +++ b/gluon/api/hooks/context.py @@ -23,18 +23,15 @@ class ContextHook(hooks.PecanHook): """Configures a request context and attaches it to the request. The following HTTP request headers are used: - X-User-Id or X-User: - Used for context.user_id. - X-Project-Id: - Used for context.tenant_id. - X-Project-Name: - Used for context.tenant_name. - X-Auth-Token: - Used for context.auth_token. - X-Roles: - Used for setting context.is_admin flag to either True or False. - The flag is set to True, if X-Roles contains either an administrator - or admin substring. Otherwise it is set to False. + + * X-User-Id or X-User - Used for context.user_id + * X-Project-Id - Used for context.tenant_id + * X-Project-Name - Used for context.tenant_name + * X-Auth-Token - Used for context.auth_token. + * X-Roles - Used for setting context.is_admin flag to either + True or False. If the flag is set to True, if X-Roles contains + either an administrator or admin substring. Otherwise it is + set to False. """ diff --git a/gluon/conf/config.py b/gluon/conf/config.py index 4dd1fac..53cb73f 100644 --- a/gluon/conf/config.py +++ b/gluon/conf/config.py @@ -37,7 +37,7 @@ API_OPTS = [ help='the type of authentication to use'), cfg.BoolOpt('debug', default=True, - help='debug') + help='debug'), ] PATH_OPTS = [ diff --git a/setup.cfg b/setup.cfg index 4b798f2..8326465 100644 --- a/setup.cfg +++ b/setup.cfg @@ -25,9 +25,11 @@ data_files = models = models/* [build_sphinx] +builder = html source-dir = doc/source build-dir = doc/build all_files = 1 +warning-is-error = 1 [entry_points] console_scripts = diff --git a/tox.ini b/tox.ini index e9c18a2..247b7cc 100644 --- a/tox.ini +++ b/tox.ini @@ -12,6 +12,7 @@ commands = find gluon -type f -name "*.pyc" -delete ostestr '{posargs}' whitelist_externals = + bash find [testenv:pep8] @@ -24,7 +25,10 @@ commands = {posargs} commands = python setup.py test --coverage --testr-args='{posargs}' [testenv:docs] -commands = python setup.py build_sphinx +commands = + bash -c "rm -rf doc/build" + bash -c "rm -rf doc/source/api" + python setup.py build_sphinx [testenv:releasenotes] commands =