From e95afc3f3fb0fbc0c94fc6f2e58ff0abf70c8c4b Mon Sep 17 00:00:00 2001 From: Bin Hu Date: Sun, 12 Feb 2017 16:02:22 -0800 Subject: [PATCH] Gluon Documentation Update This is a set of documentation for Gluon in Ocata Release, including - README.rst for overview of Gluon project, and related resources - In doc/source: updated * index.rst for overall documentation * contributing.rst with license header information * readme.rst with license header information * installation.rst that covers Gluon installation procedure * usage.rst that is Gluon's User Guide - In doc/source/installation: added for details of installation procedure * install_etcd.rst * install_gluon.rst * install_contrail.rst - In doc/source/devref: updated Developers' Guide, including * index.rst with current developer documentation * high_level_design.rst with updated high level design of Gluon Ocata * repo_structure.rst with current repository structure Change-Id: I1aa04d3a85bbda6c5a8114115c34042e43a4a0c9 Signed-off-by: Bin Hu --- README.rst | 13 +- doc/source/contributing.rst | 21 + doc/source/devref/high_level_design.rst | 219 +++++---- doc/source/devref/index.rst | 25 +- doc/source/devref/repo_structure.rst | 176 ++------ doc/source/index.rst | 45 +- doc/source/installation.rst | 51 ++- doc/source/installation/install_contrail.rst | 62 +++ doc/source/installation/install_etcd.rst | 167 +++++++ doc/source/installation/install_gluon.rst | 173 +++++++ doc/source/readme.rst | 21 + doc/source/usage.rst | 451 ++++++++++++++++++- 12 files changed, 1168 insertions(+), 256 deletions(-) create mode 100644 doc/source/installation/install_contrail.rst create mode 100644 doc/source/installation/install_etcd.rst create mode 100644 doc/source/installation/install_gluon.rst diff --git a/README.rst b/README.rst index 721b3d2..993854d 100644 --- a/README.rst +++ b/README.rst @@ -3,19 +3,18 @@ Gluon ===== A Model-Driven, Extensible Framework for Networking APIs - Gluon provides a framework for specifying, using a model file, -APIs for network forwarding. It is intended to offer devs +APIs for network forwarding. It is intended to offer developers the possibility of quickly prototyping any networking forwarding system that describes how a packet moves from port to port or port to the outside world. * Free software: Apache license -* Documentation: http://docs.openstack.org/developer/gluon +* Wiki: http://wiki.openstack.org/gluon * Source: http://git.openstack.org/cgit/openstack/gluon * Bugs: http://bugs.launchpad.net/python-gluon +* Documentation: doc/source/index + * Installation: doc/source/installation + * User Guide: doc/source/usage + * Developers: doc/source/devref/index -Features --------- - -* TODO diff --git a/doc/source/contributing.rst b/doc/source/contributing.rst index 1728a61..375d062 100644 --- a/doc/source/contributing.rst +++ b/doc/source/contributing.rst @@ -1,3 +1,24 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Gluon documentation: + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + (Avoid deeper levels because they do not render well.) + ============ Contributing ============ diff --git a/doc/source/devref/high_level_design.rst b/doc/source/devref/high_level_design.rst index 163c37d..c360810 100644 --- a/doc/source/devref/high_level_design.rst +++ b/doc/source/devref/high_level_design.rst @@ -1,100 +1,165 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Gluon devref: + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + (Avoid deeper levels because they do not render well.) + ================= High Level Design ================= - Summary ------- -Gluon brings a Networking Service Framework that enables Telecom Service -Providers to provide their customers with networking services on-demand. Gluon -uses a model-driven approach to generate Networking Service APIs (including -objects, database schema, and RESTful API endpoints) from a YAML file which -models the Networking Service. When a Telecom Service Provider needs to launch -a new Networking Service, it only needs to model the new service in a YAML -file. The Gluon framework generates the APIs accordingly. Thus Gluon helps -Telecom Service Providers accelerate the time-to-market and achieve business -agility through its extensibility and scalability in generating APIs for new -use-cases and services. - - -Nova ----- - -Gluon currently integrates in Nova by means of a Nova network service plugin -mechanism. This replaces the traditional Neutron or Nova-Networking plugins. -The Gluon plugin uses (imports) the Gluon Client (gluonlib Module). The Gluon -Client enables Nova to perform bind and unbind operations between ports and -VMs. To this end, the Gluon Client uses the Gluon REST API model for -communications between Nova and Gluon. - -In the Mitaka release this “plugin mechanism” has been removed allowing Nova -only to communicate to Neutron. This issue will have to be addressed with the -Nova team. - +**Gluon** brings a Networking Service Framework that enables Telecom Service +Providers to provide their customers with networking services on-demand. +**Gluon** uses a model-driven approach to generate Networking Service APIs +(including objects, database schema, and RESTful API endpoints) from a YAML +file which models the Networking Service. When a Telecom Service Provider needs +to launch a new Networking Service, it only needs to model the new service in a +YAML file. The **Gluon** framework generates the APIs accordingly. Thus +**Gluon** helps Telecom Service Providers accelerate the time-to-market and +achieve business agility through its extensibility and scalability in +generating APIs for new use-cases and services. Gluon ----- -Gluon is the port arbiter that maintains a list of ports and bindings of -different networking backends. Inside Gluon there is a backend driver used to -speak to Protons. Currently this driver for L3VPN is called net_l3vpn. This -driver in turn updates the “base-port” object which is stored in the Proton -object database upon the bind and unbind operations. There is room for -improvement with the net_l3vpn driver. A more generic driver should be -created. This type of port modeling is currently being driven through the -NetReady project in OPNFV. +**Gluon** is the port arbiter that maintains a list of ports and bindings of +different networking backends. A **Proton Server** is the API server that hosts +multiple **Protons**, i.e. multiple sets of APIs. **Gluon** uses backend +drivers to interact with the **Proton Server** for port binding and other +operations. The backend drivers are specified in ``setup.cfg``, and loaded +at runtime. +For example, the driver for **L3VPN** is named ``net-l3vpn``, and implemented +in ``gluon.backends.models.net_l3vpn:Provider``. This driver in turn updates +the ``Port`` object which is stored in the **Proton Server's** object +database upon the ``bind``, ``unbind`` and other operations. -Proton ------- +When the **Proton Server** receives port binding and other operations requests, +it broadcasts those requests to ``etcd``. The **Shim Layers** of respective SDN +Controllers listen to ``etcd``, and get the notification from ``etcd``. Based +on the type of operations, parameter data, and its own deployment and policy +configuration, SDN Controllers act upon accordingly. This mechanism is similar +to Neutron's Hierarchical Port Binding (HPB), and provides the flexibility and +scalability when a port operation needs to be supported by multiple SDN +Controllers in collaborative and interoperable way. -A Proton is a model driven specification for Networking APIs. A Proton is -created using a YAML descriptor file fed into the particle generator. As a -result from the particle generator, the Proton containing the objects, -database schema, and REST APIs are created. A Proton uses the Gluon REST API -to register itself to Gluon. As a result, the Proton specific drivers would -be loaded into Gluon. In the L3VPN case this is the net_l3vpn driver. -Currently the net_l3vpn driver is created manually. Automation of this could +Integration with Neutron +~~~~~~~~~~~~~~~~~~~~~~~~ + +**Gluon** currently integrates with Neutron by means of extending Neutron's +core plugin as a subclass, namely **Extended ML2 Plugin for Gluon** (a.k.a. +Gluon Wrapper Plugin). This replaces the original Neutron's core plugin in +``neutron.conf``. The **Gluon Plugin** differentiates **Proton** ports from +Neutron ports based on a Proton's record in ``etcd``, and sends the port +binding and other operations requests to either **Proton Server** or its +superclass (i.e. the original Neutron's core plugin). + +Proton and Proton Server +------------------------ + +A **Proton** is a set of APIs of a particular NFV Networking Service. + +A **Proton Server** is the API server that hosts multiple **Protons**, i.e. +multiple sets of APIs. + +A **Proton** is created by **Particle Generator** based on a YAML file modeled +for this particular NFV Networking Service. When a **Proton** is created, the +objects, database schema, and RESTful APIs of this **Proton** are created. Then +the **Proton** specific driver would be loaded into **Gluon**. In case of +L3VPN ``net-l3vpn``, the driver is ``gluon.backends.models.net_l3vpn:Provider``. + +Currently the ``net-l3vpn`` driver is created manually. Automation of this could be future work. -A port is created using the northbound REST API of the Proton. The Proton will -store the port in its database, update etcd and use the Gluon REST API to -register the port in Gluon. The Shim Layer which is discussed below will pick -up the information from etcd. This port (base-port information) is stored in -the Proton database itself. By storing the base-port information inside the -Proton the user is free to “describe” a port however they want. The -base-ports can be viewed using the :command:`port-list` command. As -previously mentioned, when a bind action happens Gluon uses the net_l3vpn -driver to bind a baseport to a VM. Currently the base-port model was built -modeling what Neutron requires to ensure compatibility. For different -use-cases it would be possible to reduce a portion of the base-port model for -layer 3 use-cases. +A port, namely ``port``, is created when an application uses the northbound +RESTful API of the **Proton**. The **Proton** will store the port and all +related information in its database and update the record in ``etcd``. The +**Shim Layers** of respective SDN Controllers listen to ``etcd``, and get the +notification from ``etcd``. Then the **Shim Layer** will pick up the +information from ``etcd``. -When a VPN is created through a Proton, the Proton creates the VPN object and -stores this in its database. This can viewed using the :command:`vpn-list` -command. Furthermore, the API of the L3VPN allows for creating service -bindings between a baseport and a VPN service. These service binding can be -viewed using the :command:`vpnservice-list` command. +The ``ports`` can be viewed using the command: -All objects (VPNs, Base-ports, and Bindings) are stored into the Proton -database. This information is then copied into etcd. The shim layers (see -below) monitor the etcd data store and take appropriate actions in the -networking backend upon an update. Currently, the Proton database is not -redundant, but it can be stored in the same database backend as the other -OpenStack services, thereby inheriting the same level of redundancy as those -services. +.. code-block:: bash + $ protonclient --api net-l3vpn port-list +More generic command is something like: + +.. code-block:: bash + + # protonclient --api [OPTIONS] COMMAND [ARGS] ... + +Please refer to **User Guide** [1]_ for more details. + +As previously mentioned, when a ``bind`` operation is requested, Gluon uses the +driver of the selected **Proton** (e.g. ``net-l3vpn``) to bind a port to a VM. + +Proton of L3VPN +~~~~~~~~~~~~~~~ + +When an L3VPN is created through a L3VPN **Proton**, the Proton creates a +``vpn`` object and stores it in its database. This can be viewed using the +command: + +.. code-block:: bash + + $ protonclient --api net-l3vpn vpn-list + +For more generic command, please refer to **User Guide** [1]_. + +Furthermore, the API of the L3VPN allows for creating service bindings between +a ``port`` and a ``vpn`` service. This service binding, namely ``vpnbinding``, +can be viewed using the command: + +.. code-block:: bash + + $ protonclient --api net-l3vpn vpnbinding-list + +All objects (``interfaces``, ``vpns``, ``ports``, and ``vpnbindings``) are +stored into the **Proton Server's** database. This information is then copied +into ``etcd``. The **Shim layers** monitor the ``etcd`` data store and take +appropriate actions in the networking backend upon an update. Currently, +**Proton Server** database is not HA, but it can be stored in the same database +backend as the other OpenStack services, thereby inheriting the same level of +HA as those services. Networking Backends (SDN Controllers) ------------------------------------- -A Proton is built to enable the API for each networking backend. A networking -backend can be considered OpenDaylight, Neutron or others. For a networking -backend to be able to use the Proton a shim layer has to be created. The shim -layer monitors changes in the data model stored in etcd and performs -appropriate actions in the respective SDN controller backend, for instance -creating a VPN service or binding a port. In an example of using -OpenDaylight, if a bind occurs the Shim Layer is responsible for seeing the -request in the data model and updating the Flow Entries on the OVS of that -particular compute where the Virtual Machine resides. +A **Proton** is built to enable the set of APIs for a particular NFV Networking +service that is supported by one or multiple networking backends. A +networking backend can be considered Open Daylight, or others. A **Shim Layer** +is created for a networking backend to be able to use the **Proton**. The +**Shim Layer** monitors changes in the data model stored in ``etcd``, and +performs appropriate actions in the respective SDN Controller backend, for +instance creating a VPN service or binding a port. In an example of using Open +Daylight, if a ``bind`` operation request occurs, the **Shim Layer** is +responsible for understanding the request in the data model and updating the +Flow Entries on the OVS of that particular compute where the Virtual Machine +resides. + +The data model of **Shim Layer**, e.g. L3VPN, and respective backend drivers of +**ShimLayer** for specific SDN Controllers are specified in ``setup.cfg``, and +loaded at runtime. + +References + +.. [1] ../usage.rst diff --git a/doc/source/devref/index.rst b/doc/source/devref/index.rst index a09e45f..30464b2 100644 --- a/doc/source/devref/index.rst +++ b/doc/source/devref/index.rst @@ -1,3 +1,24 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Gluon devref: + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + (Avoid deeper levels because they do not render well.) + :tocdepth: 2 #################### @@ -10,6 +31,6 @@ Gluon Developer Docs .. include:: plugin_wrapper.rst .. include:: service_binding_model.rst .. include:: database_migration.rst +.. include:: gluon-auth.rst +.. include:: gluon-api-spec.rst .. include:: repo_structure.rst -.. include:: task_list.rst - diff --git a/doc/source/devref/repo_structure.rst b/doc/source/devref/repo_structure.rst index 0bba7c6..316b5fd 100644 --- a/doc/source/devref/repo_structure.rst +++ b/doc/source/devref/repo_structure.rst @@ -11,7 +11,6 @@ License for the specific language governing permissions and limitations under the License. - Convention for heading levels in Gluon devref: ======= Heading 0 (reserved for the title in a document) ------- Heading 1 @@ -23,164 +22,45 @@ ========================== Gluon Repository Structure ========================== -This document describes the repository structure that was used in the initial -development of Gluon and a proposed structure going forward. + +This document describes the repository structure that is currently used in the +development of Gluon. Current Repository Structure ---------------------------- -- **gluon-nova** - - Networking plugin code for nova +- **doc** + - samples # Sample policy.json and proton.conf files + - source # All documentation + - devref # Developer Guides + - testcase # Test Case proposals - - Currently only supports Gluon. - - All Neutron-releated code is stripped out. +- **etc** # Config options for model handlers and backends + - proton # Config options for Protons + - shim # Config options for Shims - - Uses gluonclient module in gluonlib to communicate to the gluon-server - via REST API +- **gluon** # Gluon code base + - api # Proton API model and control + - backends # Proton backend handlers, including ``net_l3vpn`` model handler + - cmd # CLI API generator and other tools + - common # Common libraries + - db # Database handler + - managers # API manager, including ``net_l3vpn`` API manager + - models # Proton data model, including base model and ``net_l3vpn`` model + - particleGenerator # Particle Generator to generate APIs from YAML + - plugin # Extended ML2 Plugin for Gluon, a.k.a. Gluon Wrapper Plugin + - shim # Shim Layer, including ``net-l3vpn`` model, sample backend and ODL backend + - sync_etcd # Make hosts of ``etcd`` configurable + - tests # Unit tests -- **gluon** +- **releasenotes** # Enable release notes translation. Initiated by cookiecutter when repo was created - - Gluon core library - - - Particle generator, CLI generator, sync-etcd and other common code - - - gluon-server code - - - Main entry point for gluon-server process - - Uses particle generator to generate the REST API from model files - (like a proton) - - Uses particle generator to generate sqlalchemy database classes fro - model files (like a proton) - - Has a manager class to support API request processing behavior - - Supports multiple backends to process messages to/from individua - protons (using stevedore plugin mechanism) - - - gluonclient code - - - Main entry point for CLI script - - - Contains custom extensions for bind/unbind operations (i.e. no - generated) - - Uses CLI common code from Gluon core library - - Parses model files to generate commands and argument processing o - the fly - -- **gluonlib** - - - Contains gluonclient module that is used by gluon-nova - - Contains old CLI code that is currently not used. - -- **proton** - - - proton-server code - - - Main entry point for proton-server process - - Uses particle generator from gluon package to generate the REST API - from model files - - Uses particle generator from gluon package to generate sqlalchemy database - classes from model files - - Has a manager class to support API request processing behavio - (generic and could be generated ) - - Uses sync-etcd module in Gluon core to synchronize database update - with etcd backend - - Currently only supports one set of model files (l3vpn) - - protonclient code - - - Main entry point for CLI script - - Uses CLI common code from Gluon core library - - Parses model files to generate commands and argument processing on - the fly - - -Proposed Repository Structure ------------------------------ -- **gluon-lib** - - - Gluon core library - - - Particle generator, CLI generator, sync-etcd and other common code - -- **gluon-nova** - - - Networking plugin code for nova - - - Needs to be updated to support Gluon and Neutron - - - Uses gluonclient module in python-gluonclient to communicate to the - gluon-server via REST API - -- **gluon** - - - gluon-server code - - - Main entry point for gluon-server process - - Uses particle generator from gluon-core to generate the REST - API from model files (like a proton) - - Uses particle generator from gluon-core to generate sqlalchemy database - classes from model files (like a proton) - - Has a manager class to support API request processing - - Supports multiple backends to process messages to/from individual - protons (using stevedore plugin mechanism) - -- **proton** - - - proton-server code - - - Main entry point for proton-server process - - Uses sync-etcd module in Gluon core to synchronize database updates - with etcd backend - - Supports one or more API endpoints - - - Each API will have a unique namespace and subdirectory - - The configuration file will have a field to specify the list of - APIs the proton should serve - - For example, suppose we have two APIs defined (l3vpn and evpn) - - - \http://:2704/l3vpn/baseport - - \http://:2704/evpn/baseport - - For each API subdirectory - - - Model files for the API - - Uses particle generator from gluon-core to generate the REST - API from model files - - Uses particle generator from gluon-core to generate sqlalchemy - database classes from model files - - Has a manager class to support API request processing behavior - (generic and could be generated ) - -- **python-gluonclient** - - - Main entry point for CLI script - - Uses CLI common code from Gluon core library - - Parses model files to generate commands and argument processing on the - fly - - Contains gluonclient module that is used by gluon-nova - -- **python-protonclient** - - - Main entry point for CLI script - - Uses CLI common code from Gluon core library - - Supports one or more API endpoints - - - Each API will have a unique namespace - - A command argument will identify which API (i.e. model files) to process - - - For example, suppose we have two APIs defined (l3vpn and evpn) - - - proton —api l3vpn baseport-create …. - - proton —api evpn baseport-create ... - - Parses model files to generate commands and argument processing on the - fly +- **scripts** # Proton Server script and configuration Notes ----- -- Right now the gluon-server uses a model file to generate its API and database - classes. This may not be the best approach. The API between Nova and Gluon - should be fairly stable and may benefit from not being constrained to the - limitations of the particle generator. -- The proton code was not originally designed to handle multiple API endpoints - . A substantial amount of rework will be needed to support this. - A script should be created to help with the creation of new API endpoints. - The script should create the directory tree, default files and skeleton code needed to support the API. + The script should create the directory tree, default files and skeleton code + needed to support the API. diff --git a/doc/source/index.rst b/doc/source/index.rst index 6a02c60..4cdff0d 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -1,24 +1,43 @@ -.. gluon documentation master file, created by - sphinx-quickstart on Tue Jul 9 22:26:36 2013. - You can adapt this file completely to your liking, but it should at least - contain the root `toctree` directive. +.. + Gluon documentation master file. + Copyright 2017, AT&T + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Gluon documentation: + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + (Avoid deeper levels because they do not render well.) ================================= -Welcome to gluon's documentation! +Welcome to Gluon's documentation! ================================= Contents: .. toctree:: - :maxdepth:2 + :maxdepth:2 - readme - high_level_design - repo_structure - installation - usage - contributing - devref/index + readme.rst + devref/high_level_design.rst + devref/repo_structure.rst + installation.rst + usage.rst + contributing.rst + devref/index.rst ================== Indices and tables diff --git a/doc/source/installation.rst b/doc/source/installation.rst index c29cff2..d9e60cc 100644 --- a/doc/source/installation.rst +++ b/doc/source/installation.rst @@ -1,12 +1,53 @@ +.. + Copyright 2017, AT&T + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Gluon documentation: + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + (Avoid deeper levels because they do not render well.) + ============ Installation ============ -At the command line:: +There are 3 steps to install Gluon: - $ pip install gluon +* Step 1: Install ``etcd`` +* Step 2: Install Gluon Plugin and Proton Server +* Step 3 (Optional): Set up Mechanism Driver for Open Contrail -Or, if you have virtualenvwrapper installed:: +In addition, OPNFV Danube Release will provide a scenario in which Step 1 and +Step 2 can be done automatically to install Gluon. However, Step 3 (optional) +still needs to be manually set up. - $ mkvirtualenv gluon - $ pip install gluon +Contents: + +.. toctree:: + :maxdepth:2 + + installation/install_etcd.rst + installation/install_gluon.rst + installation/install_contrail.rst + +================== +Indices and tables +================== + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` diff --git a/doc/source/installation/install_contrail.rst b/doc/source/installation/install_contrail.rst new file mode 100644 index 0000000..77218f9 --- /dev/null +++ b/doc/source/installation/install_contrail.rst @@ -0,0 +1,62 @@ +.. + Copyright 2017, Juniper Networks + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Gluon documentation: + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + (Avoid deeper levels because they do not render well.) + +============ +Installation +============ + +Deploy Contrail Mechanism Driver +-------------------------------- + +Contrail Mechanism Driver code is available here: + +.. code-block:: bash + + https://github.com/codilime/ContrailMechanismDriver. + +Please follow the instructions there to deploy Contrail Mechanism Driver. + +Install Dependencies +-------------------- + +Contrail dependency is required: ``https://github.com/Juniper/contrail-python-api`` + +.. code-block:: bash + + git clone https://github.com/Juniper/contrail-python-api + cd contrail-python-api + sudo python setup.py install + +Configure Contrail Mechanism Driver +----------------------------------- + +* In file ``/etc/neutron/plugins/ml2/ml2_conf.ini``: + * Make sure that in section **ml2** key ``mechanism_drivers`` have value **contrail_driver** in list + +* In file ``/opt/stack/neutron/neutron.egg-info/entry_points.txt`` + * In section **neutron.ml2.mechanism_drivers** set key ``contrail_driver`` to **neutron.plugins.ml2.drivers.contrail_driver:ContrailMechanismDriver** + +Running +------- + +Neutron service need to be restarted diff --git a/doc/source/installation/install_etcd.rst b/doc/source/installation/install_etcd.rst new file mode 100644 index 0000000..1c49f49 --- /dev/null +++ b/doc/source/installation/install_etcd.rst @@ -0,0 +1,167 @@ +.. + Copyright 2016 and 2017, Nokia + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Gluon documentation: + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + (Avoid deeper levels because they do not render well.) + +========================== +Install ``etcd`` for Gluon +========================== + +The following instructions are from reference [1]_. + +You need at least three nodes for the ``etcd`` cluster to work, e.g. one +controller and two computes. + +On Each Node +------------ + +**STEP-1**: Download ``etcd`` Package + +.. code-block:: bash + + curl -L https://github.com/coreos/etcd/releases/download/v2.3.6/etcd-v2.3.6-linux-amd64.tar.gz -o etcd-v2.3.6-linux-amd64.tar.gz + Unzip/Untar the downloaded file + +**STEP-2**: Copy executables to ``/usr/local/bin`` + +.. code-block:: bash + + cd etcd-v2.3.6-linux-amd64 + sudo cp etcd /usr/local/bin + sudo cp etcdctl /usr/local/bin + +**STEP-3**: Create a directory for ``etcd`` data + +.. code-block:: bash + + sudo mkdir /var/etcd + +**STEP-4**: Create upstart ``init`` file: + +In ``/etc/init`` directory, create a file called ``etcd.conf``, and paste the +following into it: + +.. code-block:: bash + + description "etcd 2.0 distributed key-value store" + author "Scott Lowe " + start on (net-device-up + and local-filesystems + and runlevel [2345]) + stop on runlevel [016] + respawn + respawn limit 10 5 + script + if [ -f "/etc/default/etcd" ]; then + . /etc/default/etcd + fi + chdir /var/etcd + exec /usr/local/bin/etcd >>/var/log/etcd.log 2>&1 + end script + +**STEP-5**: Create an override file for ``etcd`` parameters: + +In ``/etc/init`` directory, create a file called ``etcd.override`` and paste +the following into it: + +.. code-block:: bash + + # Override file for etcd Upstart script providing some environment variables + env ETCD_INITIAL_CLUSTER="etcd-01=http://10.2.0.32:2380,etcd-02=http://10.2.0.102:2380,etcd-03=http://10.2.0.101:2380" + env ETCD_INITIAL_CLUSTER_STATE="new" + env ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-1" + env ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.2.0.32:2380" + env ETCD_DATA_DIR="/var/etcd" + env ETCD_LISTEN_PEER_URLS="http://10.2.0.32:2380" + env ETCD_LISTEN_CLIENT_URLS="http://10.2.0.32:2379,http://127.0.0.1:2379" + env ETCD_ADVERTISE_CLIENT_URLS="http://10.2.0.32:2379" + env ETCD_NAME="etcd-01" + + +**NOTE**: + +* The IP Addresses will need to be changed for your own machines! +* For each node in the cluster, the file will be slightly different, i.e. the + IP address of "advertise" and "listen" URLs, and ``etcd`` names will be for + each specific node. + +For instance, the files on the other two nodes would look like: + +.. code-block:: bash + + # Override file for etcd Upstart script providing some environment variables + env ETCD_INITIAL_CLUSTER="etcd-01=http://10.2.0.32:2380,etcd-02=http://10.2.0.102:2380,etcd-03=http://10.2.0.101:2380" + env ETCD_INITIAL_CLUSTER_STATE="new" + env ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-1" + env ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.2.0.102:2380" + env ETCD_DATA_DIR="/var/etcd" + env ETCD_LISTEN_PEER_URLS="http://10.2.0.102:2380" + env ETCD_LISTEN_CLIENT_URLS="http://10.2.0.102:2379,http://127.0.0.1:2379" + env ETCD_ADVERTISE_CLIENT_URLS="http://10.2.0.102:2379" + env ETCD_NAME="etcd-02" + +.. code-block:: bash + + # Override file for etcd Upstart script providing some environment variables + env ETCD_INITIAL_CLUSTER="etcd-01=http://10.2.0.32:2380,etcd-02=http://10.2.0.102:2380,etcd-03=http://10.2.0.101:2380" + env ETCD_INITIAL_CLUSTER_STATE="new" + env ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-1" + env ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.2.0.101:2380" + env ETCD_DATA_DIR="/var/etcd" + env ETCD_LISTEN_PEER_URLS="http://10.2.0.101:2380" + env ETCD_LISTEN_CLIENT_URLS="http://10.2.0.101:2379,http://127.0.0.1:2379" + env ETCD_ADVERTISE_CLIENT_URLS="http://10.2.0.101:2379" + env ETCD_NAME="etcd-03" + +**STEP-6**: Adjust ``iptables``: + +.. code-block:: bash + + sudo iptables -A INPUT -p tcp -m multiport --ports 2380,2379 -m comment --comment "etcd" -j ACCEPT + sudo invoke-rc.d iptables-persistent save + +**STEP-7**: Start the ``etcd`` server: + +As root: + +.. code-block:: bash + + initctl start etcd + +Or on ``ubuntu 14.04``, run: + +.. code-block:: bash + + sudo start etcd + +**STEP-8**: Verify the cluster is healty: + +.. code-block:: bash + + $ etcdctl cluster-health + member 5cd8baf7fb9d49b7 is healthy: got healthy result from http://10.2.0.102:2379 + member 9e95400273fd2acb is healthy: got healthy result from http://10.2.0.101:2379 + member ce8a4cd91a34b3f2 is healthy: got healthy result from http://10.2.0.32:2379 + cluster is healthy + +References + +.. [1] http://blog.scottlowe.org/2015/04/15/running-etcd-20-cluster/ diff --git a/doc/source/installation/install_gluon.rst b/doc/source/installation/install_gluon.rst new file mode 100644 index 0000000..e9dd760 --- /dev/null +++ b/doc/source/installation/install_gluon.rst @@ -0,0 +1,173 @@ +.. + Copyright 2016 and 2017, Nokia + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Gluon documentation: + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + (Avoid deeper levels because they do not render well.) + +====================================== +Install Gluon Plugin and Proton Server +====================================== + +Here the assumption is that ``etcd`` cluster is already installed. Otherwise, +please refer to [1]_. + +On Controller +------------- + +Assume the user logged in with sudo privileges. On an Ubuntu system: + +**STEP-1**: Clone Gluon Repository: + +.. code-block:: bash + + $ cd ~ + $ git clone https://github.com/openstack/gluon.git + $ git tag -l + # You should see the tag 1.0.0.0 + $ git checkout tags/1.0.0.0 + +Or you can clone from ``stable/ocata`` branch: + +.. code-block:: bash + + $ cd ~ + $ git clone https://github.com/openstack/gluon.git -b stable/ocata + +**STEP-2**: Create user and group for gluon and proton users + +.. code-block:: bash + + $ sudo adduser --system --group proton + +**STEP-3**: Create directories + +.. code-block:: bash + + $ sudo mkdir /opt/proton + $ sudo chown proton /opt/proton + $ sudo mkdir /etc/proton + +**STEP-4**: Setup ``iptables`` + +.. code-block:: bash + + $ sudo iptables -A INPUT -p tcp -m multiport --ports 2705 -m comment --comment gluon -j ACCEPT + $ sudo invoke-rc.d iptables-persistent save + + # Note: for Ubuntu 16.04, you may have to use netfilter-persistent as follows: + # sudo apt-get install netfilter-persistent + # sudo invoke-rc.d netfilter-persistent save + +**STEP-5**: Create config files and set permissions + +.. code-block:: bash + + $ sudo cat > /etc/proton/proton.conf < [OPTIONS] COMMAND[ARGS]... + + Options: + --api TEXT Name of API, one of ['net-l3vpn', 'test'] + --port INTEGER Port of endpoint (OS_PROTON_PORT) + --host TEXT Host of endpoint (OS_PROTON_HOST) + --help Show this message and exit. + +Mandatory Parameters +-------------------- + +``--api `` are mandatory parameters. For example, ``--api net-l3vpn``. + +Just typing the ``protonclient`` command shows you that those mandatory +parameters are required, and gives you general help information too: + +.. code-block:: bash + + $ protonclient + --api is not specified! + + Usage: protonclient --api [OPTIONS] COMMAND[ARGS]... + + Options: + --api TEXT Name of API, one of ['net-l3vpn', 'test'] + --port INTEGER Port of endpoint (OS_PROTON_PORT) + --host TEXT Host of endpoint (OS_PROTON_HOST) + --help Show this message and exit. + +Using L3VPN Proton +------------------ + +**NOTE** that there is a KNOWN BUG in the **Usage** message where the mandatory +parameters ``--api net-l3vpn`` are missing. The **examples** show you the +correct command line usage. + +.. code-block:: bash + + $ protonclient --api net-l3vpn + Usage: protonclient [OPTIONS] COMMAND [ARGS]... + + Options: + --help Show this message and exit. + + Commands: + interface-create + interface-delete + interface-list + interface-show + interface-update + port-create + port-delete + port-list + port-show + port-update + vpn-create + vpn-delete + vpn-list + vpn-show + vpn-update + vpnafconfig-create + vpnafconfig-delete + vpnafconfig-list + vpnafconfig-show + vpnafconfig-update + vpnbinding-create + vpnbinding-delete + vpnbinding-list + vpnbinding-show + vpnbinding-update + +Create ``Interface`` Object +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: bash + + $ protonclient --api net-l3vpn interface-create --help + Usage: protonclient interface-create [OPTIONS] + + Options: + --segmentation_id INTEGER Segmentation identifier [required] + --name TEXT Descriptive name of Object + --id TEXT UUID of Object + --segmentation_type [none|vlan|tunnel_vxlan|tunnel_gre|mpls] + Type of segmentation for this interface + [required] + --port_id TEXT Pointer to Port instance [required] + --port INTEGER Port of endpoint (OS_PROTON_PORT) + --host TEXT Host of endpoint (OS_PROTON_HOST) + --help Show this message and exit. + +There is a default ``Interface`` which is automatically created when a ``Port`` +is created. The UUID of this default ``Interface`` will be the same as the +UUID of the parent ``Port``. + +**For example: list the default ``Interface`` Object**: + +.. code-block:: bash + + $ protonclient --api net-l3vpn interface-list + { + "interfaces": [ + { + "name": "TestVPNPort_default", + "segmentation_id": 0, + "created_at": "2017-02-14T20:35:47.760126", + "updated_at": "2017-02-14T20:35:47.760126", + "port_id": "fe338d4c-2aef-4487-aa25-cb753bf02518", + "segmentation_type": "none", + "id": "fe338d4c-2aef-4487-aa25-cb753bf02518" + } + ] + } + +Create ``VPNAFConfig`` Object +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: bash + + $ protonclient --api net-l3vpn vpnafconfig-create --help + Usage: protonclient vpnafconfig-create [OPTIONS] + + Options: + --vrf_rt_value TEXT Route target string [required] + --export_route_policy TEXT Route target export policy + --import_route_policy TEXT Route target import policy + --vrf_rt_type [export_extcommunity|import_extcommunity|both] + Route target type [required] + --port INTEGER Port of endpoint (OS_PROTON_PORT) + --host TEXT Host of endpoint (OS_PROTON_HOST) + --help Show this message and exit. + +**For example: create a ``VPNAFConfig`` Object**: + +.. code-block:: bash + + $ protonclient --api net-l3vpn vpnafconfig-create --vrf_rt_type both --vrf_rt_value 1000:1000 + { + "vrf_rt_type": "both", + "vrf_rt_value": "1000:1000" + } + +Create ``VPN`` Object +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: bash + + $ protonclient --api net-l3vpn vpn-create --help + Usage: protonclient vpn-create [OPTIONS] + + Options: + --id TEXT UUID of VPN instance + --name TEXT Name of VPN [required] + --ipv4_family TEXT Comma separated list of route target strings + (VpnAfConfig) + --ipv6_family TEXT Comma separated list of route target strings + (VpnAfConfig) + --route_distinguishers TEXT Route distinguisher for this VPN + --description TEXT About the VPN + --port INTEGER Port of endpoint (OS_PROTON_PORT) + --host TEXT Host of endpoint (OS_PROTON_HOST) + --help Show this message and exit. + +You must specify the ``ipv4_family`` and ``ipv6_family`` attributes. The +values should match the ``vrf_rt_value`` of the ``vpnafconfig`` object. +The UUID of VPN instance ``id`` is generated by Proton and returned. + +.. code-block:: bash + + $ protonclient --api net-l3vpn vpn-create --name "TestVPN" --ipv4_family 1000:1000 --ipv6_family 1000:1000 --route_distinguishers 1000:1000 --description "My Test VPN" + { + "description": "My Test VPN", + "route_distinguishers": "1000:1000", + "created_at": "2017-02-14T20:37:58.592999", + "updated_at": "2017-02-14T20:37:58.592999", + "ipv6_family": "1000:1000", + "ipv4_family": "1000:1000", + "id": "b70b4bbd-aa40-48d7-aa4b-57cc2fd34010", + "name": "TestVPN" + } + +Create ``Port`` Object +~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: bash + + $ protonclient --api net-l3vpn port-create --help + Usage: protonclient port-create [OPTIONS] + + Options: + --device_id TEXT UUID of bound VM + --id TEXT UUID of Object + --host_id TEXT binding:host_id: Name of bound host + --mac_address TEXT MAC address for Port [required] + --vlan_transparency BOOLEAN Allow VLAN tagged traffic on Port + [required] + --device_owner TEXT Name of compute or network service (if + bound) + --mtu INTEGER MTU [required] + --vnic_type [normal|virtual|direct|macvtap|sriov|whole-dev] + Port should be attached to this VNIC type + [required] + --vif_details TEXT binding:vif_details: JSON string for VIF + details + --tenant_id TEXT UUID of Tenant owning this Port [required] + --admin_state_up BOOLEAN Admin state of Port [required] + --name TEXT Descriptive name of Object + --vif_type TEXT binding:vif_type: binding type for VIF + --profile TEXT JSON string for binding profile dictionary + --status [ACTIVE|DOWN] Operational status of Port [required] + --port INTEGER Port of endpoint (OS_PROTON_PORT) + --host TEXT Host of endpoint (OS_PROTON_HOST) + --help Show this message and exit. + +These values should be specified. + +The ``tenant_id`` should be obtained from OpenStack. + +The UUID of the object ``id`` is generated by the Proton and returned. + +**For example: create a ``Port`` Object**: + +.. code-block:: bash + + $ protonclient --api net-l3vpn port-create --mac_address c8:2a:14:04:43:80 --mtu 1500 --admin_state_up True --name "TestVPNPort" --vlan_transparency True --vnic_type normal --vif_type ovs --status ACTIVE --tenant_id 5205b400fa6c4a888a0b229200562229 + { + "profile": null, + "status": "ACTIVE", + "vif_type": "ovs", + "name": "TestVPNPort", + "device_owner": null, + "admin_state_up": true, + "tenant_id": "5205b400fa6c4a888a0b229200562229", + "created_at": "2017-02-14T20:35:47.749427", + "vif_details": null, + "updated_at": "2017-02-14T20:35:47.749427", + "mtu": 1500, + "vnic_type": "normal", + "vlan_transparency": true, + "mac_address": "c8:2a:14:04:43:80", + "host_id": null, + "id": "fe338d4c-2aef-4487-aa25-cb753bf02518", + "device_id": null + } + +As we mentioned earlier, a default ``interface`` object is created too, and +attached to this ``port`` object. + +At this point you have a ``port`` object, default ``interface`` object and a +``vpn`` service object created. + +View ``VPN`` and ``Port`` Objects +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can view the values with the following commands: + +.. code-block:: bash + + $ protonclient --api net-l3vpn vpn-list + { + "vpns": [ + { + "description": "My Test VPN", + "route_distinguishers": "1000:1000", + "created_at": "2017-02-14T20:37:58.592999", + "updated_at": "2017-02-14T20:37:58.592999", + "ipv6_family": "1000:1000", + "ipv4_family": "1000:1000", + "id": "b70b4bbd-aa40-48d7-aa4b-57cc2fd34010", + "name": "TestVPN" + } + ] + } + $ + $ protonclient --api net-l3vpn port-list + { + "ports": [ + { + "profile": null, + "status": "ACTIVE", + "vif_type": "ovs", + "name": "TestVPNPort", + "device_owner": null, + "admin_state_up": true, + "tenant_id": "5205b400fa6c4a888a0b229200562229", + "created_at": "2017-02-14T20:35:47.749427", + "vif_details": null, + "updated_at": "2017-02-14T20:35:47.749427", + "mtu": 1500, + "vnic_type": "normal", + "vlan_transparency": true, + "mac_address": "c8:2a:14:04:43:80", + "host_id": null, + "id": "fe338d4c-2aef-4487-aa25-cb753bf02518", + "device_id": null + } + ] + } + +Create ``VPNBinding`` Object +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You need to create a ``vpnbinding`` object to tie the ``Interface`` and the +``Service`` together in order to achieve service binding. + +.. code-block:: bash + + $ protonclient --api net-l3vpn vpnbinding-create --help + Usage: protonclient vpnbinding-create [OPTIONS] + + Options: + --interface_id TEXT Pointer to Interface instance [required] + --gateway TEXT Default gateway + --ipaddress TEXT IP Address of port + --subnet_prefix INTEGER Subnet mask + --service_id TEXT Pointer to Service instance [required] + --port INTEGER Port of endpoint (OS_PROTON_PORT) + --host TEXT Host of endpoint (OS_PROTON_HOST) + --help Show this message and exit. + +The ``vpnbinding`` object is created by using an ``interface_id`` and a +``service_id``. In our example, a default ``interface`` object was +automatically created and attached to a ``port`` object when the ``port`` +object was created. The ``Service`` is ``vpn``. Thus we use the ``id`` of the +default ``interface`` object, and the ``id`` of the ``vpn`` object. + +**For example: create a ``VPNBinding`` Object**: + +.. code-block:: bash + + $ protonclient --api net-l3vpn vpnbinding-create --interface_id fe338d4c-2aef-4487-aa25-cb753bf02518 --service_id b70b4bbd-aa40-48d7-aa4b-57cc2fd34010 --ipaddress 10.10.0.2 --subnet_prefix 24 --gateway 10.10.0.1 + { + "created_at": "2017-02-14T20:39:52.382433", + "subnet_prefix": 24, + "updated_at": "2017-02-14T20:39:52.382433", + "interface_id": "fe338d4c-2aef-4487-aa25-cb753bf02518", + "service_id": "b70b4bbd-aa40-48d7-aa4b-57cc2fd34010", + "ipaddress": "10.10.0.2", + "gateway": "10.10.0.1" + } + +View ``VPNBinding`` Objects +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: bash + + $ protonclient --api net-l3vpn vpnbinding-list + { + "vpnbindings": [ + { + "created_at": "2017-02-14T20:39:52.382433", + "subnet_prefix": 24, + "updated_at": "2017-02-14T20:39:52.382433", + "interface_id": "fe338d4c-2aef-4487-aa25-cb753bf02518", + "service_id": "b70b4bbd-aa40-48d7-aa4b-57cc2fd34010", + "ipaddress": "10.10.0.2", + "gateway": "10.10.0.1" + } + ] + } + +At this point you have had all of the information needed for an L3VPN Port in +Proton. + +Create VM and Bind our L3VPN Port +--------------------------------- + +.. code-block:: bash + + $ nova --debug boot --flavor 1 --image cirros --nic port-id=fe338d4c-2aef-4487-aa25-cb753bf02518 TestGluon + +When bound, the ``etcd`` data will look like: + +.. code-block:: bash + + $ etcdctl --endpoint http://192.0.2.4:2379 ls / --recursive + /proton + /proton/net-l3vpn + /proton/net-l3vpn/Port + /proton/net-l3vpn/Port/fe338d4c-2aef-4487-aa25-cb753bf02518 + /proton/net-l3vpn/Interface + /proton/net-l3vpn/Interface/fe338d4c-2aef-4487-aa25-cb753bf02518 + /proton/net-l3vpn/VpnService + /proton/net-l3vpn/VpnService/b70b4bbd-aa40-48d7-aa4b-57cc2fd34010 + /proton/net-l3vpn/VpnBinding + /proton/net-l3vpn/VpnBinding/fe338d4c-2aef-4487-aa25-cb753bf02518 + /gluon + /gluon/port + /gluon/port/fe338d4c-2aef-4487-aa25-cb753bf02518 + $ + +You may use other command in ``etcd`` to check specific data record, such as: + +.. code-block:: bash + + # etcdctl --endpoint http://192.0.2.4:2379 get /proton/net-l3vpn/Port/fe338d4c-2aef-4487-aa25-cb753bf02518 + +To Use Gluon in a Project +------------------------- + +.. code-block:: bash import gluon + +References + +.. [1] installation +