From 555fdf71aa5a461357ad6d2202d37c6754ed7a8f Mon Sep 17 00:00:00 2001 From: Alexander Gordeev Date: Thu, 1 Oct 2015 20:43:25 +0300 Subject: [PATCH] Add README.md and LICENSE This change provides: * the README to include more useful information about fuel-agent as an open source project. * LICENSE file (carefully borrowed from fuel-web repo) Change-Id: I234a448673bbfcbddbe57c76d1d5449968c0bded Closes-Bug: #1500058 --- LICENSE | 176 +++++++++++++++++++++++++++++++++++++++++++++++ README.md | 202 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 378 insertions(+) create mode 100644 LICENSE create mode 100644 README.md diff --git a/LICENSE b/LICENSE new file mode 100644 index 00000000..68c771a0 --- /dev/null +++ b/LICENSE @@ -0,0 +1,176 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + diff --git a/README.md b/README.md new file mode 100644 index 00000000..c323e101 --- /dev/null +++ b/README.md @@ -0,0 +1,202 @@ +fuel-agent README +================= + +## Table of Contents + +- [Overview](#overview) +- [Structure](#structure) +- [Usage](#usage) +- [Development](#development) +- [Core Reviewers](#core-reviewers) +- [Contributors](#contributors) + +## Overview + +fuel-agent is nothing more than just a set of data driven executable +scripts. +- One of these scripts is used for building operating system images. One can run +this script on wherever needed passing a set of repository URIs and a set of +package names that are to be installed into the image. +- Another script is used for the actual provisioning. This script being installed +into a ramdisk (live image) can be run to provision an operating system on a hard drive. +When running one needs to pass input data that contain information about disk +partitions, initial node configuration, operating system image location, etc. +This script is to prepare disk partitions according to the input data, download +operating system images and put these images on partitions. + + +### Motivation +- Native operating installation tools like anaconda and debian-installer are: + * hard to customize (if the case is really non-trivial) + * hard to troubleshoot (it is usually quite difficult to understand which log file + contains necessary information and how to run those tools in debug mode) +- Image based approach to operating system installation allows to make this + process really scalable. For example, we can use BitTorrent based image + delivery scheme when provisioning that makes the process easily scalable up + to thousands of nodes. +- When provisioning we can check hash sum of the image and use other validation + mechanisms that can make the process more stable. + + +### Designed to address requirements +- Support various input data formats (pluggable input data drivers) +- Support plain partitions, lvm, md, root on lvm, etc. +- Be able to do initial node configuration (network, mcollective, puppet, ntp) +- Be able to deploy standalone OS (local kernel, local bootloader) +- Support various image storages (tftp, http, torrent) +- Support various image formats (compressed, disk image, fs image, tar image) + +### Design outline +- Use cloud-init for initial node configuration +- Avoid using parted and lvm native python bindings (to make it easy to + troubleshoot and modify for deployment engineers) +- No REST API, just executable entry points (like /usr/bin/fa_*) +- Passing input data either via file (--input_data_file) or CLI parameter (--input_data) +- Detailed logging of all components + + +## Structure + +### Basic Repository Layout + +``` +fuel-agent +├── cloud-init-templates +├── contrib +├── debian +├── etc +├── fuel_agent +│   ├── cmd +│   ├── drivers +│   ├── objects +│   ├── openstack +│   ├── tests +│   ├── utils +├── openstack-common.conf +├── README.md +├── LICENSE +├── requirements.txt +├── run_tests.sh +├── setup.cfg +├── setup.py +├── specs +├── test-requirements.txt +``` + +### root + +The root level contains important repository documentation and license information. +It also contais files which are typical for the infracture of python project such +as requirements.txt and setup.py + +### cloud-init-templates + +This folder contains Jinja2 templates to prepare [cloud-init](https://cloudinit.readthedocs.org/en/latest/) related data for [nocloud](http://cloudinit.readthedocs.org/en/latest/topics/datasources.html#no-cloud) [datasource](http://cloudinit.readthedocs.org/en/latest/topics/datasources.html#what-is-a-datasource). + +### contrib + +This directory contains third party code that is not a part of fuel-agent itself but +can be used together with fuel-agent. + +### debian + +This folder contains the DEB package specification. +Included debian rules are mainly suitable for Ubuntu 12.04 or higher. + +### etc + +This folder contains the sample config file for fuel-agent. Every parameter is well documented. +We use oslo-config as a configuration module. + +### fuel_agent + +This folder contains the python code: drivers, objects, unit tests and utils, manager and entry points. + +- fuel_agent/cmd/agent.py + * That is where executable entry points are. It reads input data and + instantiates Manager class with these data. +- fuel_agent/manager.py + * That is the file where the top level agent logic is implemented. + It contains all those methods which do something useful (do_*) +- fuel_agent/drivers + * That is where input data drivers are located. + (Nailgun, NailgunBuildImage, Simple etc.) + Data drivers convert json into a set of python objects. +- fuel_agent/objects + * Here is the place where python objects are defined. fuel-agent manager + does not understand any particular data format except these objects. + For example, to do disk partitioning we need PartitionScheme object. + PartitionScheme object in turn contains disk labels, plain partitions, + lvm, md, fs objects. This PartitionScheme object is to be created by input + data driver. +- fuel_agent/utils + * That is the place where we put the code which does something useful on the OS + level. Here we have simple parted, lvm, md, grub bindings, etc. + +### specs + +This folder contains the RPM package specfication file. +Included RPM spec is mainly suitable for Centos 6.x or higher. + + +## Usage + +### Use case #1 (Fuel) + +fuel-agent is used in Fuel project as a part of operating system provisioning scheme. +When a user starts deployment of OpenStack cluster, the first task is to install +an operating system on slave nodes. First, Fuel runs fuel-agent on the master node +to build OS images. Once images are built, Fuel then runs fuel-agent on slave nodes +using Mcollective. Slave nodes are supposed to be booted with so called bootstrap ramdisk. +Bootstrap ramdisk is an in-memory OS where fuel-agent is installed. + +Detailed documentation on this case is available here: +* [Image based provisionig](https://docs.mirantis.com/openstack/fuel/fuel-master/reference-architecture.html#image-based-provisioning) +* [fuel-agent](https://docs.mirantis.com/openstack/fuel/fuel-master/reference-architecture.html#fuel-agent) +* [Operating system provisioning](https://docs.mirantis.com/openstack/fuel/fuel-master/reference-architecture.html#operating-system-provisioning) +* [Image building](https://docs.mirantis.com/openstack/fuel/fuel-master/reference-architecture.html#image-building) + +### Use case #2 (Independent on Fuel) + +fuel-agent can easily be used in third party projects as a convenient operating system +provisioning tool. As described above fuel-agent is fully data driven and supports +various input data formats using pluggable input data drivers. Currently there are three +input data drivers available. Those are + +- NailgunBuildImage and Nailgun + * Build image and provisioning input data drivers used in Fuel project. To use them + independently read Fuel documentation. +- NailgunSimpleDriver + * fuel-agent native partitioning input data driver. It is just a de-serializer for + fuel-agent PartitionScheme object. + +In order to be able to use another specific data format one can implement his own data +driver and install it independently. fuel-agent uses stevedore to find installed drivers. +A new driver needs to be exposed via fuel_agent.driver setuptools name space. See for example +setup.cfg file where entry points are defined. + +One can also take a look at ```contrib``` directory for some additional examples. + + +### How to install + +fuel-agent can be installed either using RPM/DEB packages or using ```python setup.py install```. + + +## Development + +fuel-agent currently is a subproject of Fuel project. So, we follow the same development +practices as Fuel itself. + +* [Fuel Development Documentation](https://docs.fuel-infra.org/fuel-dev/) +* [Fuel How to Contribute](https://wiki.openstack.org/wiki/Fuel/How_to_contribute) + + +## Core Reviewers + +* [fuel-agent cores](https://review.openstack.org/#/admin/groups/995,members) + + +## Contributors + +* [Stackalytics](http://stackalytics.com/?release=all&project_type=all&module=fuel-agent&metric=commits)