Update Readme
Change-Id: I0bdd1808fce344156f2791e38832122a9aad30f0
This commit is contained in:
Xicheng Chang
parent
9777aa9df8
commit
c946aeb254
10
README.md
10
README.md
|
|
@ -14,7 +14,7 @@ tools (Cobbler, Chef, Ansible) and allows third-parties (vendors) plugins for ha
|
|||
|
||||
##Quick Guide to Developers
|
||||
|
||||
### Get started with coding and contributing
|
||||
###Get started with coding and contributing
|
||||
|
||||
** Before everything, setup your environment:**
|
||||
|
||||
|
|
@ -30,15 +30,15 @@ ii. Dedicatedly create a virtual environment for your development. You can use
|
|||
|
||||
_Note: the above assumes you use virtualenvwrapper to manage your virtualenv_
|
||||
|
||||
** 1. Checkout source **
|
||||
**1. Checkout source**
|
||||
|
||||
(compass-core-env)$git checkout https://github.com/stackforge/compass-core.git
|
||||
|
||||
** 2. Run the tests making sure you are working on a clean and working code base** (i.e., did someone else break the code)
|
||||
**2. Run the tests making sure you are working on a clean and working code base** (i.e., did someone else break the code)
|
||||
|
||||
``` (compass-core-env)$tox -r ```
|
||||
|
||||
** 3. Write your change, and make sure test your code thoroughly** (otherwise, reviewers won't approve it)
|
||||
**3. Write your change, and make sure test your code thoroughly** (otherwise, reviewers won't approve it)
|
||||
|
||||
``` (compass-core-env)$git branch -b [TOPIC]```
|
||||
|
||||
|
|
@ -46,7 +46,7 @@ working, working, working on the [TOPIC] branch ...
|
|||
|
||||
``` (compass-core-env)$tox -r ``` (make sure your new code still works)
|
||||
|
||||
** 4. Submit for review **
|
||||
**4. Submit for review**
|
||||
|
||||
```(compass-core-env)$git review ```
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,66 @@
|
|||
Compass Binaries and Scripts
|
||||
=============================
|
||||
|
||||
bin/ contains compass heavy-lifting utility scripts and binaries. These scripts are often called by different components. Some are from core python modules and some are from compass daemon and other services. Most files in `bin/` are placed under `/opt/compass/bin/` after install.sh is complete. Some of them will go to `/usr/bin/` or `/etc/init.d/` as system binaries or services.
|
||||
|
||||
###Directories and Files
|
||||
|
||||
Below is a walkthrough of all directories and files.
|
||||
|
||||
* ansible-callbacks/ - contains callback scripts for ansible installer.
|
||||
* playbook_done.py - triggered by ansible when all playbooks are successfully executed.
|
||||
Then the script will call compass API to report ansible "complete" status.
|
||||
* chef/ - utility scripts for chef installer.
|
||||
* addcookbooks.py - upload all chef cookbooks to the chef server.
|
||||
* adddatabags.py - (deprecated) upload all chef databags to the chef server.
|
||||
* addroles.py - upload all chef roles to the chef server.
|
||||
* clean_clients.sh - remove all chef clients on the chef server.
|
||||
* clean_environments.sh - remove all chef environments on the chef server.
|
||||
* clean_nodes.sh - remove all chef nodes on the chef server.
|
||||
* cobbler/ - utility scripts for cobbler installer
|
||||
* remove_systems.sh - remove all systems on the cobbler server.
|
||||
* clean_installation_logs.py - remove all the installation logs.
|
||||
* clean_installers.py - remove all configurations and data from all installers.
|
||||
* client.sh - sample client script to call client.py
|
||||
* client.py - compass python client that calls API and deploy a cluster based on given configurations.
|
||||
* compass_check.py - binary file that is placed as /usr/bin/compass. This is the main entrance of compass check CLI.
|
||||
* compassd - (deprecated) old compass daemon file
|
||||
* compass_wsgi.py - compass wsgi module.
|
||||
* csvdeploy.py - script that enable the deployment of clusters from spreadsheets.
|
||||
* delete_clusters.py - script that deletes all given clusters and their underlying hosts.
|
||||
* manage_db.py - utility binary that manages database.
|
||||
* poll_switch.py - utility script to poll machine mac addresses that are connected to certain switches.
|
||||
* progress_update.py - main script to run as a service to update hosts installing progresses.
|
||||
* query_switch.py - independent script to query a switch.
|
||||
* refresh.sh - refresh compass-db, restart compass services and clean up all installers.
|
||||
* runserver.py - manually run a compass server instance.
|
||||
* switch_virtualenv.py.template - template of switch_virtualenv.py. This script enables switching between python
|
||||
virtual environments.
|
||||
|
||||
###Script Location and Calling Modules
|
||||
Script name | Location | Called by
|
||||
--- | --- | ---
|
||||
ansible-callbacks/playbook_done.py | /opt/compass/bin/ansible-callbacks/playbookd_done.py | ***ansible-playbook***
|
||||
chef/addcookbooks.py | /opt/compass/bin/addcookbooks.py | ***install/chef.sh***
|
||||
chef/adddatabags.py(deprecated) | /opt/compass/bin/addcookbooks.py | None
|
||||
chef/addroles.py | /opt/compass/bin/addroles.py | ***install/chef.sh***
|
||||
chef/clean_clients.sh | /opt/compass/bin/clean_clients.sh | ***compass.tasks.clean_package_installer***
|
||||
chef/clean_environments.sh | /opt/compass/bin/clean_environments.sh | ***compass.tasks.clean_package_installer***
|
||||
chef/clean_nodes.sh | /opt/compass/bin/clean_nodes.sh | ***compass.tasks.clean_package_installer***
|
||||
cobbler/remove_systems.sh | /opt/compass/bin/remove_systems.sh | ***compass.tasks.clean_os_installer***
|
||||
clean_installation_logs.py | /opt/compass/bin/clean_installation_logs.py | ***bin/refresh.sh***
|
||||
clean_installers.py | /opt/compass/bin/clean_installers.py | ***bin/refresh.sh***
|
||||
client.sh | /opt/compass/bin/client.sh | sample client
|
||||
client.py | /opt/compass/bin/client.py | ***regtest/regtest.sh***
|
||||
compsas_check.py | /opt/compass/bin/compass_check.py | ***compass check cli***
|
||||
compassd(deprecated) | None | None
|
||||
compass_wsgi.py | /var/www/compass/compass.wsgi | ***Apache daemon***
|
||||
csvdeploy.py | /opt/compass/bin/csvdeploy.py | command-line script
|
||||
delete_clusters.py | /opt/compass/bin/delete_clusters.py | command-line script
|
||||
manage_db.py | /opt/compass/bin/manage_db.py | ***install/compass.sh*** and command-line script
|
||||
poll_switch.py | /opt/compass/bin/poll_switch.py | command-line script
|
||||
progress_update.py | /opt/compass/bin/progress_update.py | ***compass-progress-updated daemon***
|
||||
query_switch.py | /opt/compass/bin/query_switch.py | command-line script
|
||||
refresh.sh | /opt/compass/bin/refresh.sh | command-line script
|
||||
runserver.py | /opt/compass/bin/runserver.py | command-line script
|
||||
switch_virtualenv.py.template | /opt/compass/bin/switch_virtualenv.py | ***all scripts using this library***
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
Compass Core Python Modules
|
||||
===========================
|
||||
`compass/` is where all the core python modules of compass are, including API, DB, tasks and installers and so on.
|
||||
|
||||
##Direcotries and Files
|
||||
|
||||
* actions/ - compass heavy-lifting actions. This directory includes wrappers of heavy-lifting utility functions,
|
||||
such as clean.py, delete.py and deploy.py. It also has complete modules, e.g. health_check/ that does a health
|
||||
check on compass and output diagnoses.
|
||||
* api/ - API related files go here.
|
||||
* api.py - defines compass RESTful API.
|
||||
* api.raml - raml file to display/document compass RESTful api.
|
||||
* auth_handler.py - handles API authentication.
|
||||
* exception_handlers.py - handles API exceptions.
|
||||
* utils.py - utilities for API.
|
||||
* v1/ - deprecated directory.
|
||||
* apiclient/ - API client.
|
||||
* example.py - example code to deploy a cluster by calling compass API client.
|
||||
* restful.py - compass API client library.
|
||||
* v1/ - deprecated directory.a
|
||||
* db/ - compass database modules.
|
||||
* api/ - Database level API interfaces, which includes all compass primitive data types.
|
||||
* callback.py - metadata callback methods.
|
||||
* config_validation/ - configuration validation module.
|
||||
* exception.py - compass defined exceptions for database module.
|
||||
* models.py - database model file, defining all compass database tables.
|
||||
* validator.py - database validation methods.
|
||||
* vi/ - deprecated directory.
|
||||
* deployment/ - backend deployment module that handles upstream data and dumps to installers
|
||||
* deployment_manager.py - deployment dispatcher that defines interfaces and retrieves/updates configurations.
|
||||
* installers - contains base installers and its children installers such as os_installer and pk_installer
|
||||
(package_installer). These installers here are the "plugin" kind of files that interact with installer
|
||||
tools(e.g. chef) that do the real installation tricks.
|
||||
* utils/ - utility module that contains a constant.py which defines all keyword variables for deployment.
|
||||
* hdsdiscovery/ - hardware discovery module, mainly for mac-address retrieval from switches.
|
||||
* base.py - a base class that can be extended by vendors under vendor/ directory.
|
||||
* error.py - hdsdiscovery module error handling.
|
||||
* hdmanager.py - manages hdsdiscovery functionalities.
|
||||
* SNMP_CONFIG.md - instructions on how to install and configure snmp related modules/packages.
|
||||
* utils.py - contains utility functions for hdsdiscovery.
|
||||
* vendors/ - switch/hardware vendor specific plugins. To support a new vendor:
|
||||
1. Make sure the switch product supports SNMP.
|
||||
2. Make sure the switch product uses standard MIB.
|
||||
3. Add a corresponding plugin under vendors/.
|
||||
using arista as an example, first a python file with the same name of its parent directory should be added
|
||||
(e.g: arista.py). Define the class_name in the vendor file as "Arista" and give the class identical name.
|
||||
Then create a subdirectory called "plugins" and place mac.py under it.
|
||||
* log_analyzor/ - compass progress tracking module.
|
||||
* adapter_matcher.py - module to provide installing progress calculation for the adapters.
|
||||
* file_matcher.py - updates installing progresses by processing the log files.
|
||||
* line_matcher.py - updates/gets progress when a matched line is found in log files.
|
||||
* progress_calculator.py: provides functions to update installing progresses.
|
||||
* tasks/ - all celery tasks are defined here.
|
||||
* client.py - module to setup celery client.
|
||||
* tasks.py - defines all celery tasks.
|
||||
* tests/ - all unittest code
|
||||
* test_serverside/ - tests for installers(now only supports chef).
|
||||
* utils/ - compass core module utility functions.
|
||||
|
|
@ -0,0 +1,63 @@
|
|||
Install Scripts for Compass
|
||||
===========================
|
||||
|
||||
You can install Compass to a CentOS-6.5/6.6 system by running install scripts.
|
||||
|
||||
##Quick steps to install Compass
|
||||
|
||||
i. run ```./install.sh```
|
||||
|
||||
ii. install.sh will pop up questions to understand your configurations. It will provide default values based on calculated results. For a more detailed explanation of these questions please [visit Compass user doc on Compass official site](http://www.syscompass.org/install.html#step-one). In most cases, using the default values will do the trick
|
||||
|
||||
iii. After answering all the questions, install.sh will take over the installation. The whole process may take 30-45 minutes depending on your network latency and the choices you made.
|
||||
|
||||
##Files
|
||||
|
||||
* ansible.sh - script to install ansible package, called by install.sh.
|
||||
* chef.sh - script to install and configure chef-server and knife, called by install.sh.
|
||||
* cobbler.sh - script to install and configure cobbler server, called by install.sh
|
||||
* compass.sh - script to install and configure compass and its components, called by install.sh
|
||||
* compass_web.sh - script to setup compass front-end UI and http server for local repositories, called by install.sh
|
||||
* dependency.sh - script to install all the dependency packages for compass, called by install.sh
|
||||
* install.conf - conf file that contains all the variables used by all sh files. You can give default values to
|
||||
variables so that install script will not ask questions. Examples will be given in next section.
|
||||
* install.conf.template - a template/example for install.conf, this file is used by compass regtest to install compass.
|
||||
* install_func.sh - contains the common functions for all scripts.
|
||||
* install.sh - main entrance.
|
||||
* prepare.sh - script to prepare environments, pre-download ISOs and update services/packages.
|
||||
* setup_env.sh - script to cache all the user inputs and persist the inputs into an env.conf file for scripts to use.
|
||||
This makes sure that when somehow the install process is aborted, running each script file separtely still works.
|
||||
|
||||
##Example of install.conf variables(for developers)
|
||||
For example, in install.conf there's no default value set for NIC variable, thus install.sh will ask you to fill in during the script run:
|
||||
|
||||
|
||||
export NIC=${NIC:-}
|
||||
If you give it a default value, install.sh will take it and skip the question:
|
||||
|
||||
export NIC=$(NIC:-eth0}
|
||||
Now *eth0* will be used by install.sh to answer the question of `Please enter the NIC:` If you wish to contribute to compass and add your own variables for compass install scripts to use, you can simply call `loadvars()` function in `install.sh`:
|
||||
|
||||
loadvars foo "bar"
|
||||
Adding this line to `install.sh` will result in a new question being asked as:
|
||||
"```Please enter the foo```" and the default value for `foo` is `bar`.
|
||||
|
||||
You would also need to add
|
||||
|
||||
foo=\${foo:-$foo}
|
||||
into `setup_env.sh` to make sure the user input value gets persisted into `env.conf`.
|
||||
|
||||
##Q&A
|
||||
Q: I am a new user of Compass and I do not know too much about DevOps. I am not sure how to answer those questions asked by `install.sh`. What should I do?
|
||||
|
||||
A: `install.sh` can give default answers based on the calculated results. If you are not sure about your environment, you can just simply follow the default values(press ENTER).
|
||||
|
||||
Q: Some questions ask me to provide `Y/N` and won't accept `ENTER` as an answer?
|
||||
|
||||
A: Yes those questions are flags for supporting different distributions. If you are not familiar with those distributions, simply answer `Y` to the ones you would like to support. Also answer `Y` to all those questions is not a bad idea.
|
||||
|
||||
Q: Installation was aborted for some reason and I do not wish to answer those questions all over again, can I run scripts one by one?
|
||||
|
||||
A: Yes, please run the scripts in this order:
|
||||
|
||||
`dependency.sh -> prepare.sh -> cobbler.sh -> chef.sh -> ansible.sh -> compass-web.sh -> compass.sh`
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
Regression Test for Compass
|
||||
===========================
|
||||
Compass uses Zuul-Jenkins combination for its continuous integration. All compass regtests are defined in this directory. The main entrance for regtests is `regtest.sh` all the `.conf` files are different test cases. This example table shows the relationship of `regtest/`, `zuul server` and `Jenkins server`.
|
||||
|
||||
Jenkins job name | regtest conf file | enabled in zuul |
|
||||
--- | --- | ---
|
||||
compass-ci-1 | regtest1.conf | master branch
|
||||
compass-daily-2 | regtest8.conf | dev branch
|
||||
compass-periodic-check-5 | regtest10.conf | none
|
||||
compass-weekly | regtest3.conf | both
|
||||
|
||||
The above example shows the relationship. The direcotry `ansible` is for ansible installer's regtest only.
|
||||
|
|
@ -58,7 +58,7 @@ Currently customized service credentials are not supported(will be soon). Defaul
|
|||
|
||||
### Ansible code
|
||||
|
||||
* All ansible-related code on github: [https://github.com/stackforge/compass-adatpers](https://github.com/stackforge/compass-adapters), switch to branch *dev/experimental* and ansible files are under /ansible directory
|
||||
* All ansible-related code on github: [https://github.com/stackforge/compass-adatpers](https://github.com/stackforge/compass-adapters).
|
||||
|
||||
* On the Compass virtualbox, to access ansible playbooks, go to ```/var/ansible/openstack_juno```. To check/modify the ansible code on any existing environment, go to ```/var/ansible/run/{{ cluster_name }}```
|
||||
|
||||
|
|
@ -70,6 +70,6 @@ To clean up an existing compass cluster, do ```sudo /opt/compass/bin/refresh.sh`
|
|||
|
||||
To make sure refresh has cleaned up the ansible running directory, check if ```/var/ansible/run``` is empty
|
||||
|
||||
### DO NOT:
|
||||
### NOTE:
|
||||
|
||||
* Select adapters other than "OpenStack Juno"
|
||||
* To make sure the `box` file is at its minimum size, it only contains OpenStack Juno release related packages and ISOs. Selecting adapters other than "OpenStack Juno" will result in a failure deployment.
|
||||
|
|
|
|||
Loading…
Reference in New Issue