Merge "Adds minimum documentation into README.rst file"

This commit is contained in:
Jenkins 2015-12-18 10:16:26 +00:00 committed by Gerrit Code Review
commit a50dd7bb69
2 changed files with 230 additions and 1 deletions

1
README
View File

@ -1 +0,0 @@
init

230
README.rst Normal file
View File

@ -0,0 +1,230 @@
Fuel Plugin CI
==============
Overview
--------
Components, concept
Jenkins and web server for logs installation
--------------------------------------------
First you should install puppet master and run the manifests.
All nodes described in ``manifests/site.pp`` file.
CI needs the following amount of nodes:
* one node for jenkins master
* one more for jenkins slave
* one for log publication.
These nodes should be described in ``manifests/site.pp`` with necessary classes:
``class { '::fuel_project::jenkins::slave':}``
``class { '::fuel_project::jenkins::master':}``
``class {::fuel_project::web:}``
Run install script ``sudo puppet-manifests/bin/install_puppet_master.sh`` on every node.
Gerrit Integration overview
+++++++++++++++++++++++++++
In general, the installation should meet the following
requirements:
* Anonymous users can read all projects.
* All registered users can perform informational code review (+/-1) on any project.
* Jenkins can perform verification (blocking or approving: +/-1).
* All registered users can create changes.
* Members of core group can perform full code review (blocking or approving: +/- 2)
and submit changes to be merged.
#. Make sure you have gerrit account on review.openstack.org::
ssh -p 29418 review.openstack.org "gerrit create-account \
--group 'Third-Party CI' \
--full-name 'Some CI Bot' \
--email ci-bot@third-party.org \
--ssh-key 'ssh-rsa AAAAB3Nz...zaUCse1P ci-bot@third-party.org' \
some-ci-bot
Jenkins gerrit plugin configuration
+++++++++++++++++++++++++++++++++++
#. The settings looks as follows:
<pic>
#. It's main gerrit configuration window. You should add a gerrit server.
<pic>
#. Vote configuration.
#. Log publication
The result of job are artifacts - logs and packages. Logs should be published on special web servers, where it can be accessible via gerrit. Web server deploys with puppet class fuel_project::web. Logs copy via ssh by job fuel-plugins.publish_logs. You should add new user with rsa key installed and necessary path accessible for write (like /var/www/logs). REPORTED_JOB_URL variable is responsible for url of logs in gerrit.
Jenkins plugins installation
-----------------------------
We recommend to install these plugins for Jenkins.
Some of them are necessary for CI and other is just useful and make your jenkins experience easier:
`AnsiColor <https://wiki.jenkins-ci.org/display/JENKINS/AnsiColor+Plugin>`_
`Ant Plugin <https://wiki.jenkins-ci.org/display/JENKINS/AnsiColor+Plugin>`_
`build timeout plugin <https://wiki.jenkins-ci.org/display/JENKINS/Build-timeout+Plugin>`_
`conditional buildstep <https://wiki.jenkins-ci.org/display/JENKINS/Conditional+BuildStep+Plugin>`_
`Copy Artifact Plugin <https://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin>`_
`Credentials Plugin <https://wiki.jenkins-ci.org/display/JENKINS/Credentials+Plugin>`_
`CVS Plug-in <https://wiki.jenkins-ci.org/display/JENKINS/CVS+Plugin>`_
`description setter plugin <https://wiki.jenkins-ci.org/display/JENKINS/Description+Setter+Plugin>`_
`Email Extension Plugin <https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin>`_
`Environment Injector Plugin <https://wiki.jenkins-ci.org/display/JENKINS/EnvInject+Plugin>`_
`External Monitor Job Type Plugin <https://wiki.jenkins-ci.org/display/JENKINS/Monitoring+external+jobs>`_
`Gerrit Trigger <https://wiki.jenkins-ci.org/display/JENKINS/Gerrit+Trigger>`_
`GIT client plugin <https://wiki.jenkins-ci.org/display/JENKINS/Git+Client+Plugin>`_
`GIT plugin <https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin>`_
`Groovy <https://wiki.jenkins-ci.org/display/JENKINS/Groovy+plugin>`_
`Heavy Job Plugin <https://wiki.jenkins-ci.org/display/JENKINS/Heavy+Job+Plugin>`_
`HTML Publisher plugin <https://wiki.jenkins-ci.org/display/JENKINS/HTML+Publisher+Plugin>`_
`Javadoc Plugin <https://wiki.jenkins-ci.org/display/JENKINS/Javadoc+Plugin>`_
`Job Configuration History Plugin <https://wiki.jenkins-ci.org/display/JENKINS/JobConfigHistory+Plugin>`_
`JUnit Plugin <https://wiki.jenkins-ci.org/display/JENKINS/JUnit+Plugin>`_
`LDAP Plugin <https://wiki.jenkins-ci.org/display/JENKINS/LDAP+Plugin>`_
`Locale plugin <https://wiki.jenkins-ci.org/display/JENKINS/Locale+Plugin>`_
`Mailer Plugin <https://wiki.jenkins-ci.org/display/JENKINS/Mailer>`_
`MapDB API Plugin <https://wiki.jenkins-ci.org/display/JENKINS/MapDB+API+Plugin>`_
`Matrix Authorization Strategy Plugin <https://wiki.jenkins-ci.org/display/JENKINS/Matrix+Authorization+Strategy+Plugin>`_
`Matrix Project Plugin <https://wiki.jenkins-ci.org/display/JENKINS/Matrix+Project+Plugin>`_
`Maven Integration plugin <https://wiki.jenkins-ci.org/display/JENKINS/Maven+Project+Plugin>`_
`Multijob plugin <https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin>`_
`Multiple SCMs plugin <https://wiki.jenkins-ci.org/display/JENKINS/Multiple+SCMs+Plugin>`_
`OWASP Markup Formatter Plugin <https://wiki.jenkins-ci.org/display/JENKINS/OWASP+Markup+Formatter+Plugin>`_
`PAM Authentication plugin <https://wiki.jenkins-ci.org/display/JENKINS/PAM+Authentication+Plugin>`_
`Parameterized Trigger plugin <https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Trigger+Plugin>`_
`Publish Over SSH <https://wiki.jenkins-ci.org/display/JENKINS/Publish+Over+SSH+Plugin>`_
`PWauth Security Realm <http://wiki.hudson-ci.org/display/HUDSON/pwauth>`_
`Run Condition Plugin <https://wiki.jenkins-ci.org/display/JENKINS/Run+Condition+Plugin>`_
`SCM API Plugin <https://wiki.jenkins-ci.org/display/JENKINS/SCM+API+Plugin>`_
`Script Security Plugin <https://wiki.jenkins-ci.org/display/JENKINS/Script+Security+Plugin>`_
`Self-Organizing Swarm Plug-in Modules <https://wiki.jenkins-ci.org/display/JENKINS/Swarm+Plugin>`_
`Simple Theme Plugin <http://wiki.jenkins-ci.org/display/JENKINS/Simple+Theme+Plugin>`_
`SSH Agent Plugin <https://wiki.jenkins-ci.org/display/JENKINS/SSH+Agent+Plugin>`_
`SSH Credentials Plugin <https://wiki.jenkins-ci.org/display/JENKINS/SSH+Credentials+Plugin>`_
`SSH Slaves plugin <http://wiki.jenkins-ci.org/display/JENKINS/SSH+Slaves+plugin>`_
`Subversion Plug-in <http://wiki.jenkins-ci.org/display/JENKINS/Subversion+Plugin>`_
`Throttle Concurrent Builds Plug-in <http://wiki.jenkins-ci.org/display/JENKINS/Throttle+Concurrent+Builds+Plugin>`_
Timestamper
`Token Macro Plugin <http://wiki.jenkins-ci.org/display/JENKINS/Token+Macro+Plugin>`_
`Translation Assistance plugin <http://wiki.jenkins-ci.org/display/JENKINS/Translation+Assistance+Plugin>`_
`Windows Slaves Plugin <http://wiki.jenkins-ci.org/display/JENKINS/Windows+Slaves+Plugin>`_
`Workflow: Step API <https://wiki.jenkins-ci.org/display/JENKINS/Workflow+Plugin>`_
Jenkins jobs installation
-------------------------
Jenkins Job Builder takes simple descriptions of `Jenkins <http://jenkins-ci.org/>`_
jobs in `YAML <http://www.yaml.org/>`_ or `JSON <http://json.org/>`_
format and uses them to configure Jenkins.
To install JJB, run the following commands::
git clone https://git.openstack.org/openstack-infra/jenkins-job-builder
cd jenkins-job-builder && sudo python setup.py install
JJB requires config file to running::
[jenkins]
user=jenkins
password=1234567890abcdef1234567890abcdef
url=https://jenkins.example.com
Running::
jenkins-jobs --conf yourconf.ini update path_to_repo/jjb
We have some examples in our repo, it depersonalized copy of our real
jobs. Dont install it without reworking. You should replace necessary paths and variables.
Plugin test templates
---------------------
Most of necessary functions can be found in `fuel-qa <https://github.com/openstack/fuel-qa>`_
framework.
All functional tests should be stored in plugins git repository.
There should be special folder named plugin_test.
Fuel-qa framework should be submodule in plugin_test folder. You can add submodule by this command:
git submodule add https://github.com/openstack/fuel-qa
<pic>
There is simple test in repository.
It deploys an openstack cluster, installs the test plugin and enables it.
There are two folders: helpers and tests.
Helpers consists two files with important functions:
* prepare_test_plugin - install the plugin to master node
* activate_plugin - activate the plugin
* assign_net_provider - allow to choose network type for cluster
* deploy_cluster - deploy a cluster
Next folder is tests and it includes tests.
There is only one important file named test_smoke_bvt.py.
It describes a class of test plugin and 2 tests.
First of them just install the plugin and enable it, another try to deploy a cluster.
Hardware test examples
----------------------
Main problem of hardware configuration is authorization.
SSH does not allow enter password in script non interactively. But we can use expect utility for resolve this problem.
You should install the util on jenkins-slave first::
apt-get install expect
Example of script that use expect for auth on cisco switch and show its configuration::
spawn ssh "root@$switch_ip"
set timeout 500
expect "yes/no" {
send "yes\r"
expect "*?assword" { send "$switch_pass\r" }
} "*?assword" { send "$switch_pass\r" }
expect "# " { send "show run" }
expect "# " { send "exit\r" }
Fuel iso updating
-----------------
There is a script ``fuel-plugin-ci/iso-updater/get_iso.sh``.
It should be added to cron and run every 2-3 hours.
This script checks for new community build of Fuel and if there is new version available, it downloads it.
You can run the script on jenkins-slave node or any web server if you have many slave nodes.
Steps:
#. Check for the latest community iso. Using w3m utility script checks ``https://www.fuel-infra.org/release/status`` url and choose right tab:
* the first tab is 8.0 now, we need 2nd tab with Fuel 7.0.
* Then we parse it and get Fuel release string.
.. note:: if new Fuel version is available, you should fix the
script and change tab number. Also output may change between
linux distros and last cut field may change.
#. Download torrent file from `http://seed.fuel-infra.org/fuelweb-iso/` via aria2 console torrent client.
#. Check for errors and delete folder if there is an error.
#. Sync downloaded iso with jenkins slave. You should have necessary users with rsa keys set.