Add fresh-box-setup script for zuul-reproducer first run
On first run or a fresh setup there are some required steps like creating a user, a clouds.yaml and ssh keys for gerrit. This adds a script into the create reproducer role which can be used for this initial setup Change-Id: I6d70a87006800447a12b4609ccc81400741290db
This commit is contained in:
Marios Andreou
parent
1b88449454
commit
a04f3c986b
|
|
@ -1,5 +1,6 @@
|
|||
---
|
||||
reproducer_zuul_based_quickstart_script: reproducer-zuul-based-quickstart.sh.j2
|
||||
reproducer_fresh_box_setup_script: reproducer-fresh-box-setup.sh.j2
|
||||
reproducer_zuul_based_quickstart_readme_file: "{{ artcl_collect_dir }}/README-reproducer.html"
|
||||
reproducer_zuul_based_launcher_playbook: launcher-playbook.yaml.j2
|
||||
launcher_env_setup_playbook: launcher-env-setup-playbook.yaml.j2
|
||||
|
|
|
|||
|
|
@ -76,6 +76,12 @@
|
|||
dest: "{{ artcl_collect_dir }}/{{ zuul_reproducer_dir }}/reproducer-zuul-based-quickstart.sh"
|
||||
mode: 0755
|
||||
|
||||
- name: Create the reproducer-fresh-box-setup script
|
||||
template:
|
||||
src: "{{ reproducer_fresh_box_setup_script }}"
|
||||
dest: "{{ artcl_collect_dir }}/{{ zuul_reproducer_dir }}/reproducer-fresh-box-setup.sh"
|
||||
mode: 0755
|
||||
|
||||
- name: Copy the tripleo-quickstart install-deps script
|
||||
copy:
|
||||
src: "/home/{{ undercloud_user }}/src/opendev.org/openstack/tripleo-quickstart/{{ item }}"
|
||||
|
|
|
|||
|
|
@ -54,125 +54,152 @@ issue.</p>
|
|||
<button class="collapsible">Setting up to run the Zuul-based reproducer</button>
|
||||
<div class="content">
|
||||
<p>
|
||||
Prior to running the Zuul-based reproducer, the following setup steps are needed -
|
||||
( only need to be done once per local system):
|
||||
To setup and run the reproducer you must first decide <b>where</b> it will run
|
||||
- on a <b>pet</b> (your laptop) or on <b>cattle</b> (rdo-cloud vm or other
|
||||
temporary environment)? For each case see notes below for required setup.
|
||||
|
||||
<p>
|
||||
<b>If this is the first time you are running the reproducer</b> you must
|
||||
ensure you have an externally routable network in your RDO-cloud tenant called
|
||||
'private'. That is you will need a router attached to the external network that
|
||||
is also attached to your network called 'private' (which you may need to create).
|
||||
Assuming you have a clouds.yaml that let's you talk to RDO cloud you can use
|
||||
the following commands to create the required setup:
|
||||
<pre>
|
||||
<code>
|
||||
openstack --os-cloud rdo-cloud network create private
|
||||
openstack --os-cloud rdo-cloud subnet create private --network private --gateway 192.168.0.1 --subnet-range 192.168.0.0/24
|
||||
openstack --os-cloud rdo-cloud router create privaterouter
|
||||
openstack --os-cloud rdo-cloud router set --external-gateway 38.145.32.0/22 privaterouter
|
||||
openstack --os-cloud rdo-cloud router add subnet privaterouter private
|
||||
</code>
|
||||
</pre>
|
||||
</p>
|
||||
|
||||
Alternatively log in to <a href="https://phx2.cloud.rdoproject.org/dashboard/auth/login/?next=/dashboard/project/networks/">RDO cloud horizon</a> and create the network and router
|
||||
that way.
|
||||
|
||||
<hr></hr>
|
||||
<h1>Cattle</h1>
|
||||
If you're using a temporary environment like RDO-cloud vm to run the
|
||||
reproducer you can use the script inside the reproducer-quickstart/ directory
|
||||
from the logs of the job you are reproducing. The script is acalled
|
||||
<code>reproducer-fresh-box-setup.sh</code>:
|
||||
|
||||
<pre><code><b>
|
||||
curl -o reproducer-fresh-box-setup.sh http://logs.openstack.org/79/657779/2/check/tripleo-ci-centos-7-containers-multinode/67d8eb6/logs/reproducer-quickstart/reproducer-fresh-box-setup.sh
|
||||
chmod 775 reproducer-fresh-box-setup.sh
|
||||
./reproducer-fresh-box-setup.sh -u marios -p password -c reprozuul
|
||||
</b></code></pre>
|
||||
|
||||
<p><b>note:</b> If you're using an RDO-cloud vm be sure to use at least m1.large2
|
||||
flavour as we've hit hard disk limits with m1.large</p>
|
||||
<p> There are two required parameters <code>-u USER</code/> and <code>-p PASS</code>
|
||||
which correspond to the RDO cloud username and password. These are written to
|
||||
the generated <code>$HOME/.config/openstack/clouds.yaml</code> file expected by the
|
||||
reproducer.
|
||||
</p>
|
||||
<p>This script creates the default user 'reprozuul' with passwordless sudo
|
||||
privileges, creates $HOME/.config/openstack/clouds.yaml with the -u USER and
|
||||
-p PASS parameters and creates the required ssh keypair. It must run as root,
|
||||
or as a user with passwordless sudo.
|
||||
</p>
|
||||
<p> The script finally switches to the newly created user and outputs the newly
|
||||
created public key. You must upload this key to gerrit (<b>both</b>
|
||||
<a href="https://review.opendev.org/#/settings/ssh-keys">opendev.org</a> as well
|
||||
as <a href="https://review.rdoproject.org/r/#/settings/ssh-keys">rdoproject.org</a>).
|
||||
</p>
|
||||
<hr></hr>
|
||||
<h1>Pet</h1>
|
||||
|
||||
If you are running the reproducer on a non-transient machine, that is your
|
||||
laptop or any other box which isn't temporary (a pet), then you need:
|
||||
|
||||
<ul>
|
||||
<li>Create a clouds.yaml - for example:
|
||||
<li> Ensure user running reproducer has passwordless sudo </li>
|
||||
<li> That user has $HOME/.config/openstack/clouds.yaml with "rdo-cloud" entry </li>
|
||||
<li> A ssh keypair generated with <b><code>-m PEM -t rsa</code></b> and uploaded
|
||||
to gerrit (<b>both </b><a href="https://review.opendev.org/#/settings/ssh-keys">opendev.org</a>
|
||||
as well as <a href="https://review.rdoproject.org/r/#/settings/ssh-keys">rdoproject.org</a>).
|
||||
</li>
|
||||
<li> A externally routable network in your RDO-cloud tenant named 'private'.
|
||||
See example commands above if you need to create this or log in to
|
||||
<a href="https://phx2.cloud.rdoproject.org/dashboard/auth/login/?next=/dashboard/project/networks/"> RDO cloud horizon</a> and verify your setup or create as needed.
|
||||
</ul>
|
||||
|
||||
You only need to do these things once on your pet and then rerun reproducer or
|
||||
update as needed.
|
||||
|
||||
<ul>
|
||||
<li>Example clouds.yaml:</li>
|
||||
<pre>
|
||||
<code>mkdir -p ~/.config/openstack
|
||||
vi ~/.config/openstack/clouds.yaml
|
||||
<code>CLOUDS_YAML_PATH=$HOME/.config/openstack/
|
||||
mkdir -p $CLOUDS_YAML_PATH
|
||||
/bin/bash -c "cat <<EOF>>$CLOUDS_YAML_PATH/clouds.yaml
|
||||
clouds:
|
||||
rdo-cloud:
|
||||
identity_api_version: 3
|
||||
region_name: regionOne
|
||||
auth:
|
||||
auth_url: https://phx2.cloud.rdoproject.org:13000/v3
|
||||
password: xxxxxxx
|
||||
project_name: user_name
|
||||
username: user_name
|
||||
password: $RDO_PASS
|
||||
project_name: $RDO_USER
|
||||
username: $RDO_USER
|
||||
user_domain_name: Default
|
||||
project_domain_name: Default
|
||||
regions:
|
||||
- name: regionOne
|
||||
values:
|
||||
networks:
|
||||
- name: 38.145.32.0/22
|
||||
routes_externally: true
|
||||
- name: private
|
||||
|
||||
- name: regionOne
|
||||
values:
|
||||
networks:
|
||||
- name: 38.145.32.0/22
|
||||
routes_externally: true
|
||||
- name: private
|
||||
EOF"
|
||||
</code>
|
||||
</pre>
|
||||
</ul>
|
||||
<ul>
|
||||
<li> Test your cloud.yaml by running:
|
||||
<pre>
|
||||
<code>
|
||||
openstack --os-cloud rdo-cloud image list
|
||||
</code>
|
||||
</pre>
|
||||
</ul>
|
||||
<ul>
|
||||
<li>Note that you will need a network named 'private' in the tenant connected to an
|
||||
router that is connected to the external network.</li>
|
||||
<li>Create an ssh key pair without a pass phrase:</li>
|
||||
<br/>
|
||||
<li> Create an ssh key pair without a pass phrase:</li>
|
||||
<ul>
|
||||
<li>The ssh key pair is configurable - see variables <code>ssh_path</code></li>
|
||||
<li><code>user_pri_key</code>. The default key is:<code>~/.ssh/id_rsa[.pub]</code></li>
|
||||
<li> <b>Note:</b> Reference FAQ below when creating ssh keys, some keys fail when used w/ gerrit
|
||||
<li>The ssh key pair is configurable - see variables <code>ssh_path</code>
|
||||
<code>user_pri_key</code>. The default key is:<code>~/.ssh/id_rsa[.pub]</code></li>
|
||||
<li> <b>Note:</b> Use the following to create keys: <b><code> ssh-keygen -m PEM -t rsa </code></b>
|
||||
See FAQ below for more information about ssh keys requirements.
|
||||
</ul>
|
||||
</p>
|
||||
</ul>
|
||||
</br>
|
||||
</div>
|
||||
|
||||
<button class="collapsible">Running the Zuul-based reproducer - using the Bash script</button>
|
||||
<div class="content">
|
||||
<p>
|
||||
<ul>
|
||||
<li> wget or curl the <b>reproducer-quickstart/reproducer-zuul-based-quickstart.tar</b> file. </li>
|
||||
<li> Note that the <code>launcher-env-setup-playbook.yaml</code> will set up
|
||||
clone repos and set up 'roles', 'playbooks', and 'library' directories
|
||||
from which the tasks will be executed.</li>
|
||||
<li> Once the files are extracted from the archive, you can run the Bash script
|
||||
with the various options. Run the script with <code>--help</code> option to
|
||||
output a list of available run options. </li>
|
||||
</ul>
|
||||
<b>Note:</b> the local $USERNAME idealy matches the openstack and rdo gerrit user id's
|
||||
If not pay extra attention to the -ug and -rg options.
|
||||
<ul>
|
||||
<code>
|
||||
<pre>
|
||||
Options:
|
||||
-w, --workspace dir
|
||||
directory where the repos and directories
|
||||
are created. Defaults to creating a directory in /tmp
|
||||
-l, --libvirt
|
||||
Runs a 2-node multinode job or singlenode job on a
|
||||
single virthost using libvirt to create the nodes.
|
||||
If a singlenode reproducer is run, two VMs will still be created.
|
||||
-c, --cloud-name
|
||||
Host cloud, specified in the clouds.yaml, to target
|
||||
Defaults to rdo-cloud
|
||||
-cp, --clouds-yaml-path
|
||||
Full path to the clouds.yaml file
|
||||
Defaults to /home/$USERNAME/.config/openstack/clouds.yaml
|
||||
-ok, --ovb-key-name
|
||||
Name of the key to use in the host tenant for OVB deployments
|
||||
Defaults to tripleo-ci-team
|
||||
-f, --force-post-failure
|
||||
Force job to fail so that the nodes will be held.
|
||||
Temporary solution
|
||||
-ug, --upstream-gerrit-user
|
||||
Set the upstream gerrit user required to clone repos.
|
||||
Defaults to the current $USERNAME
|
||||
-rg, --rdo-gerrit-user
|
||||
Set the upstream RDO user required to clone RDO-related repos.
|
||||
Defaults to the current $USERNAME
|
||||
-k, --ssh-key
|
||||
private ssh key used to set up an access nodes.
|
||||
Defaults to id_rsa.
|
||||
-kp, --ssh-key-public
|
||||
public ssh key used to set up an access nodes.
|
||||
Defaults to id_rsa.pub
|
||||
-skp, --ssh-key-path
|
||||
Path to directory where user ssh keys are stored.
|
||||
Defaults to /home/$USERNAME/.ssh
|
||||
-ugk, --upstream-gerrit-key
|
||||
Set the upstream gerrit private key.
|
||||
Defaults to the id_rsa.
|
||||
-rgk, --rdo-gerrit-key
|
||||
Set the upstream RDO user key.
|
||||
Defaults to the id_rsa.
|
||||
-e, --extra-params
|
||||
File or/and parameters used to override default
|
||||
parameters for playbooks. Multiple files
|
||||
can be passed [-e @file1.yml -e @file2.yml ...]
|
||||
and arguments [-e var=value -e var2=value2 ...]
|
||||
-h, --help print this help and exit
|
||||
wget or curl the <b>reproducer-quickstart/reproducer-zuul-based-quickstart.tar</b> file.
|
||||
<pre><code><b>
|
||||
wget http://logs.openstack.org/79/657779/2/check/tripleo-ci-centos-7-standalone-upgrade/92fd476/logs/reproducer-quickstart/reproducer-zuul-based-quickstart.tar
|
||||
tar -xvf reproducer-zuul-based-quickstart.tar
|
||||
</b></code></pre>
|
||||
One of the extracted files is the script called <code>./reproducer-zuul-based-quickstart.sh</code>
|
||||
and you can run it with <code>--help</code> to see available options.
|
||||
</br>
|
||||
</br>
|
||||
<b>Note:</b> If the local $USERNAME does not match the gerrit user IDs for
|
||||
<a href="https://review.opendev.org/#/settings/ssh-keys">opendev.org</a> or
|
||||
<a href="https://review.rdoproject.org/r/#/settings/ssh-keys">rdoproject.org</a>)
|
||||
then you will need to supply the -ug (upstream gerrit) and -rg (rdo gerrit)
|
||||
parameters:
|
||||
|
||||
</code>
|
||||
</pre>
|
||||
<ul>
|
||||
</ul>
|
||||
<pre><code><b>
|
||||
mkdir WORKSPACE
|
||||
./reproducer-zuul-based-quickstart.sh --workspace WORKSPACE -ug slim_shady -rg slim_shadier
|
||||
</b></code></pre>
|
||||
</p>
|
||||
Monitor the output and follow any instructions - for example the script will
|
||||
exit and require re-run once the user is added to the docker group.
|
||||
</br>
|
||||
</br>
|
||||
</div>
|
||||
|
||||
<button class="collapsible">Running the Zuul-based reproducer - using the playbooks directly</button>
|
||||
|
|
@ -183,9 +210,12 @@ The playbooks can be run directly using <code>ansible-playbook</code>.
|
|||
The Bash script contains some package installation and Docker user/group setup
|
||||
that you will need to cover prior to running the playbooks directly.
|
||||
Please see the Bash scripts for details, especially if you are running the
|
||||
reproducer for the first time on teh local environment.
|
||||
reproducer for the first time on the local environment.
|
||||
The playbooks can be run as follows:
|
||||
<ul>
|
||||
<li> Note that the <code>launcher-env-setup-playbook.yaml</code> will clone
|
||||
repos and set up 'roles', 'playbooks', and 'library' directories
|
||||
from which the tasks will be executed. </li>
|
||||
<li><code>ansible-playbook ./launcher-env-setup-playbook.yaml</code></li>
|
||||
<li><code>ansible-playbook ./launcher-playbook.yaml $OPTIONS</code></li>
|
||||
</ul>
|
||||
|
|
|
|||
|
|
@ -0,0 +1,127 @@
|
|||
#!/bin/bash
|
||||
|
||||
REPRO_USER=${REPRO_USER:=reprozuul}
|
||||
REPRO_DEPS="tar wget screen"
|
||||
RDO_USER=""
|
||||
RDO_PASS=""
|
||||
CLOUDS_YAML_PATH=".config/openstack/"
|
||||
usage () {
|
||||
echo "Usage: $0 [options]"
|
||||
echo ""
|
||||
echo "Options:"
|
||||
echo " -u, --rdo-user"
|
||||
echo " Username for RDO cloud to be written to"
|
||||
echo " $CLOUDS_YAML_PATH/clouds.yaml. REQUIRED."
|
||||
echo " -p, --rdo-pass"
|
||||
echo " Password for RDO cloud to be written to"
|
||||
echo " $CLOUDS_YAML_PATH/clouds.yaml. REQUIRED."
|
||||
echo " -c, --create-user"
|
||||
echo " Local user to create for running reproducer."
|
||||
echo " Defaults to $REPRO_USER"
|
||||
echo " -h, --help print this help and exit"
|
||||
}
|
||||
|
||||
set -e
|
||||
|
||||
# Input argument assignments
|
||||
while [ "x$1" != "x" ]; do
|
||||
|
||||
case "$1" in
|
||||
--rdo-user|-u)
|
||||
RDO_USER=$2
|
||||
shift
|
||||
;;
|
||||
|
||||
--rdo-pass|-p)
|
||||
RDO_PASS=$2
|
||||
shift
|
||||
;;
|
||||
|
||||
--create-user|-c)
|
||||
REPRO_USER=$2
|
||||
shift
|
||||
;;
|
||||
|
||||
--help|-h)
|
||||
usage
|
||||
exit
|
||||
;;
|
||||
|
||||
--) shift
|
||||
break
|
||||
;;
|
||||
|
||||
-*) echo "ERROR: unknown option: $1" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
;;
|
||||
|
||||
*) break
|
||||
;;
|
||||
esac
|
||||
|
||||
shift
|
||||
done
|
||||
|
||||
# check required params - rdo user/pass
|
||||
for param in "$RDO_USER" "$RDO_PASS" ; do
|
||||
if [[ $param = "" ]]; then
|
||||
echo "error --rdo-user and --rdo-pass are required parameters"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
# create user
|
||||
sudo adduser $REPRO_USER
|
||||
echo "$REPRO_USER ALL=(root) NOPASSWD:ALL" | sudo tee -a /etc/sudoers.d/$REPRO_USER
|
||||
|
||||
# install packages
|
||||
sudo yum -y install $REPRO_DEPS
|
||||
|
||||
# write clouds.yaml
|
||||
CLOUDS_YAML_PATH="/home/$REPRO_USER/.config/openstack/"
|
||||
sudo mkdir -p $CLOUDS_YAML_PATH
|
||||
sudo /bin/bash -c "cat <<EOF>>$CLOUDS_YAML_PATH/clouds.yaml
|
||||
clouds:
|
||||
rdo-cloud:
|
||||
identity_api_version: 3
|
||||
region_name: regionOne
|
||||
auth:
|
||||
auth_url: https://phx2.cloud.rdoproject.org:13000/v3
|
||||
password: $RDO_PASS
|
||||
project_name: $RDO_USER
|
||||
username: $RDO_USER
|
||||
user_domain_name: Default
|
||||
project_domain_name: Default
|
||||
regions:
|
||||
- name: regionOne
|
||||
values:
|
||||
networks:
|
||||
- name: 38.145.32.0/22
|
||||
routes_externally: true
|
||||
- name: private
|
||||
EOF"
|
||||
|
||||
sudo chown -R $REPRO_USER:$REPRO_USER /home/$REPRO_USER/.config
|
||||
|
||||
# create keys
|
||||
mkdir keystemp
|
||||
pushd keystemp
|
||||
ssh-keygen -m PEM -t rsa -N "" -f id_rsa
|
||||
ssh_path=/home/$REPRO_USER/.ssh
|
||||
sudo mkdir $ssh_path
|
||||
sudo mv id_rsa* $ssh_path
|
||||
sudo chmod 700 $ssh_path
|
||||
sudo chmod 600 $ssh_path/id_rsa
|
||||
sudo chmod 644 $ssh_path/id_rsa.pub
|
||||
sudo chown -R $REPRO_USER:$REPRO_USER $ssh_path
|
||||
popd
|
||||
rm -rf keystemp
|
||||
|
||||
echo "Fresh box setup complete - now go upload the public key in"
|
||||
echo "$CLOUDS_YAML_PATH to Gerrit - both upstream and rdo. Then you are ready "
|
||||
echo "to run the reproducer. Here is the public key you must upload:"
|
||||
sudo cat /home/$REPRO_USER/.ssh/id_rsa.pub
|
||||
echo "Switching to $REPRO_USER"
|
||||
sudo su - $REPRO_USER
|
||||
|
||||
Loading…
Reference in New Issue