Merge "Reorganize admin manual"
This commit is contained in:
commit
be64ed2777
|
@ -20,9 +20,9 @@ Zuul and Nodepool.
|
|||
|
||||
To get started, ssh to your machine as the ``centos`` user.
|
||||
|
||||
.. code-block:: console
|
||||
.. code-block:: shell
|
||||
|
||||
$ ssh centos@<ip_address>
|
||||
ssh centos@<ip_address>
|
||||
|
||||
Environment Setup
|
||||
-----------------
|
||||
|
@ -41,15 +41,15 @@ dependencies are handled by the bindep program, but a few additional
|
|||
dependencies are needed to install bindep, and for other commands
|
||||
which we will use in these instructions.
|
||||
|
||||
.. code-block:: console
|
||||
.. code-block:: shell
|
||||
|
||||
$ sudo yum update -y
|
||||
$ sudo systemctl reboot
|
||||
$ sudo yum install -y https://centos7.iuscommunity.org/ius-release.rpm
|
||||
$ sudo yum install -y git python35u python35u-pip python35u-devel java-1.8.0-openjdk
|
||||
$ sudo alternatives --install /usr/bin/python3 python3 /usr/bin/python3.5 10
|
||||
$ sudo alternatives --install /usr/bin/pip3 pip3 /usr/bin/pip3.5 10
|
||||
$ sudo pip3 install python-openstackclient bindep
|
||||
sudo yum update -y
|
||||
sudo systemctl reboot
|
||||
sudo yum install -y https://centos7.iuscommunity.org/ius-release.rpm
|
||||
sudo yum install -y git python35u python35u-pip python35u-devel java-1.8.0-openjdk
|
||||
sudo alternatives --install /usr/bin/python3 python3 /usr/bin/python3.5 10
|
||||
sudo alternatives --install /usr/bin/pip3 pip3 /usr/bin/pip3.5 10
|
||||
sudo pip3 install python-openstackclient bindep
|
||||
|
||||
Install Zookeeper
|
||||
-----------------
|
||||
|
@ -60,7 +60,7 @@ Nodepool for nodes.
|
|||
|
||||
.. code-block:: console
|
||||
|
||||
$ sudo bash -c "cat << EOF > /etc/yum.repos.d/bigtop.repo
|
||||
sudo bash -c "cat << EOF > /etc/yum.repos.d/bigtop.repo
|
||||
[bigtop]
|
||||
name=Bigtop
|
||||
enabled=1
|
||||
|
@ -69,7 +69,7 @@ Nodepool for nodes.
|
|||
baseurl=http://repos.bigtop.apache.org/releases/1.2.1/centos/7/x86_64
|
||||
gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
|
||||
EOF"
|
||||
$ sudo yum install -y zookeeper zookeeper-server
|
||||
$ sudo systemctl start zookeeper-server.service
|
||||
$ sudo systemctl status zookeeper-server.service
|
||||
$ sudo systemctl enable zookeeper-server.service
|
||||
sudo yum install -y zookeeper zookeeper-server
|
||||
sudo systemctl start zookeeper-server.service
|
||||
sudo systemctl status zookeeper-server.service
|
||||
sudo systemctl enable zookeeper-server.service
|
||||
|
|
|
@ -22,6 +22,7 @@ which is described below.
|
|||
Merger -- Gearman
|
||||
Executor -- Gearman
|
||||
Web -- Gearman
|
||||
Finger -- Gearman
|
||||
|
||||
Gearman -- Scheduler;
|
||||
Scheduler -- Gerrit;
|
||||
|
|
|
@ -23,17 +23,22 @@ ACL enabled.
|
|||
|
||||
.. TODO: Instructions to create the ssh key used here
|
||||
|
||||
As the admin user, create the ``zuul`` user, and import an SSH key for ``zuul``::
|
||||
As the admin user, create the ``zuul`` user, and import an SSH key for
|
||||
``zuul``:
|
||||
|
||||
$ cat $PUBKEY | ssh -p 29418 $USER@localhost gerrit create-account \
|
||||
.. code-block:: shell
|
||||
|
||||
cat $PUBKEY | ssh -p 29418 $USER@localhost gerrit create-account \
|
||||
--group "'Registered Users'" --ssh-key - zuul
|
||||
|
||||
``$PUBKEY`` is the location of the SSH public key for the ``zuul``
|
||||
user. ``$USER`` is the username for the admin user.
|
||||
|
||||
The ``zuul`` user should now be able to stream events::
|
||||
The ``zuul`` user should now be able to stream events:
|
||||
|
||||
$ ssh -p 29418 zuul@localhost gerrit stream-events
|
||||
.. code-block:: shell
|
||||
|
||||
ssh -p 29418 zuul@localhost gerrit stream-events
|
||||
|
||||
Configure Gerrit
|
||||
----------------
|
||||
|
@ -64,27 +69,35 @@ The admin user can create new projects in Gerrit, which users can then clone
|
|||
and use to submit code changes. Zuul will monitor the Gerrit event stream for
|
||||
these submissions.
|
||||
|
||||
To create a new project named 'demo-project'::
|
||||
To create a new project named 'demo-project':
|
||||
|
||||
$ ssh -p 29418 $USER@localhost gerrit create-project demo-project --empty-commit
|
||||
.. code-block:: shell
|
||||
|
||||
ssh -p 29418 $USER@localhost gerrit create-project demo-project --empty-commit
|
||||
|
||||
Modify the Project
|
||||
------------------
|
||||
|
||||
* Clone the project::
|
||||
* Clone the project:
|
||||
|
||||
$ git clone ssh://$USER@localhost:29418/demo-project.git
|
||||
.. code-block:: shell
|
||||
|
||||
* Install the change ID hook that Gerrit requires::
|
||||
git clone ssh://$USER@localhost:29418/demo-project.git
|
||||
|
||||
$ cd demo-project
|
||||
$ scp -p -P 29418 $USER@localhost:hooks/commit-msg .git/hooks/
|
||||
* Install the change ID hook that Gerrit requires:
|
||||
|
||||
* Now you are ready to modify the project and push the changes to Gerrit::
|
||||
.. code-block:: shell
|
||||
|
||||
$ echo "test" > README.txt
|
||||
$ git add .
|
||||
$ git commit -m "First commit"
|
||||
$ git push origin HEAD:refs/for/master
|
||||
cd demo-project
|
||||
scp -p -P 29418 $USER@localhost:hooks/commit-msg .git/hooks/
|
||||
|
||||
* Now you are ready to modify the project and push the changes to Gerrit:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
echo "test" > README.txt
|
||||
git add .
|
||||
git commit -m "First commit"
|
||||
git push origin HEAD:refs/for/master
|
||||
|
||||
You should now be able to see your change in Gerrit.
|
||||
|
|
|
@ -60,22 +60,29 @@ Go back to the `General` settings page for the app,
|
|||
https://github.com/organizations/my-org/settings/apps/my-org-zuul
|
||||
and look for the app `ID` number, under the `About` section.
|
||||
|
||||
Edit ``/etc/zuul/zuul.conf`` to add the following::
|
||||
Edit ``/etc/zuul/zuul.conf`` to add the following:
|
||||
|
||||
[connection github]
|
||||
driver=github
|
||||
app_id=<APP ID NUMBER>
|
||||
app_key=/etc/zuul/github.pem
|
||||
webhook_token=<WEBHOOK SECRET>
|
||||
.. code-block:: shell
|
||||
|
||||
sudo bash -c "cat >> /etc/zuul/zuul.conf <<EOF
|
||||
|
||||
[connection github]
|
||||
driver=github
|
||||
app_id=<APP ID NUMBER>
|
||||
app_key=/etc/zuul/github.pem
|
||||
webhook_token=<WEBHOOK SECRET>
|
||||
EOF"
|
||||
|
||||
Upload the private key which was generated earlier, and save it in
|
||||
``/etc/zuul/github.pem``.
|
||||
|
||||
Restart all of Zuul::
|
||||
Restart all of Zuul:
|
||||
|
||||
sudo systemctl restart zuul-executor.service
|
||||
sudo systemctl restart zuul-web.service
|
||||
sudo systemctl restart zuul-scheduler.service
|
||||
.. code-block:: shell
|
||||
|
||||
sudo systemctl restart zuul-executor.service
|
||||
sudo systemctl restart zuul-web.service
|
||||
sudo systemctl restart zuul-scheduler.service
|
||||
|
||||
Go to the `Advanced` tab for the app in GitHub,
|
||||
https://github.com/organizations/my-org/settings/apps/my-org-zuul/advanced,
|
||||
|
@ -92,7 +99,9 @@ and ``zuul-test``, respectively.
|
|||
Visit the public app page on GitHub,
|
||||
https://github.com/apps/my-org-zuul, and install the app into your org.
|
||||
|
||||
Edit ``/etc/zuul/main.yaml`` so that it looks like this::
|
||||
Edit ``/etc/zuul/main.yaml`` so that it looks like this:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
- tenant:
|
||||
name: quickstart
|
||||
|
@ -114,13 +123,17 @@ you can immediately use in your Zuul installation.
|
|||
|
||||
The second section is your GitHub configuration.
|
||||
|
||||
After updating the file, restart the Zuul scheduler::
|
||||
After updating the file, restart the Zuul scheduler:
|
||||
|
||||
sudo systemctl restart zuul-scheduler.service
|
||||
.. code-block:: shell
|
||||
|
||||
sudo systemctl restart zuul-scheduler.service
|
||||
|
||||
Add an initial pipeline configuration to the `zuul-test-config`
|
||||
repository. Inside that project, create a ``zuul.yaml`` file with the
|
||||
following contents::
|
||||
following contents:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
- pipeline:
|
||||
name: check
|
||||
|
@ -152,7 +165,9 @@ following contents::
|
|||
Merge that commit into the repository.
|
||||
|
||||
In the `zuul-test` project, create a `.zuul.yaml` file with the
|
||||
following contents::
|
||||
following contents:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
- project:
|
||||
check:
|
||||
|
|
|
@ -11,12 +11,12 @@ provides to in-project configuration.
|
|||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
tutorial
|
||||
quick-start
|
||||
zuul-from-scratch
|
||||
installation
|
||||
zuul-from-scratch
|
||||
components
|
||||
connections
|
||||
tenants
|
||||
monitoring
|
||||
client
|
||||
troubleshooting
|
||||
|
|
|
@ -1,5 +1,5 @@
|
|||
Installation
|
||||
============
|
||||
Installation Reference
|
||||
======================
|
||||
|
||||
Install Zuul
|
||||
------------
|
||||
|
@ -17,9 +17,42 @@ interaction with other python packages installed on a system, you may
|
|||
wish to install Zuul within a Python virtualenv.
|
||||
|
||||
Zuul has several system-level dependencies as well. You can find a
|
||||
list of operating system packages in `bindep.txt` in Zuul's source
|
||||
list of operating system packages in ``bindep.txt`` in Zuul's source
|
||||
directory.
|
||||
|
||||
Zuul Components
|
||||
---------------
|
||||
|
||||
Zuul provides the following components:
|
||||
|
||||
- **zuul-scheduler**: The main Zuul process. Handles receiving
|
||||
events, executing jobs, collecting results and posting reports.
|
||||
Coordinates the work of the other components. It also provides
|
||||
a gearman daemon which the other components use for
|
||||
coordination.
|
||||
|
||||
- **zuul-merger**: Scale-out component that performs git merge
|
||||
operations. Zuul performs a large number of git operations in
|
||||
the course of its work. Adding merger processes can help speed
|
||||
Zuul's processing. This component is optional (zero or more of
|
||||
these can be run).
|
||||
|
||||
- **zuul-executor**: Scale-out component for executing jobs. At
|
||||
least one of these is required. Depending on system
|
||||
configuration, you can expect a single executor to handle up to
|
||||
about 100 simultaneous jobs. Can handle the functions of a
|
||||
merger if dedicated mergers are not provided. One or more of
|
||||
these must be run.
|
||||
|
||||
- **zuul-web**: A web server that receives "webhook" events from
|
||||
external providers, supplies a web dashboard, and provides
|
||||
websocket access to live streaming of logs.
|
||||
|
||||
- **zuul-fingergw**: A gateway which provides finger protocol
|
||||
access to live streaming of logs.
|
||||
|
||||
For more detailed information about these, see :ref:`components`.
|
||||
|
||||
External Dependencies
|
||||
---------------------
|
||||
|
||||
|
@ -68,10 +101,73 @@ the correct version will be installed automatically with Zuul.
|
|||
Because of the close integration of Zuul and Ansible, attempting to
|
||||
use other versions of Ansible with Zuul is not recommended.
|
||||
|
||||
Zuul Setup
|
||||
----------
|
||||
|
||||
At minimum you need to provide ``zuul.conf`` and ``main.yaml`` placed
|
||||
in ``/etc/zuul/``. The following example uses the builtin gearman
|
||||
service in Zuul, and a connection to Gerrit.
|
||||
|
||||
**zuul.conf**::
|
||||
|
||||
[scheduler]
|
||||
tenant_config=/etc/zuul/main.yaml
|
||||
|
||||
[gearman_server]
|
||||
start=true
|
||||
|
||||
[gearman]
|
||||
server=127.0.0.1
|
||||
|
||||
[connection my_gerrit]
|
||||
driver=gerrit
|
||||
server=git.example.com
|
||||
port=29418
|
||||
baseurl=https://git.example.com/gerrit/
|
||||
user=zuul
|
||||
sshkey=/home/zuul/.ssh/id_rsa
|
||||
|
||||
See :ref:`components` and :ref:`connections` for more details.
|
||||
|
||||
The following tells Zuul to read its configuration from and operate on
|
||||
the *example-project* project:
|
||||
|
||||
**main.yaml**::
|
||||
|
||||
- tenant:
|
||||
name: example-tenant
|
||||
source:
|
||||
my_gerrit:
|
||||
untrusted-projects:
|
||||
- example-project
|
||||
|
||||
Starting Zuul
|
||||
-------------
|
||||
|
||||
You can run any zuul process with the **-d** option to make it not
|
||||
daemonize. It's a good idea at first to confirm there's no issues with
|
||||
your configuration.
|
||||
|
||||
To start, simply run::
|
||||
|
||||
zuul-scheduler
|
||||
|
||||
Once run you should have two zuul-scheduler processes (if using the
|
||||
built-in gearman server, or one process otherwise).
|
||||
|
||||
Before Zuul can run any jobs, it needs to load its configuration, most
|
||||
of which is in the git repositories that Zuul operates on. Start an
|
||||
executor to allow zuul to do that::
|
||||
|
||||
zuul-executor
|
||||
|
||||
Zuul should now be able to read its configuration from the configured
|
||||
repo and process any jobs defined therein.
|
||||
|
||||
.. _web-deployment-options:
|
||||
|
||||
Web Deployment Options
|
||||
======================
|
||||
----------------------
|
||||
|
||||
The ``zuul-web`` service provides an web dashboard, a REST API and a websocket
|
||||
log streaming service as a single holistic web application. For production use
|
||||
|
@ -104,7 +200,7 @@ Reverse Proxy. Where rewrite rule examples are given, they will be given
|
|||
with Apache syntax, but any other Reverse Proxy should work just fine.
|
||||
|
||||
Basic Reverse Proxy
|
||||
-------------------
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Using Apache as the Reverse Proxy requires the ``mod_proxy``,
|
||||
``mod_proxy_http`` and ``mod_proxy_wstunnel`` modules to be installed and
|
||||
|
@ -119,7 +215,7 @@ simplest reverse-proxy case is::
|
|||
|
||||
|
||||
Static Offload
|
||||
--------------
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
To have the Reverse Proxy serve the static html/javascript assets instead of
|
||||
proxying them to the REST layer, register the location where you unpacked
|
||||
|
@ -146,7 +242,7 @@ the web application as the document root and add rewrite rules::
|
|||
|
||||
|
||||
Sub directory serving
|
||||
---------------------
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The web application needs to be rebuild to update the internal location of
|
||||
the static files. Set the homepage setting in the package.json to an
|
||||
|
@ -188,7 +284,7 @@ add the following rewrite rules::
|
|||
|
||||
|
||||
White Labeled Tenant
|
||||
--------------------
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Running a white-labeled tenant is similar to the offload case, but adds a
|
||||
rule to ensure connection webhooks don't try to get put into the tenant scope.
|
||||
|
@ -223,7 +319,7 @@ Assuming the zuul tenant name is "example", the rewrite rules are::
|
|||
|
||||
|
||||
Static External
|
||||
---------------
|
||||
~~~~~~~~~~~~~~~
|
||||
|
||||
.. note::
|
||||
|
||||
|
|
|
@ -10,15 +10,15 @@ First we'll create the nodepool user and set up some directories it
|
|||
needs. We also need to create an SSH key for Zuul to use when it logs
|
||||
into the nodes that Nodepool provides.
|
||||
|
||||
.. code-block:: console
|
||||
.. code-block:: shell
|
||||
|
||||
$ sudo groupadd --system nodepool
|
||||
$ sudo useradd --system nodepool --home-dir /var/lib/nodepool --create-home -g nodepool
|
||||
$ ssh-keygen -t rsa -b 2048 -f nodepool_rsa # don't enter a passphrase
|
||||
$ sudo mkdir /etc/nodepool/
|
||||
$ sudo mkdir /var/log/nodepool
|
||||
$ sudo chgrp -R nodepool /var/log/nodepool/
|
||||
$ sudo chmod 775 /var/log/nodepool/
|
||||
sudo groupadd --system nodepool
|
||||
sudo useradd --system nodepool --home-dir /var/lib/nodepool --create-home -g nodepool
|
||||
ssh-keygen -t rsa -b 2048 -f nodepool_rsa # don't enter a passphrase
|
||||
sudo mkdir /etc/nodepool/
|
||||
sudo mkdir /var/log/nodepool
|
||||
sudo chgrp -R nodepool /var/log/nodepool/
|
||||
sudo chmod 775 /var/log/nodepool/
|
||||
|
||||
Installation
|
||||
------------
|
||||
|
@ -27,21 +27,21 @@ Clone the Nodepool git repository and install it. The ``bindep``
|
|||
program is used to determine any additional binary dependencies which
|
||||
are required.
|
||||
|
||||
.. code-block:: console
|
||||
.. code-block:: shell
|
||||
|
||||
# All:
|
||||
$ git clone https://git.zuul-ci.org/nodepool
|
||||
$ pushd nodepool/
|
||||
git clone https://git.zuul-ci.org/nodepool
|
||||
pushd nodepool/
|
||||
|
||||
# For Fedora and CentOS:
|
||||
$ sudo yum -y install $(bindep -b compile)
|
||||
sudo yum -y install $(bindep -b compile)
|
||||
|
||||
# For openSUSE:
|
||||
$ sudo zypper install -y $(bindep -b compile)
|
||||
sudo zypper install -y $(bindep -b compile)
|
||||
|
||||
# All:
|
||||
$ sudo pip3 install .
|
||||
$ popd
|
||||
sudo pip3 install .
|
||||
popd
|
||||
|
||||
Service File
|
||||
------------
|
||||
|
@ -49,18 +49,18 @@ Service File
|
|||
Nodepool includes a systemd service file for nodepool-launcher in the ``etc``
|
||||
source directory. To use it, do the following steps.
|
||||
|
||||
.. code-block:: console
|
||||
.. code-block:: shell
|
||||
|
||||
$ sudo cp etc/nodepool-launcher.service /etc/systemd/system/nodepool-launcher.service
|
||||
$ sudo chmod 0644 /etc/systemd/system/nodepool-launcher.service
|
||||
sudo cp etc/nodepool-launcher.service /etc/systemd/system/nodepool-launcher.service
|
||||
sudo chmod 0644 /etc/systemd/system/nodepool-launcher.service
|
||||
|
||||
If you are installing Nodepool on ``CentOS 7`` and copied the provided service
|
||||
file in previous step, please follow the steps below to use corresponding
|
||||
systemd drop-in file so Nodepool service can be managed by systemd.
|
||||
|
||||
.. code-block:: console
|
||||
.. code-block:: shell
|
||||
|
||||
$ sudo mkdir /etc/systemd/system/nodepool-launcher.service.d
|
||||
$ sudo cp etc/nodepool-launcher.service.d/centos.conf \
|
||||
/etc/systemd/system/nodepool-launcher.service.d/centos.conf
|
||||
$ sudo chmod 0644 /etc/systemd/system/nodepool-launcher.service.d/centos.conf
|
||||
sudo mkdir /etc/systemd/system/nodepool-launcher.service.d
|
||||
sudo cp etc/nodepool-launcher.service.d/centos.conf \
|
||||
/etc/systemd/system/nodepool-launcher.service.d/centos.conf
|
||||
sudo chmod 0644 /etc/systemd/system/nodepool-launcher.service.d/centos.conf
|
||||
|
|
|
@ -10,14 +10,18 @@ Before starting on this, you need to download your `openrc`
|
|||
configuration from your OpenStack cloud. Put it on your server in the
|
||||
staging user's home directory. It should be called
|
||||
``<username>-openrc.sh``. Once that is done, create a new keypair
|
||||
that will be installed when instantiating the servers::
|
||||
that will be installed when instantiating the servers:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
cd ~
|
||||
source <username>-openrc.sh # this may prompt for password - enter it
|
||||
openstack keypair create --public-key nodepool_rsa.pub nodepool
|
||||
|
||||
We'll use the private key later wheen configuring Zuul. In the same
|
||||
session, configure nodepool to talk to your cloud::
|
||||
session, configure nodepool to talk to your cloud:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
umask 0066
|
||||
sudo mkdir -p ~nodepool/.config/openstack
|
||||
|
@ -48,7 +52,7 @@ configuration file:
|
|||
* flavor-name
|
||||
* image-name - from your cloud
|
||||
|
||||
::
|
||||
.. code-block:: shell
|
||||
|
||||
sudo bash -c "cat >/etc/nodepool/nodepool.yaml <<EOF
|
||||
zookeeper-servers:
|
||||
|
|
|
@ -19,38 +19,42 @@ the following requirements:
|
|||
|
||||
When setting up your nodepool.yaml file, you will need the host keys
|
||||
for each node for the ``host-key`` value. This can be obtained with
|
||||
the command::
|
||||
the command:
|
||||
|
||||
$ ssh-keyscan -t ed25519 <HOST>
|
||||
.. code-block:: shell
|
||||
|
||||
ssh-keyscan -t ed25519 <HOST>
|
||||
|
||||
Nodepool Configuration
|
||||
----------------------
|
||||
|
||||
Below is a sample Nodepool configuration file that sets up static
|
||||
nodes. Place this file in ``/etc/nodepool/nodepool.yaml``.
|
||||
nodes. Place this file in ``/etc/nodepool/nodepool.yaml``:
|
||||
|
||||
::
|
||||
.. code-block:: shell
|
||||
|
||||
zookeeper-servers:
|
||||
- host: localhost
|
||||
sudo bash -c "cat > /etc/nodepool/nodepool.yaml <<EOF
|
||||
zookeeper-servers:
|
||||
- host: localhost
|
||||
|
||||
labels:
|
||||
- name: ubuntu-xenial
|
||||
labels:
|
||||
- name: ubuntu-xenial
|
||||
|
||||
providers:
|
||||
- name: static-vms
|
||||
driver: static
|
||||
pools:
|
||||
- name: main
|
||||
nodes:
|
||||
- name: 192.168.1.10
|
||||
labels: ubuntu-xenial
|
||||
host-key: "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGXqY02bdYqg1BcIf2x08zs60rS6XhlBSQ4qE47o5gb"
|
||||
username: zuul
|
||||
- name: 192.168.1.11
|
||||
labels: ubuntu-xenial
|
||||
host-key: "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGXqY02bdYqg1BcIf2x08zs60rS6XhlBSQ5sE47o5gc"
|
||||
username: zuul
|
||||
providers:
|
||||
- name: static-vms
|
||||
driver: static
|
||||
pools:
|
||||
- name: main
|
||||
nodes:
|
||||
- name: 192.168.1.10
|
||||
labels: ubuntu-xenial
|
||||
host-key: "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGXqY02bdYqg1BcIf2x08zs60rS6XhlBSQ4qE47o5gb"
|
||||
username: zuul
|
||||
- name: 192.168.1.11
|
||||
labels: ubuntu-xenial
|
||||
host-key: "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGXqY02bdYqg1BcIf2x08zs60rS6XhlBSQ5sE47o5gc"
|
||||
username: zuul
|
||||
EOF"
|
||||
|
||||
Make sure that ``username``, ``host-key``, IP addresses and label names are
|
||||
customized for your environment.
|
||||
|
|
|
@ -22,13 +22,14 @@ dependencies are handled by the bindep program, but a few additional
|
|||
dependencies are needed to install bindep, and for other commands
|
||||
which we will use in these instructions.
|
||||
|
||||
::
|
||||
.. code-block:: shell
|
||||
|
||||
sudo zypper install -y git python3-pip
|
||||
|
||||
Then install bindep
|
||||
|
||||
::
|
||||
.. code-block:: shell
|
||||
|
||||
pip3 install --user bindep
|
||||
# Add it to your path
|
||||
PATH=~/.local/bin:$PATH
|
||||
|
@ -48,7 +49,7 @@ To download follow the directions on `Zookeeper's releases page
|
|||
<https://zookeeper.apache.org/releases.html>`_ to grab the latest
|
||||
release of zookeeper. Then:
|
||||
|
||||
::
|
||||
.. code-block:: shell
|
||||
|
||||
sudo zypper install -y java-1_8_0-openjdk
|
||||
tar -xzf zookeeper-3.4.12.tar.gz # Tarball downloaded from Zookeeper
|
||||
|
|
|
@ -1,123 +1,466 @@
|
|||
Quick Start Guide
|
||||
=================
|
||||
.. _quick_start:
|
||||
|
||||
This provides a very simple overview of Zuul. It is recommended to
|
||||
read the following sections for more details.
|
||||
Quick-Start Installation and Tutorial
|
||||
=====================================
|
||||
|
||||
Install Zuul
|
||||
------------
|
||||
Zuul is not like other CI or CD systems. It is a project gating
|
||||
system designed to assist developers in taking a change from proposal
|
||||
through deployment. Zuul can support any number of workflow processes
|
||||
and systems, but to help you get started with Zuul, this tutorial will
|
||||
walk through setting up a basic gating configuration which protects
|
||||
projects from merging broken code.
|
||||
|
||||
You can get zuul from pypi via::
|
||||
This tutorial is entirely self-contained and may safely be run on a
|
||||
workstation. The only requirements are a network connection and the
|
||||
ability to run Docker containers. For code review, it provides an
|
||||
instance of Gerrit, though the concepts you will learn apply equally
|
||||
to GitHub. Even if you don't ultimately intend to use Gerrit, you are
|
||||
encouraged to follow this tutorial to learn how to set up Zuul and
|
||||
then consult further documentation to configure your Zuul to interact
|
||||
with GitHub.
|
||||
|
||||
pip install zuul
|
||||
Start Zuul Containers
|
||||
---------------------
|
||||
|
||||
Zuul Components
|
||||
---------------
|
||||
Before you start, ensure that some needed packages are installed.
|
||||
|
||||
Zuul provides the following components:
|
||||
.. code-block:: shell
|
||||
|
||||
- **zuul-scheduler**: The main Zuul process. Handles receiving
|
||||
events, executing jobs, collecting results and posting reports.
|
||||
Coordinates the work of the other components.
|
||||
# Debian / Ubuntu:
|
||||
|
||||
- **zuul-merger**: Scale-out component that performs git merge
|
||||
operations. Zuul performs a large number of git operations in
|
||||
the course of its work. Adding merger processes can help speed
|
||||
Zuul's processing. This component is optional (zero or more of
|
||||
these can be run).
|
||||
sudo apt-get install docker-compose git git-review
|
||||
|
||||
- **zuul-executor**: Scale-out component for executing jobs. At
|
||||
least one of these is required. Depending on system
|
||||
configuration, you can expect a single executor to handle up to
|
||||
about 100 simultaneous jobs. Can handle the functions of a
|
||||
merger if dedicated mergers are not provided. One or more of
|
||||
these must be run.
|
||||
# Red Hat / Fedora / SUSE:
|
||||
|
||||
- **zuul-web**: A web server that currently provides websocket access to
|
||||
live-streaming of logs.
|
||||
sudo yum install docker-compose git git-review
|
||||
|
||||
- **gearman**: optional builtin gearman daemon provided by zuul-scheduler
|
||||
Clone the Zuul repository:
|
||||
|
||||
External components:
|
||||
.. code-block:: shell
|
||||
|
||||
- **gearman**: A gearman daemon if the built-in daemon is not used.
|
||||
git clone https://git.zuul-ci.org/zuul
|
||||
|
||||
- **zookeeper**: A zookeeper cluster (or single host) for
|
||||
communicating with Nodepool.
|
||||
Then cd into the directory containing this document, and run
|
||||
docker-compose in order to start Zuul, Nodepool and Gerrit.
|
||||
|
||||
- **nodepool**: Provides nodes for Zuul to use when executing jobs.
|
||||
.. code-block:: shell
|
||||
|
||||
cd zuul/doc/source/admin/examples
|
||||
docker-compose up
|
||||
|
||||
Zuul Setup
|
||||
----------
|
||||
All of the services will be started with debug-level logging sent to
|
||||
the standard output of the terminal where docker-compose is running.
|
||||
You will see a considerable amount of information scroll by, including
|
||||
some errors. Zuul will immediately attempt to connect to Gerrit and
|
||||
begin processing, even before Gerrit has fully initialized. The
|
||||
docker composition includes scripts to configure Gerrit and create an
|
||||
account for Zuul. Once this has all completed, the system should
|
||||
automatically connect, stabilize and become idle. When this is
|
||||
complete, you will have the following services running:
|
||||
|
||||
At minimum you need to provide **zuul.conf** and **main.yaml** placed
|
||||
in **/etc/zuul/**. The following example uses the builtin gearman
|
||||
service in Zuul, and a connection to Gerrit.
|
||||
* Zookeeper
|
||||
* Gerrit
|
||||
* Nodepool Launcher
|
||||
* Zuul Scheduler
|
||||
* Zuul Web Server
|
||||
* Zuul Executor
|
||||
* Apache HTTPD
|
||||
|
||||
**zuul.conf**::
|
||||
And a long-running static test node used by Nodepool and Zuul upon
|
||||
which to run tests.
|
||||
|
||||
[scheduler]
|
||||
tenant_config=/etc/zuul/main.yaml
|
||||
The Zuul scheduler is configured to connect to Gerrit via a connection
|
||||
named ``gerrit``. Zuul can interact with as many systems as
|
||||
necessary, each such connection is assigned a name for use in the Zuul
|
||||
configuration.
|
||||
|
||||
[gearman_server]
|
||||
start=true
|
||||
Zuul is a multi-tenant application, so that differing needs of
|
||||
independent work-groups can be supported from one system. This
|
||||
example configures a single tenant named ``example-tenant``. Assigned
|
||||
to this tenant are three projects: ``zuul-config``, ``test1`` and
|
||||
``test2``. These have already been created in Gerrit and are ready
|
||||
for us to begin using.
|
||||
|
||||
[gearman]
|
||||
server=127.0.0.1
|
||||
Add Your Gerrit Account
|
||||
-----------------------
|
||||
|
||||
[connection my_gerrit]
|
||||
driver=gerrit
|
||||
server=git.example.com
|
||||
port=29418
|
||||
baseurl=https://git.example.com/gerrit/
|
||||
user=zuul
|
||||
sshkey=/home/zuul/.ssh/id_rsa
|
||||
Before you can interact with Gerrit, you will need to create an
|
||||
account. The initialization script has already created an account for
|
||||
Zuul, but has left the task of creating your own account to you so
|
||||
that you can provide your own SSH key. You may safely use any
|
||||
existing SSH key on your workstation, or you may create a new one by
|
||||
running ``ssh-keygen``.
|
||||
|
||||
See :ref:`components` and :ref:`connections` for more details.
|
||||
Gerrit is configured in a development mode where passwords are not
|
||||
required in the web interface and you may become any user in the
|
||||
system at any time.
|
||||
|
||||
The following tells Zuul to read its configuration from and operate on
|
||||
the *example-project* project:
|
||||
To create your Gerrit account, visit http://localhost:8080 in your
|
||||
browser and click `Become`.
|
||||
|
||||
**main.yaml**::
|
||||
.. image:: images/become.png
|
||||
:align: center
|
||||
|
||||
- tenant:
|
||||
name: example-tenant
|
||||
source:
|
||||
my_gerrit:
|
||||
untrusted-projects:
|
||||
- example-project
|
||||
Then click `New Account` under `Register`.
|
||||
|
||||
Starting Zuul
|
||||
-------------
|
||||
.. image:: images/register.png
|
||||
:align: center
|
||||
|
||||
You can run any zuul process with the **-d** option to make it not
|
||||
daemonize. It's a good idea at first to confirm there's no issues with
|
||||
your configuration.
|
||||
Enter your full name and click `Save Changes`.
|
||||
|
||||
To start, simply run::
|
||||
.. image:: images/name.png
|
||||
:align: center
|
||||
|
||||
zuul-scheduler
|
||||
Enter the username you use to log into your workstation in the
|
||||
`Username` field and click `Select Username`.
|
||||
|
||||
Once run you should have two zuul-scheduler processes (if using the
|
||||
built-in gearman server, or one process otherwise).
|
||||
.. image:: images/username.png
|
||||
:align: center
|
||||
|
||||
Before Zuul can run any jobs, it needs to load its configuration, most
|
||||
of which is in the git repositories that Zuul operates on. Start an
|
||||
executor to allow zuul to do that::
|
||||
Copy and paste the contents of ``~/.ssh/id_rsa.pub`` into the SSH key
|
||||
field and click `Add`.
|
||||
|
||||
zuul-executor
|
||||
.. image:: images/sshkey.png
|
||||
:align: center
|
||||
|
||||
Zuul should now be able to read its configuration from the configured
|
||||
repo and process any jobs defined therein.
|
||||
Click the `Continue` link at the bottom of the page.
|
||||
|
||||
Troubleshooting
|
||||
---------------
|
||||
.. image:: images/continue.png
|
||||
:align: center
|
||||
|
||||
You can use telnet to connect to gearman to check which Zuul
|
||||
components are online::
|
||||
At this point you have created and logged into your personal account
|
||||
in Gerrit and are ready to begin configuring Zuul.
|
||||
|
||||
telnet <gearman_ip> 4730
|
||||
Configure Zuul Pipelines
|
||||
------------------------
|
||||
|
||||
Useful commands are **workers** and **status** which you can run by just
|
||||
typing those commands once connected to gearman.
|
||||
Zuul recognizes two types of projects: :term:`config
|
||||
projects<config-project>` and :term:`untrusted
|
||||
projects<untrusted-project>`. An *untrusted project* is a normal
|
||||
project from Zuul's point of view. In a gating system, it contains
|
||||
the software under development and/or most of the job content that
|
||||
Zuul will run. A *config project* is a special project that contains
|
||||
the Zuul's configuration. Because it has access to normally
|
||||
restricted features in Zuul, changes to this repository are not
|
||||
dynamically evaluated by Zuul. The security and functionality of the
|
||||
rest of the system depends on this repository, so it is best to limit
|
||||
what is contained within it to the minimum, and ensure thorough code
|
||||
review practices when changes are made.
|
||||
|
||||
Zuul has no built-in workflow definitions, so in order for it to do
|
||||
anything, you will need to begin by making changes to a *config
|
||||
project*. The initialization script has already created a project
|
||||
named ``zuul-config`` which you should now clone onto your workstation:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git clone http://localhost:8080/zuul-config
|
||||
|
||||
You will find that this repository is empty. Zuul reads its
|
||||
configuration from either a single file or a directory. In a *Config
|
||||
Project* with substantial Zuul configuration, you may find it easiest
|
||||
to use the ``zuul.d`` directory for Zuul configuration. Later, in
|
||||
*Untrusted Projects* you will use a single file for in-repo
|
||||
configuration. Make the directory:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
cd zuul-config
|
||||
mkdir zuul.d
|
||||
|
||||
The first type of configuration items we need to add are the Pipelines
|
||||
we intend to use. In Zuul, a Pipeline represents a workflow action.
|
||||
It is triggered by some action on a connection. Projects are able to
|
||||
attach jobs to run in that pipeline, and when they complete, the
|
||||
results are reported along with actions which may trigger further
|
||||
Pipelines. In a gating system two pipelines are required:
|
||||
:term:`check` and :term:`gate`. In our system, ``check`` will be
|
||||
triggered when a patch is uploaded to Gerrit, so that we are able to
|
||||
immediately run tests and report whether the change works and is
|
||||
therefore able to merge. The ``gate`` pipeline is triggered when a code
|
||||
reviewer approves the change in Gerrit. It will run test jobs again
|
||||
(in case other changes have merged since the change in question was
|
||||
uploaded) and if these final tests pass, will automatically merge the
|
||||
change. To configure these pipelines, copy the following file into
|
||||
`zuul.d/pipelines.yaml`:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/zuul.d/pipelines.yaml
|
||||
:language: yaml
|
||||
|
||||
Once we have bootstrapped our initial Zuul configuration, we will want
|
||||
to use the gating process on this repository too, so we need to attach
|
||||
the ``zuul-config`` repository to the ``check`` and ``gate`` pipelines
|
||||
we are about to create. There are no jobs defined yet, so we must use
|
||||
the internally defined ``noop`` job, which always returns success.
|
||||
Later on we will be configuring some other projects, and while we will
|
||||
be able to dynamically add jobs to their pipelines, those projects
|
||||
must first be attached to the pipelines in order for that to work. In
|
||||
our system, we want all of the projects in Gerrit to participate in
|
||||
the check and gate pipelines, so we can use a regular expression to
|
||||
apply this to all projects. To configure the ``check`` and ``gate``
|
||||
pipelines for ``zuul-config`` to run the ``noop`` job, and add all
|
||||
projects to those pipelines (with no jobs), copy the following file
|
||||
into ``zuul.d/projects.yaml``:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/zuul.d/projects.yaml
|
||||
:language: yaml
|
||||
|
||||
Every real job (i.e., all jobs other than ``noop``) must inherit from a
|
||||
:term:`base job`, and base jobs may only be defined in a
|
||||
:term:`config-project`. Let's go ahead and add a simple base job that
|
||||
we can build on later. Copy the following into ``zuul.d/jobs.yaml``:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/zuul.d/jobs.yaml
|
||||
:language: yaml
|
||||
|
||||
Commit the changes and push them up for review:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git add zuul.d
|
||||
git commit -m "Add initial Zuul configuration"
|
||||
git review
|
||||
|
||||
Because Zuul is currently running with no configuration whatsoever, it
|
||||
will ignore this change. For this initial change which bootstraps the
|
||||
entire system, we will need to bypass code review (hopefully for the
|
||||
last time). To do this, you need to switch to the Administrator
|
||||
account in Gerrit. Visit http://localhost:8080 in your browser and
|
||||
then:
|
||||
|
||||
Click on your name in the top right corner then click `Switch
|
||||
Account`.
|
||||
|
||||
.. image:: images/switch-example.png
|
||||
:align: center
|
||||
|
||||
Click `admin` to log in as the `admin` user.
|
||||
|
||||
.. image:: images/become-select.png
|
||||
:align: center
|
||||
|
||||
In the top left corner of the page, click `All` and `Open` to see the
|
||||
list of open changes, then click on the change you uploaded.
|
||||
|
||||
.. image:: images/open-changes.png
|
||||
:align: center
|
||||
|
||||
Click `Reply...` at the top center of the change screen. This will
|
||||
open a dialog where you can leave a review message and vote on the
|
||||
change. As the administrator, you have access to vote in all of the
|
||||
review categories, even `Verified` which is normally reserved for
|
||||
Zuul. Vote Code-Review: +2, Verified: +2, Workflow: +1, and then
|
||||
click `Post` to leave your approval votes.
|
||||
|
||||
.. image:: images/review-1001.png
|
||||
:align: center
|
||||
|
||||
Once the required votes have been set, the `Submit` button will
|
||||
appear; click it. This will cause the change to be merged
|
||||
immediately. This is normally handled by Zuul, but as the
|
||||
administrator you can bypass Zuul to forcibly merge a change.
|
||||
|
||||
.. image:: images/submit-1001.png
|
||||
:align: center
|
||||
|
||||
Now that the initial configuration has been bootstrapped, you should
|
||||
not need to bypass testing and code review again, so switch back to
|
||||
the account you created for yourself. Click on `Administrator` in the
|
||||
top right corner then click `Switch Account`.
|
||||
|
||||
.. image:: images/switch-admin.png
|
||||
:align: center
|
||||
|
||||
Click your username.
|
||||
|
||||
.. image:: images/become-select.png
|
||||
:align: center
|
||||
|
||||
Test Zuul Pipelines
|
||||
-------------------
|
||||
|
||||
Zuul is now running with a basic :term:`check` and :term:`gate`
|
||||
configuration. We can now begin adding Zuul configuration to one of
|
||||
our :term:`untrusted projects<untrusted-project>`. Start by cloning
|
||||
the `test1` project which was created by the setup script.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
cd ..
|
||||
git clone http://localhost:8080/test1
|
||||
|
||||
Every Zuul job that runs needs a playbook, so let's create a
|
||||
sub-directory in the project to hold playbooks:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
cd test1
|
||||
mkdir playbooks
|
||||
|
||||
Start with a simple playbook which just outputs a debug message. Copy
|
||||
the following to ``playbooks/testjob.yaml``:
|
||||
|
||||
.. literalinclude:: examples/test1/playbooks/testjob.yaml
|
||||
:language: yaml
|
||||
|
||||
Now define a Zuul job which runs that playbook. Zuul will read its
|
||||
configuration from any of ``zuul.d/`` or ``.zuul.d/`` directories, or
|
||||
the files ``zuul.yaml`` or ``.zuul.yaml``. Generally in an *untrusted
|
||||
project* which isn't dedicated entirely to Zuul, it's best to put
|
||||
Zuul's configuration in a hidden file. Copy the following to
|
||||
``.zuul.yaml`` in the root of the project:
|
||||
|
||||
.. literalinclude:: examples/test1/zuul.yaml
|
||||
:language: yaml
|
||||
|
||||
Commit the changes and push them up to Gerrit for review:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git add .zuul.yaml playbooks
|
||||
git commit -m "Add test Zuul job"
|
||||
git review
|
||||
|
||||
Zuul will dynamically evaluate proposed changes to its configuration
|
||||
in *untrusted projects* immediately, so shortly after your change is
|
||||
uploaded, Zuul will run the new job and report back on the change.
|
||||
|
||||
Visit http://localhost:8080/#/dashboard/self and open the change you
|
||||
just uploaded. If the build is complete, Zuul should have left a
|
||||
Verified: +1 vote on the change, along with a comment at the bottom.
|
||||
Expand the comments and you should see that the job succeeded, but
|
||||
there are no logs and no way to see the output, only a `finger` URL
|
||||
(which is what Zuul reports when it doesn't know where build logs are
|
||||
stored).
|
||||
|
||||
.. image:: images/check1-1002.png
|
||||
:align: center
|
||||
|
||||
This means everything is working so far, but we need to configure a
|
||||
bit more before we have a useful job.
|
||||
|
||||
Configure a Base Job
|
||||
--------------------
|
||||
|
||||
Every Zuul tenant needs at least one base job. Zuul administrators
|
||||
can use a base job to customize Zuul to the local environment. This
|
||||
may include tasks which run both before jobs, such as setting up
|
||||
package mirrors or networking configuration, or after jobs, such as
|
||||
artifact and log storage.
|
||||
|
||||
Zuul doesn't take anything for granted, and even tasks such as copying
|
||||
the git repos for the project being tested onto the remote node must
|
||||
be explicitly added to a base job (and can therefore be customized as
|
||||
needed). The Zuul in this tutorial is pre-configured to use the `zuul
|
||||
jobs`_ repository which is the "standard library" of Zuul jobs and
|
||||
roles. We will make use of it to quickly create a base job which
|
||||
performs the necessary set up actions and stores build logs.
|
||||
|
||||
.. _zuul jobs: https://zuul-ci.org/docs/zuul-jobs/
|
||||
|
||||
Return to the ``zuul-config`` repo that you were working in earlier.
|
||||
We're going to add some playbooks to the empty base job we created
|
||||
earlier. Start by creating a directory to store those playbooks:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
cd ..
|
||||
cd zuul-config
|
||||
mkdir -p playbooks/base
|
||||
|
||||
Zuul supports running any number of playbooks before a job (called
|
||||
*pre-run* playbooks) or after a job (called *post-run* playbooks).
|
||||
We're going to add a single *pre-run* playbook now. Copy the
|
||||
following to ``playbooks/base/pre.yaml``:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/playbooks/base/pre.yaml
|
||||
:language: yaml
|
||||
|
||||
This playbook does two things; first it creates a new SSH key and adds
|
||||
it to all of the hosts in the inventory, and removes the private key
|
||||
that Zuul normally uses to log into nodes from the running SSH agent.
|
||||
This is just an extra bit of protection which ensures that if Zuul's
|
||||
SSH key has access to any important systems, normal Zuul jobs can't
|
||||
use it. The second thing the playbook does is copy the git
|
||||
repositories that Zuul has prepared (which may have one or more
|
||||
changes being tested) to all of the nodes used in the job.
|
||||
|
||||
Next, add a *post-run* playbook to remove the per-build SSH key. Copy
|
||||
the following to ``playbooks/base/post-ssh.yaml``:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/playbooks/base/post-ssh.yaml
|
||||
:language: yaml
|
||||
|
||||
This is the complement of the `add-build-sshkey` role in the pre-run
|
||||
playbook -- it simply removes the per-build ssh key from any remote
|
||||
systems. Zuul always tries to run all of the post-run playbooks
|
||||
regardless of whether any previous playbooks have failed. Because we
|
||||
always want log collection to run and we want it to run last, we
|
||||
create a second post-run playbook for it. Copy the following to
|
||||
``playbooks/base/post-logs.yaml``:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/playbooks/base/post-logs.yaml
|
||||
:language: yaml
|
||||
|
||||
This tutorial is running an Apache webserver in a container which will
|
||||
serve build logs from a volume that is shared with the Zuul executor.
|
||||
That volume is mounted at `/srv/static/logs`, which is the default
|
||||
location in the `upload-logs`_ role. The role also supports copying
|
||||
files to a remote server via SCP; see the role documentation for how
|
||||
to configure it. For this simple case, the only option we need to
|
||||
provide is the URL where the logs can ultimately be found.
|
||||
|
||||
.. note:: Zuul-jobs also contains a `role
|
||||
<https://zuul-ci.org/docs/zuul-jobs/roles.html#role-upload-logs-swift>`_
|
||||
to upload logs to an OpenStack Object Storage (swift)
|
||||
container. If you create a role to upload logs to another
|
||||
system, please feel free to contribute it to the zuul-jobs
|
||||
repository for others to use.
|
||||
|
||||
.. _upload-logs: https://zuul-ci.org/docs/zuul-jobs/roles.html#role-upload-logs
|
||||
|
||||
Now that the new playbooks are in place, update the ``base`` job
|
||||
definition to include them. Overwrite ``zuul.d/jobs.yaml`` with the
|
||||
following:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/zuul.d/jobs2.yaml
|
||||
:language: yaml
|
||||
|
||||
Then commit the change and upload it to Gerrit for review:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git add playbooks zuul.d/jobs.yaml
|
||||
git commit -m "Update Zuul base job"
|
||||
git review
|
||||
|
||||
Visit http://localhost:8080/#/dashboard/self and open the
|
||||
``zuul-config`` change you just uploaded.
|
||||
|
||||
You should see a Verified +1 vote from Zuul. Click `Reply` then vote
|
||||
Code-Review: +2 and Workflow: +1 then click 'Post'.
|
||||
|
||||
.. image:: images/review-1003.png
|
||||
:align: center
|
||||
|
||||
Wait a few moments for Zuul to process the event, and then reload the
|
||||
page. The change should have been merged.
|
||||
|
||||
Visit http://localhost:8080/#/dashboard/self and return to the
|
||||
``test1`` change you uploaded earlier. Click `Reply` then type
|
||||
`recheck` into the text field and click `Post`.
|
||||
|
||||
.. image:: images/recheck-1002.png
|
||||
:align: center
|
||||
|
||||
This will cause Zuul to re-run the test job we created earlier. This
|
||||
time it will run with the updated base job configuration, and when
|
||||
complete, it will report the published log location as a comment on
|
||||
the change:
|
||||
|
||||
.. image:: images/check2-1002.png
|
||||
:align: center
|
||||
|
||||
Follow the link and you will be able to browse the console log for the
|
||||
job. In the middle of the log, you should see the "Hello, world!"
|
||||
output from the job's playbook.
|
||||
|
|
|
@ -0,0 +1,10 @@
|
|||
Troubleshooting
|
||||
---------------
|
||||
|
||||
You can use telnet to connect to gearman to check which Zuul
|
||||
components are online::
|
||||
|
||||
telnet <gearman_ip> 4730
|
||||
|
||||
Useful commands are ``workers`` and ``status`` which you can run by just
|
||||
typing those commands once connected to gearman.
|
|
@ -1,464 +0,0 @@
|
|||
Installation and Setup Tutorial
|
||||
===============================
|
||||
|
||||
Zuul is not like other CI or CD systems. It is a project gating
|
||||
system designed to assist developers in taking a change from proposal
|
||||
through deployment. Zuul can support any number of workflow processes
|
||||
and systems, but to help you get started with Zuul, this tutorial will
|
||||
walk through setting up a basic gating configuration which protects
|
||||
projects from merging broken code.
|
||||
|
||||
This tutorial is entirely self-contained and may safely be run on a
|
||||
workstation. The only requirements are a network connection and the
|
||||
ability to run Docker containers. For code review, it provides an
|
||||
instance of Gerrit, though the concepts you will learn apply equally
|
||||
to GitHub. Even if you don't ultimately intend to use Gerrit, you are
|
||||
encouraged to follow this tutorial to learn how to set up Zuul and
|
||||
then consult further documentation to configure your Zuul to interact
|
||||
with GitHub.
|
||||
|
||||
Start Zuul Containers
|
||||
---------------------
|
||||
|
||||
Before you start, ensure that some needed packages are installed.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
# Debian / Ubuntu:
|
||||
|
||||
sudo apt-get install docker-compose git git-review
|
||||
|
||||
# Red Hat / Fedora / SUSE:
|
||||
|
||||
sudo yum install docker-compose git git-review
|
||||
|
||||
Clone the Zuul repository:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git clone https://git.zuul-ci.org/zuul
|
||||
|
||||
Then cd into the directory containing this document, and run
|
||||
docker-compose in order to start Zuul, Nodepool and Gerrit.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
cd zuul/doc/source/admin/examples
|
||||
docker-compose up
|
||||
|
||||
All of the services will be started with debug-level logging sent to
|
||||
the standard output of the terminal where docker-compose is running.
|
||||
You will see a considerable amount of information scroll by, including
|
||||
some errors. Zuul will immediately attempt to connect to Gerrit and
|
||||
begin processing, even before Gerrit has fully initialized. The
|
||||
docker composition includes scripts to configure Gerrit and create an
|
||||
account for Zuul. Once this has all completed, the system should
|
||||
automatically connect, stabilize and become idle. When this is
|
||||
complete, you will have the following services running:
|
||||
|
||||
* Zookeeper
|
||||
* Gerrit
|
||||
* Nodepool Launcher
|
||||
* Zuul Scheduler
|
||||
* Zuul Web Server
|
||||
* Zuul Executor
|
||||
* Apache HTTPD
|
||||
|
||||
And a long-running static test node used by Nodepool and Zuul upon
|
||||
which to run tests.
|
||||
|
||||
The Zuul scheduler is configured to connect to Gerrit via a connection
|
||||
named ``gerrit``. Zuul can interact with as many systems as
|
||||
necessary, each such connection is assigned a name for use in the Zuul
|
||||
configuration.
|
||||
|
||||
Zuul is a multi-tenant application, so that differing needs of
|
||||
independent work-groups can be supported from one system. This
|
||||
example configures a single tenant named ``example-tenant``. Assigned
|
||||
to this tenant are three projects: ``zuul-config``, ``test1`` and
|
||||
``test2``. These have already been created in Gerrit and are ready
|
||||
for us to begin using.
|
||||
|
||||
Add Your Gerrit Account
|
||||
-----------------------
|
||||
|
||||
Before you can interact with Gerrit, you will need to create an
|
||||
account. The initialization script has already created an account for
|
||||
Zuul, but has left the task of creating your own account to you so
|
||||
that you can provide your own SSH key. You may safely use any
|
||||
existing SSH key on your workstation, or you may create a new one by
|
||||
running ``ssh-keygen``.
|
||||
|
||||
Gerrit is configured in a development mode where passwords are not
|
||||
required in the web interface and you may become any user in the
|
||||
system at any time.
|
||||
|
||||
To create your Gerrit account, visit http://localhost:8080 in your
|
||||
browser and click `Become`.
|
||||
|
||||
.. image:: images/become.png
|
||||
:align: center
|
||||
|
||||
Then click `New Account` under `Register`.
|
||||
|
||||
.. image:: images/register.png
|
||||
:align: center
|
||||
|
||||
Enter your full name and click `Save Changes`.
|
||||
|
||||
.. image:: images/name.png
|
||||
:align: center
|
||||
|
||||
Enter the username you use to log into your workstation in the
|
||||
`Username` field and click `Select Username`.
|
||||
|
||||
.. image:: images/username.png
|
||||
:align: center
|
||||
|
||||
Copy and paste the contents of ``~/.ssh/id_rsa.pub`` into the SSH key
|
||||
field and click `Add`.
|
||||
|
||||
.. image:: images/sshkey.png
|
||||
:align: center
|
||||
|
||||
Click the `Continue` link at the bottom of the page.
|
||||
|
||||
.. image:: images/continue.png
|
||||
:align: center
|
||||
|
||||
At this point you have created and logged into your personal account
|
||||
in Gerrit and are ready to begin configuring Zuul.
|
||||
|
||||
Configure Zuul Pipelines
|
||||
------------------------
|
||||
|
||||
Zuul recognizes two types of projects: :term:`config
|
||||
projects<config-project>` and :term:`untrusted
|
||||
projects<untrusted-project>`. An *untrusted project* is a normal
|
||||
project from Zuul's point of view. In a gating system, it contains
|
||||
the software under development and/or most of the job content that
|
||||
Zuul will run. A *config project* is a special project that contains
|
||||
the Zuul's configuration. Because it has access to normally
|
||||
restricted features in Zuul, changes to this repository are not
|
||||
dynamically evaluated by Zuul. The security and functionality of the
|
||||
rest of the system depends on this repository, so it is best to limit
|
||||
what is contained within it to the minimum, and ensure thorough code
|
||||
review practices when changes are made.
|
||||
|
||||
Zuul has no built-in workflow definitions, so in order for it to do
|
||||
anything, you will need to begin by making changes to a *config
|
||||
project*. The initialization script has already created a project
|
||||
named ``zuul-config`` which you should now clone onto your workstation:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git clone http://localhost:8080/zuul-config
|
||||
|
||||
You will find that this repository is empty. Zuul reads its
|
||||
configuration from either a single file or a directory. In a *Config
|
||||
Project* with substantial Zuul configuration, you may find it easiest
|
||||
to use the ``zuul.d`` directory for Zuul configuration. Later, in
|
||||
*Untrusted Projects* you will use a single file for in-repo
|
||||
configuration. Make the directory:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
cd zuul-config
|
||||
mkdir zuul.d
|
||||
|
||||
The first type of configuration items we need to add are the Pipelines
|
||||
we intend to use. In Zuul, a Pipeline represents a workflow action.
|
||||
It is triggered by some action on a connection. Projects are able to
|
||||
attach jobs to run in that pipeline, and when they complete, the
|
||||
results are reported along with actions which may trigger further
|
||||
Pipelines. In a gating system two pipelines are required:
|
||||
:term:`check` and :term:`gate`. In our system, ``check`` will be
|
||||
triggered when a patch is uploaded to Gerrit, so that we are able to
|
||||
immediately run tests and report whether the change works and is
|
||||
therefore able to merge. The ``gate`` pipeline is triggered when a code
|
||||
reviewer approves the change in Gerrit. It will run test jobs again
|
||||
(in case other changes have merged since the change in question was
|
||||
uploaded) and if these final tests pass, will automatically merge the
|
||||
change. To configure these pipelines, copy the following file into
|
||||
`zuul.d/pipelines.yaml`:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/zuul.d/pipelines.yaml
|
||||
:language: yaml
|
||||
|
||||
Once we have bootstrapped our initial Zuul configuration, we will want
|
||||
to use the gating process on this repository too, so we need to attach
|
||||
the ``zuul-config`` repository to the ``check`` and ``gate`` pipelines
|
||||
we are about to create. There are no jobs defined yet, so we must use
|
||||
the internally defined ``noop`` job, which always returns success.
|
||||
Later on we will be configuring some other projects, and while we will
|
||||
be able to dynamically add jobs to their pipelines, those projects
|
||||
must first be attached to the pipelines in order for that to work. In
|
||||
our system, we want all of the projects in Gerrit to participate in
|
||||
the check and gate pipelines, so we can use a regular expression to
|
||||
apply this to all projects. To configure the ``check`` and ``gate``
|
||||
pipelines for ``zuul-config`` to run the ``noop`` job, and add all
|
||||
projects to those pipelines (with no jobs), copy the following file
|
||||
into ``zuul.d/projects.yaml``:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/zuul.d/projects.yaml
|
||||
:language: yaml
|
||||
|
||||
Every real job (i.e., all jobs other than ``noop``) must inherit from a
|
||||
:term:`base job`, and base jobs may only be defined in a
|
||||
:term:`config-project`. Let's go ahead and add a simple base job that
|
||||
we can build on later. Copy the following into ``zuul.d/jobs.yaml``:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/zuul.d/jobs.yaml
|
||||
:language: yaml
|
||||
|
||||
Commit the changes and push them up for review:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git add zuul.d
|
||||
git commit -m "Add initial Zuul configuration"
|
||||
git review
|
||||
|
||||
Because Zuul is currently running with no configuration whatsoever, it
|
||||
will ignore this change. For this initial change which bootstraps the
|
||||
entire system, we will need to bypass code review (hopefully for the
|
||||
last time). To do this, you need to switch to the Administrator
|
||||
account in Gerrit. Visit http://localhost:8080 in your browser and
|
||||
then:
|
||||
|
||||
Click on your name in the top right corner then click `Switch
|
||||
Account`.
|
||||
|
||||
.. image:: images/switch-example.png
|
||||
:align: center
|
||||
|
||||
Click `admin` to log in as the `admin` user.
|
||||
|
||||
.. image:: images/become-select.png
|
||||
:align: center
|
||||
|
||||
In the top left corner of the page, click `All` and `Open` to see the
|
||||
list of open changes, then click on the change you uploaded.
|
||||
|
||||
.. image:: images/open-changes.png
|
||||
:align: center
|
||||
|
||||
Click `Reply...` at the top center of the change screen. This will
|
||||
open a dialog where you can leave a review message and vote on the
|
||||
change. As the administrator, you have access to vote in all of the
|
||||
review categories, even `Verified` which is normally reserved for
|
||||
Zuul. Vote Code-Review: +2, Verified: +2, Workflow: +1, and then
|
||||
click `Post` to leave your approval votes.
|
||||
|
||||
.. image:: images/review-1001.png
|
||||
:align: center
|
||||
|
||||
Once the required votes have been set, the `Submit` button will
|
||||
appear; click it. This will cause the change to be merged
|
||||
immediately. This is normally handled by Zuul, but as the
|
||||
administrator you can bypass Zuul to forcibly merge a change.
|
||||
|
||||
.. image:: images/submit-1001.png
|
||||
:align: center
|
||||
|
||||
Now that the initial configuration has been bootstrapped, you should
|
||||
not need to bypass testing and code review again, so switch back to
|
||||
the account you created for yourself. Click on `Administrator` in the
|
||||
top right corner then click `Switch Account`.
|
||||
|
||||
.. image:: images/switch-admin.png
|
||||
:align: center
|
||||
|
||||
Click your username.
|
||||
|
||||
.. image:: images/become-select.png
|
||||
:align: center
|
||||
|
||||
Test Zuul Pipelines
|
||||
-------------------
|
||||
|
||||
Zuul is now running with a basic :term:`check` and :term:`gate`
|
||||
configuration. We can now begin adding Zuul configuration to one of
|
||||
our :term:`untrusted projects<untrusted-project>`. Start by cloning
|
||||
the `test1` project which was created by the setup script.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
cd ..
|
||||
git clone http://localhost:8080/test1
|
||||
|
||||
Every Zuul job that runs needs a playbook, so let's create a
|
||||
sub-directory in the project to hold playbooks:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
cd test1
|
||||
mkdir playbooks
|
||||
|
||||
Start with a simple playbook which just outputs a debug message. Copy
|
||||
the following to ``playbooks/testjob.yaml``:
|
||||
|
||||
.. literalinclude:: examples/test1/playbooks/testjob.yaml
|
||||
:language: yaml
|
||||
|
||||
Now define a Zuul job which runs that playbook. Zuul will read its
|
||||
configuration from any of ``zuul.d/`` or ``.zuul.d/`` directories, or
|
||||
the files ``zuul.yaml`` or ``.zuul.yaml``. Generally in an *untrusted
|
||||
project* which isn't dedicated entirely to Zuul, it's best to put
|
||||
Zuul's configuration in a hidden file. Copy the following to
|
||||
``.zuul.yaml`` in the root of the project:
|
||||
|
||||
.. literalinclude:: examples/test1/zuul.yaml
|
||||
:language: yaml
|
||||
|
||||
Commit the changes and push them up to Gerrit for review:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git add .zuul.yaml playbooks
|
||||
git commit -m "Add test Zuul job"
|
||||
git review
|
||||
|
||||
Zuul will dynamically evaluate proposed changes to its configuration
|
||||
in *untrusted projects* immediately, so shortly after your change is
|
||||
uploaded, Zuul will run the new job and report back on the change.
|
||||
|
||||
Visit http://localhost:8080/#/dashboard/self and open the change you
|
||||
just uploaded. If the build is complete, Zuul should have left a
|
||||
Verified: +1 vote on the change, along with a comment at the bottom.
|
||||
Expand the comments and you should see that the job succeeded, but
|
||||
there are no logs and no way to see the output, only a `finger` URL
|
||||
(which is what Zuul reports when it doesn't know where build logs are
|
||||
stored).
|
||||
|
||||
.. image:: images/check1-1002.png
|
||||
:align: center
|
||||
|
||||
This means everything is working so far, but we need to configure a
|
||||
bit more before we have a useful job.
|
||||
|
||||
Configure a Base Job
|
||||
--------------------
|
||||
|
||||
Every Zuul tenant needs at least one base job. Zuul administrators
|
||||
can use a base job to customize Zuul to the local environment. This
|
||||
may include tasks which run both before jobs, such as setting up
|
||||
package mirrors or networking configuration, or after jobs, such as
|
||||
artifact and log storage.
|
||||
|
||||
Zuul doesn't take anything for granted, and even tasks such as copying
|
||||
the git repos for the project being tested onto the remote node must
|
||||
be explicitly added to a base job (and can therefore be customized as
|
||||
needed). The Zuul in this tutorial is pre-configured to use the `zuul
|
||||
jobs`_ repository which is the "standard library" of Zuul jobs and
|
||||
roles. We will make use of it to quickly create a base job which
|
||||
performs the necessary set up actions and stores build logs.
|
||||
|
||||
.. _zuul jobs: https://zuul-ci.org/docs/zuul-jobs/
|
||||
|
||||
Return to the ``zuul-config`` repo that you were working in earlier.
|
||||
We're going to add some playbooks to the empty base job we created
|
||||
earlier. Start by creating a directory to store those playbooks:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
cd ..
|
||||
cd zuul-config
|
||||
mkdir -p playbooks/base
|
||||
|
||||
Zuul supports running any number of playbooks before a job (called
|
||||
*pre-run* playbooks) or after a job (called *post-run* playbooks).
|
||||
We're going to add a single *pre-run* playbook now. Copy the
|
||||
following to ``playbooks/base/pre.yaml``:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/playbooks/base/pre.yaml
|
||||
:language: yaml
|
||||
|
||||
This playbook does two things; first it creates a new SSH key and adds
|
||||
it to all of the hosts in the inventory, and removes the private key
|
||||
that Zuul normally uses to log into nodes from the running SSH agent.
|
||||
This is just an extra bit of protection which ensures that if Zuul's
|
||||
SSH key has access to any important systems, normal Zuul jobs can't
|
||||
use it. The second thing the playbook does is copy the git
|
||||
repositories that Zuul has prepared (which may have one or more
|
||||
changes being tested) to all of the nodes used in the job.
|
||||
|
||||
Next, add a *post-run* playbook to remove the per-build SSH key. Copy
|
||||
the following to ``playbooks/base/post-ssh.yaml``:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/playbooks/base/post-ssh.yaml
|
||||
:language: yaml
|
||||
|
||||
This is the complement of the `add-build-sshkey` role in the pre-run
|
||||
playbook -- it simply removes the per-build ssh key from any remote
|
||||
systems. Zuul always tries to run all of the post-run playbooks
|
||||
regardless of whether any previous playbooks have failed. Because we
|
||||
always want log collection to run and we want it to run last, we
|
||||
create a second post-run playbook for it. Copy the following to
|
||||
``playbooks/base/post-logs.yaml``:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/playbooks/base/post-logs.yaml
|
||||
:language: yaml
|
||||
|
||||
This tutorial is running an Apache webserver in a container which will
|
||||
serve build logs from a volume that is shared with the Zuul executor.
|
||||
That volume is mounted at `/srv/static/logs`, which is the default
|
||||
location in the `upload-logs`_ role. The role also supports copying
|
||||
files to a remote server via SCP; see the role documentation for how
|
||||
to configure it. For this simple case, the only option we need to
|
||||
provide is the URL where the logs can ultimately be found.
|
||||
|
||||
.. note:: Zuul-jobs also contains a `role
|
||||
<https://zuul-ci.org/docs/zuul-jobs/roles.html#role-upload-logs-swift>`_
|
||||
to upload logs to an OpenStack Object Storage (swift)
|
||||
container. If you create a role to upload logs to another
|
||||
system, please feel free to contribute it to the zuul-jobs
|
||||
repository for others to use.
|
||||
|
||||
.. _upload-logs: https://zuul-ci.org/docs/zuul-jobs/roles.html#role-upload-logs
|
||||
|
||||
Now that the new playbooks are in place, update the ``base`` job
|
||||
definition to include them. Overwrite ``zuul.d/jobs.yaml`` with the
|
||||
following:
|
||||
|
||||
.. literalinclude:: examples/zuul-config/zuul.d/jobs2.yaml
|
||||
:language: yaml
|
||||
|
||||
Then commit the change and upload it to Gerrit for review:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
git add playbooks zuul.d/jobs.yaml
|
||||
git commit -m "Update Zuul base job"
|
||||
git review
|
||||
|
||||
Visit http://localhost:8080/#/dashboard/self and open the
|
||||
``zuul-config`` change you just uploaded.
|
||||
|
||||
You should see a Verified +1 vote from Zuul. Click `Reply` then vote
|
||||
Code-Review: +2 and Workflow: +1 then click 'Post'.
|
||||
|
||||
.. image:: images/review-1003.png
|
||||
:align: center
|
||||
|
||||
Wait a few moments for Zuul to process the event, and then reload the
|
||||
page. The change should have been merged.
|
||||
|
||||
Visit http://localhost:8080/#/dashboard/self and return to the
|
||||
``test1`` change you uploaded earlier. Click `Reply` then type
|
||||
`recheck` into the text field and click `Post`.
|
||||
|
||||
.. image:: images/recheck-1002.png
|
||||
:align: center
|
||||
|
||||
This will cause Zuul to re-run the test job we created earlier. This
|
||||
time it will run with the updated base job configuration, and when
|
||||
complete, it will report the published log location as a comment on
|
||||
the change:
|
||||
|
||||
.. image:: images/check2-1002.png
|
||||
:align: center
|
||||
|
||||
Follow the link and you will be able to browse the console log for the
|
||||
job. In the middle of the log, you should see the "Hello, world!"
|
||||
output from the job's playbook.
|
|
@ -1,9 +1,14 @@
|
|||
Zuul From Scratch
|
||||
=================
|
||||
|
||||
.. note:: This is a work in progress that attempts to walk through all
|
||||
of the steps needed to run Zuul on a all-in-one server, and
|
||||
demonstrate running against either a GitHub or Gerrit project.
|
||||
This document details a fully manual installation of Zuul on a
|
||||
all-in-one server. If you want to learn all the details about how to
|
||||
install Zuul without the aid of any existing installation tools, you
|
||||
may find this a useful reference.
|
||||
|
||||
If, instead, you want to get Zuul running quickly, see the
|
||||
:ref:`quick_start` which runs all of the Zuul services in containers
|
||||
with a single command.
|
||||
|
||||
Environment Setup
|
||||
-----------------
|
||||
|
@ -40,7 +45,7 @@ server to listen on all public addresses. This is so that Zuul may
|
|||
receive webhook events from GitHub. You may wish to proxy this or
|
||||
further restrict public access.
|
||||
|
||||
::
|
||||
.. code-block:: shell
|
||||
|
||||
sudo bash -c "cat > /etc/zuul/zuul.conf <<EOF
|
||||
[gearman]
|
||||
|
@ -68,7 +73,7 @@ Starting Services
|
|||
-----------------
|
||||
|
||||
After you have Zookeeper, Nodepool, and Zuul installed and configured, you can
|
||||
start Nodepool and Zuul services with::
|
||||
start Nodepool and Zuul services with::
|
||||
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
|
@ -93,7 +98,9 @@ Zuul provides a `standard library`_ of jobs and roles. To take advantage
|
|||
of these jobs, add the ``zuul-jobs`` repo, which is hosted by the Zuul
|
||||
project, to your system.
|
||||
|
||||
Add to ``/etc/zuul/zuul.conf``::
|
||||
Add to ``/etc/zuul/zuul.conf``:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
sudo bash -c "cat >> /etc/zuul/zuul.conf <<EOF
|
||||
|
||||
|
@ -102,7 +109,9 @@ Add to ``/etc/zuul/zuul.conf``::
|
|||
baseurl=https://git.zuul-ci.org/
|
||||
EOF"
|
||||
|
||||
Restart executor and scheduler::
|
||||
Restart executor and scheduler:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
sudo systemctl restart zuul-executor.service
|
||||
sudo systemctl restart zuul-scheduler.service
|
||||
|
|
Loading…
Reference in New Issue