This change removes ceph* Dockerfiles and relevant kolla python
code bits.
Kolla-Ansible part has been removed, and K-A was the only consumer.
Change-Id: I9568261fdbbe4b9156d0e07c414ec911ca2e8557
Disable external repositories by default and enable only when needed.
Depends-on: https://review.opendev.org/696480
Implements: blueprint repos-off-by-default
Change-Id: Icf2a8397a8349e0fe849d88d160409fd234480a9
This patch will add new function in extend_start.sh for OSD
creation. Not only support loop device but also others that
disk dev layout is end with numbers.
Change-Id: Iee5f8b8581d70166de6eba1bdc9e42766fe8cb48
Closes-Bug: #1847014
This applies the fix made in kolla-ansible (see [1])
to kolla ceph mon image directly.
[1] https://review.opendev.org/670035
Change-Id: I1822a8c610db72654bbacc7148e25a837090b680
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
The changes in the following commit did not completely solve the
problem of osd initialization failure:
https://review.opendev.org/652612
In order to solve this problem, the partprobe command consistent with
ceph-disk is added, and then a loop is added to detect the partition,
which is acquired every 1s. When the partition appears, the osd
initialization is continued.
Change-Id: I0ca255c6358132d9e3acfa6b610b70a78756512c
Closes-bug: #1824787
The startup script requires to be run as root, not ceph.
All other ceph images follow this rule.
Also removed redundant blank line.
Change-Id: I1d69ec0ca97dbbacccfe737fca03c9739244b900
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
fetch_ceph_keys.py is using insecure sha1 function, this patch adds
sha256 beside sha1 - in order to remove sha1 safely in U release.
Change-Id: I32ba1abd204c9dba4bbe3132b5650b0e391616af
Closes-Bug: #1826327
The CentOS Stein OpenStack distribution depends on the Ceph Nautilus
release, while Kolla currently deploys Luminous on CentOS.
This change switches CentOS and OracleLinux builds to use Ceph Nautilus.
Support for auth UID has been removed in the Nautilus release [1][2], so
we have removed the --set-uid argument from calls to ceph-authtool.
Also ceph-osd bootstrap mode fails when no mon config is provided, so
we added --no-mon-config (mon config is injected later after bootstrap).
Due to ceph-nfs build issues (ceph upstream nfs-ganesha rpms relied on older
version of userspace-rcu, than CentOS Storage SIG packages) we need to move
to ceph upstream rpm repos.
[1] d6def8ba11
[2] http://docs.ceph.com/docs/master/releases/nautilus/
Co-Authored-By: Michal Nasiadka <michal.nasiadka@nokia.com>
Change-Id: I000398f587c5f4d6cc8995e34e162eebc77bc3e3
Implements: blueprint centos-ceph-nautilus
When 4 or 5 mons are deployed, all mons will always be in the election
state, and the ceph cluster cannot be accessed.
After comparing with ceph-deploy, I found that when the cluster was
initialized, it did not use monmap. When adding a new mon node, it
needs to use monmap. According to the above situation, I fixed the
problem.
Change-Id: Idbfc6741d371adcb06ed97d578d75d3bce862743
Closes-bug: #1824783
When deploying osd, if the user does not use the extra block
partition, the kolla will automatically partition the disk and then
clean up the data on the disk partition. Sometimes the disk partition
will not be updated, there will be an error not finding the partition.
This commit fixes the problem.
Change-Id: I14708f38614dcb75268c2f460ae3d921748c2d10
Closes-bug: #1824787
When deploying ceph osd with kolla, each osd deployment will move the
host bucket to default in the crush map.
If we adjust the crush map, we have to re-adjust the crush map after
adding osd or fixing osd.
This commit fixes this problem.
Change-Id: Ifdc3a1fd5fe37da529b2aee9811b12f744cff3bf
Closes-bug: #1821681
This change updates the docker files to use base_package_type instead
of doing specific distro checks for the rhel/deb generic cases. The
base_distro is still available and is used when a specific distro needs
a customization but if the differences are purely rpm vs deb, then the
base_package_type can be used.
Change-Id: I8d720bb185df65a0178061ccf20b1ab2265da2c5
Starting with 4.4.1 release btrfs-tools were renamed to btrfs-progs so
*-tools package became transitional one.
Next stable Debian release will not have 'btrfs-tools' package at all.
Change-Id: I188eaad233c1dbdbfd1c115e02b0e1c66e307c20
The ceph version supported by kolla is now luminous, but the latest
ceph version of ubuntu's apt source is mimic, so there will be
problems with ceph deployment.
This patch solves this problem by specifying the version of the ceph
package installed.
Change-Id: Ibea3498589b6c3c1512e558919c6ea00c95686cb
Closes-bug: #1793667
The current bluestore disk label naming is inconsistent with the
filestore. The filestore naming format is that the disk prefixes
belonging to the same osd are the same and the suffixes are
different.
This patch keeps the bluestore's disk naming as well.
Change-Id: I090cf055ebedc555b5ada35e140b7a7bb2a4cf8f
Enhance the deployment of Kolla-Ceph bluestore OSD.
Deploy bluestore OSD including up to 4 partitions:
* one partition is for bluestore OSD information
* one partition is for bluestore block
* one partition is for bluestore block.wal
* one partition is for bluestore block.db
Deploy bluestore OSD deployment with LOOP devices.
Partially-Implements: blueprint kolla-ceph-bluestore
Change-Id: I00eaa600a5e9ad4c1ebca2eeb523bca3d7a25128
Signed-off-by: tone.zhang <tone.zhang@arm.com>
Support Kolla Ceph to deploy blustore OSDs. With the patch, Kolla
Ceph can deploy bluestore OSDs on ONE, TWO or THREE storage
devices.
Before deploy bluestore OSD, please prepare devices. The detailed
Please refer to [1] for devices initialization.
extend_start.sh: initialize and start bluestore OSD
find_disk.py: search the devices for bluestore OSD
[1]: specs/kolla-ceph-bluestore.rst
Partially-Implements: blueprint kolla-ceph-bluestore
Change-Id: I832f490de63e1aeb68814697cda610a51b622c1f
Signed-off-by: Tone Zhang <tone.zhang@arm.com>
Ubuntu 18.04 has been released. We need upgrade to it this cycle.
Changes made in this patch
- Bump ubuntu repo from xenial to bionic. Some repos do not have bionic
packages, so still using xenial repos.
- Drop mariadb, percona and nfs-ganesha repo, because these packages are
already provided by Ubuntu official repo.
- Since qdrouterd does not have bionic repo and xenial packages does not
work on Ubuntu bionic, just add it into ignore image list.
- Use python-rtslib-fb and targetcli-fb rather than python-rtslib
and targetcli
- use xen-utils package name rather than with package version one.
- Seems unsigned repo won't work on Ubuntu bionic, stop using infra
mirrored unsigned repo in gate building job.
Needed-By: https://review.openstack.org/568728
Implement blueprint: ubuntu-bionic
Change-Id: I4e3b0ca669cfbf6316bf591f2d8428fa1a0d6182
centos based images have wrong label info,
these changes fix own image's name and build-date.
Change-Id: I1d13f8f386c8db12b5fbe5f8ecbbf9e3fbb4ba1c
Closes-Bug: #1680341
'ceph-fuse' package is not available for all architectures.
In Debian/Ubuntu 'ceph-fuse' package is built only for x86_64
architecture. CentOS lacks 'ppc64le' package.
Partially-Implements: blueprint multiarch-and-arm64-containers
Change-Id: I06cea15166bee77dca7b418e85d3a4080892068e
Use LABEL instruction instead of MAINTAINER (deprecated) instruc-
tion as suggested by Docker's official dockerfile guide.
docs.docker.com/engine/reference/builder/#maintainer-deprecated
Closes-Bug: #1683652
Change-Id: Ie87a1ddf31aefcd0b623fd2837d78de420e76898
Debian support is not maintained in Kolla so it got a bit behind Ubuntu
one. This changeset enables Debian for all images. Jessie (even with
backports) may be too old for some images though.
Also unify distro check to ['debian', 'ubuntu'] to keep alphabetical order
like it is done for RPM distributions.
Partially-Implements: blueprint multiarch-and-arm64-containers
Change-Id: I056233fbfa277e0e2360c07c3f80d9558c554357
Some images have packages sorted alphabetically and some not.
Unify common style between all images.
Change-Id: I906ed89c10b12886665618752f525ba71d83d991
include_header and include_footer parameter is already removed, remove
them in all Dockerfiles.
Add missing footer block.
Change-Id: I90da03eb9f95a3827361d5f5ede65fde7d6be2b3
This centralizes all user and group creation into a single source. This
will fix any current and furture uid/gid mismatches (such as with
nova-libvirt).
In the process, we also unify users between the distros in a standard
way. The users in the following containers change from thier defaults:
Ubuntu: _chrony user is now chrony
Ubuntu: memcache user is now memcached
All: qemu user is used for ownership and socket permissions
All uid and gid numbers are customizable via kolla-build.conf
Co-Authored-By: Kris Lindgren <klindgren@godaddy.com>
Change-Id: I120f26ab0683dc87d69727c3df8d4707e52a4543
Partially-Implements: blueprint static-uid-gid
Docker will change the name volume folder permission to image already
set. /var/log/kolla is not created and well configured in image.
This patch set creates and configures proper permission /var/log/kolla
in base image.
This patch set also fixed the ceph image by adding ceph user to kolla
group.
TrivialFix
Change-Id: Ib5ef0187e90de2699b3cda31e819b4babb07e0af
Change needed to add header blocks to all Dockerfiles, similar to the
base.
Use case is to easily run something before packages are installed, e.g.
to COPY a local rpm in that can be added to the package list.
Change-Id: I1bbfdf0b762da0a392aa8bf47781315b45377bee
Closes-Bug: 1618969
This patch changes version of ceph from hammer to jewel. Also removed
versionlock as it seems we don't use it in ubuntu, and actually might be
risky if we miss security patch on ceph.
Change-Id: Ib8f88c2f914a4b635e59a509fa0194605eb73165
Implements: blueprint upgrade-ceph-to-jewel
Currently if the install_packages macro is run with an empty
package list, it will add a yum or apt-get command with no
packages listed.
This bug fix aims to omit this line when no packages have
been given, or, the operator wants to use the "_override" /
"_remove" functionality to disable all packages being
installed in a Dockerfile.
Co-Authored-By: Paul Bourke <paul.bourke@oracle.com>
Change-Id: Ifaaaebfccc3adb0f2f68a35ac08e59378bc87fdb
Closes-bug: 1612446
In most of case, the disks used by ceph have different size. Use the
default value 1 may block the ceph when one disk is full. Use the disk
size as osd weight will more reasonally.
TrivialFix
Change-Id: Ib875c7289188cbb9380355baf0c8048f1eb09332
This allows us to specify external journals for osds which can greatly
improve performance when the external journals are on the solid-state
drives.
The new lookup and startup methods fix the previous races we had
preventing osds from being created properly.
This retains the same functionality as before and is completely
compatible with the previous method and labels, however this does set
new labels for all new bootstrap OSDs. This was due to a limitation
in the length of the name of a GPT partition.
Closes-Bug: #1558853
DocImpact
Partially-Implements: blueprint ceph-improvements
Change-Id: I61fd10cb35c67dabc53bd82270f26909ef51fc38
This was an attempt to get storage_interface to work properly but that
work will not be completed and functional this cycle. There are design
topics that need to be discussed about it that were brought to light
by the RAX gate failing for it.
TrivialFix
Change-Id: I65579f9e0e0dcf3fa51c0ea031ff474145457c40
Michal Nasiadka