Commit Graph

194 Commits

Author SHA1 Message Date
OpenDev Sysadmins 79f1ad40b2 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:34:28 +00:00
Alexander Gordeev 57145b1d88 fs utils: force xfs creation with '-f' option
mkfs.xfs will not write to the device if it suspects that there is a
filesystem or partition table on the device already.

'-f' option should added to force overwriting when an existing
filesystem is detected on the device.

Change-Id: I6dabe592629d017de3a7e775c92f819968b9ab64
Closes-Bug: #1482210
2015-08-06 18:01:59 +03:00
Jenkins 1fe47720ba Merge "Always take into account config file values" 2015-07-31 14:13:22 +00:00
Jenkins b27076b8d8 Merge "fuel-agent: use different md metadata version" 2015-07-30 18:00:14 +00:00
Alexander Gordeev 753a87c4bb fuel-agent: use different md metadata version
0.90 will be used for centos only letting GRUB-legacy to boot.
Otherwise, default (1.2) will be used as GRUB2 is able to deal with it.

Change-Id: I89c5178d991970de4682c1e239e0ecb014ba61bd
Closes-Bug: #1430432
2015-07-30 15:35:58 +03:00
Alexander Gordeev fab5b20f46 Always take into account config file values
Some methods were declared with the default parameters specified as
CONF.<foo> values. Those values will be evaluated only once during
module build time. So, the values speficied in config file won't be
passed into those methods as config file values will be available
only after config file gets parsed in run-time.

Change-Id: Ic8e24be4fdc0ce165d8142d47405373acaf4a803
Closes-Bug: #1479307
2015-07-30 14:43:59 +03:00
Alexander Gordeev dee9f2eb7e Move udev rules blacklisting to utils layer
Also add udevadm_settle() to utils

Change-Id: I9bce65fa545c93b3f9b556171c9030baa3795b3e
Closes-Bug: #1477537
2015-07-29 21:02:45 +03:00
Alexander Gordeev 355c08a049 Do not land /boot on NVMe disks
There's no point to place /boot partition on NVMe disk
as grub2 without UEFI support is unable to recognize it and
read any data from it.

Unfortunately, fuel-agent doesn't work with grub2-efi.
It always installs BIOS version of grub2.

It's just a work around that will allow to land /boot partition
on non-NVMe disk.

The absence of non-NVMe disks on a node will cause a failure of
provisioning.

DocImpact

Change-Id: I166c94416ccb152ccd8d1dc780dfb21a774a4f1d
Co-Authored-By: Atze de Vries <atze.devries@naturalis.nl>
Related-Bug: #1474970
2015-07-28 12:05:13 +03:00
Jenkins 2a65f11c10 Merge "Change method signature for mdcreate from md utils" 2015-07-23 12:57:10 +00:00
Jenkins bc25d3b728 Merge "fix 'udevd: can not read' warnings" 2015-07-22 15:31:07 +00:00
Alexei Sheplyakov 76f00d90f4 Add cloud init templates to Debian package
fuel-agent needs those for provisioning to work properly.

blueprint fuel-bootstrap-on-ubuntu

Change-Id: I2fce1a4b1f485bf509bfe2ea81291ceebf57e4ef
2015-07-22 01:24:59 +03:00
Alexander Gordeev 24af224618 Change method signature for mdcreate from md utils
Ditch 'device' and '*args' from its signature and
substitute them with 'devices' list.

Change-Id: I99a90c26c9a9b36ed4bfabbc7f882e4031d52899
Closes-Bug: #1476621
2015-07-21 20:06:00 +03:00
Alexander Gordeev 81ff5ad15d fix 'udevd: can not read' warnings
Those warnings appear when fuel-agent performs udev rules
blacklisting.

Adding a syncing with udev via 'udevadm settle' to the lines which
actually manipulate with these rules, totally eliminates those scary
warnings.

Change-Id: I9e528f948f83d97e9cf084e3b2c3250eef518c5e
Closes-Bug: #1422717
2015-07-21 19:12:59 +03:00
Alexei Sheplyakov fd16b972dd Build fuel-agent for Ubuntu and Debian
Necessary for making an Ubuntu/Debian bootstrap node.

blueprint fuel-bootstrap-on-ubuntu

Change-Id: Ifc799a77eb2b32e5582e13a5aa4ff1d6bfbd8f77
2015-07-21 13:47:35 +03:00
Maciej Kwiek 080e612218 Pin oslo.config version to 1.15 or less
CI broke after releasing oslo.config v2.0.0.

Change-Id: I3d857858201b59c414603d08d027769aada529e0
2015-07-21 11:52:09 +02:00
Jenkins 4e9fe5563e Merge "Load nf_conntrack_proto_gre kernel module default" 2015-07-20 12:53:47 +00:00
Jenkins ad0b102fca Merge "fuel-agent: check if lvcreate supports --yes switch" 2015-07-20 12:17:24 +00:00
Sam Stoelinga cdb13c5dde Load nf_conntrack_proto_gre kernel module default
Add nf_conntrack_proto_gre to the default loaded modules
for CentOS and Ubuntu. This enables VMs on OpenStack
that use GRE to provide GRE connectivity over floating
IPs.

Closes-bug: #1474210

Change-Id: I265a30d698396bcce64b139dfa8696a440cc2a3b
2015-07-20 14:40:43 +08:00
Jenkins 8ef6b9810f Merge "IBP: add do_bootloader unit tests for fuel-agent" 2015-07-17 15:01:23 +00:00
Jenkins 5fb8c971de Merge "All tests in fuel_agent use unittest2 now" 2015-07-17 13:47:27 +00:00
Jenkins 7629d78abd Merge "Split objects/partition.py into smaller files" 2015-07-17 12:55:47 +00:00
Jenkins 0bd81d37c3 Merge "fuel_agent: image build directory can be passed via command line" 2015-07-17 11:47:49 +00:00
Alexander Gordeev 548a94e4c9 FA: add xz, coreutils for fuel-agent spec
Those packages come from eliminated fuel-image package.

Change-Id: Id4421b27755bae0c6f5d7f2c1d807d734747e98e
Closes-Bug: #1475335
2015-07-16 18:03:33 +03:00
Alexander Gordeev 07f4346604 FA: add debootstrap package to spec
Change-Id: I5588daee9eecf5da257c2c1f3c8257d30a8d013d
Closes-Bug: #1475335
2015-07-16 17:36:08 +03:00
Sebastian Kalinowski 27d3501030 Split objects/partition.py into smaller files
Change-Id: I9bc3716b596c426ba4ff3f9348ecf1f0c5116e70
Implements: blueprint volume-manager-refactoring
2015-07-16 15:18:37 +02:00
Alexander Gordeev dd37eac342 IBP: add do_bootloader unit tests for fuel-agent
First portion of its tests was introduced by
https://review.openstack.org/#/c/201639/

Change-Id: I69d5731a6fe3c1979f89fece660fab07bc8c6e09
Closes-Bug: #1449609
2015-07-16 16:16:18 +03:00
Maciej Kwiek ff0fbf5810 All tests in fuel_agent use unittest2 now
unittest2 backports some nice features from newer versions of unittest
for python 2.7 (like assertRaises context manager). Moreover tests now
are using either oslotest or testtools directly, switching to one test
framework is going to simplify tests.

Change-Id: Ibce9403b20958f9db2a59db9c6b7479a4fc57857
Implements blueprint: volume-manager-refactoring
2015-07-16 12:42:52 +02:00
Jenkins 22b566f916 Merge "Add processing of keep_data flag" 2015-07-16 10:24:29 +00:00
Jenkins 180dda07a2 Merge "Add simple data driver for partitioning info" 2015-07-16 10:21:10 +00:00
Artur Svechnikov 68494fb0dd Add processing of keep_data flag
In case when some partition has keep_data = True
need to elevate flag to file system level and
do not to create new partition table. Do not clean
file system when it has keep_data = True.
There should not be a problem if some raw data
will not be cleaned.

Change-Id: Ifccc9256b12944283219cffeedb8c8da8ab420f8
Implements: blueprint partition-preservation
2015-07-16 12:53:45 +03:00
Sebastian Kalinowski 9e22866f3e Add simple data driver for partitioning info
This new "simple" data driver takes serialized partitioning info
that is provided before provisioning from external tool.
It does not make any calculations for that data and expect it to have
all required informations about partitioning.
Other changes:
* added unittest2 to use some of it features
* added requests_mock to mock http requests
* added objects conversion to/from dictionary to make serialization
  easier
* added a common "interface" for data drivers
* fixed test for "do_build_image" - now a correct data driver is used

Change-Id: I673cde6f0ead9945919a87cd1cfce7ed09c6e593
Implements: blueprint volume-manager-refactoring
2015-07-16 10:11:02 +02:00
Alexander Gordeev 9061c2bff4 fuel-agent: do not use lazy umount
Using "umount -l" is a potentially dangerous act as
fuel-agent doesn't wait until any process stops using files
on that mounted fs.

Use of lazy umount should be disabled by default.
In case if regular umount failed,
it means abnormal situation where manual intervention is needed as
lazy umount couldn't sort out the issue by itself.

Change-Id: If258535c43402142a8dcad0c87bc519fcc6904d2
Closes-Bug: #1454366
2015-07-15 14:23:09 +03:00
Jenkins ce4bfd0037 Merge "Add errors=panic to rootfs" 2015-07-15 10:52:34 +00:00
Jenkins d35f7d3172 Merge "fuel_agent: unit tests for umount_target, mount_target" 2015-07-15 10:48:33 +00:00
Alexei Sheplyakov da8709891e fuel-agent: check if lvcreate supports --yes switch
lvcreate --yes switch has been added in the recent versions of lvm2
which are not available on Ubuntu 14.04.

blueprint fuel-bootstrap-on-ubuntu

Change-Id: Iedef0131176c42f51b8254f3561b084e6f294cad
2015-07-15 11:33:40 +03:00
Alexei Sheplyakov d28f340a7c fuel_agent: image build directory can be passed via command line
Make it possible to set the image build directory via the command
line. Also create the image build directory if it doesn't exist.

The default image build directory (/tmp) can be too slow/small.
In particular if fuel_agent runs in a container (which is the case
on the Fuel master node) /tmp resides in the container filesystem
which is backed by dm-snapshot. dm-snapshot is known to have a poor
write performance, especially for large(er) files like OS images.
By default docker storage is a loopback file residing in
/var/lib/docker. Thus the data written to the OS image being built
goes through a number of layers:

ext4 -> loop -> dm-snapshot (/var/lib/docker-> loop ->
     -> ext4 (/var/lib/docker) -> dm-linear (os-varlibdocker) ->
     -> hard drive

which causes substantial IO overhead. If the master node runs on a VM
rather than a physcial hardware the above chain gets even more
complicated (and write performance gets really afwul):

ext4 -> loop -> dm-snapshot (/var/lib/docker-> loop ->
     -> ext4 (/var/lib/docker) -> dm-linear (os-varlibdocker) ->
     -> virtual hard drive (qemu) -> qcow2 -> ext4 (host)

Using a host volume (mounted at /var/lib/fuel/ibp) as an image build
directory eliminates the extra layers so the written data goes through

ext4 -> loop -> ext4 -> dm-linear (os-var) -> hard drive

so the IBP image build time gets reasonable.

Related-Bug: #1457643
Change-Id: Idcafd23e659fcc7c0d7e6087501799834234b34c
2015-07-15 09:48:41 +03:00
Oleksiy Molchanov fca0ea6535 Add errors=panic to rootfs
* Add errors=panic to rootfs in order to panic and
not to go to 'ro' when there are any errors on the disk
* Set parameter kernel.panic in sysctl to 60 (By default,
the kernel will not reboot after a panic, but this option
will cause a kernel reboot after 60 seconds.)

Doc-Impact

Change-Id: Ib24f13d3cbbf792e7ee81a9b4054e084f4ec1b5e
Closes-Bug: 1371689
2015-07-14 18:49:03 +03:00
Alexander Gordeev e005f7b41a fuel_agent: unit tests for umount_target, mount_target
Change-Id: I7f3d7fa0aa1b95afbc0cd7a4580704e8cb9490f8
Closes-Bug: #1445400
2015-07-14 16:33:33 +03:00
Alexander Gordeev 3c02dbf2e9 IBP: add more tests for fuel_agent.utils.grub
* improve test coverage;
* add exception raising for methods which should return something;

Change-Id: I4aa7ee05e37c8a7935d8759875f76f98d30388fa
Closes-Bug: #1452158
2015-07-14 16:21:47 +03:00
Vladimir Kozhukalov 21f4b55f16 Added infrastructure files
* .gitreview
* .gitignore
* run_tests.sh
* specs/fuel-agent.spec

Change-Id: Id4f3ca3f44dbb40bd8da8a51f76d42459ea61c85
2015-07-14 14:54:56 +03:00
Sebastian Kalinowski 8ebe1f0c54 Pin mock to version supporting Python 2.6
Change-Id: Ic4cfd65dc6e7ada2adc847ec4c2b1a1cfcb9627a
Partial-Bug: #1473294
2015-07-10 08:33:07 +02:00
Jenkins c14c67a1d4 Merge "Remove pbr import from fuel_agent's setup.py" 2015-07-10 06:09:15 +00:00
Vladimir Kozhukalov 0f46753eed FA: fixed oslo config autogeneration
CONF initialization is moved into method.
Autogeneration script does not work when
CONF is initialized in global scope.

Closes-Bug: #1472576
Change-Id: I7810d0befb0f8006592299424af1b773066785e3
2015-07-08 14:35:48 +03:00
Sebastian Kalinowski 5e552e4b0a Remove pbr import from fuel_agent's setup.py
Importing pbr in setup.py prevents installing
fuel_agent into clean environment (without pbr).

Change-Id: I2734b3cd97ddde7d243b71c3a7112e2519b7b7e2
Closes-Bug: #1472082
2015-07-07 14:48:59 +02:00
Vladimir Kozhukalov 06c936d5ad FA: oslo.serialization -> oslo_serialization
The issue is that oslo log module tries to
import oslo.serialization which is depricated.
Correct way to import this is to use
oslo_serialization module name.

The correct way how to fix this would be
installing oslo_log module instead of using
log module from oslo-incubator. But this way
is going to take long time because there are
no corresponding rpm and deb packages available.

TODO: use oslo_log instead of log module from oslo-incubator
when rpm and deb packages are available

Change-Id: I866fa0e19c5b872a82af6739d3537c24916bccc1
Closes-Bug: #1472018
2015-07-07 12:54:58 +03:00
Jenkins fe00c365d1 Merge "IBP: disable ext4 journaling during the image building" 2015-07-02 12:01:57 +00:00
Jenkins 264895cdc8 Merge "Change testrunner to py.test" 2015-06-30 08:46:19 +00:00
Jenkins 2ba3dab7af Merge "Rename all "utils/*_utils.py"" 2015-06-30 08:46:06 +00:00
Jenkins 2f8de8cc6a Merge "Upgrade hacking rules and fix new issues" 2015-06-30 08:43:52 +00:00
Alexander Gordeev 69d5d9780d IBP: disable ext4 journaling during the image building
In order to speed-up image building process, fuel-agent will
turn off journalling. That should noticeably reduce the cost of
fsync calls. Expected speed-up is about 2x-3x.

Additionaly this change will help image building job to fit within
3600 secs timeout.

Change-Id: I24fcc8ba16c092b1bc936928890717c6e9501aa7
Related-Bug: #1441679
2015-06-29 17:15:24 +03:00