Commit Graph

115 Commits

Author SHA1 Message Date
Benjamin Michael Cooper 45eccc4e25 Improve validation error message
If a sub section of a deploy schema is one of multiple options
python-jsonschema, which used to verify the schema, is unable to
determine which option is intended so cannot produce a useful
error message. As "type" is required by all partitions and volumes
this is be used to determine the correct sub schema, which can then
be validated recursively using python-jsonschema. The validation
error message then gives specific details on each incorrect
parameter.

Change-Id: Icb22b88b40f0fa45a3fe3932d69ba32e4a360edd
2017-08-31 16:17:41 +00:00
Benjamin Michael Cooper 7f71449315 Add bootstrap for ramdisk func tests on centos
Use an ansible playbook to configure a centos host so that the
Bareon functional tests can be run.

Change-Id: I846be89759d73ac72bdaa41246315716d44d858f
2017-08-24 16:58:21 +00:00
Jenkins f7a9da7065 Merge "Configurable virtualisation" 2017-08-17 09:00:10 +00:00
Benjamin Michael Cooper 0c72e265f1 Configurable virtualisation
Allows qemu to be specified instead of kvm when hardware
virtualisation is not available.

Change-Id: Ib220c344f4cecc665930d51aa56c7fbfe749b7ac
2017-08-17 08:48:08 +00:00
Benjamin Michael Cooper aae59d92ea Fix directory in centos diskimage builder script
Change-Id: Ibc5404878c204e6927a4eefe3afbece33622fe86
2017-08-17 08:47:04 +00:00
Dmitry Bogun 9cacc64c59 Solve test object naming in functional tests
During upstreaming functional test, some test object names was
updated(or not updated). As result several functional tests eds with
error.

Change-Id: I45699a1b4ea809b620cc96bb3ef274e619840612
2017-03-03 13:10:01 +02:00
Jenkins a15361393e Merge "Register all config options in one place" 2017-03-02 10:25:29 +00:00
Jenkins 245cbf02a6 Merge "Improve ironic-callback" 2017-03-02 10:22:08 +00:00
Jenkins ef5b8fe977 Merge "Make ssh-key optional" 2017-03-01 15:46:14 +00:00
Jenkins 9d2c7dbb0c Merge "Rename GDisk objects family" 2017-03-01 15:45:36 +00:00
Dmitry Bogun 1f33e227f8 Improve ironic-callback
Add support to new bareon-ironic communication "protocol" - extension
for vendor passthru API. This protocol allow to receive generic
tasks(steps) from bareon-ironic driver, process them on bareon "side"
and send back results.

Right now only one step is implemented - step to inject SSH key.

Change-Id: I9ea828b24085fa72df470eef41ad32d9096f6b40
2017-03-01 16:03:28 +02:00
Dmitry Bogun 4b734a3673 Make ssh-key optional
Bareon is capable to fetch ssh-key from ironic conductor, after start.
So we can avoid preinstalled ssh-key in bareon image. But ssh-key is
extremelly helpfull for debugging. This change switch ssk-key from
mandatory to optional.

It also add "hook" for injecting root passwords inside bareon-image. It
can/should be used for debugging too. (In case if you will have access
to node's console).

Change-Id: Icb27d40c9872703d9b01a8c7705ad7b185277e6b
2017-03-01 12:43:01 +02:00
Dmitry Bogun 937eb632e5 Register all config options in one place
Make a single point responsible for config option registration to
increase control over config initialization and avoid conflicts.

Usually such conflicts become visible when sharing code with externa
callers. In our case it was the deployment config validator (part of
data drivers), exported as part of the ironic data driver via the
setuptools entry point.

Change-Id: Ibba18db61c222d910d8dca8866ea74b14ce011c8
2017-02-28 13:12:47 +00:00
Jenkins 62b05760ef Merge "Add docstting for bareon.utils.block_device.Disk" 2017-02-24 16:05:09 +00:00
Dmitry Bogun 4ee9bbe3e5 Rename GDisk objects family
Rename GDisk int SGDisk, to have a "direct" match with wrapped tool. All
"satalite" objects acquire "s" preffix too.

Change-Id: I59847bb84a288073358560c58885f925d8589f04
2017-02-24 15:58:25 +02:00
Dmitry Bogun 1872b734bd Certain subset of hardware may lead to failue
The old way to detect SCSI address of block devices rely on incorrect
assumption. It rely that folder
/sys/class/scsi_device/{scsi_address}/device/block exist for any SCSI
device. And because this assumption is incorrect, if fails with
unhandled exception, on such hardware.

Rewrite core responsible for detection SCSI address of block devices.

Change-Id: I3c9938e145fc8c6d3036fdb769fd20f8b8f63678
2017-02-22 16:54:49 +02:00
Dmitry Bogun 2463be48b0 Add docstting for bareon.utils.block_device.Disk
Describe destination and used "methods" into Disk class.

Change-Id: I1e7e7ce1ee71556216dbef71a08686e7280cbd84
2017-02-14 17:14:46 +02:00
Dmitry Bogun 726e510897 Improve spelling
Change-Id: I61a9544f9b6ca41c029eda84f81191b7dca22468
2017-02-08 16:14:26 +02:00
Dmitry Bogun 0f1d1e7f22 Actualize comment messages in fucntional tests
Some comments are outdated - don't represent code actions. Some comments
should be replaced with docstrings. Some should be reworded.

This change is going to fix all this "comment" issues.

Change-Id: Ie518a2f25dbc9c27b0a7db35a7533bde8dec278f
2017-02-08 13:56:29 +02:00
Dmitry Bogun 1127e89243 Add additional functional tests
These are a set of functional tests developed to test known issues and
patches for those issues. It's better to keep these tests public, in
case someone else faces functional degradation of bareon code.

Change-Id: I17899f115df7f81786681ddb97db18cf2a01689a
2017-02-08 13:37:43 +02:00
Jenkins af8929b990 Merge "Avoid modification into global objects" 2017-02-07 14:51:39 +00:00
Dmitry Bogun ec374ed13b Avoid modification into global objects
Now it is not an issue. But it can become issue in any time. "hidden"
modification of global objects is a time bomb.

Change-Id: I4e9d9984cb5d0af3b03b6436ddb7c7849703acbb
2017-02-07 16:02:03 +02:00
Jenkins baf3a85477 Merge "Avoid persistent storage modification by request" 2017-02-07 13:47:31 +00:00
Dmitry Bogun 8504a657ac Avoid persistent storage modification by request
Ironic driver support "check and live as is" partitioning operation. It
is used to in "redeploy" node after hardware failure. Ususally some
files are changes since time when node was deployed. And usually we need
to keep this files "as is" too.

The problem appers in "do_install_os" deployment step. It revert all
files to state stored into deploing image.

This update disable "do_install_os" when "check and live as is"
partitioning operation is requested.

Change-Id: I978059ec28444389e5a1b18ae0a20af0e1a71a45
2017-02-06 16:39:56 +02:00
Dmitry Bogun 6d2dc82a70 Use cloudinit to put ssh-key onto deployed image
Change-Id: I23ae8a19cfc89a6eba1c52cd13d97d24273ae1b5
2017-01-26 15:14:59 +02:00
Dmitry Bogun 8ebb6bdfea Remove old naming evidence
Change-Id: Ie546d9d976c061c567302493bc2d6487a48a2557
2017-01-26 15:06:32 +02:00
Jenkins 09fba8dca8 Merge "Rename kernel parameters expected by callback" 2017-01-23 15:15:27 +00:00
Jenkins bd478a8db9 Merge "Remmoved unnecessary utf-8 encoding" 2017-01-23 15:07:59 +00:00
Andrei V. Ostapenko 901f96866f Rename kernel parameters expected by callback
Change-Id: I1e754de2aec89e6d6140792565791c1b523ec344
2017-01-20 20:52:31 +00:00
Dmitry Bogun 5104e6ae5f Correct handling of "remaining" size
Deprecated storage scheme didn't take into account "remaining" size
claims. As and percentage sizes too. This change make it capable to
handle suze size "units".

But it calculate size based on disk size received from caller. Same as
it was done in old storage scheme.

Change-Id: I4d6a59065a1546163615ac4b65411898b942afe2
2017-01-20 14:13:37 +02:00
Rajiv Kumar acdfab941b Remmoved unnecessary utf-8 encoding
Change-Id: Idd1aeb81d1012a63c530f9571e72bbf2dba0c522
2017-01-10 17:33:47 +05:30
Dmitry Bogun 525051d0bc Fix "provisioning" functional tests
"Provisions" family of bunctional tests depend of "hardcoded"
SSH-key. This fix removes this dependency.

Change-Id: I2c7a1ef8ebd57a3b402bb1534a6f01221a8cd32b
2017-01-10 09:19:15 +02:00
Dmitry Bogun 477933a85d Adopt ramdisk-func-test changes
Change https://review.openstack.org/415709 changes environment object
API. This patch adopts our tests to this changes.

Change-Id: I2fb3e0d03ea8b10013fd648f51bc7b4526071603
2017-01-10 09:18:45 +02:00
Dmitry Bogun 355667dc2e Restore missing argument in logging call
Change-Id: Idb44fac8961a4bded4ced48d2a1052bacc8d8447
2017-01-10 09:18:45 +02:00
Dmitry Bogun 0139a010e3 Fix code incompotable with python2.7
Change-Id: I10c3ccd7572b10e4f87232832743ff52ee5e6e3d
2017-01-10 09:18:45 +02:00
Dmitry Bogun c2d78e6b73 Fix "config object already initialized" error
Change-Id: I865e082215dbdb231e0ef968e76b4925b776da2a
2017-01-10 09:18:45 +02:00
Dmitry Bogun 328dd6cd0c Convert block device size into nailgun data driver
Now all partition related tools expect "size" in bytest(not in MiB) or
instances of SizeUnit object. Nailgun was last data driver that
represented "size" of block devices in MiBs.

Change-Id: I03170969407d33a8921fc0b4b32739d1865c10e0
2017-01-10 09:18:45 +02:00
Dmitry Bogun 3dc8f1c411 Avoid huge metadata reservations in LVMpv
By some reason, pvcreate method allocates two 64MiB metadata chunks by
default.  We can/should make test how many vg/lv can be allocated using
default two 2MiB metadata chunks(this LVM defaults not our). But I
believe that it will be enough for any reasonable amount ov vg/lv.

We still can pass required metadata size, if we need to.

Update functional tests expected constants to match metadata size
update.

Change-Id: I0106fa35b7b21f16ce6a9a8794e6c88ba61ceeca
2017-01-10 09:18:45 +02:00
Dmitry Bogun 0d52a375cc Use new partitioning tools and remove unused partition code
Update data and deploy drivers to enable usage of new objects model and
new partitioning tools.
Remove a lot of old partition code, become unused/never reached code.

Change-Id: I9f01304632be28626abaf8fb322dd4fda0017589
2017-01-10 09:18:39 +02:00
Dmitry Bogun 9dfef0641a Property to attribute
Convert several properties into class level attribute in data-drivers
class hierarchy.

It decreases amount of code and makes workflow more obvious.

Change-Id: Iba6ec79ab4266acbcc90133356953c8aee7180fc
2017-01-09 22:42:09 +02:00
Dmitry Bogun a70b832447 Implement new partioning mechanism
This mechanism is capable to work with new objects model and new block
device utils.

Change-Id: I16cb7ec25aa4c6a6fabffc047f50a5758bb19c06
2017-01-09 22:42:09 +02:00
Dmitry Bogun 2bd7569719 Some cleanup into irnonic json schema
Sort field, change formating to increase readability.
No definitions were changed.

Change-Id: I884edc8172fb3d6b212b66b5c7419c390e919147
2017-01-09 22:42:09 +02:00
Dmitry Bogun 9ab003ca02 Parser for old data model
Builds old data model using data stored in new data model. It is
required to keep at least part of existing API of data-driver.

We can't use existing code to build old data model, because it conatins
at least 2 serious mistake(located during code analisys while creating
parser for new object model). It is also tangled and and wry.

Change-Id: Ic2418a1c2a6334aafb2048fd057d35ca7945c31e
2017-01-09 22:42:09 +02:00
Dmitry Bogun 1786de8c21 Parser for new object model
Parser that processes deployement config and produces "storage claim".

Change-Id: I8a250f911144776f2cf7ae9b561095cbcd067637
2017-01-09 22:42:06 +02:00
Dmitry Bogun 36f518cc3b New data model to represent storage subsystem
Change-Id: I194cf6bd3e5bfd7b8747326c1ffb3433e614474e
2017-01-06 12:05:15 +02:00
Dmitry Bogun 75cc77a588 Teach SpaceClaim to claim space
Change-Id: I47aeaf5a514bb26105de44f83bed1c86eda50758
2017-01-06 12:00:29 +02:00
Dmitry Bogun 2f48694c8d Add object to plan LVM vg partition
Change-Id: I8fe41d5472df9b30a5f204985dd25ab6971ddc56
2017-01-06 12:00:07 +02:00
Dmitry Bogun c3b441e01b Improve API for disk partition planning
Extract some portion of "Disk" objects into abstract class. Later it will
be used as base class for other kinds of storages(at least for LVM).

Change-Id: I3fa9cb230f80a6d7bb3285700d12e6f88a9018c7
2017-01-06 11:57:56 +02:00
Dmitry Bogun 31486e41d4 Add SpaceClaim object
We need to "transfer" non numeric sizes from deployment config to
deployment driver. Now we have only one such "size" - this is
"ramaining" constant.

Also this object knows(will know), how to pass this claim to the storage
object.

Change-Id: I1956a0fcf60bfa5c0c3eda7ecb7c3dda176880ca
2017-01-06 11:51:53 +02:00
Dmitry Bogun 9abb3c5d80 Add DeviceFinder object
This object keeps info about existing(on object creation time) block
devices and provice API to lookup these block devices. Lookup support
of all schemas used in deployment config to define block devices.

Change-Id: I15b3345733f09b04c6656c3cabe75fbfab85acb5
2017-01-06 11:50:15 +02:00