nova/nova/tests/unit
Lee Yarwood caca71376f libvirt: Report the allocated size of preallocated file based disks
At present the Libvirt driver can preallocate file based disks using the
fallocate command and importantly the `-n` option. This option allocates
blocks on the filesystem past the initial EOF of a given file:

```
$ touch test.img ; fallocate -n -l $(( 1024 * 1024 )) test.img
$ ll -lah test.img
-rw-rw-r--. 1 stack stack 0 Apr 16 13:28 test.img
$ du -h test.img
1.0M	test.img
```

This results in a miscalculation of the total disk overcommit for file
based (excluding ploop) disks as os.path.getsize is currently used to
determine the allocated size of these disks:

```
>>> import os
>>> os.path.getsize('test.img')
0
```

Using the above example the disk overcommit would be reported as 1.0M as
the disk appears empty yet will report a potential (virtual) size of 1.0M.

However as the required blocks have already been allocated on the
filesystem the host will report disk_available_least as missing an
additional 1.0M, essentially doubling the allocation for each disk.

To correct this the allocated size of file based (excluding ploop) disks
is reported using `disk_size` from the `qemu-img info` command. This
should ensure blocks allocated past the EOF of the file are taken into
account and correctly reported as allocated.

A future change should ultimately remove the use of the `-n` option with
fallocate, however as this would not help disks that have already been
allocated this has not been included in this change to simplify backports.

Conflicts:
        nova/tests/unit/virt/libvirt/test_driver.py

NOTE(lyarwood): I11e329ac5f5fe4b9819fefbcc32ff1ee504fc58b made
get_domain private in Queens.

Change-Id: If642e51a4e186833349a8e30b04224a3687f5594
Closes-bug: #1764489
(cherry picked from commit 23bd8f6263)
(cherry picked from commit 2d50f6e785)
(cherry picked from commit d88b75e81e)
2018-04-17 23:52:23 +01:00
..
api Merge "fix nova accepting invalid availability zone name with ':'" into stable/ocata 2017-12-13 19:27:51 +00:00
api_samples_test_base Fix invalid import order 2016-07-04 16:03:42 +05:30
cells Remove straggling use of main db flavors in cellsv1 code 2017-02-11 18:54:48 +00:00
cert conf: remove deprecated cert_topic option 2016-11-23 10:53:36 -06:00
cmd Cleanup some issues with CONF.placement.os_interface 2017-02-15 09:26:48 -05:00
compute Raise MarkerNotFound if BuildRequestList.get_by_filters doesn't find marker 2018-01-03 12:48:04 -05:00
conductor Merge "Don't try to delete build request during a reschedule" into stable/ocata 2018-03-23 02:11:42 +00:00
console Merge "tests: Replace use of CONF with monkey patching" 2017-01-24 12:03:07 +00:00
consoleauth conf: Move consoleauth options to a group 2016-10-07 13:43:43 +01:00
db Fix joins in instance_get_all_by_host 2017-10-30 21:34:03 -04:00
fake_loadables
image Set size/status during image create with FakeImageService 2017-04-06 21:33:26 -04:00
keymgr
monkey_patch_example
network Handle exception on adding secgroup 2017-11-14 18:48:01 +00:00
notifications Merge "Update notification for flavor" 2017-01-11 21:58:06 +00:00
objects Lazy-load instance attributes with read_deleted=yes 2018-02-19 16:38:08 -05:00
pci Merge "Changed NUMACell to InstanceNUMACell in test_stats.py" 2017-01-04 17:43:54 +00:00
scheduler Fix an error in _get_host_states when deleting a compute node 2017-12-21 11:15:22 -05:00
servicegroup Don't use 'updated_at' to check service's status 2016-12-08 09:51:03 +00:00
ssl_cert
virt libvirt: Report the allocated size of preallocated file based disks 2018-04-17 23:52:23 +01:00
volume libvirt: Re-initialise volumes, encryptors, and vifs on hard reboot 2018-01-09 01:09:59 +00:00
README.rst
__init__.py
cast_as_call.py
conf_fixture.py Block starting compute unless placement conf is provided 2017-01-27 16:35:09 -08:00
fake_block_device.py Replace uuid4() with uuidsentinel 2016-10-05 18:27:50 +05:30
fake_build_request.py Create Instance from BuildRequest if not in a cell 2016-08-22 16:52:53 -04:00
fake_console_auth_token.py
fake_crypto.py
fake_flavor.py
fake_hosts.py
fake_instance.py Replace uuid4() with uuidsentinel 2016-11-29 11:49:20 +05:30
fake_ldap.py
fake_network.py network id is uuid instead of id 2016-12-09 16:48:55 +08:00
fake_network_cache_model.py
fake_notifier.py support polling free notification testing 2016-12-05 16:16:17 +01:00
fake_pci_device_pools.py
fake_policy.py Separate CRUD policy for server_groups 2016-11-21 11:43:13 -05:00
fake_processutils.py Merge "Stop using mox in unit/fake_processutils.py" 2016-10-14 16:24:17 +00:00
fake_request_spec.py Don't persist could-be-stale InstanceGroup fields in RequestSpec 2018-01-03 10:39:30 +00:00
fake_server_actions.py
fake_volume.py Replace uuid4() with uuidsentinel 2016-11-29 11:49:24 +05:30
fake_xvp_console_proxy.py tests: Replace use of CONF with monkey patching 2017-01-06 14:54:47 +00:00
image_fixtures.py
matchers.py [PY3] byte/string conversions and enable PY3 test 2016-11-17 14:50:35 +08:00
policy_fixture.py Separate CRUD policy for server_groups 2016-11-21 11:43:13 -05:00
test_api_validation.py Add query parameters white list for server list/detail 2017-01-17 15:54:15 +00:00
test_availability_zones.py Avoid lazy-load error when getting instance AZ 2017-05-31 16:00:42 +00:00
test_baserpc.py conf: Remove deprecated service manager opts 2016-12-05 23:28:49 +08:00
test_block_device.py
test_cache.py
test_cinder.py Fix ksa mocking in test_cinderclient_unsupported_v1 2017-01-05 19:52:10 -05:00
test_conf.py conf: Remove 'virt' file 2016-12-21 16:45:08 +00:00
test_configdrive2.py Fixes python 3 unit tests 2016-09-23 20:10:24 +00:00
test_context.py Cache database and message queue connection objects 2017-05-19 21:54:25 +00:00
test_crypto.py Replace uuid4() with uuidsentinel 2016-11-29 11:49:20 +05:30
test_exception.py do not include context to exception notification 2017-03-17 13:00:21 -04:00
test_fixtures.py Add a PlacementFixture 2017-01-20 17:41:37 +01:00
test_flavors.py Merge "Move some flavor fakes closer to where they are being used" 2016-10-18 12:13:40 +00:00
test_hacking.py hacking: Use uuidutils or uuidsentinel to generate UUID 2016-11-29 11:49:24 +05:30
test_hash_ring.py Ironic: allow multiple compute services 2016-08-04 23:51:13 +00:00
test_hooks.py
test_instance_types_extra_specs.py objects: Move 'arch' to 'fields.Architecture' 2016-11-25 16:19:41 +00:00
test_iptables_network.py
test_ipv6.py
test_loadables.py
test_matchers.py Port test_matchers.TestDictMatches.test__str__ to Python 3 2016-12-20 11:11:44 +08:00
test_metadata.py Handle keypair not found from metadata server using cells 2017-09-15 20:51:29 +01:00
test_notifications.py conf: added notifications group 2017-01-03 14:38:57 +00:00
test_notifier.py conf: added notifications group 2017-01-03 14:38:57 +00:00
test_nova_manage.py Add 'delete_host' command in 'nova-manage cell_v2' 2018-02-13 00:44:41 +00:00
test_pipelib.py conf: Move cloudpipe options to a group 2016-08-22 09:31:03 +01:00
test_policy.py Separate CRUD policy for server_groups 2016-11-21 11:43:13 -05:00
test_profiler.py Integrate OSProfiler and Nova 2017-01-18 15:00:14 +07:00
test_quota.py Move quota options to a config group. 2017-01-04 22:57:14 -06:00
test_rpc.py Integrate OSProfiler and Nova 2017-01-18 15:00:14 +07:00
test_safeutils.py
test_service.py Merge "test: drop unused config option fake_manager" 2016-12-12 08:41:03 +00:00
test_service_auth.py Add service_token for nova-cinder interaction 2017-01-11 15:49:16 +00:00
test_signature_utils.py Fix invalid import order 2016-07-04 16:24:10 +05:30
test_test.py Cleanup before removal of conductor local apis 2016-10-18 14:26:06 +02:00
test_test_utils.py test:Remove unused method _test_get_test_network_info 2016-09-25 16:19:29 +08:00
test_utils.py Use six.text_type() when logging Instance object 2017-05-22 10:57:28 +00:00
test_uuid_sentinels.py Fix invalid import order 2016-07-04 16:24:10 +05:30
test_versions.py
test_weights.py
test_wsgi.py
utils.py Fix missing instance.delete notification 2017-03-13 15:02:06 -04:00

README.rst

OpenStack Nova Testing Infrastructure

This README file attempts to provide current and prospective contributors with everything they need to know in order to start creating unit tests for nova.

Note: the content for the rest of this file will be added as the work items in the following blueprint are completed: https://blueprints.launchpad.net/nova/+spec/consolidate-testing-infrastructure

Test Types: Unit vs. Functional vs. Integration

TBD

Writing Unit Tests

TBD

Using Fakes

TBD

test.TestCase

The TestCase class from nova.test (generally imported as test) will automatically manage self.stubs using the stubout module and self.mox using the mox module during the setUp step. They will automatically verify and clean up during the tearDown step.

If using test.TestCase, calling the super class setUp is required and calling the super class tearDown is required to be last if tearDown is overridden.

Writing Functional Tests

TBD

Writing Integration Tests

TBD

Tests and Exceptions

A properly written test asserts that particular behavior occurs. This can be a success condition or a failure condition, including an exception. When asserting that a particular exception is raised, the most specific exception possible should be used.

In particular, testing for Exception being raised is almost always a mistake since it will match (almost) every exception, even those unrelated to the exception intended to be tested.

This applies to catching exceptions manually with a try/except block, or using assertRaises().

Example:

self.assertRaises(exception.InstanceNotFound, db.instance_get_by_uuid,
                  elevated, instance_uuid)

If a stubbed function/method needs a generic exception for testing purposes, test.TestingException is available.

Example:

def stubbed_method(self):
    raise test.TestingException()
self.stubs.Set(cls, 'inner_method', stubbed_method)

obj = cls()
self.assertRaises(test.TestingException, obj.outer_method)

Stubbing and Mocking

Whenever possible, tests SHOULD NOT stub and mock out the same function.

If it's unavoidable, tests SHOULD define stubs before mocks since the TestCase cleanup routine will un-mock before un-stubbing. Doing otherwise results in a test that leaks stubbed functions, causing hard-to-debug interference between tests1.

If a mock must take place before a stub, any stubs after the mock call MUST be manually unset using self.cleanUp calls within the test.


  1. https://bugs.launchpad.net/nova/+bug/1180671↩︎