In order to add support of transferring an ISO from one datastore
to another, below changes are done:
- Return and close connection for read_connection
- Add FileReadHandle similar to FileWriteHandle
- Add common headers to _create_connection
- Accept pre-build cookie for FileReadHandle/FileWriteHandle
- DatastoreUrl as parameter for FileReadHandle/FileWriteHandle
Closes-bug: #1975618
Change-Id: I311c98201e3a89db561b7a0c64592803b32a8b31
We're using 'requests.PreparedRequest' in one place in the tests.
However, the requests docs have a warning that one shouldn't do this:
Instances are generated from a Request object, and should not be
instantiated manually; doing so may produce undesirable effects.
It seems we're now seeing just such an effect, as requests has started
attempting to do proxy-related things resulting in the following error:
Traceback (most recent call last):
File "/usr/lib64/python3.6/unittest/mock.py", line 1183, in patched
return func(*args, **keywargs)
File ".../oslo.vmware/oslo_vmware/tests/test_service.py", line 518, in test_send_with_local_file_url
resp = transport.session.send(request)
File ".../oslo.vmware/.tox/py36/lib/python3.6/site-packages/requests/sessions.py", line 636, in send
kwargs.setdefault('proxies', self.rebuild_proxies(request, self.proxies))
File ".../oslo.vmware/.tox/py36/lib/python3.6/site-packages/requests/sessions.py", line 301, in rebuild_proxies
if 'Proxy-Authorization' in headers:
TypeError: argument of type 'NoneType' is not iterable
Do what we should have done from the beginning and use
'requests.Request.prepare' instead.
[1] https://docs.python-requests.org/en/master/api/#requests.PreparedRequest
Change-Id: I072c64904298cf83cb113e402d85fb62626f39f7
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
When we try and release a lease, we might get an object not found
exception meaning that the lease doesn't exist. So there is no need
to release something that doesn't exist.
Change-Id: I7912a7adf95f97074ee41441d7bfb8d2405b6533
This function converts a SUDS object to a dictionary.
Originally implemented in Nova as "object_to_dict()", this function
moves to oslo.vmware to stop leaking the abstraction around the backing
SOAP library into depending projects. This is especially necessary since
we want to switch the backing SOAP library in [1].
[1] https://specs.openstack.org/openstack/oslo-specs/specs/victoria/oslo-vmware-soap-library-switch.html
Change-Id: Ie1d42609104e604f9386c1b1a46be7dcd286e855
Introduced changes:
- pre-commit config and rules
- Add pre-commit to pep8 gate, Flake8 is covered in the pre-commit hooks.
- Applying fixes for pre-commit compliance in all code.
Also commit hash will be used instead of version tags in pre-commit to
prevend arbitrary code from running in developer's machines.
pre-commit will be used to:
- trailing whitespace;
- Replaces or checks mixed line ending (mixed-line-ending);
- Forbid files which have a UTF-8 byte-order marker
(check-byte-order-marker);
- Checks that non-binary executables have a proper
shebang (check-executables-have-shebangs);
- Check for files that contain merge conflict strings
(check-merge-conflict);
- Check for debugger imports and py37+ breakpoint()
calls in python source (debug-statements);
- Attempts to load all yaml files to verify syntax (check-yaml);
- Run flake8 checks (flake8) (local)
For further details about tests please refer to:
https://github.com/pre-commit/pre-commit-hooks
Change-Id: I593196cf0d099fe1dcc3a6298fd6cc6b7a883ea2
Signed-off-by: Moisés Guimarães de Medeiros <moguimar@redhat.com>
Having the cookiejar available as attribute on the client instead of
some child objects exposes an interface which depending code can rely
on. This will help with upcoming efforts to switch the SOAP library
backing oslo.vmware.
This is part of phase 1 of
https://specs.openstack.org/openstack/oslo-specs/specs/victoria/oslo-vmware-soap-library-switch.html
Change-Id: I72082f10a184a2451dfda3d002a9288fefcef961
Since the attribute access on a ManagedObjectReference can differ
depending on the SOAP library used in the backend, we and especially
depending projects should not rely on it, because that breaks the
abstraction layer oslo.vmware is supposed to provide. To help
transitioning to a newer backend library, we introduce two methods for
retrieving a ManagedObjectReference's value and type: get_moref_value()
and get_moref_type().
We cannot handle this by just returning our own ManagedObjectReference
object with uniform access, because it's hard to get control of all
morefs returned by the API.
In that spirit, we also change all references inside of oslo_vmware to
using `get_moref_value()` and `get_moref_type()` instead of directly
accessing the attributes.
This is part of phase 1 of
https://specs.openstack.org/openstack/oslo-specs/specs/victoria/oslo-vmware-soap-library-switch.html
Change-Id: I0b0a3d37f6c7d0c750b48596bc3d79b8b2cb0cee
Validate the completion of VMDK upload in ImportVApp. Otherwise, an
incomplete upload is falsely marked as successful which results in
corrupted VM.
Change-Id: I0608afd31daf95b564d3fc32028401d4aa526055
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: I4d8eada1df0ec43162d4729bf75147eb40252b72
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.
Fix:
F601 dictionary key 'cr_to_rp' repeated with different values
F841 local variable 'excep' is assigned to but never used
F821 undefined name 'excep'
E305 expected 2 blank lines after class or function definition, found 1
E731 do not assign a lambda expression, use a def
Blacklist:
W504 line break after binary operator
Update hacking extension for newer flake8 version.
Change-Id: I9f0d7b166e285091111c42fe978d29d7dd3bf5db
Cinder has a current implementation to support glance colocation
of volume images[1].
To support this feature for vmdk driver, we need to pass the
base image reference (from which the volume was originally created)
to the image service update method which this patch addresses.
[1] https://review.opendev.org/#/c/697636/
Change-Id: Iba7cbdd3f50877d360201391e87f1748433348a7
This patch adds the ability to properly translate and raise
VimExceptions when polling the state of the lease.
Change-Id: Ie22808471d4c72e26607e817167e0e2283630b5a
This patch adds some standard file handle functions to the
FileHandle class.
This adds
seek
fileno
tell
flush
Change-Id: I2ae5af9f79732a757788a03f2aabdf784f461d40
This patch adds the ability to change the read chunk
size during a call to VmdkReadHandle.read calls.
The read size defaults to READ_CHUNKSIZE, which was
the previous behavior.
Change-Id: I2480a2dfc42b51a22a2fce071c62c97d84555cf1
Glance supports multiple stores now. In order to use this feature, on
patch #661676, it added parameter "store_id" when call function
"image_transfer.upload_image". We need to parse this parameter and pass
it to "image_service.update" function.
Closes-Bug: #1857094
Change-Id: I03747f54545ae3f91bf10de76d07de6828a419d6
oslo.vmware library does not need to emit localized messages from
vCenter. Explicitly setting the locale to 'en' during session
establishment to avoid UnicodeDecodeError while handling non-
english vCenter messages.
Change-Id: Ifb26a7832c15af3ed3227c94131fe0b22579e889
Adding a looping call to periodically update NFC lease
progress during virtual disk copy from ESXi to file
handle.
Change-Id: Id562d200c92af94151b08d3ae4217da41edea55e
Bandit is flagging one of the lines in unit test as a security
issue which is false positive. Adding nosec to skip that line.
Change-Id: I36feb93bc23ba5ef0cb497859140bb5f962f2bee
This makes debugging terribly difficult when there are random
legitimate errors, for example a file is not found.
Change-Id: Iec76b95a70552de8e27683c0ea388b55b519db9f
Python 3.6 is more strict about special characters in regular strings if
they are invalid escape sequences. This most often happens in regex
strings. This fixes one instance by switching the regex string to use a
raw string.
Change-Id: I73a7393fe04c920c7d82a2fa68f1cf0c7d453a7d
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
In the case of a datastore cluster, if no datastor is found then
return None. Do not fail on a key exception for accessing a field
that does not exist
Change-Id: I036f8022e2b9f454f7cd10846b408153a18b7b60
Adding a method to return SDRS recommendation key for creating
a VM.
Change-Id: If759114357c2929c813f55d57df3775f0eb9533d
Authored-by: Vipin Balachandran <vbala@vmware.com>
There are cases when there is no request context set when calling an
async task in vCenter. Don't update the local store in those cases.
Change-Id: Ieef237bd5de0ad1c12afbf57426c29e3ca678dfc
There are various useful log messages which come when polling vCenter
tasks which are missing request context and thus are hard to correlate
to API requests.
This patch fixes this by passing the request context to the async
jobs which poll the vCenter tasks.
Change-Id: I8efbc343e5073d3deb18b5f87c378c271ea4d816
Under Python 3.7, a trailing comma is no longer added to
the init parameters generated by a repr() call:
>>> repr(Exception('It Works'))
"Exception('It Works')"
vs
>>> repr(Exception('It Works'))
"Exception('It Works',)"
Support pre and post Python 3.7 formats in test cases.
Change-Id: Idff8c841fd6c489e887bd72b49e91ef6f0e2d4f3
Closes-Bug: #1784487
Add utility function which returns the most recommended datastore from a
datastore cluster where a VM can be cloned to.
Change-Id: Ic532bb9c52270ba34f70d2b7977de49f8753d1d1
Add new function which returns reference and name of the specified
datastore cluster. The cluster can be specified either by name or by its
moid.
Change-Id: I73df95294accc1954da571437efc8302edf544d2
StoragePods (a.k.a datastore clusters) are located under
Datacenter.datastoreFolder, so we need a new traversal spec in order to
retrieve them.
Change-Id: I899f8f6ef05a94d21d13a55ca0a0f8fbd042a448
Generating the documentation fails because the inherited docstring for
HTTPAdapter.send() gives a warning and warnings are treated as errors.
This patch fixes this by putting a sane docstring for the overridden
method.
Change-Id: Ief6c7964848e4acec29457bc29c327ce161524f7
We have been using FileHandle classes mostly in cases when we have to
establish secure connection and we have an SSL thumbprint of the host we
connect to. However, there are also cases when we don't have a
thumbprint and we need CA store. This patch uses the requests library to
provide such CA store.
Change-Id: I8567c8c273a3bff41c4b80a77e1fa8af743bf98c
Xiangfei Zhu