Use the original importer for mock. Otherwise, the unit tests that
use importlib with context are not running in an isolated manner,
leading to various transient failures if `stestr` is not run with the
`--isolated` flag.
The unittest.mock._get_target has been changed in Python 3.11, by commit:
ab7fcc8fbd
This change is needed to add support for Python 3.11 and Python 3.12
unit testing.
Change-Id: Ic73f5280116673f771f5460cdfd3702f5fbe16b5
If `instance_id` is set in the metadata as an integer, cast it as a
string when used to uniquely identify plugin sections.
Fixes-Bug: https://github.com/cloudbase/cloudbase-init/issues/124
Change-Id: I3a9335fb3885c5f3191051aaaeeec94f89a6b89a
Match vFAT label using a given label instead of hardcoding 'config-2'.
The NoCloud and ConfigDrive metadata services both attempt to find vFAT
formatted drives but the label used was hardcoded to 'config-2' which
only works with ConfigDrive. BaseConfigDriveService which both inherit
from already has a drive_label property set appropriately for both so
let's use that for the vFAT finding logic as well.
Change-Id: I8004a8565338b0615450bb28cecc86901be94766
Check if the configuration value of bsdtar_path does exist as a path
before trying to execute the binary.
Updated the tutorial reference of bsdtar to FreeBSD instead of Ubuntu.
Change-Id: Ieba5da2f330aa11c40cce6c2ae9de40155f33b07
Use cryptography instead of the flaky openssl libraries loading.
If the libssl DLLs were not present or were present in another order
then the required one, there could be errors when securing the password
before sending it to the metadata service.
Fixes: https://github.com/cloudbase/cloudbase-init/issues/34
Change-Id: I1a2245e199f65f4665071ada9576dcae77a3a432
With Python 3.7+, the Python installer has the libcrypto and libssl dlls
in the DLLs Python folder, namely libcrypto-1_1.dll and libssl-1_1.dll,
which can be used directly.
Change-Id: I245c377dc8a9ec9a2e8548806bacd757bfdf27b6
With Python 3.8, the xml.etree.ElementTree.ElementTree function now
preserves the attribute order specified by the user.
This change broke the winrm enable functionality, throwing the erorr:
```
TypeError: Objects of type 'bytes' can not be converted to Unicode.
```
The fix was to set the correct parameter to return an unicode string and
not bytes.
Change-Id: I10f7fa357267f36dfed03ccb708694f6fc1cdd97
* Added more retries for wireserver discovery
* Added logging to wireserver discovery errors
* Fallback to the well known wireserver IP in case of discovery failure
Github Issue: https://github.com/cloudbase/cloudbase-init/issues/94
Change-Id: I91460f9ad674294a83c9b3c3f05bc04ab61de56f
Python 3.6/3.7 are no longer supported for the latest releasese of
the OpenStack packages like oslo or coverage, thus switching to the Zed
OpenStack release gates, which have support for 3.8/3.9/3.10 Python
versions.
For gates to pass, the docs required a defined language to be set ("en"),
while the crypto unit tests required a method signature fix when running
on Python 3.9/3.10.
The version test__check_latest_version* unit tests started failing
in a transient pattern because of the
pbr.version.VersionInfo().release_string() throwing the following error:
"NotImplementedError: cannot instantiate 'WindowsPath' on your system".
This required to have the cloudbaseinit.version.get_version mocked for a
reliable unit test run on Linux.
Change-Id: I4748d1258c072c377825474e1116347b0a085c56
When a small amount of disk space remains to be extended,
the 'Virtual Disk Service' can fail with
VDS_E_EXTENT_SIZE_LESS_THAN_MIN error code.
This happens on environments where a volume has already been extended
and there is a small amount of bytes that somehow remain residual after
the initial extension.
Change-Id: I072ed568ef1f2790e95851b45afb8ffcc0acce0e
Fixed Windows osutils DHCP setting, as the former implementation that
used registry keys was no reliable. Using MSFT_NetIPInterface should fix
any reliability issues.
Change-Id: I94cf70b092da706cdfaf57bc78d7b266064b97b0
Allows giving an instance an individual cloud username, e.g. for license servers, while still retrieving SSH public keys for that user from OpenStack.
Change-Id: I078380ef8f74b30105fd44378045df8b1260acbc
On Windows, the load user profile may fail on laggy systems, if the
Windows subsystems are not ready at that moment.
Retrying the load should fix the issue most of the times.
Change-Id: I28cc564ebeac6d901dcbbef7cebe882a5ccb41b1
The WMI implementation for NETLBFO is sometimes flaky and does not fail
when a bond cannot be created.
Added 5 retries for the team creation, just in case the WMI call to
create the bond did not fail, but the bond was not created.
In this scenario, the _wait_for_nic method is called and it fails,
as it does not find the team nic and the retry is performed.
Updated the wait for team nic retry interval to 10 seconds,
as the team nic may be retrieved with WMI after a longer time.
Change-Id: I7622a4a30b867f335ed8f3288ca2fea830daf94b
Removed the functools partial from being mocked, as it is used
internally in the mocking implementation and generates unit test error.
Updated tox / unit test requirements according to new OpenStack gate
requirements.
Updating the tox requirements added a new set of flakes rules that had
to be skipped for now, which will be addressed in upcoming patches.
Change-Id: I231ca3b15313541e2c43785e80d9db5659aa1cc3
Move baseopenstackservice._ip_netmask_to_cidr to utils/network so that
it can be reused by other network parsers.
Change-Id: Iacca02cda75fd5d5b80f6200e7d2f26a3381b737
It is necessary to catch yaml and json errors that come from invalid
json / yaml streams like '{}}'.
Having an enumeration after a key value was set is a yaml parser error.
Change-Id: Ia8b298c3f1b36c6dee29326955d1e76ade3104b1
If a trial license key was set, the KMS default product key and host was
not properly configured, as the execution of the licensing plugin ended
abruptly.
The fix is to set the kms and product key without checking the evaluation
date corresponding for the trial key.
Change-Id: I45e9364661208c454ddf2be0ff925d149fe0a6b0
Add support for the Packet features:
* phone home - after provisioning is ready, an empty HTTP POST request
is performed to the phone_home_url
* post user password - a random password is generated for the default
user, the password gets encrypted using the encryption public SSH key
and posted to the phone_home_url
Change-Id: Ib3e405b6b5bb4ca65d047c756513b9d6f87cf6de
Method osutils.set_static_network_config expects dnsnameservers as list.
Use an empty list if the dnsnameservers is None.
Change-Id: I3b567a941336b61f3a505306f738f14e8b3db04f
Add support for NoCloud metadata provider, where the metadata
is provided on a config-drive (vfat or iso9660) with the label
cidata or CIDATA.
The folder structure for NoCloud is:
* /user-data
* /meta-data
The user-data and meta-data files respect the EC2 metadata
service format.
Supported features for the NoCloud metadata service:
* instance id
* hostname
* plublic keys
* static network configuration (Debian format)
* user data
More information:
cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html
Change-Id: Ib434cf2b2b21bf9faa58e05ba40eb0135385c9ea
Implements: blueprint nocloud-metadata-support
Move and rename VMwareGuestInfoService._parse_data to
cloudbaseinit.utils.serialization.parse_json_yaml so that it can be
easiliy reused.
Change-Id: I3b86efd5e2b2062f89ffc7745f6f0481ec665078
Refactor the existing ConfigDrive Metadata service so that another
metadata format implemenation like NoCloud can be easily added.
Now, the drive label and the metadata file can be set in the
constructor, making it easy to add another config drive metadata service
with different label or path.
Change-Id: I8dd8160dfbe9f529bb8f30ab85181f264c18833e
Add cloudbaseinit.metadata.services.gceservice.GCEService that
supports Google Cloud Engine.
Supported features for the metadata service:
* instance id
* hostname
* userdata
* ssh keys
Change-Id: I9e5e2cbcaa0953fc0c0ae8117e258713ac6443b7
On Py27, the yaml module implementation uses a normal dictionary for
implementation, resulting in arbitrary order for the parsed yaml.
If the config option cloud_config_plugins is not set or empty, fallback
to the default order defined in the factory.
Config option `cloud_config_plugins` is now used for filtering the cloud
config plugins to be executed, which was the original intended behaviour.
Change-Id: I59ea56e473ab343a826ff8c38787cb7f11cebd27
VMwareGuestInfoService is a metadata service which uses VMware's
rpctool to extract guest metadata and userdata configured for machines
running on VMware hypervisors.
The implementation is similar to:
https://github.com/vmware/cloud-init-vmware-guestinfo
Supported features for the metadata service:
* instance id
* hostname
* admin username
* admin password
* public SSH keys
* userdata
Configuration options:
```ini
[vmwarequestinfo]
vmware_rpctool_path=%ProgramFiles%/VMware/VMware Tools/rpctool.exe
```
The VMware RPC tool used to query the instance metadata and userdata
needs to be present at the config option path.
Both json and yaml are supported as metadata formats.
The metadata / userdata can be encoded in base64, gzip or gzip+base64.
Example metadata in yaml format:
```yaml
instance-id: cloud-vm
local-hostname: cloud-vm
admin-username: cloud-username
admin-password: Passw0rd
public-keys-data: |
ssh-key 1
ssh-key 2
```
This metadata content needs to be sent as string in the guestinfo
dictionary, thus needs to be converted to base64 (it is recommended to
gzip it too).
To convert to gzip+base64 format:
```bash
cat metadata.yml | gzip.exe -9 | base64.exe -w0
```
Co-Authored-By: Rui Lopes <rgl@ruilopes.com>
Change-Id: I6a8430e87ee03d2e8fdd2685b05e60c5c0ffb5be
Fixes base_template.load and remove_template_definition which fail when
the userdata is empty / None or when the userdata has only one line.
Change-Id: I2ba0e16489049be390b2c9bc1fdf1e1059f54eb1
If the userdata starts with the text '## template:jinja',
Jinja2 templating engine will be used to render that
userdata.
This feature leverages the get_instance_data metadata
service implementation and tries to be as close as possible
with the cloud-init's implementation.
Change-Id: I79cb0066f7d65ae27867ac01b443e9be432b4dc3
Add support for cloud-config users and groups creation.
For Windows, the following format is supported:
groups:
- windows-group: [user1, user2]
- cloud-users
users:
-
name: brian
gecos: 'Brian Cohen'
primary_group: Users
groups: cloud-users
passwd: StrongPassw0rd
inactive: False
expiredate: 2020-10-01
ssh_authorized_keys:
- first key
- second key
The passwords for Windows users are required to be in
plain text. On *nix systems, the passwords are hashed.
If the password is not present, a random password will
be set.
Fixes: https://github.com/cloudbase/cloudbase-init/issues/26
Change-Id: I035f92849a59a8370df30a6de41f66f5fb2300af
Some plugins, like the userdata execution plugin, need a standard model
for the instance data, which can be applied by templating engines like
jinja to the userdata.
This way, a user can use this feature to customize her userdata scripts
with values that are specific to that instance runtime env or cloud platform.
The instance data structure is based on the cloud-init specifications:
https://cloudinit.readthedocs.io/en/latest/topics/instancedata.html
The v1 namespace contains a subset of the cloud-init specs for the instance
data.
The ds.meta_data namespace contains all the values the v1 namespace
contains, in order to be compatible with cloud-init, plus a subset
of other instance data.
Change-Id: I5c529498f06fe3c86f7fa3c20fdf3091840c4041
The cloud config plugins were configured as a normal Python dictionary,
thus executing them in a random order, defined by the Python
implementation.
We require runcmd plugin to run last so that it is the last
configuration stage. In the future, we need the order as the groups
plugin is required to run before the users plugin.
Change-Id: I754ef7b15d5c7a12d08a12c876cfb52de64a06cc
If the userdata is of type cloud-config, the runcmd entry can contain
multiple entries with commands that will be executed, in the order
of their definition.
The commands can be given as a string or as an array of strings, the
first item being the binary to be executed and the rest being the
parameters of that binary.
The commands will be aggregated and written into one single shell file,
in the order of their definition.
On Windows, the file will be executed by the native Windows
shell cmd.exe.
Example userdata file:
runcmd:
- 'dir C:\\'
- ['echo', '1']
Fixes: https://github.com/cloudbase/cloudbase-init/issues/27
Change-Id: Ie307e08f8c4108c7bf9108543cc90b6a7fa2e7ae
The empty metadata service can be used to run plugins that do not
rely on metadata service information, like setting NTP, MTU,
extending volumes, local scripts execution, licensing, etc.
It can be used also as a fallback metadata service, in case no other
previous metadata service could be loaded.
EmptyMetadataService does not support the following plugins:
* cloudbaseinit.plugins.windows.createuser.CreateUserPlugin
* cloudbaseinit.plugins.common.setuserpassword.SetUserPasswordPlugin
* cloudbaseinit.plugins.common.sshpublickeys.SetUserSSHPublicKeysPlugin
* cloudbaseinit.plugins.windows.winrmcertificateauth.ConfigWinRMCertificateAuthPlugin
If any of the plugins defined above are executed,
they will fail with exception NotExistingMetadataException. The reason
for the hardcoded failure is that these plugins rely on metadata to execute
correctly. If metadata like username or password is not provided,
these plugins can lock or misconfigure the user, leading to unwanted problems.
Implements: blueprint empty-metadata-provider
Change-Id: I5fb88a07bf72321d0f66b16d151bf6059b448580