provision_node is called from thread pool workers so the call order is
not deterministic. This causes py38 job failures sometimes.
This change asserts that the expected calls are occuring, but not
their order.
Change-Id: Id9f870e35dc9c92f0456d85c087514fb53053b76
When listing instances fetch all allocations to a cache,
fetch each nodes allocation from the cache. This is
significantly faster than calling get_allocation for each
node.
time metealsmith list # 250 nodes:
real 0m54.970s vs real 1m45.048s without "cached".
Story: 2010571
Task: 47318
Change-Id: Ic4cef61e338fb57bea5de4f96eb3584386cc5152
By retriving ports from neutron by 'binding:host_id', i.e
ironic node uuid we can remove the list_node_vifs() call
for each node.
Story: 2010571
Task: 47302
Change-Id: I1d85f3341e04ddf969027a6f6839f4a807666258
When listing NICs for an instance populate a class global
cache of networks by network_id. When listing nodes with VIFs
on the same network this reduces the amount of calls to the
network service.
250 Nodes without patch: real 6m29.342s
250 Nodes with patch: real 5m26.544s
Story: 2010571
Task: 47301
Change-Id: I99a2d8ecab657c8e4c852c73e816a5a8f2856471
Remove or update the corresponding CI jobs. Use the new custom Cirros
image in the HTTP job to avoid relying on CentOS.
Temporary increase complexity requirements while we have a deprecation
warning in the provisioner.
Change-Id: I5e65f9949ee3b78430309017800b88c1ba21dd02
A hostname or name is passed to the unprovision call, and the node is
looked up by allocation name (hostname), falling back to node name.
This can result in the wrong node being unprovisioned in the following
scenario:
* node and hostname naming scheme are identical
* node and hostname names are mixed because allocation placement
doesn't take the node name into account
* a node has already been unprovisioned by hostname (allocation)
lookup
* unprovision is performed a second time, this time matching the
hostname with a different node's name
* the wrong node is unprovisioned
The unprovision_node method can also take a node object instead of a
string identifier, so this change does a precise node lookup before
calling it. This means only hostnames are used for allocation lookups,
and node names for node lookups.
Change-Id: I9507f8d30c871ae62a250148789393695d59183a
Resolves: rhbz#2092444
The node lookup for unprovision first attempts the allocation name
(hostname)[1] but the metalsmith_instances unprovision passes the node
name.
This means in cases where the node naming scheme and the allocation
naming scheme are the same, the wrong node may be unprovisioned.
This change switches to passing the hostname for unprovision, only
passing the node name if the hostname is missing from the instances
entry.
[1] https://opendev.org/openstack/metalsmith/src/branch/master/metalsmith/_provisioner.py#L653
Change-Id: Ie6b989f8d67c03606be37310777175cfb8d9303e
Resolves: rhbz#2092444
The allocation name/hostname is set during the provision step, but it
is also possible to set it during the reserve step, which is done in
this change.
Setting the name during reservation is more correct, and triggers some
extra sanity checks during provision. But it also enables this change[1]
which attempts to reserve the node with the same name as the
allocation.
[1] https://review.opendev.org/c/openstack/ironic/+/845856
Change-Id: I0da4644a955b8e9fbf509c074b939b97c64739cb
Currently if a port is missing, running metalsmith list will fail with
an error. This change handles missing ports by not showing any IP
address for that VIF, and logging a warning.
Change-Id: I94924ebc81b46a2aa4c3b6f82bd46480f37b3150
Resolves: rhbz#2092444
Fixes and issue where a port cannot be created on
a specific subnet if there are multiple subnets
with the same name on different networks.
Allows both 'network' and 'subnet' in NIC information,
when looking up the subnet filter on the network_id when
both 'network' and 'subnet' is provided.
Story: 2009732
Task: 44152
Change-Id: Ied2d16ec33fe71522c3461d3df6e70fbfdd976b2
Set the 'binding:host_id' on neutron port VIFs when
they are created, or update it for existing ports.
This ensures IP allocation is not deferred by neutron.
If IP allocation is deferred, the config-drive network
metadata will not include enough information to properly
bootstrap the instance networking.
Story: 2009715
Task: 44089
Closes-Bug: #1952573
Change-Id: I7411c1f2fbc57dad0504fe61e278308030f059bf
cloud-init will use fall-back config when network metadata
is *not* present. This works fine if the first NIC on the node is
connected to a network with DHCP. But, when the first NIC is not
used for provisioning, cloud-init will only write a fallback config
for the first NIC. This causes the provisioned node to be unavailable.
Extend instance configuration to include network metadata so that
cloud-init can configure node networking.
Story: 2009238
Task: 43378
Change-Id: I70f1a972a6d5a0398cd348f00308957386d66067
ABCs in collections should be imported from collections.abc and direct
import from collections is deprecated since Python 3.3.
Change-Id: Ibca1d5756d7598e961e21c5ba9bd6e17bb895285
Node instance_info `image_disk_format` needs to be set to `raw` for
large raw images or the deployment may fail with a memory check error
or a full node tmpfs. Even when there is no error, without
image_disk_format=raw the image will not be streamed directly to disk.
image_disk_format is auto-detected and set by ironic for glance and
file sourced images, but this does not occur for direct HTTP based
images.
This change will set image_disk_format=raw when the URL file extension
ends with .raw, which is enough to support TripleO's conventions for
the overcloud raw image.
Change-Id: I6a9c225fc2d14b2d07cd0bf2379cd2c8c548f312
This has sub-options ``cloud_config` and ``meta_data`` and allows
customization of data packaged in the config-drive to be consumed by
cloud-init.
This is needed by TripleO to provide an equivalent to the current
heat/nova based firstboot customization support.
Change-Id: Ie384292a3310cb06e908dd9027e9300ca108e7dd
Instead of wrapping the metalsmith CLI, the metalsmith_deployment role
now uses the metalsmith_instances module. There are differences between
the instances formats of the role and the module which are partially
resolved with a simple transformation module called
metalsmith_deployment_defaults.
A 'candidates' attribute is added to metalsmith_instances, but keeping
the single 'name' attribute to remain compatible with the TripleO
usage.
Unresolved differences between the 2 are described below:
metalsmith_instances doesn't have a per-instance state attribute,
instead it has one state attribute for all instances. I propose that
support for per-instance state is dropped. This was never documented
in the README.rst anyway.
extra_args only applies to a CLI. Apart from --dry-run these arguments
are either for output formatting or Ironic API authentication. I propose
that this option is dropped. A metalsmith_debug arg is added to make
the ouput more verbose.
Change-Id: Ia30620821182c58050813e807cdde50a27d03c15
testtools was used to provide a smoother transition to Python 3.
fixtures seems no longer actively maintained and is mostly redundant.
Change-Id: I3fff3beafca13a783bcb4d5a811099ad491ac7ac
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: I97af946c244559513689c1ea9ac2d736d55dbe0f
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Since we've dropped support for Python 2.7, it's time to look at
the bright future that Python 3.x will bring and stop forcing
compatibility with older versions.
This patch removes the six library from requirements, not
looking back.
Change-Id: I19384db89fc2a68376efc3f9f2e05fbd05046fd6
It is useful to see the purpose of a port when viewing a port list or
port details. This change will set the port name to
<hostname>-<net-name> when the hostname is provided to the NICs
constructor.
Change-Id: Ic5789c0c3808afaa6b45408bce51d6c4ca86dbe5
This makes metalsmith consistent with what the nova ironic driver
does, and provides a way of discovering the hostname for a node
without needing to query the allocation API.
Change-Id: I40bbf155421cd2b382f8262eb38393c3300478cd
This change switches from a mix of metalsmith and SDK exceptions
to consistently using metalsmith exceptions.
Change-Id: I43339686a42644877dbe40d1116db1585a8a0800
With the allocation API we will need to provide the hostname as
the allocation name. Thus, we have to do it earlier.
Change-Id: I8afd8af23ad929fd9768e95a82fecd114fdcbfd9
Currently stale values can be left in instance_info (and reused next time)
if a deployment attempts fails. This change ensures that we:
1) Purge instance_info completely on failures
2) Only keep traits and capabilities in instance_info on deploy.
Change-Id: I52a85620d9ac2f471bca6498294871f3bb16d47f
After the switch to openstacksdk we no longer have a sufficient check
on node's availability or maintenance. This patch restores it.
Change-Id: I2c85cf0adb02061b3dd85f19dd10c8a5af1118da
Julia Kreger