EC2: Do not tag an instance until it is running

Sometimes an instance stays as a reservation for a long time, sometimes
it's only a reservation for a split second. Before this change, the tag
would usually work, because the reservation turned to an instance fast
enough. But many times we've observed the instance conversion taking 5 -
10 seconds, and this is long enough that the tag operation fails, and
then nodepool loses track of the instance entirely.

Change-Id: Iaa5c230976625d8a5c5afd7970691e235f0f77f7
This commit is contained in:
Clint Byrum 2019-01-14 17:20:41 -08:00
parent 7b640f7f48
commit b328c6de45
1 changed files with 2 additions and 2 deletions

View File

@ -49,8 +49,6 @@ class AwsInstanceLauncher(NodeLauncher):
attempts += 1
time.sleep(1)
instance.create_tags(Tags=[{'Key': 'nodepool_id',
'Value': str(self.node.id)}])
instance_id = instance.id
self.node.external_id = instance_id
self.zk.storeNode(self.node)
@ -60,6 +58,8 @@ class AwsInstanceLauncher(NodeLauncher):
state = instance.state.get('Name')
self.log.debug("Instance %s is %s" % (instance_id, state))
if state == 'running':
instance.create_tags(Tags=[{'Key': 'nodepool_id',
'Value': str(self.node.id)}])
break
time.sleep(0.5)
instance.reload()