Improve retry behaviour of _copy_layer_registry_to_registry

There are failures occuring in CI which result in missing layers after
a registry->registry transfer.

A job.exception() check is added to ensure the whole image transfer
fails if a single layer fails.

_copy_layer_registry_to_registry now retries on IOError. This is
the parent class of RequestException, and could plausably be raised
during a layer transfer.

Change-Id: Ibe9c046ed9172dd0bc276c6400496392b2c69dda
Partial-Bug: #1815576
This commit is contained in:
Steve Baker 2019-02-21 09:24:05 +13:00
parent fe8dd5c907
commit 67a3c4d8b3
1 changed files with 4 additions and 1 deletions

View File

@ -1099,7 +1099,7 @@ class PythonImageUploader(BaseImageUploader):
@tenacity.retry( # Retry up to 5 times with jittered exponential backoff
reraise=True,
retry=tenacity.retry_if_exception_type(
requests.exceptions.RequestException
IOError
),
wait=tenacity.wait_random_exponential(multiplier=1, max=10),
stop=tenacity.stop_after_attempt(5)
@ -1181,6 +1181,9 @@ class PythonImageUploader(BaseImageUploader):
target_session=target_session
))
for job in copy_jobs:
e = job.exception()
if e:
raise e
image = job.result()
if image:
LOG.debug('Upload complete for layer: %s' % image)