Commit Graph

12 Commits

Author SHA1 Message Date
Pierre-Samuel Le Stang 480ea3825f Implement glance-download internal plugin
Add a new import method called glance-download
that implements a glance to glance download in
a multi-region cloud with a federated Keystone.

This method will copy the image data and
selected metadata to the target glance, checking
that the downloaded size match the "size" image
attribute in the source glance.

Implements: blueprint glance-download-import
Co-Authored-By: Victor Coutellier <victor.coutellier@gmail.com>
Change-Id: Ic51c5fd87caf04d38aeaf758ad2d0e2f28098e4d
2022-08-23 08:26:52 -07:00
Zuul 111f1acffc Merge "Make web-download revert all stores on fail" 2021-03-06 17:33:58 +00:00
Zuul 1bef6a32af Merge "Pass ImageActionWrapper to internal plugins" 2021-03-06 17:33:44 +00:00
Abhishek Kekane d54449af44 Utilize newly added tasks database fields
Made provision to pass image_id, request_id and user_id information
while creating new task.

Partially-Implements: blueprint messages-api
Change-Id: I299a222eeef81431143db3ba7fc08365c924326b
2021-02-24 05:17:43 +00:00
Dan Smith cbb2af6e32 Make web-download revert all stores on fail
If web-download fails to stage the image, no subsequent import to
store tasks could have run. Thus, we should remove all of them from
"importing" and add them to "failed" so that it is obvious from the
outside that we failed.

This is another good reason for having the $image/tasks API, but we
also should not continue advertising "yes we're importing to $store"
when we are not.

Change-Id: Iebbb2dcb767ecf3c965f34f1ca04af20a2039be1
Closes-Bug: #1914826
2021-02-08 09:43:14 -08:00
Dan Smith 68646db35d Pass ImageActionWrapper to internal plugins
The _internal_plugins/* tasks are not getting passed the
ImportActionWrapper, and thus are doing things against the image
that do not respect the task lock, nor apply multiple updates
atomically.

This fixes that and brings them in line with the base tasks, using the
wrapper for things like getting the image_id. The copy_image task
looks at the image for a few other things that are not exposed out
of the wrapper, so I left the image_repo.get() in that task until I
can extend it. They are read-only though, so not a big deal.

The web-download test also was not passing the right options to
that task (confusing task_repo and image_repo), which this cleans
up as well. It was also missing validation of the code that reverts
the state to 'queued' when web-download fails, so that is added here
as well.

Change-Id: I6db86b3e17a6a2f78745b40381b9419fb4404a4e
Related-Bug: #1914826
2021-02-05 15:10:31 -08:00
Victor Coutellier 922c2ed5ad Fix cleaning of web-download image import
If import flow fail before reaching the end it never execute
the _DeleteFromFS task and the node_staging_uri is never cleaned up.

Implement the revert() function of the _WebDownload task to remove the
temporary file.

Change-Id: I6dd6a6e2a95a5bd17a80b6256852bb9fac5fa339
Co-Authored-By: Grégoire Unbekandt <gregoire.unbekandt@gmail.com>
Co-Authored-By: Abhishek Kekane <akekane@redhat.com>
Closes-Bug: #1795950
2020-09-22 20:31:30 +02:00
Grégoire Unbekandt 68c202d38b Image import "web-download" check downloaded size
If the downloaded data size is different from the expected one, the
task "web-download" in the image import process will now fail.

Change-Id: Ie260486d795a6f4af1632f6f3708abc92fb47a3a
Closes-Bug: #1895663
2020-09-22 14:04:15 +00:00
Sean McGinnis 94b0876429 Use unittest.mock instead of third party mock
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.

Change-Id: I44e7b6f76e2d12f620ec602afc77ce11ba6b9d9a
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-20 15:07:00 +00:00
Abhishek Kekane c927246085 Data remains in staging area if 'file' store is not enabled
When operator has not enabled 'file' store and using other stores like ceph,
swift etc. the uploading to staging area works as we explicitly
build 'file' store during this operation, while cleaning up we directly
use 'glance_store.delete_from_backend' which only works if 'file'
store is enabled.

Modified '_DeleteFromFS' task and _unstage call which will use os
module to unlink the file present in staging area explicitly to
delete the data from staging area.

Closes-Bug: #1803498
Change-Id: If0b3b0af9300301291758c67267890e0959ebb3c
2019-03-15 12:32:00 +00:00
Abhishek Kekane 94d3a0a14d Failure in web-dowload kept image in importing state
As of now if import using web-download method fails due to invalid
uri, image remains in 'importing' state. As the image is in 'importing'
state it is not of any use to end-user.

Resetting it to queued state, so that user can try again uploading/importing
data to the same image.

Change-Id: Iaa26922a9cf5a419c3da23bdfc347a6fdf6cb549
Closes-Bug: #1803299
2019-03-04 05:01:05 +00:00
Corey Bryant c58e5e02af Rename async package to async_
In Python 3.7, "async" is a keyword. To prevent it from
conflicting, rename the async package to async_.

Change-Id: I1eaf87eedb86679d9ca9323aac05f0770c33efea
Closes-Bug: #1781617
2018-08-07 14:42:14 -04:00