Add plug-in modules for direct downloads of glance locations

Glance can expose direct URL locations to its clients.  In current
versions of nova the only URL that can be accessed directly is file://.
This patch adds a notion of download plug-ins.  With this new download
protocol modules can be added without disruption to the rest of the
code base.  Based on the scheme of the URL returned from Glance a
plug-in will be loaded and used to download the data directly, instead
of first routing it through Glance.  If anything fails in the process
the image will be downloaded by way of Glance.

Handlers are loaded with stevedore.  To add a new module follow the example
in nova.image.downloads.file.py.  The module is required to have two
functions:

get_download_hander():
This must return a child of nova.image.download.TransferBase

get_scheme():
Return the URL scheme that this module handles (ex: 'file')

If additional configuration is needed it can be added by the specific
plug-in (as shown by file_download_module_config included with this
patch submission).  Once the module is created it must be added as an
entry point to the python installation.  For those included with nova
this can be done by adding the following it setup.cfg:

[entry_points]
nova.download.modules =
    file = nova.image.xfers.file

Additionally, as part of the multiple-locations work in Glance meta data
comes back with each location describing it.  As an example,
this is needed for direct access to file URLs.  Nova cannot assume that
every file URL is accessible on its mounted file systems, nor can it
assume that the mount points are the same.  This patch solves that
problem for direct access to files.

blueprint image-multiple-location

Change-Id: I79b863c0075cebaadce5b630f22b81d2959ddbb1

nova/exception.py

@@ -1331,3 +1331,30 @@ class InstanceGroupPolicyNotFound(NotFound):
 
 class PluginRetriesExceeded(NovaException):
     msg_fmt = _("Number of retries to plugin (%(num_retries)d) exceeded.")
+
+
+class ImageDownloadModuleError(NovaException):
+    msg_fmt = _("There was an error with the download module %(module)s. "
+                "%(reason)s")
+
+
+class ImageDownloadModuleLoadError(ImageDownloadModuleError):
+    msg_fmt = _("Could not load the module %(module)s")
+
+
+class ImageDownloadModuleMetaDataError(ImageDownloadModuleError):
+    msg_fmt = _("The metadata for this location will not work with this "
+                "module %(module)s.  %(reason)s.")
+
+
+class ImageDownloadModuleNotImplementedError(ImageDownloadModuleError):
+    msg_fmt = _("The method %(method_name)s is not implemented.")
+
+
+class ImageDownloadModuleMetaDataError(ImageDownloadModuleError):
+    msg_fmt = _("The metadata for this location will not work with this "
+                "module %(module)s.  %(reason)s.")
+
+
+class ImageDownloadModuleConfigurationError(ImageDownloadModuleError):
+    msg_fmt = _("The module %(module)s is misconfigured: %(reason)s.")

setup.cfg

@@ -27,6 +27,8 @@ packages =
     nova
 
 [entry_points]
+nova.image.download.modules =
+    file = nova.image.download.file
 console_scripts =
     nova-all = nova.cmd.all:main
     nova-api = nova.cmd.api:main