Merge "implement basic-auth support for user image download process"

This commit is contained in:
Zuul 2023-11-28 16:05:52 +00:00 committed by Gerrit Code Review
commit 28b3f56b2c
9 changed files with 364 additions and 29 deletions

View File

@ -33,6 +33,7 @@ the services.
Hardware Burn-in <hardware-burn-in>
Vendor Passthru <vendor-passthru>
Servicing <servicing>
Basic Auth Support For User-image Servers <user-image-basic-auth>
Drivers, Hardware Types and Hardware Interfaces
-----------------------------------------------

View File

@ -0,0 +1,54 @@
======================================================
HTTP(s) Authentication strategy for user image servers
======================================================
How to enable the feature via global configuration options
----------------------------------------------------------
There are 3 variables that could be used to manage image server
authentication strategy. The 3 variables are structured such a way that 1 of
them ``image_server_auth_strategy`` (string) provides the option to specify
the desired authentication strategy. Currently the only supported
authentication strategy is ``http_basic`` that represents the HTTP(S) Basic
Authentication also known as the ``RFC 7616`` internet standard.
The other two variables ``image_server_password`` and ``image_server_user``
provide username and password credentials for any authentication strategy
that requires username and credentials to enable the authentication during
image download processes. ``image_server_auth_strategy`` not just enables the
feature but enforces checks on the values of the 2 related credentials.
Currently only the ``http_basic`` strategy is utilizing the
``image_server_password`` and ``image_server_user`` variables.
When a authentication strategy is selected against the user image server an
exception will be raised in case any of the credentials are None or an empty
string. The variables belong to the ``deploy`` configuration group and could be
configured via the global Ironic configuration file.
The authentication strategy configuration affects the download process
for ``disk`` images, ``live ISO`` images and the ``deploy`` images.
Example
-------
Example of activating the ``http-basic`` strategy via
``/etc/ironic/ironic.conf``::
[deploy]
...
image_server_auth_strategy = http_basic
image_server_user = username
image_server_password = password
...
Known limitations
-----------------
This implementation of the authentication strategy for user image handling is
implemented via the global Ironic configuration process thus it doesn't
provide node specific customization options.
When ``image_server_auth_strategy`` is set to any valid value all image
sources will be treated with the same authentication strategy and Ironic will
use the same credentials against all sources.

View File

@ -73,6 +73,12 @@ You need to specify image information in the node's ``instance_info``
and ``file://``. Files have to be accessible by the conductor. If the scheme
is missing, an Image Service (glance) image UUID is assumed.
* In case the image source requires HTTP(s) Basic Authentication ``RFC 7616``
then the relevant authentication strategy has to be configured as
``http_basic`` and supplied with credentials in the ironic global config
file. Further infromation about the authentication strategy selection
can be found in :doc:`/admin/user-image-basic-auth`.
* ``root_gb`` - size of the root partition, required for partition images.
.. note::

View File

@ -74,6 +74,60 @@ class BaseImageService(object, metaclass=abc.ABCMeta):
class HttpImageService(BaseImageService):
"""Provides retrieval of disk images using HTTP."""
@staticmethod
def gen_auth_from_conf_user_pass(image_href):
"""This function is used to pass the credentials to the chosen
credential verifier and in case the verification is successful
generate the compatible authentication object that will be used
with the request(s). This function handles the authentication object
generation for authentication strategies that are username+password
based. Credentials are collected from the oslo.config framework.
:param image_href: href of the image that is being acted upon
:return: Authentication object used directly by the request library
:rtype: requests.auth.HTTPBasicAuth
"""
image_server_user = None
image_server_password = None
if CONF.deploy.image_server_auth_strategy == 'http_basic':
HttpImageService.verify_basic_auth_cred_format(
CONF.deploy.image_server_user,
CONF.deploy.image_server_password,
image_href)
image_server_user = CONF.deploy.image_server_user
image_server_password = CONF.deploy.image_server_password
else:
return None
return requests.auth.HTTPBasicAuth(image_server_user,
image_server_password)
@staticmethod
def verify_basic_auth_cred_format(image_href, user=None, password=None):
"""Verify basic auth credentials used for image head request.
:param user: auth username
:param password: auth password
:raises: exception.ImageRefValidationFailed if the credentials are not
present
"""
expected_creds = {'image_server_user': user,
'image_server_password': password}
missing_creds = []
for key, value in expected_creds.items():
if not value:
missing_creds.append(key)
if missing_creds:
raise exception.ImageRefValidationFailed(
image_href=image_href,
reason=_("Missing %s fields from HTTP(S) "
"basic auth config") % missing_creds
)
def validate_href(self, image_href, secret=False):
"""Validate HTTP image reference.
@ -96,6 +150,7 @@ class HttpImageService(BaseImageService):
verify = CONF.webserver_verify_ca
try:
auth = HttpImageService.gen_auth_from_conf_user_pass(image_href)
# NOTE(TheJulia): Head requests do not work on things that are not
# files, but they can be responded with redirects or a 200 OK....
# We don't want to permit endless redirects either, thus not
@ -104,8 +159,8 @@ class HttpImageService(BaseImageService):
# HTTPForbidden or a list of files. Both should be okay to at
# least know things are okay in a limited fashion.
response = requests.head(image_href, verify=verify,
timeout=CONF.webserver_connection_timeout)
timeout=CONF.webserver_connection_timeout,
auth=auth)
if response.status_code == http_client.MOVED_PERMANENTLY:
# NOTE(TheJulia): In the event we receive a redirect, we need
# to notify the caller. Before this we would just fail,
@ -161,14 +216,17 @@ class HttpImageService(BaseImageService):
"""
try:
verify = strutils.bool_from_string(CONF.webserver_verify_ca,
strict=True)
except ValueError:
verify = CONF.webserver_verify_ca
try:
auth = HttpImageService.gen_auth_from_conf_user_pass(image_href)
response = requests.get(image_href, stream=True, verify=verify,
timeout=CONF.webserver_connection_timeout)
timeout=CONF.webserver_connection_timeout,
auth=auth)
if response.status_code != http_client.OK:
raise exception.ImageRefValidationFailed(
image_href=image_href,

View File

@ -27,6 +27,27 @@ opts = [
cfg.StrOpt('http_root',
default='/httpboot',
help=_("ironic-conductor node's HTTP root path.")),
cfg.StrOpt('image_server_auth_strategy',
default='noauth',
mutable=True,
help=_("Used to select authentication strategy against the "
"image hosting HTTP(S) server. When set to http_basic "
"it enables HTTP(S) Basic Authentication. "
"Exception is thrown in case of missing credentials. "
"When this option has a valid value such as http_basic"
", the same single set of credentials will be used "
"against all user-image sources! Currently only the "
"http_basic option has any functionality.")),
cfg.StrOpt('image_server_user',
mutable=True,
help=_("Can be used by any authentication strategy that "
"requires username credential. Currently utilized by "
"the http_basic authentication strategy.")),
cfg.StrOpt('image_server_password',
mutable=True,
help=_("Can be used by any authentication strategy that "
"requires password credential. Currently utilized by "
"the http_basic authentication strategy.")),
cfg.StrOpt('external_http_url',
help=_("URL of the ironic-conductor node's HTTP server for "
"boot methods such as virtual media, "

View File

@ -508,6 +508,14 @@ class AgentDeploy(CustomAgentDeploy):
'stream_raw_images': CONF.agent.stream_raw_images,
}
if (CONF.deploy.image_server_auth_strategy != 'noauth'
and CONF.deploy.image_server_auth_strategy is not None):
image_info['image_server_auth_strategy'] = \
CONF.deploy.image_server_auth_strategy
image_info['image_server_user'] = CONF.deploy.image_server_user
image_info['image_server_password'] =\
CONF.deploy.image_server_password
if node.instance_info.get('image_checksum'):
image_info['checksum'] = node.instance_info['image_checksum']

View File

@ -42,7 +42,7 @@ class HttpImageServiceTestCase(base.TestCase):
self.service.validate_href(self.href)
path_mock.assert_not_called()
head_mock.assert_called_once_with(self.href, verify=True,
timeout=60)
timeout=60, auth=None)
response.status_code = http_client.NO_CONTENT
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href,
@ -60,7 +60,7 @@ class HttpImageServiceTestCase(base.TestCase):
response.status_code = http_client.OK
self.service.validate_href(self.href)
head_mock.assert_called_once_with(self.href, verify=False,
timeout=60)
timeout=60, auth=None)
response.status_code = http_client.NO_CONTENT
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href,
@ -77,7 +77,7 @@ class HttpImageServiceTestCase(base.TestCase):
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href, self.href)
head_mock.assert_called_once_with(self.href, verify=False,
timeout=60)
timeout=60, auth=None)
head_mock.side_effect = requests.RequestException()
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href, self.href)
@ -90,7 +90,7 @@ class HttpImageServiceTestCase(base.TestCase):
response.status_code = http_client.OK
self.service.validate_href(self.href)
head_mock.assert_called_once_with(self.href, verify=True,
timeout=60)
timeout=60, auth=None)
response.status_code = http_client.NO_CONTENT
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href,
@ -108,7 +108,7 @@ class HttpImageServiceTestCase(base.TestCase):
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href, self.href)
head_mock.assert_called_once_with(self.href, verify=True,
timeout=60)
timeout=60, auth=None)
head_mock.side_effect = requests.RequestException()
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href, self.href)
@ -122,7 +122,7 @@ class HttpImageServiceTestCase(base.TestCase):
self.service.validate_href(self.href)
head_mock.assert_called_once_with(self.href, verify='/some/path',
timeout=60)
timeout=60, auth=None)
response.status_code = http_client.NO_CONTENT
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href,
@ -132,6 +132,43 @@ class HttpImageServiceTestCase(base.TestCase):
self.service.validate_href,
self.href)
@mock.patch.object(requests, 'head', autospec=True)
def test_validate_href_valid_path_valid_basic_auth(self, head_mock):
cfg.CONF.set_override('webserver_verify_ca', '/some/path')
cfg.CONF.set_override('image_server_auth_strategy',
'http_basic',
'deploy')
cfg.CONF.set_override('image_server_user', 'test', 'deploy')
cfg.CONF.set_override('image_server_password', 'test', 'deploy')
user = cfg.CONF.deploy.image_server_user
password = cfg.CONF.deploy.image_server_password
auth_creds = requests.auth.HTTPBasicAuth(user, password)
response = head_mock.return_value
response.status_code = http_client.OK
self.service.validate_href(self.href)
head_mock.assert_called_once_with(self.href, verify='/some/path',
timeout=60, auth=auth_creds)
response.status_code = http_client.NO_CONTENT
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href,
self.href)
response.status_code = http_client.BAD_REQUEST
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href,
self.href)
@mock.patch.object(requests, 'head', autospec=True)
def test_validate_href_valid_path_invalid_basic_auth(self, head_mock):
cfg.CONF.set_override('webserver_verify_ca', '/some/path')
cfg.CONF.set_override('image_server_auth_strategy',
'http_basic',
'deploy')
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href,
self.href)
@mock.patch.object(requests, 'head', autospec=True)
def test_validate_href_custom_timeout(self, head_mock):
cfg.CONF.set_override('webserver_connection_timeout', 15)
@ -140,7 +177,7 @@ class HttpImageServiceTestCase(base.TestCase):
response.status_code = http_client.OK
self.service.validate_href(self.href)
head_mock.assert_called_once_with(self.href, verify=True,
timeout=15)
timeout=15, auth=None)
response.status_code = http_client.NO_CONTENT
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href,
@ -160,7 +197,7 @@ class HttpImageServiceTestCase(base.TestCase):
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href, self.href)
head_mock.assert_called_once_with(self.href, verify='/some/path',
timeout=60)
timeout=60, auth=None)
@mock.patch.object(requests, 'head', autospec=True)
def test_validate_href_verify_error(self, head_mock):
@ -169,7 +206,7 @@ class HttpImageServiceTestCase(base.TestCase):
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href, self.href)
head_mock.assert_called_once_with(self.href, verify='/some/path',
timeout=60)
timeout=60, auth=None)
@mock.patch.object(requests, 'head', autospec=True)
def test_validate_href_verify_os_error(self, head_mock):
@ -178,7 +215,7 @@ class HttpImageServiceTestCase(base.TestCase):
self.assertRaises(exception.ImageRefValidationFailed,
self.service.validate_href, self.href)
head_mock.assert_called_once_with(self.href, verify='/some/path',
timeout=60)
timeout=60, auth=None)
@mock.patch.object(requests, 'head', autospec=True)
def test_validate_href_error_with_secret_parameter(self, head_mock):
@ -191,7 +228,7 @@ class HttpImageServiceTestCase(base.TestCase):
self.assertIn('secreturl', str(e))
self.assertNotIn(self.href, str(e))
head_mock.assert_called_once_with(self.href, verify=False,
timeout=60)
timeout=60, auth=None)
@mock.patch.object(requests, 'head', autospec=True)
def test_validate_href_path_forbidden(self, head_mock):
@ -202,7 +239,7 @@ class HttpImageServiceTestCase(base.TestCase):
url = self.href + '/'
resp = self.service.validate_href(url)
head_mock.assert_called_once_with(url, verify=True,
timeout=60)
timeout=60, auth=None)
self.assertEqual(http_client.FORBIDDEN, resp.status_code)
@mock.patch.object(requests, 'head', autospec=True)
@ -219,7 +256,63 @@ class HttpImageServiceTestCase(base.TestCase):
url)
self.assertEqual(new_url, exc.redirect_url)
head_mock.assert_called_once_with(url, verify=True,
timeout=60)
timeout=60, auth=None)
def test_verify_basic_auth_cred_format(self):
self.assertIsNone(self
.service
.verify_basic_auth_cred_format(self.href,
"SpongeBob",
"SquarePants"))
def test_verify_basic_auth_cred_format_empty_user(self):
self.assertRaises(exception.ImageRefValidationFailed,
self.service.verify_basic_auth_cred_format,
self.href,
"",
"SquarePants")
def test_verify_basic_auth_cred_format_empty_password(self):
self.assertRaises(exception.ImageRefValidationFailed,
self.service.verify_basic_auth_cred_format,
self.href,
"SpongeBob",
"")
def test_verify_basic_auth_cred_format_none_user(self):
self.assertRaises(exception.ImageRefValidationFailed,
self.service.verify_basic_auth_cred_format,
self.href,
None,
"SquarePants")
def test_verify_basic_auth_cred_format_none_password(self):
self.assertRaises(exception.ImageRefValidationFailed,
self.service.verify_basic_auth_cred_format,
self.href,
"SpongeBob",
None)
def test_gen_auth_from_conf_user_pass_success(self):
cfg.CONF.set_override('image_server_auth_strategy',
'http_basic',
'deploy')
cfg.CONF.set_override('image_server_password', 'SpongeBob', 'deploy')
cfg.CONF.set_override('image_server_user', 'SquarePants', 'deploy')
correct_auth = \
requests.auth.HTTPBasicAuth('SquarePants',
'SpongeBob')
return_auth = \
self.service.gen_auth_from_conf_user_pass(self.href)
self.assertEqual(correct_auth, return_auth)
def test_gen_auth_from_conf_user_pass_none(self):
cfg.CONF.set_override('image_server_auth_strategy', 'noauth', 'deploy')
cfg.CONF.set_override('image_server_password', 'SpongeBob', 'deploy')
cfg.CONF.set_override('image_server_user', 'SquarePants', 'deploy')
return_auth = \
self.service.gen_auth_from_conf_user_pass(self.href)
self.assertIsNone(return_auth)
@mock.patch.object(requests, 'head', autospec=True)
def _test_show(self, head_mock, mtime, mtime_date):
@ -230,7 +323,7 @@ class HttpImageServiceTestCase(base.TestCase):
}
result = self.service.show(self.href)
head_mock.assert_called_once_with(self.href, verify=True,
timeout=60)
timeout=60, auth=None)
self.assertEqual({'size': 100, 'updated_at': mtime_date,
'properties': {}, 'no_cache': False}, result)
@ -256,7 +349,7 @@ class HttpImageServiceTestCase(base.TestCase):
}
result = self.service.show(self.href)
head_mock.assert_called_once_with(self.href, verify=True,
timeout=60)
timeout=60, auth=None)
self.assertEqual({
'size': 100,
'updated_at': datetime.datetime(2014, 11, 15, 8, 12, 31),
@ -280,7 +373,7 @@ class HttpImageServiceTestCase(base.TestCase):
self.assertRaises(exception.ImageRefValidationFailed,
self.service.show, self.href)
head_mock.assert_called_with(self.href, verify=True,
timeout=60)
timeout=60, auth=None)
@mock.patch.object(shutil, 'copyfileobj', autospec=True)
@mock.patch.object(requests, 'get', autospec=True)
@ -297,7 +390,7 @@ class HttpImageServiceTestCase(base.TestCase):
)
req_get_mock.assert_called_once_with(self.href, stream=True,
verify=True,
timeout=60)
timeout=60, auth=None)
@mock.patch.object(shutil, 'copyfileobj', autospec=True)
@mock.patch.object(requests, 'get', autospec=True)
@ -315,7 +408,42 @@ class HttpImageServiceTestCase(base.TestCase):
)
req_get_mock.assert_called_once_with(self.href, stream=True,
verify=False,
timeout=60)
timeout=60, auth=None)
@mock.patch.object(shutil, 'copyfileobj', autospec=True)
@mock.patch.object(requests, 'get', autospec=True)
def test_download_success_verify_false_basic_auth_sucess(
self, req_get_mock, shutil_mock):
cfg.CONF.set_override('webserver_verify_ca', 'False')
cfg.CONF.set_override('image_server_auth_strategy',
'http_basic',
'deploy')
cfg.CONF.set_override('image_server_user', 'test', 'deploy')
cfg.CONF.set_override('image_server_password', 'test', 'deploy')
user = cfg.CONF.deploy.image_server_user
password = cfg.CONF.deploy.image_server_password
auth_creds = requests.auth.HTTPBasicAuth(user, password)
response_mock = req_get_mock.return_value
response_mock.status_code = http_client.OK
response_mock.raw = mock.MagicMock(spec=io.BytesIO)
file_mock = mock.Mock(spec=io.BytesIO)
self.service.download(self.href, file_mock)
shutil_mock.assert_called_once_with(
response_mock.raw.__enter__(), file_mock,
image_service.IMAGE_CHUNK_SIZE
)
req_get_mock.assert_called_once_with(self.href, stream=True,
verify=False, timeout=60,
auth=auth_creds)
def test_download_success_verify_false_basic_auth_failed(self):
cfg.CONF.set_override('webserver_verify_ca', 'False')
cfg.CONF.set_override('image_server_auth_strategy',
'http_basic',
'deploy')
file_mock = mock.Mock(spec=io.BytesIO)
self.assertRaises(exception.ImageRefValidationFailed,
self.service.download, self.href, file_mock)
@mock.patch.object(shutil, 'copyfileobj', autospec=True)
@mock.patch.object(requests, 'get', autospec=True)
@ -333,7 +461,7 @@ class HttpImageServiceTestCase(base.TestCase):
)
req_get_mock.assert_called_once_with(self.href, stream=True,
verify=True,
timeout=60)
timeout=60, auth=None)
@mock.patch.object(shutil, 'copyfileobj', autospec=True)
@mock.patch.object(requests, 'get', autospec=True)
@ -351,7 +479,7 @@ class HttpImageServiceTestCase(base.TestCase):
)
req_get_mock.assert_called_once_with(self.href, stream=True,
verify='/some/path',
timeout=60)
timeout=60, auth=None)
@mock.patch.object(shutil, 'copyfileobj', autospec=True)
@mock.patch.object(requests, 'get', autospec=True)
@ -376,8 +504,8 @@ class HttpImageServiceTestCase(base.TestCase):
self.assertRaises(exception.ImageDownloadFailed,
self.service.download, self.href, file_mock)
req_get_mock.assert_called_once_with(self.href, stream=True,
verify=False,
timeout=60)
verify=False, timeout=60,
auth=None)
@mock.patch.object(shutil, 'copyfileobj', autospec=True)
@mock.patch.object(requests, 'get', autospec=True)
@ -394,7 +522,7 @@ class HttpImageServiceTestCase(base.TestCase):
self.service.download, self.href, file_mock)
req_get_mock.assert_called_once_with(self.href, stream=True,
verify='/some/path',
timeout=60)
timeout=60, auth=None)
@mock.patch.object(shutil, 'copyfileobj', autospec=True)
@mock.patch.object(requests, 'get', autospec=True)
@ -410,7 +538,7 @@ class HttpImageServiceTestCase(base.TestCase):
self.service.download, self.href, file_mock)
req_get_mock.assert_called_once_with(self.href, stream=True,
verify='/some/path',
timeout=60)
timeout=60, auth=None)
@mock.patch.object(shutil, 'copyfileobj', autospec=True)
@mock.patch.object(requests, 'get', autospec=True)
@ -426,7 +554,7 @@ class HttpImageServiceTestCase(base.TestCase):
self.service.download, self.href, file_mock)
req_get_mock.assert_called_once_with(self.href, stream=True,
verify='/some/path',
timeout=60)
timeout=60, auth=None)
@mock.patch.object(shutil, 'copyfileobj', autospec=True)
@mock.patch.object(requests, 'get', autospec=True)
@ -444,7 +572,7 @@ class HttpImageServiceTestCase(base.TestCase):
)
req_get_mock.assert_called_once_with(self.href, stream=True,
verify=True,
timeout=15)
timeout=15, auth=None)
class FileImageServiceTestCase(base.TestCase):

View File

@ -1259,6 +1259,33 @@ class TestAgentDeploy(CommonTestsMixin, db_base.DbTestCase):
'no_proxy': '.eggs.com'}
)
def test_wirte_image_basic_auth_success(self):
cfg.CONF.set_override('image_server_auth_strategy',
'http_basic',
'deploy')
cfg.CONF.set_override('image_server_user',
'SpongeBob',
'deploy')
cfg.CONF.set_override('image_server_password',
'SquarePants',
'deploy')
self._test_write_image(
additional_expected_image_info={
'image_server_auth_strategy': 'http_basic',
'image_server_user': 'SpongeBob',
'image_server_password': 'SquarePants'
}
)
def test_wirte_image_basic_auth_success_blocked(self):
cfg.CONF.set_override('image_server_user',
'SpongeBob',
'deploy')
cfg.CONF.set_override('image_server_password',
'SquarePants',
'deploy')
self._test_write_image()
def test_write_image_with_no_proxy_without_proxies(self):
self._test_write_image(
additional_driver_info={'image_no_proxy': '.eggs.com'}

View File

@ -0,0 +1,32 @@
---
features:
- |
Introducing basic authentication and configurable authentication strategy
support for image and image checksum download processes. This feature
introduces 3 new configuration variables that could be used to select
the authentication strategy and provide credentials for authentication
strategies. The 3 variables are structured in way that 1 of them
``[deploy]image_server_auth_strategy`` (string) provides the ability to
select between authentication strategies by specifying the name of the
authentication strategy.
Currently the only supported authentication strategy is the ``http-basic``
which will make IPA use HTTP(S) basic authentication also known as the
``RFC 7617`` standard. The other 2 variables are
``[deploy]image_server_password`` and ``[deploy]image_server_user``
provide username and password credentials for image download processes. The
``[deploy]image_server_password`` and ``[deploy]image_server_user``
are not strategy specific and could be reused for any username + password
based authentication strategy, but for the moment these 2 variables are
only used for the ``http-basic`` strategy.
``[deploy]image_server_auth_strategy`` doesn't just enable the feature but
enforces checks on the values of the 2 related credentials. When the
``http-basic`` strategy is enabled for image server download workflow the
download logic will make sure to raise an exception in case any of the
credentials are None or an empty string.
Example of activating the ``http-basic`` strategy can be found in
`HTTP(s) Authentication strategy for user image servers` section of the
admin guide.