Allow run metadata api per cell

Adds configuration option ``[api]/local_metadata_per_cell``
to allow user run Nova metadata API service per cell. Doing
this can avoid query API DB for instance information each
time an instance query for its metadata.

Implements blueprint run-meta-api-per-cell

Change-Id: I2e6ebb551e782e8aa0ac90169f4d4b8895311b3c
This commit is contained in:
Kevin_Zheng 2018-12-12 17:08:54 +08:00 committed by Matt Riedemann
parent a1c01f97ae
commit e2e372b2b1
7 changed files with 102 additions and 14 deletions

View File

@ -22,7 +22,9 @@ Description
===========
:program:`nova-api-metadata` is a server daemon that serves the Nova Metadata
API.
API. This daemon routes database requests via the ``nova-conductor`` service,
so there are some considerations about using this in a
:ref:`multi-cell layout <cells-v2-layout-metadata-api>`.
Options
=======

View File

@ -291,14 +291,35 @@ documentation
<configuration/opts.html#oslo_messaging_notifications.transport_url>` for more
details.
.. _cells-v2-layout-metadata-api:
Nova Metadata API service
~~~~~~~~~~~~~~~~~~~~~~~~~
The Nova metadata API service should be global across all cells, and
thus be configured as an API-level service with access to the
``[api_database]/connection`` information. The nova metadata API service must
not be run as a standalone service (e.g. must not be run via the
nova-api-metadata script).
Starting from the Stein release, the Nova Metadata API service
can be run either globally or per cell using the
:oslo.config:option:`api.local_metadata_per_cell` configuration option.
**Global**
If you have networks that span cells, you might need to run Nova metadata API
globally. When running globally, it should be configured as an API-level
service with access to the :oslo.config:option:`api_database.connection`
information. The nova metadata API service must not be run as a standalone
service in this case (e.g. must not be run via the nova-api-metadata script).
**Local per cell**
Running Nova metadata API per cell can have better performance and data
isolation in a muli-cell deployment. If your networks are segmented along
cell boundaries, then you can run Nova metadata API service per cell. If
you choose to run it per cell, you should also configure each
`Neutron metadata-agent`_ to point to the corresponding nova-metadata-api.
The nova metadata API service must be run as a standalone service in this
case (e.g. must be run via the nova-api-metadata script).
.. _Neutron metadata-agent: https://docs.openstack.org/neutron/latest/configuration/metadata-agent.html?#DEFAULT.nova_metadata_host
Consoleauth service and console proxies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

View File

@ -680,6 +680,12 @@ def get_metadata_by_instance_id(instance_id, address, ctxt=None):
'metadata', 'system_metadata',
'security_groups', 'keypairs',
'device_metadata']
if CONF.api.local_metadata_per_cell:
instance = objects.Instance.get_by_uuid(ctxt, instance_id,
expected_attrs=attrs)
return InstanceMetadata(instance, address)
try:
im = objects.InstanceMapping.get_by_instance_uuid(ctxt, instance_id)
except exception.InstanceMappingNotFound:

View File

@ -17,6 +17,7 @@ import six
from six.moves import range
from webob import exc
import nova.conf
from nova import context
from nova import exception
from nova.i18n import _
@ -24,6 +25,8 @@ from nova import objects
from nova import utils
CONF = nova.conf.CONF
CHUNKS = 4
CHUNK_LENGTH = 255
MAX_SIZE = CHUNKS * CHUNK_LENGTH
@ -68,12 +71,18 @@ def handle_password(req, meta_data):
msg = _("Request is too large.")
raise exc.HTTPBadRequest(explanation=msg)
im = objects.InstanceMapping.get_by_instance_uuid(ctxt, meta_data.uuid)
with context.target_cell(ctxt, im.cell_mapping) as cctxt:
try:
instance = objects.Instance.get_by_uuid(cctxt, meta_data.uuid)
except exception.InstanceNotFound as e:
raise exc.HTTPBadRequest(explanation=e.format_message())
if CONF.api.local_metadata_per_cell:
instance = objects.Instance.get_by_uuid(ctxt, meta_data.uuid)
else:
im = objects.InstanceMapping.get_by_instance_uuid(
ctxt, meta_data.uuid)
with context.target_cell(ctxt, im.cell_mapping) as cctxt:
try:
instance = objects.Instance.get_by_uuid(
cctxt, meta_data.uuid)
except exception.InstanceNotFound as e:
raise exc.HTTPBadRequest(explanation=e.format_message())
instance.system_metadata.update(convert_password(ctxt, req.body))
instance.save()
else:

View File

@ -195,6 +195,19 @@ metadata caching is disabled entirely; this is generally not recommended for
performance reasons. Increasing this setting should improve response times
of the metadata API when under heavy load. Higher values may increase memory
usage, and result in longer times for host metadata changes to take effect.
"""),
cfg.BoolOpt("local_metadata_per_cell",
default=False,
help="""
Indicates that the nova-metadata API service has been deployed per-cell, so
that we can have better performance and data isolation in a multi-cell
deployment. Users should consider the use of this configuration depending on
how neutron is setup. If you have networks that span cells, you might need to
run nova-metadata API service globally. If your networks are segmented along
cell boundaries, then you can run nova-metadata API service per cell. When
running nova-metadata API service per cell, you should also configure each
Neutron metadata-agent to point to the corresponding nova-metadata API
service.
"""),
]

View File

@ -1617,6 +1617,23 @@ class MetadataHandlerTestCase(test.TestCase):
mock_get_im.assert_called_once_with(ctxt, 'foo')
imd.assert_called_once_with(inst, 'bar')
@mock.patch.object(objects.Instance, 'get_by_uuid')
@mock.patch.object(objects.InstanceMapping, 'get_by_instance_uuid')
def test_get_metadata_by_instance_id_with_local_meta(self, mock_get_im,
mock_get_inst):
# Test that if local_metadata_per_cell is set to True, we don't
# query API DB for instance mapping.
self.flags(local_metadata_per_cell=True, group='api')
ctxt = context.RequestContext()
inst = objects.Instance()
mock_get_inst.return_value = inst
with mock.patch.object(base, 'InstanceMetadata') as imd:
base.get_metadata_by_instance_id('foo', 'bar', ctxt=ctxt)
mock_get_im.assert_not_called()
imd.assert_called_once_with(inst, 'bar')
class MetadataPasswordTestCase(test.TestCase):
def setUp(self):
@ -1655,7 +1672,10 @@ class MetadataPasswordTestCase(test.TestCase):
@mock.patch('nova.objects.InstanceMapping.get_by_instance_uuid')
@mock.patch('nova.objects.Instance.get_by_uuid')
def _try_set_password(self, get_by_uuid, get_mapping, val=b'bar'):
def _try_set_password(self, get_by_uuid, get_mapping, val=b'bar',
use_local_meta=False):
if use_local_meta:
self.flags(local_metadata_per_cell=True, group='api')
request = webob.Request.blank('')
request.method = 'POST'
request.body = val
@ -1667,12 +1687,19 @@ class MetadataPasswordTestCase(test.TestCase):
save.assert_called_once_with()
self.assertIn('password_0', self.instance.system_metadata)
get_mapping.assert_called_once_with(mock.ANY, self.instance.uuid)
if use_local_meta:
get_mapping.assert_not_called()
else:
get_mapping.assert_called_once_with(mock.ANY, self.instance.uuid)
def test_set_password(self):
self.mdinst.password = ''
self._try_set_password()
def test_set_password_local_meta(self):
self.mdinst.password = ''
self._try_set_password(use_local_meta=True)
def test_conflict(self):
self.mdinst.password = 'foo'
self.assertRaises(webob.exc.HTTPConflict,

View File

@ -0,0 +1,10 @@
---
features:
- |
Added configuration option ``[api]/local_metadata_per_cell`` to allow
users to run Nova metadata API service per cell. Doing this could provide
performance improvement and data isolation in a multi-cell deployment.
But it has some caveats, see the
`Metadata api service in cells v2 layout`_ for more details.
.. _Metadata api service in cells v2 layout: https://docs.openstack.org/nova/latest/user/cellsv2-layout.html#nova-metadata-api-service