Obliterate all references to the aforementioned service. This mostly
consists of removing the core service and any references to the now
removed '[workarounds] enable_consoleauth' configuration option.
Part of blueprint remove-consoleauth
Change-Id: I0498599fd636aa9e30df932f0d893db5efa23260
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Depends-On: Icfc175c49a1fc650d1c9ad06b77209a70c6386db
There were a lot of workarounds here to ensure we didn't switch to the
new model (vs. the old 'nova-consoleauth' service) if users were on
cells v1. These can go now, along with the old 'nova-consoleauth'
service (though that's a later, separate change).
Part of blueprint remove-cells-v1
Change-Id: I1b8f411b050d34e4e77e9a4f1e613135eb5f74b7
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
If you use redis as a caching backend and you delete a server with
no consoleauth tokens you'll get a stacktrace as an empty list is
passed down to the redis client and ultimately the redis server
which responds with an error, complaining about a wrong number
of arguments for the del command.
The code now checks if the list of tokens is empty and only calls
the caching backend if there are tokens available to delete.
This also may improve performance, as it no longer hands down an
empty list.
Closes-Bug: #1794812
Change-Id: Iffdd4e251bfa2bac1bfd49498e32b738843709de
The i18n team has decided not to translate the logs because it
seems like it not very useful; operators prefer to have them in
English so that they can search for those strings on the internet.
Partially fix on nova/conductor, nova/console, nova/consoleauth,
nova/db and nova/image other paths will be fixed on next commits
Change-Id: I132f60cb4743f85c96ec5231d6f860cfb0815758
All RPC topic opts were deprecated in Ocata.
This patch removes all *_topic opts from the code.
Change-Id: I41a6be48380999c49d6db3ee5221174a41014002
Implements: blueprint centralize-config-options-pike
When target_cell was originally written, the intent was to yield the
context that should be used. It currently mutates the input context,
which means you don't have to actually use the yielded one, because
we have a lot of stuff that would break otherwise. This fixes all the
current uses of it to be proper, and adjusts tests accordingly. This
is separate from changing target_cell's behavior to not mutate the
input context specifically to isolate the mechanical changes from ones
that actually need different behavior.
In addition to code that was already using target_cell() but not
depending on the yielded context, the _create_block_device_mapping()
method in conductor/manager.py was still depending on the shared
context switching to target the BDM objects on create. Since these
were prepared with the context prior to having determined where the
instanace was going to end up, we need to explicitly target the
object context on create (like the rest of the boot workflow does
for other objects in schedule_and_build_instances()).
Related to blueprint cells-aware-api
Change-Id: I35206e665f2c81531a2269dd66f8c5c0df834245
Since we look up the instance and call to its cell in consoleauth,
we need to look up the InstanceMapping and properly target the
database and rpc operation.
Related to blueprint cells-aware-api
Change-Id: I80013fa59b221f70376d6e1d4080ca699ff6caeb
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
1) It checks that person who want to trace is trusted and knows
secret HMAC key.
2) It starts tracing in case of proper trace headers
and adds the first wsgi trace point with info about the HTTP request
* Add initialization of osprofiler on start of a service
Currently that includes oslo.messaging notifier instance creation
to send Ceilometer backend notifications.
oslo-spec: https://review.openstack.org/#/c/103825/
python-novaclient change: https://review.openstack.org/#/c/254699/
based on: https://review.openstack.org/#/c/105096/
Co-Authored-By: Boris Pavlovic <boris@pavlovic.me>
Co-Authored-By: Munoz, Obed N <obed.n.munoz@intel.com>
Co-Authored-By: Roman Podoliaka <rpodolyaka@mirantis.com>
Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com>
Implements: blueprint osprofiler-support-in-nova
Change-Id: I82d2badc8c1fcec27c3fce7c3c20e0f3b76414f1
This adds upgrade level aliases for newton for all of our services,
including the ones we expect to remove soon.
Change-Id: If97f8f87f3d8b4db8f6b6b826c5c1f5d0b3f10a8
Rename them to remove unnecessary prefixes. This allows us to remove
the final TODO.
Change-Id: Iede35bbbf0124cfc7521cb91dda323b4d425e630
Implements: bp centralize-config-options-ocata
With the introduction of oslo_cache, there is no feedback anymore about
whether set()ting a cache item was successful, so we stop generating
warnings about this all the time.
Change-Id: I2ca0cdd3dd30498a23da2ef4f352afd199496862
Partial-Bug: 1572062
Move config options of nova.conf section "upgrade_levels" to new
centralized location nova/conf/upgrade_levels.
Also, previously some config options for "upgrade_levels" were scattered
in different conf files. Moved these options to the central
location: nova/conf/upgrade_levels.
Change-Id: I45651e8527dd98635f25e9586f68c288203d8cf5
Implements: blueprint centralize-config-options-newton
The config options of the section
"nova/consoleauth" got moved to the new central
location "nova/conf/consoleauth.py"
Change-Id: Ia9d875b62cc97065e102aeb17c19bdc074e35208
Implements: blueprint centralize-config-options-newton
Per our usual procedure, this sets version aliases for the current RPC
versions of all our internal RPC APIs for mitaka.
Change-Id: I455f3b80a9b64dfea42ee1c74685d11ae0178513
After centralizing cells options, importing nova.conf module is
enough for using cells options. import_opt(<opt_name>, <module>)
functions are not necessary.
Blueprint centralize-config-options
Change-Id: I9aabe8f2cc337c1237c2a73475cc1a3d3f4a1575
Common memorycache was replaced by analogous tool
from oslo.cache lib. In-memory cache was replaced
by oslo.cache.dict backend. Memcached was replaced
by dogpile.cache.memcached backend.
Implements blueprint oslo-for-mitaka
Closes-Bug: #1483322
Co-Authored-By: Sergey Nikitin <snikitin@mirantis.com>
Co-Authored-By: Pavel Kholkin <pkholkin@mirantis.com>
Change-Id: I371f7a68e6a6c1c4cd101f61b9ad96c15187a80e
This adds SIGHUP handlers to console, consoleauth, and conductor
services to reload the compute_rpcapi. This is required for auto
version pinning to be dynamic at runtime.
Related to blueprint service-version-behavior
Change-Id: I83a25bda681daa249ffca8bc1f38edcd70dd7cc5
Update all of the rpc client API classes to include a version alias
for the latest version implemented in Liberty. This alias is needed when
doing rolling upgrades from Liberty to Mitaka. With this in place, you can
ensure all services only send messages that both Liberty and Mitaka will
understand.
Change-Id: I3eabae03483806602d7dea0e4ab737db4917d0fd
Update all of the rpc client API classes to include a version alias
for the latest version implemented in Kilo. This alias is needed when
doing rolling upgrades from Kilo to Liberty. With this in place, you can
ensure all services only send messages that both Kilo and Liberty will
understand.
Change-Id: I2952aec9aae747639aa519af55fb5fa25b8f3ab4
Closes-Bug: #1444745
Related-bug: 1409142
As part of the fix for the related bug - we've added protocol checking
to mitigate MITM attacks, however we base protocol checking on a config
option that is normally only intended for compute hosts.
This is quite user hostile, as it is now important that all nodes
running compute and proxy services have this option in sync.
We can do better than that - we can persist the URL the client is
expected to use, and once we get it back on token validation, we can
make sure that the request is using the intended protocol, mitigating
the MITM injected script attacks.
This patch makes sure that the access_url is persisted with the token -
the follow-up patch makes consoles use that info.
Change-Id: I02a377f54de46536ca35413b615d3298967afc33
Mutating a list while iterating over it is a big python no-no. Let's not
do it in consoleauth service anymore.
Change-Id: I50d201f3f39576d7a32b722ead0667fd9abaeb75
Convert the use of the incubated version of the log module
to the new oslo.log library.
Sync oslo-incubator modules to update their imports as well.
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
Change-Id: Ic4932e3f58191869c30bd07a010a6e9fdcb2a12c
oslo.log does not have AUDIT level. This was a conscious
decision documented in blueprint:
https://blueprints.launchpad.net/oslo.log/+spec/remove-context-adapter
which was implemented in:
I10240f8af6c42508124659b3ed62c5ab93552953
So to prep for switching to oslo.log, this changeset removes
usage of LOG.audit and replaces it with LOG.info
Change-Id: I9cb1293529b2079b8a4778e27d60c6c760dfb622
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
This commit includes:
- using oslo_utils instead of oslo.utils
- using oslo_serialization instead of oslo.serialization
- using oslo_db instead of oslo.db
- using oslo_i18n instead of oslo.i18n
- using oslo_middleware instead of oslo.middleware
- using oslo_config instead of oslo.config
- using oslo_messaging instead of "from oslo import messaging"
- using oslo_vmware instead of oslo.vmware
Change-Id: I3e2eb147b321ce3e928817b62abcb7d023c5f13f
oslo.i18n uses different marker functions to separate the
translatable messages into different catalogs, which the translation
teams can prioritize translating. For details, please refer to:
http://docs.openstack.org/developer/oslo.i18n/guidelines.html#guidelines-for-use-in-openstack
There were not marker fuctions some places in directory network.
This commit makes changes:
* Add missing marker functions
* Use ',' instead of '%' while adding variables to log messages
Added a hacking rule for the warning about checking
translation for it and checking logging level `warning` instead
alias `warn`.
Change-Id: I2bced49dc5a0408a94d5d20d85b20c682886edbe
Since we only support upgrading from N-1 to N, there's no need to keep
around this older upgrade related code.
Change-Id: I253db2fe161add62cb248665c7920126f0d7455f
Very simple import change in each file, just touches a
whole lot of files. jsonutils has graduated into a stand alone
library and has been removed from the oslo-incubator repository.
We should be using the library for all projects.
Change-Id: I09d780a47d377871f357654603ee4b0fa0169d66
Update all of the rpc client API classes to include a version alias
for the latest version implemented in Juno. This alias is needed when
doing rolling upgrades from Juno to Kilo. With this in place, you can
ensure all services only send messages that both Juno and Kilo will
understand.
Change-Id: Ia81538130bf8530b70b5f55c7a3d565903ff54b4
oslo.i18n provides the i18n functions that were provided by
oslo-incubator's gettextutils module. Some tests that were
using internal details of the library were removed.
Change-Id: I44cfd5552e0dd86af21073419d31622f5fdb28e0
This switches the remaining occurences of
nova.objects.instance.Instance* to nova.objects.Instance*.
Partial-Blueprint: object-subclassing
Change-Id: I3017c149cc9fbc6b1fbecb003dc55455b1dcd12c
LOG a warning message if the key not memcached successful. This will
happen if memcached module is used but the memcached servers are not
reachable.
Change-Id: Ia3c51fe8fc82060586a014f5cb1083d5695ab455
Closes-bug: 1213660
Now that Icehouse has been branched, add aliases for the rpc version
implemented by Icehouse. These aliases can be used in the
configuration options for doing outbound rpc version control.
DocImpact - 'icehouse' is now a valid value to provide to all of the
options in the [upgrade_levels] section of nova.conf.
Closes-Bug: #1309334
Partial-Bug: #1309324
Change-Id: Iad96d8ba7e8e85346fe0b3275f220314bf76a7e3
Moves the consoleauth_manager option into nova.service like the other
manager options in commit 39ce4032.
The thinking for having it in nova.service is that's where
CONF.get('%_manager'...) is called. It also makes no sense for the
option to be declared in nova.consoleauth.manager because if you change
this config option, then you don't want nova.consoleauth.manager loaded.
Closes-Bug: #1276398
Change-Id: I85e089239228920e9e58284cf6ff52e43bf85ab0
The consoleauth manager looks up and passes a SQLite model to the
compute_rpcapi.validate_console_port() method, which is expecting
an Instance object now. This fixes that by replacing the direct call
with an object lookup.
Change-Id: I48a63805e4691899396559f48b29bef3030d5ef8
We don't need to have the vi modelines in each source file,
it can be set in a user's vimrc if required.
Also a check is added to hacking to detect if they are re-added.
Change-Id: I347307a5145b2760c69085b6ca850d6a9137ffc6
Closes-Bug: #1229324
The oslo.messaging library takes the existing RPC code from oslo and
wraps it in a sane API with well defined semantics around which we can
make a commitment to retain compatibility in future.
The patch is large, but the changes can be summarized as:
* oslo.messaging>=1.3.0a4 is required; a proper 1.3.0 release will be
pushed before the icehouse release candidates.
* The new rpc module has init() and cleanup() methods which manage the
global oslo.messaging transport state. The TRANSPORT and NOTIFIER
globals are conceptually similar to the current RPCIMPL global,
except we're free to create and use alternate Transport objects
in e.g. the cells code.
* The rpc.get_{client,server,notifier}() methods are just helpers
which wrap the global messaging state, specifiy serializers and
specify the use of the eventlet executor.
* In oslo.messaging, a request context is expected to be a dict so
we add a RequestContextSerializer which can serialize to and from
dicts using RequestContext.{to,from}_dict()
* The allowed_rpc_exception_modules configuration option is replaced
by an allowed_remote_exmods get_transport() parameter. This is not
something that users ever need to configure, but it is something
each project using oslo.messaging needs to be able to customize.
* The nova.rpcclient module is removed; it was only a helper class
to allow us split a lot of the more tedious changes out of this
patch.
* Finalizing the port from RpcProxy to RPCClient is straightforward.
We put the default topic, version and namespace into a Target and
contstruct the client using that.
* Porting endpoint classes (like ComputeManager) just involves setting
a target attribute on the class.
* The @client_exceptions() decorator has been renamed to
@expected_exceptions since it's used on the server side to designate
exceptions we expect the decorated method to raise.
* We maintain a global NOTIFIER object and create specializations of
it with specific publisher IDs in order to avoid notification driver
loading overhead.
* rpc.py contains transport aliases for backwards compatibility
purposes. setup.cfg also contains notification driver aliases for
backwards compat.
* The messaging options are moved about in nova.conf.sample because
the options are advertised via a oslo.config.opts entry point and
picked up by the generator.
* We use messaging.ConfFixture in tests to override oslo.messaging
config options, rather than making assumptions about the options
registered by the library.
The porting of cells code is particularly tricky:
* messaging.TransportURL parse() and str() replaces the
[un]parse_transport_url() methods. Note the complication that an
oslo.messaging transport URL can actually have multiple hosts in
order to support message broker clustering. Also the complication
of transport aliases in rpc.get_transport_url().
* proxy_rpc_to_manager() is fairly nasty. Right now, we're proxying
the on-the-wire message format over this call, but you can't supply
such messages to oslo.messaging's cast()/call() methods. Rather than
change the inter-cell RPC API to suit oslo.messaging, we instead
just unpack the topic, server, method and args from the message on
the remote side.
cells_api.RPCClientCellsProxy is a mock RPCClient implementation
which allows us to wrap up a RPC in the message format currently
used for inter-cell RPCs.
* Similarly, proxy_rpc_to_manager uses the on-the-wire format for
exception serialization, but this format is an implementation detail
of oslo.messaging's transport drivers. So, we need to duplicate the
exception serialization code in cells.messaging. We may find a way
to reconcile this in future - for example a ExceptionSerializer
class might work, but with the current format it might be difficult
for the deserializer to generically detect a serialized exception.
* CellsRPCDriver.start_servers() and InterCellRPCAPI._get_client()
need close review, but they're pretty straightforward ports of code
to listen on some specialized topics and connect to a remote cell
using its transport URL.
blueprint: oslo-messaging
Change-Id: Ib613e6300f2c215be90f924afbd223a3da053a69
These were initially noted during import into the manuals at
https://review.openstack.org/67502
Fixes include rework of text, missing or extra spaces, wrong
capitalization and no sentence style capitalization.
A hacking check has been added that tests that all config options
start with a capital letter.
Change-Id: I0a1c333bbb3a812932d3972a7403f94da0bff341
Co-Authored-By: Daniel Berrange <berrange@redhat.com>
Co-Authored-By: Diane Fleming <diane.fleming@rackspace.com>
A previous change adding support for the 2.x rpc interface but retained
compatibility with 1.x as a transition point. This commit removes the
old API from the server side.
UpgradeImpact
Part of blueprint rpc-major-version-updates-icehouse
Change-Id: I523c3fd605352cb83463a6040ba1f6eb30630810
48dd520958 added v2 of the consoleauth rpc
API and made the client side use it. However, I forgot to leave in
support for talking the havana API on the client side. This is needed
for rolling upgrade support.
Related to bleuprint rpc-major-version-updates-icehouse
Change-Id: I30f5651b3f83dcc25639a01513987798a6cdcbb8
In 48dd520958 support for the 2.0
consoleauth rpc API was added which missed the return statement in the
check_token method. Because of this all the components that had to
verify a token against consoleauth component would fail to do so.
This patch fixes the check_token method and extends the tests to cover
the new v2.0 class.
Closes-Bug: #1244220
Change-Id: I4e5186943d28609083f723f4b43ab6f0ecbffd27
Add support for the 2.0 consoleauth rpc API. This commit retains
compatibility with the older 1.X API to allow continuous deployment
without any downtime.
UpgradeImpact - Deployments doing continuous deployment should follow
this process to upgrade without any downtime with the consoleauth
service:
1) Sync all consoleauth services to this commit, so that they handle
both new and old messages.
2) Upgrade everything else to this commit so that all clients start
sending the new messages.
3) Upgrade past this, where support for the old messages will be
dropped.
Part of blueprint rpc-major-version-updates-icehouse
Change-Id: Iad71d3c2801f9355968e69bce3308b84f922b239
Now that Havana has been branched, add aliases for the rpc version
implemented by Havana. These aliases can be used in the configuration
options for doing outbound rpc version control.
This is one of the items that should be done after every release, and
has been added to the new release checklist wiki page:
https://wiki.openstack.org/wiki/Nova/ReleaseChecklist
DocImpact - 'havana' is now a valid value to provide to all of the
options in the [upgrade_levels] section of nova.conf.
Change-Id: Ifa0b3dcc00843ad3f045a450ceb0f47f0b843f98
Add a temporary nova.rpcclient.RPCClient helper class which translates
oslo.messaging.rpc.RPCClient compatible calls into calls on a RpcProxy
object.
Use this new class to port all of the rpcapi modules over to the new
RPCClient so that the final port of Nova over to oslo.messaging will be
smaller and easier to review.
This patch contains no functional changes at all, except that all client
side RPCs go through this temporary helper class.
blueprint: oslo-messaging
Change-Id: Iee86c36bcc474a604993618b8a2255af8c3d2f48
Instance console auth tokens never get deleted from the cache
before the instnace is deleted, this is a waste of memory. now
we check if tokens are expired then remove them from the cache
before store new tokens.
Fix bug #1209134
Co-authored-by: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Change-Id: I8dd5089ebaed3b3d91932f1f1558bbe302cd5675
Stephen Finucane