This change adds a simple [cinder]debug configurable to allow
cinderclient and os_brick to be made to log at DEBUG independently of
the rest of Nova.
Change-Id: I84f5b73adddf42831f1d9e129c25bf955e6eda78
Introduce a new 'nova.db.api.api' module to hold API database-specific
helpers, plus a generic 'nova.db.utils' module to hold code suitable for
both main and API databases. This highlights a level of complexity
around connection management that is present for the main database but
not for the API database. This is because we need to handle the
complexity of cells for the former but not the latter.
Change-Id: Ia5304c552ce552ae3c5223a2bfb3a9cd543ec57c
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Merge these, removing an unnecessary layer of abstraction, and place
them in the new 'nova.db.main' directory. The resulting change is huge,
but it's mainly the result of 's/sqlalchemy import api/main import api/'
and 's/nova.db.api/nova.db.main.api/' with some necessary cleanup. We
also need to rework how we do the blocking of API calls since we no
longer have a 'DBAPI' object that we can monkey patch as we were doing
before. This is now done via a global variable that is set by the 'main'
function of 'nova.cmd.compute'.
The main impact of this change is that it's no longer possible to set
'[database] use_db_reconnect' and have all APIs automatically wrapped in
a DB retry. Seeing as this behavior is experimental, isn't applied to
any of the API DB methods (which don't use oslo.db's 'DBAPI' helper),
and is used explicitly in what would appear to be the critical cases
(via the explicit 'oslo_db.api.wrap_db_retry' decorator), this doesn't
seem like a huge loss.
Change-Id: Iad2e4da4546b80a016e477577d23accb2606a6e4
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
There are multiple places where nova set the new default
value for other (than nova) namespace config option for
example oslo lib config options. Examples:
- 20572809f2/nova/config.py (L62)
- 20572809f2/nova/policy.py (L48)
and these defaults are reflected in code also which is working
as expected. But when config sample file is generated via
oslo-config-generator tool (tox -egenconfig) then these defaults
which are set by Nova are not reflected as this tool take the
raw defaults. To solve this issue oslo config provide a option
to add hook to reflect the new default in config generator
- https://docs.openstack.org/oslo.config/latest/cli/generator.html#modifying-defaults-from-other-namespaces
We already doing it for middleware cors option
- 20572809f2/setup.cfg (L41)
Fixing existing default value and if we find more or any future
change for default value, we need to add those config generator hook.
Change-Id: Ib0e926aec069dfb4ffd1bbfe8506d62464511b51
Closes-Bug: #1904250
This change removes support for supressing the
heartbeat message from olso.messaging before 9.8.0
Change-Id: I2035d5df31e43b730cd472cc438ec863bb538d62
Related-Bug: #1825584
When the nova api is executing under uWSGI or MOD_WSGI
the lifetime of the amqp heartbeat thread is controlled
by the wsgi server. As a result when the nova api is run
in this configuration we expect that the heartbeat thread
will be suspended and heartbeats will be missed when the wsgi
server suspends execution of the wsgi application.
This change adds a python logging filter to suppress
the reporting of heartbeat warnings as this behavior is
expected. Since the operator cannot do anything to address
the issue the warning is just noise and many operators
and customers find it to be off-putting.
Change-Id: I642b1e3ed6de2be4dcc19fe214f84095d2e1d31a
Closes-Bug: #1825584
There was only one sub-module in 'nova.common' but its presence
interferes with tab complete for the majority of people that want
'nova/compute'. Just move that sub-module to its own top-level module
and be done.
Change-Id: Iee886d915577f347e1ee4f54133ae0f87ae75841
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This finalizes the removal of the placement code from nova.
This change primarily removes code and makes fixes to cmd,
test and migration tooling to adapt to the removal.
Placement tests and documention were already removed in
early patches.
A database migration that calls
consumer_obj.create_incomplete_consumers in nova-manage has been
removed.
A functional test which confirms the default incomplete
consumer user and project id has been changes so its its use of
conf.placement.incomplete_* (now removed) is replaced with a
constant. The placement server, running in the functional
test, provides its own config.
placement-related configuration is updated to only register those
opts which are relevant on the nova side. This mostly means
ksa-related opts. placement-database configuration is removed
from nova/conf/database.
tox.ini is updated to remove the group_regex required by the
placement gabbi tests. This should probably have gone when the
placement functional tests went, but was overlooked.
A release note is added which describes that this is cleanup,
the main action already happened, but points people to the
nova to placement upgrade instructions in case they haven't
done it yet.
Change-Id: I4181f39dea7eb10b84e6f5057938767b3e422aff
The create_incomplete_consumers online data migration was added in
Rocky via change Id609789ef6b4a4c745550cde80dd49cabe03869a. That
relies on hitting some tables in the API (or placement) database.
The DB API code for that migration relies on a placement context
manager which looks like it was regressed with change
I2fff528060ec52a4a2e26a6484bdf18359b95f77 (also in Rocky). This
results in a DB error trying to query the projects table but
because of a generic try/except in _run_migration, the failure
was missed in CI testing.
Similarly, the nova-status upgrade check "_check_resource_providers"
routine also uses the placement DB API context manager to count the
number of compute resource providers in the API (or placement) DB,
which is returning 0 because it's not using the proper DB connection.
This was not caught in the nova-status CLI tests because they use
the DatabaseFixture which *does* configure the global placement DB
API context manager.
This adds the configuration of the global placement DB API context
manager so we can properly query the placement-related tables.
The blanket problematic try/except from _run_migration is left
as-is in this change but will be addressed in a separate patch.
Integration testing of this fix is being performed with devstack:
https://review.openstack.org/599847/
Change-Id: I9d97b7a904e2b7d15c763e2a067cc5909cc6c9c5
Closes-Bug: #1790701
Closes-Bug: #1790721
This avoids us logging the output of potentially-sensitive operations
performed with privilege.
Change-Id: I706fbd883f7640bebb3ba0b5848b863a157a9794
Closes-Bug: #1784062
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
1) It checks that person who want to trace is trusted and knows
secret HMAC key.
2) It starts tracing in case of proper trace headers
and adds the first wsgi trace point with info about the HTTP request
* Add initialization of osprofiler on start of a service
Currently that includes oslo.messaging notifier instance creation
to send Ceilometer backend notifications.
oslo-spec: https://review.openstack.org/#/c/103825/
python-novaclient change: https://review.openstack.org/#/c/254699/
based on: https://review.openstack.org/#/c/105096/
Co-Authored-By: Boris Pavlovic <boris@pavlovic.me>
Co-Authored-By: Munoz, Obed N <obed.n.munoz@intel.com>
Co-Authored-By: Roman Podoliaka <rpodolyaka@mirantis.com>
Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com>
Implements: blueprint osprofiler-support-in-nova
Change-Id: I82d2badc8c1fcec27c3fce7c3c20e0f3b76414f1
It's currently not possible to configure debug logging
for glanceclient, it's hard-coded to warning level. It
would be great to be able to just change config and
restart nova-compute, for example, when needing to debug
issues with glance.
Change-Id: I964e40085b68561af48b476c82288fc84d02bcc4
Currently markus_z have centralized database section in
nova/conf/database.py. But in nova/config.py, there is a
part of database section that set the default options of
oslo_db. The set_default_opt include register_opt and should be
moved to centralized place.
Change-Id: Ic94991b009dfaa076a5f285b43018e473e9e074f
Implements: blueprint centralize-config-options-newton
The config options of the "nova.conf" section "cache" got
moved to the new central location "nova/conf/cache.py".
Change-Id: I1b7aebb9f7d44ff8312b98c88093bf01f44dfa58
Implements: blueprint centralize-config-options-newton
This change moves all of the configuration options previously defined
in "nova/paths" to the new centralized "nova/conf" directory.
A subsequent patch will then improve the help texts.
As the helper functions to create specific directories are only used
for the default values of other config options, they got moved too.
The default value of the config option "pybasedir" gets determined by
the directory which contains it. The old directory was "nova/" and
new directory is "nova/conf/". That made it necessary to move one
additional directory upwards with "../../" to preserve the correct
default value.
bp centralize-config-options-newton
Change-Id: I65fc8d83da660e131e89ce5849e7535e969adcb2
The config options of the "nova/debugger" got
moved to the new central location
"nova/conf/remote_debug.py"
Implements: blueprint centralize-config-options-newton
Change-Id: Ic0c858aac92c95dc20992c06218f9823e7e683e5
Simplified the definition of Nova specific log levels by extending the
oslo.log standard levels used of all projects. This olso.log default
includes all existing values including suds=INFO.
* Removed logging context format as this matches oslo.log default.
* Reordered log initiation calls to match documented order.
See http://docs.openstack.org/developer/oslo.log/usage.html
Change-Id: Ibb4bf8b50419a578db49953e2fc69f45c5e3edb3
This patch makes use of oslo.config's genconfig hooks to add
nova-specific default config values to the generated configuration.
It also adds this same method to nova's configuration parsing logic,
to ensure that these default values are also the ones used at runtime.
Closes-bug: 1551836
Change-Id: Ic799aea89f681ef8626e996f0357ade388f76ca9
setUp calls rpc.init() twice for each test, once in each of:
self.useFixture(conf_fixture.ConfFixture(CONF))
self.useFixture(nova_fixtures.RPCFixture('nova.test'))
Stop calling rpc.init() in ConfFixture, and defer to RPCFixture for
initialization.
The calls to rpc.init() in ConfFixture were taking 0.01 seconds each.
0.01 seconds * 14784 tests = 2.5 minutes
Change-Id: I710399b9b9286884f3ed71522d2697680951c20e
Common memorycache was replaced by analogous tool
from oslo.cache lib. In-memory cache was replaced
by oslo.cache.dict backend. Memcached was replaced
by dogpile.cache.memcached backend.
Implements blueprint oslo-for-mitaka
Closes-Bug: #1483322
Co-Authored-By: Sergey Nikitin <snikitin@mirantis.com>
Co-Authored-By: Pavel Kholkin <pkholkin@mirantis.com>
Change-Id: I371f7a68e6a6c1c4cd101f61b9ad96c15187a80e
Use the new oslo.report config option for specifying a
log_dir in which the GMR reports can be generated in.
consolidate the gmr related setup code in config.py
Introduced in oslo.reports change id:
I9ed0191628f2d552acd8130687b636671bc07a3e
Change-Id: Ib5edf65fd33ce86757f7cb1a6546a6f00738c9a3
'get_session' and 'get_api_session' methods are still
needed for backward compatibility.
Implements: blueprint new-oslodb-enginefacade
Closes-Bug: #1502104
Co-Authored-By: Pavel Kholkin <pkholkin@mirantis.com>
Change-Id: I8ceb9f939470f744f0d616d7db065a4d5d80202b
As per the review comments from the recent revert of suds logging
changes (I43b66aa6be6fc1b2ed43650bd5cfe69963d8a16e), note that
suds is used by the vmware driver and removing the suds logging
config causes heaps of extra log lines for their tempest runs.
Change-Id: I2dd7126f895eb862bdb2a5b93d17be6571e01369
This reverts commit bcc0028098.
Each call to the VC has a ton of information from suds that is logged.
This makes debugging and troubleshooting terribly difficult.
Closes-bug: #1491256
Change-Id: I43b66aa6be6fc1b2ed43650bd5cfe69963d8a16e
Suds was removed as a dependency in
I28f29d3e5745c21b99dc0cf9eb0cfe4f95dffd17
So there is no reason to have default log values for suds since we don't
use it
Change-Id: I104edc80213cee8278c2641d2c391d2788bfa0f6
We previously set the default log level for oslo.messaging to INFO, when
we switched over to oslo_messaging this didn't get changed.
This should significantly reduce the amount of logging, in one tempest
run, nova-compute logs 13k oslo_messaging debug logs.
Change-Id: I560c679af420f1da8054f163f80ae1e39bb7d089
Convert the use of the incubated version of the log module
to the new oslo.log library.
Sync oslo-incubator modules to update their imports as well.
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
Change-Id: Ic4932e3f58191869c30bd07a010a6e9fdcb2a12c
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
This commit includes:
- using oslo_utils instead of oslo.utils
- using oslo_serialization instead of oslo.serialization
- using oslo_db instead of oslo.db
- using oslo_i18n instead of oslo.i18n
- using oslo_middleware instead of oslo.middleware
- using oslo_config instead of oslo.config
- using oslo_messaging instead of "from oslo import messaging"
- using oslo_vmware instead of oslo.vmware
Change-Id: I3e2eb147b321ce3e928817b62abcb7d023c5f13f
When capturing a snapshot for example, glanceclient logs
the whole uploaded image into the nova compute logs as
glanceclient's log level is set to DEBUG by default
Change-Id: Ie764517c2a4b91cb02c15096f7f0b4138900c92d
Oslo's log module has a set_defaults which Nova can use
to set the context logging format and default log levels.
We should not have to fix default levels in oslo and then
sync across to Nova. This becomes all the more important
when we move to oslo.log library so we should get started
now
Change-Id: Id47099d5a755bc2923b05fe4bef270a7c0f99389
Replace common oslo code nova.openstack.common.db by usage
of oslo.db library and remove common code.
Replaced catching of raw sqlalchemy exceptions by catches
of oslo.db exceptions(such as DBError, DBDuplicateEntry, etc).
Co-Authored-By: Eugeniya Kudryashova <ekudryashova@mirantis.com>
Closes-Bug: #1364986
Closes-Bug: #1353131
Closes-Bug: #1283987
Closes-Bug: #1274523
Change-Id: I0649539e071b2318ec85ed5d70259c949408e64b
Thanks to I255a68fc60963386e8fefe65c3ffd269795adbf4, the two modules we
explicitly set log levels for are now in oslo-incubator, so we don't
need the extra code introduced in
Ib5164a22c5cbfa4f9d881b97bb1f623cd4d2f3f3
Change-Id: Iec7bd35da36135fcaf63a384fa35f849a1319738
Thanks to I49a82c5b3446784d254ca558f51b6c3e5c2028eb, we can augment
oslo's set of default log levels with project specific ones.
Add keystonemiddleware as warn level since its verbose and nova isn't
trying to debug keystonemiddleware.
Add routes.middleware since this generates 3 log entries per API request
and doesn't help make debugging easier.
Change-Id: Ib5164a22c5cbfa4f9d881b97bb1f623cd4d2f3f3
CLI options must be registered before the command line is parsed or a
cfg.ArgsAlreadyParsedError is raised.
Since commit 2cbea24 we are registering two 'remote_debug' arguments
in nova.service. In commit b506529, we see a hack being added to
ensure that nova.service is imported and the CLI opts registered
before config.parse_args() is called.
To clean this up somewhat, refactor all the elements of this remote
debugger support into nova.debugger. Avoid importing modules at
module import time since we need to import this before monkey
patching. Add a function for registering the CLI options just before
config.parse_args() is called. And fail gracefully if the CLI options
aren't registered when we called debugger.init().
Note that because the options aren't registered at module import time
any more, the config generator doesn't pick them up. However, that's
actually a good thing since this support doesn't work if you enable it
via config files since debugger.enabled() only looks at the command
line.
DocImpact: remote_debug options are not available via the config file,
only via the command line.
Change-Id: I97f747a2fb9222137203657df1d86ba89f3219e2
This sync contains the following commits from olso-incubator:
7959826 db: move all options into database group
dda24eb Introduce mysql_sql_mode option, remove old warning
0b5af67 Introduce a method to set any MySQL session SQL mode
8dccc7b Handle ibm_db_sa DBDuplicateEntry integrity errors
0f24d82 Fix migration.db_version when no tables
ac84a40 Update log translation domains
c0d357b Add model_query() to db.sqlalchemy.utils module
84254fc Fix a small typo in api.py
b8a676c Remove CONF.database.connection default value
86707cd Remove None for dict.get()
0545121 Fix duplicating of SQL queries in logs
fcf517d Update oslo log messages with translation domains
fa05b7c Restore the ability to load the DB backend lazily
630d395 Don't use cfg.CONF in oslo.db
ce69e7f Don't store engine instances in oslo.db
35dc1d7 py3kcompat: remove
b4f72b2 Don't raise MySQL 2013 'Lost connection' errors
271adfb Format sql in db.sqlalchemy.session docstring
0334cb3 Handle exception messages with six.text_type
eff69ce Drop dependency on log from oslo db code
7a11a04 Automatic retry db.api query if db connection lost
11f2add Clean up docstring in db.sqlalchemy.session
1b5147f Only enable MySQL TRADITIONAL mode if we're running against MySQL
39e1c5c Move db tests base.py to common code
986dafd Fix parsing of UC errors in sqlite 3.7.16+/3.8.2+
bcf6d5e Small edits on help strings
ae01e9a Transition from migrate to alembic
Due to API changes in oslo.db code we have to change Nova code a bit
in order to reuse oslo-incubator changes (oslo.db no longer stores
SQLAlchemy Engine and sessionmaker instances globally and it's up to
applications to create them).
Change-Id: I4aaa7151f66e0292ff66c29330f93d78c6bf78a6
We don't need to have the vi modelines in each source file,
it can be set in a user's vimrc if required.
Also a check is added to hacking to detect if they are re-added.
Change-Id: I347307a5145b2760c69085b6ca850d6a9137ffc6
Closes-Bug: #1229324
The oslo.messaging library takes the existing RPC code from oslo and
wraps it in a sane API with well defined semantics around which we can
make a commitment to retain compatibility in future.
The patch is large, but the changes can be summarized as:
* oslo.messaging>=1.3.0a4 is required; a proper 1.3.0 release will be
pushed before the icehouse release candidates.
* The new rpc module has init() and cleanup() methods which manage the
global oslo.messaging transport state. The TRANSPORT and NOTIFIER
globals are conceptually similar to the current RPCIMPL global,
except we're free to create and use alternate Transport objects
in e.g. the cells code.
* The rpc.get_{client,server,notifier}() methods are just helpers
which wrap the global messaging state, specifiy serializers and
specify the use of the eventlet executor.
* In oslo.messaging, a request context is expected to be a dict so
we add a RequestContextSerializer which can serialize to and from
dicts using RequestContext.{to,from}_dict()
* The allowed_rpc_exception_modules configuration option is replaced
by an allowed_remote_exmods get_transport() parameter. This is not
something that users ever need to configure, but it is something
each project using oslo.messaging needs to be able to customize.
* The nova.rpcclient module is removed; it was only a helper class
to allow us split a lot of the more tedious changes out of this
patch.
* Finalizing the port from RpcProxy to RPCClient is straightforward.
We put the default topic, version and namespace into a Target and
contstruct the client using that.
* Porting endpoint classes (like ComputeManager) just involves setting
a target attribute on the class.
* The @client_exceptions() decorator has been renamed to
@expected_exceptions since it's used on the server side to designate
exceptions we expect the decorated method to raise.
* We maintain a global NOTIFIER object and create specializations of
it with specific publisher IDs in order to avoid notification driver
loading overhead.
* rpc.py contains transport aliases for backwards compatibility
purposes. setup.cfg also contains notification driver aliases for
backwards compat.
* The messaging options are moved about in nova.conf.sample because
the options are advertised via a oslo.config.opts entry point and
picked up by the generator.
* We use messaging.ConfFixture in tests to override oslo.messaging
config options, rather than making assumptions about the options
registered by the library.
The porting of cells code is particularly tricky:
* messaging.TransportURL parse() and str() replaces the
[un]parse_transport_url() methods. Note the complication that an
oslo.messaging transport URL can actually have multiple hosts in
order to support message broker clustering. Also the complication
of transport aliases in rpc.get_transport_url().
* proxy_rpc_to_manager() is fairly nasty. Right now, we're proxying
the on-the-wire message format over this call, but you can't supply
such messages to oslo.messaging's cast()/call() methods. Rather than
change the inter-cell RPC API to suit oslo.messaging, we instead
just unpack the topic, server, method and args from the message on
the remote side.
cells_api.RPCClientCellsProxy is a mock RPCClient implementation
which allows us to wrap up a RPC in the message format currently
used for inter-cell RPCs.
* Similarly, proxy_rpc_to_manager uses the on-the-wire format for
exception serialization, but this format is an implementation detail
of oslo.messaging's transport drivers. So, we need to duplicate the
exception serialization code in cells.messaging. We may find a way
to reconcile this in future - for example a ExceptionSerializer
class might work, but with the current format it might be difficult
for the deserializer to generically detect a serialized exception.
* CellsRPCDriver.start_servers() and InterCellRPCAPI._get_client()
need close review, but they're pretty straightforward ports of code
to listen on some specialized topics and connect to a remote cell
using its transport URL.
blueprint: oslo-messaging
Change-Id: Ib613e6300f2c215be90f924afbd223a3da053a69
Fixes bug #1138844
Pass the version kwarg to cfg.CONF() so that --version shows the
version number.
Change-Id: I589ebffec67ef00f8ba24f553bf3e81806aa50b6
The cfg API is now available via the oslo-config library, so switch to
it and remove the copied-and-pasted version.
Add the 2013.1b4 tarball to tools/pip-requires - this will be changed
to 'oslo-config>=2013.1' when oslo-config is published to pypi. This
will happen in time for grizzly final.
Add dependency_links to setup.py so that oslo-config can be installed
from the tarball URL specified in pip-requires.
Remove the 'deps = pep8==1.3.3' from tox.ini as it means all the other
deps get installed with easy_install which can't install oslo-config
from the URL.
Make tools/hacking.py include oslo in IMPORT_EXCEPTIONS like it already
does for paste. It turns out imp.find_module() doesn't correct handle
namespace packages.
Retain dummy cfg.py file until keystoneclient middleware has been
updated (I18c450174277c8e2d15ed93879da6cd92074c27a).
Change-Id: I4815aeb8a9341a31a250e920157f15ee15cfc5bc
Bring in the new database code from oslo.
Uses get_session() from oslo as well as changing NovaBase to derive from
a common class.
Remove test_sqlalchemy.py now that this code is test in oslo.
Implements blueprint db-common.
Change-Id: I090754981c871250dd981cbbe1a08e7181440120
The my_ip, host and use_ipv6 options are used all over the codebase
and they're pretty well related to each other. Create a new netconf
module for them to live in.
There are now no options registered globally in nova.config!
blueprint: scope-config-opts
Change-Id: Ifde37839ae6f38e6bf99dff1e80b8e25fd68ed25
Add a factory function to nova.common.memorycache which consolidates
the code to choose between real or fake memcache. This then means that
memcached_servers is used in just one place and we can move the config
option there.
blueprint: scope-config-opts
Change-Id: I67c191e0db58364eda4162b9e881606063509b9d
Apart from one use in the EC2 API, vpn_key_suffix is only used in
pipelib.
blueprint: scope-config-opts
Change-Id: If90b9e77905161ea8d9263e8bae2c47ac5f79e91
Apart from checking whether a given image is the cloudpipe image, the
vpn_image_id option is only used within pipelib itself.
Add a is_vpn_image() helper method and move the option into pipelib.
Some rejiggering of how pipelib imports ec2 opts is required to avoid
circular imports.
blueprint: scope-config-opts
Change-Id: Ie984b2bb81681c24d3cee803082960083992a535
The enabled_apis option is only used in nova-api and nova-all but
there's no more obvious place to put it than nova.service. At least
this location is consistent, since we quite have a number of other
options related to the APIs in nova.service.
blueprint: scope-config-opts
Change-Id: I86e200fa5ad91ac0b7ab6837cc61e5927b2ebeb7
The default_ephemeral_format and use_cow_images options are used by a
number of virt drivers and nowhere else.
blueprint: scope-config-opts
Change-Id: I4ab2891e156589ae7bb44e6ce69cba3680a070c5
Apart from the tests, this option is only used in
nova.compute.instance_types so move it there.
blueprint: scope-config-opts
Change-Id: Iadf4011739e50455b7ad4214a778a02e244fef35
Lee Yarwood