We want this module for use elsewhere. Given there's only a single
caller (nova.service) we can simply move the code to the caller.
Change-Id: I2c3887db8b3f6833bf24f5114fd955e1af590d03
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
this is the inital patch of applying codespell to nova.
codespell is a programing focused spellchecker that
looks for common typos and corrects them.
i am breaking this into multiple commits to make it simpler
to read and will automate the execution of codespell
at the end of the series.
Change-Id: If24a6c0a890f713545faa2d44b069c352655274e
The ComputeNode object already has a service_id field that we stopped
using a while ago. This moves us back to the point where we set it when
creating new ComputeNode records, and also migrates existing records
when they are loaded.
The resource tracker is created before we may have created the
service record, but is updated afterwards in the pre_start_hook().
So this adds a way for us to pass the service_ref to the resource
tracker during that hook so that it is present before the first time
we update all of our ComputeNode records. It also makes sure to pass
the Service through from the actual Service manager instead of looking
it up again to make sure we maintain the tight relationship and avoid
any name-based ambiguity.
Related to blueprint compute-object-ids
Change-Id: I5e060d674b6145c9797c2251a2822106fc6d4a71
This just adds the service ref to init_host() so that we can pass it
(if it exists) at startup. On the first run, this will be None, so
we know that we don't have an existing service record and thus do not
need to do any migration.
Related to blueprint stable-compute-uuid
Change-Id: I1491c9b234ef0c262b5ed01d4c138eba8dedff77
We recently added a hard failure to nova service startup for the case
where computes were more than one version old (as indicated by their
service record). This helps to prevent starting up new control
services when a very old compute is still running. However, during an
FFU, control services that have skipped multiple versions will be
started and find the older compute records (which could not be updated
yet due to their reliance on the control services being up) and refuse
to start. This creates a cross-dependency which is not resolvable
without hacking the database.
This patch adds a workaround flag to allow turning that hard fail into
a warning to proceed past the issue. This less-than-ideal solution
is simple and backportable, but perhaps a better solution can be
implemented for the future.
Related-Bug: #1958883
Change-Id: Iddbc9b2a13f19cea9a996aeadfe891f4ef3b0264
Before Ie15ec8299ae52ae8f5334d591ed3944e9585cf71 if the compute was
started before the conductor then the compute retried the connection
until the conductor was up. The Ie15ec8299ae52ae8f5334d591ed3944e9585cf71
break this behavior as the service version check runs before this RPC
retry mechanism and therefore the compute simply fails to start without
a retry if no conductor is started.
This patch moves the service version check after the RPC connection
retry mechanism.
Change-Id: Iad0ba1a02868eebc2f43b1ac843fcc5096cd5c47
Closes-Bug: #1904181
Nova services only support computes that are not older than
the previous major release. This patch introduces a check in the
service startup that prevents staring the service if too old computes
are detected.
Change-Id: Ie15ec8299ae52ae8f5334d591ed3944e9585cf71
Once again, do what we did for '_LE' and '_LW' and remove the final
remnants of the log translation effort.
Change-Id: Id6cf7a9bfbe69d6d3e65303e62403d1db9188a84
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
There are only a few of these remaining in the code base. Remove them.
Change-Id: I33725e2439b0f39c1e9bec9e33a37bf3e24944fb
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We've been slowly removing these as we go. Remove the final few '_LE'
occurrences now.
Change-Id: I75ebd2e95a0c77585d7b4329ca01e4bacc1dd7c4
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The monotonic package was needed for monotonic time operations when
running under Python runtimes older than 3.3. Since we now only support
versions higher than this, this third party package requirement can now
be removed.
Change-Id: I955117b838cdd8a1858cfccbc3caa33ba6ad59f7
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
We have had a gate bug for a long time where occasionally the scheduler
service gets into a state where many requests fail in it with
CellTimeout errors. Example:
Timed out waiting for response from cell <cell uuid>
Through the use of much DNM patch debug logging in oslo.db, it was
revealed that service child processes (workers) were sometimes starting
off with already locked internal oslo.db locks. This is a known issue
in python [1] where if a parent process forks a child process while a
lock is held, the child will inherit the held lock which can never be
acquired.
The python issue is not considered a bug and the recommended way to
handle it is by making use of the os.register_at_fork() in the oslo.db
to reinitialize its lock. The method is new in python 3.7, so as long
as we still support python 3.6, we must handle the situation outside of
oslo.db.
We can do this by clearing the cell cache that holds oslo.db database
transaction context manager objects during service start(). This way,
we get fresh oslo.db locks that are in an unlocked state when a child
process begins.
We can also take this opportunity to resolve part of a TODO to clear
the same cell cache during service reset() (SIGHUP) since it is another
case where we intended to clear it. The rest of the TODO related to
periodic clearing of the cache is removed after discussion on the
review, as such clearing would be unsynchronized among multiple
services and for periods of time each service might have a different
view of cached cells than another.
Closes-Bug: #1844929
[1] https://bugs.python.org/issue6721
Change-Id: Id233f673a57461cc312e304873a41442d732c051
This was only applying some iptables rules when running under
nova-network. Since that's no longer possible, we don't need to keep the
manager around. So don't!
Change-Id: I4350b6f3ca3fd7de1bc6db3e14fe261f2746b002
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This legacy service was only compatible with the XenServer driver and
has effectively been replaced by the noVNC console proxy service. Remove
the service. The API that provided remote access to this service,
'os-consoles', was removed in a previous change. Note that
'os-remote-consoles' is unrelated and therefore is not removed, though
it will now reject requests for XVP VNC consoles.
This was previously discussed and agreed on openstack-dev [1] and
openstack-discuss [1].
Part of blueprint remove-xvpvncproxy
[1] http://lists.openstack.org/pipermail/openstack-dev/2018-October/135413.html
[2] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005369.html
Change-Id: Ib1ff32f04b16af7981471f67c8e0bf04e6ecb6be
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Obliterate all references to the aforementioned service. This mostly
consists of removing the core service and any references to the now
removed '[workarounds] enable_consoleauth' configuration option.
Part of blueprint remove-consoleauth
Change-Id: I0498599fd636aa9e30df932f0d893db5efa23260
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Depends-On: Icfc175c49a1fc650d1c9ad06b77209a70c6386db
We're going to start unpicking this stuff from the top down. Start with
the 'nova-cells' executable itself.
Part of blueprint remove-cells-v1
Change-Id: I5bd1dd9f1bbae7a977ab9e032c4f4d200c35e193
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The Service base class already waits for conductor to be responsive
before allowing service startup to continue. However, it does this
after the manager __init__, which could do some things expecting
conductor to be up (and does now in compute). This moves the check
and wait loop above the manager init so we don't start anything in
the manager until we know conductor is alive.
Change-Id: Ib56fdb38839434d504fadccbeda65ab2e267b514
Closes-Bug: #1789484
This sets the timer on removing support for running
API services under eventlet. Nova has supported running
under WSGI since Pike, and does run with Apache by default
in the gate.
Change-Id: I83d6c100d0d2628f38a029cffa22d0c2bc15d667
ec2 was deprecated in commit
f098398a83
back to mitaka release, this patch
starts to remove ec2 services in nova.
Change-Id: I5939ef73904f1bb66ed8453c900bfb50cdb5b8b6
The file nova/api/openstack/__init__.py had imported a lot of
modules, notably nova.utils. This means that any code which
runs within that package, notably the placement service, imports
all those modules, even if it is not going to use them. This
results in scripts/binaries that are heavier than they need
to be and in some cases including modules, like eventlet, that
it would feel safe to not have in the stack.
Unfortunately we cannot sinply rename nova/api/openstack/__init__.py
to another name because it contains FaultWrapper and FaultWrapper
is referred to, by package path, from the paste.ini file and that
file is out there in config land, and something we prefer not to
change. Therefore alternate methods of cleaning up were explored
and this has led to some useful changes:
Fault wrapper is the only consumer of walk_class_hierarchy so
there is no reason for it it to be in nova.utils.
nova.wsgi contains a mismash of WSGI middleware and applications,
which need only a small number of imports, and Server classes
which are more complex and not required by the WSGI wares.
Therefore nova.wsgi was split into nova.wsgi and nova.api.wsgi.
The name choices may not be ideal, but they were chosen to limit
the cascades of changes that are needed across code and tests.
Where utils.utf8 was used it has been replaced with the similar (but not
exactly equivalient) method from oslo_utils.encodeutils.
Change-Id: I297f30aa6eb01fe3b53fd8c9b7853949be31156d
Partial-Bug: #1743120
This adds a pep8 function that will check for redundant import aliases.
Any imports of the forms below will not be allowed.
from x import y as y
import x as x
import x.y as y
Change-Id: Iff90f0172d97bd1d49d54c811a70c8af11776da4
If system time is adjusted first forward and then backward while a
nova service is running (e.g. nova-compute), then there is a high
probability, that periodic tasks will stop for the duration of time
the system clock was adjusted backward.
This was supposed to be fixed by the following patch to oslo.service
https://review.openstack.org/#/c/286838/ , but the order of imports
in unit tests and production code is different, so nova services
end up starting with the default eventlet hub, that does not use a
monotonic clock and, thus, is affected by changes of system time.
Testing this is problematic, as it's a subject of imports order and
is not reproduced in functional or unit tests (oslo_service is always
imported earlier than eventlet hub is initialized, so it just does
"the right thing"). The alternative is to make an assertion when
services start.
Closes-Bug: #1510234
Change-Id: I110cf31ad2a0c74a0cf30ec08bd94d3a56727b39
make some modifications to nova db reference and add some
basic information about some functions such as nova.service.start
Change-Id: I70c1a5eac91ab61ecd4ebc1c8ed95e4e4ba6b9fc
The compute API binary service name is a constant source of
confusion, sometimes called nova-api or nova-os_compute, which are
both wrong, since it's actually nova-osapi_compute. It comes from
nova.cmd.api.
Change-Id: I50befc34730296d3908b6dd25536394608bae0ec
I8f09f9482736b496e3ad7117f170e476e92c2b7d changed the way SIGHUP is
handled by nova-api by telling oslo_service to reload config files
and run 'mutation' hooks for a set of supported options.
Make sure we use the very same mechanism for other service like
nova-compute, that do not use process_launcher() directly, but rely
on launch() helper from oslo_service instead.
Change-Id: Ie4e7c06814310f661d054d2181340418fd0df7bc
This commit removes nova-cert which has been deprecated since change
Id7a1fc943cbe6d860a50d3cc776717b55351004b. The APIs have been hard coded
to return a 410 whenever they're called now. For the API ref a new
section for obsolete apis is added to the bottom of the page and the
certificates api ref is moved there.
Implements bp remove-nova-cert
Change-Id: I2c78a0c6599b92040146cf9f0042cff8fd2509c3
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
1) It checks that person who want to trace is trusted and knows
secret HMAC key.
2) It starts tracing in case of proper trace headers
and adds the first wsgi trace point with info about the HTTP request
* Add initialization of osprofiler on start of a service
Currently that includes oslo.messaging notifier instance creation
to send Ceilometer backend notifications.
oslo-spec: https://review.openstack.org/#/c/103825/
python-novaclient change: https://review.openstack.org/#/c/254699/
based on: https://review.openstack.org/#/c/105096/
Co-Authored-By: Boris Pavlovic <boris@pavlovic.me>
Co-Authored-By: Munoz, Obed N <obed.n.munoz@intel.com>
Co-Authored-By: Roman Podoliaka <rpodolyaka@mirantis.com>
Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com>
Implements: blueprint osprofiler-support-in-nova
Change-Id: I82d2badc8c1fcec27c3fce7c3c20e0f3b76414f1
compute_manager config options were deprecated in
13.0.0, we can remove it now.
Depends-On: I93b62372e357db4f7c39992c244b2ec9e594ad9c
Implements: bp centralize-config-options-ocata
Change-Id: I84503655bd9675ff441073e51059202142db22ab
The following service manager config options were deprecated in
13.0.0 and remove them now:
- metadata_manager
- console_manager
- consoleauth_manager
- cert_manager
- scheduler_manager
- conductor.manager
Implements: bp centralize-config-options-ocata
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
Change-Id: Ia50d18c350dbc4ca88da9543e1236e3398ea28ee
This change removes the deprecated conductor local mode and the
'use_local' flag that was used to control it.
UpgradeImpact
Change-Id: Icc1624606ecb561781a971467443e24b1643f535
This patch allows the user to specify a tag for a virtual network
interface and/or block device mapping when booting an instance.
Implements: blueprint bp/virt-device-role-tagging
Change-Id: I89247200f4cf1f644daf476727b4a6acb22b0cf6
New releases of oslo.config support a 'mutable' parameter to Opts.
This is only respected when the new method mutate_config_files is
called instead of reload_config_files. Nova delegates making this call
to oslo.service, so how do we switch?
Icec3e664f3fe72614e373b2938e8dee53cf8bc5e allows services to tell
oslo.service they want mutate_config_files to be called by passing a
parameter, which this patch does.
This allows Nova to benefit from
I1e7a69de169cc85f4c09954b2f46ce2da7106d90, where the 'debug' option
(owned by oslo.log) is made mutable. IE when all these patches merge,
we should be able to turn debug logging on and off by changing the
config and sending SIGHUP.
Further patches will make selected Nova options mutable.
Part of bp:mutable-config
Change-Id: I8f09f9482736b496e3ad7117f170e476e92c2b7d
When new code is installed with a newer service version, the version in
the DB should get updated.
The fix here is to:
1) eliminate the special case for 'version' in save()
2) cause version to be saved on startup
Change-Id: I96fa9dabfb9b7a5f1703baf80534d8b104dab4e6
Closes-Bug: 1579839
The config options of the section
"nova/netconf" got moved to the
new central location
"nova/conf/netconf.py"
Change-Id: I8a17b6f00b15e03de55385fc0206bdc82441304a
Depends-On: I0da2ad7daa942b85c3395dc4861c6e18368ece88
Implements: blueprint centralize-config-options-newton
The class "Service" doesn't have a "__repr__" method which results
in log outputs like:
DEBUG [...] service = <nova.service.Service object at 0x56eb890>
This is not helpful when analyzing logs. Therefore this change adds a
"__repr__" method.
Change-Id: I49c8b7f0dde298de780808c988e4b50278eed665
Move options from 'service' to 'conf/service.py'.
Change-Id: I621481590d93b806ea17d498a562da87207001f3
Co-Authored-By: Stephen Finucane <stephen.finucane@intel.com>
Implements: blueprint centralize-config-options-newton
Skip creating the formatted log message
if the message is not going to be emitted
because of the log level.
TrivialFix
Change-Id: Iba9f47163a0ac3aca612818272db6d536b238975
We should deprecate the manager classes used for all of our
services. This is not a thing we actually expect or want to support
people replacing. If we want modular plug points at any of these all
the options should be in tree and specified by constants to switch
between.
Change-Id: Id25bd4870c6e2fda08dc0177b7ed61a8a6091838
In Id7936be290b6febd18deb4c2db8ea4d678d4d9b1, we removed
entries from api-paste.ini for EC2 API service. In this
review we remove all the unnecessary code, docs and tests
associated with objectstore and ec2 service. Note that this
does not cleanup the Instance object or change any of the
versioned objects. We just drop any code associated with
testing the REST endpoint(s) that are no longer needed.
Also added shims such that the api-paste.ini from liberty
will still work (grenade job) and added logs and response
messages for prompting administrators to cleanup their
old api-paste.ini and switching to the stand alone EC2 API
project for their needs.
Change-Id: I8bf7cbaa7015bb61656ab90ccc8f944aaeebb095
Tempest has already removed the EC2 API tests in:
Ib5e24e19bcba9808a9f49fe7f328668df77fe4f9
We have a review in devstack below that removes EC2
API service by default.
In this review, we are removing support for the really
old EC2 API support in Nova. Anyone who needs EC2 support
should use the new EC2-API project:
http://git.openstack.org/cgit/openstack/ec2-api/
The necessary changes in Tempest and DevStack are
already merged. In a follow on review, will remove
the EC2 specific options, code and tests from Nova.
DocImpact
Change-Id: Id7936be290b6febd18deb4c2db8ea4d678d4d9b1
This links Service.reset() to the manager, which for compute will
rebuild the compute_rpcapi and thus re-determine the appropriate
RPC versions.
Change-Id: Ifec7f6ff604d1e5f3663633065e9a55baacffec8
This adds service records for the API services for the purpose of
version-tracking. Unlike other services, these records are not periodically
updated, so this adds no additional runtime database burden. The API
services update the version (if necessary) of their record at startup.
This also adds filtering in three places to prevent us from exposing
API service records out of the hosts and availability-zone APIs.
Related to blueprint service-version-behavior
Change-Id: I66becb14788de42bee35064d1155dd919059bcb7
The latest oslo.config supports a minimum and maximum value for
IntOpt options. This patch utilizes this for a number of options
with well known ranges.
Change-Id: I897d8870fdf67a095915c8fcdf239024b092e9e3
Modules eventlet_backdoor, loopingcall, periodic_task,
service, sslutils, systemd, threadgroup were removed
from nova. These modules were imported from oslo.service
library.
Co-Authored-By: Marian Horban <mhorban@mirantis.com>
Depends-On: I305cf53bad6213c151395e93d656b53a8a28e1db
Change-Id: Iaef67e16af3d69f845742f7bdcb43667bf1576ee
Zuul