Thankfully the bulk of this is neatly organized in a single directory
and can be removed, now that the bulk of the references to it have been
removed. The only complicated area is the tests, though effort has been
taken to minimise the diff here wherever possible.
Part of blueprint remove-cells-v1
Change-Id: Ib0e0b708c46e4330e51f8f8fdfbb02d45aaf0f44
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This was being called within the 'Instance' object. Remove it in its
entirety, given that nothing should be calling it now.
This also allows us to remove some special handling of the
'Instance.cell_name' field, given that nothing should be setting this
anymore.
Part of blueprint remove-cells-v1
Change-Id: I4f2ddf523d9ebfbc6f21a64687b2edb33d0ed1df
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
These were being called within the 'Instance' object. Remove them in their
entirety, given that nothing should be calling them now. They're pretty
closely coupled, hence removing the two of them at the same time. Note
that this removes a good chunk of functionality from cells v1 which turn
necessitates removing some more functional unit tests.
Part of blueprint remove-cells-v1
Change-Id: Ic89352a9900515484bffe961475feb1cefc6b2a9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This was being called within the 'InstanceMetadata' object. Remove it in
its entirety, given that nothing should be calling it now.
Part of blueprint remove-cells-v1
Change-Id: I92bd4c8f465b81a941885f9db4f708a34731de8b
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This was being called within the 'InstanceInfoCache' object. Remove it
in its entirety, given that nothing should be calling it now.
An 'update_cells' argument is removed from a number of functions as
these all resulted in a call to the cells v1 RPC API, a call that is now
removed.
Part of blueprint remove-cells-v1
Change-Id: Ia8da8cd040d104d745bd7adf20438b8eec558680
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This was being called within the 'InstanceFault' object. Remove it in
its entirety, given that nothing should be calling it now. As with the
removal of the 'bdm_(update_or_create|destroy)_at_top' APIs, this breaks
part of cells v1 but we don't really care.
Part of blueprint remove-cells-v1
Change-Id: Ia7b0166fce8c9ad5baa2a9ed6df8169f898522f2
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
These were being called within the 'BlockDeviceMapping' object. Remove
them in their entirety, given that nothing should be calling them now.
Note that this effectively breaks cells v1 but that shouldn't matter,
given that it's no longer possible to use the thing.
Part of blueprint remove-cells-v1
Change-Id: I13fb0575317015128853ac093bccffe233f3c79f
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This spec proposes to add ability to allow users to use
``Aggregate``'s ``metadata`` to override the global config options
for weights to achieve more fine-grained control over resource
weights.
blueprint: per-aggregate-scheduling-weight
Change-Id: I6e15c6507d037ffe263a460441858ed454b02504
We currently don't record snapshot instance
actions. This is useful for auditing and debugging.
This patch adds instance snapshot actions.
partial-implements: blueprint fill-the-gap-for-instance-action-records
Change-Id: I9ce48e768cc67543f27a6c87c57b47501fff38c2
We currently don't record backup instance
actions. This is useful for auditing and debugging.
This patch adds instance backup actions.
Note that because of how backup works for cellsv1, the API
call from the top isn't replayed in the child cell, so we
have to create the instance action in nova/cells/messaging
so the action is in the child cell database when nova-compute
looks it up via wrap_instance_event.
Change-Id: I878015fa211a5f2b5062902b3bfbd571d56efb76
partial-implements: blueprint fill-the-gap-for-instance-action-records
There are quite a few changes here as this is not only handling
uuids for the hypervisor id but it's also a refactor in several
APIs for consistency.
The main changes are detailed in the REST API Version History
doc in this change, but to summarize the changes:
* Hypervisor and service IDs are handled as the UUIDs for those
resources; this is necessary for accurately working with these
resources across multiple cells.
* The 'servers' and 'search' routes are deprecated and folded into
the index and detail methods as query parameters, validated using
json schema.
* The show method will also be able to return the list of servers
hosted on the given hypervisor using the with_servers query
parameter.
* The marker used when paging over lists of hypervisors is the
compute node UUID.
* Using the hypervisor_hostname_pattern query parameter will not
work with paging parameters.
* API reference docs are updated for the detailed changes.
* Functional and unit tests are provided for all changes.
Part of blueprint service-hyper-uuid-in-api
Change-Id: I828350c179df8bcfa4739910abeafaba2f96982b
This patch introduces a new microversion to identify services by uuid
instead of id, to ensure uniqueness across cells. GET /os-services
returns uuid in the id field, and uuid must be provided to delete a
service with DELETE /os-services/{service_uuid}.
The old PUT /os-services/* APIs are now capped and replaced
with a new PUT /os-services/{service_uuid} which takes a uuid path
parameter to uniquely identify the service to update. It also restricts
updates to nova-compute services only, since disabling or forcing-down
a non-compute service like nova-scheduler doesn't make sense as it
doesn't do anything.
The new update() method in this microversion also avoids trying to
re-use the existing private action methods like _enable and _disable
since those are predicated on looking up the service by host/binary,
are confusing to follow for code flow, and just don't really make sense
with a pure PUT resource update method.
Part of blueprint service-hyper-uuid-in-api
Co-Authored-By: Matt Riedemann <mriedem.os@gmail.com>
Change-Id: I45494a4df7ee4454edb3ef8e7c5817d8c4e9e5ad
All RPC topic opts were deprecated in Ocata.
This patch removes all *_topic opts from the code.
Change-Id: I41a6be48380999c49d6db3ee5221174a41014002
Implements: blueprint centralize-config-options-pike
This is a preliminary change for the counting quotas series to make
any legacy quota "reservations" arguments default to None so that in
the next change, we can simply not pass any reservations instead of
passing empty reservations. Reservations are being deprecated as part
of the counting quotas series.
Part of blueprint cells-count-resources-to-check-quota-in-api
Change-Id: I86e56c1abd979ac9968a3ce7daad07530080bde4
This allows looking up a compute node by its uuid to uniquely
identify it in a list of cells. A later REST API microversion
change will rely on this.
Note that the existing compute_node_get method is used to
minimize the impact to cells v1 (we would have to plumb through
all new support to cells v1 if we added a new HostAPI method).
Since the HostMapping in the API DB does not have the
ComputeNode.uuid we still have to loop over the cells to
find a match. This could be optimized later if needed.
Part of blueprint service-hyper-uuid-in-api
Change-Id: I705119442552e3d507e8d514218466942c5767f5
The get_transport method was deprecated in the oslo.messaging
5.24.2 release in favor of get_rpc_transport and
get_notification_transport. We are already using get_notification_transport,
we just need to change to use get_rpc_transport.
Change-Id: Ie276fedef35f411cafbaf25c2148fea42ae01410
We're seeing a trace in gate jobs, for example:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
402: ordinal not in range(128)
when attempting to log an Instance object with a unicode display name.
This resurfaced relatively recently because of the change in devstack
to use the new OSJournalHandler with use_journal=True which is
suspected of causing some deadlock issues [1] unrelated to this bug.
The problem occurs in code that logs an entire Instance object when
the object has a field with unicode characters in it (display_name).
When the object is sent to logging, the UnicodeDecodeError is raised
while formatting the log record here [2]. This implies an implicit
conversion attempt to unicode at this point.
I found that with the Instance object, the conversion to unicode fails
with the UnicodeDecodeError unless the encoding 'utf-8' is explicitly
specified to six.text_type(). And when specifying an encoding to
six.text_type(), the argument to convert must be a string, not an
Instance object, so this does the conversion in two steps as a utility
function:
1. Get the string representation of the Instance with repr()
2. Call six.text_type(instance_repr, 'utf-8') passing the encoding
if not six.PY3
Closes-Bug: #1580728
[1] https://review.openstack.org/#/c/462163
[2] https://github.com/python/cpython/blob/2e576f5/Lib/logging/__init__.py#L338
Change-Id: I0fc3ae02cb2e401b3240faf0d8b6aa5dc52b91fc
The oslo.messaging rpc client call() method does not actually make
any use of a "timeout" parameter, so drop it from the call. (It's
actually set up in the preceeding prepare() call.)
Change-Id: I662948cbb0c1fb89d8cb1e1ffad1909dc5b1dc68
Those are remnants from the oslo-incubator times. Also, oslo.messaging
deprecated [1] transport aliases since 5.2.0+ that is the minimal
version supported for stable/newton. The patch that bumped the minimal
version for Nova landed 3 months+ ago, so we can proceed ripping
those aliases from the code base.
[1] I314cefa5fb1803fa7e21e3e34300e5ced31bba89
Change-Id: Id8c0bc6733e930803b8e9fcf42a9a1f81dcbb356
Closes-Bug: #1424728
This remaining use of the flavor query routine from the cellsv1 code
still looks at the main database. This patch converts it to use the
object which looks in the right place.
Closes-Bug: #1663729
Change-Id: I418988ae1d72d541ace867a22b2c9267164bf61b
* Add osprofiler wsgi middleware. This middleware is used for 2 things:
1) It checks that person who want to trace is trusted and knows
secret HMAC key.
2) It starts tracing in case of proper trace headers
and adds the first wsgi trace point with info about the HTTP request
* Add initialization of osprofiler on start of a service
Currently that includes oslo.messaging notifier instance creation
to send Ceilometer backend notifications.
oslo-spec: https://review.openstack.org/#/c/103825/
python-novaclient change: https://review.openstack.org/#/c/254699/
based on: https://review.openstack.org/#/c/105096/
Co-Authored-By: Boris Pavlovic <boris@pavlovic.me>
Co-Authored-By: Munoz, Obed N <obed.n.munoz@intel.com>
Co-Authored-By: Roman Podoliaka <rpodolyaka@mirantis.com>
Co-Authored-By: Tovin Seven <vinhnt@vn.fujitsu.com>
Implements: blueprint osprofiler-support-in-nova
Change-Id: I82d2badc8c1fcec27c3fce7c3c20e0f3b76414f1
1.As mentioned in [1], we should avoid using
six.iteritems to achieve iterators. We can
use dict.items instead, as it will return
iterators in PY3 as well. And dict.items/keys
will more readable. 2.In py2, the performance
about list should be negligible, see the link [2].
[1] https://wiki.openstack.org/wiki/Python3
[2] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html
The patch list:
1. cells.
2. compute api.
3. image.
4. network.
5. objects.
6. scheduler.
7. virt.
8. other resources.
Partial-Implements: blueprint replace-iteritems-with-items
Change-Id: Ic6e469eb80ee1774de1374bb36f38b5134b6b311
The field 'free_disk_gb' in Objects.ComputeNode can be nullable.
In CellStateManager._get_compute_hosts, we compute free_disk_mb
by 'free_disk_gb * 1024'. If the free_disk_gb is None, it will raise
a TypeError exception because None type can't multiply by an integer.
We should check this value, and if it's None, we just let it return
zero.
Closes-Bug: #1648983
Change-Id: I558d544ff838807c1d605cb5ed8d0e0fa97ab00e
This adds upgrade level aliases for newton for all of our services,
including the ones we expect to remove soon.
Change-Id: If97f8f87f3d8b4db8f6b6b826c5c1f5d0b3f10a8
Fixes volume related unit tests.
Fixes non-sortable None items.
Fixes __getattr__ infinite recursion.
Fixes is_dict_like method. Dicts in python 3.4 do not
have the 'has_key' method.
Partially Implements: blueprint goal-python35
Change-Id: I97efc09f7657436f706b08e0b2795f0e59ac1dcd
Fixes up the get_migrations cells method and corresponding tests. The
tests were not in line with reality since the MigrationList object was
introduced.
This patch removes the [] + MigrationList operation that was occurring
since the operation is poorly defined and only works by accident. Future
objects work may break this so it should be removed now.
Change-Id: I5dabd8c745cc287322a1ef0f0dd2d4733484188d
This patchset fixes the typo errors in nova root(non recursively),
nova/api, nova/cells and nova/cmd directory.
TrivialFix
Change-Id: Id4e5dbeafb6fdb63bb733eef203a676f94143f32
It has become clear that it will be necessary to store a copy of the
block_device_mapping on the BuildRequest object so that it can be
returned via the API(os-extended-volumes). In order to prepare for doing
so the block_device_mapping needs to be validated and fully set, but not
persisted to the database. This patch refactors some methods to allow
that.
Change-Id: If2020f00e15ca1b067620f880a6f7d3a8ad5281c
Registers in-code the last remaining policy rules.
Adds missing 'discoverable' rules. Without them,
the extension_info API can fail, as it tries to check the
os_compute_api:os_server_tags:discoverable rule. As it wasn't
previously registered, when listing the available extensions,
an exception of type PolicyNotRegistered is encountered.
In order to validate this, functional/api_sample_tests/test_extension_info.py
now runs without mocking policy.authorize.
Switches extension_info to context.can.
Switches nova.cells.filters to context.can.
Switches network.neutronv2.api to context.can.
Removes the rest of the entries in etc/policy.json.
Removes DefaultPolicyTestCase, as it tests the default
policy rule, which is not registered.
Removes rules from fake_policy.py that brings no value,
that are the same as the default values.
Removes extensions authorizer factories.
Removes nova.policy.enforce.
Change-Id: Ie7771768f4f3efe0edc787c12f297aa93d533d7e
Partially-Implements: bp policy-in-code
Some comments on `Improve the help text for cells options`
reviews are implemented with this patch.
- nova.cells.opts:list_ops function removed.
Implements blueprint centralize-config-options-newton
Change-Id: Idcaf66f0db1695d4fff34a656df2934c1b6d2ede
Cold migrate and resize can now get the Spec object and pass it
down the wire to the conductor so that it could use it for the
move.
Partially-Implements: blueprint check-destination-on-migrations-newton
Co-Authored-By: Pawel Koniszewski <pawel.koniszewski@intel.com>
Change-Id: Ic3968721d257a167f3f946e5387cd227a7eeec6c
This makes compute_api create Instance objects with keypairs set,
so that they are persisted in the instance_extra table and thus
available without hitting the API DB once we move keypairs up there.
Related to blueprint cells-keypairs-api-db
Change-Id: I8d055265c8900b7fff8a209bb48884c9a734ceaf
If a host's:
* resources are unavailable
* in a unusable state
the system should:
* report 0 resources
* show 0 resources
* not be scheduled to
Change-Id: Ia1c2f6f161dde1e23acce85a54566d07805d13df
Closes-Bug: 1503453
This change removes NovaObjectDictCompat from the ComputeNode object.
The only changes needed were changing dict syntax over to either use
dot syntax or getattr/setattr. In cases where the dict function update()
was used, the update dictionary was unchanged, but was not passed into
a new update helper function that applies the updates using setattr() on
the given kwargs.
Change-Id: Iba5303d23492db55f46df8b8f6b3dbae5b4971c1
Partially-Implements: bp rm-object-dict-compat-newton
Per our usual procedure, this sets version aliases for the current RPC
versions of all our internal RPC APIs for mitaka.
Change-Id: I455f3b80a9b64dfea42ee1c74685d11ae0178513
Right now, we only create the ec2_ids for an Instance when they are first
accessed, not at instance creation time. However, we always touch them
later for building metadata, configdrive, etc, which means we end up with
an expensive lazy-load later.
This makes us always create ec2_ids at instance create time. Since it's in
instance.create(), it runs on conductor, straight to the database, avoiding
a round-trip later (assuming this instance is passed to the compute node
for create and/or that the field is specified in a later call under
expected_attrs).
Partial-Bug: #1540526
Change-Id: I29bc317f990bfa0f01b4f9615699debcc3a3c2c6
The BuildRequest object represents a boot request before an instance has
been created and written to the database. It will be required to respond
to API requests to list/show an instance in cellsv2 where the instance
is not written to the database immediately. An upcoming change will
modify the boot process to defer instance creation until after
scheduling has picked a cell/host and having this BuildRequest object
persisted is a prerequisite for that work. The BuildRequest object
lifetime ends as soon as the instance has been written to a database and
a mapping is in place for it. Until instance creation moves the
BuildRequest object is destroyed rather quickly.
Note there is a small optimization added here such that when creating
instances in the API, only populate the SecurityGroupList once rather
than each pass in the for loop (if building multiple instances).
Change-Id: I080fa53d13f7301f6f7f2977a8e3cb287a46c02c
Partially-implements: bp cells-scheduling-interaction
Since cells v1 adds a proxy to a call to ComputeTaskAPI for live_migrate
and rebuild_instance, we need to augment the Cells RPC API to accept
the request_spec argument for both methods.
That said, since cells v1 is quite feature-freeze, it's okay to not pass
the RequestSpec down to the pipe because the cell conductor will then
act as if the instance was not having a Spec object, which is a compat
behaviour we want to keep for a very long time.
Closes-Bug: #1550018
Change-Id: I1c2aaedfc83e4e97f53c9c504e4585b1f0510d4b
Stephen Finucane