Snapshot.get_all does not return all snapshots of the project but
returns all snapshots associated with a single stack, so its name
should contain _by_stack for consistency.
Change-Id: Ic6b93b7cfc84793077672b3f1052f03519e4c5a1
When sorting resource candidates in `_get_best_existing_rsrc_db`,
resources with the same score are sorted by `updated_at`, which can be
`None`. If that is the case, use `created_at` instead.
Task: 43815
Story: 2009653
Change-Id: Ic0265fcf7ceb811803cdebaa8932fe80dc59a627
There are situations when the auth_encryption_key changes
and customer wants to delete old stacks. We should allow
deleting those stacks.
Task: #42055
Change-Id: Ifc8c19e181902566d4f295fa979ab6869a4e0852
The abstract base classes previously defined in 'collections' were moved
to 'collections.abc' in 3.3. The aliases will be removed in 3.10.
Preempt this change now with a simple find-replace:
$ ag -l 'collections.($TYPES)' | \
xargs sed -i 's/\(collections\)\.\($TYPES\)/\1.abc.\2/g'
Where $TYPES is the list of moved ABCs from [1].
[1] https://docs.python.org/3/library/collections.abc.html
Change-Id: Ia282479bb1d466bd2189ebb21b51d91e89b9581e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Regenerate trust when update stack with different user
We will regenerate (and delete old trust) when user credential
doesn't match with current context (means different user is
operating).
Story: #1752347
Task: #17352
Change-Id: I39795bdbd8ab255150153bf8b1e165b49e1a7027
Now that we only support Python 3, use the 'yield from' keyword instead
of the @wrappertask decorator for calling co-routines from a co-routine.
This should greatly simplify backtraces when calling nested co-routines.
Change-Id: If9beaff74cf4facbc4aa4b30f31a3a087bdcad8a
A simple matter of having log lines fit
with all the others. It will now match the
ambiguous standard.
Also change comments inline to alias with log fixes.
Change-Id: I4a2ed6134f70c2965811f75ccb6ab8221fa0e960
Story: #2007319
Task: 38830
Six is in use to help us to keep support for python 2.7.
Since the ussuri cycle we decide to remove the python 2.7
support so we can go ahead and also remove six usage from
the python code.
Review process and help
-----------------------
Removing six introduce a lot of changes and an huge amount of modified files
To simplify reviews we decided to split changes into several patches to avoid
painful reviews and avoid mistakes.
To review this patch you can use the six documentation [1] to obtain help and
understand choices.
Additional informations
-----------------------
Changes related to 'six.b(data)' [2]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
six.b [2] encode the given datas in latin-1 in python3 so I did the same
things in this patch.
Latin-1 is equal to iso-8859-1 [3].
This encoding is the default encoding [4] of certain descriptive HTTP
headers.
I suggest to keep latin-1 for the moment and to move to another encoding
in a follow-up patch if needed to move to most powerful encoding (utf8).
HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5].
Note that this commit message is autogenerated and not necesserly contains
changes related to 'six.b'
[1] https://six.readthedocs.io/
[2] https://six.readthedocs.io/#six.b
[3] https://docs.python.org/3/library/codecs.html#standard-encodings
[4] https://www.w3schools.com/charsets/ref_html_8859.asp
[5] https://www.w3schools.com/html/html_charset.asp
Patch 13 of a serie of 28 patches
Change-Id: I09aa3b7ddd93087c3f92c76c893c609cb9473842
When we delete a legacy stack, the delete call may return when a previous
delete is still IN_PROGRESS and we are waiting for it to be cancelled - for
up to 250s by default plus the time it takes to kill any remaining threads.
This means that if the client subsequently sees a DELETE_FAILED status, it
can be ambiguous as to whether it refers to the previous delete or the one
it just commanded.
Work around this when Heat itself is the client (i.e. we are deleting a
nested stack).
First, put the time the delete started into the status_reason message when
the stack goes into the DELETE_IN_PROGRESS state, so that we can
distinguish.
Then, require the stack to report its status as DELETE_FAILED twice in a
row, 10s apart, before we mark it as failed - unless we have definitively
seen a different delete commence in the meantime.
Change-Id: Ie23ba5f4538a8f5c2e7d64caa6cbe530c013b520
Story: #1669608
Task: 17214
Avoid loading the tags from the DB and then re-saving every time
the stack is stored when the stack has tags. Avoid attempting to
lazy-load the tags from the DB multiple times when there are no tags.
Avoid lazy-loading the existing tags on an update when we intend to
overwrite them anyway. Avoid writing the same set of tags multiple
times.
e.g. in a legacy update, previously we rewrote the current set of tags
when changing the state to IN_PROGRESS, then wrote the new set of tags
regardless of whether they had changed, then wrote the new set of tags
again when the update completed. In a convergence update we also did
three writes but in a different order, so that the new tags were written
every time. With this change we write the new set of tags only once.
This could also have prevented stacks with tags from being updated from
legacy to convergence, because the (unchanged) tag list would get
rewritten from inside a DB transaction. This is not expected so the
stack_tags_set() API does not pass subtransactions=True when creating a
transaction, which would cause a DB error.
Change-Id: Ia52818cfc9479d5fa6e3b236988694f47998acda
Task: 37001
We keep the new template in the prev_raw_template_id. When setting
the stacks to FAILED we should also merge the templates for both
existing and backup stack.
Change-Id: Ic67a4833672d1c562980ee19fd8071f84dd9500a
Task: 35842
In convergence we were loading resources from the database using the
current environment. This is incorrect when a previous update has
failed, meaning the resources in the database were created with a
non-current template and environment. If an attempt was made to change
the type of a resource but that resource was never updated, this will
result in us loading a resource with the wrong type. If the type has
been removed then it can result in errors just trying to show the stack.
Note that the Resource.load() method used during a convergence traversal
already does the Right Thing - it only uses the new type if it is a
valid substitution for the old type, and UpdateReplace is later raised
in Resource.update_convergence() if the type does not match in that
specified in the new environment. So we don't see any problems with
stack updates, just with API calls.
Since we cannot change the signature of Resource.__new__() without also
modifying the signature of __init__() in every resource plugin that has
implemented it (many of which are out of tree), instead substitute the
stack definition for the duration of creating the Resource object. This
will result in stack.env returning the environment the resource was last
updated with.
Change-Id: I3fbd14324fc4681b26747ee7505000b8fc9439f1
Story: #2005090
Task: 29688
When we specify a sd on delete action, os-collect-config will not
get authentication because we didn't load access_allowed_handlers
after stack enter stack delete phrase. This patch will make sure
we load necessary access_allowed_handlers even if in stack delete
phrase.
Change-Id: I43c1a865f507f7cb7757e26ae5c503ce484ee280
Story: #2004661
Task: #28628
Rank all existing versions of a resource in a convergence stack to improve
the likelihood that we find the best one to update.
This allows us to roll back to the original version of a resource (or
even attempt an in-place update of it) when replacing it has failed.
Previously this only worked during automatic rollback; on subsequent
updates we would always work on the failed replacement (which inevitably
meant attempting another replacement in almost all cases).
Change-Id: Ia231fae85d1ddb9fc7b7de4e82cec0c0e0fd06b7
Story: #2003579
Task: 24881
Use a single write to the database to convert each resource. Add a
method to the versioned object class that encapsulates the DB-specific
information, and get rid of the Resource.set_requires() classmethod that
just calls a method on the versioned object instance that's passed to
it.
Change-Id: Ieca7e0f0642c38c44fb8d7729333a0ccd93c9cb4
When migrating stacks to convergence, if there are resources in
the database that are not in the current_template_id of the stack,
they are possibly of no isue, so it would better to delete those
resources from db to avoid any future update issues.
Change-Id: Ica99cec6765d22d7ee2262e2d402b2e98cb5bd5e
Story: #2004071
Task: 27092
Use '!=' instead of 'is not' to compare strings.
In practice, short strings that appear in the source code are interned
in CPython, but this is implementation-specific.
Change-Id: If3f305c2d647fcd7515cb0a326a30f4eda93acd3
Ensure that the StackDefinition is populated with all of the resource data
(i.e. refid and attribute values) prior to doing a legacy update. This
ensures that it is available for the purposes of obtaining the frozen
definitions of the existing resources prior to starting the update.
Obtaining the frozen definition may rely on resolving live data when there
are no properties data stored (hopefully only very old stacks) or when
fields other than properties (e.g. metadata) contain intrinsic functions.
Change-Id: Ib443cebc7b832a6cdb6583bb6cc830452357c392
Story: #2003987
Task: 26930
If there are resources in a template that don't exist in the database at
the time of a stack check (or suspend or resume) operation, running the
stack check will cause them to be stored in the database. Since these
operations have not been converted to convergence (story 1727142), they do
not set the current_template_id as a convergence update would. If this
occurs then the stack will be unrecoverable.
To avoid this, when convergence is enabled store any missing resources in
the same manner that a convergence update would, prior to running the stack
check/suspend/resume.
Just in case, make sure the stack doesn't get stuck if we do end up in the
wrong state, by not trying to load a template with None as an ID.
Change-Id: Iedba67c5de39dc2d58938da5505dda5dd147c130
Story: #2003062
Task: 23101
When we hold a StackLock, we defer any persistence of COMPLETE or FAILED
states in state_set() until we release the lock, to avoid a race on the
client side. The logic for doing this was scattered about and needed to be
updated together, which has caused bugs in the past.
Collect all of the logic into a single implementation, for better
documentation and so that nothing can fall through the cracks.
Change-Id: I6757d911a63708a6c6356f70c24ccf1d1b5ec076
If an exception occurs while doing the initial setup of a convergence
traversal (including sending the check_resource messages for leaf nodes),
mark the stack as failed. If we don't do this then any error or exception
in this method will cause the stack to hang IN_PROGRESS.
Change-Id: Ib8231321e823634a3dc23cff9a1c7d560f64fd6e
Story: #2003125
Task: 23245
Wait for the legacy stack to move to the IN_PROGRESS state before returning
from the API call in the stack update, suspend, resume, check, and restore
operations.
For the stack delete operation, do the same provided that we can acquire
the stack lock immediately, and thus don't need to wait for existing
operations to be cancelled before we can change the state to IN_PROGRESS.
In other cases there is still a race.
Change-Id: Id94d009d69342f311a00ed3859f4ca8ac6b0af09
Story: #1669608
Task: 23175
Previously when doing a delete in convergence, we spawned a new thread to
start the delete. This was to ensure the request returned without waiting
for potentially slow operations like deleting snapshots and stopping
existing workers (which could have caused RPC timeouts).
The result, however, was that the stack was not guaranteed to be
DELETE_IN_PROGRESS by the time the request returned. In the case where a
previous delete had failed, a client request to show the stack issued soon
after the delete had returned would likely show the stack status as
DELETE_FAILED still. Only a careful examination of the updated_at timestamp
would reveal that this corresponded to the previous delete and not the one
just issued. In the case of a nested stack, this could leave the parent
stack effectively undeletable. (Since the updated_at time is not modified
on delete in the legacy path, we never checked it when deleting a nested
stack.)
To prevent this, change the order of operations so that the stack is first
put into the DELETE_IN_PROGRESS state before the delete_stack call returns.
Only after the state is stored, spawn a thread to complete the operation.
Since there is no stack lock in convergence, this gives us the flexibility
to cancel other in-progress workers after we've already written to the
Stack itself to start a new traversal.
The previous patch in the series means that snapshots are now also deleted
after the stack is marked as DELETE_IN_PROGRESS. This is consistent with
the legacy path.
Change-Id: Ib767ce8b39293c2279bf570d8399c49799cbaa70
Story: #1669608
Task: 23174
When deleting a snapshot, we used the current resources in the stack to
call delete_snapshot() on. However, there is no guarantee that the
resources that existed at the time the snapshot was created were of the
same type as any current resources of the same name.
Use resources created using the template in the snapshot to do the
snapshot deletion.
This also solves the problem addressed in
df1708b1a8, whereby snapshots had to be
deleted before the stack deletion was started in convergence because
otherwise the 'latest' template contained no resources.
That allows us to once again move the snapshot deletion after the start of
the stack deletion, which is consistent with when it happens in the legacy
path. Amongst other things, this ensures that any failures can be reported
correctly.
Change-Id: I1d239e9fcda30fec4795a82eba20c3fb11e9e72a
Deleting a tenant that has active stacks would have to issue
a stack delete twice per stack to delete those stacks,
that was because the delete_trust would fail, but credentials were still
cleared.This change just logs the error but does not fail the stack delete.
Change-Id: I9d770a91b20d1db137b3fc313c794fcee4a5e4bf
Story: 2002619
Task: 22248
When purging events we are only able to (efficiently) search for
references to resource properties data from events and resources in the
same stack. This results in foreign key constraint errors if the
resource properties data is referenced from a backup stack. To avoid
this, don't reuse resource properties data IDs after moving a resource
between the backup and main stacks, but duplicate the data instead.
Change-Id: I93329197c99a2dba37b0e1dbd7efe7b2b17bc036
Story: #2002643
Task: 22510
During the original prototype development, I missed the fact that the
`needed_by` field of a resource is no longer needed for the actual
convergence algorithm:
c74aac1f07
Since nothing is using this any more, we can avoid an unnecessary DB
write to every resource at the start of a stack update. For now, just
write an empty list to that field any time we are storing the resource.
In future, we can stop reading/writing the field altogether, and in a
subsequent release we could drop the column from the DB.
Change-Id: I0c9c77f395db1131b16e5bd9d579a092033400b1
In convergence, resources can continue to exist in the database that are
no longer part of the latest template. When calling
Resource.required_by() for such resources, we still want to get a list
of those resources that depend on them. Previously we did this using the
`needed_by` field in the resource. Since this is the only actual use of
needed_by, get the information from the Stack's convergence graph
instead (which is generated from the resources' `requires` field and
ignores `needed_by`). This eliminates any risk that the `requires` and
`needed_by` fields get out of sync, and allows us to get rid of
`needed_by` altogether in the future.
Change-Id: I64e1c66817151f39829d5c54b0a740c56ea8edad
At the beginning of a convergence traversal, log the traversal ID along
with the dependency graph for the traversal. This could be useful in
debugging. Also, log it at the DEBUG, not INFO level.
Change-Id: Ic7c567b6f949bdec9b3cface4fa07748fbe585eb
This refactors the building of schema from parameter validation to use a
new method (which doesn't keep stacks in memory), and use that new
method for providing proper schema for resource group when the size is
0.
Change-Id: Id3020e8f3fd94e2cef413d5eb9de9d1cd16ddeaa
Closes-Bug: #1751074
Closes-Bug: #1626025
Because of quotas, there are times when creating a resource and then
deleting another resource may fail where doing it in the reverse order
would work, even though the resources are independent of one another.
When enqueueing 'check_resource' messages, send those for cleanup nodes
prior to those for update nodes. This means that all things being equal
(i.e. no dependency relationship), deletions will be started first. It
doesn't guarantee success when quotas allow, since only a dependency
relationship will cause Heat to wait for the deletion to complete before
starting creation, but it is a risk-free way to give us a better chance of
succeeding.
Change-Id: I9727d906cd0ad8c4bf9c5e632a47af6d7aad0c72
Partial-Bug: #1713900
The OS::Heat::HARestarter has been deprecated since Kilo. Now is the
time to eliminate support and hide it from the documentation.
This replaces it with a placeholder resource (like OS::Heat::None) and
marks it as hidden.
Change-Id: I56cd1f2d0b3323399ef02c3a0a05d79cc69af956
Takashi Kajinami