Always loading the raw template in situations where we didn't need it -
e.g. in identify_stack, where we just want the name + id (given one of
them), or when getting the summary stack list - uses up DB bandwidth and
memory unnecessarily.
This partially reverts commit 3ab0ede98c.
* The eager_load option to get_stack() is reinstated, but with the default
flipped to True. In places where we explicitly do not want to load the
template, we pass False.
* stack_get_by_name() no longer eagerly loads the template. There were no
instances of this where we subsequently use the template.
* stack_get_all() acquires an eager_load option, with the default set to
False. Direct users of objects.stack.Stack.get_all() will not eagerly
load by default, but users of engine.stack.Stack.load_all() will get the
template eagerly loaded. This practically always corresponds to what you
want.
Change-Id: I1f156c25ea26322be5b606a61dd77d80cadd65fc
Related-Bug: #1626675
We used to try to acquire the stack lock in order to find out which engine
to cancel a running update on, in the misguided belief that it could never
succeed. Accordingly, we never released the lock.
Since it is entirely possible to encounter a race where the lock has
already been released, use the get_engine_id() method instead to look up
the ID of the engine holding the lock without attempting to acquire it.
Change-Id: I1d026f8c67dddcf840ccbc2f3f1537693dc266fb
Closes-Bug: #1624538
We can use admin_context to have access to stacks
and software configs across projects. This removes
the tenant_safe flag from rpc and db api. This is
backward compatible with older rpc clients.
We still support use of global_tenant flag for listing
stacks and software configs. However, by default
an admin(user with admin role in admin_project)
would not need that.
Change-Id: I12303dbf30fb80290f95baba0c67cdf684f5f409
Context is added as the first argument to stack_lock_* db functions
but is currently ignored for getting the session used for stack lock
operations.
This is required later for bug 1479723 but is added in its own change
here to ease reviewing load.
Change-Id: Ieb3e4e2ee67150777cbe1e961d0d1806cf1f7e46
Related-Bug: #1479723
We initially didn't implement this class as a context manager because it
was being used exclusively to lock operations that were happening in an
asycnronous thread (hence the thread_lock() context manager). However, we
now have some uses for it in the same thread (see
Ifa48b179723a2100fff548467db9e162bc669d13), so it makes sense to be able to
use the lock object itself as a context manager.
Change-Id: Ib90df8f8a149b57b0588d4fc9951f7f035ad7567
Having a stack which is wedged UPDATE_IN_PROGRESS due to another bug,
restarting heat-engine should have resulted in the stack being reset to
UPDATE_FAILED. However the reset was skipped because the lock engine_id
was None.
This change removes the engine_id check since it is possible in practice
for an UPDATE_IN_PROGRESS stack to not be locked.
Since it is possible for the stack to change state in the unlocked
window, a check is added that the stack is still IN_PROGRESS before the
reset is performed.
Change-Id: I1739ccbdf75af35aac5be16b99200975df58b8e2
Closes-Bug: #1514615
When the engine starts and tries to reset the status of stacks in
progress, it tries to acquire the stack lock on them. The stack lock
queries the database to retrieve the name of the stack, but does a
tenant-scope query, thus fails when as we use the admin context during
startup. This fixes it by passing tenant_safe=False.
Change-Id: I4393fdb601eedb36d7288afd57fb5a1c96af7216
Closes-Bug: #1516712
Only acquire() function in stacklock needs to load whole stack,
other function only needs stack id. This patch only pass
stack id for stacklock so that in some place we don't need
to load whole stack.
Change-Id: Iaad5f4a871f925c4052d159a5d95184681d5bd28
Closes-Bug: #1441972
When stack is in status IN_PROGRESS and engine service went down,
the status of stack will forever remain in IN_PROGRESS. This patch
add a db apid to get engine_id from stacklock and try to reset the
stack status to FAILED when engine is back.
Closes-Bug: #1382320
Change-Id: Ica856bb0d56c23a4423fb9476c1986aaacf24108
This was only working for one heat engine as it was sending
the listener messages to a topic of "engine_id".
Change-Id: Ib542d1ebb85b004f5a67213e678882c5a723dc0e
Closes-bug: 1433821
In oslo.messaging to identify a "rpc server" we have to set the
server attribut of the target. But currently, the engine-listener uses
the hostname of the host, that make all workers on the host to be
identified has same "rpc server" by oslo.messaging, by chance because
heat uses a different topics for each workers, that works.
This change uses the engine id instead of the hostname to identify the
"rpc server" and use the same topic for all engine listeners.
Related-bug: #1414674
Change-Id: I359709ab15ee24ac2459826264adf2373dbb067f
The oslo-incubator log modlule has been removed, so port to the oslo_log
library. Note this uses the new (non namespaced, e.g oslo.log) import
convention, we'll need to align other imports in a future commit.
Some import reordering was required due to pedantic H30[57] checks, and
the services have all been converted to initialize the oslo_log library
as this is done differently to the log.py in incubator.
Change-Id: Ib5a97123fe1b287bc531e42d7887c13ba6205628
Should not to release the stack lock if acquire failed,
such as stack action in progress.
Change-Id: I2a519c47306065b0a3412a8c6faa64006dea691f
Closes-Bug: #1418360
Use parentheses instead of backslash for line continuation.
Plus, usage of escaped quotes in long strings is avoided where it makes
sense.
Change-Id: If2e78012b85a4430c6f03f65784cac2d032cf116
Currently, Log translation is motivated by oslo's move to prioritized
translation of strings, as documented at
https://wiki.openstack.org/wiki/LoggingStandards#Log_Translation
- add log translation hints for warning, error and info levels
- move from LOG.warning to LOG.warn
- remove use of % as a string formatter, use the log functionality
instead
Partial implements blueprint log-translation-hints
Change-Id: Ib91983e6e61b4d6ddbbed0f95dc98cbde1d4f673
H201 got stricter in hacking 0.9, so fix new violations and
re-enable gating.
Change-Id: Ie79b04bc1ae2d4784318c19661584d02b7bee088
Implements: blueprint reduce-flake8-ignore-rules
Move from oslo RPC to oslo.messaging.
Implements: blueprint oslo-messaging
Co-Authored-By: sdake@redhat.com
Change-Id: I2d222c248dd2cd405b8ec35c4c8198ed001fb69f
This is to give the engine service methods a means to use the stack lock
that guarantees it will be released if there is an exception.
Change-Id: I6d0e1e7c22f4e74eeb069f980bc09a14f3740beb
Partial-Bug: #1321011
Commit https://review.openstack.org/#/c/94255/1 introduces check for
translating log messages. This check uses in pattern name 'LOG', so in
heat code also should be used this name intead of 'logger'. Also same
name is used in other projects.
Change-Id: Iba24c7eb1c13e68a91d090d7bcdbcb40d6e16071
Partial-Bug: #1321283
According to the OpenStack translation policy available at
https://wiki.openstack.org/wiki/LoggingStandards debug messages
should not be translated. Like mentioned in several changes in
Nova by garyk this is to help prioritize log translation.
Change-Id: I770dfc2fd474123a8ccc37311ef69d0c03e85545
Closes-Bug: #1318713
This patch is one in a series to re-enable H306 style check rule
(imports are in alphabetical order).
It touches engine files (without resources).
Implements: blueprint reduce-flake8-ignored-rules (partial)
Change-Id: Ifd36fc49199c48af891bd8e7cc2d2262a53032eb
Decorators for expected exceptions in comomn/rpc are handled by
allowed_rpc_exception_modules config option and should not be needed
in engine.py
Change-Id: I44cad13c1f717a12db6eca6bb7ccad312ce306be
blueprint: oslo-messaging
Check that an engine is still alive before trying to stop the stack
during a stack-delete.
Change-Id: I3ce87dcefd5c4b5c542c3dc0d288655433bf69c4
Closes-Bug: #1279010
Since multi-engine was merged, it is no longer possible to delete a
stack which is create/update IN_PROGRESS. This re-introduces that
ability.
If the current engine has an active stack lock, stop any active
threads, remove the lock, and delete the stack. If another engine has
the active stack lock, send an RPC command to the remote engine to stop
any active threads and remove the lock, and then delete the stack
locally. If no engines have the an active stack lock, acquire the lock
and delete the stack.
Closes-Bug: #1262012
Change-Id: I188e43ad88b98da7d1a08269189aaefa57c36df2
Previously, stack locks would not be released after a stack-delete
because the stack ID was being passed to db_api.stack_lock_release() as
None. To resolve this, pass the stack ID to the greenthread-linked
release function while it still exists.
Change-Id: I260c4c5512bd2069f134e9d6ac6ea07565e443b0
Closes-Bug: #1276336
Introduce a ThreadGroupManager class to manage active ThreadGroups for
engine services and move the thread-related EngineService methods to
ThreadGroupManager. This will make it easier for multiple engine
services to manage thread groups.
Partial-Bug: #1262012
Change-Id: I1ce7785021726c274a294ef402b036540d395a29
Jenkins occasionally fails in the 'check-tempest-dsvm-postgres-full'
test. The tempest tests sometimes try to delete the same stack twice
and an ActionInProgress exception is raised. This results in ERROR
messages in the log file, which causes the tempest test to fail.
The fix is to decorate the StackLock.acquire method with
client_exceptions, which "allows the declaration of expected exceptions
that the RPC layer should not consider fatal, and not log as if they
were generated in a real error scenario."
Closes-Bug: #1261433
Change-Id: I0692b9d5cf03501ba532b40abf4e567134cda057
Implements a distributed stack lock using the database to avoid race
conditions when multiple engines are deployed.
Blueprint multiple-engines
Change-Id: If3c47dafcc5bc1b2188b7737205291bbab8bc231
This is a short term backout until it gets fixed.
This reverts commit c30530f126.
Closes-bug: #1250797
Change-Id: Ide1dc4a9d1469dee033e8bc08d8ab4da96f79fc0
Implements a distributed stack lock using the database to avoid race
conditions when multiple engines are deployed.
Blueprint multiple-engines
Change-Id: I7fba8808d8a4efbe7fd8fdff94fd9a53f3841c8b