Well, we actually remove it but keep the entry point to avoid breaking
scripts. This has been around since 2016 and anyone that hasn't upgraded
now ain't upgrading.
This is one less set of APIs to worry about when removing the use of
autocommit.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I43051b3ca6e292f1164d750b04d7a440012b2ef5
We don't have another ORM to content with here. Simplify
'heat.db.sqlalchemy' to 'heat.db'.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Id1db6c0ff126859f436c6c9b1187c250f38ebb62
This is significantly easier than Cinder, Nova etc. as Heat hasn't had
any database migrations in multiple cycles, meaning we don't need to
worry about having to apply any older sqlalchemy-migrate migrations
before switching to alembic. Instead, we simply need to determine we're
upgrading a deployment that was previously using sqlalchemy-migrate,
upgrading a deployment that has already migrated to alembic, or
deploying a new deployment, adjusting accordingly.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I808af9cb21ba21808209b1daddac7426f4cad310
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:
1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.
2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.
Also replace policy.json to policy.yaml ref from doc and code.
[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Change-Id: I1aa12bcd2638390f25d57ce8abeeec248121dc02
Monkey patch the original current_thread to use the up-to-date _active
global variable. This solution is based on that documented at:
https://github.com/eventlet/eventlet/issues/592
Change-Id: I0540923755ac3969b584eeba2e19c037a7f2c261
Story: 2007614
A simple matter of having log lines fit
with all the others. It will now match the
ambiguous standard.
Also change comments inline to alias with log fixes.
Change-Id: I4a2ed6134f70c2965811f75ccb6ab8221fa0e960
Story: #2007319
Task: 38830
Six is in use to help us to keep support for python 2.7.
Since the ussuri cycle we decide to remove the python 2.7
support so we can go ahead and also remove six usage from
the python code.
Review process and help
-----------------------
Removing six introduce a lot of changes and an huge amount of modified files
To simplify reviews we decided to split changes into several patches to avoid
painful reviews and avoid mistakes.
To review this patch you can use the six documentation [1] to obtain help and
understand choices.
Additional informations
-----------------------
Changes related to 'six.b(data)' [2]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
six.b [2] encode the given datas in latin-1 in python3 so I did the same
things in this patch.
Latin-1 is equal to iso-8859-1 [3].
This encoding is the default encoding [4] of certain descriptive HTTP
headers.
I suggest to keep latin-1 for the moment and to move to another encoding
in a follow-up patch if needed to move to most powerful encoding (utf8).
HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5].
Note that this commit message is autogenerated and not necesserly contains
changes related to 'six.b'
[1] https://six.readthedocs.io/
[2] https://six.readthedocs.io/#six.b
[3] https://docs.python.org/3/library/codecs.html#standard-encodings
[4] https://www.w3schools.com/charsets/ref_html_8859.asp
[5] https://www.w3schools.com/html/html_charset.asp
Patch 2 of a serie of 28 patches
Change-Id: I2795dee87f0e27b64820686acfc614ac2ba19a4f
This adds basic framework for heat-status upgrade
check commands. For now it has only "check_placeholder"
check implemented.
Real checks can be added to this tool in the future.
Change-Id: I83629184b49a6cf91928df702db23156433d99f6
Story: 2003657
Task: 26131
Stack and its nested stacks should be in *_COMPLETE state for
it to be migrated. Also, we should not allow migration of a
nested stack.
This also changes stack_get_all_by_owner_id() to not select the
backup stacks. This seems to be used only in convergence code
path and migrate_convergence_1().
Change-Id: Icd54465d0c593557a12d853ddee4ee8ce6483499
Closes-Bug: #1767962
Story: #1767962
Task: #17363
Most files are using getLogger(__name__) so that logging statements
are associated with the file they came from, but there are several
places where this practice has not been followed. This changes to
use __name__ consistently for most code. Per review comments,
commands and wsgi apps are instead updated to use the command/app
name.
Change-Id: Ic2594b88dd818df5486800e4759848d43edc24d1
This patch removes the API, the next set of patches in the
series would remove stack watch service and related
WatchRule implementation.
Change-Id: I8b0472be862907298c8da51f435b5d8b19610ec3
Partial-Bug: #1743707
Alphabetically order your imports by the full module path.
Organize your imports according to the Import order template and
Real-world Import Order Examples below.
{{stdlib imports in human alphabetical order}}
\n
{{third-party lib imports in human alphabetical order}}
\n
{{project imports in human alphabetical order}}
\n
{{begin your code}}
Change-Id: Ie3a9f1dd36272f5512f719f4db8a219015c99fec
New releases of oslo.config support a 'mutable' parameter to Opts.
This is only respected when the new method mutate_config_files is
called instead of reload_config_files. Heat delegates making this call
to oslo.service, so how do we switch?
Icec3e664f3fe72614e373b2938e8dee53cf8bc5e allows services to tell
oslo.service they want mutate_config_files to be called by passing a
parameter, which this patch does.
This allows Heat to benefit from
I1e7a69de169cc85f4c09954b2f46ce2da7106d90, where the 'debug' option
(owned by oslo.log) is made mutable. We should be able to turn debug
logging on and off by changing the config and sending SIGHUP.
Part of bp:mutable-config
Change-Id: I885f5e088440d3d8b6fb3506506598d7667e7083
Add a subcommand to heat-manage to migrate resource and events
properties data from the legacy db locations to the new. I.e., migrate
properties data out of the legacy columns in the resource and event
tables into the recently added resource_properties_data table. No
attempt at de-duplication between resources and events is made for the
migration: a new row is created in resource_properties_data for every
row that has legacy properties data in the resource or event tables.
Change-Id: I364d509c357539d1929eb2e40704e60049469ea2
This changes newly added heat-manage command for migrating
non-convergence stacks to convergence, to use '_' like other
commands. Also updates doc with the command.
Change-Id: I2ea5beb1c71cf51dceee7df6af6a6827d920c179
The db.api module provides a useless indirection to the only
implementation we ever had, sqlalchemy. Let's use that directly instead
of the wrapper.
Change-Id: I80353cfed801b95571523515fd3228eae45c96ae
Avoid large sql "in" clauses by operating on smaller batches of stacks
at a time.
To avoid transaction overhead and contention on the resource table,
the first deletions occur outside of a transaction (are
autocommitted). This is OK because the purge is re-rentrant -- we
won't lose any stack_id's to delete if something goes wrong before the
conn.begin() block. That is, we will not orphan any rows if the purge
is run multiple times where an error occurs.
Change-Id: I9edf0558ed54820842193560e323df6501411d1d
This patch implements a new heat-all command that can be used
to launch a single process version of the configured heat services.
The end user can control which services are launched by setting
'enabled_services' in the heat config file:
[heat_all]
enabled_services = api,engine
One use case for this launcher would be to launch a single process heat
using rpc_backend = fake, connection=sqlite://heat.db, to have access
to a minimal all in one Heat API/Engine for TripleO undercloud
deployments via Heat templates.
Change-Id: Ic35bdc455fe1397ac575522af0c8fca96dd95676
Run `heat-manage migrate-convergence-1 <stack_id>` to migrate
legacy stack to convergence engine.
Heat engine is used for doing migration i.e. migration can't
be done offline.
Change-Id: Ie7c2498b37937438f16d154b154b3a6ecbf9ff74
Implements-bp: convergence-migrate-stack
Add project-id argument to heat-manage purge_deleted command in order
to be able to hard delete DB entries for a specific project.
Change-Id: Ifffe5657a40ce97db9d059ff1516b8e1eb801132
Implements: bp heat-manage-purge-deleted-tenant
The raw_input() raises NameError: name 'raw_input' is not defined in python3.
This patch fixes it by replacing raw_input with input to make PY3 compatible.
Change-Id: I50aad3161893250dab1d60af4569bb65d7edef5a
Closes-Bug: #1595827
In case the "heat-manage update-params" utility hangs in the middle of
a run, it is useful to know the exact raw_template or resource that
was being processed. Verbose logging of each raw_template or resource
that is being processed will aid in debugging and checking for corrupt
data (if the heat-manage process has to be killed).
Change-Id: Ic552bdaf234c7b9a6a76c345aab816a171b404ab
Adds a new heat-manage reset_stack_status to recover from specific
crashes that leaves resources in progress. It removes resource hooks and
stack locks as well.
Closes-Bug: #1561214
Change-Id: I70fa5857c959bc5f1424d562ff8b7740331b5328
The default values needed for heat's implementation of cors
middleware have been moved from paste.ini into the configuration
hooks provided by oslo.config. Furthermore, these values have been
added to the default configuration parsing. This ensures
that if a value remains unset in heat.conf, it will be set
to use sane defaults, and that an operator modifying the
configuration file will be presented with a default set of
necessary sane headers.
Change-Id: Ie3791007b33788829417ce508a3c719ae626bbce
Closes-Bug: 1551836
Downstream test environments are frequently having failing stacks with
error messages like:
MessagingTimeout: resources[0]: Timed out waiting for a reply to
message ID ...
These environments generally have 1 or 2 cores, so only spawn one or two
engine workers. This deadlocks with stacks that have many nested stacks
due to engine->engine RPC calls.
Even our own functional tests don't work reliably with less than 4
workers, and the workaround has been to set that explicitly in
pre_test_hook.sh.
This change sets the default minimum number of workers to 4, but still
matches workers to cores for larger servers.
This change also moves the default evaluation to heat.cmd.engine so that
generated configuration doesn't get a inappropriate default value.
Change-Id: Iae6b3956bad414406d901bb2213c9ec230ff4304
Closes-Bug: #1526045
During the troubleshooting of issues, admin needs to
decrypt the values stored in the resource_data table.
This patch addes below command to address this need.
heat-manage resource_data_list <resource_id>
Closes-bug: #1524295
Change-Id: I5bb814c0873f58b3968f312c96bcabcd3024085d
Previously we used "nargs='?'" parameter for 'crypt_operation' option.
It leads to situation, when we execute this command without any result
or error.
Now when user will try to execute this command he will get message, that
should specify 'crypt_operation' option.
Note, that we used same parameters for other heat-manage commands.
However, it can be used, because if user will not provide these options,
then commands will be executed with default values (sometimes it's None).
All these default values are expected in utility functions or db
functions.
Also comments were added for clarification commands and their arguments.
Change-Id: I561bc7fc83e6d845c9b59d85c59650e640001190
db_sync command has two positional arguments, but in code we use only
one. Also original db_sync command for db API get only one parameter
"version".
Change-Id: I65732b16fbdc093ad2c7a3a521c3b75ae7183585
As a follow-up to Ibfd54e7b1bae700e4fb49a32e5bf7c00d1f104d9,
moving all the scripts from bin/ to setuptools entry points.
The devstack change is here: Iffb6d09bfef593d854b38e68200ae6039c4727e7
partial blueprint upgrade-tests
Change-Id: Ic8f5b8dc85098de752bbf673c4b15c06fdc4162a
Adding support to encrypt/decrypt parameters through heat-manage
command.
Change-Id: I2cd7b8837156ebe892d591f48ad2b1c6ecca5f9d
Implements: blueprint encrypt-hidden-parameters
Co-Authored-By: Jason Dunsmore <jasondunsmore@gmail.com>
The oslo-incubator log modlule has been removed, so port to the oslo_log
library. Note this uses the new (non namespaced, e.g oslo.log) import
convention, we'll need to align other imports in a future commit.
Some import reordering was required due to pedantic H30[57] checks, and
the services have all been converted to initialize the oslo_log library
as this is done differently to the log.py in incubator.
Change-Id: Ib5a97123fe1b287bc531e42d7887c13ba6205628
Adds required REST API, Db model and engine service
changes for reporting the heat engine service status.
Change-Id: I3ef29c1efd2015d62eec1033ed3a8c9f42e7a6e2
Implements: blueprint heat-manage-service-list
"DocImpact"
Since i18n.install() is deprecated, remove it from heat codes and
import i18n._() to where it needed.
blueprint oslo-i18n
Change-Id: Icefada18b5a33112b425cd90d31d3a6a5f06188a
Common db code was updated in oslo. The most important thing is that
engine instances don't stored anymore in oslo.db - ce69e7f.
This patch moves methods `get_engine` and `get_session` to module
`heat.db.sqalchemy.api`.
Latest commit in oslo related to db module:
2fd457bf2ccbeb2b84ffb204778b6417cd5405ba
Change-Id: Iaa2e9ba26e824c678c698914170e3dffbf1c5c95
Stephen Finucane