Commit Graph

278 Commits

Author SHA1 Message Date
Takashi Kajinami ebe15a1ba5 Enable healthcheck middleware
The healthcheck middleware from oslo.middleware library is now commonly
enabled in OpenStack services, for service healthcheck (often used by
load balancers such as haproxy). This enables the middleware so that
operators can use the healthcheck endpoint more easily. This also
replaces the oslo.middleware config entry point by more specific sub
entry points, to avoid rendering options from unused middlewares such
as sizelimit.

Change-Id: If3296e651e7f20a3f5314c085cc99ce4b004c065
2024-02-26 01:41:31 +09:00
likui 773115adce Remove the idle_timeout option.
The option was replaced 4 years ago[1] by connection_recycle_time. The
option is not anymore present in sqlalchemy. It's the good time to
remove it.

[1] 6634218415
[2] https://review.opendev.org/c/openstack/oslo.db/+/778441

Change-Id: Id288e09688062b007d57f09aebcd7c447e56f04d
2023-07-20 14:26:40 +08:00
wu.chunyang 44c3cdef5c Rename app.wsgi to app_wsgi.py
1. app.wsgi is a python script and should be end with py extension.
2. Generate trove-wsgi script which is needed by kolla to support
https and ipv6.
3. Ignore build directory in git
4. update docs

Change-Id: Ibbd6ecb8db25aa3a443d3f526f4cf6d418eb26f3
2022-08-25 11:10:33 +08:00
Ghanshyam Mann bcb8992f99 [goal] Deprecate the JSON formatted policy file
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to do two things:

1. Change the default value of '[oslo_policy] policy_file''
config option from 'policy.json' to 'policy.yaml' with
upgrade checks.

2. Deprecate the JSON formatted policy file on the project side
via warning in doc and releasenotes.

Also replace policy.json to policy.yaml ref from doc and tests.

[1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html

Change-Id: I790409da69df8479ad2fe152b15c32ba45067c23
2020-12-26 05:30:18 +00:00
Lingxian Kong aa1d4d2246 Datastore containerization
Significant changes:

* Using docker image to install datastore.
* Datastore image is common to different datastores.
* Using backup docker image to do backup and restore.
* Support MariaDB replication
* Set most of the functional jobs as non-voting as nested
  virtualization is not supported in CI.

Change-Id: Ia9c97a63a961eebc336b70d28dc77638144c1834
2020-05-27 10:31:50 +12:00
Lingxian Kong 429c39890e Support ip address type for instances
Include address type in getting instance response.

* Deprecate confip option network_label_regex as we don't reply on Nova
  to get addresses, network names don't make any sense.
* Add 'addresses' in instance API response, keep 'ip' as is but mark
  it deprecated in API doc, python-troveclient shouldn't break.

Story: 2007562
Task: 39445

Change-Id: Ia0458b5ddae8959ce29c17e444e1a51a026283cd
2020-04-26 09:56:50 +12:00
Hirotaka Wakabayashi 2f7f2c743b Fixes the following syntax error of etc/apache2/trove apache conf
$ apache2ctl configtest
AH00526: Syntax error on line 16 of /etc/apache2/sites-enabled/trove-api.conf:
Invalid command 'through', perhaps misspelled or defined by a module not included in the server configuration
Action 'configtest' failed.
The Apache error log may have more information.

I refere the manual to configure my trove env.
https://docs.openstack.org/trove/latest/install/apache-mod-wsgi.html

Here are three steps to reproduce the error.

Step1. Install apache2 and mod_wsgi
-----------------------------------
$ sudo apt-get install apache2 libapache2-mod-wsgi-py3

Step2. Git clone and Configure apache
-------------------------------------
$ git clone https://opendev.org/openstack/trove.git; cd trove
$ sudo cp etc/apache2/trove /etc/apache2/sites-available/trove-api.conf
$ sudo a2ensite trove
ERROR: Site trove does not exist!

I files this in https://bugs.launchpad.net/trove/+bug/1867811

$ sudo a2ensite trove-api
Enabling site trove-api.
To activate the new configuration, you need to run:
  systemctl reload apache2

Step3. Syntax error happens
----------------------------
Before starting apache2, Add "stack" user and mkdir "/var/log/httpd" directory.
$ sudo useradd -s /bin/bash -d /opt/stack -m stack
$ sudo mkdir -p /var/log/httpd

$ apache2ctl configtest
AH00526: Syntax error on line 16 of /etc/apache2/sites-enabled/trove-api.conf:
Invalid command 'through', perhaps misspelled or defined by a module not included in the server configuration
pAction 'configtest' failed.
The Apache error log may have more information.

$ apache2ctl configtest
AH00526: Syntax error on line 16 of /etc/apache2/sites-enabled/trove-api.conf:
Invalid command 'through', perhaps misspelled or defined by a module not included in the server configuration
Action 'configtest' failed.
The Apache error log may have more information.

$ sudo vi /etc/apache2/sites-enabled/trove-api.conf
$ diff  /etc/apache2/sites-enabled/trove-api.conf /etc/apache2/sites-enabled/trove-api.conf.orig
16c16
< # through mod_wsgi
---
> through mod_wsgi

$ apache2ctl configtest
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
Syntax OK

I have tested on Ubuntu 18.04.4 LTS.

$ uname -a
Linux ubuntu-bionic 4.15.0-76-generic #86-Ubuntu SMP Fri Jan 17 17:24:28 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
$ cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.4 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.4 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

Signed-off-by: Hirotaka Wakabayashi <hiwkby@yahoo.com>
Change-Id: Ia010941fd9fde2036672f6e6ae8ac488d63c1d4a
2020-03-19 00:50:15 +00:00
Lingxian Kong 602c4d42de Improve the function tests
- Execute test groups in serial to make sure no more than 2 database
  instance are created at the same time.
- Remove some unneccesary tests
- Remove unneeded datastore, e.g. 'Test_Datastore_1'
- Remove unsupported trovestack subcommands
- Move unsupported DIB elements to the 'deprecated-elements' folder
- Decrease default value of 'agent_call_high_timeout' to 5min
- Add initial_deplay for pooling task
- Use socket file to connect with database instead of using localhost
  IP

Change-Id: Ie5030a671fbeb453eafa6cbe04e08da7b52e33c9
2020-01-28 14:27:52 +13:00
Lingxian Kong c33fa67066 Support to create public trove instance
- The users need to specify the network to create Trove instance, but
  trove-taskmanager will create port in that network for Nova instance
  creation. Using port gives Trove more capabilities to define how the
  database service is exposed.
- Deprecate ICMP protocol for the instance.
- Restrict 'nics' parameter for creating instance.
- Add 'access' parameter for creating instance.
- Add 'public_network_id' option in order to create floating IP for the
  instance.
- Do not create records for security groups, but Trove can still delete
  existing instances for backward compatibility.
- Delete unreasonable Host, Account, Storage API.

Story: 2006500
Task: 36468
Task: 36466
Change-Id: I80827e1ad5e6b130cbf94c2bb7a909c44d5cf1e5
2019-09-11 11:28:00 +12:00
Lingxian Kong 0ef474a70c Support keypair in devstack
Since Trove already supports to specify a Nova keypair when creating
instance for management convenience, devstack needs to be changed to
create the management keypair and add to Trove config file.

One extra change in this patch is to use a single config file for Trove
API, task-manager and conductor.

Change-Id: I1e6c4f4305104815bdf89b31776a4955de61bc89
Story: 2005429
Task: 30463
2019-08-29 15:41:58 +12:00
Lingxian Kong 44fbcfaaa6 Deprecate the config option default_neutron_networks
Use `management_networks` instead. `management_networks`will be used
as admin networks which will be attached to Trove instance
automatically.

Change-Id: I5c6004b568c3a428bc0f0a8b0e36665d3c5b3087
2019-03-26 14:14:31 +13:00
Marcin Piwowarczyk dc7303a135 Be compilance with latest oslo.messaging
Support for the [oslo_messaging_rabbit] section has been removed in
oslo.messaging==9.0.0 package [1].
That's why integration tests failures on build devstack step.

This is the moment where we need to use the transport_url directive
in the [DEFAULT] section instead.

Moreover rpc_backend property (which was used in trove to enable fake
RPC backend) has been removed from DEFAULT section and API tests
fails on TIME_OUT as they starts on rabbit RPC backend.

Fake RPC can be now configured as described here [2]:
transport_url = 'fake:/'

[1] https://docs.openstack.org/releasenotes/oslo.messaging/unreleased.html
[2] https://docs.openstack.org/oslo.messaging/latest/configuration/conffixture.html

Change-Id: Id6c5a9198d5a213cb085407a1d8b534e7c755f69
Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
2018-10-13 20:00:16 +02:00
caoyuan 38a5679dac Update auth_uri option to www_authenticate_uri
Option auth_uri from group keystone_authtoken is deprecated[1].
Use option www_authenticate_uri from group keystone_authtoken.

[1]https://review.openstack.org/#/c/508522/

Change-Id: I46bb0acaf1dec9bb4b91d4a2180b3e7ed66741c2
2018-04-11 22:28:23 +08:00
melissaml f83a247bd6 fix a typo in documentation
Change-Id: Ic53adfb74c628c8ca85b2fa2107999c1ae816833
2018-03-23 08:08:18 +08:00
Zuul 15c867672a Merge "Add functionality to define different Message and Notification destination" 2018-03-19 13:48:16 +00:00
Zhao Chao 71ebd353ca Generate policy sample file automatically.
A new entrypoint in setup.cfg and a config file are added for
using olso.policy helper script to generate the sample file.

A new tox target also is added to simplify the environment
setting up. Now policy sample file can be generated
automatically, so the in-repo sample file is no longer needed.

Co-Authored-By: Andrew Laski <andrew@lascii.com>
Partial-Implements: blueprint policy-in-code
Change-Id: Ic336fa154ccc05b5e9db3a8e751a484b1cc5aa9c
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
2018-02-12 18:11:22 +08:00
Zhao Chao 29362a18a4 Remove support of creating volume from Nova.
Current Nova server volume support is broken. Nova also declared the
'os-volumes_boot' will be deprecated in the future. As creating volumes
by cinderclient has been supoorted for a long time, we could just drop
support of Nova server volume.

This patch also migrate to the new block_device_mapping_v2 parameter of
Nova servers creating API.

Closes-Bug: #1673408
Change-Id: I74d86241a5a0d0b1804b959313432168f68faf89
Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
2018-02-09 14:30:50 +08:00
ZhongShengping 6ff047e743 Remove the heat related documents
This patch[1] removed heat related support.

[1]https://review.openstack.org/#/c/385513/

Change-Id: Ib1b6f9f36097ae043d92fef3b32cf1f0ebac9672
2018-01-17 16:17:25 +08:00
wangjun 1dcdf38bc2 Add functionality to define different Message and Notification destination
Sometime,huge notification messages will effect the rabbitMQ cluster's
performance.This commit adds functionality do define different Message
and Notification destination.

Closes-Bug: #1734823
Change-Id: I2ef4d008c9ca91a03d8c7e1380e0188bf6792595
2018-01-05 10:31:49 +08:00
Dai Dang Van dd6b22d47a Remove policy.json file
We already had default rule in code, so we should not
still define all of them again in policy file.
Besides, we should you yaml format for now instead json.

Another thing, we don't need to config policy file in
Devstack enviroment.

Change-Id: I783ba51695271d358764557899fe91e84620556d
2017-12-07 16:38:03 +07:00
Zuul 290cecb627 Merge "Comment out the option oslo_messaging_rabbit.rabbit_password" 2017-11-24 01:36:09 +00:00
Zuul c3c6b7fdc7 Merge "Open the volume_support of redis" 2017-11-09 08:08:31 +00:00
jiansong 5b7930446f Open the volume_support of redis
In the past, we support volume_support when we support
redis-cluster[1], but we did not update to conf[2].
we are making the sample conf files match the 
default configuration in code.

[1]:https://review.openstack.org/#/c/203702/
[2]:https://review.openstack.org/#/c/107117/

Closes-Bug: #1701469
Change-Id: I428e274fdb85621b21453780e90a8a0ab401ffd9
2017-11-03 02:14:05 +00:00
Zuul d48bb35cd1 Merge "Remove exists_notification_ticks from sample conf" 2017-10-23 17:43:05 +00:00
Stephen Ma 68e02bbdf7 Comment out the option oslo_messaging_rabbit.rabbit_password
The option oslo_messaging_rabblt.rabbit_password has been
deprecated. Therefore the option is commented out in the
sample config files etc/trove/ directory.

Change-Id: I613087217c88b8a5dbf8dfc3f51e48b8fe4c84c8
2017-08-21 11:17:16 -07:00
Fan Zhang 327eb48384 Remove inexistent option in install guide.
Remove notifier_queue_* because the options are no longer needed and
should be removed.

Closes-bug: #1706528

Change-Id: Idfd205473f1756a69d0d9607e9f8c1dd9780553c
2017-08-07 09:33:58 +08:00
Samuel Matzek 9f4d0f7f13 Remove exists_notification_ticks from sample conf
The exists_notification_ticks conf option was removed under
commit 08dc866fb2 but was not removed from the
sample configuration file.

Change-Id: I3608929bd1ce4cd6ec767665b6f60fd5d86127b4
2017-07-26 10:05:46 -05:00
Jenkins a42496e68d Merge "Wrong comment symbol in configuration file" 2017-06-16 05:26:11 +00:00
jiansong dbeff81d50 Add port 16379 to conf.sample
This is a point that is easily hidden. In the case of not open
port 16379, redis cluster can still be deployed successfully,
but this success is a false success, there is no data communication
between the various nodes. This is because 16379 is the port for
data port[1]. We set the default value in cfg, but in the
case of conf,it is easy to be covered. Adding 16379 to conf.sample
helps the user to reduce some problem caused by the redis
configuration.

[1]:https://redis.io/topics/cluster-tutorial

Change-Id: If517072c1c875df68106af14dac1802bb959d17e
2017-04-26 03:08:28 +00:00
Amrith Kumar 20d51467c7 enable trove-api behind mod-wsgi
This change enables behind mod-wsgi as part of the community pike goal
goal-deploy-api-in-wsgi.

The change includes:

    - the wsgi script files to run trove api under apache
    - updates to the devstack plugin
    - a basic deploy doc which explains this change

Change-Id: Icdd39b47a1be426e87a5d09f9e9d567af1974a9c
Depends-On: I3d6f6649430ee40879de15fee0b215dc32e8b666
Closes-Bug: #1681478
2017-04-25 06:54:29 -04:00
Amrith Kumar 191e3036e1 fix-gate: change trove auth URL's to reflect new URL settings
Trove's code was setup to use :5000 and :35357 for keystone's auth end
points. Change that to reflect the new settings that are /identity/
and /identity_admin/. See also [1]

[1] https://review.openstack.org/#/c/456344/

Change-Id: I3d6f6649430ee40879de15fee0b215dc32e8b666
Related: I46294fb24e3c23fa19fcfd7d6c9ee8a932354702
2017-04-23 00:17:13 +00:00
Peter Stachowski 83089aa5cc Add support for module-reapply command
Server side support for the new 'reapply' command.
This reapplies a given module to all instances that it had
previously been applied to.

Originally, a module designated live-update would automatically
be re-applied whenever it was updated.  Adding a specific
command however, allows operators/users more control over
how the new payload would be distributed.  Old 'modules'
could be left if desired, or updated with the new command.

Scenario tests were updated to test the new command.

DocImpact: update documentation to reflect module-reapply command

Change-Id: I4aea674ebe873a96ed22b5714263d0eea532a4ca
Depends-On: Ic4cc9e9085cb40f1afbec05caeb04886137027a4
Closes-Bug: #1554903
2017-02-27 18:43:32 +00:00
LiuYang 7be9e623d6 Wrong comment symbol in configuration file
Change-Id: Icf6fa8296ae3bfa05a8e4f07c38ede1fb951d18d
2017-02-24 14:56:03 +08:00
Petr Malik 21250cf20c Add support for Oslo Policies to Trove
The Oslo Policy library provides support for RBAC policy
enforcement across all OpenStack services.

Update the devstack plugin to copy the default policy file
over to /etc/trove in the gate environments.

Note: Not adding a rule for 'reset-password' instance
action as that API was discontinued years ago
and is now just waiting for removal (Bug: 1645866).

DocImpact
Co-Authored-By: Ali Adil <aadil@tesora.com>

Change-Id: Ic443a4c663301840406cad537159eab7b0b5ed1c
Implements: blueprint trove-policy
2016-12-06 21:51:21 +00:00
Jenkins 39ec28436e Merge "Use http_proxy_to_wsgi middleware" 2016-11-21 11:03:52 +00:00
Mariam John ff2a52038b Implement full online backups for DB2
Currently Trove supports full offline backups for DB2. In this
implementation, we have added support for full online backups for DB2
using archival logging.

Change-Id: I30b5b0b85120fd105cc3db57983b062fad5fab5a
Implements: blueprint db2-online-backup
2016-09-13 20:35:01 +00:00
Jenkins 4603d75b0a Merge "Trove workbook for scheduled backups" 2016-09-09 21:34:05 +00:00
Morgan Jones caab14ecf1 Trove workbook for scheduled backups
A Mistral workbook adding a workflow for creating a backup.
This is used by the "trove schedule-create" command to schedule
a backup.

Change-Id: I68d997abf4ec7c32044dd18cf2a70e687c0fed9f
Implements: blueprint schedule-backups
2016-08-25 17:00:48 -04:00
Jenkins 40da584699 Merge "Add trove-guestagnet.conf sample for log rotation" 2016-08-23 01:59:36 +00:00
Amrith Kumar 3b0d1ea25d Adds the api-ref migrated RST + YAML files
With this email[0], you must migrate API reference docs into RST. The
conf.py and the tox environment are also cribbed from nova.

Still need to retain the install_command in tox.ini, otherwise the
api-ref job fails.

[0] http://lists.openstack.org/pipermail/openstack-dev/2016-May/093765.html

Co-Authored-By: Anne Gentle <agentle@cisco.com>
Co-Authored-By: Amrith Kumar <amrith@tesora.com>

Change-Id: I3315261aa18729fa7a6aa79d4a1d6c24de1e2c6b
2016-08-17 17:46:41 -04:00
Masaki Matsushita f80dd43297 Add trove-guestagnet.conf sample for log rotation
This change adds a configuration sample of guestagent
to use log rotation.

Change-Id: I2f2fb0d42f2f9d674daf3adff8725a365cc1a5b3
Closes-Bug: #1269615
2016-08-09 13:24:11 +09:00
Greg Retkowski f7b8805c3a guest_id missing err, switch config_drive default
This changes the default setting for use_nova_server_config_drive
from False to True. By default, nova does not use a config
drive for launched instances. Therefore trove must implicitly
request the config drive be used so that it can inject
guest_info.conf into the guest VM.

This commit also adds an error to guestagent when guest_id
is missing. If the guest_id configuration parameter is not
set at guestagent startup it will throw a RuntimeError. This
case typically occurs because guest_info.conf was
not injected into the guest, or was not included in the set
of configs that guestagent reads at startup.

And finally, this commit adds a section in the 'building guest
images' documentation describing how configuration is injected
into the guest agent.

Change-Id: I084c841472183893a63ca7b70d560f44a2f08901
Partial-Bug: 1609915
2016-08-06 15:16:20 -07:00
Masaki Matsushita 1584f198a5 Introduce "icmp" option for security group rule
This change introduces new datastore option "icmp" to
configure whether to permit ICMP. It helps users to
check DB instance health in different way from access
DB ports.

DocImpact
Closes-Bug: #1485884
Change-Id: I61edeb38ded5543b7976a01363108a7b5b4fc5b5
2016-08-04 13:58:56 +09:00
Masaki Matsushita 583d5cd428 Use http_proxy_to_wsgi middleware
This commit enables to handle HTTP_X_FORWARDED_PROTO by using
http_proxy_to_wsgi middleware of oslo.middleware.

Change-Id: I6a11c8470205ca78bdb027fa9a06fec3acda33ad
Closes-Bug: #1590608
2016-08-03 09:29:29 +09:00
Jeremy Stanley 42de1e7f7e Switch from MySQL-python to PyMySQL
As discussed in the Liberty Design Summit "Moving apps to Python 3"
cross-project workshop, the way forward in the near future is to
switch to the pure-python PyMySQL library as a default.

https://etherpad.openstack.org/p/liberty-cross-project-python3

BaseMySqlRootAccess.enable_root(): catch also InternalError because
the PyMySQL error is not wrapped into a SQLAlchemy OperationalError,
but a generic SQLAlchemy InternalError. Similar change is made in
026_datastore_versions_unique_fix.py.

This change requires a trove integration change to add the PyMySQL to
the guest image: Id4d013d174ba40a453819f900aaa316a93e59b48.

Partially implements: blueprint trove-python3
Co-Authored-By: Victor Stinner <vstinner@redhat.com>
Depends-On: Id4d013d174ba40a453819f900aaa316a93e59b48
Change-Id: I65e8a8d5dc251a8b00529cdfb1a6ada3d5720f68
2016-06-30 08:47:01 +02:00
Matt Fischer bd360fb3c3 Switch test config file to use os_region_name
The old value of nova_region_name has not been used in some time.

Change-Id: I8fdd6d8ae4f0982b37cff7b5583429ccb6bd4aaf
2016-06-05 20:22:04 -06:00
Amrith Kumar 02d9f9e57a remove verbose option in Trove config files
This option is now deprecated and will be removed in Newton. The
default is True, time to get rid of it in Trove.

Partial-Bug: 1586066
Change-Id: I63e3a2832d0862254548be36fea77d5d4fa381b3
related: https://review.openstack.org/#/c/314573/
ref: http://lists.openstack.org/pipermail/openstack-dev/2016-May/095166.html
2016-05-26 14:46:05 +00:00
Jenkins dfa7e86265 Merge "Add MongoDB config server port to cfg" 2016-03-18 13:59:47 +00:00
Mariam John 65917f3ec0 Implement Backup and Restore for CouchDB
The recommended method for doing full backups in CouchDB
has been a simple filesystem copy of the data files. This
is because CouchDB stores data in wholly contained append
only files. For example, when a user creates a database,
a corresponding <database-name>.couch file is created in
the database directory.

The backup functionality has been implemented by compressing
the database directory and then encrypting it and sending it
over to store in Swift. Similarly, the restore functionality
has been implemented by fetching the files from Swift and
uncompressing them into the database directory. After this,
the ownership of the directory needs to be updated.

To test the changes, follow the steps:
- Create a CouchDB instance
- Access the admin console called Futon using the following
  url: http://10.0.0.5:5984/_utils/
- Create a database from there and create one or more documents
- Create a backup of this CouchDB instance
- Create another CouchDB instance from the backup created above
- Access the admin console for this new instance and verify that
  the database created above is there

couchdb client library for the integration tests has been added
to global-requirements: https://review.openstack.org/#/c/285191/

Change-Id: Iad6d69bf60ace73825819081964a43ad53d6c6fc
Implements: blueprint couchdb-backup-restore
2016-03-15 14:37:35 -05:00
Jenkins 8d9af8c4c8 Merge "Addition of DB2 backup & restore functionality" 2016-03-15 18:43:02 +00:00