Commit Graph

25 Commits

Author SHA1 Message Date
Matt Riedemann 1516e720ce Set ETCD_USE_RAMDISK=True by default
Cinder and etcd are enabled by default and by default
cinder uses etcd as a distributed lock manager with
tooz as an intermediary. We see a lot of ToozConnectionErrors [1]
in the cinder logs when etcd is backed up [2] which results in
cinder operations timing out causing test failures, like
when a volume is not deleted within a given time.

This changes ETCD_USE_RAMDISK=True by default to try and
alleviate some of the pressure. An alternative is if we know
we're in a single-node job we could just not use a DLM for
Cinder.

[1] http://status.openstack.org/elastic-recheck/#1810526
[2] etcd[26824]: sync duration of 12.076762123s, expected less than 1s

Change-Id: I5f82aa40e9d84114e7b7b5cf19ec4942d6552490
Partial-Bug: #1810526
2019-02-05 11:21:39 -05:00
Michał Dulko 6757a9c5de Add option to place etcd data dir on RAM disk
Seems like for etcd-heavy services like Kubernetes, the fsync
performance of gate VM's are too low [1]. This commit implements an
option to put etcd data directory on RAM disk (tmpfs) to work this
around.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-January/001849.html

Change-Id: I5a17099cb9d6941b1a009dc82daefd2c7946d892
2019-01-24 13:43:47 +01:00
Michał Dulko 119ee66cde Log debug on etcd if $ENABLE_DEBUG_LOG_LEVEL is on
Change-Id: I452a2e4882377d165e3c28fcec18e237e45db8a4
2018-12-20 18:55:29 +01:00
Zuul a6a36d11d8 Merge "Allow overrides for ETCD ports" 2018-03-11 05:43:44 +00:00
Davanum Srinivas 27367bea48 Allow overrides for ETCD ports
This will enable us to run one etcd for devstack and another
for kubernetes in the same box if necessary

Change-Id: Ib71ded24727b80afd4d98eb68bade0f8c0f72311
2017-11-30 20:04:05 +00:00
Hongbin Lu ad180e0e35 Install etcdctl along with etcd3
It is better to have this command line tool for debugging etcd.

Change-Id: Ie0eb79d4e543df29ce6a38b57c8ef57a5d2211b4
2017-11-29 13:21:30 -05:00
Jenkins b0a08c1b04 Merge "Make etcd3 setup work with IPv6 addresses" 2017-09-28 05:28:36 +00:00
yuanke wei c652a498fb Fix partial download problem
(1) when checksum fails, better delete the broken files and try the second time;
(2) amazon s3 is not good in mainland China, better try one more time with wget

Change-Id: I24ee73f216b78bd80564863cd335e5d5a9b56360
2017-09-22 19:44:43 +08:00
Jens Harbott 146332e349 Make etcd3 setup work with IPv6 addresses
The client are told to connect to SERVICE_HOST instead of HOST_IP, so
we need to start etcd3 with matching listening parameters.

Change-Id: I96389090180d21d25d72df8f9e8905b850bcaee9
Partial-Bug: 1656329
2017-09-20 08:56:51 +00:00
Monty Taylor d8bb220606 Make list_images.sh emit the etcd3 tarball
We need this for every devstack run now, so downloading it from github
every time isn't the most awesome thing in the world.

Add an extra variable EXTRA_CACHE_URLS which will be appended to the
output of tools/image_list.sh.  This way, these files will be
downloaded during the daily nodepool build, but they will not be in
the IMAGE_LIST and hence be considered as images to upload.

Add a function get_extra_file which echos the path to a file given the
URL.  It will first check the cache at $FILES, and if not present
download it.

Update the documentation in image_list.sh to reflect what's happening.

Move the defaults for etcd variables into stackrc, since it is a base
service now.

Change-Id: I86104824a29d973a6288df1f24b7891feb86267c
2017-09-05 09:40:40 +10:00
Sławek Kapłoński 08367bac0b Switch from $DEST/data/etcd to $DATA_DIR/etcd
Etcd should use $DATA_DIR/etcd instead of "hardcoded"
$DEST/data/etcd directory for its data.

Change-Id: Icdc65f52a9d75981b63789036248e00d8ab72f11
2017-08-27 08:46:28 +00:00
Kevin Zhao eca7ce7492 ETCD need to add UNSUPPORT environment in AArch64
Closes-bug: #1708575

Change-Id: I77e78389ac7b8df9ba9f84b072f446d2e03d84d1
Signed-off-by: Kevin Zhao <kevin.zhao@arm.com>
2017-08-04 11:54:41 +08:00
TommyLike 599ecfbd4c Download etcd3 only zip file not exists
Give a change to manually prepare the zip file
for etcd.

Change-Id: I54283cf2ed62bb716af34df21e8c168014264c8c
2017-06-22 08:43:27 +00:00
Jenkins 3603bf59c3 Merge "rework etcd3 install to be like other devstack services" 2017-06-20 16:14:43 +00:00
Sean Dague 62b56601a9 rework etcd3 install to be like other devstack services
This separates out the install phase early from the start phase to
make this mirror other services in devstack.

Depends-On: I4124dc7e3fd3b4d973979da85209ec991c0f8c4b

Change-Id: I76f8740448b25a48869ee80006e826baa6cafc2b
2017-06-20 10:24:01 +00:00
Antoni Segura Puimedon 19279b0f87
etcd3: Allow for multi-host deployments
In Multi host deployments, it is possible to run ETCD in a different
host than the SERVICE_HOST (where all the controllers run). This patch
brings that distinction.

Change-Id: I15fe6f25eedf1efebaab81cce26b080577b856cc
Signed-off-by: Antoni Segura Puimedon <antonisp@celebdor.com>
2017-06-16 16:04:59 +02:00
Jenkins 3742b14622 Merge "Change "files" directory in etcd project" 2017-06-07 13:33:58 +00:00
Jenkins e1a2448970 Merge "Expose etcd port as a variable" 2017-06-05 18:28:11 +00:00
Rodolfo Alonso Hernandez 6f962a2ee5 Change "files" directory in etcd project
Function "_install_etcd" is trying to use "files" directory
to download a file. Instead of this, this directory should be
$FILES, which is defined previously in parent script.

TrivialFix
Change-Id: I643ce3b9aba1f65f03524430c748bf120d071509
2017-05-31 13:36:22 +01:00
Hongbin Lu de8580691d Expose etcd port as a variable
This allows devstack plugins to retrieve the etcd port from devstack
instead of hard-coding it.

Change-Id: I106b559b8ac0fb99a0426bce97a27f67e32d264d
2017-05-27 03:54:45 +00:00
Davanum Srinivas 853b475bfd Fix typo
should be etcd3

Change-Id: Icfa24654699a4e4e4be8a53f7bbe4634badbff7b
2017-05-25 13:03:58 -04:00
Jenkins f0442dd592 Merge "Use sha256sum instead of gpg for verification" 2017-05-24 14:57:29 +00:00
Sean Dague bba924121c Use sha256sum instead of gpg for verification
gpg verification requires network connectivity which is non
mirrorable. We try to avoid that in devstack whenever possible. A
sha256sum is a totally reasonable way of knowing if the downloaded
package is valid.

Closes-Bug: #1693092

Change-Id: Id496ab53f76444f08dc6961f1ecd25f450cc96d7
2017-05-24 08:04:18 -04:00
Davanum Srinivas d8283fde66 Avoid installing etcd3 in subnodes
We need to handle this better, for now, just don't install
etcd in the sub nodes. We need to setup the proper clustering
mechanism if we want to have etcd3 running in multiple nodes

Change-Id: I8dd385e3c993942473e67d04367cdf74495dbeef
2017-05-23 22:12:42 -04:00
Davanum Srinivas 546656fc05 etcd3 as a base service
ETCD_DOWNLOAD_URL is set to github url, in our CI, we can point
ETCD_DOWNLOAD_URL to a url in tarballs.openstack.org possibly
in devstack-gate

Download the etcd binaries and drop them into /opt/stack/bin and
use it from there. Cache the tgz for subsequent use (local workflow)

daemon-reload is called twice once from inside the write_user_unit_file
and then when we adjust the entries with additional things recommended
by the etcd team. We need a better way to do this in the future.

Added a TODO to verify the downloaded artifact later. The etcd team
posts gpg signature, we could verify that or run sha256sum and hard
code that in lib/etcd3 file. We would have to update it whenever we
bump the etcd3 version.

We use the public key "CoreOS Application Signing Key <security@coreos.com>"
with ID FC8A365E to verify the integrity of the downloaded file

Any jobs that need to be run on architectures where v3.1.7 is not available
should rey the v3.2.0-rcX release candidates. We can switch to v3.2.0
when it gets released.

Initial version of this code was borrowed from the dragonflow
repo:
http://git.openstack.org/cgit/openstack/dragonflow/tree/devstack

Change-Id: Ibbb430fb1dbf66942168e0cb52d990ab6a2eb8d7
2017-05-22 21:57:48 +00:00